Now you have to count defects

You have to do that anyway, and in fact you probably were already doing that. If you do not track this then you are leaving a lot on the table.

I was more thinking in terms of creating a benchmark which would optimized during training. For regular projects, I agree, you have to count that anyway