> In our first 30 days, we've come up with a replacement for resume screens, and shown that it works well.
What's the metric that shows it works well?
> The really exciting point comes when we can re-run all this analysis, basing it on actual job performance, rather than interview results.
And how precisely do you measure job performance? If this is achievable, I've got a line of companies out my door that would love to pay for a service that systematically measures job performance.
Measuring job performance is hard. A good analogy is measuring intelligence. The IQ test is clearly bullshit (it does not come close to reflecting the complexity and high dimensionality of human intelligence). That said, it's a useful research tool. Across a large number of people in aggregate, it can be used to learn things (like that leaded gas was a really bad idea). We'll be doing the same sort of (very imperfect) measurement of job performance.
I'll be very interested in hearing what your measures are, and how you'll account for things like employees being in a bad environment/having bad bosses. It's a complicated thing to measure.
Are you guys planning on publishing any of this material in a journal or research venue, or will you keep the results to blog posts?
For this purpose, it shouldn't be too hard. You don't need the evaluation results for deciding raises or promotions, and you don't need to compare employees to each other. You just need to determine if the hiring decision was correct. We can do it with a binary score:
0. Oops, shouldn't have hired.
1. AAA would hire again.
I suppose you could make finer distinctions (need to fire this person/we regret hiring but will work on it/meets expectations/exceeds expectations), but my point is that the golden standard for evaluating hiring decisions is whether the hire is adequate or not for the job. I mean, if someone gave you an oracle or a time machine that allowed you to evaluate the employee on the job for a while before hiring them, all your hiring problems would be solved. Now, if you cannot evaluate if someone is good enough for the job, then you have a bigger problem.
What's the metric that shows it works well?
> The really exciting point comes when we can re-run all this analysis, basing it on actual job performance, rather than interview results.
And how precisely do you measure job performance? If this is achievable, I've got a line of companies out my door that would love to pay for a service that systematically measures job performance.