I believe we are misunderstanding each other, as you are making some assumptions. We do spend time training junior developers. We are hands on during work.
I'm talking about one problem at the start of the process. In the example, I didn't touch on their choice of sort algorithm. I was pointing out that they believed that re-sorting the same array, for every single GetIndexAt(x) call, was an ok thing to do. The majority of boot campers did, the majority of CS grads did not.
So if we have two distinct groups applying for a junior level position, and one group has a stronger grasp on fundamentals, I fail to understand how a company is at fault for choosing the better candidate. Objectively, the junior level CS grad had a better understanding of the problem.
Are they saying that it's an actual okay thing to do? ...like, you challenged them on it and had a discussion and they said it was perfectly okay regardless of size of array?
Or it was just allowed to let lie and they said it's an okay thing to do because that's what's in their code?
I think it's a huge mistake to conflate whether someone does know and whether they can know something. Some of the best programmers I've ever seen started off pretty clueless - most of that group learns things absurdly quickly too.
I don't think that exercise gives you good information other than to confirm your expectations. I don't know whether that can tell you if they'll bring value to your organization or not.
There were a few hundred applications to sort through for the position. Applicants all received the same test. The initial response is used to filter the pool and decide with whom to speak with further. Only with a filtered applicant pool is it reasonable to sit and discuss their solution with. Some criteria is needed to filter the pool, so obviously those that have submitted a better initial solution will be selected. The exercise shows that some applicants will require more training than others.
There are many reasons why I think this process is broken that others have written so much at length about enough that I'm not going to bother taking the time.
If it's enough for you to say that because you have a test that everyone takes and it filters your candidates so that's proof of ability, without even going into the results and asking if your test is optimizing for the correct things, there's nothing I can say to convince you otherwise.
Anecdotal, but I know about a half dozen people with experience and CS degrees who would blow your filter but can really get shit done and don't really need any training.
Hiring is hard and the real shame of this industry is that collectively we don't approach this problem with the same level of rigor that we do everything else because by and large we consider this sort of work to be beneath us and not worth a significant investment of our time.
I just want to leave my response to your now-deleted followup:
I don't advocate talking to every candidate but I don't think it's possible to estimate someone's skill without talking to them (even email is a form of conversation).
I find much better filters for that sort of thing are ability to follow directions and attention to detail. If you have hundreds of applications, I guarantee they're not all great spellers or communicate effectively. I value the soft skills way above programming ability (since we have to communicate effectively every day) in the early stage.
I find that it's pretty easy to demonstrate that you "have it" from portfolio, resume, cover letter and the application process.
Personally it's a huge pet peeve of mine in an interview process whenever I have to write code and there isn't some discussion about that code - to the point that I feel it a waste of my time and a strong signal not to work there.
I'm talking about one problem at the start of the process. In the example, I didn't touch on their choice of sort algorithm. I was pointing out that they believed that re-sorting the same array, for every single GetIndexAt(x) call, was an ok thing to do. The majority of boot campers did, the majority of CS grads did not.
So if we have two distinct groups applying for a junior level position, and one group has a stronger grasp on fundamentals, I fail to understand how a company is at fault for choosing the better candidate. Objectively, the junior level CS grad had a better understanding of the problem.