the problem with the way Google interviews is that, despite it being heavily standardized, it is not sensitive to non-algorithmic skills and talents that interviewees have. It will _only_ pass candidates who are unusually good at algorithm puzzles, on whiteboards, under time pressure.
When I was interviewed for a job at Google I was told there would be 5 different interviews measuring 5 different things:
Pure Algo (which I view as "let's see that your CS grades were earned and not bought")
Pure Coding (let's see if you have a reasonable coding style and design and are proficient with at least one language)
Algo & Coding (a bit of both that makes sure you can solve a simple problem and code it [i.e. work entirely through a problem])
Software Design (so more on your skill in designing code - finding the right abstractions and interfaces etc.)
Systems Design (your ability to design complete, large scale systems at a very high level)
And this is more or less what I got, which seemed fair and logical to me. I agree that it is not tailored to the interviewee's individual skills (for example, your special training in cyber security is unlikely to give you an extra edge), but it makes sense for "good overall software engineer", and if you aren't also a good overall software engineer in addition to your special training in cyber security, then you are probably not what Google is trying to find.
Also, it does look at your skills in coding and design which are both non-algorithmic.
Now whether or not this experience is shared with other interviewees, or matches what you are looking for is something else :X
Maybe it's not what they are looking for. Maybe it's only the easiest test, like looking for the keys under the lamp post. It could also be that this way of hiring people turned the company into a specific direction, more attention to the sw and less to the customers.
> it is not sensitive to non-algorithmic skills and talents that interviewees have.
I work at Google and do interviews (though I don't enjoy them). We do ask questions around domain expertise, software design, etc. It's not all just coding and algorithms.
A good question:
1. Has a low enough floor that a poor candidate can still make some progress and not feel like they are doing poorly and get stressed out.
2. Has a high enough ceiling that a strong candidate doesn't blow through it in five minutes.
3. Has a smooth ramp between those points.
4. Doesn't rely on too much domain-specific knowledge so that a candidate who happens to have a random gap in their background that leaves them totally hosed.
5. Is concrete enough that the interviewer can capture that feedback in a way that the hiring committee can easily understand.
6. Isn't well-known outside of Google as stock interview question so that candidates can game us by just learning the answer.
7. Isn't asked by any of the other interviewers the candidates sees.
8. Can be explained and worked through in about twenty minutes.
In case it isn't obvious, it is really really hard to find good questions that pass that gauntlet. Questions do tend to skew towards smaller-scale algorithm coding questions because I think those tend to survive that gauntlet better than most other questions.
> unusually good at algorithm puzzles
Interviewers are trained to not ask "puzzle" or "trick" questions. Not only are trick questions a shitty experience for the candidate, they are a shitty experience for the interviewer too. My job in an interview is to get as much data as I can about a candidate in order to provide information to the hiring committee. If I ask you a trick question, I get about one bit—in the binary sense—of data from you: did you find the trick or not?
I don't think you have to be unusually good at algorithms. I basically read some Wikipedia articles and spent a few hours in the hotel cramming Algorithms in a Nutshell, and I managed to squeak through.
That time was incredibly well-spent. Since then, I have relied on that algorithm knowledge way more than I expected too, and have since spent more time learning algorithms and data structures because I can clearly see it's made me a better programmer.
> on whiteboards,
That part is hard. We allow candidates to use a laptop too, if they prefer, or both. My experience is that candidates who use the whiteboard, at least for the earlier "design" parts of the question tend to do better than the ones that go straight to typing.
We need to learn how you think, and putting a screen in front of people tends to make them clam up. If all I see is the code you write and you don't explain your thinking behind it, I don't get much data.
> under time pressure.
That part is really hard too. The reality is that interviewers and candidates have a limited amount of time they can put into this process. Keep in mind that most candidates are currently employed and don't want their job to know they are interviewing. Many of them travel to interview. There are only so many hours.
My experience hiring at (YouTube.. at the google campus), was similar to the original post. 3 phone screens (which I must have done ok on), and then they flew me out to CA. I went through the interview process.. got tripped up on the 'puzzle' questions (which I thought was bullshit for a Python programming job.. but whatever), and then didn't get the job. I actually agree with what was said in the original post: this process probably works well for the type of employee Google/Youtube wants to hire.. shame on me for not doing my homework.. I guess. However, I can tell you what definitely was a complete turn-off, was the odd obsession of seemingly everyone there with where I/(you) went to school. At least 5 times: me: 'I live near Princeton'. they: 'oh, did you go to Princeton??!!! (elation)'. me: 'No.. I went to a state school'. they: 'oh.. ' me: (silently) 'yeah.. I'm sorry. It gets better.' Again, whatever.. in truth, I appreciated the opportunity to interview.. no big deal.
What was even weirder after that was, for no less than 6 years, to be 'actively' recruited (again) by Google. My position each time was: thanks, but not interested in going through another bullshit interview where I'm continuously reminded that I didn't go to Princeton. I'm just not Google material.
In all seriousness, in my opinion, Google is one of the best companies in the world and is run and staffed by the best minds in the Industry. My hat's off to them for all of their accomplishments. I just don't want to work there (even if I could :)
> However, I can tell you what definitely was a complete turn-off,
> was the odd obsession of seemingly everyone there with where I/(you)
> went to school.
Ugh, that's really frustrating. I'd like to think they were just searching for common ground, but jeez.
Candidates should be judged based on what they know, not where they happened to have acquired that knowledge.
I am a college dropout and I had no idea what a rare breed I was at Google until after I got hired. (I came from the game industry where college degrees weren't as important.) There are so many over-achievers here, that I think many Googlers have never even considered that someone may not have gone to one of the top ten CS schools in the US. Especially in Mountain View, literally everyone they know probably has.
Speaking as someone who's done these interviews (but was never, for the record, asked about anything other than coding/algorithms): I think the core premise is flawed. The idea that a good developer is somebody who can regurgitate knowledge they learned on Wikipedia in a few hours from memory is just not a good base for determining if the engineer is good. More importantly, a developer who cannot regurgitate the knowledge from memory is not necessarily a bad developer, or, frankly, in any way otherwise distinguishable from the guy who can.
> The idea that a good developer is somebody who can regurgitate knowledge they learned on Wikipedia
I don't think Google's hiring people thing that that's what a good developer is, as much as they think it positively correlates to good developers.
(I don't know to what degree that's true, but it's how I assume they arrived at the current interview process.)
> More importantly, a developer who cannot regurgitate the knowledge from memory is not necessarily a bad developer
This is also a valid concern, but Google is much more concerned about false positives than false negatives. Missing out on a good candidate is a bummer. Hiring a bad one can be a nightmare. So the process is skewed to avoid the latter even if it costs the former.
> I don't think Google's hiring people thing that that's what a good developer is, as much as they think it positively correlates to good developers.
I agree. However, I think they are wrong... I'll expand below.
> This is also a valid concern, but Google is much more concerned about false positives than false negatives. Missing out on a good candidate is a bummer. Hiring a bad one can be a nightmare. So the process is skewed to avoid the latter even if it costs the former.
This is an oft-repeated line about Google's hiring process, and in fairness, I think it's oft-repeated because insofar as it reflects Google's belief that their process results in good developer hires, it is true.
However, my suspicion is that the phenomena going on here is not "losing out on some, but not all good developers in order to weed out bad ones," rather, it's "losing out on a certain kind of good developer in order to weed out bad ones." That is, I'd conjecture that the good developers who can (or want to) memorize algorithms and regurgitate basic CS knowledge are one kind of capable dev, and the good developers who rely on tools to a greater degree are another kind of dev. Call them types A and B.
I don't have the two segregated into neat categories - because this is just a suspicion, based on people I know who work at Google, and my own experiences - but I think it's roughly along this line: Developers in category A have an innate desire to learn about and understand computer science on a theoretical level, as much as or moreso than a practical level. Such a person may or may not enjoy building things as much as they enjoy learning about how to possibly build things. Developers in category B (I'd include myself in that group) don't care about theory as much as practice; they do what is necessary to get the job done. Now, neither group hates theory or practice, they just have preferences about which one to spend their time on.
For an organization to really be successful, I'd argue, you need a mix of types A and B (tending more towards one or the other depending on the type of entity). If Google is weeding out most or all of type B, they are doing themselves a disservice - and I would argue that insofar as many of the common complaints about Google (services created then abandoned, poor support, poor attention to bugs/issues, etc.) are true, if this theory is also true, it would help explain why. The practical-preference developer wants to make things work and keep them working; the theoretical-preference developer wants to discover new things and constantly expand her knowledge. Both aims are good, but you cannot have one to the exclusion of the other as an organization.
> That is, I'd conjecture that the good developers who can (or want to) memorize algorithms and regurgitate basic CS knowledge are one kind of capable dev, and the good developers who rely on tools to a greater degree are another kind of dev. Call them types A and B.
I really dislike this characterization. I'm a googler. I've never tried to, or needed to, memorize algorithms. I don't know if this characterization comes from misunderstanding, rationalization, or what, but in my experience at least, neither do most of my coworkers.
That is, I at least don't recall a rote algorithm when interviewing In fact in one of my interviews (not at google, but I could see a similar question happening there) I had to derive a solution to a problem in a space I was totally unfamiliar with (locking and multithreading).
It seems like, if you assume (incorrectly) that somehow you can't cheat the system, Google is selecting for people who can solve unfamiliar, complex, problems by applying first principles. That is, I think, orthogonal to the idea of 'theoretical or practical' computer scientist.
We have something interesting here, in that I'm looking at this from the perspective of someone the Google system rejected (and who later decided/rationalized/realized participating wasn't really worthwhile) and you're looking at it from the perspective of somebody it embraced.
Naturally, it wouldn't boil down so easily in practice to these two types, even if they are the correct ones. Everybody is a mix of both (and plenty of other things); plus, we have to assume that, like you say, occasionally somebody incompetent or strongly type B "cheats" the system and gets hired.
But I guess the question is: How, to you, does "selecting for people who can solve unfamiliar, complex, problems by applying first principles" not seem to jive with my definition of category A?
So, I think my biggest issue with your A/B dichotomy is that I don't see it. That is, in school, in myself, in coworkers, there isn't this large group of people who are trying to do all the theory at the expense of practicality, as opposed to this group of stuff-accomplishers who aren't theoreticians.
I mean, occasionally those people exist, both the "fuck it I'm going to sit down and type until it works" people and the "I must understand this concept before I write a single line of code" people. But as you say, everybody is a mix of both, and I think most people are nearer the middle than the sides, which makes the possibility of mild bias towards one side or the other a lot less harmful than you maybe expect.
The thing is that as an outsider looking in, I see a lot of evidence that Google has this dichotomy and suffers from it. I mentioned some of it in my posts above, but in general, a lot of the company's pain points from my perspective - and from the perspective of employees less satisfied with their experiences[1] - fit what we'd expect from such an enterprise.
Here's an example of what I mean: Google is a company with incredible tech and quite a lot of money, but its products tend to fall into clear patterns. First, create something awesome, then fail to support it, then fail to monetize it, and eventually, it fall to the wayside and is discontinued. There are so many apps Google has created that fit this mold that people have done meta-analyses on how long the average Google product lasts.[2] Obviously, not all Google's products fit this mold - but I think Google is unique in being able to support this pattern, fiscally and from a development fatigue standpoint.
This pattern is exactly what I would expect to find at a company that is mostly based around theory and concept, and that is either inexperienced at or disinterested in support work after the initial execution.
It's probably true that most people, even at Google, are somewhere in the mid-range. However, a large enough group of people with a small bias will result in a shift in the policy of an organization. I still think it's fair to say that, if my dichotomy exists, it would effect Google's institutional behavior even if the degree of difference between people is low on average. That is to say that while you may not see it, it also may not be obviously or even at all visible from the level of one person in one part of the organization, whereas its effects are visible from both within and without. It's kinda like dark matter - you don't know what's happening in someone's head, so you can't observe it directly, but you can predict outcomes based on hypotheses and see what comes true.
Interesting, when I read those answers, what I see is more or less people saying "management can sometimes be downright bad, and is often stupid". I very much doubt that many of the people (broadly) deciding which products to support and which to drop were hired via a new-grad-like interview process.
For things like moonshots/other bets, the "build something awesome but don't monetize it" makes a lot of sense, since the entire point of that division seems to be "build something awesome and see if it is sustainable too". More often than not, unfortunately, the answer is no.
In fact, if anything, I'd reverse the cause and effect in your idea. If we presuppose that there is this dichotomy in people and it affect google's motives and goals as a company, then this would influence the tech hiring practices to be the way they are, not vice versa.
> I very much doubt that many of the people (broadly) deciding which products to support and which to drop were hired via a new-grad-like interview process.
I'm confused as to why you doubt that. Google's been around for almost 20 years. Their interviewing process has been around since at least 2003.[1] It's 100% possible that somebody was hired, ended up in higher management, and graduated to making driving decisions (at least over a particular area) in that time, unless your suggestion is that nobody who enters as a new grad ever stays long enough to get to that level, which at Google (vs other SV companies) seems unlikely.
> For things like moonshots/other bets, the "build something awesome but don't monetize it" makes a lot of sense, since the entire point of that division seems to be "build something awesome and see if it is sustainable too". More often than not, unfortunately, the answer is no.
It does make sense, it's more the messaging around those kinds of projects that tends to get lost. You end up in situations where thousands or sometimes millions of users are relying on a "beta" product, which has no monetization strategy, and then gets scrapped. It's a pattern that's still unpleasant for end users and not great for Google's reputation.
> If we presuppose that there is this dichotomy in people and it affect google's motives and goals as a company, then this would influence the tech hiring practices to be the way they are, not vice versa.
That's a great point, and likely, assuming of course that Google started this way (I think it likely did, given its founders' backgrounds). It creates a self-perpetuating cycle, though, which still ends up being problematic.
> I don't think you have to be unusually good at algorithms. I basically read some Wikipedia articles and spent a few hours in the hotel cramming Algorithms in a Nutshell, and I managed to squeak through.
Either a lie or you are unusually good at algorithms. People work > 50 hours on Leetcode, CTCI, etc. to try and get a job at Big 4. A couple of hours just reading Wikipedia articles isn't even close to the amount of effort people put into 'gaming' the system these days.
That was the only prep I did for the interview. I was a senior software engineer at EA at the time, with about a decade of professional software experience.
I don't have much of an academic background, but I had written and shipped quite a lot of code by that point in time.
The cramming did help—several of the algorithms I read about were either new to me or I hadn't seen in ages—and some of them did come up on the interview. (I've also used almost everyone of them at my work at Google since, strangely enough.)
My point was just, if you are already a good enough engineer to be successful at Google but don't "interview well" because of whiteboard experience or algorithms it's pretty easy to shore up those two things. You don't need to spend a decade at a monastery meditating on Knuth.
If you're a decent coder, you've already done way harder things. Breaking down a big messy problem into pieces you can code and test is hard. Shipping applications is hard. Breadth-first search is not hard.
Here's the problem I have with interviewing, speaking as someone already at a big 4 company and has gone through this process already. I know exactly what types of things I need to prep for and even as someone who's competed in competitions like ICPC and TopCoder, I'm always worried that I'll get the type of interviewer who will just ask obscure questions and not go to the level of depth you mentioned when it comes to working with the interviewer to assess their problem solving skills vs just checking to see if they know the solution to some trick question - the kind that most people not knowing the solution beforehand would almost never be able to figure out within 30-60 minutes.
I'm planning on interviewing with Google in a few months just because of all the great things I've heard about their infrastructure and engineering culture, but due to these concerns I've budgeted to use my current vacation time just to get back into prepping the way I did when training for ICPC just so that I'll have confidence knowing that most interviewers probably have never studied algorithms at that level and that I'll have more confidence in not being potentially tripped up by these types of interviewers.
Most of my friends, even those at Google, have tried to convince me that this plan is overkill but I just can't get myself to leave the fate of my future into the hands of some random interviewer and then regret not doing this level of preparation if I get rejected...
you're missing something important about my comment. the issue isn't the hair-splitting difference between what you call "puzzle or trick questions" and what you describe as "a good question" (with your eight criteria). the issue is that Google style interviews ONLY ONLY ONLY ask coding questions in a pressure cooker environment.
I've interviewed at Google and the interview experience was sufficient to convince me that I'm not a culture fit. Maybe that was the point. I tend to thrive in work environments where my "soft skills" matter more. Seems like that's not even on the radar at Google.
> the issue is that Google style interviews ONLY ONLY ONLY ask coding questions in a pressure cooker environment.
By "pressure cooker", to you mean in terms of time, or more social intensity? There's not too much Google can do about time. Like I said, it's not fair to ask candidates to dump weeks of their life into the hiring process.
If it's about the social intensity, I agree, that's hard. Many programmers are introverted and being "on" for an interview is really really difficult. It's one of the reasons I don't like doing interviews—it's hard for me on the other end too.
We are trained to try to make it as relaxing and pleasant of an experience as we can, but there's only so much we can do. The reality is you're in a 1-1 conversion with someone with a fixed time bound and where your performance may alter the trajectory of your life.
That is a stressful experience, no two ways about it.
