While I'm sure we have different definitions of "superficial understanding" one thing I've noticed as I've gotten more interested in ML/datamining during the final stages of my master's in CS is that solving real world problems with these techniques is often a very different experience than deeply understanding the theory behind them.
For example I couldn't implement an SVM library from scratch to save my life, but I do understand what it means to be a 'maximum margin' classifier, from a high level how the 'kernel trick' works, and why you would tune regularization and cost parameters. However this knowledge has been enough to help me in quite a few interesting problems.
Reading accounts of how others have solved real world data mining issues it's amazing how often a very simple model will do the job, and also how often, even among more serious researches, there's a bit of intuition in finding the right combination of parameters, and lots of trial and error in searching for which model/blend of models really does the job.
I think there's a lot of room for more people approaching data mining with the 'hacker' mentality. Sure you don't want 'data scientists' using a randomForest whose eyes glaze over when you mention the word "ensemble", or someone who couldn't explain in plain terms what a "maximum margin hyperplane" is. But, there is a growing space for practitioners in this space, that aren't necessarily as strong in the theory as people working in the pure research space.
Slightly off topic, but I just completed the online learning from data course offered by Cal Tech (for free), and am pleased to say that 1) I understand this thread, and 2) have implemented an SVM starting with a quadratic programming package (not quite from scratch!). I highly recommend the class for anyone interested, a re-run of the course is starting soon: http://work.caltech.edu/telecourse.html
Much of what you say resonates with what I have seen from a different vantage point - old school software dev who has dabbled in data mining. Simple models often work and are preferable - easier to explain. More data and simple models generally provides superior results to small amounts of data and complex models.
It seems like "ensemble" methods - combining the results of several different algorithms - is generally a less-than-rigorous exercise that involves throwing a bunch of different approaches at the problem and averaging the results.
It is good to hear that there is "a growing space for practitioners in this space, that aren't necessarily as strong in the theory." But the term "Data Scientist" seems a bit lofty for folks doing this sort of work.
The thing is that "less-than-rigorous exercise" is true in many areas of ML. Take for example neural nets, which are very popular and successful, even among real expert's there a lot of 'magic' behind why they really work. SVMs are loved partially because they work well, but also they are very sound from a theoretical standpoint, if you know the math you can show that it will work, this is not necessarily true with many other successful techniques.
Interesting side note for ensembles: 'averaging' is usually not one of the best methods for blending results. More successful approaches include using either a perceptron or a simply training a linear model to find appropriate weights for predictions from each individual model. I've even had a case where simply picking the MIN of each set of predictions worked surprisingly well for a particular problem.
The above btw is something that I think a "Data Scientist" should know, and is well out of the scope of a software engineer who just plugs values into prepackaged algorithms. A "data scientist" should be able to read papers [1] that explain these things, which is more than many software engineers do.
Now I'm not a data scientist, but while I can't write an SVM from scratch, when I'm working on data mining problems I am reading several academic papers a week. I really think we're looking at two sincerely distinct areas of expertise and it's not too lofty to look at someone who has to read academic papers to do his job as a "scientist".
For example I couldn't implement an SVM library from scratch to save my life, but I do understand what it means to be a 'maximum margin' classifier, from a high level how the 'kernel trick' works, and why you would tune regularization and cost parameters. However this knowledge has been enough to help me in quite a few interesting problems.
Reading accounts of how others have solved real world data mining issues it's amazing how often a very simple model will do the job, and also how often, even among more serious researches, there's a bit of intuition in finding the right combination of parameters, and lots of trial and error in searching for which model/blend of models really does the job.
I think there's a lot of room for more people approaching data mining with the 'hacker' mentality. Sure you don't want 'data scientists' using a randomForest whose eyes glaze over when you mention the word "ensemble", or someone who couldn't explain in plain terms what a "maximum margin hyperplane" is. But, there is a growing space for practitioners in this space, that aren't necessarily as strong in the theory as people working in the pure research space.