Maybe I'm missing something here, but how exactly is it "unlabeled" data if they're specifically feeding it millions of pictures of "faces"? I mean, if you make a specific selection of the type of images you train the network on, isn't that basically equivalent to labeling them?
The aim of the paper was to produce an unsupervised system that would generate high level features from noisy data.
These high level features could then be used in supervised systems where labelled data is added.
Thus, the paper is about using an unsupervised system to help a later supervised system. An advantage of this is that, as the unsupervised system isn't trained to recognise object X, it instead learns features that are discriminative. This same network could be used to recognise arbitrary objects (which is what they do later on in the paper with ImageNet).
In other words: imagine a baby. She sees 100k "images" of faces. Thanks to the statistical regularity of the world, she now has a "subsystem" that recognizes a face in the absence of her knowing what it's called. Then, when she is told "this is a face" she pins this thing pointed-to to the existing, unnamed representation.
Nope. Not all the images contained faces (cats, bodies, etc.). There was no specific face-detection code. The system just learned the concepts from the data. http://en.wikipedia.org/wiki/Unsupervised_learning
You're correct: it isn't an unlabelled system, and the article author is deeply confused about basic topics in artificial intelligence.
What he's trying to talk about is "this is an unsupervised feature detector in a large dataset which is only categorized, and where no human has provided correct answers up front to verify progress."
The reason this matters (and it doesn't matter very much) is that that means that in cases where it's prohibitive to provide training sets, such as where you don't know the good answer yourself, or where giving a decent range of good answers would be difficult, this sort of approach can still be used.
"isn't that basically equivalent to labeling them?"
Yes. It is. The original poster is confused.
What he meant to say was "there is no training set."
The article authors of the paper? How can you say they are deeply confused - have you not seen their previous work and presentations? Everything else you say I agree with.
