Today the UConn CSE department was hosting a guest lecture by Dr. Vladimir Vapnik who was presenting Learning using privileged information (LUPI), an extension to his popular support vector machine (SVM) method. While the underlying Math is rather uninviting, the concept itself is stunningly convincing. It it based around the the idea that the training phase for a learning problem can be supplemented and improved with additional data from a privileged information space. Intuitively, this can be described as similar to a teacher giving a student subjective explanations on top of the text book examples.
The surprising point is to see that this information does not necessarily make any sense in the actual problem space. A provoking example Vapnik used during his talk was an experiment where poems describing the graphically representation of 8 and 5 were used to improve the pattern recognition of those digits. Although those informal descriptions appear to be nonsense when one looks at the graphically oriented problem, the LUPI extension of SVM is actually able to leverage them to improve the training phase of the recognition problem.
Details and several other examples related to biomedical informatics and statistics can be found in the related publication. I recommend at least skipping through it.