RSOS paper

Lingfei’s paper “Accurate wisdom of the crowd from unsupervised dimension reduction” has been published in Royal Society Open Science. In this paper it is shown that wisdom of the crowd, the collective intelligence derived from responses of multiple individuals to the same questions, is analogous to one-dimensional unsupervised dimension reduction in machine learning. This means that many of-the-shelf dimension reduction methods, such as good old PCA, can be repurposed as crowd-wisdom methods, usually with (much) better performance than existing default crowd-wisdom methods. Perhaps one of the more surprising results concerned the classification of skin images as being cancerous or not. As part of the hype surrounding deep learning, it was recently found that a deep neural network trained on 130,000 images was better at classifying a test set of 111 skin images than 21 individual dermatologists. However, we found that by doing a simple PCA of the predictions of these 21 dermatologists, they collectively outperformed the deep neural network. As The Economist put it in their recent ad, “not all intelligence is artificial”. In fact some of it is collective.