Moneycontrol
HomeNewsOpinionThe reverse swing: Small data more important than Big Data

The reverse swing: Small data more important than Big Data

In the context of investing, the current fad for Big Data and machine learning seems overblown. Successful investing entails understanding basic probabilities well. While machines can be used to crunch these probabilities, the old-fashioned method of reading annual reports has worked pretty well.

July 19, 2019 / 15:54 IST
Story continues below Advertisement
Representative Image

Saurabh Mukherjea

Small data is big data in disguise. The reason we can often make good predictions from a small number of observations… is that our priors [prior experiences] are so rich. Whether we know it or not, we appear to carry around in our heads, surprisingly accurate priors about movies grosses… poem lengths, political terms of office…and human life spans. We don’t need to gather them explicitly; we absorb them from the world.”– Brian Christian & Tom Griffiths in ‘Algorithms to Live By: The Computer Science of Human Decisions’ (2016) [square brackets are our insertions]

Story continues below Advertisement

The Big Data craze on the London Underground

Last week, while travelling on the London Underground, I was whiling away a train journey reading the adverts pasted on the walls of the subway train. When I saw that most of the adverts for professional education courses centered around Big Data (e.g. “Learn Python in 7 days” or “Masters in Big Data” or “Advanced Courses in Statistics”, etc), I couldn’t help remembering how 20 years ago, the same adverts on the London Underground trains were for courses in Java, web design or graphic design.