Premonitions. Psychic abilities. Crystal balls.
People have fascinated over prediction and uncertainty for centuries. Just think of the last movie you saw that showcased someone with those powers. Minority Report, Star Wars, etc. It’s imbued into our cultural psyche and zeitgeist.
Human beings have always had a fascination with the desire to predict the future. Being able to predict the future gives you unimaginable powers and inspire prestige and awe from those around you. With the ability to predict the future, we can wield infinite fortunes by being at the right place and the right time with the right people.
Imagine if you could predict how a particular stock is going to perform tomorrow. If you somehow knew that it’s going to go up 5% and you can do this with 100% accuracy for any given day, you’d be set for life.
The future is always an unknown and knowing the future with incredible accuracy means we have potentially limitless ways to act.
The Signal and the Noise
In engineering and science, the terms, Signal and Noise, are used to distinguish the purity of the information and its accuracy. The more the information is pure and accurate the more it is a signal. The more noise there is, the less accurate it is.
Just think of the static when you hit a channel on your radio or television that doesn't broadcast a clear transmission; the resulting sound and image is noise. It’s fuzzy, it’s crude, and it’s not consumable.
One man that’s been catapulted to the forefront of late is Nate Silver, who’s been deemed capable of predicting the future. His rise to this pedestal came from his uncanny ability to predict the 2008 Presidential Election with some measure of accuracy. Silver was able to correctly predict the outcome of 49 of the total 50 state elements for the 2008 election. And again in 2012, Silver was able to accurately predict the outcome of the elections for all 50 states.
Silver describes part of his methodology and theories in his book, The Signal and the Noise – Why so Many Predictions Fail and Some Don’t.
It is a fascinating read and I encourage everyone out there who is interested in making accurate predictions to start by reading his book.
As a mathematician, Silver deals with numbers. More importantly, as a statistician, he works in the world of probabilities. And while he posits that no one can predict the future, there are methods in which we can help better our chances.
- One is to use better data (and the ability to process loads of it).
- The second way is to simply model our predictions more accurately by introducing an uncertainty factor. And by uncertainty we mean the possibility and probability that the predicted outcomes could’ve been wrong, or that the data could’ve been misleading in the first place.
So following these general rules, what can be do today to improve our prediction ability? Big data is part of the answer to that. Corporations and enterprise level organizations have been and are starting to utilize the full potential by tapping into data about their customers.
For example, Amazon.com uses past patterns in your internet viewing on their website to help build their “Recommendation” product list to you. It’s not rocket science if you’re able to track that you looked at you read a Margaret Atwood book and that you may way to buy another book of hers.
Predicting the Future with Google Trends
So Amazon collects data about their customers (with their consent of course). How do the little guys like you and me, get data like this?
There is another company out there that is the leader in information. You probably use them every day, mostly likely their search engine. What most people don’t know about is that they also happen to have other products out there for the layman to use as an aid to help parse through the noise and get some preliminary data on predicting the future.
One of these products is Google Trends.
Essentially Google Trend is a form of voting. Millions of people around the world vote on what they're interested in by typing into the Google search bar every second of the day. Their collective interests via their keywords can be useful as a powerful predictor of what is interesting to the populous.
What’s great about Google trends is that it’s able to visually graph out the specific keywords since 2004. This means you have a decade of data to work with.
What Google Trends allows you to do is to visually see ...well… the trend. You can see if the interest is waning or if the interest is building and this serves as an excellent database for what has happened in the past. Often trends go in one direction and then another. And of course past performance is not indicative of future performance.
Crystal Ball is in your Hands
As Nate Silver puts it, modern prediction and forecasting is both a science and an art. While the crystal ball remains an artifact of the imagination in literature and film, the science has moved prediction tools into reality.
By following Silver’s advice in his book and by leveraging Google Trends, we are closer to knowing the pulse of the world, right in palm of your hands.