The Scale of Information Overload
August 21st, 2017
We have faster, easier access to information than ever before, but we are also generating information at a remarkable rate. The New York Times alone publishes about 230 pieces of content, plus several hundred stories from wire services every day. The Wall Street Journal publishes around 250 stories, and The Washington Post around 1,200 pieces of content. So at a conservative estimate, these three sources alone are publishing some 2,000 items every day – that’s 600,000 per year. There are hundreds of news outlets across the world – some 25 major newspapers in Canada alone.
Less established sources are also generating content at an extraordinary rate – Buzzfeed publishes around 6,000 stories a month, but it is the blogosphere that is really mind-boggling. Over 3 million blog posts are published every single day – and the number seems to be rising.
Add to that newsletters, whitepapers, books, podcasts, journals, conference proceedings, social media – and it is not surprising that the feeling of being ‘well informed’ has become elusive, even for professional researchers.
Limiting your research to specific subject areas helps a little, but the pace of change and the volumes of associated data is breathtaking. For example, if you want to survey the business landscape in Canada, you will find that there were almost 80,000 new businesses established in 2013, and the number appears to be growing each year. Even an extremely narrow area – cryptocurrency innovation, for example – moves at breakneck speed. There are now over 900 cryptocurrencies, all of which have come into existence over the past few years. Compare that to the mere 180 fiat currencies – a figure that has been fairly stable for decades (an exception being at the start of the 1990s, following the collapse of the Soviet Union, when newly independent nations established their sovereign currencies).
Keeping up with financial instruments is no easier. Even though there are only 60 stock exchanges in the world, the number of financial products is huge. The London Stock Exchange lists over 4,000 financial instruments (including 1,500 ETFs).
It is no wonder that researchers tell us they want to use the most cutting-edge techniques in machine learning and AI to help them select, sort, filter, and analyse the ‘news’.