We are, all of us, in unprecedented times. Still many folks looking for guidance on how to manage through these unprecedented times are looking for lessons in the financial crisis of 2008, and the recession that followed. Looking to 2008 and 2001, as well as (and man do I hate saying this) the Great Depression and World Wars is worthwhile. Still it’s good to keep an old adage in mind – generals are always fighting the last war. This adage is likely centuries old but really gripped the consciousness of everyday folks when the cavalrymen of World War I met a new menace, the tank. Technology made this adage truer than it ever was before.
I have no unique insight into how the pandemic will be defeated nor how bad the economy will be, let alone when we get back to some kind of new normal. Still I would like to share some key areas where, in 2020 the field of data and analytics is massively different than it was in 2008. And all of us in the field should think about how to leverage these differences so the pandemic is defeated more quickly (where a ton of great work is being done already), so that the economy suffers a little less and so that we find a healthier and happier new normal a little sooner.
For starters, we have more and better data, more and better compute power, increased focused and knowledge across enterprises, and better ways of working.
Not only do we have more data, we are smarter about how we understand that data. The four or the five V’s of data have been established for a while and certainly pre-date 2008, and in 2017 George Firican published 10 V’s providing content and context to the existing five, and added five more, including the very relevant “volatility”. This development of thinking was an outcome of the understanding that we collectively developed in the mid 2000’s, much of it after 2008. Since 2008 we have had enormous growth in sensor, geolocation and social data, to name three. One small example, in 2008 Facebook had 100 million, mostly US, mostly younger users and now has 2.5 billion spread across age groups and demographics. An area of where this growth in volume and variety simultaneously can be used is the application of predictive algorithms at lower and lower levels of geographic specificity, whether for disease tracking or supermarket shelf stocking.
Compute power has grown so quickly for so long we take it for granted. And that growth in capacity has come with a massive decline in cost as well. Storage capacity and cost are not the only things that matter in large scale analytics, but they do play a big role in the cost and speed to deploy and scale analytics. The cost to store a gigabyte of data has fallen between 60% and 80% since 2008. This cost reduction does not even include the lower operational costs and near infinite scaling potential of cloud computing (just ask Zoom). And while we are in the 2008 way back machine looking at cloud, keep in mind that the 800-pound gorilla of cloud, AWS, then called Elastic Storage, had revenues of approximately $25 million in 2008. In 2019 it was $40 billion. Clearly lower cost storage and more powerful computing is good for Amazon and it’s a boon for analytics, allowing for things like telemedicine to ensure that only those who really need to visit doctors in-person, and to architect the data exhaust of those calls to track COVID hotspots. Even those that don’t use telemedicine can leave a data trail for analysts to track emerging patterns of disease outbreak.
This data knowledge and the exponential improvements in computing have corresponded with a growth in maturity around the field of analytics, enabling more people to learn essential data skills and more companies to leverage those skills for competitive advantage. It was back in 2007 when IIA cofounder Tom Davenport published Competing on Analytics. So needless to say, very few were doing just that in 2008. Since then the growth in enterprise analytics has progressed, with the formalization of terms like data science, and the growth of educational programs to support the discipline. In addition to formal academics and printed books, evidence of the growth of analytics since 2008 is most clearly seen in the open source world, where things like Github and Kaggle have spread knowledge far and wide, and are currently an amazing resource in the current crisis. Still, as late as 2017, only 53% percent of large companies claimed some measure of success in leveraging big data. So, as a field we have come a long way, but we still have a way to go.
Inspired by software development, many data and analytics teams are oriented in product teams, or agile set ups, so that they can deliver quickly and improve and adapt incrementally. In the current environment where the situation changes constantly and the needs of the stakeholders shift with them, this orientation is ideal for responding quickly. Additionally, this set-up often allows ideas to surface from a variety of team members. And since we are in a time where historical knowledge can be detrimental, fresh perspectives are even more valuable. We at the IIA have covered this in both a whitepaper and a webinar, calling it “One of the 5 Areas to Obsess About”. (Note, with the sentence ending in a preposition, grammar is clearly not one of the other four areas). There are a lot of companies, including big established ones who have leveraged this approach, but I continue to be inspired by the effort to continuously redefine this way of working at Spotify, which coincidentally launched two weeks after Lehman Brothers filed for bankruptcy back in 2008.
As we search for insights into what our new normal will look like, it’s natural to look to the past for ideas, so long as we don’t forget all the great things we have seen and been a part of developing in the 12 years since we had to find our old new normal.
Drew Smith is the Executive Director for IIA’s Analytics Leadership Consortium (ALC) and has been with IIA since June 2019. The ALC is a closed network of senior analytics executives from diverse industries who meet to share and discuss best practices, as well as discover and develop analytics innovation, all for the purpose of improving the business impact of analytics at their firms. With close to 20 years of experience, Drew has worked on both the business side of analytics, leveraging insights for business performance, and on the delivery side of analytics driving the use of enterprise analytics. Before joining IIA, he led the Data Analytics and Governance team at IKEA’s global headquarters in Europe.
You can view more posts by Drew here.
Follow IIA on LinkedIn, Twitter and Facebook for more updates.