Analytics and data science have been disrupting business processes and how companies compete for many years now. There has been a lot of focus on the challenges that business and technology organizations face when attempting to incorporate even more sophisticated and pervasive analytics processes into the mix, and all of that is true.
However, as we head toward 2020, it is becoming increasingly clear that analytics and data science organizations must recognize, acknowledge, and react to the fact that they themselves are being disrupted even as they aim to disrupt the business. This blog will discuss some of the disruption that analytics organizations face as we head into 2020.
Organizational Disruption
Most large companies have a rapidly expanding analytics and data science organization. With that expansion comes stress across all aspects of managing the organization. New job roles are required, new layers of management are necessary, more procedures need to be put in place to ensure consistency of both approach and quality of output. These are all big challenges that require much time and attention from everyone up and down the chain. It is quite uncomfortable to be in an organization that is growing faster than the current structure and processes can effectively handle. Yet, it will take time to implement the changes required to adapt to today’s new reality. Thus, the organizational disruption can be expected to last for some time.
Methodological Disruption
It’s no secret that the breadth and depth of analytical methods has rapidly expanded in recent years. Just 10 – 15 years ago, most analytics at large organizations were traditional predictive and forecasting models built on top of traditional structured data. That is no longer true. There has been an explosion of growth within the realms of machine learning and artificial intelligence. There have also been a wide range of new domains opened up for analysis, such as the internet of things (IoT) enabling analytics where there was no ability to pursue them before. In short, analytics and data science organizations are being challenged to quickly integrate a wide range of new methods into their portfolio while still maintaining and building on the portfolio of traditional analytics. This, too, causes disruption to the organization and it raises additional issues of prioritization and new skill requirements.
Data Disruption
Helping contribute to both of the previous areas of disruption is the complete disruption of the data being used for analytics today. With the rise of big data, data volume and data types have exploded. Marketing analytics is no longer based on just sales and campaign response history. We now have web browsing data, app data, sensor data, and more to add into the mix. The capture of much of this new data is challenging enough. But, figuring out how to cleanse, prepare, and analyze it all is a massive effort. These new data sources have been disrupting analytics organizations for years, and there isn’t an end in sight.
Regulatory & Policy Disruption
As analytics has gained visibility, practitioners have been forced to disrupt how things are done due to new and stricter regulatory and policy requirements. GDPR, CCPA, and other legislation requires organizations to be more transparent about what they are doing and to provide more justification for how and why models were built. There is also internal pressure to have better documented, more robust checks and balances that ensure ethical processes and outcomes. This all requires time and attention from the analytics organization moving forward, while many older processes have to be updated to match the new requirements. This is a painful type of disruption that can only be handled with an intentional approach.
Technological Disruption
Last, but certainly not least, analytics and data science organizations are seeing rapid and massive change in the tools and technologies required to meet their goals and mandates. No longer are a few common desktop or server tools hitting a data warehouse enough. Today, a wide mix of commercial and open source tools are required for success. In addition, these tools must be applied across a range of data repositories both local and cloud based. Finally, the supporting technology behind the development and deployment of data science and analytics processes is evolving rapidly with the rise of technologies like containerization and serverless protocols. Keeping up with all these new options would be hard enough without all of the other disruptions in play.
Handling the Disruption In 2020
The main takeaway here is that if you, as an analytics and data science professional, are feeling a bit overwhelmed . . . you should be! Disruption is all around you even as you focus on disrupting your business partners. It is important to recognize and adapt to the disruption you are experiencing because to ignore it is to fail. Similarly, it is important to ensure that your business and IT partners understand the level of disruption you are going through. They certainly see the ways you are pushing their limits but may not understand how much your own limits are also being pushed. You’re all in this together.
There is no way to avoid or stop the disruption outlined in this blog, but is it possible to work through it as long as you and your organization are focused and intentional about identifying and then planning for how to address all the dimensions of disruption being experienced. It will sometimes be painful and chaotic but over time, progress will be made, and you will successfully transform your organization. I wish you good luck and much progress toward that goal in 2020.
Originally published by the International Institute for Analytics
Bill Franks, Chief Analytics Officer, helps drive IIA's strategy and thought leadership, as well as heading up IIA's advisory services. IIA's advisory services help clients navigate common challenges that analytics organizations face throughout each annual cycle. Bill is also the author of Taming The Big Data Tidal Wave and The Analytics Revolution. His work has spanned clients in a variety of industries for companies ranging in size from Fortune 100 companies to small non-profit organizations. You can learn more at http://www.bill-franks.com.
You can view more posts by Bill here.