One way to explore what trends may be emerging is to talk to people about what has them most excited or worried about the future. In the analytics and data science space, a recurring theme among experienced leaders is the concern of not being able to keep up with all of the rapid change taking place – both individually and as a team. New algorithms, platforms, data, business partners, and more are constantly challenging analytics leaders’ ability to stay current on everything they oversee.
The Rise of Complexity and Disruption
Until well into the 2000’s, the number of tools and platforms for performing analytics was relatively small. Virtually all analytic logic was coded using SAS, SQL, or (sometimes) SPSS. Most data use for analysis was stored in a relational database or (sometimes) a mainframe. The majority of analytics being pursued at major corporations involved classic statistical and forecasting models. Nothing was easy, but skill needs were concentrated in a few core areas. Analytics generalists ruled the day, and generalists filled roles from the bottom to the top of the analytics organization.
Given the past stability of the space, even executives who had not done hands-on work for a number of years were still mostly current (if rusty) and could still review and understand the code and analytical logic being created by their teams. Leaders were comfortable that they could stay on top of the details of what was going on. They could even jump in and get their hands dirty if they needed to! The generalist skills that leaders grew up with still represented the bulk of their team’s skill sets.
As we neared 2010, an explosion of complexity hit through a combination of, among other things, big data, open source, the cloud, and artificial intelligence. Suddenly there was more data, more algorithms, more tools, and more platforms than ever before. They were all evolving rapidly, and many were not mature. The analytics space was, and continues to be, disrupted heavily while simultaneously analytics was being used to disrupt business models.
This poses tremendous opportunities for analytics organizations, but also tremendous challenges. No individual, whether entry level data scientist or senior leader, can possibly keep up with it all from a technical perspective. There just isn’t enough time in a day to become an expert on all the data, tools, and technologies that were sprung upon us so quickly and surround us today.
The Impact on Analytics and Data Science Organizations
As a result of the complexity and disruption, analytics leaders began hiring larger teams, with a broader range of skills, and a lot of specialists. The breadth and complexity of the analytics and data science processes being built and deployed has evolved far beyond anything in the past. While productivity is enhanced with all of the pre-packaged functionality now readily available for use, deploying and scaling processes requires many pieces working well together and those individual pieces are often understood and managed by different people.
This causes major stress for analytics and data science leaders. They are now responsible for many varied and complex analytical processes. At the same time, there may be nobody on the executive’s team who truly understands how all of the technical details for a given process work from start to finish. Instead, different people understand distinct pieces of the process. For example, a data engineer might make available a data pipeline that a data scientist can then make use of. The engineer and scientist may not understand the details of what the other is doing, but simply understand what handoffs are required.
What Keeps Analytics And Data Science Executives Up At Night?
For the typical analytics executive who is detail oriented and likes a sense of control, the lack of end to end understanding is disconcerting, and it brings us to the question in the title of this blog – what keeps analytics executives up at night? Time and again, when comfortable that they can speak freely and in confidence, analytics executives confess their insecurity as it relates to keeping on top of everything their team is doing. They simply aren’t up to speed on all the latest tools, platforms, protocols, and techniques. Sure, they understand the concepts and know how things work at a high level – at a generalist level. However, they are no longer able to jump in and personally quality check all the work their team has done, nor are they any longer able to do most of the work themselves if they had to. There is simply too much specialist work now being incorporated. As a result, analytics and data science executives now must fully rely on and trust their teams.
While this level of trust may be common for some executive roles (most notably CEOs who can’t possibly know the details of what everyone does within a large organization), it was not common for analytics executives until recently. It is a very big adjustment because it is one thing to lead a team as an “Alpha” generalist resource who, like a good drill sergeant, everyone knows can still jump in and show folks how it is done. It is another thing to lead a team as a guide, coach, and mentor who helps the team go the right direction and set the right priorities, but who everyone knows can’t jump into the trenches to help do the dirty work.
Many analytics executives still long for the ability to learn everything their team is doing and to be current on all of the details. However, the smart ones have realized that this isn’t possible. Furthermore, it isn’t desirable. An executive is being paid to lead and that is where their focus should be. It isn’t a bad thing to understand many of the details, but that is what the team is there for.
To be successful today, an analytics executive should hire a good team and let them do what they do best. In turn, the executive must focus on guiding the strategy, managing the politics, and selling the team’s capabilities to the organization. The requirement today is no longer for an executive who is a technical star and also has some leadership skills. Rather, the requirement is for a leadership star who also has some technical skills. Failure to recognize and adapt for this difference will cause both an executive and an organization to pay a price.
Originally published by the International Institute for Analytics
Bill Franks, Chief Analytics Officer, helps drive IIA's strategy and thought leadership, as well as heading up IIA's advisory services. IIA's advisory services help clients navigate common challenges that analytics organizations face throughout each annual cycle. Bill is also the author of Taming The Big Data Tidal Wave and The Analytics Revolution. His work has spanned clients in a variety of industries for companies ranging in size from Fortune 100 companies to small non-profit organizations. You can learn more at http://www.bill-franks.com.
You can view more posts by Bill here.