The Death of Excel (and other false gods)

It all started with good intentions. An early child of GenX computer software developers, Microsoft Multiplan was born into a free-range 1980s household, in response to Lotus 1-2-3. The proliferation that followed was strongly tied to the methodical take-over of Microsoft Windows into business and personal computing when Multiplan became Excel and was included as part of the “Office for Windows 95” package. Microsoft’s commitment to regular software releases, feature pushes, and bug fixes has maintained their dominance, and now virtually every cubicle worker in the world knows how to perform a few basic functions from within the software. While I cannot deny the utility the program has provided to so many people, Excel’s largely unsupervised coming-of-age story has come with a significant cost to businesses, namely the rampant siloing and distrust of data throughout the modern enterprise, not to mention an extreme amount of duplicated data. As Excel reaches its mid-life crisis and a new generation of software queues in the waiting area, it’s time to settle affairs, learn from mistakes, and prepare for the death of Excel.

In order to fully grapple with what went wrong, it’s important to understand the context into which Excel arrived. In 1980, computers would’ve been 16-bit DOS-centric machines used by fringe early adopters and computer science brainiacs. It would’ve been impossible to predict in those early years how ubiquitous personal computers would become, especially in the Excel-addicted realm of big business and financial services. This mass adoption was seen only as a great success to Microsoft and the Excel team during this time. Excel’s widespread use accelerated alongside, but never fully acknowledged, the more specialized technologies like relational databases and Structured Query Language (SQL). By the time Ralph Kimbal published his first instructional book on Data Warehousing in 1996, there were already over 30 million users of Microsoft Excel, a freight train adoption curve that stands at almost 1 billion users today. Ultimately, this Excel-lent train missed the boat on structured data.

So, what exactly broke down along the way? It’s difficult to substantiate that Microsoft did anything inherently wrong or somehow designed the program with a responsible flaw. Rather, the blame lies with all of us “users” and takes shape as a fascinating study in human nature. In 1982, calculating assets over liabilities was a job that could be done just as accurately with a pen, paper, and a predisposition to mathematics, so why was this software tool welcomed with open arms? At its core, Excel made math faster… and arguably less … math-y. Suddenly, a new cohort of desk jockeys were empowered to crunch numbers quickly, and even make basic charts and graphs, without breaking out the abacus, ruler, or compass. Who needs to know Trigonometry when you can plot a curve or a trend line in a few minutes with Excel? A big victory for the average knowledge worker, with a bigger drawback: decentralization. As we, the Excel users, have all strived to climb the corporate ladder and prove our value to respective stakeholders, we’ve all developed our own ways of profiling, cleaning, transforming, and presenting data as gospel truth with varying degrees of success and virtually no centralized accountability. There are some incredible Excel workbooks, one might say even Excel applications out there designed by a particularly ingenious person in a cube. But for every success there is a myriad of failure to add to the creative graveyard that Excel has enabled amidst its users.

How then should a new generation of software plan to succeed in Excel’s wake? Do we look to the likes of Power BI, Tableau, or Domo to carry this torch and improve on Excel’s mistakes? While these tools do support next-gen capabilities with data, the same problem exists in that the analysts who use them are free to forge their own pathways and create non-standard definitions of business logic and data quality. The solution lies in the careful delegation of freedom, creating ecosystems where creativity can flourish within certain necessary bounds. This responsibility falls to bigger players in the data tools space like Talend or Informatica, and to newer, more nimble tools like DBT, or Data.World to form the backbone of data accountability within organizations. Gartner defines these platforms under a few different umbrellas, such as: Data Integration Tools, Data Quality Solutions, and Master Data Management Solutions. The common thread is that this software aims to fill the gap of centralizing data cataloging, data transformation, and data privacy in a way that ties the output of the knowledge worker to a consistent, incrementally improving blueprint within their business.

What if every business question had a glossary of terms that could be defined and rated for data quality? How would it change your ability to analyze if you could track the lineage of certain data from source, to target, to presentation? How could we better protect patients and consumers by inherently understanding the impact of PII/PHI within our datasets? None of these functions are available out of the box with Excel, however, an aggressive implementation of Data Governance within an organization can mitigate these missing features. It has been said that Data Governance is between 80 and 95 percent communication, but who moderates these conversations in our businesses today? With the mountains of data available within the modern enterprise, our Analysts and IT Professionals are often drowning in the table-stakes tasks of storing and querying the data, having no time or training to educate and evangelize topics like Data Governance to their user base. It’s time for a change.

As we look towards its twilight years, what responsibilities do we have to prepare for the death of Excel? It will not come swiftly in the night, rather it will dwindle slowly in value, even as its user base continues to grow, until it becomes a legacy freeware platform, a monument to the wild-west days of data analysis. Users will be relegated to using it as a notepad to experiment and brainstorm, but gone will be the days of Excel-driven data marts, with nineteen different attachments of the same document floating ungoverned in your email inbox or poorly organized SharePoint repository. As to what comes next, there is ample time and space for one or more disruptors to claim succession to the throne, but all will carry the weight of educating and advocating for the value and impact of Data Governance to a cohort of desk jockeys all too comfortable building their feudal data castles devoid of necessary accountability. It all starts with good intentions.