Data decrepitude and guarding against drift

As a business operator, you wake up one day and realize that you're dreading work. Not because of the company's mission, not even because of your role, but because the drudgery of day-to-day decision making is consuming your calendar. Among the many activities:

• Creating tickets for the data team who may come back with an answer in ~15 days (after some long-drawn clarification threads on Slack).
• Working with four dashboards simultaneously, each of which were last updated ~18 months ago with definitions you have no idea about, hoping to construct a single cohesive narrative of what-the-hell happened to the business. The only reason you are here is because your bosses can't make head-or-tail of this as well (so it falls on you, a slightly more data-savvy user).
• Wrangling an SQL query that an engineer threw at you, just hoping you can use a new column or table that a Confluence page mentions (created by an employee that left the company 3 years ago) so that you have one additional cut of the data that may or may not be useful for next week's management meeting.
• Cutting and pasting visualizations from a dashboard into a deck, only to realize that numbers reported for the previous month in this update don't match the previous update, and the numbers in your "smart" and "v16_final" spreadsheet have REF errors with no way to trace back to source.
• Meeting with the data-guy, the engineer handling the CRM, and the project manager to figure out why CRM numbers don't tally with what you see in the dashboard.
• [...and many other such wonderful activities]

Done with the day (or rather, the day now being done with you), you wonder: how did we get here? What set of events could possibly have made you do these activities, rather than your day job of [growing revenue / cutting cost]?

Most chalk this up to corporate drudgery, the sad set of activities that creep in when you're working in a larger company. But that's not true, is it? What of the many LinkedIn posts that gloat about the shining improvements in the "modern data stack" that rendered these as problems of the past? What of the hundreds of thousands of dollars that your company spent on the transformation project that got us new and fancy BI tools which promised a single source of truth? Why did these advancements not materialize in making your life better?

It's because it wasn't one moment that got you here. It wasn't one failed decision, or one ill-informed investment. It wasn't architecture missed by one individual. It was a slow compounding of suboptimal decisions that aggregated over time. Just as no engineer will tell you whether one specific code change ruined their codebase and introduced tech debt, no single data investment (or lack thereof) got you here. It was the slow decrepitude, the incremental decisions that added one more step, one more workaround, one less thoughtful decision in the interest of speed and growth.

Data decrepitude is not random, or accidental. It's a structural problem that the company's composition, culture, and investments haven't thought about enough. It's the real villain in the data maturity movie.

So let's discuss how we got here, what needs to change, and what you can do about it.

How did we get here

Like we discussed, it wasn't one thing. In our experience, these are the common culprits:

• People & Org:

  • The company threw money at the problem by hiring a dedicated individual that "deals with data stuff" because people kept complaining that data is hard.
  • The entire company was (implicitly) expected to be well-versed with all data structures, rather than being careful with which access points to expose based on requirements of the role and the individual's skills.
  • No collaboration mechanisms to share best practices and learn from each other (around data, amongst other things).

• Process & Prioritization:

  • One-off requests gave analysts a false sense of progress, taking away time from the prioritization framework that makes good product teams.
  • No one spent the time mapping data actions to company priorities.
  • Data team adopted a service mindset to queries, ticket-in, change-out. No roadmap, just a backlog that needs to be addressed.
  • Dashboards were built as a cure for curiosity rather than as a medium that facilitates action, by people who don't need to take actions from its output.

• Tooling & Infrastructure:

  • Engineers asked users to run queries directly on production data.
  • Pipelines were built on vendor software, with proprietary technology that created a migration project with too many expectations, bound to fail in some regard.
  • No way to find something (safely, without exposing sensitive information).
  • Lack of traceability in why a number is the way it is.

Unfortunately, suboptimal action compounds. Every time someone "pulls stuff together" in a spreadsheet, the spreadsheet becomes more entrenched. It grows worse when numbers from the database yesterday don't match today. It's even worse when a dashboard is built on top of this spreadsheet, and the dashboard builder then publishes it for the whole company, for an ask that only served one of the leaders in a specific function. Before you know it, undoing the problem suddenly feels unattainable. And even when you do take action, it feels like swimming against the tide; data debt is asymmetric. Like an investment that's down 10% needing an 11% gain just to break even, cleaning up data mess takes disproportionately more effort than the shortcuts that created it.

What needs to change

It's difficult to rewrite history. So where do you go from here? Every organization is unique, and it's rare to have a one-size-fits-all solution that is specific enough. But here are strategies organized around the same three dimensions:

• People & Org:

  • Ask for governance parity when it comes to the wider application access strategy at the company. Frameworks like RACI may already be used when allocating access at a systems level, and the same thinking applies to data assets, platforms, and tools.
  • Push for data literacy to be structured, not assumed. Define which roles need which access points, and build onboarding around that rather than expecting everyone to figure it out.

• Process & Prioritization:

  • Adopt a Jobs-to-be-Done (JTBD) lens when building reports or pipelines: ask "what decision does this person need to make?" before building anything. (JTBD is a framework that maps work to the specific outcome a user is trying to achieve, rather than to feature requests.) This surfaces the most common denominators and reveals where data readiness can provide outsized returns.
  • Bake data debt into your projects rather than treating it as a standalone initiative with dubious ROI. When revamping your accounting system, include a "payroll reconciliation" report as part of the same project. Shared definitions then evolve alongside existing systems rather than lagging behind them.
  • Push the data team toward a product mindset with a prioritized roadmap, not just a service queue. One-off tickets should feed into patterns, not replace them.

• Tooling & Infrastructure:

  • Be extremely wary of new data investments where a vendor will lock you in and raise switching costs. If you are an ambitious company, your business will change. Flexibility to support evolving use cases is the hallmark of good technology.
  • Demand descriptions and documentation from the data artifacts made available to you. Bridging the gap between the technical domain and the business domain is a shared responsibility, and requires active prioritization.
  • Insist on data discoverability and traceability as baseline requirements, not nice-to-haves. If you can't find a data asset or trace a number to its source, it's a gap worth surfacing.

These are long-burn efforts, but incremental progress has a higher chance of stickiness than a one-off transformation project. The key is intention with systems rather than daily/weekly motivation. And the stakes are higher than ever: generative AI promises to automate much of what makes your day tedious, but only if your data house is in order. Decrepitude is the very thing that keeps those efficiencies out of reach.

What can you do about it today

While the changes above require organizational buy-in, there are several things you can start doing immediately without needing anyone's permission:

• Each time you make a decision that isn't supported by data, call it out explicitly. "My gut feel is..." makes for a more inclusive conversation and starts building a record of where data gaps actually live.
• Keep a "data aspirations" scratchpad. Every time you hit a dead end — a missing column, an inconsistent number, a dashboard with no owner — log it. This is a goldmine for your data team and creates a cohesive case for prioritization that ad-hoc requests never will.
• Find others like you who struggle with these problems every day, and talk about them openly as limiters of productivity. Tangible, shared examples are what pave the way for broader change.
• Get into quick 10-15 min huddles with someone you trust when wrestling with spreadsheets or dashboards. Data problems are deeply contextual, and a second pair of eyes can help you zoom out or debug fast.
• Annotate slides, spreadsheets, and docs with context data source, when it was pulled, what the metric means. It takes 30 seconds and breaks the cycle of tribal knowledge.
• Add links on your dashboards to operational systems so that dashboard usage goes hand-in-hand with actions in your CRM / ERP / etc.

Conclusion

The through-line is that thoughtful data investments were an afterthought, creating the perfect conditions for drift to seep in. One suboptimal decision becomes part of the "normal." Before you know it, you're staring down enough debt that requires extremely painful and expensive resets. Incurring some debt is unavoidable, but not having an eye for what good looks like is a recipe for failure.

The tools to fix this exist. The AI to automate the tedious parts exists. What's missing is the organizational will to treat data decrepitude as the structural problem it is, rather than something to route around. Let's start elevating the conversation, step away from the drudgery, and be the guardians against drift.

We've been doing data systems thinking for a long time, and know that it is as much a technical problem as it is a people problem. Book some time for a no-obligation chat with us here.

---

Idea credits: Thanks to this excellent Chasing Excellence post (and corresponding podcast) for the inspiration behind this post.