Overcoming Data Silos: The Biggest Barrier to Energy Industry Digital Transformation by Rhys Jacob, CTO at D55

Expert View
January 7, 2026
January 7, 2026
|
1
min read
author

Energy organisations are not short of data, but they are short of clarity, speed, and confidence in the numbers they rely on.

Billing systems, CRM platforms, settlement systems, and trading and forecasting tools generate data continuously. Despite this, leadership teams are still waiting days or weeks for answers to basic commercial questions around cost, performance, risk, and growth.

In a market defined by volatility, regulation, and margin pressure, that delay is no longer a minor inconvenience. It is a direct threat to profitability and competitive position.

Why Data Silos Persist in Modern Energy Organisations

Most energy data estates were not deliberately designed. They evolved over time.

New systems were introduced to solve immediate problems such as billing upgrades, regulatory tooling, customer platforms, and trading systems. Each was optimised locally, and rarely architected as part of a coherent, end to end platform.

Over time, this has created a fragile operating model built on:

  • Ageing legacy billing and operational systems
  • Disconnected SaaS platforms procured by function rather than strategy
  • Manual processes used to stitch systems together
  • Spreadsheets filling the gaps between systems of record

The organisation continues to function, but insight does not flow.

The Hidden Cost of Fragmentation

When data is fragmented, the symptoms are consistent across the sector.

  1. Decision cycles slow down
    Insight lags behind events, forcing leaders to act on partial or outdated information.
  1. Product and platform backlogs never shrink
    Engineering capacity is consumed by reconciliation, data fixes, and brittle integrations rather than roadmap delivery.
  1. Manual effort becomes structural
    Skilled teams spend their time checking, correcting, and reconciling data instead of creating value.
  1. Operating and cloud costs rise without clarity
    Spend increases across platforms and pipelines, but visibility into cost drivers and value creation does not.
  1. AI initiatives stall before production
    This is not because models are unavailable, but because data quality, lineage, and governance are not production ready.

The result is a familiar state of being data rich and insight poor. When this happens, first mover advantage is often lost before it is even visible.

Why Traditional Data Strategies No Longer Work

Traditional data strategies assume:

  • Structured and predictable datasets
  • Batch based reporting
  • Centralised ownership
  • Static business logic

Modern energy organisations operate in a very different reality.

AI, advanced analytics, and automation require:

  • Event driven and near real time data pipelines
  • Access across both structured and unstructured data sources
  • Clear data lineage, ownership, and measurable quality metrics
  • Governance and security enforced by design rather than process

Without these foundations, AI does not quietly underperform. It amplifies inconsistency. Models can produce outputs, but teams do not trust the inputs, and without trust, nothing scales.

What an AI Ready Energy Data Foundation Looks Like

An AI ready data foundation is not defined by the tools chosen. It is defined by how data is treated across the organisation.

Across energy organisations making real progress, we consistently see the following characteristics:

  • Unified data platforms, often implemented using lakehouse style architectures that can store, govern, and serve structured, semi structured, and unstructured data from a single foundation
  • Event driven ingestion and processing, replacing fragile, batch heavy ETL pipelines with automated workflows that keep insight close to real time
  • Clear data product ownership, with domain aligned teams accountable for data quality, semantics, and availability rather than just ingestion
  • Built in security and compliance, including encryption, least privilege access, automated PII detection and masking, and auditable data lineage
  • AI ready data access patterns, allowing analytics, automation, retrieval augmented generation, and future agent based workflows to operate safely

When these foundations exist, AI becomes a force multiplier rather than a risk amplifier.

From Data Risk to Commercial Advantage

We see two distinct patterns when energy organisations approach AI.

Some organisations rush to adopt models without addressing fragmentation. They generate proofs of concept and experiment with large language models, but struggle to move anything into production because trust, governance, and operational readiness are missing.

Others work backwards from business outcomes. They unify their data, establish governance, and introduce AI only once trust, lineage, and access controls are firmly in place.

The difference between these approaches is not ambition. It is architecture.

When data and platforms are aligned, we consistently see:

  • Decision cycles compress from weeks to hours
  • Manual assurance and reconciliation replaced by automated, event driven workflows
  • Forecasting shift from lagging indicators to predictive insight
  • Cloud costs stabilise as architectures align to workload patterns
  • AI solutions move from experimentation into sustained production use

Speed compounds, and confidence follows.

Read our Energy 2025 Whitepaper

Read our whitepaper on 'The Opportunity for Data-Led Innovation in the UK Energy Industry'

Platform Modernisation Is Infrastructure for Decisions

This is not about doing data better for its own sake. It is about building infrastructure that supports decision making.

When applications and data are unified on a modern, well governed platform:

  • Data lineage and observability restore trust in the numbers
  • Human in the loop controls and feedback logging improve data and model quality over time
  • Reporting, analytics, and AI draw from the same trusted foundation
  • Teams can measure not just performance and cost, but also accuracy, consistency, and adoption

The platform stops constraining strategy and starts amplifying it.

How We Approach This at D55

At D55, we do not treat data strategy, platform engineering, and AI as separate initiatives. They operate as one system.

Our work in energy is built around three integrated services:

  • Application Modernisation, re architecting legacy billing, settlement, and operational platforms so software becomes a commercial accelerator rather than a bottleneck
  • Data Foundations and Engineering, building unified, real time data platforms with governance, lineage, and automation built in
  • Managed Platform Services, providing observability, FinOps, Dev-SecOps, and continuous improvement so platforms remain secure, cost controlled, and production ready as the business evolves

Real world proof from the Energy Industry

1) Read the Amber Case Study.

2) Read the Switch 2 Case Study.

Where to Start: Clarity Before Commitment

Most modernisation programmes fail because they start with assumptions.

That is why we begin with a Data Strategy Diagnostic, a short and focused engagement that:

  • Maps the current data and integration landscape
  • Identifies blockers to performance, automation, and AI readiness
  • Surfaces hidden cost centres and technical debt
  • Defines a lowest risk, highest return roadmap aligned to business outcomes

No jargon. No technology for its own sake. Just clarity before commitment.

Final Thought: Architecture Is Strategy

Energy organisations do not lose momentum because they lack ambition. They lose momentum because their architecture cannot support it.

Fix the silos and everything accelerates, including decision speed, delivery pace, cost control, innovation, and market leadership. Leave them in place and every new initiative becomes slower, more expensive, and carries more risk.

If this feels familiar, I am always open to a direct conversation about data foundations, platform modernisation, and how clarity becomes a strategic advantage.

Contact us today!

Rhys Jacob
Chief Technology Officer
D55

Heading