Transforming your organisation to succeed with data: Part four

In this four-part series, we offer practical advice on how organisations can successfully realise value from data by running through the detail of our eight-layer data operating model.

In our final instalment, we address the seventh and eighth layers: ‘Data Technology' & ‘Financial considerations’. We will provide an overview of the challenges faced in these areas and explain the important factors to consider when designing and implementing your data operating model.

Read next: Whitepaper: Building a data-driven enterprise of the future

Data technology: Enabling change and realising value

Technology has brought about a significant change to how data is accessed, stored, analysed, managed, and maintained, making it easier to work with and derive value from. Cloud technology, for example, offers the on-demand flexibility needed for organisations to meet their data requirements and build easily adaptable systems. At the same time, data is becoming productised and being broken down into a series of “data products” available for use and re-use depending on consumer demand. Technology supports this with separation of storage and compute, the use of microservices that can be independently optimised and scaled, and off-the-shelf applications designed to plug in and perform immediately.

Data products can also benefit from the application of continuous integration and deployment (CI/CD) as well as provisioning and managing environments through an “infrastructure-as-code” process, automating setup, removal, and updates to environments as required. When evaluating your current and target technology landscape to support your data strategy, there are several key areas of consideration:

Data ingestion

Operational systems can be disparate and operate independently. Ingestion is the process of moving data from these sources into a data platform. Typically arriving from various sources, the data is often in different formats and volumes. This often leads to extract, transport, and load (ETL) rules being applied upon ingestion to ensure that data is in the same usable format. Gathering this information will help to drive the technology selection process that enables ingestion.

General data storage

We are generating more data than ever before. The right storage options depend on the characteristics of the data and how it is to be consumed. Object data (e.g. photographs) may require one solution whilst transaction-oriented systems and semi-structured data storage will require different database solutions. For data that requires parallel analytical processing, a data warehousing solution may be required.

Data cataloguing

Cataloguing and making data visible through the use of metadata (contextual data) can help data analysts, engineers, and scientists find and understand data sets that are relevant to specific business operations through the use of data management tooling. Data stored and catalogued in the right way can be accessed by other analytics tools to support interrogation.

Data analytics and visualisation

Analytics involves bringing data together to create new business insights, discover trends, patterns, and data correlation. We can perform data analysis by slicing, joining, and aggregating the data, or leveraging advanced techniques such as machine learning and natural language processing. Presentation of analysed data in a way that enables patterns to be drawn, data correlation to be monitored, and dashboards published to support business decision making can be very powerful and insightful. The intelligence can be used to improve decision making and increase the agility of the organisation in meeting changing market demands.

Data product assembly

With high demand for data and the need to deliver data quickly, standard data product patterns and common services could be adopted to provide building blocks that enable fast product assembly whilst maintaining product quality. Examples of common services that could be built and standardised include:

• Standardised data product development environments: A service that enables auto-provision and consistent tooling. The service includes source control for tracking and managing changes to code/data, a deployment pipeline to automate product build and deployment in a consistent and secure fashion, artefacts storage and management, and a standardised service to store, version, and deploy artefacts.

• Ingestion of files and streaming: A service that allows the configuration and receipt of files from, for example, SFT software. This type of service can also support ingestion for data streaming, enabling consumption, storing, and configuration capabilities (tagging, alerting, decrypting, etc.) of data.

• Monitoring, logging, and alerting: A service that includes collection and ingestion of logs and information directly before submission to a centralised infrastructure. This also allows aggregating, filtering, searching, and configuration capabilities. The service can also provide monitoring analysis including searching, viewing, and filtering capture of performance metrics, whilst enabling appropriate users to receive alerts and notifications triggered by configured criteria. This also includes raising, viewing, and filtering alerts.

• Secrets management: A service for managing digital authentication credentials including passwords, keys, digital signatures, and encryption. It also includes enabling, disabling, tagging, and revoking functionalities.

• Data privacy: A service that could include data masking, de-identification for data at rest and in stream, and data classification capabilities. Identification, tagging, and classification of PII, PCI, regulatory and other sensitive data, as well as exposing masked and un-masked view and read-only and read-write access, protecting and preserving data.

Financial considerations : Investment, monetisation, and opportunity

Some financial considerations associated with a data operating model include:

• Investment in architecture and technology that will drive growth in the products and services offered by the organisation.

• Managing cost of data processing and storage, including the management of periodic CapEx relating to IT infrastructure investment or OpEx expenditure with potential variation in costs as data processing and storage demands fluctuate;

• Monetising data products and selling data and insights as part of the organisations commercial offering;

• Managing financial risks and prevention of exposure to fines and reputational damage for data protection or security breaches;

• Considering data governance as an enabler and not an overhead by delivering value through better data management, decision making, controls, and guidance.

A business case which includes investment in technology is likely to be the first step to both releasing value or managing risk associated with data and can lead to operational changes. For example, investing in cloud technology can bring about a change in the budget, funding, and control of costs of TCO (Total Cost of Ownership). Traditional IT finance models largely focus on predictable CapEx spending & infrastructure, whilst cloud models rely on less predictable, OpEx-based, consumption-led spending. While most cloud transformations seek to lower IT costs, investments can rapidly escalate and should be carefully managed in order to maintain control. This may include the need to have people dedicated to managing ongoing costs.

The business case may also consider how data can be used to deliver value to increase sales, analyse processes to improve efficiency, sell data insights to customers, and empower the salesforce with enhanced existing customer data. It can also address how new customers might be targeted with marketing and advertising data and consider the data itself as an asset that can be recorded on the balance sheet.

Managing data risks

The data operating model must ensure that the organisation manages financial risk through strong data governance, robust data processes (for acquiring, validating, storing, protecting, and processing data) and solid data security to prevent data breaches and cyberattacks. Without this, the organisation could be subject to regulatory fines, legal costs, lost sales/ reduced profit and loss of data and business continuity, resulting in damage to reputation and brand.

As such, the following data related vulnerabilities should be considered and addressed:

• Data deletion failure: Attempts to remove data has failed creating risks particularly where the failure has gone undetected.

• Data misuse: Employees may take advantage of resources or privileges for a malicious or unintended purpose. Included in this category are policy violations, and use of data for non-approved purposes.

• Data leakage: Unauthorised electronic or physical transmission of data or information from within a company to an external destination or recipient.

• Hidden data: Data that is either not used or no one is aware it is collected. Storing this data can unnecessarily open up security risks, compliance issues, and storage concerns if data isn’t catalogued and managed appropriately.

• Supplier managed data: Data is put at risk when a change from one supplier to another leads to challenges or limits on data transfer. The movement in data, applications, infrastructure, resources, or knowledge from one supplier to another may not be sufficiently covered within contracts, leading to cost increases and delays.

• Security attacks: Cybercriminals look for weaknesses in security, ranging from unpatched software to employees becoming subject to a social engineering attack. These weakness in security put data at

A preventive approach must be taken to address each of these risks. Backing up data and ensuring failsafe storage is part of the solution. An organisation must ensure that their people are educated in and confident enough, to identify scenarios where these risks may occur. This could include testing the organisation by, for example, running staged internal phishing attacks or by running a mock data breach drill. Business and technical processes must be designed, implemented and continually reviewed to either remove the potential for the risk to occur or to make sure that appropriate checks and monitors are in place to identify a potential issue. If issues are found, corrective action must be applied efficiently, root causes identified and lessons learnt to ensure the same issue does not occur again.

In a nutshell

Data is becoming productised and is being broken down into a series of “data products” available for use and re-use across an organisation. If implemented carefully, Agile frameworks and cloud technology used in software development are applied to data product development with all of the advantages of flexibility and scalability. Having standard patterns and services in place to support data product development enables assembly of products at speed. Alongside the opportunities to realise value from data, a preventive approach must be taken to managing all potential data risks. This should start with education on what these risks are and how they can be avoided or mitigated.

Closing words

In this series, we have covered a range of areas that organisations need to consider when implementing a data operating model. Although there is a lot of ground to cover, the good news is that those who are prepared will capitalise on data and its benefits in the most effective way.

If you are looking for a partner to help with your transformation, our 30+ years of experience delivering and assuring such programmes strongly positions us to assist you, wherever you are in your journey.

Please get in touch today.