SaaS Delivery Maturity

With modern tooling SaaS can be up and running very quickly and easily but the journey to maturity can be a long and winding one. My goal with this overview is to run through the operational differences between a mature and immature service and the key areas to be considered to progress through that divide.

Defining the S in SaaS

To start with a simple definition though, an organisation may have a readily identifiable service which is their principle offering. Beneath the hood there may be many independent applications which operate together to enable the offering. A modern architecture known as micro-services is commonly used which results in a proliferation of services and from a maturity point of view they could be at varying points along the maturity spectrum. Whether it makes sense to consider each micro-service independently or to consider them collectively will be context specific. Needless to say if a collective approach is taken the system is only as strong as the weakest link so scoring progress should recognise that.

What are the attributes of a mature service?

That would include the service covering off the requirements of all functions of a modern business (finance, marketing, commercial, product, engineering etc) whilst facilitating the growth and constant change a service is expected to accomodate, being dependent on unreliable and changing 3rd party services to run, in as efficient and cost effective manner as possible. For a technical operational service owner the focus will most naturally be on deploying new releases (’the build pipeline’), performance, cost and (if we’re lucky) security. These are all critical concerns and ones the technical service manager is the principle stakeholder for, but at a holistic organisational level, there are other questions and demands the business will have of a service which it’s important to also cover off.

Creating a roadmap whilst the journey is underway

Defining a specification for service maturity for a live system might feel like working out a roadmap for a trip whilst already underway but it’s a natural approach as it can be optimally done only with an understanding of actual operational demands. It is an important step for every organisation in progressing to an operational state which can provide certainty in a highly dynamic environment. Each organisation will have a different view on what maturity looks like and will put different weights on different elements of that map. And of course, these operational requirements will change over time so they should be maintained on a regular basis.

Modern cloud providers have their guidance for what well-architected services look like (such as AWS or Azure) and although these are useful for sure they only provide the cloud view of service maturity and not one which will generally cover off the needs of business operations.

Is there a correct level of maturity?

Actually no, the progress to maturity should simply match what’s appropriate for that service in that context. Getting to maturity consumes time and money, and like any investment doing so too early can result in wasted resources. It’s easy to anticipate that as a product matures from prototype to launch and then scale, we expect the delivery maturity to also progress in a similar cadence.

The 8 Focus Areas

What follows is my proposed starting point for that roadmap to enable the pathway to maturity. There are 8 areas of focus and a number of elements within each that could be scored to create a heat map of progress for each service or overall for the organisation.

1. Functional
2. Monitoring and Alerting
3. System Reliability (SRE)
4. Logging
5. Build Pipeline
6. Cost Management
7. Business Continuity (Backup and Restore)
8. Security / Compliance

Invariably progressing to maturity will require an investment of time or the utilisation of services which both cost money, and the ROI of that will vary case by case. So my preference is to track two metrics for each item:

• What is the current status of this item?
• What is required to progress this item forward?

Focus Area 1: Functional

Why?

To inform people working on or with this service what its goals are in order to provide context for its operational requirements. It is these operational requirements that allow many of the following focus areas to be scored as to whether they meet the required standard.

Focus Area 2: Monitoring and Alerting

Why?

In order to have confidence the service is fulfilling its commitments there must be oversight and then action taken when that is not achieved.

Focus Area 3: System Reliability Engineering (SRE)

Why?

This focus area probably requires the least explanation except to clarify that I have chosen to take a narrower interpretation than what some definitions would describe. Populated by Google, like DevOps, SRE can be a job title, an org unit, a program or initiative, or just a collection of practices or goals related to reliable service delivery. In this narrower interpretation the specification is simply defined as follows:

Focus Area 4: Logging

Why?

Application and system logs have been a standard part of computing since its origin. They are a record of activity and are used for audit/compliance, investigating bugs or user behaviour, and as part of understanding current system behaviour to enable SRE. Most commonly due to the operational characteristics of modern logging (e.g. high volumes and combining streams of data from multiple sources) and to enable improved creation of metrics and alerts logs are consumed through an instrumentation tool. The challenge will invariably be capturing information needed to resolve a future problem without storing unnecessary content that costs money and reduces the ability of the tooling we build around those logs to work effectively.

Focus Area 5: Build Pipeline

Why?

The ability to release code with minimal effort, risk and as quickly as possible is an important feature for SaaS businesses to enable growth. By reducing the overhead of doing a release you can reduce the delta of change which justifies a release, and therefore reduce the risk of every release. And then by releasing more quickly you can deliver value sooner, get feedback more quickly and more generally facilitate agility across the business. The processes that formalise best practice are known as continuous integration and delivery (CI/CD). They are enabled by trust in the delta which is traditionally driven by QA-automation practices and to a lesser extent the ability to control change independent of code through feature flag tooling.

Focus Area 6: Cost Management

Why?

It should go without saying that cost management is a critical element of SaaS delivery. The ability in a modern cloud native business to adjust infrastructure settings at will provides tremendous flexibility and power to solve problems. It also provides the ability to drive cost quickly and in unexpected ways so care must be taken to ensure the operational setup is in alignment with the business strategy that is being pursued. This can come down to a risk assessment where no (or low) risk costs significantly more than a higher risk setting and finding the right level should be done with transparency between all stakeholders in order to enable the business strategy and achieve the SRE settings that have been agreed to.

Focus Area 7: Business Continuity

Why?

Related to SRE is the more specialised area of understanding what infrastructure and services enable SaaS delivery, developing scenarios under which they could fail, and then building out and testing plans to allow for continuity of service delivery through these scenarios.

Focus Area 8: Security & Compliance

Why?

SaaS typically depends on the key-tenets of information security (Confidentiality , Integrity and Availability) to operate well so there is a natural synergy between the two. Building trust with customers or suppliers in order to achieve commercial goals requires ensuring security is taken seriously and is provable. For that reason companies typically adopt a certifiable and well regarded framework to achieve their security goals.

The mantra of a modern security focussed organisation is to “shift left” on security by building awareness with the team, and taking steps to consider security from the outset of product development. There are now many modern tools and processes which can assist to automate this which modern organisations should definitely be adopting.

The culmination of a suite of cloud security tools when delivered as a platform (which can be quite efficient and effective from an integration point of view) is known as a Cloud Native Application Protection Platform (CNAPP).