Which KPIs should you use when outsourcing software development?

 KPIs for outsourced software development help organizations track productivity, software quality and client satisfaction when working with an external development partner.  When developing software with an external IT partner, maintaining control throughout the process is crucial. What KPIs can you apply to ensure the success of outsourced software development? Nearshore software developer NetRom and Software Improvement Group (SIG) explain in this blog post how to effectively manage this process.

NetRom and SIG have been working together for many years for various clients and share a common vision on KPIs for outsourcing software development. This article will help guide you in thinking about KPIs when managing both the process and the final outcome of software development by an external party. We will cover the productivity of the development process, code quality, the health of the development process, and client satisfaction.

First of all: KPIs are never an end in themselves, but should provide insight into the impact of your actions. To what extent have you achieved the intended goals? Useful KPIs give you the ability to make adjustments. Making IT-related KPIs practically useful sometimes requires extra thinking about the actual end result you want to achieve. Instead of measuring "application uptime," you might also think of "no flight delays due to IT," as mentioned by Marco Gianotten of Giarte in the IT Xperience monitor 2021.

KPIs for productivity in outsourced software development 

We begin with the productivity of a software development project – the speed at which software is created. Productivity is crucial for more strategic business goals, such as a quick time-to-market. You can also monitor what is delivered compared to the plan and keep track of how much functionality is being completed. These variables are relevant to both the product owner and the development team.

How do you measure productivity?

When measuring productivity, you'll get the most benefit from three KPIs in practice: sprint burndown, team velocity, and release burndown.

• Sprint burndown shows how much work remains during a sprint. It helps teams monitor whether work is progressing as expected and identify deviations or scope changes early.

• Team velocity indicates how much work a team completes on average per sprint, often expressed in story points. It can support planning within the same team, but it should not be used to compare different teams directly.

• Release burndown provides insight into the progress of the release by comparing the remaining work at the end of each sprint against the planned workload. The remaining work can be expressed in story points, ideal days, team days, etc.

KPIs and metrics for software quality 

The second group of KPIs focuses on the quality of the software. Some managers believe that high software quality conflicts with functionality and speed, but the opposite is true. Poor software quality can lead to higher maintenance costs and a longer time-to-market. Maintainable, high-quality code can contribute to lower maintenance costs and a lower total cost of ownership (TCO). Many metrics are general measurements, unlike KPIs, which are specifically focused on strategic objectives. Metrics can be measured automatically, allowing developers to receive alerts if threshold values, such as those related to quality, are exceeded.

How do you measure software quality? 

ISO/IEC 25010:2023 defines an international product quality model for ICT and software products, consisting of nine quality characteristics and related subcharacteristics. ISO/IEC 25002:2024 provides a framework and guidance for defining and applying quality models. SIG is the first fully certified company in the world to measure according to the ISO 25010 standard. Here, we present the key variables (and sub-variables) that play a role in measuring and evaluating software quality.

Maintainability 

Maintainability, the ease with which software can be modified and maintained, includes analyzability, modifiability, testability, modularity, and reusability. These components can be seen as a representation of the stages involved in performing maintenance tasks.

Other quality aspects within ISO/IEC 25010 

In addition to maintainability, ISO/IEC 25010 includes other quality characteristics such as reliability and flexibility. Relevant aspects include: 

• Availability: the degree to which a system is operational and accessible when required. 

• Faultlessness: the degree to which a product performs its specified functions without faults under normal conditions.

• Fault tolerance: the degree to which a system continues to operate as intended despite hardware or software faults. 

• Adaptability: the degree to which a product can be adapted to different or changing hardware, software or usage environments. 

•  Replaceability: the degree to which a product can replace another product for the same purpose. 

Performance as part of software quality 

Performance plays a crucial role in software quality. Consider the following factors:

• Response time: Does the speed and processing capacity of the system meet the required standards?

• Capacity: What is the maximum growth that can occur within the response time requirements?

• Resources: How many resources are needed for the software to meet the performance requirements for response speed and capacity?

Part of ISO 25010 also includes the usability of software. This refers to the extent to which a solution can be used by users to achieve specific goals. Along with effectiveness and efficiency, user satisfaction is a key aspect of usability.

Additionally, essential are the security features. Key components include the integrity of the software (the degree to which software can prevent unauthorized access to applications and data), as well as non-repudiation and accountability. These concepts refer to the ability to demonstrate actions or events and the irrefutable ability to trace actions back to the entity performing them.

KPIs for software development process health 

In addition to the productivity of development teams and the quality of their delivered products, it's also essential to evaluate the software development and implementation process.

• Lead time refers to the time between the start and execution of a process. It also reveals something about the partnership, highlighting areas where delays may occur or need to be discussed and resolved.

• Cycle time indicates how much time a team spends working on a specific task. It’s a measure of how quickly the team delivers new functionality to the client.

• The third and final health indicator is flow efficiency, which refers to the percentage of time spent actively working on your backlog items compared to the total available time. A low flow efficiency may indicate bottlenecks in the process - such as a developer having to wait before completing a task.

KPIs for client satisfaction in software outsourcing 

In our opinion, there are four key factors that determine client satisfaction in a partnership.

"Do what you promise" is the starting point. A commonly used project-management rule of thumb is that quality, speed and cost must be balanced against one another. Taking responsibility for commitments at the sprint level is how you ensure you deliver what you promised. To continuously monitor this, you can use a metric like sprint burndown.

The second success factor for client satisfaction is employee retention. High employee turnover can put project continuity and knowledge retention under pressure. 

The third factor is domain expertise. Having and applying domain knowledge contributes to the creation of software with an excellent user experience. Our tip: ask your development partner for proposals to test the domain expertise of developers.

And finally: never compromise on quality. This contributes to future-proof products that are reliable, secure, and easily adaptable.

How to use KPIs with your software partner 

Ultimately, in a successful partnership, you should jointly decide which KPIs to apply. Your product owners should be able to work with the selected metrics and KPIs. The team must also be able to validate and interpret the metrics related to code quality and architectural decisions in collaboration with an architect. It's wise to make agreements with your development partner regarding the standards applied for code quality. You can supplement this with external audits or certifications like SIG TÜViT certification. Lastly, make use of KPIs in ongoing dialogue, sprint retrospectives, and sprint cycle evaluations.

Download the whitepaper on KPIs for outsourced software development 

Want to use KPIs more systematically when managing outsourced software development? This whitepaper from NetRom Software and Software Improvement Group (SIG) provides a more detailed framework for productivity, software quality, process health and client satisfaction. Fill in your details to receive the whitepaper.