Beyond gut feeling: a data-driven approach to developer experience

Matt Saunders

10 April 2025

5 min read

Matt Saunders

10 April 2025

5 min read

Learn how to improve developer experience with data, not just feelings. Measure pain points, DORA metrics, and gather feedback for software delivery success.

It's all very well saying that an organisation should focus on developer experience (DevEx), but as a discipline that rightfully relies heavily on anecdotal feelings, it can be left unloved in many organisations that require a data-driven approach.

Actively measuring aspects of the developer workflow, and not just the traditional outputs such as the number of features shipped or the number of lines of code committed, is a key part of understanding the impact of the developer experience. We've found that DevEx isn't just about tooling or process; it's about understanding the entire developer journey.

The DevEx evolution - playbook front cover

Deliver with direction–get the DevEx evolution playbook

Ready to evolve your development experience? Discover actionable strategies to align delivery with strategic goals in our new playbook.

Download your copy today

The findings in our DevEx evolution playbook, produced in collaboration with Network Perspective, suggest that surveys can be a highly effective way of measuring this — so long as they are the right surveys. We caution against generic questions related to satisfaction, which may be easy to use as vanity metrics, and look instead at surveying specific pain points often felt by developers — such as ease of setting up an environment, quality of documentation, and responsiveness of support. Looking at both quantitative metrics in combination with qualitative feedback gives a richer understanding of developer sentiment, going some way to revealing why some issues exist.

A well-designed survey doesn't just issue sentiment scores. A good one identifies the key drivers behind developers' interactions with people, metrics, tools, and processes to find points of friction and frustration and starts on the road to finding ways of fixing them. Doing this at an early stage of a product development process gives some early warnings of problems that may come further down the line, particularly around developer flow and their cognitive load. This allows organisations to mitigate risks to software delivery.

We've noticed success when organisations focus on the metrics that actually correlate to key business outcomes — the DORA metrics of cycle time (the time it takes to move a feature from idea to production), deployment frequency, change failure rate, and time to resolution for critical bugs give a much more accurate picture of how effective developers are and, by extension, of how healthy the overall software delivery pipeline is.

DevOps culture & collaboration–explore our knowledge hub

Discover how integrating your IDEs and improving team collaboration can transform your DevOps culture. Access valuable resources and expert insights. Click through to learn more.

Learn more

Having ways to collect data about the developer experience passively can help form a more accurate picture. Tools such as IDEs, source code repositories and collaboration platforms can generate a lot of data that can help identify patterns and trends. This, in turn, can help explain developer behaviour in various scenarios without needing developers to participate actively. It's vital to have active feedback, too, though - to unravel the context and nuance involved in interpreting passively collected data and build the big picture of Developer Experience.

All in all, putting the correct metrics in place, collected actively and passively, gives valuable data on the efficiency of a software delivery pipeline. Organisations who want to make the best of this know that aceing developer experience in tandem provides the best chances of success.

In conclusion, prioritising a data-driven approach to developer experience, rather than relying solely on subjective feelings, is crucial for optimising software delivery. By actively measuring specific pain points, tracking DORA metrics, and combining quantitative data with qualitative feedback, organisations can gain a comprehensive understanding of the developer journey. This strategy, which includes both active surveys and passively collected data from tools, allows for the identification of friction points, mitigation of risks, and, ultimately, the enhancement of the software delivery pipeline's efficiency and success.

Have DevEx challenges?

Get in touch to discover how Adaptavist can help.