DevOps insights from the CrowdStrike outage: boosting software resilience

An oft-repeated observation of the CrowdStrike incident is "Why was this not tested properly?" and DevOps offers many of the answers to this. A key tenet of DevOps is to focus hard on feedback loops—where iterative improvements to software are measured, tested and fed back to the developers, and the kneejerk reaction to the CrowdStrike incident suggests that this wasn't done. But a detailed analysis of the root cause of the incident does not support this—instead suggesting that a niche set of hard-to-test and hard-to-predict circumstances caused the outage.

The vast majority of our survey respondents say they will expand technical teams, focusing on DevOps and testing, and it's clear that this presents an opportunity to level up on both for all organisations. Organisations must avoid falling into the trap of just adding more and more tests—as these add to the build and testing cycle time and can lead to a lengthening of the feedback loop, the opposite of what we want. Rather, organisations should look to a more strategic DevOps approach—relying less on unit tests and expanding capabilities of integration and acceptance tests. This, in turn, helps ensure that those feedback loops are representative of the real world and rapid enough to fit into a highly iterative development process.