Don’t overreact to the CrowdStrike outage; being more nuanced will help you win

Whilst "we don't want to be the next CrowdStrike" is not the most positive way to frame operational improvements, it's a great lever for organisations to focus in on SRE (site reliability engineering), the groundwork for which is now well-established, and an opportunity for those who were under the mistaken belief that "it couldn't happen here".

Testing strategies have come under the spotlight, too. The answer to the cliched response of "why wasn't this update tested?" is of course "it was!" in the majority of organisations we surveyed. But one of the core reasons the CrowdStrike outage happened was because the set of circumstances that caused the outage was actually very hard to test, and our research confirms suspicions that nuanced discussions around proper testing are now taking place, with about half of them now intending to do more work on unit and integration tests. The outage has shone a light on situational awareness–and we're encouraging organisations to take time to understand the path less trodden when writing tests - particularly integration tests-where perhaps the software isn't being used in exactly the situation the authors intended.

Nearly everyone who responded to our survey said that they were expanding their technical teams in reaction to the CrowdStrike outage. We urge organisations to look strategically at these extra hires and bulk up in engineering competence around site reliability, testing, and security through a DevOps lens of continuous improvement. As highlighted by the Head of DevOps, Jobin Kuruvilla, in the report, this is the only way to ensure that lasting change is made. So, rather than hiring lots of dedicated siloed testers, organisations should look to hire people who can improve quality across the whole lifecycle of an application.

Many of the responses we saw emphasised a positive outcome-with light shone on usually underfinanced and underrepresented areas of competency-such as operations, security and testing. But the CrowdStrike outage highlighted to the world how absolutely life-critical some IT systems can be, and the long-term reaction to the emotions we all felt due to an outage of this scale can set back cultural initiatives around psychological safety and implementing a culture of learning over blame. An incident of this magnitude in an organisation can lead to micro-management, excessively constrained additional processes, and a culture where innovation is stifled due to a perception that change must be heavily scrutinised to avoid incidents. Strategic epithets such as "right first time every time" can easily reduce delivery momentum to a crawl.