Mission control: Lessons in agile risk management

To understand the broader implications of this outage, we can draw a familiar comparison with something like a mission control launch failure. Space agencies like NASA, which uses the Scrum agile methodology to develop software for projects like the Space Launch System (SLS), regularly deal with incredibly high stakes during rocket launches, where split-second decisions and cross-functional collaboration can mean the difference between success and failure. These ambitious launch projects involve numerous complex components, teams and stakeholders, making interdepartmental collaboration crucial.

In mission control, engineers and mission managers face the highest level of pressure when systems fail, but their response isn't based on panic. Instead, it's grounded in agile principles that prioritise rapid iteration, collaboration, and the ability to adapt to new information. NASA uses SCRUM with its iterative feedback loops, transparent communication, and failover systems to ensure that if something goes wrong, there's a path forward. Could these agile practices have made a difference with CrowdStrike?

In an agile organisation, adopting this iterative problem-solving mindset often speeds up response time during outages. For example, a more agile approach would involve setting up a cross-functional incident response team that regularly cycles through diagnosis, remediation, and validation without waiting for the 'perfect' fix. The goal is to get something working quickly, test it in real time, and iterate rapidly.

By proactively embracing these agile feedback loops, issues are detected earlier, and the team can deploy a more immediate solution rather than waiting for a larger, system-wide fix that takes longer to implement. This mirrors the way mission control constantly runs diagnostic checks during a mission, making tweaks in real time to ensure success.

Similarly, when systems fail within an organisation, they may have benefited from a more agile, cross-functional response in hindsight. An agile organisation places great emphasis on collaboration and breaking down silos. Having cross-functional teams working together could have led to a faster, more efficient resolution or even prevented it. Real-time communication and shared accountability are crucial in high-stakes environments, whether in mission control or cybersecurity operations.

Regardless of industry, organisations can benefit from this approach by providing customers with more frequent and transparent updates about the progress of an outage. Agile practices emphasise continuous communication—internal teams should be in constant touch. At the same time, external stakeholders (in CrowdStrike's case), and customers should be kept informed by the organisation of what's going on and when they can expect a resolution. This kind of openness helps reduce customer anxiety and creates a feedback loop that can help accelerate problem-solving.

An organisation's response to an outage can benefit from an agile fail-fast culture by testing potential solutions quickly and discarding ineffective ones sooner. The agility to fail quickly, learn from it without blame, and improve can enable the ability to resolve the issue faster, even if they don't have the perfect solution right away. By treating any incident as a learning opportunity, organisations can minimise both downtime and customer impact.

Agile risk management includes proactive 'risk sprints', which are short, focused cycles in which the team dedicates a short period within a regular sprint to identifying, testing, and mitigating potential vulnerabilities in smaller, more manageable chunks. In an agile context, risk management involves continuously identifying, assessing, and mitigating risks throughout the development and operational processes. The CrowdStrike outage serves as a reminder of the potential risks associated with technology and the need for proactive measures to mitigate them. Continuous risk assessments could mean an organisation is better prepared for incidents like this one, and downtime will be shorter or (even prevent them entirely) as part of robust risk management practices.

The CrowdStrike incident has also driven a shift in attitudes to regulation. 42.5 percent of organisations we surveyed are now more supportive of software industry regulations due to the outage. Incident reporting and testing regulations are gaining the most traction.

An agile approach to compliance management involves integrating security practices into the development lifecycle. The CrowdStrike incident highlights the importance of considering security as a fundamental aspect of product development rather than an afterthought.