Verica, the company using Continuous Verification to make systems more secure and less vulnerable to costly incidents, today announced the findings of the Second Annual Verica Open Incident Database (VOID) Report. The VOID makes public incident reports in a single database to generate open discussion about how to tackle software-based failures and outages.
Over the past two years, the VOID has scrutinized and analyzed nearly 10,000 incidents from just under 600 companies from MAANG and Fortune 100s to startups alike, making this report the largest and most comprehensive study of incident analysis as a whole, to-date. The report was written by Verica and is sponsored by Indeed and Jeli.
This version of the VOID extends the vision of being more than just standard company post-mortems or status updates. With the greater number of incident analyzes, the community can gain a broader understanding of how individuals, companies, media, and others treat these types of events by collecting a set of meta-data. Among some of the key findings:
- No company is immune from incidents. Incidents happen in organizations of all sizes, from startups to the Fortune 10. Software is mission-critical in every possible industry including banking, travel, agriculture, commerce, and more.
- Length isn’t as cut and dry as it appears: there are many insightful metrics to measure in an incident. Duration of incidents conveys little meaning about the incidents themselves, in part because it can be very tricky to attribute when incidents start or stop.
- SREs and others in similar roles should retire MTTR as a key metric. This year’s report confirms that MTTR isn’t a viable metric for the reliability of complex software systems for a myriad of reasons, particularly because averages of duration data lie.
- Common assumptions around incident duration and severity are debunked. Incident duration and severity are not related, and we have the in-depth data to prove it.
- Organizations are moving away from shortsighted approaches like RCA. Root Cause Analysis appears to be on the decline in orgs of all sizes, as they move toward more meaningful metrics and analysis.
The key findings provide insight for companies to learn where the flaws may be and how to fix them. It can now be confirmed that accepted metrics for incidents aren’t reliable and aren’t giving them the correct information. Additionally, it’s worth the time and effort to invest in analyzing and writing up incidents. This practice helps organizations better understand their systems and how to ideally make them less impactful in the future.
A new partner in this report is Jeli. Jeli grew from the Learning from Incidents (LFI) community, where openness and transparency in how engineers were addressing the gaps in incident response allowed the team to create products that surfaced the relationships between the people in an organization and the technology. Like the VOID, Jelli is also on a mission to change how the software industry thinks about incidents, software reliability, and the critical role people play in keeping their systems running.
In the past, the publication of software incident reports have been scattered across the Internet—it’s often difficult to link directly to them, or they are sequestered in corners of company websites. The VOID and Verica solved this problem and structured the process for collecting these reports to help improve the software running key areas such as transportation, infrastructure, power grids, healthcare devices, voting systems, autonomous vehicles, and many critical societal functions.
Community members can make the database more comprehensive by submitting any reports that aren’t included in the VOID with this short form. Download the full report here.
Supporting Quotes
“Bringing this to life for a second year is truly a remarkable milestone for our company. Our mission is rooted in transparency and bringing that to the software industry, and we are dedicated to delivering on that promise,” said Casey Rosenthal, Co-founder & CEO, Verica. “As we grow the community, partnering with Nora Jones, the Jeli team and the Learning from Incidents community will enhance our ability to address many of the issues articulated in this version of the report as we continue to bring safety to the forefront of the tech space.”
“The VOID report marks a remarkable advancement in how our community will look at and fix incidents moving forward.” said Nora Jones, Founder and CEO of Jeli. “Upon seeing the emerging key findings of the report, Jeli was excited to support Verica’s research across these large datasets. Through extrapolating the key findings of the report, we are all able to build more resilient systems with greater collaboration.”
“We were surprised to find no relationship between the length of an incident and how “bad” it was. We have heard from many people who suspected that longer incidents were perhaps somehow worse/harder to resolve—conversely, some people thought that for really severe incidents, a company would have all hands on deck and resolve such incidents more quickly,” said Courtney Nash, lead research analyst, Verica & Creator of The VOID. “Companies can have long or short incidents that are very minor or quite serious, and every combination in between. Not only can duration not tell a team how reliable or effective they are, it also doesn’t convey anything useful about the impact of the event or the effort required to deal with it.”
About Verica
Verica uses the next step in the evolution of chaos engineering, Continuous Verification, to make systems more secure and less vulnerable to costly incidents. Verica Continuous Verification Platform provides out-of-the-box verifications that proactively uncover system weaknesses and security flaws before they disrupt business outcomes. All companies running complex systems experience failure, but as systems become more complex, Verica will be there to help maintain confidence in those systems. With Verica, you can trust that your software is working how it’s meant to. Learn more at www.verica.io.
About The VOID
Now an industry standard yearly report, the VOID is the largest and most comprehensive of incident analysis to date, with nearly 10,000 incidents from just under 600 companies analyzed and scrutinized. This data comes from nearly 600 companies ranging from mega cap tech and Fortune 100s to startups. The mission of the VOID is to make public incident reports in a single database to generate open discussion about how to tackle software-based failures and outages. Anyone can submit an incident to the VOID or become a member.
View source version on businesswire.com: https://www.businesswire.com/news/home/20221213005501/en/