Article Blog Image

Running Post-Mortems

Best Practices

This article continues the discussion on how your team can learn from failure after a production incident. While write-ups are very important in capturing and documenting what took place, the real value is created from an open and deliberate conversation with the team to identify the lessons learned and the improvements needed to create a more reliable system. That conversation is the post-mortem.

Post-mortems are the primary mechanism for teams to learn from failure....

Article Blog Image

Incident Write-ups

Best Practices

When is an incident considered ‘done’? Is it when the production impact has been addressed and the on-call goes back to bed? If that were true, teams would pass up a huge opportunity to learn and improve from what the incident can teach them, and the on-call (and more importantly, customers) would continue to have a sub-par experience from repeat incidents.

This post discusses the importance and process of the write-up, which is documenting an...

Article Blog Image

Oncall Retrospectives

Best Practices

Last time I shared my thoughts on blameless postmortems and how they create a safe space for revealing process and technology gaps contributing to past incidents.

Today I want to introduce another opportunity for teams to learn and improve from: the ‘oncall retrospective’, which:

  • Keeps the team in touch with the operational reality of their service(s);
  • Reveals opportunities to improve the oncall experience.

I was introduced to this practice by Jos Visser while...