My best learnings came from mistakes - mine, my team's, my company's, other companies - but almost always, it was when those mistakes were promptly followed up with a thoughtful incident report.
For example, when I launched what is now Uber One, two days later, I got a call that I broke something. It turned out I had forgotten an edge case in a completely different service. Yes, this was 2016 Uber, 3000 microservices, but I don't think the users who didn't get their discount really cared.
The takeaway I still remember from that today is about always looking for "works by coincidence"; I often ask myself about the preconditions that must be in place for some class of mistake to be impossible.
The framing of preconditions that dissolve a class of mistakes is intriguing. I keep re-reading it, trying to picture how it actually exists day to day: is it something you run in your head when you're in the code, or is it structured into something your team shares (e.g. a checklist or similar)? I haven't had much success closing that gap.
This is very much a shared pain, though my experience is mostly outside the programming world.
I see it as a fixable pattern, but the inertia is strong.
- Lessons learned and retrospectives don't take place as often as they should
- The sessions themselves sometimes get a bit defensive, so the real nuggets get missed
- Even if a good session takes place, it ends up being static data buried in another archive
Taking an example from the construction world, I saw project after project suffer from poor planning and design. Common enough issues to warrant a moment of investigation, but not, then later being passed off as "black swan events". Because who would have thought the ground conditions would be suboptimal near a swamp?
There's something to take from the Health & Safety world. Talk about stuff. An organisation I worked for had a rule that every meeting, irrespective of purpose, had to start with a safety conversation. Something new, a reminder, a good story - just something safety focussed - and a reminder to look in the H&S system for updates and other stories. There's strong correlation between proactive conversations and reduction in safety incidents.
My thinking is that this approach should work across domains. I've done it with teams in the non-programming space with some success. A few minute knowledge-share at the start of stand-ups and regular meetings seemed to reduce the number of times known issues/challenges happened a third time. Just be prepared to lead the conversation with "I was looking through old lessons learned and saw <thing>... good reminder to <action>..." or something.
> Or we ship a feature and never circle back to check whether it's used the way we intended, so we keep shipping more on the same untested assumptions.
It's not on you to track the whole lifecycle of a feature, that belongs to PO+UX.
Analytics and A/B testing are part of good feature planning and should be considered well before things get to dev.
With two decades at the same company, I mostly see this at an organizational level. New management comes in and is expected to fix things, but they try the same things that have already been tried and it fails for the same reason if failed the last 3 times it was tried. Lessons aren’t learned because there is a lack of continuity.
Leadership isn’t allowed to learn from their mistakes, they get replaced. It appears this is common, so ultimately, the industry has these people “transforming” organizations, running projects, or consulting, who just bounce around and implement. They never stick around to see the changes run. They aren’t there for the failure and therefore can never learn from their mistakes. As soon as the project is far enough for them to declare success, they start using that as the springboard into their next position. They get new job based on their “successful” implementation… and then the whole thing falls apart, because they were never worried about how it would run for the next 10 years, just about making it look good long enough to get their bonus and get out. This is how they can avoid ever having to answer for mistakes and can delude themselves into thinking everything they’ve done actually worked.
The other issue I see, that is semi-related, is not giving things long enough to work. Massive cultural shifts take time. If they don’t catch on fast enough, they are abandoned for something else. This teaches everyone that it’s not worth getting on board with these attempts. For something to succeed, it now needs to be the norm for long enough to overcome the lack of trust this pattern created. We got close with agile, they but a lot of weight behind it and management was very consistent. Then when we were on the verge of really turning the corner and running smooth, even with resistant team, they did a re-org and the new guy threw it away and replaced it with nothing, destroying all that progress and teaching people, yet again, that it’s not worth adopting these models.
I often kept a personal "notes" file while I was working on a project/feature. I now think that what I should have done is check that file into the source tree, at the top level directory for whatever feature I was doing.
I like the "check it into the tree" instinct. I've tried tracking observations and lessons in README files across a monorepo, but I felt they went stale almost immediately. Maybe your more focused files would work better. But checked in or not: have you found yourself reopening those earlier notes when you start the next feature? Or do they mostly just sit there?
My best learnings came from mistakes - mine, my team's, my company's, other companies - but almost always, it was when those mistakes were promptly followed up with a thoughtful incident report.
For example, when I launched what is now Uber One, two days later, I got a call that I broke something. It turned out I had forgotten an edge case in a completely different service. Yes, this was 2016 Uber, 3000 microservices, but I don't think the users who didn't get their discount really cared.
The takeaway I still remember from that today is about always looking for "works by coincidence"; I often ask myself about the preconditions that must be in place for some class of mistake to be impossible.
The framing of preconditions that dissolve a class of mistakes is intriguing. I keep re-reading it, trying to picture how it actually exists day to day: is it something you run in your head when you're in the code, or is it structured into something your team shares (e.g. a checklist or similar)? I haven't had much success closing that gap.
This is very much a shared pain, though my experience is mostly outside the programming world.
I see it as a fixable pattern, but the inertia is strong.
- Lessons learned and retrospectives don't take place as often as they should
- The sessions themselves sometimes get a bit defensive, so the real nuggets get missed
- Even if a good session takes place, it ends up being static data buried in another archive
Taking an example from the construction world, I saw project after project suffer from poor planning and design. Common enough issues to warrant a moment of investigation, but not, then later being passed off as "black swan events". Because who would have thought the ground conditions would be suboptimal near a swamp?
There's something to take from the Health & Safety world. Talk about stuff. An organisation I worked for had a rule that every meeting, irrespective of purpose, had to start with a safety conversation. Something new, a reminder, a good story - just something safety focussed - and a reminder to look in the H&S system for updates and other stories. There's strong correlation between proactive conversations and reduction in safety incidents.
My thinking is that this approach should work across domains. I've done it with teams in the non-programming space with some success. A few minute knowledge-share at the start of stand-ups and regular meetings seemed to reduce the number of times known issues/challenges happened a third time. Just be prepared to lead the conversation with "I was looking through old lessons learned and saw <thing>... good reminder to <action>..." or something.
> Or we ship a feature and never circle back to check whether it's used the way we intended, so we keep shipping more on the same untested assumptions.
It's not on you to track the whole lifecycle of a feature, that belongs to PO+UX.
Analytics and A/B testing are part of good feature planning and should be considered well before things get to dev.
I have my own personal formal debrief process for larger projects: https://github.com/EternityForest/THUNDERWARS
Usually if I notice anything of interest to others, it just winds up in a group chat.
With two decades at the same company, I mostly see this at an organizational level. New management comes in and is expected to fix things, but they try the same things that have already been tried and it fails for the same reason if failed the last 3 times it was tried. Lessons aren’t learned because there is a lack of continuity.
Leadership isn’t allowed to learn from their mistakes, they get replaced. It appears this is common, so ultimately, the industry has these people “transforming” organizations, running projects, or consulting, who just bounce around and implement. They never stick around to see the changes run. They aren’t there for the failure and therefore can never learn from their mistakes. As soon as the project is far enough for them to declare success, they start using that as the springboard into their next position. They get new job based on their “successful” implementation… and then the whole thing falls apart, because they were never worried about how it would run for the next 10 years, just about making it look good long enough to get their bonus and get out. This is how they can avoid ever having to answer for mistakes and can delude themselves into thinking everything they’ve done actually worked.
The other issue I see, that is semi-related, is not giving things long enough to work. Massive cultural shifts take time. If they don’t catch on fast enough, they are abandoned for something else. This teaches everyone that it’s not worth getting on board with these attempts. For something to succeed, it now needs to be the norm for long enough to overcome the lack of trust this pattern created. We got close with agile, they but a lot of weight behind it and management was very consistent. Then when we were on the verge of really turning the corner and running smooth, even with resistant team, they did a re-org and the new guy threw it away and replaced it with nothing, destroying all that progress and teaching people, yet again, that it’s not worth adopting these models.
I often kept a personal "notes" file while I was working on a project/feature. I now think that what I should have done is check that file into the source tree, at the top level directory for whatever feature I was doing.
I like the "check it into the tree" instinct. I've tried tracking observations and lessons in README files across a monorepo, but I felt they went stale almost immediately. Maybe your more focused files would work better. But checked in or not: have you found yourself reopening those earlier notes when you start the next feature? Or do they mostly just sit there?
Well, see, I've never actually done it. I decided that it would be a good idea. And then I retired...