Best Methods for Analyzing Risks Before Deciding

Most projects fail. But there’s a simple technique to give yours a fighting chance. It’s not a to-do list. It’s not a fancy tool. It’s not a 12-step system. It’s a single question that flips the way you think. Here’s how it works: It’s called a “premortem.” You’ve heard of a postmortem what went wrong after a project dies. A premortem asks: What if we ran that analysis now? Before anything dies. Before the first misstep. Before failure sets in. The premortem comes from psychologist Gary Klein. Here’s how to run one: → Gather your team. → Imagine it’s 2 years in the future. → The project has completely failed. → Ask: What went wrong? No sugarcoating. No happy talk. Start listing the causes of failure. Budget misfire? Wrong team? Lack of buy-in? Scope creep? Missed deadlines? You’ll be shocked how quickly people identify risks—once they feel safe predicting failure. Why this works: It defeats irrational optimism. • It turns hindsight into foresight. • It makes risk visible. • It aligns the team before chaos hits. Because the best time to fix a problem… is before it happens. Pre-mortems don’t require special skills. Just a shift in mindset: Don’t assume success. Assume failure—and reverse-engineer your way out. Ask: What will future-you wish you had done? Then… do that now. I run a premortem for every big project I take on. Writing a book? Premortem. Launching a podcast? Premortem. Planning an event? Premortem. It never guarantees success—but it always makes success more likely. Summary: The Premortem Playbook → Imagine future failure. → List the causes. → Turn those risks into action steps. → Adjust your plan today. It’s one of the most underrated tools in your productivity toolkit. Try it before your next project. You won’t regret it.

Product development entails inherent risks where hasty decisions can lead to losses, while overly cautious changes may result in missed opportunities. To manage these risks, proposed changes undergo randomized experiments, guiding informed product decisions. This article, written by Data Scientists from Spotify, outlines the team’s decision-making process and discusses how results from multiple metrics in A/B tests can inform cohesive product decisions. A few key insights include:  - Defining key metrics: It is crucial to establish success, guardrail, deterioration, and quality metrics tailored to the product. Each type serves a distinct purpose—whether to enhance, ensure non-deterioration, or validate experiment quality—playing a pivotal role in decision-making.  - Setting explicit rules: Clear guidelines mapping test outcomes to product decisions are essential to mitigate metric conflicts. Given metrics may show desired movements in different directions, establishing rules beforehand prevents subjective interpretations during scientific hypothesis testing.  - Handling technical considerations: Experiments involving multiple metrics raise concerns about false positive corrections. The team advises applying multiple testing corrections for success metrics but emphasizes that this isn't necessary for guardrail metrics. This approach ensures the treatment remains significantly non-inferior to the control across all guardrail metrics. Additionally, the team proposes comprehensive guidelines for decision-making, incorporating advanced statistical concepts. This resource is invaluable for anyone conducting experiments, particularly those dealing with multiple metrics. #datascience #experimentation #analytics #decisionmaking #metrics – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gj6aPBBY    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/gewaB9qC

Hi - I'm Steve. I am a professional fail-er. Many times data teams are asked questions that pertain to things that the business has never done before. This might be creating opportunity sizing for a new feature, forecasting adoption or performance of our customers, or building recommendations, either for the business or the customers to suggest relevant improvements or features to adopt. The challenge with many of these problems is that there's not always a black-or-white answer, and in addition, we tend not to have complete datasets that enable us to paint the full picture. As a result, we end up having to build assumptions into our models, basing this off of past experience, similar features, user behavior and other correlational analysis. Data teams that are not comfortable with the concept of failing fast can fall into the pitfall of 'paralysis by analysis', whereby we fail to make a recommendation due to the uncertainty that implicitly exists in the data. The easiest thing to do to delay a project or deliverable is to ask for more data, which inevitably will beget more questions and sometimes cause us to lose sight of the goal we were trying to accomplish in the first place. A much more effective approach, I have found, is to clearly draw out what assumptions we must make to size the feature, or conduct the analysis. Establish clearly the risk of us being wrong on any of those assumptions, and clearly evaluate one-way (irreversible) and two-way (reversible) decisions. The goal is to have enough 'low stakes' experiments, where we can easily roll back the change to gain enough confidence in the assumptions you must make for the 'high stakes' decisions where reversing the change is either incredibly costly, or sometimes infeasible. Through this approach, we're able to dedicate a lion share of the analysis time firming up the hypothesis we must make for the 'high risk' decisions, and apply the highest level of rigor in terms of experimentation and burden of evidence. 'Low risk' areas enable us to broaden our scope of knowledge of the product, building confidence in our assumptions, and creating data for us to explore 'why wasn't my assumption accurate?' Creating controlled environments to fail fast will not only enable you to learn faster, but it will enable teams to build confidence in their abilities to test their assumptions and debug when the stakes are high. If you create an environment where *every* decision requires an insurmountable burden of evidence, you run the risk of stifling innovation and having a data team that's not equipped to debug situations when our assumptions were inevitably wrong. My suggestion to data teams is to embrace (controlled) failure. No one asks the question 'why did this roll-out go so well?', but certainly the question always arises 'what went wrong' when our predictions do not materialize. Ensure you're prepared for those situations by learning *how* to fail.

Taking risks doesn’t make you brave—it just means you’re reckless if you’re not prepared. When you’re creating a project or a business, you’ve probably thought about the risks. And if you haven’t, you should. People love to say, “Take the leap! No risk, no reward!” And sure, that’s true. Without risks, there’s no progress, no learning, no growth. But taking risks without being prepared isn’t bravery—it’s stupidity. It’s like jumping into a pool without checking if there’s water. And if you’re not careful, you’re going to hit the ground hard. That’s where the pre-mortem analysis comes in. It’s not about avoiding risks—it’s about understanding them. It’s about imagining your project has failed and working backward to figure out why. Because if you can predict how you might fail, you can prevent it from happening. Here’s how it works: 1️⃣ Imagine the project has failed. Gather your team and ask: “What went wrong?” Map out every possible reason. 2️⃣ Identify the risks. Categorize them: internal (team, resources) vs. external (market, competition). 3️⃣ Create a prevention plan. For each risk, outline actionable steps to mitigate it. The benefits? Uncovers hidden risks before they become problems. Encourages open, honest communication within teams. Builds a culture of proactive problem-solving. I always take this step when building out long-term plans for our teams because I remember the feeling of being terrified of failure and not comprehending what that looks like. It was my first ever “startup” in high school and I couldn’t shake the feeling that we were gonna crash and burn, but I just didn’t know how to describe it. Then my mentor put me on to the pre-mortem analysis. When I started describing what failure looked like, it became a lot less frightening and I built out plans to steer clear from failure. Again, taking risks is necessary, just make sure you’re not going into them blindly. Plan for failure to ensure success. Because the best way to win is to know how you might lose. #entrepreneurship #leadership #founders #problemsolving #growthmindset #startups #strategy