Evidence Guided

1. Evidence-guided development: Supercharge decisions with data and testing

Evidence-guided development is not about ceding decisions to data; it's about supercharging our judgment and helping us make better decisions.

Embrace uncertainty. Evidence-guided development acknowledges that most product ideas fail and that human judgment is flawed when dealing with complex, uncertain situations. Instead of relying solely on opinions, intuition, or rank, this approach combines human expertise with data-driven insights.

Learn and adapt. The core principle is to continuously test assumptions and ideas, collecting evidence to inform decisions. This iterative process allows teams to:

• Validate or refute hypotheses quickly
• Reduce the risk of costly failures
• Improve products based on real user feedback
• Make data-informed decisions rather than relying on guesswork

By adopting an evidence-guided mindset, companies can innovate faster, reduce waste, and create products that truly resonate with users. This approach is not about eliminating human judgment but enhancing it with objective data and systematic testing.

2. The GIST model: Goals, Ideas, Steps, and Tasks for product development

GIST is not a radically new concept. I consider it a meta-framework that combines tried-and-tested product methodologies with models and tools I created along the way.

Four interconnected layers. The GIST model provides a comprehensive framework for evidence-guided product development:

1. Goals: Define measurable outcomes aligned with company strategy
2. Ideas: Generate and prioritize potential solutions
3. Steps: Validate ideas through small, iterative experiments
4. Tasks: Execute the day-to-day work required to complete steps

Bridging strategy and execution. GIST connects high-level business objectives with tactical implementation, ensuring that every task contributes to meaningful outcomes. This alignment helps teams:

• Stay focused on what matters most
• Adapt quickly to new information
• Balance short-term delivery with long-term vision
• Foster collaboration across different organizational levels

By implementing GIST, companies create a structured yet flexible approach to product development that promotes continuous learning and improvement.

3. Goals: Focus on outcomes, not outputs, using metrics trees and OKRs

Goals define the outcomes we wish to achieve—measurable benefits for customers and for the business.

Shift from output to outcomes. Traditional goal-setting often focuses on deliverables (e.g., "Launch feature X"). GIST emphasizes measurable outcomes that create value for users and the business. This shift helps teams:

• Align efforts with true business impact
• Stay flexible in how they achieve goals
• Measure progress more meaningfully

Key tools for outcome-based goals:

• North Star Metric: A single measure of overall value delivered to users
• Top Business Metric: The most important measure of business success
• Metrics Trees: Hierarchical breakdowns showing how lower-level metrics contribute to top-level goals
• Objectives and Key Results (OKRs): A goal-setting framework combining aspirational objectives with measurable key results

By focusing on outcomes, teams gain clarity on what truly matters and have the freedom to discover the best ways to achieve those results.

4. Ideas: Evaluate objectively using ICE (Impact, Confidence, Ease) scoring

ICE moves us from shallow, subjective thinking toward deliberate, objective analysis.

Systematic idea evaluation. The ICE framework provides a structured method to assess and compare product ideas:

• Impact: Estimated effect on key metrics
• Confidence: Level of supporting evidence
• Ease: Effort required to implement

Data-driven prioritization. By scoring ideas across these dimensions, teams can:

• Reduce bias in decision-making
• Compare ideas objectively
• Prioritize high-potential opportunities
• Identify areas requiring more validation

The Confidence Meter tool helps teams assess the strength of evidence supporting an idea, from weak (e.g., personal opinion) to strong (e.g., A/B test results). This encourages a culture of evidence-seeking and helps teams invest resources more wisely.

5. Steps: Validate assumptions through build-measure-learn loops

Steps are activities or mini-projects designed to develop an idea somewhat (sometimes just in concept) and test it.

Iterative validation. Steps break down the development process into small, testable increments. This approach allows teams to:

• Learn quickly and cheaply
• Reduce risk of large-scale failures
• Adapt ideas based on real-world feedback
• Invest more as confidence grows

The AFTER model provides a framework for validation techniques:

• Assessment: Quick, low-cost evaluations (e.g., expert reviews)
• Fact-Finding: Research to gather supporting data
• Tests: Early-stage prototypes and user testing
• Experiments: Controlled tests (e.g., A/B experiments)
• Release Results: Gradual rollouts and post-launch monitoring

By systematically progressing through these validation stages, teams can develop products with a higher likelihood of success and avoid wasting resources on unproven ideas.

6. Tasks: Empower teams with context and shared understanding

GIST doesn't dictate how you manage tasks, but it strives to connect the work to steps, ideas, and goals, allowing the team to work with full context and deep focus on the customers and the business.

Bridging strategy and execution. GIST connects high-level goals to day-to-day work, ensuring that every task contributes to meaningful outcomes. This approach helps teams:

• Understand the "why" behind their work
• Make informed decisions autonomously
• Collaborate more effectively across functions

Key practices:

• GIST Boards: Visual tools showing current goals, ideas, and steps
• Step Backlogs: Prioritized lists of validation activities
• Regular Reviews: Team meetings to assess progress and adjust plans

By providing rich context and fostering shared understanding, GIST empowers teams to take ownership of product development and make better decisions at all levels.

7. Scaling GIST: Adapt the model for different company sizes and stages

GIST works well for companies of different sizes and life stages, though the main value can vary.

Flexible implementation. GIST can be adapted to fit various organizational contexts:

• Startups: Focus on rapid learning and pivoting to find product-market fit
• Scale-ups: Balance growth with maintaining agility and customer focus
• Enterprises: Foster innovation within complex structures

Key considerations for scaling:

• Startups: Emphasize quick experiments and pivots
• Scale-ups: Establish clear product teams and ownership
• Enterprises: Balance autonomy with strategic alignment

By tailoring GIST to the company's specific needs and constraints, organizations can maintain agility and customer focus as they grow.

8. GIST patterns: Apply evidence-guided thinking across various product types

GIST is an adaptable model, and one that can work in a broad set of circumstances.

Versatile application. GIST principles can be applied across diverse product types and industries:

• Enterprise B2B: Balance customer requests with scalable product development
• Consumer/SMB: Leverage rapid testing and iteration
• Internal platforms: Focus on delivering value to internal customers
• Multi-sided marketplaces: Coordinate development across different user types
• Physical products: Apply software-like iteration to hardware development

Key adaptations:

• B2B: Develop early adopter programs and co-creation partnerships
• Consumer: Emphasize qualitative research and rapid experimentation
• Internal platforms: Clarify value metrics for internal customers
• Marketplaces: Coordinate testing across multiple user types
• Physical products: Leverage prototyping and modular design

By understanding how to adapt GIST to specific product contexts, teams can apply evidence-guided principles even in challenging environments.

9. Adopting GIST: Overcome challenges and drive organizational change

Switching to an evidence-guided mode of work is rarely easy.

Common adoption challenges:

• Lack of trust between management and teams
• Missing resources or infrastructure for testing
• Resistance to change from various stakeholders

Strategies for successful adoption:

• Start small: Begin with pilot projects or teams to demonstrate value
• Build trust: Emphasize transparency and evidence-based decision-making
• Focus on quick wins: Showcase early successes to build momentum
• Provide training: Equip teams with necessary skills and mindsets
• Adapt gradually: Implement GIST elements incrementally

Driving change requires persistence, clear communication, and a willingness to adapt the model to fit the organization's unique needs. By addressing common challenges and following proven change management practices, companies can successfully transition to an evidence-guided way of working.

Last updated: October 3, 2024