Lies, damned lies, and the 4 levels of validation

Dante’s Inferno has nine circles of Hell. Fortunately, product teams don’t have to navigate them all to address their planning challenges without getting burned. The main goal of product planning is to identify and prioritize candidate features most likely to succeed in the market given resource and time constraints. How do you know what’s most likely to succeed? Validation.

Validation ensures that what’s in your plans will meet important business and product objectives as well as market needs. What’s the best approach to validation? There’s no “one size fits all” as every product organization has to operate within its abilities and constraints. 

We’ve identified four levels of validation to consider as you develop your plans. Ideally, product plans include all four levels of validation. Realistically, product managers and leaders should do what they can to increase their odds of product success.

4 levels of validation

Level 1: Business & Product Objective Validation

The first step in validating product plans is to measure what’s in them against your business and product objectives. Most companies have a set of company “imperatives,” which usually includes things like revenue, customer acquisition, customer retention, customer satisfaction, and new markets. There may also be a set of product objectives, set around things like adoption, competitive advantage, innovation, and scalability. 

To validate product plans with your business and/or product objectives, establish a consistent rating scale for each objective, and then rate the features or functionality components in your plan against each objective. Rarely does any one feature rate top scores against each objective.

For example, your product is a restaurant reservation system and you’re considering adding a feature that lets your product’s mobile application users find and book reservations with a group of friends. You believe that feature will increase adoption (those friends will need to download and use the app) and customer satisfaction, and drive revenue by attracting more restaurants to support your system. It’s also innovative, as none of your competitors has the capability. But you don’t think it will have much impact on customer retention or your penetration of new markets and it has zero impact on scalability. So you’d rate that feature pretty high on the scales for adoption, revenue, customer satisfaction, and innovation, and low on retention, new markets, and scalability. 

Once you’ve rated the features you’re considering for your product plan, you’ll be able to group, sort, and rank them to see which contribute most to each top objective. 

Showing the extended product team (including engineering, sales, marketing, and support) as well as executives how different features stack up against an agreed-upon set of core objectives helps guide and support your decisions. At minimum, with Level 1 validation, everyone will know that you validated the features that made it into the plan using a consistent set of business and/or product objectives.

TIP: If you aren’t sure what your company’s business and product objectives are, ask. Senior product leaders or executives should be able to tell you what they are. Alternatively, circulate a list of proposed objectives you want to use for product planning, and request input from key people to rank and weight them. For more information, download our Drivers Field Guide

Benefits of Level 1 validation: When you validate your product plans with your business and product objectives, it’s easier to sell your plan to colleagues and executives. 

Level 2: Internal Stakeholder Validation

Next-level validation is validating with the teams and individuals involved in building and bringing your products to market as well as selling and supporting them. Include sales, marketing, customer success and support, engineering, and anyone else whose input is valuable. We recommend getting stakeholder validation in a few ways:

  1. Schedule time to interview the internal people whose input you value the most. Conduct these interviews just as you would a customer or prospect interview where you’re trying to uncover what matters the most. Ask questions that will help you discover pain points and problems you could address. Show and tell them what you’re considering in your plans and ask them what they think. Take notes and keep them with your plans. 
  2. Ask your internal stakeholders to rate all of the features and functionality under consideration against business and product objectives -- “team sourcing” feature value assessment.  Keep a “scorecard” that shows the points of view of different teams and roles. You can share that with everyone so they see who values what; it’s useful and may stimulate interesting conversations.
  3. Send out a feature preference survey (aka a “best worst” survey), which asks respondents to pick one “best” (most important) and “worst” (least important) feature from subsets of those features you’re considering. The result is a calibrated, ranked list of features, from most important to least important. 

Different internal teams and individuals have different levels of product knowledge and understanding. So make sure those you survey have sufficient background information. We’ve noticed that it can be hard to get time from internal teams to get the validation and feedback you need for product planning. Let them know how important their input is – especially if they want to influence the product.

Benefits of Level 2 validation: By engaging with and gathering input from the teams and individuals most responsible for product delivery and success in the market, you’ll gain buy-in and support for your plans.

Level 3: External Stakeholder Validation

Now you’re reaching outside your organization to people who have a stake in your product plans: customers, partners (including channel partners, if channels are important to your business), and perhaps investors and board members or advisors. If your company has a customer advisory council, be sure to get their input. How? 

  1. Interviews: Schedule time to interview a representative set of external stakeholders. You may want to select from your customer advisory council (if you have one), your top strategic partners (including channel partners), and the board members or advisors who are most knowledgeable about your products and your markets. Prioritize customers first, followed by strategic partners. Ask questions, show and tell them what you’re considering, and take notes you can keep with your plans.
  2. Send out a feature preference survey (aka a “best worst” survey). Explain to your external stakeholders why their responses matter, how you’ll use them, and that you know there are some tough calls because all of the features you’re considering are good ones. You may also want to incent external stakeholders to respond to your survey by offering them a small reward, like a gift card.

When you reach Level 3, you’ll have data in hand that shows how different features and functionality stack up against your most important business and product objectives, as well as what matters most to key internal and external constituents who care about your product plans. 

Benefits of Level 3 validation: At Level 3, you’ve got data from your existing customers and external influencers to back up your plans. It’s hard for people to argue with a plan that’s validated by customers.

Level 4: Market Validation

Now we’re cooking with gas. You’ve aligned your product plans with your most important objectives, and you’ve validated features and functionality with your most important internal and external stakeholders. Reaching out to the broader market of prospects and targets that have yet to buy your products gets you the highest possible validation for your product plan. Why? Because now you’re validating that people and companies that don’t have an existing relationship with you, your company, or your product or service actually need and want what you’re planning to build.

See our blog post 3 things you must do before your next product release.

There are three ways to validate your plans with your prospects and non-customers in your target markets:

  1. Interviews: Schedule time to interview prospective customers in your target market segments. Once again, ask questions, show and tell them what you’re considering, find out what would motivate them most to buy (from you), and take notes you can keep with your plans.
  2. Survey your prospect list. Work with your marketing team to identify the best prospects in your marketing database. Send out a feature preference survey (aka a “best worst” survey) with a follow-up feature value question. You may want to incent prospects to respond to your survey by offering them a small reward.
  3. Conduct a market panel survey. There are firms that provide panels of people that represent every target market segment imaginable, from consumers to business buyers from all roles and levels, industries, and geographies. Obo has built-in access to panel providers that can provide vetted respondents rapidly and at a low cost. Once you field a survey, depending on how targeted your respondents are and how many responses you need, you’ll get results in a few hours or days, rather than the months that external market research projects can take.

Achieving Level 4 validation puts you in the “best practice” class of product planning. Your plans are validated with your company objectives, your internal and external stakeholders, customers, and the market. Kudos all around.

Benefits of Level 4 validation:  Your plan is near bullet proof, with solid business, stakeholder, and market validation. You’re likely to gain plan approval with minimal friction, and everyone involved in delivering on the plan will know what you’re building and why. When things change, you’ll also have the information you need to go back and re-plan, pulling in the “next most valuable” features and capabilities when something is taken off the table or shifted to a later release.  

 

At Obo, we’ve engaged with and surveyed over 2,000 companies to understand their product process challenges and develop solutions that will help drive product success. We are releasing a report of the findings of our latest survey soon. Request your copy here.

Validate for product success

No one likes the idea of pouring precious time and effort into...

Read More

5 steps to sell your product plan to your stakeholders

Getting agreement on your product plans can be an uphill,...

Read More

Are idea portals a bad idea?

Idea portals aren’t a bad idea -- but they aren’t necessarily a...

Read More

Have a comment?