Feature scoring: Using Drivers to make data-driven decisions

The method you use to evaluate potential features is critical to your product’s success. Done well, you identify and deliver the gems, those features most likely to boost product success in the market. Done poorly, your product releases fall short – which, from the point of view of your financial investment in product, is the same as leaving money on the table.

If deciding what products or product capabilities isn’t a challenge for you, you’re in luck: You probably don’t need a scoring method.

What is scoring?

“Feature scoring” is a general term for methods of assigning a value rating, or score, to product features. Ideally, those scores help you prioritize the best features to work on. In this blog, we are using the term “features” broadly – they could be called epics, capabilities, initiatives, projects, etc. depending on your development approach.

You can use feature scoring for a variety of purposes. Here we focus on feature scoring for product planning: That is, features with a larger scope – the so-called boulders and large rocks, or epics, as opposed to pebbles and sand, or tasks.

Why does scoring matter? You need a way to assess potential features to ensure that you rise above opinions, biases, and emotions to make better, more data-driven decisions on what goes into your product plan.

Scoring method matters

Many scoring methods simply put a number on an opinion. They’re inconsistent, unclear, unmeasurable, and unaligned with business objectives. For example, one PM gives a score of 5 for a feature based on usability; another gives a score of 3 based on revenue. But what do the 5 and 3 actually mean? Five is greater than three, but does your company value usability more than revenue? If a PM assigns a 3 for revenue, what should you expect in terms of return once that feature is in market? How do you track whether your feature investment is going to pay off?

Scoring Best Practices

Feature scoring should be consistent, clear, measurable, and aligned with business objectives.

Here’s how we do it at Obo, and we’ve built the Obo system (our product) to make it easy for our customers to do.

  1. Choose consistent Drivers aligned with your business objectives

You want to evaluate features using the criteria (we call them “Drivers”) that matter most to the business. So start with how you measure product success. If revenue, growth, reducing churn, or market share matter, then assess each feature in terms of its expected impact against those objectives

Choose your Drivers. This exercise is typically done collaboratively, with senior-level approval. Drivers should be consistent for each product, though some may vary by product or product line, and should be aligned with business objectives. If you have more than five Drivers, consider grouping them into sets, for example Corporate Drivers and Product Drivers. Each set provides a perspective on feature value.

  1. Define the scale for each Driver

Numeric feature scores can be misleading if you are not clear about the scale you are using. Obo uses a scale with levels from 0 to 5 for each Driver. Some Drivers are quantitative and some may be qualitative, so we document and communicate what they are. Here are two examples:

  Quantitative Example:
Monthly Recurring Revenue
Qualitative Example:
Competitive Advantage
0 None None
1 Less than $15K Catches up
2 $15k - $30k Gets to par
3 $30k - $45k Joins leaders
4 $45k - $60k Establishes small lead
5 More than $60k Establishes substantial lead
  1. Define relative Driver weights

At any point in time, some Drivers are more important than others to your business and product. For example, a corporate initiative to expand into new, international markets means that your International Expansion Driver (for assessing expected impact against ability for your product to support international expansion) is now relatively more important than some other Drivers, such as market share in existing markets.

Within each Driver Set, assign a rank or relative weight for your Drivers. We use values between 10 and 100. Here’s an example:

Driver Relative Weight
Revenue 100
Customer Acquisition 80
New Markets 60
Customer Retention 40

If you can, choose different weights for each Driver to clarify priorities.

  1. Score each potential feature against all relevant Drivers

Use all Drivers relevant to your product, even if their values are 0. No making up new Drivers on the side. If your Drivers aren’t working for you, then revisit them per step 1.

At Obo, our product manager leads the extended product team in a collective exercise to assign Driver values for each feature we are considering, either in person or using a survey. We store scores in Obo, so they’re available for everyone to see.

Our Drivers are a framework for evaluation and rational discussion. We use the data we have to forecast the impact of each feature against our business objectives. Some features that we really want to build don’t make the cut – now – because they are not aligned with current objectives. Those features will still be in the system, available for future planning sessions.

This process helps minimize bias and keep decisions “honest” – based on the data we have and our collective knowledge, not on the latest sales call. It also clarifies where we need to do more research with our customers, prospects, and our market.

  1. Calculate final, weighted score

The final, weighted score of each feature is the weighted sum of the values you set for it. In Obo we normalize to 200, which helps us compare driver scores to preference survey results. So, using a 0 to 5 scale: ((Driver Weight)*200)/(Sum of Driver Weights)) * (Level# / 5).

What scoring isn’t

Scoring isn’t enough for product planning

Even with the best scoring method, building the highest rated features and going down the list is a flawed approach: Decisions about what goes into the product plan should consider both value aligned with business objectives and cost, as well as other constraints such as interdependencies and deadlines. But feature scoring (assigning value) should be independent of feature cost, so you ensure you are making the best decisions based on overall value given your constraints.

Things to avoid when scoring features:

  • Cost estimates – A feature’s value is independent of cost. Cost estimates can bias a feature’s driver ratings. If you know a potential feature is easy to build, you may be more inclined to give it higher ratings. Don’t. You may have multiple features that would take that same amount of time, and rating them all as 5s doesn’t help you decide which to include in your product plan.
  • Dependencies – Though important for product planning, a feature’s dependencies on other features or products does not change its value to the market or your business.
  • Tech debt – Our best practice at Obo is that we don’t use tech debt as a driver (because it’s not a business objective). Tech debt is handled within engineering and in sprint planning, but not as part of our product planning process.

Drive the CAR to put your best product forwardTM

Here’s a handy mnemonic for Driver best practices:

  • C Consistent drivers and Driver values are critical.
  • A – Drivers align with business objectives.
  • RRelative weights reflect the importance of each Driver.

Validate for product success

No one likes the idea of pouring precious time and effort into...

Read More

5 steps to sell your product plan to your stakeholders

Getting agreement on your product plans can be an uphill,...

Read More

Lies, damned lies, and the 4 levels of validation

Dante’s Inferno has nine circles of Hell. Fortunately, product...

Read More

Have a comment?