Have you a method for taking a product strategy and translating it into an OKR set? What are the best practices and things to avoid from your experience?
Short answer: It's straightforward to create OKRs from your SMT framework and roadmap. Simply add a forecast for how much you'd like to improve your proxy metric in that quarter or year.
WOOT! After two months, I have 2,500 subscribers. I’d love it if you shared the newsletter with friends and colleagues. More subscribers generate more questions and more upvotes, which helps me deliver more relevant answers to you. The newsletter is fun to write, and I am building a daily writing habit that keeps me focused on the issues that product leaders care about most. Thanks for being part of the early journey!
Today’s answer
I joined Electronic Arts in 1991, and they were very diligent about OKRs (Objectives and Key Results). Each quarter, everyone spent lots of time creating quarterly OKRs for each function, team, and individual.
When I arrived at Netflix in 2005, one of my first questions to Neil Hunt, the CPO, was, “Do you do OKRs?”
Neil hesitated, then explained, “No. We don’t find it worth the time. OKRs also introduce false precision. We know what metrics we need to measure, but it’s anyone’s guess how far we will advance the metric in any given time period.”
I haven’t used OKRs since. Below I outline what I did find helpful and how you can use my tools to develop OKRs relatively quickly. Like OKRs, my goal is to ensure there’s alignment within the company and all employees know how to contribute to the company’s success.
The Quarterly Product Strategy Meeting
Each quarter I ran a Quarterly Product Strategy Meeting. (Learn more about this meeting here.) The product leader of each swimlane provided an update of the product strategy, along with the proxy metrics they used to validate their hypotheses and projects against each strategy. I call this the SMT model, demonstrating the relationship between Strategy, Metrics, and Tactics. (With product development tactics and projects are synonymous.)
Here’s an example of the SMT model for the overall Netflix product today. It’s a SWAG — a stupid, wild-ass guess— to illustrate the concept:
Beyond articulating the Strategy, Metrics, and Tactics for the overall company, we did the same for each swimlane in the company. In this case, assume that each of the strategies is a swimlane. That’s how I typically organize product organizations.
I also asked each team to create an updated four-quarter rolling roadmap. Here’s a simplified version of what the overall roadmap might look like:
The roadmap is a high-level summary of key projects. It only includes the most important projects to help tell a story of what each strategy means and how things might come to life over time.
So, each quarter, I would share the SMT model and the high-level rolling roadmap across the company. I would list the most important projects for the quarter. Many of these projects required light coordination within the product organization. The implied message was, “if this project hits your inbox, keep it moving.”
SWAG of Product Team’s Q1 Objectives & Key Results
Below I show how I would translate my product strategy tools into an OKR structure. The only additional work required is to add an estimate of how much you’d like to move each proxy metric in a given timeframe. I quibble with this estimate because it’s really just a guess, while most consider it a forecast. That’s the “false precision” part.
Here’s what the OKRs might look like:
Q1 Product OKRs (all goals by the end of the quarter)
Personalization
Measurement: RMSE (delta between predicted rating and actual member rating)
Key Q1 project: launch mood-based algorithm (a set of recommendations based on the best guess of member’s mood at that time)
Objective: Drive RMSE from .795 to .790 as demonstrated by AB test. (This would be a substantial improvement in predicting member taste for this “smaller is better” metric.)
Original content
Measurement: Percentage of customers who watch at least 10 hours/month of original content.
Key project: Cold start merchandising test (a set of presentation layer tactics to build awareness and trial of new original content, plus collect early member rating data to decide which cohorts to promote the title to).
Objective: Drive metric from 10% to 11%, as demonstrated via AB test.
Watching experience
Measurement: Percentage of members who watch at least 40 hours of content each month.
Key project: Launch “Netflix Party.”
Objective: Engage at least 5% of members in the “Netflix Party” feature by the end of the quarter. Drive watch percentage from 50 to 51%, as demonstrated by AB test.
Interactive story-telling
Measurement: Percentage of members who watch at least 40 hours of content/month.
Key project: Real-time branching prototype tool to enable artists to build interactive stories.
Objective: Engage at least 1% of studio partners in building interactive story prototypes by the end of the quarter. (This is a long-term project—it will take years to impact the member-facing metric through more interactive story content.)
Note: I indicate “progress against metric as demonstrated by AB test” because many other projects will affect the same metric. The AB tests prove the ability of that project to move the metric in isolation.
Last note: I don’t work at Netflix but I assume most folks know the product. All of this is made-up to illustrate how to create OKRs. Please don’t buy/sell Netflix stock based on this!
The Value of OKRs
I have outlined how you can generate quarterly OKRs from my SMT and roadmap frameworks. (You can learn a lot more about these product strategy frameworks here.)
There’s room for debate about the value of OKRs, and I acknowledge that the majority of companies use them. OKRs help build high-level alignment across a company and give individuals a sense of how their work contributes to success. Most feel that a specific objective — how far you will advance your proxy metrics over a specific time period—is key to motivating teams.
I’m suspicious of forming specific objectives. Imagine it’s 2005, and you’re the head of product at Netflix. Your job is to improve monthly retention, which is the high-level engagement metric that all the other proxy metrics feed. In 2005, 95% of members continue with the service each month. How much will you improve retention each quarter? That year? How good can monthly retention get in the long-term? Is there a natural rate of monthly churn you can’t improve on? These are hard questions, especially when you are still trying to discover the levers that impact retention.
I’m happy to make guesses to test intuition, but creating a specific objective feels a lot like a forecast. A forecast communicates more certainty than actually exists.
I can answer the question today. From 2005 to today, monthly retention improved from 95% to 98%, and Netflix keeps trying to improve this topline metric. The progress was glacially slow, and it would have been nearly impossible to predict specific progress in any given quarter or year.
As far as the value of setting objectives to motivate teams, I choose to trust that individuals are self-motivated to build a better product that will “dent the universe.” Reed Hastings, the co-CEO of Netflix, highlights the need for this as he explains what he got wrong at his first startup:
"What we failed to understand was that by dummy-proofing all the systems, that we would have a system where only dummies wanted to work, which was exactly what happened." - Reed Hastings.
This initial failure motivated Reed to build a culture of “freedom and responsibility” at Netflix. This is why I choose to trust that individuals are sufficiently self-motivated, well-formed adults.
Sometimes OKRs feel a bit like “dummy-proofing” to me. But if your company finds OKRs to be a helpful tool that provides sufficient value, given the time and energy they consume, I won’t argue. There are many paths to success.
If you’d like this newsletter delivered to your inbox each day, click the button below:
Thanks,
Gib
PS. Click here to give feedback on this essay. (It only takes one minute.)
PPS. Click the button below to ask and upvote questions: