Why do we care about OKR ?

Managing a company’s performance relative to goals is important. OKRs (Objectives and Key Results) have been found by the tech industry to be a good way to do that. See https://en.wikipedia.org/wiki/OKR for a history of OKRs. See Measure What Matters, by John Doerr for a great read on how they started to be used at Google. 

By using OKRs correctly, we can align both within groups and across the company on our strategic direction and our tactical implementation of that direction. We can also chart how that direction changes during a period, and we can measure ourselves on how we did. Just like a feedback-and-performance culture is imperative to have at the individual employee level, it’s also essential to have at the team/group/org/company level.

OKR Creation

Creating OKRs should be a natural part of leading your team. OKRs, at their essence, are a way of understanding company priorities, planning what you’re going to get done, communicating that to others, tracking what others say they’re going to get done, and then everybody getting together and saying how they did. That’s just good leadership.

An OKR list can be in the Wiki / CODA, a downloadable Google Drive Word Doc or Excel sheet, or a Gdoc file, among other things.

Definitions and Guiding Principles

• An Objective (O) is something we’re trying to accomplish, e.g.:
  • Deliver our services such that customers find them generally available, and don’t find that any lack of stability interferes meaningfully with their use of our services. 
• A Key Result (KR) is something that we wish to accomplish in service of our Objective. This is a goal which the team can commit to and they are able/eager to be measured on. Some guidelines:
  • Should be SMART: Specific, Measurable, Achievable, Relevant, and/or Time Based. 
  • Should be something which you can say, without a doubt, was accomplished or not.
    • Some KRs are binary – i.e. “Have no outages during the period”. 
    • Most KRs are number-based, i.e. “Have no more than 15 minutes of downtime per calendar month of the period.”
    • Typically either an absolute value of a single KPI or set of KPIs or a delta in them. E.g. “Increase the MRR from X to Y”.
  • Less is more – focus on the big issues, the “whales”
  • Good Key Results measure: 1) Outcomes, not output, 2) Results, not Tasks. 3) Values, not Activities.
  • Customer/Stakeholder/Employee Focused
    • Customers: We must delight our Customers.
    • Stakeholders: We must respect our investors for the faith they have shown in us
    • Employees: We must create environments and jobs which fulfill our employees desire for a productive, safe, and enjoyable workplace. 
  • Integrates the whole product as seen by customers: Tech+Product+Business+Operations
  • Is NOT a feature. Shipping a feature may enable a key result or even an objective. Instead, phrase your key result as what you are trying to accomplish, and shipping that feature helps it (or may not!). 
• An Aspirational Key Result (AKR) is a goal which the team set which is aspirational i.e. a stretch goal. Typically there is no defined or realistic way to achieve these Aspirational Key Results at the beginning of the OKR period. Often, we need to make a step change to reach the required final outcome. An Aspirational KR should typically be binary (met or not met), rather than having a set of Green, Yellow, or Red criteria. 
• A Key Performance Indicator [KPI] is the business or technical indicator being measured – it is almost always a number. See https://en.wikipedia.org/wiki/Performance_indicator for definitions of this. A KPI value in itself is not good or bad, or green or yellow or red. It’s an unemotional, unbiased fact, agreeable to by all.
• KPIs must be measurable on the first day of the measuring period, and preferably should be measurable in the prior period in order to measure delta values.

Grading OKRs

Objectives are not gradable, as they are meant to be worded in a way which doesn’t reduce to numeric or binary measurement but is instead visionary. However, each Key Result must be gradable – to see whether the “Key Result” was achieved.

When choosing the value of the KPI(s) to be used to grade the KR, the value(s) should be set slightly above what the teams think they can achieve with the current and expected/planned software quality and operational procedures. Achieving this result means the KR has been achieved. Most KRs should have a (subjective) 70%+ chance of being achieved i.e. they should not be certain, but still be somewhat stretch goals. In return, leadership and stakeholders understand that the business goal of KRs is not to grade employees, which will produce negative outcomes, but is instead to produce both the right leaning forward behavior and won’t look for more than 70% completion as successful delivery. I.e. a culture of being proud of being right and proud of being aggressive on goals vs a culture of being scared of being found wrong and graded – which will produce a culture of sand-bagging and conservatism. Nothing we do at our current company is sure, and pretending that we can set goals that are sure of being achieved is the path towards slowness and mediocrity. If something that would not have been reasonably predicted and was largely out of your control happens that makes you miss a KR or Objective that you and your team won’t be unduly penalized.

Each team is also encouraged to put forth Aspirational Key Results (AKRs), which are results which would happen if either the team was very lucky, or if their operational excellence efforts paid off handsomely. The AKR is meant to be the North Star of what the team would like to achieve to delight customers.

One incorrect assumption is that each KR should automatically be better/more strict than the period before. This is not true. For the good of our business and our customers, we may make short-term decisions to sacrifice small amounts of stability for longer-term/strategic goals. Or, we may make a mistake and deploy something which makes the service fundamentally less stable – and need to fix it. As stated above, the goal of the KRs is to reflect the reality of what our current software and operational procedures can provide, while incenting every team to be better in order to delight our customers.

Writing OKRs

Writing an OKR is reducing to written form what you probably already know on your head. That said, having something that you can focus on, track, and that others focus on and track the same way can be challenging. It breaks down into 1) What Objectives are you trying to accomplish, 2) What Key Results will you create to show you have? I.e. what do you need to change to accomplish that, and 3) What Key Performance Indicators will you use to prove the Key Result has been achieved?

What Objectives are you trying to accomplish? 

• Determine the needs represented in the Objectives and Key Results of your stakeholders. These are often the company or divisional OKRs. You need to consider whether each one should be “inherited” to your team – i.e. do you have a role to play in achieving that objective.
  • This is particularly important for Platform/Centralized Engineering Teams (e.g. DevOps, SSO) or other teams that don’t have customer-facing objectives. You must understand the needs of the groups who use your outcomes and create OKRs that satisfy them.
• Determine the needs of your team, independent of those stakeholders. These can be tech debt your team needs to pay, features you want to add, operational excellence initiatives you need to execute on, etc.
• Determine what needs to be done either for you by somebody else, or by you for them (interdependencies).
• Avoid discussing what you will do, or how you will accomplish it. Stick with sentences that could start with “In 2020 Q4, I am proud that we…”.

These are your Objectives.

What Key Results will you create to show success? 

In order to accomplish your Objectives, something almost always needs to start, stop, or change in some way:

• What tangible things can you change?
• Can you reduce the number of defects released to the production system?
• Can you increase the NPS?
• While Objectives should not change during the period, the HOW of doing them should be allowed to change as needed.

These are your Key Results. 

Note that even for these, you should resist the urge to get too much into the “how” – i.e. specific features. Delivering a Feature or Task list is not a Key Result – rather,  the outcome of the Feature or Task list should be. Deriving the feature or task list that will achieve the Key Results should be left to your team and you as your brainstorm and should be flexible during the OKR period. Using the example above, if you think that you can increase the NPS by doing one thing, but halfway through the period, you come up with another way, that’s totally ok (and in fact encouraged) – because you’re still accomplishing the Objective – and the Key Result was to increase NPS. 

Resisting making your KRs a list of tasks or features is the hardest part of writing OKRs for most people.

What Key Performance Indicators will you use to prove the Key Result has been achieved?

• Though there are some KRs that are amenable to purely binary interpretation, most are better measured with numbers that can then be used to determine whether the KR was achieved.
  • The best measurement is a dashboard (think Datadog graph) that you can encode the value of the KPI into, and put a binary red/green of whether the KR is met onto. Then, for each grading, you and others can just look at it. No need for transcribing anything into a spreadsheet. Just put the link and be done.
  • Second to that is a measurement or value on a dashboard which you sample regularly and transcribe into your Gdoc, Wiki, or whatever. 
  • Third is a measurement which takes manual work to create, such as a SQL script that has to be run
• A key (and required) element of a KPI is that it be available at the beginning and end of a period and able to be sampled throughout the period. It is even better if it has a value from the prior period that can be used as a baseline.
• Others must be able to measure the KPI – and it must be precise and explained. For example, if we just have a metric of “Apdex”, some readers of the O/KR/KPI will think “Apdex with Threshold X”, while others will think “Apdex with Threshold Y”. This might result in some feeling that your OKR was met, while others disagree. Not only that, but if you give others freedom to misunderstand your KPI, you’re giving them freedom to change the goal posts through that misunderstanding, during the review period.
• A motivated team will constantly find more accurate and lower-work ways of measuring KPIs.
• Note that you can change which KPIs you use to measure success of your KRs during the period. If you find a different way of measuring the Key Result, you’re encouraged to always measure better. This will be difficult to change if the new measurement wasn’t available at the beginning of the period, but if it’s a better measurement, ask anyways.

Things that can be easily and precisely measured and that show whether a Key Result has been achieved are your KPIs.