Working In Uncertainty

Participation and Key Performance Indicators (KPIs)

by Matthew Leitch, first published 14 December 2007.

Introduction

This article is about involving people from multiple organizational levels in selecting and revising Key Performance Indicators (KPIs), specifically to benefit from knowledge sharing. Involvement in KPI selection is often driven by negotiation about targets and related pay, but negotiation is not addressed in this article.

The main reasons why you might want to increase knowledge sharing participation in selecting KPIs include:

• developing a better understanding of what drives performance and selecting more effective KPIs;

• managing uncertainty around the choice of KPIs by bringing more knowledge to bear initially and over time, and by encouraging actions that will provide better knowledge in future;

• increasing every individual's understanding of what drives performance so that their actions and decisions from day to day are guided by this understanding; and

• developing a culture of mutual respect and sharing in which more employees are encouraged to participate usefully in management, but without a shift in power.

Organizations run by centralized command and control may struggle to make this work well, while organizations that tie performance pay to achievement of performance targets that are set in advance limit their freedom to change KPIs.

That does not mean that performance tied to achieving fixed targets is entirely incompatible with participation. One reason for using participation to share knowledge is to compensate for certain problem areas that may arise with target based pay. If it is difficult to measure something that is important to performance, or if people need to be reminded of long term implications of their actions, then motivation by rewarding performance against targets may not work well on its own. Participation is a complementary way to get people thinking and caring about what really drives performance. In organizations that do not pay for achievement of targets it is participation that needs to do most of the work.

If you want to empower employees while still leaving an active role for the leadership team then participation of the kind suggested here should be of interest.

Where is the uncertainty around KPIs?

It may be obvious to you already that there is uncertainty around KPIs generally, or specifically around the KPIs of your organization. Perhaps there is work being done now to select KPIs, with ideas sought from anyone who might be able to help. Perhaps KPIs were selected some time ago but increasingly people are questioning whether they are appropriate.

Alternatively, it may be that a set of KPIs has been selected and generally people feel that they are a good choice and not about to change. Even so, there is some important uncertainty to consider, for these reasons.

A set of KPIs amounts to a theory about what drives performance. It says ‘If we can improve performance on these KPIs then that's a good thing.’ What ‘good’ means is itself one of the hardest things to be certain of and is also part of the KPI set because, typically, some KPIs are thought to be drivers of others, and some are considered the nearest thing to an ultimate measure of success.

But, what drives the KPIs? Sometimes it is thought to be other KPIs, but not always. The things that drive KPIs are themselves candidate KPIs, and what in turn drives them? At some stage the thinking typically goes from confidently selected KPIs to tentative theories about how things might work and what perhaps could be measured.

Uncertainty is found on the boundary of even the most obvious and well established sets of KPIs. It is often the case that management teams at some levels feel they have a solid set of KPIs, but management teams at other levels do not.

Typically, our tendency is to behave as if we are more certain than we really are or should be. Perhaps a critical review of existing KPIs would reveal more doubts, alternative theories, and potential opportunities for improvement than are currently acknowledged.

Commitments

Given that some degree of uncertainty exists, at some levels, at some times, it follows that we should be looking for better ideas and that our commitment to a particular set of KPIs should last only until we have a better set we can implement.

In view of the difficulty of understanding the world, knowing what will make us successful, and making strategies, I do not think it is reasonable to be unconditionally committed to:

• a particular set of goals;

• a particular strategy;

• a particular set of metrics; or

• a particular view of how things work.

Nor should leaders expect people to commit to these relatively short lived and uncertain choices. Instead they need to be continually under review. Becoming attached to them could be a problem.

However, it is reasonable to commit to:

• learning;

• working together cooperatively;

• listening to colleagues (up, down, across) and others;

• contributing to the thinking of colleagues (up, down, across);

• adapting and improving;

• keeping an open mind; and

• carrying through strategies and working with goals and metrics in a spirit of experimentation – determined to learn the lessons but not committed to continuing with a strategy, set of goals, or set of metrics whatever the results.

From this point of view, a leader in an organization should be happy to work with people who go with these principles. The temptation to try to persuade people to commit to particular sets of goals, strategies, and so on should be resisted. However, leaders should expect their input to be listened to carefully because of their high ability to contribute.

The value of sharing knowledge

Considering uncertainty also helps to explain why participation can be so useful. The uncertainties around KPI choices are different for different people because of their knowledge.

Participation does not have to involve everyone, but there are reasons to aim for a wide range of people.

Typically, someone at a high level in an organization thinks in high level terms and is exposed to aggregated information. They are well placed to see the big picture. They may also have past experience with operational details and may even take time to refresh this experience from time to time. Nevertheless, the longer one spends in a senior position the harder it is to be in touch with details.

In contrast, a new employee in a very junior position may have little real understanding of the big picture but even this employee is an expert in at least one thing: what it's like to be a new employee in a very junior position. That may be crucial expertise, particularly in businesses that have a lot of relatively inexperienced employees and puts them in continuous contact with customers.

When it comes to analytical and creative ability the situation is likely to be similar. The senior, high level employee has usually spent more time thinking about the strategic possibilities, looking at research, making plans and so on. It is also quite possible that someone in a senior position is better educated than someone in a very junior position.

But, again, even the most inexperienced and ill-educated employee has one special ability: the ability to identify ideas that are too complicated or subtle to be grasped and acted on by inexperienced, ill-educated employees.

Between these extremes there are countless reasons why an individual may have a piece of information that is crucial to strategy and KPI selection. For example, perhaps they have close contact with key customers, or personal relationships with lenders, or a special understanding of a technology with huge potential implications.

Any of this knowledge could turn out to be crucial to success so, ideally, we would like to make use of it if we can do so without being overwhelmed by information and relationships.

At the very least, adjacent management teams should participate with each other. It is also possible to involve representatives from organizational levels not usually involved in management.

Starting participation

Whether participation is sought through large face-to-face meetings, by circulating documents, or by some other way of getting people together the keys to success must include overcoming the natural caution that usually keeps people from speaking freely in the presence of people higher up the pecking order.

Understandably, the most common reaction to a senior executive asking ‘What do you think?’ is a polite but awkward silence, perhaps broken by a rebel raising a familiar negotiation battleground or trying to score some points. This may even lead to a heated argument but it is not the useful participation needed to share knowledge.

One good way to start things off is for the leaders to present their initial views and identify very clearly where they themselves have doubts and would like input. They should stress the value of the knowledge people have in different positions and from their different experiences. They might also mention that often the biggest strategy is based around small but crucial details.

They should also make clear that once consideration of KPIs has started it will continue indefinitely, particularly as new experience and experiments will often be the best way to settle doubts. They should invite people to come forward at any time in the future to supply fresh information or ideas, especially if they concern one of the major areas of uncertainty.

Just explaining your thinking can be useful, if it is done in an open and honest style, acknowledging past mistakes and current limitations, in a way that respectfully invites others to think through the same issues.

Following on

Suppose the top team starts the ball rolling with their choice of metrics, based on some (preferably explicit) mental models of how things work. Their theory is that if values on certain metrics are improved (raised/lowered as appropriate) then that will be a good thing.

If they pass this thinking on to the next level(s) of management teams, particularly if they invite them to participate appropriately, then the reactions of those teams will usually include:

• questioning the theories of the top team, perhaps based on evidence that the top team did not have;

• thinking up their own theories about what metrics the top team should have chosen; and

• accepting at least some of the top team's metrics and trying to extend the thinking to what would bring about improvement in them, leading to new theories about what further metrics would indicate progress towards improving performance.

They should be encouraged to feel welcome to feed all this back to the top team. When it does go back to the top team their reactions will include:

• sharing more of the evidence on which their theories were based (to defend them or stimulate more participation), but perhaps also reconsidering in the light of new evidence; and

• questioning the theories of the lower team, perhaps based on evidence that the top team has. (Sometimes the wider view is informative.)

The top team should also feel safe to feed their reactions back, particularly where this involves being open about what they do not know.

All of this is purely the objective, rational side of life with no agency problems, and yet still there is dissent and doubt. This doubt will and should continue even as plans are laid and carried out, because we always need to be learning.

Teams at all levels can have ideas about what the metrics should be at any level of detail, but they will often have complementary evidence. So, it's good to get people talking and ideal if it is to share knowledge and work together rather than just to negotiate.

Developing the KPIs

Typically what happens in performance measurement projects is that an initial set of ‘top level’ KPIs is chose and these then contribute to the choice of further sets of KPIs relevant to management teams at different levels and in different activities. The idea is to link these in some way.

In this article I'll call this ‘development’ of the KPIs.

Even if there is no disagreement and a set of KPIs and its underlying causal logic remain completely consistent with those of another management level or area, it can still develop. These are at least three mechanisms for this:

• Mathematical decomposition: A single KPI can be split into more than one using a mathematical formula that defines the initial KPI. For example, overall profit for the year is the sum of the profit of each business unit so this split is obvious. Sometimes a KPI is calculated from a bundle of related measurements, giving another obvious way to decompose the main KPI.

• Causal extension: The causal model underpinning the selection of KPIs (which may be only implicit) can be extended by adding causes, effects, or chains of cause and effect between existing nodes. What people often call ‘root cause analysis’ can help this along. The idea is to develop the thinking to introduce KPIs that are more directly under the control of the management team that is to use the KPIs, but are causally related to others.

• Partitioning: As the set of KPIs and its underlying causal model grow it may be necessary to split them into smaller parts. A common way to do this is on the basis of the organizational units most able to influence the KPIs in each set. Quite often this can be done because KPIs form clusters, within which there are many strong causal links, but which are not so strongly linked to other clusters. This is called loose coupling and suggests natural boundaries for splitting KPI sets and their underlying models.

In practice, additional thinking often is not entirely consistent with the initial thinking and often for a good reason. We should expect feedback from teams to cause revisions to the initial thinking but usually with a time delay that means inconsistency exists for a time.

Changes over time

Through participation it should be possible to arrive at a reasonable starting set of KPIs, or multiple sets for different teams, with an understanding of the areas of doubt and even some ideas about how to learn more.

Over time several good things can happen that will lead to new ideas for KPIs and the thinking that goes with them:

• More time to think: It can take time to think through the many connected issues, and more time for people to get comfortable with the idea of participation.

• Data to study: Sometimes it is feasible to study statistical relationships between KPIs and also involve other potential indicators. (Is it even correct to regard something as a ‘Key’ indicator if its relationship to performance is no more than a gut feeling? For more on this and how to study relationships read ‘Better management of large scale financial and business processes using predictive statistics.’)

• Experiment: Studying statistical relationships identifies correlated variables, but to understand the direction of causality and discover what can be done to exploit relationships it is usually necessary to experiment. This is rarely experiment in the strictly scientific sense, but should still be with the deliberate intention of learning from experience.

• Changes in circumstances: Changes in circumstances may reveal new information, trigger new insights, bring in new people with new ideas, and so on. KPIs should adapt.

Ideas for improving or adapting KPIs can happen at any time, not just to a convenient annual schedule. Obviously, all other things being equal, the sooner you can move to using improved KPIs the better. You can make the changes as soon as possible (by next month, typically) or defer changes and introduce them annually. Which you choose to do depends on a number of factors, including these:

• Has the KPI been fixed by pay negotiations?: Sometimes a KPI has been used as part of a formula for performance related pay. Typically, this involves setting targets in advance and paying some kind of bonus if targets are met. In this case it is very difficult to change the KPI mid-year. Other organizations judge performance in retrospect, given conditions as they actually emerged. This makes it possible to revise KPIs during the year so that by the year end when performance is evaluated it is with the aid of the best available KPIs.

• Is there a risk of KPIs being manipulated?: One of the best known problems with performance targets is that people learn to deliver improvements in the things measured but not necessarily through genuine improvement. Sometimes it is by fudging the figures and sometimes by making cynical trade offs. Curiously, both these tricks are more difficult to do if the KPIs might be revised during the year towards KPIs that better reflect true performance. Fixed KPIs are more vulnerable.

• Can we calculate past actuals?: KPIs are much easier to understand if you can see how their values have changed over time. If a new KPI is introduced but past values are not available it can be hard to understand the KPI. Happily, modern computer systems usually store far more data than is used and it is often possible to work out past values of new KPIs and show them as if the KPI had been in use for some time. An alternative is to introduce the new KPI immediately but retain any KPIs it will eventually replace while a track record builds up.

If you want the flexibility to change KPIs at any time then it makes sense to choose technology that will make that possible. IT system developers can be very good at delivering efficient solutions that precisely meet stated requirements, but on this occasion such narrow delivery is not helpful. Operational systems usually hold vast amounts of data that never get used in performance measurement, so what you need is a simple, flexible way to calculate numbers from those data.

For speed and efficiency, do not allow people to assume that a KPI, once chosen, is fixed forever and needs a comprehensive and fully automated collection and presentation system. Perhaps you can find out what you want to know with a limited manual survey for a few months after which you realise that the indicator is not as helpful as expected.

Appropriate language

If this kind of participation is what you are aiming for then avoid some of the jargon often used in performance management to describe this activity. Here are some phrases to avoid, with suggested alternatives:

• 'Cascading': People often talk about ‘cascading’ objectives or KPIs down an organization. This word suggests a purely downward flow and subtly reinforces the idea that each organizational level has its own territory. Neither is really true as we can often contribute to the thinking of people above and below us, and should do. Say ‘developing’ instead.

• 'Decomposing': As with ‘cascading’ this gently promotes the idea that the upper levels are fixed and so all subordinates can do is work within them. Say ‘developing’ or use a word for the specific type of development involved (i.e. revising, extending, mathematically decomposing, or partitioning).

• 'Buy in': If someone says they want to involve people in order to get ‘buy in’ then alarm bells should ring. Too often this is a persuasion ploy designed to promote a particular choice of KPIs rather than genuine participation. It is probably best to avoid expressing this idea in any form of words.

Summary

The approach to participation described above means that leaders do not rule by command and control, but neither do they stand back as mere facilitators or coaches and allow subordinates to run the organization. Everyone has a role because everyone has a unique perspective and the big picture is tightly connected to the details.

In my time as an employee I had the opportunity to participate on a number of occasions and look back on them all as happy, productive experiences. Sadly, I was also subjected to many more experiences where the leaders clearly had no interest in what their subordinates might think or know, and instead were focused on promoting their own ideas by whatever persuasive devices they could use, including mock participation.

One of the hardest parts of participation for a leader is to be open about what you do not know. Often we feel under pressure to be ‘confident’ and knowledgeable to an unreasonable extent. Perhaps the key is to be confident that learning and adapting are good, even if we do not know what we will learn or how we will adapt.

I hope this article has encouraged you to prefer genuine participation and the good experiences it brings.

Further reading

For a broader look at ways to manage uncertainty in performance management have a look at ‘How to embed risk management into performance management and strategy making’.

Made in England