Working In Uncertainty

The basics of managing risk and uncertainty

Contents

Common sense good management

Have you noticed that things rarely turn out as expected? The most likely outcome is rarely what happens. Usually it is something else - one of the innumerable alternatives, each less likely than the expected outcome, but collectively more likely than the favourite.

Modern society and systems are so large, complex, and fast moving that we face an unprecedented problem of uncertainty, and it's getting tougher. Companies, charities, and governments are under intense pressure to handle uncertainty better and are trying various methods and programmes to do so.

In the last two decades it has become fashionable to talk about 'risk management' but 'risk management' is not a method or technique, or even an activity you should do. 'Risk management' is just what you can achieve if you manage well despite conditions of uncertainty. It just means your 'risk' is managed, where 'risk' is something to do with bad things that might happen.

Techniques for managing risk better have been developing for at least 350 years - starting perhaps when early mathematicians first applied their techniques to gambling games.

In the last few decades two particular techniques have been talked about a lot. Starting in the 1950s, one technique is to make probabilistic forecasts about the future then calculate summary statistics from those distributions and call them 'risk'. Over the years several different formulae for 'risk' have been suggested and used but none of these is entirely satisfactory. Happily, most of the work goes into the useful part, which is making the probabilistic forecast, and the risk statistics are a tiny extra step at the end that may or may not add something useful.

The other technique, rising to prominence from the early 1990s, is to make lists of 'risks' and try to 'manage' them. This unfortunately tends to lead people to spend time on separate activities limited to decisions about actions seen as responses to 'risks' instead of getting better at doing core management under uncertainty. Risk Listing is also undermined by several serious logical and practical flaws.

When I talk about the need to manage risk better most people have the same reaction: "Dealing with risk and uncertainty is everyone's job! It's just good management. I do it all the time." Quite right. The daily decisions and actions of people throughout organizations are where most of the risk management action is. Special corporate programmes with workshops and risk registers are comparatively low in importance and frequently a waste of time that could be spent doing something more useful and less confusing instead.

It should be obvious that the 'risk management' skills of individual managers are important to an organization and that recognizing the contribution of everyday 'risk management' decisions should be an important part of any organization's formal 'risk' programme.

The scope for improvement

What is less obvious is that most of us are not as good at 'risk management' as we think. There are three reasons for this:

  • Natural reasoning flaws: Psychologists have discovered many persistent flaws in the way we think about likelihood. Even Professors of statistics make big errors when they rely on their judgment. Typically, we have an overly narrow view of what might happen in the future. Finding ways to counter these subtle but important errors has been so difficult some have concluded that they are innate and inevitable.

  • Pressure from other people: The impact of our individual mental limitations is probably small compared to the effect of pressure from colleagues at work. Almost daily we are pressured to appear more certain than we really are and to fix our minds on one future outcome only. The result is that we suppress uncertainty. We don't talk about it, think about it, or manage it as much as we could and should.

  • Lack of knowledge: Finally, we get little education or training in effective techniques for managing under uncertainty. Things are changing but there's still a long way to go. In my education more time was spent trying to teach me Latin than uncertainty. Most people don't even know what they don't know.

Thanks to these issues uncertainty/risk is frequently overlooked or mis-managed by individuals and large organizations alike, with a variety of consequences from the stress you feel at the end of the day to the occasional collapse of industries and economies.

So, what to do about it?

How to improve

The way to improve is simple in principle: change the way you do things so that you perform better in uncertainty. Change habits, tools, procedures, etc so that these changes are sustained.

But beware, if you just review the way you work, looking for possible improvements, it is quite likely that you won't find much. Even if you conduct a rigorous analysis to find the most important areas of uncertainty in your life and think about them hard the improvements you come up with might not be ones you find attractive.

This is because many of the improvements you can think of now are already incorporated in the way you work. To find something new and worth doing you need to take on board some new ideas and insights before you start looking at your work again.

This is a massive topic but you can make progress with just a little learning, and make more progress the more you learn. The next sub-section has some examples of where uncertainty is important for different people. The sub-section after that illustrates just a few possible improvements.

Some illustrative uncertainty

The first illustration is set in a working environment where uncertainty is reduced as far as possible - a factory.

Illustration: A factory job. Imagine someone who works in a factory supervising assembly of electronic parts to make audio gadgets. The assembly work is largely automated. A factory is a special environment where, traditionally and quite correctly, a lot of effort goes into making the inside of the factory as stable and predictable as possible. For example, the difference between building a house on site from small components and building one from large pre-fabricated sections is that the prefabricated sections can be made indoors in a familiar, controlled environment.

The remaining uncertainty in factory operations, though small, is important and needs constant attention. Every software glitch, every machine breakdown, every input that doesn't meet specification, and every item produced that does not conform to the tolerances specified for it is a problem that needs to be fixed urgently. When will the problems happen, and where?

More uncertainty comes into play when changes are made to the factory, and factors like customer demand, competitive actions, and technological developments are obvious sources of huge uncertainty.

Experts in manufacturing may have disagreed with the above claim that every effort is made to drive uncertainty out of factories, and it is indeed a rather traditional view. In 'lean' manufacturing the inside of the factory is exposed to uncertainty around demand and supply through reducing or eliminating stocks. Instead of producing to a plan and using stocks to insulate the factory from surprises about customer demand and supplier deliveries, the factory is set up so that customer demand 'pulls' production and supplies are delivered by suppliers 'just in time' (or else). The stock holding efficiency advantages of this outweigh the reduction in regularity within the factory.

Near the other extreme, consider the situation of a freelance photographer.

Illustration: A freelance photographer. It's not easy being a freelance photographer, unless you've established a strong and enduring position, which most have not. Customer habits are changing along with technology. The business used to revolve around selling prints - remember those? In the early days most people did not have a camera. Now most people have more than one, not forgetting the one in their mobile phone, the webcam in their netbook, and the compact digital that's taken along for special occasions. These cameras do all that fiddly exposure stuff for you. Consequently, many people have thousands of pictures they have taken and no wall or shelf space left for yet more photos of their loved ones.

Specific photography services are sometimes little more than fads. Favoured styles come and go. The standard portrait used to be a straightforward head shot with a mushy grey or blue swirly pattern behind. Now it's all black or white backgrounds and funky or moody shots with artistic pretensions. Photographers at events like balls and proms used to take your details and promise to send your prints later. These days the prints are done on a portable printer while you wait.

I've given this detail not just to get some sympathy for freelance photographers, but also to underline the fact that the biggest uncertainty for them is what service or services to focus on. Like most small businesses, the freelance photographer is always searching for a winning formula. If the photographer was part of a large partnership of similar freelancers the partnership could compensate a handful of individual members to experiment with new services and report their findings to the others. However, working alone the freelancer must find some way to navigate the torrent of sales literature from equipment manufacturers and other providers of services to photographers, sift the feedback of his/her own experiences with customers, and decide what to do. If this search goes on for a long time without much financial reward the freelancer has dwindling resources to experiment and becomes increasingly constrained and under pressure. This situation will be familiar to many small business people.

From day to day the photographer faces other decisions under uncertainty. Which sales leads to follow up, and how? Should he/she take up an invitation to take photos at an event knowing that the results could be anywhere between a reasonable day's pay and a total wash out? What stocks of picture frames are worth investing in? Is it worth investing in a new piece of kit? And is this really what he/she wants to be doing until retirement?

Knowing the common reasons for uncertainty can make it easier to spot. In principle, we are uncertain because we have limited control and limited knowledge. However, there are some factors to watch out for that indicate that uncertainty should be high:

  • Novelty - particularly where new conditions are different from conditions that have been unchanging for some time and which you have become accustomed to. It can be difficult to identify and review all the ways in which assumptions about that factor have become embedded in your current behaviour so errors are common.

  • Complexity - particularly when the complexity is from numerous, interconnected elements interacting.

  • Speed - because if you have less time to gather information and think then you will have to guess more.

  • Intangibles - especially psychological things. Predicting human behaviour is extremely difficult.

One of the most common areas of uncertainty concerns what you should be trying to achieve. What would be good objectives, or at least good things to try to influence? What is in your interests? Who else cares? Even if very specific sounding objectives have been laid down by someone more important than you there will still be significant uncertainty there. Those objectives may change, or they may not change even though they should. They may be so high level that they don't help much with the everyday decisions you need to make, so lower level objectives or an objective function are needed, and what should they be?

If you think it will help to focus on the most important areas of uncertainty you face then it is easy to do so by just making deductions from the most important facts you know. Here's an illustration of finding important limitations of knowledge in a company project:

Illustration: Easy deductions. Imagine the organization you work for, which is not a retailer, decides to open its first shop and to locate it in New Bond Street, London. What can you deduce about the areas of uncertainty this involves? Firstly, as this is the first shop the odds are that your employer has little prior experience of retailing to draw on. So that's a fairly suggestive fact straight away. The value of the venture is questionable and so is your competence to make it work. Next, the location. New Bond Street is an area chock full of very expensive shops selling clothes, jewellery, art, and so on. How will shoppers in this area react to your shop? Will they be in the right mood? Are they people you want to have coming into your shop? Easy isn't it! Start with the most important, most distinctive facts ("a shop" and "in New Bond Street" for example) and work from there. You are almost guaranteed to be focused on the right things straight away.

Knowing about the uncertainty in generic management processes can also help. The uncertainty involved in decision making can be illustrated by shopping for chocolate biscuits in a supermarket. (If you don't like chocolate biscuits imagine it's something else that you do like.)

Illustration: Shopping for chocolate biscuits. Typically, we go to the shelf where we usually get the biscuits, pick up the usual product, and put it in the basket. End of shopping. This illustrates an important point, which is that many of our actions are the result of decisions made some time ago that simply become our standard policy (or habit) until something prompts us to revisit the decision. That also means that when we consider decisions we should often be thinking about choosing policies rather than one-off actions.

But, imagine that on this day something prompts you to think again. Perhaps there is no stock of your usual product but you need something. Perhaps there's a big promotion on a similar biscuit that catches your eye and prompts a rethink. Or perhaps the last pack you bought tasted funny, so you've lost confidence in your usual brand.

On the shelves you see a wide range of chocolate biscuits of different types and at very different prices. The prices are clear and certain. However, the biscuits are usually wrapped so that you cannot see them, and even when you can see them, or a photograph on the pack, it is not certain how enjoyable they will be to eat. In some cases you can't even see how large each biscuit is, though you do know the overall weight of biscuits in the pack if you look carefully.

There will be some biscuits you have tried before, some that are very similar to biscuits you have tried before, and others that are less similar.

This is another common situation in decision making under uncertainty: important factors in the decision are uncertain to an important extent, perhaps more so for some alternatives than for others.

You need to make a choice. What are you really looking for from these biscuits? Typical attributes of biscuits that might be important include low price, size (calories or feeling of being full without calories?), salt content and other health-related factors, enjoyment from flavour and texture, a sense of luxury (perhaps for entertaining guests), organic production, and ease of storage/transport (individual wrapping for lunchboxes).

With attributes of alternatives, typically:

  • there are many potentially relevant attributes but not all are relevant so you have to think about what matters at all;

  • most attributes are best thought of as a continuous scale of measure (e.g. price, quantity), while some are best thought of as categories (e.g. individually wrapped or not, organic or not);

  • you don't necessarily have a required level of performance on each attribute, though you might know you prefer higher (e.g quantity) or lower (e.g. price);

  • the attributes are not the same, in principle, as the value you might place on the bundle of attribute values provided by each alternative, but the relevant attributes contribute to that value.

Analyzed out this way it is easy to see that there are lots of things you don't know for sure about many of the biscuits you might choose.

Now warming to your task you start to read the labels more carefully, checking contents and price per 100g, and looking at 'serving suggestions' close up.

This is typical decision making under uncertainty: you can usually reduce the uncertainty to some extent by research and thought.

Meanwhile, thinking about what you are looking for you realize that the biscuits need to look a bit luxurious because you will be offering them to an important guest. Some biscuits are just not impressive enough so they are out of consideration.

You pick up a smart pack of organic Duchy Originals and as you do you reach for your wallet to check you have enough money. A moment of panic follows as you think that you have lost your wallet but then realise you failed to bring it with you. All you have is a £2 coin, which is not enough for the Duchy Originals.

This illustrates another aspect of decision making. You might think that each attribute can be weighted to show how important it is. Not true. What you can weigh is exact levels of each attribute. To prove that this is the case, imagine two types of biscuit, one of which is £2 and the other is £2.05. It's a tiny difference in price but unfortunately it's the crucial difference because you only have £2. Conversely, custard creams and chocolate digestives vary a lot in luxury level, but neither is impressive enough for an important guest so the difference between them is unimportant, despite the fact that it is important to get something that is luxurious enough. In another situation, had you brought your debit card, there would be no special region of price, but just a desire to keep price down.

In short, the relationships between attribute levels and their value to you are usually non-linear and can be quite interesting and varied. They can be changed by the situation and are not a fixed aspect of your personality.

In this example of biscuit buying, gaining more information about the context and thinking about it (i.e. available cash and the impending guest) helped to clarify values and improve the decision.

Imagine that you now weigh your alternatives and decide that the chocolate coated macaroons at £1.65, if served on a nice plate, are the best available alternative. You reach for them, but wait, there's another shop in the shopping centre that might have something better. Is it worth holding off and visiting it before making a final decision?

Once again you are confronted with an important uncertainty. You think through the facts you know about the other shop. It's a specialist bakery with hand-made biscuits. There could be some very different and pleasing alternatives, but you're worried about the prices.

Almost all real life decisions involve this situation. Rather than deciding between a fixed set of alternatives we have to decide between the current alternatives and a further alternative, which is to carry on studying the decision and creating new alternatives.

You check your watch and find you have just enough time to go to the bakery and come back to the supermarket if necessary. The supermarket alternatives will still be available if you try the bakery. For the sake of illustration, let's imagine that you decide to take a chance on the bakery and head over there without buying anything in the supermarket. At first you are disappointed to find that they offer very few biscuits and they are rather expensive. Happily, their macaroons are nicer and cost £1.99 and you are just about to ask for them when you spot some decorative buns at £2 for four. You never intended to buy cake. Biscuits were always your plan. But they're so attractive and at £2 it seems like fate. You buy the buns.

And so this illustration ends with yet another observation that is typical of decision making under uncertainty. The available alternatives can cause you to revise the way you value alternatives and expand your ideas about what alternatives to consider. In this case, the thought that you could serve the macaroons on a plate led you to ignore the design of the packaging (potentially saving quite a lot of money), but then the sight of the buns jolted your thinking even more. When you first saw the buns you didn't even have a way to value the buns, which were outside your biscuit evaluation approach, but you quickly adapted it and recognized the buns as a happy discovery.

Looking back over this little episode the points where important uncertainty appeared were:

  • in assessing the attribute levels of the alternatives (where we got extra information and thought about what it implied);

  • in valuing the attribute levels of the alternatives (where we thought about circumstances and the implications of different attribute levels); and

  • in deciding whether to extend the decision making exercise and explore more alternatives (where we again weighed up the chances, using relevant information to make rough probabilistic predictions about what might happen).

Despite these uncertainties and the unusual challenges posed by the wallet mistake, the second shop, and the guest, you and most others would tackle this problem competently if not perfectly. The underlying logic is something that can be seen in our behaviour, even though few can explain it in words.

You already know how to deal with uncertainty in decision making, but you can probably learn to do it better and more consciously, especially when other people are involved and a lot is at stake.

Some illustrative improvements

I've spent a lot of time working in consulting organizations where there is a tendency to think that organizations are best managed according to simplistic prescriptions that seem logical to the consultants, whose thinking is unhindered by reality. The first illustration is inspired by this experience.

Illustration: Super-Organized Stuart and Risk-Aware Rob. Imagine two companies are vying with each other to buy a third company. Each of the competitors hires accountants to perform a 'due diligence' review in which they will interview senior people and look at documents to try to find out more about the company on offer. They have one week, starting Monday with their reports both due on the following Monday morning. The two teams of accountants both have the same job to do, but the team leaders have very different styles of working.

Super-Organized Stuart quickly establishes a written scope and gets it signed off. He gets hold of an organization chart for the target company and draws up a plan of interviews, which his secretary starts to arrange. As usual, he starts at the top with the Chief Executive. He plans the work in a series of logical stages. Interviews will be performed and results documented in the file. A report will be drafted starting on Thursday afternoon ready for an initial review on Friday afternoon. He'll have to work over the weekend but he's confident that the schedule has enough slack in it to cope with anything unexpected.

Compared to Stuart, Risk-Aware Rob seems to be a bit of a mess. No schedule of interviews, no timetable, and a scope document still marked "DRAFT". Rob spent his planning time going through areas of uncertainty for the bid, the scope, and his review. He has already had a telephone interview with the Chief Finance Officer to get information that helped to clear up a lot of his ignorance and requested two meetings as a result of that. The other meetings have still to be decided. Like Stuart, he will start with the CEO on Monday.

Monday morning and both teams get their hour with the CEO. The CEO is impressed by Stuart's confident manner and detailed planning, but also by Rob's realism and intelligent responses. Rob in particular seems to be using information gained to plan the rest of his review. The meeting with Rob ends with identifying the right people to speak to about the issues Rob appears to be interested in. They're not all the same as Stuart's interviewees. Rob ends by saying that he expects to learn a lot and may need to come back for other contacts or explanations. The CEO tells both teams about the importance of the company's treasury operations and their progress since a problem a year ago. Both Stuart and Rob respond that this is outside the scope of their work, but Rob says he intends to check that and asks if it will be ok to ask questions in that area if the scope is changed.

Monday afternoon and Rob decides to cover the possibility that his scope may be changed to include the treasury questions. He calls his office to find a treasury expert. After a few minutes talking with the expert Rob decides it probably should be added to the scope. He puts his views to the client who agrees to come back with an answer as soon as possible. Without waiting for a reply Rob calls his own office again and starts a search for a treasury expert who could be available at short notice to do a day of investigation.

That evening Rob and his team are still hard at work long into the evening as they work through the information gained in the first interviews, considering the implications for their review and for the bid. Rob wants to manage down the uncertainties of the review as quickly as possible and does not want to be caught out discovering problems near the deadline. Stuart and his team are at their hotel, having dinner, on schedule.

Wednesday morning, and Stuart's office calls to say they have heard about a nasty treasury issue and could he cover it in his review? "Out of scope" says Stuart but agrees that, if he receives a revised scope he will include it in his review. He calls his office to try to find a treasury expert at short notice. He finds one, but very senior. Stuart had a tight budget, but it's too late to worry about costs, he decides. Rob's treasury expert finishes his review that afternoon.

Also on Wednesday morning Rob starts to draft his final report, starting with the executive summary. It's too early to finalize any words, but Rob knows that attempting a first draft is one of the best ways to identify information needs, reducing that uncertainty and reducing the risk of reaching the weekend without key facts needed to finish the report. By that afternoon he has 40% of his text and has added some items to his team's target issues list.

Thursday morning and Stuart hears from his treasury expert. Apparently, the key people in the treasury department are in a training workshop in Paris for two days but might be contactable by mobile at lunchtime. Stuart is aghast. He spends much of the morning trying to make contact or find alternative approaches to reviewing the treasury area.

By Thursday afternoon Stuart is still not in a position to begin drafting his report. There are key interviews outstanding for treasury, and also it has emerged that the IT director was unable to provide any detailed information and that further interviews are needed with two managers closer to the action. Knowing that time is running out Start starts drafting anyway and instructs his team members to begin drafting their sections as best they can. Most are still doing the file notes, but recognize the need to start drafting the report.

That night Super-Organised Stuart's team work late into the evening, but Rob is at the hotel with his team, unwinding over dinner, having managed down their uncertainties earlier in the week.

Friday, and more problems for Stuart. Nobody in his team has submitted anything completed for the report and on talking to them he discovers why. His original plan committed the team to a particular level of detail, but now it is clear that they don't have time to do as much. Rob was also disappointed at the difficulty of getting good information but always knew it could happen. His goal was to do as much as possible in the time. Another difference is that on Thursday afternoon Rob deliberately produced a first draft with limited detail but a complete structure. From then on, he could not fail to deliver something adequate and further work will just make it better.

Friday afternoon and things are looking up for Stuart. One of the treasury people has taken time out of the Paris training for a telephone conference and the treasury section has been covered. What a relief. If he works over the weekend he should have something by Monday.

That weekend Stuart works on Saturday and Sunday to re-write almost all the report. He gets review comments from his boss on Sunday. It's not comfortable but in this line of work such desperate measures are normal. There are some points where Stuart wishes he had more relevant facts to clear up issues he is only now noticing, but overall he is happy with the result.

Rob relaxes over the weekend, apart from spending some time on Sunday morning reading through his work. He knows it is easy to make mistakes in the pressure of a review and that Friday is bad time for being careful. He corrects some typos and improves the section on IT. His team also review the report that weekend because Rob says there is a chance of his mis-understanding what they have told him or written in their draft sections.

Monday morning and Rob presents his findings to the client while Stuart rearranges his client meeting for the afternoon. Nobody will ever see both reports side by side, but if a comparison was made it would show that Rob's report is far more focused on the big issues, more insightful, and more detailed where it matters. Stuart's report, by contrast, seems like a fairly even coverage of a checklist of standard issues without much tailoring.

The world of project management is another one with a tradition of simplistic thinking about management. "If only people would define their objectives up front!" says the project manager, forgetting an important fact: people often have unclear objectives at the start because they don't know for sure what they should be trying to achieve. That's something they need to learn through the project. The next illustration is set in the world of projects.

Illustration: Best Practices Bob and Risk Savvy Rachael. Two similar research companies embark on similar projects to implement "knowledge management". Each begins with the idea of creating a bespoke computer system to harness the power of knowledge more fully in their organizations. This is a strategic objective and has good support in both companies. Both hire top project management talent to steer their project to success.

Best Practices Bob is in charge of one of the projects. He is an impressive figure with many years of experience behind him. His knowledge of project management techniques is formidable and his confident manner is just what his steering committee wanted. Bob likes to apply "best practices" on his projects and feels he has an impressive track record of success as a result. Of course some projects fail for reasons outside his control, like the last one, where the customers changed their mind late in the day.

Bob sets up a project office to help him run his project. He's got quality management, requirements management, resources management, earned value reporting, risk management, change management, configuration management, benefits management, release management - you name it. (Project managers will know I'm not joking.) Bob's plan features a pilot and global roll out, so with the risk management and risk budgeting he knows he has risk covered.

Risk Savvy Rachael would agree that Bob has just about everything - except, that is, for a realistic plan. She is in charge of the other project and knows that a two year project like this is much more of a challenge than that. She too has management systems and in fact spends longer on looking at the benefits than Bob. Where Bob was intent on establishing measurable goals for benefits and getting commitment to them, Rachael knows that people have little clear or reliable idea of benefits so early in a change project. She also wants to know how the various interested parties would value other, potential outcomes.

The most obvious difference with Rachael is that she has submitted a radically different plan. Her plan shows 34 sub-projects, each of which will deliver something of use to the company and is thought to be worthwhile on its own, except for 3 which are marginal or worse. Her Steering Committee were initially taken aback by this and even more so when she explained that she expected to change the list of sub-projects as she proceeded. "So what is it you are asking us to approve, Rachael?" asked the Chairman. As Rachael explained the inherent difficulty of anticipating benefits and outcomes of large change projects and gave examples of similar projects and their outcomes the Committee began to see that Rachael was talking a lot of sense. Rachael showed how, with her evolutionary approach, there was no risk of spending money for a year and having nothing to show for it. Rachael's metric of progress was not earned value, but delivered value.

By the end of month 3 Bob's project is going well. Some tasks fell behind schedule but by reallocating resources and moving some tasks off the critical path he has put things back on course. Rachael is doing even better. She has finished the first two projects. One has improved backups for the office servers that will run the new system while the other has put in place a new training course in knowledge management for research team leaders. Already they are being encouraged to share knowledge more effectively using existing systems.

Month 8 and Bob's systems have been specified and some coding has started. He is slightly behind schedule because of user indecision over some requirements. They can't agree on what they want. Eventually Bob had to exert pressure to get a decision. Now they are under time pressure, but Bob feels he should be able to pull things back later and remains confident in his dealings with the Steering Committee. No need to get them worried unnecessarily.

Rachael too found that users were not sure what they wanted. The reason was simply that they didn't know what would actually work for them. By this stage Rachael already has a working system - it's not bespoked yet and its functionality is very limited, but it is usable and some content is already in place. She suggests that a set of experimental features be added and that different ways of working should be tried out by users for three months to see what worked. Seeing a way past the impasse, the Steering Committee agree to this.

Month 10 and already Rachael's users have found two things that seem to work well and attract popular support. First, using the systems to allow people to find out who shares their interests turns out to be simple but vital. Second, they like to have pages on the system owned by named individuals, who are motivated to contribute by the chance to raise their profile and show some knowledge. Bob's users did not have the benefit of experience and agreed to the more conventional approach of collecting documents in a database. Their databases are nearly ready.

Month 13 and Bob's users are testing his databases. Rachael's work is far from over but knowledge management is already much more developed than in Bob's company. Then, to everyone's amazement, both companies are bought by a mega-group from Japan. Both companies come under the same regional head and he wastes little time in explaining that, in his opinion, knowledge management systems are over-rated. Investment in databases is to cease but investment in training and facilitation for knowledge management is to be stepped up.

Bob appears philosophical. "There's nothing you can do about that sort of thing", he says. "If the users change their minds who are we to say they are wrong?" Privately he is devastated and can hardly believe what he sees as his bad luck. His training plans were geared entirely to the system under development. The system cannot be delivered. He has been spending money for a year and has nothing to show for it. Suddenly, his statistics on "earned value" look like a joke.

Rachael's position is completely different. Throughout the project she has been getting together a group representing interested parties to review the discoveries from sub-projects and revisit the original ideas about benefits and goals. As a result her projects have been changing throughout. For example, once it had been recognized that putting people in touch with each other was a vital benefit of the system, the benefits of this and of the activities people do when they then get together had received more attention in her later sub-projects. The implications of the new instruction to stop work on databases are minimal as she already has a working system - though not perfect - and already has plans for several training and facilitation sub-projects, all of which were designed to stand alone.

Finally, here's an example of using some simple techniques to select and run conferences:

Illustration: Running conferences. Imagine you have a job in the Conferences department of a thriving society for people who study lichens. Your role is to gather ideas for conferences, arrange them, sell them, run them, and report on the results obtained. You've been doing the job for a year and it's turned out to be a lot more difficult than you expected. On top of all the stress of getting everything ready for each conference there is the problem of deciding whether to go ahead with a conference idea. It all depends on how many people attend, which is proving very difficult to predict and nobody in your department seems to have more than a vague idea, even though they still seem confident they can predict demand and always look surprised when things turn out differently from their expectation.

One of the most awkward situations is to have to abandon a conference because of poor take up after it has been advertised and some people have signed up. This has happened three times in your first year and it is more costly than you expected because the venue is usually paid for in advance and only a small part of the fee is refundable.

Thinking about the uncertainty you face it is obvious that it is mainly uncertainty about demand. This is generated by the intangible nature of the interests of the scientists who might come, and is worse for conferences that are on unusual topics not previously explored. By contrast, the annual conference on Scandanavian tree lichens has been running for 12 years and attendance is pretty steady.

However, this uncertainty can be reduced by appropriate research. You decide to try a survey using a market research method called "conjoint analysis" at the next big conference and also on the society's web site to find out what the members value in a conference (including rating specific topics), and exactly how much. For example, what is the impact of the venue, the time of the year, the narrowness/breadth of the topic area, the style of the presentations, and the reputation of the main speakers? Using the analysis will give you a much better chance of proposing conferences likely to at least break even.

The uncertainty can also be reduced by waiting more time to see how many people actually do sign up. However, the problem of non-refundable venue fees has to be reduced at least. You survey suitable conference venues to find out their cancellation fees and discover that some will refund closer to the date of the conference than others. By favouring these venues for novel conferences, encouraging members to sign up early, and monitoring the growth of committed attendees you can avoid the venue fee problem much more often than in the past.

To give a taste of the sort of techniques that can be applied to just the design of projects, here are some useful tactics:

  • Multiple-stage projects where each stage is valuable in itself, with opt outs: In this pattern there are two helpful things going on. Firstly, each project stage is designed so that it is worthwhile in itself and would be desirable even if it was decided not to go on with further stages. Secondly, there is the option to quit after each stage, and do no more stages. Both contribute to average returns, and improve them provided excessive sacrifices were not made to design the project in that way. In Evolutionary Project Management projects are divided into roughly 50 sub-stages, achieving a dramatic improvement in 'risk profile'.

  • Portfolios: The portfolio pattern means putting your eggs in several baskets, instead of just one. This is the normal approach with investing in company shares, but also applies to research projects, new products, new systems, and so on. The idea is often to try lots of things and monitor closely for signs of successes that can then be pushed harder, while failures are pruned away. The benefit of diversification in this way is greatest if the performance of the individual investments is not correlated. Sometimes people standardize something in an organization for perceived efficiency benefits without realizing that the loss of variety will reduce learning and the chances of discovering success.

  • Reusable blocks: Building or achieving something that could be re-used in other projects or to meet other objectives is another useful pattern. Reusable blocks tend to be useful even when conditions and the goalposts have moved. Reuse also has good environmental credentials.

  • Cheap commitments to create options: Keeping your options open is nearly always valuable. Generally it is better to make low commitments if you can. Avoid up-front investment wherever possible. Delay commitments. Make your commitments low.

  • Pareto value sequencing: Often, you can divide the work and prioritise it so that the items of greatest value can be done first, with less important items left until later. If time or resources run out before the lesser items are done this is less damaging than if an important item is left undone. The project degrades gracefully. For example, it is often safer to create a rough but complete version of a deliverable first, then do a series of refinements, than to do half the deliverable to full refinement, then start working on the second half. Another example would be trying to get everyone in a team to learn Risk Management by Christmas. Not everyone's skill in Risk Management will be equally important, so start where Risk Management is most useful and move out from there.

  • Systematic risk reduction through variable costs: Systematic risk is risk that comes from general economic conditions and affects just about all businesses though to different extents. Some investors use a number called beta to represent how much a company's results are affected by general economic conditions. These investors consider a low beta desirable and worth paying extra for because it shows that returns from the company are steadier whatever the economic climate. One way to reduce your beta is to do what you can to keep fixed costs a low proportion of your total costs. Fixed costs are costs that do not vary with your output or sales.

  • Systematic risk reduction through counter cyclical activities: Another way to get a lower beta is to combine two activities that react in opposite ways to economic conditions. For example, insolvency plus services to create new companies, or selling new and used cars, or housebuilding and renovation, or taking deposits and making loans, or a recruitment agency that also does outplacement advice.

Basic terms and concepts

One thing that may surprise you is how chaotic the jargon of risk and uncertainty is. The experts do not agree on the meaning of 'risk', 'uncertainty', or even that mathematical word 'probability'. I often use the phrase 'risk and uncertainty' because it encompasses the general area of interest (almost) regardless of which definitions you happen to prefer. I recommend not using the word 'risk' at all, if you can, because it just tends to add confusion and nothing much else.

It's probably easiest to stay with the normal meanings and associations most people have to these words as far as possible.

Here are some fundamental concepts everyone should understand:

  • Limited control: One reason we are uncertain about the past, present, and future is that our control is limited.

  • Limited knowledge: The other reason we are uncertain is that our knowledge of the past, present, and future is limited. We don't know everything about what will happen in future and can't predict it from what we know of the past and present. Because our knowledge is limited it seems to us that more than one future is possible and we get caught out from time to time by surprises. (It is possible that reality is fundamentally random, perhaps at sub-atomic level, but in practice our knowledge is so limited that quantum randomness isn't much of an excuse.)

  • Uncertainty: Strictly speaking, uncertainty is a feeling we experience when we don't know something for sure, or think we don't. It is possible to feel uncertain even if your knowledge is perfect. More often, we have imperfect knowledge but still don't feel uncertain. That is a mistake and we should try to be aware of relevant limitations of our control and knowledge.

  • A risk: A risk is very different from some risk. In documents about Risk Listing (one approach to risk management) a risk is a set of possible outcomes that are important to someone. This is almost identical to the mathematical concept of an event, which is a set of outcomes. In both cases there can easily be infinitely many outcomes in the set.

  • Some risk: When people say 'some risk' they have in mind a quantity representing the importance of some possible nasty event(s). The quantity may be calculated or just felt. If calculated it could be according to one of many different formulae that have been invented over the years, but these typically reflect our preferences for different outcomes and our probabilities for each outcome happening.

  • Risk management / uncertainty management: This is what we can achieve by good management under uncertainty (or, limited control and knowledge, to be more precise). Our risk is managed.

  • Issue: An issue is something bad about the current situation. It is not a risk.

  • Opportunity: An opportunity is something good about our current or future situation that makes it possible, or just easier than usual, for us to do something we find attractive. If you ask people to think of opportunities they will usually list many opportunities inherent in the current situation.

  • Future opportunity: If you want people to think about opportunities that might arise in future you need to make that clear, so I suggest talking about 'future opportunities' or 'potential opportunities'.

  • Probability / likelihood: In the past there were two major philosophies on this. The Frequentist view is that probabilities are a natural phenomenon based on relative frequencies of occurrence. For example, the probability of a person getting heart disease is something we can find from the proportion of people who get heart disease. The Bayesian view is that probabilities represent our degree of certainty about the truth of propositions (i.e. statements that can be true or false). The trouble with the Frequentist view is that we are nearly always uncertain about what the real frequencies are. The modern Bayesian view is often that there is uncertainty about frequencies and both can be represented by probabilities and calculated using the familiar logic of probability theory. Most risk management effectively assumes a Bayesian view of probability.

  • Expected value vs expected outcome: "Expected value" is a mathematical concept and different from the expected outcome. The expected value is the probability weighted average outcome. For example, if you play a game where you roll a die and receive £10 for any number between 2 and 6, but lose £50 for throwing 1, the expected value of each throw is 10 * 5/6 - 50 * 1/6 = zero. However, we would normally think of winning £10 as the expected outcome with a risk of losing £50.

Conclusions

Those are the basics of managing risk/uncertainty. Change the way you work to perform better under uncertainty and do it by taking new ideas on board then thinking about changes you could make. Focus on limited control and knowledge. Don't say 'risk' if you can avoid it.






Made in England

 

Words © 2003, 2012 Matthew Leitch. First published 18 March 2003; greatly updated 6 February 2012.