Working In Uncertainty

Results of a survey on 'risk management'


Many thanks to everyone who participated in this survey. Once again, without your generous donations of time and thought this would have been impossible.


Is it right to define the phrase 'risk management' in risk-listing terms? In other words, do most people agree that 'risk management' should mean, by definition, the method of managing risk that involves making lists of 'risks' and managing the 'risks' on the lists (Leitch, 2012a)?

No, they do not. The results of this survey show that a clear majority of people think that the phrase 'risk management' should just mean managing risk and should not imply a particular method. They do not agree that 'risk management' should mean specifically the risk-listing approach. This was true for the respondents as a whole but there was a clear difference between respondents whose first language is English and other respondents, with the native English speakers most often wanting the more open definition of 'risk management'.

This finding may come as a surprise to many readers because most respondents thought other people define 'risk management' specifically as risk-listing. Interestingly, people who thought that 'risk management' should mean just risk-listing were much more likely to realise that most other people think differently. Also, people whose knowledge on this point was informed by systematic research were more likely to think that most people prefer the open definition.

Most people also thought that publications tended to define 'risk management' as risk-listing and those whose knowledge was based on systematic research were more likely to think this.

These results have profound implications for anyone producing guidance, standards, or regulations relating to 'risk management' because there is a danger of defining 'risk management' in risk-listing terms without realising that there are alternatives, including better, more comprehensive alternatives. Risk-listing is not a generic approach to risk management.

A note on the statistics and graphs

Most of the graphs in this report look like a dark red bar chart with a grey overlay at the end. This overlay represents uncertainty about the true value in the population, based on the sample evidence, using Bayesian methods.


The dark red part of the bar shows the range up to the 10th percentile of a probability distribution representing belief about the true percentage or average of the entire population (i.e. people with an interest in management). The pink part shows the range from the 10th to the 50th percentile. The light grey part shows the range from the 50th to the 90th percentile. In effect, the graph says that, given various assumptions (as usual in statistics), we can be 80% certain that the truth lies within the grey overlay area (i.e. pink and grey segments combined).

In all cases the initial prior distribution was uniform. For questions requiring a choice between two answers the prior distribution was the beta distribution. For questions requiring a choice between more than two answers the prior distribution was the Dirichlet distribution.

Where percentages do not add up to exactly 100% this is, of course, due to rounding.

The survey questions and answers

In this section the text in italics is the text used in the survey itself. Other text discusses the results and their implications.

The first two questions in the survey were about the first language and skills of respondents. These are given and discussed in a later section. The survey then continued as follows.

3. First, which of the the following best expresses what you think the phrase 'risk management' should mean?

Meaning% of respondents
'risk management' means managing risk, and does not imply any particular method of doing so72%
'risk management' means specifically the method of managing risk that involves making a list of 'risks' and managing them21%
'risk management' means managing risk using a particular method, but not the method of making a list of 'risks' and managing them7%

Implications: Clearly, most people think 'risk management' should mean just managing risk, without implying any particular method.

To explore the factors that might be linked to this view the job roles and whether English was a first language were transformed into binary variables and regressed against the answers to question 3. This showed that having English as a first language was the strongest predictor, by far, and was associated with preference for the open definition that does not imply a particular method. A secondary finding was that being an auditor was associated with thinking that 'risk management' should imply risk-listing by definition.

Perhaps a strong grasp of the principles of English language provides protection against publications advocating a risk-listing definition.

4. Now, which of the following best expresses what you think the phrase 'risk management' most often means to other people?

Meaning% of respondents
'risk management' means managing risk, and does not imply any particular method of doing so27%
'risk management' means specifically the method of managing risk that involves making a list of 'risks' and managing them68%
'risk management' means managing risk using a particular method, but not the method of making a list of 'risks' and managing them5%

Implications: In contrast to the results of the previous question, most people think that other people think 'risk management' refers specifically to risk-listing. Either we don't realise that others are, mostly, as enlightened as we are, or we distinguish between what we think the phrase should mean and what it usually means.

However, respondents who thought 'risk management' should imply risk-listing were much less likely to think that others agreed. Only 30% of respondents who thought 'risk management' should imply risk-listing thought that others used this definition, compared to 78% for respondents who did not think 'risk management' should imply risk-listing.

Perhaps people who argue for risk-listing provoke a reaction that other people rarely see.

5. How confident were you in answering the previous question?

Level of information% of respondents*
I just guessed16%
I know a bit about this from general reading, conversations, and so on66%
I have already read about or performed a survey to find out what people think 'risk management' means19%

Implications: The results from the previous two questions suggest that most people do not realise that other people have the same preference for the open definition that they have. This should be, in some way, related to the level of information informing their judgements.

Others' viewGuessedGeneral knowledgeSurvey

As can be seen from the above table, respondents with all levels of evidence tended to misread the views of others. (Remember that 72% of respondents in this survey thought 'risk management' should mean managing risk with no method implied.) Those using survey evidence underestimated least often but having some knowledge from conversations and reading seemed to create the false impression that others are thinking of risk-listing - a false impression that people who said they just guessed did not suffer from.

6. And finally, which of the following best expresses how 'risk management' is most often defined in publications?

Meaning% of respondents
'risk management' means managing risk, and does not imply any particular method of doing so33%
'risk management' means specifically the method of managing risk that involves making a list of 'risks' and managing them58%
'risk management' means managing risk using a particular method, but not the method of making a list of 'risks' and managing them8%

Implications: Again, most respondents thought that publications tended to favour the risk-listing definition.

7. How confident were you in answering the previous question?

Level of information% of respondents*
I just guessed21%
I know a bit about this from general reading, conversations, and so on60%
I have already read about or performed a survey to find out what people think 'risk management' means19%

Implications: Again, the answers should be related to the level of evidence used by the respondent.

Definition in publicationsGuessedGeneral knowledgeSurvey

Here the pattern seems to be that the more evidence the respondent had the more often they thought publications tended to define 'risk management' as risk-listing.

What is the truth? What do publications that define 'risk management' usually say it means? The truth is hard to know because the most obvious method of research, which is to search the internet, provides a strong bias towards documents published in the last twenty years or so and where the phrase 'risk management' appears in the title, which is typical of risk-listing documents.

However, to give you an indication of the extent to which influential documents currently define 'risk management' in risk-listing terms despite the common view of English speaking people that this is wrong, here are some examples:

Document Source Definition Evidence for this
A risk management standard AIRMIC, IRM, ALARM (originally) As risk-listing The section on risk management begins with a definition: risk management "is the process whereby organisations methodically address the risks attaching to their activities with the goal of achieving sustained benefit within each activity and across the portfolio of all activities." It carries on immediately, "The focus of good risk management is the identification and treatment of these risks." The method set out in the 'standard' is classic risk-listing with no acknowledgement of alternatives.
Enterprise Risk Management - Integrated Framework PwC via COSO As risk-listing In the section titled 'Enterprise Risk Management defined' (p4 of summary) the definition given is: "Enterprise risk management is a process, effected by an entity’s board of directors, management and other personnel, applied in strategy setting and across the enterprise, designed to identify potential events that may affect the entity, and manage risk to be within its risk appetite, to provide reasonable assurance regarding the achievement of entity objectives." The process then set out is entirely a risk-listing one.
Guide 73: 2009 ISO Open The definition given in this international glossary for ISO standard writers is: "coordinated activities to direct and control an organization with regard to risk". However, note that its idea of what a 'risk management process' does is expressed in risk-listing terms, with no alternatives acknowledged.

Also, on 6 January 2014 I used Google to search for: definition "risk management". All the first 10 documents presented included a definition of 'risk management' and 8 of these were as risk-listing. This is a situation that should be corrected.

Rank Source Definition Type of definition
1 Wikipedia "the identification, assessment, and prioritization of risks (defined in ISO 31000 as the effect of uncertainty on objectives, whether positive or negative) followed by coordinated and economical application of resources to minimize, monitor, and control the probability and/or impact of unfortunate events or to maximize the realization of opportunities." risk-listing
2 Investopedia "The process of identification, analysis and either acceptance or mitigation of uncertainty in investment decision-making." open (just)
3 The Economic Times (India) "the practice of identifying potential risks in advance, analyzing them and taking precautionary steps to reduce/curb the risk." risk-listing
4 Business Dictionary "The identification, analysis, assessment, control, and avoidance, minimization, or elimination of unacceptable risks." risk-listing
5 Department of Homeland Security Risk Lexicon "process of identifying, analyzing, assessing, and communicating risk and accepting, avoiding, transferring or controlling it to an acceptable level considering associated costs and benefits of any actions taken" open (just)
6 Praxiom " a coordinated set of activities and methods that is used to direct an organization and to control the many risks that can affect its ability to achieve objectives." risk-listing (by mis-translating ISO 31000:2009)
7 A Risk Management Standard "the process whereby organisations methodically address the risks attaching to their activities with the goal of achieving sustained benefit within each activity and across the portfolio of all activities." risk-listing
8 SearchCIO "Enterprise risk management (ERM) is the process of planning, organizing, leading, and controlling the activities of an organization in order to minimize the effects of risk on an organization's capital and earnings. Enterprise risk management expands the process to include not just risks associated with accidental losses, but also financial, strategic, operational, and other risks." risk-listing
9 Entrepreneur "Decisions to accept exposure or reduce vulnerabilities by either mitigating the risks or applying cost effective controls." risk-listing
10 Marquette University "the continuing process to identify, analyze, evaluate, and treat loss exposures and monitor risk control and financial resources to mitigate the adverse effects of loss." risk-listing (probably)

The wider situation, including influential documents that do not come up on Google searches for 'risk management' is rather different, as documented in Leitch (2012b).

8. This box is for any comments, explanations, or suggestions you would like to make. Include your email address if you would like me to respond.

Comments received (other than friendly greetings) were as follows:

"I wouldn't be surprised if most people answer Q3 and Q4 the same way as I do. But what I observe is that when companies create risk management processes they are almost invariably about making lists and (perhaps) managing them. In other words, there is a distinction between risk management as a concept and risk management as an organised activity." (The respondent was right that others would tend to give the same answers to questions 3 and 4 i.e. open and risk-listing respectively.)

"Definitions of risk management are generally too specific. Work by Mike Power and also Felix Kloman may be of interest, if you have not come across them before."

"In my experience and reading, risk itself is often poorly defined. Although 'management' often incorporates a number of elements any 'guidance' is usually qualitative in nature utilising a combination of PIM and lists. It is rare to find evidence of quantitative models and the use of probabilities/distributions."

"For me Risk Management is optimising long term decisions, actions and choices whilst considering unknowns, risks, ambiguities etc. There are some words that go along with Risk Management that are part of the approach if not what it is: Balanced, Strategic, Open Minded (one of yours), Considered, Use numbers and calculations but acknowledge importance of feelings and judgements, Humbleness when considering own ability to foresee & influence the future, Pragmatic, Creative."

"Really interesting question. Are you managing risk when you remember the Green Cross Code when crossing the road? Yes, but do people associate that everyday decision with risk management? Probably not. Has risk management tried too hard to become a discipline/science in its own right that just means people only think about the process and worry about the potential for it becoming a cottage industry? My organisation struggles with, on the one hand wanting to manage risk, while on the other worrying about the bureaucracy of 'risk management', and also not knowing what type of 'risk management' would actually work. Also - 'in publications' is a bit broad, do you mean specialist publications, or do you mean in all publications, including daily newspapers etc?"

"Q3 The way you phrase the question and options rather betrays your dislike of lists of risks but I don't think that they are unhelpful. You need to document what you are up to somewhere and review progress and state etc. Blinkered by them and totally without them are equally bad. Q4 Either because they know no more than that or because they recognize Q3 above, so ideally result needs caveat or clarification. Q5 and Q7 options don't really allow for extensive experience in this field! Q6 The options don't really allow for the oft reality of 'badly described, randomly described and confusion encouraged'. Option selected was selected for this reality." (This respondent answered Q3 and Q4 in the usual way, suggesting that extensive experience does not help with knowing what others think.)

"I believe that risk registers (RRs), as currently used, lack rigour. But they are simple and appealing, especially to supposedly 'practical' people. They work in Excel. When we come to the literature the coverage is much more mixed. As far as I can see although RRs have found a 'best practice' role this did not come directly from the old COSO work in the 1990s. RRs are beloved of people who want to sell software and advise on processes and auditors who use them to do 'risk-based' audits. My personal view is that RRs can play a useful organisational role - the list! They are *much* weaker when reliance is placed on the probability-impact method of risk assessment c.f. Hubbard's The failure of risk management and many other sources. Currently RRs tend to equate risks to events. Worse, different 'risk owners' assess the probabilities of these events at different points on the (underlying probability) curve, especially in the absence of guidance. IMHO the only hope of saving risk registers is to have an assessment process which is (a) complete, (b) internally consistent and (c) able to be recalibrated without asking assessors for another lot of work."

"People not involved in the risk and internal audit profession would favour generally understood definitions which means something like 'bad things could happen that would hurt you'. Regulators have pushed rules that increasingly cause people to associate it with risk registers. What internal auditors mean when they say 'risk-based' plan may be the least precise of all."

"Our problem is that the word 'risk' itself means many different things to different people. How can we possible 'manage' something we misunderstand?"

"Risk management means quite different things to professionals in different venues, which makes it difficult to answer these cross-venue questions coherently."

"I would have thought that the value in managing risk is in our ability to understand measure, treat and monitor the causes and effects of attaining business outcomes, at variance to expectation."

"In answering your survey I have assumed you are referring to the systematic management of risk. I would argue that a lot of people, as individuals, are managing risks in a less than systematic fashion. The approach involving making-lists and managing-those-listed-risks is probably better-considered than raw pattern-recognition-and-reaction. Both are sub-optimal in their own ways."

"In question 4 I interpret 'Other people' as ordinary people, who have no familiarity in the subject area."

"My context deals primarily with project management and risks on projects."

"Half the time, I seem to find that 'risk management' means 'management of a portfolio of insurance contracts'. But then, I live in North America."

"Q6 - it depends on the publications you read so difficult to answer in general. Could be any one of 3 options given."

"Risk management is important. It comes more naturally to people who are more proactive. Its application/impact changes with context (people, situation, level of knowledge, etc). It is also misunderstood/misapplied/undervalued due to lack of a better mental model/awareness. It depends on a good/healthy management (maturity/paradigm). One perfect area to make a significant change is for HR to focus on managers' failure modes and effects and how to mitigate/provide contingency to those i.e. how to grow people."

"The term risk management has come to stay but beyond the terminology, the profession is looking at the core puzzle of humanity: how to live a happy life in a world of uncertainties. Organizations are faced with the same challenge. Mathematical models, psychological/cultural considerations and scenario planning are all good ways of forging a way through the unknown. The debate is focused on whether to go strictly mathematical or strictly risk listing or working in uncertainty. Working in uncertainty seems the best option to me, however, there is a nuance. Beyond the right mind-set and method to tackle uncertainties, much focus should be placed on strengthening the expertise of the risk manager and creating an environment fostering genuine human dignity in organizations. The risk manager is to the organization what the general practitioner is to the human body. The risk manager's training should be extended in years to incorporate in-depth knowledge about key business units that characterize every organization (Finance, HR, Operations, Production, Legal, etc). Then, roundtable discussions about ways to effect incremental changes in organizations will be more effective. The second key requirement rests on genuine human dignity. Creating an environment where every single person in the organization feels they matter and that without them the organization is not complete. Beyond monetary reward, a smile, a genuine concern about an employee's working conditions and family well-being can foster better contributions of employees at roundtable discussions regarding the business uncertainties. Finally, a company that breaks the traditional top down management process can impact considerably on risk attitudes. An environment where the employee can feel a questionnaire about his line manager will create an atmosphere of flexibility where one knows that his ideas will not cause him to lose his job but will be assessed critically. The open-mindedness culture of uncertainty management will be given life."


Respondents were recruited by direct email requests to about 800 people who have shown an interest in my work over the years.

The survey was completed by 96 people.

1. Your first language:

Most respondents had English as a first language:

% of
First language
1%Finnish, Swedish

2. Do you consider yourself to be a professional in any of these? (click all that apply)

Role% of respondents
Performance management27%
Risk analysis39%
Risk management59%

Limitations of the survey

The survey suffered from a number of typical limitations. In particular, respondents were self-selected and from a group that is not entirely representative of people with an interest in management. They were people with an interest in risk who had shown a specific interest in my work. That makes them, to some extent, a special group. However, the analysis of different respondents in the survey shows that even the group most oriented towards risk-listing methods (the auditors) overwhelmingly preferred to define 'risk management' as management of risk without implying a particular method. Previous studies using this same pool of respondents have given similar results.

Wider implications

The definition of 'risk management' is not just a matter of pedantry. If 'risk management' is defined as risk-listing then it leaves no room for other methods. Risk-listing is like a cuckoo chick, ejecting the legitimate occupants from the nest to favour itself alone. Using 'risk management' in this way is an unfair and unethical sale trick, typical of consulting sales pitches. Offer someone 'risk management' and it is hard for them to refuse it without seeming to reject responsible management. Offer them 'risk-listing' and they can reject it and ask for a better way to manage risk.

The fair way to manage this situation is to define 'risk management' in an open way and let proponents of alternative methods for risk management offer their ideas under names that suggest a method rather than an aspiration. For example, they can offer 'risk-listing', 'decision trees', 'probabilistic forecasting models', 'scenario planning', and so on, but not 'risk management' per se.

All regulators, standard writers, and other influential authors should review their current practice and, if they find that risk-listing has insinuated itself into their definition of 'risk management', they should put an open definition in place instead. This can be done by simply defining 'risk management' as 'managing risk'. They might also like to point out, in a note perhaps, that risk management is usually one outcome of managing well, and that many different approaches may be applicable.


Leitch, M. (2012a). The risk-listing school. Available online at:

Leitch, M. (2012b). Relevant authoritative guidance. Available online at:


Company: The Ridgeway Expertise Company Ltd, registered in England, no. 04931400.

Registered office: 29 Ridgeway, KT19 8LD, United Kingdom.

Words © 2014 Matthew Leitch