Matthew Leitch, educator, consultant, researcher SERVICES OTHER MATERIAL |
Working In UncertaintyResults of a survey on risk phrases
This study was first reported in March 2010 on www.internalcontrolsdesign.co.uk and has been reformatted for Working In Uncertainty. It explores the ease with which phrases better than 'risk appetite' can be devised and was presented as consultation feedback to the Financial Reporting Council in the UK. Consequently, it uses more 'risk' vocabulary than would be ideal for a Working In Uncertainty approach. SummaryHow can leaders communicate to others how to incorporate risk in decision making and, in particular, how much importance to attach to risk? One answer that has been talked about over the last few years is that they should 'define their risk appetite' but progress on this has been slow, with many people still wondering what this advice might mean. The results of this survey confirm what many of us have long suspected, that the phrase 'risk appetite' is inherently confusing, now has multiple interpretations, and it is easy to think of better phrases that we could use from now on. With plainer, more self-explanatory terminology we can stop debating what words mean and get on with doing something useful. The survey described eight concepts related to control of risk taking and exposure that have been linked to the phrase 'risk appetite' by some writers, or established de facto by the behaviour of people in organizations. For each concept, respondents were offered a choice of three phrases and asked to pick the clearest, most self-explanatory phrase to name each concept. For each concept, one of the three alternative names incorporated the phrase 'risk appetite' while the other two were chosen to be names that might prove clearer alternatives. The results showed that phrases containing 'risk appetite' performed poorly across all eight concepts, in that they were chosen as clearest the least often for seven concepts and nearly the least for the eighth. In some cases the differences are very large, with other phrases chosen by almost all respondents. The phrases chosen as the clearest and most self-explanatory by most respondents were as follows, compared to the 'risk appetite' alternative (with the third alternative not shown in this summary):
In the relatively rare cases where a respondent chose a phrase containing 'risk appetite' as the clearest and most self-explanatory there was little consistency between respondents. 'Risk appetite' was applied to all eight concepts by one or more people, though the distribution was not equal. This implies that if the phrase 'risk appetite' is used it is likely to be interpreted differently by different people. Being exposed to guidance on 'risk appetite' appears to have had an effect. 'Risk appetite' phrases were selected more often by auditors and risk managers than by respondents in less risk focused roles, and these phrases were selected more often by respondents from the financial services and public sectors, where 'risk appetite' has had a high profile. In addition, further analysis suggests that education changes the interpretation people put on the phrase 'risk appetite'. The interpretation of 'risk appetite' that most often occurs to people spontaneously is that it refers to a personal propensity for risk taking, otherwise known as 'risk attitude'. However, people more likely to have been exposed to documents that talk about 'risk appetite' (auditors, risk managers, people involved in related policy making or regulation, people in the financial sector, and people in the public sector) were more likely to prefer it when referring to policy statements on what risk exposure is acceptable. This is probably the impact of documents such as COSO's ERM Framework and, in the UK, documents from HM Treasury. While education does seem to have communicated something about the meaning of the phrase 'risk appetite', most people in all sectors, including auditors and risk managers, found other phrases clearer and more self-explanatory. This is true overall and concept by concept except in one case. The exception is that, for the subset of auditors, risk managers, and policy/regulation people all in the public sector (a subset chosen specifically because of their higher than usual selection of 'risk appetite' phrases) concept 2 (risk exposure policy) was thought best described by the 'risk appetite' phrase by 9 out of 20 respondents, while 'risk exposure policy' was selected by 8 respondents. Regulators and guidance writers thinking of using the phrase 'risk appetite' in their documents should take note of these results and avoid the phrase. Attempts to establish a clear, universally accepted definition for the phrase 'risk appetite' have failed. Instead of trying to define the phrase 'risk appetite' we should be selecting the clearest, most self-explanatory, least misleading phrases we can for each of the concepts we want to talk about, and using those. Background"What is risk appetite?" is a question often asked. Various definitions have been offered and guidance from COSO, the UK's Treasury department, and the International Organization for Standardization has been in existence for some years. And yet confusion persists. Could it be that the phrase 'risk appetite' is inherently unclear and has contributed to a persistent failure to develop one clear concept to go with it? The phrase 'risk appetite' is metaphorical rather than literal. It draws a parallel between basic human appetites for food, drink, and sex with our approach to risk. This is despite the fact that risk is generally defined as something we do not want if we can avoid it, while food, drink, and sex are all things most of us want, at least up to a point. As far as I know, no previous research has tried to establish what people think this phrase means or if there are clearer alternatives that would be better to use. Because it is very hard for people to put into words what they think a phrase like 'risk appetite' means this survey provided a list of possible meanings and asked people to choose from short lists of alternative names, among which was a phrase using 'risk appetite'. The eight concepts used were derived from a previous study of published definitions and from the author's own observations of how people seem to use the phrase in conversation. The phrases, and their sources, were as follows. (In this report I have used the names that respondents to the survey most often chose as the clearest and most self-explanatory on the list. This is to help you understand this report. In the survey itself the concepts were just numbered one to eight.) Concept 1: Control of risk exposureThis concept was described as follows: "The overall effort to try to guide people in taking risk or exposing the organization to risk, so that bad or excessive risks are not taken and appropriate efforts are made to put controls in place." This concept was chosen because it appears to be the main aim of methods used under the banner of 'risk appetite' and because people who want to tackle reckless risk taking often suggest that 'defining our risk appetite' might be a solution. To be exact, these 'risk appetite' programmes focus on controlling risk taking/exposure using methods that try to tell people how much importance to place on risk. This is only one way to control risk exposure. Concept 2: Risk exposure policyThis concept was described as follows: "A collection of general statements telling employees and other stakeholders how the company/organization views taking various types of risk, making some completely prohibited, while others are to be limited to various extents, though these are expressed only in words, not with quantitative limits." This concept has been written about extensively by HM Treasury and Lloyds of London, among others, and appears in COSO's ERM Framework. Concept 3: Risk exposure control systemThis concept was described as follows: "A control system using limits placed on various measures of investment, business activity, expenditure, and risk, which will be monitored and acted upon (much in the same way as other control-by-numbers systems such as budgetary control)." This is the main interpretation linked to 'risk appetite' in practice in the financial sector. Concept 4: Risk limit settingThis concept was described as follows: "The act of setting or revising limits in a risk control system based on limits on measures of investment, business activity, expenditure, and risk." This continues to focus on the main interpretation of 'risk appetite' in practice in the financial sector. Given the task of 'articulating its risk appetite' banks and insurance companies usually do this, though they may describe it in different terms. Concept 5: Overall risk limitThis concept was described as follows: "An overall limit used in a limit based control system." This also continues to focus on the main interpretation of 'risk appetite' in practice in the financial sector. However, this is also very close to the definition promoted in ISO Guide 73:2009 and in publications by HM Treasury that say 'risk appetite' is a single, overall limit on risk. Concept 6: Risk attitudeThis concept was described as follows: "Personal propensity in regard to risk taking, capturing individual differences in decision making given identical perceptions of risk and reward." This concept is one I have observed in conversations, usually where auditors say that management decided to do nothing about a risk that had worried the auditor, and this is put down to the risk being 'within management's risk appetite'. There is usually a definite implication that this is a personal decision that cannot be challenged on rational grounds, and it would be impolite to try. The phrase 'risk appetite' reminds people of personal needs and perhaps is the reason that this interpretation appears, despite having no official support. Concept 7: Maximum tolerated risk levelThis concept was described as follows: "A limit applied to an individual risk item on a risk register such that if the magnitude of the risk is greater than the limit some control action must be taken that reduces the remaining risk to below the limit." This concept appears frequently in guides to risk management based on using risk registers and Probability-Impact matrices. It was enshrined in AS/NZS 4360, the risk management standard from Australia and New Zealand. (However, the phrase 'risk appetite' was dropped when this standard was revised and adopted as the international standard, ISO 31000:2009, where the phrase 'risk criteria' is used instead.) Concept 8: Risk weighting function/formula/rule/tableThis concept was described as follows: "A function/formula/rule/table that is used to incorporate risk in quantitative decision making, e.g. to calculate the total cost of risk, or the capital cost of a risk item, to calculate risk adjusted performance measures, or to set the discount rate to use in evaluating projects and other business investments." Other views of risk recognise that the maximum level of risk that can be tolerated in an investment decision without changing the decision depends on the decision. If the incentives are increased, usually we will put up with more risk. There is no one level of risk that is correct for every decision. Consequently, risk levels are weighted or costed in some way in the hope that one rule can be applied in every decision. This is probably most common in banking and insurance, where risk is assigned a cost. Survey designThe survey was conducted online with respondents invited to participate in a variety of ways. When participants arrived at the survey page they were asked the following questions: "What country are you currently living in?" "What best describes your role and your background?" "What best describes your employment background?" The survey then gave the following instructions: "Please read each of the following descriptions of different concepts and select the word or phrase on the list beside each one that, for you, is the clearest and most self-explanatory name for that concept on the list." "It does not matter what phrase you currently use, what seems familiar, or what is used in your organization. Just pick the most self-explanatory, least misleading, clearest phrase offered." "You may be able to think of something better but please just pick the best on each list. If you want to make suggestions then there is a comments box at the end of the survey that you can type into." "Thank you for participating." After the eight concepts there was a text entry box for any comments the respondent might have. Once the results had been sent a second page appeared and thanked the respondent for their views. It also offered the chance to think about the issues a bit more by participating in an entirely optional follow on survey. This follow on asked respondents to think of the clearest, most self-explanatory phrase they could for each of the eight concepts. They were given the concept descriptions again and two ways to build phrases. One was to select from drop down lists of words and phrases that were designed to go together. The other was to just type directly into a free text field. Once again the final part of the survey page was a text box for any comments. RespondentsThe numbers of respondents from each source were as follows:
The overall response rate is unknown and response rates will have varied greatly by source. The number of respondents in the main survey analysed by role and employment background is as follows:
Analysed by country, the respondents in the main survey were as follows:
ResultsNearly half of respondents chose no phrases including 'risk appetite' (RA) at all.
The outliers in the above table are two respondents who selected the 'risk appetite' phrase for seven out of the eight concepts. One of them wrote a comment, which was 'Seems like risk appetite can have many definitions depending on the context in which it is used.' It may have been that the design of the survey actually led him/her to think that the phrase 'risk appetite' was applicable in all these situations. There is no way to know what the respondents' views were before seeing the survey. A few respondents commented that the survey seemed biased towards 'risk appetite'. However, it may also have been that the design of the survey discouraged people from selecting the 'risk appetite' phrase for different concepts because this would have seemed inconsistent. In other situations, where the concepts involved are not so clear, respondents might have used the 'risk appetite' phrases more freely, unaware of the potential for confusion. The results for the most commonly selected phrases contrast with those for the 'risk appetite' phrases.
Selection of the 'risk appetite' phrase varied by role as follows:
The impact of education can be seen in this table. The risk specialists (auditors and risk managers) differ from the others (ignoring Policy making/regulation because they are a very small group whose focus on risk is unclear) in that (1) the risk specialists select 'risk appetite' phrases more often overall, and (2) they select concept 2, risk exposure policy, in preference to concept 5, risk attitude, whereas the non-risk focused roles prefer the risk attitude interpretation. Selection of the 'risk appetite' phrase varied by employment background as follows:
Here are the results in detail, concept by concept. As mentioned above, the concepts have been named retrospectively using the names that came out best in the survey. In the survey itself the concepts were just labelled 1 to 8. Concept 1: control of risk exposureThe concept description was as follows: "The overall effort to try to guide people in taking risk or exposing the organization to risk, so that bad or excessive risks are not taken and appropriate efforts are made to put controls in place." The percentages of respondents selecting each name as the clearest were as follows:
Phrases created to be the clearest in the follow on survey were as follows:
Concept 2: risk exposure policyThe concept description was as follows: "A collection of general statements telling employees and other stakeholders how the company/organization views taking various types of risk, making some completely prohibited, while others are to be limited to various extents, though these are expressed only in words, not with quantitative limits." The percentages of respondents selecting each name as the clearest were as follows:
Phrases created to be the clearest in the follow on survey were as follows:
Concept 3: risk exposure control systemThe concept description was as follows: "A control system using limits placed on various measures of investment, business activity, expenditure, and risk, which will be monitored and acted upon (much in the same way as other control-by-numbers systems such as budgetary control)." The percentages of respondents selecting each name as the clearest were as follows:
Phrases created to be the clearest in the follow on survey were as follows:
Concept 4: risk limit settingThe concept description was as follows: "The act of setting or revising limits in a risk control system based on limits on measures of investment, business activity, expenditure, and risk." The percentages of respondents selecting each name as the clearest were as follows:
Phrases created to be the clearest in the follow on survey were as follows:
Concept 5: overall risk limitThe concept description was as follows: "An overall limit used in a limit based control system." The percentages of respondents selecting each name as the clearest were as follows:
Phrases created to be the clearest in the follow on survey were as follows:
Concept 6: risk attitudeThe concept description was as follows: "Personal propensity in regard to risk taking, capturing individual differences in decision making given identical perceptions of risk and reward." The percentages of respondents selecting each name as the clearest were as follows:
Phrases created to be the clearest in the follow on survey were as follows:
Concept 7: maximum tolerated risk levelThe concept description was as follows: "A limit applied to an individual risk item on a risk register such that if the magnitude of the risk is greater than the limit some control action must be taken that reduces the remaining risk to below the limit." The percentages of respondents selecting each name as the clearest were as follows:
Phrases created to be the clearest in the follow on survey were as follows:
Concept 8: risk weighting function/formula/rule/tableThe concept description was as follows: "A function/formula/rule/table that is used to incorporate risk in quantitative decision making, e.g. to calculate the total cost of risk, or the capital cost of a risk item, to calculate risk adjusted performance measures, or to set the discount rate to use in evaluating projects and other business investments." The percentages of respondents selecting each name as the clearest were as follows:
Phrases created to be the clearest in the follow on survey were as follows:
Comments by respondentsExcluding comments purely about the survey and polite messages to the researchers, the respondent comments were as follows, presented in the order they arrived. From the main survey"My background is in human health risk assessment, which may explain why much of the above terminology did not resonate with me. I must confess to having an aversion to the term appetite (when in reference to risk), which certainly affected some of my responses." "Risk appetite doesn't belong as a descriptor for any of these statements, and, in my opinion, doesn't belong in the risk lexicon at all. For the first question, none of the descriptors is particularly good. Target risk isn't much better than risk appetite." "1 should be "Risk Management". None were suitable for 8." "1) I would say something like effort for risk management (or control). The effort aspect of the description is not well covered in any of the provided terms. Questions 2 to 4 seem to be directly related to the answer 1 flavor. 8) I would call it a risk model." "Your statements emphasise financial measures / aspects of risk - but there are times were other risks precede the financial consequences, and where the financial consequences are pretty hard to either assess or estimate." "I believe it is important to distinguish between Uncertainty, mathematically a random variable, and Risk, which is the way that uncertainty effects you, and is therefore in the eye of the beholder." "Seems like risk appetite can have many definitions depending on the context in which it is used." "I hope these aren't all definitions of risk appetite from different companies!" "Don't confuse risk appetite and risk tolerance. Most of these describe tolerance more than appetite." "I would regard the process outlined in 7 (a popular approach to defining risk appetite using P-I matrices) as being invalid. The reason for this is that it is possible to take almost any risk and break it into down into lower level 'child' risks that then each appear to have become acceptable according to the method. This approach does not alter the overall risk one iota, but is often misapplied by people who think that working at lower levels of information give them better data." "It might be good to try to define a risk appetite that may move up or down depending on management postures and other factors within and outside of the organization." "In practice, I haven't used many of these terms." "It seems clear to me that many of the statements above could be interpreted differently as they use terminology which is in itself open to different interpretations; this implies to me that there is probably no agreed definition of terminology acceptable by all within the sector and those reliant upon it. I also suspect that different sectors will view the statements differently only confirming the confusion we all face. This, if true, contributes to the continual confusion we all face in implementing guidance, governance, regulation, etc and increases the difficulty we have as a sector in influencing our businesses. I would be very interested to see the results to see whether my views are likely to be true or not." "A "stab in the dark" thought - are the above components of risk appetite?" "... do bear in mind that the terminology can vary with context. I've used risk appetite for corporate and personal tendencies; but they're unlikely to be confused in context." "An interesting choice of words none of which really define risk appetite - a much used concept and apparently a cornerstone of operational risk. The problem is that while consultants speak of it frequently, it's such a nebulous concept that I have never heard it clearly defined and I have no idea either what it means or how you would set it. A clear case of (to use a horrid Americanism) talking the talk but not being able to walk the walk!" "Questions 7 and 8 seem to be geared to direct the intervention of Risk management techniques. Q7 to direct RM to specific items and Q8 to determine the function and goals of the overall RM program within an entity." "Concept #4 - Would be better named 'Risk Exposure Calibration'. Concept #5 - Revise the descriptive phrase to 'Maximum Acceptable Risk'." "There appears to be confusion in the two distinct pillars of risk between Qualitative and Quantitative." "My concern is that risks are identified and managed, rather than certain terminology be used. However, I do find that some of the qualitative phrases in your survey are used to put a positive spin on risky ventures. For example, 'this is within our overall risk appetite' sounds reasonable, whereas 'this exposes us to a £1m risk, with 60% certainty of occurrence' might prompt more questions from a board." "Interesting that I nearly always incline to the longer, more detailed but perhaps less ambiguous form." From the follow on survey"I completed this survey while thinking in the context of OH&S. The use of accurate terminology is one of out greatest challenges. I am sure that this is in part caused by OH&S professionals' resistance to admit that 100% freedom from risk does not exist, therefore they like to speak in terms of relative safety rather than relative risk." "As with other ares of business the terminology moves about. Basic risk terminology in the world of engineering and project risk management addresses all the concepts/constructs well and clearly and eliminates the bull. However, it can be designed to obfuscate and does very successfully. Sometimes it comes about because someone powerful doesn't understand a concept - often to do with probability - and rationalises it in a half-baked way with a new 'better term'." "My guess is that the thesis behind this survey is that the concept of 'risk appetite' is easy to refer to in a standard or policy, but difficult to clarify in practice or to use for decision making. If so, I would agree. I believe that risk appetite will only normally be meaningful if related to a forecast of 'overall risk', e.g. to an organisation's profitability or a project's outcome. Trying to apply the concept on a risk-by-risk basis is fundamentally flawed unless there are risks that are genuine potential showstoppers." "I believe the greatest risk is dishonest people who find ways to steal funds entrusted to them - and then get away with it, maybe even receiving a bonus as insult to the stakeholder's injury. The best risk management would be to take the money back from the crooks and make a spectacle of them. Then we could get serious about managing business risks." "The above may be taken differently by every visiting expert." Appendix: Other pointsVarious other points may be of interest to readers so please get in touch with the author if you want to know more (matthew@workinginuncertainty.co.uk). In this appendix I mention some points for which there isn't space to go into detail. Is the UK different from other countries? No, UK respondents answered in broadly the same way as respondents from other countries. Was the survey biased because some concept names had words in common with the concept descriptions? Some bias is possible, but looking at all concepts and all names there is no obvious pattern. Some of the preferred names do not match the descriptions with as many of their words as names they were pitched against. For practical purposes it is inevitable and desirable that the name of a concept should use the words that best explain the concept. For this survey the descriptions needed to be as clear and as self-explanatory as possible and it is unsurprising that the clearest names often use the same words. Was the survey biased because some concept names were longer than others? Again, some bias is possible, but looking at all concepts and all names there is no obvious pattern. Some of the clearer, more self-explanatory names are shorter than others they were pitched against. Are people involved in policy making or regulation more likely to think that 'risk appetite' phrases are clear and self-explanatory? In percentage terms that is what the data point to, but as we only received responses from 4 people with this employment background it is unsafe to draw any conclusions. Are the eight concepts really different from each other? Concepts 3, 4, and 5 were really the same concept in different situations. Other concepts were different. If the two outliers are removed from the data set does it make a big difference? The numbers for the subsets of respondents are noticeably changed by removing the outliers, but always in the direction of 'risk appetite' being less favoured as a clear and self-explanatory phrase. The observations about the impact of education remain valid but the differences are slightly smaller. Since the response rate for the survey is unknown, doesn't this invalidate the conclusions? The overall tendency to select phrases that do not include 'risk appetite' clearly depends on who responded to the survey. The results show that risk control specialists are more likely to think that 'risk appetite' is clear and self-explanatory. Most respondents were risk control specialists. If a wider audience had participated the overall selection of 'risk appetite' phrases would have been even lower than it was. Some of the concept descriptions are hard to understand. Does that make the results unreliable? Two respondents commented that they found the descriptions were hard to understand and one person emailed to the effect that he had abandoned the survey for that reason. While it is unlikely that the concept descriptions are perfect I believe them to be unusually clear and precise statements of concepts in circulation today. The difficulty at least some respondents experienced is in part due to some concepts being unfamiliar to them prior to reading the survey and to the generally confused thinking in this area. Fortunately, the overall results showing 'risk appetite' to be a relatively unclear name for concepts in this area are clear cut, with very large differences across all eight concepts. |
Company: The Ridgeway Expertise Company Ltd, registered in England, no. 04931400.
Registered office: 29 Ridgeway, KT19 8LD, United Kingdom.
Words © 2012 Matthew Leitch