Working In Uncertainty

Guidelines for managing uncertainty/risk during sense-making/diagnosis/investigation

Introduction to the guidelines

These guidelines are to be used when thinking about how to perform sense-making, diagnosis, or investigation. The guidelines concern only managing uncertainty/risk so they should be considered alongside any other guidelines or suggestions based on other factors.

The guidelines are designed to apply to any situation where sense-making/diagnosis/investigation is done at a level where an agreed approach is considered worthwhile. They are worded very carefully to be as widely applicable as possible and that means they are not always as short as they would be if you were just writing for your own organization. Consequently, you may need to read each guideline more than once, carefully, and read any related examples, explanations, and even other documents referenced from this page. A lot of knowledge about how to manage risk has been condensed onto this page.

Usually it will be easy to think of ways to follow the guidelines but sometimes, if your situation is demanding, it may require quite a lot of thought and experimentation. What should not be difficult is justifying the guidelines. As far as possible, these are guidelines that most people think are obvious good sense.

The scope of sense-making, diagnosis, and investigation

These activities concern finding out what is happening now or what happened in the past. Examples include investigating crimes, performing audits, diagnosing illnesses, and firefighters arriving at a fire and asessing the situation.

Guidelines

The approach to sense-making/diagnosis/investigation should meet the following practice guidelines:

  1. Effort is made to gain information that helps to reveal the truth and to present it in ways that facilitate identifying and understanding significant features.

  2. Uncertainty around information gained is explicitly represented.

    Note 1: This is simply uncertainty around the information, not its implications for the alternative hypotheses. An example would be measurement uncertainty around measurements of physical properties or attitudes.

  3. The search for relevant information is guided by an understanding of what is uncertain, what further information might be obtained, and what information would help to establish the truth.

  4. Ways to gain information more efficiently and accurately are considered.

    Note 1: These may include simply logging information systematically rather than relying on memory and anecdotes, and may include automation.

  5. Effort is made to create a set of hypotheses about the truth that is organized and covers all or most possibilities.

    Note 1: Ideally the set of hypotheses will be exhaustive and mutually exclusive, at least in logic if not in the way the set is written.

  6. Information is used to reach a more informed view of how likely it is that each hypothesis is the best through considering how consistent the information is with each hypothesis and comparing the hypotheses with each other.

    Note 1: This approach encourages more explicit and sustained response to uncertainty than trying to choose the best hypothesis, which may not be possible at all without discarding hypotheses that, collectively, still have a significant chance of being the best.

  7. Multiple possibilities are kept under consideration for as long as possible or until a certain conclusion is reached.

  8. Opportunities to (partially) automate the task of combining evidence are considered.

  9. The revised view of how likely each hypothesis is to be true is clearly presented and communicated clearly and promptly in a way that facilitates use of the view in future decision-making.

    Note 1: It is not true that a 'best' hypothesis has to be chosen for decision-making purposes. Keeping multiple hypotheses open and weighing each appropriately based on the evidence is at the core of good risk management.






Made in England

 

Words © 2015 Matthew Leitch.