Working In Uncertainty

Fixing a process and controls mess

First published 15 May 2003.

How messy?

From time to time most organisations have internal processes that get into a mess, resulting in wasted time, aggravated customers, or lost money. The symptoms range from chronic inefficiency and aggravation that people have got used to, through to complete melt down.

For example, in telecoms companies it was discovered a few years ago that most were losing between 2% and 10% of their revenues simply because they failed to bill for all the services their customers used. The technical difficulty of the billing and the pace of the industry had led to process and controls mess in nearly every company's billing.

In government departments and related organisations there have been some huge messes (in the UK at least) over the last few years affecting tax credits, passports, child support, exam results, health care, and so on.

Often these ghastly situations are the result of big projects that did not achieve everything necessary for success. Other common causes are rapid expansion, prolonged financial distress, and repeated reorganisations or changes of ownership.

In an ideal world this would never happen because organisations would skillfully design the processes and internal controls they need to operate efficiently and reliably. (That's the subject of other guidance on this website.) In the real world, there will always be organisations with mess to sort out.

Sorting out process and controls mess is not easy. It takes strong nerves and sheer intellectual horsepower. However, the right approach can make a huge difference. Many people who get the job of sorting out a process and controls mess spend weeks or even months getting nowhere. This need not happen. On this page I'll explain what to expect and how to succeed.

Elements of the task

Work to sort out process and control mess usually involves doing the following:

• Investigating mysteries: Why does this reconciliation fail? What are our customers complaining about? Is it true that we're losing money? How did the data get corrupted? Which database is right? Where do we start? These are the sort of question that come up and need to be investigated. Skillful investigation makes a huge difference to the success of the project overall.

• Cleansing data: Mess nearly always results in incorrect data in files and databases. Some of it is reference data that will continue to generate incorrect transactions until it is corrected. Some of it is transactions – perhaps millions of them.

• Payments to or from another party: Often the errors mean that mistakes have been made with money. Investigating and correcting data often leads to the discovery that incorrect invoices have been sent or received, or that invoices were not sent, or sent twice, and so on. Sometimes it is just internal management accounting that is wrong. Often it is another party who is demanding payment, and perhaps also compensation, or from whom more money can be claimed. The obvious financial significance of this aspect of a process and controls mess tends to generate strong feelings and pressure.

• Process, system, and control improvements: The intention is to stop the mess happening again. The changes could include retraining, software changes, management changes, IT support improvements, redesigned forms, extra audit tools – even a new IT system.

• Assurance activity: In other words, work to reassure people such as the Board and the external auditors that the mess is being sorted out and the impact on financial results is understood and limited.

• Lessons learned: Management often want to know what can be learned from the experience and how a similar nightmare can be avoided in future.

The work is tough, but not just because there are several aspects to what needs to be done.

What to expect

The work is tough because of what it requires from your mind. Messes vary, but my experience is that you can expect the following:

• They have usually developed over time and were nearly always predictable: By the time you get the job of sorting out a mess it is usually well established with a long history of problems. The answer to ‘Why did this happen?’ is usually that it was the inevitable result of doing something ambitious under difficult circumstances and not even realising the risks involved. The exceptions are the sudden messes that sometimes develop within about 3 months of a new computer system or process being implemented. These can be particularly scary.

• Other people have already tried and failed to sort the mess out: You might be the second or third person to grasp the poisoned chalice. It is quite possible that nobody mentioned this before you took on the challenge.

• Bad feelings have grown around the mess: Departments start to blame other departments and especially outside IT companies for the problems. Usually everyone is partly to blame, but that is not how they see it. With processes that pass through more than one department the errors tend to come to light towards the end of the process, leaving departments at the start of the process to feel that the problems are nothing to do with them, when in reality they are usually the original cause.

• Some people do not want you to succeed: Some people fear that they will be found guilty of causing, or failing to prevent, the mess. They may be right. Either way, they do not want you to make progress.

• Inadequate and incomplete information: It is very likely that there will not be enough information available for you to solve the puzzle completely. Logs may have been deleted, papers are missing, nobody ever checked until last week, and so on. Some items will be too much work to investigate for the small amount of money involved.

• Everyone has issues to tell: The first day is what I call the ‘paracetamol day’ because I usually need some to get through it. Once you start your investigation, after the minimum small talk, people start to unburden themselves of problem after problem. The systems and technical details are typically very complicated, the jargon may be unfamiliar, and the problems and theories seem endless.

• Most of what you are told is wrong: Despite the effort you will have to put into noting issues and thinking about them, most theories will eventually prove to be unfounded and a lot of information will be inaccurate. If someone gives you a diagram to show what is happening it is almost certain to be technically inept, incomprehensible, and misleading, even if it comes from the IT team. The exception to this is when you get to the original documents used to develop computer systems, but they will be out of date and too detailed so not much more use.

• Only a very few people know what they are talking about: People usually want to help you and, for that reason, sometimes explain things that are not their area of expertise. Perhaps they used to do the job, or a close colleague does. This is one of the reasons that so much of what you are told is at least slightly misleading.

• People are too busy to sort out the mess: Process and controls mess creates a heavy workload of corrections and checking. People are often too busy fighting these fires to sort out their problems. This places very tight constraints on what you can propose as a way forward.

• Departmental and expertise barriers get in the way: To sort out a mess you need to have a good understanding of the business and financial angles and be able to talk technically with the computer people. If you cannot do this, or cannot cross departmental boundaries, it will be very difficult to make progress. You will have to accept what people tell you if you have no real understanding of their area. The perfect trouble shooter is someone with no departmental attachments, who understands the business, is a skilled accountant, process and controls expert, writes software, and knows how to secure a Unix server. Nobody can fob this bloodhound off.

• The mess is more costly than people think: Messes are usually long established, which means people have got used to them. Their performance expectations and ideas about how much fire fighting is reasonable assume mess. Once you see what things could be like, and also examine all the knock on effects of the mess, the true cost looks much higher.

• Many people will think a new computer system is the only solution: Flawed computer systems are a big cause of problems, but in a mess people tend to think that fixing or replacing the system will solve all the other problems. It is rarely so, and often a dramatic improvement can be made with minor changes to systems and the procedures people follow.

What not to do

Common sense suggests that a number of things will be effective in sorting out a mess, but they are not:

• The key is to have a well managed project: No it isn't. It helps, especially later in the project, but the main reason people fail is ineffective work early on and managing inadequate resources better will not make much difference.

• Follow a phased plan – analyse the problem, devise a solution and an implementation plan, then carry it out: Too slow. Elements of the solution should flow from your early investigations. You need a plan that accomplishes both at once, particularly as it is often impossible to sort out the mess without putting some new checks in place and studying the errors they reveal.

• Collect all the issues together and prioritise them: This is only sensible when you have done a proper investigation of those issues. If they are just the issues people have told you then collecting them together and presenting them back is not much help. Also, it is common to find many additional issues that have not previously come to light.

• Follow a planned programme of work: You need a basic tactic that will work, but it is not efficient to stick to your original plan in detail. It is better to adapt your approach as new evidence comes to light. For example, while investigating mysteries keep a short league table of hot leads and review them every day to see that time is being spent on the most promising activities.

• Do a diagram on one sheet of paper: This is helpful if it can be done, but unfortunately trying often results in over-simplified ‘conceptual’ diagrams with inadequate labels that are no good for anything.

How to do it right

To succeed at sorting out a process and controls mess you need to do this:

• Sequence activity to accelerate benefits and exploit learning:

  • Combine investigation and solution building.
  • Make a bootstrapping implementation plan.
  • Establish early on the controls that indicate overall controls effectiveness.
  • Implement process control monitoring.

• Use clarification and order to maximise your mental power:

  • Clarify terminology.
  • Do your own diagrams, following rules of clarity.
  • Design clear reconciliations
  • Organise the ‘information’ you get.
  • Use statistics in bulk, and graphs.

• Use critical thinking to cut through misinformation:

  • Search for unlikely or inconsistent information continually.
  • Anticipate errors
  • Assess whether there could be more problems still to find.

• Do things others forget or feel they cannot do:

  • Walk around; do not respect organisational boundaries.
  • Look in the manual.
  • Be alert for poor cognitive ergonomics.

The following subsections discuss these in more detail.

Sequence activity to accelerate benefits and exploit learning

Combine investigation and solution building

There are two reasons for getting on with solutions before you have finished investigating the problems. Firstly, there is rarely enough information already available to explain all the mysteries and starting new control checks or starting to cleanse some data is a way to generate new information. Secondly, the sooner you get on with preventing more problems the better. You may not have a complete answer immediately but you can very quickly identify parts of it.

For example, suppose you are searching for lost data and take a copy of someone's reconciliation spreadsheet to see what you can learn from it. The spreadsheet has large amounts of useful data on it, but the design of the reconciliation is poor. After a day's work you have improved it so that you can examine the data more powerfully. A couple of hours extra work makes it user-friendly enough to give back to the owner as their new reconciliation spreadsheet. From now on they are checking more powerfully and that is likely to be a part of the final, improved process and controls design. Even if it is ultimately replaced with something even better you have made a fast improvement that has also helped you investigate the issues and may throw up new clues as you are working.

As another example, imagine the initial mystery is a reconciliation difference suggesting lost invoices. The reconciliation looks interesting but its results are hard to interpret. Your approach could be to design a reconciliation that is easier to interpret and more powerful, and that will probably explain the mystery. This same reconciliation is also something that will be a key control in the new scheme so once you have done it once it will become a regular task for someone.

Make a bootstrapping implementation plan

Being in a mess often makes people too busy to solve their problems. It is vital to plan improvements with this in mind. Start with things that can be done easily – like stopping doing some useless activity – but which release time for other things.

The most important thing to focus on is removing the causes of errors. Prioritise the problems if you can so that you start with the most prevalent and time consuming errors that are also relatively easy to eliminate.

For example, one billing team facing a huge mess realised that a small change to some error checking software would cut down the number of false or trivial alarms they were having to deal with, releasing time for someone to investigate the reasons for the errors that worried them. Later, success in reducing the incidence of some types of error released further time, which was put to the same good use, and so the virtuous circle continued. In the first 6 months the size and composition of the team changed dramatically and improvements continued for some time after that.

Establish early on the controls that indicate overall controls effectiveness

Everyone wants to know if things are improving or not. It is helpful to put into place early on controls that provide information about overall controls effectiveness. It is rarely possible to get perfect information, but useful indications are usually practical.

For example, if the problem is thought to be loss of information between two departments, then an overall comparison in total between the two departments is an obvious way to see the difference and as the difference gets smaller it is evidence of progress, though not proof.

Implement process control monitoring

Processes in a mess rarely have any kind of process management going on. Nobody knows how healthy or unhealthy the process is. There may be little information about how much work is being done, let along what the error rate is, or where the backlogs are.

Since this is vital to managing the health of a process and you also want to watch your progress one of the first control improvements to start implementing is nearly always process control monitoring and reports to support it.

The reports will usually show end-to-end workload, error rates (as measured by the results of controls), backlogs (e.g. uncleared suspense items, unprocessed orders), systems support (e.g. down time, progress on bug fixing), and resources used (e.g. headcount). Some statistics might be gathered daily, some weekly, and some just monthly. Graphs are essential if people are to use the statistics and good design of the reports makes a huge difference to their usefulness.

Often the process passes through more than one department, so a committee is needed to monitor the process end-to-end. If this has not happened before you are well on your way to a lasting solution when you set one up and get them working.

Use clarification and order to maximise your mental power

Since the mess has probably been looked at by others before you came along, how are you going to succeed where others have not? The key to your success is to think more clearly than anyone has before. Where others have said ‘OK, I think I understand.’ you will say ‘Let me just check that I've understood exactly what you are saying’. Where others have accepted the jargon people are using you will insist on jargon that is precise and consistently used. Where others have tolerated badly designed reconciliations and useless diagrams you will not.

Clarify terminology

The way people talk and think about the messy processes and controls will tend to be messy. Jargon will be out of control, with many different ways of saying the same thing, and no ways to say certain concepts that are important. Some jargon will be needlessly confusing or even misleading. This jargon is part of the fog of confusion that surrounds mess and to think your way through the mess you must sort out the jargon, at least within your own mind or team.

It is vital to clarify terminology early on, and helpful to invent notation for numerical aspects of your analysis. For example, if you are trying to reconcile information at one stage in the process to the same information later on it is important to be clear exactly where any figures have come from, and to distinguish between the front and back end of a computer application, for example.

It is particularly important to agree names for reports used as the source of information, and to state them as sources rather than just saying ‘from the XYZ system’. The XYZ system may give conflicting information on its various reports. When sorting out a mess, trust nothing.

Do clear diagrams

It does not matter so much what style of diagram you use but sticking to certain rules makes diagrams of any type easier to follow. If diagrams are easier to follow they are easier to inspect for omissions and errors. They help you identify what you still do not know. Clarity is vital.

Here are the rules:

• Annotate all boxes and arrows. For example, on a data flow show what data is flowing (not the physical form). On a process box say what is happening.

• Annotate using the nearest thing to plain English you can. Don't just type in an acronym; add a couple of words of explanation. E.g. Instead of writing just ‘EXX system’ write ‘EXX – monthly bill production’.

• Organise transaction data flows so they go left to right across the page, or top to bottom down it. It should be easy for the reader to see where to start and to follow the main path through the diagram.

• Show reference data feeds coming in from the side of the transaction data flow.

• Do not fudge notation: Bad diagrams have invented notation with no explanation, typically dotted lines between or across boxes, odd shading, and outlines to divide up the picture.

• Do not get conceptual: Keen to make a diagram clearer, people often try to create simplified pictures that are not strictly accurate, or involve abstractions of notation that are not explained. Triangles, concentric circles, layers, pictures of jigsaws – all signal a conceptual diagram that can be misleading.

• Fill the page but do not compromise on the clarity of annotation or the orientation of flows. Don't put more than about 10 boxes on a sheet of A4.

• Prefer to layer diagrams rather than join the edges of pages. Layering is a technique used by proper systems analysts. A box on the top level can be ‘exploded’ so that more detail within it becomes visible on a lower level page.

Design clear reconciliations

Most reconciliations are incomprehensible. This means they are hard for others to understand, hard to check, and often contain errors because the designer himself is muddled. It is usually worth refining the design so that it works intuitively and labels/descriptions need no further explanation. As before it is vital to show exactly where data has come from.

On MS Excel it is possible to add comments to give further information. These do not display normally but when you put your mouse pointer over the little red corner of a cell the comment pops up. This is useful for clarifying sources without creating too much clutter.

Organise information

Having neat filing and perhaps a database is helpful but it is even more important to organise information so that your brain can grasp it more easily and precisely. There is a lot of information to take in and think about so to help your brain it is important to lay information out neatly and in a way that helps bring out structures and patterns. It sometimes takes several tries to get something just right. Tables, diagrams, and graphs are the main tools. Effort spent refining the layout to compress more information onto the page, and make notation more compact and intuitive will be repaid.

Use statistics in bulk, and graphs

It is a mistake to stare at a few clues trying to work out what they might mean. Grab as much data as you easily can, organise it, format it, and display it so you can easily spot breaks in patterns, and trends.

For example, if you are trying to fathom the meaning of a monthly reconciliation difference one of the easiest ways to get ideas is to pull together a large volume of easily accessible data, put it on a spreadsheet and look at graphs. Instead of looking at a monthly difference you could be looking at daily differences, which might give you clues about when the difference(s) arose.

Plot different statistics over time on the same graph to see if their movements seem to be linked in any way.

Use critical thinking to cut through misinformation

Search for unlikely or inconsistent information continually

The best defence against the misinformation you will be given is to evaluate it critically. Is it likely that it is true? Is it something that could easily be misunderstood? Does it seem inconsistent with other information? If it seems doubtful it's probably wrong.

If I had to pick out the one skill that has helped me most through this kind of project I would have to say that spotting dodgy information by critical thinking is it.

Anticipate errors

The problems people tell you about could be the tip of an iceberg. Weakly controlled processes have insufficient checks built into them so errors pass unnoticed.

Once again, thinking can tell you where to look for problems. If you see some work being done in a way that must be difficult to get right all the time expect errors, even if none have been reported. For example, a table of thousands of meaningless reference numbers typed in by hand over a period of months with no checks performed is going to contain errors, even if none have been found to date.

If any of the usual causes of mess are present (e.g. chronic financial distress, rapid expansion, repeated reorganisation and change of ownership, very old computer applications, large numbers of databases with seemingly similar functions) but people say everything is fine do not believe them until you see proof.

Have confidence in your inferences. Many times I have anticipated a problem, sent an assistant to investigate it, and been told that everything is fine. I have been known to send the assistant back two or three times to search harder and so far have always found the problem I expected.

Decide if there are more problems still to find

Think critically about possible explanations to mysteries.

A beginner's mistake is to find a cause of errors and think ‘Aha! I've found it.’ Process and control messes usually involve multiple error causes, not just one. Try to assess whether there could be more error causes still to find. Often this is easy. For example, you might have a mysterious reconciliation difference of about 2% and uncover an explanation for the differences. However, some rough calculations might show that the explanation can only account for differences of around 0.01 – 0.5%, so there must be more to find.

Do things others forget or feel they cannot do

Walk around

As you have fact finding conversations with people it helps to meet them at their desks rather than in a meeting room, unless you need white boards. See screens, reports, and papers with your own eyes if possible, though it can be much more efficient to make enquiries by telephone in a very large organisation spread across more than one site.

If organisational boundaries force you to leave an issue with someone else for investigation without the opportunity to push them along the chances are that nothing will happen very quickly.

Look in the manual

Since many people you might interview for information have not bothered to look in the software manuals it is often possible to become quite an expert in just a few hours. Well worth it.

Check usability

One of the most common reasons for errors and omissions is human error resulting from confusing software and paper forms. Be super critical of the design of forms, and their wording. Badly designed paper forms can often be sorted out very quickly and easily, saving time for people who have to use them and cutting down errors and the time they waste.

A variant on this theme is the untidy desk. Many years ago I used to audit an accountant whose desk was always piled high with papers. Perhaps ‘piled’ is misleading – the papers were one heap, spilling out onto the floor, up to 30 cm high in some places. All the drawers were full, and the space under the desk, and there were boxes of mess around his desk. His staff sometimes tidied up when he went on holiday, against his wishes. For some reason, although everyone could see this was an accounting blunder waiting to happen nobody felt it was quite right to insist that he tidy his desk up and start filing things or dealing with them when they arrived at his desk. Today I would have no hesitation in making that recommendation.

What to do if you are on the Steering Committee

If you are involved in clearing up a mess and need to delegate the task to someone and monitor their progress, the guidance above may be very helpful in showing what to look for and what is likely to be a mistake. Here are some key points for Steering Committee members:

• Choose the right people: Progress is blocked if the team cannot cross organisational boundaries or does not possess the range of expertise needed to cover all aspects of the task – process, finance, and IT. Multi-skilled people are particularly useful but you will probably have to settle for a multi-skilled team.

• Look for a sensible plan: In other words, reject something that assumes all investigation will be done first, and then be followed by implementation of solutions. This kind of plan makes the investigation too slow and inefficient, and delays the arrival of obvious improvements.

• Expect fast results: If the team seems to be wallowing around for several weeks but the progress they are so proud of amounts to very little it is past time for the Steering Committee to act. These are the signs of a team that is bogged down and failing.

• Question hard on details and clarity: Clarity is the key to progress in these projects. If you find it hard to understand progress reports and explanations of problems the chances are that team members are not clear enough in their own minds, or lack the communication and thinking skills to perform the project well. In a clear up project, never assume people know what they are talking about even if you don't understand them.

• Be forgiving: These projects are very challenging. Even a good team can struggle at times. Frequently there is not enough information to avoid writing off some data as impossible to rescue.

Finally

Process and controls mess ranges from the light breeze of the occasional customer complaint to the terrifying hurricane of a dismally failing Finance function. Whatever the amount of mess you are taking on the guidance above should help you. Do not let such a mess destroy your health and alienate your family. Calm, clear thinking is your most powerful weapon in reaching the truth, and early action is your route to the sunny uplands of a healthy process with sound controls.

Made in England