In the autumn of 2019, at a specialty chemical plant in the southern United States, researchers fixed small cameras to the brims of operators' hard hats and then left them alone to work. The task was filling operations, moving product out of storage tanks and into rail cars and trucks, somewhere around thirty to forty written steps per run. Fourteen operators took part across two sites, one of them working from paper and the other from screens. Over forty procedures, the cameras captured 1,422 individual steps.1

Three researchers then coded every one of those steps against what the document said should have happened. Just under a third came back different. The published figure is 32.9 percent.1 Every observed task was completed successfully.1

Those two findings arrived together, and that is the part worth slowing down for.

The deviations had a grammar to them

The team sorted what the cameras caught into three kinds of change. Operators reordered steps. They took actions the procedure had not called for. Occasionally they left a step out. Reordering was the most common of the three at around fifteen percent of steps, with action changes near ten and skips near eight.1 The authors give those as approximations and I am going to leave them that way, since the whole study covers one company and one type of operation. They say as much themselves, describing the result as analytical rather than statistical generalization.1

What holds my attention is the shape of the distribution. Someone trying to get out of the plant early leaves a step out. Reordering is what a person does when the sequence in front of them has stopped matching the sequence they were handed and they have decided to finish the job anyway. The most common form of departure from the written procedure was the one that takes the most thought.

A study published the following year in biomedical laboratories found something with a similar texture from a different direction. Fifteen workers were interviewed about how closely they held to their standard operating procedures, and all of them described running an on-the-spot mental risk assessment and then departing from the written procedure based on what that assessment told them.2 Fifteen interviews is a small sample, and the author runs a biorisk consultancy, so I would not build much on it standing alone. Set beside 1,422 coded steps in a chemical plant, it does suggest the pattern is not local to one facility.

The number that travels upward is a different number

Here is the operational problem. Almost nothing in the way a large organization is wired carries that 32.9 percent to a person who could do anything about it. What travels upward is an adherence rate, or a count of closed audit findings. A senior leader reading those believes they are reading a description of how the work went.

An adherence number measures the distance between a document and a plant. It will not tell you which of the two is wrong, and most organizations have already decided.

A smaller version of this turned up at an organization I advise. Someone had built a change management form months earlier to capture every application and website change, each one carrying its backup steps and a rollback plan. The form had gone months without being used with any consistency. When we went looking for the process the team was running instead, we found it living mostly inside one person's head, which made it fast and also made it a single point of failure. The first instinct in the room was to enforce the form. The more useful question was why the undocumented path kept winning, and what the form was demanding that the person's head had already handled without being asked.

What the noncompliance framing costs you

Call that 32.9 percent noncompliance and you have made a decision about what happens next. The reorder stops getting mentioned. The adaptation goes somewhere you cannot see it, and it keeps happening anyway, because the conditions that produced it have not changed. Now you are carrying both the departure from procedure and the loss of any visibility into it. The paper's authors put the cost plainly, writing that framing deviation as error prevents an organization from learning the situated expertise those adaptations contain.1 They are also careful to say that not every adaptation makes the work safer.1 Both halves matter, and only one of them is comfortable.

Worth naming who paid for the research. The study was funded by the Next Generation Advanced Procedures Consortium, an industry group whose interest runs toward the redesign of procedures.1 The authors declare no conflict, and what they report is an observation of what people did rather than an evaluation of anybody's product. You should still know whose money was behind the cameras.

I wrote a couple of weeks back about why dropping a degree requirement barely changed who actually got hired, and I am not going to run that argument again here beyond noting that the two sit back to back. That one followed a rule that outlived the deletion of its own document. In this one, the work is what persists and the document is what drifts away from it. The same instinct shows up in safety reporting too, where leaders account for missing hazard reports by blaming the reporting form while the people not filing describe something else. In each case the paper and the behavior have come apart, and in each case the paper is the only one of the two that a leader can see from where they sit.

So the question I would put to anyone running an operation of any size. When somebody on your floor changes the order of two steps because the order in the document does not survive contact with the actual equipment, where does that fact go? If the honest answer is nowhere, or into a corrective action and then nowhere, your procedures will keep drifting from the work and you will find out about the drift at the worst available moment. If you are not sure what the answer is, that is worth an afternoon of somebody's time.