Does any man in this room know how the machine works or did we only learn how to click the button on the screen? This is the question that sits in the back of the mind and it stays there during the morning meeting. It stays there when the manager opens the laptop and the light from the screen hits his face.

The manager is happy and he points at the wall. There is a chart on the wall and the chart is full of green bars. The bars show that ninety-four percent of the dealers are now certified. Two years ago the number was sixty-one percent. This looks like progress and the people in the room want to believe in progress. They want to believe that the network is stronger and they want to believe the customers are happy.

But the field reports tell a different story and the field reports do not care about the chart. The number of broken compressors is the same as it was two years ago. The number of leaked flares is the same. The technicians go to the houses and they fix the machines but they make the same mistakes they made when the chart was red. The training is finished and the certificates are printed and the wall is green but the work is exactly the same.

The Beautiful Drawer of Colored Folders

I spent my morning in the museum basement and I organized my files by color. I used red folders for the broken exhibits and blue folders for the tours and green folders for the files that are finished. It took four hours and the drawer looks beautiful. My boss walked past and he saw the colors and he smiled.

He did not ask if the exhibits were fixed and he did not ask if the donors had sent the checks. He saw the green folders and he felt that the world was in order. This is the danger of the proxy. We pick a thing that is easy to see and we pretend it is the same as the thing that is hard to do.

Training completion was picked as a proxy for capability. It was picked because capability is hard to measure and training completion is easy to track. You can see a man sit in a chair and you can see him watch a video. You can see him pass a quiz. But a ninety-four percent success rate on a screen is a perfect score and it is also a perfect lie. It does not measure what the man can do with his hands when the sun is hot and the customer is angry. It only measures his patience for the “Next” button.

The Illusion of the Progress Bar

The technician sits in his truck and the air conditioner in the truck is on. He has a tablet on his lap and the tablet shows a video about the cooper hunter tech support system. He does not listen to the audio but he watches the progress bar move from the left to the right.

He knows that he must wait for the bar to finish before he can click the button. He thinks about the lunch he will eat and he thinks about the next job on his list. When the video ends he takes the quiz. The questions are simple and he has seen them before. He clicks the answers and he gets a hundred percent. He is now a certified pro-tech dealer and the dashboard in the main office turns a little more green.

The technician goes to a house and he carries his tools. He has a manifold gauge and he has a vacuum pump and he has a roll of copper pipe. He cuts the pipe and he makes a flare. He does not use a torque wrench because the torque wrench is in the back of the truck and the truck is down the hill. He tightens the nut by feel and he thinks it is tight enough.

He does not perform a nitrogen purge because he did not bring the tank inside. He turns on the machine and the air is cold and he leaves. Three months later the refrigerant is gone and the compressor is dead. The man had the certificate in his pocket but the machine is broken on the ground.

The Cheapest Way to Win the Game

We have optimized the training for completion and that is what we received. Every dealer hit the target because hitting the target was the only thing the company asked them to do. When you set a target on an input the people will find the cheapest way to meet that input. The cheapest way to meet training completion is to finish the training without learning anything. It is a tax on time and it is not an investment in skill.

The program has excellent numbers and it has no more information about its network than it had before the program started. A proxy survives only while nobody is trying to win the game. As soon as the proxy becomes the goal it stops being a good measure. This is a law of the world and it is a law that every manager forgets.

If you measure a man by how many calls he takes he will take many calls and he will help no one. If you measure a man by how many miles he drives he will drive in circles. If you measure a man by a certificate he will get the certificate and he will remain the same man he was before.

Messy Measurements and Hard Truths

The organizations that handle this well are the ones that are never satisfied with the map. They change what they measure and they do it often. This looks like indecision to the people who want the world to stay still. It looks like a lack of focus. But it is the only defense against the people who want to win the game without doing the work.

You must measure the flare and you must measure the vacuum and you must measure the time the machine runs without a repair. These things are hard to track and they are messy and they do not make a pretty chart on a Monday morning.

In the museum we have a training for how to handle old bones. Every person on the staff has a certificate that says they are an expert. Last week a man in the basement picked up a skull from the Ming Dynasty. He held it with one hand and he used the other hand to check his phone. He had the certificate in his file but the skull is now in ten pieces on the floor. The certificate did not hold the bone and it did not stop his hand from reaching for the phone. It only told the insurance company that we had done our part.

The Brand as a Ghost

The Pro-Tech Dealer Program is a good program but its value depends on the truth. If the certification tracks field capability then the brand is strong. If the certification only tracks the ability to finish a module then the brand is a ghost. The heat pump is a complex machine and the cooper and hunter ac unit requires a man who cares about the details.

You cannot teach a man to care with a video and you cannot measure his care with a quiz. You must go to the field and you must look at the work. We look at the dashboard and we see the ninety-four percent and we feel safe. We feel that we have built a wall against failure.

But the wall is made of paper and the rain is coming. The real work happens in the dark and it happens in the crawlspaces and it happens on the roofs. It happens when no one is watching and there is no “Next” button to click. The only thing that matters is the machine and the man and the way the two interact.

Red Files to the Front

The dashboard shines with the light of a thousand finished lessons but the compressor remains cold in a house that stays hot.

We should stop looking at the screen for a moment. We should look at the warranty claims and we should look at the repeat visits. We should look at the dealers who never call for help because they know the answers already. Those are the people who are certified by the work itself.

The others are just people who are good at taking tests. We have spent two years building a system that rewards the test-takers and we wonder why the machines are still breaking. It is time to change the metric and it is time to admit that we do not know as much as the chart says we do.

I will go back to my office and I will look at my colored folders. I will take the red files and I will move them to the front. They are the problems that are not fixed and they are the truth of the museum. The green folders can wait. The green folders are just the things I have already decided to ignore.

We must do the same with the dealers. We must look at the red and we must find out why the work is not changing. Only then will the certification mean anything at all. Only then will the map match the land.