The Numbers Do Not Speak for Themselves

When I was in college, I had one of the most coveted jobs in the store.

I was in charge of the markdown team.

This meant I carried the pricing gun.

For anyone who has never experienced the strange power of retail markdowns, this little handheld device could, with enough luck and patience, turn a $7.99 shirt into a $0.99 shirt.

The prices themselves were controlled on the back end. My job was to scan the merchandise. If a SKU had been discounted, the scanner printed a new ticket for the tag.

Simple enough.

Except the store contained an enormous number of items.

Scanning everything could take the better part of a week. And while I was technically assigned to markdowns, I was also trained in customer service. So I still had to help customers, relieve cashiers, process returns, answer questions, and generally participate in the reality of running a store.

None of that changed the fact that I had a scan quota.

Somewhere, somebody had decided that a certain number of scans represented a productive markdown shift.

So I did what a mildly mischievous, sufficiently pressured college employee might do when confronted with a metric that did not quite match the work.

I tested it.

If I was behind on my quota, I would sometimes double- or triple-scan full-price merchandise.

I did not print duplicate markdown tickets.

I did not change the price.

I did not steal anything.

I just scanned the same item more than once.

Click.

Click.

Click.

The scanner recorded three actions.

The store still had one shirt.

I suspected, even then, that the system was probably recording each scan and that this might create some strange-looking data somewhere downstream.

I was also perfectly willing to play dumb and find out what the process permitted.

This was not Ocean’s Eleven.

It was retail.

But it taught me something I have never forgotten:

Metrics are not trustworthy until you understand how they were produced.

What Was the Job Actually Supposed to Be?

The real objective of the markdown process was not to pull the trigger on the scanner a certain number of times.

The objective was to make sure the store’s unique inventory had been reviewed and that any discounted merchandise received the correct price ticket.

Those are not the same thing.

One measures coverage.

The other measures activity.

The quota turned a practical assignment into a counting exercise.

If I scanned 1,000 unique items, that probably represented useful work.

If I scanned 700 unique items and pulled the trigger an additional 300 times on merchandise I had already checked, the system might still show 1,000 scans.

The number looked productive.

The store was not 30 percent more complete.

Management thought it was measuring productivity.

What it had really created was an incentive to manufacture the appearance of productivity.

That distinction matters in nearly every kind of operation.

We often measure what is easy to count because easy-to-count things are convenient.

Calls handled.

Tickets closed.

Orders entered.

Units processed.

Tasks completed.

Lines of code.

Hours billed.

Cases resolved.

Transactions posted.

Time to first response.

Some of these may be useful metrics.

But usefulness depends on whether the number is connected to the outcome we actually care about.

If it is not, people will still respond to the number.

They just may not improve the work.

Quotas Change the Assignment

A metric does not sit quietly on a dashboard observing human behavior.

Once people know it matters, the metric becomes part of the process.

Hard quotas do not merely measure behavior. They distort behavior toward whatever is easiest to count.

That does not mean employees are dishonest.

It means employees are adaptive.

People learn what the organization rewards, what gets questioned, what gets escalated, and what gets ignored.

If volume matters more than quality, people find volume.

If speed matters more than completeness, people find speed.

If closing the ticket matters more than solving the problem, tickets get closed.

If the target is scan count, somebody eventually discovers that the scanner does not appear to care whether the shirt is new.

That somebody happened to be me.

Poorly designed systems do not eliminate human creativity. They redirect it toward increasingly creative nonsense.

The important managerial lesson is not, “Employees will cheat.”

That is too easy, and it lets the system designer off the hook.

The better question is:

What behavior did we make rational?

If someone can satisfy the metric without satisfying the intended outcome, the measurement is incomplete.

If everyone begins optimizing around that gap, the measurement can become actively misleading.

When leadership applies pressure without understanding the work, people do not necessarily improve the outcome. They improve the appearance of the outcome.

And sometimes they do it with impressive efficiency.

Measurement Integrity Starts Before the Number

We tend to treat metrics as if they arrive fully formed.

They do not.

A number is the end of a process.

Before trusting it, we need to understand everything that happened upstream.

Does a process exist?

Was it communicated?

Was it taught consistently?

Does everyone using the metric mean the same thing by the words surrounding it?

If we measure “resolved,” does resolved mean the problem is fixed, a workaround was provided, the customer stopped responding, or somebody changed the status field?

If we measure “completed,” does that mean the task was actually finished, or that someone clicked the button that says it was?

If we measure “productivity,” what exactly counts as productive?

Definitions matter.

So do competing responsibilities.

My markdown quota did not disappear because a customer needed help finding a size, a cashier needed a break, or someone returned half their wardrobe ten minutes before closing.

Those activities were still work.

They just were not the work being counted by that metric.

That creates a problem because operational performance rarely happens in isolation.

People work inside systems of competing obligations.

A customer service representative may be expected to respond quickly and solve the issue thoroughly.

A warehouse worker may be expected to move quickly and verify accuracy.

A project manager may be expected to hit the date and manage risk.

A support engineer may be expected to close cases and investigate deeply.

Those goals can coexist, but they do not always cooperate.

When the measurement rewards one side of the tradeoff and ignores the other, people receive a very clear message about what matters most.

Even if leadership says otherwise.

What Does the Metric Encourage?

Every metric has behavioral consequences.

Some are intentional.

Some are not.

Before applying pressure to a number, ask what someone could reasonably do to improve that number without improving the underlying outcome.

Could they split one task into several?

Could they close and reopen work?

Could they defer complicated cases?

Could they overproduce something easy?

Could they scan the same shirt three times?

Could they change when or where something is recorded?

Could they avoid work that threatens their average?

Could they perform the visible part while quietly skipping the part nobody measures?

This is where loopholes become operationally useful.

Not because we want people exploiting them, but because loopholes expose the gap between what we said we wanted and what the system actually rewards.

The employee who finds the loophole may be doing the organization a favor.

Especially if somebody bothers to ask why the loophole exists.

Management thought the quota would whip everyone into shape.

It did.

Just not the shape they intended.

Everyone ended up all bent out of shape.

The Testing Mindset: Putting Yourself in the System

This is also where I’ve realized something important from another part of how I think: testing.

When I test things—whether it’s a system, a process, or even a simple workflow—I don’t just check whether it works under ideal conditions.

I put myself in the position of the person using it.

And then I try to break it.

Not out of malice, but out of curiosity.

What happens if I do this faster than expected?

What happens if I repeat an action?

What happens if I skip a step?

What happens if I misunderstand the instruction on purpose?

What happens if I behave like someone who is tired, rushed, or incentivized to optimize the wrong thing?

In other words, I try to exploit the system—not to harm it, but to reveal its assumptions.

That is exactly what happened with the pricing gun.

I wasn’t trying to cheat the store in any meaningful sense.

I was stress-testing the metric.

And the system immediately showed me something important: it was counting activity, not understanding work.

That is what good testing reveals.

Not just whether something works, but how it behaves when reality is messy, human, and slightly adversarial.

And in organizations, reality is always slightly adversarial—even when no one intends it to be.

People are constantly adapting to constraints, incentives, and ambiguity.

Testing simply makes that adaptation visible.

The Workaround Matters Too

There is another important question:

Does the workaround create downstream harm?

My extra scans may have been harmless at the point of action.

I did not change prices or duplicate tickets.

But what happened to the data?

Did someone later use total scan counts to estimate inventory coverage?

Did repeated scans inflate activity reporting?

Did the system interpret each scan as evidence that a unique item had been reviewed?

I did not know.

And that uncertainty is exactly the point.

Local optimization can create downstream distortion even when the person doing it sees no immediate consequence.

This happens constantly in complex organizations.

Someone creates a workaround to get an order processed.

Someone changes a date because the system will not accept the real sequence.

Someone closes a task because the reporting deadline is approaching.

Someone copies information into a spreadsheet because the official system is too slow.

Someone enters a placeholder because a required field does not fit reality.

Each workaround may make sense locally.

Each one may also change what later appears in the data.

This is why measurement integrity is not just a reporting concern.

It is an operational concern.

You cannot interpret the output honestly if you do not understand the behaviors that produced it.

Can the Result Be Independently Validated?

A strong metric should have some relationship to reality that can be checked independently.

If the scanner says 10,000 items were reviewed, can we verify unique inventory coverage?

If the dashboard says 98 percent of cases were resolved, can we determine whether the issues were actually solved?

If the report shows productivity increased, did output improve, quality improve, customer experience improve, or cost decrease?

If none of those things changed, what exactly improved?

A metric becomes more trustworthy when it can survive contact with another source of evidence.

That may be a physical count.

A downstream result.

A customer outcome.

A quality check.

An audit.

Another system.

Or simply the people doing the work saying, “That number does not match what is happening here.”

Perspective is data.

So is contradiction.

If the dashboard says everything is healthy and everyone closest to the process says it is falling apart, the answer is not automatically that the employees are negative.

The dashboard may be measuring the wrong thing.

AI Will Not Rescue a Bad Metric

This becomes even more important as organizations use AI to analyze operational data.

AI could analyze every scan perfectly and still completely misunderstand the work.

It could identify that certain SKUs were scanned multiple times.

It could flag the pattern.

It could calculate the frequency.

It could identify which employees did it most often.

It could compare shifts.

It could probably create a lovely summary showing that something unusual was happening.

But pattern recognition is not process understanding.

Without context, the same pattern could represent fraud, failure, friction, a bad incentive, a training issue, an equipment problem, or perfectly normal work.

The AI does not know that the employee was covering register breaks.

It does not know the markdown process took a week.

It does not know whether the quota measured unique inventory or trigger pulls.

It does not know what management communicated.

It does not know whether the scanner was designed to prevent duplicates.

It does not know whether repeated scans created any actual business harm.

Those questions require process understanding.

AI can help us find the pattern faster.

It cannot make a broken measurement meaningful simply because the analysis is sophisticated.

Bad context does not become good context because it arrives in a prettier dashboard.

The Number Is an Output, Not an Explanation

This is the discipline I mean when I talk about measurement integrity.

Before trusting a metric, ask:

Does the process actually exist?

Was it communicated and taught?

Are definitions consistent?

Does the metric measure the intended outcome or merely an easy activity?

What competing responsibilities affect performance?

What loopholes and behaviors does the quota encourage?

Does a testing mindset—asking how the system behaves when people try to break or exploit it—reveal hidden assumptions or failure modes?

Does a workaround create downstream harm?

Can the result be independently validated?

Those questions are not an argument against measurement.

They are an argument for better measurement.

Numbers are useful precisely because they can help us see things we might otherwise miss.

But numbers do not speak for themselves.

People interpret them.

Systems produce them.

Processes shape them.

Incentives influence them.

Definitions constrain them.

Workarounds distort them.

And sometimes a bored college student with a pricing gun gives them a little extra personality.

The lesson is not that measurement is futile.

The lesson is that responsible measurement requires understanding the work well enough to know what the number actually represents.

A good metric helps us see reality more clearly.

A bad one gives us confidence in a story that was never true.

A dashboard is only as honest as the process that produced it.

I Am Here for YOU.