A pilot is a small bet: one workflow, a few weeks, a limited budget. It is only worth something if, at the end, everyone agrees on whether it worked. That agreement has to be made before the pilot starts, not after.
Step 1: record the baseline while the work is still manual
The baseline is how the workflow performs today. Measure it for at least one normal week, with the people who do the work.
| Measure | What it tells you | How to get it |
|---|---|---|
| Hours | How much time the workflow takes | Ask the team to log it for a week, or time a sample |
| Cost per operation | What one order, ticket or invoice costs to handle | Hours × cost of an hour, divided by the number of operations |
| Errors | How often something has to be redone | Corrections, returns, re-sent documents, complaints |
| Processing speed | How long a customer or colleague waits | Time from the request arriving to the answer or the action |
| Throughput | How many cases the team handles | Operations per day or per person |
Write down how each number was measured. At the end you will measure it again in exactly the same way.
Step 2: agree what success means
Pick two or three criteria, not ten. Each one needs a number and a direction. For example:
- hours spent on the workflow down by half;
- first answer to a customer within ten minutes instead of the next morning;
- errors no higher than today.
These are illustrations; the right targets depend on your baseline. Put the criteria in writing, with the names of the people who will judge them.
Step 3: keep the scope small
One workflow, one team, the systems it already uses. A pilot that tries to fix everything at once cannot be measured, because nobody can say which change made the difference.
Step 4: measure during the pilot, not only at the end
- Log every automated step, so you can check what happened and why.
- Count the cases passed to a person, and the reason. A high number is not a failure: it shows where the rules need work.
- Note problems as they happen, not from memory at the end.
Step 5: judge the result together
At the end, measure the same numbers the same way and put them next to the baseline. Then there are three honest outcomes:
- It met the criteria: plan the rollout to daily operations.
- It came close: find out why, adjust, and decide whether to continue.
- It fell short: stop. A pilot should not commit you to a rollout.
Common mistakes
- No baseline, so nobody can say whether anything improved.
- Success criteria agreed after the results are in.
- Measuring the pilot in a quiet week and the baseline in a busy one.
- Counting only time saved and ignoring errors and waiting time.
This is how we run Razum AI pilots: baseline first, criteria agreed before we start, results reported against them. See how a pilot works.
