# A scoring sheet for comparing order automation suppliers

Compare proposals on reading orders, holding unclear lines, the link to your order system and who owns the rules. Must haves first, then twelve weighted questions scored on evidence.

Updated 5 November 2026 . 8 min read . Alexey Shurov

## The short answer

Judge a proposal for automating emailed orders on four things. How well it reads your real orders, what it does with the lines it cannot read, how it reaches your order system, and whether you can change it or leave without the supplier. Check a few must haves first. Then score twelve questions from 0 to 3 against evidence on your own test orders, multiply each score by its weight and compare the totals. The scores on this page are made up to show the method.

## How to score them, in two minutes

In under three minutes you will know how to compare suppliers on your own orders, what to score, and why the weights come first.

Every example in the video is made up to show the method.The words of the video

Are you comparing suppliers who offer to enter your emailed orders for you? Give me three minutes before you choose. I'll show you a simple scoring sheet that compares them on your own orders.

First, what are they offering? An order arrives by email, often as a spreadsheet or a scanned page. Software reads every line. And the order goes into your order system, ready for your team to check.

Here's what I've learned. Every supplier looks good in a demo, because the demo uses their own orders. What matters is what happens to yours, especially the messy ones. So test them all on the same set of your real orders.

Here's a made up example with two suppliers. Supplier A scores 58 points, and Supplier B scores 66. That gap is smaller than one important question, so the sheet calls them close. And it tells you what to test next.

So here's the sheet. It scores four things. How well it reads your orders, and what it does with lines it can't read. How it reaches your order system, and whether you can change the rules or leave without the supplier.

Most people give accuracy the most weight. But software that reads most lines and enters its guesses makes more work, not less. I'd check two things together, how well it reads and what it does with a doubt. A blank should stay blank, and go to a person.

One more thing I always check. If you set the weights after you've seen the scores, it's easy to favour the supplier you already like. So set them before you see any scores. And only a test on your own orders counts as evidence.

So that's the sheet. You get a fair comparison, where they really differ, and what to test next. And a choice you can explain later, with the evidence next to it.

The scoring sheet and two made up suppliers are free on shurco.ai. Score your shortlist on the same test orders.

### 1. Agree the must haves before any scoring

Write down the few rules a proposal has to meet before it is scored at all. Nothing is posted without a person approving it during the pilot. Unclear lines are held, never guessed. Every original email and file is kept. You can take your data with you. A proposal that fails one of these is out, however well it scores elsewhere.

### 2. Ask every supplier for the same evidence

Give each supplier the same test pack of twenty orders, made up or with customer details removed. Include a two page PDF, a spreadsheet, an order typed into the email itself, a scanned page, a repeat of an earlier order and a line where the customer's unit differs from yours. Ask for the result line by line, with every line read, held or missed. A slide about accuracy is not evidence.

### 3. Score each question from 0 to 3

Give 0 when there is no answer or only a promise. Give 1 for a clear answer with nothing shown. Give 2 when it is shown on the supplier's own demo orders. Give 3 when it is shown on your test pack and the result is written down. Score with two people from the order desk and settle any difference by looking at the evidence again.

### 4. Multiply by the weight and compare the totals

Each question carries a weight from 1 to 3. A weight of 3 means the order desk cannot trust the system without it. Multiply each score by its weight and add up the points. Treat a gap smaller than one heavy question, 9 points on this sheet, as a tie, and send the next test to the area where the proposals differ most.

## The scoring sheet with two made up proposals

**Made up example.** Every score here is made up to show the method, for Supplier A and Supplier B. Points are the score multiplied by the weight.

| Area | Question | Weight | What earns a 3 | Supplier A score | Supplier A points | Supplier B score | Supplier B points |
|---|---|---|---|---|---|---|---|
| Reading | Reads every line of your test orders, tables split across pages included | 3 | A line by line result on your test pack with every line accounted for | 3 | 9 | 2 | 6 |
| Reading | Matches customer item codes to your own codes from your mapping | 3 | Shows the mapping used for each line and flags codes it has not seen | 2 | 6 | 3 | 9 |
| Reading | Handles units and pack sizes, a box of ten against single items | 2 | Converts only with a rule you gave it and holds the line otherwise | 1 | 2 | 3 | 6 |
| Held lines | Sends unclear lines to a named person with the reason | 3 | Every held line shows the source text and why it was held | 3 | 9 | 2 | 6 |
| Held lines | Never guesses a missing quantity or price | 3 | A blank stays blank and is held, shown on a test order | 2 | 6 | 3 | 9 |
| Held lines | Catches the same order sent twice | 2 | Holds a repeated order number or repeated lines for review | 1 | 2 | 2 | 4 |
| Order system | Writes orders into your order system the way your team does today | 3 | Uses your system's own import or connector, shown on a test company | 3 | 9 | 2 | 6 |
| Order system | Keeps the original email and file next to each order | 2 | Every order links back to its source | 2 | 4 | 3 | 6 |
| Order system | Works without changing how customers send orders | 1 | No new portal or template asked of customers | 3 | 3 | 3 | 3 |
| Ownership | You can export your rules, mappings and history at any time | 2 | An export shown in a standard file format | 1 | 2 | 3 | 6 |
| Ownership | Your team can change a mapping or rule without the supplier | 2 | Someone on your side made a change during the test | 2 | 4 | 2 | 4 |
| Ownership | Support, change and exit terms are written down | 1 | A named contact, response times and an exit plan in the proposal | 2 | 2 | 1 | 1 |
| Total | All twelve questions | 27 | Best possible is 81 |  | 58 |  | 66 |

[Download this sheet as a file for Excel](data:text/csv;charset=utf-8,%EF%BB%BFArea%2CQuestion%2CWeight%2CWhat%20earns%20a%203%2CSupplier%20A%20score%2CSupplier%20A%20points%2CSupplier%20B%20score%2CSupplier%20B%20points%0D%0AReading%2C%22Reads%20every%20line%20of%20your%20test%20orders%2C%20tables%20split%20across%20pages%20included%22%2C3%2CA%20line%20by%20line%20result%20on%20your%20test%20pack%20with%20every%20line%20accounted%20for%2C3%2C9%2C2%2C6%0D%0AReading%2CMatches%20customer%20item%20codes%20to%20your%20own%20codes%20from%20your%20mapping%2C3%2CShows%20the%20mapping%20used%20for%20each%20line%20and%20flags%20codes%20it%20has%20not%20seen%2C2%2C6%2C3%2C9%0D%0AReading%2C%22Handles%20units%20and%20pack%20sizes%2C%20a%20box%20of%20ten%20against%20single%20items%22%2C2%2CConverts%20only%20with%20a%20rule%20you%20gave%20it%20and%20holds%20the%20line%20otherwise%2C1%2C2%2C3%2C6%0D%0AHeld%20lines%2CSends%20unclear%20lines%20to%20a%20named%20person%20with%20the%20reason%2C3%2CEvery%20held%20line%20shows%20the%20source%20text%20and%20why%20it%20was%20held%2C3%2C9%2C2%2C6%0D%0AHeld%20lines%2CNever%20guesses%20a%20missing%20quantity%20or%20price%2C3%2C%22A%20blank%20stays%20blank%20and%20is%20held%2C%20shown%20on%20a%20test%20order%22%2C2%2C6%2C3%2C9%0D%0AHeld%20lines%2CCatches%20the%20same%20order%20sent%20twice%2C2%2CHolds%20a%20repeated%20order%20number%20or%20repeated%20lines%20for%20review%2C1%2C2%2C2%2C4%0D%0AOrder%20system%2CWrites%20orders%20into%20your%20order%20system%20the%20way%20your%20team%20does%20today%2C3%2C%22Uses%20your%20system%27s%20own%20import%20or%20connector%2C%20shown%20on%20a%20test%20company%22%2C3%2C9%2C2%2C6%0D%0AOrder%20system%2CKeeps%20the%20original%20email%20and%20file%20next%20to%20each%20order%2C2%2CEvery%20order%20links%20back%20to%20its%20source%2C2%2C4%2C3%2C6%0D%0AOrder%20system%2CWorks%20without%20changing%20how%20customers%20send%20orders%2C1%2CNo%20new%20portal%20or%20template%20asked%20of%20customers%2C3%2C3%2C3%2C3%0D%0AOwnership%2C%22You%20can%20export%20your%20rules%2C%20mappings%20and%20history%20at%20any%20time%22%2C2%2CAn%20export%20shown%20in%20a%20standard%20file%20format%2C1%2C2%2C3%2C6%0D%0AOwnership%2CYour%20team%20can%20change%20a%20mapping%20or%20rule%20without%20the%20supplier%2C2%2CSomeone%20on%20your%20side%20made%20a%20change%20during%20the%20test%2C2%2C4%2C2%2C4%0D%0AOwnership%2C%22Support%2C%20change%20and%20exit%20terms%20are%20written%20down%22%2C1%2C%22A%20named%20contact%2C%20response%20times%20and%20an%20exit%20plan%20in%20the%20proposal%22%2C2%2C2%2C1%2C1%0D%0ATotal%2CAll%20twelve%20questions%2C27%2CBest%20possible%20is%2081%2C%2C58%2C%2C66%0D%0A)

## The must haves, checked before scoring

**Made up example.** Every result here is made up to show the method. A single fail removes a proposal.

| Must have | How to check it | Supplier A | Supplier B |
|---|---|---|---|
| Nothing is posted without a person approving it during the pilot | Ask to see the approval step on a test order | Pass | Pass |
| Unclear lines are held, never guessed | Send a test order with a blank quantity | Pass | Pass |
| Every original email and file is kept | Ask where the originals are stored and for how long | Pass | Pass |
| You can take your data with you | Ask for an export of the test week | Pass | Pass |

[Download this sheet as a file for Excel](data:text/csv;charset=utf-8,%EF%BB%BFMust%20have%2CHow%20to%20check%20it%2CSupplier%20A%2CSupplier%20B%0D%0ANothing%20is%20posted%20without%20a%20person%20approving%20it%20during%20the%20pilot%2CAsk%20to%20see%20the%20approval%20step%20on%20a%20test%20order%2CPass%2CPass%0D%0A%22Unclear%20lines%20are%20held%2C%20never%20guessed%22%2CSend%20a%20test%20order%20with%20a%20blank%20quantity%2CPass%2CPass%0D%0AEvery%20original%20email%20and%20file%20is%20kept%2CAsk%20where%20the%20originals%20are%20stored%20and%20for%20how%20long%2CPass%2CPass%0D%0AYou%20can%20take%20your%20data%20with%20you%2CAsk%20for%20an%20export%20of%20the%20test%20week%2CPass%2CPass%0D%0A)

## Reading the made up result

Supplier B scores 66 of a possible 81 and Supplier A scores 58. The gap is 8 points, less than one heavy question, so the sheet calls them close. Supplier A reads lines well and holds unclear ones clearly, but loses points on units, repeat orders and taking your data with you. Supplier B does better on the hard cases. The next test should send both a batch of orders with mixed units and a repeated order, because that is where they differ most.

## What can change the answer

The weights belong to your order desk. If most orders arrive as spreadsheets, give reading spreadsheets more weight. If customers often order in boxes and you sell single items, give units the top weight. If a wrong order is hard to reverse, raise the weight of the held lines questions. Set the weights before you see any scores, never after.

## When this is the wrong tool

This sheet compares how proposals handle your orders. It does not compare prices, contract length or the supplier's finances, so judge those separately. With only a handful of emailed orders a week, a full comparison costs more time than it saves, so start with the five questions tool linked below. If most customers already send orders through a portal or a direct electronic link to your system, compare those routes first. No sheet replaces a pilot on live orders with a person approving each one.

## What to do with the result

Shortlist the one or two proposals that pass every must have and lead on the heavy questions. Run a pilot with a few customers, a fixed test pack, a written rule for what counts as a pass and a person approving every order. Keep the sheet and the evidence together, so the decision can be explained later and checked again at renewal.[

Client result . Distribution

10 → 1

About 90 percent less manual work, over 95 percent line accuracy

People needed for order entry at a wholesale distributor, client-confirmed.

Read how the system works](https://shurco.ai/solutions/sales-order-entry-automation/)

## Common questions

### Should accuracy carry the most weight?

Not on its own. A system that reads most lines but posts its guesses causes more work than one that reads fewer and holds every doubt for a person. Weight reading and holding together.

### What counts as evidence for a score of 3?

A result on your own test orders, written down line by line, that you can check against the source. A demo on the supplier's own orders earns a 2 at most.

### Can a ready made product and a custom build share one sheet?

Yes. The questions are about what happens to your orders, not the technology behind them. Read the ownership answers carefully, because changing rules and leaving work very differently for the two.

### How many suppliers should I score?

Two or three. The evidence step takes real time, and needing more than three usually means the must haves were not strict enough.

## Useful next[GuideBest sales order automation software in 2026Distributors, wholesalers and manufacturers still key emailed purchase orders into their ERP by hand. Here is what each tool actually automates, what it costs, and when it is the wrong buy.](https://shurco.ai/guides/best-sales-order-automation-software/)[WorksheetThe customer part number mapping workbookA customer's part number only helps if it points to the right catalogue item and selling unit. Use this workbook to find the gaps and give each one an owner.](https://shurco.ai/guides/customer-part-number-mapping-workbook/)[Free toolWhat to automate first, a scoring toolPick a task your team repeats every week and answer five quick questions. You get a plain answer on whether it is worth automating, what helps and what to watch. Score up to four tasks and it tells you where to start, and you can keep the result as a private plan page.](https://shurco.ai/tools/what-to-automate-first/)

## Field notes on this[OperationsAutomating Order Entry Is a Data Problem Before It Is an AI Problem](https://shurco.ai/insights/automating-order-entry-is-a-data-problem-before-it-is-an-ai-problem/)[ReliabilityEdge Cases Are Where Production Agents Live or Die](https://shurco.ai/insights/edge-cases-are-where-production-agents-live-or-die/)

## More guides[WorksheetThe incident worksheet for an agent that keeps repeating an actionWhen an agent repeats the same action instead of stopping, you need three things before you fix anything. A trace of what it did, a clear statement of when it should have stopped, and a test case you can run again after the fix.](https://shurco.ai/guides/agent-loop-incident-worksheet/)[GuideAI agent pricing in 2026, what vendors actually chargeVendors now charge for agents by the conversation, the outcome, the action or the credit. Here are the published prices, what each unit really means, and how to compare them on your own workload.](https://shurco.ai/guides/ai-agent-pricing/)[ServiceA monthly test for your customer facing AI assistantOnce a month, I ask your website or phone assistant ordinary customer questions. You get what it got wrong, its exact words and what to change. The first test is free.](https://shurco.ai/guides/ai-assistant-monthly-test/)[PlaybookHow to automate daily site reports across many sitesThe report takes days because the data lives in many places and someone has to chase it. Here is how to gather it in parallel, catch the numbers that disagree and produce a report a manager can check.](https://shurco.ai/guides/automate-daily-site-reports/)[PlaybookHow to automate quoting for a manufacturer or distributorSlow quotes lose work. There are three quite different kinds of quoting automation, and most buying mistakes come from picking the wrong kind. Here is how to tell which one your quotes need.](https://shurco.ai/guides/automate-quoting-for-manufacturers/)[PlaybookHow to automate sales order entry in Business CentralCustomers email orders as PDFs, spreadsheets and free text, and someone keys each line into Business Central. Here are the three ways to take that work away, with what Microsoft's own agent does and where it stops.](https://shurco.ai/guides/automate-sales-order-entry-business-central/)

## Check your own customer orders in two minutes

Five questions about how the work reaches you and how it is checked. You get a plain answer on whether it is worth automating, with no sign up. [Start with question one.](https://shurco.ai/tools/what-to-automate-first/?k=orders) Or [tell me about the work](https://shurco.ai/#contact) and I will reply myself.

Web version https://shurco.ai/guides/order-automation-supplier-scoring-sheet/