Quick answer
Build your ERP requirements matrix around business outcomes, not feature names. Give each requirement an owner, an observable acceptance test, a priority and an evidence field. Evaluate mandatory requirements as pass/fail gates, then compare eligible solutions using weighted scores. Give every vendor the same demonstration scenarios and record what was actually shown.
Start with: a shared spreadsheet, representatives from the affected business functions, and synthetic examples of your important transactions.
The method below is a suggested selection framework for small and midsize organizations. Its scoring rules are illustrative—not an industry standard—and should be approved before vendors demonstrate their software.
1. Interview process owners before listing features
SAP’s ERP evaluation guidance recommends gathering requirements across stakeholders and tailoring checklists to the business rather than relying on generic feature lists. It also distinguishes business-wide priorities from individual departments’ preferences. (sap.com)
Interview the people doing the work as well as those managing it. For order-to-cash, include customer service, warehouse operations and finance. Ask them to walk through a normal transaction and an exception.
Use these questions:
- What starts the process, and what marks it complete?
- Where do people re-enter information or reconcile spreadsheets?
- Which exceptions require a supervisor?
- What information must remain connected across departments?
- What happens when a step fails?
- What evidence would prove that a replacement process works?
Record the problem separately from the proposed solution.
For example, “We need a returns module” is a feature request. “Finance cannot reliably trace a customer credit to the returned goods and original invoice” identifies the business problem.
Do not require a new system to reproduce every existing screen or workaround. Microsoft’s fit-to-standard guidance recommends examining standard processes before deciding where extensions are justified, including their development and maintenance implications. (learn.microsoft.com)
Practical check: Each requirement should have a business owner who can explain the consequence of not meeting it.
2. Turn each problem into a testable matrix row
Use one row per independently assessable outcome. Assign a stable identifier so the requirement can follow the project from selection through acceptance testing.
A useful requirement follows this pattern:
Given a business condition, when a named role performs an action, then a specific result must be observable.
For example:
Given an order for 10 units and only 6 available, when an authorized warehouse user posts the shipment, the system must record 6 shipped and leave 4 outstanding without creating a replacement order.
Build your spreadsheet with these fields:
| Field | What to record |
|---|---|
| ID and process | Stable reference, such as O2C-01; order-to-cash |
| Problem and outcome | Why the requirement exists and what must improve |
| Owner | Person authorized to accept the result |
| Scope | Relevant entities, locations, roles and transaction types |
| Acceptance test | Preconditions, actions and expected result |
| Priority | Mandatory, desirable or deferred |
| Weight | Relative importance within the scored comparison |
| Delivery approach | Standard, configured, add-on, custom or proposed |
| Evidence | Demo reference, document, screenshot or follow-up test |
| Assessment | Gate status, score, gap and next action |
Keep delivery approach separate from functional fit. An add-on might meet the outcome, but its licensing, implementation and support arrangements still need review.
Include nonfunctional requirements too. Microsoft’s testing guidance covers performance, usability, operability, maintainability and business continuity alongside functional processes. (learn.microsoft.com)
For these rows, define your own test conditions. “Fast reporting” is not testable until you specify the report, representative data, operating conditions and acceptable response.
3. Separate mandatory gates from weighted preferences
Use two decision layers.
Mandatory gates determine whether a solution remains eligible. Examples might include a required access restriction, a critical integration or an essential transaction outcome. The business must justify each gate.
Desirable requirements distinguish acceptable solutions. These might include clearer exception handling, fewer manual handoffs or more useful operational views.
Mark deferred requirements explicitly so vendors do not assume they belong in the initial scope.
For mandatory gates, use:
- Pass: agreed evidence confirms the requirement.
- Fail: evidence shows the requirement is not met.
- Unverified: evidence is incomplete; assign an owner and deadline.
Do not treat “unverified” as a pass. If the steering team permits a conditional shortlist, document what must be demonstrated before selection.
For scored requirements, allocate weights totaling 100. First allocate importance across process areas, then divide each area’s allocation among its requirements. This prevents a department with many spreadsheet rows from receiving disproportionate influence.
Avoid double-counting: If “linked return records,” “credit traceability” and “invoice-to-credit navigation” describe the same outcome, merge them or divide one fixed weight between genuinely distinct tests. Do not score both a parent category and its child rows at full weight.
Keep gate decisions outside the weighted total. Attractive extras should not compensate for a failed mandatory requirement.
4. Give every vendor the same scenario-based demo script
SAP recommends providing shortlisted vendors with the same script in advance and including business exceptions, rather than allowing the session to consist only of the vendor’s preferred presentation. (sap.com)
For each scenario, provide:
- Requirement IDs and business objective.
- Synthetic customers, products, quantities and document references.
- Starting data and configuration assumptions.
- Roles that must perform each action.
- Required steps and expected outcomes.
- Evidence reviewers must capture.
- Exceptions or prohibited actions to test.
Microsoft’s test-planning guidance similarly identifies prerequisites, execution steps and expected outcomes as core test-case elements, and recommends both positive and negative scenarios. (learn.microsoft.com)
Ask vendors to disclose the demonstrated product, edition, release, regional setup and any add-ons. Record which components are included in the proposed scope and which remain assumptions.
During the session, have reviewers request specific records and outputs: “Show the remaining order quantity” is more useful than “Does the system handle backorders?”
Use a controlled demonstration environment. Do not post test transactions to production or distribute real customer information merely to make the demonstration look realistic.
5. Score demonstrated fit and capture the evidence
Approve a common scale before demonstrations. The following is one workable model:
| Score | Meaning |
|---|---|
| 0 | Demonstrated failure to meet the requirement |
| 1 | Major gaps; proposed approach is not acceptable as shown |
| 2 | Partly meets it; material workaround remains |
| 3 | Meets the agreed outcome through an acceptable demonstrated process |
| 4 | Meets it and demonstrates a predefined additional benefit |
Define what earns a 4 for each applicable row before seeing vendors. Otherwise, an impressive but irrelevant feature can inflate the score.
Use U for unverified requirements rather than silently converting missing evidence into a numerical result. Do not publish a final comparative total until scored rows have comparable evidence. If a requirement is removed, revise the scope and weights for every vendor.
Calculate:
Weighted points = requirement weight × score ÷ 4
With weights totaling 100, the maximum total is 100 points.
For every score, retain an evidence note. A suggested format is:
Requirement ID | demo date | environment/version | role | action observed | result | evidence reference | gap | follow-up owner
Have reviewers score independently before discussing differences. Require the agreed score to reference evidence, not presenter confidence.
Record implementation effort, recurring cost and support dependencies separately. If these receive scores in the wider selection model, ensure the same disadvantage is not penalized repeatedly without a deliberate reason.
6. Worked example: partial shipment, return and credit note
Hypothetical example only: A distributor is comparing two unnamed ERP proposals. The transactions, weights and scores below are synthetic, not product-test results.
The company supplies this scenario:
- Create an order for 10 units at $100 each.
- Ship and invoice 6 units, leaving 4 outstanding.
- Receive 2 of the shipped units as a customer return.
- Create a $200 credit note linked to the relevant original transaction.
- Show the remaining order quantity, returned quantity and customer balance.
Assume no tax, freight, discounts, restocking fees or currency differences. The invoice is $600; after the $200 credit, the unpaid balance is $400. The remaining 4 units have not been shipped or invoiced. Finance must define any additional posting expectations for a real evaluation.
The distributor sets these mandatory gates:
- Partial shipment preserves the 4 outstanding units.
- The return and credit remain traceable to the original transaction.
- The designated restricted role cannot post the credit.
Assume both proposals pass those gates with evidence.
The following table scores only the desirable differentiators within this scenario:
| Criterion | Weight | Proposal A | A points | Proposal B | B points |
|---|---|---|---|---|---|
| Exception visibility beyond basic quantity accuracy | 30 | 4 | 30 | 3 | 22.5 |
| Return-handling usability beyond required traceability | 25 | 3 | 18.75 | 4 | 25 |
| Credit-review usability beyond the access gate | 25 | 3 | 18.75 | 3 | 18.75 |
| End-to-end operational view | 20 | 4 | 20 | 2 | 10 |
| Total | 100 | 87.5 | 76.25 |
For example, Proposal A’s return-handling points are 25 × 3 ÷ 4 = 18.75.
The higher scores must correspond to predefined observations. A 4 for exception visibility might require an agreed view of outstanding quantities and a demonstrated route to resolve them—not merely a colorful dashboard.
These are scenario scores, not overall ERP rankings. Give the scenario a fixed allocation in the wider evaluation; do not let this table accidentally outweigh procurement, finance or other critical processes.
7. Resolve gaps before turning the score into a decision
Before making a recommendation, run these checks:
- Roadmap promises: Mark undemonstrated future functionality as unverified.
- Hidden dependencies: Identify required add-ons, interfaces, configuration and specialist work.
- Missing negative tests: Check what an unauthorized user cannot do.
- Unequal treatment: Give every vendor the same clarification and retest opportunity.
- Changed priorities: Version any weight changes and rescore all proposals consistently.
- Overlapping criteria: Confirm that each weighted row represents a distinct benefit.
- Incomplete scope: Review technical, operational and commercial matters outside the demonstration.
Prepare a decision pack containing the matrix, gate outcomes, evidence, unresolved gaps, delivery assumptions and cost comparison. Ask process owners to approve their assessments and the steering team to document its trade-offs.
Finally, carry the requirement IDs and scenario scripts into implementation testing. Microsoft’s guidance recommends connecting test cases to business processes and obtaining business-user sign-off on readiness. A selection demonstration should inform that work, not replace it. (learn.microsoft.com)
The matrix is ready to support a decision when the team can explain what matters, what was demonstrated, what remains uncertain and who accepts the remaining risk.