How We Test
The evaluation framework StackPipeline requires for new and revised software scores.
- Effective:
- Last updated:
This page is the evaluation framework for new and revised scored content. It is not proof that every historic page has completed every step. A score should only claim hands-on testing when the page can document the actual scope, environment and date.
The three rules
- Document the plan and environment. A review should state whether it used a paid plan, trial, demo or another access route, and identify material limits.
- Separate evaluation from monetization. Potential affiliate compensation must not decide a score, rank or editorial conclusion.
- Compare like with like. A comparison should use the same documented workflow and volume assumptions for every product being compared.
The scoring rubric
Where StackPipeline publishes a score, this is the intended weighting. A page should explain any meaningful deviation from the rubric rather than imply a false precision.
| Criterion | Weight | What we measure |
|---|---|---|
| Core capability | 30% | Does the product perform its primary job under a documented workflow and test environment? |
| Ease of use | 20% | How long does a documented user take to reach a working workflow from a new account? |
| Integrations | 20% | How capable and reliable are the connectors relevant to the category and stated use case? |
| Value for money | 20% | What is the realistic cost at the stated volume, including required add-ons and usage limits? |
| Support | 10% | What can be observed from documented support interactions, response times and resolution quality? |
Workflow testing
For automation products, a credible test should publish enough detail for readers to understand the workflow: trigger, actions, source and destination systems, expected volume, failure handling, and any behavior that could change the result. Useful scenarios include lead routing, CRM enrichment, notifications, scheduled synchronization and a webhook flow with a deliberately failing downstream request.
Data enrichment testing
Match-rate or accuracy claims should identify the benchmark size, relevant geography or segment mix, sampling method, date and manual-validation method. We will not present a vendor-reported match rate as an independently observed result without saying so.
Minimum evidence for future scored content
- Full review — state the access level, testing period, hours or sessions, and limitations.
- Comparison entry — apply the same named workflow and assumptions to every contender.
- Integration tutorial — describe the integration steps, assumptions and known failure modes.
Updates and limitations
Scores and price-sensitive claims should be rechecked after a material vendor change or a documented error. An update date should describe a real review event, not a cosmetic rewrite. No evaluation can cover every tier, region, volume or long-term reliability outcome; those limits should be named on the relevant page.
Questions about a score
A verified public corrections channel has not yet been configured. Before scored content or advertising expands, StackPipeline will publish a monitored way to challenge an error and a process for recording substantive corrections.