Skip to main content

How We Test

The evaluation framework StackPipeline requires for new and revised software scores.

Effective:
Last updated:

This page is the evaluation framework for new and revised scored content. It is not proof that every historic page has completed every step. A score should only claim hands-on testing when the page can document the actual scope, environment and date.

The three rules

  1. Document the plan and environment. A review should state whether it used a paid plan, trial, demo or another access route, and identify material limits.
  2. Separate evaluation from monetization. Potential affiliate compensation must not decide a score, rank or editorial conclusion.
  3. Compare like with like. A comparison should use the same documented workflow and volume assumptions for every product being compared.

The scoring rubric

Where StackPipeline publishes a score, this is the intended weighting. A page should explain any meaningful deviation from the rubric rather than imply a false precision.

StackPipeline scoring rubric and criterion weights
Criterion Weight What we measure
Core capability 30% Does the product perform its primary job under a documented workflow and test environment?
Ease of use 20% How long does a documented user take to reach a working workflow from a new account?
Integrations 20% How capable and reliable are the connectors relevant to the category and stated use case?
Value for money 20% What is the realistic cost at the stated volume, including required add-ons and usage limits?
Support 10% What can be observed from documented support interactions, response times and resolution quality?

Workflow testing

For automation products, a credible test should publish enough detail for readers to understand the workflow: trigger, actions, source and destination systems, expected volume, failure handling, and any behavior that could change the result. Useful scenarios include lead routing, CRM enrichment, notifications, scheduled synchronization and a webhook flow with a deliberately failing downstream request.

Data enrichment testing

Match-rate or accuracy claims should identify the benchmark size, relevant geography or segment mix, sampling method, date and manual-validation method. We will not present a vendor-reported match rate as an independently observed result without saying so.

Minimum evidence for future scored content

  • Full review — state the access level, testing period, hours or sessions, and limitations.
  • Comparison entry — apply the same named workflow and assumptions to every contender.
  • Integration tutorial — describe the integration steps, assumptions and known failure modes.

Updates and limitations

Scores and price-sensitive claims should be rechecked after a material vendor change or a documented error. An update date should describe a real review event, not a cosmetic rewrite. No evaluation can cover every tier, region, volume or long-term reliability outcome; those limits should be named on the relevant page.

Questions about a score

A verified public corrections channel has not yet been configured. Before scored content or advertising expands, StackPipeline will publish a monitored way to challenge an error and a process for recording substantive corrections.