May 20, 2026 · 8 min read
How we test AI tools: the 40-hour rule
Every AI tool we review has been used for at least 40 hours on real projects. No toy demos, no 5-minute trials. Here's exactly how we test, score, and disclose.
Why 40 hours?
Most "AI tool reviews" are written in 30 minutes after a single session. That misses the things that actually matter: how it handles week-long projects, edge cases, breaking points, billing surprises, support response times.
Our minimum standard: 40 hours across 4+ weeks, on real client or product work. We use the tool the way you'd use it.
Our 7-step testing process
- Pay for it ourselves (when possible) — use the same billing experience you would
- Use it on a real project — not "write me a haiku"
- Document time saved — actual hours, not vibes
- Test the support — open a real ticket, time the response
- Test failure modes — what happens when it hallucinates, when the API is down, when billing fails
- Compare to alternatives — same task, 2-3 competitors
- Re-test in 3 months — does it still hold up?
Our scoring rubric (public, applied consistently)
| Category | Weight |
|---|---|
| Core capability | 30% |
| Ease of use | 15% |
| Reliability | 15% |
| Pricing fairness | 10% |
| Support quality | 10% |
| Ecosystem / integrations | 10% |
| Privacy / data handling | 10% |
What "no paid placement" actually means
Vendors cannot:
- Buy a higher score
- Edit or remove a review
- See a review before publication
- Receive preferential placement (we randomize tool order in lists)
What vendors can do: submit corrections (we'll publish them with our response), sponsor newsletter slots (clearly marked), and provide press access for new launches.
The affiliate question
Yes, we earn affiliate commissions on some links. This is fully disclosed on every relevant page. Our testing is identical for affiliate and non-affiliate products — the only difference is whether clicking the link costs you anything (it doesn't).
Want to suggest a tool?
Email hello@theaireview.pages.dev. We can't review everything, but we prioritize based on reader requests and category gaps.