ToolPassed (hands-on MCP & AI tool tests) case study: how an AI agent business made $0 (October 2026)

Share on X Share card (PNG)Get the badgeFeatured @ATown_Jonesy

TL;DR

ToolPassed is a directory of MCP servers, agent skills and AI tools that Scott Jones hands-on tests in Claude Code, Cursor, Claude Desktop and ChatGPT (Codex), with a dated pass/fail [1][4]. Grok bots built the site, pull new tools, run tests and talk to makers; Scott does major approvals [10][11]. Launch week drew 14.8K visitors; by ~2 weeks: ~32k uniques, 110 free submissions, a ~4-month free test queue, and $0 from $29/$99 priority tests [6][7]. October revenue for the project was still $0 [9].

Visitors from launch (to Nov 2)43k[12]

Uniques, first ~2 weeks~32k[7]

Launch-week visitors14.8K[6]

Free submissions (Oct 29)110[7]

MCP listed / tested (Nov 3)275 / 65[2]

Passed all 4 clients34[2]

Priority test$29[3]

Priority + 30-day retests$99[3]

October revenue$0[9]

AI analysis

Business model analysis

Written by AI on Nov 4, 2026 from the 12 sources at the bottom of this page. Lines with a [n] link are reported facts; lines tagged Inference are the model's own reasoning. No revenue figures are added or estimated here.

VerdictNot earning yet

Stopped
No revenue
Loss-making
Early
Earning

Tens of thousands of visitors and a long free test queue, but $0 paid priority and $0 October revenue [7][9].

Reported revenue · October 2026 (through Nov 1)$0Self-reported

How it makes money

List AI tools free, then charge makers to jump a hands-on test queue with a dated public verdict [3][4].

Who pays

Makers who want a test this week: $29 or $99 [3]
Makers who want ongoing Verified retests: $29–49/mo [3]
Sponsors for labelled slots (never change results) [2]

Pricing

Free listing: $0, marked Not tested yet [3]
Priority test: $29 this week [3]
Priority + retests: $99 for 30 days [3]
Verified: $49/mo; founding $29/mo first 10 [3]

Cost structure

Agents: ~95% Grok bots; plan not disclosed [11]
Human time: Major approvals and pricing calls [10]

Growth engine

Launch + SEO directory14.8K week one; 43k from launch [6][12]
Maker submissions110 free by Oct 29 [7]
Build in publicDaily numbers including $0 [8][9]

Moat

Dated, multi-client hands-on results [4]
Inference The queue proves demand for tests; converting it to paid is the unfinished job.

Risks

Free users won't pay to skip the line yet [7]
Inference Official registries could out-trust an indie list.
Wrong pass labels would destroy the brand [4]

Why it hasn't worked (yet)

01Traffic and submissions are real [6][7]
02Paid conversion is the open problem he posted about [7]

What does ToolPassed (hands-on MCP & AI tool tests) sell, and to whom?

Free listings plus paid priority tests ($29 / $99) and Verified plans; dated pass/fail in real AI clients.

Builders choosing an MCP server or AI tool who want a dated, client-by-client result instead of a README claim [1][4]. Makers who want a free listing or a paid fast-track test within 7 days [3].

How was ToolPassed (hands-on MCP & AI tool tests) built?

Scott says Grok bots built ToolPassed, grab new items for the site and test queue, run tests and communicate with tool owners; he handles major approvals [10]. A bot named Scoop runs the newsletter [10]. As of Nov 3 the about page counted 275 MCP servers listed (65 tested hands-on, 34 passed every step in all 4 clients), plus skills, AI tools, agents and use-case listings [2]. Tests install the tool in real clients, run a scripted task and publish passed / partial / failed with versions and dates [4].

Grok BotClaude CodeCursorClaude DesktopCodexStripe

What is the business model?

Directory with paid testing. Free same-day listing marked 'Not tested yet' [3]. $29 priority test this week (one client, written report); $99 adds 30 days of retests; Verified badge plan $49/mo (founding $29/mo for the first 10) [3]. Payment never buys a better verdict; paid tests refunded if not done in 7 days [2][3]. Sponsors are labelled and don't change results [2].

Where does the traffic come from?

Launch-day traffic (5K+ of 14.8K in week one) [6], daily build-in-public posts [5][8][9], maker submissions, SEO/'what's new' pages [1], and the Scoop newsletter [10].

What revenue has ToolPassed (hands-on MCP & AI tool tests) reported?

Reported revenue: $0 (October 2026), per @ATown_Jonesy Day 48: October revenue $0 (2026-11-01).

2026-10-19ToolPassed launches [5]
2026-10-2018 tools submitted (8 MCP), revenue $0 [5]
2026-10-2114.8K visitors in 7 days [6]
2026-10-23Confirms Grok bots built/run the site; Scoop newsletter [10]
2026-10-29~32k uniques, 110 free submissions, $0 paid priority [7]
2026-11-01October revenue $0 [9]
2026-11-0243k visitors from launch; ~95% run by Grok bots [11]

What does the agent do, and what does the human do?

Agent

Built the site and keeps adding tools to the queue [10]
Runs tests and messages tool owners [10]
Scoop bot drafts/sends the newsletter [10]

Human

Scott Jones does major approvals [10]
Sets pricing experiments and asks X what to do about the free-queue problem [7]
Owns other products (WRONG!, iOS apps) alongside ToolPassed [9]

What does it cost to run?

Not disclosed. He says the site is about 95% run by Grok bots [11]; plan tier isn't stated.

What worked and what didn't?

Worked

Strong top-of-funnel: 14.8K week one, 43k from launch [6][12]
Makers submit faster than free testing can clear (110 in queue) [7]
Clear independence rules: pay never buys a better grade [2][3]

Didn't

$0 from paid priority despite a multi-month free queue [7][8]
October revenue $0 across shipped products [9]
Free demand without paid conversion is the open problem [7]

AI guide

How to copy ToolPassed fast

A starter path written by AI from this case. Budgets and timings are our estimates, not the operator's numbers.

DifficultyMedium

BudgetEst.~$30/month plus client seats

TimeEst.A week to a tested MVP, then daily test ops

Grok Bot templateNone known

Inference One Grok Bot plan (standard ~$30 assumption) plus whatever you pay for Claude/Cursor seats to run real installs. No close public Grok Bot template for an MCP test lab; treat this as a custom ops workflow, not a one-click template.

Tools & agents

Grok BotSite updates, queue, maker mail, newsletter [10]
Claude Code / Cursor / CodexWhere each tool is actually installed [4]
StripePriority test checkout [3]

Build path5 steps

1
Pick the clients that matter
Claude Code, Cursor, Desktop, Codex [1].
2
Define a scripted pass/fail
Install, connect, agent task, date it [4].
3
List free, test slow
Same-day Not tested yet rows [3].
4
Sell the skip
$29/$99 with a 7-day refund promise [3].
5
Post the awkward funnel
Queue length vs $0 paid [7].

Week 17 days

D1
Write the test protocol for one client
D2
Ship a directory with 20 hand-tested rows
D3
Add free submit + Stripe for priority
D4
Have bots scrape new MCP repos into the queue
D5
Launch post with visitor and submission counts
D6
Publish one failed test in public
D7
Ask makers what would make them pay to skip

Pitfalls

A long free queue without paid conversion [7]
Never let payment change the grade [2]
Unpinned dependencies can flip a pass overnight — record versions [4]

Key moves from the original case

Sell dated, hands-on proof — not another unverified directory row
List free same day; charge only to skip the test queue
Publish the awkward metric (queue length vs $0 paid)
Let bots operate; keep verdicts and refunds human-owned

What are the risks and the moat?

Free testers may never pay to skip the line [7]. MCP registries and official lists compete. A wrong 'passed' label would hurt trust; the site leans on dated retests [4].

FAQ

Who runs ToolPassed?

Scott Jones (@ATown_Jonesy), with Grok bots doing most of the operating work [2][10].

What does a test cost?

Free listing; $29 or $99 to jump the test queue; Verified from $29–49/mo [3].

Has it made money?

Not by Nov 1: October revenue $0; $0 paid priority as of Oct 29 [7][9].

Sources

Featured on AgentBuiltIs this your business? Get your badge.

Add the badge to your site or README. It links back to this case, and updates itself.

Get your badge Share on X

HTML

<a href="https://agent-built.vercel.app/biz/toolpassed?ref=badge" target="_blank" rel="noopener"><img src="https://agent-built.vercel.app/badge/toolpassed.svg" alt="ToolPassed (hands-on MCP & AI tool tests) is featured on AgentBuilt" height="54" /></a>

Markdown

[![Featured on AgentBuilt](https://agent-built.vercel.app/badge/toolpassed.svg)](https://agent-built.vercel.app/biz/toolpassed?ref=badge)

#grok-bot#mcp#directory#testing#build-in-public#pre-revenue

Related cases

Developer toolsShipLatch (a security scanner run by an AI CEO on Grok Bot)$0 (141 scans, 0 paid, first week)

Developer toolsGetPaid Skills (an agent-skill store built by a Grok Bot team)$0 at launch (Nov 2, 2026)

Developer toolsCrowdSpark Agent Tools (41 x402 tools built with Grok Bot)$0.14 from outside wallets (first 5 days)

Directory / MediaAgentTask Atlas (a Grok Bot agent running an MCP directory to pay its own $30 subscription)$0 since Oct 19, 2026; 276 visits, 12 clicks on a $19 pack; checkout not live yet

Numbers come from the operators or the press and haven't been audited. Anything missing is marked "not disclosed". Methodology · Report a correction.