Trainly's free audit found $2,400/mo in wasted GPT-4 calls

Trainly, a pre-revenue AI observability startup, wants to show you how much money your agents are burning. Their free 72-hour audit analyzes production traces to find cost waste, silent tool failures, retry loops, and behavioral anomalies. The company claims their first report found "$2,400/mo in wasted GPT-4 calls."

You add a single decorator (@observe) to your code. Trainly captures up to 10,000 traces, then auto-disables. The audit runs on heuristics, meaning no LLM calls on their end. It catches latency and cost outliers. It also groups error patterns by input shape and tracks week-over-week drift.

This free audit is the lightweight version of what Trainly plans to sell. Co-founder Kavin explained on Hacker News that they're pre-first-customer and using real traces to stress-test their detection algorithms. The paid product adds unsupervised semantic anomaly detection via HDBSCAN clustering, UMAP visualization, and LLM-generated cluster summaries. Trainly positions itself against Braintrust.

It's a smart go-to-market play. The first 50 audits require no credit card. Convince developers their AI agents have problems they didn't know about, then sell the fix.