What we got wrong about AI-written cold email this year
We shipped four versions of an AI email generator. Two failed. Here is what failed and why, and what we will do differently in 2026.
We have shipped four distinct versions of the Blacksmith AI email generator since the company started. Two of them were genuinely useful. Two of them were quiet failures that nobody outside the team noticed because we caught them before they hit broad release. The two failures taught us more than the two wins, and since end of year is the season for honest retrospectives, here they are.
Failure one: the elaborate persona builder
Our v2 generator asked the user to define a sender persona in fifteen fields. Tone, voice, signature, lexical preferences, the kind of jokes that were okay, the kind that were not. We thought richer personas would produce more distinctive emails. They did not. They produced emails that all sounded vaguely like the same person trying too hard. The output got worse the more rules we added.
The signal we missed was simple. The data the LLM needed to write a good email was not about the sender. It was about the recipient. The trigger, the company context, the specific reason this email exists. The persona stuff was vanity.
Failure two: the variant explosion
Our v3 generator produced six variants per send and let the user pick. We thought this gave operators agency. What actually happened was reps picked the same variant 78 percent of the time. The other five variants were waste. Worse, the rep stopped trusting any single output because they always wondered if the other five were better. Choice was hurting confidence.
More variants does not equal more confidence. It usually means less. People want one good answer they can edit, not five mediocre answers they have to compare.
What worked
- Generating from the signal, not the persona. The trigger is the spine of the email
- Returning one email, with a visible breakdown of why each paragraph exists
- Letting the operator edit inline with the AI assist, instead of regenerating the whole body
- Refusing to send anything where the first sentence does not reference a specific fact about the account
What we will do differently in 2026
We will narrow further. We will stop generating any email that cannot cite at least one specific event in the last 120 days. We will surface the signal as a chip on the message, so the rep can see what the AI thinks the email is about. We will quietly retire the feature that lets you ask for a 'shorter version'. People always pick the shorter version, which means we should just be writing shorter versions to begin with.
“Generative tools fail when they try to be a writer. They work when they act like a researcher who happens to draft well.”
If this resonated, it'll feel familiar in the product.
Try Blacksmith against your real territory for 14 days. No card, no metered AI credits, no surprises.
Summit seats are limited and allocated to qualifying founding members. Perks subject to final terms.