A/B testing cold emails without overcomplicating it
ok so if your reply rates are sitting below 2% and you keep tweaking subject lines hoping thatll fix it... it probably wont. the problem is almost never the copy first. let me explain what i mean.
im 4 months into cold email. came from teaching high school english which is about as far from outbound sales as you can get lol. a friend showed me his setup back in january and i was like wait you make money just sending emails? anyway fast forward to now and im pulling about $10k/mo doing lead gen for small agencies. still feels surreal honestly.
but heres the thing i want to share - my actual numbers from testing, because i think people overcomplicate A/B testing and end up learning nothing.
my numbers from the last 90 days across all campaigns:
total emails sent: around 14,200 average open rate: 67% average reply rate: 4.3% (it was 1.1% when i started) bounce rate: 1.8% now, was 6.4% in my first month cost per booked meeting: roughly $38 monthly tool spend: about $340 total inboxes: 8 on Maildoso ($64/mo) sending: Instantly ($97/mo on the growth plan) verification: Bouncer (like $30-40/mo depending on volume) CRM: Close CRM ($49/mo starter) the rest goes to enrichment and random stuff
ok so with that context. when i first started testing i was doing what every youtube video tells you to do - write version A and version B of your email, change the subject line, send both, see which wins. and for the first 6 weeks i was convinced that was how you improve reply rates. just keep writing better emails.
turns out the copy was maybe 20% of my problem. the other 80% was list quality and deliverability. took me almost two months of spinning my wheels to figure that out.
so heres how i actually A/B test now and its way simpler than what most people describe.
step one is i only test ONE variable at a time. i know everyone says this but almost nobody does it. for the first two weeks of a new campaign i dont touch the copy at all. i send the same email to two different list segments - one built from LinkedIn Sales Nav filtered tight (like specific job title + company size + geography), and one built from a broader scrape. same exact email. the tight list almost always wins by 2-3x on replies. every single time. so before you test a single word of copy, test your list.
step two is verification and this is where i was bleeding early on. my first month i was just sending to whatever emails i found and my bounce rate was over 6%. thats when Instantly started throttling my sends and i had campaigns just sitting there doing nothing for days. now i run every list through Prospeo for enrichment and then verify everything through Bouncer before it touches a sending tool. takes maybe 20 extra minutes per list but my bounce rate dropped to under 2% and hasnt gone above that since.
step three - and this is where actual copy testing starts - i only test after i have at least 200 sends per variant. not 50. not 100. 200 minimum. i know thats not statistically perfect or whatever but its enough that i can see a real pattern vs just noise. before i figured this out i was making decisions off like 40 sends per version and just chasing randomness.
for the actual copy tests i keep it really simple. i test three things in this order:
first the opening line. not the subject line. the opening line. i found that subject lines matter way less than i assumed early on because most people read the preview text anyway. so i test two different opening lines - one thats personalized to their company and one thats personalized to their role. the company-specific one wins for me about 60% of the time but honestly its close.
second i test the CTA. soft ask vs direct ask. "would it make sense to chat for 15 min" vs "im free tuesday at 2pm if you want to talk about this." the specific time slot thing works better for smaller companies. for bigger companies the soft ask does better. i have no idea why.
third i test length. and this surprised me - my best performing email right now is 4 sentences. total. my worst performing one was this beautiful 3-paragraph thing i spent an hour writing. i think when i was a teacher i had this instinct to explain everything thoroughly and it just doesnt translate to cold email at all.
the whole cycle for testing one campaign takes me about 3-4 weeks. week one is list testing. week two is opening line testing. week three is CTA testing. week four i take the winner and scale it up. if reply rates are above 3% i keep it running. below that i rebuild the list from scratch before touching the copy again.
one more thing - i track everything in a google sheet, not in the sending tool. Instantly has decent analytics but i wanted to see trends across campaigns over time so i just export the data weekly and dump it into a sheet. takes 10 minutes. i color code by campaign and by test variable. its ugly but it works and i can actually see what moved the needle vs what didnt.
im still very new to this and i get imposter syndrome every time i see people in here talking about sending 50k emails a month or running 40 inboxes. im at 8 inboxes and like 3,500 sends a month and thats my whole world right now. but the testing process above took me from 1.1% reply rate to 4.3% in about 10 weeks so i figured it was worth sharing even if im not operating at some massive scale.
also i should mention warmup - i do 14 days minimum on every new inbox before sending a single cold email. Maildoso handles the warmup automatically which is nice. i tried cutting it to 10 days once and my open rates tanked immediately so... dont do that.
anyway thats basically what ive been doing. probably missing stuff. still learning every week honestly