Call to action
A call to action (CTA) is the one step you ask the reader to take next, such as replying, picking a time or confirming a detail.
Read more3. Outreach
What is A/B testing?
A/B testing sends two versions of a message, differing in one element, to similar groups at the same time to see which version gets the better result. It replaces opinion with a comparison you can read.
The parts
- One changeVary only one thing, such as the subject line or the closing ask. If two things differ you cannot tell which one mattered.
- Equal, random groupsSplit the list so both groups look alike. Sending one version to your best prospects spoils the comparison.
- Same timeSend both versions together, so the day or hour is not the reason for a difference.
- A result you can judgeChoose the measure in advance, such as reply rate, and wait for enough messages before deciding.
Worked example: two subject lines at a bookkeeping firm
Alder and Finch, a fictional bookkeeping firm, tests two subject lines on 150 delivered messages each. The body of the email is identical.
By hand
- Version A: 6 replies from 150. Reply rate: 4%.
- Version B: 9 replies from 150. Reply rate: 6%.
- Difference: 2 percentage points in favor of B, which is 3 replies out of 150.
- With so few replies, the gap could be chance. The firm repeats the test on a fresh group before changing its template.
Reply rate by subject line (invented figures)
How to apply it
- Pick one element to test and write the question the test should answer.
- Split the list randomly into two equal groups.
- Send both versions at the same time and note the measure you will judge by.
- Repeat on a new group before you make the winner your standard.
Mistakes to avoid
- Changing the subject line and the body together, so the cause of a difference is unknown.
- Declaring a winner from a handful of replies.
- Testing endlessly instead of keeping the winner and moving to the next question.
Related terms
Reply rate
Reply rate is the portion of delivered emails answered by the recipient.
Read moreCold email
A cold email is a first message to a person you have not spoken with, sent because their role or company suggests you could help.
Read moreQuestions
- How many messages does an A/B test need?
- More is better. Small groups swing widely by chance, so treat a result from a few dozen replies as a hint and confirm it with a second round.
- What should I test first?
- Start with the element most likely to change the outcome, often the opening line or the ask.
- Can I test more than two versions?
- Yes, but each extra version needs its own group, so you need a longer list for the same confidence.
- Which measure should I use?
- Use the one closest to your goal. For outreach that is usually reply rate or meetings booked, not opens.
Tell us what you sell and who buys it
What you sell
Who buys it
The button opens a draft in your own mail app. Nothing is sent from this page.
hello@coldmeetings.comEmail us
- Software companies
- Agencies
- Service businesses