How SmartLead & Instantly’s Reply Rate Metrics Can Be Harmful

In the previous lesson, I mentioned that LeadTables displays Positive Reply Rate a bit differently than most cold email platforms do.

Other platforms, such as Instantly and SmartLead, define their reply rate metrics like this:

• 👎 Reply Rate (Other Platforms):
  • Any replies you get, including out-of-office replies, people saying “that person doesn’t work here anymore,” etc.
  • Math: replies / leads emailed
• 👎 Positive Reply Rate (Other Platforms):
  • The percentage of those above replies that are favorable and interested
  • Math: positive replies / replies

And here’s how LeadTables tracks it:

• 🚫 Reply Rate (LeadTables):
  • Not tracked.
• 👍 Positive Reply Rate (LeadTables):
  • The percentage of leads you email you respond positively
  • Math: positive replies / leads emailed 👈 Different!

Why The Difference?

The reason LeadTables cares primarily about “actual funnel-wide positive reply rate” vs. “phase-centric positive reply rate” is because I believe it’s the #1 holistic indicator of “email resonance.”

That is to say, if you send 1,000 emails and get 10 positive replies, and a different campaign got 20 out of 1,000, I believe that’s the best indicator of the performance / resonance of your cold emails. (Assuming no deliverability issues, same lead quality, etc.)

The reason the metrics from SmartLead and Instantly bother me are because of situations like the one below:

• Suppose you send a campaign in March, and you get see these metrics in your Instantly account:
  • 1,000 Leads
  • Instantly Reply Rate: 1.24%
  • Instantly Positive Reply Rate: 82.2%
• And then one on December 26th with these metrics:
  • 1,000 Leads
  • Instantly Reply Rate: 5.27%
  • Instantly Positive Reply Rate: 15.76%

Which campaign has the better actual net positive reply rate, a.k.a. more actual positive replies in your inbox?

If you said “The December 26th one” you’re wrong.

If you said something like, “I’d have to do the math to find out,” you’re right — and you’ve highlighted the problem.

You’d essentially have to do…

• March: 1.24% * 0.822 = 1.02% actual net positive reply rate
• December: 5.27% * 0.1576 = 0.83% actual net positive reply rate

Okay, So When Is Raw Reply Rate Useful?

The Instantly-style “raw reply rate” IS useful in some contexts, e.g. when diagnosing deliverability issues.

One of the hardest things with cold email is knowing if your emails are even making it to the inbox, or if they’re getting auto-filed into the “marketing” or “spam” folders.

And at the end of the day, if you’re getting “out of office” emails, it means your emails are at least hitting peoples inboxes (even if they’re just going to the “marketing” tab) — so seeing that reply rate can help you figure out whether you’re running into issues with soft bounces or if your emails just aren’t resonating.

But, with that said, LeadTables isn’t concerned with deliverability, so we keep the metrics in here focused on your holistic campaign performance.

Maybe I’ll change my mind and add the reply rate metric later; we’ll see 🙂 Reach out if you think it’s bad not to have it, e.g. if its absence might mask a campaign whose performance suffered due to deliverability issues vs. email quality/resonance.