Monitoring Email Deliverability through Seeding


Monitoring an email stream can be difficult without the use of seeded messages. These messages are a small sample of emails that are sent to email addresses that you have access to check whether the email landed in the inbox or spam folder. At larger sample sets the results from these seed accounts can begin to provide insight into which mailbox providers are placing your email in the junk / spam folder or inbox.

There are a couple factors that you’ll want to take into account when determining how to seed your mail streams, since how you integrate your seeds into your mail stream can significantly impact the results. The following sections describe the main items you should consider when integrating seeding into bulk or transactional mail.

Bulk or Campaign Messages
When seeding bulk or campaign based email, when and where you place the seeded messages in comparison to the remainder of your email may impact the results. This is because delivering larger mail volumes can take hours to deliver to mailbox providers and filters may change based on positive or negative customer engagement of that mail throughout the send. Consider the following items when deciding how to seed your email:

  • At the Beginning – Placing seeds at the beginning of your mail stream may allow changes in filtering during your sending time to be unmonitored. If you only send and measure seed results at the beginning of sending your bulk or campaign message you would miss this change in results.
  • At the End – By placing seeded messages at the end of your mail stream you will be sure to measure the end state of any changes of filtering throughout the time of your message send. This will provide a conservative estimate as to how your bulk or campaign message performed with your recipient base.
  • Throughout the send – Monitoring a mail stream throughout a send will provide additional insight as to how filters reacted through the entire time of the send. This will show how reputation at the beginning of the send was impacted by recipient engagement and email filter time lag. This method will provide the most insight into deliverability but will also consume the most resources from seeds and reporting.

Since email lists are normally randomly ordered, thinking about what order your list is in will be important as well. I’d suggest having each ISP spaced from the start of send to the end of send. This will allow your delivery rate at each ISP to remain constant. Additionally, in the past some have tried to implore tricks like front loading your most engaged contacts in the front of the list to try and ‘fool’ spam filters. This is a bit outdated and I’d suggest spreading all mail to get the best results.

Transactional Messages

Mail streams that are more transactional in nature are much more difficult to characterize as they are continuously changing based on customer engagement. Most transactional messages need a custom seed deployment method designed to adequately monitor the results for that stream. The following methods are typical methods to consider when monitoring your transactional mail stream.

  • Periodic flat deployment – A periodic flat deployment method is deploying seeds similar to a small bulk campaign at a set period, possibly once per day. This will take a single snapshot of your deliverability at that time of day. The same factors as described in the bulk or campaign section above can be used to determine if the beginning or end of deployment should be used in relation to the majority of your transactional email traffic. This method is not as accurate for transactional mail but is the easiest to implement.
  • Weighted by Volume – Weighting the amount of seeded messages you send as a ratio to the volume of transactional mail you send will begin to tailor your results more closely to your actual inbox placement. Make sure to keep the ISP distribution the same throughout the day, but begin to break down the seed list by time segments of the day, perhaps for every hour, and weight the volume of seeds sent to your typical transactional volume each hour.
  • Weighted by Mailbox Provider or ISP – Similar to weighting by volume, actually weighting the mailbox providers or ISPs on your seed list by the amount of mail you send to each receiver will begin to provide you with more accurate overall inbox success percentages. This is important since most decisions that may impact deliverability are done on the overall mail stream, but filters are unique at each mailbox provider. You will want the total inbox percentage weight to accurately reflect the deliverability you’re receiving overall.

As you can see, transactional mail streams begin to present a different challenge for seeded mail. Be sure to accurately reflect the content in these transactionally seeded messages to the content you wish to measure the results on. If various messages with differing contents are desired to be monitored, a separate seed stream should be used.

In order to gain results, you need to have someone who monitors and takes action based on seed results. Some examples of actions we’ve seen or done ourselves have been the following:

  • Check Headers or Content – By replacing out parts of the email message like content, subject line, headers, from address, return path, etc… you’ll begin to see if it’s a content item causing your mail to go to the spam folder, or whether it’s your reputation. If it’s reputation, go to number 2 below, but if it’s content keep iterative on the message content until you’re able to find the root cause and remove that from your email.
  • Change process at certain mailbox providers – If it’s not content, and your seeds are reporting you’re reaching inbox at the majority of mailbox providers you’re doing well and you can begin to optimize at each mailbox provider individually. For instance, if Gmail is the only provider showing you’re being sent to the spam folder at times, try being a bit more conservative at Gmail by removing unengaged users within a shorter amount of time from Gmail.

To provide any thoughts or comments, feel free to tweet @MikeVeilleux!