Gmail Breaks Email Marketing?

Has Google moved the email marketing cheese yet again? No we are not talking Gmail Tabs. If you keep a close eye on the industry and have your finger on the pulse of email then you just may be under the impression that it has. If you are one to listen to the rumours then you may be of the belief your unable to track opens at Gmail anymore at all, or that you can track unique opens, but not repeat opens: that this problem is ubiquitous across the industry. There was discussion on an industry list today, that intimated there were big problems with open tracking, I was obviously concerned.

EDITOR UPDATE: The fix for Gmail Open Tracking is revealed here and also here

It may well be the case depending on what email platform you use, that your open tracking has been disrupted at Gmail, however it is NOT a given, and you should test and/or ask your provider yourself. I have spoken with ESP’s that say image tracking appears completely ‘broken’, others who say they can only record unique opens. In my own tests I found at least one large self service ESP that only recorded one open per recipient, unless that recipient changed device. As the Postmaster for Campaign Monitor, sending tens of thousands a message to Gmail every hour, I have not seen any issues with open tracking. However some of our more advanced features such as geo-location tracking have been affected. This is a very small subset of any senders recipient base (see earlier discussions about the impact of tabs for more background on this).

SmartFocus  & Campaign Monitor are both completely unaffected in regards to open tracking and the latest Gmail changes, further ExactTarget & Lyris report they have no issues. To see a video illustrating this check out my Youtube Upload.

Laura Atkins blogged early and provided some extremely valuable initial intelligence. I however needed to clarify what the situation was exactly for my clients who entrust me to have their mail delivered, their lists managed, and their activity tracked. There are a number of issues at play here, and by digging into the data available to me at Campaign Monitor I was able to crunch some numbers and draw some conclusions, I hope by sharing this it can prove useful to some of my colleagues and counterparts in the industry.

Why would Gmail change things?
I rather suspect user experience, they are betting they can do things faster this way. From my personal experience and the anecdotal evidence of others, Gmail has been much faster the last day or so.Anything that improves the customer experience of the email channel is generally a very good thing for the legitimate email marketer.

What has changed at Gmail?
Images are being served via a proxy for Gmail users. From what I can see from logs (and the numbers do not lie) this issue started to occur 3 days ago.

So what?
There are four issues at play:

1) Cache; Content caching which has been around for a while
2) Proxy; IP address masked caused by the requests being proxied
3) User Agent; Google appears to grab with their own User Agent
4)Hard Limits;  The proxy/cache does not seem to support files in excess of 10MB in size

Point 1 is one that many fixated on, however this has been going on for some time now, Gmail has been caching images client side for some time now. Reloading your browser window gets round this and the new change that took place recently does not impact this in any way at all from what I and others can see.

Point 2 Gmail is proxying requests for images and in doing so is Proxying their requests and thereby masking the original IP address. For reporting on geo-location, or serving up different images based on geo-location this IS a problem. For example Campaign Monitor This does not seem to affecting our ability to track opens, however it is affecting our Geo Location lookups with everything opened via the Gmail client seeming to come from Mountain View, CA.

Steve Atkins from Word to the Wise recorded the http headers from a request and it seems to indicate NO x-forward header (HTTP_FORWARDED or HTTP_X_FORWARDED) is being issued. If Gmail would add this, EDM platforms and providers could fix these issue of their own accord.

Point 3 Gmail appears to provide the original user-agent within the request, however code changes are likely to be required at various EDM, Tracking and Reporting Platforms to ensure this issue can be worked around. However it is not out of the control of EDM platforms to resolve this issue without any further change or assistance from Gmail. This is a problem still, see comments from Harald…

Point 4 Honestly I know some people are doing some real ‘smart’ things with email, and video is oh so popular, but – that was not what email was designed for. I see no issue with this.

When did it start?

From what we can see 3 days ago. The following stats for the geo-location of opens show a significant increase from Mountain View CA since the 1st of Nov.
The numbers below represent a small subset of Campaign Monitor client lists (by taking a statistically significant portion of the client databases, I can be sure of accurate results but utilise far less computing time). What we can see is an unnatural spike in opens from Mountain View, so either tens of thousands of people moved there this week, or something strange is happening. In this case, this subset of clients all happen to have their campaigns housed in a DataCentre thats fastest rout to Google is straight into Mountain View, CA. For other vendors, platforms and marketers the opens may not spike in Mountain View, it depends on the location of your DataCentre and what interconnectivity and peering arrangements you may or not have in place.

DATE        :  Opens from Mountain View, CA (Google DC)
1/11/2013:   490
2/11/2013:   14816
3/11/2013:   51732
4/11/2013:   81751
5/11/2013:   87018

So in short, I think of the 4 issues that people are discussing there is only one that is proving to be an issue that EDM platforms cannot resolve for themselves and would need some assistance from the Gmail team. That is “Point 2 Gmail is proxying requests for images” It may be that Google is playing some kind of ‘Privacy card’, and that they had every intention of achieving this, and it was not an unintended consequence. I have even heard the suggestion that this may in some way relate to limiting the amount of information that could be provided to the NSA by Google, though I am not sure I truly understand that concept.

I have a direct contact on the Gmail team and will forward my synopsis, a request that they include an x-header that identifies the original IP and a link to this post. I believe if it is not privacy concerns that are at stake they may make a change. I don’t however expect any feedback from the team, and definitely no statement would be released by way of EmailExpert.org.  Hopefully the Gmail Team may blog about this, especially so if it is their intention to obfuscate the recipients IP address.

I plan to continue to monitor the situation closely, I will see and report on any changes.

edit, read the latest post on the subject here: The fix for Gmail Open Tracking is revealed here 

Please Note: This POST does NOT represent the views of my employers. They have a great blog here, and will likely make an official statement of their own.

Andrew Bonar

The founder of emailexpert.org, Andrew Bonar currently resides not far from Sydney in Australia where he performs his primary role as Postmaster for self-service ESP Campaign Monitor

In the past two years alone Andrew has been responsible for the delivery of in excess of 120 Billion messages. With more than 15 years of industry experience, Andrew is widely recognised as a leader in the field of message sending, deliverability and compliance.

In 1996, he co-founded the UK’s oldest privately held ISP, Cheapnet Ltd. In 1998, launched the UK’s first privately held eCommerce payment systems: eBanx Ltd, and in 2003 he launched two of the very first ESP’s in Europe: MailPhoenix and eMailGenie.

From 2006 Andrew served as an independent consultant at organisations throughout Europe, the Middle East, Asia Pacific and the US. More recently serving as Worldwide Director of Deliverability at Emailvision, managing deliverability operations in 22 Countries. Andrew continues developing and evangelising best practices in permission-based marketing with clients and industry associations and travels extensively in Asia, Europe and North America to fulfil these obligations.

7 Comments

  1. Harald Walker said:

    Point 2 is also an issue with Outlook.com (Hotmail), where Microsoft also uses a proxy sever for images unless they use https.
    At least the Outlook.com proxy passes through the original user-agent but also here the IP address has a geo-location of a server in Redmond and from the user agent you can’t detect that it is the Outlook.com proxy.
    Workaround would be to use https for images that shouldn’t be cached. Unfortunately the Google proxy also caches https images.

    • Andrew Bonar said:

      Thanks for that extra information, really good stuff and much appreciated!

  2. Harald Walker said:

    I can’t confirm point 3. The user agent is always:
    Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (via ggpht.com)
    So reporting on OS, browser, mobile device etc. is not correct any longer for Gmail recipients. The same proxy with the same user agent is also being used when using the Gmail application on a mobile phone.

    • Andrew Bonar said:

      Thanks Harald, I should have double checked my source of information in regards to that, it was passed to me by a colleague

  3. Walt Mann said:

    Regarding your Point 1, while Google has been caching its own resources for a long time, with this change Gmail is now using its own caching server for all resources in the sender’s HTML content — every image request in an email you send now goes through Google.

    This appears to be URL-based, so if two end-users open the same email that contains the same resource URL, they both will request the resource from Google’s proxy, but that doesn’t mean both requests will go back to the original resource. In fact, likely only the first one will.

    So if you use non-unique URLs for campaign tracking (that is, there’s no unique hash or user id in the tracking pixel URL), this caching is definitely an issue for you.

    • Andrew Bonar said:

      Agreed, if your not hashing then you are most definitely going to be having issues

Comments are closed.

Top