Knowledge Base/General Features/Marketing Knowledge

Accuracy Problems With the Open Rate Measurement

Conor McCluskey
posted this on March 29, 2010 12:41

Open Rate is the most widely used email metric. However, Open Rate has several accuracy issues:

  • Preview panes – if HTML images are turned on, an open is recorded whenever an email appears in a preview pane. Preview panes increase Open Rates regardless of whether the recipient actually viewed the email.
  • Deliverability – Most organizations calculate Open Rate as a percentage of delivered emails. However, many emails are filtered out before they get to the inbox and not reported as bounced. Deliverability adds an uncertainty to the Open Rate calculation.
  • Blocked images – many email clients block images by default, and in doing so, fail to execute the piece of HTML code which allow Open Rates to be measured. Blocked images artificially lowers Open Rates.

Reporting Behavior of Email Service Providers†

Browser/ESP Tested Test Case: Click on email in inbox and display images. Update trcking. shows 1 unique and 1 total open? Click off email in inbox then back on. Upd trkng. 1 unique and 2 total Opens? Test Case: Click the link. Upd trckng shows 1 click? Click link again. Upd trkng. Clicks=2? Test Case: Click the video. Upd trckng, shows 1 play? Play again. Upd trkng. Plays=2?
Chrome 17 Opens Clicks Plays
Gmail Ok1 Ok Only shows 1 play
Hotmail Ok Ok Only shows 1 play
Yahoo Ok Ok Only shows 1 play
AOL Ok Ok Only shows 1 play
Zoho Ok Ok No plays
Firefox
Gmail One per session2 Ok Only shows 1 play
Hotmail One per session Ok No plays
Yahoo One per session Ok Only shows 1 play
AOL Ok Ok Only shows 1 play
Zoho One per session Ok No plays
Internet Explorer
Gmail One per session Ok Only shows 1 play
Hotmail One per session Ok No plays
Yahoo One per session Ok Only shows 1 play
AOL Ok Ok Ok
Zoho One per session Ok No plays
Safari 5
Gmail One per session Ok Only shows 1 play
Hotmail One per session Ok Only shows 1 play
Yahoo Ok Ok Only shows 1 play
AOL One per session Ok Only shows 1 play
Zoho One per session Ok No plays
Outlook 2010 (IMAP connected, except with Hotmail which uses Outlook Connector)
Gmail One per session Ok Only shows 1 play
Hotmail Ok Ok Only shows 1 play
AOL Ok Ok Only shows 1 play
Zoho One per session Ok No plays

† Tested Mar 2012.

1 Ok = count in BombBomb is incremented by 1 each time item is clicked.
2 A session equals each login to your email account. If you log out and log back in, that begins a new "session."