A/B testing and experimentation in the Obama 2008 and 2012 campaigns

Tue Oct 25 2022

Jack Virag

Editor in Chief, Statsig

The complete digital strategy breakdown to both Obama campaigns (and why they were so successful!)

⚠️ Don’t worry. This post is politically neutral. You are safe from biased undertones.

The Obama campaigns are often cited as the modernization of political data. In 2008, the Obama campaign drastically outperformed the opposing campaign (365 v 173 electoral votes), causing anger and frustration in Republican voters.

The source of that frustration was that, to those whose predictions were based on the television debates, the election seemed like it would be close: The candidates both answered questions very well. Thanks to the power of wielding data, however, the results were anything but close.

Again in 2012 the Obama campaign emerged victorious over a younger, more equitable opponent. Many voters were shocked, again. Incendiary statements about voter fraud were tossed around. But was it voter fraud, or was it the results of the first spectacularly-managed data-driven campaign?

(Spoilers, it’s the latter.)

Despite both being data-driven, Obama’s 2008 and 2012 election campaigns were as different as night and day. The first election was a proof-of-concept, and the second went all-in on the digital campaign strategy. Like true experimenters, let’s compare and contrast!

Sidecar no-code A/B testing

Get the Statsig Sidecar Chrome extension and run A/B tests and experiments on your website without writing code.
sidecar cta image

2008: Proving the theory

Website optimization

It all begins with website optimization, which was done primarily with A/B testing. In general terms, A/B testing websites usually goes down like this:

  1. Create two slightly differing web pages

  2. Determine a conversion metric like CTA (call to action) button clicks

  3. Randomly* split inbound users between the two variants

  4. See which variant had the better metrics

The digital manager of the 2008 election campaign Dan Siroker usually gets credit for this tactic, and the results of A/B tests were used to make decisions about verbiage, colors, styling, etc…more on this later.

Sequential donation form workflows

Another groundbreaking change was the implementation of sequential donation form workflows. Typically, upon clicking the donation button, a user would be presented with an enormous, daunting web form. Back in 2008 these looked particularly horrific on mobile devices.

The sequential workflows only showed one portion of the form at a time, stringing pages together with a “next” button. These are the norm today, with e-commerce sites typically handling buyer info, shipping info, and payment info all on different pages.

The theory, proven true, was that the entire form was a large hurdle for users, but presenting it one page at a time made the workflow easier, and thus would lead to more donations.

Opt-in forms everywhere

The Obama 2008 campaign newsletter strategy is easily summed up in one word: Relentless.

Please allow me to interject some statistics for context before proceeding:

  • Studies show that users read as little as 3.5% of words on a web page, and spend fewer than 15 seconds reading blog posts.

  • Users also spend 71% of time on a web page scanning for pieces of important information instead of reading

  • Best marketing practices say to disperse 2-3 CTAs across content—even if they’re identical—instead of just one, due to how severely users skim

  • If you’re part f the 19% who read word-by-word, I appreciate you

To optimize against these odds, the Obama campaign stuffed their web pages, social media posts, news articles, blog posts, etc., with CTAs to sign up for the newsletter.

Again, this is something marketers practice today, but back in 2008 the hunch was that—despite campaign donations being the most desirable outcome of a web page visit—it still makes sense to present website visitors with the opportunity to sign up for a newsletter so that they can be kept warm for future conversions.

And being subscribed to the newsletter meant that you were constantly being asked for donations, with donation requests being the campaign’s primary email content. In fact, even upon signing up for the newsletter, the “thank you” exit page featured a donation CTA. Talk about foreshadowing.

Introducing Product Analytics

Learn how leading companies stay on top of every metric as they roll out features.
an isometric module showing graphs with the word "query"

A/B Testing

Obama’s former 2008 campaign manager, Dan Siroker, estimates that A/B testing and website experimentation resulted in an additional 2,880,000 email signups and $60,000,000 in donations.

The campaign mostly experimented with one element on a page at a time, in true A/B testing fashion. For instance, one landing page might have a black and white photograph of Barack Obama, and another had a full-color version. After analyzing the behavior of users on each variant, a “winner” can be selected, which is installed as the web page’s “permanent” photo.

This is also known as split testing.

A/B/n testing, however, is like A/B testing but for many different variants. Like a website with multiple variations of the headline, multiple variations of the hero image, multiple variations of the button text, and so on. A powerful system (like Statsig) can handle all of those variations, mitigate bias, and deliver the best combination of variables.

An example of a donations site with multiple options for each field.

The campaign also A/B tested email subject lines. Their methodology was pretty simple, but set email marketing standards that are still used to this day:

  1. Send variant A to 10% of the total email audience at random

  2. Send variant B to an additional 10%

  3. After an hour, send the winning subject line to the remaining 80% of the audience

Obama email campaign headlines

A lot of cool digital stuff happened in the 2008 Obama campaign, but 2012 is when the real magic happened.

Get a free account

Get a free Statsig account today, and ping us if you have questions. No credit card required, of course.
an enter key that says "free account"

2012: Doubling down

“What we will see in 2012 will make 2008 look primitive.”

Mark Sullivan, Voter Activation Network

The digital campaign

And it did make the 2008 election look primitive.

The Obama 2012 campaign drew inspiration from the corporate sector. At the heart of the digital operation was a centralized database that was designed from the ground up solely for the election.

This database allowed all levels of employees access to information including data about voters that could be used to craft custom messaging or build mailing list audiences.

It also featured a dashboard—much like what a sales representative would see on their company’s Salesforce dashboard—empowering employees and volunteers to make calls, send emails, take action on social media, view upcoming events, and more. Each campaigner had this power at their fingertips.

With every Obama 2012 campaign staffer possessing modern digital tools and tactics, it’s easy to see in hindsight how the election was won. It is said that the Obama 2012 campaign could tailor a single donation request to 26 distinct segments of American voters.

Jeff Chester of the Center for Digital Democracy said “This is beyond J Edgar Hoover’s dream. In its rush to exploit the power of digital data to win re-election, the Obama campaign appears to be ignoring the ethical and moral implications."

At the time, it may have seemed like the Obama campaign was ignoring ethics, but in reality the campaign is only guilty of using data in a manner that was already becoming commonplace in the corporate world. The same year, Target’s algorithms figured out a woman was pregnant before her family did, for instance.

Data-driven social psychology

Social media and online messaging were the crux of the 2012 Obama campaign.

Users, volunteers, and recipients, were urged to log into the Obama re-election campaign website using their Facebook credentials. The year was 2012, and I guess a lot of people didn’t know any better.

“This is the Moneyball moment for politics. If you can figure out how to leverage the power of friendship, that opens up incredible possibilities.”

Sam Graham-Felsen, Obama Chief Blogger

Upon logging in with their Facebook credentials, all of users’ personal information was scraped from their profiles, including location, age, interests, likes, profession, friends’ names, and so on. It is worth noting that some people claim that Donald Trump won the 2016 election solely due to his use of Facebook data.

Fun fact: The term “influencer” was first coined by the Obama 2012 campaign regarding Facebook users, as influencers’ data was handled differently.

As Teddy Goff, The Obama 2012 Campaign Digital Director famously said: “People trust their friends, not political rhetoric.”

During the election buildup, the Obama campaign authored 614 social media posts, compared against the competitor’s 168 posts. The Obama campaign understood the powerful social psychology behind social media apps, and leveraged every scrap of data they could.

The payoff

The 2012 digital campaign was the most successful fundraising campaign of all time, raking in over one billion dollars in donations—almost $700,000,000 of which was online. Furthermore, it registered more than one million American voters online, thanks to the help of hundreds of thousands of volunteers.

The campaign also built a Facebook audience of 45 million followers, a Twitter audience of 33 million followers, and amassed more than 133 million video views across its sites.

So the next time you’re writing email subject lines and you feel the urge to test out a few different variants, understand that that very urge will lead you on the path to digital excellence, and maybe even a billion dollars.

Request a demo

Statsig's experts are on standby to answer any questions about experimentation at your organization.
request a demo cta image

Build fast?

Subscribe to Scaling Down: Our newsletter on building at startup-speed.

Try Statsig Today

Get started for free. Add your whole team!
We use cookies to ensure you get the best experience on our website.
Privacy Policy