Skip to content

Online Advertising Effectiveness Measurement (Part 9 of 9)

November 1, 2008

There are seven areas I considered for the measurement of Advertising effectiveness they are listed below.

The universe of online measurement can be bewildering. There are competing methodologies, competing products, and competing claims that create a great deal of confusion in the marketplace. To address some of this confusion, we need a better understanding of the measurement landscape. This post discusses some of the most prominent measurement services available (from the broad array of possibilities), their product offerings, methodologies, and areas of expertise.

  1. Offline Audience Measurement

  2. Online Audience Measurement

  3. Online Campaign Measurement

  4. Site Analytic’s

  5. Brand Campaign Measurement

  6. Emerging Measurement

  7. Business Metrics for Online Advertising

Offline Audience Measurement

The universe of advertising and audience measurement for offline media such as TV, radio, and print is far less bewildering, but it has not always been that way. Whenever a new medium emerges, competing measurement systems are typically launched, with each hoping to become the standard currency for ratings. In more established media those battles played out long ago, and advertisers and media companies have agreed to a single standard to measure audience size, cumulative reach, and ratings.

Audience measurement and advertising effectiveness are never an exact science, and are always disputed at some point. Both media buyers and sellers complain bitterly about methodologies, functionality, cost, and accuracy. Advertisers would like to know more precisely what is working to grow the brand. There is a broad array of measurement services available, and the following pages examine their methodologies, offerings, and core competencies.

I begin by looking at the audience measurement landscape in the online space, using metered tracking, surveys, and internet service provider (ISP) data for national and local markets. These audience measurement companies are most useful in the prebuy phase for media planning and competitive analysis. I then profile companies in the ad serving space, which provide real-time campaign data analysis for advertisers, followed by companies that provide site-side optimization analytics.

Pre-Buy Planning > Campaign Measurement > Site Optimization

Emerging niche measurement companies in the brand assessment, buzz quantification, and cross-media measurement industries, which do not fit neatly into the standard categories. Depending on your online advertising goals, budget, and degree of sophistication, you may need to utilize several of these services to get a holistic view of your online marketing.

Online Audience Measurement Major Players

There are two prominent methods of measuring online audience activity: metered tracking using panels, and telephone surveys. Both methodologies have their merits and weaknesses, and the major audience measurement companies such as Nielsen//NetRatings and comScore Networks use a combination of both, depending on the service. Metered tracking data services utilize technology that records which websites an internet user visits via software that panel members voluntarily install on their computers. While metered panels can get a more accurate view of online media consumption than telephone surveys might because they track actual behavior rather than self-reported visitation, metered data typically comes under fire for underrepresenting the at-work audience—an audience that is crucial for many publishers. Few companies permit metering software to be installed on their company network, especially medium-sized to large companies. As a result, much of the at-work data is collected from smaller businesses and then projected to the full at-work audience.

  • · ComScore Networks
  • · Nielsen//NetRatings
  • · Hitwise
  • · Scarborough Research
  • · The Media Audit

Audience surveys such as the ones conducted by The Media Audit and Scarborough Research, to measure local usage, employ aided recall (e.g., respondents are asked which websites they visited and are given a list to choose from) or unaided recall (no list is provided and respondents must answer from memory). While both methodologies probably skew slightly toward larger, more well-known brands and properties since respondents self-report their visitorship, this is one of the few ways to understand cross-media usage.

Online Campaign Measurement

Despite the internet entering its second decade as an advertising vehicle, there is still some debate over how to count an impression. To advance the medium toward standardization, the Internet Advertising Bureau (IAB) launched a Measurement Task Force in November 2004, with the support of many of the leading corporations in advertising and research. The goals of the task force were to develop a global standard for counting online advertising impressions and to provide industry transparency to the systems that measure ads. While the impression argument is not completely settled, the standard established by the IAB has put to rest much of the debate.

The guidelines have the support of many major online publishers as well as close to 40 major proprietary third-party ad-serving technologies. In addition to proposing a detailed definition for counting an ad impression, the task force also recommends third-party independent auditing and certification guidelines in the United States for ad-serving applications.

Companies that have completed the certification process include Atlas Solutions, DoubleClick, CNET Networks, Walt Disney Internet Group, and Yahoo!, among others.

This post examines two companies that are crucial to understanding an advertising campaign’s online effectiveness: DoubleClick and Atlas Solutions.

Atlas Solutions

At the most basic level, Atlas Solutions is a third-party ad server for advertisers, agencies, and publishers. Its origins can be traced to the late 1990s as a proprietary ad-serving solution for the interactive agency Avenue A. Around 2000, the strategic decision was made to separate Atlas from Avenue A (although both are today owned by the same corporate parent, aQuantive) and sell Atlas’ services as a stand-alone company. Born out of an ad agency, the product was geared toward media buyers more than publishers and remains so to this day.

Both Atlas and DoubleClick use a similar technology to track actions such as sweepstakes entries, registrations, and sales conversions. The third-party ad servers use a beacon to track action by placing a 1 x 1 pixel invisible graphic interchange format (GIF) image on landing pages, with a cookie. Suddenly, the ad servers became much more valuable in terms of offering fairly sophisticated campaign analytics and optimization tools, capable of telling a much more complex story for online advertisers.

Atlas provides marketers with the tools to measure impressions, clicks, and, if they’re using action tags, conversions as well. For a large campaign with multiple creative units on diverse placements, deployment and optimization pose a logistical challenge.

Atlas provides an automated solution for creative optimization at the campaign level, weighting a creative unit’s performance based on a particular placement on a particular site. By adopting a champion versus challenger mentality (setting benchmarks for the top-performing ad placements), Atlas’ system recommends which creative units to drop because of underperformance and which units should increase their impression load.

According to Institute, there are two major trends in online campaign measurement taking place today. The first is cross-channel analysis between discrete forms of online advertising, such as paid and organic search and display advertising. As marketers increase their search marketing budget at the expense of display advertising, Atlas has undertaken research to understand the synergy between the two methods. Attempting to quantify the impact of display advertising on search marketing, Atlas found a 22% higher possibility of conversion when advertisers employ both search and display in conjunction with each other.

The second and related trend also attempts to tell a more complex advertising story by rethinking conversion attribution. With more media bought on a cost-per-action (CPA) basis, it becomes critical for a marketer to correctly attribute action to the proper source, or rather sources. The prevailing mentality typically attributes the conversion to the last click that occurred before an action took place; budgets and creative departments tend to be optimized accordingly. Atlas is currently attempting to develop metrics that shed light on what happens before the last click, or ad view.


Although DoubleClick offers a suite of advertiser and publisher management tools, for the purposes of this discussion, we are focusing on tools that help marketers measure their campaign effectiveness. Dart for Advertisers (DFA), like Atlas, is a hosted, third-party advertising management and serving product. While marketers can access their campaign data from any web browser, the hardware and infrastructure for delivering ads are centrally managed by DoubleClick.

Some of DFA’s central features include a creative library, allowing advertisers to manage their creative assets in a central repository and schedule start and end dates as well as test periods for campaigns. Marketers can also batch upload and assign assets to multiple ads, saving time trafficking and deactivating creative units. DFA allows marketers to roll up multiple ad placements into a single roadblock for better reporting and more accurate pricing. DoubleClick has also designed DFA to support myriad pricing models, including cost-per-click, cost-per-action, and flat rates. The system can also be customized or expanded by clients and integrated into existing in-house billing systems.

DFA also includes a reporting feature, ReportCentral, providing a centralized tool to track campaign metrics such as click-through rates, costs, conversions, and return on investment (ROI). The company claims its interface streamlines the time it takes to generate customized reports, and includes reporting features such as cross-site duplication, frequency to conversion, and time lag to conversion. Additionally, the reporting tool features network-level reporting, allowing users to perform trend analysis across advertisers and campaigns, and planned media queries, comparing actual costs to planned campaign costs.

Site-Side Analytics

The audience measurement companies, like comScore and NetRatings, provide valuable insight for marketers on where to find their target audience, the size of the audience, and the audience’s online behaviors. The campaign measurement companies, like DoubleClick and Atlas, give marketers a real-time view into how their online advertising is performing, based on clicks and responses. The third piece in a marketer’s online measurement toolkit should be a site-side analytics platform, offering a comprehensive view of not just how his or her advertising performs post-click, but also search and affiliate marketing, organic search performance, and the myriad ways consumers find a site.

The audience measurement companies, like comScore and NetRatings, provide valuable insight for marketers on where to find their target audience, the size of the audience, and the audience’s online behaviors. The campaign measurement companies, like DoubleClick and Atlas, give marketers a real-time view into how their online advertising is performing, based on clicks and responses. The third piece in a marketer’s online measurement toolkit should be a site-side analytics platform, offering a comprehensive view of not just how his or her advertising performs post-click, but also search and affiliate marketing, organic search performance, and the myriad ways consumers find a site.

The analytics companies listed here, as well as others, produce reports that companies can use to redesign their websites to sell more effectively, alter market segmentations, and simplify processes to boost consumer response. The tools can tell a manager which search words generate the most visits or the highest rate of conversion or what to offer in a follow-up promotion.

Brand Campaign Measurement

Not long ago, the internet seemed to be the exclusive domain of direct marketing companies, and the impact on brand image or trial of the medium was considered dubious. However, as the number of brand-oriented marketers using the internet increased, it became clear that online advertising plays an important role in brand building, capable of increasing awareness, interest, and engagement. There are several companies that specialize in quantifying the branding impact of online advertising.

Dynamic Logic

One of the leading online brand measurement companies is Dynamic Logic, founded in 1999 and today owned by Millward Brown Intelli-Quest. Dynamic Logic helps marketers measure advertising effectiveness online through traditional brand metrics, beyond clicks, helping to identify the most effective creative executions. The company methodology measures effectiveness through control/exposed surveys. Essentially, an exposed group of users who have seen an ad is compared to a control group of users who have not. The exposed group is randomly intercepted and surveyed immediately after the ad has appeared on their browsers. Typically, they are notified by pop-up or floating windows, with an incentive to win a small cash reward in exchange for their participation. The control group is recruited on the same sites during the same time period. Dynamic Logic then uses the survey to gauge metrics such as brand awareness, ad recall, message association, and purchase intent.

There are several advantages to the control/exposed methodology. Marketers only need several hundred respondents to achieve statistically significant results. This methodology doesn’t measure the effectiveness of the entire campaign, since most of the testing is done on a subset of the total creative executions in a campaign. Additionally, to test each creative unit in a campaign would be expensive for most advertisers and overwhelming for the publisher and audiences.

Dynamic Logic’s flagship brand measurement product is AdIndex, which typically uses the control/exposed methodology to collect attitudinal data from users. Dynamic Logic has designed the tool to be easy to implement, and allows advertisers, agencies, and publishers to customize and filter the data they collect. Additionally, Dynamic Logic has partnered with rich media providers to integrate its technology into products such as PointRoll, Bluestreak, and Eyeblaster. To ensure marketers make informed decisions, the company uses a minimum significance level of 90% to reduce the likelihood of a sampling error.


InsightExpress is a full-service online market research organization. Leveraging a combination of innovative technology and market research expertise, InsightExpress enables its clients to harness the power of online market research to make the right business decisions. Founded in 1999, the company is headquartered in Stamford, Connecticut, and maintains offices in New York, Los Angeles, and San Francisco. InsightExpress is affiliated with General Atlantic LLC.

InsightExpress offers a range of research solutions, including AdIn-sights, an industry-leading online media measurement offering. In addition to the in-market testing via AdInsights, the company also offers precampaign testing with the Creative Pre-Test and postcampaign analysis. With its latest solution, ADI Plus, a joint offering with Marketing Evolution (reviewed later), InsightExpress also provides clients with Interactive Gross Rating Points (iGRPs) that, for the first time, allow them to include online ad measurement in traditional marketing mix models.

AdInsights is a customizable research solution that measures key attitudinal components of live campaigns, such as unaided/aided brand awareness, persuasion, brand favorability, and purchase intent. AdInsights employs a standard test/control design, inviting users to participate in a brief survey while on a given website where the online ad may appear. Survey participants are invited via banners, leader-boards, and skyscraper units. The differences between the responses from the two groups are measured, and any lift can be attributed directly to the online ad campaign. Because this all happens while the campaign is live, and respondents have the same potential for exposure to other online media, this methodology allows for the online advertising to be isolated as the single driver of results.

Emerging Measurement

Nielsen BuzzMetrics

Nielsen BuzzMetrics started as three separate companies that consolidated in 2005, all with similar goals: developing technologies and tools “to help companies measure, understand, and leverage the increasing amount of consumer activity, influence, and content—the ‘buzz’—that was migrating to the internet and new forms of media” (Nielsen BuzzMetrics 2006). BuzzMetrics was founded in 1999 to create technologies that would help companies leverage word-of-mouth activities. Intelliseek was established to help marketers delve into immense amounts of company feedback and consumer-generated content to form real-time measurement of buzz about the brand. The third company, Trendum, developed linguistic analysis technologies.

In 2005, the three companies came together through a partnership with VNU, Inc. to form Nielsen BuzzMetrics.

Today the company’s objective is to create a new measurement standard for consumer-generated media. The intelligence gleaned from Nielsen BuzzMetrics’ services is valuable not just for media planning and buying but also for new product launches, customer service, and crisis management.

For the most part, Nielsen BuzzMetrics is selling its services to the brand or marketing teams within an organization. Within the market research department, BuzzMetrics tends to augment traditional brand equity tracking. For instance, within the television sector, a network launching new shows for the fall season might compare its half dozen shows to 40 others launching in the past year and in the same season, identifying their share of online conversation before, during, and after their marketing campaigns. The service can be used to track the effectiveness of a specific new advertising campaign to evaluate how much buzz the campaign has generated. More recently, corporate communication and public relations departments have partnered with Nielsen BuzzMetrics to monitor potential crisis situations, using the trend data on the amount and tenor of conversation about the event and the company to help spot early warnings of an impending crisis before it enters the orbit of mainstream media.

Marketing Evolution

One of the burgeoning areas of measurement that is important to measuring the value of online advertising is cross-media marketing measurement. A significant challenge marketers face is accurately gauging the effectiveness of their marketing dollars across television, print, radio, out-of-home, internet, wireless, and other advertising and marketing vehicles. Additionally, marketers must figure out how best to optimize their spending, creative aspects, and positioning across multiple media vehicles. Marketers need to understand how the internet fits into the overall marketing mix. There are now several companies centered on developing research protocols and tools for more accurate cross-media measurement and optimization.

Marketing Evolution is a company that specializes in cross-media measurement, providing its clients with dollar-for-dollar comparisons among their TV, radio, print, direct mail, email, web-sites, events, and internet spending levels. The company provides custom research and consulting services to help its clients test their marketing efforts and provide learning from those tests.

ROMO begins by understanding a marketer’s goals and objectives—what is the marketer ultimately trying to achieve? For instance, is a marketer’s strategy to build brand awareness and differentiation or to reinforce the brand message and generate immediate sales? Marketing Evolution translates these goals into a measurement design that works hand-in-hand with the media plan (creating exposed and control groups) and measurements of marketer’s goals (ranging from sales to branding). The measurement design almost always includes a survey, and the survey questions ask branding-related questions. To get a more complete understanding of effectiveness, Marketing Evolution has integrated sales metrics into the ROMO analysis, at the client’s request. The end goal of the analysis is to paint a detailed picture of each channel (including online) in the marketing mix and understand its contribution in terms of relative cost efficiency.


Integration is a consultancy firm founded in 1994, dedicated to designing tools that measure the effectiveness of marketing communications. Its core product is Market ContactAudit (MCA). Essentially, it audits both the effectiveness and the cost efficiency of a brand’s marketing plan across media. The goal of the audit is to provide marketers with a common currency for comparison so companies can manage the total brand experience across channels.

The first step in an MCA audit is what the company calls calibration: “mapping contact influence at category level before measuring brand-contact associations to determine brand performance in the category” (Integration 2006). The cost for this phase of the audit is $35,000. The initial phase is designed to establish benchmarks for the Brand Experience Point, the common currency that will be used to compare marketing across channels and brands. Integration believes this step is critical in order to make informed comparisons between different marketing channels, and between a brand and its competitors. It sets the groundwork for future Integration studies.

For most marketers, to get a truly holistic view of their online advertising performance they need three core measurement tools, each designed to strengthen a particular phase in their campaigns. First are the audience measurement services, which are critical for the prebuy phase and are used for media planning and competitive analysis. Second are the ad-serving companies, which provide real-time campaign data analysis, allowing marketers to see what is going on in their campaigns at that moment. Third are the site-side analytics companies that bridge the gap between what takes place in advertising and marketing campaigns and what takes place on a company’s website.

Business Metrics for Online Advertising

Metrics need to be tailored for each brand and company with respect to its particular objectives. There is no magic number. Instead, smart marketers use a constellation of metrics that are guided by seven considerations:

· Customer satisfaction. Search marketers look to understand and improve the linkages between the text ad or listing and the brand experience.

· Business process improvements. Marketers reengineer problematic processes, like newsletter sign-up, checkout, or customer service, for example, in order to create or sustain compelling brand experiences and assess their impacts on criteria like customer satisfaction, revenue, or customer retention.

· Financial indicators. These measure the economic value of the search advertising campaign. They often include return on investment metrics. The most common ones are campaign related: cost-per-lead, cost-per-acquisition, cost-per-sale (or transaction), revenue generated, and profitability. (The Glossary defines many of these and related measures.) After these, metrics typically become more personalized to brand requirements. They can include subscriber renewals or cancellations for publishers, lead-to-close ratios for e-commerce or B2B sites, or changes in cost factors that result from moving a business process to the web or improving an existing one.

1. Determine your online advertising goals. Define what you’re looking to accomplish with your online campaign—site traffic, lead generation, online sales, offline sales, and so forth. Ask if your goals are relevant to the business results you seek. If they’re not, why measure? Don’t confuse tactics with objectives.

2. Make sure they’re realistic. Are the goals achievable within the time frames you’ve established? Are you able to assign the right resource levels to them with the right level of capability and focus? Do they require some future event to occur in order for them to be accomplished?

3. Ensure that they’re measurable. We all know that business objectives need to be stated clearly and quantifiably. Objectives that specify capturing 300 qualified leads in 30 days or gross product sales of $1 million in the next quarter leave no room for interpretation. Progress against them is easily measured, and you can use a variety of tools for tracking, from local spreadsheets to rich analytical systems.

4. Be certain you have the right metrics to measure your objectives. There’s no point in tracking a specific metric, say average sale per visitor, if you don’t have the sales data, visitor data, and dates of sale at hand. Recognize that some metrics are easily available, some are easy to compute, some may need to be derived from multiple data sources, some may be proxies because direct measurement is not possible, and data quality may be quite variable. Be very clear about why you’re measuring, what you’re measuring, the quality of your metrics, and the confidence you have in them. Don’t waste time with metrics that don’t matter. Merely accepting data, no matter how cool and persuasive its presentation, will not give you the guidance you need to manage your brand, search advertising campaign, or budget effectively.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: