This is the second in a series of posts presenting an interview I conducted with Paul McDonald, Product Manager, Google Consumer Surveys.
In this second part of the interview, Paul talks about the reaction of the market to the launch of Google Consumer Surveys, indicating it has been a success so far in Google’s view. He also talks about the reaction of the market research industry, saying mostly people have been curious about Google’s offering and stating that he sees Google as a partner rather than a competitor to the market research industry. He said Google has no plans to screen respondents for future survey participation along the lines of an internet survey panel.
Paul also explained in a fair amount of detail how targeting and sampling work in Google Consumer Surveys. In a departure from traditional market research industry practice, Google “infers” demographics rather than asking respondents to self-report them. He said that Google Consumer Services is not integrated with other Google services and does not incorporate respondents’ data from their use of other Google products. He also confirmed that those who commission the research are the owners of the data.
He also commented briefly on LinkedIn’s experiment in the market research industry and said a few words about potential other large technology companies entering the market.
Here’s Part 2 of the interview.
So how have things been going since the launch?
Paul McDonald
Surprisingly well. To be honest we had no idea what the market was going into this. And it turns out that not only are the traditional businesses that do market research, the large CPG companies and market research firms and the creative agencies doing research, but also the hundreds of thousands of small and medium sized businesses that we have as AdWords advertisers are also finding value from this because they’ve never really had access to professional market research tools before. It was either too expensive to use or they spent too much time doing the work to get it up and running. And so what we’ve seen is that the small and medium sized businesses are really using this in a way that we didn’t expect. They’re running different types of questions, and trying to solve real business needs for them. And they’re getting the answers back very quickly, within 24 hours, 48 hours, which makes them able to have real data driven business decisions instead of going off of their gut, which is really interesting to see and has been insightful to us.
And what’s been the reaction from the market research industry?
I think more than anything people are curious. They want to know what we’re up to. Why we’re doing it and why we’ve made some of the choices we’ve made. I think a lot of the market research industry sees that they can provide value on top of just the raw results that we provide. And so just like we did with agencies in AdWords, we’re providing a platform to get the data they want back, but the agencies or the market research firms or the folks who are experienced in the industry are seeing ways that they can provide value on top of that data, to pull out the insightful data and really focus on the customer’s needs. We don’t have folks here at Google that will put together research for a customer. We don’t do syndicated research. We don’t do any of that. So really we’re just a platform and a sample provider. And we think we have – of any online provider at least – the most accurate, representative sample of the US population.
And what would you say to the market research industry? Are there areas of collaboration? What message would you send to them?
Absolutely. I think there are ways to collaborate, ways to use our platform to provide additional value to customers on top of the data. Really I don’t see us as much of a competitor rather than a partner. We’re trying to provide an easy way to get accurate data. And I think the researcher’s role is really to understand the customer’s business needs and get that data from any way, any population that they can get a hold of. And I guess I see us as that way for a good chunk of the industry.
OK. Tell me how the targeting and the sampling works.
Sure. So what we do is kind of interesting. It’s based on our ad targeting. So for AdWords and our Display Network, what we do is we cookie users who view ads across our Display Network or for the content network. These are hundreds of thousands of publishers that have ads on Google. And so we can understand the websites that a user goes to, and then create a demographic profile of them in our cookies that the ads can use to target.
So for example we know, based on the sites they’ve seen, with pretty good accuracy, the age and gender of the user. We know the location of the user based on their IP address. And then from the IP address or the location we can derive some other data like whether or not they live in an urban or suburban or rural area, and what the income for that area is. And from there – what we use is this ads data and what we call the DoubleClick cookie. This is the cookie that we drop for all of our ad stuff. We actually use that cookie to infer the demographic data of the respondent as they come in. So in real time, when we see a user, we know their age and gender and location and all these other demographic data.
And so what we do is we score all of the questions that need to be answered in some way, based on the demographics that are needed to answer the question to get to a representative sample. So if the user comes in and is needed, quote needed, to answer one of the survey questions, we serve them the question that is most needed of their demographic.
We do that all real-time. And it’s not really a quota system. It’s rather, because even if we need to serve more, let’s say, 18 to 24-year-old males for this particular survey, it’s not guaranteed that we’ll get a certain number of 18 to 24-year-old males. We just try to score it the best that we can so that we get the most accurate or representative sample as possible.
And then you can see in each of the results. For each question you can see what we call our bias table, which splits out the sample by age and gender and location. And you can see how that compares to the census data, using the US census, and what our bias for that particular sample is. So it’s really kind of an interesting way to target. And we have really three ways to target the population.
One is you can target the general population, so we get a representative sample of the general US population. Or you can target a specific demographic. So you could say males who are 18 to 24-year-old in the West, the Western region which includes California and Arizona and several other states in the West. And so that’s a separate way of targeting. And the final way of targeting is through what we call screener questions. So if you want to target, say, people who own dogs, you could ask them a two-part question. In the first part it would be are you a dog owner or do you have animals or something like that. And then if the person answered yes, or they answered the target answer, in this case it would be yes, we would show them another question from that survey. So you can screen out all the people you don’t want for your survey, which allows you to get at really unique populations of folks. If you want to target, for example, people who watch Hispanic television shows, you could do that. And ask them a bunch of questions. And because we have a very diverse set of respondents, you can almost always get the people that you want. Although some populations may take a little longer to actually survey.
The inferred demographics – that’s certainly an approach that has not been done traditionally. And it is a more limited set of variables than what market research surveys typically have. Do you anticipate moving toward inferring more variables? Or moving away from inference and toward another means of determining people’s demographics? I assume that you’re not looking to add actual demographic questions as is traditional.
Yeah, that’s true. I think we have to look at this from the ecosystem point of view of our product. We have both publishers who are putting these surveys on their sites to get paid for their content, we have researchers, and we have users. Now when a user comes to a publisher’s site and they’re asked more personal questions, things like their sexual preference, for example, they’re less willing to answer those types of questions. And we see that in the response rate. And the publishers are less comfortable with showing those types of questions on their site. Because they don’t really want to upset their users in that sort of way. So we think that inferred demographics, for both the cost and the increase of speed in which you get back the results, is a good trade off for researchers. And we use it to target our ads so we are pretty confident in the accuracy of that data. And in fact we know a lot more about the user then we show in the survey analytics, or the reporting that we showed today. So over time I think you’ll probably see more of those types of variables, including things that you generally wouldn’t get in a typical market research survey like interests or like types of sites they’ve visited. Those sorts of things you can do. You can do things you wouldn’t be able to do without asking a lot of different questions in a typical market research study.
Do you anticipate incorporating some of the data that is available from people’s usage of different Google services?
Not Google services, but rather the sites that they visit on the internet. And again, this is all based on our ads demographic data. And we sort of understand where people have visited online by the ads that they’ve seen. Does that make sense?
So it’s not integrated with any other Google services?
No.
And when you serve questions to different partners for them to display on their sites in exchange for premium content, do you factor in the content of the question, match it, or take it into consideration whether it’s congruent or not with the site content?
No, we actually don’t. If we were to do something like that, it would end up biasing the results in ways that would not be acceptable to researchers. So up until this point we’ve decided not to do that. Although we may do some experiments in the future along those lines.
OK. So you neither match nor try to avoid a match of content?
That is correct.
And who owns the data that are collected?
The people who pay for it, the researchers.
Do you have plans to help people target populations that are lower incidence than 5%?
We’d like to. The reason that we do 5% or any percentage as a minimum barrier is that there’s a cost to serving the screener questions without getting the answer that you want. So if you serve 10,000 questions and collect “no” instead of “yes” to those questions, then we are still paying out the publisher for those “no” answers. So we basically priced it and modeled the pricing around a way to keep us break even or even lose a little bit of money for a minority of screener questions. And so that’s kind of why we set the limits down lower.
One of the things that we thought about initially was pricing it based on the incidence rate. But it requires some technology that we don’t quite have yet. So in particular, we’d have to bill after the fact, or we would have to run a small number of screener questions to understand the incidence rate before we actually billed. And those are some challenges that we’re working through. So we’d like to go to smaller incidence rates, but at this point it’s not economically feasible to do so.
Do you have any plans to use the existing service that you have or evolve the service that you have to ask people questions that could later be used to target them for a survey invitation? For example, if someone was looking for people that have a particular type of bicycle, you could screen them and then you’re essentially doing something along the lines of what panel companies do right now.
No, we have no plans to do something like that.
Another company that is a large technology company that had some experience in the market research industry was LinkedIn. They entered the market research industry and ultimately decided to exit it. Did you look at their experience when you were planning Google Consumer Surveys?
We did. I mean, we looked at a lot of different companies and what they were doing. In the end, we thought that, based on our relationships with the advertisers that we had, we could make a real market out of this. Or there was a market to be had. And so we decided to enter the market and understand whether or not there is a market for what we’re providing. And so that’s kind of the road that we had taken. Yeah, I mean I think a lot of companies have tried different things, and we’re just another one of those companies trying something new.
Do you have an expectation about other large technology players entering this market?
I mean, of course we’ve thought about, or thought through, what the options are for a lot of these players. Yeah. I think that this data is valuable and I think the other companies will find it interesting.
Let me ask you about a few other possible directions that you could go and see what are your thoughts on those.
To be continued…
The third and final post will cover Paul’s answers to a series of questions I asked him about Google’s plans to pursue specific directions in its market research offering, including qualitative, text analytics and adding languages and geographies. Finally, he answers my question about what we can expect from Google Consumer Surveys in the next three to six months.
About Dana Stanley
