This is the first in a series of posts presenting an interview I conducted with Paul McDonald, Product Manager, Google Consumer Surveys.
In the interview we covered a wide range of topics, including the genesis and development of Google Consumer Surveys, Google’s corporate philosophy, the product’s methodology, the market research industry, competition, and possible future directions for the product.
In this first part of the interview, Paul explains how Google Consumer Surveys was conceived and how it developed internally prior to launch.
Importantly, he indicates that the core idea behind the project is giving Google’s publisher network a way to keep their premium content online rather than putting it in apps or behind pay walls.
In an earlier iteration of the project, consumers “paid” for premium content by completing tasks (such as categorizing images) which humans do better than computers. However, people didn’t like those tasks. They do, however, like answering questions – to a point.
Paul also explained how Google’s philosophy of “faster is always better” informed the development of the Consumer Surveys product. He discussed some of the tradeoffs made in developing the product as well as some ways in which the product is different from traditional market research.
Here’s Part 1 of the interview.
You’ve certainly made a big splash with the launch of Google Consumer Surveys. What was your background prior to being part of the Google Consumer Surveys team?
Paul McDonald
Sure, so I came right out of school and joined Google about nine years ago. And I worked on a whole bunch of different things here at Google. Anything from ads to our commerce initiatives. I was part of the team that started Google Checkout. I also worked on our developer tools, so tools for external developers, and our APIs. I was part of the team that launched Google App Engine. And then I moved back to ads and worked on some optimization things for advertisers. And most recently I was the product lead for Gmail. So I basically ran the product strategy and design for Gmail for about two years before moving off to start this project.
And how long has this project been in the works?
Well, it actually started as what we call a “20 percent project” at Google. A friend of mine, who is an engineer I worked with on Google App Engine, had some ideas around helping publishers monetize their content. And he and I went back and forth a few times and sort of stumbled upon this idea that we could have users spend their time instead of their money for access to content. That was in July of 2010 when the idea kind of first came up. We put it on hold for a long while because we were both doing other things. I was the product lead of Gmail and had a very busy schedule. So we worked on it on the side. And then around March of last year we kind of decided this is something that we really wanted to look into and do. And we started really focusing on this project.
And why did you decide to enter the market when you did?
I think it was really more about the product that we wanted to create. And what the minimal viable product we thought was to get out the door. Again, this all comes from a different angle than traditional market research. It started with an idea around figuring out ways for publishers to monetize content, because at Google it’s in our interest, of course, to have publishers keep their content free and online for users. And what we saw was a lot of the publishers were sort of packaging up their data, the stories and articles, into applications for phones, mobile phones, and tablets. And at Google we don’t really have access to that data anymore once it’s kind of packaged up and sold to a user that way.
So we wanted to keep the publishers putting their content online. And the only way that we saw that was possible is if we could solve the monetization problem for them. If you look at the major newspapers of the last five or six years, the revenue from the physical paper subscriptions has gone way down. And the online revenue from ads on their online sites hasn’t really made up for that loss of revenue. And so a lot of them are turning to paywalls. That’s like the New York Times and Wall Street Journal are known for these paywalls. And we didn’t really like that user experience. We didn’t like that users had to pay for the content or pull out their wallet. It took a lot of time and it wasn’t always clear what the value you were getting for that money was for some of the smaller sites.
And so what we tried to do is figure out a way that users could spend their time instead of their money getting access to this content. And at first it wasn’t about market research at all. Instead, what we were trying to do was have users do things that computers weren’t very good at doing, human computation tasks is what we called them initially. These are things like labeling images for search results or maybe picking the best set of search results or websites for a given query. Things that it’s hard to do, Google finds particularly difficult to do. We thought we could get users to do it for us, because humans are better than computers.
It turns out there isn’t really much of a market in that, at least right now. And users took a long time to get these things right, and didn’t like to do them. But when we put in some market research questions just to kind of test the viability of them for users on our first publisher, it worked really well and we found that we got really surprisingly accurate and consistent data back from these questions. And so we really sort of focused in that area. And that’s kind of how the idea germinated and became what it is today.
And how did you go about designing the product?
So initially we built a product that we wanted to use. At Google we have a bunch of different products and questions that we have about them for our users. So we started with the groups that do research at Google and asked them, what would you like to see out of a tool like this? And we actually have a lot of market research scientists employed at Google to provide teams with support. And so we relied on them to build a product that they thought was viable in the industry and was usable. And really we tested it out on ourselves for a long time before we opened it up to external companies to try.
And I think at Google there are a couple things that are really important to us. One is speed. We have a maxim here at Google that faster is always better. And we really focus on speed in a lot of different areas, particularly the user experience or the respondent experience. That they could answer these questions very quickly, and get access to the content quickly. And for the researcher, too. We wanted them to be able to create and field surveys and get responses back within hours instead of days or weeks that we’ve seen in other platforms. And we wanted them to be able to go through the data very quickly and be able to analyze it and pull out insights in that data very quickly. So those are the things that we sort of focused on initially.
And I think you’ll see in the product, we made some choices that may seem strange from a market research perspective. For example, we saw that users really weren’t willing to do more than two questions at a time in order to get access to content. So we limited the number of questions shown to a user at any one time to two. Now that also limits us from the research side. We can’t do cross-question correlation beyond two questions. But we also found when talking to a lot of researchers and companies that do research that if you had the demographic data supplied for you already, the cross-question correlation beyond two questions was actually fairly rare. There were definitely use cases, but they were rare in most cases that we’ve seen. And so we thought maybe that was OK. And we decided to go for that.
There are also limitations on the number of characters you can put in a question or the answer options. And we did that because we thought users needed to be able to read and answer the question quickly. And it sort of forced researchers to be more precise, or concise, in their questions and their answers, which led to better questions we found after putting some of those limitations in. And finally we have a limited number of question formats today. We don’t have grids, we don’t have complicated branching logic, we don’t have rating questions with tens of different ratings, or multiple choice questions with tens of different options. We make it pretty simple to do. And again, there are some limitations there from the research side, but we’ve seen a lot of creative workarounds for those limitations. And we think those have led to better questions and more accurate and consistent data from our users.
So how have things been going since the launch?
To be continued…
The next post will cover the market reaction to Google Consumer Surveys, thoughts on the market research industry and more.
About Dana Stanley
Pingback: Everything You Ever Wanted To Know About Google Consumer Surveys But Hadn't Had A Chance To Ask | GreenBook