Measuring the Effectiveness of Your Blog

December 13, 2011 By 1 Comment

After you’ve been blogging for a few months, it’s a good idea to take a step back, examine what’s worked, and refine your blogging strategy. Especially if you have only recently begun blogging, it’s important to frequently review your strategy and successes, to make sure that you are progressing toward your goals. This can be hard to do if you aren’t sure how to go about measuring your blog accurately, or how to draw the correct judgement out of your information. Let’s examine one methodology to review your blog’s analytics and determine successes. To get started, log into whatever analytics platform that you prefer for your blog, be it Google Analytics, HubSpot, or whatever else is available to you.

Once you’ve signed into your blog, gather all of your blog’s data and export it to a spreadsheet so that you can slice it up and look at it from different angles. Your blog software can usually provide you with this general data, including the author name for each of your posts. If not, you may need to combine data from different systems. This way, you can easily build averages for each of your blog authors for how many views their posts get, or other criteria that you’re interested in. Depending on what data your blog can export, you have many options here – If it exports the tags for each post for example, you could also cut across the tags and see which of your blog’s tags generates the most views, tweets, or conversions on a regular basis.

Break Down Your Post Success By Author

Begin reviewing your blog’s performance by looking at clear breakdowns of your posts, such as by the author. This is an easy to access piece of data, and can show you who your real star authors are. Here is a Google Doc that can help you visualize how to break down your blog analytics by author. If you’d like to use it to get started, you should download it or save it on your own.

While the left contains the raw data from some sample blog analytics, the right hand side breaks down blog post performance by author. In this example, you can see that the average post by Kurt just isn’t performing as well as the other authors. This sheet can’t divine why Kurt’s posts aren’t performing as well, but it can point out the trends that you need to make your own observations. You can take this spreadsheet with you for your analysis – Just paste in your own blog’s data for post title, date, views, and authors into the left, and then fill in the right with the name of each blogger who writes for you regularly.

Look At Other Variables

This process works just as well if if you replace the content of Column D with another attribute that you know – For example, the post category, time of day, or other information. For example, you can do some surface-level research very easily into what times of day are most successful for your blog if you’ve previously tried out publishing posts at different times of the day. Go through your last two months of blog posts and categorize them into rough times of day, like “Morning”, “Afternoon”, “Evening” and “Weekend”. That way, you can break down the post averages by when you post them and see if different times of day lead to more successful posts for you. If you haven’t tried this out yet, spend a month varying up your posting times for your posts, and then check out if there are any consistent patterns.

Consider Your Business Goals

Finally, consider the business goals for your blog. Are you trying to generate conversions or leads? Or establish a presence as an authority on a subject? Review what metric you are trying to change by having a blog, and then look at how you can bring this into the analysis. Remember that the most important part of blogging is how it plays into moving your goals as a business organization. Do not just blog for the sake of doing it; if something is not working in your strategy, or if your marketing analytics show that your hard work is not translating into success, change things up until you find something that makes you successful.

Filed Under: Featured, How-To, Marketing, Social Media Tagged With: , , , ,

Exactly How Responsible Are We For Privacy?

December 7, 2011 By Leave a Comment

privacyFacebook recently entered a settlement with the Federal Trade Commission admitting all manner of fraud in its privacy policies.  The list of violations admitted are shocking enough to make the most hardened among us blush.

Here is the list of violations to which Facebook admitted, from the FTC announcement.

  • “Facebook changed its website so certain information that users may have designated as private – such as their Friends List – was made public. They didn’t warn users that this change was coming, or get their approval in advance.”
  • “Facebook represented that third-party apps that users’ installed would have access only to user information that they needed to operate. In fact, the apps could access nearly all of users’ personal data – data the apps didn’t need.”
  • “Facebook told users they could restrict sharing of data to limited audiences – for example with ‘Friends Only.’ In fact, selecting ‘Friends Only’ did not prevent their information from being shared with third-party applications their friends used.”
  • “Facebook had a ‘Verified Apps’ program & claimed it certified the security of participating apps. It didn’t.”
  • “Facebook promised users that it would not share their personal information with advertisers. It did.”
  • “Facebook claimed that when users deactivated or deleted their accounts, their photos and videos would be inaccessible. But Facebook allowed access to the content, even after users had deactivated or deleted their accounts.”
  • “Facebook claimed that it complied with the U.S.- EU Safe Harbor Framework that governs data transfer between the U.S. and the European Union. It didn’t.”

We shouldn’t be surprised that Facebook would use every means at its disposal to gain a business advantage (especially if you’ve see The Social Network), but the sheer impunity of the violations is still shocking.

Facebook agreed to a long list of reforms, but do you still feel good about using social media data in your research analysis?

If you have any grounding in the history of market research and its respect for respondent privacy, you should pause to consider the implications of the Facebook settlement for the future of market research.

Research-Live.com editor Brian Tarran writes about this dilemma in his recent post, “What the Facebook FTC settlement means for market research.”

“The question is, how do researchers respond knowing that errors of technology and ethical judgement might be commonplace?” Tarran writes. “Can they – and more importantly, should they – trust the promises a site makes to its users about its terms of service or privacy policy? Legal recourse for misuse of data might land on the website itself, but does that mean researchers are absolved of all ethical and moral responsibility to the people they are taking data from?”

These are important issues, indeed. As an industry, we should keep our eyes wide open and continue to be the skeptical data consumers we have been trained to be.

We should use our professional judgement to exclude data where there is a reasonable supposition that privacy violations exist.

We should lend our strong support to proper regulatory efforts like the FTC’s investigation as privacy advocates and as professionals with an interest in data integrity.

Once we’ve taken those steps, though, we’ve done our part.

We should then keep forging ahead and using social media data in our analyses.

It would be a shame if our noble concern for privacy were to stop us from innovating and taking advantage of new data sources, data collection methodologies and analytical techniques.

The reality is we are limited in our ability to control the privacy policies and practices of other organizations.  We must rely on the proper authorities to enforce privacy violations.  We should have a critical approach to our use of data.

Beyond that, we have satisfied our responsibility, and we should proceed boldly.

Photo Credit:  Alan Cleaver

Filed Under: Essay, Featured, Government, Market Research, Social Media Tagged With: , , , , , ,

Now We Have Smartphones; Shouldn’t We Try to Be Smarter about Surveys?

November 16, 2011 By Leave a Comment

This is a presentation from Survey Analytics‘ President, Andrew Jeavons, from the Market Research in the Mobile World Conference in Atlanta in July 2011.

(By the way, it’s pronounced JEH-vons, not JEE-vons, as Andrew explains in the video…)

Andrew presented guidelines for conducting mobile surveys, and he made suggestions for adapting Net Promoter Scores (NPS) in a mobile survey environment.

Enjoy!

Filed Under: Featured, Featured Video, How-To, Mobile, Video Tagged With: , , , , , ,

QR Code-Enabled Mobile Surveys: An Example

November 14, 2011 By Leave a Comment

 

 

 

 

 

 

 

 

 

 

 

[Editor's Note:  this post was originally published on the Survey Analytics Blog.]

My friend Scott Liang from Parametric keeps telling me – the quicker you collect feedback from the Point-of-Transaction, the better the recall and quality is. While I have absolutely no way of verifying his assertion, in todays Blackberry and iPhone induced ADHD world, it seems logical – and anything that passes the sniff test, works for me!

If we accept that premise, then what the Washington State Ferries in conjunction with the Washington State Transportation Commission is doing is pretty innovative — collecting feedback directly from commuters while they are in the ferry. As passengers are commuting in the ferry, they have options for using their smartphones via QR Codes to give feedback on the ride.

Few innovative options:

Thumbs UP and Thumbs Down:

Instead of having just one QR code that takes them to a survey, the Thumbs Up/Down model has 2 QR Codes – each representing positive or negative emotion espoused by the passenger overall. This is similar to the the universal facebook “like” button that we are all accustomed to in the web world.

FROG on Board Poster

Integrated with MicroPanel:

Users who choose to give feedback are then asked to join a panel for future surveys and feedback. This allows the commission to build a long term relationship with the passengers — the commission can then use this panel for pricing, satisfaction and other kinds of research.

Once the feedback was collected, users are given the option to go to the mobile-optimized page for MicroPanel;

Completely Turnkey:

No custom development. This entire solution is off-the-shelf. This reduces cost and complexity. SurveyAnalytics has as question type that supports creating QR Codes. This enables you to create a survey with multiple QR Codes for each option:




click-here-to-download-the-case-study



Filed Under: How-To, Innovation, Market Research, Mobile Tagged With: , , ,

How Not to Get Market Research Clients

November 11, 2011 By 4 Comments

I’ve worked in many roles in market research, but never as an in-house market researcher in a corporation.  I’ve always felt bad for them in that I know they get a constant stream of suitors among would-be market research suppliers.  In particular, at conferences as it appears they’re being accosted left and right by hungry supply-side sales people.

This Tuesday, November 8th, Tiffany McNeil, Strategy & Insights Manager, Innovation at Del Monte Foods was interviewed by Ray Poynter of Vision Critical on Radio NewMR.  She spoke about a number of topics, including how to – and how not to – get on her radar.

Note:  this interview has been lightly edited.  You can listen to the original interview on the Radio NewMR archive page.

Ray Poynter: What would you define as innovation in market research?

Tiffany McNeil: I’d say that, for me, good innovation is one of two things. Besides a general willingness to be sensible and try new things, which is actually one of the great things about BrainJuicer, I know Tom (Ewing) is the other guest today – we work with BrainJuicer a lot – they have a lot of proprietary tools that are trademarked, and those tools are fantastic, but if you have a question that doesn’t fit with one of the tools that they already have in their toolchest as it were, they are more than happy to try and find a new solution for you. And I think that a lot of other companies do have these trademarked tools, and kind of no matter what you ask them, try to shoehorn whatever your question is into one of those boxes. So I think that’s one thing – just a general willingness to be flexible and try things. And the other is sort of a more traditional answer, which is, kind of a better way to answer questions, either faster, more efficient, or maybe even just more engaging, so that when I present back to my internal partners, it’s something that’s more compelling for them to listen to or to pay attention to.

Ray Poynter: OK, we’ve mentioned one of your suppliers, but in general, when you’re looking for suppliers, what are you looking for?

Tiffany McNeil: It’s hard to say. I think that the suppliers who earn – I can’t speak for my company or my team, but for myself. The suppliers that I’m excited to go back to over and over again are the ones that are just great to work with – the ones who, when you call them, there’s always someone really smart and engaged on the other side of the phone, who are really responsive, who do lots of follow-up work without sort of claiming “scope creep” the second you ask a question you didn’t know the first time. And there aren’t actually that many people who fit that mold, unfortunately. There’s a lot of little mistakes, there’s a lot of sort of people who are really engaged when you’re trying to win the work and they disappear once you get it. So we have probably as a group a pretty small list of suppliers who we feel are always consistently delivering excellent work, and good partnerships.

Ray Poynter: So you have a good list, a small list of people – now, please do not everybody contact Tiffany offering your services…

Tiffany McNeil: [laughter]

Ray Poynter: …otherwise we’ll get nobody wanting to appear on the radio show, from the client side – but, what are the good ways of people getting onto your radar? If somebody is truly innovative, how would you like to find out about them?

Tiffany McNeil: It’s a really interesting question. I was thinking about this a lot yesterday. And I think that – how about this? The way not to do it is to call me and leave me a message, or send me kind of cold emails. I get – you know, everybody gets – but I get tons of them every day. And, not only do I not have time for them, but I also have a lot of guilt about them. It’s just not making me feel good about myself. But I think that when we’re looking for a new methodology or we’re looking for a new supplier, typically the first thing I’ll do is ask around on my team. There is a lot of word-of-mouth – people who have worked in other companies, and may have answered a similar question somewhere else and so might have an idea.

For example, I’m working on a question right now, which isn’t something that I’ve done before – the methodology’s going to require some pretty smart analytics, it’s going to be kind of analytically tough. So in my head I go through the roster of companies that fit that mold. So, you know, who do I know that’s really smart about analytics, and if I come up short, the next thing is I kind of walk around and ask everybody on my team. If I fall short there, I’d say I probably look to thought leadership next. So, Vision Critical is a good example. They are a company that I didn’t really know anything about, and we don’t use them – we do have a relationship with them now, but the way they got on my radar is two things: they showed up at the conference that Lenny (Murphy) chaired last summer – the NewMR new methods conference – but before that, I had gotten a piece of mail that was a list of the top – it actually wasn’t from them – I think it was either from BrainJuicer or from Synovate – one of those two – were also on the top of the list, but Vision Critical I think was top – and it just stuck out as a company that we don’t know, that apparently people like working with. So, that’s an example of how a company kind of got on our radar – now, frankly, we haven’t done any work with them, but I had a conversation with them, and I kind of feel like I know what they do and what they’re good at. And if the right question came up we would reach out to them proactively.

Ray Poynter: Super – and I think probably the list you’re talking about was a thing called GRIT, which is a study that GreenBook run in conjunction with lots of other people, and it has got some interesting questions in there, “Who do you think is innovative?” and yes, BrainJuicer do very well and a number of other companies do very well, including some of the large ones, which I think was an interesting part of that. I’m going to change tack now, and Tiffany, thanks for that, and I hope you don’t receive even more emails especially now that you said that’s not the way to reach you.

Here is Tiffany’s biography:

Tiffany McNeil is a client-side research manager whose circuitous career path brought her most recently to a Strategy and Insights role in the CPG world. Before that, she spent time in New York and London, where she worked primarily in the television industry, including content and editorial research roles at UKTV and Channel Five in London. She is passionate, smart, and opinionated – um . . . and modest charming, but she wrote this herself, so take that how you will. She lives in San Francisco(ish) with her family and spends most her time making lists.

Filed Under: Featured, How-To, Innovation, Market Research Tagged With: ,

Election Polls: 5 Tips For Navigating the Clutter

November 7, 2011 By Leave a Comment

vote buttonTomorrow is Election Day here in the States.  The big vote (for President) isn’t until next year, but we have the usual spate of races for local offices as well as a wide range of citizen-initiated referenda on everything from the mundane (bond measures) to the highly divisive (social issues).

As the expected avalanche of survey results washes over me and everybody else (accompanied by a barrage of television advertising, internet electioneering, and attack mailers), I thought I’d share some tips for understanding which poll results to heed, and which to take with the proverbial grain of salt.

My experience in the world of election polling has given me a bit of insight into this topic; I hope these tips help you find your way on your journey through the sea of election polling data.

1. Understand the Methodology

The best way to judge an election poll – indeed, any survey – is to have a good understanding of the study’s methodology.  Unfortunately, reporting on survey methodology is often woefully inadequate, scarcely going beyond a reporting of the margin of error.  However, sometimes one can read between the lines of a story to gain a better understanding of the circumstances under which the poll was conducted, including: timing, data collection mode and survey length.

If the methodology is completely unclear from the article, be a good citizen and email the editor requesting clarification.  The rest of us will thank you!

2. Think Random

Generally speaking, in election surveys, relative to market research surveys, it is particularly important to have sampling methods that give all likely voters as equal as possible an opportunity to be surveyed.  Look for efforts to ensure a representative sample, including:

- surveying voters at different times of day and on different days of the week
- compensating for sampling limitations such as telephone coverage, cell phone coverage, and internet access
- using a “likely voter” screening question, ensuring only those both registered and likely to vote are considered.

Also, be wary of  polls conducted using automated telephone interviews rather than trained interviewers.

3. Evaluate the News Source

As good a data consumer as you are, you cannot realistically check the fine details of methodology on every study.  Therefore, pay attention to the reputation and track record of both the publishing entity.

More reputable organizations tend to higher publishing standards than less reputable ones; they have more to lose if they publish bad reporting.  Put more stock in a study reported by the Washington Post or the Pew Research Center than a study sponsored by a smaller or less reputable newspaper, website, non-profit organization.

In local elections, sometimes the most reputable (though hardly infallible) source is the state or locality’s largest newspaper.  However, look for critical analysis rather than simple reporting of results.

4. Pay Attention to the Sponsor.  Often a group with a vested interest in an election will privately commission a survey to be used for internal strategy; however, if some of the results support their public relations efforts, they will release an (often-misleading) subset of the data in order to influence the electorate.  Sometimes entire carefully-worded polls will be conducted which are meant for public release.  Be very skeptical of any data paid for by a group or persons with an interest in the election’s outcome.

5. Learn from the Experts.  There is a wealth of great content created by smart people who focus on analyzing election data.  Take advantage of their wisdom!  I’m partial to HuffPost Pollster (formerly Pollster.com) – check out my friends Mark Blumenthal (a.k.a. “@MysteryPollster“) and Margie Omero – and Nate Silver’s Five Thirty Eight blog at the New York Times.

These resources are primarily focused on U.S. Elections; there are undoubtedly many more good resources out there both for the U.S. and everywhere else; please suggest others in the comments section below.

I hope these tips are useful to you.  And if you’re in the U.S., don’t forget to vote tomorrow!

Filed Under: Government, How-To Tagged With: , ,

How to Plus or Minus: Understand and Calculate the Margin of Error

November 4, 2011 By Leave a Comment

iPhone - Plus or MinusSometimes in the day-to-day work of conducting and interpreting market research, it’s easy to forget that many people who work with surveys on a daily basis have not had formal training in statistics. Even for those who have been trained, it can be useful to have a refresher from time to time.

UNDERSTANDING MARGIN OF ERROR

One of the most basic concepts in market research is the confidence interval, commonly referred to as the “margin of error.”  The confidence interval is a range of values within which a survey result can be assumed to accurately represent the underlying construct being measured.

Technically the margin of error is half the confidence interval; plus or minus 5 percentage points represents a confidence interval of 10 percentage points

The general public has a basic if vague understanding of this concept. Indeed, media reports of election surveys often report a result “plus or minus” a certain number of percentage points.

The confidence interval is important because it helps us as marketers and researchers understand the limitations of our survey results. The confidence interval estimates the inaccuracy of our results due to “sampling error,” that is, error stemming from the limitation of conducting our survey among a single sample of the population of interest (rather than the impractical or impossible alternative of conducting a census of the entire population).

Sampling error is distinct from other types of survey error – including measurement error, coverage error, and non-response error – but those are topics for another time.

Here are the factors that affect the margin of error:

  • confidence level
  • proportion in the sample
  • sample size

Confidence level.  You must choose how statistically certain you want to be.  The most common confidence level is 95%.  The conceptual meaning of a 95% confidence level is as follows. If you were to conduct your survey one hundred times with randomly drawn samples and everything else were equal, the result of your survey question would be expected to fall within the confidence interval ninety-five of those times and outside it five times.

Proportion in the sample.  Proportional estimates closer to 50% are subject to more variability than estimates near the ends of the spectrum, e.g. 10% or 90%.

Sample size.  The greater the sample size, the lower the margin of error because variability due to sampling anomaly is reduced.

CALCULATING MARGIN OF ERROR

There are three ways to calculate the margin of error:  use a formula, use a look-up table, or use an online calculator.

Use a formula.  There are a number of formulae you can use with slightly varying assumptions.  If you want to go through the calculations yourself using a formula, I refer you to this web page: “Guide to Computing Margins of Error for Percentages and Means” from Professor Ted Goertzel’s at Rutgers University, who explains the calculations better than I can hope to do.

Use a look-up table.  Here’s a table that will be appropriate in most circumstances.  This table is based on a 95% confidence level.  In order to find the confidence interval (the “plus or minus” amount) for a particular proportion, go the the row closest to the proportion of interest and the column closest to the sample size of interest.  For example, if an N=500 election poll showed a race tied at 50% to 50%, you would go to the 50% row and the N=500 column, yielding a margin of error of plus or minus five percentage points.

 N N N N  N 
Proportion 1,000 750 500 250 100
10% 2% 2% 3% 4% 6%
20% 3% 3% 4% 5% 9%
30% 3% 4% 4% 6% 10%
40% 3% 4% 5% 7% 10%
50% 3% 4% 5% 7% 11%
60% 3% 4% 5% 7% 10%
70% 3% 4% 4% 6% 10%
80% 3% 3% 4% 5% 9%
90% 2% 3% 3% 4% 6%

Use an online calculator.  The above exercises are great, but guess what, you’re in luck!  There are many online calculators out there.  Here are two examples:

American Research Group
Relevant Insights

I hope this post is useful as you navigate the world of survey research.  Good luck, and happy polling!

Note:  Table reproduced from The Roper Center at the University of Connecticut.

Filed Under: Featured, How-To, Market Research Tagged With: , ,

The GreenBook Research Industry Trends (GRIT) Study is Now Available

October 6, 2011 By Leave a Comment

Greenbook Research Industry Trends ReportGood news! The Fall 2011 GreenBook Research Industry Trends (GRIT) study is now available.

In addition to key issues that have been trended since the initial 2003 study, new questions on research technology, anticipated staffing characteristics and skill sets, and anticipated changes to marketing research methodologies and business models were asked. Specific probes on influential and/or authoritative industry organizations were also included. The GreenBook staff also investigated spending levels, the overall levels of optimism vs. trepidation, and how the industry perceives and is reacting to change.  Even the moniker “marketing research” itself was a subject of this most recent wave, along with the standard complement of annual GRIT tracking questions.

So go grab your copy. And be sure to share your thoughts and opinions with us on the latest trends. You can add your comments to this post, or find us on Twitter (@researchaccess).

The Fall 2011 GreenBook Research Industry Trends (GRIT) Study

EXECUTIVE version (32 pages)

FULL version (50 pages)

Filed Under: Market Research Tagged With: , , , , , , ,

Generalizing: The Bane of Insights

September 29, 2011 By Leave a Comment

stereotype[Editor's Note: The  following post by Ron Sellers was originally published by and is syndicated with permission by The GreenBook Blog.]

I often wonder whether, in research, we spend so much time navigating the complexities of gathering the data that we neglect the all-important field of communicating what we find.  Issues such as online representativeness, phone response rates, and newer forms of data collection (mobile MR, social media sampling, etc.) take up so much of our mental bandwidth that it can be easy to give short shrift to clarity and accuracy in reporting.

One of the biggest and most potentially toxic issues is generalizing.  Marketers dream about homogeneous populations – segments composed of consumers who are all looking to buy a new minivan, or who all have price as the number one criterion when choosing a cell phone provider.  Because of the lure of homogeneity, it’s very tempting to generalize a segment that shows a greater proportion of certain people as being comprised solely of those people.

Geodemographic clustering falls prey to this quite easily.  When I first learned about this technique a couple of decades ago, I was initially quite impressed that companies could identify clusters of people who were all “upscale Caucasians who are early adopters of technology.”  It was a huge disappointment to find out that this segment, rather than being exclusively comprised of these people, simply contained 20% of these people, rather than the 8% who could be found in the general population (I’m making these numbers up).  Although many purveyors of clustering clearly identify their methodology and how the technique is built, I’ve seen how this process is often used by marketers and researchers.  Rather than discuss a cluster with a higher proportion of the desired target, they discuss the cluster as containing nothing but the desired target.
[Read more...]

Filed Under: Market Research Tagged With: , , ,

The Physical and The Digital

September 23, 2011 By Leave a Comment

Marketing Punters and Pundits alike make stupid predictions.  One of the most laughable of late was the prediction of the early demise of “physical” Marketing- it will all be digital, in the ether, non-corporeal the big-thinkers said.

And like the seers of a “paperless office,” they were dead wrong.

Case in point is a superb piece of Marketing that just crossed my desk today.  The sexy brand Ferrari built their website on Microsoft’s Sharepoint for Internet Sites (FIS) platform, as have thousands of other companies.  Seeing an opportunity, Austin-based Catapult Systems decided to build a Practice out of planning, developing and maintaining websites built on FIS; in order to drive (excuse the pun) the business forward, they decided to do a Marketing campaign leveraging the Ferrari brand in a smart, fun way.

They sent a physical mailer with a nice Ferrari keychain attached.  A VIN number was etched on the keychain.  With simple text, the physical mailer drove (there we go again) folks to a website where one can watch a very simple-yet-compelling video about Sharepoint FIS and how Catapult systems can help companies engage their customers with the Sharepoint technology.  After watching the video, one had to register (with only a few information fields.)

And guess what?  No matter what happens, you get to keep the keychain (and who doesn’t love Ferrari products!)

In one fell swoop, the consumer got a very compelling digital and physical experience.

Now, I’m not trying to stump for Catapult; that’s not my intent or the intent of this blog.  But I do want to heap praise where praise is due.

But more than that, I want to share a bias with you all- I still love physical marketing.  A good giveaway is as compelling as any website.

So throwaway the bathwater but keep that gorgeous and gurgling baby!

[Editor's Note: This post originally appeared on our sister site, MarketingAccess.com]

Filed Under: Marketing Tagged With: , , ,
« Older Posts