Survey Tip: Pay Attention to the Details

January 13, 2012 By Leave a Comment

blueprintWhy survey creators need to pay more attention to the details of wording, question types and other matters that not only affect results but also how customers view the company. A recent survey from Sage Software had quite a few issues, and gives me the opportunity to share some pointers.

The survey was for follow up satisfaction after some time with a new version of ACT! Call me a dinosaur, but after experiments with various online services, I still prefer a standalone CRM. Still, this post isn’t really about ACT! – I’m just giving a little background to set the stage.

  • The survey title is ACT! Pro 2012 Customer Satisfaction Survey. Yet one of the questions asks the survey taker to compare ACT 2011 with previous versions. How dumb does this look?

  • This same question has a text box for additional comments. The box is too small to be of much use, but also the box can’t be filled with text. All the text boxes in the survey have the the same problem.

  • If you have a question that should be multiple choice, set it up correctly.

Some survey tools may use radio buttons for multiple choice (not a good idea), but this isn’t one of them. This question should either be reworded along the lines of “Which of these is the most important social networking site you use“, or – probably better – use a multiple choice question type.

  • Keep up to date.

What happened to Quickbooks 2008, or more recent versions? It would have been better to simply have Quickbooks as an option (none of the other products had versions). If the version of Quickbooks was important (I know that integration with Quickbooks is a focus for Sage) then a follow up with the date/version would work, and would make the main question shorter.

There were a couple of questions about importance and performance for various features. I could nitpick the importance question (more explanation about the features or an option something like “I don’t know what this is” would have been nice), but my real issue is with the performance question. 20 different features were included in both importance and performance. That’s a lot to keep in mind, so it’s good to try to make the survey taker’s life easier by keeping the order consistent between importance and performance. The problem was that the order of the performance list didn’t match the first. I thought at first that the lists were both randomized separately, instead of randomizing the first list and using the same order for the second. This is a common mistake, and sometimes the survey software doesn’t support doing it the right way. But after trying the survey again, I discovered the problem was that both lists were fixed orders, different between importance and performance. Be consistent. Note, if your scales are short enough, and if you don’t have a problem with the survey taker adjusting their responses as they think about performance and importance together (that’s a topic of debate among researchers) you might consider showing importance and performance together for each option. QuestionPro and Survey Analytics have a special question type just for this.

  • Keep up to date – really! The survey asked whether I used a mobile computing device such as a smartphone. But the next question asked about the operating system for the smartphone without including Android. Unbelievable!

There were a few other problems that I noted, but they are more related to my knowledge of the product and Sage’s stated directions. But similar issues to those above occur on a wide variety of surveys. Overall, I score this survey 5 out of 10.

These issues make me as a customer wonder about the competence of the people at Sage. A satisfaction survey is designed to learn about customers, but should also create the opportunity to make the customers feel better about the product and the company. However, if you don’t pay attention to the details you may do more harm than good.

Idiosyncratically,
Mike Pritchard

[Editor's Note:  This post was originally published on 5circles.com]

Photo Credit

Filed Under: Featured, How-To, Market Research Tagged With: , ,

We’re Here, We’re QR, Get Used to It

December 16, 2011 By Leave a Comment
Research Access QR Code

QR Code for ResearchAccess.com

On a recent United Airlines flight I was reading Hemispheres, the in-flight magazine – something I am wont to do as I wait for the go-ahead to turn on my iPad.

As I scanned the magazine (and read an interesting interview with comedian Amy Poehler, by the way), I noticed quite a few QR codes in the magazine’s print advertisements.

Ah, QR codes.  You’re familiar with those little things, right?

No? Don’t feel bad. A smart person familiar with my work with QR codes in the context of mobile survey apps recently asked me what they are.

QR stands for “quick response.”  QR codes are patterns that can be scanned using a mobile device using a QR code reader app.  Many free readers are available, and increasingly QR code reader capabilities are being baked into other apps.

Wikipedia describes QR codes as “a type of matrix barcode (or two-dimensional code) first designed for the automotive industry. More recently, the system has become popular outside of the industry due to its fast readability and comparatively large storage capacity. The code consists of black modules arranged in a square pattern on a white background. The information encoded can be made up of any kind of data.”

The scanning of the QR code triggers an action in the mobile device. Typically, the browser opens to a specific URL, or the app store/market opens to the download page for a particular app.

Essentially the QR code makes it easier for a mobile user to take a desired action.  No more writing down URLs on slips of paper that end up in the wash.

There is a wide range of opinions about the QR code phenomenon and whether it will last.

I don’t have quantitative evidence, but anecdotally I’ve been seeing QR codes in a lot more places lately.

In Hemispheres, I counted fifteen separate QR codes in print advertisements.  Here are the diverse advertisers whose ads had the codes in this single issue:

  • Bose
  • Chancellor University
  • Dial 7 Car and Limousine Service
  • Embry-Little Aeronautical University
  • Hemispheres iPad app
  • K2 “Rolling Stones” Edition Skis
  • Mammoth Lake Resort
  • Parallels Software
  • Riedel Wine Glasses
  • Septodont Dental Products
  • Sheldon Gate Jewelry Designs
  • Solmar Hotels & Resorts
  • Sportube Ski & Snowboard Transportation Cases
  • Texas Center for Cosmetics and Implant Dentistry
  • United Mileage Plus

Fliers are a tech-savvy audience of course, but that still seems to me to be a lot of ads with QR codes.

We could be seeing QR codes a lot more.  Indeed, Microsoft recently started offering QR codes as part of its Microsoft Tag program, “allowing marketers or small businesses to direct people in the physical world to more information.”

Survey Analytics is one of a number of companies in the research space that incorporates QR codes into its technology.  Research Access recently posted a case study of a Survey Analytics mobile panel using QR codes called the Ferry Riders Opinion Group (F.R.O.G.) for Washington State Ferries.

So to finish my story – right when I got off the United flight I saw a poster exhorting me to participate in a United Airlines customer survey, complete with a URL, and…no QR code.

Cue the record scratch sound effect.

But I’m not discouraged.  It won’t be long before customer survey posters without QR codes will be relics.

Filed Under: Featured, Market Research, Mobile Tagged With: , , ,

4 Kinds of Survey Error: Sampling, Measurement, Coverage and Non-Response

November 24, 2011 By Leave a Comment

family feud error

There are 4 generally-accepted types of survey error.  By survey error, I mean factors which reduce the accuracy of a survey estimate.

It’s important to keep each type of survey error in mind when designing, executing and interpreting surveys.  However, I suspect some of them are more ingrained in our thinking about research, while others are more often neglected.

Imagine if we interviewed 100 researchers and asked each of them (“Family Feud”-style) to name a type of survey error.

Which type of survey error do you think would be mentioned most frequently?  Which type would be most overlooked?

Here is my predicted order of finish in our hypothetical example.

Note for the “Feud”-challenged:  Number 1 represents the most commonly named type of error in our hypothetical survey of researchers, while number 4 represents the least commonly named.

1. Sampling Error.

My guess is that sampling error would be the most commonly named type of survey error.

In a recent Research Access post, “How to Plus or Minus: Understand and Calculate the Margin of Error,” I explained the concept of sampling error and gave 3 ways of calculating it.

Sampling error is essentially the degree to which a survey statistic differs from its “true” value due to the fact that the survey was conducted among only one of many possible survey samples.  It is a degree of uncertainty that we are willing to live with.  Even most non-researchers have a basic understanding, or at least awareness, of sampling error due to the media’s reference to the “margin of error” when reporting public survey results.

2. Measurement Error.  

I believe measurement error would be the second most frequently named type of error.  Measurement error is the degree to which a survey statistic differs from its “true” value due to imperfections in the way the statistic is collected.  The most common type of measurement error is one researchers deal with on a daily basis:  poor question wording, with faulty assumptions and imperfect scales.

3. Coverage Error.

Coverage error is another important source of variability in survey statistics; it is the degree to which statistics are off due to the fact that the sample used does not properly represent the underlying population being measured.

There was generally more concern about coverage error in the past; these days, the combination of increasing internet penetration and fast/easy/cheap online survey panels has made it possible to accurately represent many target populations.  Concern about coverage error is still an important conversation; however, it is being discussed more in academic and thought-leadership circles than by the average day-t0-day research practitioner.

4. Non-Response Error.

My guess is that non-response error would be the least named type of error in our hypothetical survey.  Telephone survey houses historically have routinely made 20 or more call-backs to households that do not answer the telephone.  This practice has dwindled due to a combination of the expense of conducting so many call-backs, and the dramatic growth of online surveys, where it is just easier to replace non-responders with fresh sample.  It is also not considered acceptable in an online context to conduct scores of follow-up emails; that would get the sender sent to a blacklist post haste.

Filed Under: Featured, How-To, Market Research Tagged With: , ,

Meet the Data Triplets: Data, Metadata and Paradata

November 23, 2011 By Leave a Comment

tripletsThere are three sorts of data, and very often you need all three to understand and use the data you collect from your survey.

Here are the three sorts:

1. The Data.

The data is the data, that is the actually numbers, codes or open ended text that the respondent enters into the survey. There should probably be a better way of describing this than “the data.”, maybe raw data is a better term to use ?

2. The Metadata

Metadata describes the raw data.

For instance the raw data for question 2 may be a “1” or a “2”. The metadata would say that the value “1” means male and the value “2” means female.  Or the metadata may say that the values for question 3 could be a range of 1-100, or 32-78.

Metadata gives meaning to the raw data, and so it is vital to the analysis process of the raw data that the metadata is present.  Otherwise the raw data is just a collection of numbers with no meaning.

One of the problems with metadata is keeping it connected to the right raw data. The wrong metadata with raw data can be a disaster.

Over the years data formats have got more complex, and one of the big reasons is to keep the metadata with the data. More recent data exchange formats/protocols such as JSON (Javascript Object Notation for the technically minded) have capabilities for attaching metadata to the data, which is a very good thing.

Raw data with no metadata is just a load of junk.

3. The Paradata

Paradata is the least well known of the data triplets. In the past decade or so it has become much more important for the survey research world.

Paradata is data which describes something about the way the raw data was collected.

It is data about data.

The most commonly used form of paradata used at the moment is data about questionnaire and question timings. That is, the time a respondent takes to complete a question or questionnaire.

This type of data is now one of the cornerstones of quality measurements for web surveys.

Obviously there can be many different sorts of paradata. For open ended text questions the length of text entered by the respondent can be measure, as well as the “level of vocabulary” contained in the text.

One metric used for web surveys is that of “speeders,” that is, the number of people who complete the survey extremely quickly. The paradata for time take to complete the questionnaire is used here.

Paradata can also be useful in revealing hidden biases; for instance, using paradata in the gamificaton of surveys is a rising trend. The time taken to do something in a gamified survey as well the action can have a great deal of meaning. Some researchers claim that hidden racism, some times unknown to the subject themselves, can be revealed by measuring someone’s reaction time to specific questions.

In a future post we will delve more into exactly how paradata can be used for quality control of web surveys.

Filed Under: Featured, How-To, Market Research Tagged With: , ,

Liquor Privatization Initiative Accurately Pegged by Pre-Election Online Survey

November 21, 2011 By Leave a Comment

liquorSurvey Analytics is pleased to report that our recent poll of King County, Washington voters called the outcome of the State of Washington’s liquor privatization initiative with a high degree of precision.

Our political poll marks an exciting and innovative, new approach whereby public opinion researchers, public affairs firms, political consultants and political campaigns themselves can cost-effectively and efficiently take the pulse of the electorate.

1183

Back in late October through early November, we invited likely voters residing in King County and Seattle to weigh in on various ballot measures, candidates and other matters. One of the most prominent issues on the November 8 General Election ballot was Initiative 1183, which will privatize the sales and distribution of liquor.

A total of 2,001 likely King County voters took part in our survey. When asked how they would vote “if the election were held today,” 61% said yes and 33% said no, with 6% undecided. In the actual election results, 60% of ballots cast voted for the initiative and 40% against. A match-back analysis of the survey sample suggests that those who participated were closely representative of the King County electorate, in terms of party affiliation, gender and age.

Liquor Chart

Unlike full-blown telephone surveys typically used by pollsters, our unique approach can be fielded within minutes and produce meaningful results within hours. Complete cross-tabulation data and topline results are available immediately. A complete analysis of our survey and its results is coming soon.

Note: For more information on this survey, check out this post on the SurveyAnalytics Blog:  ”Voter Panels – a real-world application in predicting outcomes of voter initiatives.”  

Filed Under: Government, Innovation, Market Research Tagged With: , , , ,

Now We Have Smartphones; Shouldn’t We Try to Be Smarter about Surveys?

November 16, 2011 By Leave a Comment

This is a presentation from Survey Analytics‘ President, Andrew Jeavons, from the Market Research in the Mobile World Conference in Atlanta in July 2011.

(By the way, it’s pronounced JEH-vons, not JEE-vons, as Andrew explains in the video…)

Andrew presented guidelines for conducting mobile surveys, and he made suggestions for adapting Net Promoter Scores (NPS) in a mobile survey environment.

Enjoy!

Filed Under: Featured, Featured Video, How-To, Mobile, Video Tagged With: , , , , , ,

How to Use Facebook for Market Research Surveys

November 15, 2011 By 1 Comment

It’s an understatement to say that there’s tremendous interest in using Facebook for market research.  Indeed, among the most popular posts on Research Access is one written last year by Survey Analytics‘ CEO Vivek Bhaskaran, entitled “Social Media Research – Using Facebook for Survey Invitations and Market Research.”

What not everybody realizes is that companies are using the power of Facebook’s large audience to conduct research every day.

While Facebook-fueled surveys are not right for every situation, they can be extremely powerful in the right circumstance.  The biggest advantage is access to a massive audience of people who do not normally complete surveys.  However, even Facebook’s large audience will not necessarily yield a sample from the target audience you are trying to reach.  In addition, sampling through Facebook Ads can be expensive, depending on the particulars of your study.

Since Vivek wrote his Facebook sampling post last year, there have been many changes to Facebook, but the fundamental principle outlined in that post still holds true.   So it’s time for an update.

Also, I will explain how to use company or brand fan pages to get valuable feedback.

1) Use Facebook Pages to Reach Your Customers and Fans.

You can ask followers of your company or brand fan page (or your personal page, for that matter) to provide feedback in several ways.

  • Post an open-ended question asking for direct feedback.  For example, “We are looking for feedback on Research Access’ new look and feel.  What do you think?”   You can add language encouraging people to post their comments on Facebook, or you can give an email address for them to contact you directly.  The feedback you receive will be useful but will not be generalizable to all customers or fans.
  • Post a poll.  Facebook now has a “Question” option in the status update box allowing you to post a poll to your fans.  Please note: you can only do one question at a time, and the results will be visible to all fans.  Interestingly, there is an option to allow your fans to add responses which you didn’t necessarily consider when creating your question.

Ask a Question

  • Post a link to a survey.  Instead of using Facebook’s built-in question function, you can simply share a link to a survey.  You should also include explanatory text in the post.  Here’s a hypothetical example Research Access could use: “Please take 5 minutes to give us feedback on Research Access’ new look and feel. Everyone who completes the survey will receive a free eBook copy of QuestionPro for Dummies.”
Post a Link

2) Use Facebook Ads to Reach a Wider Audience.

Using Facebook Ads, you can open your survey up to a massive audience which can be targeted in very specific ways.  Here are the steps for directing Facebook users to your survey using Facebook.

  • Start creating a Facebook by clicking the “Create an Ad” link in the “Sponsored” section in the right-hand column of your page.
Create an Ad
  • Create an ad with an image and a message that will drive the right type of traffic and redirect those who click on the ad to an externally hosted survey.  Select “External URL” in the “Destination” drop-down list.  Put your custom survey URL in the “URL” field.  Use the “Title” and “Body” fields to create a compelling call-to-action for survey-takers.  Be sure to include an image that will garner attention.  In the “Targeting” section, you can target your survey by geography, age, specific interests and more.
  • Define your budget and schedule.  With Facebook Ads you have a great deal of control over your ad’s schedule.  Importantly, you can define a daily budget which will not be exceeded.
  • Finally, preview your ad, then start your campaign!  Good luck.
Filed Under: How-To, Market Research, Social Media Tagged With: , , , ,

Election Polls: 5 Tips For Navigating the Clutter

November 7, 2011 By Leave a Comment

vote buttonTomorrow is Election Day here in the States.  The big vote (for President) isn’t until next year, but we have the usual spate of races for local offices as well as a wide range of citizen-initiated referenda on everything from the mundane (bond measures) to the highly divisive (social issues).

As the expected avalanche of survey results washes over me and everybody else (accompanied by a barrage of television advertising, internet electioneering, and attack mailers), I thought I’d share some tips for understanding which poll results to heed, and which to take with the proverbial grain of salt.

My experience in the world of election polling has given me a bit of insight into this topic; I hope these tips help you find your way on your journey through the sea of election polling data.

1. Understand the Methodology

The best way to judge an election poll – indeed, any survey – is to have a good understanding of the study’s methodology.  Unfortunately, reporting on survey methodology is often woefully inadequate, scarcely going beyond a reporting of the margin of error.  However, sometimes one can read between the lines of a story to gain a better understanding of the circumstances under which the poll was conducted, including: timing, data collection mode and survey length.

If the methodology is completely unclear from the article, be a good citizen and email the editor requesting clarification.  The rest of us will thank you!

2. Think Random

Generally speaking, in election surveys, relative to market research surveys, it is particularly important to have sampling methods that give all likely voters as equal as possible an opportunity to be surveyed.  Look for efforts to ensure a representative sample, including:

- surveying voters at different times of day and on different days of the week
- compensating for sampling limitations such as telephone coverage, cell phone coverage, and internet access
- using a “likely voter” screening question, ensuring only those both registered and likely to vote are considered.

Also, be wary of  polls conducted using automated telephone interviews rather than trained interviewers.

3. Evaluate the News Source

As good a data consumer as you are, you cannot realistically check the fine details of methodology on every study.  Therefore, pay attention to the reputation and track record of both the publishing entity.

More reputable organizations tend to higher publishing standards than less reputable ones; they have more to lose if they publish bad reporting.  Put more stock in a study reported by the Washington Post or the Pew Research Center than a study sponsored by a smaller or less reputable newspaper, website, non-profit organization.

In local elections, sometimes the most reputable (though hardly infallible) source is the state or locality’s largest newspaper.  However, look for critical analysis rather than simple reporting of results.

4. Pay Attention to the Sponsor.  Often a group with a vested interest in an election will privately commission a survey to be used for internal strategy; however, if some of the results support their public relations efforts, they will release an (often-misleading) subset of the data in order to influence the electorate.  Sometimes entire carefully-worded polls will be conducted which are meant for public release.  Be very skeptical of any data paid for by a group or persons with an interest in the election’s outcome.

5. Learn from the Experts.  There is a wealth of great content created by smart people who focus on analyzing election data.  Take advantage of their wisdom!  I’m partial to HuffPost Pollster (formerly Pollster.com) – check out my friends Mark Blumenthal (a.k.a. “@MysteryPollster“) and Margie Omero – and Nate Silver’s Five Thirty Eight blog at the New York Times.

These resources are primarily focused on U.S. Elections; there are undoubtedly many more good resources out there both for the U.S. and everywhere else; please suggest others in the comments section below.

I hope these tips are useful to you.  And if you’re in the U.S., don’t forget to vote tomorrow!

Filed Under: Government, How-To Tagged With: , ,

How to Plus or Minus: Understand and Calculate the Margin of Error

November 4, 2011 By Leave a Comment

iPhone - Plus or MinusSometimes in the day-to-day work of conducting and interpreting market research, it’s easy to forget that many people who work with surveys on a daily basis have not had formal training in statistics. Even for those who have been trained, it can be useful to have a refresher from time to time.

UNDERSTANDING MARGIN OF ERROR

One of the most basic concepts in market research is the confidence interval, commonly referred to as the “margin of error.”  The confidence interval is a range of values within which a survey result can be assumed to accurately represent the underlying construct being measured.

Technically the margin of error is half the confidence interval; plus or minus 5 percentage points represents a confidence interval of 10 percentage points

The general public has a basic if vague understanding of this concept. Indeed, media reports of election surveys often report a result “plus or minus” a certain number of percentage points.

The confidence interval is important because it helps us as marketers and researchers understand the limitations of our survey results. The confidence interval estimates the inaccuracy of our results due to “sampling error,” that is, error stemming from the limitation of conducting our survey among a single sample of the population of interest (rather than the impractical or impossible alternative of conducting a census of the entire population).

Sampling error is distinct from other types of survey error – including measurement error, coverage error, and non-response error – but those are topics for another time.

Here are the factors that affect the margin of error:

  • confidence level
  • proportion in the sample
  • sample size

Confidence level.  You must choose how statistically certain you want to be.  The most common confidence level is 95%.  The conceptual meaning of a 95% confidence level is as follows. If you were to conduct your survey one hundred times with randomly drawn samples and everything else were equal, the result of your survey question would be expected to fall within the confidence interval ninety-five of those times and outside it five times.

Proportion in the sample.  Proportional estimates closer to 50% are subject to more variability than estimates near the ends of the spectrum, e.g. 10% or 90%.

Sample size.  The greater the sample size, the lower the margin of error because variability due to sampling anomaly is reduced.

CALCULATING MARGIN OF ERROR

There are three ways to calculate the margin of error:  use a formula, use a look-up table, or use an online calculator.

Use a formula.  There are a number of formulae you can use with slightly varying assumptions.  If you want to go through the calculations yourself using a formula, I refer you to this web page: “Guide to Computing Margins of Error for Percentages and Means” from Professor Ted Goertzel’s at Rutgers University, who explains the calculations better than I can hope to do.

Use a look-up table.  Here’s a table that will be appropriate in most circumstances.  This table is based on a 95% confidence level.  In order to find the confidence interval (the “plus or minus” amount) for a particular proportion, go the the row closest to the proportion of interest and the column closest to the sample size of interest.  For example, if an N=500 election poll showed a race tied at 50% to 50%, you would go to the 50% row and the N=500 column, yielding a margin of error of plus or minus five percentage points.

 N N N N  N 
Proportion 1,000 750 500 250 100
10% 2% 2% 3% 4% 6%
20% 3% 3% 4% 5% 9%
30% 3% 4% 4% 6% 10%
40% 3% 4% 5% 7% 10%
50% 3% 4% 5% 7% 11%
60% 3% 4% 5% 7% 10%
70% 3% 4% 4% 6% 10%
80% 3% 3% 4% 5% 9%
90% 2% 3% 3% 4% 6%

Use an online calculator.  The above exercises are great, but guess what, you’re in luck!  There are many online calculators out there.  Here are two examples:

American Research Group
Relevant Insights

I hope this post is useful as you navigate the world of survey research.  Good luck, and happy polling!

Note:  Table reproduced from The Roper Center at the University of Connecticut.

Filed Under: Featured, How-To, Market Research Tagged With: , ,

Fundamentally Speaking: Back To The Basics in Marketing Research

August 22, 2011 By Leave a Comment

In marketing research, we are frequently faced with getting answers faster and for less investment. In the current business climate, we have become all too familiar with the term“doing more with less.” During a recent NY / Philadelphia MRA conference, we heard repeatedly that it is not only the marketing research supplier facing these issues but also client companies and their end users. In the real world, when facing such aforementioned constraints, what might help in delivering on objectives while continuing to provide credible information?

Throughout many aspects of life, we find success by returning to fundamentals. Although getting back to basics sounds obvious, it is often overlooked. When a baseball player experiences a hitting slump, what does he do? He works with a hitting coach to determine if his fundamentals are correct (i.e. stance, swing, mental approach, etc.) The financial markets are no different. Often we read that the financial market needs to turn to fundamentals in order to back on track or grow. CEO’s of large corporations often mention that their respective companies have solid fundamentals. Solid fundamentals translate into success.

In marketing research, returning to fundamentals and applying superior practices can lead to efficiency. Instead of rushing to find answers, we should not lose sight of basics such as what is it we want to learn. The “how we go about it” ought to come later. Simply obtaining information does not serve stakeholders competently.

It is very tempting and easy in the digital world to go online and get information. Digital is built for speed. However, the old adage “garbage in/garbage out” has never been more valid. More information should never be our goal but, instead, meaningful information.Fundamentals in research tie directly back to getting it right the first time. Today there are many resources from which to garner information. Whether we are conducting primary or secondary research, qualitative or quantitative, there are fundamentals that must be deployed in order to capture findings that meet our goals. “Back to the basics” comes down to applying the right resource against what we want to learn. The mistake of placing the cart before the horse is not new.

Successful solutions are not solely about research technique but in applying optimal methodology to yield credible outcomes. Our challenge is not to be induced by the speed associated with the latest and/or so called greatest but to leverage wide ranging knowledge in order to get it right. Poor designs associated with problem definition, sampling frame, questionnaire development, execution or analysis will guarantee dire results. Starting with fundamentals gives us a foundation from which to build a successful project.

[Editor's Note: This post originally appeared on Steve Levine's Blog, and is syndicated here with permission.]

Filed Under: Research Base Tagged With: , , , ,
« Older Posts