5:13pm
Carol Rozwell of Gartner moderated a panel of experts on innovation in sentiment analysis:
- Leslie Barrett of TheLadders
- Bing Liu of the University of Illinois at Chicago
- Romi Mahajan of Metavana
Barrett predicted that sentiment analysis would move more toward free flowing emotion detection.
Mahajan said the two major innovations are the organizational and the philosophical. Organizationally, we will make the vast amount of social data available to all in an organization rather than the traditional “cloistered priesthoods” who control information now. Philosophically, we will move away from serial and linear information toward a constant dialectic.
Liu gave an example of how companies are using social sentiment when considering acquisition targets. He also spoke of how his wife and daughter ask him how social data is useful to them – using the example of gathering information for a mattress purchase.
Mahajan said that even if part of what we are discussing comes to fruition it will be a major change.
An audience member stated that we have not discussed today who the people are whose sentiment we are measuring, and that we need to take this information into account.
Barrett said her organization does take this type of segmentation into acccount. Liu mentioned he has analyzed gender differences. Mahajan said that looking at causation is a tall order; rather, we should be satisfied with strong associations.
Liu mentioned that sentiment analysis can make it easier to process large amounts of information; for example, who wants to read every Amazon.com comment about a product?
An audience member asked about government use of sentiment analysis data. Mahajan mentioned that it would be useful for national security purposes. Another audience member who is a professor at George Washington University in Washington, DC said he is aware that the Defense Intelligence Agency makes use of some type of sentiment analysis.
Barrett said she believes technology will not so much put people out of business but rather create additive business opportunity. Mahajan added that changes in technology lead to the demise of some types of jobs and the rise of otehrs.
Mahajan said we would be wise to plan for how to deal with “blowback” by opponents of using sentiment analysis.
Barrett said each of us should go back to our organizations and look at our data and figure out at least one thing we will do differently based on what we have learned today.
4:29pm
Ronen Feldman of Hebrew University and Digital Trowel talked about his company’s solution called Visual Care. He stated the benefits include reduced development time and increased accuracy.
4:17pm
Kevin Cocco of SproutLoop talked about the way his company uses crowdsourcing to categorize data from the Twitter and Google API feeds. He compared the crowdsourced predictions to those of the Google Prediction API.
The Google Prediction API is ppor for batch predictions and model tuning; it is good for real-time data, sccaling and easy integration. The crowdsourcing model works well when human agree; however, in this experiment they agreed only 44% of the time. Cocco concludes tweet sentiment analysis can be confusing for humans.
4:06pm
Zack Kass of CrowdFlower said that sentiment analysis has been reduced to “PNN” (positive, negative, neutral), machines can’t even accurately define PNN, and crowdsourcing provides rich feedback. He presented an analysis of Radian6 data about former U.S. presidential candidate Herman Cain which was 27% accurate according to human audit. His company’s performance of the same task had 94% accuracy, he stated.
First they ask the human coder whether the piece of data is relevant. Then they rate the sentiment polarity relative to the target. Then they code options for finer gradations of meaning.
3:53pm
Max Yankelevich of CrowdControl discussed cognitive surplus and crowdsourcing. He spoke of ideas from the book Cognitive Surplus by Clay Chirky.
We have a lot of cognitive surplus, as evidenced by the following jusxtaposed facts:
- 200 billion hours per year spent watching TV by US adults
- 100 million hours to create Wikipedia
Crowdsourcing can be thought of as “crowd computing.”
Challenges with getting things done with cognitive surplus:
- complex tasks
- accuracy rate
- lack of attention
- lack of commitment
Yankelevich advocates combining artificial intellicenge (for the computer component) and crowdsourcing (for the human element).
3:19pm
Seth McGuire of Gnip said it’s not just about Twitter. Gnip aggregates and sells social media data across multiple platforms.
McGuire asked what is the right combination of data, that is, what is the right social cocktail?
There were two key dimensions they discovered:
- Reaction time – Twitter, Facebook and Google+ were faster, while WordPress, DISQUS and IntenseDebate were slower.
- Depth – Deeper platforms were YouTube, Tumblr and Flickr; more concise networks were Twitter, Facebook and Google+
Public Relations and supply chain professionals need faster, more concise data. Product development and brand mangagement professionals need deeper, not necesarily faster data.
3:13pm
Jeff Catlin of Lexalytics gave examples of tweets his company trained its sentiment engine to “solve” which are easier for humans to understand:
- Citigroup allows leniency for victims of foreclosure
- I loveeeeeee my evo
- I have an iPhone, but I am not really feeling very happy about my iPhone
- In my opinion right now, Apple is making money on a smart marketing strategy
Here are some they have not been able to solve:
- It was awesome – for the week that it worked.
- i thought i saw a previous for that on mtv movie awards which was a joke
- I don’t get why they call it the droid incredible
- That backflip was so sick
2:50pm
Frank Cotignola of Kraft Foods said sentiment analysis is not at all ingrained in market research. He asked whether we are truly listening. It is a cultural shift in how we interact with consumers. This is a difficult change.
A big mistake some make is just listen to what is being said about brands. However, people often do not talk about brands. The better approach is to listen to what consumers are saying; then see where brands fit into the conversation.
Common objections to sentiment analysis include:
- not representative
- missing demographics
- not my consumers
- too much to read
- no time
- not what I’m used to
A way to convince people of the utility of sentiment analysis is to give examples.
One example is the ability to predict questions around the economy. Typically we look at traditional economic measures. What if we used social media to assess the economy. What is the online sentiment about things like gas prices, unemployment and food prices. You can also look at search data – for example, searches on the term “unemployment” track the unemployment rate (presumably people are looking for information about benefits).
2:43pm
Sobhan Hota of Fidelity Investments discussed how his company uses sentiment analysis of their Voice of the Customer data for direct customer outreach, identifying influential customers, and customer retention. They do coding and analysis of data that identifies the top positive and the top negative words and phrases.
2:33pm
Ryan Sager of the Wall Street journal discussed their ongoing series of sentiment analysis data presented in their weekend newspaper under the title “Sentiment Tracker: A Computational Analysis of the Conversation on Social Networks.” He gave an example of an infographic they published analyzing the reaction on Twitter and Facebook to Tim Tebow becoming a member of the New York Jets football team.
2:24pm
David Nadeau of Media Miser discussed cross-lingual media sentiment analysis. Possible solutions to the cross-lingual challenge are: creating a system for each language or applying the same system after machine translation.
They did an experiment comparing:
- English sentiment analysis on French texts
- French sentiment analysis on French texts
- English sentiment analysis on machine translated French texts
The machine translation approach worked best. Further analysis showed that combining approaches worked best.
2:19pm
Michael Tupanjanin said his company Metavana has come up with a scientific breakthrough that will wipe the slate clean. It is a break from Natural Language Processing. They apply the principles of Chaos Theory to sentiment analysis. He said their algorithm has very high accuracy and is automated.
In the past sentiment analysis has been labor intensive, with low accuracy rates and heavy in professional services. There is now an historic business opportunity because of the explosion of the social web.
2:12pm
Andera Gadeib of Dialego AG talked about her company’s process of online ideation followed by classification. Gadeib starts with the divergent – creating ideas, followed by the convergent, adding layers of complexity to the analysis.
They created an ontology of areas addressed by sentiment analysis:
- emotional
- advertising
- person
- product
- action
- location
- brand
- time
- functional
The process of divergence includes concept testing, co-creation and crowdsourcing.
Measuring emotions is important for communications, product development and more. Gadeib gave a case study for a vacuum cleaner product; they found more emotion in this space than they expected. Blogs yielded more positive emotion and engagement; Twitter had more general content and skewed more negative.
They look at sentiment focus over time in a graphic they call the “long tail.”
12:05pm
Srini Bharadwaj of RAGE Frameworks talked about his company’s provision of its “Real Time Intelligence” product to enable a major financial institution to monitor of borrowers globally. He also gave a case study of the use of the same product by a pharmaceutical company to monitor drug safety and competitive activity.
11:55am
Catherine Van Zuylen of Attensity said the growth of social media has led to renewed interest in sentiment analysis. Sentiment analysis used to be more simple. But there is a change in what is meant by sentiment analysis.
Relative sentiment: “I bought an iPhone” is positive for Apple but negative for Apple’s competitors.
Also, negative sentiment is not always bad; Sarah Palin’s sentiment ratings were very negative when she hosted the Today Show; however, the the TV ratings for that show were very high.
Compound sentiment example using a tweet about the TV show Mad Men: “I love the show but hate the misleading episode trailers.”
Another trend is ambiguity and new uses for negative words. For example: “hate” is positive when used in the phrase “I hate to see her cry.”
It is also important to take emoticons into account; and emoticons vary culturally.
It is important for your team to be on the same page with respect to the definition of sentiment analysis and its specific operationalization.
11:44am
Banafsheh Ghassemi of the American Red Cross talked about her organization’s brand – which she described as one of the most recognized in the world. Its brand value is twice that of most American non-profits. They are also leaders in mobile text donations. They are also strong in social media. They have a partnership with Dell and Radian6 to track reaction to large-scale disasters in real time. They also have a growing mobile app presence with a focus on first aid and disaster response.
The number of charities has increased by 60% in the past decade. The Red Cross is a strong brand, but it still needs to win hearts, minds and dollars. Traditional advertising such as television spots is less effective than it used to be, and in this domain, the influence of friends and relatives is more influential. Influencers have big megaphones via social media, and they have the multiplier effect on their side. Ghassemi mentioned the recent Susan G. Komen controversy as a negative example of the effect of this multipler effect on a charitable organization.
The American Red Cross cares about the experience people have at touch points, change in sentiment, and executive visibility to systemic issues and investment prioritization.
It’s not just Twitter and Facebook. Yelp, for example, has user feedback on blood-giving touchpoints.
Advantages of analyzing social data are:
- real-time feedback
- it is a leading indicator
- competitive intelligence
- best practices
Opportunities with social data include:
- outreach (particularly youth and minorities)
- new policies
- product ideation
- process ideation
The death of the survey is overrated. Surveys give the American Red Cross lots of detailed feedback.
Beware of channel bias – different data sources tend to yield different flavors of data.
Be segment-appropriate. “Red Bull is not Red Cross.”
Beward of “Google Translate Syndrome” – sentiment platforms can lead to machine-applied incorrect information. In a recent disaster response, only 26% of positive comments were coded as positive in a sentiment platform (as compared to live coders).
Take a balanced approach, and do not lose sight of your traditional channels as you explore new ones.
11:15am
Chris Frank of American Express and Paul Mangone of Opnet Telecom are the authors of “Drinking from the Fire Hose.” They discussed their approach to online sentiment.
They apply the concept of the election “swing voter” to that of sentiment. Who are the people with neutral sentiment, who have the opportunity to move either in a positive or a negative direction. Which are the neutrals that lean in either a positive or a negative direction.
Frank and Mangone outlined a taxonomy of increasing involvement with a brand online. The steps, in order, are:
- Like it
- Know it
- Buy it
- Advocacy
Influence = power x platform
Power is derived in three ways:
- positional
- expert
- informational
They showed an “influence map” with platform (relevance, reach and amplitude) on the y axis and power (positional, expert, informational) on the x axis.
10:17am
Richard Brown of Thomson Reuters discussed his company’s provision of “news analytics” to financial markets in order to predict equity movements.
They provide 82 fields of data on all manner of financial news items, including:
- time stamp
- company identifier
- attribute
- type
- genre
- headline
- relevance
- sentiment
- degree of positive, negative or neutral
- first mention of the company
- topic codes
They are also coming out with something called Market Response Indicators which apply machine learning to determine which of the 82 fields are the most important at the stock, market and sector levels.
Thomson Reuters is now plugging in social media data. Markets are now more automated, and traditional analysts are doing what quants used to do.
You have to have big data in your business plan – forget it if you don’t.
Thomson Reuters is planning to take the problem of big data and turn it into an opportunity. They compare signals from internet news and social media output to signals generated from premium news (Reuters).
9:52am
Carol Haney of Toluna described text analysis as looking for the right needles in the haystack. She also noted that much of the data is negative in nature.
There is quite a lot of noise when selecting the data to analyze. It is important to gain an understanding of whether particular information is applicable.
Planning up front is important when embarking on an analysis. The steps are:
- plan your analysis
- harvest the data
- structure and understand the data
- validate the data with a quantitative survey
Haney noted it is important to weight to census rep and use a quality panel. Also, where to scrape depends on where you are in the world.
She presented a case study about Victoria’s Secret’s Dream Angels and Pink brands. Data were harvested from Facebook, Twitter and blogs. Clustering was used to identify and remove promotions. Then a classification scheme of brand and style was created.
Data were very domain specific. For example, the word “ass” less negative in this context because the product is underwear.
Haney also only looked at stronger setiment.
Issues identified from the analysis were thus:
- 2% said stores are not carrying the right size in swimwear
- 7% said Victoria’s Secret isn’t addressing the needs of women outside the 18-24 age group
- 1% said Victoria’s Secret merchandise is ugly
Haney then validated the comments about carrying the right size by conducting a survey of VS customers. Seventeen percent agreed about need to carry bigger sizes in store.
9:22am
Professor Jan Wiebe of the University of Pittsburgh described the process of “supervised machine learning,” as part of Natural Language Processing.
In this process there is a set of training data which is analyzed to create a learning algorithm. That algorithm is then applied to a set of data for which predictions are made for labeling the text sentiment.
Disadvantages to supervised machine learning are that it is expensive and time consuming to create training data expensive and time consuming.
Further, the meanings of words are domain dependent. Performance of machine learnning suffers when training and test data come frome different domains. Cross-domain sentiment analysis methods can help increase accuracy when analyzing data across domains.
Wiebe also discussed “sense level processing.” Senses are different meanings of words depending on context. Many words have multiple senses – for example, “interest,” “alarm” and “trust.” Further some senses of words are opinion-bearing while others are non-opinion bearing. When analyzing sentiment, non-opinion bearing senses are false hits. Data show that simply analyzing opinion polarity rather than each specific sense of a word can lead to higher accuracy.
Wiebe also described data acquisition, including the use of data annotators through the Amazon Mechanical Turk (AMT) service of Amazon.com. Data show that expert annotators perform better over time than those contracted through AMT.
“Active learning” is a process that can be used to reduce the amount of training data needed to train reliable systems. The most informative, least redundant data are analyzed first, then les efficient data are analyzed, and the analyst iterates through more data until satisfied.
