Put very simply, sentiment analysis (sometimes known as opinion mining) uses technologies including Natural Language Processing (NLP), text analysis and computational linguistics to identify intent or attitudes. Many of the use cases for sentiment snalysis are to be found in marketing - such as analyzing customer service interactions or for consumer marketing purposes.

How Is Sentiment Analysis Actually Used?

In customer facing situations, sentiment analysis is often used to identify unhappy customers from an email trail or a conversation with a Customer Service Representative (CSR) – and to ensure that these are escalated and dealt with promptly before the situation gets out of hand. It can also be used to understand if particular CSRs tend to generate unusually high amounts of positive (or negative) sentiment; or to identify if a particular stage of a process – either online or a call center – comes in for particular criticism and therefore requires the web site or script to be enhanced.

In a consumer setting, sentiment analysis is often applied to understanding the voice of the customer. This could be in a social media context: in fact, one survey found that just over half (51%) of respondents said they had ‘called out’ a company on social media. Sentiment analysis can therefore serve as an early warning system to identify negative comments that have the potential to go viral and threaten the reputation of your company. It could also be used to distil information from a review site: so, for example, if you have launched an app, you could use sentiment analysis to understand how well or poorly that app had been received. It can also be used just to monitor general sentiment about your brand – both positive and negative.

Is There Anything New About Sentiment Analysis?

Sentiment analysis has been around in one form or another for at least a couple of decades. It started out as a rules-based system that looked for and counted the frequency of particular words that would indicate the sentiment of the customer. These tools were pretty unsophisticated and largely applied only to text-based content.

Over the past five or so years, there has been a huge improvement in natural language understanding and neural networks; and machine learning algorithms have emerged which have changed the game completely. These now look not only at the words being used but at the context in which the words appear – providing  a much more nuanced and more accurate picture of the sentiment behind them.

Today, you can run sentiment analysis on pretty much any type of content – verbal or written. In fact, it has even been used to analyze dreams and investigate bias among journalists.

 

 

How Does Sentiment Analysis Work?

These algorithms have to be ‘trained’ in a process of supervised learning through which they infer rules from pairs of words provided to them, then ascribe a positive, negative or neutral ranking to each pair. How you build those tests is the ‘secret sauce’ – the way in which this supervised learning is provided is what makes the difference between an effective or ineffective algorithm which in turn dictates the accuracy of the sentiment analysis. And of course, the language of words we use to describe, say, a financial service product and a food item are completely different, so the tools might need to be trained up in the vocabulary appropriate to the industry in which they are being applied.

What Are the Limitations of Sentiment Analysis?

It’s not a perfect tool – yet. Sentiment analysis struggles to identify exaggerations, humor, and sarcasm. And any analysis of sentiment is subject to a certain amount of subjectivity – even human beings might come up with different interpretations of individual comments. And while there is a lot of web-based content in English, the same is not true of all languages – if the pool of content for a particular language is shallow, then training the algorithm will be more difficult. All of this might have the effect of reducing the accuracy of the results.

At some point, the algorithms will improve to the point where they can analyze any content with near 100% accuracy – but that’s still some way off.

What’s the Future for Sentiment Analysis?

As we’ve said previously, the tools will increase in sophistication over time – and other applications will sit above them to provide richer analysis of customer sentiment. But we will always need to understand what our customers are saying about us. And sentiment analysis is a good entry point into NLP and other forms of Artificial Intelligence. So, overall, the interest in this topic is likely to remain high for the foreseeable future.

How Can I Start Using Sentiment Analysis in My Own Organization?

If the prospect of training your own algorithm feels like a lot of work, the chances are that you needn’t be worried. Only those working in the most highly specialized fields will have to acquire the skills to do this themselves. For most people, it is perfectly possible to buy ‘off-the-shelf’ sentiment analysis tools that will allow you to start using them immediately.

If you are an OutSystems customer, then you can find a wide range of sentiment analysis tools – including those from third parties and some specific to platforms such as Microsoft Azure – in the Forge (this is a kind of app store for pre-build OutSystems application components).

If you are using the OutSystems platform, the Language Analysis tools in the Forge come with a pre-built wrapper that allows them to be included in your application in only a few clicks. (It is entirely possible to use other third-party tools, but these may require additional integration effort).

Finally, one of the benefits of using the OutSystems platform for sentiment analysis is that you can use these tools in the environment to which you are already accustomed – with no need to hire in any expensive and scarce data science resources. 

To find out more about using sentiment analysis as part of the OutSystems platform, explore our AI Language Analysis offer or watch our recent Tech Talk on OutSystems AI capabilities.