Test Your Video For Emotional Reaction

22 Feb 2018 | 10.55 am

Test Your Video For Emotional Reaction

Limerick startup EmotionReader analyses revealing facial expressions

22 Feb 2018 | 10.55 am

Artificial Intelligence is being deployed by two Irish ventures that seek to understand what consumers really feel, writes Gerry Byrne


When University of South Florida professor Robert Plutchik produced his groundbreaking work in the 1950s on the human expression and reception of emotions in, he could not in his wildest dreams have imagined that his innovative Emotion Wheel, a flower-shaped graphic which grades emotions, would one day lead to the evaluation of memes on Facebook.

Shortly before his death in 2006, Plutchik admitted that there were limitations on the interpretation of emotions. Self-reported feelings by individuals are often couched in euphemisms which make up in drama what they lack in clarity. What stage of grief, for example, is described by a ‘broken heart’?

There were arguments that emotions, being inner feelings separate from observable behaviours, were beyond the realm of empirical science. Yet previously unreadable emotions are what a brand new marketing discipline, spliced with a measure of artificial intelligence, is bringing to the table.

Pioneers include Belfast company, Adoreboard, and EmotionReader, a Limerick startup. Both firms are uncovering deeper horizons in human sentiment, attitudes which conventional market research cannot plumb. Their offerings may cause many companies to seriously reconsider their approach to consumer behaviour.

“We know that 80% of consumer motivation is driven by emotion,” explains Adoreboard’s founder and chief executive, Chris Johnston. “But a lot of the traditional tools used to measure it are quite crude, and some of these metrics are pointless because none of them get to the root cause of what is really driving people to make decisions.”

Hidden Nuances

Adoreboard, a spin-out from Queen’s University, works on finding out what emotions people really feel about companies, as opposed to what they say. Text from emails, tweets, Facebook postings and communications with customer relations department are fed into Adoreboard’s proprietary software, which is primed to reveal the nuances hidden by customers’ choice of words.

Emotion analysis goes beyond the quantitative measure of emotion to drive qualitative benefit. As a business application, emotion analysis is an aggregate measure of the emotional intensity of text, a measure of individual emotions, the topics or themes stimulating those emotions, and then the extracted business insights

“The internet and social media have transformed the way people express their opinions, experiences and knowledge,” says Johnston. “This provides an opportunity for brands and organisations to access unsolicited content to gain previously untapped insights.

“We don’t just identify trends that only tell you what ‘could be’ an issue. By defining and identifying feelings expressed, as well as the topics driving those feelings, our analysis drills right down to the cause. We can tell you precisely which key emotions are contributing to the success or the under-performing aspects of your product or service.”

Johnston cites the example of a public transport client that Adoreboard helped to prioritise customer experience improvements. A consensus had formed in the company that issues like overcrowding, smoking on platforms, staff rudeness and punctuality problems would be leading candidates for remedial action. Adoreboard’s evaluation discovered poor WiFi was the greatest factor driving malcontent, picked up from the relative vehemence of the online chatter.

“After a further investment in WiFi connectivity, total customer complaints dropped by 70%,” says Johnston explains. “The company had advertised WiFi heavily and the service or lack of it had a personal impact on passengers. They were becoming more intense in their negative attitudes to the brand.”

Customer Satisfaction

Johnston comes to emotional analysis from a background in reputation management, not algorithms. He has noted how brands increasingly campaign by offering improved consumer satisfaction rather than proffering any unique core product offering.

Banking, transport, and many other sectors, are all being commoditised, he explains, so customer experience is the new battleground.  “The conventional way of measuring that is redundant but we are instead providing a science based technology that allows people to make better decisions,” he says.

Where traditional attitudinal research might have unearthed lukewarm consumer approval for a product innovation, Adoreboard is discovering emotions as extreme as ‘rage’ and ‘disgust’.

“We did a survey of UK and Irish banks and found that what really drove intense annoyance with customers was having their cards blocked when they went abroad. It was taking up to 90 minutes to get a service call answered at roaming rates, and people were tweeting about this.” In other research, golf star Rory McIlroy was found to generate far more positive emotions than the brands that sponsor him.

Adoreboard’s algorithms are a perpetual work in progress and the tech team is now adding emojis and acronyms to the ever growing list of the words and phrases we use to express ourselves, which in turn are emotionally graded by Adoreboard’s constantly evolving iterations of Artificial Intelligence.

Facial Twitches

Broadly speaking, the approach is not dissimilar to the AI-led algorithms generated by Limerick-based EmotionReader although its product offering is more intensely focussed, in more than one sense of the word.

The brainchild of University of Limerick PhD graduates Padraig O’Leary and Stephen Moore, EmotionReader offers an evaluation of TV adverts and corporate videos using computerised analysis of the sometimes microscopic movements of the viewer’s face.

The computerised analysis registers emotions displayed, often imperceptibly, on the viewer’s face. It draws on the work of Professor Paul Ekman, named by the American Psychological Association as one of the most influential psychologists of the 20th century.

Over a long career, the former child prodigy devised an enormous chart of facial expressions to which he mapped a wide range of emotions. Ekman’s findings have been the inspiration for several TV series, as well as a more controversial face mapping application deployed by US Homeland Security in the hope of snaring terrorists.

EmotionReader’s algorithms can detect expressions displayed by testers while watching the screen, and a frame-by-frame analysis side by side with the video determines what emotions are generated by each segment.

“Analytics of video ads at present is very binary stuff,” says O’Leary. “You get numbers of views, likes, and shares etc, but it doesn’t capture the emotional aspects, which is what we provide. Recent advances in AI have meant that hard tasks like training an algorithm to accurately capture facial expressions is easier than before — and highly accurate. We can identify multiple emotions experienced from a 30-second video and then make predictions about its virality.”

EmotionReader’s analysis tracks the moment-by-moment emotional highs and lows of the viewing experience, and then a finding on the total experience of watching the video. Did the viewers pay attention, were they distracted or did they like it and find it interesting?

“Multiple muscles are engaged when laughing, or displaying happiness,” O’Leary explains. “We use mapping to align motion detected in a face with what is happening at that particular point in time. We can analyse demographics too, and inform clients which target group has the best reaction. We will also help clients decide what version of a video to promote, and what channels to use, and where.”

EmotionReader, with funding from Enterprise Ireland, is currently trialling its digital platform in the marketplace and hopes to launch fully next year. “This technology is not cheap to build and we are speaking with potential investors,” says O’Leary. The company’s target market is the US, where $70 billion in broadcast TV advertising has shifted to digital platforms, including social media, where it is targeted at specific audience sectors.

“Video is becoming more important to advertisers and social video generates 1,200% more shares than text and images combined. Using EmotionReader, marketers can allocate their media spend more effectively. We test ads based on emotion and attention, and we want to become a data supplier in that new programmatic buying ecosystem. Emotion is the data we supply.”

Photo: EmotionReader tracks moment-by-moment emotional response to video content


Comments are closed.