BBC Tomorrow's World: The photo analysis tool

In collaboration with

,
,
,

Recognition

,

The backstory

To hype the launch of BBC’s exciting new digital hub for Tomorrow’s World, Socialyse wanted to target teens and Millenials with future facing technology in a place they felt at home: social media. Knowing this audience understands social trends and cares a lot about the content they share, we created something just for them: a photo analysis tool.

Chatbot as the medium

We decided to develop the tool as a Messenger Chatbot. Such approach has not only increased the reach, but also has been a great way of delivering an asynchronous experience - one where the AI needs a little bit of time to analyse the content and deliver results.

Users would send their pictures to the bot, which would trigger the AI analysis in the background. Meanwhile, the bot would serve them with supporting content, or the user could decide to close Messenger and switch to another task. Once the AI results were ready, the bot would message the user.

Exactly the same experience as when a message from a friend comes.

The results

Rather than just telling the user how many likes their photo will receive, the bot would serve the users with a beautifully animated video that provides a detailed analysis explaining the key components of the photo and how they affect the score.

The AI analysis

There has not been any Machine Learning algorithm that would estimate the number of likes an image will get. So we developed our own.

We started by dissecting the image into key features that are likely to affect an image's social success. Like faces, emotions, landmarks, colors or secondary items visible in the image. To do this, we used Google Cloud Vision API.

AI Training Crowdsourcing

Being able to understand what is visible in the images, we used Amazon Mechanical Turk to crowdsource people's perception of image likeability. We asked them to submit both good and bad pictures, along with a number of likes they think these images would get.

This has allowed us to use Google Tensorflow to develop and train a custom Artificial Neural Network that was able to gain the understanding of how people perceive photos and estimate the number of likes an earlier unseen photo would get.

AI Accuracy

Even though the problem appears to be very subjective, we managed to build an AI that is interestingly accurate. We tested it on images that we already knew the number of likes they achieved in order to compare actual data against the AIs prediction. The correlation between the actual data and the AI exceeded our expectations:

We’ve created an algorithm that understands what makes a photo popular. You can’t understate the significance of this - Mark Vatsel, Creative Director, UNIT9

The photo analysis tool brings millenials that key insight on how make trending content with their own social posts and unpacks a little of the mystery behind how AI works.