Social networks: predicting photo identities based on social relationships

By December 05, 2013
jeune femme recherchant des personnes sur internet avec des photos

Researchers in Canada have come up with a new algorithm that can predict the identity of people pictured in photos posted on online networks by analyzing the relationships between other people – or even objects – shown in the photos.


Nowadays social networks play an increasingly important role in everyone’s lives and photo sharing has become a favorite social network activity. Now two researchers at the University of Toronto have developed a more efficient means of finding people and photos in an online search, which also enables you to visualize the social relationships between people in a highly representative way. The algorithm they have developed is basically able to predict with a high degree of accuracy the identity of an ‘untagged’ person in a given photo, by analyzing his/her relationship with other people in the photo. This new approach – which you might describe as a sort of social network for ‘tags’ – may well change the way we look for photos and people among the billions of images posted on social networks such as Facebook and Flickr.

Identification algorithm based on tags

The algorithm, which the inventors have dubbed ‘relational social image search’, has achieved high reliability without using computationally intensive object- or facial-recognition software. It is simply based on the number of ‘tags’, not on the number of photos, which makes it more efficient than standard approaches to search. The new search tool uses localization tags to ‘quantify’ relationships between individuals shown, even those not tagged in any given photo. The algorithm can for instance predict that the third person in a photo of a couple and another person is in fact their offspring. This supposes that this third person has previously been identified and that there is a sufficient volume of tagged images of the people concerned to enable the algorithm to precisely weigh the close relationship links in a given group. With its ability to recognize relationships between people, the algorithm provides an overview of the ties and close connections between people, based on photos illustrating moments they have spent together.

New way of determining relationships between people

The basic idea behind the new tool is that the easiest way to find images and people online is through an understanding of their relationships. Professor Parham Aarabi, of the Edward S. Rogers Sr. Department of Electrical & Computer Engineering at the University of Toronto, who developed the technology together with one of his former Master’s students, envisages the interface being used as a search tool such as that used on Facebook, but with better results. Ever-increasing numbers of photos are circulating on the social networks and current ways of identifying photos are unsatisfactory and time-consuming. This is why Facebook has put such a lot of effort into creating facial recognition software, so that people can be identified automatically with very little intervention on the part of the user. The main added value of the Toronto algorithm is that, in addition to being a more efficient person-search tool, it can actually construct a sort of social ‘graph’ from photos, providing an accurately representative view of the relationships of each person with others on their social networks. This month, the United States Patent and Trademark Office is due to issue a patent on this technology, which has also received approbation from the National Science and Engineering Research Council of Canada, and is scheduled to be presented at the IEEE (Institute of Electrical and Electronic Engineers) International Symposium on Multimediain Anaheim, California on 10 December.

Legal mentions © L’Atelier BNP Paribas