New Relational Social Image Search Algorithm Finds You Even In Untagged Facebook Photos

A new image search tool developed by Parham Aarabi, a professor in electrical and computer engineering at the University of Toronto and his former student Ron Appel makes spotting you in your untagged Facebook photos possible. So, the next time you tell someone to not tag you in a photo being uploaded to Facebook, because you look the worst in it, there's a chance that someone will still spot you. The new algorithm called the 'Relational Social Image Search' modifies as well as simplifies the way you find photos among the billions of pictures uploaded on social media sites such as Facebook and Flickr.

So how does this trick work? Well, if you and your mother are pictured together in one photo while you and your father in another. All of you are tagged in the photos. Now, because of your close 'tagging' relationship with both your mother in the first picture and your father in the second, the algorithm can determine that a relationship exists between those two and quantify how strong it may be. In a third photo, you fly a kite with both parents, but only your mother is tagged. Given the strength of your 'tagging' relationship with your parents, when you search for photos of your father the algorithm can return the untagged photo because of the very high likelihood he's pictured.


Simple put, "The location of tags in the images tell us a story." So, what the algorithm does is - it helps you search images better all the while understanding relationships between people and analyzing it. According to the researcher duo, their algorithm achieves high reliability without using computationally intensive object - or facial-recognition software. "If you want to search a trillion photos, normally that takes at least a trillion operations. It's based on the number of photos you have," said Aarabi. He continued, "Facebook has almost half a trillion photos, but a billion users - it's almost a 500 order of magnitude difference. Our algorithm is simply based on the number of tags, not on the number of photos, which makes it more efficient to search than standard approaches."

"I envision the interface would be exactly like you use Facebook search-for users, nothing would change. They would just get better results," said Aarabi. But here is where it gets interesting. In the course of testing their image search algorithm, Aarabi and Appel discovered an unforeseen application: a new way to generate maps. They tagged a few photographs of buildings around the university campus and ran them through the system with a bunch of untagged campus photos. "The result we got was of almost a pseudo-map of the campus from all these photos we had taken, which was very interesting," said Aarabi.

When every new technology that comes along, it does so with a set of limitations. This one relies on the fact that it requires images where at least two or three objects are tagged. Secondly, because the model can find images where users were not tagged by looking at images where they were tagged, there are privacy implications. Now, who would want to be found in a Facebook or Flickr photo they weren't tagged in or chose not to add tag in?