Facial recognition for the public: Yandex

You can use Yandex Image Search right now as a pretty good facial recognition system for anyone who has labelled photos on the Web. I believe this is the first generally accessible facial recognition system with a large database. Yandex isn’t designed for this purpose. The trick is to upload photos cropped to a face and it’ll work more or less to find similar faces.

This post was inspired by Bellingcat’s recent post on image search which noted Yandex’s image search as something particularly good and particularly called out the facial recognition. I’m just elaborating on what he found.

Update: see this article about ClearView, a company doing facial recognition with scraped photos from Facebook, etc. Also this Dec 2019 video reporting about Yandex facial recognition. The Daily Mail also published an article based on this blog post.

Demonstration with a selfie

Here’s a selfie I took today. It’s never been on the web before this post and appears in no databases as I do these searches. This is a good clear photo of me, which makes the results better, but it works with less good photos too.

Now pretend you didn’t know my name. Put the image in to Yandex. Here’s the result you get.

The first two images are me. Click on either of them and the web page it takes you to has my name on it. Now you know who I am!

It’s not perfect; there’s no name provided directly, someone has to read the linked page to find one. And a person still has to sift through the near-matches of faces. But it’s pretty good. The only limit is Yandex’s database. That’s kind of “the whole web”, but their index is nowhere as comprehensive as Google.

Demonstration with Andrew Yang

Recognizing myself is easy. How about a stranger? I used this today to identify who the “Yang” was in this image from CNBC. It’s supposed to be Andrew Yang, the presidential candidate, but it is not.

Crop the image to Yang’s face, put it in Yandex, and you get this

All five of the first images look like our guy. The first result on the Yandex result page also leads directly to his home page: Geoff Yang. Some CNBC intern is gonna be in so much trouble! (This post is not an endorsement of Andrew Yang.)

Facial recognition vs image search

The new thing here is that Yandex is working not just as a reverse image search tool; it seems to be doing facial matching. None of the images it finds look much like the original image; different aspect ratio, different background, different details. But the faces match. It even matches my selfie both to a much younger version of me without a beard and to a version of me holding a box up obscuring part of my face. That’s pretty wild.

FWIW, here’s what other image search engines do with my selfie. I tested Google, Baidu, Bing, and TinEye. TinEye returned 0 results.

Google
Baidu
Bing

Look at all the beardy white guys with some yellow in the image! None of them are me. It’s not a bad guess; I gave it a head shot, it found me similar head shots. But it didn’t find my face. Yandex finds my face. (Arguably TinEye’s 0 results is the right answer for this query; this image does not exist online.)

Why this matters

We’re on the cusp of a major change in day to day privacy with facial recognition algorithms. AI has now gotten good enough that it can do excellent facial recognition; modern software can take a blurry photo of half a face and match it to other faces of the same person. It’s a pretty scary future, particularly as already put into place by authoritarian governments like in Xinjiang, but it is inevitable and unstoppable.

Right now an ordinary person still can’t, for free, take a random photo of a stranger and find the name for him or her. But with Yandex they can. Yandex has been around a long time and is one of the few companies in the world that is competitive to Google. Their index is heavily biased to Eastern European data, but they have enough global data to find me and Andrew Yang.

If you use Google Photos or Facebook you’ve probably encountered their facial recognition. It’s magic, the matching works great. It’s also very limited. Facebook seems to only show you names for faces that people you have some sort of Facebook connection to. Google Photos similarly doesn’t volunteer random names. They could do more; Facebook could match a face to any Facebook user, for instance. But both services seem to have made a deliberate decision not to be a general purpose facial recognition service to identify strangers.

There is at least one public recognition service out there: FindClone (hat tip to Brad F). You can read more about FindClone here. (It used to be known a FindFace, and SearchFace.) However it’s limited to photos from VKontakte, the Facebook for the Russian speaking world. It’s a big dataset. But it’s not international like the Yandex search engine is.

I imagine within a few years this “look up a name from a face” will become as common as “look up a name from a phone number”. Not quite ubiquitous because everyone finds it creepy, but easy enough to access if you need it.