Facial recognition tech turns our faces into big business

Facial recognition tech (FRT) has moved from the realm of sci-fi movies into people’s daily lives. We unlock smartphones, pay in stores, and expedite airport procedures with our faces. Authorities use this tech to find dangerous criminals that hide among thousands of citizens on city streets. In fact, FRT tools like the one developed by the Chinese tech firm SenseTime can “identify a person out of a crowd of 10,000”. And advertisers are already experimenting with using our facial expressions to personalise ads displayed on digital billboards, in taxis, and many other places.

The spread of facial detection, however, comes at a cost. Companies often train smart algorithms with people’s online photos without their consent. Biometric data is used by multiple parties, stored in various databases, and is at risk of being stolen by hackers and used in identity fraud. Authoritarian governments already use facial recognition to spy on citizens and monitor the movement of political dissidents. And to make matters worse, there’s no unified approach to regulating this technology on a global scale.

AI and facial data fuelled the growth of the facial recognition industry

Patrick Grother, a computer scientist at the National Institute for Standards and Technology (NIST), a US agency that promotes industrial competitiveness, says that “facial recognition is undergoing something of a revolution”. The first factor behind the revolution is the emergence of deep learning, a subfield of artificial intelligence that replicates the functioning of a human brain. It enables companies to create powerful software that spots patterns quickly. And the second factor is the wide availability of billions of images that can be stored and used to train algorithms.

Some companies have resorted to dubious actions to acquire large volumes of much-needed facial data. EverRoll, for instance, was launched in 2012 as an app that helps users organise photo collections. But a few years later, the company rebranded into Ever AI, a facial recognition firm that now boasts 13 billion images with which it trains deep learning algorithms. Many of its users were unaware for what purpose Ever AI uses their photos. IBM has recently stirred controversy, too, by scraping millions of photos from Flickr, a photo-sharing site, to train its facial recognition system. Another source of images that companies use is databases of police mugshots that can be bought from state agencies and private companies. And the availability of data isn’t a problem for a country like China, where 176 million cameras record citizens and constantly feed algorithms with new facial data.

The race to build the best facial recognition software boils down to the question of who has the largest and most diverse training dataset. Amazon, Facebook, and Google are some of the leaders in this field, and so are Chinese companies like SenseTime and Alibaba. And the way they monetise their technology is usually through licensing. For instance, Amazon licenses its software to law enforcement agencies that use it to run their own operations. SenseTime does the same by licensing its facial recognition tech to companies such as Honda, which uses it in driverless car development.

From retailers to hotels, businesses embrace new tech solutions

The various uses of facial recognition technology spurred the growth of the global market, which was worth $530 million in 2018 and is expected to reach $1.43 billion by 2025. Retail giants such as Walmart, Target, and 7-Eleven are especially keen on using FRT to prevent theft and fraud and identify loyal customers and criminals. In China, a number of stores and hotels allow customers to pay by simply scanning their faces, and we can expect to see more similar uses worldwide. Dong Xu, the chair of computer engineering at the University of Sydney, says that smart algorithms are more reliable at identifying people than fingerprints.

But AI technologies are also skillful at reading our emotions and serving personalised ads. This area will be the focus of advertisers in the coming years, says dr Pippa Malmgren, an economist and founder of the drone maker H Robotics. She envisages a world in which cameras hidden inside TVs scan viewers’ reactions to ads and movies and feed that information to companies. Facial recognition tech will enable companies to target consumers in ways that go “beyond anything Cambridge Analytica could ever dream of”.

Cameras analyse shoppers and their emotions

Westfield, for instance, uses cameras to take photos of customers and determine the gender, age, and mood using face detection technology. The system is installed in 41 Westfield shopping centres across Australia and New Zealand and the data is then used to serve relevant ads on 1,600 digital billboards and inform advertisers of people’s reactions. The French software firm Quividi provided the facial detection tech that powers Westfield’s Smartscreen network. Quividi claims that its program distinguishes shoppers’ gender with 90 per cent precision and determines their mood with 80 per cent accuracy. And although Westfield doesn’t use the images to determine the actual identity of a person, it didn’t dismiss the idea of doing so in the future.

Combining blockchain technology and facial recognition

The UK tech firm Bidooh took a slightly different approach to billboard advertising in places like high streets and shopping centres. For instance, its platform is available to all advertisers, and any digital billboard in the world can display ads by plugging a Bidooh Box into the HDMI port. Also, the company launched its own cryptocurrency named DOOH Token that’s used to purchase ad space at a rate that’s 28 times cheaper than traditional methods. And the platform stores transaction data and verifies advertisers using blockchain technology.

Furthermore, a Bidooh advertising screen uses facial recognition tech to determine the age, gender, and the brands of clothing that passers-by wear. This information determines which ads the screen shows, and the demographic data of the viewers is shared with advertisers. Abdul Alim, the co-founder and CEO of Bidooh, hopes that new out-of-home advertising models can transform the digital advertising industry, which is worth $34.8 billion.

Showing ads to passengers in the backseat

Companies also hope to catch your attention as you sit in the backseat of taxis. In Manila, the capital of the Philippines, the tech start-up AdMov installed tablets in a number of vehicles that belong to various ride-hailing firms like Grab. Using facial recognition tech, the tablets estimate the gender, age, and mood of the passenger and then display a mix of useful content and ads. For instance, they can show a brief history of the area through which the vehicle is passing, as well as nearby restaurants and shopping malls. And if the passengers get tired of the ads, the software takes notice. Ellard Capiral, the founder and CEO of AdMov, explains that the “program also tracks eyeballs. If you are no longer paying attention to the ad, we mix it up and offer content like trivia and that pulls the passenger’s attention back.”

Similar trends are taking hold in Japan, too. The e-commerce platform DeNa Co Ltd, for instance, operates the Premium Taxi Vision service that uses tablets to scan the faces of people sitting in the backseat of taxis. The facial recognition software determines the gender and age of the passengers and sends the data to Geniee Inc., which delivers relevant ads. The facial recognition footage is then deleted, DeNa claims. Premium Taxi Vision portrays this service as a way to make “riding a taxi more comfortable and convenient”, although it remains to be seen whether passengers share the same enthusiasm.

Facial data can be used for malicious purposes

The increasing use of facial recognition tech means that new challenges will emerge. Stalkers and voyeurs, for instance, could use FRT to identify people they record in public spaces. Merchants and landlords might use the tech to identify what they consider as undesirable clients and deny them housing or services. The cybersecurity expert Andrei Barysevich says that hackers already stole profiles from India’s national biometrics database and offered them on ‘dark web’. And “it’s just a matter of time” before databases of American faces are hacked, too, he argues. Those images would enable criminals to commit fraud or identity theft. And countries like China already use FRT to spy on citizens and penalise unwanted behaviour. For instance, Malmgrem claims that  11 million Chinese citizens “can not take a flight” because the surveillance system assigned them a low score.

The industry offers several solutions on how to prevent the hacking of facial recognition systems. Doug Aley, the CEO of Ever AI, says that the first step is for technology providers to work more closely with their clients end users to make sure they follow high security standards. A second solution is to make software store faces in the form of alphanumeric hashes. Even if hackers breach the database, they wouldn’t be able to use hashes to reconstruct faces and identities. And the third solution is to develop new ways of training AI with less facial data, which is an approach that the Miami-based tech startup Kairos took. But some regulators went one step further and imposed rules on how facial data can be collected and processed.

The EU and the US took a different regulatory approach

The European Union (EU) adopted the General Data Protection Regulation, or GDPR, that covers the issue of facial recognition. Under this regulation, it’s prohibited to process facial images when used to identify a person, unless “the data subject has given explicit consent”. This poses a critical obstacle for various uses of FRT, whether in marketing or security. The EU, however, allows member states to introduce derogations. The Netherlands, for instance, passed a law that allows the use of biometric data for security or authentication purposes, while Croatia exempted surveillance security systems from the data protection law.

Unlike the EU, the US has no federal law that regulates the collection and use of biometric data, including facial data. The US Federal Trade Commission didn’t endorse regulation, and only recommended best practices such as building in privacy by design or creating strong security measures. Washington, and Texas have passed their own biometric legislation. The Illinois Biometric Information Privacy Act, for example, prescribes a timeframe in which the collected data must be destroyed. Also, the law prohibits the sale of biometric data and defines that individuals must be informed when biometric data is being collected and for what purposes it’s used. And most importantly, it allows private parties to file a lawsuit, and companies like Facebook and Google were already taken to court for their lax use of FRT.

Big Brother has arrived

Nonetheless, the massive adoption of facial recognition technology seems like an inevitable outcome. It provides convenience in many different ways and is a growing industry driven by demand from both the private and public sector. And companies are finding new ways to monetise FRT solutions, whether through personalised ads or faster hotel check-in. But the key issue is how to prevent the misuse of this futuristic technology while promoting its legitimate use. Europe, China, and the US took different paths in that regard, and it remains to be seen whether the world will remain divided or find a middle path. Whatever the final outcome is, it’s clear that Big Brother has arrived and is here to stay.

This article was originally published in 2019 on the Richard van Hooijdonk’s blog.