Facial Recognition Technology in India

Executive Summary

This article aims to explain what facial recognition technology is and analyse issues in its implementation. The issues covered cover both the legal and technical aspects of implementing this technology. It goes over legislation and judgements relevant to this topic and also addresses algorithmic biases and how they develop. Finally, the article covers recommendations that address some of the concerns surrounding this technology.

Keywords: Facial Recognition, Privacy, Algorithmic Bias, Personal Data Protection Bill.


On the 22nd of July 2020, the National Crime Records Bureau (NCRB) put out a Request For Proposal (RFP) [1] concerning the creation of an Automatic Facial Recognition System (AFRS) in India. The stated purpose of the system as per the RFP is to ensure the “Availability of relevant and timely information” to “improve outcomes in Criminal identification and verification by facilitating easy recording, analysis, retrieval and sharing of information between different organisations.”

The RFP then goes on to describe the requirements of the AFRS that the NCRB is looking for. This move by the NCRB has raised concerns for digital rights activists due to several factors such as the inaccuracy of facial recognition technology, infringement of citizens' right to privacy, etc.

This paper endeavours to explain what facial recognition technologies are, the state of facial recognition worldwide and in India, and examine arguments surrounding their implementation in state infrastructure.

A Primer on Facial Recognition Technologies

For this piece to make sense, we first need to understand what Facial Recognition Technologies (hereinafter FRTs) are, how they work, where they are used, and how their performance is measured. This will allow us to understand the technical aspects of FRTs before diving into other areas.

What are FRTs?

FRTs can be broadly classified based on the functions they perform:

  1. Whether the technology can detect a face.

  2. Identification of features that the detected face possesses.

  3. Identification of the person who possesses the detected face.

Face Detection Technology

The process of face detection (i.e., technologies that fall under category (1)) does not determine the individual's identity or what facial features (i.e., race, gender, and other features) they possess. The technology can only detect if a face exists within a given image. This is usually the first step in any kind of facial analysis since the subsequent analysis of features or finding the person's identity is predicated on the detection of a face.

Any facial detection technology has two possible errors [2]:

  • False Negatives: Not recognising a face.

  • False Positives: Recognising an object that is not a face as a face.

Feature Detection

Feature detection technologies identify a particular feature of an individual such as race, sex, emotional state, etc., but not their identity. Feature detection can be further subdivided into two categories:

  1. Facial Attribute Classification [3]:

  2. This is the task of classifying various attributes of a facial image - e.g., whether someone has a beard, is wearing a hat, and so on.

  3. This is a challenging problem because faces can vary dramatically from one person to the other, and can be viewed under a variety of different poses, occlusions and lighting conditions.

  4. Facial Expression Classification:

  5. Classifying the face based on the expression it presents.

  6. This too is a complex problem due to the wide variety of faces, and also because of the variety of expressions that the human face can make.


The final type of facial recognition technology is a technology that can identify a person using their face. This technology first detects a face, classifies a face based on the attributes, and finally uses this information to identify the person.

This type of technology is divided into two sub-categories:

  1. Face Verification (one type of face recognition):

  2. It attempts to determine whether an image shows a particular person.

  3. When presented with the face, the system determines whether the face belongs to someone that the system knows.

  4. This is used, for example, to unlock some phones, as a security mechanism etc.

  5. Face Identification:

  6. This system aims to understand whose face is the one that has been detected.

  7. Once the face has been detected, it then proceeds to match the information to someone whose image information it already has.

  8. Essentially, the software aims to answer the question: “Whose face is this?”

Facial Recognition Process

Now that we have established the different types of FRTs, we will now proceed to understand how they work. The process of facial recognition involves the use of the aforementioned technologies. The steps are:

  1. Image capture and detection:

  2. A face is photographed, which has a twofold purpose:

  3. Building a repository of faces for the technology to compare to.

  4. The photograph is compared with the repository in order to identify the individual.

  5. Next, the face is detected.

  6. If the face is not detected, then the photograph needs to be retaken.

  7. Enrolment into the system:

  8. Recognition of a face is contingent on the prior registration of the face in the database.

  9. The process by which an individual's face is stored in the system is called enrolment.

  10. Digital representation of the face:

  11. The face capture process transforms the analogue information (the face) into a set of digital information (data) that is based on the person's facial features.

  12. This data is called a faceprint.

  13. In the same way that thumbprints are unique, each person has their faceprint.

  14. Comparison:

  15. Comparison of the faceprint with the database.

  16. The comparisons generate match scores to identify the closest possible match to the given faceprint.

  17. Decision:

  18. Output is given. This could be the detection of a face, feature or full facial identification.

General uses of FRTs

FRTs are seeing more use today. For example, we see their use in:

  • Banks: Some financial institutions use FRTs as an added layer of authentication.

  • Consumer products: Some products use FRTs to grant access to the user.

  • Housing: Individual homeowners can install camera systems that use FRTs in products like smart doorbells.

  • Workplaces: Employers can use FRTs as access control measures i.e., restricting access to certain employees only.

Other areas of use are: Educational Institutions, Policing, Events etc.

With this foundation established, we will now examine India's use of FRTs.

India and FRTs

What is the status of FRTs in India?

We see the use of the FRTs in the following states. The list is not a complete list of FRTs in India:

  • Telangana:

  • As per the Internet Freedom Foundations Project Panoptic [5] (which aims to bring transparency and accountability to the relevant government stakeholders involved in the deployment and implementation of FRT projects in India) Telangana has the largest number of FRT projects being rolled out.

  • Project Panoptic lists out six projects that are present in Telangana.

  • The ones that stand out are the TSCOP [6] and the Crime and Criminal Network and Tracking System (CCTNS)[7].

  • TSCOP aims to empower front-line police officers with actionable intelligence to increase their efficiency.

  • The CCTNS provides services such as “Petition Management, FIR Management, Investigation Management, Courts and Prosecution, Station House Management, Higher Officers Module, Police Messaging System, Enterprise Search, Online Monthly Crime Review, and Criminal Intelligence System.”

  • Maharashtra:

  • The Maharashtra Police have begun rolling out the Automated Multi-Modal Biometric Identification System (AMBIS) [8].

  • Officials said that AMBIS was designed to identify suspects at the click of a mouse and provide information about criminal elements to other police forces, be it within the country or abroad.

  • The system integrates with CCTV systems in Maharashtra to apply FRT to CCTV footage.

  • Gujarat:

  • The Surat police have been provided FRT by the NEC.

  • The NEC’s NeoFace® Reveal [9] is a software solution for forensic investigation that provides law enforcement and crime laboratory agencies with the ability to enhance poor quality latent face images, search them against their mugshot databases, and locate potential suspects, while the NeoFace® Watch integrates with existing video surveillance systems and matches faces in real-time against a watch list of individuals to trigger an alert.

  • The stated aim is to increase the efficiency of the Surat police.

  • Tamil Nadu:

  • The Tamil Nadu Police has started utilizing FaceTagr[10] to reduce crime and increase efficiency.

  • The police have already made several arrests using this technology.

Legal Status of FRTs in India

The main law that covers this is the Personal Data Protection Bill of 2019[11]. Under Clause 3(7) of the same bill, facial image data comes under the category of “biometric data” along with “fingerprints, iris scans, or any other similar personal data resulting from measurements or technical processing operations carried out on physical, physiological, or behavioural characteristics of a data principal, which allow or confirm the unique identification of that natural person;”

Chapter VIII of the Bill lays out the exemptions where the provisions of the bill do not apply. Clause 35 states that:

Where the Central Government is satisfied that it is necessary or expedient, —

  1. in the interest of sovereignty and integrity of India, the security of the State, friendly relations with foreign States, public order; or

  2. for preventing incitement to the commission of any cognizable offence relating to sovereignty and integrity of India, the security of the State, friendly relations with foreign States, public order,

it may, by order, for reasons to be recorded in writing, direct that all or any of the provisions of this Act shall not apply to any agency of the Government in respect of the processing of such personal data, as may be specified in the order subject to such procedure, safeguards and oversight mechanism to be followed by the agency, as may be prescribed.”

The clause essentially provides a broad mandate for government agencies to skirt the terms of this bill in the interest of national security, and to mitigate threats against the sovereignty and integrity of the country.

Clause 36 also extends this exemption to when “personal data is processed in the interests of prevention, detection, investigation and prosecution of any offence or any other contravention of any law for the time being in force;”

These two clauses combined, essentially allow the government and its associated agencies to access the personal data of its citizens as and when they decide that a threat to the country exists. This extremely broad mandate has raised concerns with digital rights advocates. In the next section, we will examine these concerns and analyse the case against FRTs.

The Case Against FRTs

There are two aspects to this case:

  1. The legal aspects and challenges of using FRTs.

  2. The technological challenges and flaws of FRTs.

Legal Issues

The major cause for concern is the infringement of the right to privacy. Other than the Personal Data Protection Bill, Indian laws that regulate this technology are scarce. The Information Technology Act (2000) [12] Section 43A provides the conditions for compensation if there was a failure to protect data by a “body corporate”, and Article 72A provides the terms of compensation in the event of improper disclosure of data that is in breach of a lawful contract. The Aadhaar (Targeted Delivery of Financial and Other Subsidies) Act 2016[13], provides some protection as well. Article 30 of the same Act classifies information gathered as “sensitive personal information”, meaning that the provisions provided in the Information Technology Act for the protection of this data apply to information collected under the Aadhaar Act as well.

There have also been several cases and judgements that deal with privacy. The most famous, of course, is the case of Justice K. S. Puttaswamy (Retd.) vs Union of India [14] (hereinafter the Aadhaar Judgement). This landmark judgement by the Supreme Court of India laid out the contours of the right to privacy in India. The judgement established that the right to privacy is a fundamental right. This right includes autonomy over personal decisions (e.g., consumption of beef), bodily integrity (e.g., reproductive rights), as well as the protection of personal information (e.g., the privacy of health records).

The key aspect of this judgement is the proportionality test that was established by the court. The test [16] has three aspects:

  1. The procedure established by law:

  2. The first requirement is that there must be a law in existence to justify an encroachment on privacy (as stated in Article 21).

  3. This means that the infringement must be in accordance with the procedure established by law.

  4. Reasonable classification:

  5. Article 14 of the Indian Constitution mandates a reasonable classification test to guard against arbitrary state action.

  6. What this means is that if the right to privacy is infringed upon or restricted by using the law, that law must fall in the “zone of reasonableness”.

  7. The test of reasonable classification has two criteria:

  8. Classification based on intelligible differentia i.e., there is a clear difference between those in the group and those out of the group.

  9. The classification must be related to the objective of the law or act.

  10. Proportional benefit:

  11. The aims employed need to be proportionate to the benefit sought.

While the AFRS system would be in accordance with the procedure established by law if the Personal Data Protection Bill is passed, without it there exists no anchoring legislation for the AFRS currently.

As per the RFP, “The AFRS will be a centralized web application hosted at the NCRB Data Centre in Delhi with DR in a non-seismic zone which will be made available for access to all the police stations of the country.”

This essentially means that the AFRS would be active all over the country, and every citizen of India would fall under its purview. The AFRS does mention that the intended targets of the system are “criminals, missing children/persons, unidentified dead bodies and unknown traced children/persons.” But as the Supreme Court stated in the Aadhar Judgement, sweeping provisions that target every person in the country cannot be implemented under the guise of preventing crime. As a result, the AFRS system fails the reasonable classification test as well, because there is no differentiation provided.

The system is also slated to have identification capabilities, meaning that the government would have to collect sensitive data and enrol it into the recognition system to make it work. Data collection at this scale would not be proportionate to the slated end objective as it is highly likely that a majority of the data would go unused. Therefore, the system also fails the proportionality criterion of the test.