Anonymize the photographs by Blurring Out Facial Identities

Protect personal data in photos by identifying and eliminating faces automatically. Can be run in google colab

1. Problem Scoping¶

Description:¶

Although it is not necessary for the processing of the image, many pictures include people’s faces.
Artificial intelligence (AI) may be used to detect faces in a picture and mask them for more secure processing.

Problem:¶

It’s almost hard to manually regulate the context of photographs when hundreds or thousands are posted to a website or processed into a system.
It would be extremely time-consuming and costly to hand choose images and apply masks to remove individuals from them.

Solution:¶

A face identification AI service can identify human faces in photos and provide bounding boxes that include pixel data for each identified face.
After obtaining this pixel data, the picture may be altered by, say, blurring out the offending regions or masking them.

Benefit:¶

Images that have been anonymized may be utilized in:

Users’ privacy may be improved by removing persons from photographs used in regulated procedures, but only if such processes do not need personally identifiable information (IIR).

Using AI techniques to blur faces has several benefits:

Speed: Faster than a human can manually blur faces, AI algorithms can analyse pictures and videos. When working with a big volume of photographs, this may be extremely helpful.

Accuracy: Face-blurring may be improved by using AI algorithms that have been taught to effectively recognise faces in a variety of settings and situations.

Uniformity: AI algorithms can reliably apply the blur effect to all of the faces in a given picture or video.

2. Building the AI Model¶

Here we will do an activity on how to blur faces using two open-source tools:

OpenCV, a computer vision software library, and
RetinaFace, a deep-learning facial detector.

References:

%pip install ipython-autotime
%load_ext autotime

# Installing required retina-face Libraries

# Importing required libraries
from retinaface import RetinaFace
import numpy as np
import requests
import os
import cv2
from IPython.display import display
from PIL import Image

To know more about openCv click here

To know more about os click here

# Define sample images folder
sample_img_folder = "sampleImagesFolder"
if not os.path.exists(sample_img_folder):
    os.makedirs(sample_img_folder)

# URLs of Images over the Internet    
image_urls = ['https://images.pexels.com/photos/3184291/pexels-photo-3184291.jpeg?cs=srgb&dl=pexels-fauxels-3184291.jpg&fm=jpg',
              'https://media.istockphoto.com/id/1346125184/photo/group-of-successful-multiethnic-business-team.jpg?s=612x612&w=0&k=20&c=5FHgRQZSZed536rHji6w8o5Hco9JVMRe8bpgTa69hE8='
              ]

# Let’s download two images
for idx, image_url in enumerate(image_urls, 1):
    response = requests.get(image_url)
    if response.status_code:
        fp = open(f'{sample_img_folder}/image_{idx}.jpg', 'wb')
        fp.write(response.content)
        fp.close()
        
print("Images are successfully downloaded!")

# Let’s take a look at the first image.
path=f'{sample_img_folder}/image_1.jpg'
display(Image.open(path))

The RetinaFace detect_faces function allows us to identify human faces.

In addition to the coordinates of common facial cues like the eyes, nose, and mouth, RetinaFace also provides a likelihood score for each recognized face that measures the accuracy with which the face was identified.
The facial_area coordinates provide us the face’s bounding box in [x1, y1, x2, y2], where (x1, y1) and (x2, y2) are the coordinates for the top left and bottom right corners of the box, respectively.

result = RetinaFace.detect_faces(f'{sample_img_folder}/image_1.jpg')
# print the result

Let’s use OpenCV’s rectangle function to visualize the bounding boxes around the identified faces.

The bounding box’s upper left corner (x1, y1) and lower right corner (x2, y2) are represented by start_coord and end_coord, respectively.
These coordinates are returned by OpenCV’s rectangle function together with the picture, color, and thickness of the bounding box.

3. Detect Faces¶

original_image1 = cv2.imread(f'{sample_img_folder}/image_1.jpg')
result = RetinaFace.detect_faces(f'{sample_img_folder}/image_1.jpg')
color = (255,0,0) #BGR

for k, v in result.items():
    start_coord = tuple(v['facial_area'][0:2])
    end_coord = tuple(v['facial_area'][2:4])
    im_output = cv2.rectangle(original_image1, start_coord, end_coord, color = color, thickness = 10)
display(Image.fromarray(im_output))

All the faces, including those seen just from the side, have been recognized by RetinaFace.

4. Blur Faces¶

Now that we know where the faces are in the photograph, we can look at methods for concealing their identities.

To hide a person’s identity in a picture, the pixels that make up their face must be altered such that the area that used to contain their face is unidentifiable.

[I] Filled Rectangle¶

The simplest method is to change the pixel values within the bounding boxes to a solid color, such as dark grey.

We can achieve this by setting the thickness value to -1.

original_image1 = cv2.imread(f'{sample_img_folder}/image_1.jpg')
result = RetinaFace.detect_faces(f'{sample_img_folder}/image_1.jpg')
color = (50,50,50) #BGR

for k, v in result.items():
    start_coord = tuple(v['facial_area'][0:2])
    end_coord = tuple(v['facial_area'][2:4])
    im_output = cv2.rectangle(original_image1, start_coord, end_coord, color = color, thickness = -1)

# Display the im_output array using appropriate function.

The goal of hiding identities has been met, however the black squares stick out unattractively.

Now let’s see if we can’t blur the faces in a way that’s a little more attractive to the eye.

[II] Gaussian Blur¶

original_image1 = cv2.imread(f'{sample_img_folder}/image_1.jpg')

im_blurred = cv2.GaussianBlur(original_image1, (31, 31), 0)

mask = np.zeros(original_image1.shape, dtype=np.uint8)
color = (255,255,255)

for k, v in result.items():
    start_coord = tuple(v['facial_area'][0:2])
    end_coord = tuple(v['facial_area'][2:4])
    mask = cv2.rectangle(mask, start_coord, end_coord, color = color, thickness = -1)
    out = np.where(mask==np.array([255, 255, 255]), im_blurred, original_image1)
    
display(Image.fromarray(out))

Gaussian blur is a result of blurring an image using a gaussian function.

When an image is blurred using a Gaussian function, the result is known as Gaussian blur.

im: The picture before gaussian blur was applied is seen here.

im_blurred: The whole original picture, im, is blurred using a Gaussian blur in im_blurred.

mask: This masking layer is used to show where the face boundary boxes are located. Zones containing face bounding boxes are indicated by the pixel values (255,255,255), whereas the rest of the image is indicated by the values (0,0,0).

out: The final picture, once the mask has been applied, is seen below. The final picture may be thought of as a blurred version of the original image (im) that has been superimposed over the face bounding boxes (im_blurred).

To learn more about Gaussian Blur click here

[III] Pixelated¶

By “pixelating” the face, we can hide people’s identities.

The pixelate function has a degree argument that may be used to control the pixelation level.
More pixelation is produced with lower degrees of accuracy.
Degree = 20 pixelation was used on the top picture, while degree = 1 was used on the bottom.

def pixelate(image, degree):
    """
    image(array): array that contains the image
    degree(int): amount of blurring. smaller value creates more pixlation.
    """
    # Get input size
    height, width, _ = image.shape
    h,w = degree, degree
    # Resize image to "pixelated" size
    temp = cv2.resize(image, (w, h), interpolation=cv2.INTER_LINEAR)
    # Initialize output image
    return cv2.resize(temp, (width, height), interpolation=cv2.INTER_NEAREST)

result = RetinaFace.detect_faces(f'{sample_img_folder}/image_1.jpg')
original_image1 = cv2.imread(f'{sample_img_folder}/image_1.jpg')
for k, v in result.items():
    x1,y1,x2,y2 = tuple(v['facial_area'])
    # Extract ROI
    ROI = original_image1[y1:y2, x1:x2]
    # Pixelate ROI
    pixelated_ROI = pixelate(ROI, 10)
    # Paste pixelated ROI back into original image
    original_image1[y1:y2, x1:x2] = pixelated_ROI

    
display(Image.fromarray(original_image1))

# Lets try pixelating another image: image_2.jpg

result = RetinaFace.detect_faces(f'{sample_img_folder}/image_2.jpg')
# Read image_2.jpg

print("Before Pixelating:")
display(Image.fromarray(original_image2))

# Implement a for loop with Extact ROI, Pixelate ROI, and paste the pixelated ROI back into original image.

print("After Pixelating:")
display(Image.fromarray(original_image2))

In conclusion:

We utilize RetinaFace to find faces in photos and pinpoint their locations, then OpenCV to blur the faces using various techniques.
While we can automate the process with a high degree of accuracy, even the most sophisticated deep learning models cannot recognise faces with absolute certainty, thus there may be cases when anonymization fails.
The shortcomings of deep learning models may be worked around by including a person in the loop of the validation and correction of automated system output.