Asignment 01 – Course COMPUTER VISION II: APPLICATIONS (PYTHON)

 

Asignment 01 – Course COMPUTER VISION II: APPLICATIONS (PYTHON)

Instructions

In this assignment, you will be working on Smile Detection using Dlib's facial landmarks model. You can download the video from this link and check out the video on your local system.

Task

You will have to complete the smile_detector function. In the function, you will receive an image and you have to write the logic for smile detection and return a boolean variable indicating whether the frame has a smiling person or not.

We have provided the supporting code for keeping track of the smiling frames and we will use it to compare with the ground truth. If the percentage of overlap is high enough, you will obtain full marks.

Finally, we also write the output video in a avi file which you can download and see from the jupyter dashboard whether the smile is getting detected correctly.

You can see the output video to check whether the code detects smile or not and help to debug your code.

You will have a total of 5 attempts. This assignment carries 30 marks.

Some hints:

1. Find the lip and/or jaw coordinates using the facial landmarks.
2. For a person to be smiling, the ratio of width of the lip and the jaw width should be high.
3. Return True if a smile is detected, else return False.

Also, When you complete the assignment, you can use your own video to generate the output and share on your social media channels for fun.

 https://youtu.be/Pngwsy-Swz0 

I have to write a code snippet inserted in the TODO section(s)

Landmarks numbers

The program steps:

1.- Import all the necessary libraries, in this case cv2, dlib and numpy.

2.- Initialize dlib’s face detector

3.- Create the facial landmark predictor

4.- Define the function smile_detector(image_dlib):

                This function get an input image in dlib format. (color space RGB), detects faces in it.

                If there is not any face, the function return false (There isn’t face à there isn’t smile)

                If there is a face the function continues.

                Assign False to the variable isSmiling

                ###

                ### Here is the place for our code

                ### YOUR CODE HERE

                ###

#  My code explanation:

# The program find the square distance "lips_square_distance" between the 

#  external corners of the lips, landmarks 48 and 54.

# The program find the square distance "jaws_square_distance" between two jaw 

# points in the line of the corners of the lips, landmarks 3 and 13.

# The ratio between this two values "lips_square_distance/jaws_square_distance" 

# varies when someone smile.

# The distance between the points in the jaws varies very little but the distance  

# between the corners of the mouth increase a lot when someone smiles., 

# another fixed distance would have been the distance between 

# the eyes external corners landmarks 36 and 45.

# I looked to the numbers to find a person smiling and the ratio is greater than 0.21

# The open smile is for a ratio value greater than 0.23  

If ratio greater than 0.21:  isSmiling=True

### end of my code

 

# Return True if smile is detected

return isSmiling

 

# The main function is supplied

# Initialize the video capture and writer cv2 objects.

# Capture a video, in this case load a video file

# Initialize frame_number to zero and the list smile_frames to empty

# Loop the video file

                # grab the next frame using capture.read()

                # convert frame from BGR to RGB

                # call the function smile_detector(image_dlib) and save return in frame_has_smile

                # if( frame_has_smile == True):

                               # put text on frame and smile_frames.append(frame_number)

# after 50 frames, print the number of processed frames and the “order number” of # frames with smiles detected.

                # save the processed frame

                # increase frame_number in 1

 

# clean the buffers

# destroyAllWindows

# release capture and smileDetectionOut