Find similar face based on storage files

Fortnight ago, I have completed to detect faces on the photo using face API from MS cognitive site and then introduce the article in previous. Basic scenario is home security camera but it has more functionalities like motion detect, face find, register users, find similar faces based on users and etc. The size of Home security solution is getting bigger and bigger. But I am pretty much sure the experience is able to increase my views and many people can use it and develop together. Once I would like to describe current status what it is done.

# Scenario

 If someone roams near my house, it will be automatically detected and captures the image to identify who it is. If he or she is not registered in the user list, the solution will notify the caution to registered users and saves the photo to prove.

For this scenario. I have wrote a few article to complete and utilize in real world. At First, I tested with PIR sensor in order to detect motion.(http://hero-space.tistory.com/33). Second, tested web cam how to preview(http://hero-space.tistory.com/40). During the second step, I had stuck in some vision api issue. OpenCV that I used to identify the faces in the videos or photos but it would not be worked well. I guess root cause is in opencv library which was released from MS.(Not sure so far). Third one is to identify faces based on photo(http://hero-space.tistory.com/42). In this article, I used to call MS cloud api to identify the face. Please look over the previous article what I was doing in previous steps. Today, I am going to explain 'Face similar find based on storage files. Definitely, administrator will be existed due to manage the solution, If I set up the solution on the front door, the host of the house will be supervisor. Only he or she can handle the solution and criteria to compare stranger's photo. Assuming If somebody doesn't want to get a newspaper, but every morning I can find the newspaper. How to catch the person who delivered the newspaper. The solution is helpful to identify who he or she is and If I put some TTS module, It can say the caution message like 'Do not leave the newspaper' Anyway I need to find out how to compare the faces between users and strangers.

# Find Similar Face

https://dev.projectoxford.ai/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f30395237

Please visit to the site what apis are. All of MS cloud apis are connected via http and used to json which include some information. like facedId. See the request body and response body. If we get return value from the request, we need to parse the json to get proper information like confidence.

ex. Request Body

{ "faceId":"c5c24a82-6845-4031-9d5d-978df9175426", "faceListId":"sample_list", "maxNumOfCandidatesReturned":10, "mode": "matchPerson" }

ex. Response Body

[ { "persistedFaceId" : "015839fb-fbd9-4f79-ace9-7675fc2f1dd9", "confidence" : 0.82 }, ...  ]

In UWP project, we can add up the package via Nuget package manager and use it to call. We don't need to consider the Json parsing and whatever. 

private async void checkFacefromUserLists(Guid id)
{
      var userCnt = 0;
      double percent = 0;
      try
      {
           var result = await faceServiceClient.FindSimilarAsync(id, _facesId, FindSimilarMatchMode.matchFace);

           listView.Items.Clear();
           foreach (var fr in result)
           {
               percent = fr.Confidence * 100;
               Debug.WriteLine(fr.Confidence * 100 + "% matched");
               listView.Items.Add(fr.Confidence * 100 + "% matched");
               if (fr.Confidence >= OpenAPIConstants.precision)
               {
                   userCnt++;
               }
           }

           if(userCnt == 0)
           {
               strangerDetected = true;
           }
       }
       catch(FaceAPIException ex)
       {
           Debug.WriteLine("Error Respons : {0}. {1}", ex.ErrorCode, ex.ErrorMessage);
       }

       if (strangerDetected == true)
       {
           PhotoLiveFeedText.Text = "CAUTION !!!";
       }
       else
       {
           PhotoLiveFeedText.Text = userCnt + " User Detected";
       }

       strangerDetected = false;

}

The method is calling if the photo is detected faces in order to compare user list. Let's see the the api like 'faceServiceClient.FindSimilarAsync'. The api is able to check based on face id. Already we know the faceid from the DB when the solution is started and keep it into _facesid list. id is sent by calling function which stands for detected face due to motion. So far we don't know who is detected users or strangers. As soon as we can know the face by the api return. I will create list view according to the result. Because users are possible to be 2 or more. Detected person will be checked with the all users how much he or she looks like. return value will include 'Confidence' section It is showing the threads holds. Once I set up the thread holds like 70 percents. If the matched percentage is upper 70 percent we can say the person is one of users. and the draw the text on the UI like "CAUTION!!!" or "1 User detected". I think it needs more test due to various environment like brightness, size and so on. 

This is the main view when it boots. web camera will show the current image and we can see the user list either. If somebody detects, camera will be captured and show on to the right side and calculate how much percent is matched based on users photos.

Okay, how is the result? I registered me and my wife into user list and tested it manually. The bottom of the view, we can find the icon like camera shape, If I click the Icon, camera will be captured and saved to identify the face. Independently, I have developed it to check functionalities. Captured image is showing how much matched with each user and one of user is almost the same with captured image.

 The page is user registration page, If I click the '+' icon, it goes the register view to capture ID photo and requires the name. We can keep the user images in the local folder. If I want to add up more user, I can run the view to add up easily.

I refer to the project (https://www.hackster.io/windows-iot/windows-iot-facial-recognition-door-e087ce) which shows Facial recognition Door. But my project concept is little bit different than this. Also I add up more functionalities like Find Similar, TTS and other. The project is not the end and keep to improve. Please wait more few days, I think the next step is much more exciting.