To find the suitable lipstick
There is always a problem for many people: “What kind of lipstick does the character use in the movie?” Another problem usually comes with the previous one: “Does the lipstick of this color suit me/someone?”
We hypothesize that great commercial value will be found if an application can fill the gap between potential lipstick buyers and cosmetics firms. In this project, we will build a system that mainly contains the following three functions.
Lipstick Recognition
Recognize the most possible lipsticks for given profile images
Lipstick Makeup
Give the user lip digital trial using face parsing
Lipstick Recommendation
Provide lipsticks recommendations based on the user's preference
Architecture
This flowchart shows our design to achieve our project’s objectives.
Feature
The core function of this project focuses on lipstick recognition. The target users are those who want to know what lipstick a person in the photo used. Such target users usually have sequent questions like “Does this lipstick suit me/someone?”. To have a complete pain point solving solution for the expected target user, we extend the project's function to include digital makeup and recommendation system. The project is going to build a system in the form of a mobile application that can achieve the below functions:
Lipstick Recognition
It can extract lipstick color in the user-uploaded photo and return the most similar lipstick brand and corresponding color number.
- Use a pre-trained face parsing model BiSeNet that is trained on the CelebA-HQ-img dataset to extract every pixel from the lips' region.
- With the segmentation result, calculate the “average” RGB color of the whole lip of an original image, then find the closest color from the lipstick database.
Lipstick Makeup
It can extract details of user-selected lipstick and generate convincing lipstick makeup results.
- Support to select the target lipstick from the list or the previous "the most possible lipstick".
- Using the same face parsing technique we utilized in the previous step to get the lip region of the user-uploaded image.
- Transform the target lipstick color and user-uploaded image into HSV color space. Target lipstick color’s value and user-uploaded image’s hue & saturation are discarded and then Overlaid these two images into one picture.
A real-time lipstick recommendation system
It will recommend users the lipsticks they potentially like and purchase links of them, which endows dual value to the Lipstick Finder.
- Use the collaborative filtering solution to collect users’ information and behaviors through questionnaires during user registration and users’ interaction history with the app.
- The system will recommend lipsticks using both user-to-user and item-to-item methods.
Prototype
Progress
About us
Our team members are all master candidates in computer science of the University of Hong Kong. If you are interested in our application, or are willing to make any suggestions for our project, you are very welcome to contact us.
Our Address
Department of Computer Science,
The University of Hong Kong,
Pokfulam, Hong Kong
Email Us
jiangl@connect.hku.hk
qsj1024@connect.hku.hk
Call Us
+852 61598156
+852 63569702
