• Subscribe

What can you see in a smart mirror?

Speed read
  • Penn State data science students present ‘smart mirror’ at international competition
  • Facial recognition and advanced algorithms combine to preview hairstyles in real time
  • Personalized data collection improves user and stylist experience

On May 17, a trio of Penn State graduates traveled to Paris to pitch their ‘smart mirror’ prototype at the 2018 L'Oréal Brandstorm competition.

<strong>Yuya Ong</strong> demonstrates the facial recognition technology that powers SalonAI's smart mirror. The technology maps the user's face by recognizing key points, such as the jawline's shape and distance between eyes, and displays popular hairstyles and colors seen on similar face shapes. Courtesy Penn State. <a href='https://creativecommons.org/licenses/by-nc-nd/2.0/'>(CC BY-NC-ND 2.0)</a>The members of team SalonAI, Vincent Trost, Vamshi Voruganti, and Yuya Ong, have spent several years exploring new applications of data sciences through their work at Nittany Data Labs. Recognizing that salon interactions are most often based around a mirror, SalonAI aims to bring the mirror into the 21st century using computer vision technology, machine learning algorithms, and data collection infrastructure.

To create the mirror, Ong programmed facial recognition software that identified key points on the user's face — such as the distance between the eyes and the curve of the jawline — to determine face shape.

Trost used an algorithm to determine which hairstyles were most often seen on models with different face shapes. By combining the two, the mirror can recommend and simulate hairstyles and cosmetic products that are popular among others with similar features.

"With this system, we can better analyze many features including facial shape, age, skin tone, and other critical information about a customer in real time," said Ong. "By leveraging this data we can better deliver a higher quality of services than ever before and will generate tremendous brand loyalty."

Ong built the mirror in his dorm room over two days using an old computer monitor and spare parts he found online. He then created the entire facial tracking algorithm in 24 hours during HackPSU.

<strong>Real-time feedback</strong> using facial reactions lets the stylist know whether the consumer likes the style or not. They can record the user's preferences and get feedback on their performance. Courtesy KOMUnews. <a href='https://creativecommons.org/licenses/by/2.0/'>(CC BY 2.0)</a>The mirror also serves as a pipeline for data collection, which the group views as the most valuable element of their product. For example, by indexing individual preferences, companies can anticipate and develop products that respond to trends, and stylists can improve performance.

"The entire salon experience is seen in the mirror, which allows us to capture the subtler points," said Ong. "This lets us solicit real-time feedback using facial reactions to see whether the consumer likes the style or not. The stylist can record the user's preferences and get feedback on how well they performed the services they provided."

"We are not trying to replace the stylists," said Trost. "They are the reason it works, but we are trying to enhance the experience and make it more collaborative."

Creating a shared salon experience

L'Oréal's internal research has shown that 97 percent of stylists believe they provide consultations to their clients, but only 7 percent of clients feel like they receive one. This disconnect helped the group members hone their proposal.

<strong>Team SalonAI</strong> (Vince Trost, Yuya Ong, and Vamshi Voruganti) receive their award as one of the top two US teams in the L'Oréal Brandstorm from David Greenberg, president of L'Oréal's Professional Products Division. Courtesy Penn State. <a href='https://creativecommons.org/licenses/by-nc-nd/2.0/'>(CC BY-NC-ND 2.0)</a>The team also surveyed more than 100 women to gather feedback about the salon experience. Their research showed that most people either asked the stylist to recreate a hairstyle based on a photo from their smartphone or simply let the stylist choose which style would look best. But neither is ideal.

"When you get a haircut, you spend the whole time staring in the mirror," explained Trost. "Instead of trying to describe it or zoom in on a photo on your phone, why not just display the style on the mirror? Then you and the stylist are working together, identifying what you do and don't like, simulating it on you in real time, and everyone is happy because you can see exactly what you're going to get before you get it."

Added Ong, "This isn't automated intelligence or artificial intelligence; it's augmented intelligence."

Reflecting on an idea

To make it to the world finals, more than 1,000 US registrants submitted video pitches, and 10 teams were selected to present their ideas at the national finals in New York City. The top two teams from that national competition — including SalonAI — were selected to participate in the final competition in Paris.

The team believes much of their success so far has come from their ability to capitalize on individual strengths.

"To run a successful startup, you need three people," said Trost, recalling advice given to him by Steven Haynes, associate teaching professor of IST. "One should be completely devoted to product development — that's Yuya; one should be completely devoted to marketing the product and securing funding — that's Vamshi; and one should be focused on making things run smoothly — which has been my role."

Along the way, the team received hands-on coaching and regular feedback from L'Oréal executives, industry experts, and Penn State faculty.

And while they're excited to be in the finals, the group is focusing their attention on turning their idea into an entrepreneurial business. Noticing the potential for smart mirrors in industries like high-end retail, fitness and personal health, the team is hoping additional feedback from judges will help them explore more potential opportunities.

"We're ultimately trying to start a business," said Trost. "If we lose this competition we lose nothing, but we've had the opportunity to pitch our idea to CEOs of Fortune 500 companies, refine it, and make it better."

Read the original article on Penn State's site here.

Join the conversation

Do you have story ideas or something to contribute? Let us know!

Copyright © 2023 Science Node ™  |  Privacy Notice  |  Sitemap

Disclaimer: While Science Node ™ does its best to provide complete and up-to-date information, it does not warrant that the information is error-free and disclaims all liability with respect to results from the use of the information.


We encourage you to republish this article online and in print, it’s free under our creative commons attribution license, but please follow some simple guidelines:
  1. You have to credit our authors.
  2. You have to credit ScienceNode.org — where possible include our logo with a link back to the original article.
  3. You can simply run the first few lines of the article and then add: “Read the full article on ScienceNode.org” containing a link back to the original article.
  4. The easiest way to get the article on your site is to embed the code below.