"Mirror, mirror on the wall, what lipstick suits me best of all?"

17.02.2021

by Julia Pott

Retail has debated the benefits of augmented reality for years. After the initial hype, augmented reality (AR) went through a dry spell – the technology and users were not yet ready for prime time. But now there is another B2C retail sector – apart from the furniture and fashion industries – with emerging AR use cases: enter the beauty industry.

A woman uses gestures to control an app for trying out cosmetics; copyright: Perfect Corp

Brick-and-mortar stores have generally struggled amid coronavirus lockdowns, sanitation, and social-distancing rules. More specifically, the health and beauty industry seems to grapple with the pandemic’s fallout. It is an area where in-store product testing – often involving the face, an area where practicing safe hygiene is especially tricky – is an inherent part of success. Customers want to experience, test, smell, and touch beauty products and see how they match their skin tone. How does this work during a pandemic? It’s complicated ...

Technology can come to the rescue – at least as far as the aspect of appearance is concerned. Beauty customers can use their smartphone and an augmented reality application to virtually try on products at home and see what a specific lipstick, eyeshadow or blush would look like on them.

The big advantage for customers and retailers: there is a far greater online product selection than the limited number of products that are usually available for testing at the store.

Three screenshots from the Pinterest app; Copyright: Pinterest

Social media channels: The perfect platform for AR technology?

Pinterest has spotted this user demand and jumped on the bandwagon: It launched the AR technology “Try On” powered by Google Lens. Users can find and try on over 4,000 different eyeshadow shades and shop from brands like Lancome, YSL, Urban Decay, and NYX Cosmetics. This attractive option uses Pinterest's own features such as search, filters, and personalized recommendations. “Try On” for lipstick has already rolled out. Pinterest says its users are considerably more interested in buying products after using the virtual try-on tool.

Social media app Snapchat partnered with Taiwanese Perfect Corp., a provider of beauty tech solutions to integrate AR try-on experiences. Brands can bring their products into Snapchat and turn those experiences into Snapchat Lenses for users to try on. CNBC reported in January 2021 that Snap Inc., the parent company of Snapchat, has acquired the British startup Ariel AI which focuses on artificial intelligence, augmented reality, and real-time 3D rendering of people.

Google has also teamed up with Perfect Corp. and the L'Oréal Group. L'Oréal's company ModiFace offers manufacturers, brands and retailers AR try-on capability for apps, websites, and magic mirrors. Originally developed by Stanford University, the application is said to be one of the most accurate real-time video tracking and analysis technologies that tracks facial microfeatures such as skin tone and texture, wrinkles, eye size and color, head pose and facial expressions. The better these technologies recognize the physical original and combine it with high-quality animation with AR technology, the more important and useful these applications become. There have been many improvements in this area in recent years.

In-store magic mirror solutions

Augmented reality solutions are not just a great short-term solution amid a pandemic. It is highly likely that brick-and-mortar stores will also make good use of them in the long term. Even pre-COVID-19, it wasn’t ideal that multiple customers used the same testers and applied products to their skin. Virtual try-ons can not only replace physical testers but also allow consumers to simultaneously “put on and take off” as many products as they like. And as in online retail, the virtual shelf platform allows beauty stores, salons, and drug stores to make the entire product assortment available and enables customers to try things on and get all the information they need before they make a purchase decision.

So-called magic mirrors, smart mirrors, or interactive mirrors use a screen to overlay virtual information or graphics on your mirror image – like the one you take online with your own camera.

Perfect Corp advertises its "touchless virtual makeup try-on solutions" to reimagine brick-and-mortar stores in pandemic times: customers use voice activation or gesture controls for touchless navigation of the respective programs to browse through menus and product categories. Perfect Corp's “YouCam Makeup” app lets users try on makeup and cosmetics at home and uses in-store magic makeup mirrors. According to a product video, shoppers can use touch, gestures, or voice activation to control the app. Customers can scan a barcode to get product information and AR photos of their face on their smartphone to try on virtual makeup.

Sephora and Douglas also launched virtual try-on apps in some stores. It goes without saying that magic mirrors cannot replace the real try-on experience. A virtual reproduction is not (just yet) able to imitate the way products feel or smell on your skin. That being said, these solutions can nevertheless support customers in some purchase decisions. And, according to a survey commissioned by Nosto, these tools also increase the likelihood of customers making a purchase.