Smashbox Cosmetics and ModiFace recently partnered to test a novel eye tracking technology within a mobile augmented reality app. The results? By measuring the location users are looking at during a makeup video try-on session, and providing e-commerce opportunities that are synchronized with the user’s gaze location, a 27% increase in e-commerce conversions was achieved.

Now mainstream, Augmented Reality (AR) is being embraced by nearly every beauty brand as a means of educating consumers, increasing sales and maximizing digital interactions. The ModiFace and Smashbox Cosmetics partnership has focused on testing ModiFace’s pipeline of futuristic AR capabilities starting with ModiFace’s MAKEUP mobile app. MAKEUP, with 50 million downloads, provides a unique platform for brands to try new mobile e-commerce strategies and ideas, gain insight into what works and to apply those ideas within their own web and mobile AR experiences.

One of the experimental technologies within MAKEUP is the ability to precisely track the location of the user’s iris within a makeup video try-on session. The iris location, along with the exact mapping of the face, which are enabled by ModiFace’s patented Face AI tracking technology, provide the app the location each user is looking at during the session. For example, the app can distinguish when the user is looking at their own face, or at the swatches, or reading the product name at the top of the page.

This eye gaze information was used by ModiFace and Smashbox for two purposes. First, the location which users were looking at was used to create a heat map of the areas on the screen that receive more attention. This heat map provides insights into what user experiences work best and what actions users are considering. For example, there are times when users think about a particular shade but do not tap a product or take any action. Eye tracking provides a deeper level of insight into what users are thinking about by measuring what they are looking at.

By analyzing usage data within the MAKEUP app, it was observed that users spent 80% of their time looking at their own lips and eyes while trying on video-based makeup filters. After their own face, users spent 21% of the time looking at the category and color swatch bar at the bottom of the page and 19% of the time looking at the Smashbox logo and product names at the top of the page.

Looking at the specific products that users were trying on, and then reading the name of the product (as determined by eye tracking), it was determined that the most popular product category was Smashbox’s Always On Liquid Lipstick (which represented 49% of all products focused on by users). Within this category, the most popular shades were Babe Alert, Girl Gang, Miss Conduct and Bawse. A similar analysis was performed for other product categories, including the second most popular category, which was Smashbox’s Cover Shot Eye Palettes, whose products represented 12% of all products tried-on and whose name was read by users (as determined by eye tracking).

A deeper analysis was performed on 8,819 mobile users divided into two groups. For one set of users, a call to action button for purchasing products was permanently placed visibly near the top of the page regardless of the user’s eye gaze location. For the second set of users the call to action button was hidden and only displayed when the user tried on and read the name of the product. It was observed that the conversion rate increased from 6.2% when the gaze was not included to 7.9% when the call to action button was displayed after the user had finished reading the name of the product as determined by eye gaze. This represented a 27% conversion increase attributed primarily to the eye tracking technology.

ModiFace and Smashbox Cosmetics will continue to test new technologies within the MAKEUP app throughout 2017 and will share the results of each experiment with the beauty industry when possible, with the goal that the lessons learned would be used to substantially enhance and improve the AR apps and experiences of tomorrow.