Japan supermarket chain uses AI to assess staff smiles, sparking harassment concerns

The implementation of Mr Smile has raised concerns about exacerbating workplace harassment, particularly from customers.

Japanese supermarket chain AEON has become the world's first company to implement an artificial intelligence (AI) system to assess and standardise employees' smiles, sparking renewed debate about workplace harassment. The initiative, launched on July 1, is in place across AEON’s 240 stores nationwide.

The AI system, known as “Mr Smile,” was developed by Japanese technology firm InstaVR. It evaluates over 450 factors, including facial expressions, voice volume, and the tone of greetings, to rate a shop assistant’s service attitude. The system incorporates gamification elements, encouraging staff to improve their attitude by challenging their scores.

A trial involving approximately 3,400 staff across eight stores reportedly showed a 1.6-fold improvement in service attitude over three months. AEON's objective is to “standardise staff members’ smiles and satisfy customers to the maximum.”

However, the implementation of Mr Smile has raised concerns about exacerbating workplace harassment, particularly from customers—a significant issue in Japan known as “kasu-hara.” Customer harassment manifests as abusive language and repetitive complaints. According to a survey conducted by Japan’s largest union, UA Zensen, nearly half of the 30,000 service industry workers polled reported experiencing customer harassment.

Critics argue that enforcing a standardised smile could be perceived as a new form of customer harassment. One respondent remarked, “When service industry workers are forced to smile according to a ‘standard’, it looks to me like another form of customer harassment.” Another respondent added, “Smiles should be a beautiful, heartfelt thing, and not be treated like a product.”

This initiative is reminiscent of McDonald’s Japan’s “Smile zero yen” strategy, where “Smile” is listed on menus at no cost, symbolising the expectation for employees to greet customers with a smile. This practice has been increasingly criticised as placing additional burdens on workers, who often receive the lowest hourly wages in the country.

In 2022, Japan’s Ministry of Health, Labour and Welfare issued guidelines to combat customer harassment, urging companies to uphold service standards without compromising employee well-being. In a positive development, a supermarket in Fukuoka prefecture introduced an extra-slow checkout counter to accommodate elderly and disabled customers. Despite longer checkout times, sales increased by 10 percent, demonstrating a successful balance between customer service and staff welfare.