Rosetta extracts text from more than a billion public Facebook and Instagram images and video frames and inputs it into a text recognition model
Facebook has built and deployed a large-scale machine learning system called Rosetta which can help identify harmful and inappropriate content on its site.
A significant number of the photos shared on Facebook and Instagram contain text in various forms. It can be overlaid on an image in a meme, or inlaid in a photo of a storefront, street sign, or restaurant menu.
Rosetta extracts text from more than a billion public Facebook and Instagram images and video frames (in a variety of languages), daily and in real time, and inputs it into a text recognition model.
The data distribution of textual images on Facebook and Instagram change quite rapidly and those who try to use text overlaid on images for spam or other inappropriate content continue to adapt their techniques.
As a result, Rosetta has been widely adopted by various products and teams within Facebook and Instagram.
To align with Facebook, there were a few major modifications like incorporating special rules for languages such as Arabic (right to left), and Spanish and German (diacritics), etc.To support a global platform and the number of languages in the world, investment is being carried out by extending the text recognition model for the wide number of languages used on Facebook.