Google Assistant is now able to communicate in multiple languages with ease.
The new multilingual feature allows users to speak two languages interchangeably with the Assistant. This will work on both Google Home speakers and Assistant enabled phones.
Furthermore, Assistant users can switch between languages across queries. This means you can ask questions like “Hey Google, add ‘dinner with mom’ to my calendar” and then “Ok Google, trouve un bon restaurant près d’ici.”
To activate this, simply navigate to Assistant settings, select Preferences and then Assistant languages. Then, select up to two languages.
Currently, this feature supports English, French, Spanish, German, Italian and Japanese. However, Google will add support for more languages in the coming months.
While turning on the feature is simple, building it wasn’t such an easy feat, says Google.
The multilingual challenge
A recent Google blog post outlines the difficulties in setting up the system.
In brief, Google had to train the Assistant to identify and understand multiple languages and then optimize that recognition.
To identify multiple languages, Google engineers had to build deep neural networks that distinguish between over 2,000 alternative language pairs. That work began in 2013 and has evolved into the LangID model.
On top of identifying the language, the Assistant must also understand it. That means Assistant must be able to translate human speech into an actionable command — something it already does well for monolingual speakers.
To do so, multiple computer processes must run in parallel to allow both identifications of language and command. In other words, the Assistant must run LangID to identify the language, as well as two monolingual speech recognition systems to understand what’s being said.
Naturally, that uses a lot of processing power and slows down results. To ensure things stay snappy, Google designed the system to stop identifying the language as soon as it’s confident what language is used.
Essentially, if I set Google Assistant to recognize English and French, the system will guess at which of the two languages I’m speaking. If Assistant is confident I’m speaking in English, it will drop the French identification and recognition engine. This saves processing power and reduces latency.
The Assistant team is starting work on making the Assistant trilingual as well. However, we may not see this feature for some time.
For more information on how Google made Assistant multilingual, you read Google’s full blog post here.
