In 1933, the Soviet scientist Peter Troyanskii presented a “machine for the selection and printing of words when translating from one language to another” to the USSR’s Academy of Sciences. The invention was based on a typewriter, an old film camera and cards in four different languages.

The operator took the first word from the text, found a corresponding card, took a photo, and typed its morphological characteristics (noun, plural, genitive) on the typewriter. The typewriter’s keys encoded one of the features. The tape and the camera’s film were used simultaneously, making a set of frames with words and their morphology.

Despite the breakthrough, the invention was not considered useful and no one would know about it until two Soviet scientists found his patents in 1956.

At the beginning of the Cold War, on January 7th 1954, at IBM New York headquarters, the IBM 701 computer automatically translated 600 sentences from Russian into English. A first in history, the computer was able to translate at a pace of 2 and a half lines per second.

However, the translated samples were carefully selected and tested. The system was not more suitable for everyday use than a simple phrasebook. Nevertheless, this started a race for machine translation between countries such as the US, Germany, France and Japan.

From then on, the struggle to improve MT (Machine Translation) lasted for 6 decades resulting in the creation of different models of MT from Rule-Based Machine Translation (RBMT) to Neural Machine Translation (NMT).

The first rule-based machine translation concepts emerged in the ’70s. The scientist took inspiration in interpreter’s work, trying to program very slow computers to repeat those actions. The system consisted of a bilingual dictionary (RU->EN) and a set of linguistic rules for both languages.

PROMT and Systran are the most famous RBMT systems, but even in the RBMT space, there are some nuances and sub-models.

The most basic type of machine translation. It separates the text into words, translates them, slightly corrects the morphology and fine tunes the syntax to make the sentence right, more or less. The output returned a translation of quite poor quality and this model is hardly useful.

The main difference from Direct Machine Translation is that there is a preparation process to determine the structure of the sentence to be translated. The system was meant to manipulate whole constructions, not words, in order to generate a better output. In theory.

Translation outputs were still of poor quality and even though it might have simplified grammar rules it became too complex for the system because of the increased number of word constructions compared to single words.

In this model, the source language is first transformed into an intermediate universal representation (Interlingua) which could then be converted into any target language. This was the main singularity of Interlingua Machine Translation, because of the initial conversion into the intermediate representation, it meant also that we could translate one source text into various languages which was not possible in the transfer-based model.

As good as it sounds, it was extremely hard to create such a universal interlingua and scientists dedicating their whole lives to the task did not manage to do so. Although, thanks to them, we now have different levels of language representation such as semantic, syntactic or morphological.

Japan was highly interested in Machine Translation. It was identified very soon that the lack of English speakers in the country would be an issue for the upcoming globalisation. Because of a completely different language structure, Rule-based English-Japanese machine translation is pretty much impossible.

In 1984, Makoto Nagao of Kyoto University had the idea of using ready-made phrases instead of repeated translation. Imagine we have to translate a simple sentence – “I’m going to the cinema” and we have already translated the similar sentence – “I’m going to the theatre” and we can find the word “cinema” in the dictionary.

All the system needs to do is identify the missing word and then translate it. EBMT unveiled a major breakthrough, the machine can be fed with existing translations, without having to spend years in creating rules and exceptions.

In the early 90s, at the IBM Research Center, a machine translation system was tested based not on rules and linguistics but on previous translations analysis and patterns. The model is based on the fact that, based on statistics of millions of previous translations,  the machine is able to understand a pattern and choose the most adequate translation. “If people translate that way, I will”, and so Statistical Machine Translation was born.

The method was more performant than any previous ones and zero linguists were needed. The more data the system had to calculate its statistics, the better the output was.

The SMT went through different models over the years as it struggled with different languages dimensions such as word order or when new words need to be added to the output i.e: “Ich möchte keine Persimonen essen -> I do not want to eat persimmons”. The system evolved from word-based SMT to phrase-based SMT which was able to handle word re-ordering and came with a few additional lexical hacks.

Phrase-based SMT became the state-of-the-art of Machine Translation and was used by all the high-profile online translators such as Bing, Yandex or Google Translate. The latter would then revolutionise the Machine Translation world a few years later.

In November 2016, Google made a game-changing announcement announcing the launch of the Google Neural Machine Translation System (GNMT). The idea was similar to transferring style between photos such as programs like Prisma that can turn a photo into a painting imitating famous artists’ style. The neural network is taught to recognise the artist’s paintings to stylise the picture accordingly.

If we can transfer the style to a photo, how about imposing a language to a source text? The idea was to be able to translate while keeping the essence of the source text (just like the artist’s style). The source text is encoded into a set of specific features by one neural network and then decoded back into text in the target language by another network. Both networks speak a different language and don’t know about each other but they both can understand the set of features extracted from the source text. This is quite similar to the idea of Interlingua Machine Translation.

In a few years, NMT surpassed every system that developed previously and with the implementation of deep learning, it was able to implement improvements without being taught to do so. Because the system relies on an intermediary set of features and not data from previous translations, it allows direct translation between languages where it was not possible before. For example, SMT always worked using English as the source text so to translate from French into Spanish the text had to be translated into English first and then into Spanish adding more errors to the final output.