MS to EN

This tutorial is available as an IPython notebook at Malaya/example/ms-en-translation.

This module only trained on standard language structure, so it is not save to use it for local language structure.

[1]:
%%time

import malaya
CPU times: user 5.11 s, sys: 1.06 s, total: 6.17 s
Wall time: 7.45 s

List available Transformer models

[2]:
malaya.translation.ms_en.available_transformer()
INFO:root:tested on 100k MS-EN sentences.
[2]:
Size (MB) Quantized Size (MB) BLEU Suggested length
small 42.7 13.4 0.626 256.0
base 234.0 82.7 0.792 256.0
large 815.0 244.0 0.714 256.0
bigbird 246.0 63.7 0.678 1024.0
small-bigbird 50.4 13.1 0.586 1024.0

We tested on 100k MS-EN sentences.

Load Transformer models

def transformer(model: str = 'base', quantized: bool = False, **kwargs):
    """
    Load Transformer encoder-decoder model to translate MS-to-EN.

    Parameters
    ----------
    model : str, optional (default='base')
        Model architecture supported. Allowed values:

        * ``'small'`` - Transformer SMALL parameters.
        * ``'base'`` - Transformer BASE parameters.
        * ``'large'`` - Transformer LARGE parameters.

    quantized : bool, optional (default=False)
        if True, will load 8-bit quantized model.
        Quantized model not necessary faster, totally depends on the machine.

    Returns
    -------
    result: malaya.model.tf.Translation class
    """
[3]:
transformer = malaya.translation.ms_en.transformer()
transformer_small = malaya.translation.ms_en.transformer(model = 'small')
transformer_large = malaya.translation.ms_en.transformer(model = 'large')
WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:112: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:112: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:114: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.

WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:114: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.

WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:107: The name tf.InteractiveSession is deprecated. Please use tf.compat.v1.InteractiveSession instead.

WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:107: The name tf.InteractiveSession is deprecated. Please use tf.compat.v1.InteractiveSession instead.

downloading frozen model to /Users/huseinzolkepli/Malaya/translation-ms-en/small/model.pb
101%|██████████| 41.0/40.5 [00:15<00:00, 2.64MB/s]
downloading frozen model to /Users/huseinzolkepli/Malaya/translation-ms-en/large/model.pb
100%|██████████| 777/777 [05:32<00:00, 2.34MB/s]

Load Quantized model

To load 8-bit quantized model, simply pass quantized = True, default is False.

We can expect slightly accuracy drop from quantized model, and not necessary faster than normal 32-bit float model, totally depends on machine.

[4]:
quantized_transformer = malaya.translation.ms_en.transformer(quantized = True)
WARNING:root:Load quantized model will cause accuracy drop.

Translate

Using greedy decoder

def greedy_decoder(self, strings: List[str]):
    """
    translate list of strings.

    Parameters
    ----------
    strings : List[str]

    Returns
    -------
    result: List[str]
    """

Using beam decoder

def beam_decoder(self, strings: List[str]):
    """
    translate list of strings using beam decoder, beam width size 3, alpha 0.5 .

    Parameters
    ----------
    strings : List[str]

    Returns
    -------
    result: List[str]
    """

For better results, always split by end of sentences.

[5]:
from pprint import pprint
[6]:
# https://www.sinarharian.com.my/article/89678/BERITA/Politik/Saya-tidak-mahu-sentuh-isu-politik-Muhyiddin

string_news1 = 'TANGKAK - Tan Sri Muhyiddin Yassin berkata, beliau tidak mahu menyentuh mengenai isu politik buat masa ini, sebaliknya mahu menumpukan kepada soal kebajikan rakyat serta usaha merancakkan semula ekonomi negara yang terjejas berikutan pandemik Covid-19. Perdana Menteri menjelaskan perkara itu ketika berucap pada Majlis Bertemu Pemimpin bersama pemimpin masyarakat Dewan Undangan Negeri (DUN) Gambir di Dewan Serbaguna Bukit Gambir hari ini.'
pprint(string_news1)
('TANGKAK - Tan Sri Muhyiddin Yassin berkata, beliau tidak mahu menyentuh '
 'mengenai isu politik buat masa ini, sebaliknya mahu menumpukan kepada soal '
 'kebajikan rakyat serta usaha merancakkan semula ekonomi negara yang terjejas '
 'berikutan pandemik Covid-19. Perdana Menteri menjelaskan perkara itu ketika '
 'berucap pada Majlis Bertemu Pemimpin bersama pemimpin masyarakat Dewan '
 'Undangan Negeri (DUN) Gambir di Dewan Serbaguna Bukit Gambir hari ini.')
[7]:
# https://www.sinarharian.com.my/article/90021/BERITA/Politik/Tun-Mahathir-Anwar-disaran-bersara-untuk-selesai-kemelut-politik

string_news2 = 'ALOR SETAR - Kemelut politik Pakatan Harapan (PH) belum berkesudahan apabila masih gagal memuktamadkan calon Perdana Menteri yang dipersetujui bersama. Ahli Parlimen Sik, Ahmad Tarmizi Sulaiman berkata, sehubungan itu pihaknya mencadangkan mantan Pengerusi Parti Pribumi Bersatu Malaysia (Bersatu), Tun Dr Mahathir Mohamad dan Presiden Parti Keadilan Rakyat (PKR), Datuk Seri Anwar Ibrahim mengundurkan diri daripada politik sebagai jalan penyelesaian.'
pprint(string_news2)
('ALOR SETAR - Kemelut politik Pakatan Harapan (PH) belum berkesudahan apabila '
 'masih gagal memuktamadkan calon Perdana Menteri yang dipersetujui bersama. '
 'Ahli Parlimen Sik, Ahmad Tarmizi Sulaiman berkata, sehubungan itu pihaknya '
 'mencadangkan mantan Pengerusi Parti Pribumi Bersatu Malaysia (Bersatu), Tun '
 'Dr Mahathir Mohamad dan Presiden Parti Keadilan Rakyat (PKR), Datuk Seri '
 'Anwar Ibrahim mengundurkan diri daripada politik sebagai jalan penyelesaian.')
[8]:
string_news3 = 'Menteri Kanan (Kluster Keselamatan) Datuk Seri Ismail Sabri Yaakob berkata, kelonggaran itu diberi berikutan kerajaan menyedari masalah yang dihadapi mereka untuk memperbaharui dokumen itu. Katanya, selain itu, bagi rakyat asing yang pas lawatan sosial tamat semasa Perintah Kawalan Pergerakan (PKP) pula boleh ke pejabat Jabatan Imigresen yang terdekat untuk mendapatkan lanjutan tempoh.'
pprint(string_news3)
('Menteri Kanan (Kluster Keselamatan) Datuk Seri Ismail Sabri Yaakob berkata, '
 'kelonggaran itu diberi berikutan kerajaan menyedari masalah yang dihadapi '
 'mereka untuk memperbaharui dokumen itu. Katanya, selain itu, bagi rakyat '
 'asing yang pas lawatan sosial tamat semasa Perintah Kawalan Pergerakan (PKP) '
 'pula boleh ke pejabat Jabatan Imigresen yang terdekat untuk mendapatkan '
 'lanjutan tempoh.')
[9]:
# https://qcikgubm.blogspot.com/2018/02/contoh-soalan-dan-jawapan-karangan.html

string_karangan = 'Selain itu, pameran kerjaya membantu para pelajar menentukan kerjaya yang akan diceburi oleh mereka. Seperti yang kita ketahui, pasaran kerjaya di Malaysia sangat luas dan masih banyak sektor pekerjaan di negara ini yang masih kosong kerana sukar untuk mencari tenaga kerja yang benar-benar berkelayakan. Sebagai contohnya, sektor perubatan di Malaysia menghadapi masalah kekurangan tenaga kerja yang kritikal, khususnya tenaga pakar disebabkan peletakan jawatan oleh doktor dan pakar perubatan untuk memasuki sektor swasta serta berkembangnya perkhidmatan kesihatan dan perubatan. Setelah menyedari  hakikat ini, para pelajar akan lebih berminat untuk menceburi bidang perubatan kerana pameran kerjaya yang dilaksanakan amat membantu memberikan pengetahuan am tentang kerjaya ini'
pprint(string_karangan)
('Selain itu, pameran kerjaya membantu para pelajar menentukan kerjaya yang '
 'akan diceburi oleh mereka. Seperti yang kita ketahui, pasaran kerjaya di '
 'Malaysia sangat luas dan masih banyak sektor pekerjaan di negara ini yang '
 'masih kosong kerana sukar untuk mencari tenaga kerja yang benar-benar '
 'berkelayakan. Sebagai contohnya, sektor perubatan di Malaysia menghadapi '
 'masalah kekurangan tenaga kerja yang kritikal, khususnya tenaga pakar '
 'disebabkan peletakan jawatan oleh doktor dan pakar perubatan untuk memasuki '
 'sektor swasta serta berkembangnya perkhidmatan kesihatan dan perubatan. '
 'Setelah menyedari  hakikat ini, para pelajar akan lebih berminat untuk '
 'menceburi bidang perubatan kerana pameran kerjaya yang dilaksanakan amat '
 'membantu memberikan pengetahuan am tentang kerjaya ini')
[10]:
# https://www.parlimen.gov.my/bills-dewan-rakyat.html?uweb=dr#, RUU Kumpulan Wang Simpanan Pekerja (Pindaan) 2019

string_parlimen = 'Subfasal 6(b) bertujuan untuk memasukkan subseksyen baharu 39(3) dan (4) ke dalam Akta 452. Subseksyen (3) yang dicadangkan bertujuan untuk menjadikan suatu kesalahan bagi mana-mana orang yang meninggalkan Malaysia tanpa membayar caruman yang tertunggak dan kena dibayar atau mengemukakan jaminan bagi pembayarannya. Subseksyen (4) yang dicadangkan memperuntukkan bahawa bagi maksud seksyen 39 Akta 452, “caruman” termasuklah apa-apa dividen atau caj lewat bayar yang kena dibayar ke atas mana-mana caruman.'
pprint(string_parlimen)
('Subfasal 6(b) bertujuan untuk memasukkan subseksyen baharu 39(3) dan (4) ke '
 'dalam Akta 452. Subseksyen (3) yang dicadangkan bertujuan untuk menjadikan '
 'suatu kesalahan bagi mana-mana orang yang meninggalkan Malaysia tanpa '
 'membayar caruman yang tertunggak dan kena dibayar atau mengemukakan jaminan '
 'bagi pembayarannya. Subseksyen (4) yang dicadangkan memperuntukkan bahawa '
 'bagi maksud seksyen 39 Akta 452, “caruman” termasuklah apa-apa dividen atau '
 'caj lewat bayar yang kena dibayar ke atas mana-mana caruman.')
[11]:
string_random1 = 'saya menikmati filem mengenai makhluk asing yang menyerang bumi. <> Saya fikir fiksyen sains adalah genre yang luar biasa untuk apa sahaja. Sains masa depan, teknologi, perjalanan masa, perjalanan FTL, semuanya adalah konsep yang menarik. <> Saya sendiri peminat fiksyen sains!'
pprint(string_random1)
('saya menikmati filem mengenai makhluk asing yang menyerang bumi. <> Saya '
 'fikir fiksyen sains adalah genre yang luar biasa untuk apa sahaja. Sains '
 'masa depan, teknologi, perjalanan masa, perjalanan FTL, semuanya adalah '
 'konsep yang menarik. <> Saya sendiri peminat fiksyen sains!')
[12]:
string_random2 = 'Fiksyen sains <> saya menikmati filem mengenai makhluk asing yang menyerang bumi. <> Fiksyen sains (sering dipendekkan menjadi SF atau sci-fi) adalah genre fiksyen spekulatif, biasanya berurusan dengan konsep khayalan seperti sains dan teknologi futuristik, perjalanan angkasa, perjalanan waktu, lebih cepat daripada perjalanan ringan, alam semesta selari, dan kehidupan di luar bumi .'
pprint(string_random2)
('Fiksyen sains <> saya menikmati filem mengenai makhluk asing yang menyerang '
 'bumi. <> Fiksyen sains (sering dipendekkan menjadi SF atau sci-fi) adalah '
 'genre fiksyen spekulatif, biasanya berurusan dengan konsep khayalan seperti '
 'sains dan teknologi futuristik, perjalanan angkasa, perjalanan waktu, lebih '
 'cepat daripada perjalanan ringan, alam semesta selari, dan kehidupan di luar '
 'bumi .')

Comparing with Google Translate

These printscreens taken on 4th July 2020, Google always update model, so Google Translate in the future might improved.

string_news1

[13]:
from IPython.core.display import Image, display

display(Image('string1.png', width=450))
_images/load-translation-ms-en_22_0.png

Tan Sri Muhyiddin Yassin said he did not want to touch on current political issues, instead focusing on the welfare of the people and revitalizing the country’s economy following the Covid-19 pandemic. The prime minister explained this when speaking at a Leaders’ Meetings with leaders of the Gambir State Legislative Assembly (assembly) at the Bukit Gambir Multipurpose Hall today.

string_karangan

[14]:
display(Image('string2.png', width=450))
_images/load-translation-ms-en_25_0.png

Additionally, career fairs help students determine which careers they will pursue. As we know, the job market in Malaysia is very broad and many of the jobs in the country are still vacant because it is difficult to find a truly qualified workforce. For example, the medical sector in Malaysia is facing a significant shortage of labor force, in particular by specialists due to the resignation of doctors and medical professionals to enter the private sector as well as expanding health and medical services. Upon realizing this fact, students will be more interested in the field of medicine as the career exhibitions help provide a wealth of knowledge about this profession.

string_parlimen

[15]:
display(Image('string3.png', width=450))
_images/load-translation-ms-en_28_0.png

Subsection 6 (b) seeks to introduce new subsections 39 (3) and (4) into Act 452. Subsection (3) is intended to make it an offense for any person to leave Malaysia without paying any outstanding and payable contribution or submit a guarantee for payment. The proposed subsection (4) provides that for the purposes of section 39 of Act 452, “contribution” includes any dividend or late payment chargeable on any contribution.

Translate transformer base

[13]:
%%time

pprint(transformer.greedy_decoder([string_news1, string_news2, string_news3]))
['TANGKAK - Tan Sri Muhyiddin Yassin said he did not want to touch on '
 'political issues at the moment, instead focusing on the welfare of the '
 "people and efforts to revitalize the affected country's economy following "
 'the Covid-19 pandemic. The prime minister explained the matter when speaking '
 'at a Leadership Meeting with Gambir State Assembly (DUN) leaders at the '
 'Bukit Gambir Multipurpose Hall today.',
 'ALOR SETAR - Pakatan Harapan (PH) political turmoil has not ended when it '
 "has failed to finalize the Prime Minister's candidate agreed upon. Sik MP "
 'Ahmad Tarmizi Sulaiman said he had suggested former United Nations (UN) '
 "Indigenous Party chairman Tun Dr Mahathir Mohamad and People's Justice Party "
 '(PKR) president Datuk Seri Anwar Ibrahim resign from politics as a solution.',
 'Senior Minister (Security Cluster) Datuk Seri Ismail Sabri Yaakob said the '
 'relaxation was given as the government was aware of the problems they had to '
 'renew the document. He added that for foreigners who had passed the social '
 'visit during the Movement Control Order (CPP) they could go to the nearest '
 'Immigration Department office for further extension.']
CPU times: user 23.9 s, sys: 14.5 s, total: 38.4 s
Wall time: 9.95 s
[14]:
%%time

pprint(quantized_transformer.greedy_decoder([string_news1, string_news2, string_news3]))
['TANGKAK - Tan Sri Muhyiddin Yassin said he did not want to touch on '
 'political issues at the moment, instead focusing on the welfare of the '
 "people and efforts to revitalize the affected country's economy following "
 'the Covid-19 pandemic. The prime minister explained the matter when speaking '
 'at a Leadership Meeting with Gambir State Assembly (DUN) leaders at the '
 'Bukit Gambir Multipurpose Hall today.',
 'ALOR SETAR - Pakatan Harapan (PH) political turmoil has not ended when it '
 "has failed to finalize the Prime Minister's candidate agreed upon. Sik MP "
 'Ahmad Tarmizi Sulaiman said he had suggested former United Nations (UN) '
 "Indigenous Party chairman Tun Dr Mahathir Mohamad and People's Justice Party "
 '(PKR) president Datuk Seri Anwar Ibrahim resign from politics as a solution.',
 'Senior Minister (Security Cluster) Datuk Seri Ismail Sabri Yaakob said the '
 'relaxation was given as the government was aware of the problems they had to '
 'renew the document. He added that for foreigners who had passed the social '
 'visit during the Movement Control Order (CPP) they could go to the nearest '
 'Immigration Department office for an extension.']
CPU times: user 23.5 s, sys: 13.9 s, total: 37.5 s
Wall time: 9.58 s
[15]:
%%time

pprint(transformer.greedy_decoder([string_karangan, string_parlimen]))
['In addition, career exhibitions help students determine their careers. As we '
 'know, the career market in Malaysia is very broad and there are still many '
 'job sectors in the country that are still vacant because it is difficult to '
 'find a truly qualified workforce. For example, the medical sector in '
 'Malaysia is facing a critical shortage of labor, especially specialists due '
 'to the resignation of doctors and physicians to enter the private sector and '
 'develop health and medical services. Upon realizing this fact, students will '
 'be more interested in medicine because the exhibition careers are very '
 'helpful in providing general knowledge of this career.',
 'Subclause 6 (b) seeks to introduce new subsections 39 (3) and (4) into Act '
 '452. Subsection (3) proposed to make an offense for any person leaving '
 'Malaysia without paying a deferred and payable contribution or filing a '
 'guarantee for payment. Subsection (4) proposed provides that for the purpose '
 'of section 39 of Act 452, the "contribution" includes any dividend or late '
 'payment charge payable on any contribution.']
CPU times: user 30.8 s, sys: 17 s, total: 47.7 s
Wall time: 10.6 s
[16]:
%%time

pprint(quantized_transformer.greedy_decoder([string_karangan, string_parlimen]))
['In addition, career exhibitions help students determine their careers. As we '
 'know, the career market in Malaysia is very broad and there are still many '
 'job sectors in the country that are still vacant because it is difficult to '
 'find a truly qualified workforce. For example, the medical sector in '
 'Malaysia is facing a critical shortage of labor, especially specialists due '
 'to the resignation of doctors and physicians to enter the private sector and '
 'develop health and medical services. Upon realizing this fact, students will '
 'be more interested in the medical field as the career exhibitions are very '
 'helpful to provide general knowledge of this career.',
 'Subclause 6 (b) seeks to introduce new subsections 39 (3) and (4) into Act '
 '452. Subsection (3) proposed to make an offense for any person leaving '
 'Malaysia without paying a deferred and payable contribution or to submit a '
 'guarantee for his payment. Subsection (4) proposed provides that for the '
 'purpose of section 39 of Act 452, the "contribution" includes any dividend '
 'or late payment charge payable on any contribution.']
CPU times: user 30.9 s, sys: 16.8 s, total: 47.7 s
Wall time: 10.8 s
[17]:
%%time

result = transformer.greedy_decoder([string_random1, string_random2])
pprint(result)
['I enjoy movies about aliens attacking the earth. <> I think science fiction '
 'is an incredible genre for anything. Future science, technology, time '
 "travel, FTL travel, everything is an exciting concept. <> I'm a science "
 'fiction fan!',
 'Science fiction <> I enjoy movies about aliens invading the earth. <> '
 'Science fiction (often shortened to SF or sci-fi) is a genre of speculative '
 'fiction, usually dealing with imaginary concepts such as science and '
 'futuristic technology, space travel, time travel, faster than light travel, '
 'parallel universe, and life abroad.']
CPU times: user 19.1 s, sys: 10.3 s, total: 29.5 s
Wall time: 6.77 s
[18]:
%%time

result = quantized_transformer.greedy_decoder([string_random1, string_random2])
pprint(result)
['I enjoy movies about aliens attacking the earth. <> I think science fiction '
 'is an incredible genre for anything. Future science, technology, time '
 "travel, FTL travel, everything is an exciting concept. <> I'm a science "
 'fiction fan!',
 'Science fiction <> I enjoy movies about aliens invading the earth. <> '
 'Science fiction (often shortened to SF or sci-fi) is a genre of speculative '
 'fiction, usually dealing with imaginary concepts such as science and '
 'futuristic technology, space travel, time travel, faster than light travel, '
 'parallel universe, and life abroad.']
CPU times: user 19.2 s, sys: 11.7 s, total: 30.9 s
Wall time: 6.66 s

Translate transformer small

[19]:
%%time

pprint(transformer_small.greedy_decoder([string_news1, string_news2, string_news3]))
['TANGKAK - Tan Sri Muhyiddin Yassin said he did not want to touch on '
 'political issues at this time, instead focusing on the welfare of the people '
 "and efforts to revitalize the country's economy affected following the "
 'Covid-19 pandemic. The Prime Minister explained the matter when speaking at '
 'the Leaders Meeting with the leaders of the Gambir State Assembly (DUN) '
 'community at the Bukit Gambir Multipurpose Hall today.',
 'ALOR SETAR - Pakatan Harapan (PH) political turmoil has not been expected '
 "when it still fails to finalize the Prime Minister's candidate agreed "
 'together. Sik MP Ahmad Tarmizi Sulaiman said the party had suggested former '
 'United Nations Indigenous Party (UN) chairman Tun Dr Mahathir Mohamad and '
 "President of the People's Justice Party (PKR), Datuk Seri Anwar Ibrahim "
 'resigned from politics as a solution.',
 'Senior Minister (Security Cluster) Datuk Seri Ismail Sabri Yaakob said the '
 'relaxation was given as the government was aware of the problems they faced '
 'to renew the document. He said in addition, for foreigners who had passed '
 'the social visit expired during the Movement Control Order (PKP) could go to '
 'the nearest Immigration Department office for further time.']
CPU times: user 3.46 s, sys: 871 ms, total: 4.33 s
Wall time: 1.5 s
[20]:
%%time

pprint(transformer_small.greedy_decoder([string_karangan, string_parlimen]))
['In addition, career exhibitions help students determine their careers. As we '
 'know, the career market in Malaysia is very broad and many employment '
 'sectors in the country are still vacant because it is difficult to find a '
 'truly qualified workforce. For example, the medical sector in Malaysia is '
 'facing critical labor shortages, especially specialists as specialists '
 'resign as well as medical professionals to enter the private sector and the '
 'development of health and medical services. After realizing this fact that '
 'students will be more interested in medicine as the exhibition of careers is '
 'helping to provide general knowledge of this career in this career as this '
 'career is especially in providing general knowledge of this career as this '
 'career as this career is difficult to gain knowledge of this career as it is '
 'difficult to gain knowledge of this career is difficult to find the career '
 'as it is difficult to find the career as it is difficult to enter this '
 'career as it is difficult to enter this career as it is difficult to gain '
 'through this career is difficult to enter the private sector and medical '
 'career as it is difficult to find the field of career as it is difficult to '
 'gain knowledge of this career as it is difficult to find the field of career '
 'as it is difficult to find the field of career as it is especially in the '
 'field of career as it is difficult to find the field of this career as it is '
 'difficult to find the field of career as it is difficult to find that it is '
 'difficult to find that it is difficult to',
 'Subclaal 6 (b) aims to include a new subsection of 39 (3) and (4) into Act '
 '452. Subsection (3) proposed aimed at making a mistake for any person who '
 'leaves Malaysia without paying for outstanding contributions and paid or to '
 'provide guarantees for his payment. Subsection (4) proposed provides that '
 'for the purpose of section 39 of Act 452, "contributions" including any '
 'dividend or late payment charge to be paid to any contribution.']
CPU times: user 9.94 s, sys: 1.8 s, total: 11.7 s
Wall time: 3.14 s
[21]:
%%time

result = transformer_small.greedy_decoder([string_random1, string_random2])
pprint(result)
['I enjoy movies about aliens attacking the earth. <> I think science fiction '
 'is a great genre for whatever future science, technology, travel, FTL '
 'travel, all of which is an interesting concept. <> I personally love science '
 'fiction!',
 'science fiction <> I enjoy movies about aliens who attack the earth. <> The '
 'science fiction (often shortened to SF or sci-fi) is a speculative fiction '
 'genre, usually dealing with the concept of imaginary science and futuristic '
 'technology, space travel, travel, faster than light travel, parallel '
 'universe, and outer life.']
CPU times: user 2.64 s, sys: 402 ms, total: 3.04 s
Wall time: 773 ms

Translate transformer large

[ ]:
%%time

pprint(transformer_large.greedy_decoder([string_news1, string_news2, string_news3]))
[ ]:
%%time

pprint(transformer_large.greedy_decoder([string_karangan, string_parlimen]))
[ ]:
%%time

result = transformer_large.greedy_decoder([string_random1, string_random2])
pprint(result)