Long Text Translation#

This tutorial is available as an IPython notebook at Malaya/example/long-text-translation.

This module only trained on standard language structure, so it is not save to use it for local language structure.

[1]:
%%time
import malaya
CPU times: user 4.71 s, sys: 916 ms, total: 5.63 s
Wall time: 6.2 s

List available Transformer models#

[2]:
malaya.translation.ms_en.available_transformer()
INFO:root:tested on 100k MS-EN sentences.
[2]:
Size (MB) Quantized Size (MB) BLEU Suggested length
small 42.7 13.4 0.626 256.0
base 234.0 82.7 0.792 256.0
large 815.0 244.0 0.714 256.0
bigbird 246.0 63.7 0.678 1024.0
small-bigbird 50.4 13.1 0.586 1024.0

If you look at Suggested length, bigbird 4x longer than normal transformer models, able to infer longer text without need to partition. Let’s we do some examples.

We are going to compare small and base models with small-bigbird and bigbird models for MS-EN translation task. Feel free to test on EN-MS translation.

Load Transformer models#

def transformer(model: str = 'base', quantized: bool = False, **kwargs):
    """
    Load Transformer encoder-decoder model to translate MS-to-EN.

    Parameters
    ----------
    model : str, optional (default='base')
        Model architecture supported. Allowed values:

        * ``'small'`` - Transformer SMALL parameters.
        * ``'base'`` - Transformer BASE parameters.
        * ``'large'`` - Transformer LARGE parameters.
        * ``'bigbird'`` - BigBird BASE parameters.
        * ``'small-bigbird'`` - BigBird SMALL parameters.

    quantized : bool, optional (default=False)
        if True, will load 8-bit quantized model.
        Quantized model not necessary faster, totally depends on the machine.

    Returns
    -------
    result: model
        if `bigbird` in model, return malaya.model.bigbird.Translation
        else, return malaya.model.tf.Translation
    """
[3]:
transformer = malaya.translation.ms_en.transformer()
transformer_small = malaya.translation.ms_en.transformer(model = 'small')
[4]:
bigbird = malaya.translation.ms_en.transformer(model = 'bigbird')
quantized_bigbird = malaya.translation.ms_en.transformer(model = 'bigbird', quantized = True)
WARNING:root:Load quantized model will cause accuracy drop.
[5]:
bigbird_small = malaya.translation.ms_en.transformer(model = 'small-bigbird')
quantized_bigbird_small = malaya.translation.ms_en.transformer(model = 'small-bigbird', quantized = True)
WARNING:root:Load quantized model will cause accuracy drop.

Long text examples#

[6]:
# https://www.bharian.com.my/berita/nasional/2021/02/785400/kabinet-tidak-pernah-bangkitkan-isu-konflik-kepentingan-najib
string = """
KUALA LUMPUR: Mahkamah Tinggi di sini, hari ini, diberitahu Kabinet tidak pernah membangkitkan isu mengenai konflik kepentingan Datuk Seri Najib Razak dalam 1Malaysia Development Berhad (1MDB).

Menurut Kod Etika Bagi Anggota Pentadbiran dan Ahli Parlimen adalah menjadi amalan ahli Kabinet untuk mengisytiharkan konflik kepentingan sekiranya mempunyai pembabitan di dalam hal yang dibincangkan dalam Mesyuarat Jemaah Menteri.

Perkara itu dimaklumkan bekas Timbalan Ketua Setiausaha (Kabinet) Bahagian Kabinet Perlembagaan dan Perhubungan Jabatan Perdana Menteri (JPM), Tan Sri Mazidah Abdul Majid, dalam keterangannya pada perbicaraan kes 1MDB yang dihadapi Najib.

Kod etika itu antara lain turut menyatakan bahawa ahli Kabinet yang mempunyai kepentingan peribadi dan bercanggah dengan kepentingan kerajaan, atau membabitkan ahli keluarga, perlu meninggalkan mesyuarat dan merekodkan ketidakhadiran mereka.

Di dalam 1MDB, Najib memegang tiga jawatan iaitu Perdana Menteri, Menteri Kewangan dan Pengerusi Pengerusi Badan Penasihat 1MDB

Menjawab soalan peguam Tan Sri Muhammad Shafee Abdullah, sama ada terdapat ahli Kabinet yang membangkitkan isu bahawa Menteri Kewangan tidak seharusnya membabitkan diri dalam perbincangan itu kerana konflik kepentingan, Maziah menjawab: "Tiada."

Muhammad Shafee: Ada sesiapa yang membangkitkan perkara berhubung hal 1MBD dengan Najib?

Maziah: Tidak

Muhammad Shafee: Timbalan Perdana Menteri (ketika itu) adalah Tan Sri Muhyiddin Yassin, manakala Datuk Seri Ahmad Husni Hanadzlah adalah bekas Menteri Kewangan II. Mereka juga tidak pernah membangkitkan hal konflik kepentingan?

Mazidah: Ya, tidak pernah.

Sementara itu, menjawab soalan Timbalan Pendakwa Raya, Ahmad Akram Gharib sama ada beliau mengetahui bahawa Najib mempunyai kepentingan peribadi dalam 1MDB, Mazidah berkata: "Tidak"

Ahmad Akram: Adakah anda mengetahui bahawa Najib secara peribadi menerima duit daripada 1MDB?

Maziah: Tidak

Ahmad Akram: Sekiranya Najib secara peribadi menerima wang daripada 1MDB adakah itu konflik kepentingan dan melanggar Kod Etika Bagi Anggota Pentadbiran dan Ahli Parlimen.

Maziah: Menurut pandangan peribadi saya, ya, namun kerana ia membabitkan menteri dan perdana menteri, saya cadangkan untuk dapatkan pandangan Peguam Negara.

Terdahulu, di awal prosiding, Maziah turut memberitahu mahkamah bahawa Najib tidak pernah menyebut nama ahli perniagaan dalam buruan, Low Taek Jho atau Jho Low pada mesyuarat Kabinet sebagai individu yang membantu beliau mendapatkan sumbangan daripada kerabat diraja Arab Saudi.

"Sekiranya perkara itu dimaklumkan kepada Kabinet, maka, ia akan dicatat dalam minit mesyuarat," katanya.

Tambah Maziah, beliau hanya mendengar dan mengetahui mengenai nama Jho Low selepas timbul isu membabitkan 1MDB.

Najib, 68, menghadapi empat pertuduhan menggunakan kedudukannya untuk memperoleh suapan berjumlah RM2.3 bilion daripada dana 1MDB dan 21 pertuduhan pengubahan wang haram membabitkan jumlah wang yang sama.

Perbicaraan di hadapan Hakim Collin Lawrence Sequerah bersambung Isnin ini.
"""
[7]:
# https://www.astroawani.com/berita-malaysia/laporan-audit-1mdb-najib-gagal-gugurkan-sri-ram-daripada-pasukan-pendakwaan-283003
string2 = """
KUALA LUMPUR: Datuk Seri Najib Tun Razak hari ini gagal dalam satu lagi cubaannya untuk menggugurkan bekas Hakim Mahkamah Persekutuan Datuk Seri Gopal Sri Ram daripada mengetuai pasukan pendakwaan dalam kes meminda laporan audit 1Malaysia Development Berhad (1MDB) melibatkan bekas perdana menteri itu.
Ini merupakan cubaan kali ketiga Najib untuk menggugurkan Sri Ram sebagai pendakwa dalam kes jenayah berkaitan 1MDB itu. Sebelum ini, satu permohonan difailkan dalam satu lagi kes 1MDB di hadapan hakim berbeza dan cubaan kedua menerusi prosiding sivil.
Ketika menolak permohonan Ahli Parlimen Pekan itu, Hakim Mahkamah Tinggi Mohamed Zaini Mazlan berkata dakwaan Najib bahawa Sri Ram terlibat dalam siasatan terhadapnya sebagai tidak ada merit.
“Tidak ada bukti kukuh untuk menyokong dakwaan pemohon dan kekal sebagai hipotesis semata-mata. Isu ini telah disiasat oleh pihak pendakwaan selaku responden semasa permohonan pemohon (Najib) yang terdahulu.
“Isu ini sudah dibincangkan dan diputuskan. Keputusan oleh mahkamah lain sebelum ini kekal dan tidak boleh diterbalikkan,” kata hakim.
Mohamed Zaini ketika memberi alasan penolakan berkata pemohon, antara lain, menjadikan komunikasi di antara bekas Peguam Negara Tan Sri Mohamed Apandi Ali dan Sri Ram sebagai bukti berat sebelah Sri Ram terhadap pemohon dan kebimbangan pemohon berhubung perkara itu adalah tidak berasas.
“Seperti juga individu lain, Sri Ram berhak mempunyai pandangan peribadi. Itu sahaja. Namun, pertimbangan berbeza akan dibuat jika beliau menunjukkan sikap berat sebelah semasa melaksanakan tugas sebagai pendakwa raya kanan. Pandangan peribadi beliau tidak boleh dianggap akan menghalang tanggungjawab beliau sebagai pendakwa raya kanan.
“Tambahan pula, kejadian itu, seperti yang dikemukakan oleh responden, berlaku ketika sebelum pelantikan Sri Ram sebagai pendakwa raya kanan. Turut penting ialah pemohon tidak membuat sebarang aduan mengenai tindak-tanduk Sri Ram ketika menjalankan perbicaraan melibatkan pemohon. Ini mengukuhkan hujah responden bahawa Sri Ram bersikap terbuka semasa menjalankan tugas sebagai pendakwa raya kanan,” kata hakim.
Mohamed Zaini seterusnya berkata pada akhirnya, mahkamah bertanggungjawab memastikan sesebuah perbicaraan dilaksanakan secara adil demi mendapatkan keadilan.
“Mahkamah akan membantu mana-mana pihak yang dilayan secara tidak adil, jika perkara tersebut berlaku. Sehubunggan itu, permohonan pemohon ditolak,” katanya.
Perbicaraan kes laporan audit 1MDB itu akan bersambung pada 22 Feb ini.
Pada prosiding hari ini, Timbalan Pendakwa Raya Ahmad Akram Gharib bertindak bagi pihak pendakwaan, manakala Najib diwakili peguam Nur Syahirah Hanapiah.
Najib, 67, dan bekas Ketua Pegawai Eksekutif 1MDB Arul Kanda Kandasamy, 45, dibicarakan atas tuduhan meminda laporan audit 1MDB.
Ahli Parlimen Pekan itu dituduh menggunakan kedudukannya untuk mengarahkan pindaan ke atas laporan audit akhir 1MDB sebelum dibentangkan kepada Jawatankuasa Kira-Kira Wang Negara bagi mengelakkan sebarang tindakan diambil terhadapnya, sementara Arul Kanda didakwa bersubahat dengan Najib dalam membuat pindaan ke atas laporan tersebut bagi melindungi Najib daripada dikenakan tindakan.
"""
[8]:
# https://miasa.org.my/blog/2018/08/artikel-sokongan-rakan-senasib-dapat-mengurangkan-kemasukan-semula/
string3 = """
3 Ogos 2018 – Kajian terbaharu yang dijalankan oleh University College London (UCL) mendapati khidmat penjagaan kesihatan mental daripada pekerja sokongan (support worker) yang pernah mengharungi sendiri penyakit ini mungkin boleh membantu mengurangkan kebarangkalian pesakit yang baru keluar dari penjagaan kesihatan mental akut (acute mental health care) daripada dimasukkan semula ke unit berkenaan.



Kertas penyelidikan tersebut yang diterbitkan pada hari ini dalam jurnal The Lancet mendapati bilangan pesakit mental yang dimasukkan semula ke unit penjagaan akut dalam tempoh setahun ialah 24% jauh lebih rendah bagi kumpulan pesakit yang ditawarkan pekerja sokongan berbentuk rakan senasib, berbanding dengan kumpulan pesakit yang hanya diberi buku kerja pemulihan peribadi sahaja.



“Pesakit yang dibenarkan keluar (discharge) dari perkhidmatan krisis komuniti sering dimasukkan semula ke unit penjagaan akut. Ini bukan sahaja membantutkan pemulihan, malah menggunapakai sumber yang sepatutnya dikhususkan untuk penambahbaikan jangka panjang fungsi dan kualiti hidup pesakit. Pekerja sokongan daripada golongan rakan senasib berupaya memberikan sokongan dan dorongan yang benar-benar mesra dan berempati kerana ianya datang daripada pengalaman peribadi mereka sendiri, di samping menjadi contoh tauladan (role model) yang baik untuk pemulihan pesakit,” kata pengarang utama, Profesor Sonia Johnson (Psikiatri, UCL).

Di United Kingdom (UK), lebih separuh daripada jumlah pesakit yang dimasukkan ke unit penjagaan akut dimasukkan semula ke unit berkenaan dalam tempoh setahun, namun tiada bukti kukuh yang menjelaskan kaedah untuk mengurangkan jumlah ini.



Sebenarnya, sokongan daripada individu yang pernah mengalami masalah kesihatan mental telah dipraktikkan dalam program seperti Pelaksanaan Pemulihan melalui Perubahan Organisasi NHS (UK) dan juga Pelan Tindakan Pemulihan Kesihatan Amerika Syarikat. Penyelidikan ini merupakan ujian terkawal rawak pertama yang seumpamanya untuk menilai keberkesanan program sokongan rakan senasib. Walaupun begitu, masih banyak penyelidikan yang perlu dijalankan sebelum strategi ini boleh dilaksanakan secara menyeluruh di UK, contohnya untuk memahami sebab-sebab mengapa program ini berkesan.



Pastinya, campur tangan atau intervensi dalam pengurusan kendiri (self-management) mungkin boleh membantu pesakit menguruskan kesihatan mental mereka dengan lebih baik. Dalam kajian ini, penyelidik menyediakan peserta sama ada dengan buku kerja pemulihan peribadi sahaja (220 orang) atau buku kerja berserta bantuan pekerja sokongan rakan senasib (221 orang). Peserta juga dibenarkan meneruskan penjagaan biasa mereka.



Kajian ini dijalankan menerusi enam buah pasukan penyelesaian krisis kesihatan mental (crisis resolution team) di England dan peserta dipilih hanya selepas mereka dibenarkan keluar dari unit krisis oleh pasukan penyelesaian krisis. Peserta kajian terdiri daripada pelbagai diagnosis antaranya termasuk skizofrenia, bipolar, psikosis, kemurungan, kecelaruan keresahan, kecelaruan stres pascatrauma (PTSD), dan kecelaruan personaliti.



Peserta yang menerima sokongan rakan senasib ditawarkan 10 sesi pertemuan mingguan selama satu jam. Dalam sesi pertemuan ini, pekerja sokongan mendengar masalah peserta dan bermatlamat menyemaikan harapan dengan berkongsi kemahiran dan strategi pengurusan penyakit (coping strategies) yang dikuasai sewaktu mereka sendiri berada dalam proses pemulihan. Kesemua pekerja sokongan diberikan latihan terlebih dahulu dalam kemahiran mendengar, kesedaran budaya, pendedahan kendiri, dan kerahsiaan, serta cara menggunakan buku kerja.

Rekod kesihatan peserta dipantau oleh penyelidik selama satu tahun untuk menentukan sama ada peserta dimasukkan semula ke unit penjagaan akut (seperti wad pesakit akut, pasukan penyelesaian krisis, rumah krisis, dan perkhidmatan penjagaan harian akut) atau tidak.



Selepas tamat tempoh satu tahun, kajian mendapati peratusan kemasukan semula ke unit penjagaan akut adalah lebih rendah bagi kumpulan peserta yang menerima intervensi berbanding dengan kelompok kawalan— dengan 29% peserta dimasukkan semula setelah menerima sokongan rakan senasib, berbanding dengan 38% peserta yang hanya menerima buku kerja.



Kadar penyerapan intervensi juga baik— 72% daripada peserta yang ditawarkan khidmat pekerja sokongan rakan senasib dan buku kerja menghadiri sekurang-kurangnya tiga sesi pertemuan, manakala satu pertiga menghadiri kesemua 10 sesi pertemuan.

“Kami sedia maklum bahawa banyak pengguna perkhidmatan penjagaan krisis kesihatan mental merasakan perkhidmatan ini terhenti secara mendadak kerana kurangnya penjagaan susulan yang siap sedia. Kajian kami menunjukkan bahawa pekerja sokongan rakan senasib dapat membantu mengisi kelompangan ini dengan membantu pengguna perkhidmatan (pesakit) membangunkan strategi pengurusan kendiri dan pelan pemulihan dengan cara tersendiri dan lebih bermakna, seterusnya dapat membantu pesakit terus bertahan sesudah berlakunya sesuatu krisis,” kata pengarang bersama, Dr Brynmor Lloyd-Evans (Psikiatri, UCL).
"""

do simple cleaning, remove newline and weird characters.

[9]:
import re
from unidecode import unidecode

def cleaning(string):
    return re.sub(r'[ ]+', ' ', unidecode(string.replace('\n', ' '))).strip()
[10]:
string = cleaning(string)
string2 = cleaning(string2)
string3 = cleaning(string3)
[11]:
len(string.split()), len(string2.split()), len(string3.split())
[11]:
(406, 433, 649)

Translate using greedy decoder#

[12]:
from pprint import pprint
[13]:
pprint(transformer.greedy_decoder([string]))
['KUALA LUMPUR: The High Court here today said the Cabinet had not raised the '
 "issue of Datuk Seri Najib Razak's conflict in 1Malaysia Development Berhad "
 '(1MDB). According to the Code of Ethics for Administrators and Members of '
 'Parliament it is customary practice for Cabinet members to declare conflicts '
 'of interest if they have been involved in matters discussed at the 1MDB '
 'Cabinet. The matter was announced by the former Deputy Chief Secretary '
 '(Cabinet) of the Constitutional and Liaison Division of the Prime Minister, '
 'Tan Sri Mazidah Abdul Majid, in his statement at the 1MDB hearing. The code '
 'of ethics is also stated that the Cabinet members have personal interests '
 'and conflicts with the government, or involving family members, had to leave '
 'meetings and record their absence. In 1MDB, Najib held three positions, '
 "namely the Prime Minister's Office of Finance and the Prime Minister's "
 "Chairman of the Prime Minister's Office of the Prime Minister, namely Tan "
 'Sri Mazram Abdullah Abdullah Abdullah Shafee Mazidah Mazghani Mazram, who '
 'had been informed in the former cabinet of the matter of the 1MDB.']
[14]:
pprint(transformer_small.greedy_decoder([string]))
['KUALA LUMPUR: The High Court here today, said that the Cabinet had never '
 "raised the issue of Datuk Seri Najib Razak's interest in 1Malaysia "
 'Development Berhad (1MDB). According to the Code of Ethics For Members of '
 'the Administration and Member of Parliament is the practice of Cabinet '
 'members to declare conflict of interest if he had been discussed in matters '
 'discussed at the Cabinet. Article. The matter was informed that former '
 'Secretary-General (Cabinet) of the Cabinet, he said. Tan Sri Mazidah Abdul '
 'Majid, in his statement at the 1MDB trial, in his statement at the 1MDB '
 'case, Najib, the case of the 1MDB, he said. Najib, he had not been informed '
 'that the case of the case, Najib, he said. Najib, he said. Najib, he had not '
 'been informed that the case, he had informed that the case, Najib had '
 'informed that the 1MDB, he had not been informed that the case, he had '
 'informed that the 1MDB, he had informed that the case, he had given that he '
 'had informed that the case, he had informed that the case, he had informed '
 'of the case, Najib, he had given that the case, he had informed that the '
 'case, Najib, he had not been informed that the 1MDB, he had not been '
 'informed that he had informed that the case, he had given the case, he had '
 'given him that the case, he had not been informed that the case, Najib, he '
 'had informed that he had informed that he had informed that he had been '
 'informed that he had informed that the 1MDB, he had informed of the 1MDB, he '
 "had informed that the 1MDB, he had informed that the 1MDB's case, Najib, he "
 'had informed of the case. Najib, he had informed that the case. Najib, he '
 'had been that the case, he had not been informed that he had been informed '
 'that he had informed of the case, he had informed that the 1MDB, he had been '
 'informed that the case, he had been informed that the 1MDB, Najib, he had '
 'been that he had informed of the case. Najib, he had informed of the case, '
 'he had been informed of the 1MDB, he had not been informed of the case. '
 'Najib, he had not been informed of the case. Najib, he had not been informed '
 'of the case. Najib, he had been informed of the case, he had been that the '
 'case, he had informed that the 1MDB, he had informed of the case, he had '
 'been informed of the case. Najib, he had been informed of the case. Najib, '
 'he had been informed that the case. Najib, he had not been informed that the '
 'case. Najib, he had informed of the case. Najib, he had informed that the '
 'case. Najib, he had informed that the case. Najib, he had been informed that '
 'he had been informed that the 1MDB, he had informed that the case, in his '
 'case. Najib, in the case. Najib, and Najib, he had informed that the case. '
 'Najib, he had informed that the case, he had informed that the case, he had '
 'informed that the case. Najib, he had informed that the case. Najib, he had '
 'given him that the case, he had not been informed that the case, he had been '
 'that the case, he had not been informed that he had not been informed that '
 'the case, he had informed that he had not been informed that he had informed '
 'that he had been informed that the case, he had been that the case. Najib, '
 'he had informed that he had informed that the case. Najib, he had informed '
 'that he had been informed that he had been informed of the case. Najib, he '
 'had been informed of the case. Najib, he had been informed that he had been '
 'informed that he had been informed that the case. Najib, he had been '
 'informed that the case. Najib, he had been informed that he had repeatedly '
 'informed that the 1MDB, he had been informed that the 1MDB, he had been '
 'informed of the case. Najib, he had informed that the case. Najib, and '
 'Najib, he had informed that the 1MDB, he had informed that the case. Najib, '
 'and Najib, he had not been that the case. Najib, and that he had informed '
 'that the case. Najib, he had informed that he had informed that he had '
 'repeatedly informed that he had informed that the case, he had informed that '
 'the 1MDB, he had been informed that the case of the 1MDB, he had been '
 'informed that he had informed that he had informed that he had informed that '
 'he had not been informed that he had been informed that the case. Najib, and '
 "that the 1MDB's case. Najib, he had informed that he had not informed that "
 'the case. Najib, and that the case. Najib, he had not been informed that the '
 'case. Najib, he had informed of the case of the case. Najib, he had informed '
 'that the case. Najib, he had not been informed that the case. Najib, he had '
 'informed that he had not been informed that he had repeatedly informed of '
 'the 1MDB, he had not been informed that he had informed of the case. Najib, '
 'he had been that the case, he had informed that the case. Najib, and Najib, '
 'he had been informed that the case, he had repeatedly informed that the '
 'case. Najib, he had given the case. Najib, and that the case. Najib, he had '
 'repeatedly, he']

Normal transformer models not able to infer long texts, we need to partition the text, example, split the text by full stop.

[15]:
partition_string = malaya.text.function.split_into_sentences(string)
partition_string
[15]:
['KUALA LUMPUR: Mahkamah Tinggi di sini, hari ini, diberitahu Kabinet tidak pernah membangkitkan isu mengenai konflik kepentingan Datuk Seri Najib Razak dalam 1Malaysia Development Berhad (1MDB).',
 'Menurut Kod Etika Bagi Anggota Pentadbiran dan Ahli Parlimen adalah menjadi amalan ahli Kabinet untuk mengisytiharkan konflik kepentingan sekiranya mempunyai pembabitan di dalam hal yang dibincangkan dalam Mesyuarat Jemaah Menteri.',
 'Perkara itu dimaklumkan bekas Timbalan Ketua Setiausaha (Kabinet) Bahagian Kabinet Perlembagaan dan Perhubungan Jabatan Perdana Menteri (JPM), Tan Sri Mazidah Abdul Majid, dalam keterangannya pada perbicaraan kes 1MDB yang dihadapi Najib.',
 'Kod etika itu antara lain turut menyatakan bahawa ahli Kabinet yang mempunyai kepentingan peribadi dan bercanggah dengan kepentingan kerajaan, atau membabitkan ahli keluarga, perlu meninggalkan mesyuarat dan merekodkan ketidakhadiran mereka.',
 'Di dalam 1MDB, Najib memegang tiga jawatan iaitu Perdana Menteri, Menteri Kewangan dan Pengerusi Pengerusi Badan Penasihat 1MDB Menjawab soalan peguam Tan Sri Muhammad Shafee Abdullah, sama ada terdapat ahli Kabinet yang membangkitkan isu bahawa Menteri Kewangan tidak seharusnya membabitkan diri dalam perbincangan itu kerana konflik kepentingan, Maziah menjawab: "Tiada".',
 'Muhammad Shafee: Ada sesiapa yang membangkitkan perkara berhubung hal 1MBD dengan Najib?',
 'Maziah: Tidak Muhammad Shafee: Timbalan Perdana Menteri (ketika itu) adalah Tan Sri Muhyiddin Yassin, manakala Datuk Seri Ahmad Husni Hanadzlah adalah bekas Menteri Kewangan II.',
 'Mereka juga tidak pernah membangkitkan hal konflik kepentingan?',
 'Mazidah: Ya, tidak pernah.',
 'Sementara itu, menjawab soalan Timbalan Pendakwa Raya, Ahmad Akram Gharib sama ada beliau mengetahui bahawa Najib mempunyai kepentingan peribadi dalam 1MDB, Mazidah berkata: "Tidak" Ahmad Akram: Adakah anda mengetahui bahawa Najib secara peribadi menerima duit daripada 1MDB?',
 'Maziah: Tidak Ahmad Akram: Sekiranya Najib secara peribadi menerima wang daripada 1MDB adakah itu konflik kepentingan dan melanggar Kod Etika Bagi Anggota Pentadbiran dan Ahli Parlimen.',
 'Maziah: Menurut pandangan peribadi saya, ya, namun kerana ia membabitkan menteri dan perdana menteri, saya cadangkan untuk dapatkan pandangan Peguam Negara.',
 'Terdahulu, di awal prosiding, Maziah turut memberitahu mahkamah bahawa Najib tidak pernah menyebut nama ahli perniagaan dalam buruan, Low Taek Jho atau Jho Low pada mesyuarat Kabinet sebagai individu yang membantu beliau mendapatkan sumbangan daripada kerabat diraja Arab Saudi.',
 '"Sekiranya perkara itu dimaklumkan kepada Kabinet, maka, ia akan dicatat dalam minit mesyuarat," katanya.',
 'Tambah Maziah, beliau hanya mendengar dan mengetahui mengenai nama Jho Low selepas timbul isu membabitkan 1MDB.',
 'Najib, 68, menghadapi empat pertuduhan menggunakan kedudukannya untuk memperoleh suapan berjumlah RM2.3 bilion daripada dana 1MDB dan 21 pertuduhan pengubahan wang haram membabitkan jumlah wang yang sama.',
 'Perbicaraan di hadapan Hakim Collin Lawrence Sequerah bersambung Isnin ini.']
[16]:
pprint(transformer.greedy_decoder(partition_string))
['KUALA LUMPUR: The High Court here today said the Cabinet had never raised '
 "the issue of Datuk Seri Najib Razak's conflict of interest in 1Malaysia "
 'Development Berhad (1MDB).',
 'According to the Code of Ethics for Administrative Members and Members of '
 'Parliament it is the practice of Cabinet members to declare conflicts of '
 'interest if they have any involvement in the matter discussed at the Cabinet '
 'Meeting.',
 'The matter was informed by former Deputy Secretary-General (Cabinet) of the '
 "Constitutional and Liaison Division of the Prime Minister's Department "
 '(JPM), Tan Sri Mazidah Abdul Majid, in his testimony at the 1MDB trial '
 'hearing facing Najib.',
 'The code of ethics, among others, states that Cabinet members who have '
 'personal interests and conflict with the interests of the government, or '
 'involve family members, must leave meetings and record their absence.',
 'In 1MDB, Najib held three positions namely Prime Minister, Finance Minister '
 'and Chairman of 1MDB Advisory Board Answering the question of lawyer Tan Sri '
 'Muhammad Shafee Abdullah, whether there were Cabinet members who raised the '
 'issue that the Minister of Finance should not be involved in the discussion '
 'because of conflict of interest, Maziah replied: "None".',
 'Muhammad Shafee: Is there anyone raising questions about 1MBD with Najib?',
 'Maziah: Muhammad Shafee: Deputy Prime Minister (then) was Tan Sri Muhyiddin '
 'Yassin, while Datuk Seri Ahmad Husni Hanadzlah was the former Minister of '
 'Finance II.',
 'They have never raised any issues of conflict of interest?',
 'Mazidah: Yes, never.',
 "Meanwhile, responding to Deputy Public Prosecutor Ahmad Akram Gharib's "
 'question as to whether he knew Najib had personal interests in 1MDB, Mazidah '
 'said: "No" Ahmad Akram: Do you know that Najib personally received money '
 'from 1MDB?',
 'Maziah: No Ahmad Akram: If Najib personally receives money from 1MDB whether '
 'it is a conflict of interest and violates the Code of Ethics for '
 'Administrative Members and Members of Parliament.',
 'Maziah: In my personal opinion, yes, but because it involves ministers and '
 "prime ministers, I suggest to get the Attorney General's views.",
 'Earlier, at the beginning of the proceedings, Maziah also told the court '
 'that Najib had never mentioned the name of the businessman in the game, Low '
 'Taek Jho or Jho Low at a Cabinet meeting as an individual who helped him get '
 "donations from Saudi Arabia's royal family.",
 '"If the matter is communicated to the Cabinet, then it will be recorded at '
 'the minutes of the meeting," he said.',
 "Maziah added that he only heard and knew about Jho Low's name after the 1MDB "
 'issue.',
 'Najib, 68, faces four charges of using his position to obtain RM2.3 billion '
 'in bribes from 1MDB funds and 21 counts of money laundering involving the '
 'same amount.',
 'The trial before Judge Collin Lawrence Sequerah continues this Monday.']
[17]:
pprint(transformer_small.greedy_decoder(partition_string))
['KUALA LUMPUR: The High Court here today said the Cabinet had never raised '
 'the issue of conflict of interest between Datuk Seri Najib Razak in '
 '1Malaysia Development Berhad (1MDB).',
 'According to the Code of Ethics For Members of the Administration and MPs is '
 'the practice of Cabinet members to declare conflicts of interest if they '
 'have involvement in matters discussed at the Cabinet Meeting.',
 'The matter was informed by former Deputy Secretary-General (Cabinet) of the '
 "Cabinet of the Constitution and the Relations of the Prime Minister's "
 'Department (JPM), Tan Sri Mazidah Abdul Majid, in his statement at the 1MDB '
 'case trial against Najib.',
 'The code of ethics also states that Cabinet members with personal interests '
 'and conflict with the interests of the government, or involving family '
 'members, need to leave the meeting and record their absence.',
 'In 1MDB, Najib held three positions, Prime Minister, Finance Minister and '
 'Chairman of the 1MDB Advisory Agency Responding to lawyer Tan Sri Muhammad '
 'Shafee Abdullah, whether Cabinet members raised the issue that the Minister '
 'of Finance should not involve the discussion because of the conflict of '
 'interest, Maziah replied: "No".',
 'Muhammad Shafee: Someone who raises things about 1MBD with Najib?',
 'Maziah: Muhammad Shafee: Deputy Prime Minister (then) was Tan Sri Muhyiddin '
 'Yassin, while Datuk Seri Ahmad Husni Hanadzlah was the former Minister of '
 'Finance II.',
 'They have never raised conflicts of interest?',
 'Mazidah: Yes, never.',
 "Meanwhile, responding to Deputy Public Prosecutor Ahmad Akram Gharib's "
 'question whether he knew that Najib had personal interests in 1MDB, Mazidah '
 'said: "No" Ahmad Akram: Do you know that Najib personally receives money '
 'from 1MDB?',
 'Maziah: Ahmad Akram: If Najib personally receives money from 1MDB whether it '
 'is a conflict of interest and violates the Code of Ethics for Members of the '
 'Administration and Member of Parliament.',
 'Maziah: According to my personal view, yes, but because it involves '
 'ministers and prime ministers, I propose to get the views of the Attorney '
 'General.',
 'Earlier, in the early proceedings, Maziah also told the court that Najib had '
 'never mentioned the name of a businessman in the hunt, Low Taek Jho or Jho '
 'Low at a Cabinet meeting as an individual who helped him get a donation from '
 "Saudi Arabia's royal family.",
 '"If the matter is communicated to the Cabinet, then it will be recorded in '
 'the minutes of the meeting," he said.',
 "Maziah added that he only heard and learned about Jho Low's name after the "
 'issue of involving 1MDB.',
 'Najib, 68, faces four charges of using his position to obtain a RM2.3 '
 'billion bribe from 1MDB funds and 21 counts of money laundering involving '
 'the same amount of money.',
 'The trial before Judge Collin Lawrence Sequerah continues this Monday.']

Problem with partitioning#

Problem with partition, the model assumed N + 1 element has no relationship with N element, vice versa. Attention mechanism will not work across partition.

So we introduced BigBird to solve this problem.

[18]:
pprint(bigbird_small.greedy_decoder([string]))
['KUALA LUMPUR: The High Court here today, was told the Cabinet never raised '
 'issues about the conflict of interest of Datuk Seri Najib Razak in 1Malaysia '
 'Development Berhad (1MDB). According to the Code of Ethics for Members of '
 'Administration and Member of Parliament is the practice of Cabinet members '
 'to declare conflicts of interest if they have involvement in matters '
 'discussed in the Cabinet Meeting. The matter was informed by the former '
 'Deputy Secretary General (Cabinet) of the Cabinet of the Constitution and '
 "Relations of the Prime Minister's Department (JPM), Tan Sri Mazidah Abdul "
 'Majid, in his statement at the 1MDB case trial faced by Najib. The code of '
 'ethics, among others, also stated that Cabinet members who have personal '
 'interests and contradict the interests of the government, or involving '
 'family members, should leave the meeting and record their absence. In 1MDB, '
 'Najib holds three positions, namely the Prime Minister, Finance Minister and '
 'Chairman of the 1MDB Advisory Body Answering a question from lawyer Tan Sri '
 'Muhammad Shafee Abdullah, whether there are Cabinet members who raise the '
 'issue that the Minister of Finance should not engage in the discussion '
 'because conflicts of interest, Maziah replied: "No." Muhammad Shafee: '
 'Someone raised matters regarding 1MBD with Najib? Maziah: Not Muhammad '
 'Shafee: Deputy Prime Minister (then) is Tan Sri Muhyiddin Yassin, while '
 'Datuk Seri Ahmad Husni Hanadzlah is a former Finance Minister II. They also '
 'never raised matters of interest conflict? Mazidah: Yes, never. Meanwhile, '
 'answering a question from Deputy Public Prosecutor Ahmad Akram Gharib '
 'whether he knew that Najib had personal interest in 1MDB, Mazidah said: "No" '
 'Ahmad Akram: Do you know that Najib personally received money from 1MDB? '
 'Maziah: No Ahmad Akram: If Najib personally receives money from 1MDB is it a '
 'conflict of interest and violates the Code of Ethics for Members of '
 'Administration and Member of Parliament. Maziah: According to my personal '
 'opinion, yes, but because it involves ministers and prime minister, I '
 'suggest to get the views of the Attorney General. Earlier, at the beginning '
 'of the proceedings, Maziah also told the court that Najib had never '
 'mentioned the name of a businessman in the hunt, Low Taek Jho or Jho Low at '
 'a Cabinet meeting as an individual who helped him get donations from the '
 'royal family of Saudi Arabia. "If the matter was informed to the Cabinet, '
 'then, it will be recorded in the minutes of the meeting," he said. Maziah '
 "added that he only heard and found out about Jho Low's name after the issue "
 'involving 1MDB. Najib, 68, faced four charges of using his position to '
 'obtain bribes amounting to RM2.3 billion from 1MDB funds and 21 charges of '
 'money laundering involving the same amount of money. The trial before Judge '
 'Collin Lawrence Sequerah continues this Monday.']
[19]:
pprint(quantized_bigbird_small.greedy_decoder([string]))
['KUALA LUMPUR: The High Court here today, was told the Cabinet never raised '
 'issues about the conflict of interest of Datuk Seri Najib Razak in 1Malaysia '
 'Development Berhad (1MDB). According to the Code of Ethics for Members of '
 'Administration and Member of Parliament is the practice of Cabinet members '
 'to declare conflicts of interest if they have involvement in matters '
 'discussed in the Cabinet Meeting. The matter was informed by the former '
 'Deputy Secretary General (Cabinet) of the Cabinet of the Constitution and '
 "Relations of the Prime Minister's Department (JPM), Tan Sri Mazidah Abdul "
 'Majid, in his statement at the 1MDB case trial faced by Najib. The code of '
 'ethics, among others, also stated that Cabinet members who have personal '
 'interests and contradict the interests of the government, or involve family '
 'members, should leave the meeting and record their absence. In 1MDB, Najib '
 'holds three positions, namely the Prime Minister, Finance Minister and '
 'Chairman of the 1MDB Advisory Body Answering a question from lawyer Tan Sri '
 'Muhammad Shafee Abdullah, whether there are Cabinet members who raise the '
 'issue that the Minister of Finance should not engage in the discussion '
 'because conflicts of interest, Maziah replied: "No." Muhammad Shafee: '
 'Someone raised matters regarding 1MBD with Najib? Maziah: Not Muhammad '
 'Shafee: Deputy Prime Minister (then) is Tan Sri Muhyiddin Yassin, while '
 'Datuk Seri Ahmad Husni Hanadzlah is a former Finance Minister II. They also '
 'never raised matters of interest conflict? Mazidah: Yes, never. Meanwhile, '
 'answering the question of Deputy Public Prosecutor, Ahmad Akram Gharib '
 'whether he knew that Najib had personal interest in 1MDB, Mazidah said: "No" '
 'Ahmad Akram: Do you know that Najib personally received money from 1MDB? '
 'Maziah: Not Ahmad Akram: If Najib personally receives money from 1MDB is it '
 'a conflict of interest and violates the Code of Ethics for Members of '
 'Administration and Member of Parliament. Maziah: According to my personal '
 'opinion, yes, but because it involves ministers and prime minister, I '
 'suggest to get the views of the Attorney General. Earlier, at the beginning '
 'of the proceedings, Maziah also told the court that Najib never mentioned '
 'the name of a businessman in the hunt, Low Taek Jho or Jho Low at a Cabinet '
 'meeting as an individual who helped him get donations from the royal family '
 'of Saudi Arabia. "If the matter was informed to the Cabinet, then, it will '
 'be recorded in the minutes of the meeting," he said. Maziah added that he '
 "only heard and found out about Jho Low's name after the issue involving "
 '1MDB. Najib, 68, faced four charges of using his position to obtain bribes '
 'amounting to RM2.3 billion from 1MDB funds and 21 charges of money '
 'laundering involving the same amount of money. The trial before Judge Collin '
 'Lawrence Sequerah continues this Monday.']
[20]:
pprint(bigbird.greedy_decoder([string]))
['KUALA LUMPUR: The High Court here today was told that the Cabinet had never '
 'raised the issue of the conflict of interest of Datuk Seri Najib Razak in '
 '1Malaysia Development Berhad (1MDB). According to the Code of Ethics for '
 'Administrative Members and Members of Parliament, it is the practice of '
 'Cabinet members to declare conflicts of interest if they have involvement in '
 'matters discussed in the Cabinet Meeting. The matter was informed by the '
 'former Deputy Secretary General (Cabinet) of the Cabinet Division of the '
 "Constitution and the Relations of the Prime Minister's Department (JPM), Tan "
 'Sri Mazidah Abdul Majid, in his testimony at the 1MDB case trial faced by '
 'Najib. The code of ethics, among others, also states that Cabinet members '
 'who have personal interests and are contrary to the interests of the '
 'government, or involving family members, have to leave the meeting and '
 'record their absence. In 1MDB, Najib held three positions, namely the Prime '
 'Minister, Finance Minister and Chairman of the Chairman of the 1MDB Advisory '
 'Body, answered a question from lawyer Tan Sri Muhammad Shafee Abdullah, '
 'whether there were Cabinet members who raised the issue that the Minister of '
 'Finance should not be involved in the discussion due to conflicts of '
 'interest, Maziah replied: "No." Muhammad Shafee: There are anyone who raised '
 'the matter of 1MBD with Najib? Maziah: No Muhammad Shafee: The Deputy Prime '
 'Minister (then) is Tan Sri Muhyiddin Yassin, while Datuk Seri Ahmad Husni '
 'Hanadzlah is the former Minister of Finance II. They have never raised the '
 'conflict of interest? Mazidah: Yes, never. Meanwhile, answering a question '
 'from Deputy Public Prosecutor Ahmad Akram Gharib whether he knew that Najib '
 'had a personal interest in 1MDB, Mazidah said: "No" Ahmad Akram: Do you know '
 'that Najib personally receives money from 1MDB? Maziah: No Ahmad Akram: If '
 'Najib personally receives money from 1MDB is that a conflict of interest and '
 'violates the Code of Ethics for Administrative Members and Members of '
 'Parliament. Maziah: According to my personal view, yes, but because it '
 'involves ministers and prime ministers, I propose to get the views of the '
 'Attorney General. Earlier, at the beginning of the proceedings, Maziah also '
 'told the court that Najib had never mentioned the name of a businessman in '
 'the hunt, Low Taek Jho or Jho Low at a Cabinet meeting as an individual who '
 'helped him get donations from the royal family of Saudi Arabia. "If the '
 'matter is informed to the Cabinet, then, it will be recorded in the minutes '
 'of the meeting," he said. Maziah added that he only heard and knew about Jho '
 "Low's name after an issue involving 1MDB. Najib, 68, faces four charges of "
 'using his position to obtain bribes amounting to RM2.3 billion from 1MDB '
 'funds and 21 charges of money laundering involving the same amount of money. '
 'The trial before Judge Collin Lawrence Sequerah continues this Monday.']
[21]:
pprint(quantized_bigbird.greedy_decoder([string]))
['KUALA LUMPUR: The High Court here today was told that the Cabinet had never '
 'raised the issue of the conflict of interest of Datuk Seri Najib Razak in '
 '1Malaysia Development Berhad (1MDB). According to the Code of Ethics for '
 'Administrative Members and Members of Parliament, it is the practice of '
 'Cabinet members to declare conflicts of interest if they have involvement in '
 'matters discussed in the Cabinet Meeting. The matter was informed by the '
 'former Deputy Secretary General (Cabinet) of the Cabinet Division of the '
 "Constitution and the Relations of the Prime Minister's Department (JPM), Tan "
 'Sri Mazidah Abdul Majid, in his testimony at the 1MDB case trial faced by '
 'Najib. The code of ethics, among others, also states that Cabinet members '
 'who have personal interests and are contrary to the interests of the '
 'government, or involving family members, have to leave the meeting and '
 'record their absence. In 1MDB, Najib held three positions, namely the Prime '
 'Minister, Finance Minister and Chairman of the Chairman of the 1MDB Advisory '
 'Body, answered a question from lawyer Tan Sri Muhammad Shafee Abdullah, '
 'whether there were Cabinet members who raised the issue that the Minister of '
 'Finance should not be involved in the discussion due to conflicts of '
 'interest, Maziah replied: "No." Muhammad Shafee: There are anyone who raised '
 'the matter of 1MBD with Najib? Maziah: No Muhammad Shafee: The Deputy Prime '
 'Minister (then) is Tan Sri Muhyiddin Yassin, while Datuk Seri Ahmad Husni '
 'Hanadzlah is the former Minister of Finance II. They have never raised the '
 'conflict of interest? Mazidah: Yes, never. Meanwhile, answering a question '
 'from the Deputy Public Prosecutor, Ahmad Akram Gharib whether he knew that '
 'Najib had a personal interest in 1MDB, Mazidah said: "No" Ahmad Akram: Do '
 'you know that Najib personally receives money from 1MDB? Maziah: No Ahmad '
 'Akram: If Najib personally receives money from 1MDB is that there is a '
 'conflict of interest and violates the Code of Ethics for Administrative '
 'Members and Members of Parliament. Maziah: According to my personal view, '
 'yes, but because it involves ministers and prime ministers, I propose to get '
 'the views of the Attorney General. Earlier, at the beginning of the '
 'proceedings, Maziah also told the court that Najib had never mentioned the '
 'name of a businessman in the hunt, Low Taek Jho or Jho Low at a Cabinet '
 'meeting as an individual who helped him get donations from the royal family '
 'of Saudi Arabia. "If the matter is informed to the Cabinet, then, it will be '
 'recorded in the minutes of the meeting," he said. Maziah added that he only '
 "heard and knew about Jho Low's name after an issue involving 1MDB. Najib, "
 '68, faces four charges of using his position to obtain bribes amounting to '
 'RM2.3 billion from 1MDB funds and 21 charges of money laundering involving '
 'the same amount of money. The trial before Judge Collin Lawrence Sequerah '
 'continues this Monday.']
[22]:
pprint(bigbird_small.greedy_decoder([string2]))
['KUALA LUMPUR: Datuk Seri Najib Tun Razak today failed in another attempt to '
 'drop former Federal Court Judge Datuk Seri Gopal Sri Ram from leading the '
 'prosecution team in the case of amending the 1Malaysia Development Berhad '
 "(1MDB) audit report involving the former prime minister. This is Najib's "
 'third attempt to drop Sri Ram as prosecutor in the 1MDB-related criminal '
 'case. Earlier, an application was filed in another 1MDB case before a '
 'different judge and a second attempt through civil proceedings. While '
 'rejecting the application of the Pekan Member of Parliament, High Court '
 "Judge Mohamed Zaini Mazlan said Najib's allegation that Sri Ram was involved "
 'in the investigation into him as there was no merit. "There is no solid '
 "evidence to support the applicant's allegation and remain a mere hypothesis. "
 'This issue has been investigated by the prosecution as the previous '
 'respondent during the application of the applicant (Najib)." This issue has '
 'been discussed and decided. The decision by other courts previously remained '
 'and could not be reversed, "said the judge. Mohamed Zaini when giving '
 'reasons for rejection said applicants, among others, made communication '
 'between former Attorney General Tan Sri Mohamed Apandi Ali and Sri Ram as '
 "evidence of Sri Ram's biased side of the applicant and the applicant's "
 'concern regarding the matter was baseless. "As well as other individuals, '
 "Sri Ram has the right to have personal views. That's all. However, different "
 'considerations will be made if he shows a biased attitude while performing '
 'his duties as a senior prosecutor. His personal view cannot be considered to '
 'prevent his responsibilities as a senior prosecutor." Furthermore, the '
 'incident, as presented by the respondent, happened when before the '
 'appointment of Sri Ram as a senior prosecutor. Also important is that the '
 "applicant did not make any complaint about Sri Ram's actions while "
 'conducting a trial involving the applicant. This strengthened the '
 "respondents' argument that Sri Ram was open while carrying out his duties as "
 'a senior prosecutor, "said the judge. Mohamed Zaini further said that in the '
 'end, the court was responsible for ensuring that a trial was implemented '
 'fairly for justice." The court will help any party treated unfairly, if the '
 "matter happened. The relationship, the applicant's application was rejected, "
 '"he said. The trial of the 1MDB audit report case will continue on Feb 22. '
 "At today's proceedings, Deputy Public Prosecutor Ahmad Akram Gharib acted on "
 'behalf of the prosecution, while Najib was represented by lawyer Nur '
 'Syahirah Hanapiah. Najib, 67, and former 1MDB Chief Executive Officer Arul '
 'Kanda Kandasamy, 45, were tried on charges of amending the 1MDB audit '
 'report. The Pekan Member of Parliament was accused of using his position to '
 'direct amendments to the 1MDB final audit report before being tabled to the '
 'Public Accounts Committee to prevent any action from being taken against '
 'him, while Arul Kanda was charged with abetting Najib in making amendments '
 'to the report to protect Najib from being prosecuted.']
[23]:
pprint(quantized_bigbird_small.greedy_decoder([string2]))
['KUALA LUMPUR: Datuk Seri Najib Tun Razak today failed in another attempt to '
 'drop former Federal Court Judge Datuk Seri Gopal Sri Ram from leading the '
 'prosecution team in the case of amending the 1Malaysia Development Berhad '
 "(1MDB) audit report involving the former prime minister. This is Najib's "
 'third attempt to drop Sri Ram as prosecutor in the 1MDB-related criminal '
 'case. Earlier, an application was filed in another 1MDB case before a '
 'different judge and a second attempt through civil proceedings. While '
 'rejecting the application of the Pekan Member of Parliament, High Court '
 "Judge Mohamed Zaini Mazlan said Najib's allegation that Sri Ram was involved "
 'in the investigation into him as there is no merit. "There is no solid '
 "evidence to support the applicant's allegation and remain a mere hypothesis. "
 'This issue has been investigated by the prosecution as the previous '
 'respondent during the application of the applicant (Najib)." This issue has '
 'been discussed and decided. The decision by other courts previously remained '
 'and could not be reversed, "said the judge. Mohamed Zaini when giving '
 'reasons for rejection said applicants, among others, made communication '
 'between former Attorney General Tan Sri Mohamed Apandi Ali and Sri Ram as '
 "evidence of Sri Ram's biased side of the applicant and the applicant's "
 'concern regarding the matter was baseless. "As well as other individuals, '
 'Sri Ram has the right to have a personal view. That alone. However, '
 'different considerations will be made if he shows a biased attitude while '
 'performing his duties as a senior prosecutor. His personal view cannot be '
 'considered to prevent his responsibility as a senior prosecutor." '
 'Furthermore, the incident, as presented by the respondent, happened when '
 'before the appointment of Sri Ram as a senior prosecutor. Also important is '
 "that the applicant did not make any complaint about Sri Ram's actions while "
 "conducting trial involving the applicant. This strengthened the respondents' "
 'argument that Sri Ram was open while carrying out his duties as a senior '
 'prosecutor, "said the judge. Mohamed Zaini further said that in the end, the '
 'court was responsible for ensuring that a trial was implemented fairly for '
 'justice." The court will help any party treated unfairly, if the matter '
 'happened. The connection, the applicant\'s application was rejected, "he '
 'said. The trial of the 1MDB audit report case will continue on Feb 22. At '
 "today's proceedings, Deputy Public Prosecutor Ahmad Akram Gharib acted on "
 'behalf of the prosecution, while Najib was represented by lawyer Nur '
 'Syahirah Hanapiah. Najib, 67, and former 1MDB Chief Executive Officer Arul '
 'Kanda Kandasamy, 45, were tried on charges of amending the 1MDB audit '
 'report. The Pekan Member of Parliament was accused of using his position to '
 'direct amendments to the 1MDB final audit report before being tabled to the '
 'Public Accounts Committee to prevent any action from being taken against '
 'him, while Arul Kanda was charged with abetting Najib in making amendments '
 'to the report to protect Najib from being prosecuted.']
[24]:
pprint(bigbird.greedy_decoder([string2]))
['KUALA LUMPUR: Datuk Seri Najib Tun Razak today failed in another attempt to '
 'drop former Federal Court Judge Datuk Seri Gopal Sri Ram from leading the '
 'prosecution team in the case of amending the 1Malaysia Development Berhad '
 "(1MDB) audit report involving the former prime minister. This is Najib's "
 'third attempt to drop Sri Ram as a prosecution in the 1MDB-related criminal '
 'case. Previously, an application was filed in another 1MDB case before a '
 'different judge and a second attempt through civil proceedings. While '
 "rejecting the Pekan Member of Parliament's application, High Court Judge "
 "Mohamed Zaini Mazlan said Najib's allegation that Sri Ram was involved in "
 'his investigation into him as no merit. "There is no solid evidence to '
 "support the applicant's allegations and remain a mere hypothesis. This issue "
 'has been investigated by the prosecution as the respondent during the '
 'application of the applicant (Najib) earlier." This issue has been discussed '
 'and decided. The decision by other courts previously remains and cannot be '
 'reversed, "said the judge. Mohamed Zaini when giving the reason for refusing '
 'said the applicant, among others, made communication between former '
 'Attorney-General Tan Sri Mohamed Apandi Ali and Sri Ram as proof of Sri '
 "Ram's bias towards the applicant and the applicant's concern over the matter "
 'was baseless. "Like other individuals, Sri Ram has the right to have a '
 "personal view. That's all. However, different considerations will be made if "
 'he shows a biased attitude while performing his duties as a senior '
 'prosecutor. His personal views should not be considered to hinder his '
 'responsibility as a senior prosecutor. "Furthermore, the incident, as '
 'submitted by the respondent, occurred before the appointment of Sri Ram as '
 'senior prosecutor. It is also important that the applicant did not make any '
 "complaints about Sri Ram's actions during the trial involving the applicant. "
 "This strengthens the respondents' argument that Sri Ram was open while "
 'carrying out his duties as senior prosecutor," said the judge. Mohamed Zaini '
 'further said that in the end, the court was responsible for ensuring that a '
 'trial was held fairly in order to obtain justice. "The court will assist any '
 'party treated unfairly, if the matter happens. In relation to that, the '
 'applicant\'s application is rejected," he said. The trial of the 1MDB audit '
 "report case will continue on Feb 22. In today's proceedings, Deputy Public "
 'Prosecutor Ahmad Akram Gharib acted on behalf of the prosecution, while '
 'Najib was represented by lawyer Nur Syahirah Hanapiah. Najib, 67, and former '
 '1MDB Chief Executive Officer Arul Kanda Kandasamy, 45, were tried on charges '
 'of amending the 1MDB audit report. The Pekan Member of Parliament was '
 'accused of using his position to order amendments to the 1MDB final audit '
 'report before being tabled to the Public Accounts Committee to prevent any '
 'action being taken against him, while Arul Kanda was accused of abetting '
 'Najib in amending the report to protect Najib from being prosecuted.']
[25]:
pprint(quantized_bigbird.greedy_decoder([string2]))
['KUALA LUMPUR: Datuk Seri Najib Tun Razak today failed in another attempt to '
 'drop former Federal Court Judge Datuk Seri Gopal Sri Ram from leading the '
 'prosecution team in the case of amending the 1Malaysia Development Berhad '
 "(1MDB) audit report involving the former prime minister. This is Najib's "
 'third attempt to drop Sri Ram as a prosecution in the 1MDB-related criminal '
 'case. Previously, an application was filed in another 1MDB case before a '
 'different judge and a second attempt through civil proceedings. While '
 "rejecting the Pekan Member of Parliament's application, High Court Judge "
 "Mohamed Zaini Mazlan said Najib's allegation that Sri Ram was involved in an "
 'investigation into him as no merit. "There is no solid evidence to support '
 "the applicant's allegations and remain a mere hypothesis. This issue has "
 'been investigated by the prosecution as the respondent during the applicant '
 '(Najib) application." This issue has been discussed and decided. The '
 'decision by other courts has previously remained and cannot be reversed, '
 '"said the judge. Mohamed Zaini when giving the reason for his rejection said '
 'the applicant, among others, made communication between former '
 'Attorney-General Tan Sri Mohamed Apandi Ali and Sri Ram as proof of Sri '
 "Ram's bias towards the applicant and the applicant's concern over the matter "
 'was baseless. "Like other individuals, Sri Ram has the right to have a '
 "personal view. That's all. However, different considerations will be made if "
 'he shows a biased attitude while performing his duties as a senior '
 'prosecutor. His personal views should not be considered to hinder his '
 'responsibility as a senior prosecutor. "Furthermore, the incident, as '
 'submitted by the respondent, occurred before the appointment of Sri Ram as '
 'senior prosecutor. It is also important that the applicant did not make any '
 "complaints about Sri Ram's actions during the trial involving the applicant. "
 "This strengthens the respondents' argument that Sri Ram is open while "
 'carrying out his duties as senior prosecutor," said the judge. Mohamed Zaini '
 'further said that in the end, the court was responsible for ensuring that a '
 'trial was held fairly in order to obtain justice. "The court will assist any '
 'party treated unfairly, if the matter happens. In relation to that, the '
 'applicant\'s application is rejected," he said. The trial of the 1MDB audit '
 "report case will continue on Feb 22. At today's proceedings, Deputy Public "
 'Prosecutor Ahmad Akram Gharib acted on behalf of the prosecution, while '
 'Najib was represented by lawyer Nur Syahirah Hanapiah. Najib, 67, and former '
 '1MDB Chief Executive Officer Arul Kanda Kandasamy, 45, were tried on charges '
 'of amending the 1MDB audit report. The Pekan Member of Parliament was '
 'accused of using his position to order amendments to the 1MDB final audit '
 'report before being tabled to the Public Accounts Committee to prevent any '
 'action being taken against him, while Arul Kanda was charged with abetting '
 "Najib in amending Najib's report to protect Najib from being prosecuted."]
[21]:
pprint(bigbird_small.greedy_decoder([string3]))
['3 August 2018 - The latest study conducted by University College London '
 '(UCL) found that mental healthcare services from support workers who have '
 'been facing the disease themselves may help reduce the probability of '
 'patients who have just left out of acute mental health care from '
 're-introduced to the unit. The research paper published today in the journal '
 'The Lancet found that the number of mental patients re-induced into acute '
 'care units within a year was 24% much lower for the group of patients '
 'offered by support workers in the form of senasib friends, compared to '
 'groups of patients who are only given personal rehabilitation work books. '
 '"The pain allowed to go out (discharge) from community crisis services is '
 'often re-entered into acute care units. This not only helps recovery, but '
 'also uses resources that should be reserved for long-term improvement of '
 "patients' functions and quality of life. Support workers from the group of "
 'senasib friends are able to provide truly friendly and empatth support and '
 'encouragement because they come from their own personal experience, in '
 'addition to being a good example of role model for patient rehabilitation, '
 '"said the main author, Professor Sonia Johnson (Psychiiatri, UCL). In the '
 'United Kingdom (UK), more than half of the number of patients included in '
 'the acute care unit is re-entered into the unit within a year, but there is '
 'no solid evidence to explain the method to reduce this number. In fact, '
 'support from individuals who have experienced mental health problems has '
 'been practiced in programs such as the Implementation of Rehabilitation '
 'through NHS Organizational Change (UK) as well as the United States Health '
 'Rehabilitation Action Plan. This research is the first random controlled '
 'test to evaluate the effectiveness of senasib partner support programs. '
 'However, there are still many research that needs to be carried out before '
 'this strategy can be implemented comprehensively in the UK, for example to '
 'understand the reasons why this program is effective. Certainly, '
 'intervention or intervention in self-management (self-management) may help '
 'patients manage their mental health better. In this study, researchers '
 'provide participants either with personal rehabilitation work books only '
 '(220 people) or working books with help with senasib partner support workers '
 '(221 people). Participants are also allowed to continue their regular care. '
 'This study was conducted through six mental health crisis solution teams in '
 'England and participants were selected only after they were allowed to leave '
 'the crisis unit by crisis resolution team. Study participants consisted of '
 'various diagnosiss including schizophrenia, bipolar, psychosis, depression, '
 'anxiety confusion, post-trauma stress disorders (PTSD), and personality '
 'confusion. Participants who received support from senasib friends were '
 'offered 10 weekly meeting sessions for an hour. During this meeting session, '
 "support workers heard participants' problems and aim to instill hope by "
 'sharing the skills and strategies of disease management (coping strategies) '
 'which were mastered when they themselves were in the process of recovery. '
 'All support workers are given prior training in listening skills, cultural '
 'awareness, self-disclosure, and confidentiality, as well as how to use work '
 "books. Participants' health records were monitored by researchers for one "
 'year to determine whether participants were re-entered to acute care units '
 '(such as acute patient wards, crisis solutions, crisis homes, and acute '
 'daily care services) or not. After completing a year, the study found that '
 'the percentage of re-entry into acute care units is lower for the group of '
 'participants who receive intervention compared to the control group - with '
 '29% of participants re-in-reduced after receiving support from senasib '
 'friends, compared to 38% participants who only received working books. The '
 'intervention absorption rate also well - 72% of participants offered the '
 'services of senasib support employees and working books attended at least '
 'three meeting sessions, while one third attended all 10 meeting sessions. '
 '"We are aware that many users of mental health crisis care services felt '
 'this service stopped sharply due to lack of ready-up care. Our study shows '
 'that senasib partner support workers can help fill this gap by helping '
 'service users (accuraration) to develop self-management strategies and '
 'recovery plans in their own way and more meaningful, in turn can help '
 'patients continue to survive after a crisis, "said the joint author, Dr '
 'Brynmor Lloyd-Evans (Psikiatrialt) to help in turn can help patients '
 'continue to survive after a crisis after a crisis after a crisis," said '
 'after a crisis," said. together," said that. (Piatri, (s after receiving job '
 'management services of the services, Dr imized service users with at least '
 'three-inforceditionary support services (Philly, and more meaningfully and '
 'more meaningfully, Dr (Planties, in turn help patients with a crisis, in '
 'turn can help patients continue to help']
[22]:
pprint(quantized_bigbird_small.greedy_decoder([string3]))
['3 August 2018 - The latest study conducted by University College London '
 '(UCL) found that mental healthcare services from support workers who have '
 'been going through the disease themselves may help reduce the probability of '
 'patients who have just left from acute mental health care from re-increased '
 'to the unit. The research paper published today in the journal The Lancet '
 'found that the number of mental patients re-induced into acute care units '
 'within a year is 24% much lower for the group of patients offered by support '
 'workers in the form of dustry partners, compared to groups of patients who '
 'are only given personal rehabilitation work books. "The pain allowed to go '
 'out (discharge) from community crisis services is often re-entered to acute '
 'care units. This not only hinders recovery, but also uses resources that '
 "should be reserved for long-term improvement of patients' functions and "
 'quality of life. Support workers from the group of senasib friends are able '
 'to provide truly friendly and empath support and encouragement because they '
 'come from their own personal experience, in addition to being a good example '
 'of role model for patient rehabilitation, "said the main author, Professor '
 'Sonia Johnson (Psychiiatri, UCL). In the United Kingdom (UK), more than half '
 'of the number of patients included in the acute care unit is re-entered into '
 'the unit within a year, but there is no solid evidence to explain the method '
 'to reduce this number. In fact, support from individuals who have '
 'experienced mental health problems has been practiced in programs such as '
 'the Implementation of Rehabilitation through NHS Organizational Change (UK) '
 'as well as the United States Health Recovery Action Plan. This research is '
 'the first random controlled test to evaluate the effectiveness of senasib '
 'support programs. Even so, there are still many research that needs to be '
 'carried out before this strategy can be implemented comprehensively in the '
 'UK, for example to understand the reasons why this program is effective. '
 'Certainly, intervention or intervention in self-management may help patients '
 'manage their mental health better. In this study, researchers provide '
 'participants either with personal rehabilitation work books (220 people) or '
 'working books with help from senasib partner support workers (221 people). '
 'Participants are also allowed to continue their regular care. This study was '
 'conducted through six mental health crisis solution teams in England and '
 'participants were selected only after they were allowed to leave the crisis '
 'unit by crisis resolution team. Study participants consisted of various '
 'diagnosiss including schizophrenia, bipolar, psychosis, depression, anxiety '
 'confusion, post-trauma stress disorders (PTSD), and personality confusion. '
 'Participants who received support from senasib friends were offered 10 '
 'weekly meeting sessions for an hour. During this meeting session, support '
 "workers heard participants' problems and aim to instill hope by sharing the "
 'skills and strategies of disease management when they themselves were in the '
 'process of recovery. All support workers are given prior training in '
 'listening skills, cultural awareness, self-disclosure, and confidentiality, '
 "as well as how to use work books. Participants' health records were "
 'monitored by researchers for one year to determine whether participants were '
 're-entered to acute care units (such as acute patient wards, crisis '
 'solutions, crisis homes, and acute daily care services) or not. After the '
 'end of one year, the study found that the percentage of re-entry to acute '
 'care units is lower for the group of participants who receive intervention '
 'compared to the control group - with 29% of participants re-increased after '
 'receiving the support of the senasib partner, compared to 38% participants '
 'who only received work books. The intervention absorption rate also - 72% of '
 'participants offered the services of senasib support employees and working '
 'books attended at least three meeting sessions, while one third attended all '
 '10 meeting sessions. "We are aware that many users of mental health crisis '
 'care services feel that this service stopped dramatically due to lack of '
 'follow-up care ready. Our study shows that senasib peer support workers can '
 'help fill this gap by helping service users (accuratic) develop '
 'self-management strategies and recovery plans in their own way and more '
 'meaningfully, in turn can help patients continue to survive after a crisis, '
 '"said the joint author, Dr Brynmor Lloyd-Evans (Pcyclist, UCL).']
[26]:
pprint(bigbird.greedy_decoder([string3]))
['3 August 2018 - A recent study conducted by University College London (UCL) '
 'found that mental healthcare services from support workers who have '
 'undergone the disease themselves may help reduce the probability of patients '
 'who have just left acute mental health care (acute mental health care) from '
 'being re-entered to the unit. The research paper published today in the '
 'journal The Lancet found that the number of mental patients re-entered to '
 'acute care units within a year was 24% much lower for the group of patients '
 'offered by support workers in the form of senib friends, compared to the '
 'group of patients who were only given personal rehabilitation workbooks '
 'only. "Patients allowed to leave (discharge) from community crisis services '
 'are often re-entered to acute care units. This not only hinders recovery, '
 'but also uses resources that should be reserved for long-term improvement of '
 'patient function and quality of life. Support workers from the senasib '
 'friends are able to provide support and encouragement that is truly friendly '
 'and empathic because it comes from their own personal experience, in '
 'addition to being a good example of models (role models) for patient '
 'recovery, "said lead author Professor Sonia Johnson (Psychiatric, UCL). In '
 'the United Kingdom (UK), more than half of the total number of patients '
 'admitted to acute care units are re-entered to the unit within a year, but '
 'no solid evidence explains the method to reduce this number.In fact, support '
 'from individuals who have had mental health problems has been practiced in '
 'programs such as Rehabilitation Implementation through NHS Organization '
 'Change (UK) and also the United States Health Rehabilitation Action '
 'Plan.This research is the first such randomized test to evaluate the '
 'effectiveness of  senmitarian support programs.However, there is still a lot '
 'of research that needs to be done before this strategy can be implemented '
 'comprehensively in the UK, for example to understand the reasons why the '
 'program is effective. Certainly, intervention or intervention in '
 'self-management may help patients manage their mental health better.In this '
 'study, the researcher provided participants either with personal '
 'rehabilitation workbooks only (220 people) or workbooks with the help of '
 'support employees of senasib friends (221 people). Participants are also '
 'allowed to continue their normal care. This study was conducted through six '
 'teams to resolve the mental health crisis (crisis resolution team) in '
 'England and participants were selected only after they were allowed to leave '
 'the crisis unit by the crisis-solving team. Study participants consisted of '
 'various diagnosiss including schizophrenia, bipolar, psychosis, depression, '
 'anxiety disorder, post-trauma stress disorder (PTSD), and personality '
 'disorder. Participants who received the support of senmiline friends were '
 'offered 10 weekly meeting sessions for an hour. During this meeting session, '
 "support workers listened to participants' problems and aimed at instilling "
 'hope by sharing skills and strategies of disease management (coping '
 'strategies) controlled while they themselves were in the process of '
 'recovery. All support workers are given prior training in listening skills, '
 'cultural awareness, self-disclosure, and confidentiality, as well as how to '
 "use workbooks. Participants' health records are monitored by researchers for "
 'a year to determine whether participants are re-entered into acute care '
 'units (such as acute patient wards, crisis-solving teams, crisis houses, '
 'crisis houses, and acute daily care services) or not. After a year, the '
 'study found that the percentage of re-entry to acute care units was lower '
 'for groups of participants who received the intervention compared to the '
 'control group - with 29% of participants re-entered after receiving support '
 'from their peers, compared to 38% of participants who only received '
 'workbooks. The intervention absorption rate is also good - 72% of '
 'participants offered the services of support partner support workers and '
 'workbooks attended at least three meeting sessions, while one-third attended '
 'all 10 meeting sessions. "We are aware that many users of mental health '
 'crisis care services feel this service ceases sharply due to lack of '
 'ready-made follow-up care. Our study shows that support employees of senied '
 'partners can help fill out this gap by helping service users (pesakit) '
 'develop self-management strategies and recovery and more meaningfully, '
 'further helping patients survive after a crisis, " said the co-winnary '
 'author, " said the co-star, Dr Brynmorynmor Lloyd-E).']
[27]:
pprint(quantized_bigbird.greedy_decoder([string3]))
['August 3, 2018 - A recent study conducted by University College London (UCL) '
 'found that mental healthcare services from support workers who have '
 'undergone the disease themselves may help reduce the probability of patients '
 'who have just left acute mental health care (acute mental health care) from '
 'being re-entered to the unit. The research paper published today in the '
 'journal The Lancet found that the number of mental patients re-entered into '
 'acute care units within a year was 24% much lower for the group of patients '
 'offered by support workers in the form of senib partners, compared to the '
 'group of patients who were only given personal rehabilitation workbooks '
 'only. "Puncharge allowed out of community crisis services are often '
 're-entered to acute care units. This not only hinders recovery, but also '
 'uses resources that should be reserved for long-term improvement of patient '
 'function and quality of life. Support workers from the senasib partners are '
 'able to provide support and encouragement that is truly friendly and '
 'empathistic because it comes from their own personal experience, in addition '
 'to being a good example of a good role model for patient recovery, "said '
 'lead author Professor Sonia Johnson (Psychiatrist, UCL). In the United '
 'Kingdom (UK), more than half of the total number of patients admitted to '
 'acute care units are re-entered to the unit within a year, but no solid '
 'evidence explains the method to reduce this amount.In fact, support from '
 'individuals who have had mental health problems has been practiced in '
 'programs such as Rehabilitation Implementation through NHS Organization '
 'Change (UK) and also the United States Health Rehabilitation Action '
 'Plan.This research is the first such random controlled test to evaluate the '
 'effectiveness of the senasib support program.However, there is still a lot '
 'of research that needs to be done before this strategy can be implemented '
 'comprehensively in the UK, for example to understand the reasons why the '
 'program is effective. Certainly, intervention or intervention in '
 'self-management may help patients manage their mental health better.In this '
 'study, the researcher provided participants either with personal '
 'rehabilitation workbooks only (220 people) or workbooks with the help of '
 'support staff of senasib friends (221 people). Participants are also allowed '
 'to continue their normal care. This study was conducted through six teams '
 'solving the mental health crisis (crisis resolution team) in England and '
 'participants were selected only after they were allowed to leave the crisis '
 'unit by the crisis-solving team. Study participants consisted of various '
 'diagnosis including schizophrenia, bipolar, psychosis, depression, anxiety '
 'disorder, post-trauma stress disorder (PTSD), and personality disorder. '
 'Participants who received the support of senasib friends were offered 10 '
 'weekly meeting sessions for an hour. During this meeting session, support '
 "workers listened to participants' problems and aimed at instilling hope by "
 'sharing skills and strategies of disease management (coping strategies) '
 'controlled while they were in the process of recovery. All support workers '
 'are given prior training in listening skills, cultural awareness, '
 'self-disclosure, and confidentiality, as well as how to use workbooks. '
 "Participants' health records are monitored by researchers for a year to "
 'determine whether participants are re-entered into acute care units (such as '
 'acute patient wards, crisis-solving teams, crisis houses, crisis houses, and '
 'acute daily care services) or not. After a year, the study found that the '
 'percentage of re-entry to acute care units is lower for groups of '
 'participants who receive interventions compared to control groups - with 29% '
 'of participants re-entered after receiving support from their peers, '
 'compared to 38% of participants who only received workbooks. The '
 'intervention absorption rate is also good - 72% of participants offered the '
 'services of support partner support and workbooks attend at least three '
 'meeting sessions, while one-thirds attend all 10 meeting sessions. "We are '
 'aware that many users of mental health crisis care services feel this '
 'service is abruptly due to lack of ready-made follow-up care. Our study '
 'shows that support staff support workers can help fill this gap by helping '
 'service users (pesakit) develop self-management strategies and recovery '
 'plans in their own way, further helping patients survive after a crisis," '
 'said the occurrence, " said the co-started author Dr Bryn of the author, Dr '
 'Brynousa.']
[ ]: