Grapheme-to-Phoneme IPA

This tutorial is available as an IPython notebook at Malaya/example/phoneme-ipa.

This module only trained on standard language structure, so it is not save to use it for local language structure.

Explanation

Phonemizer is a Grapheme-to-phoneme (G2P) conversion is the process of generating pronunciation for words using IPA format based on their written form, for an example,

husein -> husəin

%%time
import malaya

CPU times: user 6.13 s, sys: 1.35 s, total: 7.47 s
Wall time: 9.79 s

Use deep learning model

Load LSTM + Bahdanau Attention phonemizer model.

If you are using Tensorflow 2, make sure Tensorflow Addons already installed,

pip install tensorflow-addons U

def deep_model_ipa(quantized: bool = False, **kwargs):
    """
    Load LSTM + Bahdanau Attention phonetic model,
    Originally from https://github.com/open-dict-data/ipa-dict/blob/master/data/ma.txt

    Original size 10.4MB, quantized size 2.77MB .

    Parameters
    ----------
    quantized : bool, optional (default=False)
        if True, will load 8-bit quantized model.
        Quantized model not necessary faster, totally depends on the machine.

    Returns
    -------
    result: malaya.model.tf.Seq2SeqLSTM_Split class
    """

model = malaya.phoneme.deep_model_ipa()

Load Quantized model

To load 8-bit quantized model, simply pass quantized = True, default is False.

We can expect slightly accuracy drop from quantized model, and not necessary faster than normal 32-bit float model, totally depends on machine.

quantized_model = malaya.phoneme.deep_model_ipa(quantized = True)

Predict

def predict(self, strings: List[str], beam_search: bool = False):
    """
    Convert to target string.

    Parameters
    ----------
    strings : List[str]
    beam_search : bool, (optional=False)
        If True, use beam search decoder, else use greedy decoder.

    Returns
    -------
    result: List[str]
    """

If want to speed up the inference, set beam_search = False.

model.predict(['saya suka makan ayam', 'ayaq acaq kotoq'])

['saja suka makan ajam', 'ajaʔa atʃaʔa kotoʔa']

quantized_model.predict(['saya suka makan ayam', 'ayaq acaq kotoq'])

['saja suka makan ajam', 'ajaʔa atʃaʔa kotoʔa']

Limitation

Not able to convert numbers to phoneme.

model.predict(['123'])

['a']

you have to use normalization like https://malaya.readthedocs.io/en/latest/load-num2word.html

model.predict([malaya.num2word.to_cardinal(123)])

['səratus dua puluh tiga']