Dependency Parsing¶
This tutorial is available as an IPython notebook at Malaya/example/dependency.
This module only trained on standard language structure, so it is not save to use it for local language structure.
[4]:
%%time
import malaya
CPU times: user 4.98 s, sys: 944 ms, total: 5.92 s
Wall time: 6.56 s
Describe supported dependencies¶
[5]:
malaya.dependency.describe()
INFO:root:you can read more from https://universaldependencies.org/treebanks/id_pud/index.html
[5]:
| Tag | Description | |
|---|---|---|
| 0 | acl | clausal modifier of noun |
| 1 | advcl | adverbial clause modifier |
| 2 | advmod | adverbial modifier |
| 3 | amod | adjectival modifier |
| 4 | appos | appositional modifier |
| 5 | aux | auxiliary |
| 6 | case | case marking |
| 7 | ccomp | clausal complement |
| 8 | advmod | adverbial modifier |
| 9 | compound | compound |
| 10 | compound:plur | plural compound |
| 11 | conj | conjunct |
| 12 | cop | cop |
| 13 | csubj | clausal subject |
| 14 | dep | dependent |
| 15 | det | determiner |
| 16 | fixed | multi-word expression |
| 17 | flat | name |
| 18 | iobj | indirect object |
| 19 | mark | marker |
| 20 | nmod | nominal modifier |
| 21 | nsubj | nominal subject |
| 22 | obj | direct object |
| 23 | parataxis | parataxis |
| 24 | root | root |
| 25 | xcomp | open clausal complement |
List available transformer Dependency models¶
[6]:
malaya.dependency.available_transformer()
INFO:root:tested on 20% test set.
[6]:
| Size (MB) | Quantized Size (MB) | Arc Accuracy | Types Accuracy | Root Accuracy | |
|---|---|---|---|---|---|
| bert | 426.0 | 112.0 | 0.855 | 0.848 | 0.920 |
| tiny-bert | 59.5 | 15.7 | 0.718 | 0.694 | 0.886 |
| albert | 50.0 | 13.2 | 0.811 | 0.793 | 0.879 |
| tiny-albert | 24.8 | 6.6 | 0.708 | 0.673 | 0.817 |
| xlnet | 450.2 | 119.0 | 0.931 | 0.925 | 0.947 |
| alxlnet | 50.0 | 14.3 | 0.894 | 0.886 | 0.942 |
Make sure you can check accuracy chart from here first before select a model, https://malaya.readthedocs.io/en/latest/models-accuracy.html#Dependency-parsing
The best model in term of accuracy is XLNET.
Load xlnet dependency model¶
def transformer(model: str = 'xlnet', quantized: bool = False, **kwargs):
"""
Load Transformer Dependency Parsing model, transfer learning Transformer + biaffine attention.
Parameters
----------
model : str, optional (default='bert')
Model architecture supported. Allowed values:
* ``'bert'`` - Google BERT BASE parameters.
* ``'tiny-bert'`` - Google BERT TINY parameters.
* ``'albert'`` - Google ALBERT BASE parameters.
* ``'tiny-albert'`` - Google ALBERT TINY parameters.
* ``'xlnet'`` - Google XLNET BASE parameters.
* ``'alxlnet'`` - Malaya ALXLNET BASE parameters.
quantized : bool, optional (default=False)
if True, will load 8-bit quantized model.
Quantized model not necessary faster, totally depends on the machine.
Returns
-------
result: model
List of model classes:
* if `bert` in model, will return `malaya.model.bert.DependencyBERT`.
* if `xlnet` in model, will return `malaya.model.xlnet.DependencyXLNET`.
"""
[1]:
model = malaya.dependency.transformer(model = 'xlnet')
Load Quantized model¶
To load 8-bit quantized model, simply pass quantized = True, default is False.
We can expect slightly accuracy drop from quantized model, and not necessary faster than normal 32-bit float model, totally depends on machine.
[11]:
quantized_model = malaya.dependency.transformer(model = 'xlnet', quantized = True)
WARNING:root:Load quantized model will cause accuracy drop.
INFO:root:running dependency/xlnet-quantized using device /device:CPU:0
Predict¶
def predict(self, string: str):
"""
Tag a string.
Parameters
----------
string: str
Returns
-------
result: Tuple
"""
[12]:
string = 'Dr Mahathir menasihati mereka supaya berhenti berehat dan tidur sebentar sekiranya mengantuk ketika memandu.'
[13]:
d_object, tagging, indexing = model.predict(string)
d_object.to_graphvis()
[13]:
[14]:
d_object, tagging, indexing = quantized_model.predict(string)
d_object.to_graphvis()
[14]:
Voting stack model¶
[15]:
alxlnet = malaya.dependency.transformer(model = 'alxlnet')
tagging, indexing = malaya.stack.voting_stack([model, alxlnet, model], string)
malaya.dependency.dependency_graph(tagging, indexing).to_graphvis()
INFO:root:running dependency/alxlnet using device /device:CPU:0
[15]:
Harder example¶
[16]:
# https://www.astroawani.com/berita-malaysia/terbaik-tun-kita-geng-najib-razak-puji-tun-m-297884
s = "Najib yang juga Ahli Parlimen Pekan memuji sikap Ahli Parlimen Langkawi itu yang mengaku bersalah selepas melanggar SOP kerana tidak mengambil suhu badan ketika masuk ke sebuah surau di Langkawi pada Sabtu lalu"
[17]:
tagging, indexing = malaya.stack.voting_stack([model, alxlnet, model], s)
malaya.dependency.dependency_graph(tagging, indexing).to_graphvis()
[17]:
Dependency graph object¶
To initiate a dependency graph from dependency models, you need to call malaya.dependency.dependency_graph.
[18]:
graph = malaya.dependency.dependency_graph(tagging, indexing)
graph
[18]:
<malaya.function.parse_dependency.DependencyGraph at 0x1f5db7910>
generate graphvis¶
[20]:
graph.to_graphvis()
[20]:
Get nodes¶
[21]:
graph.nodes
[21]:
defaultdict(<function malaya.function.parse_dependency.DependencyGraph.__init__.<locals>.<lambda>()>,
{0: {'address': 0,
'word': None,
'lemma': None,
'ctag': 'TOP',
'tag': 'TOP',
'feats': None,
'head': None,
'deps': defaultdict(list, {'root': [7]}),
'rel': None},
1: {'address': 1,
'word': 'Najib',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 7,
'deps': defaultdict(list, {'fixed': [4]}),
'rel': 'nsubj'},
7: {'address': 7,
'word': 'memuji',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 0,
'deps': defaultdict(list, {'nsubj': [1], 'obj': [8]}),
'rel': 'root'},
2: {'address': 2,
'word': 'yang',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 4,
'deps': defaultdict(list, {}),
'rel': 'nsubj'},
4: {'address': 4,
'word': 'Ahli',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 1,
'deps': defaultdict(list,
{'nsubj': [2], 'advmod': [3], 'flat': [5]}),
'rel': 'fixed'},
3: {'address': 3,
'word': 'juga',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 4,
'deps': defaultdict(list, {}),
'rel': 'advmod'},
5: {'address': 5,
'word': 'Parlimen',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 4,
'deps': defaultdict(list, {'flat': [6]}),
'rel': 'flat'},
6: {'address': 6,
'word': 'Pekan',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 5,
'deps': defaultdict(list, {}),
'rel': 'flat'},
8: {'address': 8,
'word': 'sikap',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 7,
'deps': defaultdict(list, {'flat': [9], 'acl': [14]}),
'rel': 'obj'},
9: {'address': 9,
'word': 'Ahli',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 8,
'deps': defaultdict(list, {'flat': [10], 'det': [12]}),
'rel': 'flat'},
10: {'address': 10,
'word': 'Parlimen',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 9,
'deps': defaultdict(list, {'flat': [11]}),
'rel': 'flat'},
11: {'address': 11,
'word': 'Langkawi',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 10,
'deps': defaultdict(list, {}),
'rel': 'flat'},
12: {'address': 12,
'word': 'itu',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 9,
'deps': defaultdict(list, {}),
'rel': 'det'},
13: {'address': 13,
'word': 'yang',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 14,
'deps': defaultdict(list, {}),
'rel': 'nsubj'},
14: {'address': 14,
'word': 'mengaku',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 8,
'deps': defaultdict(list,
{'nsubj': [13],
'ccomp': [15],
'xcomp': [17],
'acl': [21]}),
'rel': 'acl'},
15: {'address': 15,
'word': 'bersalah',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 14,
'deps': defaultdict(list, {}),
'rel': 'ccomp'},
16: {'address': 16,
'word': 'selepas',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 17,
'deps': defaultdict(list, {}),
'rel': 'case'},
17: {'address': 17,
'word': 'melanggar',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 14,
'deps': defaultdict(list, {'case': [16], 'obj': [18]}),
'rel': 'xcomp'},
18: {'address': 18,
'word': 'SOP',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 17,
'deps': defaultdict(list, {'compound': [23]}),
'rel': 'obj'},
19: {'address': 19,
'word': 'kerana',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 23,
'deps': defaultdict(list, {}),
'rel': 'det'},
23: {'address': 23,
'word': 'badan',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 18,
'deps': defaultdict(list, {'det': [19]}),
'rel': 'compound'},
20: {'address': 20,
'word': 'tidak',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 21,
'deps': defaultdict(list, {}),
'rel': 'advmod'},
21: {'address': 21,
'word': 'mengambil',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 14,
'deps': defaultdict(list, {'advmod': [20], 'advcl': [25]}),
'rel': 'acl'},
22: {'address': 22,
'word': 'suhu',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 25,
'deps': defaultdict(list, {}),
'rel': 'obj'},
25: {'address': 25,
'word': 'masuk',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 21,
'deps': defaultdict(list,
{'obj': [22], 'mark': [24], 'obl': [28, 32]}),
'rel': 'advcl'},
24: {'address': 24,
'word': 'ketika',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 25,
'deps': defaultdict(list, {}),
'rel': 'mark'},
26: {'address': 26,
'word': 'ke',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 28,
'deps': defaultdict(list, {}),
'rel': 'case'},
28: {'address': 28,
'word': 'surau',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 25,
'deps': defaultdict(list,
{'case': [26], 'det': [27], 'nmod': [30]}),
'rel': 'obl'},
27: {'address': 27,
'word': 'sebuah',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 28,
'deps': defaultdict(list, {}),
'rel': 'det'},
29: {'address': 29,
'word': 'di',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 30,
'deps': defaultdict(list, {}),
'rel': 'case'},
30: {'address': 30,
'word': 'Langkawi',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 28,
'deps': defaultdict(list, {'case': [29]}),
'rel': 'nmod'},
31: {'address': 31,
'word': 'pada',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 32,
'deps': defaultdict(list, {'amod': [33]}),
'rel': 'case'},
32: {'address': 32,
'word': 'Sabtu',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 25,
'deps': defaultdict(list, {'case': [31]}),
'rel': 'obl'},
33: {'address': 33,
'word': 'lalu',
'lemma': '_',
'ctag': '_',
'tag': '_',
'feats': '_',
'head': 31,
'deps': defaultdict(list, {}),
'rel': 'amod'}})
Flat the graph¶
[22]:
list(graph.triples())
[22]:
[(('memuji', '_'), 'nsubj', ('Najib', '_')),
(('Najib', '_'), 'fixed', ('Ahli', '_')),
(('Ahli', '_'), 'nsubj', ('yang', '_')),
(('Ahli', '_'), 'advmod', ('juga', '_')),
(('Ahli', '_'), 'flat', ('Parlimen', '_')),
(('Parlimen', '_'), 'flat', ('Pekan', '_')),
(('memuji', '_'), 'obj', ('sikap', '_')),
(('sikap', '_'), 'flat', ('Ahli', '_')),
(('Ahli', '_'), 'flat', ('Parlimen', '_')),
(('Parlimen', '_'), 'flat', ('Langkawi', '_')),
(('Ahli', '_'), 'det', ('itu', '_')),
(('sikap', '_'), 'acl', ('mengaku', '_')),
(('mengaku', '_'), 'nsubj', ('yang', '_')),
(('mengaku', '_'), 'ccomp', ('bersalah', '_')),
(('mengaku', '_'), 'xcomp', ('melanggar', '_')),
(('melanggar', '_'), 'case', ('selepas', '_')),
(('melanggar', '_'), 'obj', ('SOP', '_')),
(('SOP', '_'), 'compound', ('badan', '_')),
(('badan', '_'), 'det', ('kerana', '_')),
(('mengaku', '_'), 'acl', ('mengambil', '_')),
(('mengambil', '_'), 'advmod', ('tidak', '_')),
(('mengambil', '_'), 'advcl', ('masuk', '_')),
(('masuk', '_'), 'obj', ('suhu', '_')),
(('masuk', '_'), 'mark', ('ketika', '_')),
(('masuk', '_'), 'obl', ('surau', '_')),
(('surau', '_'), 'case', ('ke', '_')),
(('surau', '_'), 'det', ('sebuah', '_')),
(('surau', '_'), 'nmod', ('Langkawi', '_')),
(('Langkawi', '_'), 'case', ('di', '_')),
(('masuk', '_'), 'obl', ('Sabtu', '_')),
(('Sabtu', '_'), 'case', ('pada', '_')),
(('pada', '_'), 'amod', ('lalu', '_'))]
Check the graph contains cycles¶
[23]:
graph.contains_cycle()
[23]:
False
Generate networkx¶
Make sure you already installed networkx,
pip install networkx
[24]:
digraph = graph.to_networkx()
digraph
[24]:
<networkx.classes.multidigraph.MultiDiGraph at 0x1f5dbc390>
[25]:
import networkx as nx
import matplotlib.pyplot as plt
nx.draw_networkx(digraph)
plt.show()
[26]:
digraph.edges()
[26]:
OutMultiEdgeDataView([(1, 7), (2, 4), (3, 4), (4, 1), (5, 4), (6, 5), (8, 7), (9, 8), (10, 9), (11, 10), (12, 9), (13, 14), (14, 8), (15, 14), (16, 17), (17, 14), (18, 17), (19, 23), (20, 21), (21, 14), (22, 25), (23, 18), (24, 25), (25, 21), (26, 28), (27, 28), (28, 25), (29, 30), (30, 28), (31, 32), (32, 25), (33, 31)])
[27]:
digraph.nodes()
[27]:
NodeView((1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33))
[28]:
labels = {i:graph.get_by_address(i)['word'] for i in digraph.nodes()}
labels
[28]:
{1: 'Najib',
2: 'yang',
3: 'juga',
4: 'Ahli',
5: 'Parlimen',
6: 'Pekan',
7: 'memuji',
8: 'sikap',
9: 'Ahli',
10: 'Parlimen',
11: 'Langkawi',
12: 'itu',
13: 'yang',
14: 'mengaku',
15: 'bersalah',
16: 'selepas',
17: 'melanggar',
18: 'SOP',
19: 'kerana',
20: 'tidak',
21: 'mengambil',
22: 'suhu',
23: 'badan',
24: 'ketika',
25: 'masuk',
26: 'ke',
27: 'sebuah',
28: 'surau',
29: 'di',
30: 'Langkawi',
31: 'pada',
32: 'Sabtu',
33: 'lalu'}
[29]:
plt.figure(figsize=(15,5))
nx.draw_networkx(digraph,labels=labels)
plt.show()
Vectorize¶
Let say you want to visualize word level in lower dimension, you can use model.vectorize,
def vectorize(self, string: str):
"""
vectorize a string.
Parameters
----------
string: List[str]
Returns
-------
result: np.array
"""
[42]:
r = quantized_model.vectorize(s)
[43]:
x = [i[0] for i in r]
y = [i[1] for i in r]
[44]:
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
tsne = TSNE().fit_transform(y)
tsne.shape
[44]:
(33, 2)
[45]:
plt.figure(figsize = (7, 7))
plt.scatter(tsne[:, 0], tsne[:, 1])
labels = x
for label, x, y in zip(
labels, tsne[:, 0], tsne[:, 1]
):
label = (
'%s, %.3f' % (label[0], label[1])
if isinstance(label, list)
else label
)
plt.annotate(
label,
xy = (x, y),
xytext = (0, 0),
textcoords = 'offset points',
)
[ ]: