Devices

This tutorial is available as an IPython notebook at Malaya/example/devices.

List available devices supported to run Malaya model

[1]:
import malaya
import logging
logging.basicConfig(level = logging.INFO)
[2]:
malaya.utils.available_device()
[2]:
[('CPU:0', '0.268 GB'),
 ('XLA_CPU:0', '17.18 GB'),
 ('XLA_GPU:0', '17.18 GB'),
 ('XLA_GPU:1', '17.18 GB'),
 ('XLA_GPU:2', '17.18 GB'),
 ('XLA_GPU:3', '17.18 GB'),
 ('GPU:0', '29.466 GB'),
 ('GPU:1', '29.469 GB'),
 ('GPU:2', '29.469 GB'),
 ('GPU:3', '29.469 GB')]

Use specific device for specific model

To do that, pass device parameter to any load model function in Malaya, default is CPU:0.

malaya.sentiment.transformer(model = 'alxlnet', device = 'CPU:0')

Or if you want to use XLA,

malaya.sentiment.transformer(model = 'alxlnet', device = 'XLA_CPU:0')

By default, device will automatically set to a gpu with the most empty memory if you have GPUs detected.

[3]:
alxlnet_cpu = malaya.sentiment.transformer(model = 'alxlnet', device = 'CPU:0')
INFO:root:running sentiment/alxlnet using device /device:GPU:1
[4]:
alxlnet_cpu = malaya.sentiment.transformer(model = 'alxlnet', device = 'GPU:1')
INFO:root:running sentiment/alxlnet using device /device:GPU:1

Disable auto GPU

Let say you do not want to use auto allocate to gpu, simply set auto_gpu to False, or set,

export CUDA_VISIBLE_DEVICES=''
[5]:
alxlnet_cpu = malaya.sentiment.transformer(model = 'alxlnet', device = 'CPU:0', auto_gpu = False)
INFO:root:running sentiment/alxlnet using device /device:CPU:0
[6]:
alxlnet_xla_cpu = malaya.sentiment.transformer(model = 'alxlnet', device = 'XLA_CPU:0', auto_gpu = False)
INFO:root:running sentiment/alxlnet using device /device:XLA_CPU:0
[7]:
string = 'saya kentut busuk tapi muka comel'
[8]:
%%time

alxlnet_cpu.predict_proba([string])
CPU times: user 4.95 s, sys: 636 ms, total: 5.59 s
Wall time: 5.14 s
[8]:
[{'negative': 0.99993134, 'positive': 6.920824e-07, 'neutral': 6.7949295e-05}]
[9]:
%%time

alxlnet_xla_cpu.predict_proba([string])
CPU times: user 49.9 s, sys: 818 ms, total: 50.7 s
Wall time: 50.3 s
[9]:
[{'negative': 0.99997425, 'positive': 2.5436142e-07, 'neutral': 2.5510788e-05}]

Again, not all Tensorflow operation support XLA.