Devices
Contents
Devices#
This tutorial is available as an IPython notebook at Malaya/example/devices.
List available devices supported to run Malaya model#
[1]:
import malaya
import logging
logging.basicConfig(level = logging.INFO)
[2]:
malaya.utils.available_device()
[2]:
[('CPU:0', '0.268 GB'),
('XLA_CPU:0', '17.18 GB'),
('XLA_GPU:0', '17.18 GB'),
('XLA_GPU:1', '17.18 GB'),
('XLA_GPU:2', '17.18 GB'),
('XLA_GPU:3', '17.18 GB'),
('GPU:0', '29.466 GB'),
('GPU:1', '29.469 GB'),
('GPU:2', '29.469 GB'),
('GPU:3', '29.469 GB')]
Use specific device for specific model#
To do that, pass device
parameter to any load model function in Malaya, default is CPU:0
.
malaya.sentiment.transformer(model = 'alxlnet', device = 'CPU:0')
Or if you want to use XLA,
malaya.sentiment.transformer(model = 'alxlnet', device = 'XLA_CPU:0')
By default, device
will automatically set to a gpu with the most empty memory if you have GPUs detected.
[3]:
alxlnet_cpu = malaya.sentiment.transformer(model = 'alxlnet', device = 'CPU:0')
INFO:root:running sentiment/alxlnet using device /device:GPU:1
[4]:
alxlnet_cpu = malaya.sentiment.transformer(model = 'alxlnet', device = 'GPU:1')
INFO:root:running sentiment/alxlnet using device /device:GPU:1
Disable auto GPU#
Let say you do not want to use auto allocate to gpu, simply set auto_gpu
to False
, or set,
export CUDA_VISIBLE_DEVICES=''
[5]:
alxlnet_cpu = malaya.sentiment.transformer(model = 'alxlnet', device = 'CPU:0', auto_gpu = False)
INFO:root:running sentiment/alxlnet using device /device:CPU:0
[6]:
alxlnet_xla_cpu = malaya.sentiment.transformer(model = 'alxlnet', device = 'XLA_CPU:0', auto_gpu = False)
INFO:root:running sentiment/alxlnet using device /device:XLA_CPU:0
[7]:
string = 'saya kentut busuk tapi muka comel'
[8]:
%%time
alxlnet_cpu.predict_proba([string])
CPU times: user 4.95 s, sys: 636 ms, total: 5.59 s
Wall time: 5.14 s
[8]:
[{'negative': 0.99993134, 'positive': 6.920824e-07, 'neutral': 6.7949295e-05}]
[9]:
%%time
alxlnet_xla_cpu.predict_proba([string])
CPU times: user 49.9 s, sys: 818 ms, total: 50.7 s
Wall time: 50.3 s
[9]:
[{'negative': 0.99997425, 'positive': 2.5436142e-07, 'neutral': 2.5510788e-05}]
Again, not all Tensorflow operation support XLA.