MADRID, June 25 (Portaltic/EP) –
Big language models (L.L.M.for its acronym in English) require a large amount of processing and energy to execute, which limits their deployment on devices with more limited features, such as mobile phones.
Qualcomm addresses this problem with quantification of LLMa technique that, as he explains, is used “to reduce the precision of the numbers used in calculations, which in turn decreases the size of the model and the resources needed to run it.”
“To the reduce size and computational demands of LLMs, AI applications can become more accessible and run efficiently on a wider range of devices, from smartphones to IoT devices,” said Mart van, engineer, senior staff/manager, Qualcomm Technologies Netherlands BV. Baalen; and Senior Director of Software Engineering at Qualcomm Technologies, Inc., Abhijit Khobare.
Of the different techniques that are included in quantification, researchers highlight vector quantization, because it leads to more efficient compression and less information loss by opting for the joint distribution of the parameters instead of quantifying each parameter independently.
Developers can access these techniques with AI Model Efficiency Toolkit, a library that offers quantification techniques and model understanding for training, and within Qualcomm AI Hubwhich already has more than one hundred optimized models, although it also allows the use of its own models.
The latter allows, as a novelty, run the models on the devices that integrate Snapdragon X Elite, a Qualcomm processor designed for computers with the capacity to run generative AI models with more than 13 billion parameters. It is compatible with Facebook Denoiser, Llama2, OpenAI Clip, Riffusion, Stable Diffusion v2.1 and Whisper, among others.
Snapdragon computers designed for the AI eralike the Copilot+ PCs powered by Microsoft Windows.
