Testing the models with LlamaCpp and ONNX
Towards Data Science 12:53 pm on May 23, 2024
Testing small language models like Phi-3 and Llama-3 with PCs and Raspberry Pi by Dmitrii Eliuseev shows feasibility in limited resources. The text discusses deploying these models using LlamaCpp, ONNX for cross-platform compatibility, and the implications on data science practices.
- Model Selection: Phi-3 & Llama-3 are chosen due to their balance between performance and resource demands.
- Testing Platforms: Models are tested on PCs and Raspberry Pi, showcasing versatility across devices.
- LlamaCpp and ONNX: Utilization of LlamaCpp for model execution and ONNX format for cross-platform compatibility.
- Implications: Highlights the trend toward more accessible AI development without full reliance on high-powered hardware.
- Authorship and Publication: By Dmitrii Eliuseev, published in 'Towards Data Science' reflecting current industry insights.
Category
https://towardsdatascience.com/small-language-models-using-3-8b-phi-3-and-8b-llama-3-models-on-a-pc-and-raspberry-pi-9ed70127fe61
< Previous Story - Next Story >