
Compiling and running a pre-trained LLM on AWS Inferentia accelerator
In this tutorial, we are going to compile and run a small llama architecture model on an EC2 instance and if we manage to pass the compilation and inference test, it means our model is compatible.
Source code in Github: https://github.com/p0o/run-models-in-aws-inf2-ml-accelerator