Rust LLM AWS Lambda

Project Link: Click this link

Description

This is an example of how to dockerize a Rust LLM project and deploy it to AWS Lambda.

Features

• Utilze llm to build a text generation function that can be deployed to AWS Lambda.
• Model in use is BLOOM
• AWS Lambda deployment is done using AWS CLI

Example Usage

We want to automatically generate a text based on the input text. Here, for example, we use "Rust with AWS" as the input text. Then we get:

curl "http://localhost:9000?text=Rust%20with%20AWS"

OR using AWS lambda, the input text is passed as a query parameter text to the lambda function. The lambda function will generate a text based on the input text and return it as a response:

curl "https://gerupw4q2kyr3jaadmkyhzx7ni0ccphs.lambda-url.us-east-1.on.aws/?text=Rust%20with%20AWS"

OR accessing the lambda function through web browser: https://gerupw4q2kyr3jaadmkyhzx7ni0ccphs.lambda-url.us-east-1.on.aws/?text=Rust%20with%20AWS

AWS Lambda Function

Using AWS CLI, we can deploy the Rust LLM project to AWS Lambda.

The lambda function is created using the following code:

docker build -t rust-llm-aws-lambda .

Then, deploy the lambda function:

docker tag rust-llm-aws-lambda:latest 309324599654.dkr.ecr.us-east-1.amazonaws.com/rust-llm-aws-lambda:latest

Finally, push the image to ECR:

docker push 309324599654.dkr.ecr.us-east-1.amazonaws.com/rust-llm-aws-lambda:latest

After the image is pushed to ECR, we can deploy the lambda function using AWS lambda by creating with container image:

Then, we need to set up the trigger for the lambda function. Here, we use API Gateway as the trigger:

Finally, we can access the lambda function by setting the function URL:

Screenshots

• Local deployment:

• Model while running on local machine:

• AWS Lambda deployment: