How to deploy your ChatGPT-like app with Chainlit and Fly.io

Willy Douhard — Wed, 31 May 2023 10:29:39 +0000

🔗💡Chainlit is an open-source Python package that allows you to create ChatGPT-like UIs on top of any Python code in just minutes! Visit the GitHub repo to get started!

This post is designed to guide you through deploying your Chainlit apps to Fly.io.

Fly is an excellent choice for two reasons:

💸 It offers a free plan
✨ It's super easy

In this tutorial, we'll be using this simple example from the Chainlit cookbook repo. Feel free to use your own application instead.

Prerequisites:

Python >= 3.8
A Fly.io account

Step 1: Install the Fly CLI 🚀

The Fly CLI enables you to deploy your app with just a couple of command lines! First, you'll need to install it.

For mac users, run:

brew install flyctl

For other operating systems, refer to Fly's detailed guide on how to install here.

Step 2: Login to Fly 🔒

After installing the CLI, you will first need to signup:

flyctl auth signup

Then run:

flyctl auth login

Step 3: Run your Chainlit app locally 🏠

Before deploying, it's a good idea to make sure your application runs locally without any problems.

If you don't have a Chainlit app ready, clone our example from the cookbook:

git clone https://github.com/Chainlit/cookbook.git chainlit-cookbook

Then, navigate to the app folder:

cd chainlit-cookbook/fly-io-deployment

Finally, run chainlit run main.py. If you're using your own application, replace main.py with the appropriate file.

You should see the application up and running:

Step 4: Add Procfile and requirements.txt 📄

A Procfile is a way to declare how to start your application. The cookbook example app we're using already includes one. If you're running your own application, create a Procfile at the root of your project with the following content:

web: python -m chainlit run main.py -h --port 8080

If you're using your own application, replace main.py with the appropriate file

A requirement.txt file lists all the Python dependencies of your app. Again, the cookbook example already has one. If you are deploying your own app make sure to create one. If any dependency is missing, the deployment will fail!

Step 5: Create the Fly project 🎉

We're almost ready to deploy!

Although you will be using Fly's free plan, you still need to provide a payment method (you will not be charged) on the Fly website before deploying your application.

Once the payment method is setup, we need to create the Fly project that will host our app. To do that, run:

flyctl launch

You’ll be prompted with several Fly.io questions. Answer NO to all of them ✋. Here's a screenshot of my setup:

Take note of the Hostname, which is the URL your app will be deployed to.

Step 6: Deploy! 🚢

The final (and easiest) step! Run:

flyctl deploy

Once the deployment finished, you have to run one final command:

flyctl scale count 1

The reason for that is that Fly does not support sticky Websocket sessions out of the box, meaning that scaling to multiple machine will prevent your Chainlit app to work.

If you need features like autoscaling, I advise you to deploy your app on AWS ECS or GCP, with an Application Load Balancer that support sticky sessions.

🎊 And voilà! Your app is available at the Hostname provided in the previous step. 🎊 In my case:

Fly demo

summer-pond-3349.fly.dev

If you have any questions or suggestions, please let me know in the comments below, or feel free to get in touch on Discord!

Forem: Willy Douhard