Forem: Nick Hehr

Practical AI - Local LLM and machine learning for plant care

Nick Hehr — Fri, 29 Mar 2024 15:23:31 +0000

Create a custom AI assistant with local data security and no usage charges in less than 50 lines of code.

Story

I've never bought a house plant. And yet, somehow my house and porch have slowly started to fill with all varieties of flora and fauna. Even with my aversion to seeking them out (thank you, all our plant-gifting friends!), I do need to keep the leafy beings alive and healthy. To help myself and anyone else who may be plant-sitting, I built an AI-powered greenery consultant that could identify the various species and provide helpful tips for taking care of the growing population within my home.

This smart machine uses speech-to-text to listen for a question, recognizes the plant type using a pre-trained machine learning model and computer vision, and passes that context into a local large language model (LLM) before sharing the response out loud using text-to-speech. This provides a natural and intuitive way of interacting with your new assistant while keeping the data and processing in your control.

The plant identification model

This project might sound like a lot to put together from scratch, but Viam's built-in and community modules make it possible to compose each of these capabilities together; adding or removing features as needed while being independent of specific hardware components, software packages or ML model types. For example, when I approached this project I knew I needed to use a machine learning model to recognize plant types, but was not sure which model.

I knew that Viam makes it easy to train and create your own custom ML models, but I did not feel comfortable enough training a model on plants I know nothing about.
Next, I looked in the Viam Registry - as a new feature was just released that allows people to share and use ML models directly on the Viam Registry. I found a model that my colleague Arielle had created at a plant store when working with Nvidia on a webinar to highlight Viam’s integration with Nvidia’s Triton server, but since it was for demo purposes it was only trained on 5 plant types.
I searched around the internet, and found a model for houseplants that was trained on over 6 thousand images - could it work with Viam? Yes, because the Viam Registry has modules that allow you to point to and use Roboflow and HuggingFace models.

To use any of these models, first choose the kind of model and add a couple of details.

Then, whether it be a model you trained yourself, a model you find on the Viam Registry, or a HuggingFace, Roboflow or DirectAI model, you can “see” the world with the same single line of code:

detections = await roboflow_plants.get_detections(image)

Large Language Models (LLMs)

LLMs (Large Language Models), like ChatGPT, have gained a lot of well-deserved attention in the past year, and have been integrated into many services in most industries. However, only now are we seeing LLMs that are both small and useful enough to run locally on portable devices. This gets interesting for a few reasons:

LLMs in the cloud can get costly, as they charge for API access.
For some use cases, you want to maintain privacy and not have to worry about your interactions with an LLM going to the cloud and potentially even be used to add to the LLMs knowledge base (imagine a custom LLM used for corporate document analysis, for example).
A local LLM can maintain its own knowledge base and does not rely on internet connectivity.

A Viam module exists that allows you to deploy a local LLM to your machine in just a few clicks. This module currently defaults to TinyLlama 1.1B, which provides reasonably accurate and performant results on small, portable single board computers like the Raspberry Pi 5 and OrangePi 5B.

For my use-case, the AI LLM can cover a wide range of topics, such as watering frequency, lighting conditions, temperature preferences, and fertilization schedules. The LLM's ability to understand context and generate personalized responses makes it an ideal tool for guiding users in their plant care journey. The model and prompt can be tuned to be more specific or broad, depending on the use case.

Supplies

Viam allows you to be mostly hardware-agnostic, so you can try this all with a Macbook, but because I wanted to create a small portable device I used the following:

A single-board computer (Orange Pi 5B or other octa-core CPU for best performance, a Raspberry Pi 5 will also work)
MicroSD card for the computer OS
USB webcam with a microphone
USB speaker
Power supply for the board

Build instructions

1. Install viam-server

The first step in this project build is to install viam-server on the single-board computer (SBC) with the Viam installation instructions. Then in the Viam App, add a new machine instance. I named mine “smart-assistant”, but you can choose another name if you want.

2. Configure the camera

We will now set up the components and services, beginning with a camera. Be sure to have the USB devices (webcam and speaker) connected to board when turning it on.

Viam has support for a variety of cameras; we chose to use a webcam, which generally can be set up with just a few clicks as Viam will auto-detect any attached cameras.

3. Configure local LLM chat module

Now, we can start adding capabilities to our machine. First, we’ll add the local LLM, which we can find in the Service configuration menu, under ‘chat’.

Configuration attributes are documented in the local LLM module README, but we can start with the defaults.

After configuring the LLM module, we can test it by running a small Python script using the Viam Python SDK. If you’ve not installed the Viam Python SDK, install it and the chat API used by the local LLM module by running this command on your terminal:

pip install viam-sdk chat_service_api@git+https://github.com/viam-labs/chat-service-api.git@main

Then, create this script called assistant.py on your machine, but substitute api_key, api_key_id, and your robot address with the correct values from the Viam app’s Code sample tab.

import asyncio
from viam.robot.client import RobotClient
from chat_service_api import Chat

async def connect():
    opts = RobotClient.Options.with_api_key(
      api_key='',
      api_key_id=''
    )
    return await RobotClient.at_address('', opts)

async def main():
    robot = await connect()

    llm = Chat.from_robot(robot, name="llm")
    response = await llm.chat("what is a robot?")
    print(response)

    # Don't forget to close the machine when you're done!
    await robot.close()

if __name__ == '__main__':
    asyncio.run(main())

Run it with python assistant.py to produce a result like:

A robot is an autonomous machine designed to perform specific tasks with limited human control. They are programmed to follow predetermined instructions or rules, making them highly efficient and reliable. Robots are used in various industries, from manufacturing and assembly to healthcare and transportation.

You now have a LLM running and generating responses all on your local machine!

4. Add a speech interface

We want to interact with our local LLM in a natural way, so the next thing to add is a module that gives our machine capabilities to:

Listen to us speak and convert that speech to text that we can be passed to the LLM
Convert the LLM response to audio so we can hear the response spoken aloud

We’ll use a speech module from the Viam registry with these capabilities. To configure it, we’ll go back to the Services configuration tab in the Viam app and choose the "speechio" module.

There are options to customize the speech module’s behavior. Since we’ll want to be able to talk to our plant assistant, we need to allow the module to listen to the microphone, and also to have a trigger prefix (the speech that will trigger a response, you can set this to whatever you prefer). Add this configuration to the attributes field of the speech service, and save.

{
  "listen": true,
  "listen_trigger_command": "hey there"
}

Once you’ve configured the module, add the speech API in the location you are running your Python scripts by running this command in the terminal:

pip install speech_service_api@git+https://github.com/viam-labs/speech-service-api.git@main

Now update the assistant.py program to import the speech API, initialize the speech service, listen, and have the speech service speak the LLM response out loud.

We will have the script running in a loop. When the speech module hears a command, it will add it to a buffer. We’ll pull that command out of the buffer and send it to the LLM.

import asyncio
from viam.robot.client import RobotClient
from chat_service_api import Chat
from speech_service_api import SpeechService

async def connect():
    opts = RobotClient.Options.with_api_key(
      api_key='',
      api_key_id=''
    )
    return await RobotClient.at_address('', opts)

async def main():
  robot = await connect()

  llm = Chat.from_robot(robot, name="llm")
  speech = SpeechService.from_robot(robot, name="speech")

  while True:
    commands = await speech.get_commands(1)
    if len(commands) > 0:
      command = commands[0]
      print(command)
             response = await llm.chat(command)
             print(response)
      await speech.say(response, True)

    # Don't forget to close the machine when you're done!
    await robot.close()

if __name__ == '__main__':
    asyncio.run(main())

You now have an AI assistant that you can converse with when you run this script and say something that is prefixed with the trigger you chose, like:

Hey there, how often should I water my ficus plant?

Hey there, does my spider plant need a lot of sun?

Hey there, my cat peed on my verbena plant, what should I do?

5. Enhance your LLM assistant with computer vision

Now that we have our speech interface to our local LLM working, the last step is to connect it to the machine learning model we set up previously so that we can leverage computer vision to see and recognize our plants.

We’ll then have it replace mentions of “this plant”, “that plant” and “the plant” with the name of the plant our computer vision service most recently saw. We can then point our assistant at one of our plants and say:

Hey there, how do I take care of this plant?

What will get updated to:

How do I take care of this snake plant?

We can make this possible with a few changes in our code. We’ll first import the ‘re’ module, which allows us to use regular expressions to replace the name of the plant in our text. Then, we’ll import and instantiate our plant detector and use it to detect any plants it sees.

import asyncio
import re
from viam.robot.client import RobotClient
from viam.services.vision import VisionClient
from chat_service_api import Chat
from speech_service_api import SpeechService

async def connect():
    opts = RobotClient.Options.with_api_key(
      api_key='',
      api_key_id=''
    )
    return await RobotClient.at_address('', opts)

async def main():
  robot = await connect()

  llm = Chat.from_robot(robot, name="llm")
  speech = SpeechService.from_robot(robot, name="speech")
  roboflow_plants = VisionClient.from_robot(robot, "roboflow-plants")
  plant = ""

  while True:
    detections = await roboflow_plants.get_detections_from_camera('cam')
    if len(detections) > 0:
      print(detections[0].class_name)
      plant = detections[0].class_name

    commands = await speech.get_commands(1)
      if len(commands) > 0:
        command = commands[0]
        if plant != "":
          command = re.sub(r'(this|that|the) plant',  "a " + plant, command)
        print(command)

        response = await llm.chat(command)
        print(response)
        await speech.say(response, True)

    # Don't forget to close the machine when you're done!
    await robot.close()

if __name__ == '__main__':
    asyncio.run(main())

You now have an AI and computer vision enabled plant care assistant, and all of this is possible with less than 50 lines of code!

Thank you to Matt Vella for recording this working demo!

Next steps

Making this project was a great way to test the possibilities of running an AI LLM locally, and combining that with other capabilities like machine learning and computer vision. Other things I am hoping to do next:

I want to try fine-tuning a local LLM like TinyLlama with a custom dataset so that it can have specialized knowledge. For example, I could train it with the Viam documentation so that it can help me build smart machines.
Despite the LLM being local, the speech service I used defaults to using Google service for text-to-speech (TTS) and speech-to-text (STT). I want to integrate the new stt-vosk and tts-piper modules to perform these tasks locally.
I want to create a portable case for the components so I truly can use this as a portable device.

I am interested in how you might use local LLMs. Leave a comment below or join us in Discord to let the community know!

This post was originally shared on the Viam blog.

A Smarter On-Air Light Using ESP32 & Viam

Nick Hehr — Tue, 19 Dec 2023 17:07:06 +0000

Want to avoid that situation where a random family member or coworker walks into the background of your Zoom meeting because they didn't know you were live?

Have you tried to program microcontrollers past blinking an LED and gave up from frustration?

Do you wish to wield the power of webhooks in your hardware project?

Join me on a journey to create an automated on-air light that updates based on my Zoom meeting status, powered by an ESP32 and Viam.

My son is starting become more mobile and explore all the different spaces around my house. I've always wanted some sort of indicator to him, my wife, and anyone else in the house that I'm currently in a meeting without needing to remember turn it on or off. It would also be useful if it could help me remember when the meeting has started; I think we've all been there when a reminder for a meeting comes up 5 minutes before it starts and then it completely slips your mind until 5 minutes after it starts. 😅

To accomplish this task, I used a cardboard box, an ESP32, an RGB LED, and a USB battery (along with the wires and cables to connect them all together). The ESP32 is a very popular microcontroller because it provides Wi-Fi, Bluetooth, plenty of GPIO, and a solid amount of memory and storage for its form factor while staying relatively low-power, so it can run all day using the USB battery. Since I didn't need to worry about connecting to an outlet, the light can be mounted wherever I like.

This also gave me a chance to try the latest release of the Viam micro-RDK, so I could securely connect to and program the device with my choice of languages. Typically, when a network-connected device in a home (or office) needs to communicate with the world wide web, there are a few methods to do it:

Some code on the device could check in with a cloud service to see if there is any relevant information with which it can take action; this is called "polling", which is a quick and valid approach when you're not worried about power or checking the service too often for updates.
The device could also "subscribe" to updates from a messaging service in the cloud using MQTT or another "pub/sub" protocol. This is more efficient for the device and can be useful for scaling those updates to many devices, which is not a concern for my project; plus it adds a layer of complexity to the stack to maintain this project over time.
There is also "hard mode": opening a public port on the network router and forwarding a request to the public IP address to the device, which could have security implications if done incorrectly and it's not guaranteed that the IP address will always stay the same since typical home service providers can reassign them at will.

Devices connected to Viam take a different approach entirely. The RDK and micro-RDK use gRPC to establish a secure peer-to-peer WebRTC connection with the Viam cloud service and any programs using one of Viam's SDKs. This means any communication goes directly to/from the device without worrying about exposing private networks to the public internet or managing additional cloud services in-between.

To control the light automatically based on my Zoom status, I created a webhooks-only app written in Python connected to my account. I deployed it to Fly since they make it so easy to launch web applications for free.

Check out the working demo video below and read the build instructions to learn how to set up a smarter on-air light for yourself!

Supplies

1 x ESP32 development board compatible with the Viam micro-RDK

1 (or more) x RGB LED

1 x breadboard to hold the components

4 (at least) x jumper wires (can be a mix of male-to-male, female-to-female, or female-to-male)

1 x USB data cable that will connect to the development board and your computer

1 x USB battery

1 x container to hold the components and battery (I used a cardboard box with "On Air" cut out )

Build Instructions

1. Wiring the Components

We'll start off by wiring the RGB LED to the ESP32:

The diagram above shows a breadboard for clarity, but the RGB LED and ESP32 could be wired directly together using male-to-male jumpers if you choose.

Take note of the GPIO pin numbers connected by red, blue, and green jumper wires to three of the LED legs, any GPIO that support digital output or pulse width modulation (PWM) will work; check your board's pinout to verify. Here we're using GPIO pins 15, 32, and 14. Those values will be important for configuring the development board in the Viam app and the Python code later. The jumper connecting the 3V pin on the ESP32 board to the LED will provide power to the component when the board is connected to your computer or the battery over USB.

2. Flash and Configure the ESP32 in Viam

Next is creating a new robot in the Viam app and following the "Setup" instructions to flash the ESP32 with the micro-RDK.

Once you've confirmed that it is connected to Viam (you should see a "Live" status under "Last online" in the header of the app), configure the board component and relevant pins using the following JSON:

{
  "components": [
    {
      "attributes": {
        "pins": [
          15,
          32,
          14
        ]
      },
      "depends_on": [],
      "name": "board",
      "model": "esp32",
      "type": "board",
      "namespace": "rdk"
    }
  ]
}

Remember to replace the numbers in the list of pins with the ones matching the physical wiring of the project.

Restart the ESP32 to confirm a working configuration. If there is a "reset" button on the development board, use that; otherwise, unplugging and plugging in the board also works. You can look at the Control tab in the app to read and write the configured pin values to control the LED. Try setting all the pins to 1 or 0 or a floating number in between like 0.5 to see what happens.

3. Web Application Setup

Now to build the web application that will accept webhook requests from Zoom and control the light, we'll use the Viam Python SDK with the Starlette async web framework. Clone the working project from GitHub.

git clone https://github.com/HipsterBrown/viam-on-air.git

To configure the application with the required API key, API key ID, and location address for the Viam SDK and the secret token from Zoom, make a copy of the .env.example file and call it .env.

Get the Viam API and location values from the "Code Sample" tab when viewing the machine details in the app. Toggle the "Include API Key" switch to reveal the API Key and API Key ID in the code where "" and "" is seen initially.

Get the Zoom secret token by creating a webhook-only app in the Zoom App Marketplace and subscribing to the following events when adding an event subscription:

Update the .env file with the relevant values, including the configured pins from Viam as RGB_PINS and your Zoom username to make sure the light only updates when you join or leave a meeting.

With a working Python development environment, run the install command to get the dependencies in the cloned project repository:

pip install -e .

Start the web application:

python -m viam_on_air

This will connect to the ESP32 and start a webserver on 0.0.0.0:8000 (a.k.a localhost:8000).

❯ python -m viam_on_air
INFO:   Started server process [96243]
INFO:   Waiting for application startup.
2023-11-22 17:40:02,956         INFO    viam.rpc.dial (dial.py:286)     Connecting to socket: /tmp/proxy-LSL4xP2F.sock
INFO:   Application startup complete.
INFO:   Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now that we've confirmed it is working, let's take a quick look at some of the code involved!

Using Starlette's Config class, the values from the .env file (as well as variables set in the execution environment, i.e. your computer or server) are set as Python variables:

app_config = Config(".env")

ZOOM_SECRET_TOKEN = app_config("ZOOM_SECRET_TOKEN", cast=Secret)
ZOOM_USERNAME = app_config("ZOOM_USERNAME", cast=str)
VIAM_API_KEY = app_config("VIAM_API_KEY", cast=Secret)
VIAM_API_KEY_ID = app_config("VIAM_API_KEY_ID", cast=Secret)
VIAM_ADDRESS = app_config("VIAM_ADDRESS", cast=str)
BOARD_NAME = app_config("BOARD_NAME", cast=str, default="board")
RGB_PINS = app_config("RGB_PINS", cast=CommaSeparatedStrings, default=["18", "5", "19"])

To share the long-running WebRTC connection to the ESP32 with each webhook request coming into the application, rather than establishing the connection each time a request occurs, Starlette's lifespan feature is handy for managing the async context:

@contextlib.asynccontextmanager
async def lifespan(_app: Starlette) -> AsyncIterator[State]:
    async with create_robot(
        api_key=str(VIAM_API_KEY),
        api_key_id=str(VIAM_API_KEY_ID),
        robot_address=VIAM_ADDRESS,
        board_name=BOARD_NAME,
        rgb_pins=tuple(RGB_PINS),
    ) as robot:
        yield {"robot": robot}

The create_robot method encapsulates the logic for creating a new RobotClient instance from the Viam Python SDK and wrapping it in a custom Robot class for managing the RGB LED controls. It will automatically close the connection to the ESP32 when the server shuts down since it is provided as an async context:

@contextlib.asynccontextmanager
async def create_robot(
    api_key: str, api_key_id: str, robot_address: str, board_name: str, rgb_pins: Tuple[str, str, str]
):
    opts = RobotClient.Options.with_api_key(
        api_key=api_key,
        api_key_id=api_key_id,
    )
    client = await RobotClient.at_address(robot_address, opts)
    board = Board.from_robot(client, board_name)

    robot = Robot(board=board, rgb_pins=rgb_pins)
    await robot.setup()
    try:
        yield robot
    finally:
        await robot.close()
        await client.close()

The Robot class controls the RGB LED by setting the pulse-width modulation (PWM) duty cycle for each GPIO pin configured by the RGB_PINS environment variable. The asyncio.gather method is used to perform each async task concurrently:


class Robot:
    def __init__(self, board: Board, rgb_pins: Tuple[str, str, str]):
        self._board = board
        self._rgb_pins = rgb_pins
        self.logger = getLogger(__name__)

    async def setup(self):
        self._pins = await asyncio.gather(
            # red, green, blue
            *[self._board.gpio_pin_by_name(pin_name) for pin_name in self._rgb_pins]
        )
        await asyncio.gather(*[pin.set_pwm_frequency(3000) for pin in self._pins])

    async def set_color(self, color: Tuple[float, float, float]):
        # (red, green, blue) = color
        await asyncio.gather(*[pin.set_pwm(color[idx]) for idx, pin in enumerate(self._pins)])

    async def blink(self, color: Tuple[float, float, float], duration: float = 5, freq: float = 0.25):
        count = duration / freq
        current = 0
        is_on = True
        while current <= count:
            current += 1
            await self.set_color(color if is_on else (1, 1, 1))
            is_on = not is_on
            await asyncio.sleep(freq)
        await self.set_color((1, 1, 1))

    async def close(self):
        await self.set_color((1, 1, 1))

The webhook request to the application is handled by a ZoomHandler class that uses the shared robot instance from the Request. Controlling the LED is executed as a background task to ensure the response is sent as quickly as possible:

async def zoom(request: Request):
    handler = ZoomHandler(request=request, zoom_token=str(ZOOM_SECRET_TOKEN), zoom_username=ZOOM_USERNAME)
    return await handler.handle()

RED = (0.0, 1.0, 1.0)
GREEN = (1.0, 0.0, 1.0)
BLUE = (1.0, 1.0, 0.0)
PURPLE = (0.0, 1.0, 0.0)

class ZoomHandler:
    _body: Mapping[str, Any] | None = None

    def __init__(self, request: Request, zoom_token: str, zoom_username: str) -> None:
        self._request = request
        self._robot: Robot = request.state.robot
        self._zoom_token = zoom_token
        self._username = zoom_username

    async def handle(self) -> Response:
        body = await self._get_body()
        event: Literal[
            "endpoint.url_validation",
            "meeting.started",
            "meeting.ended",
            "meeting.participant_joined",
            "meeting.participant_left",
        ] = body["event"]

        if event == "endpoint.url_validation":
            return self.validate_endpoint()
        if event == "meeting.participant_joined":
            return await self.welcome_participant()
        if event == "meeting.started":
            return await self.inform_participant()
        if event == "meeting.participant_left":
            return await self.dismiss_participant()
        if event == "meeting.ended":
            return await self.dismiss_meeting()
        return JSONResponse({"message": f"Event type {event} unknown"}, status_code=404)

    def validate_endpoint(self):
        assert self._body is not None

        plain_token = self._body["payload"]["plainToken"].encode()
        zoom_token = self._zoom_token.encode()
        encrypted_token = hmac.new(zoom_token, plain_token, hashlib.sha256).hexdigest()
        return JSONResponse({"plainToken": plain_token.decode(), "encryptedToken": encrypted_token})

    async def welcome_participant(self):
        participant = self._get_participant()
        if participant["user_name"] != self._username:
            return JSONResponse({})

        LOGGER.info(f"Welcome {participant['user_name']}!")
        task = BackgroundTask(self._robot.set_color, color=RED)
        return JSONResponse({}, background=task)

    async def dismiss_participant(self):
        participant = self._get_participant()
        if participant["user_name"] != self._username:
            return JSONResponse({})

        LOGGER.info(f"Goodbye {participant['user_name']}!")
        task = BackgroundTask(self._robot.blink, color=RED)
        return JSONResponse({}, background=task)

    async def inform_participant(self):
        meeting = self._get_meeting()
        LOGGER.info(f"Meeting {meeting['topic']} has started!")
        task = BackgroundTask(self._robot.set_color, color=GREEN)
        return JSONResponse({}, background=task)

    async def dismiss_meeting(self):
        meeting = self._get_meeting()
        LOGGER.info(f"Meeting {meeting['topic']} has ended!")
        task = BackgroundTask(self._robot.blink, color=PURPLE)
        return JSONResponse({}, background=task)

    async def _get_body(self):
        if self._body is None:
            self._body = await self._request.json()

        return self._body

    def _get_participant(self):
        assert self._body is not None

        return self._body["payload"]["object"]["participant"]

    def _get_meeting(self):
        assert self._body is not None

        return self._body["payload"]["object"]

4. Local Testing

To confirm this all works together and start receiving webhook events from Zoom, the webhook endpoint needs to be validated first. To do this without deploying to a live server, use a service like ngrok , zrok , or something else to provide a secure proxy to the local server.

For example, install zrok with Homebrew and start a proxy by running:

brew install zrok
zrok share public localhost:8000

The endpoint URL configured for the webhook should end in /webhooks/zoom. Click the "Validate" button to confirm the app is working as expected:

The hostname (the part before the /webhooks/zoom) should be the proxy URL returned by the secure tunnel tool.

Once the Zoom webhook-only app has been successfully activated, you can start joining some meetings and see the on-air light in action!

Green = A meeting has started
Red = You have joined the meeting
Blinking Red for 5 seconds = You left the meeting
Blinking Purple for 5 seconds = The meeting has just ended

5. Web Application Deployment

Keeping the secure tunnel running all the time is not a viable solution in the long term, so let's take a look at deploying the Python app to a service like Fly.

With the flyctl CLI installed, log in or create an account before continuing. The project repository contains the required fly.toml and Procfile to deploy the application to a Fly Machine.

#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

app = "viam-on-air"
primary_region = "ewr"

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
  processes = ["app"]

[build]
  builder = "paketobuildpacks/builder:base"
  buildpacks = ["gcr.io/paketo-buildpacks/python"]

[env]
  ZOOM_USERNAME="Your Username"
  BOARD_NAME="board"
  RGB_PINS="18,5,19"

[[vm]]
  cpu_kind = "shared"
  cpus = 1
  memory_mb = 1024

Before deploying, you can update the app value in the fly.toml to create a different hostname for the running application, as well as update the [env] section with your environment variables from the .env file. While we could include the .env file in the deployed application image, there are security concerns with storing secrets like the Viam API key and Zoom secret token in plain text on a server that could be compromised. For those secrets, use the expected fly command:

fly secrets set ZOOM_SECRET_TOKEN="" VIAM_API_KEY="" VIAM_API_KEY_ID="" VIAM_ADDRESS=""

The empty "" after each secret name should be filled in with the associated value.

Finally, deploy the application:

fly deploy

After a minute or two, you will get a success message:

Watch your deployment at https://fly.io/apps/viam-on-air/monitoring

-------
Updating existing machines in 'viam-on-air' with rolling strategy

-------
 ✔ Machine 7811110f907408 [app] update succeeded
-------

Visit your newly deployed app at https://viam-on-air.fly.dev/

You can visit the deployed app URL to see a {"message": "Ok"} JSON response. Then update the Zoom app configuration to use that new URL hostname instead of the secure tunnel and re-validate the endpoint.

6. Wrapping Up

Unplug the ESP32 from the computer and power it from the USB battery. Mount all the components in your container. I used a cardboard box with "ON AIR" cut from the front and lined it with printer paper to diffuse the RGB LED shining through.

In practically no time at all, you now have a battery-powered, automated on-air light that you can hang whenever you'd like to signal your current meeting status. If you ever want to monitor the status of the light from your computer or phone, you can view it in the Viam app and learn about extending the functionality of the project.

Share your builds with the community in Discord! Thanks for following along!