Forem: elvis kahoro

Designing a Pure Python Web Framework

elvis kahoro — Wed, 18 Sep 2024 16:33:45 +0000

Web development is one of the most popular use cases for programming. Python is one of the most popular programming languages in the world. So why can't we build web apps in Python?

Making a UI should be simple, but even you have great engineers on your team, the overhead of learning a new language and tools was a huge barrier. Often making a UI could be harder than the actual work one is doing!

TLDR

Under the hood, Reflex apps compile down to a React frontend app and a FastAPI backend app. Only the UI is compiled to Javascript; all the app logic and state management stays in Python and is run on the server. Reflex uses WebSockets to send events from the frontend to the backend, and to send state updates from the backend to the frontend.

Existing Python Solutions

There were a few ways already to build apps in Python, but none of them fit our needs.

On the one hand, there are frameworks like Django and Flask that are great for building production-grade web apps. But they only handle the backend - you still need to use JavaScript and a frontend framework, as well as writing a lot of boilerplate code to connect the frontend and backend.

On the other hand, pure Python libraries like Dash and Streamlit can be great for small projects, but they are limited to a specific use case and don't have the features and performance to build a full web app. As your app grows in features and complexity, you may find yourself hitting the limits of the framework, at which point you either have to limit your idea to fit the framework, or scrap your project and rebuild it using a "real web framework".

We want to bridge this gap by creating a framework that is easy and intuitive to get started with, while remaining flexible and powerful to support any app.

Goals of Reflex

Pure Python: Use one language for everything.
Easy to get started: Build your ideas easily without needing web development experience.
Full flexibility: Web apps should match the customizability and performance of traditional web frameworks.
Batteries included: Handle the full-stack from the frontend, to the backend, to deployment.

Now let's dive into how we built Reflex to meet these goals.

The Reflex Architecture

Full-stack web apps are made up of a frontend and a backend. The frontend is the user interface, and is served as a web page that runs on the user's browser. The backend handles the logic and state management (such as databases and APIs), and is run on a server.

In traditional web development, these are usually two separate apps, and are often written in different frameworks or languages. For example, you may combine a Flask backend with a React frontend. With this approach, you have to maintain two separate apps and end up writing a lot of boilerplate code to connect the frontend and backend.

We want to simplify this process in Reflex by defining both the frontend and backend in a single codebase, while using Python for everything. Developers should only worry about their app's logic and not about the low-level implementation details.

Frontend

We want Reflex apps to look and feel like a traditional web app to the end user, while still being easy to build and maintain for the developer. To do this, we built on top of mature and popular web technologies.

When you reflex run your app, Reflex compiles the frontend down to a single-page Next.js app and serves it on a port (by default 3000) that you can access in your browser.

The frontend's job is to reflect the app's state, and send events to the backend when the user interacts with the UI. No actual logic is run on the frontend.

Components

Reflex frontends are built using components that can be composed together to create complex UIs. Instead of using a templating language that mixes HTML and Python, we just use Python functions to define the UI.

Under the hood, components compile down to React components.

Many of our core components are based on Radix, a popular React component library. We also have many other components for graphing, datatables, and more.

We chose React because it is a popular library with a huge ecosystem. Our goal isn't to recreate the web ecosystem, but to make it accessible to Python developers.

This also lets our users bring their own components if we don't have a component they need. Users can wrap their own React components and then publish them for others to use. Over time we will build out our third party component ecosystem so that users can easily find and use components that others have built.

Styling

We wanted to make sure Reflex apps look good out of the box, while still giving developers full control over the appearance of their app.

We have a core theming system that lets you set high level styling options such as dark mode and accent color throughout your app to give it a unified look and feel.

Beyond this, Reflex components can be styled using the full power of CSS. We leverage the Emotion library to allow "CSS-in-Python" styling, so you can pass any CSS prop as a keyword argument to a component. This includes responsive props by passing a list of values.

Backend

In Reflex only the frontend compiles to Javascript and runs on the user's browser, while all the state and logic stays in Python and is run on the server. When you reflex run, we start a FastAPI server (by default on port 8000) that the frontend connects to through a websocket.

All the state and logic are defined within a State class.

The state is made up of vars and event handlers.

Vars are any values in your app that can change over time. They are defined as class attributes on your State class, and may be any Python type that can be serialized to JSON.

Event handlers are methods in your State class that are called when the user interacts with the UI. They are the only way that we can modify the vars in Reflex, and can be called in response to user actions, such as clicking a button or typing in a text box.

Since event handlers are run on the backend, you can use any Python library within them.

Event Processing

Normally when writing web apps, you have to write a lot of boilerplate code to connect the frontend and backend. With Reflex, you don't have to worry about that - we handle the communication between the frontend and backend for you. Developers just have to write their event handler logic, and when the vars are updated the UI is automatically updated.

Event Triggers

The user can interact with the UI in many ways, such as clicking a button, typing in a text box, or hovering over an element. In Reflex, we call these event triggers.

Event Queue

On the frontend, we maintain an event queue of all pending events. An event consists of three major pieces of data:

client token: Each client (browser tab) has a unique token to identify it. This let's the backend know which state to update.
event handler: The event handler to run on the state.
arguments: The arguments to pass to the event handler.

When an event is triggered, it is added to the queue.
We have a processing flag to make sure only one event is processed at a time. This ensures that the state is always consistent and there aren't any race conditions with two event handlers modifying the state at the same time. There are exceptions to this, such as background events which allow you to run events in the background without blocking the UI.

Once the event is ready to be processed, it is sent to the backend through a WebSocket connection.

State Manager

Once the event is received, it is processed on the backend.

Reflex uses a state manager which maintains a mapping between client tokens and their state. By default, the state manager is just an in-memory dictionary, but it can be extended to use a database or cache. In production we use Redis as our state manager.

Event Handling

Once we have the user's state, the next step is to run the event handler with the arguments.

State Updates

Every time an event handler returns (or yields), we save the state in the state manager and send the state updates to the frontend to update the UI.

To maintain performance as your state grows, internally Reflex keeps track of vars that were updated during the event handler (dirty vars).
When the event handler is done processing, we find all the dirty vars and create a state update to send to the frontend.

We store the new state in our state manager, and then send the state update to the frontend.
The frontend then updates the UI to reflect the new state.

Conclusion

I hope this provides a good overview of how Reflex works under the hood. We will have more posts coming out to share how we made Reflex scalable and performant through features such as state sharding and compiler optimizations.

How we designed themes for the terminal - a peek into our process

elvis kahoro — Thu, 17 Mar 2022 05:16:23 +0000

It’s no secret that people love to customize their UIs. With the rampant adoption of light mode and dark mode and the long history of themes in terminals and IDEs, the power to make your UI fit your functionality or aesthetic needs is huge. Here at Warp we knew how important it is to create a great theming system for our users. Some key goals were to ensure compatibility with existing themes, easy customization and sharing.

Let’s start with how themes work in existing terminals and what we wanted to improve.

Themes only control the text display, yet the rest of the UI remains the same. We want to deliver a more cohesive and immersive experience.
Themes are hard to control, you have to turn a ton of individual “knobs” in order to get something that looks good. It’s not easy to get what you want and add your personal preference. We want to make customization simple, flexible and quick.
Themes are hard to share. Importing and exporting themes are different and can require several steps. We want to make it a one click simple experience to import and share.

The foundation

To ensure warp could be compatible with all other existing themes we started with the standard foundation of 16 ANSI colors. Using these standards as our foundation allows practically every terminal theme to be used in warp.

We wanted to take it one step further and have the theme extend to the whole UI. Since we custom built Warp’s UI this allows us to have full control of the look and feel.

The first step of that was to consider the UI accent areas like the tab indicator and block selection. We want these areas to pop and gain your attention while also adding some visual flair. We needed to add another color attribute which we naturally called the accent color.

Being able to change the accent color gave themes a wider range of customization just from one color change without changing the look and functionality of the core theme.

UI theming

There is a bit more UI to cover within Warp like contextual menus, auto suggestion overlays, dialogs, command palette. The text coloring could easily be inherited from the theme and the accent color used for the same conditions, though we wanted the UI surface to stand out from the background. The last thing we wanted to do is to add yet another color attribute.

Starting with dark and light themes we had to handle the UI layering in different ways. To achieve separation from the background for dark themes, we added a white overlay which aligns with the core text color, and for light themes the opposite, black overlay which also aligns with the core text color.

We created a consistent style called “UI surface” which would be the background of all the overlay UI elements.

It consists of the theme background color, the opposite overlay color and an outline.

This basic system allows for a different UI surface leveraging the same color system. Seen here in dark and light mode.

The really neat part is how this easily extends to existing themes like Dracula and Solarized. For existing themes we convert, we will choose an accent color from the core 16 colors that we feel match the best. If you want to change it, no sweat, you can customize the accent color to any of the 16 theme colors or any custom color you like to add your personal preference to existing themes.

Image backgrounds

So with the standard theme color system and an added accent color attribute, we can easily customize the whole Warp UI. Yet we didn’t want to stop there, what about adding photos and advanced color treatment?

We experimented with adding photo backgrounds with color themes that matched and loved the results.

We also added the ability to support gradients, which can drastically change the look of a theme and add additional depth and visual pizzaz.

Dracula regular:

Dracula with gradients:

Customization

We want our user to have the power to make amazing themes, so we are working on a theme editor that makes customizing themes super simple. For lightweight use, you can start with our default dark or light theme and change the accent color. Choose one of many popular themes we have built into warp or import your favorite. If you want to go for even more customization, add a photo, make your accent color a gradient. With just a few customization you can radically change the way your terminal looks and feels.

Sharing

Currently, there are ton of different sites and resources to get terminal themes, each of them with totally different ways to download themes, from a file download, github repo, etc.

For Warp themes, we want to build a one stop shop for themes. One spot where you can browse, install and share themes all with one click. We also want individual themes to be easily shareable from user to user within the app.

Check out themes now

It’s been really exciting working on themes in Warp. Building a system that can accommodate existing themes, expanding on them within Warp and giving users more control to customize and share. We can’t wait to see what themes get created.

You can check out the base theme experience in Warp now, where you can choose from a set of themes and customize via an external file. Shortly gradients and image support will be rolling out. Customization and sharing are down the road.

We are all super excited to get all these theming features out, we have a bit of work ahead of us! Please let us know if there are any ideas that brew up, we would love your input!

Request early access to our beta. We look forward to your input!