Forem: Trevor Lee

PyTorch Introductory Experiments

Trevor Lee — Sat, 02 Aug 2025 09:34:10 +0000

PyTorch Introductory Experiments

In this project, I will revisit some of my previous basic TensorFlow Deep Learning experiments / training exercises

Nevertheless this time, those experiments will be retried (reimplemented)

The experiments will be using PyTorch as the deep learning framework (and still be with "dense" layers only)
They will not be targeted for microcontroller; but still be demonstrable, with the help of DumbDisplay using regular Python and PyTorch

Installation with VSCode

This project is developed with VSCode (with Python extension); here, I will assume VSCode development environment as well.

To try along, please clone this project -- PyTorchIntroductoryExperiments -- from GitHub to your local machine, and open the folder with VSCode.

Create a virtual environment by selecting the command Python: Select Interpreter from the command palette and choose to create a new virtual environment.

In the process, you will be given an option to also install the required dependent packages specified in requirements.txt file.
In case you missed installing those dependent package, you still can install them, including MicroPython DumbDisplay Library, with an opened terminal (virtual environment activated) by running pip
like

pip install -r requirements.txt

I have tested the Python code of the project to work with Python 3.12.8

For older version of Python, when installing the packages, you might see error like

ModuleNotFoundError: No module named 'setuptools.config.expand'; 'setuptools.config' is not a package

try upgrade pip like

python -m pip install --upgrade pip

then run

pip install -r requirements.txt

again.

When open the Jupyter Notebook, the needed components will be installed when necessary.

"Hello World" Deep Learning Model Training

The "Hello World" of Deep Learning here refers to the training of a DL model for the sine Mathematical function -- train_sine.ipynb.

I guess modeling the Mathematical function sine is considered the "Hello World" of Deep Learning because
1) The training data is very easy to generate.
2) The architecture of the model can simply be just a few "dense" layers.
3) When it comes to implementation, the Mathematical function sine might not be as trival as it sounds, and therefore a good demonstration of the Deep Learning "magic".

Moreover, just for fun, I extended the architecture of the sine model to output cosine values at the same time -- train_sine_cosine.ipynb.

Highlights of `train_sine.ipynb`

The target of this "Hello World" model is the sine Mathematic function for the input range from 0 to 2π (0° to 360°).

Hence, to create the data for the training the model, can generate randomized data like

x_values = np.random.uniform(low=0, high=2*math.pi, size=SAMPLES)
np.random.shuffle(x_values)
y_values = np.sin(x_values)

Since the default datatype for PyTorch is float32, first convert the data to float32, and reshape them as

x_values = x_values.astype(np.float32).reshape(-1, 1)
y_values = y_values.astype(np.float32).reshape(-1, 1)

Notes:

x_values is a two-dimensional matrix, with a single column, as the input to the model is a single value (the input angle in radian)
y_values is also a two-dimensional matrix, with a single column, as the output of the model is also a single value (the sine value of the input angle)

Then, 70% of the data will be treated as "train" data set, and the rest is treated as "test" data set

train_split = int(0.7 * SAMPLES)

x_train, x_test = np.split(x_values, [train_split])
y_train, y_test = np.split(y_values, [train_split])

Now, turn the above data into something suitable for the PyTorch DL framework.

First define the dataset class to capture the input data

class SineDataset(Dataset):
    def __init__(self, x_values, y_values):
        self.x_values = x_values
        self.y_values = y_values

    def __len__(self):
        return len(self.x_values)

    def __getitem__(self, idx):
        return (self.x_values[idx], self.y_values[idx])

__len__ -- the size of the data
__getitem__ -- given an index of the data, returns the (input, label) pair

With SineDataset, instantiate data loaders (DataLoader) for the "train" and "test" data sets

train_dataset = SineDataset(x_train, y_train)
test_dataset = SineDataset(x_test, y_test)

train_dataloader = DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True)
test_dataloader = DataLoader(test_dataset, batch_size=BATCH_SIZE)

Here, BATCH_SIZE is the size of each training batch for each training epoch.
For the "train" data set, it is better to shuffle it.

As the core, define the model class

class SineModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(1, 16),
            nn.ReLU(),
            nn.Linear(16, 16),
            nn.ReLU(),
            nn.Linear(16, 1)
        )

    def forward(self, x):
        return self.linear_relu_stack(x)

The model basically consist of:
- an input "dense" layer -- a single input (the input angle in radian); 16 output, with relu activation
- another "dense" layer -- 16 input; 16 output, with relu activation
- an output "dense" layer -- 16 input; a single output (the sine value of the input angle)
During training / inference, the method forward will be called. Here, it simply calls the stack of layers to do the forward pass
- x is the input -- from the dataset SineDataset during training; from inference input during inference

It is equally important to define the training and testing functions, which involves
the "loss function" and the "optimizer". Later on, the "loss function" and "optimizer" be selected from the standard ones that come with the PyTorch framework.

For training:

def train(device, dataloader, model, loss_fn, optimizer) -> float:
    num_batches = len(dataloader)
    model.train()
    train_loss = 0
    for batch, (X, y) in enumerate(dataloader):
        X, y = X.to(device), y.to(device)

        # Compute prediction error
        pred = model(X)
        loss = loss_fn(pred, y)
        train_loss += loss.item()

        # Backpropagation
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()

    train_loss /= num_batches
    return train_loss

The function train will be called during training (as will be apparent in later traning loop)
During training, "grad info" will be accumulated, as model.train() is called.
For each batch:
- prediction is made with the model -- pred = model(X)
- loss is calculated -- loss = loss_fn(pred, y)
- back-propagation involves:
- loss.backward()
- optimizer.step()
- optimizer.zero_grad() -- reset the "grad info", for the next batch

For testing:

def test(device, dataloader, model, loss_fn) -> float:
    num_batches = len(dataloader)
    model.eval()
    test_loss = 0
    with torch.no_grad():
        for X, y in dataloader:
            X, y = X.to(device), y.to(device)
            pred = model(X)
            test_loss += loss_fn(pred, y).item()
    test_loss /= num_batches
    return test_loss

The function test will be called during training as well, but for testing, the model will not accumulate grad info, as torch.no_grad() is called

At last, the actual training loop can be implemented as

EPOCH_COUNT = 600

loss_fn = nn.MSELoss()
optimizer = torch.optim.RMSprop(model.parameters(), lr=0.001)

for t in range(EPOCH_COUNT):
    train_loss = train(device, train_dataloader, model, loss_fn, optimizer)
    test_loss = test(device, test_dataloader, model, loss_fn)
    print(f"$ Epoch {t+1} / {EPOCH_COUNT} -- Train loss: {train_loss:.6f} / Test loss: {test_loss:.6f}")

EPOCH_COUNT -- the number of epoch for the training
The "loss function" selection is MSELoss
The "optimizer" selection is RMSprop
In lock steps, train() and test() are called for each epoch iteration.
- all batches of the "train" dataset are walked through in train(), with back-propagation to alter the model's parameters
- all batches of the "test" dataset are walked through just to find out the average of loss of the "test" dataset

To evaluate the trained model, can run the test data again like:

x_values = [math.pi * (5 * a) / 180.0 for a in range(0, 73)]
x_values = np.array(x_values).reshape(-1, 1)
x_values = torch.tensor(x_values, device=device, dtype=torch.float32)
model.eval()
pred_y_values = model(x_values).tolist()

Highlights of `train_sine_cosine.ipynb`

As previous "Hello World" model, this extended model (sine + cosine) will also have input range from 0 to 2π (0° to 360°).

Hence, to create the data for training the model, need to randomize to get both -- y_sin_values and y_cos_values

x_values = np.random.uniform(low=0, high=2*math.pi, size=SAMPLES)
np.random.shuffle(x_values)
y_sin_values = np.sin(x_values)
y_cos_values = np.cos(x_values)

Again, since the default datatype for PyTorch is float32, convert the data to float32, and reshaped, as

x_values = x_values.astype(np.float32).reshape(-1, 1)         
y_sin_values = y_sin_values.astype(np.float32).reshape(-1, 1) 
y_cos_values = y_cos_values.astype(np.float32).reshape(-1, 1)

Notes:

The input data x_values is the same as the previous model -- a two-dimensional array with a single column
Both of the output data y_sin_values and y_cos_values are also two-dimensional arrays with a single column -- the sine and cosine values respectively

As there are two sets of output values, it follows that there will also be two sets of values for the train and test data

x_train, x_test = np.split(x_values, [train_split])
y_sin_train, y_sin_test = np.split(y_sin_values, [train_split])
y_cos_train, y_cos_test = np.split(y_cos_values, [train_split])

And the two sets of output data are stacked together for the model like

y_train = np.stack((y_sin_train, y_cos_train), axis=-1).reshape(-1, 2)
y_test = np.stack((y_sin_test, y_cos_test), axis=-1).reshape(-1, 2)

Notes:

y_train is two-dimensional with two columns -- 1st column is for sine; 2nd column is for cosine
hence, y_train[0] are the sine and cosine values of the first input angle for the "train" data
similarly, y_test has the same shape as y_train, but for the "test" data

In other words, for the training, there are still two output values -- train output values y_train and test output values y_test

To capture the data set, the following classes -- SineCosineDataset and SineCosineModel -- are defined

class SineCosineDataset(Dataset):
    def __init__(self, x_values, y_values):
        self.x_values = x_values
        self.y_values = y_values

    def __len__(self):
        return len(self.x_values)

    def __getitem__(self, idx):
        return self.x_values[idx], self.y_values[idx]

class SineCosineModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(1, 16),
            nn.ReLU(),
            nn.Linear(16, 16),
            nn.ReLU(),
            nn.Linear(16, 2)
        )

    def forward(self, x):
        return self.linear_relu_stack(x)

Notice that there are 2 output values as the last layer of SineCosineModel -- one for sine and one for cosine

The rest of the code for training the model is basically the same as the sine one.

To evaluate the trained model, can run the test data again like:

x_values = [math.pi * (5 * a) / 180.0 for a in range(0, 73)]
x_values = np.array(x_values).reshape(-1, 1)
x_values = torch.tensor(x_values, device=device, dtype=torch.float32)
model.eval()
pred_y_values = model(x_values).tolist()

Mnist Dataset DL Training and Demo App

The popular Mnist dataset is also frequently used as introductory demonstration to Deep Learning -- train_mnist.ipynb.

In this project, not only will a simple DL training of Mnist dataset be presented, demonstration UI will be realized wireless on your Android mobile phone with the help of DumbDisplay -- start_dd_mnist.py

UI is coded with (and driven by) Python using MicroPython DumbDisplay Library package; and is realized wirelessly on your Android mobile phone with DumbDisplay Android App

Nowadays, code generation with AI is a common practice. Indeed, I did prompt LLM to generate a Jupyter Notebook to train a PyTorch DL model for the Mnist dataset -- train_mnist_ai.ipynb.

The AI-generated DL model is more complex (and certainly more standard) than the one presented in train_mnist.ipynb. At least, mine basically only involves "dense" layers, while the AI-generated one involves "convolutional" layers.

The actual model architecture (the simple version) is captured by the class MnistModel defined in the file model_mnist.py

class MnistModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(784, 64),
            nn.ReLU(),
            nn.Dropout(0.3),
            nn.Linear(64, 64),
            nn.ReLU(),
            nn.Linear(64, 10)
        )

    def forward(self, x):
        logits = self.linear_relu_stack(x)
        return F.log_softmax(logits, dim=1)

Please refer to train_mnist.ipynb for the complete training exercise.

Demo UI for the Trained Mnist DL Model

The Python script start_dd_mnist.py starts the DumbDisplay program which uses MicroPython DumbDisplay Library to drive a wireless UI on your Android mobile phone with DumbDisplay Android App.

After starting the Python script start_dd_mnist.py, you should see from VSCode terminal that it waits for connection from DumbDisplay Android App.

**********
*****
*** reading model output/mnist_model.pth ...
*****
**********



**********
*****
*** starting mnist with model output/mnist_model.pth ...
*****
**********

connecting socket ... listing on 192.168.0.46:10201 ...

You can open the DumbDisplay Android App installed in your Android phone -- DumbDisplay Android App -- and make connection like

When connected, the VSCode should show something like

... connected 192.168.0.46:10201 from ('192.168.0.98', 41578)


Once connected, on the [black] canvas of the UI, draw a dight you want the DL model to recognize, like the digit `8`, and press the `>>>` button to trigger inference of the drawn digit data (stored in the memory of the running Python process)

The inference with PyTorch is actually performed by the following Python function mnist_inference

def mnist_inference(inference_data, model) -> int:
    try:
        x = np.array(inference_data).reshape((1, 784))
        torch_x = torch.tensor(x, dtype=torch.float32).reshape(1, 784)
        output = model(torch_x)
        pred = output.argmax()
        return pred.item()
    except Exception as e:
        print(f"XXX error during inference: {e}")
        raise e

Note that mnist_inference is just a callback function for the DumbDisplay "driver" Python code implemented as an example of MicroPython DumbDisplay Library

Now, draw the digit 9 on the canvas ...

If you want to clear what have drawn, press the clear button.

The center button toggles whether the drawn digit will be auto centered before calling mnist_inference for inference.

It is interesting to see that even without auto-centering, the digit recognition is pretty good, especially with the AI generated model -- start_dd_mnist_ai.py.

Sliding Puzzle DL Training and Demo App

The DL model presented in this project for the classical Sliding Puzzle game is a simple and naive "next move" suggesting model -- train_sliding_puzzle.ipynb

I came up with this simple and naive "next move" suggesting model by referencing to the above-mentioned "Hello World" and Mnist DL models, just for fun.

Say, for a 4x4 board.

There will be 16 tiles, with the 0th tile being the empty space; e.g. the board "orientation" can be represented as

  |  0 |  1 |  2 |  3 |
  |  4 |  5 |  6 |  7 |
  |  8 |  9 | 10 | 11 |
  | 12 | 13 | 14 | 15 |

With respect to the empty space (i.e. the 0th tile), there can be 4 possible moves (but some might be invalid)
- 0: from left
- 1: from right
- 2: from top
- 3: from bottom
For example, the above solved board can be randomized by a single step of the move 1 (from right) to

  |  1 |  0 |  2 |  3 |
  |  4 |  5 |  6 |  7 |
  |  8 |  9 | 10 | 11 |
  | 12 | 13 | 14 | 15 |

The "next move" of this randomized board is apparently an "undo" move to undo the randomization step, in this case, the move 0 (from left)
- 1 => undo with 0
- 0 => undo with 1
- 2 => undo with 3
- 3 => undo with 2
In other words, the "undo" moves are the "next moves" toward solving a randomized board
The way to capture the "undo" moves can be as easy as
- during a randomization step, a valid "move" is select
- the "undo" move is recorded as the "next move" for the randomized board "orientation"
- so, for a board randomized by 5 steps, there will be 5 "next moves" recorded -- one for each board "orientation"
The board "orientations" is the input to the DL model. The "undo" moves is the output of the DL model.

The actual model architecture is captured by the class SlidingPuzzleModel defined in the file model_sliding_puzzle.py

class SlidingPuzzleModel(nn.Module):
    def __init__(self, tile_count):
        super().__init__()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(tile_count * tile_count, 256),
            nn.ReLU(),
            nn.Dropout(0.3),
            nn.Linear(256, 256),
            nn.ReLU(),
            nn.Linear(256, 4)
        )

    def forward(self, x):
        logits = self.linear_relu_stack(x)
        return F.softmax(logits, dim=1)

Please refer to train_sliding_puzzle.ipynb for the complete training exercise, which is a bit more involving, mostly because generation of the training data.

Demo UI for the Trained Sliding Puzzle Model

The demo UI Python script this time is start_dd_sliding_puzzle.py, which is mostly based on the example of MicroPython DumbDisplay Library

After starting the Python script start_dd_sliding_puzzle.py, you should see from VSCode terminal that it waits for connections from the DumbDisplay Android App.

**********
*****
*** reading model output/sp_model_4.pth ...
*****
**********



**********
*****
*** starting sliding puzzle with model output/sp_model_4.pth ...
*****
**********

connecting socket ... listing on 192.168.0.46:10201 ...

You can open the DumbDisplay Android App and make connection like previously.


Once connect, double-press the sliding puzzle game board to randomize it by 5 steps. You can try to solve the puzzle manually by moving / sliding the appropriate tile to the empty space. If stuck, press the `suggest` button to have the DL model suggest the "next move" for you.

When "next move" is needed, the following Python function suggest_next_move will be called

def suggest_next_move(board_manager: BoardManager, model, tile_count) -> int:
    x = torch.tensor([board_manager.board_tiles], dtype=torch.float32) / (tile_count * tile_count)
    prediction = model(x)
    move_ans = prediction.argmax(dim=1).item()
    return move_ans

Messed up or not, double-press the 🔄 Reset 🔄 button to reset the game; but this time, press the continuous button to have "next move" suggested [and made] continuously.

If things go well, the game should be solved in 5 suggested "next moves".

Once solved, double-press the board to randomize it again; but this time, it will be randomized by 10 steps (5 more than last time).

Once solved again, double-press the board again ...

It is interesting to see how randomized the board is, the DL model can still suggest the correct "next moves" to solve it.
My experience is, around 15 to 20 randomize steps.

If you are interested, try tuning the model to see if it can achieve more randomize steps!

Two Simple DumbDisplay Samples

In the Python script test_run_dd.py are two simple DumbDisplay samples

`run_blink`	`run_graphical`

Hopefully, these two samples can kick start those interested friends to implement Android apps using Python with DumbDisplay,
for purposes like that of this project.

Have Fun!

A Weather Clock (with Alarms) for ESP32 / Raspberry Pi Pico Implemented with Arduino Framework

Trevor Lee — Sat, 10 May 2025 03:47:25 +0000

Arduino Weather Clock -- `AWeatherClock` -- v1.1

This little microcontroller project AWeatherClock (Arduino Weather Clock) was inspired by pyClock, and is implemented using the Arduino framework.

VSCode with PlatformIO extension is the primarily development environment for the project, in the similar fashion as described by the post -- A Way to Run Arduino Sketch With VSCode PlatformIO Directly


Naturally, the hardware used for the initial development of the project was exactly the one mention in the `pyClock` GitHub repository.

Nevertheless, the sketch of AWeatherClock should be adaptable to other hardware configurations, with a big enough colored TFT LCD screen.

The functions of AWeatherClock are:

Display of a digital clock synchronized with NTP
Display of the current weather info gathered from OpenWeather with version 2.5 APIs [for free account]
Alarms that can be repeated daily, with selectable days [of the week] for the repeat. Note that if hardware permits, alarm sound (beep / melody / music) can be produced when alarm is due.
Idle slideshow of photos (JPEG images) uploaded to the MCU, much like the slideshow function as described by Simple Arduino Framework Raspberry Pi Pico / ESP32 TFT LCD Photo Frame Implementation With Photos Downloaded From the Internet Via DumbDisplay

AWeatherClock requires hardware with the following capabilities:

ESP32 line of MCU that supports WiFi, or Raspberry Pi PicoW
240x240 (or bigger) colored TFT LCD screen
Able to provide with simple "trigger" input, like with
- BUTTON -- note that the "boot" button for ESP32 can be used as BUTTON here
- TOUCH SCREEN
Optionally -- buzzer, speaker or audio module (like ES8311 for ESP32) -- for sounding of alarms.

Since AWeatherClock needs to gather current weather info from OpenWeather,
please, sign up for an APP_ID of their version 2.5 APIs.
You will need this APP_ID in order to setup for AWeatherClock -- please refer to the section Basic Hardcoded Configurations

Also, DumbDisplay Android app
for Android phone (or Chrome OS / Android simulator etc) is an essential part of AWeatherClock since most settings -- including alarm setups and slideshow photos upload -- can be done with remote UI realized on your Android phone with the help of DumbDisplay Android app

Photos to upload can be acquired in the following ways:
- randomly from the Internet
- picsum.photos
- loremflickr.com
- placedog.net
- randomly from unsplash.com
- for downloading photos from unsplash.com, you will need to sign up and create a demo app for an Access Key
- pick from your phone
- take with your phone's camera
Location for weather info
- according to hardcode query string (DEF_OPEN_WEATHER_API_LOCATION defined in config.h), or
- based on GPS location of your phone queried via DumbDisplay Android app
NTP timezone
- the initial timezone of your MCU is hardcoded to the macro INIT_TIMEZONE defined in config.h
- after weather info gathering, timezone of your MCU is set to the timezone returned with the weather info
When connected to DumbDisplay Android app, AWeatherClock is considered not idle, and hence slideshow will not start

Notice that the remote UI on your Android phone is driven by the sketch -- i.e. the control flow of the UI is programmed in the sketch -- hence, other than the DumbDisplay Android app, there is no specific mobile app for AWeatherClock.

You may want to refer to Blink Test With Virtual Display, DumbDisplay
for a bit more details about DumbDisplay Arduino library and DumbDisplay Android app.

Concerning "sounding" of alarm. Actually AWeatherClock not only will flash the screen when alarm is due,
if buzzer / speaker / audio module (ES8311 for ESP32) is installed, audible alarm-specific sound will be generated.
Please refer to Alarm Sound for more details.

Out-of-the-Box Supported Hardware

The sketch arduino_weather_clock.ino of this project is tailored for various compatible hardware that I can get hold of:

PlatformIO env PYCLOCK -- ESP32 C3 pyClock
- it's button on the back is configured as a BUTTON for "trigger" input
PlatformIO env AST_WATCH -- ESP32 C3 Astronaut-Clock-Watch (touch)
- it's touch screen is configured as a TOUCH SCREEN for "trigger" input
- it has a buzzer, which is used to generate alarm sound
PlatformIO env TW3 -- ESP32 LILYGO_WATCH_2020_V3
- it's touch screen is configured as a TOUCH SCREEN for "trigger" input
PlatformIO env ESP_SPARKBOT -- ESP32 S3 ESP-SparkBot
- it's right side is touchable, which is configured as a BUTTON for "trigger" input, but from my experience, this "touch-pin" mechanism doesn't work very well 😞
- it also has the ES8311 audio codec module, which is used to generate alarm sound
PlatformIO env MUMA (touch) / MUMA_NT (non-touch) -- ESP32 S3 Little MuMa
- for touch version (MUMA), it's touch screen is configured as a TOUCH SCREEN for "trigger" input
- for non-touch version (MUMA_NT), it's "boot" button on the back right side is configured as a BUTTON for "trigger" input
- it also has the ES8311 audio codec module, which is used to generate alarm sound
Platformio env PICOW_GP -- a Raspberry Pi PicoW attached to a gamepad-like LCD
- the gamepad's "start" button is configured as a BUTTON for "trigger" input
PlatformIO env PICOW -- the Raspberry Pi PicoW wiring as mentioned in Simple Arduino Framework Raspberry Pi Pico / ESP32 TFT LCD Photo Frame Implementation With Photos Downloaded From the Internet Via DumbDisplay
- a ST7789 2.8 inch 240x320 SPI TFT LCD module is used, with a FT6336U capacitive touch layer
- it's touch screen is configured as a TOUCH SCREEN for "trigger" input
- a speaker is attached to it; the speaker is used to generate alarm sound
- for more details, please refer to the later section Customizations for New Hardware PicoW Example
PlatformIO env ESP32 -- an ESP32 customized hardware
- a ST7789V 1.3 inch 240x240 SPI TFT LCD module (ST7789) is used
- a button is attached to it, as BUTTON for "trigger" input
- a speaker is attached to it; the speaker is used to generate alarm sound
- for more details, please refer to the later section Customizations for New Hardware ESP32 Example

How many slides can be stored in the MCU?

E.g., for PYCLOCK which is an ESP32-C3 with 4M of flash memory, I can store 25 to 30 photos to the flash of the MCU; note that for PYCLOCK, it is configured to use no_ota.csv partitions in platformio.ini like

  board_build.partitions = no_ota.csv

E.g., for Raspberry Pi Pico, I can also store 25 to 30 photos to the flash of the MCU as well; note that for Raspberry Pi Pico, littlefs is configured in platformio.ini like

  board_build.filesystem = littlefs
  board_build.filesystem_size = 1m

Additional notes:

For TWatch (TW3): If when compile you see some "include font" error, it might be that the project's folder path is too long. In such a case, try move the project to somewhere closer to the "root" of your filesystem

Showing of IP and Connecting DumbDisplay Android App

With BUTTON or TOUCH SCREEN, you can trigger AWeatherClock to show the IP of your MCU to connect to with Android DumbDisplay app


Be the "trigger" mechanism `BUTTON` or `TOUCH SCREEN`, you "double click" to trigger showing of IP


You will need this IP to make connection between `AWeatherClock` and your Android phone's Arduino DumbDisplay app for the remote UI, as will be described in more details next


BTW. You might see that DumbDisplay Android app requires "LOCATION" permission to make use of your phone's GPS service. You can grant the permission with the `Settings` menu option of DumbDisplay Android app

Settings UI

As mentioned previously, most of the AWeatherClock settings can be modified with the UI remotely rendered on your Android phone with DumbDisplay Android app

Settings UI -- General


With the `General` tab, you can modify the general settings / options

🌤️ -- you click the 🌤️ button to [manually] trigger refresh of the current weather info
12 Hour / 24 Hour -- you select whether the digital clock display should be in 12-hour or 24-hour format
📡 -- you select whether to sync weather location with the GPS location of your phone
Slide Show Idle -- you select the idle time (in minutes) before starting slideshow; to disable idle slideshow, select 🚫
Slide Duration -- you select the duration (in seconds) each slide should be kept shown, before switching to another one
Update Weather -- you select the gap (in minutes) between each auto update of the current weather info

Settings UI -- Alarms


With the `Alarms` tab, you can set up the alarms of `AWeatherClock`

The 🕰️ icon next to an alarm indicates that the alarm is ON; below that icon, the time of the alarm is shown, like 00:00
You can select any one of the alarms to edit. The details of the alarm being edited are shown on the right-side
- you can turn the alarm ON / OFF by selecting the ⏰ button
- you select the 🔄 button to set that the alarm is to be repeated daily; and the week days for the repeat are indicated by the Su / Mo / Tu / We / Th / Fr / Sa below the 🔄 button
- The time of the alarm is clearly shown further down below. You can press the 🔼 / 🔽 to have the alarm hour / minute changed. Alternatively, you can double-press on the hour / minute to have a pop-up dialog for you to enter the hour / minute of the alarm. Note that if you enter a value bigger than 99, like 1230, it will be interpreted as the time 12:30; if you want to enter the time, say, 00:01, enter 2401

Settings UI -- Slides

With the Slides tab, you can add / remove photos for the idle slideshow

⬅️ / ➡️ -- you use the ⬅️ / ➡️ buttons to review the slideshow photos
- you select the photo to be deleted from the slideshow
- newly uploaded photo can be saved to the slideshow, positioned after the photo selected
💾 / 🗑️ -- you delete the photo shown by double-pressing 🗑️; the 💾 is for you to add the uploaded photo to the slideshow
Acquire photo to upload:
- 🌍 / 💦 -- you trigger download of a random photo from the Internet by pressing 🌍; 💦 is specifically for downloading a random photo from Unsplash
- 📱 / 📷 -- you pick a photo from your phone by pressing 📱; you take a photo with your phone's camera by pressing 📷
- In any case, the uploaded photo will be considered "unsaved"; you press 💾 to add the photo to the slideshow
- Also notice for photo that might not fit AWeatherClock screen, a crop UI will be invoked for you to crop the photo in order to fit the screen

Alarm Sound

As mentioned previously, in addition to flashing of the screen, in case of audible alarm sound can be generated with buzzer / speaker / audio module (ES8311 for ESP32), alarm-specific selection will be available

Beep sound -- with buzzer / speaker / audio module
Melody Amazing Grace (for ESP32) -- the same melody encoding as mentioned in the YouTube video Raspberry Pi Pico playing song melody tones, with DumbDisplay control and keyboard input
and in the post Respberry Pi Pico W Generating Tones With Programmable I/O (PIO) Using MicroPython
Melody Birthday Song (for ESP32) -- similar to above melody Amazing Grace, but musical note encoding was generated with LLM
happy_birthday_melody.h

  I asked LLM to generate this file's "happy birthday" melody.

  Prompt:

  At the end are the data structures for the melody "amazing grace". Can you figure out the data for the melody "happy birthday"?

  IMPORTANT notes:
  * Each musical note MUST be composed of exactly two characters, no more and no less.
  * Hence, the data "nodenames" / "octaves" / "beats" for each musical note MUST be composed with two characters, and therefore might be padded with space " "
  * For "nodenames" -- e.g "C " for C; "C#" for C sharp; and "Cb" for C flat
  * For "octaves" -- e.g. "0 " for octave 0; "1 " for octave 1; "2 " for octave 2; note that it can be negative like "-1" (still TWO chars), for lower octaves

  -----------------

  const char* amazing_grace_nodenames = "G C E C E D C A G G C E C E D G E G E G E C G A C C A G G C E C E D C ";
  const char* amazing_grace_octaves   = "0 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 0 0 0 1 1 1 1 1 1 ";
  const char* amazing_grace_beats     = "2 4 1 1 4 2 4 2 4 2 4 1 1 4 2 8 2 1 1 1 1 4 2 4 1 1 1 4 2 4 1 1 4 2 8 ";

  const int amazing_grace_beatSpeed = 300;

Music [sample] Star Wars (for ESP32 with audio module ES8311) -- music played with Arduino Audio Tools, with data file star_wars_music.h, which is actually the same data file used by the streams-memory_raw-i2s example

Note that this per-alarm sound selection is only available for ESP32 line of MCU. Moreover, if your hardware has the audio module ES8311, you will have more
| | |
|--|--|
|Again, with the Alarms tab, you can select the per-alarm sound to alert you when the alarm is due|

|
Notes:

The sound choice Star Wars is only available for the audio module ES8311
Additionally, slider next to 🔊 allows you to change the ES8311 audio module volume

Basic Hardcoded Configurations -- `config.h`

config.h for secrets like WIFI_SSID, WIFI_PASSWORD and OPEN_WEATHER_MAP_APP_ID

...
// #####
// # you will need to either
// # . enable and complete the following "secret" block
// # . or create and complete the "secret" block in the file `_secret.h`
// #####
#if false

  // ----------------------
  // !!! "secret" block !!!
  // ----------------------
  //
  // *****
  // * you can setup WIFI_SSID / WIFI_PASSWORD; for ESP32, if WIFI_SSID not defined, will use WiFiManager to get WiFi SSID and password
  // * you will need to setup OPEN_WEATHER_MAP_APP_ID
  // * you can optionally setup UNSPLASH_CLIENT_ID
  // *****

  #define WIFI_SSID               "<wifi ssid>"
  #define WIFI_PASSWORD           "<wifi password>"

  // you MUST get APP_ID from https://home.openweathermap.org/users/sign_up
  #define OPEN_WEATHER_MAP_APP_ID "<app id>"

  // optionally, sign up and create an app to get Access Key from https://unsplash.com/developers
  // comment out UNSPLASH_CLIENT_ID if you do not want to use unsplash.com
  #define UNSPLASH_CLIENT_ID      "<client id>"

#else
  #include "_secret.h"
#endif  
...

Notes:

For ESP32 line of MCU, it is not a must to define WIFI_SSID / WIFI_PASSWORD. In case not defined, WiFiManager will be used to acquire WiFi credential. Say, you connect to the AP set up by WiFiManager running on your MCU, with AP name AWClock, as defined by AUTOCONNECT_AP_NAME in config.h
However, you MUST set your own OPEN_WEATHER_MAP_APP_ID for version 2.5 APIs which you can apply for from OpenWeather, say with a free account
Optionally, you can sign up and create an app to get Access Key from Unsplash; with UNSPLASH_CLIENT_ID defined, your are ready to download photos from Unsplash and upload them to your MCU for slideshow

config.h for other basic hardcoded configurations

...
// TIMEZONE (in hours); note that NTP timezone will be gotten from weather api, hence, this is just the initial TIMEZONE
#define INIT_TIMEZONE                       8

// In order to properly setup the openweathermap.org the endpoint
// * please head to http://api.openweathermap.org to create an account and get an APP ID (for version 2.5) 
// * the country (location) for which to retrieve whether is defined with OPEN_WEATHER_API_LOCATION
//   - please refer to https://openweathermap.org/api/geocoding-api
//   - Please use ISO 3166 country codes -- https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes
// * below DEF_SYNC_WEATHER_LOCATION_WITH_GPS ==> got location from GPS when connected to DD
#define DEF_OPEN_WEATHER_API_LOCATION       "Hong Kong"

#define DEF_SYNC_WEATHER_LOCATION_WITH_GPS  true /* got location from GPS when connected to DD */
#define DEF_SLIDE_SHOW_IDLE_DELAY_MINS      2    /* <= 0 means slide show not enabled */
#define DEF_SLIDE_DURATION_SECS             5
#define DEF_UPDATE_WEATHER_INTERVAL_MINS    30

#define NUM_ALARMS                          5
#define AUTO_ACK_ALARM_MINUTES              10
...

Notes:

INIT_TIMEZONE -- the initial timezone; as mentioned previously; your MCU's timezone will eventually be synchronized with that returned from 'get current weather' API
DEF_OPEN_WEATHER_API_LOCATION -- the location (see ISO 3166 country codes) for the initial current weather info; note that when connected to DumbDisplay Android app, your phone's GPS location can be the location for getting current weather info
DEF_SYNC_WEATHER_LOCATION_WITH_GPS -- The default setting whether "get current weather" should be based on GPS location got from your phone, (rather than based on DEF_OPEN_WEATHER_API_LOCATION)
DEF_SLIDE_SHOW_IDLE_DELAY_MINS -- The default setting for how many idle minutes to start slideshow
DEF_SLIDE_DURATION_SECS -- The default setting for how many seconds for each slideshow phone should stay displayed before switching to another one
DEF_UPDATE_WEATHER_INTERVAL_MINS -- The default setting for the interval (minutes) between each update of current weather info
NUM_ALARMS -- The fixed number of alarms AWeatherClock can set
AUTO_ACK_ALARM_MINUTES -- The number of minutes before due alarm is automatically acknowledged (stopped)

System Hardcoded Configurations -- `sys_config.h`

The system hardcoded configuration file sys_config.h not only contains most hardware pin mappings (as will be mentioned in the next section
Highlight for Customization for New Hardware);
specifically, sys_config.h contains some values that will be useful during development:

...
// comment out to DEBUG use of WIFI MANAGER for WiFi SSID / password
//#define TEST_WIFI_MANAGER

// comment out if you want the program to delay startup for 10 seconds for debugging (examine the serial monitor output)
//#define DELAY_INITIALIZE_FOR_SECONDS 10

// suggested to set the following EEPROM_HEADER to the date you want to reset the saved program settings *** INCLUDING saved slides ***
const int32_t EEPROM_HEADER = 20250505;
...

Customizations for New Hardware Highlights

There are several areas to consider for customizing AWeatherClock for new hardware:

platformio.ini for configuring PlatformIO for your MCU
TFT LCD screen. The out-of-the-box configured TFT LCD screens are
- ST7789 with Adafruit ST7735 Library
- ILI9341 with Adafruit_ILI9341
- LCD Screen of TWatch (TW3) -- TTGO_TWatch_Library
Button or touch screen. For touch screen, the out-of-the-box configured touch screens are
- CST816T with cst816t
- FT6236 with FT6236
- FT6336U with Arduino FT6336U
Buzzer or audio module. For audio module, the out-of-the-box configured audio module is
- ES8311 with arduino audio driver and arduino audio tools

For configuring PlatformIO for MCU. For example, for PYCLOCK, which is has an ESP32C3:

[env:PYCLOCK]
platform = espressif32
board = esp32-c3-devkitm-1
framework = arduino
board_build.partitions = no_ota.csv
lib_deps =
    https://github.com/adafruit/Adafruit-ST7735-Library.git
    https://github.com/adafruit/Adafruit-GFX-Library
    https://github.com/Bodmer/TJpg_Decoder.git
    https://github.com/bblanchon/ArduinoJson
    https://github.com/bitbank2/PNGdec#1.1.0
    https://github.com/tzapu/WiFiManager
    https://github.com/trevorwslee/Arduino-DumbDisplay
build_flags =
    -D ARDUINO_USB_MODE=1
    -D ARDUINO_USB_CDC_ON_BOOT=1
    -D FOR_PYCLOCK

Adafruit-ST7735-Library and Adafruit-GFX-Library for the ST7789 2.8 inch 240x320 SPI TFT LCD screen
TJpg_Decoder for decoding JPEG data (slides)
ArduinoJson for parsing the JSON got from OpenWeather
PNGdec for decoding the weather condition icon (PNG) retrieved from OpenWether
Arduino-DumbDisplay for driving DumbDisplay Android app for the UI remotely rendered with your Android phone

As for TFT LCD pin mappings (and others), they are mostly defined in sys_config.h.
For example, for PYCLOCK, which has a button

...
#if defined(FOR_PYCLOCK)
  #define TFT_CS          5
  #define TFT_DC          4
  #define TFT_SCLK        6
  #define TFT_MOSI        7
  #define TFT_RST         8
  #define TFT_X_OFF       0
  #define BUTTON_PIN      9
...

TFT_xxx -- the pin mappings for the TFT LCD screen
TFT_X_OFF -- the x offset to start the 240x240 area; note that the TFT screen can actually be wider than 240
BUTTON_PIN -- the pin number of the button; assume it is INPUT_PULLUP

In case of CST816T touch screen, like that for AST_WATCH

...
#elif defined(FOR_AST_WATCH)
  #define TFT_CS          3
  #define TFT_DC          2
  #define TFT_SCLK        5
  #define TFT_MOSI        6
  #define TFT_RST         8
  #define TFT_X_OFF       20
  #define CST_TP_BUS_NUM  0
  #define CST_TP_SCL      7
  #define CST_TP_SDA      11
  #define CST_TP_RST      10
  #define CST_TP_INT      9
  #define BUZZER_PIN      1
...

TFT_xxx -- the pin mappings for the TFT LCD screen
TFT_X_OFF -- x offset is to 20, since the TFT LCD screen is 280x240
CST_TP_xxx -- the pin mappings for the CST816T touch layer
BUZZER_PIN -- the pin number of the buzzer

Nevertheless, especially for TFT LCD screen, simply define the pin mapping normally not good enough.
Indeed, you will need other customizations to the code, like initialization of Adafruit_ST7789 object in screen_helpers.cpp

...
#if defined(FOR_PYCLOCK)
  #include <Adafruit_ST7789.h>
  SPIClass spi(FSPI);
  Adafruit_ST7789 tft(&spi, TFT_CS, TFT_DC, TFT_RST);
#elif defined(FOR_AST_WATCH)
  #include <Adafruit_ST7789.h>
  SPIClass spi(FSPI);
  Adafruit_ST7789 tft(&spi, TFT_CS, TFT_DC, TFT_RST);
#elif defined(FOR_PICOW_GP)
  #include <Adafruit_ST7789.h>
  Adafruit_ST7789 tft(&SPI1, TFT_CS, TFT_DC, TFT_RST);
#elif defined(FOR_PICOW)
  #include <Adafruit_ST7789.h>
  Adafruit_ST7789 tft(TFT_CS, TFT_DC, TFT_RST);
...
...
void screenSetup() {
#if defined(TFT_BL)
  pinMode(TFT_BL, OUTPUT);
  #if defined(TFT_BL_LOW)
  digitalWrite(TFT_BL, 0);  // light it up (LOW)
  #else
  digitalWrite(TFT_BL, 1);  // light it up
  #endif
#endif

#if defined(FOR_PYCLOCK)
  spi.begin(TFT_SCLK, -1, TFT_MOSI, TFT_CS);
  tft.init(240, 240, SPI_MODE0);
  tft.invertDisplay(true);
  tft.setRotation(2);
  tft.setSPISpeed(40000000);
#elif defined(FOR_AST_WATCH)  
  spi.begin(TFT_SCLK, -1, TFT_MOSI, TFT_CS);
  tft.init(240, 280, SPI_MODE0);
  tft.setRotation(3);
#elif defined(FOR_PICOW_GP)  
  SPI1.setSCK(TFT_SCLK);
  SPI1.setMOSI(TFT_MOSI);
  tft.init(240, 240, SPI_MODE0);
  tft.setRotation(3);
#elif defined(FOR_PICOW)  
  tft.init(240, 320, SPI_MODE0);
  tft.setRotation(1);
  tft.setSPISpeed(40000000);
  ...
}
...

Normally, customizing for BUTTON or TOUCH SCREEN code modifications should be easier, like in trigger_helpers.cpp:

...
#if defined(CST_TP_BUS_NUM)
  #include <Wire.h>
  #include "cst816t.h"
  TwoWire tpWire(CST_TP_BUS_NUM);
  cst816t touchpad(tpWire, CST_TP_RST, CST_TP_INT);
#elif defined(FT6336_INT)
  #include <Wire.h>
  #include "FT6336U.h"
  #if defined(ESP32)
    FT6336U touchpad(FT6336_SDA, FT6336_SCL, FT6336_RST, FT6336_INT);
  #else
    FT6336U touchpad(FT6336_RST, FT6336_INT);
  #endif 
#elif defined(GT911_TP_SCL)  
  #include "TAMC_GT911.h"
  TAMC_GT911 touchpad = TAMC_GT911(GT911_TP_SDA, GT911_TP_SCL, GT911_TP_INT, GT911_TP_RST, GT911_TP_WIDTH, GT911_TP_HEIGHT);
...
void triggerSetup() {
#ifdef BUTTON_PIN
  pinMode(BUTTON_PIN, INPUT_PULLUP);  // assume INPUT_PULLUP
  attachInterrupt(digitalPinToInterrupt(BUTTON_PIN), _triggered, FALLING);
#elif defined(TOUCH_PIN)
  touchAttachInterrupt(TOUCH_PIN, _triggered, TOUCH_THRESHOLD);
#elif defined(CST_TP_BUS_NUM)
  tpWire.begin(CST_TP_SDA, CST_TP_SCL);
  touchpad.begin(mode_motion);
#elif defined(FT6336_INT)
  //pinMode(FT_TP_INT, INPUT_PULLUP);
  #if !defined(ESP32)
    Wire.setSDA(FT6336_SDA);  // FT6336U will use Wire
    Wire.setSCL(FT6336_SCL);
  #endif  
  touchpad.begin();
  attachInterrupt(digitalPinToInterrupt(FT6336_INT), _triggered, CHANGE);
   ...
}
...

As hinted above with AST_WATCH, to configure a buzzer / speaker is as easy as defining the pin assignment BUZZER_PIN.
However, there are much more pins for the ES8311 audio module, as in ESP_SPARKBOT

  #define ES8311_PA             46 
  #define ES8311_I2C_SCL        5
  #define ES8311_I2C_SDA        4
  #define ES8311_I2S_PORT       35
  #define ES8311_I2S_MCLK       45
  #define ES8311_I2S_BCK        39  
  #define ES8311_I2S_WS         41
  #define ES8311_I2S_DOUT       42
  #define ES8311_I2S_DIN        40
  #define DEF_AUDIO_VOLUME         60

DEF_AUDIO_VOLUME sets the default volume of the audio module; set it if you want something different from the audio module default

Customizations for New Hardware PicoW Example

Take the above-mentioned PICOW as an example -- the Raspberry Pi PicoW wiring as mentioned in Simple Arduino Framework Raspberry Pi Pico / ESP32 TFT LCD Photo Frame Implementation With Photos Downloaded From the Internet Via DumbDisplay

A ST7789 2.8 inch 240x320 SPI TFT LCD module is used, with a FT6336U capacitive touch layer
It's touch screen is configured as a TOUCH SCREEN for "trigger" input
A speaker is attached to it -- one end of the speaker is connected to GND and the other end is connected to GP15

|Raspberry Pi Pico|SPI TFT LCD |
|-----------------|------------|
| 3V3 | VCC |
| GND | GND |
| GP21 | BL |
| GP17 | CS |
| GP16 | RS / DC |
| GP18 | CLK / SCLK |
| GP19 | SDA / MOSI |
| GP20 | RST |
| GP5 | TP_SCL |
| GP4 | TP_SDA |
| GP6 | TP_INT |
| GP7 | TP_RST |
| GP15 | SPEAKER |
| GND | SPEAKER |

platformio.ini

...
[env:PICOW]  ; ensure long file name support ... git config --system core.longpaths true
platform = https://github.com/maxgerhardt/platform-raspberrypi.git
board = rpipicow
framework = arduino
board_build.core = earlephilhower
board_build.filesystem = littlefs
board_build.filesystem_size = 1m
lib_deps =
    https://github.com/adafruit/Adafruit-ST7735-Library.git#1.11.0
    https://github.com/adafruit/Adafruit-GFX-Library#1.12.1
    https://github.com/Bodmer/TJpg_Decoder.git#V1.1.0 
    https://github.com/aselectroworks/Arduino-FT6336U
    https://github.com/bblanchon/ArduinoJson#v7.4.1
    https://github.com/bitbank2/PNGdec#1.1.0
    https://github.com/trevorwslee/Arduino-DumbDisplay#v0.9.9-r51
build_flags =
    -D FOR_PICOW
...

it appears that if using earlephilhower core in Windows, need to ensure long file name support ...

git config --system core.longpaths true
there are quite a few libraries needed:
- Adafruit-ST7735-Library and Adafruit-GFX-Library for the ST7789 2.8 inch 240x320 SPI TFT LCD screen
- TJpg_Decoder for decoding JPEG data (slides)
- Arduino-FT6336U for the FT6336U touch layer
- ArduinoJson for parsing the JSON got from OpenWeather
- PNGdec for decoding the weather condition icon (PNG) retrieved from OpenWether
- Arduino-DumbDisplay for driving DumbDisplay Android app for the UI remotely rendered with your Android phone

sys_config.h

...
#elif defined(FOR_PICOW)
  #define TFT_BL      21
  #define TFT_CS      17
  #define TFT_DC      16
  #define TFT_SCLK    18
  #define TFT_MOSI    19
  #define TFT_RST     20
  #define TFT_X_OFF   40
  #define TFT_UN_INVERTED
  #define FT_TP_SCL   5 
  #define FT_TP_SDA   4
  #define FT_TP_INT   6
  #define FT_TP_RST   7
  #define BUZZER_PIN  15
...

TFT_xxx are the pin mappings of the ST7789 TFT LCD screen
TFT_X_OFF is set to 40 since the screen is 320 wide and therefore need to offset the horizontal start by 40
TFT_UN_INVERTED defines that the TFT LCD normally un-inverted; this macro tells the normal state of the TFT LCD when flashing the TFT LCD for due alarm
FT_TP_xxx are the pin mappings of the FT6336U touch layer (normally part of the TFT LCD module)
BUZZER_PIN is the pin connected to one end of the speaker attached to the PicoW; the other end can connect to GND of the PicoW

screen_helpers.cpp

...
#elif defined(FOR_PICOW)
  #include <Adafruit_ST7789.h>
  Adafruit_ST7789 tft(TFT_CS, TFT_DC, TFT_RST);
...
void screenSetup() {
#if defined(TFT_BL)
  pinMode(TFT_BL, OUTPUT);
  #if defined(TFT_BL_LOW)
    digitalWrite(TFT_BL, 0);  // light it up (LOW)
  #else
    digitalWrite(TFT_BL, 1);  // light it up
  #endif
#endif
  ...
#elif defined(FOR_PICOW)  
  tft.init(240, 320, SPI_MODE0);
  tft.invertDisplay(false);
  tft.setRotation(1);
  tft.setSPISpeed(40000000);
  ...
}
...

the first defined(FOR_PICOW)
- include the needed header file for the TFT LCD screen <Adafruit_ST7789.h>
- instantiate the global variable tft (as the TFT LCD screen object)
the second defined(FOR_PICOW), which is inside the void screenSetup() block
- run the necessary code to setup the TFT LCD screen -- tft declared previously

Customizations for New Hardware ESP32 Example

Take the above-mentioned ESP32 as another example

A ST7789 1.3 inch 240x240 SPI TFT LCD module is used
A button is attached to it, which act as a BUTTON for "trigger" input -- one end of the button is connected to GND and the other end is connected to GIPO26
A speaker is attached to it -- one end of the speaker is connected to GND and the other end is connected to GIPO23

|ESP32 |SPI TFT LCD |
|-------------------|------------|
| 3V3 | VCC |
| GND | GND |
| GPIO18 | BL |
| GPIO15 | CS |
| GPIO2 | DC |
| GPIO14 | SCLK |
| GPIO13 | MOSI |
| GPIO4 | RST |
| GPIO26 | BUTTON |
| GND | BUTTON |
| GPIO23 | SPEAKER |
| GND | SPEAKER |

platformio.ini

...
[env:ESP32]
platform = espressif32
board = esp32dev
framework = arduino
;board_build.partitions = huge_app.csv
;monitor_filters = esp32_exception_decoder
lib_deps =
    https://github.com/adafruit/Adafruit-ST7735-Library.git#1.11.0
    https://github.com/adafruit/Adafruit-GFX-Library#1.12.1
    https://github.com/Bodmer/TJpg_Decoder.git#V1.1.0 
    https://github.com/bblanchon/ArduinoJson#v7.4.1
    https://github.com/bitbank2/PNGdec#1.1.0
    https://github.com/tzapu/WiFiManager#v2.0.17
    https://github.com/trevorwslee/Arduino-DumbDisplay#v0.9.9-r51
build_flags =
    -D FOR_ESP32
...

it appears that if using earlephilhower core in Windows, need to ensure long file name support ...

git config --system core.longpaths true
there are quite a few libraries needed:
- Adafruit-ST7735-Library and Adafruit-GFX-Library for the ST7789 1.3 inch 240x240 SPI TFT LCD screen
- TJpg_Decoder for decoding JPEG data (slides)
- ArduinoJson for parsing the JSON got from OpenWeather
- PNGdec for decoding the weather condition icon (PNG) retrieved from OpenWether
- WiFiManager for getting WiFi credential by setting up WiFi hotspot, when WiFi SSID / password not hard-coded
- Arduino-DumbDisplay for driving DumbDisplay Android app for the UI remotely rendered with your Android phone sys_config.h

...
#elif defined(FOR_ESP32)
  #define TFT_BL      18     
  #define TFT_CS      15
  #define TFT_DC      2
  #define TFT_SCLK    14
  #define TFT_MOSI    13
  #define TFT_RST     4
  #define TFT_X_OFF   0
  #define BUTTON_PIN  26    
  #define BUZZER_PIN  23
...

TFT_xxx are the pin mappings of the ST7789 TFT LCD screen
TFT_X_OFF is set to 0 since the screen is 240 wide and therefore need no horizontal offset
BUTTON_PIN is the pin connected to one end of the button attached to the ESP32; the other end can connect to GND of the ESP32
BUZZER_PIN is the pin connected to one end of the speaker attached to the ESP32; the other end can connect to GND of the ESP32

screen_helpers.cpp

...
#elif defined(FOR_ESP32)
  #include <Adafruit_GFX.h>
  #include <Adafruit_ST7789.h>
  #include <SPI.h>   
  SPIClass spi = SPIClass(HSPI);
  Adafruit_ST7789 tft(&spi, TFT_CS, TFT_DC, TFT_RST);
...
void screenSetup() {
#if defined(TFT_BL)
  pinMode(TFT_BL, OUTPUT);
  #if defined(TFT_BL_LOW)
    digitalWrite(TFT_BL, 0);  // light it up (LOW)
  #else
    digitalWrite(TFT_BL, 1);  // light it up
  #endif
#endif
  ...
#elif defined(FOR_ESP32)  
  spi.begin(TFT_SCLK, -1, TFT_MOSI, TFT_CS);
  tft.setSPISpeed(40000000);
  tft.init(240, 240, SPI_MODE0);
  tft.setRotation(3);
  ...
}
...

the first defined(FOR_ESP32)
- include the needed header file for the TFT LCD screen <Adafruit_ST7789.h>
- instantiate the global variable tft (as the TFT LCD screen object)
the second defined(FOR_ESP32), which is inside the void screenSetup() block
- run the necessary code to setup the TFT LCD screen -- tft declared previously

Have fun with AWeatherClock!

Enjoy!

ESP-IDF with Arduino Examples

Trevor Lee — Sun, 02 Mar 2025 12:29:11 +0000

ESP-IDF with Arduino Examples

ESP32 microcontroller program development with Arduino framework is fun.
Nevertheless, in order to be more flexible, it is often easier to go the route of ESP32 program development with ESP-IDF.

Say,

With microphone module installed, it is possible to develop "wake-word" recognition; however, it is not clear / easy to do so in Arduino framework. E.g. ESP-Box.
With camera module installed, it is possible to develop face recognition (not just face detection); however, it is not clear / easy to do so in Arduino framework. E.g. ESP-Eye.

Consequently, one might then choose to develop ESP32 program directly and completely with the well-supported native ESP-IDF framework.

Nevertheless, it is possible to do it the other way around, by incorporating Arduino framework style in ESP-IDF programming.
Will this make ESP32 programming looks easier? Moreover, it might be possible to use libraries written using the Arduino framework. At the very least, this way might be helpful in transition from the easier non-native Arduino framework for ESP32 to the native ESP-IDF.

As stated in Arduino as an ESP-IDF component

It is yet to see how compatible is Arduino code with ESP-IDF code; regardless, let's have some fun trying some ESP-IDF with Arduino examples, as will be described next.

Here

VSCode will be used as the development IDE, certainly, together with ESP-IDF VSCode extension
The target ESP-IDF version is v5.3.2
The target Arduino core for ESP32 is v3.0.2

Examples

The examples that we will try out are:

idf-blink: A simple ESP-IDF blink program that is essentially generated by ESP-IDF without modification; it blinks the built-in LED (pin 2)
idf_with_a_blink: From an "empty" ESP-IDF program, modified to incorporate with Arduino to blink the built-in LED, the same as idf-blink but coded in Arduino style
s3_wb2812_blink: Again, started from an "empty" ESP-IDF program, modified to incorporate with Arduino, in order to be able to use the Arduino library Freenove_WS2812_Lib_for_ESP32 for "blinking" the NeoPixel on an ESP32S3 board (pin 48)
wifi_dd_blink: Modified to incorporate with Arduino, in order to use the Arduino library Arduino-DumbDisplay. With DumbDisplay, you can realize simple UI on your Android phone. You may want to refer to the post Blink Test With Virtual Display, DumbDisplay for a brief introduction to DumbDisplay. Note that this example wifi_dd_blink uses WiFi connectivity, rather than OTG as described in the post.
le_dd_blink: Like wifi_dd_blink, but is a bit more extensive in using DumbDisplay's features, and is using Bluetooth Low Energy connectivity. Moreover, le_dd_blink deliberately uses some ESP-IDF constructs like xTaskCreate() / vTaskDelay() / printf()

Highlights

The steps I worked out to add Android to ESP-IDF is based on the above mentioned [official] Arduino as an ESP-IDF component:

Add IDF component to idf_component.yml:

  dependencies:
    idf:
      version: '>=4.1.0'
    espressif/arduino-esp32: ^3.0.2

The default of CONFIG_FREERTOS_HZ in sdkconfig is 100, change it to 1000
The C entry point app_main() should be declared in a C++ file like

  #include "Arduino.h"
  extern "C" void app_main(void) {
      initArduino();
  }

Notice:

The inclusion of the header file Arduino.h
The initialization routine initArduino() should be called the first thing in the entry point app_main()
- In order to make it more like programming with Arduino framework, setup() and loop() can be added like

  #include "Arduino.h"
  void setup() {}
  void loop() {}
  extern "C" void app_main(void) {
    initArduino();
    setup();
    while (1) {
      loop();
    }
  }

A way to use Arduino library is to download the Arduino library source code and compile the library code as if it is part of the main program. Hence
- in main, create the folder arduino_libraries for the the downloaded / cloned Arduino library source code
- modify CMakeLists.txt in main to compile the library source, like
```
idf_component_register(
    SRC_DIRS "." "arduino_libraries/Freenove_WS2812_Lib_for_ESP32/src"
    INCLUDE_DIRS "arduino_libraries/Freenove_WS2812_Lib_for_ESP32/src"
)
```
or
```
idf_component_register(
    SRC_DIRS "." "arduino_libraries/Arduino-DumbDisplay"
    INCLUDE_DIRS "arduino_libraries/Arduino-DumbDisplay"
)
```

Demonstration

You may be interested in a demonstration of steps leading to the examples as recorded in the video ESP-IDF with Arduino Examples; from IDF-Blink to Virtual Blink using WiFi / BLE

Enjoy!

Hope that you will have fun with the examples!

Do share anything you find interesting / insightful concerning ESP-IDF with Arduino development.

Enjoy!

Sliding Puzzle Next Move Suggesting Simple DL Model with ESP32 TensorFlow Lite

Trevor Lee — Sun, 08 Dec 2024 14:17:00 +0000

Sliding Puzzle 'Next Move' Suggesting Simple DL Model with ESP32 TensorFlow Lite

This project takes the game Sliding Puzzle
(with simple 'next move' suggesting 'search-directed heuristic' option),
adding to it the capability of suggesting 'next move' with a simple and naive DL model realized with ESP32 TensorFow Lite support.
The Sliding Puzzle game is implemented for Arduino framework compatible microcontroller boards with aids from DumbDisplay
to render the game remotely on your Android mobile phone.
Specifically, ESP32 / ESP32S3 is the targe microcontroller board for this experiment, since it not only supports Arduino framework, it also supports TensorFlow Lite.

The DL model of this experiment is implemented with TensorFlow Lite that I worked out by referencing to two of my previous experiments:

In addition to an ESP32 / ESP32S3 microcontroller board, a few tools are assumed:

Python 3
Git
VSCode and PlatformIO -- reference on how they are used: A Way to Run Arduino Sketch With VSCode PlatformIO Directly
DumbDisplay Android app -- reference: Blink Test With Virtual Display, DumbDisplay

Building DL Model with VSCode

First, clone this project's source from the project's GitHub repository

git clone https://github.com/trevorwslee/ESP32SlidingPuzzle

Open the cloned directory ESP32SlidingPuzzle with VSCode, then open train_model.ipynb

Run all cells of the Jupyter Notebook train_model.ipynb.
If environment is not setup already, it should first prompt you to create / select an Python environment.

Create a virtual Python environment for the project

As the last step, when asked to install dependencies, make sure to select requirement.txt

This should create the Python virtual environment .venv for the project with the needed dependencies installed.

If all the cells completed successfully, the model C header file src/esp32_sliding_puzzle/sp_model_4.h will be generated, overwriting the one already there.
Yes, there is one already included with this project, and hence it is not necessary for you to train the model again, unless you would like to tune the DL model.

The DL model is naively constructed with Keras like

tile_count = 4
batch_size = 128
epochs = 100
...
model = keras.models.Sequential()
model.add(keras.layers.Dense(256, activation='relu', input_shape=(tile_count * tile_count,)))
model.add(keras.layers.Dropout(0.3))
model.add(keras.layers.Dense(256, activation='relu'))
model.add(keras.layers.Dense(4, activation='softmax'))
...
model.compile(loss='categorical_crossentropy', optimizer=keras.optimizers.RMSprop(), metrics=['accuracy'])
history = model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_data=(x_validate, y_validate))

With accuracy 0.868, the DL model appears not very good, but should be acceptable

Building and Uploading the Sketch

The steps to build and upload the sketch is via PlatformIO (an extension of VSCode).

The configurations for developing and building of the sketch are basically written down in the platformio.ini file

[env]
monitor_speed = 115200

[env:ESP32]
platform = espressif32
board = esp32dev
framework = arduino
board_build.partitions = huge_app.csv
monitor_filters = esp32_exception_decoder
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay
    tanakamasayuki/TensorFlowLite_ESP32@^1.0.0
build_flags =
    -D FOR_ESP32

[env:ESP32S3]
platform = espressif32
board = esp32-s3-devkitc-1
framework = arduino
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay
    tanakamasayuki/TensorFlowLite_ESP32@^1.0.0
build_flags =
    -D FOR_ESP32S3

Notice that you have choices of PlatformIO environments -- ESP32 and ESP32S3

Please choose the correct one according to the microcontroller board that you have.

In either case, the program entry point is src/main.cpp

#include <Arduino.h>

#if defined(CONFIG_IDF_TARGET_ESP32S3)
  #include "_secret.h"
#else
  #define BLUETOOTH "ESP32Sliding"
#endif

#include "esp32_sliding_puzzle/esp32_sliding_puzzle.ino"

Notice that

For ESP32S3, WiFi connection to DumbDisplay Android app is used. In such a case, you will need to provide necessary WiFi login credentials in a header file src/_secret.h that you need to create with content like

  #define WIFI_SSID           "wifi-ssid"
  #define WIFI_PASSWORD       "wifi-password"

Otherwise, for ESP32, Bluetooth connection to DumbDisplay Android app is used, with the Bluetooth device name ESP32Sliding

Build and upload the sketch with VSCode menu item View | Command Palette

For ESP32 which is using Bluetooth connectivity, you should see log entries with Serial Monitor (with baud rate 115200) like

For ESP32S3 which is using Wifi connectivity, the log entries will be like

Notice that the IP to connect to the ESP32 board shown by the log entries.

To connect your Android phone to your ESP32 / ESP32S3 microcontroller board via DumbDisplay Android app, open the DumbDisplay Android app and make connection like

with Bluetooth	add WiFi	with Wifi

Sliding Puzzle Game UI


After connection, you should see the picture of the board drawn.
To start a game, double click on the board, to divide the board into grid of tiles

Double clicking on the board will randomize the board by 5 steps (5 random moves).

During game play, you can click Suggest for a suggested 'next move'.
If you click Continuous, suggested 'next moves' are continuously made until either you disabled Continuous or the puzzle is solved.

There are three options for the 'next move' suggestion:
1) AI Only -- use the trained DL model for predicting 'next move' (the highest probability one)
2) AI+Search -- largely use the trained DL model for predicting 'next move'; however, if the top 'next move' probability is not high, fallback to use original "search* algo
3) Search Only -- use original "search" algo only

After every solving of the puzzle, 5 more randomize steps are added to randomization making the game a bit harder.

Any time if you want to reset / restart the game, double click on the Reset button.

Improving the 'Next Move' Suggestion Accuracy

The 'next move' suggestion is certainly not very accurate, specially when the board is randomized for many steps.

The naive DL model can certainly be improved. Notice that intrain_model.ipynb, the training data is randomized in reverse of how a board is randomize. Hence, it makes sense that the number of randomize steps affects the accuracy of the model specially in case the game is randomized for more steps than the model is trained. Here are the parameters for training the model:
- round_count -- the number of rounds of random boards to generate; 10000 by default
- random_step_count -- the number of random moves from "solved orientation" to "randomized orientation"; 20 by default
The "search" algo can be improved
The DL + "search" can be improved

Moreover, the board size can be bigger, like 5x5.
In order to change the board size to 5:

1) Need to change tile_count of train_model.ipynb generate a sp_model_5.h for the sketch

   # tile_count is the number of horizontal / vertical tiles ... a tile_count of 4 means a 4x4 board
   tile_count = 5

2) Need to modify the sketch esp32_sliding_puzzle.ino

   #define TILE_COUNT 5

which controls

    #if TILE_COUNT == 4
      #include "sp_model_4.h"
    #elif TILE_COUNT == 5
      #include "sp_model_5.h"
    #endif
    const tflite::Model* model = ::tflite::GetModel(sp_model);

Interested friends are encouraged to try out and share how the 'next move' suggestion can be improved.

A LLM Prompt for the Moves

BTW, here is a challenging LLM prompt for you to try out (just for fun)

if a 4x4 sliding puzzle board game "reference" board layout is represented with

---------------------
|    |  2 |  3 |  4 |
|  5 |  6 |  7 |  8 |
|  9 | 10 | 11 | 12 |
| 13 | 14 | 15 | 16 |
---------------------

and move of the "missing" cell is represented with a number:
. 0: move to left
. 1: move to right
. 2: move to top
. 3: move to bottom

1:
when move the "missing" cell of the "reference" board layout with move 1, what is the board layout after the move?

2:
from the "reference" board layout, what is the board layout after the moves: 1, 1, 3, 3

3:
if given the board layout
---------------------
|  2 |  6 |  3 |  4 |
|  5 | 10 |  7 |  8 |
|    |  9 | 11 | 12 |
| 13 | 14 | 15 | 16 |
---------------------
what are the moves in order to bring the board back to the "reference" board layout?

From my experience, not all LLM models I tried can provide satisfactory response for the above prompt.
One reason might be that the prompt is not well written; another might be that the answer to question 3 above can be "endless".
GPT4o seems to provide reasonable response; but others like
GPT4 or even DeepSeek might produce long response that appears never ending;
hence, be prepared to stop the response generation.

Enjoy!

Hope that you will have fun with it! Enjoy!

Turn ESP32-CAM into a Snapshot Taker, for Selfies and Time-Lapse Pictures

Trevor Lee — Wed, 11 Sep 2024 13:42:03 +0000

Turn ESP32-CAM into a Snapshot Taker, for Selfies and Time Lapse Pictures

This project is an attempt to turn ESP32-CAM (or similar ones like LILYGO T-Camera / T-Camera Plus) microcontroller board into an Android phone-managed snapshot taker, for snapshots like selfies and time-lapse pictures ... maybe ... just for fun!

The microcontroller program (sketch) is developed with Arduino framework using VS Code and PlatformIO, in the similar fashion as described by the post -- A Way to Run Arduino Sketch With VSCode PlatformIO Directly

The remote UI -- control panel -- is realized with the help of the DumbDisplay Android app. You have a choice of using Bluetooth (classic) or WiFi for the connection. For a brief description of DumbDisplay, please refer to the post -- Blink Test With Virtual Display, DumbDisplay

Please note that the UI is driven by the sketch; i.e., the flow of the UI is programmed in the sketch. Other than installing the DumbDisplay Android app, no building of a separate mobile app is necessary.

The UI -- Ways of taking Snapshots

There are 3 ways snapshots can be captured and saved to your phone


With your phone connected to the ESP32-CAM -- certainly through the DumbDisplay Android app with Bluetooth or WiFi -- you click the *💾Save* button of the UI to save the image being shown. You can turn on / off the auto-save feature by clicking *Auto❎* / *Auto☑️*. With auto-save enabled, whenever a snapshot is captured from the ESP32-CAM, it will be saved to your phone automatically.

With your phone connected to the ESP32-CAM -- certainly through the DumbDisplay Android app with Bluetooth or WiFi -- you click the 💾Save button of the UI to save the image being shown. You can turn on / off the auto-save feature by clicking Auto❎ / Auto☑️. With auto-save enabled, whenever a snapshot is captured from the ESP32-CAM, it will be saved to your phone automatically.


Assuming you have connected your phone with the ESP32-CAM, clicking on the image canvas will pause showing snapshots, and you will be provided with a slider to slide back in time to select the previously captured snapshots to save to your phone. You will have 20 such snapshots that you can slide back in time to. Once you see the snapshots you like, click *💾Save* to save it. When you are done, you can go back by clicking *❌Cancel* (or double click on the image canvas).

Assuming you have connected your phone with the ESP32-CAM, clicking on the image canvas will pause showing snapshots, and you will be provided with a slider to slide back in time to select the previously captured snapshots to save to your phone. You will have 20 such snapshots that you can slide back in time to. Once you see the snapshots you like, click 💾Save to save it. When you are done, you can go back by clicking ❌Cancel (or double click on the image canvas).


Select *Offline📴* to enable "offline" capturing and saving of snapshots to the ESP32-CAM. With "offline" enabled, when you disconnect (i.e. offline), the ESP32-CAM will start capturing "offline" snapshots saving them to its flash memory (or SD card). Whenever you reconnect, you will be asked if you want to transfer the saved "offline" snapshots to your phone. Better yet, in SD card case, you can physically transfer the "offline" snapshot *JPEG* files saved the usual way -- insert the SD card to your computer, copy and delete the *JPEG* files on your SD card. One point about using the SD card slot of ESP32-CAM -- since the SD card module of ESP32-CAM shares the same pin 4 used by the flashlight, whenever the SD card is accessed, the flashlight will light up bright (hence it is not a feature, but it is how it is)

Select Offline📴 to enable "offline" capturing and saving of snapshots to the ESP32-CAM. With "offline" enabled, when you disconnect (i.e. offline), the ESP32-CAM will start capturing "offline" snapshots saving them to its flash memory (or SD card). Whenever you reconnect, you will be asked if you want to transfer the saved "offline" snapshots to your phone. Better yet, in SD card case, you can physically transfer the "offline" snapshot JPEG files saved the usual way -- insert the SD card to your computer, copy and delete the JPEG files on your SD card. One point about using the SD card slot of ESP32-CAM -- since the SD card module of ESP32-CAM shares the same pin 4 used by the flashlight, whenever the SD card is accessed, the flashlight will light up bright (hence it is not a feature, but it is how it is)

The snapshots will be saved in JPEG format:

Snapshots saved to your phone will be saved to the private folder of DumbDisplay app - /<main-storage>/Android/data/nobody.trevorlee.dumbdisplay/files/Pictures/DumbDisplay/snaps/ -- with file name like 20240904_223249_001.jpg (YYYYMMDD-hhmmss_seq.jpg)
Offline snapshots transferred to your phone will be saved to a subfolder of the above-mentioned private folder of DumbDisplay app. The name of the subfolder depends on the time you do the snapshots transfer, and is like 20240904_231329_OFF.
Offline snapshots are saved to your ESP32-CAM's flash memory / SD card with name like off_001.jpg. Note that the "offline" JPEG files will only be properly time-stamped if there was no reset / reboot of ESP32-CAM after connecting to your phone.


By default, the storage for DumbDisplay app is a private folder, which DumbDisplay app needs to initialize. You can do this By selecting the *Settings* menu item of DumbDisplay app, and clicking on the *Media Storage* button


And you can use a folder manager app, like *Files* by *Marc apps & software* to browse to that folder.

The UI -- Frequency of Capturing Snapshots

Actually, snapshot capturing is continuous (as fast as possible), but captured snapshot shipment to your phone is not as smooth; in fact there is a setting on how frequently a snapshot is shipped to your phone. Depending on the resolution / quality of the snapshot and the connection method / condition, it can be as frequent as 5 snapshots per second. Treat this frequency control as a feature that allows taking of time-lapse pictures at a desired frequency 😁

There are 4 quick selections of frequency / frame rate -- 5 PS / 2 PS / 1 PS / 30 PM corresponding to 5 frames per second / 2 frames per second / 1 frame per second / 30 frames per minute

And there is a custom per-hour frame rate you can set and select -- 720 PH (720 is the default). Once you click on the 720 PH selection, your phone's virtual keyword will pop up allowing you to enter the value (1 - 3600) you want (an empty value means previously set value).

The UI -- Snapshots Quality Adjustments

There are several camera adjustments that will affect the quality of the captured snapshots:

Snapshot resolution / size selections:
- QVGA (320x240)
- VGA (640x480)
- SVGA (800x600)
- XGA (1024x768)
- HD (1280x720)
- SXGA (1280x1024)
- UXGA (1600x1200)
JPEG compression quality slider 🖼️ -- from 5 (high quality) to 60 (low quality)
Brightness slider ☀️ -- from -2 (dim) to 2 (bright)

Note that the higher the snapshots quality / resolution, the more data needs be shipped from the ESP32-CAM to your phone, making the UI less responsive, especially when WiFi (not Bluetooth) since WiFi is not full-duplex.

The UI -- ESP32 Idle Sleep

Another feature is that when "offline" snapshot capturing is enabled, after a while (60 seconds) of being idle (i.e. not connected), the ESP32-CAM will be put to sleep mode in order to save power. Of course, since "offline" capturing is enabled, the ESP32-CAM will wake up in due time to take a snapshot. However if "offline" frequency is too high, higher than 12 frames per minute, the ESP32-CAM will stay on.

In any case, if you want to connect to the ESP32-CAM when it is in sleep mode, you will need to reset / reboot the ESP32-CAM first

Developing and Building

As mentioned previously, the sketch will be developed using VS Code and PlatformIO.
Please clone the PlatformIO project ESP32CamSnapper GitHub repository.

The configurations for developing and building of the sketch are basically written down in the platformio.ini file

[env]
monitor_speed = 115200

[env:ESP32CAM]
platform = espressif32
board = esp32cam
framework = arduino
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay
build_flags =
    -D FOR_ESP32CAM

[env:TCAMERA]  ; v7
platform = espressif32
board = esp-wrover-kit
framework = arduino
board_build.partitions = huge_app.csv
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay
build_flags =
    -D BOARD_HAS_PSRAM
    -D FOR_TCAMERA

[env:TCAMERAPLUS]
platform = espressif32
board = esp32dev
framework = arduino
board_build.partitions = huge_app.csv
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay
build_flags =
    -D BOARD_HAS_PSRAM
    -D FOR_TCAMERAPLUS

Please make sure you select the correct PlatformIO project environment -- ESP32CAM / TCAMERA / TCAMERAPLUS

The program entry point is src/main.cpp

// *** use Bluetooth with the following device name
#define BLUETOOTH "ESP32CamSnapper"
#include "esp32camsnapper/esp32camsnapper.ino"

The above main.cpp assumes that you prefer to use Bluetooth (classic), and defines the Bluetooth device name to be ESP32CamSnapper

#define BLUETOOTH "ESP32CamSnapper"

Note that, for the purposes of this sketch, Bluetooth (classic) is the suggested way of connecting with DumbDisplay app.

However, if you prefer to use WiFi, you will need to define WIFI_SSID and WIFI_PASSWORD rather than BLUETOOTH

//#define BLUETOOTH "ESP32CamSnapper"
#define WIFI_SSID           "your-wifi-ssid"
#define WIFI_PASSWORD       "your-wifi-password"

In case of WiFi, you will need to find out the IP of your ESP32-CAM in order to make connect to it. This can be done easily by attaching it to a Serial monitor (set to baud-rate 115200). There, when your ESP32-CAM tries to connect to DumbDisplay app, log lines like below will be printed:

binded WIFI TrevorWireless
listening on 192.168.0.155:10201 ...
listening on 192.168.0.155:10201 ...
listening on 192.168.0.155:10201 ...

That IP address is the IP address of your ESP32-CAM to make connection to.

If you have SD card installed for ESP32-CAM (or LILYGO T-Camera Plus), you can use it to store "offline" snapshots, rather than the limited flash memory of the microcontroller board.

To configure the sketch for such setup, you will need to define the macro OFFLINE_USE_SD, like by uncomment the corresponding line of the sketch

// *** uncomment the following if offline snaps are to be stored to SD (ESP32CAM / TCameraPlus), rather than LittleFS
#define OFFLINE_USE_SD

or put the #define line in main.cpp like

#define OFFLINE_USE_SD 
#define BLUETOOTH "ESP32CamSnapper"
#include "esp32camsnapper/esp32camsnapper.ino"

The Sketch

The sketch of the project is esp32camsnapper/esp32camsnapper.ino. There are several customizations you may want to make:

By default, you have 20 snapshots you can go back in time to. This number is controlled by the macro STREAM_KEEP_IMAGE_COUNT

  // *** maximum number of images to keep cached (on app side) when streaming snaps to DD app; this number affects how much you can go back to select which image to save
  #define STREAM_KEEP_IMAGE_COUNT             20

By default, you can enable "offline" snapshot capturing. If you don't want "offline" at all, you can comment out the corresponding #define

  // *** support offline snap taking; comment out if not desired
  #define SUPPORT_OFFLINE

By default, the ESP32-CAM will be put to sleep after 60 seconds being idle (i.e. not connected). You can change it to other value by changing the macro IDLE_SLEEP_SECS. If this 'idle put to sleep' behavior is not desirable, you can comment out that #define to disable such behavior

  // *** number of seconds to put ESP to sleep when idle (not connected); if ESP went to sleep, will need to reset it in order to connect; comment out if not desired
  #define IDLE_SLEEP_SECS                     60

Most settings you change in the UI will be persisted to the EEPROM of the ESP32-CAM. These settings will be effective even you re-upload the program built. In case you want to reset those settings to their defaults, change the HEADER to some other value, build and re-upload

  // *** if want to reset settings / offline snap metadata saved in EEPROM, set the following to something else
  const int32_t HEADER = 20240910;

Sketch Highlight -- EEPROM

Firstly, in order to use the EEPROM, you will first need to reserve some number of bytes for it, like:
void setup() {
Serial.begin(115200);
EEPROM.begin(32);
...
}
Here, 32 bytes are reserved for the purpose.

Most settings are persisted to the EEPROM of ESP32-CAM like

  EEPROM.writeLong(0, HEADER);
  EEPROM.writeChar(4, cameraBrightness);
  EEPROM.writeBool(5, cameraVFlip);
  EEPROM.writeBool(6, cameraHMirror);
  EEPROM.writeChar(7, currentCachePMFrameRateIndex);
  EEPROM.writeChar(8, currentFrameSizeIndex);
  EEPROM.writeBool(9, enableOffline);
  EEPROM.writeShort(10, customCachePHFrameRate);
  EEPROM.writeBool(12, autoSave);
  EEPROM.writeChar(13, cameraQuality);
  EEPROM.commit();

And these settings are read back when ESP32-CAM starts up like

  int32_t header = EEPROM.readLong(0);
  if (header != HEADER) {
    dumbdisplay.logToSerial("EEPROM: not the expected header ... " + String(header) + " vs " + String(HEADER));
    return;
  }
  cameraBrightness = EEPROM.readChar(4);
  cameraVFlip = EEPROM.readBool(5);
  cameraHMirror = EEPROM.readBool(6);
  currentCachePMFrameRateIndex = EEPROM.readChar(7);
  currentFrameSizeIndex = EEPROM.readChar(8);
  enableOffline = EEPROM.readBool(9);
  customCachePHFrameRate = EEPROM.readShort(10);
  autoSave = EEPROM.readBool(12);
  cameraQuality = EEPROM.readChar(13);

Sketch Highlight -- ESP32 Sleep Mode

ESP32 can be put to sleep like

esp_sleep_enable_timer_wakeup(sleepTimeoutMillis * 1000);  // in micro second
esp_deep_sleep_start();

When it wakes up, the sketch / program will be just like freshly started, except for variables like

RTC_DATA_ATTR int32_t tzMins = 0;
RTC_DATA_ATTR int64_t wakeupOfflineSnapMillis = -1;

Note that the variable prefixed with RTC_DATA_ATTR will keep its value through sleep.

Sketch Highlight -- Camera

The ESP32-CAM Camera is initialized with initializeCamera

bool initializeCamera(framesize_t frameSize, int jpegQuality) {
  esp_camera_deinit();     // just in case, disable camera first
  delay(50);
  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  ...
  config.pixel_format = PIXFORMAT_JPEG; 
  config.frame_size = frameSize;                
  config.jpeg_quality = jpegQuality;
  config.fb_count = 1;
  ...
  esp_err_t camerr = esp_camera_init(&config);  // initialize the camera
  if (camerr != ESP_OK) {
    dumbdisplay.logToSerial("ERROR: Camera init failed with error -- " + String(camerr));
  }
  resetCameraImageSettings();
  return (camerr == ESP_OK);
}

It is important that pixel_format be set to PIXFORMAT_JPEG.
Also notice that at the end of initializeCamera, resetCameraImageSettings is called to set the various settings of the Camera, which is also called when you change camera settings with the UI

bool resetCameraImageSettings() {
  sensor_t *s = esp_camera_sensor_get();
  if (s == NULL) {
    dumbdisplay.logToSerial("Error: problem reading camera sensor settings");
    return 0;
  }
  ...
  s->set_brightness(s, cameraBrightness);  // (-2 to 2) - set brightness
  s->set_quality(s, cameraQuality);        // set JPEG quality (0 to 63)
  ...
  return 1;
}

Concerning capturing snapshots, here is how an "offline" snapshot is taken when ESP32-CAM wakes up from sleep

void setup() {
  Serial.begin(115200);
  EEPROM.begin(32);
  initRestoreSettings();
  initializeStorage();
  ...
  if (wakeupOfflineSnapMillis != -1) {  // wakeupOfflineSnapMillis will be -1 if ESP is reset
    String localTime;
    long now = getTimeNow(&localTime);
    Serial.println("*** woke up for offline snap ... millis=" + localTime + " ***");
    framesize_t frameSize = cameraFrameSizes[currentFrameSizeIndex];
    if (initializeCamera(frameSize, cameraQuality)) { 
      delay(SLEEP_WAKEUP_TAKE_SNAP_DELAY_MS);
      if (true) {
        // it appears that the snap taken will look better if the following "empty" steps are taken        
        camera_fb_t* camera_fb = esp_camera_fb_get();
        esp_camera_fb_return(camera_fb);
      }
      camera_fb_t* camera_fb = esp_camera_fb_get();
      saveOfflineSnap(camera_fb->buf, camera_fb->len);
      esp_camera_fb_return(camera_fb);
    } else {
      Serial.println("failed to initialize camera for offline snapping");
    }
    Serial.println("*** going back to sleep ... timeout=" + String(wakeupOfflineSnapMillis / 1000.0) + "s ***");
    Serial.flush();
    esp_sleep_enable_timer_wakeup(wakeupOfflineSnapMillis * 1000);  // in micro second
    esp_deep_sleep_start();
    // !!! the above call will not return !!!
  }    
  ...
}

Notice:

the value of wakeupOfflineSnapMillis is used to determine whether the program is started due to normal boot up of wake up; it is set to the value of "how many milliseconds to sleep for next "offline" snap
certain, initializeCamera will be called to initialize the camera
after initializing the camera, the camera is given some time (2 seconds) to be ready
then the camera's buffer is retrieved by calling esp_camera_fb_get, which contains the bytes (JPEG bytes) to be saved by calling saveOfflineSnap
after saving the bytes, the buffered is returned by calling esp_camera_fb_return
at last, it goes back to sleep again

You will find that the tutorial Change ESP32-CAM OV2640 Camera Settings: Brightness, Resolution, Quality, Contrast, and More by Random Nerd Tutorials gives more details on ESP32-CAM.

Sketch Highlight -- DumbDisplay

Like all other use cases of using DumbDisplay, you first declare a global DumbDisplay object dumbdisplay

#if defined(BLUETOOTH)
  #include "esp32dumbdisplay.h"
  DumbDisplay dumbdisplay(new DDBluetoothSerialIO(BLUETOOTH));
#elif defined(WIFI_SSID)
  #include "wifidumbdisplay.h"
  DumbDisplay dumbdisplay(new DDWiFiServerIO(WIFI_SSID, WIFI_PASSWORD), DD_DEF_SEND_BUFFER_SIZE, 2 * DD_DEF_IDLE_TIMEOUT);
#else
  #include "dumbdisplay.h"
  DumbDisplay dumbdisplay(new DDInputOutput());
#endif

Then, several global helper objects / pointers are declared

DDMasterResetPassiveConnectionHelper pdd(dumbdisplay, true);
GraphicalDDLayer* imageLayer;
JoystickDDLayer* frameSliderLayer;
SelectionDDLayer* frameSizeSelectionLayer;
...
BasicDDTunnel* generalTunnel;

The life-cycle of the above DumbDisplay layers and "tunnels" are managed by the global pdd object, which will monitor connection and disconnection of
DumbDisplay app, calling appropriate C++ lambda expressions in the appropriate time. It is cooperatively given "time slices" in the loop() block like

void loop() {
  pdd.loop([]() {
    initializeDD();
  }, []() {
    if (updateDD(!pdd.firstUpdated())) {
       saveSettings();
    }
  }, []() {
    deinitializeDD();
  });
  if (pdd.isIdle()) {
    handleIdle(pdd.justBecameIdle());
  }
}

pdd.loop() accepts 3 C++ lambda expressions -- []() { ... }
- the 1st lambda expression is called when DumbDisplay app is connected, and DumbDisplay components need be initialized. Here initializeDD() is called for the purpose.
- the 2nd lambda expression is called during DumbDisplay app is connected to update the DumbDisplay components. Here, updatedDD() for the purpose. Note that pdd.firstUpdated() tells if this 2nd lambda expression was previously called after DumbDisplay component initialization (hence not of that means first call)
- the 3rd optional lambda expression is called when DumbDisplay app is disconnected. Here, deinitializeDD() is called for the purpose.
After pdd.loop() is the code that will be run whether DumbDisplay app is connected or not.
pdd.isIdle() tells if DumbDisplay is not connected (i.e. idle). In idle case, the function handleIdle() is called. Note that pdd.justBecameIdle() tells if DumbDisplay just disconnected making it idle.

The first time when the ESP32-CAM is connected, ESP32-CAM's clock is synchronized with that of your phone via generalTunnel -- a "general tunnel"

void initializeDD() {
  ...
  generalTunnel = dumbdisplay.createGeneralServiceTunnel();
  generalTunnel->reconnectTo(DD_CONNECT_FOR_GET_DATE_TIME);  // ask DumbDisplay app for current time, in order to set ESP's clock
  ...
}
...
bool updateDD(bool isFirstUpdate) {
  ...
  if (generalTunnel->pending()) {
    String response;
    if (generalTunnel->readLine(response)) {
      const String& endPoint = generalTunnel->getEndpoint();
      if (endPoint == DD_CONNECT_FOR_GET_DATE_TIME) {
        // got current time "feedback" from DumbDisplay app (initially requested)
        dumbdisplay.log("- got sync time " + response);
        DDDateTime dateTime;
        DDParseGetDataTimeResponse(response, dateTime, &tzMins);
        dumbdisplay.log("  . tz_mins=" + String(tzMins));
        Esp32SetDateTime(dateTime);
        ...
  ...
}

Other than the main scene / state, the UI has other scenes, like for selecting snapshots to save, and for transferring "offline" snapshots. When switching between the different scenes, the UI layers will be re-auto-pined by calling pinLayers

void pinLayers(State forState) {
  if (forState == STREAMING) {
    dumbdisplay.configAutoPin(DDAutoPinConfig('V')
      .beginGroup('H')
        .addLayer(frameSizeSelectionLayer)
        ...
        .addLayer(saveButtonLayer)
      .endGroup()  
      .build(), true);
  } else if (forState == CONFIRM_SAVE_SNAP) {
    dumbdisplay.configAutoPin(DDAutoPinConfig('V')
      .beginGroup('H')
        .addLayer(imageLayer)
        ...
      .endGroup()
      .build(), true);
  } else if (forState == TRANSFER_OFFLINE_SNAPS) {
    dumbdisplay.configAutoPin(DDAutoPinConfig('V')
      .addLayer(imageLayer)
      .addLayer(progressLayer)
      .addLayer(cancelButtonLayer)
      .build(), true);
  }
}

Selection and prompting for custom frame rate might be worth highlighting here. It is accomplished with the customFrameRateSelectionLayer layer

    fb = customFrameRateSelectionLayer->getFeedback();
    if (fb != NULL) {
      if (fb->type == CUSTOM) {
        int phFrameRate = fb->text.isEmpty() ? customCachePHFrameRate : fb->text.toInt();
        if (phFrameRate >= 1 && phFrameRate <= 3600) {
          customCachePHFrameRate = phFrameRate;
          ...
        } else {
          dumbdisplay.log("invalid custom frame rate!", true);
        }
      } else {
        customFrameRateSelectionLayer->explicitFeedback(0, 0, "'per-hour' frame rate; e.g. " + String(customCachePHFrameRate) , CUSTOM, "numkeys");
      }
    }

normally the type of "feedback" (fb->type) from the UI will not be CUSTOM
in such a non-CUSTOM case (i.e. "feedback" is because of clicking), call customFrameRateSelectionLayer->explicitFeedback like above
- explicitFeedback will explicitly fake a "feedback" to the layer
- but this time, the type of "feedback" is CUSTOM -- the 4th parameter
- the first 3 parameters are for x, y, and text of the "feedback"
- the last parameters -- option to explicitFeedback numkeys -- means that during the "feedback" routing, you will be prompted for numeric value with your phone's virtual keyboard, which will then replace the text of the "feedback"
- note that the initial text passed in to explicitFeedback is used as the "hint" on the virtual keyboard
in CUSTOM case, fb->text will be the value you entered

Another opportunity of prompting is confirmation for transferring "offline" snapshots to your phone. It is accomplished via generalTunnel like

bool updateDD(bool isFirstUpdate) {
  ...
  if (generalTunnel->pending()) {
    String response;
    if (generalTunnel->readLine(response)) {
      const String& endPoint = generalTunnel->getEndpoint();
      if (endPoint == DD_CONNECT_FOR_GET_DATE_TIME) {
        // got current time "feedback" from DumbDisplay app (initially requested)
        ...
        if (state == STREAMING && offlineSnapCount > 0) {
          generalTunnel->reconnectTo("confirm?title=Transfer Snaps&message=There are " + String(offlineSnapCount) + " offline snaps. Transfer them%3F&ok=Yes&cancel=No");
        }
#endif
      } else {
        if (state == STREAMING && response == "Yes") {
          setStateToTransferOfflineSnaps();  // assume the response if from the "confirm" dialog [for transferring offline snaps]
        }
        ...
      }
    }
  ...
}

Right after gotten the "sync time" from DumbDisplay app, and when need to prompt for "yes/no" confirmation for transferring "offline" snaps, generalTunnel->reconnectTo is called to handle the "confirm" request.
The "Yes" or "No" response will come back through generalTunnel again (the same way as DD_CONNECT_FOR_GET_DATE_TIME)

After the "offline" transfer, an "alert" is popped up indicating the transfer is done. This is done by calling dumbdispaly.alert

bool updateDD(bool isFirstUpdate) {
  ...
  if (state == TRANSFER_OFFLINE_SNAPS) {
    if (offlineSnapCount == 0 || clickedCancelButton) {
      ...
      dumbdisplay.alert("Transferred " + String(startTransferOfflineSnapCount - offlineSnapCount) + " offline snaps!");
      ...
  }
  ...
}

Build and Upload

Build, upload the sketch and try it out!

If it interests you, you may use a Serial monitor (set to baud-rate 115200) to observe when things are happening

Initialize offline snap storage ...
... offlineSnapCount=3 ...
... existing STORAGE ...
    $ total: 896 KB
    $ used: 180 KB
    $ free: 716 KB (79.91%)
... offline snap storage ...
    - startOfflineSnapIdx=0
    - offlineSnapCount=3
... offline snap storage initialized
bluetooth address: 84:0D:8E:D2:90:EE
*** just became idle ... millis=1067 ***
bluetooth address: 84:0D:8E:D2:90:EE
bluetooth address: 84:0D:8E:D2:90:EE
bluetooth address: 84:0D:8E:D2:90:EE
...
**********
* _SendBufferSize=128
**********
*** just connected ... millis=7133 ***
- got sync time 2024-09-07-10-17-41-+0800
  . tz_mins=480
Initializing camera ...
... initialized camera!
...
*** just became idle ... millis=93898(Saturday, September 07 2024 10:19:07+0800) ***
bluetooth address: 84:0D:8E:D2:90:EE
! written offline snap to [/off_000.jpg] ... size=48.39KB
bluetooth address: 84:0D:8E:D2:90:EE
bluetooth address: 84:0D:8E:D2:90:EE
...
*** going to sleep ... millis=153898(Saturday, September 07 2024 10:20:07+0800) ***
...
Initialize offline snap storage ...
... offlineSnapCount=2 ...
... existing STORAGE ...
    $ total: 896 KB
    $ used: 112 KB
    $ free: 784 KB (87.50%)
... offline snap storage ...
    - startOfflineSnapIdx=0
    - offlineSnapCount=2
... offline snap storage initialized
*** woke up for offline snap ... millis=542(Saturday, September 07 2024 10:20:33+0800) ***
! written offline snap to [/off_002.jpg] ... size=78.91KB
*** going back to sleep ... timeout=27.20s ***
...

Enjoy!

Hope that you will have fun with it! Enjoy!

Simple Arduino Framework Photo Frame Implementation with Photos Downloaded from the Internet via DumbDisplay

Trevor Lee — Mon, 08 Jul 2024 10:06:04 +0000

Simple Arduino Framework Raspberry Pi Pico / ESP32 TFT LCD Photo Frame Implementation with Photos Downloaded from the Internet via DumbDisplay

The target of this project is to implement, mostly the software part, a simple Arduino framework photos / images showing "photo frame" using Raspberry Pi Pico or ESP32 with photos / images downloaded from the Internet via DumbDisplay -- an Android app running on your Android phone.

The microcontroller program here is developed in Arduino framework using VS Code and PlatformIO, in the similar fashion as described by the post -- A Way to Run Arduino Sketch With VSCode PlatformIO Directly

The simple remote UI for downloading photos / images from the Internet is realized with the help of the DumbDisplay Android app. For a brief description of DumbDisplay, you may want to refer to the post -- Blink Test With Virtual Display, DumbDisplay

Please note that the UI is driven by the microcontroller program; i.e., the control flow of the UI is programmed in the sketch. Additionally, please note that the downloaded image, be it in Jpeg or PNG format, will be transferred to the microcontroller board in Jpeg format, scaled to site inside the TFT LCD screen.

For Raspberry Pi Pico board (WiFi), a ST7789 2.8 inch 240x320 SPI TFT LCD screen is attached to a Raspberry Pi Pico board.
The TFT LCD module library used is the Adafruit-ST7735-Library Arduino library.

For ESP32, LiLyGo TDisplay / TCamera Plus board is used.
The TFT LCD module library use is the bodmer/TFT_eSPI Arduino library.

In all cases, the Jpeg library used is the bodmer/TJpg_Decoder Arduino library.

A simple flash-based LittleFS file-system is allocated for storing the saved Jpeg images.

The microcontroller board has two running modes:

1) When connected to the DumbDisplay Android app (using WiFi), a simple UI is provided for downloading images from some predefined sites,
as well as for transferring the downloaded image in Jpeg format to the microcontroller board.
Note that the predefined sites is hardcoded in the sketch that you can conveniently change as desired by changing the sketch.
2) When not connected to the DumbDisplay Android app, the microcontroller cycles through the saved Jpeg images displaying them to the TFT LCD
screen one by one like a simple "photo frame". Note that since the images are stored in LittleFS, they will survive even after reboot of the
microcontroller board.

Connect for the UI; disconnect to enjoy "photo frame" slide show.

The UI

The first time connected, an image download will be initiated automatically.
After downloading an image the image will be transferred to the microcontroller board in Jpeg format and be displayed to the TFT LCD screen.

You can choose to save the transferred image by clicking the 💾Save button.
Notice that the [7-segment] number displayed next to the Saved🗂️ label will be bumped up after saving.
The number indicates the number of images saved to the microcontroller's LittleFS storage.

If you so desired, you can turn on auto-save by clicking the Auto save button. (You turn off auto-save by clicking the button again.)

If you want to initiate another download of image, click on the canvas that shows the downloaded image.

If you want to delete all the saved images, double-click on the [7-segment] number.

After you are done with downloading and saving images, disconnect the microcontroller.
After disconnection, the "photo frame" slide show begins on the microcontroller side.

Anytime you want to change the saved images, reconnect to DumbDisplay Android app.

Connect for the UI; disconnect to enjoy "photo frame" slide show.

Wiring TFT LCD Module

For LiLyGo TDisplay / TCamera Plus board, the TFT LCD screen is in-built the microcontroller board; hence, no need for additional wiring.

For Raspberry Pi Pico board, as mentioned previously, a ST7789 2.8 inch 240x320 SPI TFT LCD module is used; hence, some wiring is necessary

Raspberry Pi Pico	SPI TFT LCD
3V3	VCC
GND	GND
GP21	BL
GP17	CS
GP16	RS / DC
GP18	CLK / SCLK
GP19	SDA / MOSI
GP20	RST

Developing and Building

As mentioned previously, the sketch will be developed using VS Code and PlatformIO.
Please clone the PlatformIO project TFTImageShow GitHub repository.

The configurations for developing and building of the sketch in basically captured in the platformio.ini file

[env]
monitor_speed = 115200

[env:PICOW]  ; ensure long file name support ... git config --system core.longpaths true
platform = https://github.com/maxgerhardt/platform-raspberrypi.git
board = rpipicow
framework = arduino
board_build.core = earlephilhower
board_build.filesystem = littlefs
board_build.filesystem_size = 1m
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay
    https://github.com/adafruit/Adafruit-ST7735-Library.git
    https://github.com/adafruit/Adafruit-GFX-Library
    https://github.com/Bodmer/TJpg_Decoder.git 
    Wire
    SPI
    https://github.com/adafruit/Adafruit_BusIO 
build_flags =
    -D FOR_PICOW

[env:TDISPLAY]
platform = espressif32
board = esp32dev
framework = arduino
board_build.filesystem = littlefs
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay
    bodmer/TFT_eSPI     ; Setup25_TTGO_T_Display
    bodmer/TJpg_Decoder
    LittleFS
build_flags =
    -D FOR_TDISPLAY

[env:TCAMERAPLUS]
platform = espressif32
board = esp32dev
framework = arduino
board_build.filesystem = littlefs
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay
    bodmer/TFT_eSPI      ; modify User_Setup_Select.h ... Setup44_TTGO_CameraPlus
    bodmer/TJpg_Decoder
    LittleFS
    Wire
    SPI
    SPIFFS
build_flags =
    -D FOR_TCAMERAPLUS

Please make sure you select the correct PlatformIO project environment -- PICOW / TDISPLAY / TCAMERAPLUS

For PICOW, the platform core is download from https://github.com/maxgerhardt/platform-raspberrypi.git.
(As far as I know, this is the only PlatformIO platform core that supports the use of Raspberry Pi PicoW WiFi capability.)

It might take a long time for PlatformIO to download and install it.
If PlatformIO fails to download and install the platform core, it might be that your system doesn't have long "file name" enabled, in such a case, try

git config --system core.longpaths true

For TDISPLAY that uses bodmer/TFT_eSPI, you will need to modify the installed .pio/libdeps/TDISPLAY/TFT_eSPI/User_Set_Select.h
to use User_Setups/Setup25_TTGO_T_Display.h rather than the default User_Setup.h like

...
//#include <User_Setup.h>           // Default setup is root library folder
...
#include <User_Setups/Setup25_TTGO_T_Display.h>    // Setup file for ESP32 and TTGO T-Display ST7789V SPI bus TFT
...

For TCAMERAPLUS which also uses bodmer/TFT_eSPI, modify User_Set_Select.h similarly

...
//#include <User_Setup.h>           // Default setup is root library folder
...
#include <User_Setups/Setup44_TTGO_CameraPlus.h>   // Setup file for ESP32 and TTGO T-CameraPlus ST7789 SPI bus TFT    240x240
...

The program entry point is src/main.cpp

// ***
// the below _secret.h just define macros like:
// #define WIFI_SSID           "your-wifi-ssid"
// #define WIFI_PASSWORD       "your-wifi-password"
// ***
#include "_secret.h"

#include "tft_image_show/tft_image_show.ino"

Notice there are two included files -- _secret.h and tft_image_show/tft_image_show.ino -- in the src directory

You will need to create the _secret.h with content like

#define WIFI_SSID           "your-wifi-ssid"
#define WIFI_PASSWORD       "your-wifi-password"

With these macros for accessing your WiFi, the microcontroller board will connect to DumbDisplay Android app using WiFi.
If you do not want to use WiFi, simply don't provide them.
In such a case, connection to DumbDisplay Android app is assumed to be using serial UART (slower) via an OTG adapter.
Please refer to the above mentioned post -- Blink Test With Virtual Display, DumbDisplay

The Sketch

The sketch of the project is tft_image_show/tft_image_show.ino. You can [easily] customize some aspects of the sketch

...
// NEXT_S defines the delay (in seconds) to show next saved image
#define NEXT_S              5
...
// MAX_IMAGE_COUNT define that maximum number of images that can be saved
// set MAX_IMAGE_COUNT to 0 to force reformat the storage
#define MAX_IMAGE_COUNT     10
...
// getDownloadImageURL() returns a URL to download an image; add / remove sites as needed
// download image bigger than needed (on purpose)
const String urls[] = {
  String("https://loremflickr.com/") + String(2 * TFT_WIDTH) + String("/") + String(2 * TFT_HEIGHT),
  String("https://picsum.photos/") + String(2 * TFT_WIDTH) + String("/") + String(2 * TFT_HEIGHT),
};
const char* getDownloadImageURL() {
  int idx = random(2);
  return urls[idx].c_str();
}
...

The slide show delay is defined by the macro NEXT_S, which default to 5 seconds
The maximum number of saved images is defined by the macro MAX_IMAGE_COUNT, which default to 10. Note that if you set MAX_IMAGE_COUNT to 0, flash and run the sketch, the LittleFS storage will be reformatted. For normal running, MAX_IMAGE_COUNT should be at lease 1.
You can modify urls / getDownloadImageURL() to add / remove Internet sites for downloading images.

Sketch Highlight -- TFT LCD Library `Adafruit-ST7735-Library`

Here is how Adafruit-ST7735-Library used in the sketch.

First a global tft object is defined like

  #define A_TFT_BL    21
  #define A_TFT_CS    17
  #define A_TFT_DC    16
  #define A_TFT_SCLK  18
  #define A_TFT_MOSI  19
  #define A_TFT_RST   20
  #define TFT_WIDTH   320
  #define TFT_HEIGHT  240
  #include <Adafruit_ST7789.h>
  Adafruit_ST7789 tft(A_TFT_CS, A_TFT_DC, A_TFT_RST);

Notice the pin assignments exactly match the wiring described previously.

Then in setup()

  pinMode(A_TFT_BL, OUTPUT);
  digitalWrite(A_TFT_BL, 1);  // light it up
  tft.init(240, 320, SPI_MODE0);
  tft.invertDisplay(false);
  tft.setRotation(1);
  tft.setSPISpeed(40000000);

The back-light part is obvious.
The TFT LCD screen size is 240x320.
Why SPI_MODE0 and other settings? Simply, they work for me.

Sketch Highlight -- TFT LCD Library `bodmer/TFT_eSPI`

Here is how bodmer/TFT_eSPI used in the sketch.

First, a global tft object is defined like

  #include <TFT_eSPI.h>
  TFT_eSPI tft = TFT_eSPI();

Then in setup()

  tft.init();
  tft.setRotation(0);

Sketch Highlight -- Jpeg Library `bodmer/TJpg_Decoder`

You might be wondering why use Jpeg but not RGB565 directly. Simply because of the very high data compression ratio of Jpeg.

Anyway, here is how bodmer/TJpg_Decoder used in the sketch.

Include the needed headers

#include <TJpg_Decoder.h>

In setup()

#if defined(TFT_ESPI_VERSION)
  TJpgDec.setSwapBytes(true);
#endif
  TJpgDec.setCallback(tft_output);

Why setSwapBytes(true)? Since it seems to work that way.

And here is the callback tft_output, which is mostly copied from an example of bodmer/TJpg_Decoder

bool tft_output(int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t* bitmap) {
  // Stop further decoding as image is running off bottom of screen
  if ( y >= tft.height() ) return 0;
  // This function will clip the image block rendering automatically at the TFT boundaries
#if defined(TFT_ESPI_VERSION)
  tft.pushRect(x, y, w, h, bitmap);
#else
  tft.drawRGBBitmap(x, y, bitmap, w, h);
#endif
  // Return 1 to decode next block
  return 1;
}

Notice that in case of using bodmer/TFT_eSPI, the way to draw the decoded Jpeg chunk is

  tft.pushRect(x, y, w, h, bitmap);

If Adafruit-ST7735-Library is use, the way to draw the decoded Jpeg chunk is

  tft.drawRGBBitmap(x, y, bitmap, w, h);

After setting TJpgDec up, a Jpeg image can be drawn like

  TJpgDec.drawJpg(x, y, jpegImage.bytes, jpegImage.byteCount);

Sketch Highlight -- LittleFS

Include the needed headers

#include <FS.h>
#include <LittleFS.h>

In setup()

 LittleFS.begin();

If you want to format the LittleFS, call format() like

  LittleFS.format()

Jpeg image can be saved like

  File f = LittleFS.open(fileName, "w");
  if (f) {
    f.println(currentJpegImage.width);
    f.println(currentJpegImage.height);
    f.println(currentJpegImage.byteCount);
    f.write(currentJpegImage.bytes, currentJpegImage.byteCount);
    f.close();
  }

Notice that not only the Jpeg image bytes got written to the file, the various metadata are saved first. (I.e. the file is not a normal JPG file, but a customized one.)

And Jpeg image can be read back like

  File f = LittleFS.open(fileName, "r");
  if (f) {
    int width = f.readStringUntil('\n').toInt();
    int height = f.readStringUntil('\n').toInt();
    int byteCount = f.readStringUntil('\n').toInt();
    uint8_t* bytes = new uint8_t[byteCount];
    f.readBytes((char*) bytes, byteCount);
    f.close();
    tempImage.width = width;
    tempImage.height = height;
    tempImage.byteCount = byteCount;
    tempImage.bytes = bytes;
  }

Sketch Highlight -- DumbDisplay

Like all other use cases of using DumbDisplay, you first declare a global DumbDisplay object dumbdisplay

#if defined(WIFI_SSID)
  #include "wifidumbdisplay.h"
  DumbDisplay dumbdisplay(new DDWiFiServerIO(WIFI_SSID, WIFI_PASSWORD));
#else
  #include "dumbdisplay.h"
  DumbDisplay dumbdisplay(new DDInputOutput());
#endif

There, the new configured DDWiFiServerIO is one of the few ways to establish connection with DumbDisplay Android app.
Like in "no WiFi" case, Serial UART DDInputOutput is used.

Then, several global helper objects / pointers are declared

DDMasterResetPassiveConnectionHelper pdd(dumbdisplay);
GraphicalDDLayer* imageLayer;
LcdDDLayer* saveButtonLayer;
LcdDDLayer* autoSaveOptionLayer;
LcdDDLayer* savedCountLabelLayer;
SevenSegmentRowDDLayer* savedCountLayer;
SimpleToolDDTunnel* webImageTunnel;
ImageRetrieverDDTunnel* imageRetrieverTunnel = NULL;

The DDMasterResetPassiveConnectionHelper global object pdd is a helper for managing connect and reconnection with DumbDisplay app.
The GraphicalDDLayer pointer imageLayer is the canvas to which downloaded image is drawn to. Like other layers, they the layer is created later, in this case, when DumbDisplay app is created.
The LcdDDLayer pointers saveButtonLayer, autoSaveOptionLayer and savedCountLabelLayer are for save button, auto-save button, and saved-count label respectively.
The SevenSegmentRowDDLayer pointer savedCountLayer is for show the saved image count.
The SimpleToolDDTunnel pointer webImageTunnel is for download image to DumbDisplay Android app purpose. It will be created together with other layers and tunnels.
The ImageRetrieverDDTunnel pointer imageRetrieverTunnel is for retrieving the data of the downloaded image, in Jpeg format.

The life-cycle of the above DumbDisplay layers and "tunnels" are managed by the global pdd object, which will monitor connection and disconnection of
DumbDisplay app, calling appropriate user-defined functions as well as DumbDisplay functions in the appropriate time. It is cooperatively given "time slices" in the loop() block like

  void loop() {
    ...
    pdd.loop(initializeDD, updateDD);
    ...
  }

The initializeDD is the function defined in the sketch that is supposed to create the various layer and tunnel objects.

void initializeDD() {
  tft.fillScreen(COLOR_BG);
  // create a graphical layer for drawing the downloaded web image to
  imageLayer = dumbdisplay.createGraphicalLayer(2 * TFT_WIDTH, 2 * TFT_HEIGHT);
  ...
  // create a LCD layer for the save button
  saveButtonLayer = dumbdisplay.createLcdLayer(6, 2);
  ...
  // create a LCD layer for the auto save option
  autoSaveOptionLayer = dumbdisplay.createLcdLayer(6, 1);
  ...
  // create a LCD layer as the label for the number of saved images
  savedCountLabelLayer = dumbdisplay.createLcdLayer(8, 1);
  ...
  // create a 7-segment layer for showing the number of saved images
  savedCountLayer = dumbdisplay.create7SegmentRowLayer(2);
  ...
  // create a tunnel for downloading web image ... initially, no URL yet ... downloaded.png is the name of the image to save
  webImageTunnel = dumbdisplay.createImageDownloadTunnel("", "downloaded.png");
  ...
  // create a tunnel for retrieving JPEG image data from DumbDisplay app storage
  imageRetrieverTunnel = dumbdisplay.createImageRetrieverTunnel();
  // auto pin the layers
  dumbdisplay.configAutoPin(DDAutoPinConfig('V')
    .addLayer(imageLayer)
    .beginGroup('H')
      ...
    .endGroup()
    .build());
}

The updateDD is the function define in the sketch that is supposed to receive "time slices" to update / act-on the layer and tunnel objects.

  bool isFirstUpdate = !pdd.firstUpdated();
  bool updateUI = isFirstUpdate;
  if (autoSaveOptionLayer->getFeedback() != NULL) {
    // toggle auto save
    autoSave = !autoSave;
    updateUI = true;
  }
  if (updateUI) {
    if (autoSave) {
      autoSaveOptionLayer->writeLine("Auto✅️"); 
    } else {
      autoSaveOptionLayer->writeLine("Auto⛔");
    }
  }
  ...
  if (isFirstUpdate || state == NOTHING) {
    if (isFirstUpdate || imageLayer->getFeedback() != NULL) {
      // trigger download image
      saveButtonLayer->disabled(true);
      imageLayer->noBackgroundColor();
      state = DOWNLOADING_FOR_IMAGE;
    }
    return;
  }
  if (state == DOWNLOADING_FOR_IMAGE) {
    // set the URL to download web image 
    currentJpegImage.release();
    String url = getDownloadImageURL();
    webImageTunnel->reconnectTo(url);
    imageLayer->clear();
    imageLayer->write("downloading image ...");
    state = WAITING_FOR_IMAGE_DOWNLOADED;
    return;
  }
  if (state == WAITING_FOR_IMAGE_DOWNLOADED) {
    int result = webImageTunnel->checkResult();
    if (result == 1) {
      // web image downloaded ... retrieve JPEG data of the image
      imageRetrieverTunnel->reconnectForJpegImage("downloaded.png", TFT_WIDTH, TFT_HEIGHT);
      imageLayer->clear();
      imageLayer->drawImageFileFit("downloaded.png");
      state = RETRIEVING_IMAGE;
      retrieveStartMillis = millis();
    } else if (result == -1) {
      // failed to download the image
      imageLayer->clear();
      imageLayer->write("!!! failed to download image !!!");
      dumbdisplay.writeComment("XXX failed to download XXX");
      state = NOTHING;
    }
    return;
  }
  if (state == RETRIEVING_IMAGE) {
    // read the retrieve image (if it is available)
    DDJpegImage jpegImage;
    bool retrievedImage = imageRetrieverTunnel->readJpegImage(jpegImage);
    if (retrievedImage) {
      unsigned long retrieveTakenMillis = millis() - retrieveStartMillis;
      dumbdisplay.writeComment(String("* ") + jpegImage.width + "x" + jpegImage.height + " (" + String(jpegImage.byteCount / 1024.0) + " KB) in " + String(retrieveTakenMillis / 1000.0) + "s");
      if (jpegImage.isValid()) {
        ...
      } else {
        ...
      }
      ...
      state = NOTHING;
    }   
    return
  }

Notice

how layer "feedback" (e.g. clicking) is received using getFeedback()
how the webImageTunnel "tunnel" is used to download image with call to reconnectTo()
how the imageRetrieverTunnel "tunnel" is used to initiate retrieving of download image data with call to reconnectForJpegImage()
how the image data (DDJpegImage) is received via the tunnel imageRetrieverTunnel with call to readJpegImage()

The whole updateDD basically is a "state-machine" that handles the different states (state) of the UI processing:

NOTHING -- just started, or finished download / saving of image; will wait for imageLayer being clicked to initiate an image
DOWNLOADING_FOR_IMAGE -- this state could have been merged with previous stage; anyway, it reconnects webImageTunnel to activate an image download
WAITING_FOR_IMAGE_DOWNLOADED -- waiting for download image to complete; then will retrieve the download image to be displayed to the TFT LCD screen
RETRIEVING_IMAGE -- retrieving the download image data to be transferred to the microcontroller; once retrieved, display the image to the TFT LCD screen

The slide show is carried out when "idle" (not connected to DumbDisplay app)

void loop() {
  pdd.loop(initializeDD, updateDD);
  if (pdd.isIdle()) {
    if (pdd.justBecameIdle()) {
      // re-start slide show
      ...
    }
    unsigned long now = millis();
    if (now >= nextMillis) {
      if (MAX_IMAGE_COUNT > 0 && savedImageCount > 0) {
        ...
      } else {
        ...
      }
      ...
    }
  }
}

Build and Upload

Build, upload the sketch and try it out!

For WiFi connectivity, you will need to find out the IP address of the microcontroller board. Simply connect the microcontroller board with a Serial monitor (set to use baud rate 115200), you should see lines like

binded WIFI TrevorWireless
listening on 192.168.0.218:10201 ...
listening on 192.168.0.218:10201 ...

See that the IP address of the microcontroller board is printed out.

On DumbDisplay Android app side


You will need the microcontroller's IP address to configure DumbDisplay Android app to connect it with your microcontroller

Start the DumbDisplay app.
Click on the Establish Connection icon.
In the "establish connection" dialog, you should see the "add WIFI device" icon at the bottom right of the dialog. Click on it.
A popup for you to enter WIFI IP will be shown. Enter the IP address of your ESP board as Network Host. Click OK when done.
Back to the "establish connection" dialog, a new entry will be added, click on it to establish WIFI connection.

Have fun with it!

Enjoy!

Implement a Simple Calculator Android App by Reusing Logics in Rust via JavaScript-WASM Interfacing

Trevor Lee — Tue, 14 May 2024 16:33:25 +0000

Implement a Simple Calculator Android App by Reusing Logics in Rust via JavaScript-WASM Interfacing

As a followup of my previous work -- Implement a Simple WASM Calculator in Rust Using Leptos, and with DumbCalculator --
this time, I would like to explore a not-elegant-but-work-for-me way to reuse the logics implemented in Rust, without going through the trouble rewriting the core using Kotlin.

The idea is to use JavaScript as the "bridge" between the Android app and the WASM Rust code, which is
largely realized with the help of DumbCalculator of rusty_dumb_tools.

The binding of Rust (WASM) and JavaScript is realized with the help of wasm-bindgen and wasm-pack -- https://github.com/rustwasm/wasm-bindgen/tree/main/examples/without-a-bundler
You can refer to wasm-pack Quickstart for instruction on installing wasm-pack
For Android app to interact with JavaScript (web page), Android's WebView is the key enabler
You can refer to Install Android Studio for instruction on installing Android Studio
Other than Android Studio, VSCode is used for developing the Rust code

Starting the Project

Since it will be an Android App -- ACalculatorApp -- naturally, we will create an Android Studio project for it.
Moreover, inside the project, we will be creating a little Rust project for developing the JavaScript "bridge".

Simply, create an Android Studio project ACalculatorApp

Initialize for the JavaScript-Rust "Bridge"

To start coding for the JavaScript-Rust "bridge":

Open the created folder ACalculatorApp with VSCode
Create the folder rust (inside ACalculatorApp)

In the folder rust, create Cargo.toml

[package]
name = "dumb_calculator"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
wasm-bindgen = "0.2.92"
rusty_dumb_tools = "0.1.13"

[dependencies.web-sys]
version = "0.3.4"
    features = [
    'Document',
    'Element',
    'HtmlElement',
    'Node',
    'Window',
]

Note that in order for wasm-bindgen to work, the following configurations are required

crate-type = ["cdylib"]
[dependencies.web-sys]
wasm-bindgen = "0.2.92"

In the folder rust, create src/lib.rc

use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn get_greeting(who: String) -> String {
    format!("Hello, {}!", who)
}

In the folder rust, create simple.html

<script type="module">
  import init, { get_greeting } from './pkg/dumb_calculator.js';
  async function load() {
      await init();
      window.get_greeting = get_greeting;
  }
  load();
</script>
<button onclick="document.getElementById('msg').innerText+=get_greeting('World')">GREET</button>
<div id="msg"></div>

Try build the Rust code

Open a terminal to rust and run

  wasm-pack build --target web

This will generate the output folders target and pkg

Start Live Server VSCode extension
- visit localhost:5501/rust/simple.html
- click the GREET button

An alternative is to use Python's http.server; in rust, open a terminal and run

  python -m http.server

visit localhost:8000/simple.html

Key Takeaways of the JavaScript-WASM Bridge

Rust functions are exposed simply by annotating them like

use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn get_greeting(who: String) -> String {
  ...
}

Cargo.toml requires some special specifications
- crate-type = ["cdylib"]
- [dependencies.web-sys]
- wasm-bindgen = "0.2.92"
Building not with cargo, but with wasm-pack like

wasm-pack build --target web

HTML page that loads the WASM Rust exposed needs be in "module" like

<script type="module">
  import init, { get_greeting } from './pkg/dumb_calculator.js';
  async function load() {
      await init();
      window.get_greeting = get_greeting;
  }
  load();
</script>

Notice:

the load() async function, which calls init generated
load() invoked explicitly as the last thing of the module
after load, assign the exposed -- get_greeting in this case -- to window so that it can be accessed outside of the "module"

Android Calling the JavaScript-WASM "Bridge"

The key enabled is an Android WebView. With Jetpack Compose, it can be programmed like

(as in simple package)

val ENDPOINT: String = "http://192.168.0.17:8000/simple.html"
class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            SimpleBridgeWebView()
        }
    }
}
@Composable
fun SimpleBridgeWebView(modifier: Modifier = Modifier) {
    AndroidView(
        factory = { context ->
            WebView(context).apply {
                this.settings.javaScriptEnabled = true
                this.webViewClient = WebViewClient()
                this.loadUrl(ENDPOINT)
            }
        },
        update = {}
    )
}

Notice:

WebView is wrapped with an AndroidView
javaScriptEnabled, but currently no JavaScript is involved
loadUrl is called to load the "bridge" (web page) when the WebView is created

Important Notes:

you should change the IP and port in ENDPOINT to yours
you may get into "firewall" issue; if so, be suggested to try to use Python's http.server to serve the "bridge" since very likely your Python installation already has firewall access setup

More importantly, modify Android permission settings in AndroidManifest.xml:

allow access to the Internet:

<manifest ...>
    <uses-permission android:name="android.permission.INTERNET" />

allow WebView "clear text" traffic

<application ...
    android:usesCleartextTraffic="true"

Reminder: this repository already includes the above code in the simple package, you can use that MainActivity simply by changing AndroidManifest.xml like

    ...
    <activity
        android:name=".simple.MainActivity"
        ...

Build and run the Android app, and see that the "bridge" loads and is working

Package the "Bridge" with the App

It is possible to package the "bridge" in the app's PKG. To do so, we will need to put everything of the "bridge" to the assets folder like

Android Studio	VSCode

To copy the "bridge" over to assets, after building it by running

wasm-pack build --target web

additionally run

mkdir ../app/src/main/assets
mkdir ../app/src/main/assets/bridge
cp bridge.html simple.html ../app/src/main/assets/bridge/
cp -r pkg ../app/src/main/assets/bridge/pkg

In order to make things easier, create a rust/build.sh like

set -ex
wasm-pack build --target web
cp *.html ../app/src/main/assets/bridge/
cp -r pkg ../app/src/main/assets/bridge/pkg

Now, every time want to build the "bridge", in rust run build.sh

As for the Android app side, somethings need be changed
(as in internal package)

class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            SimpleInternBridgeWebView()
        }
    }
}
@Composable
fun SimpleInternBridgeWebView(modifier: Modifier = Modifier) {
    AndroidView(
        factory = { context ->
            val assetLoader = WebViewAssetLoader.Builder()
                .addPathHandler("/assets/", WebViewAssetLoader.AssetsPathHandler(context))
                .build()
            WebView(context).apply {
                this.settings.javaScriptEnabled = true
                this.webViewClient = object : WebViewClient() {
                    override fun shouldInterceptRequest(
                        view: WebView,
                        request: WebResourceRequest
                    ): WebResourceResponse? {
                        return assetLoader.shouldInterceptRequest(request.url)
                    }
                }
                this.loadUrl("https://appassets.androidplatform.net/assets/bridge/simple.html")
            }
        },
        update = {}
    )
}

Notes:

assetLoader is used to act as a web server that serves web contents from assets
shouldInterceptRequest is intercepted to call assetLoader
the URL is now like https://appassets.androidplatform.net/assets/bridge/simple.html

Again, build and run the Android app, and see that the "bridge" loads from assets and is working

Add Some Jetpack Compose Code to Call `get_greeting`

The first thing to realize is that we need to get hold of the WebView in order to be able to call its special tailored methods

To achieve this, we need to refactor the code a bit so that the WebView is created explicitly outside of SimpleInternBridgeWebView like

...
    setContent {
        val webView = createSimpleInternBridgeWebView(this)
        SimpleInternBridgeWebView(webView)
    }
...

With the WebView (webView), we call its evaluateJavascript method to invoke the "bridge" asynchronously like

    webView.evaluateJavascript("get_greeting('Android')") {
        greeting.value = it
    }

And here is the code
(as in internal_ui package)

class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            val webView = createSimpleInternBridgeWebView(this)
            Column() {
                val greeting = remember { mutableStateOf("...") }
                SimpleInternBridgeWebView(webView)
                Text(text = greeting.value)
                Button(onClick = {
                    webView.evaluateJavascript("get_greeting('Android')") {
                        greeting.value = it
                    }
                }) {
                    Text("Get Greeting")
                }
            }
        }
    }
}
@Composable
fun SimpleInternBridgeWebView(webView: WebView, modifier: Modifier = Modifier) {
    AndroidView(
        factory = { context -> webView },
        update = { webView.loadUrl("https://appassets.androidplatform.net/assets/bridge/simple.html") }
    )
}
fun createSimpleInternBridgeWebView(context: Context): WebView {
    val assetLoader = WebViewAssetLoader.Builder()
        .addPathHandler("/assets/", WebViewAssetLoader.AssetsPathHandler(context))
        .build()
    return WebView(context).apply {
        this.settings.javaScriptEnabled = true
        this.webViewClient = object : WebViewClient() {
            override fun shouldInterceptRequest(
                view: WebView,
                request: WebResourceRequest
            ): WebResourceResponse? {
                return assetLoader.shouldInterceptRequest(request.url)
            }
        }
        this.loadUrl("https://appassets.androidplatform.net/assets/bridge/simple.html")
    }
}

Lets Bring Some Calculator Code into the Picture

As mentioned previously, the Android App will be a simple calculator with core implementation in Rust with the help of DumbCalculator. Now, lets bring DumbCalculator into the picture and make change to the code lib.rs

use wasm_bindgen::prelude::*;
use std::{cell::RefCell, mem::{self, MaybeUninit}, sync::Once};
use rusty_dumb_tools::calculator::DumbCalculator;

#[wasm_bindgen]
pub fn get_greeting(who: String) -> String {
    format!("Hello, {}!", who)
}

#[wasm_bindgen]
struct Calculator {
    display_width: usize,
    calculator: DumbCalculator,
}
#[wasm_bindgen]
impl Calculator {
    pub fn new(display_width: u8) -> Calculator {
        Calculator {
            display_width: display_width as usize,
            calculator: DumbCalculator::new(),
        }
    }
    pub fn push(&mut self, key: &str) {
        self.calculator.push(key).unwrap();
    }
    pub fn get_display(&self) -> String {
        self.calculator.get_display_sized(self.display_width)
    }
}

Notice:

Now, we have a Calculator class to expose
Again, in order to expose, simply annotate the struct as well as the impl with #[wasm_bindgen]

And we use another "bridge" -- simple_calculator.html

<script type="module">
  import init, { Calculator } from './pkg/dumb_calculator.js';
  async function load() {
      await init();
      window.Calculator = Calculator;
  }
  load();
</script>
<script>
  function new_calc() {
    calc = Calculator.new(5);
    _sync_calc();
  }
  function _sync_calc() {
    let display = calc.get_display();
    let elem = document.getElementById('msg');
    elem.innerText = display;
  }
</script>
<div id="buttons">
  <button onclick="new_calc()">new</button>
  <button onclick="calc.push('1'); _sync_calc();">1</button>
  <button onclick="calc.push('+'); _sync_calc();">+</button>
  <button onclick="calc.push('2'); _sync_calc();">2</button>
  <button onclick="calc.push('='); _sync_calc();">=</button>
</div>
<div id="msg"></div>

Notice that in another <script> block, some JavaScript functions are defined

new_calc() should be called to crate a new Calculator instance, and assign it to the JavaScript variable calc
after creating a new Calculator instance with new_calc(), can call the methods of Calculator like calc.push('1')
_sync_calc() is there to synchronize the Calcuator's display with the "msg" <div>

Try it! Simply change the above simple.html to simple_calculator.html, like in ENDPOINT

val ENDPOINT: String = "http://192.168.0.17:8000/simple_calculator.html"

It Appears that It Will Work!

Nevertheless:

The Rust code file lib.rs need be extended in order to expose more functionalities of DumbCalculator
The "bridge" is also extended, like in bridge.html
The complete ACalculatorApp coding is quite involving, mostly due to UI coding


Without going into all the details of the implementation, I hope this exploration at this point is already enjoyable.

Enjoy!

You are more than welcome to clone the GitHub repository and build and run the complete Android app yourself.

GitHub Repository README to DEV Community Post

Trevor Lee — Sat, 06 Apr 2024 02:03:37 +0000

GitHub Repository README to DEV Community Post

The enabler of the capability is a GitHub Action workflow contributed by the publish-devto project

Hence, the picture

which is the sample that comes with the publish-devto project.

Here I will simply show how I setup GitHub repo for automatically posting README (README.md) to DEV Community (dev.to).

On Dev Community Side

As expected, you will need some "access token" from dev.to for the workflow to work.

Indeed, you need to get an API token from dev.to -- Settings -> Extension -> DEV Community API Keys

On GitHub Side

GitHub also requires some [repository] settings for security measures.

Firstly, grant "read and write permissions" to "Workflow" -- Settings -> Actions | General -> Workflow permissions set to Read and write permissions

Secondly, setup DEVTO_TOKEN secret (the API token you acquired in previous step) -- Secret and variables | Actions new Repository secrets

It appears that the repository needs be a public one. Otherwise, dev.to will fail to acquire the images.

On Repo Source Side

Firstly, you will need .github/workflows/publish.yml that you can download here, which is one that I tailored from the one that comes with the publish-devto project

And importantly, make sure that the branch specified in publish.yml is indeed the actual branch you want the action be triggered on

...
name: publish
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
...
        # (Optional) The git branch to use. Default is 'main'.
        branch: main
...

Secondly, your README.md will need to have "headers" (starting from 1st line) like

---
title: GitHub Repository README to DEV Community Post
description: Setup GitHub repo for automatically posting README to DEV Community  
tags: 'github, readme, devto'
cover_image: ./assets/cat.jpg
published: false
---

Notes on the headers:

title should be set correctly the first time, and should not be changed (unless you want to start a new post)
description is optional (i.e can be missing)
tags is optional, which is a list of comma-delimited single-words
cover_image is optional
published should be false until you want to publish the post
the first successful trigger of the publish workflow, id will be added automatically (README.md modified in the repo, and need be pulled) without the id, post will be treated as a new post

Commit Repo

That is it!

Commit and push the changes to GitHub, you should see the running of the workflow publish.

If the workflow is successful, a new draft post will be created on Dev Community (dev.to)

Be reminded that the first time successful running of the workflow, id will be automatically added to the "headers" of README.md. Pull for the addition.

Hope this helps!

Enjoy!

Arduino UNO R4 WiFi Experiments

Trevor Lee — Fri, 05 Apr 2024 09:02:01 +0000

Arduino UNO R4 WiFi Experiments

With this project, I hope to demonstrate my Arduino UNO R4 WiFi experiments with the microcontroller board's LED matrix, starting from simply turning on/off each one of the LEDs, to having a simple remote UI for controlling the LEDs of the matrix, using an Android app -- DumbDisplay -- connected using the microcontroller board's WiFi support.

The microcontroller programs here are Arduino sketches developed using VSCode with PlatformIO, in the similar fashion as described by the post -- A Way to Run Arduino Sketch With VSCode PlatformIO Directly.

Nevertheless, the sketches should be buildable with Arduino IDE.

PlatformIO Project for the Sketches

A single PlatformIO project will be used for all the sketches here.

Assuming you have created a PlatformIO project UNOR4WiFiExperiments for the board Arduino UNO R4 Wifi with the Arduino framework.

You should have the file platformio.ini similar to

[env:UNOR4]
platform = renesas-ra
board = uno_r4_wifi
framework = arduino
monitor_speed = 115200
lib_deps =
    https://github.com/trevorwslee/Arduino-DumbDisplay

Notes:

UNOR4 is simply the name of the "PlatformIO Project Environment"
The section monitor_speed specifies the baud rate for the PlatformIO's Serial Monitor to be 115200
The section lib_deps specifies that the project will be depending on the DumbDisplay Arduino Library

The file src/main.cpp should simply be the two lines like

#include <Arduino.h>
#include "INO/matrix_test/matrix_test.ino"

Yes, the first sketch will be src/INO/matrix_test/matrix_test.ino

Turning LED Matrix On/Off

src/INO/matrix_test/matrix_test.ino

#include "Arduino_LED_Matrix.h"
ArduinoLEDMatrix matrix;
void setup() {
  Serial.begin(115200);
  matrix.begin();
}
const uint32_t happy[] = {
    0x19819,
    0x80000001,
    0x81f8000
};
const uint32_t heart[] = {
    0x3184a444,
    0x44042081,
    0x100a0040
};
void loop(){
  matrix.loadFrame(happy);
  delay(500);
  matrix.loadFrame(heart);
  delay(500);
}

which is basically directly copied from the web page -- Using the Arduino UNO R4 WiFi LED Matrix

Notes:

The basic initialization for using the LED matrix is like

  matrix.begin();

To manipulate the LEDs of the matrix, a "frame" of 3 uint32_ts is needed, as in happy and heart
You load and display a "frame" like

  matrix.loadFrame(happy);

Since each one of the 96 bits of the 3 u_int32_ts corresponds to a LED of the 12x8 LED matrix, apparently, the LED on/off can be controlled by turning the proper bit on/off, with routine like the following set_bit() which sets the corresponding bits of frame on/off

...
uint32_t frame[] = { 0, 0, 0 };  // 3 32-bit unsigned ints can hold 96 bits
void set_bit(size_t bit, bool on) {
  int index = bit / 32;
  int offset = 31 - (bit % 32);
  if (on) {
    frame[index] |= (1 << offset);
  } else {
    frame[index] &= ~(1 << offset);
  }
}
...

Accordingly, let's try to turn on/off each of the LEDs one by one with sketch src/INO/matrix_obo_test/matrix_obo_test.ino

Turing the LEDs On/Off One-by-One

src/INO/matrix_obo_test/matrix_obo_test.ino

#include "Arduino_LED_Matrix.h"
ArduinoLEDMatrix matrix;
uint32_t frame[] = { 0, 0, 0 };  // 3 32-bit unsigned ints can hold 96 bits
void set_bit(size_t bit, bool on) {
  int index = bit / 32;
  int offset = 31 - (bit % 32);
  if (on) {
    frame[index] |= (1 << offset);
  } else {
    frame[index] &= ~(1 << offset);
  }
}
void setup() {
  Serial.begin(115200);
  matrix.begin();
}
int bit = -1;  // the bit last turned on
void loop() {
  if (bit != -1) {
    // turn off last bit
    set_bit(bit, false);
  }
  // advance bit
  if (bit == -1) {
    bit = 0;
  } else {
    bit = (bit + 1) % 96;
  }
  // turn on bit
  set_bit(bit, true);
  // load the frame (bits)
  matrix.loadFrame(frame);
  delay(500);
}

Notice that as it is looping through, the bit (turned on previously) is first turned off

  if (bit != -1) {
    // turn off last bit
    set_bit(bit, false);
  }

Then the bit is advanced to next bit, and turned on

  // advance bit
  if (bit == -1) {
    bit = 0;
  } else {
    bit = (bit + 1) % 96;
  }
  // turn on bit
  set_bit(bit, true);

Finally, the "frame" is loaded and displayed on the LED matrix

  // load the frame (bits)
  matrix.loadFrame(frame);

To build and upload the sketch above, you will need to modify src/main.cpp as well

#include <Arduino.h>
//#include "INO/matrix_test/matrix_test.ino"
#include "INO/matrix_obo_test/matrix_obo_test.ino"

A Simple Remote Virtual Joystick to Control the LEDs

Let's do something, hopefully, more interesting -- use a Joystick to control the LEDs, but a virtual one, and remotely on your Android phone, with the help of DumbDisplay.

For some little background on DumbDisplay, you may want to refer to the post Blink Test With Virtual Display, DumbDisplay

Note that the connection between Arduino UNO R4 WiFi and DumbDisplay Android app will be WIFI. Hence, you will first need to define macros for your WIFI credentials

src/INO/dd_joystick_test/secret.h

#define WIFI_SSID           "<WIFI-SSID>"
#define WIFI_PASSWORD       "<WIFI-password>"

src/INO/dd_joystick_test/dd_joystick_test.ino

#include "Arduino_LED_Matrix.h"
#include "wifidumbdisplay.h"
#include "secret.h"
ArduinoLEDMatrix matrix;
uint32_t frame[] = { 0, 0, 0 };  // 3 uint32_t can hold 96 bits
void set_bit(size_t bit, bool on) {
  int index = bit / 32;
  int offset = 31 - (bit % 32);
  if (on) {
    frame[index] |= (1 << offset);
  } else {
    frame[index] &= ~(1 << offset);
  }
}
DumbDisplay dumbdisplay(new DDWiFiServerIO(WIFI_SSID, WIFI_PASSWORD));
JoystickDDLayer *joystickLayer;
const size_t JOYSTICK_SIZE = 240;
void setup() {
  Serial.begin(115200);
  matrix.begin();
  // create a joystick layer
  joystickLayer = dumbdisplay.createJoystickLayer(JOYSTICK_SIZE - 1);
  joystickLayer->border(3, "darkblue", "round", 1);
  // turn on bit 0
  set_bit(0, true);
  matrix.loadFrame(frame);
}
int prev_bit = 0;
void loop() {
  const DDFeedback* fb = joystickLayer->getFeedback();
  if (fb != NULL) {
    // got "feedback" (i.e. joystick moved)
    size_t x = int((fb->x * 12) / (double) JOYSTICK_SIZE);
    size_t y = int((fb->y * 8) / (double) JOYSTICK_SIZE);
    size_t bit = x + y * 12;
    if (bit != prev_bit) {
      dumbdisplay.writeComment("x: " + String(x) + " y: " + String(y) + " bit: " + String(bit));
      set_bit(prev_bit, false);
      set_bit(bit, true);
      matrix.loadFrame(frame);
      prev_bit = bit;
    }
  }
}

To build and upload the sketch above, you will need to modify src/main.cpp as well

#include <Arduino.h>
//#include "INO/matrix_test/matrix_test.ino"
//#include "INO/matrix_obo_test/matrix_obo_test.ino"
#include "INO/dd_joystick_test/dd_joystick_test.ino"

When the sketch is running, please turn to the Serial Monitor. There you should see log lines like

listening on 192.168.0.2:10201 ...
listening on 192.168.0.2:10201 ...
listening on 192.168.0.2:10201 ...

which shows the network IP address (and port) on which your Arduino UNO R4 WiFi is listening.

Your IP will likely be different than what shown above. Note down the IP address shown. You will need this IP for DumbDisplay WIFI connection.

On your Android phone side

Start the DumbDisplay app.
Click on the Establish Connection icon.
In the "establish connection" dialog, you should see the "add WIFI device" icon at the bottom right of the dialog. Click on it.
A popup for you to enter WIFI IP will be shown. Enter the IP address of your ESP board as Network Host. Click OK when done.
Back to the "establish connection" dialog, a new entry will be added, click on it to establish WIFI connection.

Once connected, you should see the virtual joystick displayed

Moving the joystick should move the turned on LED of your Arduino UNO R4 WiFi LED matrix

Another Simple Remote UI to Draw on the LED Matrix

The sketch for the captioned UI is a bit longer, and hence will not be listed here. Instead, you can download the sketch src/INO/dd_draw/dd_draw.ino here

Alternatively, you can clone the complete repo UNOR4WiFiExperiments for all sketches mentioned

I guess by now, you will realize that all you need to switch the PlatformIO project to target for another sketch is simply to change src/main.cpp, like

#include <Arduino.h>
//#include "INO/matrix_test/matrix_test.ino"
//#include "INO/matrix_obo_test/matrix_obo_test.ino"
//#include "INO/dd_joystick_test/dd_joystick_test.ino"
#include "INO/dd_draw/dd_draw.ino"

And the resulting UI on DumbDisplay app will present you with a canvas for you to draw dots, which of course will be synchronized to your Arduino UNO R4 WiFi LED matrix.

CLEAR -- clear the dots
LOG -- show the "frame" on the "terminal" of DumbDisplay app , e.g. // {0x44444444,0x7c44444,0x44444444}

Tips:

By default, DumbDisplay app will show the "commands" sent to it to the "terminal". You can turn this off with the menu "Show Commands"
You can share the text on the "terminal" to other apps with the menu "Share Terminal Text"

Since the UI allows you to draw "frame" by "frame", with some work (maybe some hard work), you can create some basic animation to be displayed on the LED matrix, like the sketch src/INO/frames/frames.ino

#include "Arduino_LED_Matrix.h"
ArduinoLEDMatrix matrix;
void setup() {
  Serial.begin(115200);
  matrix.begin();
}
const uint32_t frames[][3] = {
  {0x20020020,0x2002002,0x200200},
  {0x20020020,0x3e02002,0x200200},
  {0x22022022,0x3e02202,0x20220220},
  {0x22422422,0x3e02202,0x20220220},
  {0x22422422,0x3e42242,0x24224224},
};
const size_t num_frames = sizeof(frames) / sizeof(frames[0]);
int frame_index = 0;  
void loop(){
  matrix.loadFrame(frames[frame_index]);
  delay(500);
  frame_index = (frame_index + 1) % num_frames;
}

Enjoy!

Implement a Simple WASM Calculator in Rust Using Leptos, and with DumbCalculator

Trevor Lee — Thu, 04 Apr 2024 11:44:32 +0000

Implement a Simple WASM Calculator in Rust Using Leptos, and with DumbCalculator

Development Environment

Here I will assume program development tools like

Of course, the Rust programming language itself.
The popular VSCode program development editor / IDE, with the extension rust-analyzer
Preferably the popular source control tool GIT.

Rust Crates Used

Leptos -- a Rust framework to develop WASM app in Rust.
DumbCalculator of rusty_dumb_tools -- a simple Rust component that acts like a simple physical calculator.

Preparation for WASM Development

WASM development in Rust can be enabled with the Trunk tool.
Indeed, Trunk is used here, and we install Trunk like

cargo install trunk

After installing Trunk, we will also need to add the Rust target wasm32-unknown-unknown, like

rustup target add wasm32-unknown-unknown

Kick-starting `wasm_calculator`

To get kick-started, create a new Rust project wasm_calculator like

cargo new wasm_calculator

Open the just created folder wasm_calculator with VSCode like

cd wasm_calculator
code .

In VSCode, open and edit Cargo.toml adding the necessary dependencies, like

...
[dependencies]
leptos = { version = "0.6.5", features = ["csr"] }
rusty_dumb_tools = "0.1.8"

Add the Trunk config file Trunk.toml with content like

[build]
target = "trunk.html"

Add trunk.html, which is sort of the template for our final output index.html

<!DOCTYPE html>
<html>
  <head><meta charset="UTF-8"></head>
  <body></body>
</html>

Note that without the above mentioned Trunk.toml config file, Trunk will in fact look for index.html as the template instead.
(However, we would like to reserve index.html for other purposes, and hence would use trunk.html instead.)

Our WASM code will be "mounted" to <body> of this trunk.html, let's see it working

trunk serve --open

This will run the Trunk server serving the trunk.html merged with whatever WASM code in main.rs

The server will keep running, and hot update the page whenever trunk.html or main.rs get changed

Say, change the <body> of trunk.html to

<body><h3>&mdash; WASM Calculator &mdash;</h3></body>

See that the browser page is updated accordingly.

The Basis of `wasm_calculator`

The initially generated main.rs is actually not WASM code to be "mounted" to <body>.
To "mount" some simple WASM code (written in Rust), can change main.rs like

use leptos::*;
fn main() {
    mount_to_body(move || {
        view! {
            <div style="color:red">Hello, World!</div>
        }
    });
}

Again, our modification will be hot-deployed, and we should see that

<div style="color:red">Hello, World!</div>

is "mounted" to <body>, after <h3>— WASM Calculator —</h3>

Here is some little insights from the above code:

mount_to_body is the function provided by Leptos to "mount" WASM code (written in Rust) to <body>
mount_to_body can accept a closure which accepts no argument and returns the result of calling the view! macro, which is of cause also provided by Leptos.
Inside view!, we write "HTML" -- like <div style="color:red">Hello, World!</div> -- which even looks like plain HTML, is in fact "legal" Rust code to be pre-processed by the macro view!.

In fact, normally, we will be coding our WASM code, in an App() function, and "mount" it like

use leptos::*;
fn main() {
    mount_to_body(move || view! { <App/> });
}
fn App() -> impl IntoView {
    view! {
        <div style="color:red">Hello, World!</div>
    }
}

As mentioned above, things inside view! will be pre-processed to be transcribed to Rust code in compile time, hence, we should be able to include regular Rust code inside view! like

fn App() -> impl IntoView {
    let color = "red";
    let who = "World"; 
    view! {
        <div style={format!("color:{}", color)}>Hello, {who}!</div>
    }
}

As shown, we can enclose regular Rust code inside {} as above {format!("color:{}", color)} and Hello, {who}!

We can even "nest" view! like

fn App() -> impl IntoView {
  let color = "red";
  let who = "World"; 
  view! {
    {
      view! {
        <div style={format!("color:{}", color)}>Hello, {who}!</div>
      }
    }
  }
}

Why "nest" view!? Hopefully, it will become apparent in later sections.

Let's extract the styling of the <div> to live else-where. Indeed, we can put our CSS in trunk.html and use it in App() like

trunk.html

<!DOCTYPE html>
<style>
  .test-class {
    color: green;
  }
</style>
...

main.rs

...
fn App() -> impl IntoView {
  let who = "World"; 
  view! {
    {
      view! {
        <div class="test-class">Hello, {who}!</div>
      }
    }
  }
}

Now, let's add two buttons to it to make it interactive.

Note that Leptos will generate HTML once only -- like the above App() will only be called once to generate initial HTML code -- any updates are triggered with "signals", and rendered with closures (more about this later).

Therefore, to make it interactive, not only we will need to add some interactive HTML elements, like <button>, we will also need to make use of "signals" as well.

First, let's add two <button>s, and be able to log to the browser's console when any of the button is clicked:

use leptos::*;
use leptos::logging::log;
use web_sys::MouseEvent;
...
fn App() -> impl IntoView {
  let who = "World"; 
  let on_clicked = |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
  };
  view! {
    {
      view! {
        <div class="test-class">Hello, {who}!</div>
      }
    }
    <button on:click=on_clicked value="1">I am 1</button>
    <button on:click=on_clicked value="2">I am 2</button>
  }
}

Notes:

Two more dependencies

    use leptos::logging::log;
    use web_sys::MouseEvent;

The closure defined by on_clicked will be called when any one of the button is clicked

    <button on:click=on_clicked value="1">I am 1</button>
    <button on:click=on_clicked value="2">I am 2</button>

Notice that the two <button>s are placed in the same level as the "nested" view!

Each button is associated with a value (e.g. value="1"), and when a button is clicked, the associated value is retrieved and printed out to the browser's console

    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);

Now, let's make use of "signal" to trigger update of the <div> content

fn App() -> impl IntoView {
  let (clicked_value, set_clicked_value) = create_signal(String::from(""));
  let on_clicked = move |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
    set_clicked_value.set(value);
  };
  view! {
    {
      move || view! {
        <div class="test-class"> {
          let value = clicked_value.get();
          format!("Hello, [{}]!", value)
        } </div>
      }
    }
    <button on:click=on_clicked value="1">I am 1</button>
    <button on:click=on_clicked value="2">I am 2</button>
  }
}

Notes:

The "signal" is created like

    let (clicked_value, set_clicked_value) = create_signal(String::from(""));

such a "signal" is composed of a "getter" clicked_value and a "setter" set_clicked_value
the type of the "signal" is String, and the "signal" is initialized to the empty String
- The "setter" set_clicked_value is called to set new value in the closure defined by on_click, which is called when any of the button is clicked

  let on_clicked = move |ev: MouseEvent| {
    ...
    set_clicked_value.set(value);
  }

Notice that the closure captures variables, like set_clicked_value, moved; and this is the requirement of using "signal"

The "user" of the "signal" is the <div> created by the nested view!

    move || view! {
      <div class="test-class"> {
        let value = clicked_value.get();
        format!("Hello, [{}]!", value)
      } </div>
    }

Notice:

the "nested" view is now a "moved" closure, since it is using the "signal" to get the new value set
the value set is returned by calling clicked_value.get()
by using the "signal", Leptos knows that the closure need be called again to update the content of the nested view! when the "signal" is updated
- Here is what will happen when a button is clicked
the closure on_click gets called
the "signal" gets updated when set_clicked_value is called
the closure of the "nested" view! gets called when the "signal" is updated, which will update the <div> generated by the "nested" view!

Let's change the buttons to ones that simulate the key presses for 1 + 2 =

  <button on:click=on_clicked value="1">1</button>
  <button on:click=on_clicked value="+">+</button>
  <button on:click=on_clicked value="2">2</button>
  <button on:click=on_clicked value="=">=</button>

And also add DumbCalculator into the picture

...
use std::cell::RefCell;
use rusty_dumb_tools::calculator::*;
...
fn App() -> impl IntoView {
  let calculator_ref = RefCell::new(DumbCalculator::new());
  let (clicked_value, set_clicked_value) = create_signal(String::from(""));
  let on_clicked = move |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
    set_clicked_value.set(value);
  };
  view! {
    {
      move || view! {
        <div class="test-class"> {
          let mut calculator = calculator_ref.borrow_mut();
          let value = clicked_value.get();
          if !value.is_empty() {
            calculator.push(value.as_str()).unwrap();
          }
          let result_value = calculator.get_display_sized(10);
          format!("[{}]", result_value)
        } </div>
      }
    }
    <button on:click=on_clicked value="1">1</button>
    <button on:click=on_clicked value="+">+</button>
    <button on:click=on_clicked value="2">2</button>
    <button on:click=on_clicked value="=">=</button>
  }
}

Notes:

An instance of DumbCalculator is created and stored in a RefCell, and assigned to calculator_ref; note that even calculator_ref is immutable, the instance of DumbCalculator can be retrieved mutable

    let calculator_ref = RefCell::new(DumbCalculator::new());

Note that since App() will only be called one, only a single instance of DumbCalculator will ever be created, unless the browser page is refreshed.

The "nested" view! code for the <div> is changed to something like

    <div class="test-class"> {
      let mut calculator = calculator_ref.borrow_mut();
      let value = clicked_value.get();
      if !value.is_empty() {
        calculator.push(value.as_str()).unwrap();
      }
      let result_value = calculator.get_display_sized(10);
      format!("[{}]", result_value)
    } </div>

The instance of DumbCalculator is "borrowed" mutable

  let mut calculator = calculator_ref.borrow_mut();

The button value, which is supposed to simulate calculator key press, is pushed to the DumbCalculator like
```
  calculator.push(value.as_str()).unwrap();
```

What the calculator display should look like is rendered in the content of the <div>

  let result_value = calculator.get_display_sized(10);
  format!("[{}]!", result_value)

Now, let's add two buttons for "AC" (all cancel) as well as "⬅" (undo)

fn App() -> impl IntoView {
  let calculator_ref = RefCell::new(DumbCalculator::new());
  let (clicked_value, set_clicked_value) = create_signal(String::from(""));
  let on_clicked = move |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
    set_clicked_value.set(value);
  };
  view! {
    {
      move || view! {
        <div class="test-class"> {
          let mut calculator = calculator_ref.borrow_mut();
          let value = clicked_value.get();
          if value == "ac" {
            calculator.reset();
          } else if value == "<" {
            calculator.undo();
          } else if !value.is_empty() {
            calculator.push(value.as_str()).unwrap();
          }
          let result_value = calculator.get_display_sized(10);
          format!("[{}]", result_value)
        } </div>
      }
    }
    <button on:click=on_clicked value="1">1</button>
    <button on:click=on_clicked value="+">+</button>
    <button on:click=on_clicked value="2">2</button>
    <button on:click=on_clicked value="=">=</button>
    <div>
      <button on:click=on_clicked value="ac">AC</button>
      <button on:click=on_clicked value="<">{"⬅"}</button>
    </div>
  }
}

Notes:

The two additional buttons are added below the 1 + 2 = buttons like

    <div>
      <button on:click=on_clicked value="ac">AC</button>
      <button on:click=on_clicked value="<">{"⬅"}</button>
    </div>

Clicking of the two buttons are handled like

    let value = clicked_value.get();
    if value == "ac" {
      calculator.reset();
    } else if value == "<" {
      calculator.undo();
    } else if !value.is_empty() {
      calculator.push(value.as_str()).unwrap();
    }

We are getting closer and closer. Let's add to the code the capability of showing the history of the calculator.

For this, we will be using another history "signal".

fn App() -> impl IntoView {
  let calculator_ref = RefCell::new(DumbCalculator::new());
  let (clicked_value, set_clicked_value) = create_signal(String::from(""));
  let (history, set_history) = create_signal(String::from(""));
  let on_clicked = move |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
    set_clicked_value.set(value);
  };
  view! {
    {
      move || view! {
        <div class="test-class"> {
          let mut calculator = calculator_ref.borrow_mut();
          let value = clicked_value.get();
          if value == "ac" {
            calculator.reset();
          } else if value == "<" {
            calculator.undo();
          } else if !value.is_empty() {
            calculator.push(value.as_str()).unwrap();
          }
          let history = calculator.get_history_string(true);
          match &history {
            Some(hist) => set_history.set(hist.to_string()),
            None => set_history.set("".to_string()),  
          }
          let result_value = calculator.get_display_sized(10);
          format!("[{}]", result_value)
        } </div>
      }
    }
    <button on:click=on_clicked value="1">1</button>
    <button on:click=on_clicked value="+">+</button>
    <button on:click=on_clicked value="2">2</button>
    <button on:click=on_clicked value="=">=</button>
    <div>
      <button on:click=on_clicked value="ac">AC</button>
      <button on:click=on_clicked value="<">{"⬅"}</button>
      {move || view! {
        <span> { history.get() } </span>
      }}
    </div>
  }
}

Notes:

The history "signal" is added like

    let (history, set_history) = create_signal(String::from(""));

The history if set after pushing value to the calculator

    let history = calculator.get_history_string(true);
    match &history {
      Some(hist) => set_history.set(hist.to_string()),
      None => set_history.set("".to_string()),  
    }

The updated history is displayed next to the "⬅" button

    {move || view! {
      <span> { history.get() } </span>
    }}

Finally

Finally, let's add all the whistles and bells and finish off our calculator ...

Simply ... please replace the corresponding file as listed here:

trunk.html -- https://github.com/trevorwslee/wasm_calculator/blob/master/trunk.html
main.rs -- https://github.com/trevorwslee/wasm_calculator/blob/master/src/main.rs

Or ... simply ... clone the GitHub repo https://github.com/trevorwslee/wasm_calculator

Manually Deploy to GitHub Pages

Now that we have the final result WASM calculator, we may want to deploy it to GitHub Pages.

Assuming a GitHub account, say like mine -- trevorwslee, I can easily post some static pages to my GitHub Pages -- https://trevorwslee.github.io/

What I needed is a GitHub repository with the "same" name trevorwslee.github.io -- https://github.com/trevorwslee/trevorwslee.github.io

In order for GitHub Pages to host our WASM Calculator app

With Trunk, build release version specifying the context to use in GitHub Pages, like mine
- The context is WASMCalculator
- Then the URL to the WASM app is https://trevorwslee.github.io/WASMCalculator/
To build such release, run Trunk like

  trunk build --release --public-url "WASMCalculator"

After building the release with Trunk, the dist folder will contain the files
- index.html
- wasm_calculator-XXX.wasm
- wasm_calculator-XXX.js to be checked in to WASMCalculator folder of the GitHub Pages repository, like mine -- https://github.com/trevorwslee/trevorwslee.github.io/tree/main/WASMCalculator
After a few moments, the WASM Calculator should be ready to try out -- https://trevorwslee.github.io/WASMCalculator/

Enjoy!

Implement a Simple WASM Calculator in Rust Using Leptos, and with DumbCalculator

Trevor Lee — Tue, 20 Feb 2024 15:18:56 +0000

Development Environment

Here I will assume program development tools like

Of cause, the Rust programming language itself.
The popular VSCode program development editor / IDE, with the extension rust-analyzer
Preferably the popular source control tool GIT.

Rust Crates Used

Leptos -- a Rust framework to develop WASM app in Rust
DumbCalculator of rusty_dumb_tools -- a simple Rust component that acts like a simple physical calculator.

Preparation for WASM Development

WASM development in Rust can be enabled with the Trunk tool.
Indeed, Trunk is used here, and we install Trunk like

cargo install trunk

After installing Trunk, we will also need to add the Rust target wasm32-unknown-unknown, like

rustup target add wasm32-unknown-unknown

Kick-starting `wasm_calculator`

To get kick-started, create a new Rust project wasm_calculator like

cargo new wasm_calculator

Open the just created folder wasm_calculator with VSCode like

cd wasm_calculator
code .

In VSCode, open and edit Cargo.toml adding the necessary dependencies, like

...
[dependencies]
leptos = { version = "0.6.5", features = ["csr"] }
rusty_dumb_tools = "0.1.7"

Add the Trunk config file Trunk.toml with content like

[build]
target = "trunk.html"

Add trunk.html, which is sort of the template for our final output index.html

<!DOCTYPE html>
<html>
  <head><meta charset="UTF-8"></head>
  <body></body>
</html>

Note that without the above mentioned Trunk.toml config file, Trunk will in fact look for index.html as the template instead.

Our WASM code will be "mounted" to <body> of this trunk.html, let's see it working

trunk serve --open

This will run the Trunk server serving the trunk.html merged with whatever WASM code in main.rs

The server will keep running, and hot update the page whenever trunk.html or main.rs changed

Say, change the <body> of trunk.html to

<body><h3>&mdash; WASM Calculator &mdash;</h3></body>

See that the browser page is changed accordingly.

The Basis of `wasm_calculator`

The initially generated main.rs is actually not WASM code to be "mounted" to <body>.
To "mount" some simple WASM code (written in Rust), can change main.rs like

use leptos::*;
fn main() {
    mount_to_body(move || {
        view! {
            <div style="color:red">Hello, World!</div>
        }
    });
}

Again, our modification will be hot-deployed, and we should see that

<div style="color:red">Hello, World!</div>

is "mounted" to <body>, after <h3>— WASM Calculator —</h3>

Here is some little insights from the above code:

mount_to_body is the function provided by Leptos to "mount" WASM code (written in Rust) to <body>
mount_to_body can accept a closure which accepts no argument and returns the result of calling the view! macro, which is of cause also provided by Leptos.
Inside view!, we write "HTML" -- like <div style="color:red">Hello, World!</div> -- which even looks like plain HTML, is in fact valid Rust code to be pre-processed by the macro view!.

In fact, normally, we will be coding our WASM code, in an App() function, and "mount" it like

use leptos::*;
fn main() {
    mount_to_body(move || view! { <App/> });
}
fn App() -> impl IntoView {
    view! {
        <div style="color:red">Hello, World!</div>
    }
}

As mentioned above, things inside view! will be pre-processed to be transcribed to Rust code in compile time, hence, we should be able to include regular Rust code inside view! like

fn App() -> impl IntoView {
    let color = "red";
    let who = "World"; 
    view! {
        <div style={format!("color:{}", color)}>Hello, {who}!</div>
    }
}

As shown, we can enclose regular Rust code inside {} as above {format!("color:{}", color)} and Hello, {who}!

We can even "nest" view! like

fn App() -> impl IntoView {
  let color = "red";
  let who = "World"; 
  view! {
    {
      view! {
        <div style={format!("color:{}", color)}>Hello, {who}!</div>
      }
    }
  }
}

Why "nest" view!? Hopefully, it will become apparent in later sections.

Let's extract the styling of the <div> to live else-where. Indeed, we can put our CSS in trunk.html and use it in App() like

trunk.html

<!DOCTYPE html>
<style>
  .test-class {
    color: green;
  }
</style>
...

main.rs

...
fn App() -> impl IntoView {
  let who = "World"; 
  view! {
    {
      view! {
        <div class="test-class">Hello, {who}!</div>
      }
    }
  }
}

Now, let's add two buttons to it to make it interactive.

Note that Leptos will render HTML once only -- like the above App() will only be called once to generate initial HTML code -- any updates are triggered with "signals", and rendered with closures (more about this later).

Therefore, to make it interactive, not only we will need to add some interactive HTML elements, like <button>, we will also need to make use of "signals" as well.

First, let's add two <button>s, and be able to log to the browser's console when any of the button is clicked:

use leptos::*;
use leptos::logging::log;
use web_sys::MouseEvent;
...
fn App() -> impl IntoView {
  let who = "World"; 
  let on_clicked = |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
  };
  view! {
    {
      view! {
        <div class="test-class">Hello, {who}!</div>
      }
    }
    <button on:click=on_clicked value="1">I am 1</button>
    <button on:click=on_clicked value="2">I am 2</button>
  }
}

Notes:

Two more dependencies

    use leptos::logging::log;
    use web_sys::MouseEvent;

The closure defined by on_clicked will be called when any one of the button is clicked

    <button on:click=on_clicked value="1">I am 1</button>
    <button on:click=on_clicked value="2">I am 2</button>

Notice that the two <button>s are placed in the same level as the "nested" view!

Each button is associated with a value (e.g. value="1"), and when a button is clicked, the associated value is retrieved and printed out to the browser's console

    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);

Now, let's make use of "signal" to trigger update of the <div> content

fn App() -> impl IntoView {
  let (clicked_value, set_clicked_value) = create_signal(String::from(""));
  let on_clicked = move |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
    set_clicked_value.set(value);
  };
  view! {
    {
      move || view! {
        <div class="test-class"> {
          let value = clicked_value.get();
          format!("Hello, [{}]!", value)
        } </div>
      }
    }
    <button on:click=on_clicked value="1">I am 1</button>
    <button on:click=on_clicked value="2">I am 2</button>
  }
}

Notes:

The "signal" is created like

    let (clicked_value, set_clicked_value) = create_signal(String::from(""));

such a "signal" is composed of a "getter" clicked_value and a "setter" set_clicked_value
the type of the "signal" is String, and the "signal" is initialized to the empty String
- The "setter" set_clicked_value is called to set new value in the closure defined by on_click, which is called when any of the button is clicked

  let on_clicked = move |ev: MouseEvent| {
    ...
    set_clicked_value.set(value);
  }

Notice that the closure captures variables, like set_clicked_value, moved; and this is the requirement of using "signal"

The "user" of the "signal" is the <div> created by the nested view!

    move || view! {
      <div class="test-class"> {
        let value = clicked_value.get();
        format!("Hello, [{}]!", value)
      } </div>
    }

Notice:

the "nested" view is now a "moved" closure, since it is using the "signal" to get the new value set
the value set is returned by calling clicked_value.get()
by using the "signal", Leptos knows that the closure need be called again to update the content of the nested view! when the "signal" is updated
- Here is what will happen when a button is clicked
the closure on_click gets called
the "signal" gets updated when set_clicked_value is called
the closure of the "nested" view! gets called when the "signal" is updated, which will update the <div> generated by the "nested" view!

Let's change the buttons to ones that simulate the key presses for 1 + 2 =

  <button on:click=on_clicked value="1">1</button>
  <button on:click=on_clicked value="+">+</button>
  <button on:click=on_clicked value="2">2</button>
  <button on:click=on_clicked value="=">=</button>

And also add DumbCalculator into the picture

...
use std::cell::RefCell;
use rusty_dumb_tools::calculator::*;
...
fn App() -> impl IntoView {
  let calculator_ref = RefCell::new(DumbCalculator::new());
  let (clicked_value, set_clicked_value) = create_signal(String::from(""));
  let on_clicked = move |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
    set_clicked_value.set(value);
  };
  view! {
    {
      move || view! {
        <div class="test-class"> {
          let mut calculator = calculator_ref.borrow_mut();
          let value = clicked_value.get();
          if !value.is_empty() {
            calculator.push(value.as_str()).unwrap();
          }
          let result_value = calculator.get_display_sized(10);
          format!("[{}]", result_value)
        } </div>
      }
    }
    <button on:click=on_clicked value="1">1</button>
    <button on:click=on_clicked value="+">+</button>
    <button on:click=on_clicked value="2">2</button>
    <button on:click=on_clicked value="=">=</button>
  }
}

Notes:

An instance of DumbCalculator is created and stored in a RefCell, and assigned to calculator_ref; note that even calculator_ref is immutable, the instance of DumbCalculator can be retrieved mutable

    let calculator_ref = RefCell::new(DumbCalculator::new());

Note that since App() will only be called one, only a single instance of DumbCalculator will ever be created, unless the browser page is refreshed.

The "nested" view! code for the <div> is changed to something like

    <div class="test-class"> {
      let mut calculator = calculator_ref.borrow_mut();
      let value = clicked_value.get();
      if !value.is_empty() {
        calculator.push(value.as_str()).unwrap();
      }
      let result_value = calculator.get_display_sized(10);
      format!("[{}]", result_value)
    } </div>

The instance of DumbCalculator is "borrowed" mutable

  let mut calculator = calculator_ref.borrow_mut();

The button value, which is supposed to simulate calculator key press, is pushed to the DumbCalculator like
```
  calculator.push(value.as_str()).unwrap();
```

What the calculator display should look like is rendered in the content of the <div>

  let result_value = calculator.get_display_sized(10);
  format!("[{}]!", result_value)

Now, let's add two buttons for "AC" (all cancel) as well as "⬅" (undo)

fn App() -> impl IntoView {
  let calculator_ref = RefCell::new(DumbCalculator::new());
  let (clicked_value, set_clicked_value) = create_signal(String::from(""));
  let on_clicked = move |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
    set_clicked_value.set(value);
  };
  view! {
    {
      move || view! {
        <div class="test-class"> {
          let mut calculator = calculator_ref.borrow_mut();
          let value = clicked_value.get();
          if value == "ac" {
            calculator.reset();
          } else if value == "<" {
            calculator.undo();
          } else if !value.is_empty() {
            calculator.push(value.as_str()).unwrap();
          }
          let result_value = calculator.get_display_sized(10);
          format!("[{}]", result_value)
        } </div>
      }
    }
    <button on:click=on_clicked value="1">1</button>
    <button on:click=on_clicked value="+">+</button>
    <button on:click=on_clicked value="2">2</button>
    <button on:click=on_clicked value="=">=</button>
    <div>
      <button on:click=on_clicked value="ac">AC</button>
      <button on:click=on_clicked value="<">{"⬅"}</button>
    </div>
  }
}

Notes:

The two additional buttons are added below the 1 + 2 = buttons like

    <div>
      <button on:click=on_clicked value="ac">AC</button>
      <button on:click=on_clicked value="<">{"⬅"}</button>
    </div>

Clicking of the two buttons are handled like

    let value = clicked_value.get();
    if value == "ac" {
      calculator.reset();
    } else if value == "<" {
      calculator.undo();
    } else if !value.is_empty() {
      calculator.push(value.as_str()).unwrap();
    }

We are getting closer and closer. Let's add to the code the capability of showing the history of the calculator.

For this, we will be using another history "signal".

fn App() -> impl IntoView {
  let calculator_ref = RefCell::new(DumbCalculator::new());
  let (clicked_value, set_clicked_value) = create_signal(String::from(""));
  let (history, set_history) = create_signal(String::from(""));
  let on_clicked = move |ev: MouseEvent| {
    let value = event_target_value(&ev);
    log!("* clicked value [{}]", value);
    set_clicked_value.set(value);
  };
  view! {
    {
      move || view! {
        <div class="test-class"> {
          let mut calculator = calculator_ref.borrow_mut();
          let value = clicked_value.get();
          if value == "ac" {
            calculator.reset();
          } else if value == "<" {
            calculator.undo();
          } else if !value.is_empty() {
            calculator.push(value.as_str()).unwrap();
          }
          let history = calculator.get_history_string(true);
          match &history {
            Some(hist) => set_history.set(hist.to_string()),
            None => set_history.set("".to_string()),  
          }
          let result_value = calculator.get_display_sized(10);
          format!("[{}]", result_value)
        } </div>
      }
    }
    <button on:click=on_clicked value="1">1</button>
    <button on:click=on_clicked value="+">+</button>
    <button on:click=on_clicked value="2">2</button>
    <button on:click=on_clicked value="=">=</button>
    <div>
      <button on:click=on_clicked value="ac">AC</button>
      <button on:click=on_clicked value="<">{"⬅"}</button>
      {move || view! {
        <span> { history.get() } </span>
      }}
    </div>
  }
}

Notes:

The history "signal" is added like

    let (history, set_history) = create_signal(String::from(""));

The history if set after pushing value to the calculator

    let history = calculator.get_history_string(true);
    match &history {
      Some(hist) => set_history.set(hist.to_string()),
      None => set_history.set("".to_string()),  
    }

The updated history is displayed next to the "⬅" button

    {move || view! {
      <span> { history.get() } </span>
    }}

Finally

Finally, let's add all the whistles and bells and finish off our calculator ...

Simply ... please replace the corresponding file as listed here:

trunk.html -- https://github.com/trevorwslee/wasm_calculator/blob/master/trunk.html
main.rs -- https://github.com/trevorwslee/wasm_calculator/blob/master/src/main.rs

Or ... simply ... clone the GitHub repo https://github.com/trevorwslee/wasm_calculator

Manually Deploy to GitHub Page

Now that we have the final result WASM calculator, we may want to deployed it to GitHub Pages.

Assuming a GitHub account, say like mine -- trevorwslee, I can easily post some static pages to my GitHub Pages -- https://trevorwslee.github.io/

What I needed is a GitHub repository with the "same" name trevorwslee.github.io -- https://github.com/trevorwslee/trevorwslee.github.io

In order for GitHub Pages to host our WASM Calculator app

With Trunk, build release version specifying the context to use in GitHub Pages, like mine
- The context is WASMCalculator
- Then the URL to the WASM app is https://trevorwslee.github.io/WASMCalculator/
To build such release, run Trunk like

  trunk build --release --public-url "WASMCalculator"

After building the release with Trunk, the dist folder will contain the files
- index.html
- wasm_calculator-XXX.wasm
- wasm_calculator-XXX.js to be checked in to WASMCalculator folder of the GitHub Pages repository, like mine -- https://github.com/trevorwslee/trevorwslee.github.io/tree/main/WASMCalculator
After a few moments, the WASM Calculator should be ready to try out -- https://trevorwslee.github.io/WASMCalculator/

Enjoy!

Forem: Trevor Lee

PyTorch Introductory Experiments

PyTorch Introductory Experiments

Installation with VSCode

"Hello World" Deep Learning Model Training

Highlights of train_sine.ipynb

Highlights of train_sine_cosine.ipynb

Mnist Dataset DL Training and Demo App

Demo UI for the Trained Mnist DL Model

Sliding Puzzle DL Training and Demo App

Demo UI for the Trained Sliding Puzzle Model

Two Simple DumbDisplay Samples

Have Fun!

A Weather Clock (with Alarms) for ESP32 / Raspberry Pi Pico Implemented with Arduino Framework

Arduino Weather Clock -- AWeatherClock -- v1.1

Out-of-the-Box Supported Hardware

Showing of IP and Connecting DumbDisplay Android App

Settings UI

Settings UI -- General

Settings UI -- Alarms

Settings UI -- Slides

Alarm Sound

Basic Hardcoded Configurations -- config.h

System Hardcoded Configurations -- sys_config.h

Customizations for New Hardware Highlights

Customizations for New Hardware PicoW Example

Customizations for New Hardware ESP32 Example

Enjoy!

ESP-IDF with Arduino Examples

ESP-IDF with Arduino Examples

Examples

Highlights

Demonstration

Enjoy!

Sliding Puzzle Next Move Suggesting Simple DL Model with ESP32 TensorFlow Lite

Sliding Puzzle 'Next Move' Suggesting Simple DL Model with ESP32 TensorFlow Lite

Building DL Model with VSCode

Building and Uploading the Sketch

Sliding Puzzle Game UI

Improving the 'Next Move' Suggestion Accuracy

A LLM Prompt for the Moves

Enjoy!

Turn ESP32-CAM into a Snapshot Taker, for Selfies and Time-Lapse Pictures

Turn ESP32-CAM into a Snapshot Taker, for Selfies and Time Lapse Pictures

The UI -- Ways of taking Snapshots

The UI -- Frequency of Capturing Snapshots

The UI -- Snapshots Quality Adjustments

The UI -- ESP32 Idle Sleep

Developing and Building

The Sketch

Sketch Highlight -- EEPROM

Sketch Highlight -- ESP32 Sleep Mode

Sketch Highlight -- Camera

Sketch Highlight -- DumbDisplay

Build and Upload

Enjoy!

Simple Arduino Framework Photo Frame Implementation with Photos Downloaded from the Internet via DumbDisplay

Simple Arduino Framework Raspberry Pi Pico / ESP32 TFT LCD Photo Frame Implementation with Photos Downloaded from the Internet via DumbDisplay

The UI

Wiring TFT LCD Module

Developing and Building

The Sketch

Sketch Highlight -- TFT LCD Library Adafruit-ST7735-Library

Sketch Highlight -- TFT LCD Library bodmer/TFT_eSPI

Sketch Highlight -- Jpeg Library bodmer/TJpg_Decoder

Sketch Highlight -- LittleFS

Sketch Highlight -- DumbDisplay

Build and Upload

Enjoy!

Implement a Simple Calculator Android App by Reusing Logics in Rust via JavaScript-WASM Interfacing

Implement a Simple Calculator Android App by Reusing Logics in Rust via JavaScript-WASM Interfacing

Starting the Project

Initialize for the JavaScript-Rust "Bridge"

Key Takeaways of the JavaScript-WASM Bridge

Android Calling the JavaScript-WASM "Bridge"

Package the "Bridge" with the App

Add Some Jetpack Compose Code to Call get_greeting

Lets Bring Some Calculator Code into the Picture

It Appears that It Will Work!

Enjoy!

Highlights of `train_sine.ipynb`

Highlights of `train_sine_cosine.ipynb`

Arduino Weather Clock -- `AWeatherClock` -- v1.1

Basic Hardcoded Configurations -- `config.h`

System Hardcoded Configurations -- `sys_config.h`

Sketch Highlight -- TFT LCD Library `Adafruit-ST7735-Library`

Sketch Highlight -- TFT LCD Library `bodmer/TFT_eSPI`

Sketch Highlight -- Jpeg Library `bodmer/TJpg_Decoder`

Add Some Jetpack Compose Code to Call `get_greeting`

Kick-starting `wasm_calculator`

The Basis of `wasm_calculator`

Kick-starting `wasm_calculator`

The Basis of `wasm_calculator`