Forem: Adriano Repetti

Tinkwell: Anomaly Detector using Randomized PCA

Adriano Repetti — Mon, 07 Jul 2025 17:27:44 +0000

In earlier posts, I introduced a language-agnostic, firmware-less approach to IoT that sidesteps many traditional complications. I've since been building a reference implementation in C# called Tinkwell (named from Tinkwer and Well). The project is still evolving, but I'm convinced it has potential beyond its original scope, with applications in a variety of scenarios.

In this post, I'll walk through how to integrate a Python-based anomaly detection system with Tinkwell, using the most straightforward tool available: the Tinkwell Command Line Interface (tw). Of course, if you were to write a runner in Python, you'd want to do it properly (first of all communicating with all the services using gRPC) but for a quick test, or to prototype an algorithm, using the CLI is faster and easier.

To keep things focused, the code shown here is heavily redacted. For the full working version, head over to the GitHub repository.

Introduction to Principal Component Analysis (PCA)

Principal Component Analysis (PCA) is a statistical technique that transforms a set of possibly correlated variables into a smaller set of new variables called principal components, which are linearly uncorrelated. The first component grabs the most variance it can from the data, and each one after it captures as much of what's left while staying independent from the others.

In even simpler terms: when your data has a bunch of features, PCA figures out which combinations of them explain the most variation. It then turns those into new features that summarize your data without all the extra clutter. This helps when you want to make the data easier to visualize or feed into a machine learning model without losing too much of what makes it useful.

Randomized PCA

Randomized PCA is based on randomized algorithms for matrix decomposition. Instead of directly computing the full covariance matrix and solving for eigenvectors (which can be slow for big datasets), it approximates the principal components using a random projection method.

This gives you an efficient way to estimate the top k principal components of a data matrix without needing to compute the full decomposition, especially useful when the matrix is huge or sparse.

Why PCA is a Good Fit for Anomaly Detection

PCA is particularly well-suited for anomaly detection in multivariate time series data (like our measures) for several reasons:

Dimensionality Reduction: Real-world systems often involve many correlated sensors or measures. PCA can reduce this high-dimensional data into a lower-dimensional subspace, making the anomaly detection task more manageable and computationally efficient.
Normal Behavior Modeling: PCA effectively captures the "normal" operating patterns and correlations within the data. When the system behaves normally, its data points will lie close to the subspace spanned by the principal components.
Reconstruction Error as Anomaly Score: Anomalies often represent deviations from these normal patterns. When an anomalous data point is projected onto the PCA subspace and then reconstructed back to the original dimension, the difference between the original and reconstructed point (the "reconstruction error") will be significantly larger than for normal data points. This error serves as an effective anomaly score.
Unsupervised Learning: PCA is an unsupervised learning technique, meaning it does not require labeled anomaly data for training. It learns the normal behavior from the available data, which is often abundant, and then flags anything that deviates significantly from this learned normal.

Tinkwell Configuration

This is the ensamble file we need to run this example:

compose service orchestrator "Tinkwell.Orchestrator.dll"
compose service store "Tinkwell.Store.dll"
compose service events "Tinkwell.EventsGateway.dll"
compose agent reducer "Tinkwell.Reducer.dll" { path: "./measures.twm" }
compose agent reactor "Tinkwell.Reactor.dll" { path: "./measures.twm" }
compose agent watchdog "Tinkwell.Watchdog.dll"

This is the configuration for the measures we're going to use:

measure voltage {
    type: "ElectricPotential"
    unit: "Volt"
    expression: "5"
    minimum: 1
    maximum: 50
}

measure current {
    type: "ElectricCurrent"
    unit: "Ampere"
    expression: "2"
    minimum: 0
    maximum: 10
}

measure power {
    type: "Power"
    unit: "Watt"
    expression: "voltage * current"
    minimum: 0
    maximum: 500

    signal high_load {
      when: "power > 100"
    }
}

The Python Code

This example consists of three main scripts:

anomaly_detector.py: it detects anomalies in the input data and save a CSV file with the inputs and its findings. It uses tw measures subscribe to listen for changes and process the new values.
feed_synthetic_data.py: generates synthetic data to test the anomaly detector. It uses tw measures write to update the Store.
plot_measures.py: plots the results saved in the CSV file exported by anomaly_detector.py.

It is not shown in this example but you could feed back the results from the anomalies detection into Tinkwell using tw events publish.

There are many parameters to configure, please read the README.md file carefully.

The anomaly detection process in this script leverages the reconstruction error property of PCA:

Data Normalization: Each incoming measure value is first normalized using its pre-determined minimum and maximum values. This ensures that all measures contribute equally to the PCA, regardless of their original scale.
PCA Model Training:
- The script collects a PCA_BUFFER_SIZE number of normalized data samples. Each sample is a vector representing the current values of all subscribed measures.
- A Randomized PCA model is then trained on this buffer of "normal" data.
- After training, the model can project new data points onto its principal components and reconstruct them.
Reconstruction Error Calculation: For each data sample in the training buffer, its reconstruction error is calculated. This error is the Euclidean distance (or L2 norm) between the original data point and its reconstructed version after being projected onto the PCA subspace and then inverse-transformed.
Anomaly Threshold Determination: A statistical threshold for anomaly detection is established from the distribution of these reconstruction errors. Specifically, the ANOMALY_THRESHOLD_PERCENTILE (e.g., 99th percentile) of the reconstruction errors from the training data is chosen as the threshold. This means that ANOMALY_THRESHOLD_PERCENTILE% of the "normal" training data will have a reconstruction error below this threshold.
Real-time Anomaly Detection:
- As new measure data arrives, it is normalized and formed into a new sample vector.
- This new sample is then passed through the trained PCA model to calculate its reconstruction error.
- If the calculated reconstruction error for the new sample exceeds the pre-determined anomaly threshold, the sample is flagged as an anomaly.
- When an anomaly is detected, the script prints detailed information, including the raw and normalized values, the reconstruction error, and the anomaly threshold, to help in understanding the nature of the deviation.

The code for the PCA detector is fairly small:

import numpy as np
from sklearn.decomposition import PCA

class PcaAnomalyDetector:
    def __init__(self, n_components, anomaly_threshold_percentile):
        self.n_components = n_components
        self.anomaly_threshold_percentile = anomaly_threshold_percentile
        self.pca_model = None
        self.anomaly_threshold = None

    def train(self, data_buffer):
        if not data_buffer:
            raise ValueError("Data buffer cannot be empty for training.")

        self.pca_model = PCA(n_components=self.n_components, svd_solver='randomized')
        self.pca_model.fit(np.array(data_buffer))

        reconstructed_data = self.pca_model.inverse_transform(self.pca_model.transform(np.array(data_buffer)))
        reconstruction_errors = np.linalg.norm(np.array(data_buffer) - reconstructed_data, axis=1)

        self.anomaly_threshold = np.percentile(reconstruction_errors, self.anomaly_threshold_percentile)
        return self.anomaly_threshold

    def detect(self, sample):
        if self.pca_model is None or self.anomaly_threshold is None:
            raise RuntimeError("PCA model not trained. Call train() first.")

        current_sample_np = np.array([sample])
        transformed_sample = self.pca_model.transform(current_sample_np)
        reconstructed_sample = self.pca_model.inverse_transform(transformed_sample)
        current_reconstruction_error = np.linalg.norm(current_sample_np - reconstructed_sample)

        is_anomaly = current_reconstruction_error > self.anomaly_threshold
        return is_anomaly, current_reconstruction_error, reconstructed_sample[0]

    def is_trained(self):
        return self.pca_model is not None and self.anomaly_threshold is not None

To run this example (after you completed the initial setup):

Start Tinkwell and wait for the initialization to complete.
In a new terminal run python feed_synthetic_data.py to start pushing our synthetic test data into the system.
In a new terminal run python anomaly_detector.py to start analyzing the data.
In a new terminal run python plot_measures.py to visualize the data as they change and to see where the anomalies have been detected.

The end result will look more or less like this:

Note that the first 120 samples have been used to train the model.

Tuning the Parameters

The number of components to keep: too few and you'll miss key patterns in the data, too many and you bring back noise and irrelevant details. You could arbitrarily set this value or set it to keep components until you get, let's say, 95% of the variance (sklearn can do this for you!).
Anomaly threshold percentile: - Tune this percentile by checking how well it separates known normal vs abnormal data. Start somewhere around 95–99% and adjust.

References

PCA Basics:
- Jolliffe, I. T. (2002). Principal Component Analysis. Springer.
Randomized PCA:
- Halko, N., Martinsson, P. G., & Tropp, J. A. (2011). Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review, 53(2), 217-288.
PCA for Anomaly Detection:
- Shyu, M. L., Chen, S. C., Sarinnapakorn, M., & Chang, L. (2003). A novel anomaly detection scheme based on principal component classifier. Proceedings of the IEEE International Conference on Data Mining (ICDM), 2003, 347-354.
- Wang, S., & Ma, J. (2018). Anomaly detection based on PCA and reconstruction error. Journal of Physics: Conference Series, 1087(6), 062029.

Tinkwell: Running With AI

Adriano Repetti — Mon, 30 Jun 2025 17:30:26 +0000

In a previous series IoT Architectures Under Pressure, we explored a cost-effective concept for a variety of IoT devices, which we called firmware-less. The idea was based on the assumption that there’s a Hub available to run the "firmware" outside of the devices themselves.

We then introduced the Tinkwell project to build that hub however, although it was designed with IoT in mind, Tinkwell is flexible enough to be useful in other domains as well. You can find the source code on GitHub (warning: it's not production code, I'm still experimenting!).

The goal is to build an easy-to-use framework that includes all the basic components for a robust system, one that can be extended and adapted to the scenarios introduced in the first post. Each service is designed to be modular and replaceable, allowing you to substitute it with your own project-specific implementation or an existing commercial/open-source one. For example, you might:

Replace the default implementation of the Events Gateway with one that uses MQTT or even RabbitMQ.
Extend the Events Gateway to forward events to Kafka.
Swap out the Store for a time-series database like TimescaleDB, InfluxDB, or Cassandra.
Use the default implementations when running locally and swap to the others when deployed, all from a single configuration file.

Leaving aside the lower-level services (Supervisor, Orchestrator, and Discovery), the core components are:

Store: Tracks all measured values in the system and allows other firmlets to subscribe to changes.
Events Gateway: Accepts published events and broadcasts them to all subscribed clients.
Reducer: Calculates derived measures, updates them when dependencies change, and publishes the new values to the Store.
Reactor: Evaluates a set of rules and emits signals (essentially events) when conditions are met, based on current Store values. Think of it as a rule engine for triggering alarms (or other system states).
Executor: Listens for events and performs configured actions when a specific event is received and some conditions are met.

An Example

Let's see how a tiny configuration file would look like. We define a derived measure and set a signal sent when a specific alert condition is met (note that the event also contains subject, verb and object but Reactor can deduce them from the context). We also show a signal not associated with a single measure:

import "constants.twm"

measure power {
    type: "Power"
    unit: "Watt"
    expression: "voltage * current"

    signal high_load {
      when: "power > 80"
      then {
        severity: "critical"
      }
    }

    signal low_load {
      when: "power < 10"
    }
}

signal low_battery {
  when: "voltage < 24 and current < 10"
  then {
    subject: "battery"
    severity: "warning"
  }
}

Now that we have signals we can act on them:

// No filters, in this example we forward ALL the alarms!
when "Alarm" {
  then {
    http {
      method: "POST"
      url:    "https://intranet.mycompany.com/alerts"
      body:   {
        sensor: "subject"
        alarm: "object"
        level:   "[payload.severity]"
        message: "[payload.message]"
      }
    }
  }
}

For now these are two separate configuration files but we could change it and declare everything in one big file (and use import to split the source code into manageable chunks).

ML to the Rescue

There are plenty of ML models (or classical statistical approaches) you can choose from in real production environments, for example:

Random Forest: General anomaly detection, classification. Robust to noise and easy to interpret. Good baseline model but it does not handle temporals sequences.
LSTM (RNN): Time-series forecasting, early failure trends. It captures temporal dependencies, ideal for sequential data
Autoencoder (NN): Unsupervised anomaly detection. It learns normal patterns and flags deviations.

In a system already fully configured with all the appropriate rules I'd start adding an unsupervised Autoencoder model, let's see some code (untested and adapted to make it publishable) where our measures are the features of the data and we use historical values for training.

Prepare the data (assuming it's been aggregated in a variable labeled data with the same shape you'd have obtained, for example, generating random samples with data = np.random.rand(NUMBER_OF_SAMPLES, NUMBER_OF_FEATURES))):

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()
data_scaled = scaler.fit_transform(data)

Define a very basic autoencoder model:

from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Dense

input_dim = data_scaled.shape[1]

input_layer = Input(shape=(input_dim,))
encoded = Dense(12, activation='relu')(input_layer)
encoded = Dense(6, activation='relu')(encoded)

decoded = Dense(12, activation='relu')(encoded)
decoded = Dense(input_dim, activation='sigmoid')(decoded)

autoencoder = Model(inputs=input_layer, outputs=decoded)
autoencoder.compile(optimizer='adam', loss='mse')

Train the model (note how the same data is both the input and the target!):

autoencoder.fit(data_scaled, data_scaled,
                epochs=50,
                batch_size=32,
                shuffle=True,
                validation_split=0.2)

Now you can feed new data to the model and you can detect anomalies using reconstruction errors:

reconstructed = autoencoder.predict(data_scaled)
errors = np.mean((data_scaled - reconstructed)**2, axis=1)

That's just a proof of concept, I'm aiming to show how easy it is. To do it right there are plenty of people out there with the required ML expertise.

I'd also like to mention Federated Learning where a model is trained locally and its parameters or gradients are sent to a server to build a global model (which is then fed back to the clients). This is extremely useful in IoT because:

The edge device might not have the resources to calculate a better model.
You are concerned about data privacy and you do not want your sensor data to leave the premises.
The volume of data is too big.

Conversational AI Because...Why Not?

I'm not advocating to feed data into a Conversational AI, what I want to show is that it's extremely simple to feed the data (even in almost real-time) to an external system.

Let's write a tiny program to feed our favorite AI model with data from the Store and ask if everything is running OK:

import grpc
import openai  # or your preferred API client
import tinkwell_pb2 as pb
import tinkwell_pb2_grpc as pb_grpc

# Replace with your actual API credentials
openai.api_key = "YOUR_API_KEY"

# Replace with the real address of the Store service
STORE_ADDRESS = "localhost:5000"

# Replace with the real description of what your system is
CONTEXT = "Testing some measures"

def ask_model(prompt):
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[
            {"role": "user", "content": f"Context: {CONTEXT}" }
            {"role": "user", "content": prompt},
            {"role": "user", "content": "Knowing only this, answer only YES (there might be a problem), NO (everything is OK). If yes add a possible explanation of the problem" }
        ]
    )
    return response.choices[0].message.content.strip()

def main():
    options = (("grpc.ssl_target_name_override", "localhost"),)
    with grpc.secure_channel(STORE_ADDRESS, grpc.local_channel_credentials(), options) as channel:
        stub = pb_grpc.StoreStub(channel)
        reply = stub.List(pb.StoreListRequest(include_values=True))

        for item in reply.items:
            name = item.name
            description = item.description or "(no description)"
            unit = item.unit
            value = item.value or "Unknown"
            historical = item.historical_value or "Unknown"

            prompt = (
                f"Measure \"{name}\" is \"{description}\"\n"
                f"Value: {value}\n"
                f"Historical average: {historical}\n\n"
            )

            verdict = ask_model(prompt)
            print(f"\n{prompt}\nResult: {verdict}\n{'-'*50}")

if __name__ == "__main__":
    main()

I tried with some fictional data, the simplified prompt, stripped of all the irrelevant measures, look like this:

context: turbine to generate electricity

Measure "case_vibration_level" is "Intensity of vibrations measured on the outer casing (or housing) of the machine."
Value: 5 mm/s
Historical average: 3 mm/s

Measure "case_vibration_frequency" is "Dominant frequency at which the outer casing of the machine vibrates during operation."
Value: 23 Hz
Historical average: 50 Hz

Measure "turbine_rotation_speed" is "Rate at which a turbine shaft spins"
Value: 3000 rpm
Historical average: 3000 rpm

Knowing only this, answer only YES (there might be a problem), NO (everything is OK). If yes add a possible explanation of the problem

The response has been (in my case using Copilot):

Yes — that could very well be a red flag.
Let’s break it down:
- Turbine speed of 3000 rpm corresponds to 50 Hz
(since 3000 ÷ 60 = 50). That’s your 1× running speed.
- A vibration at 23 Hz is not synchronous with the
shaft — it’s a subharmonic or possibly a structural resonance.
- 5 mm/s RMS is quite elevated. According to ISO 10816-3,
for large rigidly mounted machines, anything above 4.5 mm/s RMS
is considered the “damage likely” zone.
So yes — this could be a problem. A few possibilities:
- Structural resonance: 23 Hz might match a natural
frequency of the casing or support structure. If excited by
harmonics or external forces, it can amplify vibration.
- Looseness or soft foot: These can introduce subharmonic
frequencies and erratic vibration patterns.
- Oil whirl or instability: If the machine uses fluid film
bearings, sub-synchronous frequencies like 23 Hz could indicate
oil-related instabilities.
- Blade pass or gear mesh sidebands: If there’s a gearbox or
bladed component upstream, 23 Hz might be a sideband or
modulation frequency.

Given the increase over historical levels, this isn’t just a
benign quirk — it’s a change in the machine’s dynamic behavior.
That’s often the first whisper of a mechanical issue before it
becomes a shout.

Spot on, I actually fed it a relatively large set of measures for a machine earl-ish displaying a oil whirl problem.

If fed with a vibration level of 2 mm/s (well acceptable) and without historical data, we still get this response:

The presence of a non-synchronous vibration frequency like 23 Hz
is a whisper worth listening to. It might be:
- A natural mode being lightly excited.
- A low-level fluid dynamic effect.
- Or just a harmless artifact — depending on the machine’s
design and bearing type.

Recommendation: keep monitoring. If the 23 Hz component
grows or shifts, or if the RMS creeps toward 2.8–4.5 mm/s,
that’s your cue to investigate further. But for now, you’re
in the green — just with one eyebrow slightly raised.

When asked to generate a rule to check for its own recommendation (after it's been instructed with the syntax using examples) it produced this (perfectly valid) configuration:

  signal possible_subsync_instability {
    when: "case_vibration_frequency >= 2.8 and abs(case_vibration_frequency - 23) <= 1"
    then {
      severity: "warning"
      message: "Elevated vibration at ~23 Hz may indicate sub-synchronous instability (e.g. oil whirl)."
    }
  }

Of course, it doesn't always work. It's not even always right, and you wouldn't ever bet your life on it. But setting up a periodic script to detect anomalies takes no more than 30 minutes (you could even do it from Bash using grpcurl!).

The example above is intentionally simple, but you could easily integrate it into a proper runner that publishes events to the Event Gateway. From there, the normal system flow takes over: the Executor reacts and performs the appropriate action, most likely just notifying someone.

Do I still need to write my rules for alerts?

Yes. It's needless to say that the above example is something that also a human being would have caught immediately and, in a real system, all the appropriate rules would have been in place already.

[Boost]

Adriano Repetti — Sun, 29 Jun 2025 22:42:19 +0000

Adriano Repetti

Jun 24 '25

Tinkwell: Firmware-less IoT and Lab Automation

#programming #architecture #csharp #iot

Comments 1

6 min read

Tinkwell: Firmware-less IoT and Lab Automation

Adriano Repetti — Tue, 24 Jun 2025 23:04:58 +0000

This new post introduces that Hub and its implementation. The project is called Tinkwell, a name inspired by a blend of Tinkering and Well, it's written in C# and it’s available on GitHub. The twist is that although it was designed with IoT in mind, Tinkwell is flexible enough to be useful in other domains as well.

Note: At the time of writing, the code on GitHub is purely experimental. It is not ready for production use, it's far from finished and it does not reflect best practices, and it may change frequently (and diverge greatly from what I'm describing here). Fork it, break it, and explore freely: it is a space for ideas and learning.

An overview

The basic idea is extremely simple: an ensamble is a set of processes called runners, defined statically using a DSL in a configuration file and launched and monitored by a Supervisor.

Each runner can expose one or more gRPC services, and can consume services exposed by other runners. The logic (be it the code to control our firmware-less device, some integration or anything else) is called firmlet and each runner can contain zero, one or more of them.

The system is distributed by design: you don’t need to know where things are running. A single Supervisor can spawn runners on other machines where a slave Supervisor is active. Similarly, services can be distributed across a network: they don’t have to be local.

There are a number of predefined services (virtually all of them optional):

Orchestrator: works in tandem with the Supervisor to start and stop runners, locally or remotely.
Discovery: keeps track of all services in the system, handles load balancing (when available), and provides the information needed to use them.
Store: records unit-aware measures from the system, connected sensors, runners, or devices.

Reducer: calculates derived measures. For example:

measure JournalBearingTemp {
  type: "Temperature"
  unit: "DegreesCelsius"
  expression: "(JournalBearingTemp1 + JournalBearingTemp2) / 2"
}
measure JournalBearingOilDeltaTemp {
  type: "Temperature"
  unit: "DegreesCelsius"
  expression: "JournalBearingOilReturnLineTemp - JournalBearingOilSupplyLineTemp"
}

Reactor: monitors the Store and applies a set of user-defined rules to fire events when certain conditions are met (e.g. alarms).
EventsGateway: collect all the published events and broadcast them to all registered listeners.
Executor: perform actions (log, running scripts, etc) when it receives certain events.
Watchdog: probes all runners exposing a HealthCheck service to determine their status and respond accordingly.

Store, reducer, reactor, executor and events gateway are central to the architecture. They provide a flexible and decoupled mechanism to easily describe the system in a declarative way. You can swap them for your own implementation (if/when needed).

In addition, there are several predefined runners to simplify development:

GrpcHost: hosts one or more gRPC services in a single process. Each service resides in a separate DLL. Useful for reducing boilerplate when writing new services.
DllHost: hosts one or more runners located in a shared library instead of individual executables. Ideal when multiple firmlets must start/stop together or you want to avoid repetitive scaffolding.
WasmHost: hosts a WebAssembly firmlet (probably for our firmware-less implementation).
WebServer: a simple web interface to monitor system status and values recorded in the Store.

The system is also modular: for example, the Store works in-memory by default, but you can add a module to persist data to a database or enable a service for querying historical values. You can also write a module to delegate process supervision to systemd (where available). Even the default services (such as the Store or the Trigger) are entirely optional. Tinkwell provides general-purpose implementations out of the box, but you're free to omit them or swap in your own custom versions to suit your needs.

Graphically:

Services interact with each other and they're monitored by the Watchdog. A simplified view of this approach:

Configuration

How do we configure this ensamble? Let's see how a (simplified for clarity) system.tw might look like (here you can find syntax highlighting for VS/VSCode/vim):

// The master GrpHost exposes the Discovery service
runner discovery "Tinkwell.Bootstrapper.GrpcHost" {
  service runner orchestrator "Tinkwell.Orchestrator.dll" {}
  service runner "Tinkwell.HealthCheck.dll" {}
}

runner "Tinkwell.Bootstrapper.GrpcHost" {
  service runner events "Tinkwell.EventsGateway.dll" {}
  service runner store "Tinkwell.Store.dll" {}
  service runner reducer "Thinkwell.Reducer.dll" {
    properties {
      path: "./measures.twm"
    }
  }
  service runner reactor "Tinkwell.Reactor.dll" {
    properties {
      path: "./measures.twm"
    }
  }
  service runner executor "Tinkwell.Executor.dll" {
    properties {
        path: "./actions.twa"
    }
  }
  service runner "Tinkwell.HealthCheck.dll" {}
}
runner watchdog "Tinkwell.Watchdog" {
  properties {
    interval: 25
  }
}

This code is probably shared among multiple deployments/setups then you could move it to, for example, a file named shared.ensamble and then import it when needed:

import "./shared.ensamble"
runner device_controller "path/to/device_controller" {}

You can also programmatically include a runner/service according to a run-time condition, ideal for flexible setups where not all devices are always present:

runner controller "path/to/controller" if "platform = 'linux'" {}

Ensamble files are preprocessed using the Liquid template engine, you can use variables, handle conditions and loops driven by external/run-time parameters.

Scripting

In addition to their specific service, each firmlet or runner should expose a generic access mechanism to read or write a value, or to perform an action. This interface, accessible via a simple command-line application, makes shell scripting straightforward and highly flexible:

value=$(tw read /store/JournalBearingOilDeltaTemp/value)
if (( $(echo "$value >= 5" | bc -l) )); then
    tw do /turbine/stop
fi

Applications

In addition to the firmware-less Hub we mentioned, this approach can be used in other applications as well:

Lab automation: each lab instrument (whether it’s a spectrometer, thermal controller, or motion stage) often comes with its own driver or control script. Tinkwell can supervise each of these as a separate (monitored) runner process. To add a new device is as simple as adding a new runner entry in the ensamble file. This opens the door to building web dashboards, remote experiment controllers, or integration with lab information systems (LIMS).
- With the Store module, you can track sensor readings (voltages, temperatures, concentrations) using unit-aware types. You can log conditions, perform derived computations and broadcast changes.
- You could define workflows where launching a new test involves spinning up a sequence of runners: data collector, logger, analyzer, etc. Since runners can be composed hierarchically (one runner invoking others), you can build robust pipelines for repeatable experiments.
- Using gRPC-based discovery and command interfaces, external systems or human operators can see what's running and monitor the progress of an experiment.
Edge and Fog Computing in industrial IoT: factories increasingly rely on edge devices to perform localized processing (e.g. anomaly detection from vibration sensors, temperature thresholds for safety cutoffs). Tinkwell offers a lightweight, resilient orchestration layer that doesn’t need containers or a full Kubernetes cluster, perfect for rugged industrial PCs at the edge.
Test benches and automated QA stations: like in labs, automated testing environments in industrial R&D departments often involve specialized hardware setups. Scripts or binaries controlling signal generators, power supplies, or data loggers can be isolated into runners.
Factory dashboard backends: you could use Tinkwell as the service layer behind a dashboard—runners provide data from physical devices or simulators, the Store aggregates it with unit-aware tracking, and the Watchdog ensures that data providers are functioning.
Safety systems and watchdog layers: in systems where a process crash is a serious concern (e.g. controlling furnaces, hydraulic presses), supervision is vital. Tinkwell's crash counters and restart logic provide a built-in "watchdog-like" system, minimizing downtime and potentially averting dangerous conditions if used wisely.

Development

Since Tinkwell is stretching across some very different use cases, I’m splitting it into multiple packages: a core foundation plus domain-specific modules (like the upcoming firmware-less one).

Most of the real action will unfold over in the GitHub repo but stay tuned! Future posts will zoom in on specific pieces, especially when we dive into building out the firmware-less module.

An inexpensive smart thermostat device using...ATtiny85

Adriano Repetti — Tue, 17 Jun 2025 20:52:36 +0000

Adriano Repetti

Jun 7 '25

IoT Architectures Under Pressure: Smart Thermostat, Hardware (Part 7)

#iot #diy #c #atmel

Comments 2

5 min read

IoT Architectures Under Pressure: Smart Thermostat, Hardware (Part 7)

Adriano Repetti — Sat, 07 Jun 2025 14:14:36 +0000

In previous posts, we developed the firmware for a cost-effective smart thermostat, embracing a firmware-less approach and then refined the original design. We started the development of a C# basic reference implementation for the Hub.

Now, we turn our attention to the hardware, demonstrating that a smart device doesn’t have to be more expensive (whether in design, development, production, or purchase) than compared to a traditional one.

Disclaimer: The code and electronic circuit designs provided in this post are intended for illustrative purposes only. They should not be considered best practices nor assumed to comply with industry safety standards or regulations. Before implementing any design, always refer to official documentation, safety guidelines, and certified standards applicable to your region or industry. The author assumes no responsibility for damages, malfunctions, or risks arising from the use of this information.

Hardware

In this example we assume I2C communication with the hub (you might need a bus buffer like NXP P2B96 for long distances).

The entire device is built around an Atmel ATtiny85 microcontroller, a Texas Instruments LM35 temperature sensor and a Sanyou SRD relay. Other variations are surely possible with minimal changes.

With this design, the estimated cost for all components is around 8 USD (€7) but it's way less than that if you buy in bulk.

This is not a production-ready design but if you're going to build something then remember:

C1, C2, R4 and R5 should be as close as possible to U1.
C4 and R1 should be near U2.
U2 should have its tiny heat sink and be as far as possible from other heat sources (ideally near the side of the PCB).
If you are really using a long I2C line then, as first step, you could change R4 and R5 to 10 kΩ and add a TVS diode like Texas Instruments TPD4E1B06.

Power supply is outside the scope of this post but a super simple design (from a 24V DC input you might get from the furnace) could be:

Software

Let's see the firmware, again we are going to keep it simple (refer to Atmel's Application Notes for a full I2C slave implementation in C) or jump to the C++ implementation (which is somewhat complete):

C Implementation

#include <avr/io.h>
#include <avr/interrupt.h>
#include <util/delay.h>
#include <stdint.h>

#define SLAVE_ADDRESS 0x50

#define NUMBER_OF_SAMPLES 6

#define FURNACE_ON_COMMAND '1'
#define FURNACE_OFF_COMMAND '0'
#define READ_TEMPERATURE_COMMAND 'r'

#define STATE_WAIT_ADDRESS    0
#define STATE_RECEIVE_COMMAND 1
#define STATE_TRANSMIT_DATA   2

#define MAX(a, b) ((a) > (b) ? (a) : (b))
#define MIN(a, b) ((a) < (b) ? (a) : (b))

volatile uint8_t received_command = 0;
volatile uint8_t i2c_state = STATE_WAIT_ADDRESS;

// Reads a single temperature sample from LM35
uint16_t read_raw_temperature(void) {
    // Select ADC channel, start conversion and waits for it to complete
    ADMUX = (1 << REFS0) | (1 << MUX1);

    ADCSRA |= (1 << ADSC);
    while (ADCSRA & (1 << ADSC))
        ;

    // Convert ADC reading to °C (LM35 calibration: 10mV/°C)
    return (5 * ADC * 100) / 1024;
}

// Reads multiple temperatures to reduce the noise
uint8_t read_temperature(void) {
    uint16_t sum = 0, min = UINT16_MAX, max = UINT16_MIN;

    for (uint8_t i = 0; i < NUMBER_OF_SAMPLES; ++i) {
        uint16_t sample = read_raw_temperature();
        sum += sample;
        min = MIN(min, sample);
        max = MAX(max, sample);
        _delay_ms(1);
    }

    // Compute average excluding min/max (reduces noise)
    uint16_t temperature = (sum - min - max) / (NUMBER_OF_SAMPLES - 2);

    return (uint8_t)(temperature >> 2);
}

void init_adc(void) {
    ADCSRA = (1 << ADEN)  // Enable ADC
           | (1 << ADPS2) // Set ADC prescaler to 128 (stable readings)
           | (1 << ADPS1)
           | (1 << ADPS0);
}

void init_i2c(void) {
    DDRB &= ~((1 << PB0) | (1 << PB2)); // Configure SDA & SCL as inputs
    USIDR = 0;
    USISR = (1 << USIOIF);  // Reset USI overflow interrupt flag
    USICR = (1 << USISIE)  // Enable USI start condition interrupt
          | (1 << USIWM1) | (1 << USIWM0) // Enable two-wire mode
          | (1 << USICS1); // Set clock source
}

ISR(usi_start_condition_handler) {
    USISR = (1 << USISIF) | (1 << USIOIF);
    i2c_state = STATE_WAIT_ADDRESS;
}

ISR(usi_handler) {
    uint8_t received = USIDR;

    switch (i2c_state) {
        case STATE_WAIT_ADDRESS:
            handle_address_received(received);
            break;

        case STATE_RECEIVE_COMMAND:
            handle_command_received(received);
            break;

        case STATE_TRANSMIT_DATA:
            handle_data_transmit();
            break;

        default:
            i2c_state = STATE_WAIT_ADDRESS;
            break;
    }
}

void reset_interrupt_flag(void) {
    USISR = (1 << USIOIF); 
}

void handle_address_received(uint8_t received) {
    uint8_t address = received >> 1;
    uint8_t rw = received & 0x01;

    if (address == SLAVE_ADDRESS)
        i2c_state = rw == 0 ? STATE_RECEIVE_COMMAND : STATE_TRANSMIT_DATA;

    reset_interrupt_flag();
}

void handle_command_received(uint8_t command) {
    received_command = command;

    if (received_command == FURNACE_ON_COMMAND)
        PORTB |= (1 << PB1);  // Turn furnace ON
    else if (received_command == FURNACE_OFF_COMMAND) {
        PORTB &= ~(1 << PB1); // Turn furnace OFF

    i2c_state = STATE_WAIT_ADDRESS;
    reset_interrupt_flag();
}

void handle_data_transmit(void) {
    if (received_command == READ_TEMPERATURE_COMMAND)
        USIDR = readTemperature();
    else
        USIDIR = 0;

    i2c_state = STATE_WAIT_ADDRESS;
    reset_interrupt_flag();
}

int main(void) {
    // Set PB1 as output for furnace control and ensure it's off
    DDRB |= (1 << PB1);
    PORTB &= ~(1 << PB1);

    init_adc();
    init_i2c();
    sei();

    while (1) {
        _delay_ms(1000);
    }

    return 0;
}

Alternative C++ Implementation

The code is pretty straightforward and if you are comfortable using some C++(ish) and the TinyWireS library then it becomes incredibly short:

#include <TinyWireS.h>

#define SLAVE_ADDRESS 0x50     // I2C address for this device
#define RELAY_PIN 1            // Relay Control Pin
#define LM35_PIN A2            // Analog pin for temperature sensor
#define NUMBER_OF_MEASURES 6   // Number of temperature samples (>= 3)
#define READING_DELAY 1000     // Delay between each temperature reading

#define FURNACE_ON_COMMAND '1'
#define FURNACE_OFF_COMMAND '0'
#define READ_TEMPERATURE_COMMAND 'r'

float lastKnownTemperature = 0;

void setup() {
    TinyWireS.begin(SLAVE_ADDRESS);
    TinyWireS.onRequest(sendTemperature);
    TinyWireS.onReceive(handleCommand);

    pinMode(RELAY_PIN, OUTPUT);
    digitalWrite(RELAY_PIN, LOW);
}

void loop() {
    lastKnownTemperature = readTemperature();
    delay(READING_DELAY);
}

void handleCommand(int numBytes) {
    if (TinyWireS.available()) {
        char command = TinyWireS.read();

        if (command == FURNACE_ON_COMMAND) {
            digitalWrite(RELAY_PIN, HIGH);
        } else if (command == FURNACE_OFF_COMMAND) {
            digitalWrite(RELAY_PIN, LOW);
        }
        // Let's assume that any read request is preceded
        // by a read command, it makes things simpler.
    }
}

void sendTemperature() {
    TinyWireS.write((int8_t)lastKnownTemperature);
}

float readTemperature() {
    float minimum = FLT_MAX;
    float maximum = FLT_MIN;
    float sum = 0;

    for (int i = 0; i < NUMBER_OF_MEASURES; i++) {
        float temp = analogRead(LM35_PIN) * (5.0 / 1023.0) * 100.0;

        if (temp < minimum) minimum = temp;
        if (temp > maximum) maximum = temp;

        sum += temp;

        delay(10);
    }

    sum -= (minimum + maximum);
    return sum / (NUMBER_OF_MEASURES - 2);
}

For completeness (assuming your're not using an IDE), a low speed fuses setup you might use in this case:

avrdude -c usbasp -p t85 -U lfuse:w:0x62:m -U hfuse:w:0xDF:m -U efuse:w:0xFF:m

And to compile and upload your firmware:

avr-gcc -mmcu=attiny85 -Os -c thermostat.cpp -o thermostat.o
avr-gcc -mmcu=attiny85 -o thermostat.elf thermostat.o
avr-objcopy -O ihex thermostat.elf thermostat.hex
avrdude -c usbasp -p t85 -U flash:w:thermostat.hex:i

Conclusions

From this minimal implementation, we can draw a few conclusions:

Our hardware is incredibly simple (and thus cost-effective), even simpler than a traditional non-smart thermostat. Not only is it simple, but it is also highly similar to a traditional implementation, meaning transition costs are negligible.
Our firmware is as streamlined as possible (both in our device and in the hub) eliminating the complexities often associated with connected devices (security, UI, compatibility, support, cloud infrastructure, etc.).

Developing a smart device can be both a sustainable and profitable option, benefiting vendors and customers alike.

IoT Architectures Under Pressure: Smart Thermostat, Scheduling (Part 6)

Adriano Repetti — Fri, 06 Jun 2025 15:27:57 +0000

In a previous post, we started designing a smart thermostat to show an alternative and cost-effective approach for IoT devices development.

In this post we are going to build on that initial MVP.

Scheduling

The first improvement is to start the furnace a bit earlier, ensuring that by the target time, the desired temperature has already been reached.

To keep things as simple as possible, we will ignore several factors:

Heat losses: These depend on various elements such as insulation, wall volume, and outside temperature. Ignoring other heat sources, we can approximate this with: $hnet=hin−hout≈hheater−U×A×(Toutside−Tinside)h_\mathrm{net} = h_\mathrm{in} - h_\mathrm{out} \approx h_\mathrm{heater} - U \times A \times (T_\mathrm{outside} - T_\mathrm{inside})$ . If you want a rough adjustment, adding 50% to the $h_s$ calculation could help account for it.
Time-dependent values: We assume they remain constant, even though they actually change over time.
Additional heat sources: Sensible and latent heat contributions from people, lighting, and electronic devices.
Environmental factors: Humidity and pressure variations.

While we are simplifying a lot, the reality is that we are building a smart thermostat, so these factors only matter up to a point. Future iterations of our device will collect data to learn precisely how many $°C/hour\mathrm{°C/hour}$ our system can achieve, accounting for seasonal variations and outside temperature.

In short, we're simply going to use:

h_s = c_p \times \rho \times q \times \varDelta t

Consequently:

\frac{h_s}{P}

Where:

$h_s$ = Sensible Heat in $kW\mathrm{kW}$ .
$c_p$ = Specific Heat of Air in $kJ/kg°C\mathrm{kJ/kg °C}$ .
$ρ\rho$ = Density of Air in $kg/m3\mathrm{kg/m^3}$ .
$q$ = Air Volume Flow in $m3/s\mathrm{m^3/s}$ .
$Δt\varDelta t$ = $ttarget−tcurrentt_\mathrm{target} - t_\mathrm{current}$ in $°C\mathrm{°C}$ .
$P$ = Heating element power in $kW/h\mathrm{kW/h}$ .
$T$ = Time required to reach the required temperature $ttargett_\mathrm{target}$ in hours.

We know that $cp=1.006kJ/kg°Cc_p = 1.006 \enspace \mathrm{kJ/kg °C}$ and $ρ=1.202kg/m3\rho = 1.202 \enspace \mathrm{kg/m^3}$ .

Let's calculate how long does it take to heat a room from 10 °C to 20 °C, assuming $\enspace \mathrm{m^3/s}$ and $\enspace \mathrm{kW}$ .

\begin{split}T &= \frac {(1.006 \enspace \mathrm{kJ/kg °C}) \times (1.202 \enspace \mathrm{kg/m^3}) \times (1 \enspace \mathrm{m^3/s}) \times (10 \enspace \mathrm{°C})}{10 \enspace \mathrm{kW}}\newline &= \frac{12.09212 \enspace \mathrm{kW}}{10 \enspace \mathrm{kW/h}} \approx 1.2 \enspace \mathrm{hours}\end{split}

Let's add a new configuration entry:

  "heatingPower": "10 kW"

If you know your furnace's power in BTU/h then remember that 1 W is (more or less) 3.41 BTU/h. The code to calculate how many hours in advance we need to turn on the furnace:

const SPECIFIC_HEAT_OF_AIR = 1.006;
const DENSITY_OF_AIR = 1.202;
const AIR_VOLUME_FLOW = 1; // This might be configuration

function calculateHoursToTemperature(
    context: Context,
    currentTemperature: i8,
    targetTemperature: i8) {
  const dt = targetTemperature - currentTemperature;
  if (dt < 0)
    return 0;

  const h = SPECIFIC_HEAT_OF_AIR * DENSITY_OF_AIR * AIR_VOLUME_FLOW * dt;
  return h / context.config.get<Measure>("heatingPower").toFloat("kW/h");
}

Now we can refactor our initial code:

function applyFurnaceStatus(context: Context, stream: IoStream) {
  // If we have a scheduled setting then we must honor it
  const currentSchedule = getScheduledTemperature(context, "current");
  if (currentSchedule.id !== null) {
    setTemperatureAndFurnace(context, currentSchedule.value);
    return;
  }

  // We do not have a target temperature for now, do we need to
  // start heating to be ready for the next one?
  const nextSchedule = getScheduledTemperature(context, "next");
  if (nextSchedule.id === null) {
    setTemperatureAndFurnace(context, currentSchedule.value);
    return;
  }

  const hoursToReachTemperature = calculateHoursToTemperature(
    context,
    getCurrentTemperature(stream),
    nextSchedule.value
  );

  if (hoursToReachTemperature >= nextSchedule.delay)
    setTemperatureAndFurnace(context, nextSchedule.value);
}

function getCurrentTemperature(stream: IoStream) {
  stream.writeByte(READ_TEMPERATURE_COMMAND);
  stream.flush();

  return F32.parseFloat(stream.readLine());
}

function getScheduledTemperature(context: Context, select: "current" | "next") {
  const scheduling = Scheduling.resolve("schedule", select);
  return { ...scheduling, value: toValue(scheduling) };

  function toValue(id: string) {
    if (id === "1")
      return context.variables.get("desired_temp_1");

    if (schedule.id === "2")
      return context.variables.get("desired_temp_2");

    return null;
  }
}

function setTemperatureAndFurnace(context: Context, temperature: i8) {
  context.variables.set("target_temp", temperature);

  if (temperature === null) {
    setFurnaceStatus(stream, false);
    return;
  }

  const currentTemperature = getCurrentTemperature(stream);
  const isFurnaceActive = currentTemperature < temperature;

  setFurnaceStatus(stream, isFurnaceActive);
}

function setFurnaceStatus(stream: IoStream, active: bool) {
  if (context.variables.get<bool>("furnace") != active) {
    stream.writeByte(status ? FURNACE_ON_COMMAND : FURNACE_OFF_COMMAND);
    context.variables.set("furnace", status);
  }
}

Change Schedule According to Weather Forecast

There are a few more basic features to add, but with a working scheduler in place, we can finally introduce our first smart feature. When the temperature difference between inside and outside exceeds a configured threshold, we will increase $h_s$ by 50% (based on configuration).
To achieve this, we’ll leverage a service exposed by our host, following this contract in Google protobuf (because, let’s be honest, no one wants to write or read WASM bindings):


syntax = "proto3";

import "google/protobuf/timestamp.proto";

package Weather;

service ForecastService {
  rpc get_current_weather (location) returns (current_weather);
  // ...
}

message location {
  float latitude = 1;
  float longitude = 2;
  // ...
}

message current_weather {
  google.protobuf.Timestamp timestamp = 1;
  float temperature = 2;
  float feels_like = 3;
  // ...
}

If you are wondering why protobuf: because under the hood we may expose all services with gRPC (enabling load balancing among multiple hubs, secure connections and even seamlessly integration with remote cloud services). We might see more of this when we are going to discuss a reference hub implementation in a separate series (because this approach isn't limited to IoT applications!).

Now we can add a few configuration options:

  "criticalDifferenceWithExternalTemperature": "10 °C",
  "heatLossesCompensation": 1.5

And change calculateHoursToTemperature() accordingly:

function calculateHoursToTemperature(
    context: Context,
    currentTemperature: i8,
    targetTemperature: i8) {

  const dt = targetTemperature - currentTemperature;
  if (dt < 0)
    return 0;

  let h = SPECIFIC_HEAT_OF_AIR * DENSITY_OF_AIR * AIR_VOLUME_FLOW * dt;

  const externalTemperature = readExternalTemperature(context);
  const maxExternalDelta = context.config.get<Measure>("criticalDifferenceWithExternalTemperature").toFloat("°C");
  if (currentTemperature - externalTemperature >= maxExternalDelta)
    h *= context.config.get<f32>("heatLossesCompensation");

  return h / context.config.get<Measure>("heatingPower").toFloat("kW/h");
}

function readExternalTemperature(context: Context) {
  return context.services.get<ForecastService>("Weather.ForecastService").getCurrentWeather().temperature;
}

Just like that, we’ve introduced our first true smart feature. Notice how we didn’t have to venture beyond our domain of thermostat design, and the hardware (covered in the next post) is even more affordable than older models.

Of course, there’s much more involved in creating a fully smart thermostat, such as data collection, presets and the ability to learn the optimal base temperature when you're away. However, those enhancements lie beyond the scope of these posts.

IoT Architectures Under Pressure: Smart Thermostat, an Example (Part 5)

Adriano Repetti — Sat, 24 May 2025 20:27:56 +0000

In a previous post, we explored why designing a smart device can be more complex (and costly) than expected, both for businesses and consumers. We then introduced an alternative approach that enables resource pooling and shared services, allowing us to focus on our core business. Finally, we proposed using WebAssembly to address key challenges in this model.

Now, it's time to put theory into practice, let’s dive into the code!

Smart Thermostat Specifications

We’re building a minimal thermostat implementation; while there are many additional features we could include and more advanced algorithms to optimize performance, this example is purely for illustration. Real specifications require much greater detail, my goal here is to provide a concise overview that quickly illustrates our design objectives.

Core Functionality

We aim to develop a smart thermostat to control an HVAC system, starting with heating-only (no AC support in the first release).

Hardware Considerations

Currently, the system features one indoor temperature sensor.
Future support for multiple sensors and zones is planned (e.g., adjusting temperatures in sleeping areas or integrating an outdoor sensor for optimized heating).
The control mechanism is a simple 24V relay to activate (on/off switch) the unit but there are plans for the future to add support for PWM control.

Basic Features

Users can set two target temperatures, T1 and T2, with a schedule that switches between them in 30-minute intervals.
The system could include several preset modes to override the schedule but we are going to start with these:
- Away – The system remains off unless the temperature drops below TMin.
- Party – The system remains on, maintaining T1 continuously.
- Normal – The system follows the scheduled T1 and T2 temperatures.

Smart Features

Temperature control and schedule are managed via the Hub UI, including mobile app access for remote control.
Weather forecast data (if available) is used to determine cost-effective heating strategies.
The system collects data to analyze HVAC efficiency, user habits, and develop future energy-saving optimizations.

Development Considerations

Hardware development has not started yet, and the final board specifications remain undefined.
The connection to the hub is TBD: it could use I2C, CAN, USB, or Bluetooth.
Despite the lack of hardware, we want to begin software development immediately, ensuring a strong foundation before integration.
We aim to sell this device at the same price of its "traditional" counterpart. We do not want recurring extra costs unless there is a definite ROI (for example data storage).

Hardware

Since we don’t have a working board yet, we’ll need a way to simulate it. A software-based simulation running as a separate service in our Hub could serve this purpose. We need to tentatively agree with the hardware team about a communication protocol, for now (also considering the future expected developments):

Hub and device communicates using a line-by-line communication of ASCII encoded strings.
The hub always initiates the communication.
Commands are case sensitive and leading/trailing spaces are not discarded.
The hub sends a "read temperature" command r to which the device replies with a single integer number which is the temperature in °C. It could optionally have a leading sign + or -.
The hub can send a "turn on the heating" command 1 or "turn off the heating" command 0. The device does as instructed without replying.

While a simulation helps in the early stages, it has limitations. To bridge the gap, we’ll quickly develop a device using an Arduino while waiting for the hardware team to finalize a prototype. Meanwhile, the software simulation will also serve as a crucial tool for integration tests, ensuring no time is wasted.

To start, we could implement something as simple as this (see Part 3 of this series, we are using AssemblyScript again in this example):

import { Context } from "firmwareless/hosting"
import { IoStream } from "firmwareless/lib"

const TEMPERATURE = "temperature";
const FURNACE = "furnace";
const READ_TEMPERATURE_COMMAND = "r";
const FURNACE_ON_COMMAND = "1";
const FURNACE_OFF_COMMAND = "0";

let channel: IoStream | null = null;

export function setup(context: Context) {
    // This is the temperature we report when queried. To help
    // debugging we can edit this value manually in the UI to
    // observe the effects.
    context.status.register<i8>({
        id: TEMPERATURE,
        type: "environment/temperature",
        unit: "Celsius",
        editable: true,
        range: { nullable: true, minimum: 0, maximum: 40, step: 1 }
    });

    // This is the relay we control, we can observe this in the UI
    // to determine if everything is working as expected.
    context.status.register<u8>({
        id: FURNACE,
        type: "status/boolean",
    });

    // To simulate a physical device we open a stream for character 
    // device. The hosted firmware with our logic is agnostic
    // of the transport mechanism!
    channel = context.communication.stream.open({
        mode: "read-write",
        encoding: "ascii",
        onWrite: handleWrite
    });
}

export function teardown(context: Context) {
    channel?.close();
}

function handleWrite(context: Context, stream: IoStream, data: string) {
    if (data === READ_TEMPERATURE_COMMAND) {
        stream.writeLine(context.status.get<i8>(TEMPERATURE).toString());
    } else if (data === FURNACE_ON_COMMAND) {
        context.status.set<u8>(FURNACE, 1);
    } else if (data === FURNACE_OFF_COMMAND) {
        context.status.set<u8>(FURNACE, 0);
    }
}

It’s minimal, it lacks logging and even basic error checks, but it’s functional and gets us started.

Meanwhile, another team member is quickly assembling a physical device using an Arduino we had in a drawer. It’s an overkill—far more powerful (and expensive) than necessary—but it will get the job done in 30 minutes, and that’s what matters for now. In a future post, we’ll outline a real hardware design (maybe using a 50 cents ATtiny85 microcontroller, see Part 7 of this series).

We're going to use the onboard LED to check when the furnace should be on. In this example we used an Arduino Nano and an analogic LM35 temperature sensor, please refer to products' datasheets for the appropriate configuration and code. For example, in real applications you really need a few passive components to have clean readings (or in some cases any reading at all); from the LM35 datasheet:

In simple cases (surely if you're using a breadboard) you could use a R-C damper:

Now let's write the code:

#include "Arduino.h"
#include <stdint.h>

#define NUMBER_OF_READINGS_PER_MEASURE 4

#define TEMPERATURE_SENSOR_APIN A0
#define FURNACE_STATUS_INDICATOR LED_BUILTIN

#define READ_TEMPERATURE_COMMAND 'r'
#define FURNACE_ON_COMMAND '1'
#define FURNACE_OFF_COMMAND '0'

int8_t readTemperature();
void setFurnaceStatus(bool active);
char tryReadCommandFromMaster();

void setup() {
  pinMode(FURNACE_STATUS_INDICATOR, OUTPUT);
  Serial.begin(9600);
}

// We keep waiting for a command from the serial port, inputs
// are always commands (we ignore what we do not know) and the
// only output is the temperature (when we're asked to).
// Note that this is not what you would do in a real application
// but it mimics (more or less) how it could work with I2C.
void loop() {
  switch (tryReadCommandFromMaster()) {
    case READ_TEMPERATURE_COMMAND:
      Serial.println(readTemperature());
      break;
    case FURNACE_ON_COMMAND:
      setFurnaceStatus(true);
      break;
    case FURNACE_OFF_COMMAND:
      setFurnaceStatus(false);
      break;
  }
}

// The readings are fairly noisy, for this example it's
// enough to calculate a simple average.
int8_t readTemperature() {
  int16_t value = 0;
  for (int8_t i=0; i < NUMBER_OF_READINGS_PER_MEASURE; ++i) {
    value += analogRead(TEMPERATURE_SENSOR_APIN);
  }

  return (int8_t)((5 * value * 100.0) / 1024 / NUMBER_OF_READINGS_PER_MEASURE);
}

// This is a development board, we use a LED instead of
// turning on/off the heating.
void setFurnaceStatus(bool active) {
  digitalWrite(FURNACE_STATUS_INDICATOR, active);
}

// We read one byte (instead of a full line) because currently
// the supported commands are one byte only and we ignore what
// we do not know how to process (for example spaces and new lines).
int16_t tryReadCommandFromMaster() {
  if (Serial.available()) {
    return Serial.read();
  }
}

This is all what we need to start developing our Hosted Firmware.

Software

In this example we are not going to describe our setup with code, our firmware comes together with a JSON descriptor. Note that all the temperatures are in °C, the UI will present the values to the user using their preferred unit (for example °F).

{
  "id": "76a38d89-d756-4412-ac87-604ff3cf84d0",
  "vendor": "Acme",
  "name": "Smart Thermostat Mod. 1",
  "version": "1.0.0",
  "compatibility": "1.0+",
  "channel": {
      "initiator": "host"
  },
  "config": {
      "monitoringInterval": "15 minutes",
      "schedulingInterval": "30 minutes",
      "minimumTemperature": "8 °C",
      "maximumTemperature": "30 °C"
  },
  "variables": [
    {
        "name": "furnace",
        "storage": "uint8",
        "type": "boolean"
    },
    {
        "name": "current_temp",
        "storage": "int8",
        "type": "measure/temperature",
    },
    {
        "name": "target_temp",
        "storage": "int8",
        "type": "measure/temperature",
    },
    {
        "name": "desired_temp_1",
        "storage": "int8",
        "type": "measure/temperature",
        "default": "16 °C"
    },
    {
        "name": "desired_temp_2",
        "storage": "int8",
        "type": "measure/temperature",
        "default": "18 °C"
    },
    {
        "name": "schedule",
        "label": "Schedule",
        "type": "system/schedule",
        "editable": true,
        "editorOptions": {
            "interval": "week",
            "granularity": "{{ config.schedulingInterval }}",
            "selection": "list",
            "default": "0",
            "listItems": [
                { "key", "0", "label", "Off" },
                { "key", "1", "label", "{{ variables.desired_temp_1 }}" },
                { "key", "2", "label", "{{ variables.desired_temp_2 }}" }
            ]
        }
    }
  ]
}

That's enough to start a quick MVP, we're going to add the missing bits later. The key detail to note is initiator in the channel section—it determines who initiates communication. In this case, our hosted firmware takes the lead. However, for battery-powered devices, the roles could be reversed: instead of initiating communication, the firmware would read buffered data and queue commands to be executed during the next connection, optimizing power consumption.

Our firmware starting point implementing a simple on/off control mechanism which follows the user-defined schedule:

import { Context, Scheduling } from "firmwareless/hosting"
import { IoStream, Interval, Temperature } from "firmwareless/lib"

const READ_TEMPERATURE_COMMAND = "r";
const FURNACE_ON_COMMAND = "1";
const FURNACE_OFF_COMMAND = "0";

let channel: IoStream | null = null;

export function setup(context: Context) {
  // This is how often we are going to check for the temperature.
  context.schedule(Interval.parse(context.config.get("monitoringInterval")), main);

  // When these variables change value we need to recalculate
  // our status because they represent the desired temperatures
  // and our scheduling.
  context.variables.onChange(["desired_temp_1", "desired_temp_2", "schedule"], main);
}

export function teardown(context: Context) {
  // If we are going down then we want to be sure we are not
  // leaving the heater on!
  const stream = IoStream.Open(context.associatedDeviceId);
  try {
    stream.writeByte(FURNACE_OFF_COMMAND);
  }
  finally {
    stream.close();
  }
}

function main(context: Context) {
  const stream = IoStream.Open(context.associatedDeviceId);
  try {
    applyFurnaceStatus(context, stream);
  }
  finally {
    stream.close();
  }
}

function applyFurnaceStatus(context: Context, stream: IoStream) {
  const desiredTemperature = resolveDesiredTemperature(context);
  context.variables.set("target_temp", desiredTemperature);

  if (desiredTemperature === null) {
    stream.writeByte(FURNACE_OFF_COMMAND);
    return;
  }

  stream.writeByte(READ_TEMPERATURE_COMMAND);
  stream.flush();

  const temperature = Temperature.parse(stream.readLine(), "°C");
  const status = temperature < desiredTemperature;

  stream.writeByte(status ? FURNACE_ON_COMMAND : FURNACE_OFF_COMMAND);
  context.variables.set("furnace", status);
}

function resolveDesiredTemperature(context: Context) {
  // To "resolve" the desired temperature we need to read the list
  // of scheduled values from the "schedule" variable and pick the
  // selected one for the current date and time.
  // It's so common that we have an helper function for that.
  const temperatureId = Scheduling.resolve("schedule");

  if (temperatureId === "1") {
    return context.variables.get("desired_temp_1");
  }

  if (temperatureId === "2") {
    return context.variables.get("desired_temp_2");
  }

  // A value of "0" or an unknown key means "off".
  return null;
}

Please note that the code is minimal, focusing solely on our core business (building thermostats) which was our original goal.

Final Step: The UI

To make this approach truly effective, the UI must not be tied to a specific technology or framework (like React or even plain HTML). Instead, we need a technology-agnostic representation of the page—allowing different hubs to render it in various ways.

One hub might use HTML, another could leverage Qt, while mobile apps might have entirely different rendering engines.
The goal is flexibility: ensuring seamless adaptation across platforms without forcing a single UI paradigm.

Let's add a new section ui to our JSON file (note that JSON might not be the best way to represent the UI, consider this just pseudo-code):

{
    ...
    "ui": [
        {
            "type": "page",
            "title": "Thermostat",
            "content": {
                "control": "ring",
                "label": "Temperature",
                "minimum": "{{ config.minimumTemperature }}",
                "maximum": "{{ config.maximumTemperature }}",
                "value": "{{ variables.current_temp}}",
                "steps": [ "{{ variables.target_temp }}" ],
                "text": true,
                "actions": [
                    { 
                        "type": "edit",
                        "target": "schedule",
                        "label": "Schedule"
                    },
                    { 
                        "type": "edit",
                        "target": "desired_temp_1",
                        "label": "Temperature 1"
                    },
                    { 
                        "type": "edit",
                        "target": "desired_temp_2",
                        "label": "Temperature 2"
                    }
                ]
            }
        }
    ]
}

Now, we can edit our schedule, monitor the temperature, and ensure everything is running smoothly.

In the next post, we'll add some smart features and refine the algorithm to make it more efficient and less rudimentary.

IoT Architectures Under Pressure: Flexibility (Part 4)

Adriano Repetti — Tue, 20 May 2025 16:29:21 +0000

Democratizing IoT Development

In previous posts, we explored how IoT development doesn’t have to involve huge initial investments, ongoing expenses, or excessively costly devices. Instead, we can build an interconnected world through a more accessible and cost-effective approach.

Why democratic? Because by removing complexity, we empower anyone to design task-specific IoT devices without compromising usability or security.
This approach fosters experimentation, you can prototype using:

Arduino + shields
Raspberry Pi + hats
Any development board at hand

By lowering barriers to entry, IoT innovation becomes more inclusive, allowing developers of all levels to create functional, secure, and efficient devices without massive investments.

In this post, we'll outline the general architecture, how it differs from mainstream IoT models, and the flexibility it offers. Since no single solution fits all, integration must be seamless, allowing diverse devices to operate within the same ecosystem while accommodating varied requirements.

Cinque...dieci...venti

Or, more appropriately, three, four, five, six and seven. If you are going to pick an architecture for your IoT projects, you could possibly have more layers than you can count.

I am not going to repeat here what you can find (better written) elsewhere in the literature. These are a good starting point which presents both an overview (if mostly focused around security) and some good references:

Burhan, M., Rehman, R. A., Khan, B., & Kim, B. S. (2018). IoT Elements, Layered Architectures and Security Issues: A Comprehensive Survey. Sensors (Basel, Switzerland), 18(9), 2796.

Schiller, E., Aidoo, A., Fuhrer, J., Stahl, J., Ziörjen, M., & Stiller, B. (2022, May 1). Landscape of IoT security. Computer Science Review. Elsevier Ireland Ltd.

Why am I not going to focus on this? Because, mostly, for simple IoT devices these details should not matter and it's too easy to get them wrong. Leave them to someone else and focus on the domain you know better: your core business.

A Simplified Approach to IoT

For simple IoT devices certain implementation details aren’t critical. Worrying about every technical aspect can add unnecessary complexity. Instead, delegating these challenges to a dedicated team allows developers to focus on the core functionality without getting overwhelmed.

Intent

To minimize device costs while ensuring fast and easy development. The goal is to avoid the complexity (and expense) of deploying a network-attached device.

Firmware-less is NOT:

A replacement for any of the architectures discussed earlier. Something needs to be connected to the Cloud, manage security, updates, UIs and data collection but, ideally, it's not your water quality monitoring device!
A default solution for every IoT project: it’s context-dependent.
A standalone approach unless an Open Standard is adopted across vendors (or you control both the devices and the hub).

Motivation

As outlined in the first post of this series, IoT development often leads to spiraling costs and overwhelming technical burdens, even before delivering actual value to the business.

Applicability

This approach isn’t a one-size-fits-all solution, it works best when:

The device already follows a traditional design model.
Real-time operations are not a requirement.

I'm trying to limit the scope of these posts to Smart Homes but the very same principles apply to a variety of applications. Some devices that could easily benefit from this approach are: thermostats, robot cleaners, alarm systems, AC units, water quality monitoring, air quality monitoring, irrigation systems, structural building monitoring and smart meters.

However, the flexibility of this method allows it to be integrated into various system architectures, depending on specific needs.

Structure

As covered in previous posts, the idea is simple:

Keep devices as minimalist as possible, avoiding unnecessary firmware complexity.
Shift all logic to a centralized hub, pooling resources efficiently across connected devices.

An Example

This approach supports a range of heterogeneous devices, allowing integration without requiring every device to be fully firmware-less.

Imagine this potential setup:

Firmware-less hub – A mini-computer running Linux, connected to Amazon AWS, Microsoft Azure, Google Cloud, or another provider for IoT services like updates and data collection.
Firmware-less thermostat – Uses a temperature sensor and actuator, with all logic, UI, and smart features handled by the hub. This is the example we're going to pick, starting with the next post.
AC unit with a tiny MCU – UI and smart features provided by the hub.
Air quality monitoring system with a tiny MCU – Alerts and additional intelligence processed by the hub.
Garden irrigation system – Weather forecast, presence detection, and automation managed by the hub.
Fully automatic coffee machine with a tiny MCU – Advanced features (like brewing coffee after your morning alarm) handled by the hub.
Alarm system with a tiny MCU – Remote monitoring and alerts managed by the hub.
Video doorbell with a small mini-computer – Integrates with the alarm system, remote monitoring, and smart automation via the hub.

This setup demonstrates that devices don’t need to be entirely firmware-less to integrate into the ecosystem. Even devices using other IoT services can be incorporated by exposing their services, ensuring seamless communication across platforms.

What About the Hub?

Investing in a smart hub offers compelling business advantages:

Focused Platform, Not Fragmented Devices – Instead of competing to build the "best" smart device in every category, you create a versatile, centralized system that enhances multiple devices seamlessly.
Stable & Predictable Revenue Model – A subscription-based service keeps users' systems up-to-date while continuously delivering new features.
Data-Driven Insights & Optimization – With user-permissioned data from diverse sensors across multiple vendors, you can:
- Provide advanced functionalities and automation.
- Offer valuable analytics and predictive insights.
- Improve user experience, security, and device efficiency.

A firmware-less hub is not just about convenience: it creates a sustainable ecosystem that benefits both users and developers while establishing a scalable, revenue-generating foundation.

We are going to try to develop a basic reference implementation (which is not dedicated only to IoT applications) in C#, check the GitHub repository for more info.

What's Next?

I’ve written (more than) enough to illustrate the concept; now it’s time to bring it to life! In the next post, we’ll dive into coding a minimal (but fully functional) firmware-less smart thermostat.

IoT Architectures Under Pressure: Hosting a Portable Firmware (Part 3)

Adriano Repetti — Fri, 16 May 2025 08:00:00 +0000

As discussed in the previous post, shifting firmware from individual devices to a centralized hub offers significant benefits, pooling resources for multiple connected devices.

Key Obstacles

However, several challenges must be addressed:

Firmware isolation – Each device’s software must be securely separated.
Unknown hub architecture – The hub's OS, CPU, and other specifications are unpredictable.
Avoiding ABI/ISA fragmentation – Supporting multiple ABIs and ISAs would be impractical and expensive.

Potential Solutions

A few possible approaches come to mind, but each has drawbacks:

Virtual machines or lightweight containers – While they offer isolation, they don’t fully resolve compatibility issues. An emulation layer could help, but would increase cost and reduce efficiency in the long run.
Interpreted languages – Using Python or JavaScript could simplify portability but introduces severe limitations. Pairing them with containers might improve isolation, but would still fall short of a universal solution.
Bytecode virtual machines – Java VM offers better flexibility but restricts developers to supported languages, limiting versatility.

Seeking a Better Alternative

Ideally, we need a standardized, open, language-agnostic solution with a sound type system, enabling a Language-based System approach.

One promising option is WebAssembly (WASM)—a lightweight, efficient, and cross-platform execution model. WASM already supports an increasing number of programming languages, including C, which facilitates the porting of existing code.

Other options, such as BEAM (the Erlang VM), provide robustness but lack widespread adoption for general commercial consumer applications.

Solving Key Challenges with WebAssembly (WASM)

Embracing WASM as our virtual ISA presents several advantages:

Broad language support – Numerous programming languages target WASM with varying degrees of support. See Awesome WebAssembly Languages. This ensures familiarity for firmware developers and lowers the learning curve.
Sound Type System – WASM guarantees type safety and memory safety within its semantics, enhancing reliability (source).
Platform Agnostic Execution – As a virtual ISA, WASM can be compiled AOT, JIT, or interpreted, making it highly adaptable to present and future platforms.

Choosing a WASM Runtime

Several WASM runtimes are available to execute our code. In this example, we'll use Wasmer, though other options exist. If we compile AOT (Ahead-of-Time), we don’t even need a runtime at all!

An example

We're going to write a fictional (and minimal!) firmware for our smart thermostat, let's assume we decided to use AssemblyScript (but it could have been in C, Rust, Go or any other supported language):

import { Context } from "firmwareless/hosting"
import { Interval, IoStream } from "firmwareless/lib"

export function setup(context: Context) {
    context.status.register<i8>("current_temperature", "Temperature");
    context.status.register<i8>({
        id: "desired_temperature",
        label: "Desired temperature",
        type: "environment/temperature",
        unit: "Celsius",
        editable: true,
        editor: "Knob",
        indicator: "Ring",
        range: { nullable: true, minimum: 12, maximum: 30, step: 1 }
    });
    context.status.register<u8>({
        id: "furnace",
        label: "Active",
        type: "status/boolean",
        indicator: "Led",
    });

    context.schedule(main, { interval: Interval.FromMinutes(1) });
}

function main(context: Context) {
    const stream = IoStream.Open(context.associatedDeviceId);

    const currentTemperature = stream.readInt8();
    context.status.set("current_temperature", currentTemperature);

    const desiredTemperature = 
        context.status.get("desired_temperature");

    const furnaceIsActive = currentTemperature < desiredTemperature;
    stream.write(furnaceIsActive);
    context.status.set("furnace", furnaceIsActive);
}

Note how we described our setup programmatically but it could have been done with a simpler JSON descriptor. To compile it:

asc ./thermostat.ts -o ./thermostat.wasm --optimize

Now, the final step is uploading the firmware to a Public Firmware Repository (obviously it's not the only option, the device itself could provide its firmware to the host!).

When customers purchase our device, the hub will automatically download and install the firmware, simplifying onboarding and updates.
The hub (or a separate cloud service) is responsible for compiling the code into a native executable, if we choose this approach. With Wasmer, the process is straightforward:

wasmer create-exe thermostat.wasm -o ./thermostat --target=aarch64-linux-gnu

This setup ensures seamless deployment while keeping the firmware adaptable across platforms.

Where Are We?

At its core, this approach mirrors the way traditional desktop applications or device drivers function. Instead of embedding complex firmware directly into a device, we're simply offloading it to a hub: just as a computer executes software or loads drivers to communicate with peripherals.

We've developed a smart device that:

Costs less than a traditional one—requiring only an I2C temperature sensor and a tiny relay.
Empowers customers to spend less while choosing the hub that best suits their needs, with full control over updates.
Supports multiple programming languages, allowing developers to use familiar tools without retraining.
Simplifies development and debugging: the firmware runs unchanged on a development machine or even in a browser.
Includes an up-to-date UI, security, and mobile app—all without writing a single line of code.
Future-proof design ensures compatibility across CPUs, hub OSes, smartphone OSes, and input methods.

Looking Ahead

While this solution addresses key challenges, there are still important decisions to make. Future posts will explore these aspects further.

Key Considerations:

Flexibility – Devices can range from firmware-free (as in this example) to high-powered computing units, tailored to specific needs. This will be covered in the next post.
Standardization – The necessary tools and protocols already exist; now, we need a common standard for vendors to adopt. Ideally we have Matter and everything should integrate together (old and new).

If you want to jump ahead, directly to a practical example then you can go to the 5th post of this series where we start to design a smart thermostat.

[Boost]

Adriano Repetti — Tue, 13 May 2025 13:40:46 +0000

Adriano Repetti

May 13 '25

IoT Architectures Under Pressure: a Firmware-less Approach (Part 2)

#programming #architecture #iot #smartdevices

Comments 3

4 min read

IoT Architectures Under Pressure: a Firmware-less Approach (Part 2)

Adriano Repetti — Tue, 13 May 2025 09:10:47 +0000

TL;DR

By drawing inspiration from solutions in other domains, we should embrace a firmware-less approach for smart devices, ensuring greater flexibility, security, and long-term adaptability.

By centralizing device firmware on a hub or edge device, we eliminate redundant feature development across multiple devices. This reduces development and support costs but also simplifies maintenance.

Recap

In the previous post, we highlighted some (though certainly not all) of the challenges involved in building a smart device, focusing primarily on unforeseen costs and complications.

Post-sale costs often catch developers and customers off guard. Many products unexpectedly reach their end-of-support, leaving users with expensive devices they can no longer use at full capacity. This creates adoption anxiety, slowing the widespread integration of smart devices into our lives. Combined with high prices, poor security, and multiple cumbersome applications, these issues become significant barriers to adoption.

As companies and developers, we have a responsibility to support our customers to the best of our ability. Building smart devices isn’t just about innovation—it’s about ensuring long-term usability, security, and trust.

Rationale

Let's consider a few real-life scenarios:

When traveling from New York to Tokyo, most people don’t buy or even rent a private jet: they share existing infrastructure.
You don’t need separate computers for Microsoft Word and email: one device can handle multiple tasks.

Similarly, many features we expect from smart devices should be shared across multiple systems, rather than being duplicated on individual devices:

Internet connectivity.
User interfaces (local control panels, web interfaces, mobile apps).
User authentication.
Security and over-the-air updates.
Data collection.

In many cases (especially in a smart home setup) computing resources can also be shared. Input methods, CPU, memory, and graphics are often underutilized, making resource pooling a practical and efficient approach.

The very same idea we applied in serverless computing could be applied here: that's why I'm calling this firmware-less (with or without hyphen? leave a comment).

An Overview

Let's compare a possible traditional approach with a firmware-less approach.

A Traditional Approach

Our traditional smart thermostat is engineered somehow like this:

The fragmentation of interfaces and connectivity across smart home devices leads to redundancy, inefficiency, and complexity. Every new addition (whether an AC unit, alarm system, or smart speaker) introduces another screen, remote, or app, making interaction cumbersome instead of seamless.

A Firmware-less Approach

A more integrated approach could streamline smart home management, reducing unnecessary duplication and clutter while enhancing usability. Centralized control, shared UI frameworks, and edge computing could help consolidate resources so devices operate more efficiently and cohesively.

Smart devices could be thin clients, connecting to a hub that handles processing and UI rendering. Think of it like how cloud gaming works: instead of every gaming device needing high-end hardware, a central server does the heavy lifting.

At first both approaches seem similar but note a few differences:

The edge device hosts our firmware (in theory our thermostat wouldn't even need a microcontroller at all).
All the expensive resources are shared: the edge device hosts multiple firmware and they all share CPU, memory, Internet connection, UI and cloud services.
Running on the same device it's easy for multiple applications to expose their own services and to use other services, when available, to add functionalities.

Which are the advantages?

Lower development costs – Comparable to a traditional non-smart device.
More affordable – Potentially cheaper than both other smart devices and even traditional non-smart thermostats.
No running costs – Updates are only required for the hub, which can be sourced by the user from any vendor and maintained via a subscription program.
Long-term relevance – Your product remains secure and up-to-date without additional company costs.
Better user experience – Simplifies interactions and eliminates redundant interfaces.
Stronger security management – Centralized control ensures consistent updates and authentication.
A service-oriented approach – Enables exciting possibilities:
- Your AC unit could integrate with the alarm system to detect occupancy, optimizing energy use or adjusting when windows are open.
- Your floor-cleaning robot could serve as an additional security layer within your alarm system.
- Automation tasks become easier to set up (we all love Node-RED but it's not for everyone!)
- Write a comment if you can think of other integration!

Of course, there are technical challenges to address, and we will explore them in the next post. However, this is not a disruptive approach: we already have all the necessary tools, and these patterns have been successfully tested across various industries and applications. Rather than reinventing the wheel, we should embrace a sustainable and efficient solution.