Forem: Mangirdas Judeikis

What is Azure IoT Hub and how do I use it?

Mangirdas Judeikis — Tue, 09 Nov 2021 08:51:18 +0000

Synpse is an end-to-end platform to manage your device fleet that can grow to hundreds of thousands of devices, perform OTA software updates, collect metrics, logs, deploy your containerized applications and facilitate tunnel-based SSH access to any of your device. You can find a Quick Start here.

Intro into Azure IoT Hub

We often get questions how Synpse is compared or competes with Azure IoT Hub service. The short answer is that they operate in slightly different domains. Azure IoT hub focuses on application connectivity of the devices while Synpse targets deployment of the applications that may or may not be using services such as IoT Core.

The best results are achieved when solutions are used together. For example, when you build an application locally that utilizes AWS IoT Core message broker or device state services and then use Synpse to distribute your application to thousands of devices.

Example application

In this tutorial, we will deploy a simple open-source application that collects metrics and send them to AWS IoT Core for further processing. All code for this blog post can be found at:

https://github.com/synpse-hq/metrics-nats-example-app - Sample metrics application
https://github.com/synpse-hq/azure-iot-hub-example - Azure IoT Hub example

Steps:

Create Azure IoT hub
Configure rules to forward results into Azure blob storage
Create Azure "Thing"/IoT device for Synpse
Demo Synpse application from 3 microservices - Metrics demo, NATs messaging, Azure IoT python forwarder containers

Technologies used

Synpse - manage devices and deploy applications to them
NATs - a lightweight message broker that can run on-prem
Azure IoT Hub - message broker between all devices and Azure

Contrary to AWS, Azure was easy to configure using CLI. All steps here will be done using Azure CLI.

Azure IoT Hub

Create Azure IoT hub:

az iot hub create --resource-group MyResourceGroup --name MyIotHub --location eastus --tags synpse=true

Create certificate based on Azure documentation.

Upload the certificates for Azure device by creating "device-identity"

Thumbprint 1: SHA1 Fingerprint=56:0E:78:56:74:F1:1B:60:73:AA:7C:8E:12:73:C4:62:01:D5:A3:10 
Thumbprint 2: SHA1 Fingerprint=81:EB:0B:27:BB:3F:BB:D8:91:AF:38:28:BE:83:E5:46:C3:0F:4D:DE

# remove colons ':' from the fingerprints

az iot hub device-identity create -n MyIotHub -d synpse --am x509_thumbprint --ptp "560E785674F11B6073AA7C8E1273C46201D5A310" --stp "81EB0B27BB3FBBD891AF3828BE83E546C30F4DDE"

For this example, we gonna create a message route to the storage account blob.

Create storage account:

az storage account create -n MyStorageAccountName -g MyResourceGroup -l eastus

Create container/bucket for results:

az storage container create --account-name MyStorageAccountName -n metrics

Create IoT hub endpoint for message routing:

storageConnectionString=$(az storage account show-connection-string --name MyStorageAccountName --query connectionString -o tsv)

az iot hub routing-endpoint create --resource-group MyResourceGroup --hub-name MyIotHub \
        --endpoint-name storage --endpoint-type azurestoragecontainer --endpoint-resource-group MyResourceGroup \
        --endpoint-subscription-id $(az account show | jq -r .id) --connection-string $storageConnectionString \
        --container-name metrics --batch-frequency 60 --chunk-size 10 \
        --ff {iothub}-{partition}-{YYYY}-{MM}-{DD}-{HH}-{mm}

Use routing in question with our HUB (endpoint name is same as --endpoint-name)

az iot hub route create -g MyResourceGroup --hub-name MyIotHub --endpoint-name storage --source-type DeviceMessages --route-name Route --condition true --enabled true

Deploy an application

Deploy an application. Modify application YAML with your thing endpoint.

Create certificate secrets

synpse secret create azure-crt -f device1.crt
synpse secret create azure-key -f device1.key

Deploy the application. You will need to modify other values inside YAML file.

synpse deploy -f synpse-azure-example.yaml

where synpse-azure-example.yaml is

name: Azure-IoT-Hub
description: Azure IoT Hub Synpse example
scheduling:
  type: Conditional
  selectors:
    # device selector
    azure: iot
spec:
  containers:
    - name: nats
      image: nats
      restartPolicy: {}
    - name: metrics
      image: quay.io/synpse/metrics-nats-example-app
      restartPolicy: {}
    - name: azure-iot
      image: quay.io/synpse/azure-iot-hub-example
      command: /server/azure.py
      env:
        - name: NATS_HOSTNAME
          value: nats
        - name: HOSTNAME
          # IoT hub DNS name
          value: "mj-hub.azure-devices.net"
        - name: DEVICE_ID
          # device id used when creating a device
          value: "synpse"
      secrets:
      - name: azure-crt
        filepath: /server/device1.crt
      - name: azure-key
        filepath: /server/device1.key
      restartPolicy: {}

You should see messages coming into the Azure IoT Hub

Once running, you should see the application running and data coming into Azure storage account blob.

Things to look for

Certificate configuration path is painful. But this is the common pattern between cloud offering. Azure offers shared private key authentication. This makes multiple device provisioning easier. If you are not interested in cloud portability - this might be easier way to deal with complexities.

Events land into a storage account without visibility how and the default data structure cannot easily adjust. Sometimes it takes a few minutes, sometimes multiple minutes. We assume this is due to the ETL (Extract, Transform, Load) process, which is not visible to users. But on the other hand, on GCP it was very visible and VERY expensive. Not sure if both here are possible :)

Overall Azure experience was quite pleasant. API and CLI consistent, documentation is quite well written.

go run ./wrap_up.go

This is a simple way to use Azure IoT Hub with Synpse. When it comes to consuming and managing a lot of data, constructing complex applications and integrating seamlessly into your current technological infrastructure - nothing can beat the public cloud. But where the cloud is lacking is - IoT device and application management.

Public cloud providers are built on assumption that they will manage infrastructure for you. When it comes to devices themselves - they are yours and yours only. And this is where Public cloud providers lack influence and where Synpse comes into the picture.

If you have any questions or suggestions, feel free to start a new discussion in our forum or drop us a line on Discord

Originally published at https://synpse.net/blog

What is AWS IoT Core and how do I use it?

Mangirdas Judeikis — Tue, 02 Nov 2021 08:11:35 +0000

Synpse is an end-to-end platform to manage your device fleet that can grow to hundreds of thousands of devices, perform OTA software updates, collect metrics, logs, deploy your containerized applications and facilitate tunnel-based SSH access to any of your device. You can find a Quick Start here.

Intro into AWS IoT Core

We often get questions about how Synpse is compared or competes with the AWS IoT Core service. The short answer is that they operate in slightly different domains. AWS IoT Core focuses on application connectivity of the devices while Synpse targets deployment of the applications that may or may not be using services such as IoT Core.

Disclaimer: While the AWS landing page says "Easily and securely connect.." we found that it's not true. Setup is quite complicated so buckle up, backaroo!

Example application

In this tutorial, we will deploy a simple open-source application that collects metrics and send them to AWS IoT Core for further processing. All code for this blog post can be found at:

https://github.com/synpse-hq/metrics-nats-example-app - Sample metrics application
https://github.com/synpse-hq/aws-iot-core-example - AWS IoT Core example

Steps:

Set up AWS IoT Core
Configure rules to forward data into the S3 bucket
Create AWS device/Thing for Synpse
Deploy a demo Synpse application from 3 microservices - Metrics collector, NATS broker, example Python app that forwards data to AWS IoT Core

Technologies used

Synpse - manage devices and deploy applications to them
NATS - a lightweight message broker that can run on-prem
AWS IoT Core - message broker between all devices and AWS

AWS IoT Core

For this application to work we need to setup AWS IoT Core and its pre-requisites. AWS uses certificate authentication by default for all IoT devices.

AWS IoT Core page

Inside AWS IoT Core page navigate to Manage sub-page. Create a "Thing" with AWS generated certificates. Download certificates to your workstation. We will need them later.

AWS IoT S3 page

Inside the AWS IoT S3 page, create an S3 bucket for metrics to be stored. We gonna use it later

Create Act Rule

Go back on the IoT Core page navigate to the ACT subpage. We will create a new Rule for our metrics. Rule creation involved multiple steps, like creating the rule itself, granting access with policy and finalizing the setup.

Create policy and attach

We will need to create a policy to publish events, and attach it to the certificate we generated

The policy document is as below:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iot:Connect",
        "iot:Receive",
        "iot:Publish",
        "iot:Subscribe"
      ],
      "Resource": "*"
    }
  ]
}

Attach the policy to a certificate, used by "Thing"

Get endpoint

You will need an endpoint for your IoT Core. You can get it via CLI:

aws iot describe-endpoint

Or navigate via UI:

Deploy an application

We have downloaded certificates in the first step. Let's create Synpse secret with those certificates

synpse secret create aws-cert --file zzz-certificate.pem.crt
synpse secret create aws-key --file zzz-private.pem.key
synpse secret create aws-root-ca --file AmazonRootCA1.pem

Deploy Synpse application. Modify application yaml with your thing endpoint and messaging topic.

synpse deploy -f synpse-aws-example.yaml

where synpse-aws-example.yaml is:

name: AWS-IoT-Core
description: AWS IoT Core Synpse example
scheduling:
  type: Conditional
  selectors:
    aws: iot    
spec:
  containers:
  - name: nats
    image: nats
  - name: app
    image: quay.io/synpse/metrics-nats-example-app
  - name: aws-iot
    image: quay.io/synpse/aws-iot-core-example
    forcePull: true
    args:
      - --endpoint 
      - a243pu5i3wf6nw-ats.iot.us-east-1.amazonaws.com
      - --cert 
      - /server/gateway/certificate.pem  
      - --key 
      - /server/gateway/certificate.key
      - --root-ca 
      - /server/gateway/AmazonRootCA1.pem
      - --topic
      - test/topic
    command: /server/aws.py
    secrets:
      - name: aws-cert
        filepath: /server/gateway/certificate.pem
      - name: aws-key
        filepath: /server/gateway/certificate.key
      - name: aws-root-ca
        filepath: /server/gateway/AmazonRootCA1.pem

Once running, you should see the application running and data coming into the AWS S3 account

At this point, you might thing "This was not as hard as you told us". We did all the steps using AWS Console UI. Github repository contains more detail steps how to achieve same result via CLI. Good luck :)

Things to look for

AWS CLI was a disaster 5 years ago and it still is. Either API is consistent or CLI should be advanced enough to abstract inconsistencies. In AWS case API documentation is very poor, and if it is there, it just doesn't work. But if you are comfortable "clicking UI" - you are good. In my personal opinion
AWS still does a good job to keep their things proprietary.

Wrapping up

This is a simple way to use AWS IoT Core with Synpse. When it comes to consuming and managing a lot of data, constructing complex applications and integrating seamlessly into your current infrastructure - nothing can beat public cloud. But where the cloud is lacking - IoT device and application management.

Public cloud providers are built on assumption that they will manage infrastructure for you. When it comes to devices itself - hardware is owned by you so there's little incentive for them to manage fleets of machines that they can't be properly billing for.

If you have any questions or suggestions, feel free to start a new discussion in our forum or drop us a line on Discord

Originally published at https://synpse.net/blog

DIY downtime monitor with NodeRED

Mangirdas Judeikis — Tue, 26 Oct 2021 11:52:43 +0000

"Black box" monitoring is one of the key concepts, when monitoring any application. We will show how to deploy a free DIY downtime monitoring stack, using NodeRed and Synpse.

Similar pattern could be used to do almost any home automation. So go crazy! We use this pattern as its easy to modify, and not does much maintenance.

Synpse is an end-to-end platform to manage your device fleet that can grow to hundreds of thousands of devices, perform OTA software updates, collect metrics, logs, deploy your containerized applications and facilitate tunnel-based SSH access to any of your device. You can find a Quick Start here .

Technologies used

Synpse for hosting and running applications anywhere
NodeRED for instrumenting our monitoring

NodeRED

We will use NodeRED as instrumentation tool to “write code without writing any code” and monitor our web pages availability. If downtime is detected, we will notify administrators via slack or discord message.

First lets deploy NodeRED into Synpse enable IoT device we own. We use simple RPI4 under the desk at home:

name: NodeRed
description: Low-code programming for event-driven applications
type: container
scheduling:
  type: Selector
  # More on selectors https://docs.synpse.net/synpse-core/applications/scheduling
  selectors:
    nodered: master
spec:
  containers:
  - name: nodered
    image: nodered/node-red:latest-minimal
    user: root
    ports:
      - "1880:1880"
    volumes:
      - /data/nodered:/data
      - /etc/localtime:/etc/localtime
      - /root/.ssh:/root/.ssh

Access the remote device via Synpse proxy CLI command to configure it

synpse device proxy <device_name> --port 1880

Open the browser page http://localhost:1880. You should be able to see NodeRed running.

Downtime flow

We use Discord webhook integration to send a messages and alert our developers once page is down. Discord webhook documentation can be found here

Flow we created is very simple:

Every 5 seconds cloud.synpse.net health url
If response is not 200, go to next phase, else drop
Rate limit to send only 1 message every 5 min
Create a Webhook payload
Send POST request to webhook URL

Flow itself looks like this (we have left if 200 branch too, in case you want to re-use it for other purposes). Full json flow can be found bellow for you to reuse.

Press “deploy” and you are done :)

This is very simple way to monitor your applications or construct any other automation flows. In addition you can deploy same application to different agents across the globe, and you have free Region based downtime monitoring!

./wrap_up.sh

If you have any questions or suggestions, feel free to start a new discussion in our forum or drop us a line on Discord

Originally published at Synpse.net blog

Full flow

[
    {
        "id": "90902c5d92f08930",
        "type": "tab",
        "label": "NodeRed on Synpse",
        "disabled": false,
        "info": ""
    },
    {
        "id": "ef34b5db67c67385",
        "type": "http request",
        "z": "90902c5d92f08930",
        "name": "<PAGE_TO_CHECK_NAME>",
        "method": "GET",
        "ret": "txt",
        "paytoqs": "ignore",
        "url": "<PAGE_TO_CHECK_URL>",
        "tls": "",
        "persist": false,
        "proxy": "",
        "authType": "",
        "x": 370,
        "y": 260,
        "wires": [
            [
                "2109536ce55f85cf",
                "1aae7673af94fb6c"
            ]
        ]
    },
    {
        "id": "3c39826c5c2c418b",
        "type": "inject",
        "z": "90902c5d92f08930",
        "name": "ticker 5s",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "5",
        "crontab": "",
        "once": true,
        "onceDelay": "1",
        "topic": "",
        "payload": "{}",
        "payloadType": "json",
        "x": 150,
        "y": 260,
        "wires": [
            [
                "ef34b5db67c67385"
            ]
        ]
    },
    {
        "id": "2109536ce55f85cf",
        "type": "switch",
        "z": "90902c5d92f08930",
        "name": "if not 200",
        "property": "statusCode",
        "propertyType": "msg",
        "rules": [
            {
                "t": "neq",
                "v": "200",
                "vt": "str"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 1,
        "x": 580,
        "y": 200,
        "wires": [
            [
                "758156729b55665f"
            ]
        ]
    },
    {
        "id": "1aae7673af94fb6c",
        "type": "switch",
        "z": "90902c5d92f08930",
        "name": "if  200",
        "property": "statusCode",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "200",
                "vt": "str"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 1,
        "x": 570,
        "y": 300,
        "wires": [
            []
        ]
    },
    {
        "id": "758156729b55665f",
        "type": "delay",
        "z": "90902c5d92f08930",
        "name": "",
        "pauseType": "rate",
        "timeout": "5",
        "timeoutUnits": "seconds",
        "rate": "1",
        "nbRateUnits": "5",
        "rateUnits": "minute",
        "randomFirst": "1",
        "randomLast": "5",
        "randomUnits": "seconds",
        "drop": true,
        "allowrate": false,
        "x": 740,
        "y": 260,
        "wires": [
            [
                "a062d51eb49713f9"
            ]
        ]
    },
    {
        "id": "2452ffaa52e6333f",
        "type": "http request",
        "z": "90902c5d92f08930",
        "name": "discord webhook",
        "method": "POST",
        "ret": "txt",
        "paytoqs": "ignore",
        "url": "<DISCORD_WEBHOOK_URL>",
        "tls": "",
        "persist": false,
        "proxy": "",
        "authType": "",
        "x": 1110,
        "y": 260,
        "wires": [
            []
        ]
    },
    {
        "id": "a062d51eb49713f9",
        "type": "function",
        "z": "90902c5d92f08930",
        "name": "payload",
        "func": "var payload = {\n  \"username\": \"Downtime BOT\",\n  \"avatar_url\": \"https://i.imgur.com/zEGYUVs.jpeg\",\n  \"content\": \"cloud.synpse.net Downtime detected. Next check in 5 min\",\n}\n\nmsg.payload = payload\nreturn msg",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 920,
        "y": 260,
        "wires": [
            [
                "2452ffaa52e6333f"
            ]
        ]
    }
]

Access remote networks without a VPN!

Mangirdas Judeikis — Mon, 20 Sep 2021 07:50:23 +0000

Synpse is an end-to-end platform to manage your device fleet that can grow to hundreds of thousands of devices, perform OTA software updates, collect metrics, logs, deploy your containerized applications and facilitate tunnel-based SSH access to any of your device. You can find a Quick Start here.

Sometimes you get into situations, where you want to configure your home router to expose new ports, or help your close ones to configure some network device in their home network (IP camera, router, smart home system). How many times you wanted to have SSH access into your mini server at home while on holidays?

I had a similar case. I was traveling, and at some point I needed to reconfigure my home wifi router because ISP updated the firmware, and external IP address changed and I lost all port forwarding rules to my "home hosted devices". So no more free home hosted VPN (I will have separate post about this)

Luckily, I had synpse device running in my home! I will show a simple and easy way to access your remote networks in cases like this.

Proxy feature

Synpse allows you to proxy from your laptop to remote device for the local development:

synpse device proxy <device-name> <local-port>:<remote-port>

This is very helpful, when you are doing remote development and need to debug remote devices and application. But there is more to this. Synpse can act as forward proxy to other devices on the remote network (nothing what normal SSH can't do too but just makes it easier). Diagram bellow show how this could be used at the high level:

To achieve this, just change a command a bit:

synpse device proxy <device-name> <local-port>:<hostname/ip>:<remote-port>

How does this looks like?

In my case, we needed to access WIFI router and reconfigure it.

SSH into remote device to check network address space it is running in.

synpse device ssh rpi3
root@synpse:/# ip a
...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether b8:27:eb:15:c6:af brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.178/24 brd 192.168.0.255 scope global dynamic eth0
       valid_lft 85386sec preferred_lft 85386sec
    inet6 fd01::ba27:ebff:fe15:c6af/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 286sec preferred_lft 286sec
    inet6 fe80::ba27:ebff:fe15:c6af/64 scope link 
       valid_lft forever preferred_lft forever
...

We see that my device is in 192.168.0.1/24 network. We know our router is the first IP address in the network. Lets get to it:

synpse device proxy rpi3 8443:192.168.0.1:443
forwarding port (local->remote) 8443 -> 192.168.0.1:443

And this is it. Open https://localhost:8443 and you should see the router page:

./wrap_up

In short, Synpse is not only tool to manage deployments for your IoT fleet, but it is a development and debugging tool too!

If you have any questions or suggestions, feel free to start a new discussion in our forum or drop us a line on Discord

Originally published at: https://synpse.net

Welcome to Synpse — an IoT platform for everybody

Mangirdas Judeikis — Mon, 13 Sep 2021 15:46:22 +0000

Why IoT?

IoT should not be hard. Sadly we found it to be not the case. As a DYI hackers and hobbyists we are running a lot of random projects on the side. From home automation, 3D printers, heating, temperature sensors, car tracking systems and more.

Then, once upon a time, we received a request to deploy software to a place where we didn't even imagine you can place a computer with a working Linux OS 🤯.

That led us to a journey where we tried and tested a bunch of different tools for IoT fleet management. In the end we reached a conclusion - running software on the edge is not that easy with the current available technologies. The ones capable of doing so are either too expensive to run, do not scale or require custom OS'es to be installed or all those things. What is more, if you do it yourself it is an incredibly tedious and error prone process. Ansible, Chef, Puppet, Terraform and all the other tools are great, but they are not flexible enough to manage thousands of devices and especially when all of those little computers are just out of reach.

So, as true Engineers - we created a new tool for the job 😀

Why Synpse?

If you are coming from Kubernetes world, you might be thinking - "I know how to do it - let's use Kubernetes". Not so fast, cowboy 🤠. True edge devices have a bit different requirements. They need to be able to go "offline" and "online" without any need to reach the control plane, "graceful shutdown" almost never exist, your devices just lose electricity. They need to be independent from the network and they need to be able to be deployed in different environments.

So, the main reasons why we didn't choose Kubernetes:

We didn't want to run Kubernetes ourselves as we don't need most of the features it offers.
Agents should be lightweight and are not allowed to consume most of the resources (kubelets can be resource hungry).
Deployments in IoT fleets usually work more as a daemonsets where it's important to run apps on specific devices and not an X number of replicas on any of the devices (can be achieved on k8s with daemonsets and taints but it's just not the right solution).
Scale and controlplane resource consumption. With Kubernetes, when you reach >1K, controller nodes will be consuming massive amounts of CPU, RAM and Etcd will be slowing down. We wanted a system that can easily scale to thousands of nodes, running on a single core machine with several gigs of RAM.

Why not Docker Compose?

Well, Docker Compose is great for local development or deploying some simple software to a remote server. However, moving docker-compose.yaml files around can become complicated. Also, it wouldn't provide any cluster management functionality.

We didn't create the tool for sake of building something. We encountered a problem and decided to solve it.

Biggest challenges that we had to overcome

How to quickly bootstrap thousands of devices.
How to ensure that the bootstrap process is secure, reproduceable and allows the organization to group devices based on their needs.
How to access devices when they are located in the unknown network you don't control, even behind firewalls where external IPs are not available or when it's behind CGNAT.
How to deploy software to devices and update applications easily (updates over the air/OTA)
Use concepts and tools that most modern software developers are already familiar with and love - SSH, Containers, Docker, TCP tunnels.
We wanted to use any OS we choose to. Some tools are better on Ubuntu, some on Fedora or Raspberry PI OS. When you deal with embedded devices you need more flexibility as manufacturers are often not very accomodating.
It should be simple and easy to use (batteries included :)).

None of these are new problems. We just had to solve them in a different way. So we did.

This is how Synpse was born

Bespoke solution for edge device management. Easy to use without history of complexity behind it.

Whats next?

We prepared set of blog posts for you to get started:

Originally posted at synpse.net