Forem: Karolis

How to setup on-prem Jenkins with Bitbucket

Karolis — Wed, 27 Sep 2023 22:37:27 +0000

In this tutorial, we will show a Jenkins Bitbucket integration using webhooks. It will work behind a firewall, inside a private network. You can use this setup for other services too - such as GitHub, GitLab or anything else that emits webhooks.

Main advantages of Webhook Relay here are:

No delay between polling requests
Additional security layer as Jenkins is not exposed to the internet as webhooks by default are uni-directional, responses are not returned to the caller.

Prerequisites

Webhook Relay account, create one here.
Jenkins instance. We will not go into installing Jenkins itself as there are quite a few options and many articles on that. See Jenkins official docs for up-to-date instructions.
Bitbucket account and a repository that you will want to use.

Configure webhook forwarding

We will be using the Jenkins Bitbucket plugin . This plugin exposes a single endpoint to which we can send webhooks from multiple Bitbucket repositories.

Go to the internal URL forwarding setup page https://my.webhookrelay.com/new-internal-destination and enter your Jenkins address. In my case I will be running the agent on the same machine as Jenkins so the address for me is http://localhost:8080/bitbucket-hook/:

Follow the instructions to setup the agent and being forwarding webhooks. You will get your public URL that you can use in Bitbucket webhook configuration.

As you can see in the screenshot above, take the "Listening on" address.

Configure Bitbucket

For Bitbucket webhook configuration you can follow the plugin guide here: https://plugins.jenkins.io/bitbucket/#plugin-content-bitbucket-cloud-usage. You need to go to the repository settings and then to the webhooks section add "Add webhook" with the public URL that you have gotten from the previous step:

Bitbucket will be sending webhooks to Webhook Relay and our service will forwarding them to your internal Jenkins instance.

Configure Jenkins

Ensure you have the Bitbucket Jenkins plugin. Plugin instructions can be found here: https://plugins.jenkins.io/bitbucket/#plugin-content-bitbucket-cloud-usage.

In your repository configure the build trigger:

Testing

Once the agent is running, you can test by pushing a commit to your repository. You should see a build being triggered in Jenkins:

You should also see it in the terminal where you started Webhook Relay agent:



relay forward -b localhost-9Jk06s
Filtering on bucket: localhost-9Jk06s
Starting webhook relay agent...
2023-09-24 23:03:10.884 INFO    using standard transport...
2023-09-24 23:03:10.951 INFO    webhook relay ready...  {"host": "my.webhookrelay.com:8080", "buckets": ["localhost-9Jk06s"]}

Troubleshooting

There are several places to look for logs:

Webhook Relay bucket details. It will show all your webhooks and their requests, responses.
Jenkins system logs. You can find them in the Jenkins UI under "Manage Jenkins" -> "System Log".

Response "200" in Webhook Relay logs but no build

It's possible that you don't have the SCM configuration matching your Bitbucket repository. Check the system logs in Jenkins for errors such as:



PM WARNING com.cloudbees.jenkins.plugins.BitbucketJobProbe triggerMatchingJobs
No SCM configuration was found!

If you find them, add your repository to the SCM configuration in the Jenkins job.

On Webhook Relay all logs appear as "received"

You need to start the agent, follow the instructions here https://webhookrelay.com/v1/installation/. Agent is required to run in order to receive and forward webhooks.

Bitbucket server

Bitbucket server webhook to jenkins example can be found here https://plugins.jenkins.io/bitbucket/#plugin-content-bitbucket-server-usage. You mostly need to install a webhook plugin https://marketplace.atlassian.com/apps/1215474/post-webhooks-for-bitbucket?hosting=server&tab=overview and then create a Post-WebHook, which is different from WebHook and enable on push.

Originally published on https://webhookrelay.com/v1/tutorials/bitbucket/

How good are Docker containers in the IoT devices?

Karolis — Tue, 01 Feb 2022 10:50:44 +0000

Docker containers (also known as OCI containers) are now the most popular way to build and ship applications, whether it's through plain Docker, Docker-Compose, Kubernetes, or hosted services like Heroku, GCP CloudRun. The ease of building and deploying containers made it a no-brainer for developers and ops teams to adopt the new technology.

But what about IoT devices? In this post, we will look into what's good about Docker containers on the edge and things to watch out for.

Are Docker containers ready for the IoT devices?

Yes. The devices themselves are now plenty powerful to run a regular Linux distro, have a Docker engine installed, and run your containers without much overhead. Most of the devices out there, even with dated hardware are often more powerful than the small VMs provided by major cloud services.

Synpse has been deployed on large fleets of devices that were comparable to RaspberryPi 2 (much weaker than the current generation) and we haven't observed any performance-related issues so far.

For example, looking at Pi Zero 2 which is priced as low as $15 can offer you a quad-core 64-bit ARM Cortex-A53 processor clocked at 1GHz and 512MB of SDRAM. Or we can look at the regular Raspberry Pi 4 or Jetson Xavier which can provide a much higher amount of computing power.

Software deployment and OTA updates

The biggest advantage of Docker is the ability to package the application the way you want and not worry about the host machine too much (you still need to think about the OS but a lot less). Before writing software in Go I used to write backend applications in Python. Application deployment used to be complicated, updates brittle and once in a while we would get some dependency clash. Docker solved this by packaging the whole filesystem with installed dependencies. You have your packages, your static files placed in the container.

This way you can build the container, run automated tests against it and then deploy it to the servers. It also paves way for a simple rollback mechanism, where if things go wrong, you can just start the older version of the Docker container.

For IoT products with tiny applications and fewer packages, the OTA update process is usually lightweight and simple: a new image is downloaded, the running container is stopped, a new container from the new Docker image is started.

Preparing multi-arch (or at least arm) images

The main difference between x86/amd64 Docker images and the ones that you will probably need to deploy to the IoT devices is their architecture. This can sometimes be a gotcha when trying to run your containers on a RaspberryPi or some other device as the errors are often cryptic.

Thankfully, DockerHub and other registries support multi-arch images where you can push both x86, arm 32-bit, and arm 64-bit images. This greatly simplifies deployment as the Docker daemon will figure out on its own, which image to download. We have written an article on building multi-arch images.

Persisting data

An application that performs AI tasks such as image processing or has local databases need to persist data. When people just start using Docker containers, storage can be confusing as anything saved inside the container is not persisted when a container is recreated during the update. The way to persist data is to mount a volume from the device's filesystem into the container. You can read more about persistence in Docker here.

Write/read-heavy directories and extending device's filesystem lifetime

When preparing a device for an extended time in service, you need to think not only about the updates but also about the longevity of its components. SD cards or inbuilt storage can last for a long time but there are ways to reduce the load on them.

If your application needs to write and read large amounts of data, it's often good to also mount /dev/shm. You can read more about shared memory here.

Exposing hardware devices into the containers

Quite often you will need to use serial devices. Containerized applications can access them without any problems but you need to let Docker know that you want to expose these devices to your containers. On the Docker CLI it looks like this:

docker run -t -i --device=/dev/ttyACM0 ubuntu bash

Alternatively, in Synpse you can add the following into your application spec:

    devices:
      - hostPath: /dev/ttyACM0
        containerPath: /dev/ttyACM0

Security

When deploying software to thousands of devices you need to think about "day two" operations. Even before releasing your application, it's crucial to plan and test both OS and application updates. How will you update the application? If the device is connected to the internet, what are the best methods to keep it secure even years later?

By reducing the number of packages needed to be installed on the device, you are also drastically reducing the attack surface. For example, if you are deploying NodeJS, Python, Ruby based applications, your device OS doesn't need to have the runtime and can be as bare-bones as possible. This helps a lot, whatever is not installed, doesn't need to be updated.

You can also use tools to automatically scan your images for vulnerabilities:

Developer productivity

Another important aspect when deploying containerized applications to the edge is reusing the tools that are already well known to your developers and ops teams. By choosing Docker for the edge, you aren't introducing a new packaging system. You can also reuse your CI pipelines.

Summing up

If your hardware and OS allow, it's almost always better to choose Docker containers for deploying your applications. It will increase security, simplify the deployment process and reduce the overall amount of issues over a long time of operation. The important part becomes a solid strategy to update your containers.

OTA updates with Synpse

When it comes to managing large fleets of containers - Synpse was built for this. It provides everything you might need to successfully deploy and run a large scale operation:

Declarative application deployment, store configuration in GitHub or any other source control management system
Simple rollbacks
GPU support
Volume support
SSH support with Ansible integration for OS updates
Application logs for each device
Metrics collection

You can try out Synpse for free (up to 5 devices). Check out our quick start.

Putting your RaspberryPi to work with a sleek torrent client & Plex TV setup

Karolis — Wed, 29 Dec 2021 13:18:03 +0000

Intro

I guess we all know that torrents present a fantastic opportunity (amongst other things) for families to digitize their old VHS tapes that contain embarrassing videos of your childhood and then share these videos with others using an efficient BitTorrent protocol.

Then, once everyone has those videos that were supposed to die together with the old technology we also need to watch them, this is where the good guy Plex comes in.

In this short tutorial we will setup a Vuetorrent and Plex combo on a RaspberryPi. Using this installation method you will get few great things out of the box:

Drag & drop torrent files to download them. You can select individual files from a larger torrent like in any other client.
Shared directory between torrent client container and Plex so you can view them
Optional public HTTPS URL to connect to Vuetorrent from anywhere in a form https://dev-{YOUR ID}.synpse.cloud

Hardware

Software

Synpse - provides app deployment, SSH and HTTPS access.
Vuetorrent - Vuetorrent is a nice looking skin for otherwise a bit dated qbittorrent
Plex - probably the best option to stream things locally to your smart TV.

Step 1: Install Docker

Depending on your RaspberryPi OS Docker installation might vary but in most cases this script from https://docs.docker.com/engine/install/ubuntu should do the job:



curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

Step 2: Install Synpse agent on your device

Log into your Synpse account, navigate to the "Devices" section and then click on "Provision". Use the command in your RaspberryPi terminal. This will download, install and initialize the synpse agent.

Once the device has appeared in your Synpse dashboard:

Once device is visible, click on the "edit labels" from the menu. Add a label 'type: rpi':

In Synpse, applications are installed into the devices based on device labels so it's a good practice to label your devices accordingly.

Step 3: Create Vuetorrent and Plex deployment

Applications in Synpse are defined in yaml format. If you have used Docker Compose or Kubernetes before, it will feel familiar. You can read more about them here.

To create the application:

Select the namespace (should be 'default' on the left side menu)
Click on "New Application" button in the top
Copy paste the yaml from below



name: torrent
scheduling:
  type: Conditional
  selectors:
    type: rpi
spec:
  containers:
    - name: qbittorrent
      image: cr.hotio.dev/hotio/qbittorrent # Using an image that already contains vuetorrent
      ports:
        - 80:8080
      volumes:
        - /data/qbittorrent/config:/config
        - /media/downloads:/downloads
        - ./vuetorrent:/vuetorrent
      env:
        - name: PUID
          value: "1000"
        - name: PGID
          value: "1000"
        - name: TZ
          value: Europe/London
        - name: UMASK_SET
          value: "022"
        - name: WEBUI_PORT
          value: "8080"
    - name: plex
      image: lscr.io/linuxserver/plex
      networkMode: host # Using host network as it will be exposing quite a few ports for various protocols
      volumes:
        - /data/plex:/config
        - /media/downloads:/media/downloads
      env:
        - name: PUID
          value: "1000"
        - name: PGID
          value: "1000"
        - name: VERSION
          value: docker
        - name: PLEX_CLAIM
          value: "" # Optional, you can get your claim here https://www.plex.tv/claim/

Logging into the Vuetorrent:

Once deployed, it will take a few seconds or minutes (depending on your internet speed) to download and start the docker containers. Once started, you can access it on http://{YOUR DEVICE IP}:80 address. Initial username is admin and password is adminadmin.

Once logged in, go into settings and configure the UI theme:

Then, change your password:

Logging into Plex:

You can login into it using port http://{YOUR DEVICE IP}:32400

Step 4: Configuring Plex libraries

To see whatever you have downloaded via Vuetorrent to your local machine in Plex. For both Films and TV Shows the media path should be /media/downloads:

Step 5 (optional): Expose it to the internet so you can connect from anywhere

At this step we will enable remote access via HTTPS to our Vuetorrent installation on the RaspberryPi. If you have skipped password setup, please do it, otherwise malicious users or automated bots could hijack your device.

Go to your device details
In the top right corner set the port to your Vuetorrent port which should be 80
Turn on the switch

Next Steps

Connect your smart TV

Most TVs now have inbuilt Plex TV apps. You can use it to pair it with your Raspberry Pi server. As long as you are using same account, it should just work.

Download content

Add a torrent to the Vuetorrent dashboard. Once downloaded, it should appear in your Plex library.

References

https://www.qbittorrent.org/ - torrent client
https://hotio.dev/containers/qbittorrent/ - Docker image with embedded Vuetorrent
https://hub.docker.com/r/linuxserver/plex/tags - LinuxServer.io provided Plex Docker image
https://www.plex.tv/en-gb/ - streaming app

Originally published at https://synpse.net/blog/vuetorrent-plex/

Self-hosted business intelligence with Metabase

Karolis — Thu, 11 Nov 2021 19:38:24 +0000

It is always useful to know how your business or projects are doing and for that, there are a bunch of tools available such as Excel spreadsheets, Google DataStudio, Apache Superset, etc. I personally am a fan of Metabase as it is the easiest to deploy and use. When paired with the right technologies, this setup becomes trivial to any small or large organizations.

Main benefits of self-hosting Metabase (other tools as well):

Performance - compared to Cloud VMs your own hardware will not be throttled even when it runs heavy queries.
Costs - hardware like Intel NUC (or new ones from AMD) pay for themselves not in years but in months. I have mine for more than 2 years and it's been fantastic little helper.
Privacy - since you need to supply connection strings to your detabase, it's much better to do it on your own hardware where you control the risks.
Stability - managed instances can be updated at any time and can break your setup. This happened to us :)

In this article, we will use a setup that works exactly the same way on both an Intel NUC (for some of my projects) and on a large VM that is managed by a VMware.

Prerequisites

Webhook Relay account - will be used to expose the Metabase to the internet, so we can access it.
Synpse account - a lightweight and fantastic platform to manage and run software on your own hardware. Offers management of up to 5 devices free of charge.

1. Install Synpse into your server/machine

Once you log into Synpse Cloud, select your project and then head to the "Devices" page. From there you will be able to find the auto-generated command that you need to run on the device to add it to your project.

There are multiple ways to do it however initially you can just SSH into the machine via local network. Once you run the command, after a few seconds (depends on your internet speed) you should see the magic happen and device appear in your "Devices" page in Synpse :)

2. Create a Webhook Relay tunnel for your Metabase app

Our Metabase deployment will need to be reachable from outside so we can actually view reports. For this, we are creating a Webhook Relay tunnel that will be established by a webhookrelayd container.

Go to your https://my.webhookrelay.com/tunnels page and create a new tunnel with these details:

name: 'whr-metabase' (webhookrelayd agent will need to know which tunnel to use)
destination: 'http://metabase:3000' (metabase is reachable using container's name)

3. Get your access token key and secret

Head to the tokens page here https://my.webhookrelay.com/tokens and create a new pair. Save the key and secret before closing the window.

Go to the secrets page in Synpse and create both relayKey and relaySecret secrets:

4. Deploy Metabase via Synpse

Last step is to create a Synpse application.

name: metabase
description: metabase + WHR
scheduling:
  type: Conditional
  selectors:
    type: controller
spec:
  containers:
    - name: metabase
      image: metabase/metabase:latest
      volumes:
        - /data/metabase:/metabase-data
      env:
        - name: MB_DB_FILE
          value: /metabase-data/metabase.db
        - name: MB_REDIRECT_ALL_REQUESTS_TO_HTTPS
          value: "false"
    - name: relayd
      image: webhookrelay/webhookrelayd:1
      args:
        - --mode
        - tunnel
        - -t
        - whr-metabase # <--- if you have chosen a different name for the tunnel, change it here too
      env:
        - name: RELAY_KEY
          fromSecret: relayKey
        - name: RELAY_SECRET
          fromSecret: relaySecret

Once deployed, use your tunnel URL to access it. You can also configure Google OAuth to make the login easier, however that's out of scope in this article.

Enjoy!

Originally published on https://webhookrelay.com/blog/2021/11/11/setting-up-selfhosted-metabase/

Manage your meetings like a boss with self-hosted calender

Karolis — Sat, 02 Oct 2021 21:12:12 +0000

Calendly and other scheduling tools are awesome. It made our lives massively easier. We're using it for business meetings, side projects and anything else where we need some flexible coordination. However, most tools are very limited in terms of control, customization, data protection, and most important - they are not free! That's where Calendso & Synpse come in. Self-hosted, all your data sitting on your own machines. API-driven and ready to be deployed on your own domain. Full control of your events and data.

In this tutorial we will be deploying a self-hosted Calendso instance that lives under your own DNS domain. You will be able to share meeting links such as https://calendar.synpse.net/karolis/quick-chat with your colleagues, clients and friends:

You will also be able to:

Integrate it with your Google calendar
Create multiple users for your organization
Create teams

The setup

Deployment will be handled in a declarative way through Synpse. Incoming traffic will be handled by Caddy server which will terminate HTTPS (it will also automatically retrieve TLS certificates for you) and route traffic to Calendso.

You will also be able to access Prisma Studio locally to create your and your team's users.

Prerequisites

Synpse account - Deployment
Cloudflare account - DNS (or any other DNS provider)

If you don't have your own DNS domain, just jump in our Discord channel and I will help you out with some alternatives (you can either use Webhook Relay's subdomains or use services like DuckDNS).

Configure DNS

If you are using Cloudflare, creating an A record looks like this:

You can consult your DNS provider's documentation on managing A records.

Configure port forwarding on your router (optional)

If you server where you are setting this up is within your local network, you will need to expose it to the internet. The easy way to do it is just using your own router to forward the ports:

If you don't have a static IP or your network is behind a CGNAT, you can use https://webhookrelay.com instead. We will be providing a tutorial on that setup as well.

Integrating with Google calendar

Bookings are great but it would be tedious to check your Calendso instance every day. To allow Calendso to read from and write to our Google calendar, we need to create API credentials and provide them to Calendso.

Enable the API

To do that, head to the Google API console and create a new project. It might take a little bit for Google to set up your project but once that's done, head over to the project and click the Enable APIs and Services button.

Once you click the button, you'll be taken to the API library. In the search box, look for calendar and select the Google Calendar API result and then enable it.

Retrieve credentials

Next, let's head to our credentials page in the Google API console so we can create the Calendso OAuth ID and add our production origin and redirect URI.

Your key will be created and you'll be redirected back to the Credentials page where you will see your newly generated OAuth 2.0 Client ID. Select the download option and save this file to your computer. Copy all the contents of the file and add that as the value for the GOOGLE_API_CREDENTIALS environment variable:

Deploy through Synpse

With Synpse we will be deploying multiple containers to provide us with admin capabilities, database, tunneling (to expose it to the internet) and the Calendso itself.

Preparing the secrets

First, go to your secrets page and create:

calendsoPostgres with your database password
calendsoPostgresConnString with postgresql://calendso:PASSWORD@postgres:5432/calendso (just replace PASSWORD with your database password)
calendsoEncKey with a random key that you generate yourself. You can generate one in your terminal with command openssl rand -base64 32

Creating the application

The full spec looks like this:

# Name of your app
name: synpse-calendso
scheduling:
  type: Conditional
  # Need to match your labels
  # Ref: https://docs.synpse.net/synpse-core/applications/scheduling
  selectors:
    type: controller
spec:
  containers:
    - name: calendso
      image: ctadeu/calendso:0.0.17-1
      env:
        - name: BASE_URL
          value: https://calendar.synpse.net
        - name: NEXTAUTH_URL
          value: https://calendar.synpse.net
        - name: DATABASE_URL
          fromSecret: calendsoPostgresConnString
        - name: CALENDSO_ENCRYPTION_KEY
          fromSecret: calendsoEncKey
        - name: GOOGLE_API_CREDENTIALS
          fromSecret: calendsoGoogleApiCredentials
      restartPolicy: {}
    - name: prisma
      image: codejamninja/prisma-studio:latest
      ports:
        - 5555:5555
      env:
        - name: POSTGRES_URL
          fromSecret: calendsoPostgresConnString
      restartPolicy: {}
    - name: postgres
      image: postgres:latest
      volumes:
        - /data/calendso-postgres:/var/lib/postgresql/data
      env:
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        - name: POSTGRES_USER
          value: calendso
        - name: POSTGRES_DB
          value: calendso
        - name: POSTGRES_PASSWORD
          fromSecret: calendsoPostgres
      restartPolicy: {}
    # Caddy provides HTTPS
    - name: caddy
      image: caddy:latest
      args:
        - caddy
        - reverse-proxy
        - --from
        - calendar.synpse.net:7300 # This should be your domain and port, same as in your router
        - --to
        - calendso:3000
      ports:
        - 7300:7300
      volumes:
        - /data/calendso-caddy:/data
        - /data/calendso-caddy-cfg:/config
      restartPolicy: {}

Now, if you click on Allocations, you should see all containers running and in device details you can see the ports:

If you click on Prisma and if your device is locally reachable, you will be able to access the admin dashboard.

Please note, that if your device is exposed to the internet, you should not leave the Prisma container with the port exposed.

Once in Prisma, follow the official guide from self-hosting section and:

Click on the User model to add a new user record.
Fill out the fields (remembering to encrypt your password with BCrypt) and click Save 1 Record to create your first user.

Once you have logged in, go to the integrations section and connect it with the Google's calendar.

References

Calendso docs: https://docs.cal.com/
Synpse docs: https://docs.synpse.net
Webhook Relay docs: https://docs.webhookrelay.com
Cloudflare: https://cloudflare.com

Originally published on https://synpse.net/blog/self-hosting-calendso-caddy/

Showdev: deploy apps to edge devices and servers with Synpse

Karolis — Thu, 02 Sep 2021 14:07:39 +0000

Hey dev.to! I wanted to share our project Synpse that my friend and I have been working on for quite some time.

TL;DR;

Registration is now open at https://cloud.synpse.net. Free for up to 5 devices forever. We can increase your quotas if requested. Self-hosted version available as well.

Synpse provides tooling and services to:

Prepare machine images that you can burn into SD cards to quickly bootstrap hundreds or thousands of devices.
Generated simple installation script for one-off device/server bootstrap.
Group devices based on labels (used for both filtering and scheduling applications).
Automated device naming (based on hostname, random names, etc.).
Deploy containerized applications (Docker Compose/K8s Pod style).
Automated device software updates.
Easily scale to tens or hundreds of thousands of devices.
SSH into any of your devices through synpse CLI or web UI.
Kubectl style port-forward command to open up TCP tunnels.
CPU/RAM utilization metrics from device.

Problem

While it's easy to configure a systemd or some other init service in an OS image to launch your application on device boot - it won't help much for fast updates or gradual rollout. It will also not help much if during the rollout you need to collect some data from applications (logs, SSH access to check some files).

We had a need to deploy quite complex piece of software to ~1200 devices. After looking into a bunch of alternatives like Mender, Balena, Deviceplane we came to a conclusion that it's either:

Can't use my own image (we had a manufacturer that could only use their own Linux distro, new custom OS was not an option).
Self-hosting is too complicated.
Not scaleable (started considerably slowing down with very few devices).

Solution

What our testing gave us was the understanding that the ideal solution would be:

Simple self-hosting, a horizontally scaleable controller and a managed database (we chose Postgres) without disrupting your bank balance.
Must support any Linux OS.
Don't try to be compatible with neither Kubernetes or Docker Compose in the API/config manifests. These systems require different semantics so it's fine to be similar but 1:1 mapping will just damage the UX.
SSH and application logs are a must if you have an outage and you need a fast way to debug.
Everything belongs to a project. Project can have multiple users but the projects should be able to survive multiple generations of users as these systems will last longer than many individual employees.

The feel

So, one-off device provisioning script + execution parameters can be found in either "devices" or "provisioning" pages:

Once devices are added, they appear in your dashboard for SSH access, monitoring:

You can read more about device management here: https://docs.synpse.net/synpse-core/devices.

You can add various labels and then use them in the application deployment:

View application docs here: https://docs.synpse.net/synpse-core/applications.

Most applications have some shared or private configuration, luckily Synpse provides secrets (similar to Kubernetes ones) where you can reference to them from your application spec:

And when things go south, view your application logs on any of the devices:

Use cases

As mentioned in the first paragraph, we built Synpse for a medium size fleet of ~1200 IoT devices and a bunch of large servers as it appears that the UX can be much nicer than Docker Compose or Kubernetes. All of those devices are actually card readers in a large bus/trolleybus fleet in Lithuania :)

We would recommend using Synpse for:

Home lab deployments (up to 5 devices free forever).
PoS deployments.
Public transport fleets.
ML on edge (packaging and running ML inference through Docker containers).
Any kind of industrial applications where devices can be offline for a prolonged time. Synpse will work well without internet access.
Normal VMs on cloud providers that don't provide good application deployment UX (most smaller ones like Vultr, DO, Packer, etc.)

Next steps

If you run software yourself or in a company, you should definitely check it out.

Useful links:

Docs: https://docs.synpse.net/
Website: https://synpse.net
Our Discord channel: https://discord.gg/dkgN4vVNdm

CDN types and how to set one up (Vue, React)

Karolis — Mon, 27 Jul 2020 16:09:41 +0000

What is CDN? Cloudflare has a nice explanation here: https://www.cloudflare.com/learning/cdn/what-is-a-cdn/

In short:

A content delivery network (CDN) refers to a geographically distributed group of servers that work together to provide fast delivery of Internet content.

Many of you know CDNs from using various 3rd party resources such as fonts, css, certain javascript libraries (for example Stripe library for secure payments).

In this article we will have a quick look into several CDN types and potentials pros/cons that you might encounter when setting one up yourself.

Cloudflare is one of the best CDNs out there and I am using it for various landing pages. It's a great DNS configuration service too that provides rich APIs. However, it's good to understand what other types of CDNs are out there and which suits you best.

CDN types

All CDNs have different pros and cons and all solutions are trying to achieve the same thing: load content faster.

Reverse proxies with caching

Some of the CDN types you will encounter in the wild:

Cloudflare type proxies that forward all incoming traffic to your origin servers and cache as much as possible. Pros:
- Ease of use. Your application doesn't have to be aware of the CDN itself. If you are using Cloudflare as a DNS provider you just click on the button and their servers start intercepting all traffic and caching it. On top of that, they offer a bunch of other useful services like firewalls, "page rules" that can redirect Cons:
- Can be caching too much (you don't see updates once you push because index.html is also cached).
- Since they are terminating connections, if they go down together with your DNS control, it becomes harder to recover.
- Lack of control from your side and potential security implication of allowing 3rd party to terminate TLS for you.

Push CDN

Push CDN is a setup where you upload your assets to a server (or a group of servers). An example of such CDN is Google Cloud CDN. In this setup, you will have to create a load balancer and a storage bucket and upload your content assets as part of the CI/CD pipeline where you build your frontend app. In this setup, you will need to create a new domain such as cdn.example.com that points to your CDN storage location.

Pros:

You remain in control of TLS termination and have a better understanding of what content is presented when. If your frontend app uses unique IDs for the static assets, for example /js/chunk-2d22502a.0844b32d.js.
Main file index.html is served by your server so it can always point to the most up-to-date js/css files.
You can know exactly what's pushed to the CDN.

Cons:

You get a new step in your CI/CD pipeline that can fail. If your frontend is deployed but assets failed to sync, your users might get a lot of errors. You also need to ensure that the CDN static files are not simply overwritten (as you might overwrite them while the old frontend app is still using previous files).

No CDN

No CDN, just cache control headers on your web server. This option might work for many cases, however, the first load can be painful if the user is far from your server location and you have a lot of static assets.

This is still a valid option when combined with optimized asset size, PWA worker and correct caching headers.

Pull CDN

CDNs like BunnyCDN (affiliate link, great service) pull from your origin server but don't try to proxy all your traffic. In this scenario, you serve your index.html that then loads assets through the CDN domain instead of your own. Similarly, as with the "Push CDN" type, you will have to either serve assets from cdn.example.com, or if you have a fancy global load balancer, you can configure that certain paths load files directly from CDN servers.

Pros:

Ease of use. It feels like Cloudflare from the "setup" perspective. You only need to provide it with the address of your web server and then optionally configure your domain. It will pull assets and show nice stats.
Pricing. It seems like it's a lot cheaper than other CDNs while providing an excellent service. They have some comparison info on their pricing page: https://bunnycdn.com/pricing, however you would need to test it for yourself as it may well depend on your content.

Cons:

Need to ensure that your assets have unique build IDs baked into the filenames so you don't serve stale content. Fortunately, most modern javascript transpilers do this by default so in my case with Vue.js I didn't have to do anything on this front.
If CDN would go down, even though your index.html loads, your assets would fail anyway. However, in this case, you would still be able to quickly change the assets domain to your main web server.

Setting up BunnyCDN (Pull CDN) in a SPA

I couldn't immediately spot the docs but if you are doing this not for the first time, it's quite straightforward:

Create a "pull zone". You will get your pull zone domain which is a reverse proxy to your origin web server:

(Optional but recommended) Create a CNAME from your domain to the allocated pull zone domain (in our case it's cdn.webhookrelay.com -> webhookrelay.b-cdn.net). This enables you to load assets from your domain name.
Update your webpack config to add asset file prefix. Example for vue.config.js would be:

module.exports = {
  publicPath: process.env.NODE_ENV === 'production' ? 'https://cdn.your-domain-here.com/' : '/',
}

If you are using React, check out "Public Folder" docs here: https://create-react-app.dev/docs/using-the-public-folder/.

That's it, generated assets will all have the prefix to load through the CDN. If you are using Nginx to serve your app, ensure that you are providing correct headers for js and css files. For example:

location ~* \.(?:css|js)$ {
          expires 1y;
          add_header Cache-Control "public";
          access_log off;
        }

I hope you will find this useful whenever you decide to add CDN for your website!

Originally posted on Webhook Relay blog: https://webhookrelay.com/blog/2020/08/27/cdn-types-and-setup/

Easy and secure Jenkins Operator deployment on Kubernetes

Karolis — Mon, 06 Jul 2020 10:23:45 +0000

In this tutorial, we will configure a Jenkins pipeline on Kubernetes that leverages Jenkins and Webhook Relay operators. Jenkins Kubernetes operator will be creating Jenkins instances with a predefined seed job. Webhook Relay operator will ensure that GitHub webhooks on push events trigger new Jenkins builds for a fast and efficient CI/CD experience.

Advantages of this setup:

Your Jenkins instance is only accessible through kubectl port-forward while maintaining the ability to receive webhooks from public destinations.
Jenkins pipeline configuration is stored in Git.
Webhook Relay routing configuration is stored in Git, the same as the Jenkins itself.

You can read about operator pattern in Kubernetes docs.

Prerequisites

Prerequisites:

Helm
Webhook Relay account
Kubernetes environment, Minikube, k3s, GKE, AKS, etc. are fine.
Configured kubectl
Git

Installation

The installation will consist of several steps:

Installing Jenkins operator
Installing Webhook Relay operator

Create a fresh namespace

Let's start by creating a new namespace where we will put our Jenkins instance and run builds. I will call it 'jenkins' but you can choose any other name:

kubectl create namespace jenkins

And then switch to it:

kubectl config set-context $(kubectl config current-context) --namespace=jenkins

Jenkins Operator

We will install Jenkins operator using Helm. First, add the repository:

helm repo add jenkins https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/chart
helm repo update

Once the repository has been added, install it:

helm install jenkins-operator jenkins/jenkins-operator

Official docs can be found here: https://jenkinsci.github.io/kubernetes-operator/docs/installation/. The operator is not the Jenkins itself so to get our Jenkins instance, we will have to create a Custom Resource.

Custom resources are extensions of the Kubernetes API. This page discusses when to add a custom resource to your Kubernetes cluster and when to use a standalone service. It describes the two methods for adding custom resources and how to choose between them.

Start Jenkins (using Custom Resource)

We will need to create a CR. You can either use Jenkins Operator docs to create one or you can fork this https://github.com/webhookrelay/jenkins-operator-example.git repository and clone it. Then:

Update jenkins_cr.yaml file https://github.com/webhookrelay/jenkins-operator-example.git to your own repository fork (it will usually be https://github.com/<your username or organization>/jenkins-operator-example.git).
Create it with kubectl:

  kubectl apply -f jenkins_cr.yaml

Main differences in this file from the stock example:

Added github plugin as we will need it to trigger jobs
Seed job got githubPushTrigger: true set as well

Creating this PR should result in two additional containers:

kubectl get pods
NAME                                      READY   STATUS    RESTARTS   AGE
jenkins-jenkins                           1/1     Running   0          7m11s
jenkins-operator-6dbbc458c9-gmx6p         1/1     Running   0          18m
seed-job-agent-jenkins-65cc4bc684-9ztr5   1/1     Running   0          6m21s

Let's connect to Jenkins. First, get username and password:

kubectl --namespace jenkins get secret jenkins-operator-credentials-jenkins -o 'jsonpath={.data.user}' | base64 -d
kubectl --namespace jenkins get secret jenkins-operator-credentials-jenkins -o 'jsonpath={.data.password}' | base64 -d

Then, in one terminal start port forwarding:

kubectl port-forward jenkins-jenkins 8080:8080

And then just open http://localhost:8080 in your browser.

Webhook Relay

Retrieve your access token key & secret pair from https://my.webhookrelay.com/tokens and set them as an environment variables:

export RELAY_KEY=xxxxxxxxxxxx
export RELAY_SECRET=xxxxx

Add Webhook Relay Operator Helm repository and install it:

helm repo add webhookrelay https://charts.webhookrelay.com
helm repo update
helm upgrade --install webhookrelay-operator --namespace=jenkins webhookrelay/webhookrelay-operator \
  --set credentials.key=$RELAY_KEY --set credentials.secret=$RELAY_SECRET

Operator doesn't forward webhooks on its own. Each created CR will ensure an agent deployment that is configured to route specific buckets.

From the operator example repository we will need to create Webhook Relay Custom Resource:

kubectl apply -f webhookrelay_cr.yaml

Note that if you have modified Jenkins CR name you will need to update webhookrelay_cr.yaml "destination" field from destination: http://jenkins-operator-http-jenkins:8080/github-webhook/ to whatever your current Jenkins service is. Typically it will be in a format jenkins-operator-http-<CR name>.

GitHub Configuration

This step could be automated by making Jenkins automatically configure Github repositories to forwarding to this endpoint, however for simplicity and so that it's more clear how it works, we will add this URL manually.

Get your Webhook Relay public URL

To get your public endpoint you can either visit https://my.webhookrelay.com/buckets page or get it via CR status:

kubectl get webhookrelayforwards.forward.webhookrelay.com forward-to-jenkins -o 'jsonpath={.status.publicEndpoints[0]}'

Result should look something like:

$ kubectl get webhookrelayforwards.forward.webhookrelay.com forward-to-jenkins -o 'jsonpath={.status.publicEndpoints[0]}'
https://k0yv9ip5sxxp55ncsu936k.hooks.webhookrelay.com

Add public URL to GitHub repository settings

Take the public endpoint URL and add it to your GitHub repository:

Using the pipeline

First, when the pipeline is created, trigger the build manually. After that, any push to your GitHub repository will send a webhook through Webhook Relay to your Jenkins instance that's running inside a Kubernetes cluster:

This tutorial was initially published on Webhook Relay website: https://webhookrelay.com/v1/tutorials/webhooks-jenkins-operator-kubernetes.html

Simple trick to get specific elements from Kubernetes kubectl output

Karolis — Sun, 05 Jul 2020 18:09:10 +0000

Usually, most Kubernetes objects are quite big as they have namespace, name, labels, template, spec and a bunch of other fields. When you need to get only one or several fields, it can get tedious scrolling up and down in your terminal looking for it :) Luckily, kubectl supports formatting your output using JSON path that allows you to specify the format and values yourself.

In the following example I will show how to format a Custom Resource output from the operator that I have recently wrote so it's easy to retrieve data from the status field.

First, let's see what data is available:

$ kubectl get  webhookrelayforwards.forward.webhookrelay.com -o json
{
    "apiVersion": "v1",
    "items": [
        {
            "apiVersion": "forward.webhookrelay.com/v1",
            "kind": "WebhookRelayForward",
            "metadata": {
                "annotations": {
                    "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"forward.webhookrelay.com/v1\",\"kind\":\"WebhookRelayForward\",\"metadata\":{\"annotations\":{},\"name\":\"forward-to-jenkins\",\"namespace\":\"jenkins\"},\"spec\":{\"buckets\":[{\"inputs\":[{\"description\":\"Endpoint for GitHub\",\"name\":\"public-endpoint\",\"responseBody\":\"OK\",\"responseStatusCode\":200}],\"name\":\"jenkins-whr-operator\",\"outputs\":[{\"destination\":\"http://jenkins-operator-http-jenkins:8080/github-webhook/\",\"name\":\"jenkins\"}]}],\"image\":\"webhookrelay/webhookrelayd-ubi8:latest\"}}\n"
                },
                "creationTimestamp": "2020-07-05T12:52:11Z",
                "generation": 2,
                "name": "forward-to-jenkins",
                "namespace": "jenkins",
                "resourceVersion": "270615",
                "selfLink": "/apis/forward.webhookrelay.com/v1/namespaces/jenkins/webhookrelayforwards/forward-to-jenkins",
                "uid": "9d75809c-1795-45e2-a437-bb89b22bdac4"
            },
            "spec": {
                "buckets": [
                    {
                        "inputs": [
                            {
                                "description": "Endpoint for GitHub",
                                "name": "public-endpoint",
                                "responseBody": "OK",
                                "responseStatusCode": 200
                            }
                        ],
                        "name": "jenkins-whr-operator",
                        "outputs": [
                            {
                                "destination": "http://jenkins-operator-http-jenkins:8080/github-webhook/",
                                "name": "jenkins"
                            }
                        ]
                    }
                ],
                "image": "webhookrelay/webhookrelayd-ubi8:latest"
            },
            "status": {
                "agentStatus": "Running",
                "publicEndpoints": [
                    "https://k0yv9ip5sxxp55ncsu936k.hooks.webhookrelay.com"
                ],
                "ready": true,
                "routingStatus": "Configured"
            }
        }
    ],
    "kind": "List",
    "metadata": {
        "resourceVersion": "",
        "selfLink": ""
    }
}

Here, if we want to grab just the value from publicEndpoints, we specify again the resource and additional jsonpath query:

$ kubectl get webhookrelayforwards.forward.webhookrelay.com forward-to-jenkins -o 'jsonpath={.status.publicEndpoints[0]}'
https://k0yv9ip5sxxp55ncsu936k.hooks.webhookrelay.com

Go ahead and try it on your own Kubernetes cluster! :)

You can find more examples in the official docs here: https://kubernetes.io/docs/reference/kubectl/jsonpath/.

Serverless webhook transformation (DockerHub to Slack)

Karolis — Fri, 28 Feb 2020 08:49:56 +0000

Many Docker registries provide a way to notify team in chat channels when new images are pushed (if you are waiting for a build complete). Let's add this capability to the official DockerHub registry! :)

We will use newly released Webhook Relay Functions to transform webhooks while they are passing through the service.

Prerequisites:

Create a bucket and configure DockerHub notification

Create a bucket here https://my.webhookrelay.com/buckets
Once you have it, in the inputs section you will find your public input endpoint, copy it:
Add a new DockerHub webhook setting pointing at our public input endpoint (DockerHub docs):

Get a sample of DockerHub webhook

Push a new Docker image:

$ docker push karolisr/demo-webhook:latest
The push refers to repository [docker.io/karolisr/demo-webhook]
48bd38e03c42: Mounted from karolisr/webhook-demo
fd9f9fbd5947: Mounted from karolisr/webhook-demo
5216338b40a7: Mounted from karolisr/webhook-demo
latest: digest: sha256:703f2bab2ce8df0c5ec4e45e26718954b09bf4a625ab831c6556fd27d60f1325 size: 949

We should be able to see a new incoming webhook. It looks like this:

{
    "push_data": {
        "pushed_at": 1582839308,
        "images": [],
        "tag": "latest",
        "pusher": "karolisr"
    },
    "callback_url": "https://registry.hub.docker.com/u/karolisr/demo-webhook/hook/242ii4ddc2jji4a0cc44fbcdbcdecj1ab/",
    "repository": {
        "status": "Active",
        "description": "",
        "is_trusted": false,
        "full_description": "",
        "repo_url": "https://hub.docker.com/r/karolisr/demo-webhook",
        "owner": "karolisr",
        "is_official": false,
        "is_private": true,
        "name": "demo-webhook",
        "namespace": "karolisr",
        "star_count": 0,
        "comment_count": 0,
        "date_created": 1524557040,
        "repo_name": "karolisr/demo-webhook"
    }
}

Create a Function to transform the webhook

Go to the Functions page and click on a "Create Function" button. Enter a name, for example "dockerhub-to-slack" and click "Submit".

For this example we are using Lua language which provides an easy way to parse, transform payloads. Webhook Relay Functions also support WebAssembly modules, however they are currently under development and can only be added/updated using relay CLI.

You can now copy/paste webhook payload into the "request body" area for later tests. In the code editor let's add a Lua function to get repository name and prepare a Slack webhook payload:

local json = require("json")

local body, err = json.decode(r.RequestBody)
if err then error(err) end

local message = "New image pushed at: " .. body["repository"]["repo_name"] .. ":" .. body["push_data"]["tag"]

-- Preparing Slack payload
local slack = {
    response_type= "in_channel", 
    text= message}

local result, err = json.encode(slack)

-- Set request header to application/json
r:SetRequestHeader("Content-Type", "application/json")
-- Set request method to PUT
r:SetRequestMethod("POST")
-- Set modified request body
r:SetRequestBody(result)

Click "Save" and then try testing it with the "Send" button:

Connect everything together

Navigate to https://api.slack.com/messaging/webhooks and click "Create your Slack app". Select your workspace, enter a name that you will remember.
Create a new incoming webhook configuration, copy "Webhook URL" (it starts with https://hooks.slack.com/services/T3...), we will need to supply it to Webhook Relay.
Open your bucket details (via https://my.webhookrelay.com/buckets)
Open "OUTPUT DESTINATIONS" tab and create a new output called "Slack" with the Slack URL from step 2:
Once created, click on the "code" symbol and from the dropdown select dockerhub_to_slack function:

Push new image to DockerHub, you should see a new notification in your Slack channel:

That's it, feel free to continue modifying Lua function to include pusher's name and message format. Following this process you can transform any webhook into any other webhook.

Have fun!

Originally released on https://webhookrelay.com/v1/examples/convert-dockerhub-webhook-to-slack.html

Sunstone - simple templates for Kubernetes and beyond

Karolis — Tue, 19 Nov 2019 16:33:36 +0000

Sunstone is an easy to use, no-CLI solution to create templates that work with plain curl. It's targeted at users that need some templating functionality but don't want to use tools like Helm that need their own registries, directory structures and update mechanisms.

Currently Sunstone is targeted at Kubernetes users but there's actually no restriction or issues when using it with any other tools (it doesn't really care whether templates are Kubernetes manifests or Docker Compose files or anything else).

Project goals:

Users host templates on their own infrastucture that is reachable from Sunstone (GitHub, GitLab, S3, NGINX static file server).
Easy to use and open alias system where users can share their own templates. We could potentially allow hub yaml manifest to point to other hubs and discover aliases from them.
When fetch is not possible, user can create and update private repo in the Sunstone via curl or a specialized Docker container (currently works as a GitHub action but a standalone instructions will be available) that can be included as a CI step.
Dead simple installation instructions for your docs. For example, to install a Dotscience ML model deployment operator for your user account, it's as simple as:

   kubectl apply -f https://sunstone.dev/dotscience?token=my-super-secret-token

Kubernetes deployment page in dotscience.com deployers page:

Example 1: Remote template

Here we have a template hosted on GitHub at: https://github.com/sunstone-dev/example/blob/master/deployment.yaml. Contents are:

apiVersion: apps/v1
kind: Deployment
metadata: 
  name: pushwf  
  labels: 
    name: "pushwf"
spec:
  replicas: 1
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: pushwf
  template:
    metadata:
      name: pushwf
      labels:
        app: pushwf
    spec:     
      containers:                    
        - image: keelhq/push-workflow-example:{{ .version | latestRegistrySemver "keelhq/push-workflow-example" }}
          imagePullPolicy: Always
          name: pushwf
          ports:
            - containerPort: {{ .port | default 8500 }}
          livenessProbe:
            httpGet:
              path: /
              port: {{ .port | default 8500 }}
            initialDelaySeconds: 10
            timeoutSeconds: 5

(raw link is: https://raw.githubusercontent.com/sunstone-dev/example/master/deployment.yaml)

Have a look at {{ .version | latestRegistrySemver "keelhq/push-workflow-example" }}, this latestRegistrySemver template tag will actually go to that registry and retrieve the latest semver tag from the registry :) No need to update your docs when a new semver image tag is released!

Now, to render a template we need to know that Sunstone template generator API works like this:

https://sunstone.dev/<URL to link but without https>?<first arg>=<value>&<second arg>=<value>

So, to render our example template with default values, use:

https://sunstone.dev/raw.githubusercontent.com/sunstone-dev/example/master/deployment.yaml

Result should look like:

apiVersion: apps/v1
kind: Deployment
metadata: 
  name: pushwf  
  labels: 
    name: "pushwf"
spec:
  replicas: 1
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: pushwf
  template:
    metadata:
      name: pushwf
      labels:
        app: pushwf
    spec:     
      containers:                    
        - image: keelhq/push-workflow-example:0.11.0-alpha
          imagePullPolicy: Always
          name: pushwf
          ports:
            - containerPort: 8500
          livenessProbe:
            httpGet:
              path: /
              port: 8500
            initialDelaySeconds: 10
            timeoutSeconds: 5

If you add ?port=999 to the URL:

https://sunstone.dev/raw.githubusercontent.com/sunstone-dev/example/master/deployment.yaml?port=9999

Endpoint will show you different container port:

...
    ports:
            - containerPort: 8500
          livenessProbe:
            httpGet:

...

kubectl allows to install directly from this URL:

https://sunstone.dev/raw.githubusercontent.com/sunstone-dev/example/master/deployment.yaml?port=9999

Example 2: Using aliases

Constructing whole URL is usually quite difficult to remember, that's why we have a public aliases "hub" that allows everyone to map their own shorter aliases to remote repositories. Hub repository can be found here: https://github.com/sunstone-dev/hub. If you would like to contribute your own template, just fork it and submit a pull request.

To install from alias, it becomes as simple as:

kubectl apply -f https://sunstone.dev/keel

And you can also view all public aliases here: https://apps.sunstone.dev/dashboard

Clicking "Install" on any:

Entering into variable fields will automatically generate your template render URL:

Next steps

Check out Sunstone template docs: https://about.sunstone.dev/examples/#latest-docker-semver-tag
Not yet documented, but you can check out private repositories (requires login with GitHub just for authentication, no access to repos): https://apps.sunstone.dev/private-repositories
Accompanying GitHub action to automatically update your private alias on Sunstone when you push to that repo: https://github.com/marketplace/actions/sunstone-template-update

Tech stack

Golang for the backend (API, templating)
Vue.js
Database - Firestore on GCP
Running on GKE

P.S.

Sunstone templating API will be Open Source (still need to do some work to separate it from the current multi-tenant service) so you will be able to just host it anywhere. It might be possible to also run it via Cloud Run or similar services.

A simple way to keep your Vue page title in sync with the router

Karolis — Sun, 08 Sep 2019 22:11:10 +0000

I have noticed in quite a few projects that developers forget to keep page title updated with the router or maybe think that they will do it tomorrow as it is such a small feature :). It always makes sense to keep the title synchronized with the contents for several reasons:

helps users with more than one tab
important for website analytics

I will show you how to do it with the standard vue-router.

Step 1: Declare route meta in your router config

First things first, let's add some additional metadata to our standard routes:

  routes: [
    {
      path: '/',
      name: 'home',
      component: HomePage,
      meta: {
        auth: true,
        title: 'Dashboard'
      }
    }, {
      path: '/login',
      name: 'login',
      component: LoginPage,
      meta: {
        auth: false,
        title: 'Login'
      }
    }, {
      path: '/buckets',
      name: 'buckets',
      component: BucketsPage,
      meta: { auth: true, title: 'Buckets' }

More docs on the router package can be found in the official vue router website.

Step 2: Add $route watcher in your App.vue

Go to your Vue main .vue file (it should have a <router-view></router-view> component) and add a watcher:

    watch: {
      '$route' (to, from) {
        document.title = to.meta.title || 'Your Website'
      }
    },

This will set a title as your page title (if you specified a meta field for that route). I have tried to achieve the same with router navigation guards but $route watcher seemed like the simplest solution.

Looks easy, right? :) Feel free to experiment and define more fields in the router meta like page description and anything else you want to be set for that page. Then modify the watcher to also use them.

Hope this helps!