Forem: Thorsten Hirsch

Why Scale To Zero?

Thorsten Hirsch — Tue, 22 Apr 2025 11:16:32 +0000

What's the point of scaling to zero? Seriously, I don't understand why anybody would want to consider this as a goal in their architecture as it complicates things for nearly zero benefit.

Savings Are Close To Zero

Let's take a microservice application with a database, a message broker, several frontend and backend containers. Some parts scale up easily, others are more complicated, but the point is that you need at least 1 instance of each component for your whole application to work properly. We also address fault tolerance by only using managed services and a Kubernetes cluster with multiple nodes.

The costs could look like this:

$ 1,700 Kubernetes Cluster (3 nodes)
$ 100 Message Broker (as Service)
$ 150 Database (as Service)

Sum: $ 1,950

If this setup is enough for 10,000 users, we scale up for 1,000,000 users (Congratulations!) with this setup:

$ 28,000 Kubernetes Cluster (50 nodes)
$ 10,000 Message Broker (as Service)
$ 15,000 Database (as Service)

Sum: $ 53,000

...and we can scale back down to $ 1,950 just like that. No architectural change is needed, just shutdown the additional instances we used for scaling up to 1,000,000 users and we save 96,3 % of our costs.

Scaling to zero would only save an additional 3,7 %, which is equivalent to going from 960,000 users to 1,000,000 users. Did we think about this step the same way as we think about scaling from one to zero? Probably not, because it's too small - we rather focused on scaling up from 800,000 to 1,000,000 users or something like that.

Keep The Architecture Simple

Scaling up and down between the least number of instances and what's required for 1,000,000 users does not require any architectural change. Just keep the bare minimum of instances running and as soon as traffic goes up Kubernetes is able to (auto-) scale horizontally. But scaling to zero is a different kind of beast. It means you really shutdown components completely and need to wake them up somehow, e.g. by issuing an API call to Kubernetes to start the very 1st container of your frontend. This requires additional tooling or additional components. Think about the following insane approach of scaling to zero:

There's a worker queue in your message broker with a task waiting to be processed, but since you scaled down your worker containers to zero you now need to implement some kind of monitor that checks all queues for any waiting messages, which is then able to scale up the corresponding container from zero to one.

Please don't even think about shutting down your message broker if your monitor script finds all queues empty. I already heard people suggest building another message layer for such events just to inform another additional, always-running component that it now has to start the container with the "real" message broker.

What About Test Environments?

You don't want a different architecture in your test environment than in your production environment! So, you don't want to have additional components just to be able to scale to zero. If really nobody is using your test environment for quite a while, just shutdown it down completely and start it up manually as soon as you need it.

What About Infrequent Usage?

Do you mean user-driven infrequent usage? I'd say your minimal setup is still the best way to go. If the number of users is so small that they don't cover your server costs you better think about a way to attract more users instead of implementing scale-to-zero.

And what about cronjobs that only run once a month? If this is the only usage of your application you can call yourself lucky, because then you can exactly plan your usage. Just startup and shutdown your environment in the same cronjob. Notice that a planned shutdown is not what it means for an application to be able to (automatically!) scale-to-zero.

Do I Have A Clue?

Not really, so I just googled "scale to zero" and found this guy on Reddit, who explores options for scaling to zero in Kubernetes. The very first questions he asks:

What [additional] tools are required?
What do they cost?

That's exactly what I'm complaining about! No, it's worse: not only do these additional tools mess up your beautiful, simple architecture, they also eat up the tiny savings you could've achieved if they're not for free.

tl;dr

Please don't waste your time trying to make your application scalable to zero.

Who Writes Helm Charts?

Thorsten Hirsch — Mon, 03 Jul 2023 18:03:03 +0000

It seems like for Helm charts (and their values files) the whole IT department needs to provide input:

What are the dependencies of the application?
What settings are available to customize the application?
Server names, URL of DB, storage settings, maybe also message broker, and all other infrastructure/platform requirements
Initial resource allocation, scaling settings, deployment strategy, HA settings (if on application level)
Internal network settings like ports to be opened between your pods, service mesh settings & mTLS, certificates & their renewal
Usernames and passwords in the different stages, path to secrets, additional certificates that need to be imported
Backup settings, agents, storage path, interval settings
Monitoring & logging agents, forwarding settings

If you work at a small company you might only have 2 or 3 people filling all these roles. But imagine working at a big company, where you really have 10 different people. Is it even possible under these circumstances to write Helm charts (and their values files) for an application in a single sprint?

So what's the solution? Who writes Helm charts (and their values files) at your workplace? Is splitting responsibility between charts and values files a good start? Where do you draw the line between things that belong into Terraform scripts and things that go into Helm charts?

Web5: A Better Web3 Approach?

Thorsten Hirsch — Sun, 12 Jun 2022 21:33:26 +0000

tl;dr: On June 10 Jack Dorsey (ex Twitter CEO) announced Web5, which is basically Web3, but without the token scams.

The Blockchain Wars

The crypto space is somehow divided in Bitcoin maximalists, who accept no other ~~god~~ blockchain. Their position is that Bitcoin is the only valid use case for blockchains. It's also the only distributed and secure blockchain, as well as the only one that's not a Ponzi scheme. Unfortunately it is wasting enormous amounts of energy.

On the other side we have Ethereum and all the more advanced blockchains, which are basically frameworks for building decentralised apps on top. The problem: many "apps" are just coins, whose developers might or might not be scammers, using the coins to fund their projects outside of the regulated investment channels. Most of these blockchains do not kill our planet, as they use other consensus mechanisms than Bitcoin. Well, Ethereum still does, but at least they admit it's a problem and try to migrate.

Web5

Jack Dorsey is a Bitcoin maxi, but he also likes the Web3 idea of decentralised apps with self-sovereign identity and user controlled data management for privacy. So he and his company TBD are now developing the building blocks for such applications in the Bitcoin space.

The key difference to Web3 is: only identity is stored on-chain. Everything else is stored on user-run nodes. This means: no smart contracts, no "shitcoins" (I don't like this term), and no fraudulent tokenomics with air drops, yield farming, or staking.

Building Blocks

• Decentralised identifiers (DiDs). They are unique IDs for every user, like a public key (wallet address) in Web3. They are implemented as a Layer 2 protocol on Bitcoin (like Lightning) called ION.

• Verifiable credentials (VCs) are additional information for each user. This might your home address, a digital signature, your age, or something else. AFAIK access can be implemented with ZK algorithms to protect privacy.

• Decentralised Web Nodes (DWNs). DWNs are decentralised computers. They store data and facilitate communication between DiDs. Analogous to smart contracts in Web3, but run locally (I guess this means on a single node) instead of "on the blockchain" (which means on every node).

• Decentralised Web Apps (DWAs). Users interact with Web5 using a DWA. This is like a web app with a key difference: users control their own data and there is no intermediary.

Summary

Since applications cannot be implemented directly in Bitcoin (due to the lack of smart contracts) Web5 brings building blocks, such as an L2 protocol for identity and data management. This seems like a bad deal for developers, as everything gets more complex, but it might solve the problems and frauds we see in Web3 to some extent... so why not give it a try?

Further Information

You can read more about Web5 on TBD's project website: https://developer.tbd.website/projects/web5/

They also have slides (PDF).

Credits

I have to thank @ntkris, because I copied a lot from his tweets for this article.

Rails 6 with Metaprogrammed Models

Thorsten Hirsch — Sun, 06 Dec 2020 12:10:21 +0000

The Task

In the last weeks I had to migrate several databases to a new one, which was mostly copying data, but also a bit of enriching data. It was just a little bit too much for plain SQL and since Rails 6 has multi database support I wanted to give it a try.

Here's an example of the task:

So we already have both schemas, the one in the old database and the one in the new database. The table names are all the same in both databases, some models just have some additional fields in the new database like persons.country_of_phone_number. We fill this new field during migration by fetching the country code from the phone number (e.g. "+49") and doing a lookup to which country this country code belongs (e.g. in a REST service or another table).

Not The Plan

Since this migration is a one-time-procedure that has nothing to do with the evolution of the rails application we won't "pollute" the code of the rails application with our migration task. Instead we will create a new rails app just for the migration.

We will also not copy/generate the model files from the existing rails application since we don't need any model functionality. We just want to copy data. But how does database access work without (ActiveRecord based) model files?

The Plan

It's possible to generate model files on-the-fly just as we need them. Here's the code:

def generate_class_if_not_exists(namespaced_model)
  check = namespaced_model.safe_constantize
  if check.nil?
    mod = namespaced_model.split(/::/)[0]
    klass = namespaced_model.split(/::/)[1]
    eval %{
module #{mod}
  class #{klass} < BaseRecord
  end
end
}
  end
end

The indentation looks a bit ugly, but I like it better than using lots of "\n" in a single line string.

So let's look a the code. The most important thing to notice is that we generate a class that inherits from BaseRecord in a module. And that's the key to access different databases. So we need to make sure, that we have one module for the old database and another one for the new database.

Database Configuration

Rails 6 has introduced one more level in database.yml in order to define multiple databases:

development:
  primary:
    <<: *default
    database: new_db
    username: 'newuser'
    password: 'newpassword'
  old_db:
    <<: *default
    database: old_db
    username: 'olduser'
    password: 'oldpassword'
  another_old_db:
    <<: *default
    database: another_old_db
    username: 'olduser'
    password: 'oldpassword'
  end
end

As you can see, I recommend to use the primary database for the (single) new database, thus all old databases have specific names.

BaseRecords

Now in order to tell ActiveRecord which database to use we have to implement the BaseRecord for each database. Since the new database uses the primary db configuration we don't need to specify the database in app/models/new_db/base_record.rb:

module NewDb
  class BaseRecord < ApplicationRecord
    self.abstract_class = true
  end
end

But all old databases must be linked to their database.yml configuration, so let's take a look at app/models/old_db/base_record.rb:

module OldDb
  class BaseRecord < ApplicationRecord
    connects_to database: { writing: :old_db, reading: :old_db }
    self.abstract_class = true
  end
end

That's it for the setup.

The Migration

Now the migration is really easy. We can write a rake task for that:

  desc 'migrate all tables'
  task :migrate => :environment do
    %i( :persons :foos :bars ).each do |table|
      migrate_table(table)
    end
  end

  def migrate_table(table)
    table = table.to_s.gsub(/^:/, '')

    source_string = 'OldDb::'
    source_string += table.classify
    generate_class_if_not_exists(source_string)
    source = source_string.safe_constantize

    target_string = 'NewDb::'
    target_string += table.classify
    generate_class_if_not_exists(target_string)
    target = target_string.safe_constantize

    begin
      # first try to call a specific method for data enrichment
      send("table_#{table}".to_sym, source, target)
    rescue NoMethodError => _e
      # no specific method found means: no enrichment, just copy the data
      normal_table(source, target)
    end
  end

So we have two cases here:

tables that are 100% the same in old db and new db use normal_table to just copy the data
tables that need enrichment use a specific method, e.g. table_persons for adding the Country-of-Phone-Number property; we need to implement a specific method for each table that needs enrichment

Here are both methods:

  def normal_table(source, target)
    puts "migrating #{source} to #{target}..."
    begin
      source.all.each.with_index do |u|
        s = target.send(:new, u.attributes.except(:id))
        s.save!
      end
    rescue => e
      puts e
    end
    puts "#{target.count} of #{source.count} records migrated."
  end

  def table_persons(source, target)
    puts "migrating #{source} to #{target}..."
    begin
      source.all.each.with_index do |u|
        s = target.send(:new, u.attributes.except(:id))
        s.country_of_phone_number = some_magic_function(u)
        s.save!
      end
    rescue => e
      puts e
    end
    puts "#{target.count} of #{source.count} records migrated."
  end

And that's it. We can copy hundreds of tables from old_db to new_db with just two files under app/models.

Discussion: Metaprogramming

We always should ask ourselves: would another developer understand my codebase? There are several aspects we need to consider, e.g.:

How big is my codebase?
How readable is my code?
How complex is my code?
Do I make use of advanced techniques?

On the plus side our migration project is a very minimalistic rails project with no legacy code and the least amount of self written code I can think of. But we have used advanced techniques like send(:method, :param1, :param2) in order to call methods by names we determine at runtime (dynamically). This might be called "metaprogramming". There's even a book for that:

You can buy it here

One of the claims in this book is that metaprogramming in ruby is just programming, it's nothing special, because you don't need to access anything "forbidden". Other languages might have things like __method_name__ to access metaprogramming, which already tells you by the name "Caution! My double underscore prefix is reserved for language constructs. Here be dragons!"

Ruby is different. Take a look at the object model, for example, it's a real beauty - go check it out! There's no need for ruby to draw a line between programming and metaprogramming, because ruby's advanced concepts still rely on simple, yet powerful foundations, that are easy to grasp.

So I think my codebase is easy to understand mainly thanks to its brevity. You only have to know one or two advanced concepts. And IMHO that's much better than writing a lot of code explicitly, because the more code I write, the more bugs I write, and the more code another developer has to read.

What do you think?

(The cover image is "Final Train" by Jason Heeley.)

Looking for Angular Architecture Advise

Thorsten Hirsch — Sat, 07 Nov 2020 11:13:44 +0000

After building one or two Angular applications it's time to ask if I'm doing things right. There might be a lot of beginner tutorials for Angular out there, but it's hard to find architectural best practices. My most urgent question is:

How long to keep using data as an Observable? When can/should I "resolve" it to its actual data e.g. with the async pipe?

The answer might be "as long as possible", so I should always try to use an observable and only "resolve" it in the the html template where the data is finally being shown.

But I guess it's easier to use an observable as short as possible by resolving it in the parent component and providing the data as input for the child components like so:

I really don't know the answer. And here are some more questions I keep asking myself:

Since I only want to fetch data from my http server once, but use it several times - where do I use ReplaySubjects? In the Services or in the Components?
Should I instantiate Observables/Subjects when declaring their variables? Or should I instantiate in the constructor?
When it comes to subscribe/unsubscribe I should probably use the ngOnInit() and ngOnDestroy() hooks, which are available in Components only... so I guess I should never subscribe/unsubscribe in my services, right?
Is it a bad pattern to have an Observable foo$ together with its resolved data foo in the same class?

Please share your recommendations!

Attributions for the cover image go to jannoon028 from www.freepik.com.

Just JS (by Dan Abramov)

Thorsten Hirsch — Sat, 08 Aug 2020 20:07:45 +0000

This is a recommendation of a JS tutorial for beginners by Dan Abramov. It's called Just JS and I liked it very much, especially for the visualisations by Maggie Appleton. The cover image of this article is one of hers.

Unfortunately the tutorial is only available by mail and it's very shot, 9 courses only. But I can assure you: it's worth it. The mental model for how the inners of JS work are pure gold. You can subscribe here on Dan's website for Just JS.

P.S.: Dan Abramov is working on React and the (co-) creator of Redux and create-react-app.

Q: What Technologies Should I Use for Migrating a Rails Monolith to Microservices in 2020?

Thorsten Hirsch — Mon, 23 Dec 2019 17:26:55 +0000

So I have this 6 years old rails application monolith that contains all these things:

user interface
database
additional tables for statistics (snowflake schema)
API for android app
workflow engine
batch workers (sidekiq)

Currently I'm running 4 instances of this application, each is configured especially for the customer who's using the instance. Yes, that makes 4 customers, each has ~300 users, so we're talking about 1200 users. A virtual server with 2 cores and 8 GB RAM handles it easily. The web server is nginx and the ruby application server is passenger.

In 2020 a new UI (SPA) will be written, so I can get rid of the UI in rails. However I will have to provide an API for the SPA and it will be totally different than the API for the android app.

What a great opportunity to split the monolith into microservices!

I might split the microservices like this:

API for new UI
API for android app
batch workers
statistics
what's left in the old rails app (business logic, workflow engine)

So far I pretty much know what I'm doing. However I don't have much experience in the microservices world, so I have a bunch of questions for you:

What about user authentication at the API for the new UI? Are JSON web tokens the way to go?
What about the database? Should I split it along the microservices?
How do the microservices communicate with each other? I prefer a message bus over direct REST calls, so...?
- RabbitMQ?
- Kafka?
- Nats?
Is it worth learning the whole Kubernetes stack? Or is docker plus docker-compose (or helm???) enough? Any other alternatives?

Actually my monolith isn't even dockerized, yet. Currently I feel overwhelmed by all the technologies I can chose from. But what's the right technology stack for rails? And for me? This rails application is only a hobby project. Any recommendations?

Automatic Termination of Closed Source Licenses

Thorsten Hirsch — Tue, 15 Oct 2019 23:06:36 +0000

Let's slip into the role of your customer...

The Problem

So you've finally found an app for your business in the app store. It's being developed by an aspiring developer who somehow happens to know exactly what you need. Sure, the app has some rough edges, but after you give him some feedback it gets better and better. You decide to buy the pro version with an annual fee, and you integrate all your workflows in the app. Everything's great!

Some years later your business relies completely on this app, but suddenly you receive a letter from the developer:

Dear customers,

as you might have noticed, there have been no updates for the app in a while. The reason is that I've lost interest, so I've decided to abandon it.

So long, and thank you for the fish!
Your aspiring developer.

Oh shit! You immediately contact him in order to convince him to continue supporting your beloved app. You even throw cash at him, but to no avail. He says he just landed a job at Google, something with AI, and has no interest anymore to solve your worldly problems.

You keep using the (now unsupported) app, but after upgrading your Mac to Catalina it doesn't work anymore. You're afraid you have to close your business.

Open Source to the Rescue!

Enough role-play. Maybe some of us can relate to the developer in this example and say that this customer was crazy to build his business on an app from a single developer. Or maybe some of us think that the developer acts irresponsible and selfish.

We don't have to pick sides, we just have to accept that situations like these happen and useful or even important software gets abandoned. These situations can easily be resolved if the app was open source - the customer can just pay some other developer to migrate the app on Catalina. But not every developer wants to publish his apps under an open source license. Monetarization might indeed be a problem if you cannot rely on making money with support and services alone. But whatever the reasons might be - the developer is the author, and copyright law grants him to choose whatever license he wants. Then the software lives with its license happily ever after. End of the story.

...or maybe not?

Big Business (Contract Law)

The company I work for has licensed a lot of closed source software. Our lawyers address the risk of losing software support with a paragraph like the following one, which is put in all software vendor contracts:

In case of bankruptcy the vendor must provide the source code of the licensed applications in complete and compilable form.

That seems to be a reasonable solution. However I'm not sure if my employer can really exercise this claim. The software might be composed of different parts, for which the vendor might not have the rights to give others the source code.

Anyhow - here we see a customer's effort to get control over the software's source code when the vendor cannot support it anymore.

Comparison to Copyright in Music

Musicians are paid royalties for the songs they've written or performed. This is a perpetual revenue stream that can (and should) last the whole life of an artist in order to prevent him from poverty. In many countries royalties even continue after the author's death - 50 year, 70 years, maybe even 95 years after his death and are being paid to his descendants. But then the exclusive rights of the author ends and the song enters public domain, which is pretty similar to open source.

So music might be timeless, but copyright enforces "open source" after quite some time. The time of course is very, very long, but music is timeless. Software is not.

Enforcing Open Source by Law

I think it's time for a law that enforces the end of a closed source license - a law that automatically enforces its release under an open source license for circumstances that factually end support, e.g. when a software developer loses interest, when a software vendor goes bankrupt, or when a (single) developer dies.

What do you think of such a law?

P.S.: In my article the term "open source" is used synonymously for free software.

Poll: Do You Know What "Idempotent" Means?

Thorsten Hirsch — Thu, 27 Dec 2018 13:23:51 +0000

In a very good talk about TLS 1.3 at 35c3 Hanno Böck wondered if web developers know what "idempotent" means. Now before you fall into a rage, Hanno was not mean, but rather funny, so let's take his talk as an opportunity to talk about idempotence in web applications. I'm really curious, so let's (mis-) use dev.to's like system as a poll:

Do you know what "idempotent" means or - after checking out wikipedia - used the concept in your projects?

Sometimes idempotent requests to the server are required and obvious, but other times it might be tricky ...or not required at all:

dev.to's like buttons don't seem to be idempotent at first, because clicking once activates the like, clicking twice deactivates the like. You might think that the request is just a toggle. But it's not. Each request contains an action ("create" or "destroy"). So the 1st click produces: {"result": "create", "category": "like"} . And this request is idempotent, because if something goes wrong on the network layer and the server receives this request twice it will still do the right thing, it will activate the like on the 1st request and do nothing on the 2nd request.
Medium's clapping system is not idempotent. A request looks like this: {"userId":"my123id456","clapIncrement":1} . And you can also see that it's not idempotent, because clapping once turns the counter from $old-value to $old-value + 1, while clapping twice turns it to $old-value + 2. But in this case it's not a bug, you are explicitly allowed to clap multiple times.

Please vote:

heart/like = yes, I already knew what "idempotent" means or used the concept in past projects.
unicorn = no, this is new to me or I've seen no need to implement this concept in my projects.

I for myself must admit, that I've learned the term only 2 or 3 years ago. However I knew its concept and have used it for more than 10 years in past projects. I guess the poll is pretty anonymous (maybe with the exception of the dev.to admins), so please vote with honesty.

By the way: 35c3 is one of the largest hacker events in the world and the talks are streamed live. You will also find recordings of the talks later behind this link.

// cover photo by Yves Sorge / CC BY

Do You Know Mental Models in IT?

Thorsten Hirsch — Sun, 17 Jun 2018 16:29:04 +0000

It's been 17 years since I started working in my 1st real job. It was in a bank and it was introduction week. There were 50 other students in the same situation as I and we were all sitting in an awe inspiring room full of ornaments. We were waiting for the next speaker. An elderly man entered the stage. The experience of decades could be seen in the wrinkles in his face. And I will never forget what he told us:

No matter what kind of business department you will work in, always remember that it's all about ledgers.

So this old banker's mental model was the ledger. Actually he was using the T account representation of a ledger in his presentation. And he explained, that no matter what kind of financial business he was working on, he always transferred it on ledgers in his mind. Retail credit, cash transfer, trade finance, lending, money market, funding, ... you name it. By applying all problems on ledgers he could easily understand the business of each and every department in the bank.

Are there Mental Models in IT?

After reading James Clear's article about mental models (and Richard Feynman) I asked myself what mental models we have in IT. What abstractions can we use to reason about any kind of problem we're confronted with? Whenever I reason about a new system I instantly think about its data modelling, so I guess these 2 might qualify as mental models in IT:

Entity Relationships (ER)
Object Oriented Programming (OOP)

Are design patterns a mental model? I don't think so. Each pattern suits a special problem, so a pattern is by far not generic enough to work as a mental model. And what about the sum of all patterns? No, I think trying to find patterns is just human behaviour.

What about Functional Programming (FP)? Since I'm not proficient in FP I don't know if it can work as a mental model. Does it?

Ethereum Mainnet & Kovan Nodes on Same Machine

Thorsten Hirsch — Sun, 31 Dec 2017 13:32:56 +0000

Introduction

Right after installing an Ethereum node on my developer machine, the initial sync caused it to run wild, pushing the fan to its limit, and in the end I had several gigabytes of blockchain data on my tiny SSD. Sounds familiar? Then this guide is for you.

You don't need to run the node on your development machine directly, any computer in your network can do the job. So I've chosen my NAS, which is running Ubuntu 17.10 and has enough space on its HDD.

And since we're developers, it is a good idea not only to have access to the mainnet, but also to one of the test nets. So I will setup an additional parity instance for the kovan chain.

Installation

Please tell me that I was too dumb to find a PPA repo for parity yesterday. I cannot believe that it doesn't exist. Anyway, the parity installation instructions tell you to download a .deb file, so here we go with the beta release, which is 1.8.x at the time of writing:

wget "https://d1h4xl4cr1h0mo.cloudfront.net/beta-release/x86_64-unknown-linux-gnu/parity" -O parity.deb
sudo dpkg -i parity.deb

There's also a download option with curl/bash script, but since you never know what it does I prefer to download and install a .deb file.

Ethereum Setup

We will need an Ethereum account in each instance for interacting with contracts and such. I don't want these accounts (and their keys) in a system directory, since they are probably coupled to my user (I will refer to him as hans in this article). I also want to keep open the option of adding more instances that might use accounts of other users. So let's create some directories in my user's home directory:

mkdir ~/ethereum
mkdir ~/ethereum/mainnet
mkdir ~/ethereum/kovan

And wow it's time to create an account in each chain:

parity account new -d ~/ethereum/mainnet
parity account new -d ~/ethereum/kovan

If you want to import existing accounts instead, you can follow the instruction in the official wiki.

System Setup

Let's start with the directory for our config files and our log files.

sudo mkdir /etc/ethereum
sudo touch /var/log/parity-mainnet.log
sudo touch /var/log/parity-kovan.log
sudo chown hans /var/log/parity-*.log

Now put the config files for our chains into /etc/systemd. You will need sudo permissions for this file and the 3 other files in this paragraph to come. Let's start with /etc/ethereum/config-mainnet.toml.

[parity]
chain = "mainnet"
base_path = "/home/hans/ethereum/mainnet"
identity = ""

[account]
unlock = ["0x6e6dbb24dd98dd37c41a9964a766e5eb64352d9c"]
password = ["/home/hans/ethereum/mainnet/password"]

[ui]
disable = false
port = 8180
interface = "all"

[network]
port = 30303
reserved_only = false
#reserved_peers = "/etc/parity/mainnetpeers.txt"

[rpc]
disable = false
port = 8545
interface = "all"
apis = ["web3", "eth", "net", "parity", "traces", "rpc", "secretstore"]

[websockets]
disable = false
port = 8546
interface = "all"
apis = ["web3", "eth", "net", "parity", "traces", "rpc", "secretstore"]

[ipc]
disable = false
apis = ["web3", "eth", "net", "parity", "parity_accounts", "personal", "traces", "rpc", "secretstore"]

[dapps]
disable = false

[secretstore]
http_interface = "local"
http_port = 8082
interface = "local"
port = 8083

[ipfs]
enable = false
port = 5001
interface = "all"

[misc]
log_file = "/var/log/parity-mainnet.log"

And here's /etc/ethereum/config-kovan.toml.

[parity]
chain = "kovan"
base_path = "/home/hans/ethereum/kovan"
identity = ""

[account]
unlock = ["0xaa125f607bb23f41c0c45c27eca820f2b133d1b6"]
password = ["/home/hans/ethereum/kovan/password"]

[ui]
disable = false
port = 18180
interface = "all"

[network]
port = 30304
reserved_only = false
#reserved_peers = "/etc/parity/kovanpeers.txt"

[rpc]
disable = false
port = 18545
interface = "all"
apis = ["web3", "eth", "net", "parity", "traces", "rpc", "secretstore"]

[websockets]
disable = false
port = 18546
interface = "all"
apis = ["web3", "eth", "net", "parity", "traces", "rpc", "secretstore"]

[ipc]
disable = false
apis = ["web3", "eth", "net", "parity", "parity_accounts", "personal", "traces", "rpc", "secretstore"]

[dapps]
disable = false

[secretstore]
http_interface = "local"
http_port = 18082
interface = "local"
port = 18083

[ipfs]
enable = false
port = 15001
interface = "all"

[misc]
log_file = "/var/log/parity-kovan.log"

Then we need a systemd config file for each instance, so that the system will start parity at boot time (but don't worry, the processes will run under your user account). Let's start with mainnet again, the file name is /etc/systemd/system/parity-mainnet.service.

[Unit]
Description=Parity Mainnet Daemon
After=network.target

[Service]
# run as root, set base_path in config.toml
ExecStart=/usr/bin/parity --config /etc/parity/config-mainnet.toml
# To run as user, comment out above and uncomment below, fill in user and group
# picks up users default config.toml in $HOME/.local/share/io.parity.ethereum/
User=hans
# Group=groupname
# ExecStart=/usr/bin/parity
Restart=on-failure
Nice=15

# Specifies which signal to use when killing a service. Defaults to SIGTERM.
# SIGHUP gives parity time to exit cleanly before SIGKILL (default 90s)
KillSignal=SIGHUP

[Install]
WantedBy=multi-user.target

And here comes /etc/systemd/system/parity-kovan.service.

[Unit]
Description=Parity Kovan Daemon
After=network.target

[Service]
# run as root, set base_path in config.toml
ExecStart=/usr/bin/parity --config /etc/parity/config-kovan.toml
# To run as user, comment out above and uncomment below, fill in user and group
# picks up users default config.toml in $HOME/.local/share/io.parity.ethereum/
User=hans
# Group=groupname
# ExecStart=/usr/bin/parity
Restart=on-failure
Nice=15

# Specifies which signal to use when killing a service. Defaults to SIGTERM.
# SIGHUP gives parity time to exit cleanly before SIGKILL (default 90s)
KillSignal=SIGHUP

[Install]
WantedBy=multi-user.target

You can check if systemd has found your new services with the following command:

systemctl list-units parity*

Test Run

We're ready for a test ride. Just start an instance in your shell (as your user) with one of the config files we just created:

parity --config /etc/parity/config-mainnet.toml

You should see some information and after some seconds the sync process will begin, printing a new status line every few seconds. If everything's fine press Ctrl+c to stop the test run.

Starting & Enabling the Service

First we start the services, then we enable autostart at boot time:

sudo systemctl start parity-mainnet
sudo systemctl start parity-kovan
sudo systemctl enable parity-mainnet
sudo systemctl enable parity-kovan

If everything went fine, you will probably see your shell becoming less responsive. I tried to minimise this effect by giving the parity processes a low priority by setting the nice level to 15. My suggestion for the nice level in general is: since you always want the system to be responsive, never give your user processes a higher priority (i.e. a negative number). A nice level of 15 is pretty low. It can go down as low as 20. However setting the nice level to 20 might not help much, because the load that is caused by the initial (warp) sync of the chains spawns quite a lot of threads. The load on my dual core machine goes up to 8 or 9 during this task. Here is a more detailed explanation of that problem, which is being worked on, but Parity 1.8.x still comes with RocksDB, so for now we have to live with this load and a decreased responsiveness of the shell. It will get better after the initial sync is over.

Port Info

You might wonder about the ports I've chosen. Here's my reasoning:

30303 is the default ethereum port, so mainnet will use it
30304 is not used by any other module, so kovan will use it

The rpc and http ports (8xxx) can not follow the same logic, because some of the default values are consecutive. So here we will use...

8xxx for mainnet (8180 for ui, 8545 for rpc, 8546 for websockets, ...)
18xxx for kovan (18180 for ui, 18545 for rpc, 18546 for websockets, ...)

UPDATE 2018-01-20
systemd: WantedBy=multi-user.target instead of default.target

Building React Components with Bootstrap Studio

Thorsten Hirsch — Sun, 12 Nov 2017 01:10:21 +0000

Do you know Bootstrap Studio? It's a graphical HTML/Bootstrap builder. The latest version features support for Bootstrap 4 and includes a migration function for converting Bootstrap 3 projects to Bootstrap 4. It's not free, but at least the price tag is not very high. It looks like this:

Since I liked the idea of using it for building my Rails web application, I was wondering if there's a better way to use it than manually copy-and-pasting HTML and CSS code from BSS's export. Spoiler alert: Yes, there is!

I recently integrated React into my Rails app and that was just about the right decision, because a company called WIX has written react-templates, which provides a way to wrap a BSS export into a React component. So the idea is to export the BSS page to html, transform it to a React template, which is a Javascript function, and use that function as React's render function in a component. Since BSS can run a custom export script each time I hit the export button, this process can be automated.

Here's a step-by-step guide:

Add react-rails to your rails application, see github page for details.
Create an additional directory components_rt under app/assets/javascripts of your rails application for the template files. You also need to add an import line for it right before the import line of the components directory in app/assets/javascripts/application.js.coffee:
```
#= require_tree ./components_rt
#= require_tree ./components
```
Install the rt command from react-templates:
```
npm install react-templates -g
```

In BSS you can supply an export script. Use this one and change the TARGET to your needs:

#!/bin/sh
TARGET=~/path/to/rails/app/assets/javascripts/components_rt
[[ -z $1 ]]          && echo "argument error: bss export directory required" && exit 1
[[ ! -d $1 ]]        && echo "bss export directory does not exist" && exit 1
[[ ! -d ${TARGET} ]] && echo "target does not exist: ${TARGET}" && exit 1
# is rt installed?
which rt >/dev/null 2>&1
[[ $? -ne 0 ]]       && echo "rt (react-template compiler) is not installed" && exit 1
# main
RC=0
for f in "$1"/*.html; do
    RTFILE="${f%.html}.rt"
    xmllint --html --xpath '/html/body/*' "$f" | tee ${RTFILE}
    sed -i 's|<script .*script>||g' "${RTFILE}"
    sed -i 's|%7B|{|g' "${RTFILE}" # fix due to xmllint/xpath bug 788398
    sed -i 's|%7D|}|g' "${RTFILE}" # fix due to xmllint/xpath bug 788398
    mv "${RTFILE}" "${TARGET}/"
    rt "${TARGET}/$(basename ${RTFILE})"
    RC=$(($RC+$?))
done
exit $RC

Write a React component, let's call it "Hello":

class Hello extends React.Component {
    render: helloRT
}

Now you can build your component in BSS. Rename the html file to "hello.html". It will be transformed into the Javascript function "helloRT" when you click on export.
Use the component in any view of your rails project:

<%= react_component 'Hello' %>

Put your elements under /html/body in BSS. There's only 1 element allowed directly under /html/body, so wrap your stuff in a container element. And use 1 HTML file in BSS for each React component.

Have fun!

P.S.: You might want to extend the export script by the ability to bring CSS files from the BSS export into Rails.