Forem: Afri Schoedon

Ethereum node configuration modes cheat sheet

Afri Schoedon — Sat, 05 Jan 2019 22:33:32 +0000

This document is a quick cheat sheet of most common geth and parity configurations explained — usually, everything you need to know running an Ethereum node.

Light nodes

Light clients only store the header chain and request everything else on-demand from the network. They can verify the validity of the data against the state roots in the block headers.

`❯ geth --syncmode light`

Synchronizes block headers for a light client with the Ethereum network using the Light Ethereum Subprotocol (LES) starting at a trusted checkpoint inserted by the developers:

INFO [01-05|22:42:27.103] Added trusted checkpoint                 chain=mainnet block=6848511 hash=5e9f76…d9d6ac
WARN [01-05|22:42:27.123] Light client mode is an experimental feature 
INFO [01-05|22:43:13.912] Block synchronisation started 
INFO [01-05|22:43:40.953] Updated latest header based on CHT       number=6848511 hash=5e9f76…d9d6ac age=4w9h38m
INFO [01-05|22:44:01.569] Imported new block headers               count=192 elapsed=2.386s number=6848703 hash=2e78ec…851f48 age=4w8h54m
INFO [01-05|22:44:08.962] Imported new block headers               count=192 elapsed=341.059ms number=6848895 hash=4b8c89…d6b233 age=4w8h5m
INFO [01-05|22:44:12.912] Imported new block headers               count=192 elapsed=392.806ms number=6849087 hash=1b80b2…8515b4 age=4w7h15m
INFO [01-05|22:44:42.234] Imported new block headers               count=192 elapsed=379.507ms number=6849279 hash=ef9c20…605416 age=4w6h29m

Note, how it inserts the trusted checkpoint and imports remaining headers afterward.

`❯ parity --light`

Synchronizes block headers for a light client with the Ethereum network using the Parity Light Protocol (PIP) starting at a trusted checkpoint inserted by the developers:

2019-01-05 22:46:08  Running in experimental Light Client mode.
2019-01-05 22:46:08  Inserting hardcoded block #6692865 in chain
2019-01-05 22:46:08  Debug API is not available in light client mode.
2019-01-05 22:46:23  Syncing #6692865 0xa988…8132     0.0 hdr/s      0+    0 Qed   2/50 peers   664 bytes cache 0 bytes queue  RPC:  0 conn,    0 req/s,    0 µs
2019-01-05 22:47:59  Syncing #6693604 0x10bc…90fd   147.8 hdr/s    535+    0 Qed   4/50 peers   480 KiB cache 404 KiB queue  RPC:  0 conn,    0 req/s,    0 µs
2019-01-05 22:48:09  Syncing #6698114 0x1b74…9c97   451.0 hdr/s    122+    0 Qed   3/50 peers   3 MiB cache 95 KiB queue  RPC:  0 conn,    0 req/s,    0 µs

Note, how it inserts a hardcoded block and synchronizes remaining headers subsequently.

`❯ parity --light --no-hardcoded-sync`

Synchronizes block headers for a light client with the Ethereum network using the Parity Light Protocol (PIP) starting at genesis:

2019-01-05 22:48:17  Running in experimental Light Client mode.
2019-01-05 22:48:17  Debug API is not available in light client mode.
2019-01-05 22:48:17  Public node URL: enode://4959d0273c8b7410ce46a47907c52c64c32cd4fca4ce3bfbb62c1f9d2d52a64d37d94b5a9e00e61fc218af482b6e04ea2c5d21480a0a079e15077020e4870590@192.168.1.110:30303
2019-01-05 22:48:27  Syncing       #0 0xd4e5…8fa3     0.0 hdr/s      0+    0 Qed   1/50 peers   0 bytes cache 0 bytes queue  RPC:  0 conn,    0 req/s,    0 µs
2019-01-05 22:48:37  Syncing    #3356 0x4f82…dbf7   335.6 hdr/s    223+    0 Qed   2/50 peers   2 MiB cache 172 KiB queue  RPC:  0 conn,    0 req/s,    0 µs
2019-01-05 22:48:42  Syncing    #5977 0x8339…9045   524.2 hdr/s    160+    0 Qed   2/50 peers   4 MiB cache 126 KiB queue  RPC:  0 conn,    0 req/s,    0 µs

Note, how it synchronizes headers starting at #0.

Full nodes

Full nodes have the full blockchain data available on disk and can serve the network with any data on request.

`❯ geth`

The default mode. Synchronizes a full node doing a fast synchronization by downloading the entire state database, requesting the headers first, and filling in block bodies and receipts afterward.

INFO [01-05|22:56:41.759] Loaded most recent local header          number=0 hash=d4e567…cb8fa3 td=17179869184 age=49y8mo3w
INFO [01-05|22:56:41.759] Loaded most recent local full block      number=0 hash=d4e567…cb8fa3 td=17179869184 age=49y8mo3w
INFO [01-05|22:56:41.759] Loaded most recent local fast block      number=0 hash=d4e567…cb8fa3 td=17179869184 age=49y8mo3w

Once the fast sync reached the best block of the Ethereum network, it switches to a full sync mode (see below).

`❯ geth --syncmode full`

Synchronizes a full node starting at genesis verifying all blocks and executing all transactions. This mode is a bit slower than the fast sync mode but comes with increased security.

`❯ parity`

The default mode. Synchronizes a full Ethereum node using warp synchronization mode by downloading a snapshot of the 30_000 best blocks and the latest state database.

2019-01-05 23:00:33  Syncing snapshot 0/2577        #0    4/25 peers   8 KiB chain 3 MiB db 0 bytes queue 11 KiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-01-05 23:00:43  Syncing snapshot 2/2577        #0    7/25 peers   8 KiB chain 3 MiB db 0 bytes queue 11 KiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-01-05 23:00:48  Syncing snapshot 3/2577        #0    9/25 peers   8 KiB chain 3 MiB db 0 bytes queue 11 KiB sync  RPC:  0 conn,    0 req/s,    0 µs

Once the snapshot is restored, the client switches to full sync and ancient blocks are synchronized from the network in background, denoted by yellow block numbers in the logs.

`❯ parity --no-warp`

Synchronizes a full node starting at genesis verifying all blocks and executing all transactions. This mode is a bit slower than the warp sync mode but comes with increased security.

Archive nodes

In addition to the chain data which is kept by all full nodes, it is possible to build an archive of historical states.

`❯ geth --syncmode full --gcmode archive`

Synchronizes an archive node starting at genesis, thoroughly verifying all blocks, executing all transactions, and writing all intermediate states to disk ("archive").

INFO [01-05|23:04:54.981] Imported new chain segment               blocks=2 txs=0 mgas=0.000 elapsed=741.138ms mgasps=0.000 number=2 hash=b495a1…4698c9 age=3y5mo3w  cache=0.00B
INFO [01-05|23:04:56.475] Imported new chain segment               blocks=4 txs=0 mgas=0.000 elapsed=27.599ms  mgasps=0.000 number=6 hash=1f1aed…6b326e age=3y5mo3w  cache=0.00B
INFO [01-05|23:05:03.616] Imported new chain segment               blocks=86 txs=0 mgas=0.000 elapsed=206.526ms mgasps=0.000 number=92 hash=c86dcb…f8b64c age=3y5mo3w  cache=0.00B

In Geth, this is called gcmode which refers to the concept of garbage collection. Setting it to archive basically turns it off.

`❯ parity --no-warp --pruning archive`

Synchronizes an archive node starting at genesis, thoroughly verifying all blocks, executing all transactions, and writing all intermediate states to disk ("archive").

2019-01-05 23:03:21  State DB configuration: archive
2019-01-05 23:03:21  Warning: Warp Sync is disabled because of non-default pruning mode.
2019-01-05 23:03:31  Syncing    #1689 0x3c91…2cea   169.80 blk/s    0.0 tx/s    0 Mgas/s   4019+    0 Qed     #5715    9/25 peers   2 MiB chain 51 KiB db 7 MiB queue 7 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-01-05 23:03:41  Syncing    #6422 0x750c…dc88   473.30 blk/s    0.0 tx/s    0 Mgas/s   6270+    0 Qed    #12700   12/25 peers   5 MiB chain 120 KiB db 10 MiB queue 12 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-01-05 23:03:51  Syncing   #11293 0x576c…7163   487.20 blk/s    0.0 tx/s    0 Mgas/s  13332+    0 Qed    #24638   19/25 peers   7 MiB chain 166 KiB db 20 MiB queue 8 MiB sync  RPC:  0 conn,    0 req/s,    0 µs

In Parity, this is called pruning which refers to the concept of state trie pruning. Setting it to archive basically turns it off.

Thanks for reading thus far. That's it basically.

The Görli Testnet Initiative Bounties Announcement

Afri Schoedon — Thu, 22 Nov 2018 18:48:32 +0000

Update: The bounties have been paid.

The Görli testnet initiative is a development campaign to create the first cross-client proof-of-authority network for Ethereum. There are various testnets available for experimenting with contracts and d-apps before going live on the public chain. However, there is no testnet available that is both widely usable across all clients, and robust enough to guarantee consistent availability and high reliability. This is what Görli tries to be. Read more on the motivation in the previously published Görli Testnet Proposal.

Since the ETHBerlin hackathon, where the idea was born and we were disqualified, we've come a long way, and the current test-testnet is already supported by four different clients, namely Geth, Pantheon, Nethermind, and Parity Ethereum. Even though the code is not final and not all of these clients are feature-complete yet, they are happily syncing the Görli chain.

A lot of members from the Ethereum community are convinced this project is badly needed and joined our Gitter to help running nodes, block explorers, and volunteering in many other ways. Look at this; people even Tweet about the project.

"These folks are neat!" --- Austin Griffith, hacker

"More than being just another testnet, Goerli is proving to be a great environment for collaborative development and cross-team communication and coordination." --- E.G. Galano, infrastructure

"Is it too early to say I’m a #Goerli testnet maximalist?" --- Maria Paula Fernandez, mvp

The feedback we received so far is encouraging, but it does not pay anyone's bills. Therefore, we are thrilled to announce that the Görli initiative received the first $DAI 52_200 grant from the ETC Cooperative and an anonymous donation of $DAI 10_000 (who's on the multi-sig?). The funds available now can be used to help to develop this project with all the tools and infrastructure required.

The roadmap ahead will include launching two testnets, a fully Constantinople-enabled Görli testnet for Geth, Parity Ethereum, Pantheon, and all the other Ethereum clients; and a Görli Classic testnet for Geth Classic, Parity Ethereum, and Mantis mimicking Ethereum Classic mainnet conditions.

The first batch of bounties will be announced today, sorted by priority for the cause:

A specification for the Clique POA engine as EIP-225/ECIP-1048
$DAI 5_000 claimed by @karalabe
A Clique engine written in Rust for Parity Ethereum #9981
$DAI 30_000 claimable by @thefallentree and @jwasinger (chunks A, B, Gitcoin)
Genesis and chain specs for Görli
$DAI 2_500 claimed by @5chdn
Genesis and chain specs for Kotti (Classic)
$DAI 2_500 claimed by @5chdn
Native support for Görli in Geth #18121
$DAI 2_500 claimed by @karalabe
Native support for Kotti (Classic) in Multi Geth #17
$DAI 2_500 claimed by @5chdn
Native support for Görli in Parity Clique #10
$DAI 2_500 claimed by @5chdn
Native support for Kotti (Classic) in Parity Clique #45
$DAI 2_500 claimed by @5chdn
Native support for Görli in Pantheon #370
$DAI 2_500 claimed by @Nashatyrev
Native support for Görli in Nethermind #234
$DAI 2_500 claimed by @tkstanczak
A one-way throttled bridge faucet from Ropsten, Kovan, and Rinkeby
$DAI 5_000 claimed by @ChainSafe

But why are some bounties already assigned to developers before this announcement was published, you might be wondering? It's simply because people were so excited about the Görli testnet initiative that they started working on it without asking for permission; without knowing we already planned to have these bounties available in future. This is the rare instance where we have more developers than funding available.

Hopefully, more bounties will be available and announced as soon as more grants are available. Our wishlist of projects and tools is sheer endless. Open tasks that are claimable will be available on Gitcoin once the Parity Ethereum Clique engine is merged upstream, which is the primary blocker for all other tasks, and the Görli testnet is ready to relaunch with network ID 0x5 to replace Ropsten.

The Görli Testnet Proposal - A Call for Participation.

Afri Schoedon — Thu, 13 Sep 2018 09:38:22 +0000

There are many Ethereum testnets available for experimenting with smart contracts and deploying decentralised applications before going live on the main Ethereum network. However, there is no testnet available that is both widely usable across all client implementations, and robust enough to guarantee consistent availability and high reliability.

Existing Public Testnets

Olympic used to be the chain "0", the public Ethereum testnet prior to the public Frontier launch of the Ethereum main network. The Ethereum genesis block was templated based on block #1028201 of the Olympic network. This network was retired soon after the Frontier launch.

Morden was introduced as the new public proof-of-work testnet which was launched with the Frontier release of the Ethereum public main network. The Ethereum community abandoned it due to issues with the Spurious Dragon testing efforts. However, it is still used by the Ethereum Classic community, also known as Morden Classic.

Ropsten was the last public proof-of-work testnet and as of today, the last testnet that is available across different client implementations. It's the network with specifications closest to the Ethereum main network. It is mostly abandoned due to stability issues but still kept alive as this is the only way to test contracts and dapps on both Geth and Parity Ethereum.

Kovan is the first proof-of-authority testnet launched as a response to the Ropsten spam attacks. A consortium of developers and service providers stepped up to utilise Parity's Aura proof-of-authority engine to run a testnet.

Rinkeby was the second proof-of-authority network, also launched in response to the Ropsten spam attacks. However, instead of Aura, it's using Geth's Clique proof-of-authority engine which differs in its specification.

Many other public Ethereum testnets with various specifications are available, i.e., the Sokol testnet, however, none of them are available across different clients while also being highly available and robust.

As of today, only Geth implemented Clique and only Parity Ethereum implemented Aura. To run a cross-client testnet, users have to rely on the Ropsten proof-of-work network. However, proof-of-work mining on networks bearing no value is hard to be incentivised and the resulting low hash rates enable potential attackers to take over the network easily.

The Görli Testnet Proposal

Now, that EIP-1011, the hybrid Casper proof-of-work/proof-of-stake testnet idea has been abandoned by latest Ethereum research, Görli could be the next-generation public Ethereum test network.

We hereby propose,

sufficiently specifying a proof-of-authority engine, such as Aura or Clique, in an EIP,
to implement one or multiple of these engines across different clients, such as Geth and Parity Ethereum,
and to bootstrap a new simplistic Görli proof-of-authority testnet based on the available implementations that mimics main network conditions.

Outlook and Roadmap

The ChainSafe team started working on an Aura port to Geth at the #ETHBerlin hackathon and suggests also porting Clique to Parity Ethereum. This is not a political proposal to favour one technology over another; it's rather the desire to enable developers with as many diverse tools as possible.

The Aura EIP is being drafted on Github. What's missing is details on the Aura chain-score rules, how the seals are composed, and how finality works. Currently, Aidan and myself are working on the EIP with the help of the Parity Ethereum team.

A basic Aura engine is prepared in this repository on Github. While the work on the integration of the engine and the testnet configuration is done, significant amounts of work still have to be done in the actual implementation of the Aura consensus mechanism. Currently, Elizabeth, David, Dustin, and Tim are working on this but would appreciate help from engineers more confident with writing consensus-critical code in the Go-Ethereum code-base.

Parity Ethereum supports Aura by default, a working chain-spec for a pre-testnet is available here. Two validator nodes and bootnodes are prepared to enable seamless integration with Geth nodes.

All hands on deck!

There's still a lot to be done. Here's how you can help:

Join the chat on Gitter: gitter.im/goerli/testnet
Run a node: github.com/goerli/testnet
Report to the dashboard: ethstats.goerli.ethberl.in
Help with the Aura spec: eip-aura.md, gitter.im/goerli/eip-aura
Help with the Go implementation: go-ethereum-aura/tree/aura-dev, gitter.im/goerli/geth-aura

Acknowledgements

Thanks to MP and the #ETHBerlin team for hosting us at the most amazing hackathon in the world.

Thanks to Tomasz, André, and Thibaut from Parity Ethereum helping with analysing the Authority Round specification.

Thanks to Martin from the Ethereum Foundation for helping us with the Golang implementation and debugging our code all night.

Thanks to Kirill and Parity Ethereum to provide us with a node Dashboard.

Thanks to Aidan and the ChainSafe team to kick this off.

The Ethereum-blockchain size will not exceed 1TB anytime soon.

Afri Schoedon — Wed, 29 Nov 2017 11:08:11 +0000

Before diving into this article, please read the two disclosures about my involvement (1,2) and the one on data accuracy (3) at the bottom of the article.

At least once a month someone posts a chart on r/ethereum predicting the blockchain size of Ethereum will soon exceed 1 TB. I want to take that chance to clean up with some stories around the Ethereum-blockchain size in this article and try to explain why this chart is technically correct, but not the full picture.

Let's have a look at this chart first. It shows the complete data directory size of an Ethereum node (red), Geth in this case, and a Bitcoin node (blue), probably Bitcoin-Core, plotted over time. While the Bitcoin graph is moving slightly upwards in a seemingly linear inclination, the Ethereum graph reminds the reader of an exponential growing slope.

On Blocks, Block-History, States, and State-History

Users accusing Ethereum of blockchain-bloat are not far off with their assumptions. But actually, not the chain is bloated but the Ethereum state. I want to examine some terminology from the Whitepaper before proceeding.

Block. A bundle of transactions which, after proper execution, update the state. Each transaction-bundling block gets a number, has some difficulty, and contains the most recent state.
State. The state is made up of all initialized Ethereum accounts. At the time of writing, there are around 12 million known accounts and contracts growing at a rate of roughly 100k new accounts per day.
Block-History. A chain of all historical blocks, starting at the genesis block up to the latest best block, also known as the blockchain.
State-History. The state of each historical block makes up the state history. I will get into the details on this later.

If this already bores you, now please, read on.

Understanding Pruning-Modes and Sync-Modes

Early 2016, the Go-Ethereum team introduced a so-called fast synchronization mode. Since then, it was pretty famous to run geth --fast, especially after the spam-attacks on Ethereum later the same year making a full synchronization mode painful. I'm writing these modes italic because I will come back to an essential disambiguation at a later point in this article. Just keep them in mind for now.

The Parity team (formerly Ethcore) reacted to the on-chain spam by offering a warp synchronization mode at the end of 2016 to ease the chain synchronization for new users. Much as the same as Geth's fast, parity --warp soon became the de-facto standard mode for users trying to synchronize the Ethereum chain. As of today, both these options are adapted as default in both clients.

But what does it mean to fast-sync versus full-sync a Geth node? What does it actually mean to warp-sync a Parity node rather than no-warp-syncing it?

A full Geth node processes the entire blockchain and replays all transactions that ever happened. A fast Geth node downloads all transaction receipts in parallel to all blocks, and the most recent state database. It switches to a full synchronization mode once done with that. Note, that this results not only in a fast sync but also in a pruned state-database because the historical states are not available for blocks smaller than best block minus 1024. That's not an issue, but before reading on, please keep in mind that Geth synchronization modes are also pruning modes.

Looking at Parity configuration options, this gets more complex. In addition to the previously mentioned synchronization modes, Parity also offers separate pruning modes, namely fast and archive... Right, Geth fast is a sync-mode, we learned, that even prunes, however, Parity fast is pruning mode not heavily coupled to the sync mode. At this point, I have to admit, the terminology is confusing, and I might have lost you already. Let's draw something with pen and paper.

Geth's fast enables a quicker synchronization and database pruning. Geth full disables both. Parity warp, however, can be disabled without disabling the state-trie pruning! This is a significant sentence. Thus I bolded it. And I am not comparing Ethereum clients here, that's not my intention at least. I want to show you that it is possible to run a full-verifying Ethereum node with a small database. Parity just provides the proof-of-concept for this.

But why is this? Because as long as you have all historic blocks on your disk, you can compute any historical state from it by reprocessing the entire chain again. But in most use-cases, you don't need historical states at all! Therefore it is smart just to delete outdated entries from the state history and to reduce your required disk space by 95%.

So, what's the minimum Size of a full-verified Node?

Some 10's of GB by just running parity --no-warp. Earlier this fall it was less than 20 GB, but the state is growing very fast. Currently, the raw historical block data containing the blocks and transactions is approximately 12-15GB in size and the latest state around 1-2GB.

But is this to be considered a full Ethereum node? Yes:

It runs a full blockchain synchronization starting at genesis.
It replays all transactions and executes all contracts.
It recomputes the state for each block.
It keeps all historical blocks on the disk.
It keeps the most recent states on the disk and prunes ancient states.

Something an Ethereum client never does is deleting old blocks. This is a significant difference between Bitcoin and Ethereum because pruning a Bitcoin node does not leave any choice but removing old blocks. With this context available, it's easier to understand why users often think a pruned Ethereum node is not a full node. But now, dear reader, you know the opposite is true. :)

And on top of this, even a warp-synced Parity node is downloading the whole history of blocks after the initial synchronization allowing it to serve the network as a full node once completed the ancient-block synchronization.

The Full Picture: 9 Parity Configurations compared

Below is a screenshot of my nicely-colored spreadsheet trying to distinguish between node-security of different Parity operation modes.

The configurations 00 through 05 are all to be considered full nodes. Configuration 06 is a default-configuration warp-node which can be regarded as full once the ancient block download is finished. However, it does not replay all transactions; it only checks the Proof-of-Work of the historical blocks.

The configuration 07 is something users often ask for but should be highly discouraged in production use. This setting is comparable to a pruned bitcoin node as historical blocks are partially not available. This is not a full node anymore. Note, how I added a separator above this paragraph. You get the idea.

Configuration 08 is a light client, but that's worth another blog article. Thanks for scrolling this far down, here is your conclusion: An Ethereum full node does not require more than 20-30 GB disk space by default. :)

Noteworthy disclosures and bottom-line comments.

(1) I work for Parity. I'm comparing different Parity configurations not only because I sincerely know and understand them, but also because Parity allows users to configure pruning mode and synchronization mode separately.

(2) I hold some Bitcoin and some Ether. I hope this does not have any influence on the technical aspects I'm outlining in this article. Also, I'm trying not to become overly political about this.

(2) I have been running Parity in 36 different configurations over six weeks to gather the numbers. This is time- and resource-consuming, and still, it bears the issue that I can not keep all configurations running at the same time, and therefore, the accuracy of the numbers presented in this article have to be consumed with caution. I expect the results to differ up to plus/minus 20% from other nodes running the same configuration. But you get the idea:

| ID | Pruning / DB Config | Verification    | Available History          | ETH        | ETC        | MSC        | EXP        | Parity CLI Options                         |
|====|=====================|=================|============================|============|============|============|============|============================================|
| 00 | archive +Fat +Trace | Full/No-Warp    | All Blocks + States        | 385     GB |  90     GB |  25     GB |   5.6   GB | --pruning archive --tracing on --fat-db on |
| 01 | archive +Trace      | Full/No-Warp    | All Blocks + States        | 334     GB |  90     GB |  21     GB |   5.8   GB | --pruning archive --tracing on             |
| 02 | archive             | Full/No-Warp    | All Blocks + States        | 326     GB |  91     GB |  30     GB |   5.5   GB | --pruning archive                          |
| 03 | fast +Fat +Trace    | Full/No-Warp    | All Blocks + Recent States |  37     GB |  13     GB |   3.5   GB |   1.3   GB | --tracing on --fat-db on                   |
| 04 | fast +Trace         | Full/No-Warp    | All Blocks + Recent States |  34     GB |  13     GB |   3.5   GB |   1.2   GB | --tracing on                               |
| 05 | fast                | Full/No-Warp    | All Blocks + Recent States |  26     GB |   9.7   GB |   3.0   GB |   1.1   GB | --no-warp                                  |
| 06 | fast +Warp          | PoW-Only/Warp   | All Blocks + Recent States |  25     GB |   9.6   GB |   2.6   GB |   0.96  GB |                                            |
| 07 | fast +Warp -Ancient | No-Ancient/Warp | Recent Blocks + States     |   5.3   GB |   2.9   GB |   0.19  GB |   0.13  GB | --no-ancient-blocks                        |
| 08 | light               | Headers/Light   | No Blocks + No State       |       5 MB |       3 MB |       4 MB |       5 MB | --light                                    |

Meta-data:

Version: Parity/v1.8.0-unstable-7940bf6ec-20170921/x86_64-linux-gnu/rustc1.19.0 from source w/ musicoin support
Ubuntu: 17.04 Kernel 4.10.0-35-generic / September 2017 / Lenovo Thinkpad X270, Core i7-7600U, 1TB SSD, 16GB RAM

Thanks for scrolling to the bottom. <3

Update: Thanks for featuring me on dev.to and twitter. Users who enjoyed reading this article, might also find the following reddit discussion interesting.

Fun fact: While publishing this article, the price of Bitcoin broke 10_000 USD and Ethereum 500 USD. I think I will add current market prices to my articles in future, just for fun.

Update: Thanks for rating this top-1 post, dear dev.to team <3 <3 <3

Update: Here is a more controversial discussion on Hackernews.

Solving the problem with Ethereum cross-client private networks

Afri Schoedon — Fri, 01 Sep 2017 13:05:34 +0000

This post was initially an answer on Ethereum Stack Exchange, but it was worth more exposure as I keep talking to people running into this problem again and again. And, it's a very time-consuming issue to get a Geth node talking to a custom Parity network, and often also vice-versa.

Cross-client private networks: The problem

Geth is a client written for Ethereum. It was one of the first official reference implementations and was never meant to run anything else but Ethereum. If you want to configure Geth for any other chain configuration, you would have to fork the source-code and create your own client implementation based on your desired rule-set. This happened for instance for Expanse (Gexp) and Ethereum Classic (Getc).

Parity, however, was created much later than Ethereum itself, by a team that was initially involved with the C++ Ethereum client (Eth). After they (Gavin Wood, et al.) founded Ethcore (now Parity Technologies), they created a client with a much broader vision; a client that is not supposed to only run Ethereum.

Parity allows more abstraction in the core and therefore, all you need to start a new chain is a so called Chain Specification file which describes the whole chain parameters, including consensus rules, validator engines, and so on. The most visible consequence is that Ethereum, Ethereum Classic, and Expanse (just to pick up the examples form above) do not need to maintain their own copy of the Parity source-code in order to support their project and their consensus rules. Parity just works out of the box with the --chain parameters foundation, classic, or expanse.

In addition, the --chain parameter allows configuring an entirely custom chain with pluggable consenus, custom transitions, and the validator engine you prefer.

In contrast, Geth only allows to specify a custom Genesis which is only one piece of the puzzle for a complete chain specification. And therefore you will have trouble to connect a Geth node to a custom network of Parity nodes. However, you basically have two options to run a custom cross-client network with both Geth and Parity nodes, explained below.

Best-practice: "Geth First"

Now, since you have less chain configuration options in Geth than in Parity, I'll call the most obvious idea "Geth First". You chose a network ID and create a custom Genesis, and initialize your new network with:

geth init geth-genesis.json

You can fire up a miner thread and a couple more Geth nodes at this point to verify the network is doing exactly what you need.

Now, you can start integrating Parity nodes. There is a tiny rust tool keorn/parity-spec that converts your custom Geth genesis file into a full Parity chain specification file.

cargo run -- geth-genesis.json

You can pass the output parity-spec.json directly to your Parity node with the --chain parameter.

parity --chain parity-spec.json

For discovery, you can use Geth's admin.addPeer() commands or Parity's --reserved-peers functionality.

However, both Geth and Parity are actively developed code-bases and the converter tool is prone to bugs and often users report generated chain-specification files are still leading to consensus issues. If this happens to you frequently, check out the other option below.

Best-practice: "Ropsten Revived - Revived"

The probably most stable and most obvious option to run a private network compatible with Geth, Parity, and all the other clients out there, is something I would call "Ropsten Revived - Revived".

Just take a known, working network that is supported by all clients, like the Ropsten public testnet, and slightly modify it. To ease this process, I created presets for both the Geth Genesis and the Parity Chain Specification at 5chdn/crossclient-chainsepc. It contains a geth.json and a parity.json mimicking a Ropsten revival with different network ID 1337 (0x539).

# Geth

geth init --datadir ~/.ethereum/crossclient geth.json
geth --datadir ~/.ethereum/crossclient --networkid 1337 --port 31333 --rpcport 8538

# Parity

parity --chain parity.json --port 31337 --jsonrpc-port 8539

The downside of this is probably the unsatisfied desire to add more complex modifications to your custom private development network. However, the predefined genesis and chain-spec offer a good starting point to get you bootstrapped with whatever network you have in mind.

AAWWWGS! Android Apps Which Work Without Google Services!

Afri Schoedon — Thu, 16 Mar 2017 21:50:05 +0000

I try to keep my device disconnected from Google by running a custom ROM without any Google services. Here is my experience and a list with apps which work great without Google.

Set and Setting

Device: LG Google (sic!) Nexus 5 (hammerhead)
System: LineageOS 14.1 (Android N 7.1.1)
Bootloader: factory-m4b30z (HHZ20h), latest, locked, and safe boot enabled
Recovery: factory-m4b30z, latest, and (un)fortunately dysfunctional in this setup
Miscellaneous: rooted with addonsu-arm from LineageOS

The Nexus 5 is a widely used smartphone from 2013. It has a huge community and one could say, a lot of fans. That are very fortunate circumstances, paired with the fact that the Google Nexus devices used to be very good in quality, security, and software support.

However, the official support ended in October 2016, and the last factory image was Android 6.0.1 in December 2016. Thus, I use a custom ROM and was always very satisfied with CyanogenMod (now LineageOS) both in terms of stability and functionality.

App store(s)

Primary app store: F-Droid
Secondary app store: Yalp

Due to the fact that I don't flash any Google services, I also end up without the Google Play app store. That's very inconvenient for most users, and I will not exclude myself here, because a liberated phone is nice but it's really annoying if you can not even use some of the standard apps you are usually used to.

Now, I use F-Droid, the open source Android app store. It contains basically everything you need to stuff your phone with apps for everything one could desire. But that's not enough for me, because I need some apps which are (unfortunately) only available in Google play with much better experience than the open source equivalents.

Thus, it happened that I use F-Droid for only one single application: The Yalp store! Yalp is a Google Play wrapper, which allows you to search apps, download and install them, and keep track of available updates... directly from Google Play! This is the end of problematic APK-mirror sites which do not offer any evidence that the downloaded Android Packages are full of mal-, spy- or phishware.

AAWWWGS! Here's the actual list:

Adobe, everything I tried so far
Firefox, of course
Lastpass
Periscope
Signal, new on that list, now works without GCM!
Syncthing
Telegram
Twitter, can't live without the official app
VLC

It's worth to maintain such a list because a lot of apps rely on Google Services which really shouldn't need to (e.g., Snapchat, Facebook, Instagram, Pinterest). Anything important missing? Please, let me know!

Hi, this is 5chdn

Afri Schoedon — Wed, 22 Feb 2017 20:25:30 +0000

I have been coding for 1337 years.

You can find me on Twitter as @5chdn

I live in Berlin.

I work for my family.

I mostly program in these languages: Ruby, Shell.

I am currently learning more about myself.

Nice to meet you.