Forem: marpme (Marvin)

Development Update on Verge #22

marpme (Marvin) — Sun, 02 Jun 2019 12:08:14 +0000

Did you think we were all going to vanish after announcing the codebase and partying up at the Verge Meetup? Think again! We're still here, with some fresh off the keyboard development update! Without further hesitation, let's get into it.

Codebase Release

On the 16th of May, my fellow teammate Marvin Piekarek pushed the latest release of our codebase, named Lumos. In case you missed it, you can read the press release here.

Without getting too technical, we found a bug when creating a stealth address and then encrypting the wallet, which would make that stealth address unusable. This critical bug was fixed a few days later when we pushed for the 5.1 release.

Currently, we are working on a newfound issue which causes the reindex process to crash when launching the daemon in -reindex mode.

You can follow the bug resolution directly on GitHub, but apart from this everything else is working great!

iOS Wallet

After the iOS wallet release, the workload hasn’t stopped. Our iOS team is currently working on… wait for it… Moon Mode!

No, it doesn’t shoot Verge’s price to the moon (sorry, your adoption will do that), but The "Swenator" is a king for naming things and thus he has named the dark mode, Moon Mode.

Swen van Zanten✨🔭

@swenvanzanten

Lol special designs for Moon Mode to get completely into style 🌘🛰️🌌🌠🚀☄️

21:16 PM - 29 May 2019

In the meantime, our great community has struck again, already providing translations for the wallet in 12 languages! Thank you very much for helping Verge being spread across every border!

Ivan made a great video showing how easy it is to switch between different languages, here it is:

Liquid error: internal

Android Wallet

While we don’t have shiny screenshots of the Android wallet, there has been a lot of code written to prepare for the integration of our Verge Wallet Service. Once this step is completed, VWS features will be used and we will start planning a public beta for testing the wallet. If we summarize that with a sentence, it’s the last step before being able to send and receive funds!

While this sounds great, there is no time frame for that, although Manuel stated at the meetup that he wishes to be done before the end of 2019 (for the beta).

As always, I’d like to take a minute to thank our amazing community that helps us grow every day. This means translators, contributors on GitHub, users, followers and also haters.

That’s all! Thanks for reading!

Crypto Currencies - What do you think about it?

marpme (Marvin) — Mon, 29 Oct 2018 10:25:51 +0000

Lately, I came across this thesis survey:

Liquid error: internal

Which made me think about:

What do you, as a developer and also non-developers, think about certain aspects of crypto currencies?
Have you even heard of it?
Do you think it's something that you will work on in your coming years?

Please let me know and definitely check out the survey to answer those questions made by AnnaRomeroSierr

Async, Await - Sorry what JavaScript?

marpme (Marvin) — Sun, 28 Oct 2018 15:53:32 +0000

Async & Await, a pair of words you might have heard of earlier in your JavaScript career, or maybe your teacher told you to utilize Async/Await to make your function a non-blocking function and you have literally no clue what this is about!? Well then let's dive into the first chapter of this article. We'll first focus on the aspect of having an async functions and then going deeper into the usage of await as it is more a syntactic sugar for us developers.

Async

Async, also called asynchronous, is a common technique to make your code non-blocking. But what does that mean? Let's explain it with a basic NodeJS example:

const fs = require('fs')

// 1. Example, the common sync'd code execute 
// logic: everything will stop here until it has finally loaded the file (imagine a file of 2GB)
const myFilesContent = fs.readFileSync('./path/to/file')

// 2. Example Async (callback based) 
// logic: Start loading the file, and "call" me back later when you are done
fs.readFile('./path/to/file', (error, buffer) => {
    // ah we got that file read into our memory and it's read for usage
});

// 2. Example Async (promised based) 
// logic: Start loading the file and after finishing loading, then please execute the code
new Promise((resolve, reject) => fs.readFile('./path/to/file', (error, buffer) => {
    // function without a return either reject if error are happening
    // or resolve the to resolving object
    // The promise will wait until either of those two function has been called!
    if(err){
        reject(err)
    }
    resolve(buffer)
})).then((fileBuffer) => {
    // thank you for providing me with your buffer.
}).catch(error => {
    // ahh, snap something wrong happened!
})

Okay, that looks quite simple - but:

Where's the async part over here?
Are there no indicators in Javascript for working with an asynchronous function?

Basically, yes there are, but they are rarely used as they were added within the latest standardization cycle. That is the reason why most of the projects you'll come or came across will probably either apply callback based or promised based executions for asynchronous operations.

Eventually, it's up to you to remember async structures like these, when you come across callbacks and/or promises.

Let us also look into the syntactic sugar - world, where we can use the keyword async to signal a function to act as an asynchronous function. And we will also look deeper into on what it actually does in the background. (Fun-Fact: async functions will just wrap your output into a new promise)

// async function way of doing asynchronous fetching
const fetchNewsletter = async options => {
  // fetch your newsletter and await its arrival
  // Caution: fetch is already a function which returns a promise
  const response = await fetch(
    'https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty',
    options
  );
  // check if the response is ok, like the normal way you would do it
  if (!response.ok) {
    throw new Error('Ups.');
  }
  // Caution: .json() returns a new Promised value!
  // the returned value can be everything and will be wrapped into a new promise
  return await response.json();
};

// the above shown function will look something like this:
const fetchNewsletterPromised = options =>
  fetch(
    'https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty',
    options
  ).then(response => {
    if (!response.ok) {
      throw new Error('Ups.');
    }

    return response.json();
  });

// syntactic sugar function
fetchNewsletter()
  .then(newsletter => {
    // have fun reading it later ;)
  })
  .then(error => {
    // ahhh snap!
  });

// non syntactic sugar function
fetchNewsletterPromised()
  .then(newsletter => {
    // have fun reading it later ;)
  })
  .then(error => {
    // ahhh snap!
  });

As a remark there, the initial code for the async function seems to be a bit longer than the promised one, but in my opinion it isn't a problem as it is comfortable to read and more transparent then having a chain of then's which can become really really disturbing and build more overhead, if you have more than one chain to resolve there.

And what does the await syntax within the first function do? This unpacks the Promised value of the fetch function we called there. A self-explanatory example right here:

const XMASParty = async promisedPresent => {
  const myRealPresent = await promisedPresent;
  const mood = await myRealPresent.isWishedOne
    ? enjoyPresent(myRealPresent)
    : enjoyItNeverTheLess(myRealPresent);

  return mood;
};

Never the less, I really support the trend we have here and I believe we are correcting most of the core concepts of JavaScript in a very great way. Although for newcomers it might be unusual to have nearly 3 patterns of arranging async code in JavaScript. Furthermore, they also cause a lot of pitfalls within the language as you have to understand promises in depth before heading into Async/Await stuff.

Basic Functional Programming with Javascript

marpme (Marvin) — Tue, 23 Oct 2018 12:55:09 +0000

Functional programming - hmm sounds interesting - but I have actually no clue about this topic? ... Well, then you are welcome to read this article to its full extent.

It will give you some basic knowledge about how functional programming works
Basic components of functional programming
What difference is to the commonly known object-oriented programming.

Let's get actually started!

Basic Functional programming knowledge

For this, we first have to look up how functional programming is defined. The explanation for this is actually quite simple:

"Functional programming (often abbreviated FP) is the process of building software by composing pure functions, avoiding shared state, mutable data, and side-effects." - Eric Elliott (Master the JavaScript Interview: What is Functional Programming?)

Well... yeah too many buzzwords! Let's quickly decompose this sentence by its important parts, afterward, we should come up with those:

pure functions

It basically follows the principle of having the same output for any given input at any time.

This behavior is also called determinism of a function. So a normal function does have its input(s) and calculates its output(s), but those haven't to be same at any time due to changes within the global variable or certain side effects.

A pure function tho will have the same requirement as a normal function, but with the difference that it isn't allowed to access any global/external state, so it's consistent with a given input there follows the same output. (like an one-to-one relationship)

avoiding shared state

No shared state between functions. So function A shouldn't know what's done in function B and also the way around. Also not by using a global state.

avoiding mutable data

Data should never be modified while working with them. In more common functional programming languages, values are memory copied and returned as a new object. For supporting such immutable data, javascript has the following functions to offer: Map, Reduce, RightReduce, filter ...

avoiding side-effects

Side effects are being introduced by, for example, relying on reading in a text file from somewhere. The File's content could be easily changed and can, therefore, cause huge differences (aka. side-effects) without you ever noticing it.

namespace studentExample {
  const costPerStudentPerMonth: number = 200;
  const months: number = 2;
  const students: Array<{ name: string }> = [
    {
      name: 'lulu'
    },
    {
      name: 'lala'
    },
    {
      name: 'lili'
    },
    {
      name: 'lele'
    }
  ];

  //normal function
  // + side effects, when changing any variable
  // + relying on a shared state
  const calculateStudentsCost = (students: Array<{ name: string }>) =>
    `${students.length * costPerStudentPerMonth * months} $`;

  // pure function
  // - side-effects, same input = same output!
  // - no shared state
  const pureCalculateStudentsCost = (studentsCount: number) => (
    monthsToCalculate: number
  ) => (costPerStudentPerMonth: number) =>
    `${studentsCount * costPerStudentPerMonth * monthsToCalculate} $`;

  console.log(calculateStudentsCost(students)); // 1600 $
  console.log(pureCalculateStudentsCost(students.length)(months)(costPerStudentPerMonth)); // 1600 $
}

What you may already notice is one of the biggest downsides by using functional programming is that everything gets a bit more verbose and maybe also harder to wrap your head around. Your brain is really well trained to understand certain concepts by using an object-oriented approach.

So for example, if someone would come up with a task to calculate the diameter and circumference of a circle most people would just go for the object-oriented approach as it seems quite easy to understand. First, you would create a circle class which gets all necessary class variables and also the needed methods to calculate the required numbers.

By experience, fewer people would actually go for a function as it feels less natural doing it this way, but what I've also noticed is, that mathematic functions are a good example of being projectable as a functional approach. Let's try it with the circle example from above:


namespace circleExample {

  // diameter = radius * 2
  const circleDiameter = (radius: number) => radius * 2;

  // Circumference = π × Diameter
  const circumferenceWithDiameter = (diameter: number) => Math.PI * diameter;

  // Circumference = π × radius × 2
  const circumferenceWithRadius = (radius: number) => Math.PI * radius * 2;

  // area = π × radius ** 2
  const areaWithRadius = (radius: number) => Math.PI * radius ** 2;

  // Example calculation
  console.log(areaWithRadius(4))
  console.log(circumferenceWithDiameter(circleDiameter(4)))
}

To me, this seems very practical and also quite readable as you can just read the functions together with their inputs. You get a sense of what's happening there without having a look into the code at all. Beside this certain project example, math is always a good example of showing how powerful functional programming can be.

Besides this, did you notice that we never change any variable within our programming examples, that's a nice effect of using functional programming.

In the end, I hope you could gain some knowledge from reading this and I also hope you keep this in mind when touching your personal or work-related projects again. It's only a very basic insight into the functional programming paradigm and there's so much more to explore about this. Feel free to leave me some feedback on this so we can see if we can have a deeper look into it. :)

Changes in Javascript's lifecycle [ES6,ES7,ES8]

marpme (Marvin) — Sun, 21 Oct 2018 10:12:03 +0000

First off, I would like to dig a bit deeper into the current situation on how Javascript is actually built upon and then we can dive into some critical changes that have to be applied at some time to maybe maintain the sustainability of your project or to have a more readable/easier to understand code pieces.

What is Javascript actually?

Well, to determine what Javascript we have to dig a bit deeper. Did you ever hear of ECMAScript? It's the standardized language behind the actual implementation which is represented by Javascript. So all changes we are talking about will already be available within the Standard of ECMAScript, but may not be available in Javascript as of now or better to say within your current project. See babeling your project as a possible solution for this.

ECMAScript is also the reason why most developer call for ES5 (ECMAScript 5)/ES6 (ECMAScript 6) support within their project.

It's basically the level of standardization of their projects. In comparison, you are actually able to stick your USB3 supporting stick in your USB2 slot, but it might lack some features of USB3, but the way around you can also stick your USB2 device into USB3 supported slot, but that might also lack some USB3 features. In computer science, we are naming this upwards- / downwards-compatibility.

These compatibilities are all over the place. And those are ensured by the ECMAScript Standard which is run by the TC39 committee. This committee is also deciding what should reach their final standard and what not.

ES6 vs ES7 (aka. 2016)

Array.indexOf vs Array.Includes

// ES6 way of searching for values in an array
let numbers = [1, 2, 3, 4];
if(numbers.indexOf(2) !== -1) {
  console.log('Array contains value');
}

// ES7 way of searching for values in an array
if(numbers.includes(2)) {
  console.log('Array contains value');
}

Exponentiation Operator

let base = 3;
let exponent = 4;

// old way Math.pow()
console.log(Math.pow(base ,exponent)) //81

// new way
console.log(base**exponent); //81

Destructuring

// Before:
var width = 200;
var height = 400;

// After:
let [width, height] = [200, 400];

// before: 
const calculateArea = (areaParameters) => areaParameters[0] * areaParameters[1]
calculateArea([200, 400]);

// or for better naming in older functions:
const calculateArea = ([width, height]) => width * height
calculateArea([200, 400]);

Default Parameters

// before:
const myFunction = (name) => name ? `I love ${name}` : 'I love marpme'

// now:
const myFunction = (name = "marpme") => `I love ${name}`

Spreading Arguments within functions

// before with an arraylike object, very difficult:
const sumAll = () => arguments.reduce((sum, next) => sum + next, 0)

// now, real array also more way clear
const sumAll = (...numbers) => numbers.reduce((sum, next) => sum + next, 0)

Array Concat

//before: 
const arrayOne = [1,2,3]
const arrayTwo = [4,5,6]
Array.prototype.concat(arrayOne, arrayTwo) // [1, 2, 3, 4, 5, 6]

// now shorthand:
const arrayOne = [1,2,3]
const arrayTwo = [4,5,6]
const new = [...arrayOne, ...arrayTwo] // [1, 2, 3, 4, 5, 6]

Object assigning/merging

//before: 
const objectOne = { love: true, number: 6 }
const objectTwo = { es7: false, number: 8 }
Object.assign({}, objectOne, objectTwo) // {love: true, number: 8, es7: false}

// now shorthand:
const objectOne = { love: true, number: 6 }
const objectTwo = { es7: false, number: 8 }
const new = {...objectOne, ...objectTwo} // {love: true, number: 8, es7: false}

Classes, Inheritance ...

see your old java classes, we won't need that in javascript. Keep it real and go for a basic functional approach and you are going to be good.

// Before:
var Person = function(name) {
 this.name = name; 
}

Person.prototype.sayHi = function() {
 console.log(`hi, I'm ${this.name}!`); 
}

// After - this produces exactly the same as above:
class Person {
  constructor(name) {
    this.name = name; 
  }

  sayHi() {
   console.log(`hi, I'm ${this.name}!`);
  }
}

// Let's test it:
var jack = new Person('Jack');
jack.sayHi();

ECMAScript 8 (ES8/2017)

Also fully supported by any of the Javascript implementations (V8-Chrome, NodeJS etc ...). It basically comes with the async/await pattern which removes the pain of handling promises in a 'chainy' manner.

Object.values / Object.entries, which returns you either an array of all object values or all object keys.

Support for having getOwnPropertyDescriptors(obj); on Objects. This will, in the end, allow us for having the possibility to build decorators around those objects.

Also, memory shared objects like SharedArrayBuffer, which can actually be used by many threads at the same time. And also a namespace for an object called Atomic with some static methods to handle such thread mechanisms.

Stealth Addressing in depth [1/3]

marpme (Marvin) — Sat, 20 Oct 2018 09:32:22 +0000

First, we will start with the normal procedure which will occur on a daily recurring basis. This is a simple user-case situation where two individuals want to transact privately, as is their right. To do this, we need to a stealth address which makes it possible for the receiver to receive his funds from the paying person. Let’s call the payer Bob. For today’s text, we will cover the part on how to create a stealth address, and what it is made of, and what is it being used for.

How to generate a stealth addresses?

First off you will need math, a lot of math. We basically need some crypto algorithms for this entire processes including SHA256, SECP256k1 (elliptic curve crypto with this equation y² = x³ + 7) and Base58 encoding.

The data structure of our stealth addresses:

 // [version] [options] [scan_key] [N] ... [Nsigs] [prefix_length]

The overall structure of a normal stealth address is the following. It’s based on the version of the current stealth address, which simply specifies for which currency it could be used. If the version were to change, the blockchain would reject your address. The option is a simple unsigned integer, which can specify any options. You can a scan public key and multiple spend public keys (where N = number of keys; NSigs = all the spend keys) to send the funds to. The prefix length is by default zero.

Verge’s current version of the stealth address is 0x28.

Step-by-Step creation of a verge based stealth address

Generate two private random numbers with a lenght of 32 bytes (aka. secure random numbers)
One will be declared as spend secret and the other as scan secret
each, scan and spend secret, will be used to generate a public key by utilizing SECP256k1 (length should be 33 bytes). Make sure to not overflow the SECP256k1 algorithm.
Now putting everything together into a single buffer in the abovementioned order.
Generate a hashsum of the current address buffer by double hashing this buffer with SHA256 and append the first 4 bytes of the checksum to the address buffer.
finally base58 encode your address If you have done everything right you address should start with smY.

Stealth addressing with JavaScript

const { randomBytes } = require('crypto');
const secp256k1 = require('secp256k1');
const SxAddress = require('./StealthAddress');

// generates a private key from a secure/random source
generatePrivateKey = () => {
  // generate privKey
  let privKey;
  do {
    privKey = randomBytes(32);
    // check if the seed is within the secp256k1 range
  } while (!secp256k1.privateKeyVerify(privKey));

  return privKey;
};

// generate a public key based on the current seeds
generatePublicKey = privKey => {
  // get the public key in a compressed format
  return secp256k1.publicKeyCreate(privKey);
};

const scanPriv = generatePrivateKey();
const spendPriv = generatePrivateKey();
// pass all the data into the stealth address class
const address = new SxAddress(
  scanPriv,
  generatePublicKey(scanPriv),
  spendPriv,
  generatePublicKey(spendPriv)
);

console.log(address.toJson());

After this procedure, we have assembled the basic requirements needed to create a stealth address and we have gathered all the inputs needed. Namely having two pairs of keys (public and private) representing the scan and spend key. Those are needed to verify the transaction and signing them in new blocks. We will go in depth about that one later. First, we will cover the basics of stealth addressing.

With that being said, let’s finally create an encoded address by having a look at the stealth address class:

const stealth_version_byte = 0x28;
const crypto = require('crypto');
const bs58 = require('bs58');

module.exports = class StealthAddress {
  constructor(scanPriv, scanPub, spendPriv, spendPub) {
    this.scanPub = scanPub;
    this.spendPub = spendPub;
    this.scanPriv = scanPriv;
    this.spendPriv = spendPriv;
    this.options = 0;
  }

  encode() {
    const address = new Buffer.from([
      stealth_version_byte,
      this.options,
      ...this.scanPub,
      1, // size of public keys
      ...this.spendPub,
      0, // size of signatures
      0, // ??
    ]);

    const result = Buffer.concat([address, this.generateChecksum(address)]);
    return bs58.encode(result);
  }

  generateChecksum(data) {
    return crypto
      .createHash('sha256')
      .update(
        crypto
          .createHash('sha256')
          .update(data)
          .digest()
      )
      .digest()
      .slice(0, 4);
  }

  validateChecksum(modules) {
    const buffered = new Buffer.from(modules);
    const checksumModule = buffered.slice(
      buffered.byteLength - 4,
      buffered.byteLength
    );

    const informationModules = buffered.slice(0, buffered.byteLength - 4);
    const informationChecksum = this.generateChecksum(informationModules);
    return {
      valid: Buffer.compare(informationChecksum, checksumModule) === 0,
      checksum: informationChecksum.toString('hex'),
    };
  }

  isStealth(bs58address) {
    const modules = bs58.decode(bs58address);
    let checks = this.validateChecksum(modules);
    if (!checks.valid) {
      return {
        valid: false,
      };
    }

    if (modules.length < 1 + 1 + 33 + 1 + 33 + 1 + 1 + 4) {
      return {
        valid: false,
      };
    }

    checks = { ...checks, length: modules.length };

    if (modules[0] !== stealth_version_byte) {
      return {
        valid: false,
      };
    }

    checks = {
      ...checks,
      stealthVersion: `0x${modules[0].toString('16')}`,
    };

    return checks;
  }

  toJsonPrivate() {
    return JSON.stringify(
      {
        scanPub: this.scanPub.toString('hex'),
        spendPub: this.spendPub.toString('hex'),
        scanPriv: this.scanPriv.toString('hex'),
        spendPriv: this.spendPriv.toString('hex'),
        options: this.options,
        address: this.encode(),
        isStealth: this.isStealth(this.encode()),
      },
      null,
      2
    );
  }

  toJson() {
    return JSON.stringify(
      {
        scanPub: this.scanPub.toString('hex'),
        spendPub: this.spendPub.toString('hex'),
        scanPriv: 'hidden',
        spendPriv: 'hidden',
        options: this.options,
        address: this.encode(),
        isStealth: this.isStealth(this.encode()),
      },
      null,
      2
    );
  }
};

Okay, that was a big chunk for a read-through, focus on the path along the encode() method, which combines all the important buffers to one simple address buffer.

In the end, we will be covered with a checksum which is based on the address buffer. This will then merge with that address buffer, and give the possibility to verify if the address has been manually modified or corruption due to network failures, etc.

… to be continued : )

All the details explained over are based on the original idea of this stealth addressing workflow:
// Copyright © 2014 The ShadowCoin developers
// Distributed under the MIT/X11 software license, see the accompanying