Forem: Thomas Hunter II

Osgood Performance Gains

Thomas Hunter II — Wed, 31 Jul 2019 16:06:00 +0000

We recently performed several optimizations within the request/response path for the new JavaScript platform, Osgood, in an attempt to make it run faster. Osgood is a secure, fast, and simple platform for running JavaScript HTTP servers, and is distributed as a binary that can run on your server just like Node.js.

Optimizations

Some of these improvements were rather straightforward and are applicable to most codebases:

Remove unnecessary work (in our case, when returning a string response)
Lazily construct complex class instances (such as Headers)
Pass around references instead of performing a table lookup

Other optimizations depended on how V8 optimizes and runs JavaScript and wouldn't necessarily be faster in other situations:

Replace defineProperty() calls with private class fields
- Though private symbols turned out to be even faster
Use class when instantiating similar-shaped objects

Many of these optimizations are ones you wouldn't necessarily want to make within app code. However, with Osgood being a large-audience platform for running application code, it makes sense to optimize as much as possible and benefit a large number of applications.

Results

Using the wrk benchmarking tool we saw a 3.0x improvement—measured in requests per second (r/s)—when running a simple “Hello, World!” benchmark with 10 concurrent requests: osgood@0.1.0 runs at 25,261 r/s whereas osgood@0.2.1 runs at 77,450 r/s! (For reference, node@12.7.0, which also runs server-side JavaScript, runs at 31,159 r/s.)

As you can see, Osgood runs much faster as the concurrency increases. We built Osgood from the beginning with concurrency in mind so these results aren't that surprising. Under the hood, Osgood is making use of Tokio. From the Tokio homepage:

Applications built with Tokio are concurrent out of the box. Tokio provides a multithreaded, work-stealing, task scheduler tuned for async networking work loads.

Here are some raw numbers from these benchmarks which also show how response time standard deviation is an order of magnitude calmer as well:

$ wrk -d 60 -c 10 http://localhost:3000/hello # osgood 0.1.0
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.26ms    9.91ms 123.57ms   92.77%
    Req/Sec    12.69k     2.91k   16.98k    73.83%
Requests/sec:  25261.70

$ wrk -d 60 -c 10 http://localhost:3000/hello # osgood 0.2.1
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   140.86us  219.40us  15.27ms   97.41%
    Req/Sec    38.92k     2.30k   45.89k    71.38%
Requests/sec:  77449.91

$ wrk -d 60 -c 10 http://localhost:3000/hello # node v12.7.0
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   321.69us   96.95us  11.08ms   98.41%
    Req/Sec    15.66k     1.18k   17.50k    76.54%
Requests/sec:  31159.16

$ wrk --version
wrk 4.0.0 [epoll]

The code used for these benchmarks is available here.

What's Next

We're pretty happy with the performance gains we've been able to make. That said we have more plans to make it even faster. One such feature we're planning on implementing is to optionally auto-scale workers (a feature which gave a 2.5x improvement over the osgood@v0.1.0 release).

While the average latency of osgood@0.2.1 is less than half of Node.js, the max is still higher. This means there should still be some room to optimize garbage collection and get more consistent results.

As always, patches are welcome, and if you see an area to help performance we'd love to get a PR from you!

Wanna get your hands on this faster version of Osgood? Visit the releases page and download the latest version!

Osgood and CouchDB

Thomas Hunter II — Mon, 22 Jul 2019 18:53:38 +0000

We recently announced a new open source project, Osgood, which aims to be a secure platform for running JavaScript on the server. This platform applies the Principle of Least Privilege to application code. One of the ways we enforce this is by limiting the types of operations an application can perform. For example, arbitrary network connections cannot be made and child processes cannot be executed.

Outbound HTTP requests are a first class citizen thanks to the fetch() API. This means that CouchDB, a NoSQL database with an HTTP API, is a perfect match for performing application persistence with Osgood.

One of the biggest strengths of Osgood is the ability to specify policies on a per-route basis. This allows for very-fine security enforcement, allowing each Osgood Worker to only perform pre-approved operations.

Sample CRUD Application

Consider a simple CRUD application. This app represents a microservice within a larger organization. The service is essentially a facade in front of other services. It performs validation of the provided data, like enforcing username length. It limits database interactions, such as preventing arbitrary destructive queries from running. This app also decouples application code from the database implementation by transforming data into an ideal format. It also handles the database authentication, keeping the credentials on a trusted internal service and out of the client.

This microservice will have five endpoints:

List Users (GET /users)
Create User (POST /users)
Get User (GET /users/{user_id})
Delete User (DELETE /users/{user_id})
Update User (PUT /users/{user_id})

Note: If you'd like to follow along with this example, the code for the project is open source and available in our repository: IntrinsicLabs/osgood/examples/couchdb-rest

Application Configuration: `app.js`

Osgood Applications are configured using JavaScript. There is a global object called app available for setting properties. The first one is interface and is the name of the interface we want our application to bind to. The second one is port and is the port we want to listen on.

There are also some methods available on the app object for performing routing of incoming HTTP requests based on HTTP method and path patterns. For example, to route an incoming GET request to the /users endpoint, one can call app.get('/users', ...). The second argument to the routing functions is a path to the Osgood Worker file. The third argument is a function for configuring the route's policy.

Within the policy configuration functions we specify which URLs can be requested. These can be configured by calling methods like this: policy.outboundHttp.allowMETHOD(urlPattern). The urlPattern uses the glob syntax.

Note: This API is described in much more detail in our API Documentation.

This is what an Osgood Application file might look like for our CouchDB application:

app.interface = '0.0.0.0';
app.port = 8000;

app.get('/users', 'list.js', policy => {
  policy.outboundHttp.allowGet('http://localhost:5984/users/_all_docs');
});

app.get('/users/:user_id', 'view.js', policy => {
  policy.outboundHttp.allowGet('http://localhost:5984/users/*');
});

app.delete('/users/:user_id', 'delete.js', policy => {
  policy.outboundHttp.allowGet('http://localhost:5984/users/*');
  policy.outboundHttp.allowDelete('http://localhost:5984/users/*');
});

app.post('/users', 'create.js', policy => {
  policy.outboundHttp.allowPost('http://localhost:5984/users');
});

app.put('/users/:user_id', 'update.js', policy => {
  policy.outboundHttp.allowPut('http://localhost:5984/users/*');
});

We've now described all of the capabilities and have fully configured our application within a single file. With this configuration our application would not be able to, say, send an HTTP request to http://evil.co, nor would the GET /users route be able to perform a DELETE operation against the users collection in CouchDB.

Describing the capabilities up front is beneficial for two reasons. The straightforward reason is that it is secure. A side-effect is that application code is now much easier to audit. Imagine how quick those tedious GDPR audits could be if you had this list of I/O available for all your other apps.

Create User Worker: `create.js`

Our application has five operations that it can perform. In this post we'll only look at one of them: the creation of users (if you'd like to see the other examples checkout the sample application on GitHub).

This route will accept an incoming POST request, convert the body into JSON, perform some minimal validation, then pass the data to CouchDB (along with auth credentials). It'll then relay information to the client based on whether or not the operation succeeds.

const AUTH = `Basic ${btoa('osgood_admin:hunter12')}`;

export default async (request) => {
  try {
    var user = await request.json();
  } catch (e) {
    return json({"error": "CANNOT_PARSE_REQUEST"}, 400);
  }

  if (user.id || user._id) {
    return json({"error": "CANNOT_OVERRIDE_ID"}, 400);
  }

  if (!user.username || typeof user.username !== 'string'
      || user.username.length < 3 || user.username.length > 20) {
    return json({"error": "USERNAME_INVALID"}, 400);
  }

  const payload = await fetch(`http://localhost:5984/users`, {
    method: 'POST',
    headers: {
      Authorization: AUTH,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify(user)
  });

  const obj = await payload.json();

  if (obj.error) {
    return json({"error": "UNABLE_TO_INSERT"}, 500);
  }

  return json({ok: true});
}

function json(obj, status = 200) {
  const headers = new Headers({
    'Content-Type': 'application/json'
  });

  const body = JSON.stringify(obj);

  const response = new Response(body, { headers, status });

  return response;
}

If you've ever worked with Service Workers, Lambda Functions, or Express.js controllers then this code might look familiar. The file exports a single default function which accepts request and context arguments. The request argument is an instance of the Request object available in modern browsers. The context argument has some additional niceties that we don't need for this particular example. The function itself can be an async function or otherwise return a promise. If the promise rejects Osgood will respond to the client with a 500 error. If it resolves a string or a simple object then Osgood will reply with a 200 and an appropriate Content Type. But, for fine grained control, a Response object can be returned which allows for manually setting the HTTP status code and other headers.

Running Osgood

To run Osgood, first download a release for your platform. Once that's done extract the osgood binary somewhere, ideally in your $PATH.

Then, download the six files for this project (app.js, list.js, create.js, delete.js, update.js, view.js). Finally, run this command:

$ osgood app.js

This will start the Osgood Application and route requests to the five Osgood Workers. Of course, the service won't be too useful without a CouchDB instance to talk to. The following commands will run CouchDB in a Docker container:

$ docker run \
  -e COUCHDB_USER=osgood_admin \
  -e COUCHDB_PASSWORD=hunter12 \
  -p 5984:5984 \
  --name osgood-couch \
  -d couchdb
$ curl \
  -X PUT \
  http://localhost:5984/users

After that we're ready to interact with the application. The next command will send a POST request to the Osgood Application and create our first user:

$ curl \
  -X POST \
  http://localhost:8000/users \
  -d '{"username": "osgood"}' \
  -H "Content-Type: application/json"

More Information

Osgood is open source. It's written in Rust and runs JavaScript using the speedy V8 engine.

The sourcecode is hosted on GitHub and is available at IntrinsicLabs/osgood. Pull Requests welcome!

Hosting a Static Site and Contact Form with Osgood

Thomas Hunter II — Wed, 17 Jul 2019 17:51:06 +0000

Our newly released project, Osgood, is a tool for building HTTP applications that run JavaScript on the server. It has an emphasis on security and by default enforces isolation between the different routes.

One of the features we added is the ability to serve static content. In this post we'll show how easy it is to build a website which is almost entirely static but does require some server-side logic, in this case a contact form.

This application will make use of Mailgun to send email. Note that we're using the demo Mailgun API key and a demo URL—if you want to adapt this application for your own use you'll need to create an account and use real credentials.

Application File (app.js)

This file handles global configuration of Osgood, how to route requests, and the policies of the different routes. Every application requires an application file.

#!/usr/bin/env osgood

app.port = 3000;

app.route('POST', '/contact', 'contact.js', policy => {
  // Mailgun demo URL
  policy.outboundHttp.allowPost('https://api.mailgun.net/v3/samples.mailgun.org/messages');
});

// GET requests beginning with / will map to the ./static directory
app.static('/', './static');

This file specifies two different routes. The first one is rather specific and captures POST requests made to the /contact endpoint. When these requests are made it will trigger the contact.js worker file. Based on the policy configuration that route will only be allowed to make a POST request to the specified Mailgun endpoint. Any other attempts at making I/O requests, such as making a GET request to the GitHub API, would fail.

The other route is for static content. Any other request (i.e. GET requests beginning with /, aka not POST /contact) will translate into a lookup in the ./static directory. Osgood defaults to serving a file named index.html when a request to a directory has been made.

Worker File (contact.js)

This worker file will handle the only route defined by our application which isn't related to serving static content.

Osgood workers work by exporting a default function, usually with the async keyword. The value returned from this function will be used to determine the response for this particular HTTP request.

This function can essentially be thought of as a controller for your application. Within this function we perform a lot of validation logic. Once we're satisfied we generate an instance of the FormData class. Once the instance has the necessary values attached we can send the data along to Mailgun by making a fetch request. (The APIs available should look pretty familiar if you're used to writing JavaScript that runs in a web browser.)

Finally, once the request has been sent, we notify the user that their email has been successfully delivered.

export default async (request) => {
  try {
    var req = await request.json();
  } catch (e) {
    return json({error: 'CANNOT_PARSE', message: "Invalid JSON Payload Provided"}, 400);
  }

  const email = new FormData();

  if (!('email' in req) || !('name' in req) || !('message' in req)) {
    return json({error: 'MISSING_FIELDS', message: "Invalid JSON Payload Provided"}, 400);
  } else if (typeof req.email !== 'string' || typeof req.name !== 'string' || typeof req.message !== 'string') {
    return json({error: 'INVALID_FIELD_TYPES', message: "Invalid JSON Payload Provided"}, 400);
  } else if (!req.name || !req.email || !req.message) {
    return json({error: 'EMPTY_VALUE', message: "Please supply all fields"}, 400);
  }

  email.append('from', `${req.name} <${req.email}>`);
  email.append('to', 'spam@intrinsic.com');
  email.append('subject', "Contact form email");
  email.append('text', req.message);

  try {
    // Mailgun demo URL
    await fetch('https://api.mailgun.net/v3/samples.mailgun.org/messages', {
      method: 'POST',
      headers: new Headers({
        // Mailgun demo key
        Authorization: 'Basic ' + btoa('api:key-3ax6xnjp29jd6fds4gc373sgvjxteol0')
      }),
      body: email
    });
  } catch (e) {
    return json({error: 'CANNOT_SEND', message: "Cannot parse provided JSON"}, 500);
  }

  return json({success: true, message: "Email has been sent"});
}

// helper for setting status code
function json(obj, status = 200) {
  const headers = new Headers({
    'Content-Type': 'application/json'
  });

  const body = JSON.stringify(obj);

  const response = new Response(body, { headers, status });

  return response;
}

Running the Project

Our project has the following hierarchy of files to support this application:

static/
- scripts/
- contact.js
- styles/
- main.css
- index.html
app.js
contact.js

We then run the application by executing the following command:

$ osgood app.js

After that the application can be seen by visiting http://localhost:3000.

The files used in this example can be downloaded here: osgood/examples/contact

You'll also need to install a recent Osgood Release to run the example.

Checkout the next post in this series, Osgood and CouchDB.

Introducing Osgood

Thomas Hunter II — Mon, 15 Jul 2019 18:22:52 +0000

Services written today share a common flaw: Being over-privileged. Node.js applications, for example, are capable of executing child processes, sending network requests, writing to the filesystem, and sending signals to other processes. A typical application requires a small subset of these features, and as malicious modules gain in popularity, the risk of these often-unnecessary features being abused only increases.

At Intrinsic, we've spent years building a product which applies the Principle of Least Privilege to Node.js. Our core product allows our customers to provide a list of policies describing what the application is capable of doing. This includes things like which binaries can be executed, how the filesystem can be interacted with, and what URLs the application can request. If an I/O operation hasn't been whitelisted, then it fails.

Recently we asked ourselves: if a new platform was designed for building middleware / microservices, one which followed the Principle of Least Privilege, and provided enough functionality to securely replace the most common microservice use-cases, what would such a system look like?

Osgood became our attempt to build such a platform. However, we didn't want such a tool to be completely unfamiliar to developers. So, we reached for a pile of technology already familiar to many.

The platform is built using Rust, a language heralded for its safety, and JavaScript is run in V8, a ridiculously fast JavaScript engine.

How does it work?

Osgood is available as a statically-linked binary that can be downloaded for Linux and MacOS. The binary can then be called with a path to a JavaScript application defining high level configuration, how to route incoming requests to different Osgood workers, and the security policies required by each worker.

The binary can easily be run on a laptop for doing local development. Once the application is ready, the code, as well as the Osgood binary, can be uploaded to a production server.

Osgood is generally useful in situations where logic needs to be performed on a server, or some sort of secret needs to be kept from a client, or when the only required outbound I/O is via HTTP requests.

Osgood provides the necessary APIs to intuitively build CRUD applications backed with technologies like CouchDB or Elasticsearch. Clients such as mobile apps and web browsers typically shouldn't be given unfettered access to a database, both for security purposes as well as preventing tight coupling. Using Osgood to maintain database credentials is a more secure approach, and transforming data into a common format helps avoid vendor lock-in.

Another use-case is providing an HTTP facade in front of existing backend API services. For example, if a mobile application wants to access data from two internal services, Osgood can make the two calls on behalf of the client and transform the resulting data. This also makes Osgood feasible as a GraphQL API.

Guiding Principles

A few principles helped guide us while designing Osgood. Since we're a security company it's very important that the platform be secure. Nobody is going to want to use Osgood if it's slower than other technologies, so it needs to be fast. And finally, we wanted to bring the Principle of Least Privilege into the hands of more programmers. This required us to make such an implementation extremely simple.

Osgood Applications are Secure

Policies are defined within Osgood Application files using JavaScript functions. These functions use the same syntax as the Intrinsic for Node.js HTTP policies.

Here's an example of a policy file which is able to interact with a few select GitHub APIs, as well as a CouchDB database:

// app.js

// global configuration
app.interface = '0.0.0.0';
app.port = 3000;

app.get('/user/:username', './worker.js', (policy) => {
  policy.outboundHttp.allowGet('https://api.github.com/users/*/gists');
  policy.outboundHttp.allowGet('https://api.github.com/users/*/repos');

  policy.outboundHttp.allowGet('http://couchdb.local:5984/users/*');
  policy.outboundHttp.allowPut('http://couchdb.local:5984/users/*');
  policy.outboundHttp.allowPost('http://couchdb.local:5984/users');
  policy.outboundHttp.allowDelete('http://couchdb.local:5984/users/*');
});

Many backend databases expose functionality via HTTP-based APIs—think CouchDB and Elasticsearch. Many third-party services expose their APIs via HTTP as well—such as GitHub and Stripe. Needless to say a lot of these middle-layer microservices can be built by exclusively communicating via HTTP.

Osgood is Fast

With a simple Hello, World! benchmark, Osgood is able to serve around 40k requests per second.

$ wrk -c 100 -d 60 http://localhost:3000/hello
Running 1m test @ http://localhost:3000/hello
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.61ms   13.36ms 219.02ms   97.42%
    Req/Sec    20.36k     3.17k   25.04k    91.00%
  2422992 requests in 1.00m, 265.74MB read
Requests/sec:  40360.32
Transfer/sec:      4.43MB

Osgood is Simple

Osgood is available as a single statically-linked binary about 20MB in size. The only required argument for executing the binary is a single JavaScript file.

Let's take a look at a sample Osgood Application. This application represents a common task amongst microservices. The application accepts an input value in the request, makes multiple outbound HTTP requests using the value, transforms the resulting data in some manner, and responds with the combined dataset. This is the API facade pattern and is frequently used to reduce requests made by a client.

// worker.js

export default async (_request, context) => {
  const username = context.params.username;

  const [gists_res, repos_res] = await Promise.all([
    fetch(`https://api.github.com/users/${username}/gists`),
    fetch(`https://api.github.com/users/${username}/repos`),
  ]);

  const [gists, repos] = await Promise.all([
    gists_res.json(),
    repos_res.json(),
  ]);

  return { username, gists, repos };
}

Once you download a release you can then run this Osgood Application using the following command:

$ osgood app.js

In our next post we'll look at Hosting a Static Site and Contact Form with Osgood.