Forem: Stuart Harrison

Managing Javascript the easy way in Rails 7

Stuart Harrison — Mon, 25 Apr 2022 16:22:23 +0000

Rails 7 is here, and with it, comes a whole new way of managing Javscript with "UJS with import maps plus Turbo and Stimulus from Hotwire as the defaults".

I'm a pretty experienced dev, but I must admit, I struggled to wrestle with these concepts, and I had a very short greenfield project that I wanted to get shipped and out the door as quickly as possible.

I wanted an approach that mirrored what I was used to with Rails 6 and Webpacker, but while still using something that felt new, and wouldn't succumb to bitrot as soon as I shipped it.

After some background research and deliberation with colleagues, I eventually settled on JSbundling. This allows a similar approach to asset building to the rest of the Javascript world, but with the added magic from the asset pipeline we know and love.

First, add jsbundling-rails to the Gemfile:

gem "jsbundling-rails", "~> 1.0"

And run bundle install

Then run the generator (I'm using rollup, as it's marginally easier to grasp than Webpack, and is focussed wholly on building JS):

bin/rails javascript:install:rollup

This adds the following to your Rails app:

An app/assets/builds folder - This will contain your built JS, which will then be served via the asset pipeline
Add app/javascript/application.js file - This is the entrypoint for all your application's frontend Javascript
A bin/dev file - This should be how you run your application in development. It runs (and optionally installs if you don't have it) Foreman, so you can run your server and build Javsacript on the fly
A Procfile.dev file - This tells Foreman what processes to run when you run the bin/dev script
A rollup.config.js file - This tells Rollup how to build the Javascript in your application.js

In addition, it also adds some extra bits and bobs to your .gitignore (so you don't commit the built files to git), and adds the builds folder to the assets/manifest.js file. It also adds the necessary dependencies to your package.json, as well as an NPM command to build your JS.

When you start the server with bin/dev, your application will spin up as usual, whilst also watching your application.js adn rollup.config.js for any changes and building the result.

Some gotchas

This mainly worked for me off the bat, however, I had some teething problems.

Firstly, I wanted to use Babel to make sure my JS was compatible with older browsers (we still have to support IE11 as we work with Government clients). Adding this to me rollup.config.js seemed to work:

import resolve from "@rollup/plugin-node-resolve"
import { babel } from '@rollup/plugin-babel';

export default {
  input: "app/javascript/application.js",
  output: {
    file: "app/assets/builds/application.js",
    format: "es",
    inlineDynamicImports: true,
    sourcemap: true
  },
  plugins: [
    babel({
      babelHelpers: 'runtime',
      exclude: 'node_modules/**',
      presets: [
        [
          '@babel/preset-env', {
            'useBuiltIns': 'usage',
            'corejs': '3'
          }
        ]
      ],
      plugins: ['@babel/plugin-transform-runtime']
    }),
    resolve()
  ]
}

(I also had to install @babel/plugin-transform-runtime and @babel/runtime for this to work).

Secondly, I like to keep my JS nice and modular, with seperate files for each piece of functionality, and importing each file. For this to work, I had to set the rollup output format to iife(Immediately Invoked Function Expression), which wraps a function around all your code, so it runs as soon as the JS is loaded. Now, my rollup config looks something like this:

import resolve from '@rollup/plugin-node-resolve'
import commonjs from '@rollup/plugin-commonjs';

import { babel } from '@rollup/plugin-babel';

export default {
  input: 'app/javascript/application.js',
  output: {
    file: 'app/assets/builds/application.js',
    format: 'iife',
    inlineDynamicImports: true,
    sourcemap: true,
  },
  plugins: [
    commonjs(),
    babel({
      babelHelpers: 'runtime',
      exclude: 'node_modules/**',
      presets: [
        [
          '@babel/preset-env', {
            'useBuiltIns': 'usage',
            'corejs': '3'
          }
        ]
      ],
      plugins: ['@babel/plugin-transform-runtime']
    }),
    resolve(),
  ]
}

And that's it! Hopefully this will save you hours of painful googling, and prevent you from getting migraine from staring at the same config file for hours on end!

Data Migrations in NestJS

Stuart Harrison — Fri, 08 Apr 2022 10:50:55 +0000

(Header image by Patrick Coe on Flickr)

Coming to NestJS from a Ruby on Rails Background, there's a lot to like, and a lot that's familiar. There's also some stuff I miss, but I have a great time trying to replicate those missing features.

Take, for example, data migrations. When working on a live project, it's common that you'll want to change aspects of the data structure. A simple example might be if you have a books database with a single author.

Then, one day, your client informs you that some books have multiple authors, and has asked if you can make it possible for a book to have multiple authors.

For a non-technical client, this seems like a simple request, but for us devs, we know it's not that simple. We'll have to set up a new relation (probably through a join table) and transfer that data. Setting up the new relation is easy enough, but how do we move that data from the old relation to the old relation?

A naïve solution might be to make that change directly in the database migration, but, dear reader, here be dragons. Writing our data migrations in this way means that they stay around forever. If something in our codebase changes and a new developer runs the whole migration suite, this could cause the migrations to fail. The full pitfalls of this approach are outlined in this great blog post by Elle Meredith at Thoughtbot.

Another solution might be to add automated data migrations, that run in a similar way to database migrations, running at deploy time, and updating a migrations table to let the codebase know when a migration has run.

In the past, we've used data-migrate for this, but this comes with pitfalls. What happens if the deploy fails? We can often end up with strange errors and changes in state, so it's nicer to have a bit more control.

The approach I've taken to prefer recently is having small, short-lived scripts in the codebase that we run manually after deploying. In the example of changing our books table to have multiple authors, we would do the following:

Add a new authors relation
Write a data migration to move the author relation to the authors relation
Deploy the code
Run the migration in the production environment
Remove the old relation
Deploy that code

In Rails, we've added a little generator that generates a file in db/data_migrate that we can then run with rails runner. The beauty of rails runner is that any code that is run with that command has access to the whole of the Rails environment, so we have access to the database and the rest of the application.

In NestJS, we don't have the same luxury. However, I have found a neat solution that works just as well. For the uninitiated, the main entrypoint to a Nest app is main.ts. This is where the application is loaded, binded to Express and booted:

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(3000);
}
bootstrap();

We can steal this approach for our data migrations, which could then look like the following.

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function migrate() {
  const application = await NestFactory.createApplicationContext(AppModule);

  // Add migration code here

  await application.close();
  process.exit(0);
}
migrate();

Notice the use of NestFactory.createApplicationContext, this gives us a reference to the application, so we can fetch things like services from the application like so:

const application = await NestFactory.createApplicationContext(AppModule);

const booksService = application.get(BooksService);

In our books example above, we could then do something like this:

async function migrate() {
  const application = await NestFactory.createApplicationContext(AppModule);

  const bookService = application.get(BooksService);
  const books = await bookService.getAll();

  console.log(`Updating ${books.length} books`);

  for (const book of books) {
    book.authors = [book.author];
    bookService.save(book);
  }

  console.log('Done!');

  await application.close();
  process.exit(0);
}
migrate();

We can then save that file to a known location, and run it with ts-node:

ts-node --transpile-only path/to/your/migration.ts

(Here we're adding transpile-only to prevent type information from being produced, which has a lower memory footprint and prevents any weirdness in production)

So, this is all great, but copying and pasting boilerplate code is boring. How about we add a generator to generate our data migrations in a known place, with some guidance on what to do next? This could look something like this:

import * as fs from 'fs';
import { format } from 'date-fns';

async function generateDataMigration() {
  const timestamp = format(new Date(), 'yyyymmddHMS');
  const name = process.argv[2]
    .replace(/[A-Z]/g, (m) => '-' + m.toLowerCase())
    .substring(1);

  const filename = `${timestamp}-${name}`;
  const path = __dirname + `/../src/db/data/${filename}.ts`;

  const body = `// Run me with \`ts-node --transpile-only ./src/db/data/${filename}.ts\`
import { NestFactory } from '@nestjs/core';
import { AppModule } from '../../app.module';

async function migrate() {
  const application = await NestFactory.createApplicationContext(AppModule);

  // Add migration code here

  await application.close();
  process.exit(0);
}

migrate();`;

  fs.writeFile(path, body, { flag: 'a+' }, function (err) {
    if (err) {
      return console.error(err);
    }
    console.log(`Data migration ${filename} created!`);
  });
}

generateDataMigration();

This generates a boilerplate migration in a known place with a timestamp. We can then run it in development with (assuming your script is at ./utils/generate-data-migration.ts):

ts-node ./utils/generate-data-migration.ts

You could even add a data:migrate:generate to your package.json like so:

"data:migrate:generate": "ts-node ./util/generate-data-migration.ts",

And that's it! Now you can safely and carefully create data migrations on your project whenever you need to!

I'm still pretty new to Nest / Typescript, so please let me know in the comments if this is not the best approach, or share any alternative ideas. I'm all for learning.