Forem: Can Olcer

How to deploy your Jekyll site on Scalingo

Can Olcer — Fri, 04 Nov 2022 18:48:46 +0000

Jekyll is a static site generator based on Ruby. Scalingo is a hosting provider (similar to Heroku, Platform-as-a-Service). One of the great things about Scalingo is that they are an European company with a data center in France.

At Ravioli, we're moving all our apps to EU-based hosting providers to ensure the privacy of customers. This is necessary because the USA has very lax privacy laws, and it's easy for the US government to access customer data that is residing on data centers owned or operated by US legal entities. So, even if you host your app on AWS's Frankfurt server, you compromise your users' privacy.

If you're still reading this after my privacy detour, thanks! Let's get to the juicy part.

Scalingo doesn't offer a native way to serve static applications such as Jekyll or Hugo. Rather, they recommend that you run an Express server that serves the static assets.

Sounds easy enough. Why do we need a wordy blog post for this?

Because there are two gotchas which make deploying your Jekyll site not that straight-forward. The good news is that I have the solution right here for you.

Like Heroku, Scalingo works with buildpacks. Buildpacks tell Scalingo which environment to install before deploying and running your app. We are basically running an Express app that just serves the static files generated by Jekyll. Therefore, we need to choose the nodejs buildpack.

When you do that, you'll quickly realize that your build fails. The reason is simple: The nodejs build pack doesn't include Ruby, but before deploying we need Ruby to generate the static files for our website. The Jekyll command for that is bundle exec jekyll build.

Of course, we could run the command before and check in the static build output (usually the _site folder) to our version control. But this would be a major pain in the ass.

We need a buildpack that somehow contains Node and Ruby. It's our lucky day! Scalingo offers Multi Buildpacks. To combine multiple build packs, add a file called .buildpacks to your root with this content:

https://github.com/Scalingo/ruby-buildpack
https://github.com/Scalingo/nodejs-buildpack.git

The order is important. The nodejs buildpack needs to come last, because we're deploying a Node application.

Our package.json should look like this:

{
  "scripts": {
    "start": "node server.js",
    "build": "bundle exec jekyll build"
  },
  "dependencies": {
    "express": "^4.18.2"
  }
}

The nodejs buildpack runs the build task before deploying. That's where we build our Jekyll static files. And then it uses the start task to run the Express server.

Speaking of Express, our second gotcha comes to mind. In their documentation, Scalingo suggests this setup for server.js:

var express = require('express');

var app = express();
var directory = '/' + (process.env.STATIC_DIR || 'dist')
app.use(express.static(__dirname + directory));

var port = process.env.PORT || 3000;
app.listen(port, function () {
  console.log('Listening on', port);
});

With Jekyll, this works ok for the index page, but not for any other page. That's because it expects the .html extension. So, /about won't work, but /about.html will. But that's a bit too 1995 for me. Fortunately, a simple modification to server.js fixes this problem:

app.use(express.static(__dirname + directory), { extensions:['html'] });

That's it. Using an Express server and leveraging Scalingo's Multi Buildpacks, you should be able to deploy any static website on Scalingo.

This static website that you're reading is generated with Hugo and ironically still runs on Netlify. I'll make sure to migrate it over to Scalingo one of these days.

A better CSV import

Can Olcer — Sat, 07 May 2022 10:48:41 +0000

If you ever had to import data from a CSV file in a web app, there's a high chance that your experience wasn't great. You need to prepare a file with the required columns, export to .csv and make sure to get the delimiter right. (Ironically, even though CSV stands for comma separated values, the delimiter can be a different character like a semicolon or tab). The fun doesn't stop there. If there are any errors in your data, such as wrong formatting or missing values, the app will tell you to fix it before continuing. Now, you need to go back to Excel or Google Sheets and find the erroneous cells. Rinse and repeat.

When building the bulk location import feature for Mapzy, our open-source and self-hostable store finder, we wanted to provide a better import experience for our users. We wanted to spare them the pain of dealing with exporting files and choosing delimiters, and we wanted to make it simple for them to fix any import errors.

In an innovation that can only be likened to the invention of the first iPhone, we came up with the following: In the import screen, we provide an embedded spreadsheet that already has the correct headers filled out. Instead of choosing a file from their computer, users just copy and paste their CSV data into this spreadsheet and press the big red button. If there are any errors, we simply highlight the relevant cells in the spreadsheet. This way, users can easily fix them right then and there without needing to fiddle with a separate file and go through the whole process again.

Ladies and gentlemen, without further ado, I present to you - the Mapzy location importer:

Your browser does not support the video tag.

If there are errors, we handle them like this:

Your browser does not support the video tag.

For those who are interested in the technical background, more details follow.

For the spreadsheet, we use the excellent Jspreadsheet library. If there are no errors, the import happens in two steps because importing a lot of locations requires a lot of calls to the Mapbox geocoding API (geocoding is converting an address to longitude and latitude), which is rate limited.

In the first step, we only validate if the information is all there and correctly formatted. For example, if a location name is missing, we highlight that error in the spreadsheet. If everything looks good, we save the locations to the database using the awesome Activerecord-Import library, but skip calling the geocoding API. Rather, we mark these locations as "to be geocoded".

In the second step, we then start a background job (with Sidekiq and Redis) that goes through the database and checks all locations that are related to the given map and geocodes their addresses if necessary. If a geocoding attempt is not successful, we later ask the user to correct the address from their Mapzy dashboard.

If you're interested in the code, have a look at location_imports_controller.rb, location_import.rb, and batch_geocode_worker.rb.

An added technical bonus of this approach is that we don't need to deal with mapping columns from a CSV file to our data models, which can be a whole can of worms in itself.

Fellow devs: Stop building on the blockchain

Can Olcer — Mon, 07 Mar 2022 17:22:07 +0000

This is an open letter to all my fellow software developers, indie hackers and entrepreneurs: Stop building things on blockchain technologies.

This is a complex topic, and I will only touch on a few topics. If you remember one thing, let it be this: The growing popularity of cryptocurrencies makes the rich richer, promotes ultra-libertarian views and enables scams and ransomware. It does not lead to decentralization and empowerment of the little man.

By building on the blockchain, you are not a revolutionary for a better, more equitable world. You are merely a pawn in this selfish game of chess of the rich.

Why does it make the rich richer?

Cryptos are increasingly used by venture capitalists to create returns quicker than ever before. Simply said, they invest in new crypto startups that then list their crypto on a “reputable” exchange like Coinbase (which is of course also owned by VCs). Then, they create demand for the crypto and sell their initial holdings extremely high. After a while, the gullible fools who thought they are going to the moon realize that they were left holding the bag.
There is an interesting analysis by Fais Kahn where he looks at the data and concludes exactly this.

By building and promoting blockchain, you become part of this hype cycle. Even if you personally don’t pump and dump or have bad intentions, every bit of activity in this space ultimately benefits the Mr. Andreessen and his buddies.

Why blockchain doesn’t lead to decentralization

Decentralization is just a pipe dream sold to you by institutions trying to enrich themselves. Blockchain does not lead to decentralization. It's obvious if you look at major cryptos: a small number of people hold a large amount of the crypto wealth, and a few mining pools control the majority of the mining activity.

Why is this happening? The answer is economies of scale. Nvidia lead engineer David Rosenthal makes this point in his 2014 blog post Economies of Scale in Peer-to-Peer Networks. If you have some time, go have a read. But the main point is this:

Since then I've become convinced that this problem is indeed fundamental. The simplistic version of the problem is this:

The income to a participant in a P2P network of this kind should be linear in their contribution of resources to the network.

The costs a participant incurs by contributing resources to the network will be less than linear in their resource contribution, because of the economies of scale.

Thus the proportional profit margin a participant obtains will increase with increasing resource contribution.

Thus the effects described in Brian Arthur's Increasing Returns and Path Dependence in the Economy will apply, and the network will be dominated by a few, perhaps just one, large participant.

And this is the same exact principle that applies to blockchain, too. Not only to blockchain, by the way. The internet is built on decentralized protocols like SMTP and HTTP. However, most people use Gmail for their email and most web services are hosted on AWS, Microsoft Azure or Google Cloud.

But don't we need all this experimentation to find a game-changing use case for blockchain?

There will be no real use cases for blockchain. It’s true that it sometimes take a while until interesting use cases emerge for new technologies. But, this technology has been around as long as the iPhone, and look at the use cases that emerged for the iPhone. In fact, the iPhone (like the internet) had real use cases from the get go. Technologies may not be widespread or cost-effective at the beginning, but if it's a good technology, it already provides a real value add to a small group of people in its early days.

Conclusion

All that blockchain is, is an intriguing intellectual exercise, that proved to be not viable in the real world. It does not solve the trust problem in a practical way.

All projects and apps that build on blockchains, be it DeFi or NFT, keep promoting cryptocurrencies, making the initial creators richer and the people that "get in" later are left holding the bag.

I'll say it again to make it stick: By building and promoting apps and projects based on the blockchain, you become a pawn in this selfish game of chess of the rich. I’m not denying that there might be a few blockchain projects that do real good with the money raised, but they are the absolute minority and do so by promoting an incredibly destructive technology. The end does not justify the means.

My plea is: Don’t do it. Don’t be tempted by the devil to join this get-rich-quick scheme. Go back to building products and services that are not zero-sum games but add real value to the lives of people without fucking everything else up.

Recommended further reading:

Active Record Callbacks shouldn't modify other records

Can Olcer — Sat, 18 Dec 2021 10:42:46 +0000

In Ruby on Rails, After Record Callbacks allow you to hook into the lifecycle of Active Record objects. Commonly used callbacks are after_create or before_validation. In this post, I want to highlight a bad practice that is sometimes used in callbacks.

Let's say we have a User model that has an associated Account. The idea is that every user also has an account, which we use to specifiy more details like billing information.

Now, since every time we create a new User, we also want to create a new Account, we might be tempted to do something like this in our User model:

class User < ApplicationRecord
  has_one :account
  after_create :create_account

  def create_account
    Account.create(user: self)
  end
end

This is almost never a good idea. Creating or updating one model should never touch another model.

The main reason is that it tangles up concerns (remember: we like separation of concerns) and is not very explicit. A colleague working with you on this codebase might not expect that this side effect will happen. After all, in other places in your code where you create your user, you only do User.create and it magically also creates an Account for this user.

Another reason is that, while most of the time you want to create an associated Account, maybe there will be times where you don't.

A further reason is testing. If you use factories to set up your test data, creating a User factory instance will not automatically create an Account factory instance, but it will create an Account record in the test database. Things will get messy from there.

Instead of using a callback, I recommend just creating the Account separately or wrapping these two commands into a method that lives in the User model, like this:

class User < ApplicationRecord
  has_one :account

  def self.create_with_account(attributes)
    user = User.create(attributes)
    Account.create(user: user)
  end
end

Setting up Rubocop correctly with Gitpod and VSCode Ruby

Can Olcer — Fri, 29 Oct 2021 14:15:42 +0000

I've recently come across Gitpod. Gitpod provides cloud-based development environments. They're similar to GitHub Codespaces, but in my opinion offer a better experience. Setting up an environment is simpler and they are all about ephemerality. The idea is that each environment just lives for the duration of the task you're working on, and then you throw it away. Like Codespaces, Gitpod uses the Visual Studio Code editor, which you can access with your browser.

I'm slowly adding Gitpod to all of my projects, because the development experience is much better than doing it on my local machine.

For one of my Rails projects, I had some issues getting the Rubocop linter to work. Here's a short set up guide to so that others can avoid the problem I had.

To configure your environment to work with Gitpod, you need to add a file called .gitpod.yml to the root of your project. In my case it looks like this:

image:
  file: .gitpod.Dockerfile
ports:
  # Rails server
  - port: 3000
    onOpen: ignore
  # PostgreSQL server
  - port: 5432
    onOpen: ignore
tasks:
  - init: >
      bundle install &&
      rails db:setup
github:
  prebuilds:
    branches: false
    pullRequests: true
vscode:
  extensions:
    - rebornix.ruby
    - wingrunr21.vscode-ruby

You can see that it contains simple instructions that Gitpod uses to set up your cloud-based development environment. Importantly, it needs to know which Docker image it will build from. While you can specify a pre-built image, in this example, I reference another file in my project called .gitpod.Dockerfile that looks like this:

FROM gitpod/workspace-postgres
USER gitpod
# Install the Ruby version specified in '.ruby-version'
COPY --chown=gitpod:gitpod .ruby-version /tmp
RUN echo "rvm_gems_path=/home/gitpod/.rvm" > ~/.rvmrc
RUN bash -lc "rvm reinstall ruby-$(cat /tmp/.ruby-version) \
              && rvm use ruby-$(cat /tmp/.ruby-version) --default \
              && gem install rails \
                 rubocop rubocop-performance rubocop-rails rubocop-rspec"
RUN echo "rvm_gems_path=/workspace/.rvm" > ~/.rvmrc

This is a default Gitpod Docker image that works well with Rails and Postgres. Gitpod first runs the commands in the Dockerfile, and then the commands in the tasks: key in .gitpod.yml. And here is the tricky part: If you only install Rubocop with the bundle install command after the Docker image has been set up, VSCode's Ruby extension (which we use for linting) won't have access to the command. Therefore, as you can see in the above Dockerfile, you need to install rubocop in the same path as you install Ruby.

That's it! If you haven't tried Gitpod or Github Codespaces, I really recommend giving it a spin. For me, this is the next level of abstraction in programming that will increase developer productivity by a significant factor.

Jekyll and Tailwind: How to speed up build time

Can Olcer — Fri, 29 Oct 2021 14:13:12 +0000

I've been trying out Jekyll for a new side project's static website and of course added my favorite CSS framework, Tailwind, into the mix. However, after adding Tailwind with PostCSS I began to see very slow build times. Generating the site went from less than a second to more than 30 seconds. Waiting this long to see every change you do makes working with Jekyll impossible.

The problem arises because Jekyll regenerates all of the SCSS for every change, even if you use the --incremental flag and don't touch your SCSS. The Tailwind files include a lot of CSS classes, and therefore this takes forever (at least on my 2016 MacBook).

There are different ways to avoid this and build your own fancy asset pipeline, but I wanted to stick to a simple asset pipeline with PostCSS.

The idea is to take advantage of some neat config options in Jekyll and of the fact that you usually don't change your Tailwind CSS. We'll make Jekyll generate the site with Tailwind once in the beginning of our project, and after that tell Jekyll to not bother with the Tailwind CSS files.

First, add Tailwind with PostCSS to your project (you can follow this or this tutorial).

Second, in your _config.yml file, add the following to your exclude and keep_files settings.

exclude:
  # other stuff you're excluding
  - assets/css/tailwind.scss
keep_files:
  # other stuff you're keeping
  - assets/css/tailwind.css
  - assets/css/tailwind.css.map

We're telling Jekyll to ignore tailwind.scss and to not delete tailwind.css and tailwind.css.map when building the site. Why not delete the files if we ignored them in the first place? You'll see. Make sure that you only @import the Tailwind stuff in tailwind.scss and keep the rest of your CSS in other files. My tailwind.scss looks like this:

---
---
@import "tailwindcss/base";
@import "tailwindcss/components";
@import "tailwindcss/utilities";

Third, we add a second config file called _config_tailwind.yml that looks like this:

include:
  - assets/css/tailwind.scss

That's it.

The magic happens here: Whenever you want Jekyll to build Tailwind, you need to tell it to include both config files. You definitely need to do this once at the start of the project, and then everytime you change something in Tailwind. To build with both configs, run this command:

jekyll build --config _config.yml,_config_tailwind.yml

After that, you can just build or serve Jekyll normally with:

jekyll serve --incremental

The keep_files option is necessary, because otherwise Jekyll will delete our taiwind.scss file from the generated site.

Bonus: To make your live easier, add the initial build command to your package.json file:

"scripts": {
  "build-tailwind": "jekyll build --config _config.yml,_config_tailwind.yml"
}

Now, you can just go yarn run build-tailwind.

A word of caution: Once you add PostCSS to handle your asset pipeline, Jekyll's standard way of importing SASS partials from the _sass/ folder stops working. Instead, you need to import with the full path, like so:

# assets/css/main.scss
@import '/_sass/header.scss'
# this default Jekyll way will fail
@import 'header'

Rails: Prevent users from logging out after each deployment

Can Olcer — Fri, 29 Oct 2021 14:08:33 +0000

Here's a quick one, and it may be obvious to some of you but I didn't know about it. I noticed that my Rails app (Fugu) kept logging out all users after every deployment.

First, I thought it's an issue with Devise, but it turns out that it's related to a variabled called secret_key_base that Rails uses to sign and encrypt cookies (among other things).

For production, there are multiple places to define secret_key_base. A glance at the Rails soure code shows that Rails looks for it in ENV["SECRET_KEY_BASE"], credentials.secret_key_base, or secrets.secret_key_base.

In my case, I hadn't set up any credentials or secrets, nor was I providing an environment variable.

Digital Ocean (and, as it looks, Heroku) automatically sets the SECRET_KEY_BASE environment variable for you, and it changes with every deployment. And this was the problem. After each deployment, my Rails app couldn't decrypt the existing session cookies anymore beause secret_key_base had a different value, and my users needed to log in again.

To solve the problem, just provide a SECRET_KEY_BASE environment variable in your production server. The simplest way to generate it is to run rake secret in your terminal (make sure you're in a Rails project folder).