Forem: Maxence Henneron

Automatic DNS and SSL management with Traefik

Maxence Henneron — Tue, 23 Apr 2019 18:27:28 +0000

Last month, we launched a new SaaS service called upload.express. It is a fully customizable uploader for anyone who often shares files. Upload.express is fully open-source and our business model is to offer hosting services.

We quickly faced our first problem: how will we manage all the DNS records and SSL certificates?

The first solution that came to our mind was to develop a small micro-service that would call the API of our domain name hosting provider to create the records, and finally use Let’s Encrypt to generate the SSL certificate.

This is something I implemented 3 years ago in one of my last projects and we came across a huge issue: Let’s Encrypt has per-domain rate limits. You can only request 50 subdomain SSL certificates per week. Our project has a free trial plan so we expect to create more than 50 subdomains per week.

Wildcard certificates

In the past, wildcard SSL certificates used to cost $300 but in 2018 everything changed when let’s encrypt announced they would support them. A wildcard certificate is a certificate you can use on all your subdomains.
In example, if you own the domain “example.com”, you can use the same wildcard certificate “*.example.com” on all your subdomains.

Wildcard DNS records

Along with wildcard certificates, most DNS providers support wildcard DNS records. By creating an “A” entry named “*.example.com”, that points to your server’s IP address, any request that come from any of your subdomain would redirect to your server’s IP. By doing this, you remove the necessity to implement your own domain-management microservice.

Using Traefik as a load balancer

As you grow, you will need more than a single server to handle all the traffic. Also, configuring the SSL certificates at the load balancer level is easier as you do not have to configure all the different types of applications, you will only have to secure the transmission between the user and the load balancer.

On upload.express, I use docker swarm to manage the deployment. You can also use other cluster management systems with traefik, such as kubernetes. In this short tutorial, I’m going to assume you’re using docker swarm, but it should not be hard to adapt it to kubernetes.

First, I created a network to let traefik communicate with the services deployed on my cluster

docker network create --opt encrypted -d overlay webgateway

Then, I used the following stack to deploy traefik.

version: "3.5"
services:
  traefik:
    image: traefik:1.7
    command:
      - "--api"
      - "--loglevel=debug"
      - "--docker"
      - "--docker.swarmMode"
      - "--docker.domain=upload.express"
      - "--docker.watch"
      - "--configFile=/etc/traefik/traefik.toml"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /opt/traefik/traefik.toml:/etc/traefik/traefik.toml
      - /opt/traefik/acme.json:/etc/traefik/acme/acme.json
    environment:
      - AWS_ACCESS_KEY_ID=
      - AWS_SECRET_ACCESS_KEY=
    networks:
      - webgateway
      - traefik
    ports:
      - target: 80
        published: 80
      - target: 443
        published: 443
      - target: 8080
        published: 8080
        mode: host
    deploy:
      mode: global
      placement:
        constraints:
          - node.role == manager
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: on-failure
networks:
  webgateway:
    driver: overlay
    external: true
  traefik:
    driver: overlay

As you can see, I set two environment variables that correspond to my AWS credentials. This is because we need to use the DNS validation to generate the SSL wildcard certificates. As I’m using route53 to manage my domain, I had to provide my AWS credentials. You can see the list of environment variables you need to set depending on your DNS provider here.

The last step is to create the configuration file for traefik. In my swarm stack, I configured the path to “/opt/traefik/traefik.toml”. All you have to do is creating the file and pasting the following configuration.

defaultEntryPoints = ["https","http"]

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
  [entryPoints.https.tls]

[retry]

[accessLog]

[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "example.com"
watch = true
exposedbydefault = false

[acme]
email = "mail@example.com "
storage = "/etc/traefik/acme/acme.json"
entryPoint = "https"
acmeLogging = true
onHostRule = true

[acme.dnsChallenge]
  provider = "route53"
  delayBeforeCheck = 0

[[acme.domains]]
   main = "*.example.com"

Do not forget to replace the example domain by yours. Now, start your traefik stack by running the following command:

docker stack deploy -c traefik-stack.yml traefik

To finish, for each of the stacks you will deploy in your swarm cluster, set the following labels:

- traefik.port=4000
- traefik.enable=true
- traefik.docker.network=webgateway
- traefik.backend=service_name
- traefik.frontend.rule=Host:service_name.example.com

and link the webgateway network we created earlier like so:

networks:
  webgateway:
    external: true

In example, here's the docker stack I'm using to deploy a upload.express instance:

---
version: '3.7'
services:
  uploadexpress:
    image: uploadexpress/app
    networks:
    - webgateway
    deploy:
      placement:
        constraints:
        - node.role==worker
      labels:
      - traefik.port=4000
      - traefik.enable=true
      - traefik.docker.network=webgateway
      - traefik.backend=maxencehenneron
      - traefik.frontend.rule=Host:maxencehenneron.upload.express
      restart_policy:
        condition: on-failure
      mode: replicated
      replicas: 1
networks:
  webgateway:
    external: true

That's it! Traefik will automatically pick the correct wildcard certificate and you will be able to reach your service at "service_name.example.com"

One-command file sharing with upload.express

Maxence Henneron — Wed, 17 Apr 2019 11:40:59 +0000

In this short tutorial, I'm going to present the latest feature we implemented on upload.express: a one-command terminal file sharing.

If you want to upload a file in your current working directory, all you have to do is typing the following command:

curl --upload-file ./file_name.ext https://upload.express/upload/file_name.ext

It will return a link to the uploaded file.

Creating an alias

Remembering the curl command arguments can take a while. You can simplify the command by creating an alias in your ~/.bashrc or ~/.zshrc file.

Just append the following code at the end of the file:

upload() {
    if [ $# -eq 0 ]; then echo -e "No arguments specified. Usage:\necho upload /path/to/file.md"; return 1; fi
    curl --progress-bar --upload-file "$1" https://upload.express/upload/$(basename $1) | tee /dev/null;
}

alias upload=upload

You can now use the command upload file_path to share any file.

Handling stripe webhooks with Ruby on Rails

Maxence Henneron — Tue, 26 Mar 2019 18:24:05 +0000

When accepting payments on a Ruby on Rails app, if you want to be aware of every actions that happen on stripe, you will have to implement the stripe webhooks.

I found a nice way to handle them, which I'm going to share in this article.

I strongly discourage you to implement them on your own by adding a simple POST route to /webhooks, because:

You need to check stripe's signature before accepting them (someone may be trying to impersonate Stripe!)
There are gems available to simplify this task

In this tutorial, we're going to use a gem called stripe_event

Let's get started

Installing the dependencies

The first step is to add the gem in your Gemfile. If you're new to ruby, the file is located at the root of your app.

gem 'stripe_event'

Then, in your routes.rb file, located in /config, add the following line:

mount StripeEvent::Engine, at: '/stripe-webhooks' #you can change this url

This will create a POST route to handle all the webhooks. We're going to implement that in a few moments, after we setup everything in the stripe dashboard

Setting up the webhooks

First, if you want to try the webhooks on your development environment, you will need to use a tool like ngrok.

Then, in your Stripe dashboard, navigate to Developers -> Webhooks or click on this link.

Finally, switch to "test data" and add a new endpoint. In "URL to be called", add your ngrok URL followed by the route name you chose earlier.

Once you created your webhook, please note the signing secret for that webhook somewhwere.

Adding the credentials

We will now add the stripe credentials to the (relatively) new rails encrypted credentials

To edit the credentials, type the following command:

EDITOR=nano rails credentials:edit

and write the following lines:

stripe:
  development:
    publishable_key: 'pk_test_'
    secret_key: 'sk_test_'
    signing_secret: 'whsec_'
  production:
    publishable_key: 'pk_live_'
    secret_key: 'sk_live_'
    signing_secret: 'whsec_'

publishable_key and secret_key are your stripe keys, you can find them in your account settings, signing_secret is the secret we generated earlier.

Configuring stripe_events

Create a file called "stripe_events.rb" in config/initializers and paste the following code

Stripe.api_key = Rails.application.credentials.stripe[Rails.env.to_sym][:publishable_key]
StripeEvent.signing_secret = Rails.application.credentials.stripe[Rails.env.to_sym][:signing_secret]

StripeEvent.configure do |events|
  events.subscribe 'invoice.', Stripe::InvoiceEventHandler.new
end

The first two lines are setting the correct tokens, depending on your current environment (development or production)

In the last three lines, we are telling stripe_event to subscribe to all the events starting with 'invoice.' and redirect them to a class called "InvoiceEventHandler"

The next step is to implement this class.

In your app folder, create a new folder called "services" and add a folder called "stripe" in it.

Then, add a file called invoice_event_handler.rb in the stripe folder we just created.

module Stripe
  class InvoiceEventHandler
    def call(event)
      begin
        method = "handle_" + event.type.tr('.', '_')
        self.send method, event
      rescue JSON::ParserError => e
        # handle the json parsing error here
        raise # re-raise the exception to return a 500 error to stripe
      rescue NoMethodError => e
        #code to run when handling an unknown event
      end
    end

    def handle_invoice_payment_failed(event)
    end

    def handle_invoice_payment_succeeded(event)
    end
  end
end

Here, I implemented two events: invoice.payment.failed and invoice.payment.succeeded. Using 'send', I'm forwarding the event to the correct method. (Credit: @aradilopez)

That's it!

In my various projects, I'm redirecting all the useful notifications to a slack channel, so I'm always aware of what's happening.