Forem: Peter Gundel

RSpec GraphQL integration testing

Peter Gundel — Mon, 16 Jan 2023 09:35:19 +0000

While working on different Ruby projects, I noticed one pattern when writing integration tests for GraphQL: You write your query in a multiline string, get the response, parse it (probably with a helper) and write some expectations, maybe even expecting a whole multi-dimensional Hash. This could then look something like this:

RSpec.describe "Query.currentUser" do
  subject(:query_result) { MySchema.execute(query, context: context).as_json }

  let(:user) { create(:user) }
  let(:context) { { current_user: user } }
  let(:query) { <<~GRAPHQL }
      query {
        currentUser {
          id
          email
        }
      }
    GRAPHQL
  let(:expected_result) do
    { "data" => { "currentUser" => { "id" => user.id.to_s, "email" => user.email } } }.as_json
  end

  it "returns the current user" do
    expect(query_result).to eq(expected_result)
  end
end

For small queries, this is fine. But for big queries (and hence, big responses) this gets unhandy very fast. This is subjective of course ;)

Another issue is that we can't leverage the GraphQL language server while writing/maintaining these integration tests.

A solution to this

I decided to use this opportunity to write my first gem: rspec-graphql-integration

This gem tries to improve this situation by moving the query and the response in their own files with a proper file type. This way, the integration test files are smaller and can focus on mocking data/instances. Also, the GraphQL language server will give you autocompletion/linting in your GraphQL files (if you've set up your editor for it).

The simple integration test from above then looks like this:

current_user_spec.rb

RSpec.describe "Query.currentUser" do
  let(:user) { create(:user) }
  let(:context) { { current_user: user } }
  let(:response_variables) { { user_id: user.id, user_email: user.email } }

  it { is_expected.to match_graphql_response }
end

current_user.graphql

query {
  currentUser {
    id
    email
  }
}

current_user.json

{
  "data": {
    "currentUser": {
      "id": "{{user_id}}",
      "email": "{{user_email}}"
    }
  }
}

Tech documentation culture at store2be

Peter Gundel — Thu, 09 May 2019 11:51:37 +0000

Our documentation setup (photo by Michael D Beckwith, flickr)

Intro

Ink is better than the best memory. 🎓

This is the subtitle of our tech documentation!

If you haven't thought about documentation in a small team, with a growing team you eventually realise how important documentation is. Even for one person alone, it takes a lot of cognitive load to remember all the knowledge when it is not written down. In a team, documentation is even more important and acts as a communication tool and ensures that everyone has the same understanding of certain topics. Documentation also helps a lot in conducting non-automated regular tasks that occur in a low frequency.

In this article, we want to share how we approach documentation at store2be. Documentation happens on different levels:

Company-wide documentation
Tech documentation
Infrastructure as code
App-specific documentation
Code documentation

Everything that is of importance for all departments needs to go into our company-wide documentation. For example, if something technical is important for the product team as well, we will put it in the company-wide documentation.

The tech documentation is for all the technical things that are only relevant for the engineering and the DevOps team. In this part of the documentation, there is also frontend and backend related documentation that is shared across all our apps, so we don't have to repeat it in these repositories.

Our tech documentation is a Git repository with markdown files. This has the advantage that we can easily write it offline and use our favourite tools to edit and search the documentation. Also, everyone has always a local copy of the whole documentation present.

Infrastructure as code happens, when you use tools to describe your infrastructure in a declarative way and you don't manage your infrastructure manually. Some tools that we use in that context are Terraform and Kubernetes.

If something specific to an app needs to be documented, we will mostly put it into the README.md of the respective Git repository. Examples here are eg. app specific scripts/commands and what they are there for.

We won't go into details about code documentation, because this is something a whole blog post on its own can talk about. We want to mention it for completeness here anyway.

Tech Documentation

The Tech Handbook

The main topic of this blog post is our tech documentation. In this part of the documentation, we collect the most knowledge of the engineering team and this is what helps us scale the engineering team. We call it: The Tech Handbook.

Looking back into 2017, we had nearly no tech documentation at all. We were still just a team of three and we hadn't thought too much about documentation at that point. In hindsight, this is super bad and something we will avoid in the future, because now we know how valuable proper documentation is.

By the end of 2017, we had three people joining in the beginning of 2018 and we knew, we had to share a lot of knowledge with them. That's when we began the groundwork of our Tech Handbook.

In the beginning, we focused on the areas that we needed to share with the new developers from day one, eg. processes and tools. The infrastructure sections followed quickly, as infrastructure work is mostly done once and then looked at again a lot later when you forgot a lot of it probably.

Since then we continuously add new things along the way, and documentation has become part of our tech culture.

Keeping documentation up to date is a whole different challenge. At store2be, it's a collective effort. As soon as we discuss for example process changes, we assign someone with the task of updating the relevant part in the tech documentation via our project management tool. This helps us organizing documentation changes together with our normal development workflow. Thankfully, we use Git and GitHub for our Tech Handbook, everyone else in the team can review the documentation changes and comment on them. So in the end, we're all on the same page again.

We divided our Tech Handbook into the following areas:

General
Processes and tools
Infrastructure
Backend
Frontend
Post mortems

These sections should be all pretty self-explanatory. The last section, post mortems, is the newest section that we added in autumn 2018. We realized that most of the time the same people worked on finding and fixing critical issues in our systems without much transparency to the rest of the team. We occasionally had small presentations of what caused the problem, but if a similar problem occurred again, the person working on it had to remember it. This approach doesn't scale well, and even if the same person worked on a similar problem again, this person might have forgotten parts of the previous solution.

With the post mortems in our Tech Handbook, we can now share the steps on how we were able to debug an issue easily, and because everyone reviews it, everyone gets a feeling for possible issues and can account for it while coding.

Conclusion

All in all, we are very happy with our choice of building up our Tech Handbook and can really recommend it to everyone. It gives us a lot of safety regarding not losing knowledge, especially if someone goes on holidays. Also, onboarding new people has become way easier and faster.

Tested Elasticsearch on Rails with Docker

Peter Gundel — Fri, 02 Nov 2018 15:03:24 +0000

Recently, the product team has been mentioning the need for a better search solution in our apps. That’s why I took the time to look into Elasticsearch after a two year break. For me, it was amazing to see how easy it was to set up a proof of concept that was even integrated into CI. The last part is quite important for us because we are very keen on testing in general ‚Äî especially when it comes to CI.

Our development setup is already running on docker compose, which while having some disadvantages of course, mostly regarding speed, has the benefit that you can share and spin up the whole infrastructure without installing anything but the docker related things. If you only have a Rails app with PostgreSQL this might not be worth it. Sharing your development infrastructure via code however also gives you the advantage of sharing the correct version of PostgreSQL with the team. Also, the more things you add to the mix, the more annoying it gets to spin everything up. For instance, we use Redis for caching, sessions and our worker queue: yet another infrastructure dependency we have to share across the team.

Having our development and CI setup already on docker made adding Elasticsearch to our app super easy! I won't go into details about how to use Elasticsearch with Rails, because there are other articles doing this well already.

Only one thing that I wasn‚Äôt getting right from the beginning: The match parameter of searchkick (Ruby library around Elasticsearch, used in the article mentioned above). You might want to think about whether you need word_start or word_middle when deciding how to match your queries ;)**

Setup of Elasticsearch

I needed to add Elasticsearch to our development environment and to our CI setup. For the local environment, we use docker compose. So this addition to our docker-compose.yml was needed:

  ...
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.4.2
    hostname: elasticsearch
     ports:
      - "9200:9200"

This tells docker compose to spin up an Elasticsearch container and hook it up via DNS to our app container. So you just need to set environment variable ELASTICSEARCH_URL: http://elasticsearch:9200 and you are good to go.

For CI we use CircleCI with Docker. There it was just a one-liner to get it up and running:

jobs:
  build:
    docker:
      ...
      - image: docker.elastic.co/elasticsearch/elasticsearch:6.4.2

And that‚Äôs it!

I won‚Äôt talk about Elasticsearch in production here, because this is a whole other thing of course and you should think about using a managed solution if you don‚Äôt have any experience with it.

Testing the search endpoint

We use RSpec for testing. This is how a basic test of the search would look like:

# searches_controller.rb
class SearchController < ApplicationController
  def execute
    data = if params[:query].blank?
      []
    else
      Store.search(params[:query]).map do |store|
        {
          id: store.id,
          title: store.title,
        }
      end
    end

    render json: { data: data }
  end
end

# searches_spec.rb
require 'rails_helper'

describe SearchController do
  describe '#execute' do
    it 'works' do
      store = Store.create!(title: 'Alexa Shopping Center')

      # The query is misspelled on purpose!
      get '/search?query=Alxa'

      expect(response).to have_http_status(200)
      expect(JSON.parse(response.body)).to eq({
        data: [{
          id: store.id,
          title: store.title,
        },],
      })
    end
  end
end

We also use VCR for recording HTTP requests to external APIs for future test runs. However, the Elasticsearch calls are local and can be executed during tests. That‚Äôs why we needed to add an exception to VCR:

VCR.configure do |config|
  config.ignore_hosts '127.0.0.1', 'localhost', 'elasticsearch'
end

Why not just use PostgreSQL full text search?

There are good articles out there that explain how to enable a similar full-text search in PostgreSQL without adding another dependency to the infrastructure. This is definitely an option as well!

Following these tutorials was not as easy as it was for me to setup Elasticsearch with searchkick, there is still a lot of setup required, even though it‚Äôs only code and not infrastructure. The option of using a managed Elasticsearch service is great in that it doesn‚Äôt add many operational difficulties. The argument for Elasticsearch in my opinion is the reduced complexity of development, however we are still just trying things out when it comes taking our app search to the next level and might go for a PostgreSQL solution in the end.

Hi there, we’re store2be, a Berlin based startup that builds a SaaS enabled marketplace for short term retail space. If you like what we are posting you might wanna check out the store2be tech page or follow us on Twitter.

Email templates at store2be and GDPR — How we migrated away from Sendwithus

Peter Gundel — Thu, 14 Jun 2018 14:35:31 +0000

When store2be started 3 years ago we were searching for a nice way to handle email templating and sending. I came across Sendwithus, an email template service that connects to a lot of different email providers, like SendGrid, Mailjet, etc.

We decided to use Sendwithus as it decoupled the email templating from our main application and allowed non developers to handle email template changes. Furthermore, it was very helpful to have different email sending providers integrated automatically. Once, we had to switch the email provider and it took a matter of minutes with Sendwithus.

Now that the GDPR is coming into effect we have to evaluate all the services we use and check whether they are compliant. In February, Sendwithus informed its users about their way of handling GDPR compliance:

Unfortunately, we will not be making Sendwithus GDPR compliant.

This was a bummer for us. Although they now provide a new, compliant service, back then we started looking for a solution right away hearing from Sendwithus that they would not be attempting GDPR compliance. So we tried to find another service that matched our requirements but we weren’t successful.

At store2be, we are very keen on code quality and the tools around it (testing, linting, etc.). This was always a problem with Sendwithus. It kind of worked, but we were never sure whether we were going to break something and reviews only happened visually and not by looking at the actual code. Also, there was no nice Git history of the changes. Finally, there were a lot of hacks to get around the limitations of the templating possibilities of Sendwithus, eg. regarding snippets.

In the end we decided to move email templating into the hands of the developers again. The main reason for that might be the fact that Mailjet open sourced its email template markup language, MJML, which makes writing HTML email templates super easy. In the frontend we are mainly developing with React in Typescript and Jest for tests. This seemed like a perfect fit for this project regarding code quality, testability and ease of use.

Of course we lose one very important attribute with that approach: All email template changes have to be done by developers again.

The open source project Maily provided a lot of inspiration on how to get started with this service (here is a Medium post about it from the creator of Maily). Unfortunately, it is not maintained anymore and my issues and PRs are not being tackled. But in its core, Maily is just one file that creates the express server. So we copied this file into our repository and adjusted it to our needs (moving it to Typescript, satisfying the linter, updating MJML, adding more functionality).

This is what we are working with now:

Typescript: All our code for the email templates is in Typescript. Hence, a lot of bugs are caught early.
Linter: We use TSLint to comply to a coding standard we like.
Prettier: We use Prettier to format our code. No discussions about each individual coding style.
Testing: All components (snippets and email templates) are unit and snapshot tested. This means every developer feels confident about changing an email template. Furthermore, we use lorikeet for integration tests. This adds an additional layer of safety we hadn’t thought about in the beginning.
Localization: We use a very simple approach where each email template has a JSON file with the keys for each language we want to support. So the actual React component does not contain any literals but uses the translate function that reads this JSON file. Both, TXT and HTML templates use the same translations which reduces the possibility of inconsistencies.
Preview: For development, you make GET request to the local express server (without hot reloading at the moment) to see a preview of the email. Online, the product team can do the same with the staging or production server. Furthermore, we have Swagger definitions for the email templates that can be transformed into Postman collections which makes the life of the product team even easier.
Review: All code at store2be is reviewed. This also applies to the new email template service.

All in all, we are very happy with our decision of developing the email template service ourselves. Email templates are finally fun to work with.

Here is how an email template could look like now:

import { generateFetchLocale } from 'lib/utils'
import * as React from 'react'

import Button from 'templates/html/snippets/Button'
import Closing from 'templates/html/snippets/Closing'
import Footer from 'templates/html/snippets/Footer'
import FullWidthBorder from 'templates/html/snippets/FullWidthBorder'
import Greeting from 'templates/html/snippets/Greeting'
import Header from 'templates/html/snippets/Header'
import Layout from 'templates/html/snippets/Layout'
import Text from 'templates/html/snippets/Text'
import Title from 'templates/html/snippets/Title'
import locales = require('templates/locales/Welcome.json')

const Welcome: React.SFC<WelcomeProps> = props => {
  const link = props.link || 'https://www.store2be.com'
  const user = props.user || { title: '', lastname: '' }
  const { locale } = props
  const fetchLocale = generateFetchLocale(locale, locales)
  return (
    <Layout env={props.env}>
      <Header />
      <Title>{fetchLocale('title')}</Title>

      <Greeting locale={locale} lastname={user.lastname} title={user.title} />

      <Text>{fetchLocale('welcome_please_confirm')}</Text>

      <Button link={link}>{fetchLocale('button')}</Button>

      <Text>
        {fetchLocale('button_not_working') + ' '}
        <a href={link}>{link}</a>
      </Text>

      <Closing locale={locale} />

      <FullWidthBorder />

      <Footer locale={locale} />
    </Layout>
  )
}

export default Welcome

How to use SweetAlert2 for your Rails +5.1 (rails-ujs) confirms without jQuery

Peter Gundel — Sat, 02 Dec 2017 15:16:01 +0000

TL;DR see this demo project for the final solution.

Please be aware that the examples here are provided with ES6 syntax. There are different ways to get ES6 to work in Rails. The demo project has an ES5 branch for reference.

If you are reading this I assume you are familiar with Ruby on Rails and SweetAlert2. With Rails before version 5.1, when rails-ujs was still jquery-ujs, there was an easy way to hook up SweetAlert (or SweetAlert2) with the Rails confirm functionality.

One way to achieve it was to overwrite the confirm handler of Rails:

// Override the default confirm dialog of Rails
$.rails.handleConfirm = link => {
  if (link.data('confirm') === undefined){
    return true
  }
  showSweetAlertConfirmationDialog(link)
  return false
}

We had this solution in one of our apps and I wanted to use the new rails-ujs. My first thought was, that it should be an easy task to adapt. Just change $.rails. into Rails. and we’re good:

// Override the default confirm dialog of Rails
Rails.handleConfirm = link => {
  if (link.data('confirm') === undefined){
    return true
  }
  showSweetAlertConfirmationDialog(link)
  return false
}

As it turns out some things have changed.

Rails.handleConfirm can be overwritten, but that will not override the event listener that is already attached since rails-ujs was initialised. But no problem, let’s just write our own event handler and plug it into the new rails-ujs way of doing things. If you have a look at the source code of the start part of rails-ujs you see how event listeners are created. The code to add an event listener for our own method then looks like this:

const handleConfirm = link => {
  // Do your thing
}
Rails.delegate(document, 'a[data-confirm-swal]', 'click', handleConfirm)

Alright, cool. It works for links with the attribute data-confirm-swal="Are you sure?" now 🎉 …but wait, if you have a delete link, the confirm dialog never shows up because the method never gets called. 🤔 Turns out the event listener for method: :delete is called earlier because it got initialised before our event listener for SweetAlert2. This is because the event listeners of rails-ujs are hooked up directly when evaluating the code.

If we want to add our event listener by requiring our Javascript code before rails-ujs, then we get into the Problem that Rails.delegate is not defined yet. When you require rails-ujs this is what’s happening:

Define event handlers and Rails.delegate
Fire rails:attachBindings event on document
Attach event handlers (Rails.start)

So in order to get in between the definition of Rails.delegate and the execution of Rails.start we have to attach to the rails:attachBindings event. (For that to happen we need to require our script before rails-ujs!)

document.addEventListener('rails:attachBindings', element => {
  Rails.delegate(document, 'a[data-confirm-swal]', 'click', handleConfirm)
})

🎉 Now everything works as expected 🎉

For the final solution have a look at this demo project (with ES5 and ES6 version) or see the code below (only ES6).

// This file has to be required before rails-ujs
// To use it change `data-confirm` of your links to `data-confirm-swal`
(function() {
  const handleConfirm = function(element) {
    if (!allowAction(this)) {
      Rails.stopEverything(element)
    }
  }

  const allowAction = element => {
    if (element.getAttribute('data-confirm-swal') === null) {
      return true
    }

    showConfirmationDialog(element)
    return false
  }

  // Display the confirmation dialog
  const showConfirmationDialog = element => {
    const message = element.getAttribute('data-confirm-swal')
    const text = element.getAttribute('data-text')

    swal({
      title: message || 'Are you sure?',
      text: text || '',
      type: 'warning',
      showCancelButton: true,
      confirmButtonText: 'Yes',
      cancelButtonText: 'Cancel',
    }).then(result => confirmed(element, result))
  }

  const confirmed = (element, result) => {
    if (result.value) {
      // User clicked confirm button
      element.removeAttribute('data-confirm-swal')
      element.click()
    }
  }

  // Hook the event before the other rails events so it works togeter
  // with `method: :delete`.
  // See https://github.com/rails/rails/blob/master/actionview/app/assets/javascripts/rails-ujs/start.coffee#L69
  document.addEventListener('rails:attachBindings', element => {
    Rails.delegate(document, 'a[data-confirm-swal]', 'click', handleConfirm)
  })

}).call(this)

To find this all out it took me some hours as I wasn’t familiar with the codebase of rails-ujs. But I learnt a lot along the way. Hopefully with this writeup I can help some other developers as well who want to use the latest version of Rails with SweetAlert2 and without jQuery.

If you use Webpacker, there is an easy way to get in between the rails-ujs code and the start script:

import Rails from 'rails-ujs';

const handleConfirm = () => {
  // Do your thing
}

// Add event listener before the other Rails event listeners like the one
// for `method: :delete`
Rails.delegate(document, 'a[data-confirm-swal]', 'click', handleConfirm)

Rails.start()

Heroku CLI meets fzf

Peter Gundel — Mon, 14 Aug 2017 09:14:53 +0000

While working with tmux and fzf I came across another interesting fzf use case: Use a fuzzy finder to simplify the use of the Heroku CLI.

Imagine you have some apps on Heroku and now and then you want to tail their logs or you want to ssh into one of them. There is one problem with that: You always have to memorize the exact app name in order to run $ heroku logs -t -a awesome-app-staging. Or was it staging-awesome-app? Well, you see the problem with that. Another solution might be to define an alias for every app. However, this doesn’t scale too well…

But what about a fuzzy finder for your Heroku apps in the CLI? 🤖

When I switched to vim, I first heard about fzf and began using it with joy to open my files. I recently started using tmux and there fzf can be a great help as well. This stackoverflow post showed me how to use fzf for switching between tmux windows. I also used this approach to switch between tmux sessions (and shared the solution in the linked stackoverflow post). This eventually lead to the idea of using fzf in combination with the Heroku CLI.

The magic command for this is:

$ heroku apps --all | grep '(' | sed 's/ .*$//' | fzf --header='Select the app you want to tail the logs' | xargs heroku logs -t -a

Of course you should create aliases for that:

alias heroku-logs="heroku apps --all | grep '(' | sed 's/ .*$//' | fzf --header='Select the app you want to tail the logs' | xargs heroku logs -t -a"
alias heroku-bash="heroku apps --all | grep '(' | sed 's/ .*$//' | fzf --header='Select the app you want to bash into' | xargs heroku logs -t -a"

Step by step explanation

The first command should be self explanatory. It gives you a list of all your apps on Heroku. But the format here is important! The command returns two lists. Your own apps and the apps on which you are a collaborator:

$ heroku apps --all

=== peterfication@example.com Apps
obscure-tundra-15966 (us)
nameless-basin-19036 (eu)

=== Collaborated Apps
shrouded-fortress-18816 (eu)          someone-else@example.com

First, we remove all the lines by piping the output to grep. Every line with an app contains brackets so we use the brackets to filter the lines.

$ heroku apps --all | grep '('

obscure-tundra-15966 (us)
nameless-basin-19036 (eu)
shrouded-fortress-18816 (eu)          someone-else@example.com

Next, we have to ensure that we only take the app name with nothing else. So we remove everything after the app name, starting with the empty space.

$ heroku apps --all | grep '(' | sed 's/ .*$//'

obscure-tundra-15966
nameless-basin-19036
shrouded-fortress-18816

This is a nice list we can pipe to fzf. The result of fzf is then piped to the heroku command that we want to execute:

$ heroku apps --all | grep '(' | sed 's/ .*$//' | fzf --header='Select the app you want to bash into' | xargs heroku logs -t -a

After selecting the desired app with fzf, the command finally expands for example to:

$ heroku logs -t -a shrouded-fortress-18816

💪

Hi there, we’re store2be, a Berlin based startup that builds a SaaS enabled marketplace for short term retails space. If you like what we are posting you might wanna check out the store2be tech page or follow our Medium channel.

So you switched to Vim - Don't forget to install Vimium for Chrome!

Peter Gundel — Wed, 02 Aug 2017 14:58:13 +0000

A few months ago I wrote about my first steps with Vim. Now I can finally say: I am faster with Vim than I ever have been with RubyMine 🎉 As soon as you get used to the Vim key bindings you demand them from every other program as well. In my mail program for example I try doing the most things with shortcuts now, too.

And recently I started using the Chrome Plugin Vimium. It makes surfing the web much more efficient if you are used to the Vim key bindings already.

The equivalent plugin for Firefox is VimFx.

Here are my top 3 Vimium features

J/K lets you go to the previous/next tab (Way easier to type compared to <Ctrl-Tab>/<Ctrl-Shift-Tab>).

T gives you an fzf-like tab switcher. With <Ctrl-j/k> you can go up and down in the results (like in fzf). Similarly, o gives you an fzf-like menu for your history and bookmarks.

f/F gives you letters in overlays for every link of the page and opens the link in the same tab/new tab.

Vimium has even more features. So check it out :)

Finally switching to vim

Peter Gundel — Wed, 19 Apr 2017 14:33:36 +0000

This article is just a selection of things I found helpful when switching to vim. So when someone asks for help because he or she wants to switch to vim, I just pass him or her the link to this article. Some of the links in this article also describe why you should switch to vim. So I won't get into this here.

Before I started using vim, I thought switching would be just a timeframe of 2-4 weeks of being less productive and learning vim and then I would have it all set up and I am used to the vim style of doing things. But then I realized switching to vim is more like a long and ongoing journey.

The most important thing about vim is that you can't just start with a complete-guide-to-vim and then you're good to go. It is a steady process of learning new things (and be overwhelmed by them). The secret in the beginning is to give yourself some time to get up to speed and understand the brilliance behind the concepts of vim.

I recommend to start by watching this introduction talk to vim by Mike Coutermash. He describes really good what I was experience as well when starting with vim. Here are also his blog article and his slides for the talk.

In order to get a first impression I started with an interactive online tutorial. This tutorial provides a very good interactive introduction to the principles of vim. And this is a very good read about the why of vim.

After that I realized that I need a list of all these movement commands. Here is a cheat sheet to start with. Or even better you print out a more complete vim cheat sheet: This or this. The vim wiki has a getting started page that I found quite useful.

In order to train the basic movements you can start playing VIM Adventures.

This Github repository provides a long guide to vim.

Here are 68 free screencasts about vim and here are two more free ones and other paid ones.

You also might wanna have a look at neovim.

Vim has a very rich ecosystem of plugins. There are multiple ways of installing plugins. I started with pathogen.vim and it was ok. But my coworkers showed me the advantages of vim-plug and as they are using vim-plug I switched, too.

Then you have to decide which plugins to use (and not to use 😉) and how to customize your settings and keymaps. As a starting point I will list some plugins someone might have a look at. The most important thing here is: read through the documentation of each plugin and decide whether you need it or not!

Here are some plugins that I found useful for the start:

And every programming language normally has a vim plugin on its own.

Finally, start playing vimgolf from the beginning. It's never too early to start (I had that thought). Just look at the other solutions and try to understand what they did.

After one week of vim I'm not as productive as with RubyMine. If I really need to get something done fast I still open RubyMine. But most of the time I use vim now and I enjoy working with it. Two more weeks and RubyMine stays closed. What I miss the most is the awesome multi cursor implementation of RubyMine. But I already have been told that in vim you do things differently 😉

Introduction to Metabase

Peter Gundel — Thu, 06 Apr 2017 16:55:20 +0000

When you start a company, setting up a business intelligence (BI) solution is not the first thing most founding teams implement, I guess. Now that I'm past the step of implementing such a solution, I think it is something that should be implemented in the very early days to build a culture of measuring everything.

The main reason why it wasn't implemented directly from the beginning at my company was that no one had the experience with a good solution yet. Therefore, reports were conducted manually against the database in raw SQL or implemented into the internal admin area to make them a little bit more accessible (but without graphs of course because it would have cost too much time). As you might guess, this was some work and couldn't be changed or extended fast.

In the fall of 2015, AWS QuickSight was announced. It seemed to be a very good solution for us as our databases are hosted on AWS. In August 2016 we got access to the preview version. It wasn't really easy to use and the user management was not really good. Reports and datasources weren't shared accross users or I didn't find the right way to do it. In most cases we use UUIDs as our primary keys and AWS QuickSight had no support for UUIDs back then. This lead to the situation that I didn't put more time and effort in using AWS QuickSight. When AWS QuickSight was finally released, I took another look at it but I still wasn't satisfied with the look and feel. (This is very subjective and might be different if I gave it another try today!)

At the same time of the release of AWS QuickSight, my former CTO at fotograf.de, Marco Beinbrech, showed me their BI solution, an open source software called Metabase. He told me that it is super simple to setup and easy to use for all team members (especially for non-technical team members ðŸ˜‰).

And indeed, he is right. Metabase provides different ways to deploy it anywhere. Most of our servers were at Heroku then and Metabase provides a one click deployment to Heroku. Pretty neat! But at that time we were also undertaking our first steps with Kubernetes and Metabase seemed to a be a perfect fit to play around with Kubernetes. So we wrote the Kubernetes objects for it and voilÃ , it was up and running. No hassle at all!

Further setup

A collection of things you should consider when setting up Metabase

Using the production PostgreSQL user creates a security risk because this user can change data in the database. As Metabase is only for reporting, and therefore reading data, we created a read-only PostgreSQL user for our production database.

It is not a good idea to give Metabase direct access to your production database. When you create huge and slow queries, it will directly affect your production servers. Fortunately, AWS RDS makes it super easy to create a read replica of your production database.

Metabase needs a database it can read and write from for itself in order to manage data about the users, reports and so on. To make the database of Metabase persistent across Kubernetes deployments, we created another database on our RDS instance. This way, the Metabase data is also backed up by our current backup procedure. You just have to provide the PostgreSQL connection details to the Metabase deployment in Kubernetes and it works out of the box. Then you can create and rebuild the cluster without loosing any Metabase internal data.

We use Google Apps at my company and Metabase has a Google OAuth login built in. So you just have to follow the guide and all your colleagues can register with their Google Apps account.

User management can be handled with user groups and collections (folders for your reports in Metabase). Marco gave me the tip to set up collections and and user groups according to the department structure. This way you can implement user access on a collection level very easily.

First steps

I really recommend reading the docs of Metabase. They are really good, especially for the first steps.

In Metabase reports are called questions. So in order to create the first report you click on New Question, select the database, select the table, add filters, specify what you want to see (eg. count of users) and add a grouping (eg. created_at by month). And all of that in a simple click interface that can be used by a lot of users and not only the SQL pros in the company. (It still gives you the possibility to create custom SQL queries).

Me encanta! ðŸ˜Š

Finally, you can put multiple questions on a dashboard and have a nice overview of what's going on in the company.

Conclusion

All in all, Metabase was really easy to setup for the tech team. If we had used Heroku, it would have been even easier. And most importantly, it is really easy and fun to use for all team members. ðŸ˜Š

Thanks Marco, for all the tips here ðŸ˜Š In further posts I will go into detail on how we create advanced reports at store2be.

Multi AZ Kubernetes with kops in AWS regions with only 2 AZ

Peter Gundel — Thu, 30 Mar 2017 10:21:39 +0000

At the moment we are setting up Kubernetes at store2be. We are a company from Germany and most of our business is taking place in Germany. Therefore, we want our AWS servers to be in Frankfurt.

We use kops for our Kubernetes cluster management and we want our cluster to be a multi availability zone (AZ) deployment. With this in mind we tried out the following command:

$ kops create cluster \
    --node-count 4 \
    --zones eu-central-1a,eu-central-1b \
    --master-zones eu-central-1a,eu-central-1b \
    --node-size t2.medium \
    --master-size m3.medium \
    --yes \
    kubernetes-cluster.example.org

But the result was not that promising:

There should be an odd number of master-zones, for etcd's quorum.  Hint: Use --zones and --master-zones to declare node zones and master zones separately.

So unfortunately, at the moment it seems that it is not possible to use kops out of the box in an AWS region with only 2 AZ. But after some digging we found a Github issue that described our problem and even had a work around for that! So the following steps are based on the solution provided by Kamil Hristov.

1. Create a cluster.yml for a AWS region with 3 AZ

Create a cluster in eu-west (because it has 3 AZ):

$ kops create cluster \
    --node-count 4 \
    --zones eu-west-1a,eu-west-1b,eu-west-1c \
    --master-zones eu-west-1a,eu-west-1b,eu-west-1c \
    --node-size t2.medium \
    --master-size m3.medium \
    --yes \
    kubernetes-cluster.example.org

Dump the cluster and instancegroup configuration and append it:

$ kops get cluster --name=kubernetes-cluster.example.org --output yaml > cluster.yml && \
  echo "\n---\n" >> cluster.yml && \
  kops get instancegroups --name=kubernetes-cluster.example.org --output yaml >> cluster.yml

2. Adjust the cluster.yml

Replace all the occurrences of eu-west with eu-central
Replace all the zone definitions of eu-central-1c with eu-central-1a
Replace all the name definitions of eu-central-1c with something different than eu-central-1a like for example eu-central-1a2 (otherwise there will be name conflicts)

3. Create the cluster by using the adjusted cluster.yml

We have to use replace here, because the cluster spec already has

been created in step 1.

$ kops replace -f cluster.yml
$ kops create secret --name kubernetes-cluster.example.org sshpublickey admin -i ~/.ssh/id_rsa.pub
$ kops update cluster --yes
`

And ✨, there is the multi AZ cluster in Frankfurt 🎉

I created this article because it took us a while to find out about this solution and I hope we could help someone with it. Thanks to Kamil Hristov for sharing the solution. Thanks to Tom Houlé for reviewing the article.

Hi there, we're store2be, a Berlin based startup that builds a SaaS enabled marketplace for short term retails space. If you like what we are posting you might wanna check out the store2be tech page or follow our Medium channel.