Forem: Mikias Abera

Vue SPA with Rails JSON:API using Graphiti

Mikias Abera — Wed, 05 Dec 2018 17:30:42 +0000

Standards are good, they make our lives easier. Graphiti and it's client library Spraypaint make it easy to build JSON:API compliant APIs that integrate seamlessly with front-end frameworks like Vue.

I’m using graphiti in a production application to serve JSON requests to Vue components embedded in our Rails views. It's been reliable, flexible and a pleasure to use.

In this tutorial we’ll walk through setting up Vue as an SPA with a JSON:API compliant Rails 5 API using graphiti. You can clone the demo app to see the finished product.

# follow along
git clone git@github.com:mikeeus/demos-rails-webpack.git
cd demos-rails-webpack
git checkout ma-vue-graphiti

Set up Rails API with Webpacker

Create Rails app with webpacker and Vue. I use postgresql but you can use any database you like.

mkdir rails-vue
rails new . --webpack=vue —database=postgresql
rails db:create db:migrate

And… done! That was easy right? Now we can move on to setting up graphiti to handle parsing and serializing our records according to the JSON API spec.

Set up Graphiti

Install graphiti, you can find the full instructions in the docs. We’ll need to add the following gems.

# The only strictly-required gem
gem 'graphiti'

# For automatic ActiveRecord pagination
gem 'kaminari'

# Test-specific gems
group :development, :test do
  gem 'rspec-rails'
  gem 'factory_bot_rails'
  gem 'faker'
  gem 'graphiti_spec_helpers'
end

group :test do
  gem 'database_cleaner'
end

We’ll need to add Graphiti::Rails to our Application controller so graphiti can handle parsing and serializing our requests. And we’ll register the Graphiti::Errors::RecordNotFound exception so we can return 404.

# app/application_controller.rb
class ApplicationController < ActionController::API
    include Graphiti::Rails

    # When #show action does not find record, return 404
    register_exception Graphiti::Errors::RecordNotFound, status: 404

    rescue_from Exception do |e|
        handle_exception(e)
    end
end

Now lets create a Post model.

rails g model Post title:string content:string
rails db:migrate

We’ll also need to create a PostResource for graphiti and a controller to handle requests. Graphiti has a generator that makes it easy to set this up.

rails g graphiti:resource Post -a index

We’re going to declare our attributes and add ActionView::Helpers::TextHelper to format our Post content using simple_format so we can render it nicely on our client.

class PostResource < Graphiti::Resource
    include ActionView::Helpers::TextHelper

    self.adapter = Graphiti::Adapters::ActiveRecord
    primary_endpoint '/api/v1/posts'

    attribute :title, :string
    attribute :content, :string do
        simple_format(@object.content)
    end
end

The generator will also create specs and a controller at app/controllers/posts_controller.rb. We’re going to move that to a namespaced folder app/api/v1 which will allow us to manage API versions in the future.

# app/controllers/api/v1/posts_controller.rb
module Api
    module V1
        class PostsController < ApplicationController
            def index
                posts = PostResource.all(params)
                render jsonapi: posts
            end
        end
    end
end

We use render jsonapi: posts to render the posts according to the JSON:API spec so we can parse it on our client using graphiti's js client spraypaint.

Now let’s add the route.

# config/routes.rb
Rails.application.routes.draw do
    namespace :api do
        namespace :v1 do
            resources :posts, only: :index
        end
    end
end

Alright looking good! All we need now is a client to consume our API.

Setup Vue

Webpacker comes with a generator for vue which you can see in the docs. It makes it super easy to add Vue or any other front-end framework like React or Angular to our application.

bundle exec rails webpacker:install:vue

Running the above will generate files at app/javascript

We’re going to edit app/javascript/packs/application.js so that we can render our App.vue component.

// app/javascript/packs/application.js

import Vue from 'vue/dist/vue.esm'
import App from '../app.vue'

document.addEventListener('DOMContentLoaded', () => {
    const app = new Vue({
        el: '#app',
        components: { App }
    })
})

For now we can disregard the Vue component, we’ll fill it in later once we’ve setup our resources and endpoints.

Setup Rails to Serve Static Content

We can’t use our ApplicationController to serve our index.html page since it inherits from ActionController::Api and we want to keep it that way since our other controllers will inherit from it.

In order to serve our index page for the SPA, we’ll use a PagesController that inherits from ActionController::Base so it can serve html files without issue.

# app/pages_controller.rb
class PagesController < ActionController::Base
  def index
  end
end

Next we’ll add a route for our homepage and redirect all 404 requests to it so our SPA can take care of business.

# config/routes.rb
Rails.application.routes.draw do
    root 'pages#index'
    get '404', to: 'pages#index'

    namespace :api do
        # ...
    end
end

Looking good, friends! Now let’s add our index.html page which will render our Vue component.

# app/views/pages/index.html
<%= javascript_pack_tag 'application' %>
<%= stylesheet_pack_tag 'application' %>

<div id="app">
  <app></app>
</div>

It’s super simple: it just pulls in our javascript and stylesheets compiled by webpacker. Then we add a div with id=“app” and a nested <app></app> so our Vue wrapper component can pick it up and render the main component.

This is the only Rails view we need to write for our application to work.

Create Models on Client

Usually when I build an SPA I’ll write services that use libraries like axios to make Ajax requests to the backend. Graphiti comes with a client library called spraypaint that handles parsing and serializing JSON:API payloads. It supports including associations, advanced filtering, sorting, statistics and more.

Let’s set it up!

yarn add spraypaint isomorphic-fetch

Next let’s create an ApplicationRecord class that will store our spraypaint configuration.

// app/javascript/models/application_record.js

import { SpraypaintBase } from 'spraypaint';

export const ApplicationRecord = SpraypaintBase.extend({
  static: {
    baseUrl: '',
    apiNamespace: '/api/v1',
  }
})

We set the baseUrl and apiNamespace to ‘’ and ‘/api/v1‘ respectively so that spraypaint uses relative paths and avoids CORS requests. It also namespaces our requests so we can manage API versions easily.

Now the Post model

// app/javascript/models/post.model.js

import { ApplicationRecord } from './application_record';

export const Post = ApplicationRecord.extend({
    static: {
        jsonapiType: 'posts'
    },

    attrs: {
        id: attr(),
        title: attr(),
        content: attr()
    },

    methods: {
        preview() {
            return this.content.slice(0, 50).trim() + '...'
        }
    }
})

We declare the id, title and content attributes. We also add a method to return a truncated preview of the content to show how we declare methods.

The jsonapiType property is needed to generate the endpoint and parse and serialize the JSON payload.

Now we’re ready to hook up the client to the API.

Hook up SPA

To hook everything up we’ll create a Vue component that uses our spraypaint models to communicate with our endpoints.

// app/javascript/app.vue

<template>
<div>
    <h1>Posts</h1>
    <div v-if="error" class="error">{{error}}</div>
    <div class="loading" v-if="loading">Loading...</div>
    <ul>
        <li v-for="post of posts" :key="post.id">
            <h3>{{post.title}}</h3>
            <p v-html="post.preview()"></p>
        </li>
    </ul>
</div>
</template>

<script>
import {Post} from './models/post.model'

export default {
    data: function () {
        return {
            posts: [],
            error: null,
            loading: true
        }
    },

    created() {
        Post.all()
            .then(res => {
                this.posts = res.data
                this.loading = false
            })
            .catch(err => {
                this.error = err
                this.loading = false
            })
        }
}
</script>

<style scoped>
h1 {
    text-align: center;
}
ul {
    list-style: none;
}
</style>

Marvellous! If we add some posts in the console and run the application we will see the posts load and render in the page.

Notice that we import our Post model and use it in our created() call like it was a Rails model. Calling Post.all() returns a promise that we can chain to set our posts and loading data properties. The spraypaint model can chain more useful methods like where and page.

Post.where({ search: 'excerpt' })
    .stats({ total: 'count' })
    .page(1)
    .per(10)
    .order({ created_at: 'desc' })
    .all()
    .then(res => ...)

Spraypaint is a very powerful library that supports pagination, sorting, statistics, complex filtering and much more. You can check out the spraypaint docs for detailed guides.

Conclusion

Standards are good. Vue is awesome, and so is Rails. Gems and libraries like Graphiti and Spraypaint make it super easy to build scalable REST APIs that comply with said standards and integrate seamlessly with frameworks like Vue.

I hope you enjoyed the article, don’t forget to like if you did. I would love to hear your thoughts or suggestions for other articles. Just leave a comment below :)

Building Gmail/Chrome Extension with Vue.js and InboxSDK

Mikias Abera — Thu, 29 Nov 2018 16:29:11 +0000

We’re going to create a small Chrome extension that uses InboxSDK and Pipl to let you search email addresses for user information right in our Gmail. You can change this demo to use any Api that you like. For example you could:

do a sentiment analysis using apis like aylien
check the spam score of your email with spamcheck

You can clone the demo's git repository with:

git clone git@github.com:mikeeus/vue-demo-inboxsdk.git

Create the Chrome Exenstion

Creating a Chrome extension is surprisingly simple. First we need a manifest.json file that will describe our extension. You can find the Chrome manifest documentation here

// manifest.json
{
  "manifest_version": 2,
  "name": "Vue Pipl Search",
  "version": "1.0",
  "permissions": [
    "https://mail.google.com/",
    "https://inbox.google.com/",
    "https://api.pipl.com/",
    "https://cdn.jsdelivr.net/npm/vue@2.5.17/dist/vue.js",
    "https://cdnjs.cloudflare.com/ajax/libs/axios/0.18.0/axios.min.js"
  ],
  "content_scripts" : [
    {
      "matches": ["https://mail.google.com/*", "https://inbox.google.com/*"],
      "js": ["inboxsdk.js", "app.js"]
    }
  ],
  "web_accessible_resources": [
    "icon.png"
  ],
  "icons": {"128": "icon.png"}
}

We want to use the 2.0 version of the manifest. We'l call our extension "Vue Pipl Search" and make it version 1.0.

Since we want our extension to work on Gmail, we're going to add https://mail.google.com/ and https://inbox.google.com/ to our permissions. We'll also add pipl.com and our vue.js and axios.min.js cdn links because we'll be using them in our app.

Next we'll add a content_scripts key which tells Chrome that we want to run the inboxsdk.js and our app.js scripts when the brower is on mail.google.com or inbox.google.com.

Lastly we'll declare icon.png in the web_accessible_resources array and as our icon for the extension. Declaring it as web accessible allows us to load it later on using InboxSDK.

InboxSDK

Before we can use the InboxSDK we need an AppId which we can get from here. And we'll include the inboxsdk.js file which we can download from here.

Now let's create our app.js file which will use InboxSDK to add our extension to Gmail.

// app.js

InboxSDK.loadScript('https://cdn.jsdelivr.net/npm/vue@2.5.17/dist/vue.js')
InboxSDK.loadScript('https://cdnjs.cloudflare.com/ajax/libs/axios/0.18.0/axios.min.js')

// Load InboxSDK 1.0 using our AppID
InboxSDK.load('1.0', 'INBOX_SDK_APP_ID').then(function(sdk){
  // the SDK has been loaded
  // We want to register a view handler on the Compose view
  sdk.Compose.registerComposeViewHandler(function(composeView){
    // a compose view has come into existence, do something with it!
    composeView.addButton({
      title: "Vue Pipl Search",
      iconUrl: chrome.extension.getURL('icon.png'),
      onClick: function(event) {
        sdk.Widgets.showModalView({
          title: 'Vue Pipl Search',
          'el': `<div id="vue-pipl-search"></div>`,
        });

        // new Vue(...)
      },
    });
  });
});

Once the updated version of InboxSDK is loaded with InboxSDK.load we can use sdk.Compose to register a view handler and add a button on the Compose email view that will launch our Vue component. Inside the popup we'll render a div with id='vue-pipl-search' that will be picked up by the Vue component.

Vue Component

Now we can define our Vue component. We do this in the onClick handler so that it is defined after the #vue-pipl-search element exists on the page. Also we need a sample Api key from Pipl.

In order for the Vue component to get the email's recipients, we can use InboxSDK's composeView methods. composeView.getToRecipients() will return an array of the recipients, so we can get the email address with recipients[0].emailAddress.

Putting this together we get the following.

// app.js

InboxSDK.load('1.0', 'INBOX_SDK_APP_ID').then(function(sdk){
  sdk.Compose.registerComposeViewHandler(function(composeView){
    composeView.addButton({
      // ...
      onClick: function(event) {
        // ...

        const vuePiplSearch = new Vue({
          el: '#vue-pipl-search',
          template: `
            <div>
              <template v-if="recipients.length">
                <div v-if="person" style="text-align: center;">
                  <h2 style="text-align: center">
                    {{person.names[0].display}}
                  </h2>
                  <img :src="person.images[0].url" width="80px">
                  <p v-if="person.jobs[0]">{{person.jobs[0].title}}</p>
                </div>
                <div v-else>
                  Person was not found.
                </div>
              </template>
              <div v-else>
                Add an email recipient to search Pipl Api.
              </div>
            </div>
          `,

          data() {
            return {
              recipients: composeView.getToRecipients(),
              person: null
            }
          },

          created() {
            if (this.recipients.length) {
              this.loading = true

              axios.get(`https://api.pipl.com/search/v5/?email=${this.recipients[0].emailAddress}&key=[PIPL_SAMPLE_KEY]`)
                .then(res => {
                  if (res.status === 200) {
                    this.person = res.data.person;
                    this.loading = false
                  }
                })
            }
          }
        })
      },
    });
  });
});

When the component is created we check for recipients then make a request to Pipl. We store the result in the data property which will render in the template.

This is super simple, but we can expand on it to add error handling or support for multiple recipients.

If we want to use a different Api or have a different use case we can use methods like composeView.getHTMLContent() to get the email body and send that to an Api.

Check out the docs for more ideas.

Loading Extension

To run our extension we need to load it. We can zip our project and load it that way, but for this tutorial we'll just load the unpacked folder. From Chrome's extension page select 'Load Unpacked` at the top left and navigate to the extensions folder then select okay. This will add the extension to chrome.

Now if you navigate to gmail.com and compose an email, you'll see the Vue icon.

Add "clark.kent@example.com" as the email recipient and click on the icon. A modal should open up with our Vue component in it, hooray! It searches the Pipl Api and returns the person.

Magic! Now you can begin your PI career like a boss!

Don't forget to like if you enjoyed this tutorial. Leave a comment below if you have any questions. :)

React Components In Lucky With Laravel Mix and lucky-react

Mikias Abera — Mon, 30 Jul 2018 15:56:24 +0000

I just started learning React after 2 years of Angular, and I'm surprised at how fun React is and how amazing the community and supporting packages are. I'm also a huge fan of Crystal and the Lucky framework, so what could be more awesome than using these tools together?

In this post I'm going to show you how you can add React components to your Lucky applications with Laravel mix and lucky-react.

Demo

To see code for this feature you can clone the lucky_demo repo and checkout the lucky-react branch.

git clone git@github.com:mikeeus/lucky_demo.git
cd lucky_demo
bin/setup
git checkout lucky-react

Otherwise you can follow along with a fresh lucky app by running lucky init app_name.

Laravel Mix

Since we'll be using jsx to write React components we need to update Laravel mix's config to compile our JavaScript correctly. Laravel mix makes it easy to set it up by changing our mix.js() call in the configuration to mix.react().

// webpack.mix.js
mix
  // ...
  .react("src/js/app.js", "public/js") // instead of .js(...)

Babel Plugin: transform-class-properties (Optional)

In order to use arrow functions and other awesome syntax we need babel's transform-class-properties plugin. Laravel mix does not come with this plugin by default so we need to install it.

yarn add babel-plugin-transform-class-properties

Then we add a .babelrc file in the root of our project with the following content:

{
  "plugins": [
    "transform-class-properties"
  ]
}

This will be picked up by mix automatically! Dope.

Writing React Components

Now that we have support for jsx, we can write components and import them into our app.js file. For organization I put my components in the src/js/components/ directory.

Here is the Bordered component and a simplified version of the Chat component that are used in the demo app.

// src/js/components/Bordered.jsx
import React from 'react';

export class Bordered extends React.Component {  
  render() {
    return (
      <div style={{border: '2px solid'}}>
        {this.props.children}
      </div>
    )
  }
}

// src/js/components/Chat.jsx
import React from 'react';

import { ChatInput } from './ChatInput';
import { Message } from './Message';

export class Chat extends React.Component {
  ...
  render() {
    return (
      <div style={styles}>
        <h2 style={styles}>Conversation</h2>
        <div>
          {
            this.state.messages.map(message => 
              <Message
                key={message.id}
                sender={message.sender}
                text={message.text} />
            )
          }
        </div>
        <ChatInput writeMessage={this.onWriteMessage}/>
      </div>
    )
  }
}

LuckyReact

To allow rendering React components in Lucky apps I've created an npm module called lucky-react that adds event listeners on turbolinks:load and turbolinks:before-render to mount and unmount components using [data-react-class] and [data-react-props] attributes.

I've also created a crystal shard called lucky_react with helper methods for rendering elements with these attributes in your Lucky pages.

lucky-react npm module: Finds and renders React components on your pages using [data-react-class] and [data-react-props] attributes.
lucky_react crystal module: Adds helper methods for rendering elements with the right attributes so they can be found by lucky-react.

Lets use them now to render our Chat and Bordered components on our home page.

First install the npm module.

yarn add lucky-react

Then add the shard to shard.yml and run shards.

# shard.yml
...
  dependencies:
    ...
    lucky_react:
      github: mikeeus/lucky_react

Registering React Components

In order for LuckyReact to render our components we need to import and register them in our app.js file.

// src/js/app.js
...
import LuckyReact from "lucky-react";

import { Chat } from './components/Chat';
import { Bordered } from './components/Bordered';

LuckyReact.register({ Chat, Bordered });

That's all we need to do! LuckyReact will create event listeners on turbolinks:load and turbolinks:before-render to mount and unmount these components if it finds them on the page.

Note that we only need to register Chat and Bordered since they are the only root components.ChatInput and Message are nested within Chat and will be handled automatically by React.

Rendering Components on Pages

Now in our Pages we can use the LuckyReact crystal module to add elements which reference our components so they can be rendered.

# src/pages/home/index_page.cr
require "lucky_react"

class Home::IndexPage < GuestLayout
  include LuckyReact # include the module

  def content
    react "Bordered" do # call react
      h1 "React Component", style: "text-align: center;"
    end

    messages = [
      { id: 1, sender: "me", text: "Hi" },
      { id: 2, sender: "Chatbot", text: "Hi! How can I help?" },
      { id: 3, sender: "me", text: "Can you tell me the time?" },
      { id: 4, sender: "Chatbot", text: "Sure it's #{Time.now}" }
    ]

    react "Chat", { messages: messages } # with props
  end
end

We can render components without the lucky_react shard by adding the [data-react-class] and [data-react-props]. The above example would then be written like this:

# src/pages/home/index_page.cr
class Home::IndexPage < GuestLayout
  def content
    div  "data-react-class": "Bordered" do
      h1 "React Component", style: "text-align: center;"
    end

    messages = [
      { id: 1, sender: "me", text: "Hi" },
      { id: 2, sender: "Chatbot", text: "Hi! How can I help?" }
    ]

    div "data-react-class": "Chat", "data-react-props": ({ messages: messages }).to_json
  end
end

If you run the app now and visit the home page you'll see the chat component working!

Join Us

I hope you enjoyed this tutorial and found it useful. Join us on the Lucky gitter channel to stay up to date on the framework or checkout the docs for more information on how to bring your app idea to life with Lucky.

Uploading and Validating Images with Crystal and Lucky on Heroku

Mikias Abera — Sun, 29 Jul 2018 14:20:31 +0000

Today we're going to be uploading images with Lucky and crystal! To demo this I'm going to make an app that allows uploading images to a gallery that is tied to your ip address. Since we're going to host this app on heroku, we can use heroku's X-FORWARDED_FOR header to get the user's ip address.

Note that because of proxys and the potential for ip spoofing, this is not a secure method of restricting user access and I won't recommend using it for any important data.

Finished Code

To see the finished code and run it locally you can clone the repo and checkout the image-uploads branch.

git clone git@github.com:mikeeus/lucky_demo.git
git checkout image-uploads

cd image-uploads
bin/setup
lucky db.create && lucky db.migrate

And you can run the specs to see the beautiful green result :).

crystal spec spec/flows/images_spec.cr

Image Table

First lets create the Image table.

lucky gen.migration CreateImages

We'll add the following columns to hold the filename, ip address of the owner and we'll even record the number of times the image is viewed by users.

class Image::V20180728000000 < LuckyMigrator::Migration::V1
  def migrate
    create :images do
      add filename : String
      add owner_ip : String
      add views : Int32
    end

    execute "CREATE INDEX owner_ip_index ON images (owner_ip);"
    execute "CREATE UNIQUE INDEX filename_index ON images (filename);"  
  end

  def rollback
    drop :images
  end
end

We'll also add a unique index on the filename and a normal index on the owner_ip column so we can quickly get collections of images based on it.

Specs

When allowing uploads to our app we'll want to restrict the files by type and possible dimensions. We'll create specs to check this for us. Unfortunately Crystal doesn't give us information on an image's dimensions our of the box, so we'll later we'll use crymagic to get this info for us.

The limits we'll put on our uploads are the following:

formats: JPG, GIF, PNG
max dimensions: 1000x1000
max size: 250kb

I've added some images to our assets folder that break each of these rules as well as one image that is perfect.

ALSO: I got these images from this amazing site: africandigitalart.com, which I recommend checking out.

public/
  assets/
    images/
      test/
        perfect_960x981_56kb.jpg
        too_big_900x900_256kb.jpg
        too_tall_1001x1023_95kb.jpg
        wrong_format_240x245_235kb.bmp

Next we'll create an ImageBox in case we need to instantiate Images in our tests.

# spec/support/boxes/image_box.cr
class ImageBox < LuckyRecord::Box
  def initialize
    filename "perfect_960x981_56kb.jpg"
    owner_ip "0.0.0.0"
    views 1
  end
end

Lucky Flow

Lucky uses the concept of Flows which are classes that encapsulate the behavior of your browser tests. We'll create one now that uploads an image on our homepage and has two methods for checking if it succeeded or not.

We can simulate uploading a file by adding the file's full path to the file input of the form. Then theclick "@upload-image" method will look for an element with [flow_id=upload-image] tag on the page and click it.

# spec/support/flows/images_flow.cr
class ImagesFlow < BaseFlow
  def upload_image(filepath)
    visit Home::Index

    fill_form ImageForm,
      image: File.expand_path(filepath)
    click "@upload-image"
  end

  def image_should_be_created(filepath)
    image = find_image_by_filename?(File.basename(filepath))
    image.should_not be_nil
  end

  def image_should_not_be_created(filepath)
    image = find_image_by_filename?(File.basename(filepath))
    image.should be_nil
  end

  private def find_image_by_filename?(filename)
    ImageQuery.new.filename.ilike("%#{filename}%").first?
  end
end

Now we can use this flow and our test images to write our specs. Crystal has first class support for specs and we can see that by how simple it is to write them. We use Spec.after_each to clear the images with a delete! method that will also delete the underlying file after every spec.

# specs/flows/images_spec.cr
require "../spec_helper"

describe "Images flow" do
  Spec.after_each do
    ImageQuery.new.map(&.delete!)
  end

  describe "uploading" do
    it "works with valid image" do
      flow = ImagesFlow.new

      flow.upload_image(valid_image_path)
      flow.image_should_be_created(valid_image_path)
    end

    it "doesnt work with image above 250kb" do
      flow = ImagesFlow.new

      flow.upload_image(too_big_image_path)
      flow.image_should_not_be_created(too_big_image_path)
    end

    it "doesnt work with dimensions over 1000x1000" do
      flow = ImagesFlow.new

      flow.upload_image(too_tall_image_path)
      flow.image_should_not_be_created(too_tall_image_path)
    end

    it "doesnt work with image of the wrong format" do
      flow = ImagesFlow.new

      flow.upload_image(wrong_format_image_path)
      flow.image_should_not_be_created(wrong_format_image_path)
    end
  end
end

private def valid_image_path
  "public/assets/images/test/perfect_960x981_56kb.jpg"
end

private def too_tall_image_path
  "public/assets/images/test/too_tall_1001x1023_95kb.jpg"
end

private def too_big_image_path
  "public/assets/images/test/too_big_900x900_256kb.jpg"
end

private def wrong_format_image_path
  "public/assets/images/test/wrong_format_240x245_235kb.bmp"
end

Running these specs will cause them to fail since we haven't implemented anything. Let's now build out our models, actions and pages to make them work.

Image Model

We'll need to persist references to our images, their owner's ip and number of views to the database. So let's generate a model to do that.

lucky gen.model Image

And we can fill out the Image model with its columns and a some helper methods to build the path, url and handle deletion. The images will be saved at public/assets/images/..., and will be available publicly at at www.example.com/assets/images/.... We'll also add a case for test images that will be stored under the public/assets/images/test/ directory.

# src/models/image.cr
class Image < BaseModel
  table :images do
    column filename : String
    column owner_ip : String
    column views : Int32
  end

  def url
    "#{Lucky::RouteHelper.settings.base_uri}#{path}"
  end

  def path
    if Lucky::Env.test?
      "/assets/images/test/#{self.filename}"
    else
      "/assets/images/#{self.filename}"
    end
  end

  def full_path
    "public#{path}"
  end

  def delete!
    File.delete(full_path)
    delete
  end
end

Next we can fill out our ImageForm. Forms in Lucky are responsible for creating and updating models. We use fillable to declare which columns we'll be updating, and we'll declare a virtual field image to hold our uploaded image until we can save it. We'll also add needs file and needs ip because we'll be passing these in when instantiating the form.

uuid is used to make sure we have unique filenames and make it almost impossible for someone to view the image without the filename.

We put all of this together in the prepare method which saves the image and sets the columns. It currently doesn't do any validations but we'll get to that later.

require "uuid"

class ImageForm < Image::BaseForm
  fillable filename
  fillable views
  fillable owner_ip

  virtual image : String

  needs file : Lucky::UploadedFile, on: :create
  needs ip : String, on: :create

  getter new_filename

  def prepare
    save_image
    views.value = 1
    filename.value = new_filename
    owner_ip.value = ip
  end

  private def uploaded
    file.not_nil!
  end

  private def save_image
    File.write(save_path, File.read(uploaded.tempfile.path))
  end

  private def new_filename
    @new_filename ||= "#{UUID.random}_#{uploaded.metadata.filename}"
  end

  private def image_path
    if Lucky::Env.test?
      "assets/images/test/" + new_filename
    else
      "assets/images/" + new_filename
    end
  end

  private def save_path
    "public/" + image_path
  end
end

Now we need to create the UI to allow uploads and the actions to save the forms.

Home Page

Currently our app displays Lucky's default homepage. We'll create a new Home page that holds our form and allow us to upload files. Let's generate the page.

lucky gen.page Home::IndexPage

Then we'll add a form that has enctype: "multipart/form-data" and posts to Images::Create which will handle creating our Image. We add needs form : ImageForm to tell the action that renders this page to pass in a new form. We'll also render any errors in a list below the input.

class Home::IndexPage < GuestLayout
  needs form : ImageForm

  def content
    render_form(@form)
  end

  private def render_form(f)
    form_for Images::Create, enctype: "multipart/form-data" do
      text_input f.image, type: "file", flow_id: "file-input"      

      ul do
        f.image.errors.each do |err|
          li "Image #{err}", class: "error"
        end
      end

      submit "Upload Image", flow_id: "upload-image"
    end
  end
end

And let's change our Home::Index action to show our index page rather than Lucky's welcome page.

# src/actions/home/index.cr
class Home::Index < BrowserAction
  include Auth::SkipRequireSignIn
  unexpose current_user

  get "/" do
    if current_user?
      redirect Me::Show
    else
      render Home::IndexPage
    end
  end
end

Get current_ip in Actions

We won't be using current_user for authentication, instead we need to get the ip address of the request. When our app is on heroku we can use the X-FORWARDED-FOR header which is set automatically. Locally we'll just set it to local.

We'll add these methods in the BrowserAction. Since our other actions inherit from it class Home::Index < BrowserAction, it will make these methods available for us.

# src/actions/browser_action.cr
abstract class BrowserAction < Lucky::Action
  ...
  def current_ip
    current_ip?.not_nil!
  end

  private def current_ip?
    if Lucky::Env.production?
      context.request.headers["X-FORWARDED-FOR"]?
    else
      "local"
    end
  end
  ...
end

Images Create Action

Now we need an action to handle the image creation after we submit the form on the home page. Let's generate one with:

lucky gen.action.browser Images::Create

For more information on how actions work, you can check out Lucky's guides.

This action will get the file from the params which will be in the form { "image": { "image": "file is here" }}. If it's not nil we'll pass the file as well as the current_ip to the ImageForm which will validate and save our new Image.

To check that our file exists we'll make sure its not nil and that the filename exists.

# src/actions/images/create.cr
class Images::Create < BrowserAction
  include Auth::SkipRequireSignIn
  unexpose current_user

  route do # lucky expands this to: post "/images"
    file = params.nested_file?(:image)["image"]?

    if is_invalid(file)
      flash.danger = "Please select a file to upload"
      redirect to: Home::Index
    else
      ImageForm.create(file: file.not_nil!, ip: current_ip) do |form, image|
        if image
          flash.success = "Image successfuly uploaded from #{current_ip}!"
          redirect to: Home::Index
        else
          flash.danger = "Image upload failed"
          render Home::IndexPage, form: form
        end
      end
    end
  end

  private def is_invalid(file)
    file.nil? || file.metadata.filename.nil? || file.not_nil!.metadata.filename.not_nil!.empty?    
  end
end

And voila! Our app can now handle image uploads.

If we run the specs with lucky spec spec/flows/images_spec.cr we'll see that our first spec that checks valid images will pass, but since we haven't implemented image validations the rest will fail.

Validations

In order to check the images' file size, type and dimensions we're going to use a little gem of a shard called crymagick. It requires having ImageMagick installed which luckily for us is present on Heroku by default. If it's not installed on your local machine you can get it from the official site here.

Lets install the shard by adding it to the bottom of our dependencies in shard.yml and running shards.

# shard.yml
...
dependencies:
  ...
  crymagick:
    github: imdrasil/crymagick

Now we can use it in our ImageForm to validate our images. We add three methods validate_is_correct_size, validate_is_correct_dimensions and validate_is_correct_type that will use CryMagick::Image to check the file's type, size and dimensions. If there are no errors, we move on to saving the file and setting the Image's columns.

require "uuid"
require "crymagick"

class ImageForm < Image::BaseForm
  ...
  getter crymagick_image : CryMagick::Image?

  def prepare
    validate_is_correct_size
    validate_is_correct_dimensions
    validate_is_correct_type

    if errors.empty? # save if validations pass
      save_image

      views.value = 1
      filename.value = new_filename
      owner_ip.value = ip
    end
  end

  private def validate_is_correct_type
    ext = crymagick_image.type

    unless Image::VALID_FORMATS.includes? "#{ext}".downcase
      image.add_error "type should be jpg, jpeg, gif or png but was #{ext}"
    end
  end

  private def validate_is_correct_size
    size = crymagick_image.size # returns size in bytes

    if size > 250_000 # 250kb limit
      image.add_error "size should be less than 250kb but was #{size / 1000}kb"
    end
  end

  private def validate_is_correct_dimensions
  dimensions = crymagick_image.dimensions # returns (width, height)

    if dimensions.first > 1000
      image.add_error "width should be less than 1000px, but was #{dimensions.first}px"
    end

    if dimensions.last > 1000
      image.add_error "height should be less than 1000px, but was #{dimensions.last}px"
    end
  end

  private def crymagick_image # To avoid opening the file multiple times
    @crymagick_image ||= CryMagick::Image.open(uploaded.tempfile.path)
  end
  ...
end

Now if we run the specs we'll see that they all pass! Hurray!

All thats left now is to add support for deleting and viewing our images.

Displaying and Deleting Images

What we want is to display our images on the home page as a gallery. Each image should have a button to delete and should display it's url.

Let's begin with a spec that visits the homepage and checks for images on the screen, and another one that clicks the delete button and checks that the image is deleted.

# specs/flows/images_spec.cr
require "../spec_helper"

describe "Images flow" do
  ...
  describe "displays" do
    it "own images" do
      flow = ImagesFlow.new

      owned = ImageBox.new.owner_ip("local").create
      not_owned = ImageBox.new.owner_ip("not-owned").create

      flow.homepage_should_display_image(owned.id)
      flow.homepage_should_not_display_image(not_owned.id)
    end
  end

  describe "deleting" do
    it "is allowed for owner" do
      flow = ImagesFlow.new

      flow.upload_image(valid_image_path)

      image = ImageQuery.new.first

      flow.delete_image_from_homepage(image.id)
      flow.image_should_not_exist(image.id)
    end

    it "is not allowed for other ip addresses" do
      flow = ImagesFlow.new
      not_owned = ImageBox.new.owner_ip("not-local").create

      flow.delete_image_from_action(not_owned.id)
      flow.image_should_exist(not_owned.id)
    end
  end
end
...

And lets add the flows that will visit the homepage, check for images, check for images in the database, and delete images by pressing buttons or visiting the actions directly.

class ImagesFlow < BaseFlow
  ...
  def homepage_should_display_image(id)
    visit Home::Index
    image(id).should be_on_page    
  end

  def homepage_should_not_display_image(id)
    visit Home::Index
    image(id).should_not be_on_page    
  end

  def delete_image_from_homepage(id)
    visit Home::Index
    click "@delete-image-#{id}"
  end

  def delete_image_from_action(id)
    visit Images::Delete.with(id: id)
  end

  def image_should_exist(id)
    ImageQuery.find(id).should_not be_nil
  end

  def image_should_not_exist(id)
    ImageQuery.new.id(id).first?.should be_nil    
  end

  ...

  private def image(id)
    el("@image-#{id}")
  end

Our tests will be failing now, so lets add support for displaying images by updating our Home::IndexPage. We'll require that the page is rendered with an images prop that is an ImageQuery. Then we'll use the images in a new gallery method that renders each image including links to delete it and a url to display it.

class Home::IndexPage < GuestLayout
  needs form : ImageForm
  needs images : ImageQuery # ADD THIS!

  def content
    div class: "homepage-container" do
      render_form(@form)

      gallery # add gallery erhere
    end
  end

  private def gallery # define it here
    h2 "Image Gallery"

    ul class: "image-gallery" do
      @images.map do |image|
        li class: "image", flow_id: "image-#{image.id}" do
          div class: "picture", style: "background-image: url(#{image.path});" do
            div "Views: #{image.views}", class: "views"
          end

          link to: Images::Delete.with(image.id), flow_id: "delete-image-#{image.id}" do
            img src: asset("images/x.png")
          end

          div image.url, class: "image-url",  flow_id: "image-url-#{image.id}"
        end
      end
    end
  end
  ...
end

I've also added styles to src/css/app.scss which I won't include in this article.

In order for this to work we need to update our actions that render the Home::IndexPage so that they pass in the images.

# src/actions/home/index.cr
class Home::Index < BrowserAction
  ...

  get "/" do
    if current_user?
      redirect Me::Show
    else
      images = ImageQuery.new.owner_ip(current_ip)
      render Home::IndexPage, form: ImageForm.new, images: images # pass it in here
    end
  end
end

And in our Images::Create action.

# src/actions/images/create.cr
class Images::Create < BrowserAction
  ...
  route do
    if is_invalid(file)
      ...
    else
      ImageForm.create(file: file.not_nil!, ip: current_ip) do |form, image|
        if image
          ...
        else
          ...
          images = ImageQuery.new.owner_ip(current_ip)
          render Home::IndexPage, form: form, images: images # pass it in here
        end
      end
    end
  end
  ...
end

All set! The Home::IndexPage won't complain about not having images passed in. But it will complain about a link to Images::Delete which hasn't been implemented. So let's do that now.

lucky gen.action.browser Images::Delete

The Images::Delete action should check if the current_ip matches the Image's owner_ip and if so call delete!.

# src/actions/images/delete.cr
class Images::Delete < BrowserAction
  include Auth::SkipRequireSignIn
  unexpose current_user

  route do # expands to: delete "/images/:id"
    image = ImageQuery.find(id)

    if image.owner_ip == current_ip
      image.delete!
      flash.success = "Image succesfully deleted!"
      redirect to: Home::Index
    else
      flash.danger = "You are not authorized to delete this image"
      redirect to: Home::Index
    end
  end
end

Now run the tests and... Boom! Green.

Show Single Image

The last thing we'll implement is a show page for each image that updates the number of views. Let's generate the action, page and a form to update images for us.

lucky gen.action.browser Images::Show
lucky gen.page Images::ShowPage

touch src/forms/image_views_form.cr # no generater for forms atm

The form will be simple and only be used for incrementing the value. It can be used like this: ImageViewsForm.update!(image).

# src/forms/image_views_form.cr
class ImageViewsForm < Image::BaseForm
  fillable views
  fillable filename
  fillable owner_ip

  def prepare
    views.value = views.value.not_nil! + 1
  end
end

For our action we'll use a custom route so that our route parameter is available as filename instead of id. Then we check that it exists and increment the views and render the page, otherwise we redirect to the Home::Index action.

# src/actions/images/show.cr
class Images::Show < BrowserAction
  include Auth::SkipRequireSignIn
  unexpose current_user

  get "/images/:filename" do
    image = ImageQuery.new.filename(filename).first?
    if image.nil?
      flash.danger = "Image with filename: #{filename} not found"
      redirect to: Home::Index
    else
      ImageViewsForm.update!(image)
      render Images::ShowPage, image: image
    end
  end
end

The show page will be very simple. We display the filename, the views and the image using minimal style to keep everything centered while allowing the image to stretch to its full size.

# src/pages/images/show_page.cr
class Images::ShowPage < GuestLayout
  needs image : Image

  def content
    div style: "text-align: center;" do
      h1 @image.filename
      h2 "Views: #{@image.views}"
      img src: @image.path, style: "max-width: 100%; height: auto;"
    end
  end
end

To finish off we'll make the image displayed on the home page link to the Images::ShowPage.

class Home::IndexPage < GuestLayout
  ...
    ul class: "image-gallery" do
      @images.map do |image|
        li class: "image", flow_id: "image-#{image.id}" do
          link to: Images::Show.with(image.filename), # Changed this to link: ..
               class: "picture",
               style: "background-image: url(#{image.path});" do
            div "Views: #{image.views}", class: "views"
          end
          ...
        end
      end
    end
  ...
end

And we're done! The tests should all be green and the app working as expected.

Join Us

Using Materialized Views as Models in Lucky

Mikias Abera — Fri, 27 Jul 2018 12:36:57 +0000

I've got an app with over 2 million rows in a Postgresql database and I make complex queries with joins, unions and aggregate functions that take a lot of time. The results of some of these queries are used a lot, for example to render a cloropleth map on the app's homepage that changes based on filters on the map. This is a super expensive operation and can't scale!

In come Materialized Views to the rescue. A materialized view is a query that is persisted like a table and treated the same way. This means queries on this table are fast. No joins, or unions, just like querying any other table. Note that it needs to be refreshed to keep it up to date so it is not suited for realtime data.

Although we can map queries on our materialized view to a plain crystal class using DB.mapping, we would miss out on Lucky's awesome typesafe queries. So in this article we're going to look at treating materialized views just like normal tables and having Lucky model them for us!

ComplexPost

The sql for the query we'll be materializing is below. Later we'll make slight adjustments to it so it can play nicely with LuckyRecord::Model.

SELECT
  posts.id,
  posts.title,
  posts.content,
  users.name as author
FROM posts
JOIN users
ON users.id = posts.user_id

This isn't a complex query but the way we implement this feature can be expanded to include any query regardless of complexity. The only thing that matters is that the columns returned by our SELECT statement match our LuckyRecord model definition.

Setup

If you want to test this out yourself you can use my demo app, just clone the repo and checkout the matviews-0 branch to follow along, or matviews-3-complete to see the finished code.

git clone git@github.com:mikeeus/lucky_api_demo.git
cd lucky_api_demo
bin/setup
git checkout matviews-0

Create Spec

git checkout matviews-0

Although we don't have a materialized view or a model to access it, we're going to write a spec for how we want to be able to use it.

# spec/matview_spec.cr
require "./spec_helper"

describe App do
  describe "ComplexPost matview" do
    it "should refresh and query" do
      user = UserBox.new.create
      first_post = PostBox.new.title("First").user_id(user.id).create
      second_post = PostBox.new.title("Second").user_id(user.id).create

      ComplexPost.refresh

      complex = ComplexPostQuery.new.title(first_post.title).first
      complex.title.should eq first_post.title
    end
  end
end

We want to be able to refresh the materialized view using the model, and we want to query the view with Lucky's own queries which will generate helper methods for each column, eg: ComplexPostQuery.new.title(...), ComplexPostQuery.new.content.ilike(...), etc.

Create Materialized View

git checkout matviews-1-create-matview

First lets generate a migration with lucky gen.migration CreateComplexPostMatview.

class CreateComplexPostsMatview::V20180101010 < LuckyMigrator::Migration::V1
  def migrate
    execute <<-SQL
      CREATE MATERIALIZED VIEW complex_posts AS
        SELECT
          posts.id,
          posts.title,
          posts.content,
          users.name as author,
          NOW() as created_at,
          NOW() as updated_at
        FROM posts
        JOIN users
        ON users.id = posts.user_id
    SQL

    execute "CREATE UNIQUE INDEX complex_posts_id_index ON complex_posts (id)"
  end

  def rollback
    execute "DROP MATERIALIZED VIEW complex_posts"
  end
end

We need to have id, created_at and updated_at columns because LuckyRecord::Model will expect these when it generates helper methods for us. In this case we actually use id and even add a unique index on it to speed up our queries, but I should note that neither an id column or a primary key is needed for a materialized view.

Now if we migrate with lucky db.migrate it should pass without issue.

Generate ComplexPost Model and Query

git checkout matviews-2-create-model

Now that we have the materialized view in the database we could define a ComplexPost class that calls DB.mapping (like in my previous post) and just query it with LuckyRecord::Repo like this.

LuckyRecord::Repo.run do |db|
  db.query_all "SELECT * FROM complex_posts", as: ComplexPost
end

But that wouldn't give us the benefits of LuckyRecord::Model or ComplexPost::Query.

So instead we'll generate our ComplexPost model by running lucky gen.model ComplexPost. We can go ahead and delete the generated ComplexPost::Form because we'll never be inserting rows into our materialized view.

Let's fill it in as if it was a normal database model and add a class method to refresh the materialized view.

# src/models/complex_post.cr
class ComplexPost < BaseModel
  table :complex_posts do
    column title : String
    column content : String
    column author : String
  end

  def self.refresh
    LuckyRecord::Repo.db.exec "REFRESH MATERIALIZED VIEW complex_posts"
  end
end

Run The Specs

git clone matviews-3-complete

Run crystal spec spec/matviews_spec.cr and... BAM! It's working.

Although this was a simple example we can expand on it to account for complex queries. An example of a query that I use in my app is this.

class CreateCountryAnnualTradeMatview::V0000001 < LuckyMigrator::Migration::V1
  def migrate
    execute <<-SQL
    CREATE MATERIALIZED VIEW country_annual_trades AS
      SELECT
        merged.id, -- FOR LUCKY
        merged.name,
        merged.short,
        merged.year,
        COALESCE(sum(merged.total_imports_cents))::bigint as total_imports_cents,
        COALESCE(sum(merged.total_exports_cents))::bigint as total_exports_cents,
        NOW() as created_at, -- FOR LUCKY
        NOW() as updated_at -- FOR LUCKY
      FROM (
        ( SELECT ... ) -- some complex query
        UNION
        ( SELECT ... ) -- another complex query
      ) merged
      GROUP BY id, name, short, year
      ORDER BY name
    SQL

    execute <<-SQL
      CREATE UNIQUE INDEX country_annual_trades_id_year -- we can even add indexes!
        ON country_annual_trades(id, year)
    SQL
  end

  def rollback
    execute <<-SQL
      DROP MATERIALIZED VIEW country_annual_trades
    SQL
  end
end

And the model.

class CountryAnnualTrade < BaseModel
  table :country_annual_trades do
    column name : String
    column short : String
    column year : Int32
    column total_imports_cents : Int64
    column total_exports_cents : Int64
  end

  def self.refresh
    LuckyRecord::Repo.db.exec "REFRESH MATERIALIZED VIEW country_annual_trades"
  end
end

Note that I cast my aggregated columns total_imports_cents and total_exports_cents to ::bigint then define them in the model as Int64 with column total_imports_cents : Int64.

Join Us

Mapping Database Queries to Crystal Classes With crystal-db and Lucky

Mikias Abera — Thu, 26 Jul 2018 05:01:07 +0000

Sometimes we need to get a couple columns from our database, or make complex queries and return many columns that don't fit into our models. In these cases we want the framework we use to be flexible enough to allow such queries and make it easy to use the results in our app. Crystal and Lucky let us do just that.

In this post we'll look at how to use crystal-db's DB.mapping macro to map database queries to generic Crystal classes. Then we'll quickly look at how Lucky uses DB.mapping internally.

In this article we'll be using Lucky to make the database queries, but remember that crystal-db can be used alone or with any framework.

Setup

If you want to test this out yourself you can use my demo app, just clone the repo and checkout the db-mapping-0 to follow along, or db-mapping-1-complete to see the finished code.

git clone git@github.com:mikeeus/lucky_api_demo.git
cd lucky_api_demo
bin/setup
git checkout db-mapping-0

The Query

For this example we'll map this fairly simple query which fetches posts, joins users on user_id and return the user's name and email as a JSON object. Since Lucky uses the crystal-pg Postgresql driver, we can use DB.mapping to easily parse json objects from our query into JSON::Any.

SELECT
  posts.id,
  posts.title,
  ('PREFIX: ' || posts.content) as custom_key, -- custom key for fun
  json_build_object(
    'name', users.name,
    'email', users.email
  ) as author
FROM posts
JOIN users
ON users.id = posts.user_id;

The Class

crystal-db returns the results of the query as DB::ResultSet which isn't directly useful for us. So lets create the class that the result will be mapped to, and we can use the DB.mapping to handle the dirty work.

class CustomPost
  DB.mapping({
    id: Int32,
    title: String,
    content: {
        type: String,
        nilable: false,
        key: "custom_key"
    },
    author: JSON::Any
  })
end

Essentially the mapping macro will create a constructor that accepts a DB::ResultSet and initializes this class for us, as well as a from_rs class method for intializing multiple results. It would expand to something like this.

class CustomPost
  def initialize(%rs : ::DB::ResultSet)
    # ...lots of stuff here
  end

  def self.from_rs(rs : ::DB::ResultSet)
    objs = Array(self).new
    rs.each do
      objs << self.new(rs)
    end
    objs
  ensure
    rs.close
  end
end

Hooking It All Up

Now let's write a spec to ensure everything is working as planned.

# spec/mapping_spec.cr
require "./spec_helper"

describe App do
  describe "CustomPost" do
    it "maps query to class" do
      user = UserBox.new.name("Mikias").create
      post = PostBox.new
                    .user_id(user.id)
                    .title("DB mapping")
                    .content("Post content")
                    .create

      sql = <<-SQL
        SELECT
          posts.id,
          posts.title,
          ('PREFIX: ' || posts.content) as custom_key,
          json_build_object(
            'name', users.name,
            'email', users.email
          ) as author
        FROM posts
        JOIN users
        ON users.id = posts.user_id;
      SQL

      posts = LuckyRecord::Repo.run do |db|
        db.query_all sql, as: CustomPost
      end

      posts.size.should eq 1
      posts.first.title.should eq post.title
      posts.first.content.should eq "PREFIX: " + post.content
      posts.first.author["name"].should eq user.name
    end
  end
end

class CustomPost
  DB.mapping({
    id: Int32,
    title: String,
    content: {
        type: String,
        nilable: false,
        key: "custom_key"
    },
    author: JSON::Any
  })
end

We can run the tests with lucky spec spec/mapping_spec and... green! Nice.

Lucky Models

This is actually very similar to how LuckyRecord sets up it's database mapping. For example if you have a User model like this.

class User < BaseModel
  table :users do
    column name : String
    column email : String
    column encrypted_password : String
  end
end

Calls to the column method will add the name and type of each column to a FIELDS constant.

macro column(type_declaration, autogenerated = false)
  ... # check type_declaration's data_type and if it is nilable
  {% FIELDS << {name: type_declaration.var, type: data_type, nilable: nilable.id, autogenerated: autogenerated} %}
end

The table macro will setup the model, including calling the setup_db_mapping macro which will call DB::mapping by iterating over the FIELDS.

macro setup_db_mapping
  DB.mapping({
    {% for field in FIELDS %}
      {{field[:name]}}: {
        {% if field[:type] == Float64.id %}
          type: PG::Numeric,
          convertor: Float64Convertor,
        {% else %}
          type: {{field[:type]}}::Lucky::ColumnType,
        {% end %}
        nilable: {{field[:nilable]}},
      },
    {% end %}
  })
end

Just like that each of your Lucky models can now be instantiated from DB::ResultSet and have a from_rs method that can be called by your queries. Pretty simple right?

Join Us

Deploying Angular and Lucky Monorepo Using Netlify and Heroku

Mikias Abera — Thu, 12 Jul 2018 19:29:43 +0000

I like to keep my client and server together because it makes me more efficient by allowing me to open both apps in a single VS Code workspace. The difficulty comes when deploying the apps separately, but luckily we can use services like Netlify and Heroku to simplify that for us!

As a reference, my blog is made with a JSON Api built with Lucky and a front end built with Angular. You can look at the source code on github for an example of how to deploy these apps together, use nrwl and ngrx with Angular or build and test my Lucky apis.

The folder structure is something like this:

/client         # Angular Front End
/server         # Lucky Api Back End
/bin
  /deploy_server

Lucky on Heroku

Heroku doesn't allow using a non root directory for publishing your applications. I got around this by using git subtree method that I learned from this post on coderwall which allows us to deploy a sub directory of our git repository.

Prepare for Deploy

First create the heroku app, add the buildpack and set the environment variables necessary for the Lucky app to run on Heroku. You can find full instructions in the Lucky Guides, the only difference being that for the Lucky Api you only need the crysal buildpack.

Deploy!

Now the key difference is in how we deploy to heroku. Instead of git push heroku master we're going to use git subtree to only push a subdirectory of our repo.

git subtree push --prefix server heroku master

Change the --prefix parameter to the name of your sub-directory.

I put that command in a file at bin/deploy_server and made it executable with chmod u+x for convenience.

Angular on Netlify

Netlify is easily my favorite service for deploying static applications. I use it for all my Angular apps and it has great features even at the free tier.

Hooking Up to Git

To deploy all we need to do is log in to netlify, go to the sites page at app.netlify.com/account/sites and select the "New Site from Git" button.

This will prompt you to log in with Github or Bitbucket, then select a repository to link to. One done, it will watch for push events to the repo and deploy your application automatically, using magic, of course.

Joking. There is no such thing as magic. We actually need to tell Netlify how to build our app and where to find the index.html to serve.

Configuring Base Directory

Since our application is in a sub-directory we could create a package.json in our root directory and use a "postinstall" script to trigger an npm install and build (eg: "postinstall": "cd client && npm install"), then use a deploy command like "build": "cd client && ng build...".

This will deploy fine the first time but due to the way netlify caches directories like node_modules, it will not cache the packages installed from the package.json in the sub-directory and subsequent deploys will fail. To get around this we need to create a netlify.toml file to set the base directory or our application, specify our build command and our publish directory. This will make netlify cache our node_modules folder correctly and allow our build scripts to run.

The file will look like this:

# netlify.toml

# Global settings applied to the whole site.  
# 
# “publish” is the directory to publish (relative to root of your repo),
# “command” is your build command,
# “base” is directory to change to before starting build. if you set base:
#    that is where we will look for package.json/.nvmrc/etc not repo root!

[build]
  base    = "client"
  publish = "client/dist/apps/portfolio"
  command = "npm run build-netlify"

Note the publish directory path needs to be set from the root, while the build command will be run from the base directory (in this case client/)

You can find more configuration options in the netlify docs.

Building the App

The npm run build-netlify actually expands to: ng build --prod --build-optimizer && cp _redirects dist/apps/portfolio. When running a single page application you need to redirect every non-root route back to your index.html, otherwise netlify will display a 404 page which we don't want.

We use a _redirects file with a simple configuration to handle this:

# client/_redirects
/* /index.html 200

Deploy!

Now to deploy our Angular app we just need to push to our git repository with: git push origin master.

Directory Structure

Our final directory structure looks like this:

/client         # Angular Front End
  /_redirects
  /package.json
  /...
/server         # Lucky Api Back End
  /...
/bin
  /deploy_server
netlify.toml

TL;DR

heroku build command:

Setup heroku using instructions from the Lucky Guides but remember not to add the nodejs buildpack
Deploy with git subtree push --prefix server heroku master

netlify:

Create a Netlify app by connecting to your apps git repo
create netlify.toml configuration with the following:

[build]
  base    = "client"
  publish = "client/dist/apps/portfolio"
  command = "npm run build-netlify"

create _redirects file in your Angular root with the following: /* /index.html 200
create a build script in your Angular app like this: "build-netlify": "ng build --prod --build-optimizer && cp _redirects dist/apps/portfolio"
deploy with: git push origin master

Join Us

JWT Auth with Lucky Api

Mikias Abera — Sun, 08 Jul 2018 10:22:47 +0000

Although Lucky is fantastic for building complete applications, I like to build my front-end in Angular so I usually use Lucky as a JSON Api. I prefer it over Rails Api because of the type checking, separation of models from forms and queries, and the way actions and routes are organized.

One feature I usually need in a JSON api is authentication, and today we'll go over setting up JWT authentication with Lucky Api.

Starter App

To start we'll be using the lucky api demo app which has User and Post models defined. Run:

git clone git@github.com:mikeeus/lucky_api_demo.git
git checkout jwt-auth-0
bin/setup

You can follow along by switching to the branches shown under the headings for each section. Or look at the finished product by checking out jwt-auth-10-complete

Dependencies

branch: jwt-auth-0

The only dependecy we'll need is a shard for jwt encoding and decoding. We can use the crystal jwt package so lets add the following to our shard.yml and run shards.

dependencies:
  jwt:
    github: crystal-community/jwt

Begin with Tests

branch: jwt-auth-01-sign-in-test

How else would we know the app is working? Aight, in spec/blog_api_spec.cr we'll add a describe block for authentication and our first test which will be for signing in.

Note I got AppVisitor from Hannes Kaeufler's blog which is a great Lucky site that I use as reference.

# spec/blog_api_spec.cr
require "./spec_helper"

describe App do
  visitor = AppVisitor.new
  ...
  describe "auth" do
    it "signs in valid user" do
      # create a user

      # visit sign in endpoint

      # check response has status: 200 and authorization header with "Bearer"
    end
  end
end

We'll user Lucky's boxes to make generating test data easy. We'll also use Authentic's generate_encrypted_password method to generate our password.

# spec/support/boxes/user_box.cr
class UserBox < LuckyRecord::Box
  def initialize
    name "Mikias"
    email "hello@mikias.net"
    encrypted_password Authentic.generate_encrypted_password("password")
  end
end

Now we can generate a user in our test and make a post request to our sign_in endpoint using its email and password. And we'll check the response for the correct status code and Authorization header.

# spec/blog_api_spec.cr
...
    it "signs in valid user" do
      # create a user
      user = UserBox.new.create

      # visit sign in endpoint
      visitor.post("/sign_in", ({
        "sign_in:email" => user.email,
        "sign_in:password" => "password"
      }))

      # check response has status: 200 and authorization header with "Bearer"
      visitor.response.status_code.should eq 200
      visitor.response.headers["Authorization"].should_not be_nil
    end
...

Now this test will fail because we don't have an action for this route or the forms to handle user creation, so let's build them.

Sign In

branch: jwt-auth-02-sign-in-form

If we generate a normal Lucky app it will come with Authentic already configured and several forms and actions will be generated for us. Currently, Lucky api configures Authentic but doesn't generate these files so we'll need to add them ourselves and update them to fit our use case.

Form

Let's start with the SignInForm which will be used to validate the user credentials, generate a token and return it in the Authorization header of the response. This form will be the same as the one generated by Authentic in new non-api apps, and we'll also need to create the form mixin FindAuthenticable which wasn't generated.

# src/forms/mixins/find_authenticable.cr
module FindAuthenticatable
  private def find_authenticatable
    email.value.try do |value|
      UserQuery.new.email(value).first?
    end
  end
end

# src/forms/sign_in_form.cr
class SignInForm < LuckyRecord::VirtualForm
  include Authentic::FormHelpers
  include FindAuthenticatable

  virtual email : String
  virtual password : String

  private def validate(user : User?)
    if user
      unless Authentic.correct_password?(user, password.value.to_s)
        password.add_error "is wrong"
      end
    else
      email.add_error "is not in our system"
    end
  end
end

Action

branch: jwt-auth-03-complete-sign-in

Following Lucky's conventions we're going to create two actions:

lucky gen.action.api SignIn::Create

These commands will generate classes at src/actions/sign_up/create.cr and src/actions/sign_in/create.cr and two post routes to /sign_up and /sign_in.

Now we'll need a way to generate tokens from our user, we'll put this method in a GenerateToken mixin because we'll use it in several of our actions.

# src/actions/mixins/auth/generate_token.cr
require "jwt"

module GenerateToken
  def generate_token(user)
    exp = 14.days.from_now.epoch
    data = ({ id: user.id, name: user.name, email: user.email }).to_s
    payload = { "sub" => user.id, "user" => Base64.encode(data), "exp" => exp }

    JWT.encode(payload, Lucky::Server.settings.secret_key_base, "HS256")
  end
end

We also need to make our User PasswordAuthenticatable for it to be used with Authentic. Optionally you can include Carbon::Emailable and the emailable method if you plan to send emails to your users on registration, password reset, etc.

# src/models/user.cr
class User < BaseModel
  include Carbon::Emailable
  include Authentic::PasswordAuthenticatable

  table :users do
    column name : String
    column email : String
    column encrypted_password : String
  end

  def emailable
    Carbon::Address.new(email)
  end
end

Now we can include GenerateToken in our SignIn action and use our SignInForm to complete the authentication.

# src/actions/auth/sign_in.cr
class SignIn::Create < ApiAction
  include GenerateToken

  route do
    SignInForm.new(params).submit do |form, user|
      if user
        context.response.headers.add "Authorization", generate_token(user)
        head 200
      else
        head 401
      end
    end
  end
end

Run the specs with crystal spec and voila! It works! :)

Sign Up

branch: jwt-auth-04-sign-up-test

I don't allow sign ups on my blog so I return head 401 for my SignIn action but of course you may want to implement it in yours. It's going to be very similar to the SignIn feature with some slight differences. Let's get to it.

Test

Let's begin by writing a test to create a user, making sure it returns the Authorization header and that we can query our new user from the database.

# spec/blog_api_spec.cr
describe App do
  ...
  describe "auth" do
    ...
    it "creates user on sign up" do
      visitor.post("/sign_up", ({
        "sign_up:name" => "New User",
        "sign_up:email" => "test@email.com",
        "sign_up:password" => "password",
        "sign_up:password_confirmation" => "password"
      }))

      visitor.response.status_code.should eq 200
      visitor.response.headers["Authorization"].should_not be_nil

      UserQuery.new.email("test@email.com").first.should_not be_nil
    end
  ...
end

Form

branch: jwt-auth-05-sign-up-form

Now our SignUpForm will need a PasswordValidations module to check the passwords, we'll create that first.

# src/forms/mixins/password_validations.cr
module PasswordValidations
  private def run_password_validations
    validate_required password, password_confirmation
    validate_confirmation_of password, with: password_confirmation
    validate_size_of password, min: 6
  end
end

With that we can build our sign up form.

# src/forms/sign_up_form.cr
class SignUpForm < User::BaseForm
  include PasswordValidations

  fillable name, email
  virtual password : String
  virtual password_confirmation : String

  def prepare
    validate_uniqueness_of email
    run_password_validations
    Authentic.copy_and_encrypt password, to: encrypted_password
  end
end

Action

branch: jwt-auth-06-complete-sign-up

With those two things done our we can create our SignUp::Create action which will look exactly the same as our SignIn::Create action. Run lucky gen.action.api SignUp::Create and fill it in:

# src/actions/sign_up/create.cr
class SignUp::Create < ApiAction
  include GenerateToken

  route do
    SignUpForm.create(params) do |form, user|
      if user
        context.response.headers.add "Authorization", generate_token(user)
        head 200
      else
        head 401
      end
    end
  end
end

Now we can run our tests and watch them pass!

Protecting Routes

Great we can sign in and sign out, but what good does that do us if we can't protect our resources based on it? Since every action in lucky inerits from ApiAction or BrowserAction, it's very straight forward to build our own AuthenticatedAction that handles getting the current user from the Authorization header and returning head 401 if it's not valid.

Test

branch: jwt-auth-07-authenticated-action-test

First let's write test to make sure our feature works as expected. Since we are creating posts with this api, lets make sure that the endpoint is protected. We'll create two specs and update an older one that will be effected by our changes.

Make sure to include the GenerateToken module in our specs so we can mock an authenticated request.

# spec/blog_api_spec.cr
describe App do
  include GenerateToken
  ...
  describe "/posts" do
    ...
    it "creates post" do
      user = UserBox.create

      visitor.post("/posts",
                   new_post_data,
                   { "Authorization" => generate_token(user) })

      visitor.response_body["title"].should eq "New Post"
    end
  end
  ...
  describe "auth" do
    ...
    it "allows authenticated users to create posts" do
      user = UserBox.create

      visitor.post("/posts",
                   new_post_data,
                   { "Authorization" => generate_token(user) })

      visitor.response_body["title"].should eq new_post_data["post:title"]
    end

    it "rejects unauthenticated requests to protected actions" do
      visitor.post("/posts", new_post_data)
      visitor.response.status_code.should eq 401
    end
  end
end

Now our tests will definitely be failing so lets build our AuthenticatedAction to make them pass.

AuthenticatedAction

In order to do so we'll need a way to get the user from the token, so lets create a mixin called UserFromToken to do just that.

Note I chose to use mixins for generating and parsing tokens but you can also include these methods directly in the user model.

# src/actions/mixins/user_from_token.cr
module UserFromToken
  def user_from_token(token : String)
    payload, _header = JWT.decode(token, Lucky::Server.settings.secret_key_base, "HS256")
    UserQuery.new.find(payload["sub"].to_s)
  end
end

Now we can use that in our AuthenticatedAction class.

# src/actions/authenticated_action.cr
abstract class AuthenticatedAction < ApiAction
  include UserFromToken

  before require_current_user

  getter current_user : User? = nil

  private def require_current_user
    token = context.request.headers["Authorization"]?

    if token.nil?
      head 401
    else
      @current_user = user_from_token(token)
    end

    if @current_user.nil?
      head 401
    else
      continue
    end
  rescue JWT::ExpiredSignatureError
    head 401
  end

  def current_user
    @current_user.not_nil!    
  end
end

So what's happening here? We use a callback before to run require_current_user before the action is called. In that method we get the user from the Authorization token and set it to the current_user getter. If there is no token, if the user doesn't exist or if the token is expired (raises JWT::ExpiredSignatureError) we return 401.

We also add a current_user method to alias our nilable getter for convenience in our actions.

Protect Actions

branch: jwt-auth-09-complete-authenticated-action

Now we can use it in our Posts::Create action.

class Posts::Create < AuthenticatedAction # changed this
  route do
    post = PostForm.create!(params, author: current_user) # and this
    json Posts::ShowSerializer.new(post), Status::Created
  end
end

Now we can run our specs... and BOOM! Protected.

That's whats up.