Forem: Mark Aiken

CLJS ❤️'s AWS Amplify

Mark Aiken — Fri, 13 Dec 2019 16:23:11 +0000

A Brief introduction

AWS Amplify is a tool from Amazon's AWS for building applications atop AWS services like Cognito, DynamoDB, and API Gateway. Its design intent is to enable small teams to quickly build full-featured applications with nothing more than an AWS account, some configuration files, and application code written in front-end technologies i.e. Javascript, Java, or Swift. In this post I show how to get started building with Amplify using Clojurescript via Shadow-CLJS. This post will gloss over a lot of the details of the pieces it weaves together in service of showing how to get to the final product, a deployed web-app that uses a simple GQL powered DynamoDB backend.

Notes:

This guide is adapted from the AWS Amplify docs located here, if you encounter problems you can look to that guide or reach out to me.

Step 0: Pre-reqs

Make an AWS Account

Because this is targeting AWS you will need an AWS account which you can create here

Setup the amplify CLI

Next is actually installing the command line toolchain (amplify). See the instructions here.

Step 1: Setup the project

In order for us to actually use our tools we need to make sure that they have everything they need to run. Our goal is to lay our project out like this:

.
├── build
│   ├── app
│   ├── index.html
│   └── js
│       ├── main.js
│       └── manifest.edn
├── deps.edn
├── package.json
├── shadow-cljs.edn
├── src
│   ├── app.cljs
│   ├── aws-exports.js
│   └── graphql
│       └── schema.json
└── yarn.lock

This file tree contains configuration for yarn/npm (package.json), shadow-cljs (shadow-cljs.edn), and clj (deps.edn) [which is used by shadow to manage our Clojure/script dependencies] and our source files that will be transpiled into the Javascript that is our app.

To setup a rough skeleton of our project run:

$ mkdir -p amplify-js-app/src amplify-js-app/public && cd amplify-js-app
$ touch package.json shadow-cljs.edn deps.edn public/index.html src/app.cljs

Add the project configuration

Package.json

{
  "name": "amplify-js-app",
  "version": "1.0.0",
  "description": "Amplify JavaScript Example",
  "dependencies": {
    "@aws-amplify/api": "latest",
    "@aws-amplify/pubsub": "latest"
  },
  "devDependencies": {
   "shadow-cljs": "^2.8.58"
  },
  "scripts": { 
    "start": "shadow-cljs watch",
    "release": "shadow-cljs release",
    "server": "shadow-cljs server"
  }
}

shadow-cljs.edn

{:source-paths ["src"]
 :deps true 
 :nrepl {:port 64824} 
 :builds {:app {:target :browser
                :output-dir "build/js"
                :asset-path "/js"
                :modules {:main {:entries [app]}} ;; <- becomes public/js/main.js
                ;; start a development http server on http://localhost:8020
                :devtools {:http-root "build"
                           :http-port 8020}}}}

deps.edn

{:deps {thheller/shadow-cljs {:mvn/version "2.8.58"}
        thheller/shadow-cljsjs {:mvn/version "0.0.12"}
        appliedscience/js-interop {:mvn/version "0.1.20"}
        binaryage/devtools {:mvn/version "0.9.10"}}
 :paths ["src"]}

Now that our app code can actually be built using our tools its time to invoke them.

Install all the things

yarn

The HTML that will be served

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8">
        <title>Amplify Framework</title>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <style>
            html, body { font-family: "Amazon Ember", "Helvetica", "sans-serif"; margin: 0; }
            a { color: #FF9900; }
            h1 { font-weight: 300; }
            .app { width: 100%; }
            .app-header { color: white; text-align: center; background: linear-gradient(30deg, #f90 55%, #FFC300); width: 100%; margin: 0 0 1em 0; padding: 3em 0 3em 0; box-shadow: 1px 2px 4px rgba(0, 0, 0, .3); }
            .app-logo { width: 126px; margin: 0 auto; }
            .app-body { width: 400px; margin: 0 auto; text-align: center; }
            .app-body button { background-color: #FF9900; font-size: 14px; color: white; text-transform: uppercase; padding: 1em; border: none; }
            .app-body button:hover { opacity: 0.8; }
        </style>
    </head>
    <body>
        <div class="app">
            <div class="app-header">
                <div class="app-logo">
                    <img src="https://aws-amplify.github.io/images/Logos/Amplify-Logo-White.svg" alt="AWS Amplify" />
                </div>
                <h1>Welcome to the Amplify Framework</h1>
            </div>
            <div class="app-body">
                <button id="MutationEventButton">Add data</button>
                <div id="MutationResult"></div>
                <div id="QueryResult"></div>
                <div id="SubscriptionResult"></div>
            </div>
        </div>
        <script src="js/main.js"></script>
    </body>
</html>

With the HTML in place we actually need to build a /js/main.js which can be done one two ways, development aka :optimizations :none and production aka :optimizations :advanced. Because of complicated reasons and compile time constraints we will just run the development builds locally.

yarn run start app #This will produce a local, source-mapped build and start the shadow-cljs interal http server for us to test with.

Step 2: The Backend

Up until now we have mostly concerned just our frontend code. Now we will configure a simple backend using amplify which takes care of a ton of the heavy lifting.

amplify init  #accept most defaults, naming your env dev

Output:

➜ amplify init
Note: It is recommended to run this command from the root of your app directory
? Enter a name for the project amplify-js-app
? Enter a name for the environment dev 
? Choose your default editor: Visual Studio Code
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using react
? Source Directory Path:  src
? Distribution Directory Path: build
? Build Command:  yarn run release app
? Start Command: yarn run start app

And then we can run amplify status to check that everything "worked" (there are no resources attached to our backend yet, it is just scaffolded out)

Output:

➜ amplify status

Current Environment: dev

| Category | Resource name | Operation | Provider plugin |
| -------- | ------------- | --------- | --------------- |

Step 4

The next step is to add our GQL API. The only thing that we need to do is run amplify add api, answer a few questions, and wait.

➜ amplify add api   
? Please select from one of the below mentioned services: GraphQL
? Provide API name: amplifyJsApp
? Choose the default authorization type for the API API key
? Enter a description for the API key: 
? After how many days from now the API key should expire (1-365): 7
? Do you want to configure advanced settings for the GraphQL API No, I am done.
? Do you have an annotated GraphQL schema? No
? Do you want a guided schema creation? Yes
? What best describes your project: Single object with fields (e.g., “Todo” with ID, name, description)

Now that we have "added" the api to our backend we need to AWS to make those actual changes. This is a common theme throughout amplify, changes are made to local configuration files or, more commonly, through the amplify CLI, and then when those changes are ready to deployed we push them to AWS via amplify push.

➜ amplify push
...
? Are you sure you want to continue? Yes
...
? Do you want to generate code for your newly created GraphQL API Yes
? Choose the code generation language target javascript
? Enter the file name pattern of graphql queries, mutations and subscriptions src/graphql/**/*.js
? Do you want to generate/update all possible GraphQL operations - queries, mutations and subscriptions Yes
? Enter maximum statement depth [increase from default if your schema is deeply nested] 2
...
✔ Generated GraphQL operations successfully and saved at src/graphql
✔ All resources are updated in the cloud

GraphQL endpoint: <snip>
GraphQL API KEY: <snip>

This command will ask us a few questions about if want leverage the codegen facilities that come with the Amplify CLI, we do.

Next we need to actually wire up the backend that we just deployed to the page we have created. We are going to base this code off the examples found in here in the Amplify docs. For brevity I won't break down what the source code is doing other than to say it is a very literal translation of the complete code in step 4.

(ns app
  (:require ["@aws-amplify/api" :refer [graphqlOperation] :default API]
            ["@aws-amplify/pubsub" :default PubSub]
            ["/graphql/mutations" :refer [createTodo]]
            ["/graphql/queries" :refer [listTodos]]
            ["/graphql/subscriptions" :refer [onCreateTodo]]
            ["/aws-exports" :default awsconfig] ;; Important Diff, no period
            [applied-science.js-interop :as j]))

;;Setup/Config
(j/call API :configure awsconfig)
(j/call PubSub :configure awsconfig)

;;Mutations

(defn create-todo []
  (let [todo        (j/obj :name "Use AppSync"
                           :description "Realtime and Offline")
        gql-todo-op (graphqlOperation createTodo (j/obj :input todo))]
    (j/call API :graphql gql-todo-op)))

(def mutation-button (j/call js/document :getElementById "MutationEventButton"))
(def mutation-result (j/call js/document :getElementById "MutationResult"))

(j/call mutation-button :addEventListener
        "click" (fn [_] 
                  (j/assoc! mutation-result :innerHTML "MUTATION RESULTS:")
                  (-> (create-todo)
                      (.then (fn [evt]
                               (let [na       (j/get-in evt [:data :createTodo :name])
                                     descript (j/get-in evt [:data :createTodo :description])]
                                 (j/update! mutation-result :innerHTML str
                                            "<p>" na "-" descript "</p>")))))))

;; Queries
(def query-result (j/call js/document :getElementById "QueryResult"))
(defn get-data
  []
  (j/assoc! query-result :innerHTML "QUERY RESULTS:")
  (->  (j/call API :graphql (graphqlOperation listTodos))
       (.then (fn [evt]
                (let [todos (j/get-in evt [:data :listTodos :items])]
                  (mapv 
                   #(let [na       (j/get-in % [:name])
                          descript (j/get-in % [:description])]
                      (js/console.log %)
                      (j/update! query-result :innerHTML str "<p>" na " - " descript "</p>"))
                   todos))))))

(get-data)

;;Subscriptions

(def subscription-result (j/call js/document :getElementById "SubscriptionResult"))
(-> (j/call API :graphql (graphqlOperation onCreateTodo))
    (.subscribe (j/obj :next 
                       (fn [evt]
                         (j/assoc! subscription-result :innerHTML "SUBSCRIPTION RESULTS:")
                         (let [na       (j/get-in evt [:value :data :onCreateTodo :name])
                               descript (j/get-in evt [:value :data :onCreateTodo :description])]
                           (j/update! subscription-result :innerHTML str "<p>" na " - " descript "</p>"))))))

This app allows us to save a todo in DynamoDB by clicking that button, includes a list of the todo's that have been created, and finally have a live updating section that updates with last todo that was created.

Step 5: Send it live

The Final step is for us to ship the app to "prod". For this we need to add the hosting resource to our amplify backend which enablesS3 Bucket based website hosting

➜ amplify add hosting
? Select the environment setup: DEV (S3 only with HTTP)
? hosting bucket name amplify-js-app-20191210185143-hostingbucket
? index doc for the website index.html
? error doc for the website index.html

You can now publish your app using the following command:
Command: amplify publish

And then publish your app! (Note that it took about 5 minutes to upload all of the files in my build folder to S3 because we never cleaned up the CLJS runtime development files, this can be fixed by modifying the script in the package.json).

➜ amplify publish
✔ Successfully pulled backend environment dev from the cloud.

Current Environment: dev

| Category | Resource name   | Operation | Provider plugin   |
| -------- | --------------- | --------- | ----------------- |
| Hosting  | S3AndCloudFront | Create    | awscloudformation |
| Api      | amplifyJsApp    | No Change | awscloudformation |
? Are you sure you want to continue? Yes
...
✨  Done in 18.70s.
frontend build command exited with code 0
✔ Uploaded files successfully.
Your app is published successfully.

🎉🎉🎉

And Done! If you have any questions or feedback reach out on Twitter, or @royalaid on the Clojurians Slack

AWS API + CLJS

Mark Aiken — Mon, 17 Dec 2018 16:44:37 +0000

Building on our own shoulders

Previously we built a simple AWS Lambda. Today we are going mess around with the AWS Node SDK and DynamoDB. The main goal of this post is to show more serious inter-op between Clojurescript and Javascript.

AWS Services

AWS offers a lot of services for lot of different business needs and getting started can be really overwhelming. Before starting this blog anytime I needed to use AWS at work I was nervous. Coworkers through around nonsense acronyms left and right. "Check the Codebuild step in the Codepipeline to see if there is a problem with S3 or IAM" makes no sense if you haven't used AWS before. Luckily climbing over that hurdle really doesn't take more than a few hours of googling and poking around and hopefully this post will help anyone trying to get involved in the AWS ecosystem. Just to steer our exploration we will be using DynamoDB but if something else seems cool I highly encourage you to check out the intro project and docs!

🍖The actual meat🍖

For this posts example code I have ported the Node SDK's Create Table, CRUD Operations, and Delete Table.

Link to repo

royalaid / Shadow-Node-AWS

Node.js example for shadow-cljs

Develop

Watch compile with with hot reloading:

yarn
yarn shadow-cljs watch app

Start program:

node target/main.js

REPL

Start a REPL connected to current running program, app for the :build-id:

yarn shadow-cljs cljs-repl app

Build

shadow-cljs release app

Compiles to target/main.js.

You may find more configurations on http://doc.shadow-cljs.org/ .

Steps

add shadow-cljs.edn to config compilation
compile ClojureScript
run node target/main.js to start app and connect reload server

License

MIT

View on GitHub

Tools

We will obviously need a few tools:

AWS CLI We need the CLI because it gives us auth into AWS, more info on setting up the CLI here
NPM/Yarn (for package management and to use the Shadow-CLJS CLI)
Shadow-CLJS (Our CLJS Build tool of choice mainly because it makes consuming npm deps super easy)

CLJS VS JS

Note: I have basically ported the JS to it's literal, but not idiomatic, CLJS equivalent. I would use this code to help get a better understanding of how the two languages relate and how to call one from the other. I would NOT code like this when using CLJS as the primary language.

In this post I will just break down on example, createTable, because the only difference between any of the examples is the params var and the dynamodb/docClient fn call.

JS for reference

var AWS = require("aws-sdk");

AWS.config.update({
  region: "us-west-2",
  endpoint: "http://localhost:8000"
});

var dynamodb = new AWS.DynamoDB();

var params = {
    TableName : "Movies",
    KeySchema: [       
        { AttributeName: "year", KeyType: "HASH"},  //Partition key
        { AttributeName: "title", KeyType: "RANGE" }  //Sort key
    ],
    AttributeDefinitions: [       
        { AttributeName: "year", AttributeType: "N" },
        { AttributeName: "title", AttributeType: "S" }
    ],
    ProvisionedThroughput: {       
        ReadCapacityUnits: 10, 
        WriteCapacityUnits: 10
    }
};

dynamodb.createTable(params, function(err, data) {
    if (err) {
        console.error("Unable to create table. Error JSON:", JSON.stringify(err, null, 2));
    } else {
        console.log("Created table. Table description JSON:", JSON.stringify(data, null, 2));
    }
});

CLJS

(ns server.create-table
  (:require ["aws-sdk" :as AWS])) ;; Our var AWS = require statement

(AWS/config.update #js{:region "us-east-1"})
;; first example of js interop, the translates to the AWS.config.update above, the AWS/ bit is used for accessing the CLJS "Namespace" for the AWS SDK

(def dynamo (AWS/DynamoDB.  #js{:apiVersion "2012-08-10"}))
;; Second example of interop and shows constructor invocation. It can also be written as (def dynamo (new AWS/DynamoDB  #js{:apiVersion "2012-08-10"})) because the . is shorthand for new
;;Additionally def is the CLJS equivalentish of var but isn't used as often as in Clojure/script

(def params
  (clj->js {:TableName "Movies",
            :KeySchema
            [{:AttributeName "year", :KeyType "HASH"}
             {:AttributeName "title", :KeyType "RANGE"}],
            :AttributeDefinitions
            [{:AttributeName "year", :AttributeType "N"}
             {:AttributeName "title",
              :AttributeType "S"}],
            :ProvisionedThroughput
            {:ReadCapacityUnits 10,
             :WriteCapacityUnits 10}}))

(defn invoke []
 (.createTable dynamo params
               #(if %1
                  (js/console.error "Unable to create table. Error JSON:"
                                    (js/JSON.stringify %1 nil 2))
                  (js/console.log "Created table. Table description JSON:"
                                  (js/JSON.stringify %2 nil 2)))))
;; This is the one difference from the AWS example code above, the actual call to AWS is wrapped in a function so it can be call from node.js proper.

This pattern follows through all the the rest of the examples.

Node.js REPL calls

If you want to be able to test the code out for yourself you can call into from a node.js repl just compile and require

npx shadow-cljs compile app
cd target
node

then once in the repl

var m = require('./main.js');
m.aws.createTable() //Other options include getItem, createItem, readItem, deleteTable, deleteItem, updateItem, updateItemConditionally, atomicInc
//Inside of the Shadow-CLJS config is a mapping between the CLJS fn's to the m.aws object

🎉🎉🎉

And there we have it! If you have any questions or feedback reach out on Twitter, or @royalaid on the Clojurians Slack or Zulip

CLJS ❤️'s Lambda

Mark Aiken — Sat, 08 Dec 2018 21:46:57 +0000

What is Clojure and why do you care?

Because every post about Clojure or Clojurescript (CLJS) has to explain what Clojure(script) is and why you should use it:
// Detect dark theme var iframe = document.getElementById('tweet-1068978794910777344-612'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1068978794910777344&theme=dark" }

But in all seriousness a bunch of people have done better introductions to Clojure then this post will:

Clojure for the Brave and True
- How I learned Clojure (Long)
Learn Clojure in Y Minutes
- A great overview of the language in a file you can run at the REPL to test (Short)
Hypi's Intro to Clojure Post
- A good middle ground between the above

🔥Serverless🔥

Unfortunately there isn't an amazing parody commercial for Serverless so instead let's just try to actually motivate why you should care about it. Serverless is just a shorthand for describing a way to run code without worrying about infrastructure. What does that mean practically? Give them and zip and they throw it on some unused hardware, that's it!

It supported on AWS, Google Cloud, and Azure and probably other clouds. Because the examples are built with AWS that is where we will focus.

Probably the biggest motivator for migrating to or build new services with serverless is cost. The the free tier is very generous and it's pay as you go so it's free to prototype!

You also get ease of deployment/operation and native integration with your cloud provider making it super easy to tap into services like cloud provided DBs, metrics/analytics, queues, email services, etc.

CLJS + Lambda

While there exist ways for you to plug together Clojurescript and Lambda they are coupled to specific build tooling and are a bit too much like a black box for my liking. This also is a good excuse to learn how all the pieces plug together. This post is a distillation of how to do just that. Before finally getting to the meat of the post we have to introduce one more player to the stage.

Shadow-CLJS

Shadow-CLJS is an alternative frontend to the Clojurescript compiler, it is fills the same role as lein-cljsbuild or Figwheel-main if you are familiar with the space. The big advantage it has over the others tools is awesome npm support. We can use basically all of npm with CLJS via Shadow which is important if we want to leverage things like AppSync or the AWS Node SDK.

MAKE THE DAMN LAMBDA ALREADY

Setup

Okay to being lets make a lambda, eventually we will throw away everything inside of it, but for now just leave it be.

And with our basic lambda up we just need our tools and some example code, first the tools:
brew install yarn #Or your OS equiv, apt-get, yum, etc
And that's it!

Example Code

For the example code I have forked Chenyong's minimal Shadow + Node example to create a hello world lambda with Shadow:

royalaid / minimal-shadow-cljs-nodejs

shadow-cljs hot code swapping for Node.js

Node.js example for shadow-cljs

Develop

Watch compile with with hot reloading:

yarn
yarn shadow-cljs watch app

Start program:

node target/main.js

REPL

Start a REPL connected to current running program, app for the :build-id:

yarn shadow-cljs cljs-repl app

Build

shadow-cljs release app

Compiles to target/main.js.

You may find more configurations on http://doc.shadow-cljs.org/ .

Steps

add shadow-cljs.edn to config compilation
compile ClojureScript
run node target/main.js to start app and connect reload server

License

MIT

View on GitHub

git clone https://github.com/royalaid/minimal-shadow-cljs-nodejs.git /tmp/lambda-example
cd /tmp/lambda-example
yarn

and now all of your deps are setup!

Code Breakdown

While the project should be pretty self explanatory I will given a high level overview of the important bits, the shadow-cljs.edn and src/server/main.cljs files.

shadow-cljs.edn

The shadow-cljs.edn is the equivalent of project.clj for lein or boot.clj for boot, its the build configuration file for Shadow. The most important part of that file for us is the :target :node-library line which tells shadow to setup module.exports for use by node, this is bridge between the CLJS code with write and the Javascript that Node reads. The second most important line is :exports {:handler server.main/handler}. This line tell Shadow what to call the export and then what CLJS function to bind to it, in our case the handler function in the Clojurescript namespace server.main. This function is what AWS will end up calling when we execute the lambda.

main.cljs

As for the src/server/main.cljs, this is what will eventually be run by AWS after Shadow transpiles it for us. The lone function inside of the namespace is what actually invoked. Important note, the function signature has to have 3 args (2 if using async/await, which we aren't) and must "return" its value by then invoking the cb arg and passing the result to the second arg of that callback (cb) invocation.

We need to do this because Lambda expects the result to come as a promise or from the callback passed in. This took me waaaaaaaaay too long too piece together and hopefully this won't trip up anyone else either. Additionally we need to be sure to return Javascript values and not Clojurescript values hence the use of #js for the array.

Build and release time

Note: I am going to skip the process of actually developing the code but check out the earlier linked Shadow + Node example for details and workflow for how to use Shadow with Nodejs and hot-reloading.

cd /tmp/lambda-example # Get you back to the correct directory
yarn shadow-cljs release app # This transpiles and optimizes your CLJS
zip -r archive.zip target # Prep your output for upload to Lambda

Then we just upload our new zip to our old Lambda and point the handler function at our CLJS function (which will be .handler, main.handler in the recording).

🎉🎉🎉

And there we have it! If you have any questions or feedback reach out on Twitter, Mastodon, or @royalaid on the Clojurians Slack or Zulip