Forem: Netguru

Commentable.rs - Building a Serverless Comment System in Rust

Wojciech Olejnik — Mon, 20 Jan 2020 10:01:43 +0000

In this article I'm going to write about my journey of building Commentable.rs - a privacy oriented Disqus-like comment hosting solution written in Rust and running on AWS Lambda.

Motivation

A couple months ago I was working on our internal Software Radar - a web app showcasing the technologies we use and tracking the ones we're experimenting with or watching. One of the ideas for this project was that our employees should be able to have a discussion about a particular technology in its comment section, without having to look for an appropriate channel on Slack. Normally we'd integrate such comment system into the current backend, but at the time Radar's API was just a couple of Google Cloud Functions, acting as a proxy to Salesforce. We've decided to look at existing solutions and compare their capabilities but we couldn't find a perfect candidate. The closest to our needs was Disqus, the leading choice in this field. But Disqus has raised some major concerns among our team, such as the lack of data ownership, every comment being public (accessible by URL) and separate sign in/sign up process. Disqus also proved to be hard to customise and even though we managed to override it's stylesheet to match our website's theme it still didn't look very good.

With so much room for improvement, it's hard not to be inspired to build
something better. A comment hosting solution that is privacy oriented and easily customisable. In the spirit of our current backend architecture I've decided to try the Serverless approach. I've also made the decision to use Rust, mostly because I'm a huge fan of it, but also because of its speed and safety. 😉

Development

Getting started

I began by doing some research how to even run Rust code on AWS Lambda. Turns out it's quite easy thanks to a great library that became the backbone of this project - lambda_http. This crate is an extension of the lambda_runtime library and is effectively all we need to quickly develop HTTP-based microservices. A basic hello world example would look like this:

use lambda_http::{lambda, Response, Body, http::StatusCode};

fn main() {
  lambda!(|request, _| Ok(
    Response::builder()
      .status(StatusCode::OK)
      .body(Body::Text("Hello World!"))
      .unwrap()
  ));
}

This incredibly small amount of boilerplate makes me happy, but unfortunately we need to do a bit more to actually run this code on the server. The next part is going to differ depending on your operating system, because we actually need to cross-compile our code to the specific architecture AWS is using for running Lambda functions (Unless you're using Linux, then you're pretty much set!). I was using a Mac and this step gave me a headache, so here's a short summary of what needs to be done:

# 1) Install the cross-compilation tools (WARNING: can take a long time):
$> brew install filosottile/musl-cross/musl-cross
# 2) Add the new target for compilation:
$> rustup target add x86_64-unknown-linux-musl
# 3) Create a soft-link for `musl-gcc`
$> ln -s /usr/local/bin/x86_64-linux-musl-gcc /usr/local/bin/musl-gcc

The final step is to create a .cargo/config file (relative to your project directory) with the following contents:

[target.x86_64-unknown-linux-musl]
linker = "x86_64-linux-musl-gcc"

After that you should be able to compile your project using:

$> CC_x86_64_unknown_linux_musl=x86_64-linux-musl-gcc cargo build

If you don't like headaches (or you're using Windows), the best choice is to use Docker. It works great, but the downside is that build times can get really long - which effectively made me stick to the cross-compilation approach.

Testing and deploying

Having our code compiled we could now zip the binary and upload it to AWS Lambda through the AWS Management Console. That's not very elegant though and certainly not feasible for rapid development. This is where the second pillar of this project comes into play: the AWS Serverless Application Model (SAM). AWS SAM is a technology that allows us to describe the architecture of our application using a YAML (or JSON) file, create a local instance of the app using Docker and finally publish the app to a production environment with a single command. To make use of it we need two command line utilities: aws-cli and aws-sam-cli. If you follow these links you'll be able to install and configure these tools easily.

The final step is to create the AWS SAM template. Here's a very basic one that would work with our hello world example:

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Runtime: provided
    Handler: rust.binary
    Timeout: 3

Resources:
  # GET /hello
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      # I'm assuming the binary name is hello
      CodeUri: target/x86_64-unknown-linux-musl/release/hello # or just target/release/hello on Linux/Docker
      Events:
        HelloWorldEndpoint:
          Type: Api
          Properties:
            Path: /hello
            Method: get

The most important parts are specifying the rust.binary handler, CodeUri (binary location) and endpoint properties (path and HTTP method). There's a lot more you can do with SAM templates and I recommend checking the documentation for more information. With the template ready you should be able to test the application locally (Docker required) with:

$> sam local start-api --template PATH_TO_YOUR_SAM_TEMPLATE.YML

The output should contain a URL which you can visit to test your application. If everything works fine, we can deploy the app to the actual AWS servers with these two commands (make sure you're logged in to aws-cli first):

# assuming a bucket named "hello-rust-lambda" exists
# if not, you can create one with `aws s3 mb s3://hello-rust-lambda`
$> sam package --template-file PATH_TO_YOUR_SAM_TEMPLATE.YML --s3-bucket hello-rust-lambda --output-template-file package.yml
$> sam deploy --template-file package.yml --stack-name hello-rust-lambda --capabilities CAPABILITY_IAM

The first command packages the application and uploads it to S3. The second command instantiates or updates the required services and copies the code from S3 into a new Lambda function. Even though compiling and deploying takes only 3 commands I ended up writing a sizeable Makefile to speed up this process. You can check it out here.

Going forward

At this point we have everything we need to implement a working API, but we still lack some kind of storage. This is where I introduce the third major component of Commentable.rs - AWS DynamoDB and the rusoto_dynamodb create. I'm not a big fan of NoSQL databases, but this was the only choice that worked very well with AWS SAM and our Serverless approach - we can specify the database schema in the same YAML template and even set the billing to only pay per the number of database requests we make.

rusoto_dynamodb makes interacting with the database a breeze. It's not an ORM though, so I wrote a handy trait called DynamoDbModel containing methods such as new, create, find, delete etc. Here's a snippet showing how it encapsulates the methods from rusoto_dynamodb:

use rusoto_dynamodb::{
  DynamoDb,
  DynamoDbClient,
  PutItemInput,
  AttributeValue,
  // ... other imports
};

pub trait DynamoDbModel where Self: Sized + Serialize {
 // ...
  fn create(db: &DynamoDbClient, attributes: IntoDynamoDbAttributes) -> Result<Self, DbError> {
    let attributes: DynamoDbAttributes = attributes.into();
    db.put_item(PutItemInput {
      item: attributes.clone(),
      table_name: COMMENTABLE_RS_TABLE_NAME.to_string(),
      ..Default::default()
    }).sync()
      .map_err(|err| DbError::Error(err.to_string()))
      .and_then(|_| Self::new(attributes))
  }
 // ...
}

We can implement this trait for any struct in order to save it in the database. In our case we only need three models: User, Comment and Reaction. Here's the Comment model for example:

#[derive(Serialize, Debug)]
pub struct Comment {
  pub primary_key: CommentableId,
  pub id: CommentId,
  pub user_id: Option<UserId>,
  pub replies_to: Option<CommentId>,
  pub body: String,
  pub is_deleted: Option<bool>,
  pub created_at: DateTime<Utc>,
}

impl DynamoDbModel for Comment {
  fn new(mut attributes: DynamoDbAttributes) -> Result<Self, DbError> {
    Ok(Self {
      primary_key: attributes.string("primary_key")?,
      id: attributes.string("id")?,
      user_id: attributes.optional_string("user_id"),
      replies_to: attributes.optional_string("replies_to"),
      body: attributes.string("body")?,
      is_deleted: None,
      created_at: attributes.timestamp("created_at")?,
    })
  }
}

That's all there really is to DynamoDB from the Rust perspective. All we need to do now is to specify the database schema in our SAM template:

# ...
Resources:
  # ... Lambda functions definitions
  CommentableRsTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: CommentableRsTable
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: primary_key
          AttributeType: S
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: primary_key
          KeyType: HASH
        - AttributeName: id
          KeyType: RANGE

We only have to specify the required keys so that's enough. We can also create indexes here, but for that I recommend reading the documentation.

Implementing a Create Comment endpoint

At this point there's nothing left to do apart from actually implementing the comment system logic. I'm going to show some snippets from the AddComment Lambda function but I recommend checking the full source code on GitHub - there's a lot of glue code and handy utilities that I won't be able to show here.

Let's start with the entrypoint:

fn main() {
  lambda!(|request, _|
    AddComment::respond_to(request)
      .or_else(|error_response| Ok(error_response))
  );
}

This minimal piece of code immediately delegates execution to the custom AddComment service. I use this pattern for every Lambda function to help with code organisation and state management. The struct itself looks like this:

// HTTP request parameters
#[derive(Deserialize)]
struct Params {
  auth_token: AuthToken,
  replies_to: Option<CommentId>,
  body: String,
}

struct AddComment {
  db: DynamoDbClient,
  commentable_id: CommentableId,
  params: Params,
  current_user: Option<User>,
  comment: Option<Comment>,
}

The real action takes place in the impl block. I used a "pseudo" builder pattern to chain method calls on the AddComment struct. I like the readability of this approach and the bubbling error handling, but there are also some disadvantages: most methods have to take a mutable reference to self, and the methods have to be invoked in a certain order to work correctly.

impl AddComment {
  pub fn respond_to(request: Request) -> Result<Response<Body>, HttpError> {
    if let Some(commentable_id) = request.path_parameters().get("id") {
      Self::new(request, commentable_id.to_string())?
        .validate()?
        .fetch_current_user()?
        .check_reply()?
        .save()?
        .serialize()
    } else {
      Err(bad_request("Invalid params: 'id' is required."))
    }
  }

  pub fn new(request: Request, commentable_id: CommentableId) -> Result<Self, HttpError> {
    if let Ok(Some(params)) = request.payload::<Params>() {
      Ok(Self {
        db: DynamoDbClient::new(Region::default()),
        comment: None,
        current_user: None,
        commentable_id,
        params,
      })
    } else {
      Err(bad_request("Invalid parameters"))
    }
  }
  // ...
}

And here's the save method. It's mostly about preparing the JSON attributes and passing them to the database through a DynamoDbModel trait method.

impl AddComment {
  // ...
  pub fn save(&mut self) -> Result<&mut Self, HttpError> {
    let current_user_id = &self.current_user.as_ref().unwrap().id;
    let mut attributes = IntoDynamoDbAttributes {
      attributes: hashmap!{
        String::from("primary_key") => self.commentable_id.clone().into(),
        String::from("id") => comment_id(&self.commentable_id, current_user_id).into(),
        String::from("user_id") => current_user_id.clone().into(),
        String::from("body") => self.params.body.clone().into(),
        String::from("created_at") => Utc::now().to_rfc3339().into(),
      }
    };
    // replies_to can be empty so we have to insert it separately
    if let Some(parent_comment_id) = self.params.replies_to.clone() {
      attributes.attributes.insert(String::from("replies_to"), parent_comment_id.into());
    }
    match Comment::create(&self.db, attributes) {
      Ok(comment) => {
        self.comment = Some(comment);
        Ok(self)
      },
      Err(err) => Err(internal_server_error(err))
    }
  }
  // ...
}

And that's it! The rest of the code is written using the same general patterns.

Result

I'm overall very happy with how this project turned out. Rust has been a blast to work with and its safety features have saved me multiple times from introducing bugs into the code. The library support for writing Serverless applications is very good. The most problematic thing during development was understanding DynamoDB and getting around some of its limitations - I had to redesign the data layer quite a few times. The performance could also have been a bit better - Rust itself is super fast but it seems that the custom Lambda runtime is introducing some overhead to the response times. When it comes to the future of Commentable, we're aiming to soon realease the client side libraries - for React, Vue, and vanilla JS. After that it's only a matter of adding proper documentation and tests and the product will be ready to ship.

Stickyheader.js - beautiful headers in React Native

Krzysztof Jackowski — Mon, 16 Dec 2019 12:24:48 +0000

Stickyheader.js

Introduction

Stickyheader.js is a simple React Native library, enabling to create a fully custom header for your iOS and Android apps.

Preview

Features

Stickyheader.js ships with 3 different use cases for sticky headers and a possibility to create fully custom header!

Tabbed Header	Avatar Header	Details Header

In Use

Predefined headers can be accessed through headerType="HeaderName" property, each header can be configured according to your demands using the wide amount of properties. You can change all of them, or use it right out of the box with as little changes as possible to use it for you needs:)

This is how you can add them in your app:

import React from 'react'
import StickyParalaxHeader from 'react-native-sticky-parallax-header'

const TestScreen = () => (
  <React.Fragment>
    <StickyParalaxHeader headerType="TabbedHeader" />
    {/* <StickyParalaxHeader headerType="AvatarHeader" /> */}
    {/* <StickyParalaxHeader headerType="DetailsHeader" /> */}
  </React.Fragment>
)

export default TestScreen

Below are examples of those components and description of the props they are accepting.

Tabbed Header

Property	Type	Optional	Default	Description
`backgroundColor`	`string`	Yes	`#1ca75d`	Header background color
`headerHeight`	`number`	Yes	`ifIphoneX(92, constants.responsiveHeight(13))`	Sets height of folded header
`backgroundImage`	`number`	Yes	`null`	Sets header background image
`title`	`string`	Yes	`"Mornin' Mark! \nReady for a quiz?"`	Sets header title
`bounces`	`bool`	Yes	`true`	Bounces on swiping up
`snapToEdge`	`bool`	Yes	`true`	Boolean to fire the function for snap To Edge
`renderBody`	`func`	Yes	`title => <RenderContent title={title} />`	Function that renders body of the header (can be empty)
`tabs`	`arrayOf(shape({}))`	Yes	`[{title: 'Popular',content: <RenderContent title="Popular Quizes" />},...]`,	Array with tabs names and content

Details Header

Property	Type	Optional	Default	Description
`leftTopIconOnPress`	`func`	Yes	`() => {}`	Define action on left top button press
`rightTopIconOnPress`	`func`	Yes	`() => {}`	Define action on right top button press
`leftTopIcon`	`number`	Yes	`require('../../assets/icons/iconCloseWhite.png')`	Set icon for left top button
`rightTopIcon`	`number`	Yes	`require('../../assets/icons/Icon-Menu.png')`	Set icon for right top button
`backgroundColor`	`string`	Yes	`#1ca75d`	Header background color
`headerHeight`	`number`	Yes	`ifIphoneX(92, constants.responsiveHeight(13))`	Sets height of folded header
`backgroundImage`	`number`	Yes	`null`	Sets header background image
`tag`	`string`	Yes	`"Product Designer"`	Sets header tag name
`title`	`string`	Yes	`"Design System"`	Sets header title
`image`	`number`	Yes	`require('../../assets/images/photosPortraitBrandon.png')`	Sets header image
`renderBody`	`func`	Yes	`title => <RenderContent title={title} />`	Function that renders body of the header (can be empty)
`bounces`	`bool`	Yes	`true`	Bounces on swiping up
`snapToEdge`	`bool`	Yes	`true`	Boolean to fire the function for snap To Edge
`hasBorderRadius`	`bool`	Yes	`true`	Adds radius to header's left bottom border
`iconNumber`	`number`	Yes	`10`	Set amount of cards shown on icon

Avatar Header

Property	Type	Optional	Default	Description
`leftTopIconOnPress`	`func`	Yes	`() => {}`	Define action on left top button press
`rightTopIconOnPress`	`func`	Yes	`() => {}`	Define action on right top button press
`leftTopIcon`	`number`	Yes	`require('../../assets/icons/iconCloseWhite.png')`	Set icon for left top button
`rightTopIcon`	`number`	Yes	`require('../../assets/icons/Icon-Menu.png')`	Set icon for right top button
`backgroundColor`	`string`	Yes	`#1ca75d`	Header background color
`headerHeight`	`number`	Yes	`ifIphoneX(92, constants.responsiveHeight(13))`	Sets height of folded header
`backgroundImage`	`number`	Yes	`null`	Sets header background image
`title`	`string`	Yes	`"Brandon`	Sets header title
`subtitle`	`string`	Yes	`"Coffee buff. Web enthusiast. Unapologetic student. Gamer. Avid organizer."`	Sets description(subtitle) section
`image`	`number`	Yes	`require('../../assets/images/photosPortraitBrandon.png')`	Sets header image
`renderBody`	`func`	Yes	`title => <RenderContent title={title} />`	Function that renders body of the header (can be empty)
`bounces`	`bool`	Yes	`true`	Bounces on swiping up
`snapToEdge`	`bool`	Yes	`true`	Boolean to fire the function for snap To Edge
`hasBorderRadius`	`bool`	Yes	`true`	Adds radius to header's left bottom border

Custom Header

Property	Type	Required	Default	Description
`background`	`node`	No	-	This renders background component
`backgroundImage`	`number`	No	-	This renders background image instead of background component
`backgroundColor`	`string`	Yes	`""`	Header background color
`bounces`	`bool`	Yes	`true`	Bounces on swiping up
`children`	`node`	No	-	This renders all the children inside the component
`foreground`	`node`	Yes	-	This renders foreground component
`header`	`node`	Yes	-	This renders header component
`headerHeight`	`number`	No	`92`	Sets height of folded header
`headerSize`	`func`	No	-	Returns size of header for current device
`initialPage`	`number`	No	`0`	Set initial page of tab bar
`onChangeTab`	`func`	No	-	Tab change event
`onEndReached`	`func`	No	-	Tab change event
`parallaxHeight`	`number`	No	`0`	Sets height of opened header
`snapToEdge`	`bool`	No	`true`	Boolean to fire the function for snap To Edge
`scrollEvent`	`func`	No	-	Returns offset of header to apply custom animations
`tabs`	`arrayOf(string)`	No	-	Array of tab names
`tabTextStyle`	`shape({})`	No	{}	Text styles of tab
`tabTextActiveStyle`	`shape({})`	No	{}	Text styles of active tab
`tabTextContainerStyle`	`shape({})`	No	{}	Container styles of tab
`tabTextContainerActiveStyle`	`shape({})`	No	{}	Container styles of active tab
`tabsContainerBackgroundColor`	`string`	No	-	Background color of tab bar container
`tabsWrapperStyle`	`shape({})`	No	{}	Tabs Wrapper styles

Getting Started

Prerequisites

Installation

$ yarn add react-native-sticky-parallax-header

In order to make tab bar work, we have to link react-native-nested-scroll-view package.

$ react-native link react-native-nested-scroll-view

Depending on the version of React Native you use, the package can be still making issues for you, you have to install patch-package

$ yarn add patch-package postinstall-postinstall

Then you add this script to your scripts:

 "scripts": {
+  "postinstall": "patch-package"
 }

After all those steps, just copy a 'patches' folder from this repository and run yarn again to apply the patched package.
You're ready to use the package now.

Running/Development

iOS:

$ react-native run-ios

Android:

$ react-native run-android

Running Tests

$ yarn test

Creating new Pull Request

remember to add appropriate title, ticket, description
adding video or screenshot is very beneficial but it's not mandatory
additionally please remember to add appropriate Pull Request title from following:
- [RNS-XX] short description - for normal feature branches

Code structure

src/
├──assets
├──components
├──constants
├──predefinedComponents

Code Style

Make sure you are using linter with linting rules defined in ESLint config (.eslinrc)
Name branch according to your ticket following this pattern: RNS-XX-short_description
Imports and exports inside index.js files eg. screens/index.js, components/index.js should be alfabetically
Style names in ComponentName.styles.js should be ordered alfabetically

Usage

Here is a basic example of how to create a custom header

import React from 'react'
import { Text, View, Animated, StyleSheet } from 'react-native'
import StickyParallaxHeader from 'react-native-sticky-parallax-header'

const styles = StyleSheet.create({
  content: {
    height: 1000,
    marginTop: 50
  },
  foreground: {
    flex: 1,
    justifyContent: 'flex-end'
  },
  message: {
    color: 'white',
    fontSize: 40,
    paddingTop: 24,
    paddingBottom: 7
  },
  headerWrapper: {
    backgroundColor: 'green',
    width: '100%',
    paddingHorizontal: 24,
    paddingBottom: 25,
    flexDirection: 'row',
    alignItems: 'center'
  },
  headerTitle: {
    fontSize: 16,
    color: 'white',
    margin: 12
  },
  tabsWrapper: {
    paddingVertical: 12
  },
  tabTextContainerStyle: {
    backgroundColor: 'transparent',
    borderRadius: 18
  },
  tabTextContainerActiveStyle: {
    backgroundColor: 'lightgreen'
  },
  tabText: {
    fontSize: 16,
    lineHeight: 20,
    paddingHorizontal: 12,
    paddingVertical: 8,
    color: 'white'
  }
})

class TabScreen extends React.Component {
  state = {
    scroll: new Animated.Value(0)
  }

  componentDidMount() {
    const { scroll } = this.state
    scroll.addListener(({ value }) => (this._value = value))
  }

  renderContent = (label) => (
    <View style={styles.content}>
      <Text>{label}</Text>
    </View>
  )

  renderForeground = () => {
    const { scroll } = this.state
    const titleOpacity = scroll.interpolate({
      inputRange: [0, 106, 154],
      outputRange: [1, 1, 0],
      extrapolate: 'clamp'
    })

    return (
      <View style={styles.foreground}>
        <Animated.View style={{ opacity: titleOpacity }}>
          <Text style={styles.message}>STICKY HEADER</Text>
        </Animated.View>
      </View>
    )
  }

  renderHeader = () => {
    const { scroll } = this.state
    const opacity = scroll.interpolate({
      inputRange: [0, 160, 210],
      outputRange: [0, 0, 1],
      extrapolate: 'clamp'
    })

    return (
      <View style={styles.headerWrapper}>
        <Animated.View style={{ opacity }}>
          <Text style={styles.headerTitle}>STICKY HEADER</Text>
        </Animated.View>
      </View>
    )
  }

  render() {
    const { scroll } = this.state

    return (
      <StickyParallaxHeader
        foreground={this.renderForeground()}
        header={this.renderHeader()}
        parallaxHeight={200}
        headerHeight={90}
        headerSize={() => {}}
        onEndReached={() => {}}
        scrollEvent={Animated.event([{ nativeEvent: { contentOffset: { y: scroll } } }])}
        tabs={[
          {
            title: 'First Tab',
            content: this.renderContent('FIRST TAB')
          },
          {
            title: 'Second Tab',
            content: this.renderContent('SECOND TAB')
          },
          {
            title: 'Third Tab',
            content: this.renderContent('THIRD TAB')
          },
          {
            title: 'Fourth Tab',
            content: this.renderContent('FOURTH TAB')
          },
          {
            title: 'Fifth Tab',
            content: this.renderContent('FIFTH TAB')
          }
        ]}
        tabTextStyle={styles.tabText}
        tabTextContainerStyle={styles.tabTextContainerStyle}
        tabTextContainerActiveStyle={styles.tabTextContainerActiveStyle}
        tabsContainerBackgroundColor={'green'}
        tabsWrapperStyle={styles.tabsWrapper}
      >
      </StickyParallaxHeader>
    )
  }
}

Demo

Live demo videos

If you want to test package without installation you can download our showcase app

Tips

In order to nest scrollable component use scrollEnabled={false} on it and move all the logic to the header eg. by using onEndReached prop.

Technology stack

Technology	Version
React	16.8.6
React Native	0.59.8

License

The gem is available as open source under the terms of the MIT License.

Building WASM applications with C#

Wojciech Olejnik — Tue, 03 Dec 2019 08:51:41 +0000

This post is a short summary of our recent experiment with WASM and Blazor (.NET)

Here at Netguru we have high hopes for WebAssembly. It's definitely a disruptive technology with a huge potential to revolutionize modern web development. So far though, we've struggled to come up with a good use case for it or to find a way to incorporate it in our day-to-day projects. We've experimented with Machine Learning and image processing, but the results were not very satisfying. Our technology of choice was always Rust, which has out of the box support for WASM and a great ecosystem of tools. But Rust also had some drawbacks, including it's infamous steep learning curve. When it comes to writing whole web apps in Rust, the only reasonable option seems to be Yew (https://github.com/yewstack/yew) and even though it looks very promising, it's still a work in progress.

Some time ago I stumbled across Blazor, a part of Microsoft's ASP.NET framework which enables writing client side web applications in C#. There are two flavours of Blazor - one works by executing C# code on the server and passing the data back using a technology named SignalR (basically WebSockets). The second flavour of Blazor works by compiling C# into WebAssembly. I was intrigued that Microsoft adopted this technology so quickly into their core framework, so we decided to perform some research and maybe write a simple application with it.

As of today, the WASM version of Blazor is still in Preview, but the documentation and support are very solid. Getting started is as easy as installing the preview version of Visual Studio (or the dotnet core CLI), downloading the blazorwasm project template, and then creating a project. This comes down to two commands:

# Download the blazorwasm project template
$> dotnet new -i Microsoft.AspNetCore.Blazor.Templates::3.1.0-preview3.19555.2
# Build a new project from the template
$> dotnet new blazorwasm --hosted

You can skip the --hosted part if you want to make just a client-side application. I also wanted to try building a simple API in ASP.NET and this option initializes a simple project for that.

The blazorwasm template contains three examples: a counter, a todo list, and a weather forecast API example. They're minimal and easy to understand, one can also get rid of them quickly (which I did because I like to build my stuff from scratch). For our research example, I've decided to implement a simple dashboard with fake financial data. Stock values are fetched every second from an API where they are adjusted randomly, so that we get some dynamic content in an otherwise simple application. The code that we actually write in a Blazor app can be divided into three areas: Razor templates, component logic and plain C# classes. Razor templates are very similar to other template solutions like JSX or ERB. The difference is of course in the syntax and the fact that we can embed C# code in them. Components in Blazor are again very similar to React or Vue - they are stateful and have specific lifecycle methods defined on them that we can use to manage their state. In my example I had to use OnInitializedAsync (an equivalent of React's componentDidMount), where I set up recurring data fetching. The template and the component logic reside in one file that looks like this:

@page "/"
@using Stocks.Shared
@inject StocksViewModel viewModel

<div class="dashboard">
  // We can use C# inside html if we preceed it with the "@" symbol.
  @if (viewModel.Stocks == null) {
    <p><em>Loading...</em></p>
  }
  else {
    // Rest of the page...
  }
</div>

@code {
  // Component logic goes here
  protected override async Task OnInitializedAsync() {
    await viewModel.FetchStocks();
    System.Timers.Timer t = new System.Timers.Timer();
    t.Elapsed += async (s, e) => {
      await viewModel.FetchStocks();
      await InvokeAsync(StateHasChanged); // Inform the component that it should re-render;
    }
    t.Interval = 1000;
    t.Start();
  }
}

Last but not least, we write pure C# classes to separate the logic from our views and components. An amazing advantage of this approach is the fact that we can actually share the same code between the client and server applications. In my case I had to write a ViewModel class for my Stock model to decorate some of the database fields.

The developer experience overall is pretty good, even on a Mac. Visual Studio is not required (any .NET core app can be run from a terminal), but it certainly brings a lot to the table with IntelliSense and an integrated debugger. That said, the development felt quite slow at times. Error messages in the browser were often extremely unhelpful due to the obfuscated nature of WebAssembly. There's also no hot reload on C# code changes (at least to my knowledge), which meant I had to stop and restart the application every couple of minutes.
When it comes to performance and the overall result of this experiment, I'm quite happy. The app runs smoothly and the only performance bottleneck is the initial load time - since the browser needs to download the WASM equivalent of the entire .NET framework. But these files would be served from browser cache in a normal scenario so I'm actually not worried about that.

I'm certainly intrigued by this new way to develop web applications. With Blazor coming out of preview and WebAssembly constantly evolving, I'm starting to think that it might become one of the hot technologies of the next few years.

Painless NSLayoutAnchors

Marcin Siemaszko — Mon, 18 Nov 2019 20:07:26 +0000

Creating layouts is a huge part of every iOS developer work. Working with UIKit is one of the basics which we need to master. From simple login screens where you have two text fields and a button to more complex screens with nested stack views, custom collection view cells where there is also a need to implement animations improving user experience.

Positioning views can be done in a few ways:

Manually setting views frame position
Storyboards
Layout constraints created in code

The first option is probably used only by masochists. Storyboards are ideal for prototyping and building small apps. Collaboration when using storyboards can be very hard and resolving merge conflicts can make you cry. So this leaves us with the third option. In Netguru we decided that our policy is to always build our views using code. It makes collaboration easier.

Some of us may remember times when a common way of adding layout constraints was using Visual Format Language. For those who do not remember what was that about, here is a “simple” snippet:

let verticalConstraints = NSLayoutConstraint.constraints(
    withVisualFormat: "V:|-40-[logoImageView(50)]-[loginTextField]-20-[passwordTextField]-20-[loginButton]-15@750-|",
    options: [],
    metrics: nil,
    views: views
)

Simple? Not at all. It’s not very descriptive. It takes a huge amount of time to master VFL. Because of that and a fact that NSLayoutConstraint API was not very friendly, a lot of libraries for adding constraints were created. You probably heard about PureLayout, SnapKit, Masonry, EasyPeasy and many many others. All of those were trying to make adding constraints easier. I used some of those and none of them made me spend a lot of time to learn how to create constraints using those.

Some time ago, Apple decided to improve our lives and introduced NSLayoutAnchor. At first glance, it looks very nice. Every view have the same set of anchors that we can use for creating constraints. Let's take a look at the example:

let topConstraint = logoView.topAnchor.constraint(equalTo: self.topAnchor, constant: 50)
let heightConstraint = emailTextField.heightAnchor.constraint(equalToConstant: 100)
let widthConstraint = emailTextField.widthAnchor.constraint(equalToConstant: 100)

So, where is the pain?

Actually, that’s not all the code you need to write. To make it work you need to set translatesAutoresizingMaskIntoConstraints flag to false for every view that you will be setting up. Next thing is that every constraint is not active after creation. To make it all work we need code like this:

self.translatesAutoresizingMaskIntoConstraints = false
logoView.translatesAutoresizingMaskIntoConstraints = false
logoView.topAnchor.constraint(equalTo: self.topAnchor, constant: 50).isActive = true
logoView.topAnchor.constraint(equalTo: self.topAnchor, constant: 50).isActive = true
logoView.heightAnchor.constraint(equalToConstant: 100).isActive = true
logoView.widthAnchor.constraint(equalToConstant: 100).isActive = true

And remember, UIKit will not help you. If you will forget about setting correct value to translatesAutoresizingMaskIntoConstraints or isActive you can end up debugging layout issues from code that “should work”. Trust me, been there, done that.

How can we deal with layouts without using any external dependency and without all NSLayoutAnchor boilerplate. First thing that comes to mind is to write micro framework that will wrap creating constraints using NSLayoutAnchors. Before writing any helper for that I came across a post written by Chris Eidhof describing how to approach this problem. His idea is to use key paths. How great is that? I loved it from beginning. I will not describe how it works in detail, Chris did a great job explaining this in his post, there is even a free Swift Talk about it. You should check it out.

If Chris did all the necessary work, where is my part here? Actually his implementation is missing some of features that I needed. He handled case only when we want to add constraints along with adding view as subview. It was not possible to add relations between two views using his solution. I’ve got cracking and came up with small wrapper, that in my opinion helps creating constraints a lot. By small I mean, that it has around 200 lines of code of which most are docs.

Syntax looks like this:

loginTextField.addConstraints([ 
    equal(\.heightAnchor, to: 30), 
    equal(logoImageView, \.topAnchor, \.bottomAnchor, constant: 20), 
    equal(superView, \.leadingAnchor, constant: 20), 
    equal(superView, \.trailingAnchor, constant: -20) 
])

What does this do is:

makes view 30pts height
attaches top border to bottom border of logoView with 20pts spacing
attaches leading and trailing anchors to corresponding anchors of superView with offset 20pts

I hope that without my explanation you could easily tell how this works. If yes, then i can consider my goal achieved.

For more info please refer to the code and documentation for the wrapper. If you want to play with this, I’ve prepared swift playground with one simple view created.

Wrapper
SwiftPlayground

New Things in Android Fragments

Kacper Kogut — Fri, 15 Nov 2019 14:30:11 +0000

Many of Android Developers had bad experience using Fragments. There are many issues connected to them starting from the lifecycle and ending on animations. Fortunately, the Android team addressed some of these problems on the two latest releases of the Fragments library. Newest version is still the Release Candidate but in this article, I want to show what functionalities it will bring and what will be the future of Fragments.

Getting started

I've decided to check out these new features in a simple project. It allows to create and replace Fragment in the old way, and the new one. On every Fragment, there will be displayed its number, which is fetched from mocked service.

To start working with new features that I will be explaining, new dependencies have to be added to build.gradle file:

dependencies {
    def fragment_version = "1.1.0"

    // For Java
    implementation "androidx.fragment:fragment:$fragment_version"
    // For Kotlin
    implementation "androidx.fragment:fragment-ktx:$fragment_version"
    // For testing fragments
    implementation "androidx.fragment:fragment-testing:$fragment_version"
}

Fragment Container

One of the most important things introduced was a new view for holding Fragments, called FragmentContainer. Until now when we wanted to add single Fragment to our layout we had to use the tag. Since many Android developers treat it as an anti-pattern and instead of using it they would rather inflate FrameLayout with desired Fragment, we can say that until now there has not been a dedicated and well-working XML view for storing Fragments.

FragmentContainer extends FrameLayout but allows only Fragments as its children. Except for being one and true container for the Fragments, it also fixes the issues with animations between each transition.

In the project that I have created, I have added custom animation between each transition, to check how these components will behave.

parentFragmentManager.commit {
    setCustomAnimations(
        R.anim.enter_from_right,
        R.anim.exit_to_left,
        R.anim.enter_from_left,
        R.anim.exit_to_right
    )
    replace(R.id.fragment, BaseFragment)
    addToBackStack(null)
}

Below are the results of this transaction using these two components.
<fragment> tag
Fragment container

Pay attention to the bottom of each view, especially the button. It's really easy to see, that transition in FragmentContainer looks and works much better. This happens thanks to fixes with z-ordering of Fragments in the FragmentContainer.

Similarly to tag, FragmentContainer allows us to use the class tag, to inflate view with the desired Fragment. But unlike the old way, FragmentContainer uses FragmentTransaction for adding Fragments.

<androidx.fragment.app.FragmentContainerView
    android:id="@+id/fragment"
    android:name="com.example.fragmentsexample.feature.common.BaseFragment"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />

Fragment Factory

Until now, whenever you wanted to create new Fragment you had to call its no-argument constructor and put all the variables in a Bundle, which should be then passed as the Fragment arguments using setArguments setter. It was advised to do it this way because whenever the Android system would recreate this Fragment it will call this empty constructor.

Bundle is working fine if you want to pass some object or list to Fragment. But this requirement makes it impossible to implement constructor Dependency Injection.

To start working with FragmentFactory you need to create a class which will create an instance of fragments based on their class name:

class BaseFragmentFactory : FragmentFactory() {
    override fun instantiate(classLoader: ClassLoader, className: String) =
        when (className) {
            BaseFragment::class.java.name -> BaseFragment()
            SecondFragment::class.java.name -> SecondFragment()
            else -> super.instantiate(classLoader, className)
        }
}

Then, in Activity that will show these Fragments, you should attach created FragmentFactory:

class ContainerActivity : AppCompatActivity() {
   override fun onCreate(savedInstanceState: Bundle?) {
       supportFragmentManager.fragmentFactory = BaseFragmentFactory()
       super.onCreate(savedInstanceState)
   }
}

Now whenever you want to replace or add Fragments, you can do it via its class name, rather than its instance, thanks to which Fragment can have as many parameters in constructor as you want:

parentFragmentManager.commit {
    replace<BaseFragment>(R.id.fragment_container)
}

On back pressed dispatcher

When I was implementing some app a long time ago I wanted to listen for back press events in Fragment. To achieve this I had to send an event from Activity to my target class from onBackPressed method. It was annoying when I had to do this many times, and it generated a lot of boilerplate code.

Fortunately, now we can make use of BackPressedDispacher which could be used in any component that can get an instance of its Activity:

lateinit var dispatcher : OnBackPressedDispatcher
lateinit var callback: OnBackPressedCallback

override fun onAttach(context: Context) {
    super.onAttach(context)
    dispatcher = requireActivity().onBackPressedDispatcher
    callback = dispatcher.addCallback(
        //Lifecycle owner
        this
    ) {
        fragmentService.fragmentsCount--

        //Called when user should be navigated back
        callback.isEnabled = false
        dispatcher.onBackPressed()
    }
}

In the following example on every back press, I am decrementing fragment number and then calling onBackPressed to navigate the user back. Calling dispatcher methods can be done anywhere in code, so for example, you can show confirmation dialog at back arrow press and after user clicks on confirm - call the onBackPressed method.
<fragment> tag
Fragment container

Both of these use cases contain the same view, and as you can see, it doesn't work as it supposed to with old tags. After pressing the next button, then returning back and once again pressing the next button Fragment number should be shown as 2, but instead, it is displayed as 1. Both examples use the same Fragment class, and since it only works as it supposed to with FragmentContainer, it is another reason to use them instead of old tag.

Alternatively, you can inject the OnBackPressedDispatcher from Activity to Fragment. A similar example was presented on Android Dev Summit.

Fragment Scenario

FragmentScenario allows you to isolate single fragment and test how it will behave on click, recreation, state changes, etc. Thanks to the usage of FragmentFactory we can simply create FragmentScenario with MockFactory, which will provide mocked dependencies.

val scenario = 
    launchFragmentInContainer<BaseFragment>(factory = MockBaseFragmentFactory())

To test each of the use-cases you have to call appropriate method on created scenario. Test assertions could be done with onFragment method.

//Move Fragment to onCreated state
scenario.moveToState(State.CREATED)

//Recreate Fragment
scenario.recreate()

//Check fragment on click
onView(withId(R.id.nextFragmentButton)).perform(click())

//Test assertions
scenario.onFragment{ fragment ->
    //Check if fragment responded as it should
}

Future of Fragments

One more thing that was addressed on Android Dev Summit was the future of Fragments. Features, that they described are not yet available to be tested, but they look very promising. Since these are still things, that they are working on, you have to keep in mind that their plan might still change.

Multiple back stacks - Until now, there was one stack, that was responsible for holding fragments that had been started one after another. This approach made working with things that navigate to different Fragments in parallel on one screen (like BottomNavigationView) very painful. The solution that the Android team will propose is to have multiple back-stacks, each connected to starting fragments. Thanks to this, the state of all Fragments will be stored.

Returning results - For now, if you want to pass data from one fragment to another, you were supposed to use targetFragment, and keep the hard reference to it. Since we did not know in which state the referenced Fragment will be, this solution generated a lot of problems. Android team will be working on improving the method onResult, so that you could receive result from different components, not only from Activity to Activity.

Fragment lifecycle - Now there are two lifecycles connected to each Fragment. What Android team is trying to achieve is to merge these two lifecycles into one, based on the view lifecycle. So when the view will be destroyed, the Fragment lifecycle will be destroyed as well.

Summary

I think that this update is a big step forward in dealing with Fragments in Android applications. Finally developers have a dedicated view to store Fragments, and finally, they can inject variables in its constructor.

These changes might not look like a real game-changer, but it's good to know that people working on the Android platform paid attention to occurring problems, and made work easier for Android developers. I'm looking forward to the future of Fragments, and I think that you should too.

Useful links

Android Dev Summit presentation

AndroidX official documentation and changelog

Sample project source code

Photo by Patrick Tomasso on Unsplash

Getting Started With Flutter BLoC

Kacper Kogut — Fri, 15 Nov 2019 11:52:51 +0000

I have to admit it. My first experience with Flutter was not great. It was very unstable when I started working with it, and what put me off was the lack of architecture patterns. It was hard for me to easily structure my app, and I had to create custom logic for good communication between components. So I abandoned my Flutter projects and waited for what the time will bring.

Recently I had to create a multiplatform application. So it was either Flutter or React Native. Since my skills in web development ended on writing "Hello World" in HTML I have decided to give Flutter another chance. I saw that a lot of things have changed, and a lot of new architecture patterns came into play. I have tested some of them in simple projects, and the one I immediately fell in love with was BLoC.

BLoC introduction

BLoC stands for Business Logic Controller. It was created by Google and introduced at DartConf 2018. It is created based on Streams and Reactive Programming.

If you want to start creating apps with BLoC architecture I would strongly recommend two libraries that make working with it much easier: bloc and flutter_bloc. I would also recommend the official documentation of these libraries. It is well-written, with tons of examples that could be applied to most of the use-cases. I will describe briefly all of the BLoC components, but if you want to dive deeper, documentation is the best place to go.

In BLoC pattern we can distinguish four main layers of application:

UI - it holds all of the application's components, that are visible to the user and could be interacted with. Since in Flutter all parts of User Interface are Widgets, we can say that all of them belong in this layer.

BLoC - these are classes that act as a layer between data and UI components. It listens to events passed from it, and after receiving a response it emits an appropriate state.

Repository - it is responsible for fetching pieces of information from single or multiple data sources and processing it for the UI classes.

Data sources - these are classes that provide data for the application, from all of the data sources including database, network, shared preferences, etc.

So now, when we know all of the basic structures, we should understand how these layers communicate with each other. BLoC pattern relays on two main components for it:

Events that are passed from UI, that contains information about a specific action that has to be handled by the bloc.

States that show how the UI should react to change of data. Every BLoC has its initial state, which is defined when it is created.

For example, if we want to implement a login screen, we would have to pass LoginEvent with login details, when user clicks on the appropriate button. After receiving response BLoC should show the SuccessState - when login will be completed successfully, or ErrorState - when the user has entered the wrong credentials, or some other error has occurred.

App specification

Let's tackle BLoC by example. I have created a simple app for fetching song lyrics. It should enable a user to search for lyrics from the Genius API. I have also decided to allow a user to create, update and delete their lyrics to test how the BLoC pattern will work with multiple data sources. Project source code can be found here. And since I will only describe some of the BLoC specific components, you can see how I have implemented the Data Source and Repository layer there.

App preview

Lyrics fetched from the Genius are displayed on the Webview, and those added by a user are shown on the custom screen with the possibility to edit them. Removing items is done by swiping them off the list.

Getting started

To start working with Flutter BLoC library I had to add two dependencies to pubspec.yaml file.

  bloc: ^2.0.0
  flutter_bloc: ^2.0.1

My first approach to creating this app was following the TODOs example. It looked very similar to my application and had similar functionalities. Following this example, I have created one BLoC class for all operations on my lyrics data and three screens. My project structure looked like this:

It turned out not to be the best solution. It was hard to properly handle state changes in screens, and update them depending on states of other screens.

I found out that the best solution for structuring a BLoC app is to create one BLoC for one screen. It will help you to always know in which state UI component is currently in, just from the BLoC that is assigned to it.

Keeping this in mind I have refactored my project, and afterward, its structure looked like this:

You can see that not all of the screens have BLoC assigned to it. Take for example the song details screen. It will only display information about the song, which will be passed to it. So it is unnecessary to track this screen state information.

While working with BLoC it is up to you to decide, whenever each screen should have its BLoC component. Some complicated screens could even have multiple BLoCs that will communicate with each other.

Events and states

Now, when I knew how the project will be structured I could define states which every screen could be in, and which events will it sent.

I will demonstrate the process of implementing BLoC architecture by Search Screen with the possibility to search for song lyrics. Firstly, I have to define the event that this screen will send:

TextChanged - shows that input in the search field has changed, and new songs list should be fetched

Now I have to define states that this screen could be in:

StateEmpty - it should be active when there is no user input in the search bar, it would be BLoC initial state.

StateError - the state should be passed with an error message when something goes wrong.

StateLyricsLoaded - this state will be passed, with a list of songs, after successful fetching songs from repository.

StateLoading - defines that the repository is waiting for the response from the server, or is processing data.

Putting it all together

I now know how the main screen of the application will behave, and I can start implementing app functionalities. Let's start with the main app functionality - searching for lyrics.

When working with BLoC pattern you should always start from the bottom layer and then move to the upper ones based on the flow of data. So after I have implemented data sources and repository, the next step was to create BLoC.

To hold states for the main screen of the application I have to create file song_search_state. It defines all of the states that the search screen could be in.

abstract class SongsSearchState extends Equatable {
  SongsSearchState([List props = const []]) : super(props);
}

As you can see, this class extends the Equatable. It will help to check if new state differs from the current one. It simply allows us to compare objects by the list of props, that are passed in the constructor.

But why is it needed to check if the new state differs from the current? If a passed object is equal to the last one, we don't want to rebuild our screen, and this solution does that for us. So for example, if StateLoading is passed two times, one after another, UI widget that listens to it, will only receive it once.

class SearchStateEmpty extends SongsSearchState {
  @override
  String toString() => 'SearchStateEmpty';
}

class SearchStateLoading extends SongsSearchState {
  @override
  String toString() => 'SearchStateLoading';
}

class SearchStateSuccess extends SongsSearchState {
  final List<SongBase> songs;
  final String query;

  SearchStateSuccess(this.songs, this.query) : super([songs]);

  @override
  String toString() => 'SearchStateSuccess { songs: ${songs.length} }';
}

class SearchStateError extends SongsSearchState {
  final String error;

  SearchStateError(this.error) : super([error]);

  @override
  String toString() => 'SearchStateError { error: $error }';
}

As you can see, I have created states that was described before. Every state could also hold and pass different objects. Like for example, ErrorState holds an error message and SuccessState holds a list with the fetched songs.

It is also good practice to override the toString method. It will well describe the state and will be printed after a transition from one state to another. More on that later.

abstract class SongAddEditEvent extends Equatable{
  SongAddEditEvent([List props = const []]) : super(props);
}

Event class also extends Equatable. It is not necessary since by default BLoC library doesn't make use of this. But knowing whenever passed event will be different from current one allows to manipulate the stream of events in BLoC, which makes it possible to implement functionalities like debounce, distinct, etc.

class TextChanged extends SongSearchEvent {
  final String query;

  TextChanged({this.query}) : super([query]);

  @override
  String toString() => "SongSearchTextChanged { query: $query }";
}

Now I could create an event, that will inform BLoC that the user has changed the search query. Event classes look very similar to state classes, and similar rules apply to them.

When states and events were created, I could finally start implementing song_search_bloc.

class SongsSearchBloc extends Bloc<SongSearchEvent, SongsSearchState> {
  final LyricsRepository lyricsRepository;

  SongsSearchBloc({
    @required this.lyricsRepository, 
    @required this.songAddEditBloc})

  @override
  SongsSearchState get initialState => SearchStateEmpty();

  @override
  void onTransition(Transition<SongSearchEvent, SongsSearchState> transition) {
    print(transition);
  }  @override
  Stream mapEventToState(SongSearchEvent event) async* {
    if (event is TextChanged) {
      yield* _mapSongSearchTextChangedToState(event);
    }
  }
}

Song search BLoC holds an instance of LyricsRepository, which is responsible for combining network and local data source, and doing all operations on the fetched data.

As stated before, I had to override getter for field initialState, to show in which state will BLoC be after its creation.

Remember when I have advised to override the toString method in every state and event? Place where it will be useful is the onTransition. It is called whenever BLoC will change its state. And thanks to the toString method of every state, after each transition terminal would print nice output.

I/flutter (22988): Transition { currentState: SearchStateEmpty, event: SongSearchTextChanged { query: never gonna give }, nextState: SearchStateLoading }
I/flutter (22988): Transition { currentState: SearchStateLoading, event: SongSearchTextChanged { query: never gonna give }, nextState: SearchStateSuccess { songs: 10 } }
I/flutter (22988): Transition { currentState: SearchStateSuccess { songs: 10 }, event: SongSearchTextChanged { query:  }, nextState: SearchStateEmpty }

Next thing to override in BLoC is mapEventToState method. It will be called every time a new event is added to the BLoC, and it should do what its name suggests - react to a particular event with a specific state.

Every event should have their corresponding method. So as stated before TextChanged should produce state depending on the result of search.

Stream _mapSongSearchTextChangedToState(TextChanged event) async* {
  final String searchQuery = event.query;
  if (searchQuery.isEmpty) {
    yield SearchStateEmpty();
  } else {
    yield SearchStateLoading();
    try {
      final result = await lyricsRepository.searchSongs(searchQuery);
      yield SearchStateSuccess(result, searchQuery);
    } catch (error) {
      yield error is SearchResultError
          ? SearchStateError(error.message)
          : SearchStateError("Default error");
    }
  }
}

One new thing for me was the yield keyword. It adds value to the stream that was called by the yield*. You can think about it, that it acts as a return, but it doesn't stop execution of code that is afterward, thanks to which state could be changed to multiple values in one method.

One last thing that should be done is to provide created BLoC, so it could be accessed by UI widgets. To do this main MaterialApp file should be modified.

  @override
  Widget build(BuildContext context) {
    return BlocProvider<SongAddEditBloc>(
        builder: (context) 
            SongAddEditBloc(lyricsRepository: lyricsRepository),
        child: MaterialApp(
          //main app code
        ));
  }

Working with BLoC from UI

After BLoC is implemented, the next thing to do is to make use of it in UI. Firstly I have to show appropriate widget depending on the state:

@override
Widget build(BuildContext context) {
  return BlocBuilder<SongsSearchBloc, SongsSearchState>
    bloc: BlocProvider.of(context),
    builder: (BuildContext context, SongsSearchState state) {
      if (state is SearchStateLoading) {
        return  CircularProgressIndicator();
      }
      if (state is SearchStateError) {
        return Text(state.error);
      }
      if (state is SearchStateSuccess) {
        return state.songs.isEmpty
            ? Text(AppLocalizations.of(context).tr(S.EMPTY_LIST))
            : Expanded(
                child: _SongsSearchResults(
                  songsList: state.songs,
                ),
              );
      } else {
        return  Text(AppLocalizations.of(context).tr(S.ENTER_SONG_TITLE));
      }
    },
  );
}

Then in TextField, where user types song search query I have to send new event to created BLoC. We can achieve this by calling method add(Event).

TextField(
  onChanged: (text) {
     _songSearchBloc.add(TextChanged(query: text));
  }
)

If you take a look at the project source code you will see, that these functions are placed in separate files. And this is where the power in BLoC lies in. You can get the same instance of single BLoC in any Widget.

Transforming events

Now the search is working as it should be. But there is one thing that could be improved. Right now every time user changes text input, a new request is sent. So when user types name of the song very quickly there would be as many requests as many letters this title contains. Good practice in this situation is to wait for a small amount of time and cancel the previous request when new is send. This method is called debounce, you can find more information about it in ReactiveX documentation.

@override
Stream<SongsSearchState> transformEvents(Stream<SongSearchEvent> events,
    Stream<SongsSearchState> Function(SongSearchEvent event) next) {
  return super.transformEvents(
    (events as Observable<SongSearchEvent>).debounceTime(
      Duration(milliseconds: DEFAULT_SEARCH_DEBOUNCE),
    ),
    next,
  );
}

As I have mentioned before, we can extend Equatable in events class of BLoC, to know whenever the state changed to new. This gives us the possibility to override function transformEvents in BLoC and manipulate the incoming stream

Communication between BLoCs

Let's skip in time for a while. I have implemented second BLoC with a possibility to add and edit a song. Allegorically as I shown before I added events states and assigned it to screen.

Everything worked great until I had added a song and went back to my search screen. When there was inserted search query that newly added song should contain, it would not appear on the list.

My first solution was to update the list after adding and removing the song. But it generated unnecessary API calls. I found a better solution, which is listening to my second BLoC state changes in the first one.

First I had to add new events to SongSearchBloc - SongAdded and SongUpdated - which will pass the instance of added or changed song. Then create something called StreamSubscription, which is responsible for listening changes from other BLoC.

final SongAddEditBloc songAddEditBloc;

StreamSubscription addEditBlocSubscription;

SongsSearchBloc(
    {@required this.lyricsRepository, @required this.songAddEditBloc}) {
  songAddEditBloc.listen((songAddEditState) {
    if (state is SearchStateSuccess) {
      if (songAddEditState is EditSongStateSuccess) {
        add(SongUpdated(song: songAddEditState.song));
      } else if (songAddEditState is AddSongStateSuccess) {
        add(SongAdded(song: songAddEditState.song));
      }
    }
  });
}

It is also very important to remember to cancel subscription after it won't be needed anymore. Every BLoC can override method close, which is called after BLoC will no longer be used.

@override
Future<void> close() {
  addEditBlocSubscription.cancel();  return super.close();
}

Since I have created second BLoC, and since the first one depended on it, main file should be modified once again.

  @override
  Widget build(BuildContext context) {
    return MultiBlocProvider(
      providers: [
        BlocProvider<SongAddEditBloc>(
          builder: (context) =>
              SongAddEditBloc(lyricsRepository: lyricsRepository),
        ),
        BlocProvider<SongsSearchBloc>(
          builder: (context) => SongsSearchBloc(
              lyricsRepository: lyricsRepository,
              songAddEditBloc: BlocProvider.of<SongAddEditBloc>(context)),
        ),
      ],
      child: MaterialApp(
          //main app code
          ),
    );
  }

Testing with BLoC

As I have stated before BLoC helps you to easily create tests. Since this topic is really broad and could be subject for another article I will show a simple example, and if you want to see more of them you can always go to the source code, where I have prepared few tests.

Firstly there should be prepared mocked classes, that will be used in tests.

class MockLyricsRepository extends Mock implements LyricsRepository {}
class MockSongBase extends Mock implements SongBase {}

Afterwards, we can start implementing the main function of our tests. We should remember to initialize BLoC in setUp method and close it in tearDown.

void main() {
  SongsSearchBloc songsSearchBloc;
  MockLyricsRepository lyricsRepository;

  String query = "query.test";
  List<SongBase> songsList = List();

  setUp(() {
    lyricsRepository = MockLyricsRepository();
    songsSearchBloc = SongsSearchBloc(lyricsRepository: lyricsRepository);
  });

  tearDown(() {
    songsSearchBloc?.close();
  });}

Then we can write simple tests that will check whenever BLoC initial state is correct and it not emit any state after being closed.

 test('after initialization bloc state is correct', () {
    expect(SearchStateEmpty(), songsSearchBloc.initialState);
  });

  test('after closing bloc does not emit any states', () {
    expectLater(songsSearchBloc, emitsInOrder([SearchStateEmpty(), emitsDone]));

    songsSearchBloc.close();
  });

When writing tests for BLoC you have to know in which states should the BLoC be after specific events. Let's tackle search functionality as an example, after inserting text by user state from empty should go to loading, and after fetching songs list - to success. This states should be defined in order in the array and passed as an argument of function expectsLater, which checks if BLoCs states have changed accordingly.

  test('emits success state after insering lyrics search query', () {
    List<SongBase> songsList = List();
    songsList.add(MockSongBase());

    final expectedResponse = [
      SearchStateEmpty(),
      SearchStateLoading(),
      SearchStateSuccess(songsList, query)
    ];
    expectLater(songsSearchBloc, emitsInOrder(expectedResponse));
    when(lyricsRepository.searchSongs(query))
        .thenAnswer((_) => Future.value(songsList));

    songsSearchBloc.add(TextChanged(query: query));
  });

Then we just have to tell the instance of mocked LyricsRepository to return the list with mocked songs, so when our BLoC will call this function it will work as expected.

The last thing is to add an event to our BLoC, that will produce the desired state, and that's it. Now we have a working test for implemented functionality.

Summary

I think that BLoC is a great pattern, that could be used in every type of app. It helps to improve the quality of your code and makes working with it a real pleasure.

Since it uses advanced techniques like Streams and Reactive Programming, I think that it would be hard for the beginners to try it. But after understanding fundamentals, it is really simple to create a simple app using this architecture.

Photo by David Pisnoy on Unsplash

Intro to Observables with RxJS

Sebastian Hewelt — Mon, 21 Oct 2019 14:11:28 +0000

At first, I had a hard time trying to find any reliable resource on RxJS (besides the docs), which wouldn't treat it in the context of Angular. There is much less talk online about integrating RxJS with React or Vue. Then, I learned that since Angular 2 the framework relies heavily on RxJS, and that's probably why. Turns out though, it works perfectly fine with any other widely used framework, like React or Vue, too! This article though, focuses on RxJS and observables in general, without going into peculiarities of any frontend framework.

What is RxJS?

RxJS [or ReactiveX] is a library for reactive programming using Observables, to make it easier to compose asynchronous or callback-based code.

~ RxJS docs

Be aware, that there are implementations of Rx for other platforms too. There is RxJava, RxPHP, RxSwift and many others. The newest stable version of RxJS is v6. v7 is currently in alpha, so we may see it released as a stable version soon.

The theory (just a little)

The fundamental concept and the core of RxJS that we need go through, in order to better understand RxJS, is the concept of an observable. So what is an observable? In shortest terms it's data arriving over time. It comes from the observer pattern, a software design pattern that addresses the problem of automatically updating UIs when events occur. It's also sometimes called a stream. Although stream is a broader term, if you've heard about it in the RxJS context, it most likely refers to the same thing. So, observable === stream. You can subscribe to Observables with subscribers (also called observers, so subscribers === observers). A subscriber is just an object which triggers three actions upon events, next, error or complete:

    // A subscriber example

    {
    next: event => console.log(`Hey, this is ${event}.`), 
    error: error => console.log(`Oh no, ${error}.`),
    complete: () => console.log(`Now I'm complete`), 
    }

next controls the flow of events, reacting to them. error and complete are two ways of terminating the stream, which no longer emits data when any of those two happen. The difference is, complete doesn't get passed any value to it.

To put those concepts together, here's an Observable with a subscriber:

    // An Observable example with two subscriptions to the same Observable.

    import { Observable } from 'rxjs';

    const foo = new Observable(subscriber => {
      console.log('Hello');
      subscriber.next(42);
    });

    foo.subscribe(x => {
      console.log(x);
    });
    foo.subscribe(y => {
      console.log(y);
    });

    // "Hello"
    // Hey, this is 42.
    // "Hello"
    // Hey, this is 42.

The glossary of RxJS-related concepts includes also:

Subject - type of an observable. Observable and observer combined.
Event - a mouse click, a scroll, a route change, a request
Operators - They are predefined set of functions RxJS exposes, so you can manipulate values from a source with them, returning an observable of the transformed values. You import them similar to as you would import a util from Lodash or Ramda. But the comparison to Ramda is more accurate, because of its composable nature.
Example: import { map, filter } from 'rxjs/operators';

Why is it so cool?

It helps managing the control flow of many async data requests

In the old pre-ES2015 days of web development, asynchronous operations in javascript were mostly handled with callbacks. The major drawback of callbacks was the so called "callback hell", where you would nest functions deeply in one another, to be executed when its wrapping function is done. It would lead to a spaghetti code with a series of ))}})} at the end of blocks. Modern vanilla javascript uses Promise API to solve asynchronous problems. There's also async/await, but it's also Promise under the hood, just with cleaner syntax. Promises functionality is limited, though. Both Promises and Observables are push collections. But, and here's the important part, Promises operate on a single value. You create one, possibly provide a .then() that gets passed a value as soon as the Promise settles, and that's it. It's "bound" to this single value. What's more, a Promise cannot be cancelled. Observable on the other hand, can hold multiple values and the action performed upon it can be easily cancelled. For more head over to official docs which describe more differences between an Observable and a Promise.

Where does RxJS shine the most?

There are no restrictions or limitations on how and when to use RxJS. You could add it just for the sake of having a gentle ending of loading state toggle when some data arrives. But there are certain types of apps, where RxJS shines the most. Those would be all the apps that need real-time updates, relying on dynamic data. All the dashboard-centric apps, with many widgets and data from many sources, dependent on each other, or where sequence of events is important. These would be the ones where the declarative and sequential nature of the library come in handy. If you're building an app with many asynchronous requests and complex side effects this is the way to go.

Who uses RxJS?

When you browse online you can find evidence that RxJS is widely used as a dependency in many major companies like Microsoft, Github or Netflix. I'd say the last one popularizes RxJS the most recently, providing many video resources, even one on how does Netflix use RxJS internally.

Beware. It's not (yet) a part of javascript

RxJS brings observables objects to javascript. This is because Observables aren't currently a part of the language itself. You install it as a dependency. There is a proposal to add it to javascript, but it's still a Stage 1 proposal. Stage 1 is the second step (0-based index) of four stages in total, which every new feature of javascript needs to go through before being added to the standard. Current status of the proposal means, to quote the TC39 repo, that it represents problems that the committee is interested in spending time exploring solutions to. So nobody really knows, if Observables are going to be a part of ECMAScript 2021 or ECMAScript 2029, or will a completely different solution be developed to solve asynchronicity problems in JS.

src: RxJS Official Docs, https://rxjs-dev.firebaseapp.com/.

HowTo: Augmented Reality without ArCore -> kotlin + mobile sensors + Rx 💥 🚀

Mateusz — Fri, 18 Oct 2019 09:03:16 +0000

Augmented Reality

Do you ever wanted to play with AR a little but didn't want to dive deep into whole ARCore experience?
In our company as a part of R&D research we developed a little library that provides you with a view showing labels of the destinations provided by you. It's an open source project so you can contribute or write suggestions of new features at our repo https://github.com/netguru/ar-localizer-view-android

You can easily use it in your projects:

Using library

Add repository in you projects build.gradle file

repositories {
    maven {  url 'https://dl.bintray.com/netguru/maven/' }
    ...
}

Add library dependency

    dependencies {
            implementation 'com.netguru.arlocalizerview:arlocalizerview:0.1.0'
    }

Add view to your layout

 <co.netguru.arlocalizer.arview.ARLocalizerView
        android:id="@+id/arLocalizer"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        />

In in arLocalizerView onCreate method you need to provide

interface ARLocalizerDependencyProvider {
    fun getSensorsContext(): Context
    fun getARViewLifecycleOwner(): LifecycleOwner
    fun getPermissionActivity(): Activity
}

fun arLocalizerView.onCreate(arLocalizerDependencyProvider: ARLocalizerDependencyProvider)

In order to process the permission request you need to provide permission results to the view.

    fun onRequestPermissionResult(
        requestCode: Int,
        permissions: Array<out String>,
        grantResults: IntArray
    )

Finally in order to display the destination labels on the camera preview use
```
arLocalizerView.setDestinations(destinations: List<LocationData>)
```

Library in action!

Lambda - From Knowing Nothing to Creating Your First API With it in Less Than 10 Minutes

Pasquale Coretti — Fri, 04 Oct 2019 08:28:10 +0000

Serverless topic is becoming more and more hot these days. The idea behind this architecture style is that a large part of the headaches related to the server’s operational responsibilities can be delegated to a 3d-party provider, so that developers can focus entirely on writing code aligned with the business goal the application serves.

In this post, I’m going to illustrate how ridiculously easy is to go from don’t-know-even-where-to-put-my-hands, to create a simple API with AWS Lambda:

AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume - there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app

What I’m not going to do here, instead, is illustrating how to do it properly.

Why..?

The fact is that, from one side, there is tons of staff that I’m going to knowingly omit (like how to handle routing properly or how to configure serverless webpack and so forth) and from another, what could look like a smart solution for me, could be a really bad one for some other, depending on the project needs. So, I come up with the idea of keeping it as generic as possible and try to write an introduction to the subject. And the topic is so vast that it’s probably best to chunk it anyway.

Specifically, what I would like to do is to go through the process of deploying a lambda using the Serverless Framework that, as its own documentation says:

helps you build serverless apps with radically less overhead and cost. It provides a powerful, unified experience to develop, deploy, test, secure and monitor your serverless applications.

Serverless Framework is so easy to use that would be equally easy to miss some of the things it does for us under the hood. That is why in this post I'm also going to focus on the AWS console side of the process.

Now, before getting to the juicy part, there’s a couple of things we want to take care of..

1. Create an AWS Account:

First thing to do is to create an AWS Account, if you don't have one already.

2. Create AWS Access Keys:

Go to Services -> IAM:

Inside your IAM page, Go to Users -> Add Users;
Type the new user name and, since we're going to need an access key ID and a secret access key, we can give the new user a Programmatic access -> Next: Permissions
Inside the Set Permissions page, we grant the user administrator access that provides full access to AWS resources and services:

Add Tags: optional - we can ignore this part for now (unless you want to add user's information like email, job title, etc);
Review and Create User;
Save Credentials. You can download the .csv file or copy directly the access key and the secret:

We can open up the terminal and type:

serverless config credentials --provider aws --key {Access_key_ID} --secret {Secret_access_key}

And we should receive a message to inform us that our AWS access keys were successfully stored. This step is going to be necessary when we will want to deploy our Lambdas to AWS.

3. Start our first serverless project:

So let’s create a new directory and let’s say we want to call it: my-app-serveless. You got a better name, go ahed and use it (just make sure to keep it while you’re going through the steps below).

We'll want also to globally add the serverless framework itself. It’s on npm, so:

npm add serverless -g

And now we want to create the actual project, the relative configuration yml, our first function etc.

We don’t need to do it from scratch, we let Serverless do its magic for us. How? Well, for creating a new project we use the command create followed by one of the following options:

a. the template name we want to use;

b. the template url;

c. the local path to your template.

I’m going to pass in a template name, which is composed of the provider name and the runtime environment I would like to use. They are respectively AWS and Nodejs separated by a dash. So it’s going to be aws-nodejs; * you can check all the supported providers and template list on serverless documentation.

So, if we're inside the directory where the project is going to be created we can just launch:

sls create -t aws-nodejs

Otherwise we can use the path flag -p / -path to specify it.

Wait a sec... sls? What is that? And what is happening here?

sls is short for serverless. What we’re basically doing here is pretty simple: we are asking the framework to create for us a new serverless project inside the directory, using an already existing template named aws-nodejs, which also means that we’re letting serverless know that we want to deploy our code to AWS cloud provider. If we wanted, for instance, to deploy to Azure, we would have to use the appropriate template for that.

Now, looking at our directory, we've probably already noticed that serverless has done a bunch of staff under the hood. We got a .serverless dir, a configuration serverless.yml and a hello handler (inside the handler.js file) already set up. How cool is that? Basically we could already deploy it. But we won’t, not for now.

First, we need to take a look at the serverless.yml file. We can ignore for now all the comments and focus on the not-commented part.

This is how it looks like:

service: my-new-cool-serverless-project  

provider:  
  name: aws  
  runtime: nodejs10.x  

functions:  
  hello:  
    handler: handler.hello

service: This is the name space for the group of lambdas you’re going to be creating. You can totally rename with what you believe is most suitable (by default it takes the name of the directory the service has been created in).

provider -> name and runtime: AWS and nodejs - no surprise there, am I right? At the end, that’s what we’ve asked passing as the template 'aws-nodejs'. There is a couple of things we could add here like the region (if not specified, defaults to US East (N. Virginia)) and the stage (that defaults to dev). Since I like to have everything clear in my .yml, I’m gonna go ahed and add:

region: eu-central-1
stage: dev

functions: that is followed by our lambdas. Right now we got just one function: hello that can be renamed however we wish. Now, if you look at its handler, it points at the handler.js file that sls created and the exported function hello.

You can think of the handler as the name of the actual function that is going to be deployed on AWS platform. We can rename it as we wish inside our yml file, as long as we keep the handler path relative to the actual function's source. So, if we were to move the handler to a source directory /src we would have to rewrite it:

functions:hello: // the name of the lambda functionhandler: src/handler.hello // where the function source code is and the name of the function in the code

Pretty straight, right? Ok, let’s take a look at the handler function itself. At time of writing it looks like:

"use strict";module.exports.hello = async event => {  return {    statusCode: 200,    body: JSON.stringify(      {        message: "Go Serverless v1.0! Your function executed successfully!",        input: event      },      null,      2    )  };  // Use this code if you don't use the http event with the LAMBDA-PROXY integration  // return { message: 'Go Serverless v1.0! Your function executed successfully!', event };};

As you can see, this function takes in a parameter event and returns an object with status code and stringified body. About the Event: if you go and look for it in the serverless documentation for AWS provider, you’ll see that it’s described as “anything that triggers an AWS Lambda Function to execute”. It actually makes sense since we know that lambdas are event-triggered. You can take a look at all the kind of events our lambda can subscribe to in AWS, and we’re going to do it right after our deployment. And probably, the event everybody is looking forward to subscribe their lambda to is the AWS API Gateway HTTP endpoint request.

* FYI, our hello function actually also accepts context and a callback as parameters, but I won’t touch them here as they’re not relevant to what we’re trying to accomplish;

Now, even though we haven't written a single line of code yet, we can already move on testing our function - and the good news is that we don’t need to deploy it first. We can test it locally to check that everything’s fine.

We’re going to use the command invoke local followed by the name of the function. So if you haven’t renamed it, the command will look like:

sls invoke local -f hello

At this point you should see on your terminal exactly what our lambda is returning.

Why are we testing it?

Well, the invoke local command is a great tool to check that our lambda is running. We just need to be aware that it won’t always work out of the box, like it does in this situation. In some cases we will need to pass the actual event the lambda is going to respond to. We're going to check all the kind of events lambdas can subscribe to a bit ahed in this post. For now, I can anticipate that we can copy the specific event type, whatever it is, from AWS, pasting inside a JSON or YAML file and pass it to our lambda using the -p flag.

Now that we know our lambda is running, we are actually ready to deploy it.

How?

We can check the possibilities we have by typing:

sls deploy —help

As you can see we've got different options, from deploying the entire serverless service to deploy single functions; in this case, we want to deploy the service, so we’ll just type in our terminal:

sls deploy

* serverless won't be able to deploy the service if you haven't set up credentials already - in case: go back to Create AWS Access Keys section.

The process is going to be pretty fast since we’ve got just one function to deploy, but it’s going to take more time once the project grows.

If we wanted to check the deployment status, how everything's going, if anything's failed, there are two different ways to do it:

Going into your AWS console -> Services -> Cloudformation: here you'll find your stack by the name you gave it in your service (in the yml file). Clicking on it, you’ll be able to check all the updates. (Small piece of advice, be sure to be on the correct region - specifically, the region you passed to your yml configuration file and region in your AWS console needs to match - you can change it from the navigation bar on top, near your login name);
Running the deploy in verbose mode:

sls deploy -v

Once the deployment's completed, we can finally check our Lambda. From your AWS console, just go back to Services -> Lambda and you should be able to see your function under the functions list.

The name of the lambda is the result of the service name, the staging environment, and the function name. We can click on it:

Inside our lambda we see 2 tabs: Configuration and Monitoring.

Their name seem pretty intuitive. The first one is, indeed, for Configuration. You can check here the function's code, runtime, handler name, and all the stuff you’ve set up inside your serverless configuration file.

Here you could also add Environment variables, modify basic settings like memory allocation for the specific lambda (defaults to 1024 - and it can go up till 3008MB - and remember that although increasing it can positively impact your performance, it’s also going to affect costs) and timeout.

On this regard, since memory size and function execution time play (along with the cost per function's invocation) a key role in the cost calculation, a good way to minimize them is to write as quick and as low RAM consuming functions as possible. It is also a good practice to set the RAM quota and timeout to a low number on your testing AWS instance (to minimize the risk of deploying a function that e.g. falls into an endless loop and increases the resource usage and the subsequent costs that we need to pay).

You have the possibility to change settings here or in your yml file. Whatever the decision is, just remember to keep it consistent; all the configurations should be kept in one place only.

Inside Monitoring tab, instead, we can check the CloudWatch Metrics, relative to the function’s performance - so whenever we want to check how our lambdas are doing, this is the place we want to be in.

Looking up at the top right corner of the page, there’s a Test button. Yup, I said before we can test our lambda locally using the command invoke local and I also said that sometimes we will need to pass the event our lambda has subscribed to in order for the test to work, remember? Well, clicking on the Test button will open up a series of possibilities for us:

To create a new test specifying the Event we want to pass to our function, and test it;
To copy the event we want to pass to our lambda when we test it locally;

As you can see, there’s really a lot of events you can use, from Amazon CloudWatch to Alexa Smarthouse - Control. Clicking on any of this event will prompt the source code, that you can copy/past in json/yml file.

If we choose now to test our lambda we would be free to pass any event we like, give it a name, and create our first not-really-useful test. Done? Alright, now we watch the test succeed. Niice!

Another interesting thing is that if you go to the S3 service on your AWS console, you will also see the bucket name of your service (as: your service name, stage and serverlessdeploymentbucket followed by some key). At the end of the day, that's what serverless deployment actually is: packing the service/function to a .zip and uploading it to the provider specified in the yml file (AWS in our case).

That's right, here's another thing Serverless Framework is doing for us (instead of having to manually zip and upload the service by ourselves).

If, at any point, we need to remove it from s3, it's going to be better not to do it directly from here. The best way to do it it is using:

sls remove

that is going to remove the bucket and stack, allowing us to avoid out-of-sync kind of situation in which we would end up having to remove and re-deploy everything.

Why is it useful to know our lambdas are stored here?

In case anything goes wrong with them, we can always come here, download them, and check if there is any error in the source file.

Ok, I believe now you have a pretty wide idea of where you lambda-related stuff is on the AWS console and it’s time for us to finally start building our API.

There are, again, different ways of doing it. Theoretically we could use Amazon API Gateway which is, as the name suggest, a gateway for the api, which would be responsible for routing our http events to our lambda functions. We could, but we don’t have to. There’s a simpler way. How does it sound using our configuration file for achieving the same result?

We’re going to subscribe to the event called Amazon API Gateway Proxy (you can check it in Lambda -> function_name -> test).

Using the proxy implementation of API Gateway (used by default by serverless), is going to give our lambda the power to decide how the response should look like. Without it, we would need to go inside the Amazon API Gateway and configure there what the response should look like - and that’s probably not what we want to get into.

To set up the gateway locally, we have to go back to our yml file. And this time we’re going to hook our lambda to an http event.

functions:    hello:      handler: handler.hello      events:         - http:           path: /hello           method: GET

Or we can use the shortcut:

functions:  hello:    handler: handler.hello    events:      - http: GET /hello

Now, whenever someone does a get request to

wherever_your_gateway_is_hosted/hello

it’s going to execute handler.hello for hello function, it’s going to pass the event that AWS gateway formatted as the first argument, it’s going to wait for a response or an error and respond back to whoever made that request.

Now about running this function, we won’t invoke it locally as we would not be able to check if it’s working with an api. What we need instead is a way to run an offline server that can execute our lambda..

We basically need to imitate the api gateway server on our machine. Since our functions are serverless and they’re going to be executed in a different context, the api gateway on our machine is going to execute our lambdas for us just like it would on AWS.

Let’s do it. We need to add serverless-offline plugin into our dependencies and add it in our configuration:

yarn add serverless-offline// and inside our serverless.yml let's add: plugins:  - serverless-offline

Last thing we want to take care of before seeing the results of our work is modify what we're returning from our lambda:

module.exports.hello = async event => {
  return {
    statusCode: 200,
    body: JSON.stringify(
      {
        message: "Hey there, Welcome!",
        input: event
      },
      null,
      2
    )
  };
};

We need to stringify the body - that's not something is going to be done for us.

And now we can just run:

sls offline

and go check if it works on: localhost:{PORT}/api. If you didn’t set up a port it defaults to 3000. You can set it up via -P flag.

If everything works correctly you should see:

Your message and event are printed on the page.

If you want to add a parameter name to greet, you'd have to add it in the configuration file, so the event would become:

- http: GET /hello/{name}

And we can take the parameter from our event.pathParameters.name, so that our lambda would become:

module.exports.handler = async event => {
  const {
    pathParameters: { name }
  } = event;

  return {
    statusCode: 200,
    body: JSON.stringify(
      {
        message: `Hey there ${name}, Welcome!`,
        input: event
      },
      null,
      2
    )
  };
};

Now, following this logic you could add a new handler to take care of another end point, and so on... I am going to leave up to you the kind of approach you will decide to experiment with.

Those that I have been playing with so far are:

using the api gateway to proxy to one lambda and that one function to proxy to express and having express to define the routes;
using the same handler for all the lambdas and handle routing based on the functions' names (that is available in context passed to the handler);

Long story short...

We talked about the Serverless framework - how to set up a project, what it does under the hood, how to modify the serverless.yml based on your need. We went more into details on lambdas and how to test them, how to work / where to find what you may need while working with them inside the AWS console, completing it with the creation of a simple working API.

And yet, there are parts I've not even had time to touch, like lambdas's cold starts and how they can affect costs, how to build lambda functions with Webpack or how to setup a serverless application with a GraphQL endpoint. But, I am going to try to keep writing on this topic and sharing details or info that I find useful for a general deeper understanding! :)

Ruby on WebAssembly

Thu Le — Tue, 24 Sep 2019 15:30:25 +0000

WebAssembly, shortened as WASM, is a performance optimized solution enabling web applications to run at a near-native speed. It is a cutting-edge technology with the current state as a MVP (minimum viable product), not the final version of WASM. Even though the standard is still evolving and new features are being added, the current MVP version of WASM is guaranteed to be compatible with future releases. This allows the developers to start using it right away, even for big projects, without worrying about future breaking changes. This article takes a closer look at the WebAssembly roadmap and the future’s additional features in relation to Ruby language.

A brief introduction to WebAssembly
To describe briefly how WASM works, we would need to take a look at the concept of WASM modules - distributable, loadable and executable units of code in WASM.

WASM modules are delivered to the browser in binary format, and are executed by a virtual machine (VM) that works alongside the JavaScript VM, sharing resources (e.g. memory) and executing in the same thread.

To put it differently, developers can write code in high-level languages and compile it beforehand to WASM (.wasm filename suffix) which would allow it to run in the browser at a near-native speed.

(Image: hacks.mozilla.org)

In the MPV, WebAssembly has developed the core features which are available today in most modern web browsers (Google Chrome, Microsoft Edge, Apple Safari and Mozilla Firefox). The introduction of WASM module, WASM binary format, WASM text format has been proved to bring high performance to the application on a wide range of platforms, including mobile and IoT. WASM applications running on the browser would be able to keep up with users’ expectations of smooth interaction, fast execution, fast loading and security in memory management.

Where are we at with WebAssembly and Ruby
For years, JavaScript has been the dominant language for web development. Ruby developers can transpile their Ruby code to plain JavaScript through source-to-source compiler like Opal. Since the development of WebAssembly, developers now have another option to choose from compilation targets.

At this state of its development, while WASM only has preliminary support for Ruby, it fully supports compilers for languages like C/C++, Rust, Go to run in WASM themselves. Therefore, these statically typed languages can easily and directly compile to WASM in just one compilation step. This process is straightforward: just target WebAssembly in the compiler toolchain.

However, for a dynamically typed language like Ruby, there are some fundamental issues blocking the compiling process to WASM. Though these issues can be resolved by some means, there are quite a lot of certain basic steps to be executed before Ruby can be fully supported.

WASM modules currently do not have direct access to DOM. There are active movements in WASM community where experienced developers attempted to develop many libraries which allow compiling code to WASM and let developers manipulate the DOM, e.g. https://github.com/mbasso/asm-dom. However, unlike other languages like C, C++, C#, Rust and Go, Ruby does not have an active community around WebAssembly. Ruby developers would need to write a WASM module which contains a lot of glue code dedicated to manipulating DOM.
Ruby uses a garbage collector for memory management, while WASM does not have a garbage collector. In fact, it does not have any tools for memory management yet. Even statically typed languages which do not use a GC still need some sort of mechanism for managing memory allocation, e.g. https://github.com/rustwasm/wee_alloc for Rust language. Therefore, Ruby needs WASM support for garbage collection which is promised to be a coming feature in post-MVP state of WASM.
Ruby requires a direct runtime support which is not ready yet in WASM infrastructure. For now, Ruby developers need to create their own runtime and ship a required runtime alongside their code, and implement a JIT (just-in-time) compilation to give themselves a fast execution. It’s possible to create their own runtime today, but it takes some effort, and this effort will have to be duplicated across different applications. Developers would have to bundle separate runtime with their applications, which is an impediment for many use cases.

For all the above reasons, Ruby does not compile automatically, cleanly and efficiently to WASM at this state of development yet.

Conclusion
It is no doubt that WebAssembly today is MVP complete, but not feature complete. There are more additional features which are expected to be implemented and are still in progress state in the post-MVP will fundamentally change what we can do with WebAssembly. These features include garbage collection, multithreading, zero cost exceptions and fixed-width SIMD (single instruction multiple data), etc. Among these features, some can be developed rapidly while some would need much more time to achieve the desired result. With the joint effort of Google, Microsoft and Mozilla, and an active community where new ideas and proposals are openly raised and added into tracking issues, WASM is expected to become a more mature product and the future of high performance web development. See more here.

Hopefully this article has provided you with a useful brief insight into what’s the current state of WebAssembly in connection with Ruby language and where WebAssembly is heading in the future. Ruby traditionally has a comprehensive ecosystem of tools dependent on a Unix-like environment, compiler toolchain and standard libraries. So I believe when WASM reaches mature state, it will be seamlessly incorporated into Ruby's ecosystem in future.

Photo by Francisco Casero on Unsplash

Database Views and How to Use Them in a Rails Based Application?

Kamil Walkowiak — Sun, 15 Sep 2019 19:54:49 +0000

Most of the time when we need to query some data from our database in a Rails-based application, we just use ActiveRecord query interface. When the query is more complicated, we write it using the SQL language. Sometimes it may be needed to use such SQL query in more than one place. In such cases (to better stick to the DRY principle), we can consider using a database view. In this blog post, I will present what are the database views and how to easily use them in Ruby on Rails application with the help of a Scenic gem.

The database views

The database view is a result set of a query stored in the Database Management System (DBMS). Every time access to the data is required, the stored query must be executed. Though database view is not connected to any persisted data table, it is possible to query it as a regular table (that is why views are often called virtual tables).

There are a few types of database views:

read-only views - can be only used for querying purposes.
updatable views - can be used for querying and data management (INSERT, UPDATE, DELETE operations) purposes, there are some restrictions for this type of views such as only one FROM clause or no aggregate functions in view's query definition. This type of view maybe not supported by all DBMS.
materialized views - provide a static snapshot of data while accessing the view to improve the querying performance. The query related to such view is executed only after the view's creation and on-demand (materialized view's refresh operation). Materialized views can be also indexed as the regular data tables to achieve even better performance. This type of view may be also not supported by all DBMS.

Database views give us many benefits such as:

more DRY database queries - we can extract subqueries used in many places to a single view and use simple JOIN to include the view's data in the main query.
an additional level of abstraction - we can encapsulate pretty complex SQL queries into one, simple to use a view.
better-querying performance - when we know that the data will be more often read than updated or it is possible to proceed with some cached data instead of the most current one we can use materialized view (with additional indexing if needed). It will make the querying process more efficient.

The Scenic gem

Scenic is a gem that allows to easily use database views in a Ruby on Rails-based application without having to switch the database schema to SQL format. It supports versioning of views and provides out-of-the-box support for PostgreSQL. There are available additional adapters for other DBMS such as SQLite, MySQL or SQL Server. Below, we will take a look at the main features offered by the gem.

Creating a new view

After including Scenic in the Gemfile, we can use generators provided by the gem. Let's say that we have the Users table with one of its columns being called active and indicating that the user is active. We want to create a view that will return only active users (that have an active field equal to true). First, we need to create the view by typing in the console:

$ rails generate scenic:view active_users
   create  db/views/search_results_v01.sql # this is a view's SQL query file
   create  db/migrate/[TIMESTAMP]_create_active_users.rb # this is a migration file

As the effect of this command, we get two files. First, let's take look at the migration file:

class CreateActiveUsers < ActiveRecord::Migration
  def change
    create_view :active_users
  end
end

Here, we are using create_view function provided by the gem. As the argument, we need to provide the view's name. We don't need to do anything more in this file. Now let's move on to the SQL query file. There, we need to provide a SQL query of our database view. Let's write an easy query to fetch all of the active users from our database.

SELECT * FROM users WHERE active = TRUE;

It is important to mention that all the views' SQL queries are versioned. We need to create a new version if we want to modify the logic standing behind our view. Each query version can be used in many migrations (e.g. create_view function mentioned earlier takes an optional parameter version if no value is given it defaults to 1).

After providing SQL query of our view we can run the migration. After that, we are ready to use the newly created database view. Let's say that we have such records in the database.

irb(main):001:0> User.all
=> #<ActiveRecord::Relation [#<User id: 1, active: true>, #<User id: 2, active: false>]>

After calling our database view (by executing a raw SQL query) we get the only user with id 1 (as only this one is active).

irb(main):002:0> ActiveRecord::Base.connection.exec_query('SELECT * FROM active_users').rows
  SQL (1.2ms)  SELECT * FROM active_users
=> [["1", "t"]]

View as an ActiveRecord model

OK, but are we limited to use the views in SQL queries only? Fortunately, the answer is: no. We can create an ActiveRecord model based on our view and it will behave like a regular AR's model. There is only one exception: the data provided by such model are available in read-only mode. The simplest model based on our view can look like this:

class ActiveUser < ActiveRecord::Base
  self.primary_key = :id

  def readonly?
    true
  end
end

In this piece of code, we set our model's primary key to id returned by the view. It is not required but helps to better map our view to AR's model, without it we would get objects with id field always equal to nil. Then, we mark our model as read-only so AR will not even try to reach the database when we accidentally call save on an instance of such a model.

Now, we can go back to the Rails console and try to use the newly defined model:

irb(main):003:0> ActiveUser.all
=> #<ActiveRecord::Relation [#<ActiveUser id: 1, active: true>]>

Feels good, doesn't it?

Updating view

Let's say that we have added a new column (called full_name) to the Users table. Now, our data looks like this:

irb(main):004:0> User.all
=> #<ActiveRecord::Relation [#<User id: 1, active: true, full_name: 'Jan Kowalski'>, #<User id: 2, active: false, full_name: 'James Bond'>]>

Let's call out active users view:

irb(main):005:0> ActiveUser.all
=> #<ActiveRecord::Relation [#<ActiveUser id: 1, active: true>]>

As you can see, the new full_name column is not reflected in the database view. It's because some DBMS (like PostgreSQL which was used while preparing this example) is freezing the columns returned by the view. So even when we have used * selector, columns added after view creation will be not included in the result set. To get the full_name column, we need to update the view. To do so, we can use the existing SQL query version that we have used while creating the view, but this time (for academic purposes) we will create a new query version (this time without * selector to avoid confusion in the future).

Let's execute the same command that we have used for generating a view for the first time:

$ rails generate scenic:view active_users
   create  db/views/search_results_v02.sql # new SQL query version
   create  db/migrate/[TIMESTAMP]_update_active_users_to_version_2.rb # update migration file

As you can see, we got a different output than after executing the command last time. Scenic gem had recognized the existence of the requested view so it created an update migration file (instead of creating) alongside with a new SQL query version file. Let's take a look at the migration file:

class UpdateActiveUsersToVersion2 < ActiveRecord::Migration
  def change
    update_view :active_users, version: 2, revert_to_version: 1
  end
end

To update the view, we are using the update_view function. It will first drop the existing view version and then recreate it. As parameters, we are passing the current version (revert_to_version parameter) and the desired version of the view after update (version parameter).

To update the view but without dropping it at first, we can use the replace_view function. It accepts the same params as update_view but there are some restrictions resulting from usage of this function (e.g. we may only add additional new columns to the end of the columns list returned by the view). You can find more information about this function in the Scenic documentation

Now, let's move on to the newly generated SQL query file:

SELECT * FROM users WHERE active = TRUE;

The new query file has been populated with the SQL query from the previous version. Let's modify it so our view will return the id, active and full_name columns but without using * selector:

SELECT id, active, full_name FROM users WHERE active = TRUE;

Now, when we migrate the database, we should get the full_name column in the result set of our view. Let's test it in the Rails console:

irb(main):006:0> ActiveUser.all
=> #<ActiveRecord::Relation [#<ActiveUser id: 1, active: true, full_name: 'Jan Kowalski'>]>

Everything is working as expected.

Materialized views in Scenic

As mentioned earlier, materialized views can provide a performance boost by serving some kind of cached data instead of executing the query every time we refer to the view. They can be also indexed as regular DB tables. Scenic provides support for such type of views.

Let's say, that we want to migrate our active users view to a materialized form and add some indexes on top of it. The generator command and the function presented in the create view step accept an optional parameter for creating the materialized views. However, instead of removing the existing one and creating another version (with duplication of the SQL query) of active users view, let's try to migrate the existing one. For this purpose, let's create a new migration file:

class MigrateActiveUsersToMaterializedView < ActiveRecord::Migration
  def up
    drop_view :active_users
    create_view :active_users, version: 2, materialized: true

    add_index :active_users, :full_name
  end

  def down
    remove_index :active_users, :full_name

    drop_view :active_users, materialized: true
    create_view :active_users, version: 2
  end
end

As you can see, we are defining two methods: up (executed while running the migration) and down (executed while reverting the migration). In the up method, we first remove the existing view from our database and then re-create it in the materialized form (using the same SQL query version that was provided earlier). At this stage, the DBMS executes query related to the view and stores its result. When the materialized view is ready, we also add an index on the full_name column.

In the down method we do an opposed set of operations. First, we remove the index on full_name column, then we remove materialized view and finally we re-create active users as a regular database view.

After executing such migration, the active users view should be migrated to the materialized form. The querying performance boost for this view could be not so visible as this is a pretty simple view. For the more complicated ones, the improvement in the performance may be much more significant.

Summary

A database view is a powerful tool provided by the relational DBMS. We can benefit a lot when using it properly. Scenic gem allows us to use views easily in Ruby on Rails-based applications. We can also create an ActiveRecord model based on a view.

Exploring Machine Learning possibilities in WASM

Wojciech Olejnik — Mon, 09 Sep 2019 09:25:33 +0000

This post is a short summary of our recent experiment with WASM & ML.

The goal was to run a simple ML model in WASM and then maybe benchmark it against existing JavaScript ML libraries. Our Machine Learning team provided a basic grayscale filter model in three formats: CoreML, Tensorflow and PyTorch.

We tried PyTorch first as it looked very promising. We found good bindings for Rust - https://github.com/LaurentMazare/tch-rs - that seemed to be very easy to use. But after some tinkering it turned out that it’s not possible to export the whole model from PyTorch in a form that can be easily imported in Rust. We’d basically have to implement the model from scratch before loading the data the ML team provided. That was a bit too much for our experiment so we moved on.

Next we looked at CoreML, but quickly found that there were virtually no libraries for Rust or anything else that we could compile to WASM. I'm not sure if that’s due to CoreML being made by Apple (and thus being proprietary) or the community isn’t as big and there is no interest in libraries for other languages. If it's the second case than maybe it’s worth taking a look at it in the future.

Our last chance was Tensorflow and it started out great. Even though it doesn’t have first-class support for WebAssembly (it's a work in progress: https://github.com/tensorflow/tfjs/issues/1497), it does have official support for Rust - https://github.com/tensorflow/rust.
We were able to quickly prototype a simple app that loaded the Tensorflow model in a *.pb format, pass it an image as a tensor and finally save the output from the neural network to the file system. It worked really well, so we tried to port it to WebAssembly. For Rust it’s really easy thanks to wasm-pack (https://github.com/rustwasm/wasm-pack).
Unfortunatelly that’s when we hit a major blocker. Turns out that Tensorflow depends on a library called “aligned_alloc” which wouldn’t compile to WebAssembly, most likely due to some system-dependent functionality that could not be ported to the browser.

My conclusion for now is that porting big libraries to WASM can be, more often than not, tricky. The differences in memory allocation create conflicts in low-level code and some libraries simply won’t compile. Another solution would be to write a neural network from scratch, using only basic libraries and language-level constructs. This approach was explored e.g. in https://ngoldbaum.github.io/posts/python-vs-rust-nn. It would most likely compile without issues, but require more time to develop.