Forem: Alex Fallenstedt

Streaming Tweets with Go

Alex Fallenstedt — Thu, 30 Dec 2021 20:29:58 +0000

Building with free APIs is a great way to teach yourself new skills in languages you like. I’ve always found APIs as an underrated way to learn something new. Building with APIs brings challenges that force you to learn new parts of programming that video tutorials can not do.

Twitter’s API’s filtered stream endpoint allows you to filter the real-time stream of public Tweets. You can tap into twitter discussions by filtering tweets for specific attributes. You can find the latest job postings, monitor weather events, or keep on top of trends.

In this article I will discuss how to create twitter rules and manage a stream with my open source library twitterstream. This library was built for my project findtechjobs so I could find the latest tech jobs posted on twitter.

If you want a complete code example to get started, head over to the examples on twitterstream

Where do I start?

The first step is to create an app on Twitter Developers and obtain a set of consumer keys. One you have an API key and an API secret key, you can generate an access token with twitterstream

Generate an Access Token

We can use twitterstream to generate an access token. This access token will be used to authenticate all network requests going forward. In the code below, we make a network request to twitter’s oauth2/token endpoint with the The 'Basic' HTTP Authentication Scheme. Then we create an instance of twitterstream with our access token.

    tok, err := twitterstream.NewTokenGenerator().SetApiKeyAndSecret("YOUR_KEY", "YOUR_SECRET_KEY").RequestBearerToken()
// Create an instance of twitter api
    api := twitterstream.NewTwitterStream(tok.AccessToken)

Set up Streaming Rules

Streaming rules make your stream deliver relevant information. The rules match a variety of twitter attributes such as message keywords, hashtags, and URLs. Creating great rules is fundamental to having a successful twitter stream. It’s important to continue refining your rules as you stream so you can harvest relevant information.

Let’s create a stream for software engineer job postings with twitterstream. A valid job posting tweet should should be:

Posted in the english language
Not a retweet
Not a reply to another tweet
Contain the word “hiring”
And contain the words “software developer” or “software engineer”

The twitterstream package makes building rules easy. We can use a NewRuleBuilder to create as many rules as the Twitter API allows for our consumer keys.

rules := twitterstream.NewRuleBuilder().
        AddRule("lang:en -is:retweet -is:quote hiring (software developer OR software engineer)", "hiring software role").
            Build()

res, err := api.Rules.Create(rules, false)

The first part is using twitterstream to create a NewRuleBuilder.

We pass in two arguments when we add our rule with AddRule. The first is a long string with many operators. Successive operators with a space between them will result in boolean "AND" logic, meaning that Tweets will match only if both conditions are met. For example cats dogs will match tweets that contain the words “cats” and “dogs”. The second argument for AddRule is the tag label. This is a free-form text you can use to identify the rules that matched a specific Tweet in the streaming response. Tags can be the same across rules.

Let’s focus on the first argument. Each operator does something unique:

The first is the single lang:en which is BCP 47 language identifier. This filters the stream for tweets posted in the English language. You can only use a single lang operator in a rule, and it must be used with a conjunction.
Then we exclude retweets with -is:retweet. We use NOT logic (negation) by including a minus sign in front of our operator. The negation can be applied to words too. For example, cat #meme -grumpy will match tweets with the word cat, #meme, and do not include the word “grumpy”.
We also exclude quote tweets with -is:quote. Quote tweets are tweets with comments, and I’ve found this operator very useful. When I was building findtechjobs.io, I encountered a lot of people retweeting an article about automated hiring with their opinion. These quote tweets cluttered my dataset with unrelated job postings.
I then narrow my stream of tweets to words that include hiring. People who tweet about jobs would say “My team is hiring…”, or “StartupCo is hiring…”.
Finally (software developer OR software engineer), is a grouping of operators combined with an OR logic. Tweets will match if the tweet contains either of these words.

After we build our rules, we create them with api.Rules.Create. If you want to delete your rules, you can use api.Rules.Delete with the ID of each rule you currently have. You can find your current rules with api.Rules.Get.

You can learn more about rule operators here. Additionally, the endpoint that creates the rules is documented here.

Set the Unmarshal Hook

We need to create our own struct for our tweets so we can unmarshal the tweet well. Twitter’s Filtered Stream endpoint allows us to fetch additional information for each tweet (more on this later). To allow us to find this data easily, we need to create a struct that will represent our data model.

type StreamDataExample struct {
    Data struct {
        Text      string    `json:"text"`
        ID        string    `json:"id"`
        CreatedAt time.Time `json:"created_at"`
        AuthorID  string    `json:"author_id"`
    } `json:"data"`
    Includes struct {
        Users []struct {
        ID       string `json:"id"`
        Name     string `json:"name"`
        Username string `json:"username"`
        } `json:"users"`
    } `json:"includes"`
    MatchingRules []struct {
        ID  string `json:"id"`
        Tag string `json:"tag"`
    } `json:"matching_rules"`
}

Every tweet that is streamed is returned as a []bytes by default. We can turn our data into something usable by unmarshaling each tweet into the struct StreamDataExample. It’s important to set an unmarshal hook with SetUnmarshalHook so we can process []bytes in a goroutine safe way.

api.SetUnmarshalHook(func(bytes []byte) (interface{}, error) {
    data := StreamDataExample{}

    if err := json.Unmarshal(bytes, &data); err != nil {
        fmt.Printf("failed to unmarshal bytes: %v", err)
    }

    return data, err
})

If you are uncertain what your data model will look like, you can always create a string from the slice of bytes.

api.SetUnmarshalHook(func(bytes []byte) (interface{}, error) {
    return string(bytes), nil
})

Starting a Stream

After creating our streaming rules and unmarshal hook, we are ready to start streaming tweets.
By default, twitter returns a limited amount of information about each tweet when we stream. We can request additional information on each tweet with a stream expansion.

    streamExpansions := twitterstream.NewStreamQueryParamsBuilder().
        AddExpansion("author_id").
        AddTweetField("created_at").
        Build()

    // StartStream will start the stream
    err = api.StartStream(streamExpansions)

We first create some stream expansions with a NewStreamQueryParamsBuilder. This builder will create query parameters to start our stream with. Here, we are adding two additional piece of information to each tweet

AddExpansion("author_id") will request the author’s id for each tweet streamed. This is useful if you are keeping track of users who are tweeting.
AddTweetField("created_at") will request the time the tweet was tweeted. This is useful if you need to sort tweets chronologically. You can learn more about the available stream expansions here

Then we start the stream with our expansions using api.StartStream. This method will start a long running GET request to twitter’s streaming endpoint. The request is parsed incrementally throughout the duration of the network request. If you are interested in learning more about how to consume streaming data from twitter, then you should read their documentation Consuming Streaming Data

Consuming the Stream

Each tweet that is processed in our long running GET request is sent to a go channel. We range over this channel to process each tweet and check for errors from twitter. The stream will stop when we invoke api.StopStream, then we skip the remaining part of the loop, return to the top and wait for aclose signal from the channel.

   // Start processing data from twitter after starting the stream
    for tweet := range api.GetMessages() {

        // Handle disconnections from twitter
        if tweet.Err != nil {
            fmt.Printf("got error from twitter: %v", tweet.Err)

            // Stop the stream and wait for the channel to close on the next iteration.
            api.StopStream()
            continue
        }
        result := tweet.Data.(StreamDataExample)

        // Here I am printing out the text.
        // You can send this off to a queue for processing.
        // Or do your processing here in the loop
        fmt.Println(result.Data.Text)
    }

Twitter’s servers attempt to hold the stream connection indefinitely. The error from twitter is made available in the stream. Disconnections can occur from several possible reasons:

A streaming server is restarted on the Twitter side. This is usually related to a code deploy and should be generally expected and designed around.
Your account exceeded your daily/monthly quota of Tweets.
You have too many active redundant connections.
More disconnect reasons can be found here

Anticipating Disconnects from Twitter

It’s important to maintain the connection to Twitter as long as possible because missing relevant information in your stream can create poor datasets. It should be expected that disconnections will occur and reconnection logic be built to handle disconnections from twitter

We can build reconnection logic using twitterstream’s api and a defer statement. A full example of handling reconnects can be found here. Below is a snippet

// This will run forever
func initiateStream() {
    fmt.Println("Starting Stream")

    // Start the stream
    // And return the library's api
    api := fetchTweets()

    // When the loop below ends, restart the stream defer initiateStream()
    defer initateStream()

    // Start processing data from twitter
    for tweet := range api.GetMessages() {
        if tweet.Err != nil {

            fmt.Printf("got error from twitter: %v", tweet.Err)

            api.StopStream()
            continue
        }
        result := tweet.Data.(StreamDataExample)
        fmt.Println(result.Data.Text)
    }
    fmt.Println("Stopped Stream")
}

After we have started the stream and before we start processing the tweets, we defer the method itself. This will handle reconnections to twitter whenever the messages channel closes.

Final Thoughts

I hope you find this library useful in streaming tweets from twitter. Building this library was a challenge, and I learned how Go’s concurrency model works. If you liked this post, follow me on twitter as I document my journey in the software world.

Three Steps I Took to Get a Job Offer From Amazon

Alex Fallenstedt — Thu, 17 Dec 2020 21:06:10 +0000

Last year, I took the plunge and applied for a Frontend Engineering Position at Amazon. I knew little about the interview process and I had just one month to prepare for it. The result? I got an offer. In fact, the use of the interview techniques below granted me the opportunity to choose between two competing job offers. In this guide I will reveal the steps I took in order to become a competitive candidate.

Review Your Accomplishments

On the day of my Amazon interview I chatted with eight people over the period of five hours. Each interviewer was curious to know how my accomplishments have impacted companies I've worked for. To prepare, I spent hours combing through completed JIRA tickets, slack conversations, and logs to measure the impact of my contributions.

I used the interview prep grid from Cracking the Coding Interview, 6th Edition to organize information about my past accomplishments. This interview prep grid addresses common topics an interviewer would be curious about— such as leadership skills, conflict resolution, and experiences with failure. Start documenting your accomplishments with this google sheet template I made.

Completing the interview prep grid is fundamental to having a successful interview. In the grid, columns represent projects or jobs you've had and rows represent topics the interviewer will be curious about. Complete the grid to analyze the role you played in high impact projects and you will discover talking points for your interview.

I continue to fill out my grid, even now, because having a written record of my work experiences will simplify things if I ever need to prepare for another interview. It's important to keep a record of both failures and accomplishments. I focus on describing my own accomplishments as opposed to my team's accomplishments. To begin brainstorming your work experiences as a software engineer, ask yourself these questions:

What projects have you led? What was the challenging about them?
Have you taken down production before? How did you help resolve the situation?
What have you made simpler at work? What was the outcome?

For reference, below is an example of a column from my interview prep grid that I used at my Amazon interview.

Common Questions	Angular App Refactor
Challenges	* Refactor Angular app to reduce redundant network requests by 78%.
Mistakes/Failures	* Took down production with a single line of untested code being changed
Enjoyed	* Teaching the team how observables work and pub sub design pattern
Leadership	* Identified production outage via 1 star review. created lambda canary coverage to ensure we are alerted to the minute.
Conflicts	* Teaching the team how observables work and pub sub design pattern.
What you would do differently	* I would have started teaching the team how to use RxJS sooner

Prepare for Behavioral Interview Questions

A large part of the interview focuses on Amazon's Leadership Principles. Take the time to understand the meaning of these leadership principles and how you relate to them. You've probably followed these leadership principles throughout your career without necessarily being conscious of them. Be prepared to share examples of how you have followed these principles. Make an effort to really study these principles and memorize how they have guided your work experiences.

My interview process felt quite lengthy. To prepare for several hours of discussion, I searched online for sample Amazon interview questions and made flashcards. Every day, I practiced answering the flashcard questions using examples from my interview prep gird. I wanted to make sure I could confidently answer any behavioral interview question that would come my way.

Let's walk through an example interview question and formulate a response using the Situation Action Result (SAR) method:

Tell me about a time you had to quickly adjust your work priorities to meet changing demands?

You can create a response to this type of question by using the interview prep grid to focus on key points that describe your specific work experiences. As you answer, provide evidence that justifies the decisions you have made. At the end of your response, expect the interviewer to dive deeper. I cannot stress how important it is to keep your response reasonably brief so that a discussion can follow. The content of your response is just as important as the way you respond.

The SAR method is a trustworthy interview technique used to create structured and succinct responses to behavioral questions. Using this method will help your interviewer easily follow your thought process. SAR stands for situation, action, result.

Situation: Context and relevant details of your example situation
Action: The steps you took to address the situation, including your decision-making process
Result: The direct effects and measurable result of your efforts

Here is how I answered the above question using the SAR method:

(Situation) I was notified by our call center that many customers were complaining about our website being down. (Action) A quick test showed parts of our Angular components were loading poorly in production, but loading fine in development. I knew we deployed last week, so the issue must have been caused by a third party library being injected at runtime. I inspected what network events occurred as I interacted with the DOM and discovered hotjar, a behavior analytics tool, as the culprit. (Result) I made the code change to remove the 3rd party library from our app, deployed to production, and got our app back online in about 10 minutes. I can go into more detail if you'd like.

In my interview response I was able to highlight these three Amazon leadership principles:

Customer Obsession: I addressed the situation immediately because we risked losing customer trust in our product.
Ownership: Instead of saying, "That's not my job," I took ownership of the problem and quickly began working on a solution.
Bias for Action: I knew that my fix would hinder our marketing department's data collection efforts, but the damage this plugin was causing was greater than the value of analytics, and so I took action.

I recommend making flashcards of sample behavioral interview questions and practicing daily using the SAR method and referencing your interview prep grid. Here are some sample Amazon interview questions to get you started:

Tell me about a time when you were faced with a problem that had a number of possible solutions. What was the problem and how did you determine the course of action? What was the outcome of that course of action?
When did you take a risk, make a mistake, or fail? How did you respond and how did you grow from that experience?
Describe a time you took the lead on a project.
What did you do when you needed to motivate a group of individuals or promote collaboration on a particular project?
How have you leveraged data to develop a strategy?
Describe a time when you took on work outside of your comfort area. How did you identify what you needed to learn to be successful? How did you go about building expertise to meet your goal? Did you meet your goal?

Confidently Solve Whiteboard Questions

If you're interviewing for an engineering role at Amazon, then you will have whiteboard questions. You may be thinking, "Ugh, whiteboard questions are awful." But if you think whiteboard questions are gruesome, then you're probably not approaching them with the right mindset.

Amazon interviewers ask you to solve whiteboard questions because they want to evaluate your problem solving and communication skills. Interviewers want to see how you approach challenging problems that you've never encountered before. Don't worry about being unable to solve whiteboard questions—the interviewers are more interested in seeing how you work towards the solution than if you are actually able to provide the correct answer.

I had three difficult whiteboard questions that ranged from algorithms, architecture, and designing part of a game. Each question asked was unfamiliar to me despite having studied computer science algorithms in preparation. To answer these questions, I used several techniques to answer these questions to the best of my ability.

Ask questions to clarify the problem before you start

The quality of your attempt at solving a tough question depends on how well you define the problem you are solving. By asking clarifying questions about the problem, you are laying the groundwork to help you discover acceptance criteria.

In one whiteboard question, I was tasked with creating a "move" function for a board game called gobblet. I spent 10 minutes asking my interviewer clarifying questions before I started solving the question. I wrote my thoughts on the whiteboard as I asked questions about expectations, rules for a valid move, and how a winner is determined. By asking clarifying questions I discovered that the problem I was solving for was not just "moving" pieces, but designing application state and having the game be replayable. Once my questions were answered, only then did I begin solving the problem.

If you do not gather information before you start solving a problem, you can miss a lot of valuable context which can improve the quality of your answer. Clarifying questions identify the crux of the problem and steer you away from premature conclusions.

Voice your thought process so that the interviewer understands what you are thinking at all times.

It is important to communicate your thought process when solving your whiteboard question, especially if you are stuck. Amazon whiteboard questions are meant to explore your critical thinking skills as you tackle an unfamiliar problem. When you communicate your thoughts with your interviewer, you are showing them how you are planning to solve for the solution even if you are uncertain how to solve it.

One of my Amazon Interview questions was to find a value in a sorted and rotated array without using the Array object's built-in methods. I told my interviewer that I've never built a binary search algorithm before, but I had a general idea of how it worked. I explained that if I were to search for a name in a phone book, which I knew began with the letter "M", it would be ridiculous if I started at the first page. Ideally, I'd start somewhere in the middle and see if I need to go forward or backward based on where I landed. To solve for the rotated array, I continued with the phone book analogy and discovered my first step would be finding the index where the rotation starts. Knowing where the rotation started would determine on which "side" of the array my binary search should be performed.

Voicing your thoughts with your interviewer will help you discover a solution if you are unsure how to solve it. When you have constant dialogue with your interviewer, it helps them understand how you think and helps you explore solutions to unfamiliar problems.

What's next?

I hope you feel inspired to do well at your next interview, even if it's not at Amazon. My Amazon interview was the most challenging interview I've ever had, and I learned that careful review of my accomplishments, public speaking, and critical thinking are the foundation for any successful interview.

If you liked this post, follow me on twitter as I document my journey in the software world. I frequently share my knowledge about Rust, Go and Typescript and enjoy meeting others who have the desire to learn.

My Mistakes Making a Canvas Based Game with Rust and WebAssembly

Alex Fallenstedt — Fri, 09 Oct 2020 03:59:23 +0000

I began looking into WebAssembly as a means to process pixels from a video feed. That experience was very fun and rewarding. I wanted another Rust and WebAssembly challenge, so I built Gobblet, a two player game played on a 4x4 board with 12 pieces of different sizes.

Fallenstedt / rust-goblet

A canvas-based game built using Rust and WebAssembly

Gobblet is a game where players take turns to place a piece on the board or move their piece that already is on the board. Bigger pieces can be placed so that they cover smaller ones. A player wins by placing four pieces of the same color in a horizontal, vertical, or diagonal row. Basically Gobblet is advanced tic tac toe.

// Detect dark theme var iframe = document.getElementById('tweet-1289664957609472000-127'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1289664957609472000&theme=dark" }

Below is a summary of what I learned, and what I will avoid the next time I use Rust and WebAssembly.

Adding Interaction

I usually reach for RxJS when building complex user interaction, but not today. I was building Gobblet with Rust because I was curious to know where the pain points can exist.

I began by creating event listeners to process three mouse events: mouse down, mouse move, and mouse up. These three events combined represent a drag event, and can be used to influence the state of the game. After each mouse up event, I used Rust to see if either player had 4 pieces in a row.

// inside of lib.rs
#[wasm_bindgen]
pub fn start_game(canvas: HtmlCanvasElement, name1: String, name2: String) {

    // process mousedown
    {
        let closure = Closure::wrap(Box::new(move |event: web_sys::MouseEvent| {
            // do stuff
        }) as Box<dyn FnMut(_)>);
        canvas
            .add_event_listener_with_callback("mousedown", closure.as_ref().unchecked_ref())
            .unwrap();
        closure.forget();
    }

    // process mouse move
    {


        let closure = Closure::wrap(Box::new(move |event: web_sys::MouseEvent| {
            // do stuff
        }) as Box<dyn FnMut(_)>);
        canvas
            .add_event_listener_with_callback("mousemove", closure.as_ref().unchecked_ref())
            .unwrap();
        closure.forget();
    }

    //process mouse up
    {
        let closure = Closure::wrap(Box::new(move |event: web_sys::MouseEvent| {
            // do stuff
        }) as Box<dyn FnMut(_)>);
        canvas
            .add_event_listener_with_callback("mouseup", closure.as_ref().unchecked_ref())
            .unwrap();
        closure.forget();
    }
}

I'll be honest, adding an event listener is a bit over the top in Rust. If I would do this over again, I would not use Rust to add or remove event listeners. What's happening here is we are creating a closure defined in Rust and passing it to JavaScript. We pass this to JS because the closure must persist beyond the call to this one function start_game.

Also of note here is the .as_ref().unchecked_ref() chain, which is how you can extract &Function, what web-sys expects, from a Closure which only hands you &JsValue via AsRef.

👍 This is when I started to regret adding event listeners with Rust.

It's a lot. I don't enjoy it. Just creating these closures was the tip of the iceberg. I then had to create Reference Counters for my game logic and game graphics, and pass clones of these to each closure. In the majority of cases, ownership is clear: you know exactly which variable owns a given value. However, there are cases when a single value might have multiple owners. For example, in graph data structures, multiple edges might point to the same node, and that node is conceptually owned by all of the edges that point to it. A node shouldn’t be cleaned up unless it doesn’t have any edges pointing to it.

To enable multiple ownership, Rust has a type called Rc<T>, which is an abbreviation for reference counting. The Rc<T> type keeps track of the number of references to a value which determines whether or not a value is still in use. If there are zero references to a value, the value can be cleaned up without any references becoming invalid.

To complicate things further, these mouse events will mutate the state of the game logic and the game graphics. So I opted for RefCell<T> to handle the mutation of data in a single area of memory.

Then, when we use Rc<RefCell<T>>, we are saying we have shared, mutable ownership of data in our application. You can check out this pattern here in the Rust Book

//lib rs
pub fn start_game(canvas: HtmlCanvasElement, name1: String, name2: String) {
    let graphics = Rc::new(RefCell::new(Graphics::new(canvas.clone())));
    let manager = Rc::new(RefCell::new(Manager::new(name1, name2)
    // process mousedown
    {
        let graphics = graphics.clone();
        let manager = manager.clone();
//...

Managing logic

This was the probably the most fun I had when building this game. I created a simple game manager to coordinate the game state as the user interacts with the board. It consists of two players, a board, and the current turn.

// manager.rs
#[derive(Debug)]
pub struct Manager {
    player1: Player,
    player2: Player,
    board: Board,
    turn: PlayerNumber,
}

impl Manager {
    pub fn new(name1: String, name2: String) -> Self {
        let board = Board::new();
        let player1 = Player::new(name1, PlayerNumber::One);
        let player2 = Player::new(name2, PlayerNumber::Two); 

        Manager{ player1, player2, board, turn:  Manager::random_turn() }
    }
}

The Board is a struct used to manage pieces on the board. It is a two dimensional vector of Cells. Pieces can be added or removed from a cell.

pub struct Board {
    cells: Vec<Vec<Cell>>,
}

impl Board {
    pub fn new() -> Board {
        Board { cells: Board::build_cells() }
    }
    pub fn add_piece_to_board(&mut self, coord: &Coord, gobblet: Gobblet) -> Option<Gobblet> {
        let r = *coord.get_row() as usize;
        let c = *coord.get_column() as usize;
        let cell = &mut self.cells[r][c];

        return if cell.can_add(&gobblet) {
            cell.add(gobblet);
            None
        } else {
            Some(gobblet)
        }
    }

    pub fn remove_piece_from_board(&mut self, coord: &Coord, player: &PlayerNumber) -> Option<Gobblet> {
        let r = *coord.get_row() as usize;
        let c = *coord.get_column() as usize;
        let cell = &mut self.cells[r][c];

        return if cell.can_remove(&player) {
            Some(cell.remove())
        } else {
            None
        }
    }
    // Create 2 dimensonal array of cells. 
    // index in first vec represents row
    // index in second vec represent column
    // [
    //  [c, c, c, c],
    //  [c, c, c, c],
    //  [c, c, c, c],
    //  [c, c, c, c]
    // ]
    fn build_cells() -> Vec<Vec<Cell>> {
        vec![vec![Cell::new(); 4]; 4]
    }
}

Calculating the winning move was also a fun challenge. After every move, either player could win, A chosen gobblet removed from a cell could reveal the opposing player's gobblet. If that chosen gobblet is not placed well, then the opposing player could win!

I opted for a function which accepts a PlayerNumber enum as a parameter. I then counted the pieces on the board using gasp a double for-loop. This isn't ideal, I should have kept track of the player's positions per move but this was the quickest solution.

If the player has 4 pieces in any given row, column, diagonal or anti-diagonal, then they won:

// board.rs
    pub fn has_won(&self, number: PlayerNumber) -> bool {
        let mut rows: [u8; 4] = [0, 0, 0, 0];
        let mut columns: [u8; 4] = [0, 0, 0, 0];
        let mut diagonal: u8 = 0;
        let mut anti_diagonal: u8 = 0;
        for (r, row) in self.cells.iter().enumerate() {
            for (c, cell) in row.iter().enumerate() {
               if cell.is_empty() {
                   continue;
               }
                // check rows,
                // check columns,
                if player_number_match(cell.get_top_piece().get_player_number(), &number) {
                    rows[r] += 1;
                    columns[c] += 1;
                }

                // check diagonal,
                if r == c && player_number_match(cell.get_top_piece().get_player_number(), &number)  {
                    diagonal += 1;
                }

                // check anti diagonal
                if r + c == 3 && player_number_match(cell.get_top_piece().get_player_number(), &number) {
                    anti_diagonal += 1
                }
            }
        }

        return rows.contains(&4) || columns.contains(&4) || diagonal == 4 || anti_diagonal == 4
    }

Graphics

At this point, I started to question everything I was doing. I had to repaint the frame for every mouse move when the player was dragging a piece. I had to check if the player dropped the piece on a valid spot. I had to grab the "top" piece from the gobblet stack. The list of requirements for the UI seemed never ending and it was clear I spent little to no time on planning the architecture for this game.

So I kept programming

The Graphics struct kept track of what piece was clicked, and rendering all the rectangles and circles on the canvas.

#[derive(Debug, Clone)]
pub struct Graphics {
    pub interaction: Interaction,
    element: HtmlCanvasElement,
    context: CanvasRenderingContext2d,
    rectangles: Vec<Rectangle>,
    circles: Vec<Circle>,
}

impl Graphics {

    pub fn new(element: HtmlCanvasElement) -> Graphics {
        let context = element
        .get_context("2d")
        .unwrap()
        .unwrap()
        .dyn_into::<CanvasRenderingContext2d>()
        .unwrap();

        let rectangles = Graphics::create_board(&context, &element);
        let circles = Graphics::create_hand(&context);
        let interaction = Interaction::new();
        Graphics { 
            interaction,
            element, 
            context, 
            rectangles, 
            circles, 
        }
    }
}

The first challenge I had was creating the a checkerboard with Rust. I wanted it to be flexible in case I wanted to make an n * n checkerboard one day.

// graphics.rs
    fn create_board(context: &CanvasRenderingContext2d, element: &HtmlCanvasElement) -> Vec<Rectangle> {
        let light_purple: JsValue = JsValue::from_str("#6C5B7B");
        context.set_fill_style(&light_purple);
        context.fill_rect(0.0, 0.0, element.width() as f64, element.height() as f64);

        // board
        let w = 400.0;
        let h = 400.0;
        let n_row = 4.0;
        let n_col = 4.0;

        let w: f64 = w / n_row; // width of block
        let h: f64 = h / n_col; // height of block

        // colors
        let sea = JsValue::from_str("#5f506c");
        let foam = JsValue::from_str("#867297");

        let offset = (100.0, 200.0);
        let mut rectangles: Vec<Rectangle> = Vec::with_capacity(16);
        for i in 0..n_row as u8 { // row
            for j in 0..(n_col as u8) { // column
                // cast as floats
                let j = j as f64;
                let i = i as f64;

                if i % 2.0 == 0.0 {
                    if j % 2.0 == 0.0 { context.set_fill_style(&foam); } else { context.set_fill_style(&sea); };    
                } else {
                    if j % 2.0 == 0.0 { context.set_fill_style(&sea); } else { context.set_fill_style(&foam); };
                }

                let x = j * w + offset.0;
                let y = i * h + offset.1;

                let coord = Coord::new(i as u8, j as u8);
                let path = Path2d::new().unwrap();
                path.rect(x, y, w, h);
                context.fill_with_path_2d(&path);

                let rectangle = Rectangle::new(path, coord, x + (0.5 * w), y + (0.5 * h));
                rectangles.push(rectangle);
            }
        }
        rectangles
    }

This took a good minute to figure out, but once I did I was so excited. If you're curious, this is what happens when n_row and n_col are set to 40.0

Things began to take a turn for the worse when I attempted to figure out which circle was clicked. Using the coordinates of the click event, I could determine which Path2D was chosen.

It was here when realized I was using functional decomposition to address complexity, and as a result, I was adding more complexity to my program. But that's ok, because I know that I will never build something like this in Rust ever again.

//graphics.rs
    pub fn set_largest_clicked_circle(&self, x: f64, y: f64) {
        let mut index: isize = -1;
        let mut clicked_circles = Vec::new();

        for (i, c) in self.circles.iter().enumerate() {
            if self.context.is_point_in_path_with_path_2d_and_f64(c.get_path(), x, y) {
                clicked_circles.push((i, c));
                index = i as isize;
            } 
        }

        if clicked_circles.len() == 0 {
            self.interaction.set_chosen_circle(index);
            return;
        }

        // sort circles by largest -> smallest
        clicked_circles.sort_by(|a, b| b.1.get_size().partial_cmp(&a.1.get_size()).unwrap());
        index = clicked_circles.get(0).unwrap().0 as isize;
        self.interaction.set_chosen_circle(index)
    }

What did I learn?

Rust and WebAssembly are amazing tools. I can really see the impact they will have in the Web Development scene for many years. The primary goal of WebAssembly is to enable high-performance applications to run on web pages. Rust and WebAssembly are special in that they make it possible to write web applications when reliable performance is essential. With Rust and WebAssembly, you can build remote video augmented reality, scientific visualizations and simulations, games, image/video editing applications, and more.

That being said, using Rust and WebAssembly to build this canvas based game was equivalent to using a bulldozer to hammer in a nail.

Confusion is a feeling that precedes learning something. It means you should pay closer attention, not disengage. I learned so much about Rust then I ever did reading books and articles about it. When I built this game, there were days I considered giving up because I was stuck on a Rust problem. But I kept researching and experimenting. I am proud of what I built, even though it's not flexible and it's overly complex. This experience will help inform my decisions the next time I need to use Rust and WebAssembly.

Using Rust and WebAssembly to Process Pixels from a Video Feed

Alex Fallenstedt — Tue, 30 Jun 2020 23:53:05 +0000

At Streem we are on a mission to make the world's expertise more accessible. We create guidance tools to steer the discussion and ensure accurate understanding the first time. One of the guidance tools we are developing for web is a 3d cursor that can be positioned in a remote video. To accomplish this, we need to process a lot of raw pixel data and AR data per frame.

Positioning remote artifacts in AR involves a lot of computation between animation frames. It involves so much computation that it is simply too much to cover in one article. In this post, I will discuss how we used Rust to access raw pixel data from a video frame.

If you would rather jump straight to the code, then hop over here and give this repo a ⭐

What is Web Assembly?

WebAssembly (wasm) is a type of code that can be run in web browsers and mobile devices. Wasm was designed to be a compilation target for low-level languages like C, C++, and Rust. With wasm, web browsers and mobile devices can now run code written in multiple languages at near-native speeds by taking advantage of common hardware capabilities.

Wasm was introduced to all modern web browsers to help extend the capabilities of JavaScript. Since JavaScript has complete control over how WebAssembly code is downloaded, compiled and run, JavaScript developers can think of wasm as a feature for efficiently creating high-performance functions.

In this demo, we used WebAssembly to extract raw pixel data from a remote video feed. This guide will cover high level details about web assembly. It will not cover setting up a web assembly project. There are tools and tutorials to help you get started with your next web assembly project. If you are completely new to Rust, then you should watch Tensor Programming's Intro to Rust playlist

How do I process pixels from a remote video feed?

To process raw pixel data for every frame of a video, we used a video track from a MediaStream object, which was then used to create an HtmlVideoElement. The video element can then be used as a source for a canvas to draw an image with. With the image drawn onto a canvas at 60fps, we have access the raw underlying pixel data with CanvasRenderingContext2D.getImageData().

Below is a high level diagram demonstrating how you can put individual video frames onto a canvas element. With the video frame drawn onto a canvas element, you will have access to raw pixel data.

Once we knew how to access raw pixel data from a frame, we brought in Rust and wasm. We wanted the interface between JavaScript and Rust to be simple, so we had our RenderingEngine be responsible for two things

Registering target canvases for our processed video frame to render onto
Processing every frame from a video feed

Registering Target Canvases

A target canvas is where our processed video frames would render.

After dynamically loading our wasm, we can invoke add_target_canvas to register a rendering destination for our RenderingEngine

const renderingEngine = new wasm.RenderingEngine();
renderingEngine.add_target_canvas(canvas)

The RenderingEngine is a struct which consumes three private fields

canvas the buffer canvas to parse LightShow data on
render_targets A vector of canvas elements to render the final frames onto
cancel A signal to stop rendering frames onto a canvas

pub struct RenderingEngine {
  canvas: Rc<RenderingEngineCanvas>,
  render_targets: Rc<RefCell<Vec<RenderingEngineCanvas>>>,
  cancel: Rc<RefCell<bool>>,
}

Each of these fields is wrapped in Rust's Reference Counter (Rc). Rcs enable shared ownership of data. A Rc is used when we need several references to an immutable value at the same time. Rc pointers are distinct from Rust's usual references in that, while they are allocated on the heap, cloning a Rc pointer does not cause a new heap allocation. Instead, a counter inside the Rc is incremented. We will see how this is used with our animation loop. This is needed because we can't use lifetimes with wasm_bindgen. See this issue.

Inside our Rc is a RefCell, which provides us a way to mutate data when there are immutable references to that data. We will need add many render_targets and mutate our cancel flag as our application is used at runtime. In a nutshell, a RefCell let's you get &mut references of your contents. When we use Rc<RefCell<T>>, we are saying we have shared, mutable ownership of data in our application.

In Rust, add_target_canvas is a public method exposed with wasm_bindgen. It's important to note this method uses &mut self. This reference type allows you to modify self without taking ownership of it.

#[derive(Debug)]
struct RenderingEngineCanvas {
    element: HtmlCanvasElement,
    context_2d: CanvasRenderingContext2d,
}

#[wasm_bindgen]
#[derive(Debug)]
pub struct RenderingEngine {
    canvas: Rc<RenderingEngineCanvas>,
    render_targets: Rc<RefCell<Vec<RenderingEngineCanvas>>>,
    cancel: Rc<RefCell<bool>>,
}

#[wasm_bindgen]
impl RenderingEngine {

    #[wasm_bindgen(constructor)]
    pub fn new() -> RenderingEngine {
        let canvas = Rc::new(RenderingEngine::create_buffer_canvas());
        let render_targets = Rc::new(RefCell::new(Vec::new()));
        let cancel = Rc::new(RefCell::new(false));

        RenderingEngine {
            canvas,
            render_targets,
            cancel,
        }
    }

    #[wasm_bindgen(method)]
    pub fn add_target_canvas(&mut self, canvas: HtmlCanvasElement) {
// Obtain 2D context from canvas
        let context = canvas
            .get_context("2d")
            .unwrap()
            .unwrap()
            .dyn_into::<CanvasRenderingContext2d>()
            .expect("failed to obtain 2d rendering context for target <canvas>");

// Create a struct
        let container = RenderingEngineCanvas {
            element: canvas,
            context_2d: context,
        };

// Update instance of rendering engine
        let mut render_targets = self.render_targets.borrow_mut();
        render_targets.push(container);
    }
}

Processing every frame from a video feed

Processing every frame from a video feed is more involved. I will remove a lot of finer details, however, you can explore the github repo for a complete code example

From JavaScript, we can invoke our animation loop with a start method. It's only argument is MediaStream object which is obtained by requesting the user's media

const renderingEngine = new wasm.RenderingEngine();
renderingEngine.add_target_canvas(canvas)

const userMedia = await navigator.mediaDevices.getUserMedia(someContraints);
renderingEngine.start(userMedia);

In Rust, we create an HTMLVideoElement and start our animation loop. With start_animation_loop, we clone the values we will be using in our animation loop.

video is needed so we can obtain it's dimensions and frames from.
canvas is our buffer canvas so we can proccess our pixel data
cancel is a signal we can use to trigger a stop to our animation loop
render_targets are all the target canvases on JS that need render our final image onto.

There's also two new constants f and g. We want to call requestAnimationFrame every frame until our video ends. After the video source ends we want all our resources cleaned up. We will use f to store our closure we want to execute on each frame, and g to kick it off for us.

The closure we create is stored on g for the first frame. We call borrow_mut to get a mutuable reference to value inside RefCell::new(None).

We learned a lot about this from this PR at rustwasm and how to capture an environment within an anonymous function

    #[wasm_bindgen(method)]
    pub fn start(&self, media_stream: &MediaStream) {
        let video = RenderingEngine::create_video_element(media_stream);
        &self.start_animation_loop(&video);
    }

   fn start_animation_loop(&self, video: &Rc<HtmlVideoElement>) {
        let video = video.clone();
        let canvas = self.canvas.clone();
        let cancel = self.cancel.clone();
        let render_targets = self.render_targets.clone();

        let f = Rc::new(RefCell::new(None));
        let g = f.clone();

        *g.borrow_mut() = Some(Closure::wrap(Box::new(move || {
          // clean up f when cancel is set to true
          if *cancel.borrow() == true {
                let _ = f.borrow_mut().take();
                return;
          }
          // continuously animate with the value of f.
          RenderingEngine::request_animation_frame(
            f.borrow().as_ref().unwrap()
        }) as Box<dyn FnMut()>));

// start the animation loop here for 1 frame, drop g.  

RenderingEngine::request_animation_frame(g.borrow().as_ref().unwrap());
  }


// Note this method call, which uses `as_ref()` to get a `JsValue`
// from our `Closure` which is then converted to a `&Function`
// using the `JsCast::unchecked_ref` function.
   fn request_animation_frame(n: &Closure<dyn FnMut()>) {

      RenderingEngine::get_window()
         .request_animation_frame(n.as_ref().unchecked_ref())
         .expect("should register `requestAnimationFrame` OK");
   }

With a function wrapped in a Closure for JavaScript to execute, we can process our video frames' pixel data. I will make the code example below simple, however, you can find the original code here.

// inside our animation loop

// obtain video dimensions
let video_dimensions = Dimensions {
  width: video.video_width() as f64,
  height: video.video_height() as f64,
};

// draw frame onto buffer canvas
// perform any pixel manipulation you need on this canvas
canvas.element.set_width(video_dimensions.width as u32);
canvas.element.set_height(video_dimensions.height as u32);
canvas.context_2d.draw_image_with_html_video_element(&video, 0.0, 0.0).expect("failed to draw video frame to <canvas> element");


// render resulting image onto target canvas
for target in render_targets.borrow().iter() {
// Use scrollWidth/scrollHeight so we fill the canvas element.
  let target_dimensions = Dimensions {
    width: target.element.scroll_width() as f64,
    height: target.element.scroll_height() as f64,
  };
let scaled_dimensions = RenderingEngine::get_scaled_video_size(
                        &video_dimensions,
                        &target_dimensions,
                    );
let offset = Dimensions {
  width: (target_dimensions.width - scaled_dimensions.width) / 2.0,
  height: (target_dimensions.height - scaled_dimensions.height) / 2.0,
  };

// Ensure the target canvas has a set width/height, otherwise rendering breaks.                 target.element.set_width(target_dimensions.width as u32);
target.element.set_height(target_dimensions.height as u32);
target.context_2d.draw_image_with_html_canvas_element_and_dw_and_dh(
  &canvas.element,
  offset.width,
  offset.height,
  scaled_dimensions.width,
  scaled_dimensions.height,
  ).expect("failed to draw buffer <canvas> to target <canvas>");
}

If you liked this example and want to learn more about Rust, WebAssembly, and TypeScript then let me know! Leave a comment here or find me on Twitter

Compute With Flutter

Alex Fallenstedt — Sun, 29 Sep 2019 19:18:23 +0000

Dart is a single threaded language, but it comes with a handy compute function to spawn isolates. In a nutshell, the compute function is useful for doing extra work on a different "thread"--it's actually an isolate--so your flutter app does not experience "jank". Jank occurs when the UI doesn’t render smoothly. For example, every so often, a frame takes 10 times longer to render, so it gets dropped, and the animation visibly jerks.

To use compute, you need a function and data. compute will return a future of whatever your function returns.

final Future<R> result = compute(someFunction, someData)

In my app, I had to parse about 200 tweets that were scored with sentiment analysis. Each tweet has a message and created property, but the sentiment property had some intense parsing. So when I created a new TweetDto, I also had to create a new SentimentDto.

class TweetDto {
  final int id;
  final String message;
  final String created;
  final SentimentDto sentiment;

  TweetDto({
    @required this.id,
    @required this.message,
    @required this.created,
    @required this.sentiment,
  });

  factory TweetDto.fromJson(Map<String, dynamic> json) {
    return TweetDto(
      id: json['id'],
      message: json['message'],
      created: json['created'],
      sentiment: SentimentDto.fromJson(json['sentiment']),
    );
  }
}


class SentimentDto {
  final List<_SentimentSentence> sentences;
  final _SentimentSummary documentSentiment;
  final String language;

  SentimentDto({
    @required this.sentences,
    @required this.documentSentiment,
    @required this.language,
  });

  factory SentimentDto.fromJson(Map<String, dynamic> json) {
    final List<dynamic> sentencesRef = json['sentences'];
    final List<_SentimentSentence> sentences = List<_SentimentSentence>.from(
      sentencesRef.map(
        (s) => _SentimentSentence(
          text: _SentimentText(
            content: s['text']['content'],
            beginOffset: (s['text']['beginOffset']).toDouble(),
          ),
          sentiment: _SentimentSummary(
            score: (s['sentiment']['score']).toDouble(),
            magnitude: (s['sentiment']['magnitude']).toDouble(),
          ),
        ),
      ),
    );
    final documentSentiment = _SentimentSummary(
      score: (json['documentSentiment']['score']).toDouble(),
      magnitude: (json['documentSentiment']['magnitude']).toDouble(),
    );
    final language = json['language'];

    return SentimentDto(
      sentences: sentences,
      documentSentiment: documentSentiment,
      language: language,
    );
  }
}

class _SentimentSentence {
  final _SentimentText text;
  final _SentimentSummary sentiment;

  _SentimentSentence({
    @required this.text,
    @required this.sentiment,
  })  : assert(text != null),
        assert(sentiment != null);
}

class _SentimentText {
  final String content;
  final double beginOffset;

  _SentimentText({
    @required this.content,
    @required this.beginOffset,
  });
}

class _SentimentSummary {
  final double magnitude;
  final double score;

  _SentimentSummary({
    @required this.magnitude,
    @required this.score,
  })  : assert(magnitude != null),
        assert(score != null);
}

I experienced some performance when I was parsing about 200 tweets each with their own sentiment score, so I looked to compute all of this json parsing with Flutter's compute. In my TweetBloc, I had a function called loadTweets which was responsible for fetching 200 scored tweets. You'll see that I switch to new Observable with Observable.fromFuture(...) in order to keep this operation reactive.

...
 void loadTweets() {
    _authService
        .get(ApiConsts.getTweets(200))
        .switchMap((signal) => Observable.fromFuture(
            compute(TweetBloc._decodeAndParseTweets, signal.body)))
        .take(1)
        .listen(
      (List<TweetDto> signal) {
        _tweets.add(signal);
      },
    );
  }

..

Because compute takes a function that performs the computation, I created a static function decodeAndParseTweets which parses all of my json.

 void loadTweets() {
    _authService
        .get(ApiConsts.getTweets(200))
        .switchMap((signal) => Observable.fromFuture(
            compute(TweetBloc._decodeAndParseTweets, signal.body)))
        .take(1)
        .listen(
      (List<TweetDto> signal) {
        _tweets.add(signal);
      },
    );
  }

  static List<TweetDto> _decodeAndParseTweets(String jsonBody) {
    final List<dynamic> decodedJson = jsonDecode(jsonBody);
    final List<TweetDto> tweets =
        decodedJson.map((t) => TweetDto.fromJson(t)).toList();
    return tweets;
  }

Flutter's compute helped prevent jank in my flutter app by offloading intense parsing of data onto a new isolate. Consider using it in the future when you have any intense data processing.

Testing NgRx Effects

Alex Fallenstedt — Wed, 24 Apr 2019 03:51:15 +0000

You are walking down the street sipping some coffee and you ask yourself, “How can I test my ngrx@7.4.0 effects?”. If you’re asking that question because you're not satisfied with the developer experience of jasmine-marbles then this might help you.

A Basic Effect

@Injectable()
export class RouterHistoryEffects {

 @Effect()
 addRouteToHistory = this.actions.pipe(
   ofType(ROUTER_NAVIGATED),
   map((action: RouterNavigatedAction) => action.payload.routerState.url),
   switchMap((url: string) => {
     return of(new fromRouterHistoryActions.AddRouteToHistory(url));
   })
 );

 constructor(private actions: Actions<RouterHistoryActions>) { }
}

Here we have an action that is dispatched from somewhere. It’s supposed to return a new action. When this is consumed by ngrx, addRouteToHistory subscribes to the action stream and can publish to your action stream. More reading here

Knowing this, our test should perform the same scenario. We should subscribe to addRouteToHistory and listen for events to happen.

import { TestBed, async } from '@angular/core/testing';
import { provideMockActions } from '@ngrx/effects/testing';
import { ReplaySubject } from 'rxjs';

import { RouterHistoryEffects } from './router-history.effects';
import { AddRouteToHistory } from './router-history.actions';
import { ROUTER_NAVIGATED } from '@ngrx/router-store';

describe('RouterHistoryEffects', () => {
 let actions: ReplaySubject<any>;
 let effects: RouterHistoryEffects;

 beforeEach(() => {
   TestBed.configureTestingModule({
     providers: [
       RouterHistoryEffects,
       provideMockActions(() => actions)
     ]
   });

   effects = TestBed.get(RouterHistoryEffects);
 });

 it('should be created', async() => {
   expect(effects).toBeTruthy();
 });

 it('#addRouteToHistory should dispatch fromRouterHistoryActions.AddRouteToHistory with a url', async(async() => {
   actions = new ReplaySubject(1);
   const routerNavigatedAction = {
     type: ROUTER_NAVIGATED,
     payload: {
       routerState: {
         url: '/member'
       }
     }
   };
   actions.next(routerNavigatedAction);
   effects.addRouteToHistory.subscribe((result: AddRouteToHistory) => {
     expect(result.payload).toEqual('/member');
   });
 }));
});

We have a standard operating procedure for creating our testing module except for provideMockActions. This function (which looks like this) accepts an Observable, or a function that returns an Observable. Here, we are creating a factory that generates a ReplaySubject. We can publish to the ReplaySubject stream, and listen to it. This is exactly what our effect does:

…
 @Effect()
 addRouteToHistory = this.actions.pipe( // do stuff then return an observable )
…

In our test, we can create a subscription to our effect addRouteToHistory and replay an action that was ‘dispatched’.

   actions = new ReplaySubject(1);
   const routerNavigatedAction = {
     type: ROUTER_NAVIGATED,
     payload: {
       routerState: {
         url: '/member'
       }
     }
   };
   actions.next(routerNavigatedAction);
   effects.addRouteToHistory.subscribe((result: AddRouteToHistory) => {
     expect(result.payload).toEqual('/member');
   });

Hopefully this helps you. Depending if your store obtains slices of state from your store, you may need to create a mock store too! I’ll spare you the details, but it’s very similar to providing mock actions:

//use provideMockStore to provide an instance of MockStore: 
//https://ngrx.io/api/store/testing/MockStore


// configure testing module in before each


import { provideMockStore, MockStore } from '@ngrx/store/testing';
import { provideMockActions } from '@ngrx/effects/testing';

//...
    TestBed.configureTestingModule({
      providers: [
        YourEffect
        provideMockStore({ initialState }),
        provideMockActions(() => actions)
      ]
   })

    mockStore = TestBed.get(Store);
//...


 it('redirect to privacy policy page', async( () => {
// set testing state
   mockStore.setState({ [ROUTER_HISTORY_FEATURE_KEY]: { previousRoutes: [] } });

// dispatch your action
actions = new ReplaySubject(1);
   actions.next({
     type: ROUTER_REQUEST,
     payload: {
       routerState: {
         url: `/${NavigationRoutes.PrivacyPolicy}`
       },
     }
   });
// listen for stream to emit.
   sub = effects.redirectToHome.subscribe((data) => {
    expect(navigationServiceSpy.navigateToPrivacyPolicy).toHaveBeenCalled();
    expect(data).toBeTruthy(); // or whatever you’re testing
   });

 }));

Building Widgets On Demand with Flutter

Alex Fallenstedt — Sun, 10 Feb 2019 21:26:18 +0000

There are many ways to display a large list of items when building your app. Say you had to display an infinite amount of list items that a user can scroll through. Naively, you can fetch thousands of list items and build them all. This can work, however, you most likely will have frustrated users with insane load times.

So how can we build our list items, or widgets, on demand? Using ListView and FutureBuilder we can accomplish this. I'm going to start by creating a new Flutter application, deleting everything in main.dart and replace it with the following:



// main.dart
import 'package:flutter/material.dart';
import 'src/app.dart';

void main() {
  runApp(App());
}

Our App is a basic MaterialApp which holds a OnDemandList widget. This will be home to our 'infinite' list of items.



// app.dart
import 'package:flutter/material.dart';
import './screens/big_list.dart';

class App extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return child: MaterialApp(
        title: 'OnDemandList',
        home: BigList(),
      );
  }
}

Before we think about lazy loading items...let's focus first on putting lots of items in our app. Here, we are using ListView.builder() to create a scrollable, linear array of widgets that are created on demand. The neat thing is that even though we have specified 1000 items, it will only build the amount of items that are visible on the screen.

How it accomplishes this is by taking the first item in the list and measuring its height. In this example, the height of the item is 80. Flutter can measure the height of the screen and build the amount of items needed.



// big_list.dart';
import 'package:flutter/material.dart';
import 'dart:async';


class BigList extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
        appBar: AppBar(
          title: Text('Lots of Items'),
        ),
        body: buildList(),
    );
  }

  Widget buildList() {
    return ListView.builder(
      itemCount: 1000,
      itemBuilder: (BuildContext context, int index) {
        return Container(
          height: 80,
          child: Text("$index hello World"),
        );
      },
    );
  }
}

Each time we scroll, Flutter is building the list items that appear in our view. This is what we are aiming for, however, say each list item needs to make an async request to fetch data. We can use FutureBuilder. Flutter is async by nature, and Futures allows us to work with data without worrying about threads or blocking our application. FutureBuilders easily allow you to show different states while your data is loading, and on complete, show a specified Widget. Here is a very simple example without checking for errors or connection states:



// big_list.dart';

import 'package:flutter/material.dart';
import 'dart:async';

class BigList extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
        appBar: AppBar(
          title: Text('Lots of Items'),
        ),
        body: buildList(),
    );
  }

  Widget buildList() {
    return ListView.builder(
      itemCount: 1000,
      itemBuilder: (BuildContext context, int index) {
        return FutureBuilder(
          future: getFuture(), // <--- get a future
          builder: (BuildContext context, snapshot) { // <--- build the things.
            return Container(
              height: 80.0,
              child: snapshot.hasData ? snapshot.data : Text('loading..'),
            );
          },
        );
      },
    );
  }

 Future<Widget> getFuture() {
    return Future.delayed(Duration(seconds: 2), () => Text('Hello World!'));
  }
}

Notice how we see loading... for 2 seconds and then a new Text widget of "Hello World!" appearing. This is exactly what we needed, a lazy loading list of widgets. With this foundation, you now have the knowledge to build performant apps that build widgets on demand.

Have fun!

Creating environments in Flutter

Alex Fallenstedt — Mon, 04 Feb 2019 05:17:21 +0000

A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.

I started playing with Flutter and I was surprised there were not many resources on creating configs, everything that is likely to vary between deploys (staging, production, developer environments, etc). Here is a method I have found to be pretty useful.

You can always skip to see a real Flutter example, or follow along in dart pad.

Here is our basic app.

class App {
  String final someConfigMember = 'secret';
  String final someOtherConfigMember = 'some other secret';

  App() {}

  runApp() {
    print("Hello World. I am running");
    print(this.someConfigMember);
    print(this.someOtherConfigMember);
  }
}

void main() {
  new App().runApp();
}

Our main should always launch a local development environment though. Let's change the contents of this function.

//snip

void main() {
  Development();
}

This is an imaginary development environment. Let's create this development environment as a new Dart class. Our app should also be aware of the development environment through a parameter.


class App {
  final Development env;

  App(this.env) {}

  runApp() {
    print("Hello World. I am running on $env");
    print(this.env.someConfigMember);
    print(this.env.someOtherConfigMember);
  }
}


class Development {
  static Development d; 
  String someConfigMember = 'sup';
  String someOtherConfigMember = 'some private thing';

  Development() {
    // obtain snapshot of this.
    d = this;

    //shove it into our app.
    // call 'runApp'. 
    new App(d).runApp();
  }
}

void main() {
  Development();
}

Now we need a test environment so we don't do testing on production. Dart allows us to have inheritance (extending a class). Our development, testing, and production environments should extend from a base Env class. We can save ourselves from headaches by creating one class as the source of truth for many environments. Your work should now look something like this.

class Env {
  static Env d;
  String someConfigMember;
  String someOtherConfigMember;

  Env() {
    // obtain snapshot of this.
    d = this;

    //shove it into our app.
    new App(d).runApp();
  }
}


class App {
  final Env env;

  App(this.env) {}

  runApp() {
    print("Hello World. I am running on $env");
    print(this.env.someConfigMember);
    print(this.env.someOtherConfigMember);
  }
}


class Development extends Env {
  String someConfigMember = 'sup';
  String someOtherConfigMember = 'some private thing';
}


void main() => Development();

And that's it. You can create as many environments as you need and tell flutter to run specific files when building. When you decide to use this in flutter, you will need to have your app start with runApp(App(this)) in your Env class. You can see an example of this here.

Have fun

Scan Operator For Mini Redux Stores

Alex Fallenstedt — Sat, 02 Feb 2019 20:21:28 +0000

I was placed on an Angular project that did not have a state management system like Redux or ngrx. I saw this as an opportunity to gently introduce state management using RxJS.

There are n+1 blog posts about Reactive Programming. In a nutshell, reactive programming concerns itself with async data. This data can come from APIs, or from user events.

The task was to build a toast notification system. Something similar to Angular Material’s Snackbar. The requirements were:

each toast notification auto-expires
each toast notification can be closed ahead of time by a user
and there can be many toast notifications at once.

I ended up using the scan operator from RxJS to have data persistence. You can think of a scan operator as JavaScript's reduce method.

This was accomplished inside an Angular service, however, you can take these concepts in any project that uses RxJS.

// toast.service.ts
// getter so no bad developer uses the store incorrectly.
  get store() { return this._store$; }

// Dispatch "actions" with a subject
  private _action$ = new Subject();

// Create a store which is just an array of 'Toast' 
  private _store$: Observable<Toast[]> = this._action$.pipe(
    map((d: ToastAction) => (!d.payload.id) ? this.addId(d) : d), // add id to toast to keep track of them.
    mergeMap((d: ToastAction) => (d.type !== ToastActionType.Remove) ? this.addAutoExpire(d) : of(d)), // concat a hide toast request with delay for auto expiring
    scan(this.reducer, []) // magic is here!
  );

// dispatch method 
  public dispatch(action: ToastAction): void {
    this._action$.next(action);
  }

// generate ids for the toast
  private addId(d: ToastAction): ToastAction {
    return ({
      type: d.type,
      payload: { ...d.payload, id: this.generateId() }
    });
  }

// If a user does not click on the toast to clear it, then it should auto expire
  private addAutoExpire(d: ToastAction) {
    const signal$ = of(d);
    const hide$ = of({ type: ToastActionType.Remove, payload: d.payload }).pipe(delay(this.config.duration));
    return concat(signal$, hide$);
  }

// generates a random string
  private generateId(): string {
    return '_' + Math.random().toString(36).substr(2, 9);
  }

// The reducer which adds and removes toast messages.
  private reducer(state: Toast[] = [], action: ToastAction): Toast[] {
    switch (action.type) {
      case ToastActionType.Add: {
        return [action.payload, ...state];
      }
      case ToastActionType.Remove: {
        return state.filter((toast: Toast) => toast.id !== action.payload.id);
      }
      default: {
        return state;
      }
    }
  }
}

Being able to use scan was perfect for this situation. I could reduce incoming streams of data into an array of objects using the reducer function.

To use the store, you can either subscribe to the store or reference it in your Angular template.

// toast.component.ts
export class ToastComponent {
  public state$: Observable<Toast[]> = this.toastService.store;

  constructor(public toastService: ToastService) { }
  public add() {
    this.toastService.dispatch({ type: ToastActionType.Add, payload: { message: 'hi', status: ToastStatus.Info } });
  }
  public remove(payload: Toast) {
    this.toastService.dispatch({
      type: ToastActionType.Remove,
      payload
    });
  }
   ...

But is this ideal? Yes it works, however, a developer that comes by shouldn't have to care about ToastActionType and construction objects. It's a lot of work. Let's create some helper methods in our ToastService so any developer can call 'enqueueSuccess' or 'enqueueInfo' for the type of toast we want.

// toast.service.ts
@Injectable({
  providedIn: 'root'
})
export class ToastService {

  get store() { return this.store$; }

  private action$: Subject<ToastAction> = new Subject<ToastAction>();
  private store$: Observable<Toast[]> = this.action$.pipe(
    map((d: ToastAction) => (!d.payload.id) ? this.addId(d) : d),
    mergeMap((d: ToastAction) => (d.type !== ToastActionType.Remove) ? this.addAutoExpire(d) : of(d)),
    scan(this.reducer, [])
  );

  constructor() { }

  public enqueueSuccess(message: string): void {
    this.action$.next({
      type: ToastActionType.Add,
      payload: {
        message,
        status: ToastStatus.Success
      }
    });
  }

  public enqueueError(message: string): void {
    this.action$.next({
      type: ToastActionType.Add,
      payload: {
        message,
        status: ToastStatus.Error
      }
    }
    );
  }

  public enqueueInfo(message: string): void {
    this.action$.next({
      type: ToastActionType.Add,
      payload: {
        message,
        status: ToastStatus.Info
      }
    });
  }

  public enqueueWarning(message: string): void {
    this.action$.next({
      type: ToastActionType.Add,
      payload: {
        message,
        status: ToastStatus.Warning
      }
    });
  }

  public enqueueHide(payload: Toast): void {
    this.action$.next({
      type: ToastActionType.Remove,
      payload
    });
  }


  private addId(d: ToastAction): ToastAction {
    return ({
      type: d.type,
      payload: { ...d.payload, id: this.generateId() }
    });
  }

  private addAutoExpire(d: ToastAction) {
    const signal$ = of(d);
    const hide$ = of({ type: ToastActionType.Remove, payload: d.payload }).pipe(delay(this.config.duration));
    return concat(signal$, hide$);
  }

  private generateId(): string {
    return '_' + Math.random().toString(36).substr(2, 9);
  }

  private reducer(state: Toast[] = [], action: ToastAction): Toast[] {
    switch (action.type) {
      case ToastActionType.Add: {
        return [action.payload, ...state];
      }
      case ToastActionType.Remove: {
        return state.filter((toast: Toast) => toast.id !== action.payload.id);
      }
      default: {
        return state;
      }
    }
  }
}

Now our component has a much easier time calling our service:

// toast.component.ts

export class ToastComponent {
  public state$: Observable<Toast[]> = this.toastService.store;

  constructor(public toastService: ToastService) { }
  public add() {
    this.toastService.enqueueInfo('It works!');
    this.toastService.enqueueWarning('Oops!');
    this.toastService.enqueueError('Uh oh!');
    this.toastService.enqueueSucccess(':D');
  }
   ...

<!-- toast.component.html --> 
  <button 
    *ngFor="let toast of (state$ | async)"
    (click)="remove(toast)">
    <span [innerHtml]="toast.message"></span>
  </button>

And what you get are many toast notifications with a powerful service with a very nice API.