<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>Forem: Aaron Schlesinger</title>
    <description>The latest articles on Forem by Aaron Schlesinger (@arschles).</description>
    <link>https://forem.com/arschles</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F463%2FeCTtfssI.jpg</url>
      <title>Forem: Aaron Schlesinger</title>
      <link>https://forem.com/arschles</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://forem.com/feed/arschles"/>
    <language>en</language>
    <item>
      <title>Synchronizing the KEDA HTTP Addon Request Routing Table Across Hundreds of Interceptor Pods</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Fri, 29 Oct 2021 23:15:58 +0000</pubDate>
      <link>https://forem.com/arschles/synchronizing-the-keda-http-addon-request-routing-table-across-hundreds-of-interceptor-pods-4bl3</link>
      <guid>https://forem.com/arschles/synchronizing-the-keda-http-addon-request-routing-table-across-hundreds-of-interceptor-pods-4bl3</guid>
      <description>&lt;p&gt;The &lt;a href="https://github.com/kedacore/http-add-on"&gt;KEDA HTTP Addon project&lt;/a&gt; contains three major components: the &lt;a href="https://github.com/kedacore/http-add-on/tree/main/operator"&gt;operator&lt;/a&gt;, &lt;a href="https://github.com/kedacore/http-add-on/tree/main/scaler"&gt;scaler&lt;/a&gt; and &lt;a href="https://github.com/kedacore/http-add-on/tree/main/interceptor"&gt;interceptor&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Of these, the interceptor is the only component that sits in the critical path of all incoming HTTP requests. We also run them in a &lt;em&gt;fleet&lt;/em&gt; that is horizontally scaled by software.&lt;/p&gt;

&lt;p&gt;We're going to focus on how we ensure that any interceptor replica can route an incoming request to the correct backing application at any time.&lt;/p&gt;

&lt;h2&gt;
  
  
  Implications of Multi-Tenancy
&lt;/h2&gt;

&lt;p&gt;The interceptor component is designed to run in a &lt;code&gt;Deployment&lt;/code&gt; that KEDA automatically scales. This high-level design has a few implications:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;There must be a centralized, durable copy of a lookup table -- called a &lt;em&gt;routing table&lt;/em&gt; -- that maps any incoming request to the correct backing &lt;code&gt;Service&lt;/code&gt; and port.&lt;/li&gt;
&lt;li&gt;All interceptor pods must reliably stay up to date with the central routing table&lt;/li&gt;
&lt;li&gt;All interceptor pods must be able to handle any valid incoming request, regardless of application it was intended for&lt;/li&gt;
&lt;li&gt;All interceptor pods must be able to quickly execute lookups to the routing table&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Keeping up to Date with the Routing Table
&lt;/h2&gt;

&lt;p&gt;Since the interceptor needs to do a lookup to the routing table before forwarding any request (or returning an error code), lookups need to be as fast as possible. That means storing the routing table in memory and keeping each interceptor's in-memory copy up to date with the central copy.&lt;/p&gt;

&lt;p&gt;We do this wth a relatively simple event loop, outlined in the below (&lt;a href="https://golang.org"&gt;Go&lt;/a&gt;-like) pseudocode:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="n"&gt;table&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;fetch_table_from_kubernetes&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;report_alive&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;ticker&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;start_ticker&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;every_1_second&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;case&lt;/span&gt; &lt;span class="n"&gt;new_table&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt; &lt;span class="n"&gt;kubernetes_events_chan&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;table&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;new_table&lt;/span&gt;
        &lt;span class="k"&gt;case&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;ticker&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;table&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;fetch_table_from_kubernetes&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A few important points to note about this event loop:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;On startup, we fetch a complete copy of the routing table from Kubernetes before we report to the &lt;a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/"&gt;Kubernetes liveness probe&lt;/a&gt;. This means that Kubernetes doesn't consider any interceptor pod "ready" until it has a complete initial copy of the routing table.&lt;/li&gt;
&lt;li&gt;The &lt;code&gt;kubernetes_events_chan&lt;/code&gt; receives notifications about changes to the central routing table in near-real time. When we get a change notification, we immediately update the in-memory copy.&lt;/li&gt;
&lt;li&gt;The &lt;code&gt;ticker&lt;/code&gt; fires a signal every second, at which time we do a full refresh of the routing table. This mechanism ensures that all interceptor replicas receive changes to the routing table within 1 second of the change being made to the central copy.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  The Event Loop Pattern in Action
&lt;/h2&gt;

&lt;p&gt;The KEDA HTTP Addon runs a fleet of interceptors and enlists KEDA to actively scale the fleet. This means that as we send more HTTP traffic to the cluster, we expect the interceptors to automatically scale up. This behavior is one of the most important features of the HTTP Addon.&lt;/p&gt;

&lt;p&gt;We've built this event loop into the interceptors to ensure that there can be thousands [1] of them running at once, and they all stay up to date with the central routing table -- data that they need to do their job.&lt;/p&gt;




&lt;p&gt;[1] As we see in the pseudocode above, each interceptor issues requests to the Kubernetes API, so as we scale them out, we generate more consistent traffic to the cluster API. As you scale further than the low thousands of replicas, you would need to add an intermediate layer of caching between the interceptors and the API to ensure that you don't crash the cluster control plane. &lt;/p&gt;

</description>
      <category>distributedsystems</category>
      <category>eventdriven</category>
      <category>concurrency</category>
    </item>
    <item>
      <title>Fan-in / Fan-out with Go</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Mon, 11 Oct 2021 22:59:27 +0000</pubDate>
      <link>https://forem.com/arschles/fan-in-fan-out-with-go-19ah</link>
      <guid>https://forem.com/arschles/fan-in-fan-out-with-go-19ah</guid>
      <description>&lt;p&gt;Hacking on the &lt;a href="https://github.com/kedacore/http-add-on"&gt;KEDA HTTP Addon&lt;/a&gt;, I found myself having to do something familiar:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Split some work into N pieces, do them all concurrently, wait for them all to be done, and then merge all the results together.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I've done this a bunch of times before, but this time I forgot how to do it. I took a few minutes away from the computer to gather my thoughts and came back to it. So I don't have to forget how to do it again, I want to write the algorithm down here!&lt;/p&gt;

&lt;h2&gt;
  
  
  What are we doing here?
&lt;/h2&gt;

&lt;p&gt;First thing's first - we need a problem we can break down into a bunch of pieces. Sometimes it's called an "embarassingly parallel" problem. &lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Note that concurrency and parallelism aren't equivalent, but I'm going to be using the word "parallel" hereafter because I'm hoping the machine you run this algorithm on will be able to execute the work on different cores simultaneously.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The primary goal here is to run each different piece of the work in a goroutine. That's pretty easy in Go - just put &lt;code&gt;go&lt;/code&gt; before the function call that does the work. The tougher part is to get the results of the work, check for errors, and wait for them all to be done -- not necessarrily in that order 🤣.&lt;/p&gt;

&lt;p&gt;Even though the concept is simple, there is a big-ish gotcha when the rubber hits the road. Below is some code that does "fake" work, annotated with comments to explain it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="n"&gt;workToDo&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s"&gt;"do"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"some"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"work"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="k"&gt;range&lt;/span&gt; &lt;span class="n"&gt;workToDo&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c"&gt;// make sure you pass the index and work into the &lt;/span&gt;
    &lt;span class="c"&gt;// function that runs in the goroutine.&lt;/span&gt;
    &lt;span class="c"&gt;// this mechanism makes sure that the goroutine&lt;/span&gt;
    &lt;span class="c"&gt;// gets a (stack-allocated) _copy_ of the data.&lt;/span&gt;
    &lt;span class="c"&gt;// if you don't do this, idx and work will change out&lt;/span&gt;
    &lt;span class="c"&gt;// from under it as the loop progresses.&lt;/span&gt;
    &lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The biggest gotcha is in that comment inside the &lt;code&gt;for&lt;/code&gt; loop.&lt;/p&gt;

&lt;h2&gt;
  
  
  Waiting for the work to be done
&lt;/h2&gt;

&lt;p&gt;Now that we've got goroutines running with the right parameters, let's add a &lt;a href="https://pkg.go.dev/sync?utm_source=godoc#WaitGroup"&gt;&lt;code&gt;sync.WaitGroup&lt;/code&gt;&lt;/a&gt; to the mix. This mechanism will let us wait for all these goroutines to finish.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="n"&gt;wg&lt;/span&gt; &lt;span class="n"&gt;sync&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WaitGroup&lt;/span&gt;
&lt;span class="n"&gt;workToDo&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s"&gt;"do"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"some"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"work"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="k"&gt;range&lt;/span&gt; &lt;span class="n"&gt;workToDo&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c"&gt;// add 1 to the waitgroup _before_ you start the goroutine.&lt;/span&gt;
    &lt;span class="c"&gt;// you want to do this in the same goroutine as where&lt;/span&gt;
    &lt;span class="c"&gt;// you call wg.Wait() so that you're sure that, even if&lt;/span&gt;
    &lt;span class="c"&gt;// none of the goroutines started yet, you have the&lt;/span&gt;
    &lt;span class="c"&gt;// right number of pending work.&lt;/span&gt;
    &lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c"&gt;// make sure you pass the index and work into the &lt;/span&gt;
    &lt;span class="c"&gt;// function that runs in the goroutine.&lt;/span&gt;
    &lt;span class="c"&gt;// this mechanism makes sure that the goroutine&lt;/span&gt;
    &lt;span class="c"&gt;// gets a (stack-allocated) _copy_ of the data.&lt;/span&gt;
    &lt;span class="c"&gt;// if you don't do this, idx and work will change out&lt;/span&gt;
    &lt;span class="c"&gt;// from under it as the loop progresses.&lt;/span&gt;
    &lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c"&gt;// wg.Done() tells the WaitGroup that we're done in&lt;/span&gt;
        &lt;span class="c"&gt;// this goroutine. In other words, it decrements&lt;/span&gt;
        &lt;span class="c"&gt;// the internal WaitGroup counter, whereas wg.Add(1)&lt;/span&gt;
        &lt;span class="c"&gt;// above increments it.&lt;/span&gt;
        &lt;span class="c"&gt;// Most commonly, we just do this in a defer statement.&lt;/span&gt;
        &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Done&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="c"&gt;// this is the "work". in the next section, we'll be&lt;/span&gt;
        &lt;span class="c"&gt;// changing this to return a value, because we'll&lt;/span&gt;
        &lt;span class="c"&gt;// need to send that value somewhere&lt;/span&gt;
        &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="c"&gt;// wait for all the goroutines to finish. this call&lt;/span&gt;
&lt;span class="c"&gt;// blocks until the WaitGroup's internal count goes &lt;/span&gt;
&lt;span class="c"&gt;// to zero&lt;/span&gt;
&lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Wait&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Getting the results
&lt;/h2&gt;

&lt;p&gt;So, now we know when all the work is done, we need to get the results. There are two kinds of results we need to get - the actual values of the work we're doing -- we'll call this the "success value" -- and the errors that it might have returned.&lt;/p&gt;

&lt;p&gt;Let's focus on the success values first. We're going to use one group of channels, one "final" channel, and a clever way of shuttling data between the former and the latter:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Read this code on the &lt;a href="https://play.golang.org/p/CM34_zkrmrg"&gt;Go Playground&lt;/a&gt;, if you prefer&lt;/em&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="c"&gt;// this is the channel that will hold the results of the work&lt;/span&gt;
&lt;span class="n"&gt;resultCh&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="nb"&gt;make&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;chan&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="n"&gt;wg&lt;/span&gt; &lt;span class="n"&gt;sync&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WaitGroup&lt;/span&gt;
&lt;span class="n"&gt;workToDo&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s"&gt;"do"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"some"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"work"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="k"&gt;range&lt;/span&gt; &lt;span class="n"&gt;workToDo&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c"&gt;// add 1 to the waitgroup _before_ you start the goroutine.&lt;/span&gt;
    &lt;span class="c"&gt;// you want to do this in the same goroutine as where&lt;/span&gt;
    &lt;span class="c"&gt;// you call wg.Wait() so that you're sure that, even if&lt;/span&gt;
    &lt;span class="c"&gt;// none of the goroutines started yet, you have the&lt;/span&gt;
    &lt;span class="c"&gt;// right number of pending work.&lt;/span&gt;
    &lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c"&gt;// this is the loop-local channel that our first goroutine&lt;/span&gt;
    &lt;span class="c"&gt;// will send its results to. we'll start up a second&lt;/span&gt;
    &lt;span class="c"&gt;// goroutine to forward its results to the final channel.&lt;/span&gt;
    &lt;span class="n"&gt;ch&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="nb"&gt;make&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;chan&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c"&gt;// make sure you pass the index and work into the&lt;/span&gt;
    &lt;span class="c"&gt;// function that runs in the goroutine.&lt;/span&gt;
    &lt;span class="c"&gt;// this mechanism makes sure that the goroutine&lt;/span&gt;
    &lt;span class="c"&gt;// gets a (stack-allocated) _copy_ of the data.&lt;/span&gt;
    &lt;span class="c"&gt;// if you don't do this, idx and work will change out&lt;/span&gt;
    &lt;span class="c"&gt;// from under it as the loop progresses.&lt;/span&gt;
    &lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c"&gt;// this is the "work". right now, it just returns an&lt;/span&gt;
        &lt;span class="c"&gt;// int. in the next section, it will return both an int&lt;/span&gt;
        &lt;span class="c"&gt;// and an error&lt;/span&gt;
        &lt;span class="n"&gt;res&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;doSomeWork&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;ch&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt; &lt;span class="n"&gt;res&lt;/span&gt;
    &lt;span class="p"&gt;}(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c"&gt;// start up another goroutine to forward the results from&lt;/span&gt;
    &lt;span class="c"&gt;// ch to resultCh&lt;/span&gt;
    &lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c"&gt;// we want to indicate that we're done after we forward&lt;/span&gt;
        &lt;span class="c"&gt;// the result to the final channel, _not_ just when we're&lt;/span&gt;
        &lt;span class="c"&gt;// done with the actual computation. this arrangement&lt;/span&gt;
        &lt;span class="c"&gt;// will be useful below, in our final goroutine that&lt;/span&gt;
        &lt;span class="c"&gt;// runs after the for loop is done&lt;/span&gt;
        &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Done&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;res&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;ch&lt;/span&gt;
        &lt;span class="n"&gt;resultCh&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt; &lt;span class="n"&gt;res&lt;/span&gt;
    &lt;span class="p"&gt;}()&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="c"&gt;// start up a final goroutine that is going to watch for&lt;/span&gt;
&lt;span class="c"&gt;// the moment when all of the loop goroutines are both&lt;/span&gt;
&lt;span class="c"&gt;//&lt;/span&gt;
&lt;span class="c"&gt;// 1. done with their work&lt;/span&gt;
&lt;span class="c"&gt;// 2. done sending their results to the final channel&lt;/span&gt;
&lt;span class="c"&gt;//&lt;/span&gt;
&lt;span class="c"&gt;// after that, we can close the resultCh. this closure is&lt;/span&gt;
&lt;span class="c"&gt;// important for the following for loop, since ranging over&lt;/span&gt;
&lt;span class="c"&gt;// a channel will only stop after that channel is closed&lt;/span&gt;
&lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Wait&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="nb"&gt;close&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;resultCh&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}()&lt;/span&gt;

&lt;span class="c"&gt;// now that we have that final goroutine running, we can&lt;/span&gt;
&lt;span class="c"&gt;// be sure that this for loop will end after:&lt;/span&gt;
&lt;span class="c"&gt;//&lt;/span&gt;
&lt;span class="c"&gt;// 1. all goroutines are done with their work&lt;/span&gt;
&lt;span class="c"&gt;// 2. all goroutines are done sending their work to resultCh&lt;/span&gt;
&lt;span class="c"&gt;// 3. we have processed each result&lt;/span&gt;
&lt;span class="c"&gt;//  (in this case, we just print it out)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="k"&gt;range&lt;/span&gt; &lt;span class="n"&gt;resultCh&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"result:"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The code is extensively commented, but notice a few more "meta" things about it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;We're enlisting &lt;em&gt;another&lt;/em&gt; goroutine for each loop iteration, so now we're using &lt;code&gt;2N&lt;/code&gt; goroutines rather than &lt;code&gt;N&lt;/code&gt; (where &lt;code&gt;N&lt;/code&gt; is the number of work items to do).

&lt;ul&gt;
&lt;li&gt;If you're worried about the extra goroutines, remember that a Go program can run hundreds of thousands of them comfortably on a relatively modern laptop. Plan accordingly with that in mind.&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;
&lt;li&gt;We use one extra final goroutine to determine when the final goroutine should be closed&lt;/li&gt;
&lt;li&gt;We no longer use &lt;code&gt;wg.Wait()&lt;/code&gt; in the main goroutine. Instead, we range over &lt;code&gt;resultCh&lt;/code&gt; to both get the results &lt;em&gt;and&lt;/em&gt; determine when all the work items are done.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  A final wrinkle: error handling
&lt;/h2&gt;

&lt;p&gt;Now that you (hopefully) have a decent grasp over the code in the previous section, consider that, for most workloads, you'll also have to deal with error handling. It doesn't take a lot of &lt;em&gt;additional&lt;/em&gt; code to do it, but it does add a bit more complexity. Let's see how it works:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Read this code on the &lt;a href="https://play.golang.org/p/Bcv_XQwoAi6"&gt;Go Playground&lt;/a&gt;, if you prefer&lt;/em&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Note that you can reduce complexity by using the &lt;a href="https://pkg.go.dev/golang.org/x/sync/errgroup"&gt;&lt;code&gt;errgroup&lt;/code&gt;&lt;/a&gt; package. The code herein implements functionality similar to that of &lt;code&gt;errgroup&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;
&lt;/blockquote&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="c"&gt;// this is the channel that will hold the results of the work&lt;/span&gt;
&lt;span class="n"&gt;resultCh&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="nb"&gt;make&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;chan&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c"&gt;// this channel receives all the errors that occur.&lt;/span&gt;
&lt;span class="c"&gt;// for each work item, either resultCh or errCh will receive&lt;/span&gt;
&lt;span class="c"&gt;// precisely once. both channels will be closed immediately&lt;/span&gt;
&lt;span class="c"&gt;// after all receives happen&lt;/span&gt;
&lt;span class="n"&gt;errCh&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="nb"&gt;make&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;chan&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="n"&gt;wg&lt;/span&gt; &lt;span class="n"&gt;sync&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WaitGroup&lt;/span&gt;
&lt;span class="n"&gt;workToDo&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s"&gt;"do"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"some"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"work"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="k"&gt;range&lt;/span&gt; &lt;span class="n"&gt;workToDo&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c"&gt;// add 1 to the waitgroup _before_ you start the goroutine.&lt;/span&gt;
    &lt;span class="c"&gt;// you want to do this in the same goroutine as where&lt;/span&gt;
    &lt;span class="c"&gt;// you call wg.Wait() so that you're sure that, even if&lt;/span&gt;
    &lt;span class="c"&gt;// none of the goroutines started yet, you have the&lt;/span&gt;
    &lt;span class="c"&gt;// right number of pending work.&lt;/span&gt;
    &lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c"&gt;// this is the loop-local channel that our first goroutine&lt;/span&gt;
    &lt;span class="c"&gt;// will send its results to. we'll start up a second&lt;/span&gt;
    &lt;span class="c"&gt;// goroutine to forward its results to the final channel.&lt;/span&gt;
    &lt;span class="n"&gt;ch&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="nb"&gt;make&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;chan&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c"&gt;// this is the loop-local channel that our first goroutine&lt;/span&gt;
    &lt;span class="c"&gt;// will send errors on. for each loop iteration, exactly&lt;/span&gt;
    &lt;span class="c"&gt;// one of ch or errCh will receive&lt;/span&gt;
    &lt;span class="n"&gt;eCh&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="nb"&gt;make&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;chan&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c"&gt;// make sure you pass the index and work into the&lt;/span&gt;
    &lt;span class="c"&gt;// function that runs in the goroutine.&lt;/span&gt;
    &lt;span class="c"&gt;// this mechanism makes sure that the goroutine&lt;/span&gt;
    &lt;span class="c"&gt;// gets a (stack-allocated) _copy_ of the data.&lt;/span&gt;
    &lt;span class="c"&gt;// if you don't do this, idx and work will change out&lt;/span&gt;
    &lt;span class="c"&gt;// from under it as the loop progresses.&lt;/span&gt;
    &lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c"&gt;// this is the "work". right now, it just returns an&lt;/span&gt;
        &lt;span class="c"&gt;// int. in the next section, it will return both an int&lt;/span&gt;
        &lt;span class="c"&gt;// and an error&lt;/span&gt;
        &lt;span class="n"&gt;res&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;doSomeWork&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;eCh&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;ch&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt; &lt;span class="n"&gt;res&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}(&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;work&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c"&gt;// start up another goroutine to forward the results from:&lt;/span&gt;
    &lt;span class="c"&gt;//&lt;/span&gt;
    &lt;span class="c"&gt;// - ch to resultCh&lt;/span&gt;
    &lt;span class="c"&gt;// - eCh to errCh&lt;/span&gt;
    &lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c"&gt;// we want to indicate that we're done after we do the&lt;/span&gt;
        &lt;span class="c"&gt;// forward operation, similar to the code in the&lt;/span&gt;
        &lt;span class="c"&gt;// previous section&lt;/span&gt;
        &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Done&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="c"&gt;// only one forward operation will happen per loop&lt;/span&gt;
        &lt;span class="c"&gt;// iteration, so we use a select to choose exactly&lt;/span&gt;
        &lt;span class="c"&gt;// one of the channels - either the success or error&lt;/span&gt;
        &lt;span class="c"&gt;// one.&lt;/span&gt;
        &lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;case&lt;/span&gt; &lt;span class="n"&gt;res&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;ch&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;resultCh&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt; &lt;span class="n"&gt;res&lt;/span&gt;
            &lt;span class="k"&gt;case&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;eCh&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;errCh&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}()&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="c"&gt;// start up a final goroutine that is going to watch for&lt;/span&gt;
&lt;span class="c"&gt;// the moment when all of the loop goroutines are both&lt;/span&gt;
&lt;span class="c"&gt;//&lt;/span&gt;
&lt;span class="c"&gt;// 1. done with their work&lt;/span&gt;
&lt;span class="c"&gt;// 2. done sending their results to the appropriate channel&lt;/span&gt;
&lt;span class="c"&gt;//&lt;/span&gt;
&lt;span class="c"&gt;// after that, we can close both resultCh and errCh.&lt;/span&gt;
&lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;wg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Wait&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="nb"&gt;close&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;resultCh&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nb"&gt;close&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;errCh&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}()&lt;/span&gt;

&lt;span class="c"&gt;// we're now at a point where we have two "final" channels:&lt;/span&gt;
&lt;span class="c"&gt;//&lt;/span&gt;
&lt;span class="c"&gt;// - one for the successful results&lt;/span&gt;
&lt;span class="c"&gt;// - one for the errors&lt;/span&gt;
&lt;span class="c"&gt;//&lt;/span&gt;
&lt;span class="c"&gt;// we have a few choices on how to handle them, and it's&lt;/span&gt;
&lt;span class="c"&gt;// largely up to your use case how you handle errors or success&lt;/span&gt;
&lt;span class="c"&gt;// results. In our case, we'll loop through both channels,&lt;/span&gt;
&lt;span class="c"&gt;// print out the result either way, and then exit when all&lt;/span&gt;
&lt;span class="c"&gt;// receives happen.&lt;/span&gt;

&lt;span class="c"&gt;// these two booleans are going to keep track of when &lt;/span&gt;
&lt;span class="c"&gt;// each channel is closed and done receiving&lt;/span&gt;
&lt;span class="n"&gt;resultsDone&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="no"&gt;false&lt;/span&gt;
&lt;span class="n"&gt;errsDone&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="no"&gt;false&lt;/span&gt;
&lt;span class="c"&gt;// we're going to use an infinite loop and break out of it&lt;/span&gt;
&lt;span class="c"&gt;// when both channels are done receiving&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;resultsDone&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="n"&gt;errsDone&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;break&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;case&lt;/span&gt; &lt;span class="n"&gt;res&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;valid&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;resultCh&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;valid&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;resultsDone&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="no"&gt;true&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"result:"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;res&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;case&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;valid&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;errCh&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;valid&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;errsDone&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="no"&gt;true&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"error:"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A few more things to note here:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;We've handled errors and success values with equal importance. In many cases, you might want to immediately exit if you see an error. In that case, make sure that you find a way to receive the rest of the errors and success values on &lt;code&gt;errCh&lt;/code&gt; and &lt;code&gt;resultCh&lt;/code&gt; (respectively), or tell the remaining goroutines to exit.

&lt;ul&gt;
&lt;li&gt;If you intend to do the latter, I highly recommend using &lt;a href="https://pkg.go.dev/context"&gt;&lt;code&gt;context&lt;/code&gt;&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;
&lt;li&gt;There is a lot of code here! And for that reason, it's worth repeating that you can reduce complexity by using the &lt;a href="https://pkg.go.dev/golang.org/x/sync/errgroup"&gt;&lt;code&gt;errgroup&lt;/code&gt;&lt;/a&gt; package.&lt;/li&gt;
&lt;li&gt;The primary source of complexity is the parallelism (that's the reason this blog post exists!) If you're thinking of using this pattern, I encourage you to measure the serial (non-parallel) version of the algorithm first to determine whether you really need to take on this complexity&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>programming</category>
      <category>hacktoberfest</category>
      <category>go</category>
    </item>
    <item>
      <title>Next Steps for KEDA HTTP</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Wed, 09 Dec 2020 20:41:25 +0000</pubDate>
      <link>https://forem.com/arschles/next-steps-for-keda-http-5735</link>
      <guid>https://forem.com/arschles/next-steps-for-keda-http-5735</guid>
      <description>&lt;p&gt;I &lt;a href="//./2020-11-23-kedahttp.md"&gt;recently wrote&lt;/a&gt; about a new project I'm starting in the Kubernetes world. Its overarching goal is to build an open source, app developer friendly system built on top of Kubernetes. Since then, there has been a lot of interest and a few developments that I'm excited about.&lt;/p&gt;

&lt;h2&gt;
  
  
  Technical Reference and Proposal Document
&lt;/h2&gt;

&lt;p&gt;Along with the &lt;a href="https://github.com/kedacore/keda/issues/538"&gt;GitHub issue&lt;/a&gt; that started the project, there is now a &lt;a href="https://hackmd.io/@arschles/kedahttp"&gt;proposal and technical overview&lt;/a&gt; of the system. Like most proposals, this document will serve as the proposal submitted to the KEDA maintainers and also as a reference document that the project will build against.&lt;/p&gt;

&lt;p&gt;Included in this document is an overview of all the components in the system, an architectural diagram of how they're assembled, and a discussion of the extensibility of the system [1].&lt;/p&gt;

&lt;h2&gt;
  
  
  Acceptance to the KEDA Organization
&lt;/h2&gt;

&lt;p&gt;The proposal has been accepted and a &lt;a href="https://github.com/kedacore/http-add-on"&gt;new repository&lt;/a&gt; under the &lt;a href="https://github.com/kedacore"&gt;KEDA organization&lt;/a&gt; to house these components.&lt;/p&gt;

&lt;p&gt;This development is important because the people and the idea now have the support [2] of the KEDA organization and a wider discussion has been started around the direction and featureset. We also have a wider team of developers that have joined to contribute.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's Next
&lt;/h2&gt;

&lt;p&gt;I'm pleased with the direction of this project and looking forward to the next few months. Things are moving very quickly in the new repository. I did a pairing session yesterday with coworkers in which we completed the entire serving and metrics infrastructure [3], and the next component on deck is the &lt;a href="https://hackmd.io/@arschles/kedahttp#Operator"&gt;operator&lt;/a&gt; [4].&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;If you're interested in seeing this component progress -- along with the project at-large -- I live code it &lt;a href="https://twitch.tv/arschles"&gt;on Twitch&lt;/a&gt; 2-3 times per week. I encourage you to tune in and ask questions/leave comments in the chat.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Contributing
&lt;/h2&gt;

&lt;p&gt;We're not yet ready for contributions from the community at-large. We have two major tasks to complete before we'll be open for contributions from the at-large community.&lt;/p&gt;

&lt;p&gt;First, we need to finish the minimal infrastructure. The components and supporting artifacts (Helm charts, CI scripts, etc...) are being built in &lt;a href="https://github.com/kedacore/http-add-on/pull/2"&gt;PR #2&lt;/a&gt;, and once we have them completed, we will merge it [5]. Second, we need to establish a roadmap. We're beginning to outline it now and will finish it shortly after merging.&lt;/p&gt;

&lt;p&gt;We believe that when these two tasks are complete, the project will have an established foundation and be well situated to accept contributions and support a larger community.&lt;/p&gt;

&lt;p&gt;I along with others will broadcast when we're ready to welcome the wider community.&lt;/p&gt;




&lt;p&gt;[1] In addition to the discussion in there, I've had more concrete discussions elsewhere about extensibility. Most of them are around alternative levels of abstraction that this system could support. The given architecture is highly extensible, and after this particular abstraction is built, we'll explore others.&lt;/p&gt;

&lt;p&gt;[2] The word "support" has different implications in different open source projects and communities. In this case, I mean that there is a repository created and the general idea to support HTTP based autoscaling is on the roadmap of the larger KEDA project.&lt;/p&gt;

&lt;p&gt;[3] We based much of our code on the &lt;a href="https://github.com/osscda/kedahttp"&gt;prototype codebase&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;[4] This component is notable because it's being implemented in &lt;a href="https://rust-lang.org"&gt;Rust&lt;/a&gt;. The decision to do so was inspired by my former colleagues in the &lt;a href="https://deislabs.io"&gt;Deis Labs&lt;/a&gt; group at Microsoft. They are innovating in significant ways in the Cloud Native community.&lt;/p&gt;

&lt;p&gt;[5] Depending on how we progress, we may create and merge multiple PRs.&lt;/p&gt;

</description>
      <category>kubernetes</category>
      <category>cloudnative</category>
      <category>rust</category>
      <category>go</category>
    </item>
    <item>
      <title>Deploying Apps With KEDA-HTTP</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Tue, 24 Nov 2020 01:22:57 +0000</pubDate>
      <link>https://forem.com/arschles/deploying-apps-with-keda-http-1gf1</link>
      <guid>https://forem.com/arschles/deploying-apps-with-keda-http-1gf1</guid>
      <description>&lt;p&gt;When you search for &lt;a href="https://duckduckgo.com/?q=what+is+kubernetes&amp;amp;t=brave&amp;amp;ia=web"&gt;"what is Kubernetes?"&lt;/a&gt;, content abounds with technical details of the system. There's less about what Kubernetes is at a high level. Here's my attempt at that in two sentences:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Kubernetes is an operating system for your cluster of computers. Like any OS, you get a syscall API for CPU, memory, networking, storage and more, and build abstractions from there.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;If you're new to Kubernetes, let those two sentences sink in before you move on. And if you're an app developer trying to deploy to Kubernetes, remember the things you're &lt;em&gt;not&lt;/em&gt; implementing: an HTTP server/router, a database driver, a JSON parser, and so on. You're using abstractions. Kubernetes should be no different.&lt;/p&gt;

&lt;h2&gt;
  
  
  App Developers on the Web
&lt;/h2&gt;

&lt;p&gt;I'm using the term "app developer" in a very imprecise way; that's on purpose. I'm not out to define what this group developers does or is, I'm out to assert the tasks they &lt;em&gt;shouldn't have to do&lt;/em&gt; while building their app. Developers should not have to understand details of virtual machines, the Kubelet, &lt;code&gt;Pod&lt;/code&gt;s/&lt;code&gt;Deployment&lt;/code&gt;s/&lt;code&gt;Service&lt;/code&gt;s/etc..., or &lt;code&gt;kubectl&lt;/code&gt; &lt;em&gt;[0]&lt;/em&gt;. They also shouldn't need to deal with &lt;a href="https://helm.sh"&gt;Helm&lt;/a&gt; charts or watch Kubernetes dashboards after a deploy &lt;em&gt;[1]&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;In other words, they shouldn't have to know a distributed syscall API to accomplish everyday deployment and management tasks. Kubernetes is a robust platform on which to run web applications, not an adequate experience for web developers to deploy them.&lt;/p&gt;

&lt;h2&gt;
  
  
  A Better Experience
&lt;/h2&gt;

&lt;p&gt;In 2015, Rob Pike gave a talk at dotGo entitled &lt;a href="https://www.youtube.com/watch?v=rFejpH_tAHM"&gt;"Simplicity is Complicated"&lt;/a&gt;. Referring to the Go language, he explained that while Go is simple for developers, the code that provides that simplicity is very complex. Indeed, the Go standard library provides a very &lt;em&gt;simple&lt;/em&gt; and &lt;em&gt;flexible&lt;/em&gt; API to access underlying system resources.&lt;/p&gt;

&lt;p&gt;Kubernetes is the underlying system. The task we have now is (a) defining what a simple and flexible interface needs to look like, and (b) implementing it.&lt;/p&gt;

&lt;p&gt;Many cloud providers have relevant products we can draw inspiration from &lt;em&gt;[2]&lt;/em&gt;. While these platforms are diverse, most or all of them share roughly the following features.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You &lt;em&gt;focus&lt;/em&gt; on running your code, not underlying resources like VMs, containers, VNets and so forth&lt;/li&gt;
&lt;li&gt;Your app can automatically scale up and down based on demand&lt;/li&gt;
&lt;li&gt;A CLI is available to deploy your code to production with a command&lt;/li&gt;
&lt;li&gt;Your app can built in the cloud, instead of your computer&lt;/li&gt;
&lt;li&gt;It's easy to connect your code to other databases and other cloud services&lt;/li&gt;
&lt;li&gt;You can opt in to advanced features like gradual rollouts and more&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Kubernetes has the foundational technologies to build these features and more. Our challenge is to choose the right ones, manage the complexity, and expose a simple and flexible abstraction purpose-built for Kubernetes.&lt;/p&gt;

&lt;h2&gt;
  
  
  KEDA-HTTP
&lt;/h2&gt;

&lt;p&gt;That brings us to &lt;a href="https://github.com/osscda/kedahttp"&gt;KEDA-HTTP&lt;/a&gt;, a new project I'm starting. I'm building KEDA-HTTP to bring Kubernetes closer to app developers plain and simple. Knowing approximately what a good platform looks like, the technical details matter.&lt;/p&gt;

&lt;h3&gt;
  
  
  Under the Hood
&lt;/h3&gt;

&lt;p&gt;This project primarily assembles existing building blocks to build the right developer experience &lt;em&gt;[3]&lt;/em&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Standard Kubernetes &lt;code&gt;Deployment&lt;/code&gt;s, &lt;code&gt;Service&lt;/code&gt;s and &lt;code&gt;Ingress&lt;/code&gt;es and Ingress Controllers for running pods ("compute") and routing HTTP traffic to them&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://keda.sh"&gt;KEDA&lt;/a&gt; for scaling &lt;code&gt;Deployment&lt;/code&gt;s up and down&lt;/li&gt;
&lt;li&gt;The &lt;a href="https://kubernetes.io/docs/concepts/extend-kubernetes/operator/"&gt;operator pattern&lt;/a&gt; and &lt;a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/"&gt;Custom Resource Definitions (CRDs)&lt;/a&gt; for defining a comprehensive abstraction and API over KEDA and the standard Kubernetes resources&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  The Future
&lt;/h3&gt;

&lt;p&gt;Kubernetes has democratized distributed systems. Any developer can access lots of distributed primitives over a standard API, but doing so requires understanding large concepts and keeping lots of context in one's head. That requirement alone is a barrier to entry for a lot of people. Further, a large subset of applications don't need some of these primitives &lt;em&gt;[4]&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;I believe that we know what a simple and flexible interface needs to look like. As I see this system evolve, I'm optimistic that this next level of abstraction will greatly increase the number of developers who can use the technology. Work is ongoing at &lt;a href="https://github.com/osscda/kedahttp"&gt;github.com/osscda/kedahttp&lt;/a&gt; and there is a &lt;a href="https://github.com/kedacore/keda/issues/538"&gt;proposal&lt;/a&gt; open for adding this technology to the KEDA project.&lt;/p&gt;

&lt;p&gt;A small team of my Cloud Advocate peers at Microsoft, the great people in the KEDA project and me are working on this project. We're not quite ready for help, but watch the &lt;a href="https://github.com/kedacore/keda/issues/538"&gt;proposal issue&lt;/a&gt; for progress. We'll be slowly modularizing the codebase and contributing functionality piece by piece into KEDA.&lt;/p&gt;




&lt;p&gt;[0] As is the case with every abstraction, developers should somewhat understand inner workings.&lt;/p&gt;

&lt;p&gt;[1] In many cases, developers also function in SRE or DevOps roles as well. In those situations, they almost certainly would need to use these dashboards to assess the health of their code in production. There is a clear separation of roles or "personas" implied here - the developer writes application code and deploys it to production, while the SRE/DevOps engineer operates it in production.&lt;/p&gt;

&lt;p&gt;[2]  In no order, with no preference and non-exhaustively, these include &lt;a href="https://www.digitalocean.com/products/app-platform/"&gt;App Platform&lt;/a&gt;, &lt;a href="https://cloud.google.com/appengine/"&gt;App Engine&lt;/a&gt;, &lt;a href="https://aws.amazon.com/elasticbeanstalk/"&gt;Elastic Beanstalk&lt;/a&gt;, &lt;a href="https://heroku.com"&gt;Heroku&lt;/a&gt; and &lt;a href="https://azure.microsoft.com/en-us/services/app-service/"&gt;App Service&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;[3] Personally, I'm excited about the fact that the project is partially being implemented in &lt;a href="https://rust-lang.org"&gt;Rust&lt;/a&gt;. While the language has a reputation for being hard to learn, I believe that the language provides several features that make it well suited for writing Kubernetes components&lt;/p&gt;

&lt;p&gt;[4] Such as leader election and distributed leases&lt;/p&gt;

</description>
      <category>kubernetes</category>
      <category>webdev</category>
      <category>cloud</category>
      <category>cloudnative</category>
    </item>
    <item>
      <title>First Lines of Rust</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Thu, 19 Nov 2020 19:41:14 +0000</pubDate>
      <link>https://forem.com/arschles/first-lines-of-rust-ofe</link>
      <guid>https://forem.com/arschles/first-lines-of-rust-ofe</guid>
      <description>&lt;p&gt;After ~8 years of Go as the end-all-be-all of programming languages in my life, I started writing Rust code for &lt;a href="https://github.com/osscda/kedahttp"&gt;a real project&lt;/a&gt;, not just as a hobby.&lt;/p&gt;

&lt;p&gt;The project, briefly put, is a Kubernetes system to let people easily and quickly deploy HTTP-based containers to production with a CLI. There is more to the story than that, but I don't want to stray from what I'm excited about. I'm &lt;em&gt;really&lt;/em&gt; learning a new language after all this time, from the ground up [1].&lt;/p&gt;

&lt;h2&gt;
  
  
  Unlearning Go is Hard
&lt;/h2&gt;

&lt;p&gt;Lots of people come to the Go community and start writing Goava or Gython (Java-like Go or Python-like Go). I usually spend time with many of them helping un-teach that language so that they can absorb Go and build up the cognitive "muscle memory" that they need to be productive in the language.&lt;/p&gt;

&lt;p&gt;Now, the tables have turned and I'm experiencing the same thing with Rust. One of the important first steps I've taken is accepting that I will still "know" Go even if I "unlearn" it so I really get productive with Rust. I'll always be a Gopher.&lt;/p&gt;

&lt;h2&gt;
  
  
  Similarities to Go
&lt;/h2&gt;

&lt;p&gt;As I progress, I'm writing down some cognitive and structural similarities I see in the two languages. That's helping me make the transition from the Go way to the Rust way. Both are modern, systems-oriented programming languages so they view the world similarly and as such, they've landed on a few similar and important features.&lt;/p&gt;

&lt;h3&gt;
  
  
  Memory Safe
&lt;/h3&gt;

&lt;p&gt;Go uses garbage collection and Rust uses the ownership concept, the borrow checker and (opt-in) reference counting, but both languages give you confidence &lt;em&gt;at compile time&lt;/em&gt; that you won't access invalid memory. They also work hard to help you ensure that memory will be cleaned up as expected.&lt;/p&gt;

&lt;h3&gt;
  
  
  Batteries Included
&lt;/h3&gt;

&lt;p&gt;I was going to entitle this section "Built in Tests", but more is built in than just tests. Both languages ship with almost all of the tools needed to build modern software. Go has the &lt;code&gt;go&lt;/code&gt; CLI and Rust has &lt;code&gt;cargo&lt;/code&gt;. In both cases, you run an installer to get a built in test framework, extensive standard library, dependency management (in Go 1.9 and up!), and more.&lt;/p&gt;

&lt;p&gt;I can't understate the effect these out-of-the-box tools have on the community. Anyone can come to the language, run an installer, and be ready to write production level code. Other language ecosystems make you learn -- and sometimes even choose from -- a wide variety of tools before you write your first line of code. That's a big barrier to entry for someone new to the language and can turn people off.&lt;/p&gt;

&lt;p&gt;Both ecosystems do have other tools you can learn, but you can be productive with what comes out of the box.&lt;/p&gt;

&lt;h3&gt;
  
  
  Errors are Values
&lt;/h3&gt;

&lt;p&gt;One of Go's hallmarks is that "errors are values". In practical terms, that means that there is no exception system and the language makes you check any error that a function might return.&lt;/p&gt;

&lt;p&gt;Rust also lacks exceptions, but has a more expressive type system [2]. It usually represents errors using the &lt;code&gt;Result&amp;lt;T, E&amp;gt;&lt;/code&gt; type [3]. Many people use &lt;code&gt;match&lt;/code&gt; statements to "pull" values out of the &lt;code&gt;Result&lt;/code&gt; (you can do it in other ways too), and Rust has an exhaustiveness checker that effectively forces you to check every error.&lt;/p&gt;

&lt;h3&gt;
  
  
  Panic Only in Unrecoverable Situations
&lt;/h3&gt;

&lt;p&gt;Go has a &lt;code&gt;panic()&lt;/code&gt; function and Rust has a &lt;code&gt;panic!()&lt;/code&gt; macro. They look almost the same and they have approximately the same purpose; if there is no way to recover and your program can't continue under any circumstances, &lt;code&gt;panic&lt;/code&gt;-ing will exit immediately with a stack trace (optionally in Rust).&lt;/p&gt;

&lt;h3&gt;
  
  
  Polymorphism
&lt;/h3&gt;

&lt;p&gt;Polymorphism is an umbrella term to encompass many features in a programming language and Rust and Go provide a similar subset of them.&lt;/p&gt;

&lt;p&gt;In Go, you can create an &lt;code&gt;interface{}&lt;/code&gt; that specifies methods that a concrete type can implement. Any concrete type  that implements them automatically adheres to that interface, without specifying so. You can use any concrete type where the interface is expected.&lt;/p&gt;

&lt;p&gt;Rust has &lt;code&gt;trait&lt;/code&gt;s, which are very similar. You can define methods in them, and a default implementation for those methods as well (similar to an &lt;code&gt;abstract class&lt;/code&gt; in C++).&lt;/p&gt;

&lt;p&gt;One big difference in this area is that Rust requires that you explicitly state that a concrete type is implementing a trait. That small difference has a big impact in large codebases because with Go, you don't have to add a dependency to your app just to implement one of its &lt;code&gt;interface&lt;/code&gt;s [4].&lt;/p&gt;

&lt;h2&gt;
  
  
  More?
&lt;/h2&gt;

&lt;p&gt;I'm a month or so into building this app and becoming more productive by the day. I've been writing Go for a long time and I'll naturally draw more parallels to Rust as I progress. So far, I'm enjoying the process and looking forward to the future 🚀.&lt;/p&gt;




&lt;p&gt;[1] I've learned a few languages ad-hoc over the past years, but not from the ground up, and that's why this is exciting for me&lt;br&gt;
[2] Go has plans to add &lt;a href="https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-contracts.md"&gt;generics&lt;/a&gt; to the language soon, but the proposed design is different from Rust's generics design and it lacks features that Rust has.&lt;br&gt;
[3] Since &lt;code&gt;Result&lt;/code&gt; is a regular enum in the standard library, it's only the de-facto standard for error checking; you don't have to use it as such&lt;br&gt;
[4] Go is sometimes credited with allowing "duck typing" for this reason, but the feature is closer to structural typing&lt;/p&gt;

</description>
      <category>rust</category>
      <category>programming</category>
      <category>systems</category>
    </item>
    <item>
      <title>async/await: under the hood</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Tue, 30 Jun 2020 23:15:26 +0000</pubDate>
      <link>https://forem.com/arschles/async-await-under-the-hood-1j0n</link>
      <guid>https://forem.com/arschles/async-await-under-the-hood-1j0n</guid>
      <description>&lt;p&gt;I'm really interested in concurrency strategies in programming languages, and because there's a lot of written research out there on the topic, you can find lots of strategies out there. &lt;/p&gt;

&lt;p&gt;When you look at some of the more modern stuff, you'll find a &lt;a href="https://www.microsoft.com/en-us/research/wp-content/uploads/2007/01/The-Joins-Concurrency-Library.pdf"&gt;lot&lt;/a&gt; &lt;a href="https://concurnas.com/"&gt;of&lt;/a&gt; &lt;a href="https://rust-lang.github.io/async-book/03_async_await/01_chapter.html"&gt;literature&lt;/a&gt; on just about the same pattern: &lt;code&gt;async&lt;/code&gt;/&lt;code&gt;await&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;async&lt;/code&gt;/&lt;code&gt;await&lt;/code&gt; is picking up steam in languages because it makes concurrency &lt;em&gt;really&lt;/em&gt; easy to see and deal with. Let's look at how it works and why it helps, using Javascript to illustrate the concepts.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;I'm a Javascript dabbler at best, but it's a great language to illustrate these concepts with. Don't go too hard on my JS code below 😅&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h1&gt;
  
  
  What It's About 🤔
&lt;/h1&gt;

&lt;p&gt;&lt;code&gt;async&lt;/code&gt;/&lt;code&gt;await&lt;/code&gt; is about writing concurrent code easily, but more importantly, it's about writing the code so it's easy to &lt;strong&gt;read&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Solving Concurrency Three Ways 🕒
&lt;/h2&gt;

&lt;p&gt;This pattern relies on a feature called Promises in Javascript, so we're gonna build up from basics to Promises in JS, and cap it off with integrating &lt;code&gt;async&lt;/code&gt;/&lt;code&gt;await&lt;/code&gt; into Promises.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Promises are called Futures in many other languages/frameworks. Some use both terms! It can be confusing, but the concept is the same. We'll go into details later in this post.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  Callbacks 😭
&lt;/h3&gt;

&lt;p&gt;You've probably heard about callbacks in Javascript. If you haven't, they're a programming pattern that lets you schedule work to be done in the future, after something else finishes. Callbacks are also the foundation of what we're talking about here.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;The core problem we're solving in this entire article is how to run code after some concurrent work is being done.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The syntax of callbacks is basically passing a function into a function:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nx"&gt;doStuff&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;callback&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// do something&lt;/span&gt;
    &lt;span class="c1"&gt;// now it's done, call the callback&lt;/span&gt;
    &lt;span class="nx"&gt;callback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;someStuff&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nx"&gt;doStuff&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;result&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// when doStuff is done doing its thing, it'll pass its result&lt;/span&gt;
    &lt;span class="c1"&gt;// to this function.&lt;/span&gt;
    &lt;span class="c1"&gt;//&lt;/span&gt;
    &lt;span class="c1"&gt;// we don't know when that'll be, just that this function will run.&lt;/span&gt;
    &lt;span class="c1"&gt;//&lt;/span&gt;
    &lt;span class="c1"&gt;// That means that the rest of our ENTIRE PROGRAM needs to go in here&lt;/span&gt;
    &lt;span class="c1"&gt;// (most of the time)&lt;/span&gt;
    &lt;span class="c1"&gt;//&lt;/span&gt;
    &lt;span class="c1"&gt;// Barf, amirite?&lt;/span&gt;
    &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;done with doStuff&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="c1"&gt;// Wait, though... if you put something here ... it'll run right away. It won't wait for doStuff to finish&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That last comment in the code is the confusing part. In practice, most apps don't want to continue execution. They want to wait. Callbacks make that difficult to achieve, confusing, and &lt;a href="http://callbackhell.com/"&gt;exhausting to write&lt;/a&gt; and read 😞.&lt;/p&gt;

&lt;h3&gt;
  
  
  Promises 🙌
&lt;/h3&gt;

&lt;p&gt;I'll see your callbacks and raise you a &lt;code&gt;Promise&lt;/code&gt;! No really, Promises are dressed up callbacks that make things easier to deal with. But you still pass functions to functions and it's still a bit harder than it has to be.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nx"&gt;returnAPromiseYall&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// do some stuff!&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;somePromise&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// let's call it and get our promise&lt;/span&gt;
&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;myProm&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;returnAPromiseYall&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

&lt;span class="c1"&gt;// now we have to do some stuff after the promise is ready&lt;/span&gt;
&lt;span class="nx"&gt;myProm&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;then&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;result&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// the result is the variable in the promise that we're waiting for,&lt;/span&gt;
    &lt;span class="c1"&gt;// just like in callback world&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;anotherPromise&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}).&lt;/span&gt;&lt;span class="nx"&gt;then&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;newResult&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// We can chain these "then" calls together to build a pipeline of&lt;/span&gt;
    &lt;span class="c1"&gt;// code. So it's a little easier to read, but still. &lt;/span&gt;
    &lt;span class="c1"&gt;// Passing functions to functions and remembering to write your code inside&lt;/span&gt;
    &lt;span class="c1"&gt;// these "then" calls is sorta tiring&lt;/span&gt;
    &lt;span class="nx"&gt;doMoreStuff&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;newResult&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We got a few small wins:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;No more intimidating &lt;em&gt;nested&lt;/em&gt; callbacks&lt;/li&gt;
&lt;li&gt;This &lt;code&gt;then&lt;/code&gt; function implies a &lt;em&gt;pipeline&lt;/em&gt; of code. Syntactically and conceptually, that's easier to deal with&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;But we still have a few sticky problems:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You have to remember to put the rest of your program into a &lt;code&gt;then&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;You're still passing functions to functions. It still gets tiring to read and write that&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  async/await 🥇
&lt;/h3&gt;

&lt;p&gt;Alrighty, we're here folks! The &lt;code&gt;Promise&lt;/code&gt;d land 🎉🥳🍤. We can get rid of passing functions to functions, &lt;code&gt;then&lt;/code&gt;, and all that forgetting to put the rest of your program into the &lt;code&gt;then&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;All with this 🔥 pattern. Check it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nx"&gt;doStuff&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// just like the last two examples, return a promise&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;myPromise&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// now, behold! we can call it with await&lt;/span&gt;
&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;theResult&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;doStuff&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

&lt;span class="c1"&gt;// IN A WORLD, WHERE THERE ARE NO PROMISES ...&lt;/span&gt;
&lt;span class="c1"&gt;// ONLY GUARANTEES&lt;/span&gt;
&lt;span class="c1"&gt;//&lt;/span&gt;
&lt;span class="c1"&gt;// In other words, the value is ready right here!&lt;/span&gt;
&lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`the result is ready: &lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;theResult&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Thanks to the &lt;code&gt;await&lt;/code&gt; keyword, we can read the code from top to bottom. This gets translated to something or other under the hood, and what exactly it is depends on the language. In JS land, it's essentially &lt;code&gt;Promise&lt;/code&gt;s most of the time. The results to us &lt;em&gt;programmers&lt;/em&gt; is always the same, though:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Programmers can read/write code from top to bottom, the way we're used to doing it&lt;/li&gt;
&lt;li&gt;No passing functions into functions means less &lt;code&gt;})&lt;/code&gt; syntax to &lt;del&gt;forget&lt;/del&gt; write&lt;/li&gt;
&lt;li&gt;The &lt;code&gt;await&lt;/code&gt; keyword can be an indicator that &lt;code&gt;doStuff&lt;/code&gt; does something "expensive" (like call a REST API)&lt;/li&gt;
&lt;/ul&gt;

&lt;h4&gt;
  
  
  What about the &lt;code&gt;async&lt;/code&gt; keyword⁉
&lt;/h4&gt;

&lt;p&gt;In many languages including JS, you have to mark a function &lt;code&gt;async&lt;/code&gt; if it uses &lt;code&gt;await&lt;/code&gt; inside of it. There are language-specific reasons to do that, but here are some that you should care about:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;To tell the caller that there are &lt;code&gt;Promise&lt;/code&gt;s or &lt;code&gt;await&lt;/code&gt;s happening inside of it&lt;/li&gt;
&lt;li&gt;To tell the runtime (or compiler in other languages) to do its magic behind the scenes to "make it work"™&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  🏁
&lt;/h2&gt;

&lt;p&gt;And that's it. I left a lot of implementation details out, but it's really important to remember that this pattern exists more for human reasons rather than technical.&lt;/p&gt;

&lt;p&gt;You can do all of this stuff with callbacks, but in almost all cases, &lt;code&gt;async&lt;/code&gt;/&lt;code&gt;await&lt;/code&gt; is going to make your life easier. Enjoy! 👋&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>javascript</category>
      <category>serverless</category>
    </item>
    <item>
      <title>How To Make Windows Keyboards Look Like Mac</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Wed, 17 Jun 2020 22:15:46 +0000</pubDate>
      <link>https://forem.com/arschles/how-to-make-windows-keyboards-look-like-mac-329i</link>
      <guid>https://forem.com/arschles/how-to-make-windows-keyboards-look-like-mac-329i</guid>
      <description>&lt;p&gt;I wrote a bit about PowerToys &lt;a href="https://dev.to/blog/2020-05-20-powertoys"&gt;previously&lt;/a&gt;, but today we're gonna talk about my second-favorite feature: the keyboard manager.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;My first-favorite feature is Powertoys Run - the spotlight-like feature for Windows. I talked about this one last post.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h1&gt;
  
  
  Make Windows Feel Like Mac 🍏
&lt;/h1&gt;

&lt;p&gt;I &lt;a href="https://dev.to/blog/how-to-wsl-2/"&gt;switched to Windows from Mac&lt;/a&gt; a few months back, and I'm used to just about everything except the keyboard. Muscle memory keeps kicking in and my thumbs reach for the &lt;code&gt;cmd&lt;/code&gt; key instead of &lt;code&gt;ctrl&lt;/code&gt;. There's no &lt;code&gt;cmd&lt;/code&gt; on Windows - it just opens the start menu. I've opened the start menu a &lt;strong&gt;lot&lt;/strong&gt;.&lt;/p&gt;

&lt;h1&gt;
  
  
  The Keyboard Manager ⌨
&lt;/h1&gt;

&lt;p&gt;The key feature (see what I did there?) of PowerToys that lets my keyboard be all apple-ey is the keyboard manager. It looks like this:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--xfBXoi7k--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://arschles.com/images/2020-06-16-powertoys-mac/powertoys-keyboard-manager.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--xfBXoi7k--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://arschles.com/images/2020-06-16-powertoys-mac/powertoys-keyboard-manager.png" alt="powertoys keyboard manager" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;There's a lot going on in that window. Let's break it down...&lt;/p&gt;

&lt;h2&gt;
  
  
  Remap Keyboard 🗺
&lt;/h2&gt;

&lt;p&gt;This one is simple in theory. Like the heading says, you get to map a single key. Take the first one in there - I mapped Page Up to the home key (that one wasn't Mac specific - I just kept accidentally hitting it 😉).&lt;/p&gt;

&lt;p&gt;Here are the keys I remapped to Mac-ify:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;Caps Lock&lt;/code&gt; =&amp;gt; &lt;code&gt;Win (Left)&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Win (Left)&lt;/code&gt; =&amp;gt; &lt;code&gt;Ctrl (Left)&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is basically making the Windows key to the left of the spacebar into the Ctrl key, which does similar things as the Cmd key on a Mac. Since I was overriding the Windows key, I moved that up to Caps Lock, since I never use it anyway. You could also do this with the Windows key on the right, if you have one.&lt;/p&gt;

&lt;p&gt;To do these mappings, you click "Remap a Key', then the Plus icon at the bottom. Click on each box and type the keys you want to map. The "before" key goes on the left, and the "after" key on the right.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;❗ Do the mapping from &lt;code&gt;Win&lt;/code&gt; -&amp;gt; &lt;code&gt;Ctrl&lt;/code&gt; &lt;em&gt;last&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Remap Shortcuts 🍕
&lt;/h2&gt;

&lt;p&gt;This one piggybacks on key remapping. As you know, Windows has lots of key combinations that "do stuff"™. For example, alt+tab cycles through Windows, alt+tab switches Windows, win+s opens the search menu, win+d shows the desktop, win+e the explorer, and so on.&lt;/p&gt;

&lt;p&gt;Here are the shortcuts I set for mac-ifying:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;Ctrl (Left) + Tab&lt;/code&gt; =&amp;gt; &lt;code&gt;Alt (Left) + Tab&lt;/code&gt;

&lt;ul&gt;
&lt;li&gt;cmd+tab on Mac lets you cycle through Windows. This makes it feel the same. This still has a &lt;a href="https://github.com/microsoft/PowerToys/issues/3331"&gt;bug&lt;/a&gt;, but it's usable enough for now&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Ctrl (Left) + Shift (Left) + [&lt;/code&gt; =&amp;gt; &lt;code&gt;Ctrl (Left) + Shift (Left) + Tab&lt;/code&gt;

&lt;ul&gt;
&lt;li&gt;This lets me cycle through tabs quickly, just like on a Mac. This particular one cycles left.&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Win (Right) + Shift (Right) + [&lt;/code&gt; =&amp;gt; &lt;code&gt;Ctrl (Left) + Shift + Tab&lt;/code&gt;

&lt;ul&gt;
&lt;li&gt;Cycle to the left, from the right side of the keyboard&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Ctrl (Left) + Shift (Left) + ]&lt;/code&gt; =&amp;gt; &lt;code&gt;Ctrl (Left) + Tab&lt;/code&gt;

&lt;ul&gt;
&lt;li&gt;Cycle to the right, from the left side of the keyboard&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;
&lt;li&gt;- &lt;code&gt;Win (Right) + Shift (Right) + ]&lt;/code&gt; =&amp;gt; &lt;code&gt;Ctrl (Left) + Shift + Tab&lt;/code&gt;

&lt;ul&gt;
&lt;li&gt;Cycle to the right, from the right side of the keyboard&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I hope those help! It's by far not &lt;em&gt;everything&lt;/em&gt; to make Windows feel like a mac - some Windows things just don't apply to Mac and vice versa. But I think you'll find this a good setup.&lt;/p&gt;

&lt;p&gt;Enjoy!&lt;/p&gt;

</description>
      <category>tutorial</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Storage with Dapr</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Fri, 29 May 2020 18:36:48 +0000</pubDate>
      <link>https://forem.com/arschles/storage-with-dapr-36g6</link>
      <guid>https://forem.com/arschles/storage-with-dapr-36g6</guid>
      <description>&lt;p&gt;Dapr! No, not &lt;a href="https://dev.to/1kevgriff/what-is-dapper-and-why-you-should-consider-it-for-your-net-projects-cm4"&gt;dapper&lt;/a&gt;, &lt;a href="https://dapr.io" rel="noopener noreferrer"&gt;dapr&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you haven't read my past &lt;a href="https://dev.to/arschles/what-is-dapr-526a"&gt;articles&lt;/a&gt; about dapr, that's ok. Let me try to boil this big project down to a sentence:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;👋 Dapr is a process you run that does all the hard stuff for your app you'd rather not do&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;If you want to read more about it, see &lt;a href="https://cda.ms/1jF" rel="noopener noreferrer"&gt;the intro blog post&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;And with that, let's talk about (and demo!) probably the hardest thing that most (any?) apps have to deal with: storing data. &lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;🙋‍♀️ Dapr does &lt;a href="https://github.com/dapr/docs/tree/master/concepts#building-blocks" rel="noopener noreferrer"&gt;other things&lt;/a&gt; besides storage too&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h1&gt;
  
  
  Why It Matters
&lt;/h1&gt;

&lt;p&gt;Dealing with data stores is hard. You have to get the right SDK, mock out your data store for unit tests, write integration tests for round-tripping data, make sure you deal with migrations, failover, and more...&lt;/p&gt;

&lt;p&gt;Dapr helps with all that, but it also comes with some limitations too. Before we go demo the thing, some pros and cons for storage:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;➕ You read/write data with a pretty simple REST API (or gRPC!)&lt;/li&gt;
&lt;li&gt;➕ You get a lot of useful features (retries anyone?)&lt;/li&gt;
&lt;li&gt;➕ You can swap out implementations (Dapr comes with a bunch of &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#state-management-api" rel="noopener noreferrer"&gt;different implementations&lt;/a&gt; of the data API across clouds and open/closed source.)&lt;/li&gt;
&lt;li&gt;➖ You have to fit your data model into key/value. Not all apps fit into this model&lt;/li&gt;
&lt;li&gt;➕ and ➖ You have to do a bit of reading to take advantage of some useful storage features:

&lt;ul&gt;
&lt;li&gt;Strong consistency&lt;/li&gt;
&lt;li&gt;Optimistic Concurrency&lt;/li&gt;
&lt;li&gt;Retry policies&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;

&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;ℹ In Dapr land, the data storage API isn't called a database on purpose because it gives you a key/value storage API. It's not SQL or any other kind of rich query language, so keep that in mind when deciding what to use.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;If you want, you can check out more details on storage &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#state-management-api" rel="noopener noreferrer"&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;h1&gt;
  
  
  🏄‍♀️ Surf's Up, Let's Demo This Thing
&lt;/h1&gt;

&lt;p&gt;Dapr comes with a bunch of &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#state-management-api" rel="noopener noreferrer"&gt;different implementations&lt;/a&gt; of the data API across clouds and open/closed source. We'll use the Redis implementation here.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;ℹ You'll need to have &lt;a href="https://www.docker.com/products/docker-desktop" rel="noopener noreferrer"&gt;Docker&lt;/a&gt; installed and running for this demo to work&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  1️⃣ - Installing 🛠
&lt;/h2&gt;

&lt;p&gt;I followed the &lt;a href="https://github.com/dapr/docs/blob/master/getting-started/environment-setup.md#using-script-to-install-the-latest-release" rel="noopener noreferrer"&gt;installing&lt;/a&gt; instructions, using the Linux script (since I'm &lt;a href="https://dev.to/arschles/how-to-wsl-516d"&gt;using WSL2&lt;/a&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;wget &lt;span class="nt"&gt;-q&lt;/span&gt; https://raw.githubusercontent.com/dapr/cli/master/install/install.sh &lt;span class="nt"&gt;-O&lt;/span&gt; - | /bin/bash
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;I had to provide my password because the script &lt;code&gt;sudo&lt;/code&gt;s things, so if you're not into letting a random script from GitHub get superuser on your computer, you can &lt;a href="https://github.com/dapr/docs/blob/master/getting-started/environment-setup.md#from-the-binary-releases" rel="noopener noreferrer"&gt;download binaries&lt;/a&gt; instead.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;It was installed and I could test things out:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;dapr &lt;span class="nt"&gt;--help&lt;/span&gt;

         __
    ____/ /___ _____  _____
   / __  / __ &lt;span class="s1"&gt;'/ __ \/ ___/
  / /_/ / /_/ / /_/ / /
  \__,_/\__,_/ .___/_/
              /_/

======================================================
A serverless runtime for hyperscale, distributed systems

Usage:
  dapr [command]

Available Commands:
  components     List all Dapr components
  configurations List all Dapr configurations
  help           Help about any command
  init           Setup dapr in Kubernetes or Standalone modes
  invoke         Invokes a Dapr app with an optional payload (deprecated, use invokePost)
  invokeGet      Issue HTTP GET to Dapr app
  invokePost     Issue HTTP POST to Dapr app with an optional payload
  list           List all Dapr instances
  logs           Gets Dapr sidecar logs for an app in Kubernetes
  mtls           Check if mTLS is enabled in a Kubernetes cluster
  publish        Publish an event to multiple consumers
  run            Launches Dapr and (optionally) your app side by side
  status         Shows the Dapr system services (control plane) health status.
  stop           Stops a running Dapr instance and its associated app
  uninstall      Removes a Dapr installation

Flags:
  -h, --help      help for dapr
      --version   version for dapr

Use "dapr [command] --help" for more information about a command.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It even has ascii art 🤡!&lt;/p&gt;

&lt;h2&gt;
  
  
  2️⃣ Hello, World 👋
&lt;/h2&gt;

&lt;p&gt;Let's get something running. The &lt;a href="https://github.com/dapr/samples/tree/master/1.hello-world" rel="noopener noreferrer"&gt;hello world tutorial&lt;/a&gt; actually goes much farther than I expected it would.&lt;/p&gt;

&lt;p&gt;You get an app up and talking to the storage API (backed by Redis), but then you get a second app up and running and have it call an API on the first one. It's actually showing off &lt;a href="https://github.com/dapr/docs/tree/master/concepts/service-invocation" rel="noopener noreferrer"&gt;service invocation&lt;/a&gt; too 🎉&lt;/p&gt;

&lt;p&gt;I first got the hello world sample code:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;git clone https://github.com/dapr/samples.git &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;cd &lt;/span&gt;samples/1.hello-world
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The sample code has a Python app and a Node app. Both has a REST API in them.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Node App 🗼
&lt;/h3&gt;

&lt;p&gt;First, the Node app. The &lt;a href="https://github.com/dapr/samples/tree/master/1.hello-world#step-2---understand-the-code" rel="noopener noreferrer"&gt;demo README&lt;/a&gt; first goes through the Node code a bit.&lt;/p&gt;

&lt;p&gt;The really interesting bit I saw was that we're doing a &lt;code&gt;POST&lt;/code&gt; request with the &lt;code&gt;fetch&lt;/code&gt; API to the local Dapr API to store some data. What was cool was you can also just return some JSON and it'll be automatically stored.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"state"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"key"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"nomnomnom"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"value"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"pizza"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I really like that part because you don't have to manually do anything to get data into the database. It just happens.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Farschles.com%2Fimages%2Fmagic.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Farschles.com%2Fimages%2Fmagic.jpg" alt="Magic"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;On to running the thing. You do the standard thing to install the JS dependencies:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;npm &lt;span class="nb"&gt;install
&lt;/span&gt;npm WARN node_server@1.0.0 No repository field.

added 55 packages from 41 contributors and audited 55 packages &lt;span class="k"&gt;in &lt;/span&gt;0.626s
found 0 vulnerabilities



   ╭────────────────────────────────────────────────────────────────╮
   │                                                                │
   │      New patch version of npm available! 6.14.4 → 6.14.5       │
   │   Changelog: https://github.com/npm/cli/releases/tag/v6.14.5   │
   │               Run npm &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-g&lt;/span&gt; npm to update!                │
   │                                                                │
   ╰────────────────────────────────────────────────────────────────╯
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;But you then run it with the &lt;code&gt;dapr&lt;/code&gt; CLI, not &lt;code&gt;npm&lt;/code&gt; or &lt;code&gt;node&lt;/code&gt;. This spits out a lot of log lines, and that's normal. Here's approximately what it looks like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;dapr run &lt;span class="nt"&gt;--app-id&lt;/span&gt; nodeapp &lt;span class="nt"&gt;--app-port&lt;/span&gt; 3000 &lt;span class="nt"&gt;--port&lt;/span&gt; 3500 node app.js
ℹ️  Starting Dapr with &lt;span class="nb"&gt;id &lt;/span&gt;nodeapp. HTTP Port: 3500. gRPC Port: 42491
&lt;span class="o"&gt;==&lt;/span&gt; DAPR &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"2020-05-18T16:26:52.702694-07:00"&lt;/span&gt; &lt;span class="nv"&gt;level&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;info &lt;span class="nv"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"starting Dapr Runtime -- version 0.6.0 -- commit e99f712-dirty"&lt;/span&gt; &lt;span class="nv"&gt;app_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;nodeapp &lt;span class="nv"&gt;instance&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;DESKTOP-DQP07VM &lt;span class="nv"&gt;scope&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;dapr.runtime &lt;span class="nb"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;log &lt;span class="nv"&gt;ver&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0.6.0

&amp;lt;snip&amp;gt;

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Node App listening on port 3000!

&lt;span class="o"&gt;==&lt;/span&gt; DAPR &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"2020-05-18T16:26:52.7568756-07:00"&lt;/span&gt; &lt;span class="nv"&gt;level&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;info &lt;span class="nv"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"application discovered on port 3000"&lt;/span&gt; &lt;span class="nv"&gt;app_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;nodeapp &lt;span class="nv"&gt;instance&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;DESKTOP-DQP07VM &lt;span class="nv"&gt;scope&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;dapr.runtime &lt;span class="nb"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;log &lt;span class="nv"&gt;ver&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0.6.0

&amp;lt;snip&amp;gt;

ℹ️  Updating metadata &lt;span class="k"&gt;for &lt;/span&gt;app &lt;span class="nb"&gt;command&lt;/span&gt;: node app.js
✅  You&lt;span class="s1"&gt;'re up and running! Both Dapr and your app logs will appear here.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Ok, now it's running. You use the &lt;code&gt;dapr&lt;/code&gt; CLI to &lt;em&gt;also&lt;/em&gt; make calls to the web service. I like the simplicity of a single CLI.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;dapr invoke &lt;span class="nt"&gt;--app-id&lt;/span&gt; nodeapp &lt;span class="nt"&gt;--method&lt;/span&gt; neworder &lt;span class="nt"&gt;--payload&lt;/span&gt; &lt;span class="s1"&gt;'{"data": { "orderId": "41" } }'&lt;/span&gt;
✅  App invoked successfully
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;You can also use &lt;code&gt;curl&lt;/code&gt; for this if you want&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And verifying that the request actually went through, and the data was stored, we have logs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Got a new order! Order ID: 41

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Successfully persisted state.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And we can also call the Node App's API to get the new data back out of the datastore&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;curl http://localhost:3500/v1.0/invoke/nodeapp/method/order
&lt;span class="o"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"orderId"&lt;/span&gt;:&lt;span class="s2"&gt;"41"&lt;/span&gt;&lt;span class="o"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;This and the above &lt;code&gt;dapr invoke&lt;/code&gt; call are using one of the features in Dapr's service discovery building block&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  The Python App 🐍
&lt;/h3&gt;

&lt;p&gt;Now for the grande finale! The Python code calls the Node app in an infinite loop, so we should be able to see the Node app responding to requests and also see the datastore fill up at the same time.&lt;/p&gt;

&lt;p&gt;First, we set up the Python dependency:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;pip3 &lt;span class="nb"&gt;install &lt;/span&gt;requests

&amp;lt;snip&amp;gt;

Successfully installed certifi-2020.4.5.1 chardet-3.0.4 idna-2.9 requests-2.23.0 urllib3-1.25.9
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And then I ran it with the same &lt;code&gt;dapr run&lt;/code&gt; command we used for the Node app, except in a new terminal tab. I cut out all the verbose log lines this time!&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;dapr run &lt;span class="nt"&gt;--app-id&lt;/span&gt; pythonapp python3 app.py
ℹ️  Starting Dapr with &lt;span class="nb"&gt;id &lt;/span&gt;pythonapp. HTTP Port: 43891. gRPC Port: 42163
ℹ️  Updating metadata &lt;span class="k"&gt;for &lt;/span&gt;app &lt;span class="nb"&gt;command&lt;/span&gt;: python3 app.py
✅  You&lt;span class="s1"&gt;'re up and running! Both Dapr and your app logs will appear here.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now, back in the Node app logs, storage is starting to fill up!&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Got a new order! Order ID: 1

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Successfully persisted state.

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Got a new order! Order ID: 2

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Successfully persisted state.

&amp;lt;snip&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  That's It!
&lt;/h3&gt;

&lt;p&gt;From experience, getting storage and microservice-to-microservice calls working from scratch takes time, carefully crafted code, and effort.&lt;/p&gt;

&lt;p&gt;I wanted to compare this Node/Python codebase to compare my experience, and was pleasantly surprised. Let's break it down:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;87 lines of Node code, including comments

&lt;ul&gt;
&lt;li&gt;We could save a lot of lines if we returned the &lt;code&gt;"data"&lt;/code&gt; JSON dictionary instead of using &lt;code&gt;fetch&lt;/code&gt; calls&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;

&lt;li&gt;23 lines of Node code, including comments&lt;/li&gt;

&lt;/ul&gt;

&lt;p&gt;From experience with Go, I can say that a battle tested storage layer for this kind of thing would be about 50 lines. That's an apples to oranges comparison, but I know for sure that all the battle-tested bits are now behind that slick storage API the Node app is using. And funny enough, that part is written in Go 💪.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;P.S. don't forget to clean up: &lt;code&gt;dapr stop --app-id nodeapp &amp;amp;&amp;amp; dapr stop --app-id pythonapp&lt;/code&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Bonus: Behind the Scenes
&lt;/h2&gt;

&lt;p&gt;I mentioned above that you need to have Docker installed in order to get this working. That's all because Dapr needs an image running in the background for this demo to work.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cb722412a5d3        redis               &lt;span class="s2"&gt;"docker-entrypoint.s…"&lt;/span&gt;   4 weeks ago         Up 6 hours          0.0.0.0:6379-&amp;gt;6379/tcp     dapr_redis
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important one is ... Redis! By default, all the storage API calls the Node app is making get routed into Redis.&lt;/p&gt;

&lt;h2&gt;
  
  
  Wrap Up
&lt;/h2&gt;

&lt;p&gt;So we've seen the storage API &lt;em&gt;and&lt;/em&gt; a bit of the service invocation API. Honestly, those two pieces alone can get you a long way just on basic usage alone.&lt;/p&gt;

&lt;p&gt;As you get more advanced, you'll probably want to learn more about and use things like &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#concurrency" rel="noopener noreferrer"&gt;concurrency management&lt;/a&gt; and &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#retry-policies" rel="noopener noreferrer"&gt;retry policies&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Any way you go, the &lt;a href="https://github.com/dapr/docs" rel="noopener noreferrer"&gt;docs repository&lt;/a&gt; and particularly the &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management" rel="noopener noreferrer"&gt;state management section&lt;/a&gt; are great references.&lt;/p&gt;

&lt;p&gt;That's all for today. I hope you go forth and enjoy dapr-ing!&lt;/p&gt;

&lt;p&gt;🤘🚀&lt;/p&gt;

</description>
      <category>devops</category>
      <category>tutorial</category>
      <category>opensource</category>
      <category>database</category>
    </item>
    <item>
      <title>Storing Data with Dapr</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Fri, 29 May 2020 18:33:38 +0000</pubDate>
      <link>https://forem.com/azure/storing-data-with-dapr-cd8</link>
      <guid>https://forem.com/azure/storing-data-with-dapr-cd8</guid>
      <description>&lt;p&gt;Dapr! No, not &lt;a href="https://dev.to/1kevgriff/what-is-dapper-and-why-you-should-consider-it-for-your-net-projects-cm4"&gt;dapper&lt;/a&gt;, &lt;a href="https://dapr.io" rel="noopener noreferrer"&gt;dapr&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you haven't read my past &lt;a href="https://dev.to/arschles/what-is-dapr-526a"&gt;articles&lt;/a&gt; about dapr, that's ok. Let me try to boil this big project down to a sentence:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;👋 Dapr is a process you run that does all the hard stuff for your app you'd rather not do&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;If you want to read more about it, see &lt;a href="https://cda.ms/1jF" rel="noopener noreferrer"&gt;the intro blog post&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;And with that, let's talk about (and demo!) probably the hardest thing that most (any?) apps have to deal with: storing data. &lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;🙋‍♀️ Dapr does &lt;a href="https://github.com/dapr/docs/tree/master/concepts#building-blocks" rel="noopener noreferrer"&gt;other things&lt;/a&gt; besides storage too&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h1&gt;
  
  
  Why It Matters
&lt;/h1&gt;

&lt;p&gt;Dealing with data stores is hard. You have to get the right SDK, mock out your data store for unit tests, write integration tests for round-tripping data, make sure you deal with migrations, failover, and more...&lt;/p&gt;

&lt;p&gt;Dapr helps with all that, but it also comes with some limitations too. Before we go demo the thing, some pros and cons for storage:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;➕ You read/write data with a pretty simple REST API (or gRPC!)&lt;/li&gt;
&lt;li&gt;➕ You get a lot of useful features (retries anyone?)&lt;/li&gt;
&lt;li&gt;➕ You can swap out implementations (Dapr comes with a bunch of &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#state-management-api" rel="noopener noreferrer"&gt;different implementations&lt;/a&gt; of the data API across clouds and open/closed source.)&lt;/li&gt;
&lt;li&gt;➖ You have to fit your data model into key/value. Not all apps fit into this model&lt;/li&gt;
&lt;li&gt;➕ and ➖ You have to do a bit of reading to take advantage of some useful storage features:

&lt;ul&gt;
&lt;li&gt;Strong consistency&lt;/li&gt;
&lt;li&gt;Optimistic Concurrency&lt;/li&gt;
&lt;li&gt;Retry policies&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;

&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;ℹ In Dapr land, the data storage API isn't called a database on purpose because it gives you a key/value storage API. It's not SQL or any other kind of rich query language, so keep that in mind when deciding what to use.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;If you want, you can check out more details on storage &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#state-management-api" rel="noopener noreferrer"&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;h1&gt;
  
  
  🏄‍♀️ Surf's Up, Let's Demo This Thing
&lt;/h1&gt;

&lt;p&gt;Dapr comes with a bunch of &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#state-management-api" rel="noopener noreferrer"&gt;different implementations&lt;/a&gt; of the data API across clouds and open/closed source. We'll use the Redis implementation here.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;ℹ You'll need to have &lt;a href="https://www.docker.com/products/docker-desktop" rel="noopener noreferrer"&gt;Docker&lt;/a&gt; installed and running for this demo to work&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  1️⃣ - Installing 🛠
&lt;/h2&gt;

&lt;p&gt;I followed the &lt;a href="https://github.com/dapr/docs/blob/master/getting-started/environment-setup.md#using-script-to-install-the-latest-release" rel="noopener noreferrer"&gt;installing&lt;/a&gt; instructions, using the Linux script (since I'm &lt;a href="https://dev.to/arschles/how-to-wsl-516d"&gt;using WSL2&lt;/a&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;wget &lt;span class="nt"&gt;-q&lt;/span&gt; https://raw.githubusercontent.com/dapr/cli/master/install/install.sh &lt;span class="nt"&gt;-O&lt;/span&gt; - | /bin/bash
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;I had to provide my password because the script &lt;code&gt;sudo&lt;/code&gt;s things, so if you're not into letting a random script from GitHub get superuser on your computer, you can &lt;a href="https://github.com/dapr/docs/blob/master/getting-started/environment-setup.md#from-the-binary-releases" rel="noopener noreferrer"&gt;download binaries&lt;/a&gt; instead.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;It was installed and I could test things out:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;dapr &lt;span class="nt"&gt;--help&lt;/span&gt;

         __
    ____/ /___ _____  _____
   / __  / __ &lt;span class="s1"&gt;'/ __ \/ ___/
  / /_/ / /_/ / /_/ / /
  \__,_/\__,_/ .___/_/
              /_/

======================================================
A serverless runtime for hyperscale, distributed systems

Usage:
  dapr [command]

Available Commands:
  components     List all Dapr components
  configurations List all Dapr configurations
  help           Help about any command
  init           Setup dapr in Kubernetes or Standalone modes
  invoke         Invokes a Dapr app with an optional payload (deprecated, use invokePost)
  invokeGet      Issue HTTP GET to Dapr app
  invokePost     Issue HTTP POST to Dapr app with an optional payload
  list           List all Dapr instances
  logs           Gets Dapr sidecar logs for an app in Kubernetes
  mtls           Check if mTLS is enabled in a Kubernetes cluster
  publish        Publish an event to multiple consumers
  run            Launches Dapr and (optionally) your app side by side
  status         Shows the Dapr system services (control plane) health status.
  stop           Stops a running Dapr instance and its associated app
  uninstall      Removes a Dapr installation

Flags:
  -h, --help      help for dapr
      --version   version for dapr

Use "dapr [command] --help" for more information about a command.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It even has ascii art 🤡!&lt;/p&gt;

&lt;h2&gt;
  
  
  2️⃣ Hello, World 👋
&lt;/h2&gt;

&lt;p&gt;Let's get something running. The &lt;a href="https://github.com/dapr/samples/tree/master/1.hello-world" rel="noopener noreferrer"&gt;hello world tutorial&lt;/a&gt; actually goes much farther than I expected it would.&lt;/p&gt;

&lt;p&gt;You get an app up and talking to the storage API (backed by Redis), but then you get a second app up and running and have it call an API on the first one. It's actually showing off &lt;a href="https://github.com/dapr/docs/tree/master/concepts/service-invocation" rel="noopener noreferrer"&gt;service invocation&lt;/a&gt; too 🎉&lt;/p&gt;

&lt;p&gt;I first got the hello world sample code:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;git clone https://github.com/dapr/samples.git &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;cd &lt;/span&gt;samples/1.hello-world
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The sample code has a Python app and a Node app. Both has a REST API in them.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Node App 🗼
&lt;/h3&gt;

&lt;p&gt;First, the Node app. The &lt;a href="https://github.com/dapr/samples/tree/master/1.hello-world#step-2---understand-the-code" rel="noopener noreferrer"&gt;demo README&lt;/a&gt; first goes through the Node code a bit.&lt;/p&gt;

&lt;p&gt;The really interesting bit I saw was that we're doing a &lt;code&gt;POST&lt;/code&gt; request with the &lt;code&gt;fetch&lt;/code&gt; API to the local Dapr API to store some data. What was cool was you can also just return some JSON and it'll be automatically stored.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"state"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"key"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"nomnomnom"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"value"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"pizza"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I really like that part because you don't have to manually do anything to get data into the database. It just happens.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Farschles.com%2Fimages%2Fmagic.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Farschles.com%2Fimages%2Fmagic.jpg" alt="Magic"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;On to running the thing. You do the standard thing to install the JS dependencies:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;npm &lt;span class="nb"&gt;install
&lt;/span&gt;npm WARN node_server@1.0.0 No repository field.

added 55 packages from 41 contributors and audited 55 packages &lt;span class="k"&gt;in &lt;/span&gt;0.626s
found 0 vulnerabilities



   ╭────────────────────────────────────────────────────────────────╮
   │                                                                │
   │      New patch version of npm available! 6.14.4 → 6.14.5       │
   │   Changelog: https://github.com/npm/cli/releases/tag/v6.14.5   │
   │               Run npm &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-g&lt;/span&gt; npm to update!                │
   │                                                                │
   ╰────────────────────────────────────────────────────────────────╯
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;But you then run it with the &lt;code&gt;dapr&lt;/code&gt; CLI, not &lt;code&gt;npm&lt;/code&gt; or &lt;code&gt;node&lt;/code&gt;. This spits out a lot of log lines, and that's normal. Here's approximately what it looks like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;dapr run &lt;span class="nt"&gt;--app-id&lt;/span&gt; nodeapp &lt;span class="nt"&gt;--app-port&lt;/span&gt; 3000 &lt;span class="nt"&gt;--port&lt;/span&gt; 3500 node app.js
ℹ️  Starting Dapr with &lt;span class="nb"&gt;id &lt;/span&gt;nodeapp. HTTP Port: 3500. gRPC Port: 42491
&lt;span class="o"&gt;==&lt;/span&gt; DAPR &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"2020-05-18T16:26:52.702694-07:00"&lt;/span&gt; &lt;span class="nv"&gt;level&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;info &lt;span class="nv"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"starting Dapr Runtime -- version 0.6.0 -- commit e99f712-dirty"&lt;/span&gt; &lt;span class="nv"&gt;app_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;nodeapp &lt;span class="nv"&gt;instance&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;DESKTOP-DQP07VM &lt;span class="nv"&gt;scope&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;dapr.runtime &lt;span class="nb"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;log &lt;span class="nv"&gt;ver&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0.6.0

&amp;lt;snip&amp;gt;

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Node App listening on port 3000!

&lt;span class="o"&gt;==&lt;/span&gt; DAPR &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"2020-05-18T16:26:52.7568756-07:00"&lt;/span&gt; &lt;span class="nv"&gt;level&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;info &lt;span class="nv"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"application discovered on port 3000"&lt;/span&gt; &lt;span class="nv"&gt;app_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;nodeapp &lt;span class="nv"&gt;instance&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;DESKTOP-DQP07VM &lt;span class="nv"&gt;scope&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;dapr.runtime &lt;span class="nb"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;log &lt;span class="nv"&gt;ver&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0.6.0

&amp;lt;snip&amp;gt;

ℹ️  Updating metadata &lt;span class="k"&gt;for &lt;/span&gt;app &lt;span class="nb"&gt;command&lt;/span&gt;: node app.js
✅  You&lt;span class="s1"&gt;'re up and running! Both Dapr and your app logs will appear here.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Ok, now it's running. You use the &lt;code&gt;dapr&lt;/code&gt; CLI to &lt;em&gt;also&lt;/em&gt; make calls to the web service. I like the simplicity of a single CLI.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;dapr invoke &lt;span class="nt"&gt;--app-id&lt;/span&gt; nodeapp &lt;span class="nt"&gt;--method&lt;/span&gt; neworder &lt;span class="nt"&gt;--payload&lt;/span&gt; &lt;span class="s1"&gt;'{"data": { "orderId": "41" } }'&lt;/span&gt;
✅  App invoked successfully
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;You can also use &lt;code&gt;curl&lt;/code&gt; for this if you want&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And verifying that the request actually went through, and the data was stored, we have logs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Got a new order! Order ID: 41

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Successfully persisted state.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And we can also call the Node App's API to get the new data back out of the datastore&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;curl http://localhost:3500/v1.0/invoke/nodeapp/method/order
&lt;span class="o"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"orderId"&lt;/span&gt;:&lt;span class="s2"&gt;"41"&lt;/span&gt;&lt;span class="o"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;This and the above &lt;code&gt;dapr invoke&lt;/code&gt; call are using one of the features in Dapr's service discovery building block&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  The Python App 🐍
&lt;/h3&gt;

&lt;p&gt;Now for the grande finale! The Python code calls the Node app in an infinite loop, so we should be able to see the Node app responding to requests and also see the datastore fill up at the same time.&lt;/p&gt;

&lt;p&gt;First, we set up the Python dependency:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;pip3 &lt;span class="nb"&gt;install &lt;/span&gt;requests

&amp;lt;snip&amp;gt;

Successfully installed certifi-2020.4.5.1 chardet-3.0.4 idna-2.9 requests-2.23.0 urllib3-1.25.9
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And then I ran it with the same &lt;code&gt;dapr run&lt;/code&gt; command we used for the Node app, except in a new terminal tab. I cut out all the verbose log lines this time!&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;dapr run &lt;span class="nt"&gt;--app-id&lt;/span&gt; pythonapp python3 app.py
ℹ️  Starting Dapr with &lt;span class="nb"&gt;id &lt;/span&gt;pythonapp. HTTP Port: 43891. gRPC Port: 42163
ℹ️  Updating metadata &lt;span class="k"&gt;for &lt;/span&gt;app &lt;span class="nb"&gt;command&lt;/span&gt;: python3 app.py
✅  You&lt;span class="s1"&gt;'re up and running! Both Dapr and your app logs will appear here.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now, back in the Node app logs, storage is starting to fill up!&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Got a new order! Order ID: 1

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Successfully persisted state.

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Got a new order! Order ID: 2

&lt;span class="o"&gt;==&lt;/span&gt; APP &lt;span class="o"&gt;==&lt;/span&gt; Successfully persisted state.

&amp;lt;snip&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  That's It!
&lt;/h3&gt;

&lt;p&gt;From experience, getting storage and microservice-to-microservice calls working from scratch takes time, carefully crafted code, and effort.&lt;/p&gt;

&lt;p&gt;I wanted to compare this Node/Python codebase to compare my experience, and was pleasantly surprised. Let's break it down:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;87 lines of Node code, including comments

&lt;ul&gt;
&lt;li&gt;We could save a lot of lines if we returned the &lt;code&gt;"data"&lt;/code&gt; JSON dictionary instead of using &lt;code&gt;fetch&lt;/code&gt; calls&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;

&lt;li&gt;23 lines of Node code, including comments&lt;/li&gt;

&lt;/ul&gt;

&lt;p&gt;From experience with Go, I can say that a battle tested storage layer for this kind of thing would be about 50 lines. That's an apples to oranges comparison, but I know for sure that all the battle-tested bits are now behind that slick storage API the Node app is using. And funny enough, that part is written in Go 💪.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;P.S. don't forget to clean up: &lt;code&gt;dapr stop --app-id nodeapp &amp;amp;&amp;amp; dapr stop --app-id pythonapp&lt;/code&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Bonus: Behind the Scenes
&lt;/h2&gt;

&lt;p&gt;I mentioned above that you need to have Docker installed in order to get this working. That's all because Dapr needs an image running in the background for this demo to work.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cb722412a5d3        redis               &lt;span class="s2"&gt;"docker-entrypoint.s…"&lt;/span&gt;   4 weeks ago         Up 6 hours          0.0.0.0:6379-&amp;gt;6379/tcp     dapr_redis
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important one is ... Redis! By default, all the storage API calls the Node app is making get routed into Redis.&lt;/p&gt;

&lt;h2&gt;
  
  
  Wrap Up
&lt;/h2&gt;

&lt;p&gt;So we've seen the storage API &lt;em&gt;and&lt;/em&gt; a bit of the service invocation API. Honestly, those two pieces alone can get you a long way just on basic usage alone.&lt;/p&gt;

&lt;p&gt;As you get more advanced, you'll probably want to learn more about and use things like &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#concurrency" rel="noopener noreferrer"&gt;concurrency management&lt;/a&gt; and &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management#retry-policies" rel="noopener noreferrer"&gt;retry policies&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Any way you go, the &lt;a href="https://github.com/dapr/docs" rel="noopener noreferrer"&gt;docs repository&lt;/a&gt; and particularly the &lt;a href="https://github.com/dapr/docs/tree/master/concepts/state-management" rel="noopener noreferrer"&gt;state management section&lt;/a&gt; are great references.&lt;/p&gt;

&lt;p&gt;That's all for today. I hope you go forth and enjoy dapr-ing!&lt;/p&gt;

&lt;p&gt;🤘🚀&lt;/p&gt;

</description>
      <category>devops</category>
      <category>tutorial</category>
      <category>opensource</category>
      <category>database</category>
    </item>
    <item>
      <title>Crystal Fridays - May 22, 2020</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Thu, 21 May 2020 20:48:50 +0000</pubDate>
      <link>https://forem.com/arschles/crystal-fridays-is-tomorrow-mf7</link>
      <guid>https://forem.com/arschles/crystal-fridays-is-tomorrow-mf7</guid>
      <description>&lt;p&gt;I'll have some Windows 10 goodies for you as well 😉&lt;/p&gt;

&lt;p&gt;Details ⏬&lt;br&gt;
&lt;/p&gt;
&lt;blockquote class="ltag__twitter-tweet"&gt;
      &lt;div class="ltag__twitter-tweet__media ltag__twitter-tweet__media__video-wrapper"&gt;
        &lt;div class="ltag__twitter-tweet__media--video-preview"&gt;
          &lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--NVDoyL32--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://pbs.twimg.com/tweet_video_thumb/EYkbMrTU4AIvfgE.jpg" alt="unknown tweet media content"&gt;
          &lt;img src="/assets/play-butt.svg" class="ltag__twitter-tweet__play-butt" alt="Play butt"&gt;
        &lt;/div&gt;
        &lt;div class="ltag__twitter-tweet__video"&gt;
          
            
          
        &lt;/div&gt;
      &lt;/div&gt;

  &lt;div class="ltag__twitter-tweet__main"&gt;
    &lt;div class="ltag__twitter-tweet__header"&gt;
      &lt;img class="ltag__twitter-tweet__profile-image" src="https://res.cloudinary.com/practicaldev/image/fetch/s--BLOaqC02--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://pbs.twimg.com/profile_images/1028010275423707136/Y2CrNpCC_normal.jpg" alt="Aaron Schlesinger profile image"&gt;
      &lt;div class="ltag__twitter-tweet__full-name"&gt;
        Aaron Schlesinger
      &lt;/div&gt;
      &lt;div class="ltag__twitter-tweet__username"&gt;
        &lt;a class="mentioned-user" href="https://dev.to/arschles"&gt;@arschles&lt;/a&gt;
      &lt;/div&gt;
      &lt;div class="ltag__twitter-tweet__twitter-logo"&gt;
        &lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--kDgU_xDI--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev.to/assets/twitter-f95605061196010f91e64806688390eb1a4dbc9e913682e043eb8b1e06ca484f.svg" alt="twitter logo"&gt;
      &lt;/div&gt;
    &lt;/div&gt;
    &lt;div class="ltag__twitter-tweet__body"&gt;
      Tomorrow.&lt;br&gt;&lt;br&gt;&lt;a href="https://twitter.com/hashtag/CrystalFridays"&gt;#CrystalFridays&lt;/a&gt;.&lt;br&gt;&lt;br&gt;10am.&lt;br&gt;&lt;br&gt;&lt;a href="https://t.co/rTpua3yQQJ"&gt;twitch.tv/arschles&lt;/a&gt;.&lt;br&gt;&lt;br&gt;Be there for some &lt;a href="https://twitter.com/CrystalLanguage"&gt;@CrystalLanguage&lt;/a&gt; and a few surprises. 
    &lt;/div&gt;
    &lt;div class="ltag__twitter-tweet__date"&gt;
      20:43 PM - 21 May 2020
    &lt;/div&gt;


    &lt;div class="ltag__twitter-tweet__actions"&gt;
      &lt;a href="https://twitter.com/intent/tweet?in_reply_to=1263571080423157760" class="ltag__twitter-tweet__actions__button"&gt;
        &lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--OXOJJiQT--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev.to/assets/twitter-reply-action-238fe0a37991706a6880ed13941c3efd6b371e4aefe288fe8e0db85250708bc4.svg" alt="Twitter reply action"&gt;
      &lt;/a&gt;
      &lt;a href="https://twitter.com/intent/retweet?tweet_id=1263571080423157760" class="ltag__twitter-tweet__actions__button"&gt;
        &lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--foTp-unf--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev.to/assets/twitter-retweet-action-632c83532a4e7de573c5c08dbb090ee18b348b13e2793175fea914827bc42046.svg" alt="Twitter retweet action"&gt;
      &lt;/a&gt;
      &lt;a href="https://twitter.com/intent/like?tweet_id=1263571080423157760" class="ltag__twitter-tweet__actions__button"&gt;
        &lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--SFHqU4bF--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev.to/assets/twitter-like-action-1ea89f4b87c7d37465b0eb78d51fcb7fe6c03a089805d7ea014ba71365be5171.svg" alt="Twitter like action"&gt;
      &lt;/a&gt;
    &lt;/div&gt;
  &lt;/div&gt;
&lt;/blockquote&gt;


</description>
    </item>
    <item>
      <title>Windows, Now With More Toys</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Wed, 20 May 2020 23:46:16 +0000</pubDate>
      <link>https://forem.com/arschles/windows-now-with-more-toys-4377</link>
      <guid>https://forem.com/arschles/windows-now-with-more-toys-4377</guid>
      <description>&lt;p&gt;If you were around way back in the Windows 95 days, you might remember &lt;a href="https://socket3.wordpress.com/2016/10/22/using-windows-95-powertoys/"&gt;the PowerToys from then&lt;/a&gt;. Years and years later, it's back in a similar form, but for Windows 10.&lt;/p&gt;

&lt;p&gt;I just got started with it and it's another tool in my belt to help me feel comfortable using Windows every day after &lt;a href="https://deploy-preview-70--arschles-www.netlify.app/blog/coming-from-a-mac-to-windows-wsl-2/"&gt;many&lt;/a&gt; &lt;a href="https://deploy-preview-70--arschles-www.netlify.app/blog/how-to-wsl-2/"&gt;years&lt;/a&gt; of using Macs.&lt;/p&gt;

&lt;p&gt;Let me take you through it.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;You might have read that I recently &lt;a href="https://arschles.com/blog/coming-from-a-mac-to-windows-wsl-2/"&gt;came from a Mac&lt;/a&gt;. I have a follow-up post to this on how to use PowerToys Keyboard Shortcuts to make your keyboard more mac-like&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h1&gt;
  
  
  🤯❔
&lt;/h1&gt;

&lt;p&gt;PowerToys is a &lt;a href="(https://github.com/microsoft/PowerToys#current-powertoy-utilities)"&gt;few things in one package&lt;/a&gt;, and different folks describe it differently.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://github.com/microsoft/powertoys"&gt;official GitHub repository&lt;/a&gt; says this:&lt;/p&gt;

&lt;p&gt;"...set of utilities for power users to tune and streamline their Windows experience for greater productivity."&lt;/p&gt;

&lt;p&gt;Well, I love productivity (if you've seen any of my screencasts, I hit the keyboard shortcuts &lt;strong&gt;hard&lt;/strong&gt;), so I'm gonna go with that!&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;😎 By the way, PowerToys is 100% open source at &lt;a href="https://github.com/microsoft.com/PowerToys"&gt;github.com/microsoft.com/PowerToys&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I just installed this thing a few hours ago and started messing around with it. It's at version v0.18.0, but the install was not as bad as I thought. Let's go over it!&lt;/p&gt;

&lt;h2&gt;
  
  
  🍟 Get a Package Manager
&lt;/h2&gt;

&lt;p&gt;There are a &lt;a href="https://github.com/microsoft/PowerToys#installing-and-running-microsoft-powertoys"&gt;few ways to install it&lt;/a&gt;, but I decided to use &lt;a href="https://cda.ms/1hP"&gt;WinGet&lt;/a&gt; because it's also new and I hadn't used it yet.&lt;/p&gt;

&lt;p&gt;My colleague has &lt;a href="https://www.thomasmaurer.ch/2020/05/how-to-install-winget-windows-package-manager/"&gt;detailed instructions&lt;/a&gt; for installing WinGet, but the gist of it is you need to be a &lt;a href="https://cda.ms/1hQ"&gt;Windows Insider&lt;/a&gt; (free and open for anyone to join, despite how the name might sound 🎃) and then you can download it from the &lt;a href="https://dev.toms-windows-store:/pdp/?productid=9nblggh4nns1"&gt;Windows Store&lt;/a&gt;. That's the easiest way.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;WinGet is also open source &lt;a href="https://github.com/microsoft/winget-cli"&gt;here&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  🥨 Use WinGet to Install It
&lt;/h2&gt;

&lt;p&gt;When you have it installed, getting PowerToys is a command:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;WinGet&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;install&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;powertoys&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;Run this in PowerShell, not WSL2&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  🍕 Get Used to the Features
&lt;/h2&gt;

&lt;p&gt;The very first thing to do is bask in the awesomeness of "PowerToys Run". Seriously.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Just hit left-alt+space and a search bar pops up.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;You can type anything in there and quickly get to "things" on your computer. That generally means a metric ton of things that you didn't even know you had. But it also means ... YOU CAN GET TO YOUR APPS WITH NO CLICKS!!!!🎉🤘🏄‍♀️. &lt;/p&gt;

&lt;p&gt;I really missed that from my Mac days.&lt;/p&gt;

&lt;p&gt;Anyway, hit left-alt+space, type in "PowerToys", and hit enter. That will take you to something that looks like the standard Windows settings app, but it's actually the &lt;em&gt;added&lt;/em&gt; features that PowerToys gives.&lt;/p&gt;

&lt;p&gt;Check out that left bar. 7 new features plus a "General" thing at the top. I'm gonna boil each complex feature down into a single sentence, Aaron style™. Here goes:&lt;/p&gt;

&lt;h4&gt;
  
  
  🍚 General
&lt;/h4&gt;

&lt;p&gt;Go here to do general stuff (😂), most importantly getting updates to PowerToys itself.&lt;/p&gt;

&lt;h4&gt;
  
  
  🥧 FancyZones
&lt;/h4&gt;

&lt;p&gt;Automatically resize each window in different patterns on the screen, like side by side or in rows.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Yes, the name is amazing. The zones are indeed fancy 🍰! But the feature overall is pretty slick and powerful once you get the hang of it. You'll want to try out a few different setups to figure out the best one for you.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h4&gt;
  
  
  🍣 File Explorer preview
&lt;/h4&gt;

&lt;p&gt;See rendered previews of SVGs and rendered Markdown in the file explorer. Optional second sentence 💰: I imagine there will be more formats in the future.&lt;/p&gt;

&lt;h4&gt;
  
  
  🍤 Image resizer
&lt;/h4&gt;

&lt;p&gt;Resize any image with a right click from the file explorer.&lt;/p&gt;

&lt;h4&gt;
  
  
  🍎Keyboard Manager
&lt;/h4&gt;

&lt;p&gt;Remap any key or keyboard shortcut from a nice UI.&lt;/p&gt;

&lt;h4&gt;
  
  
  🍉 PowerRename
&lt;/h4&gt;

&lt;p&gt;Rename tons and tons of files by highlighting them all and right clicking.&lt;/p&gt;

&lt;h4&gt;
  
  
  🍍 PowerToys run
&lt;/h4&gt;

&lt;p&gt;The belle of the ball for me - basically Alfred or Spotlight for Mac, but for Windows.&lt;/p&gt;

&lt;h4&gt;
  
  
  🥦 Shortcut guide
&lt;/h4&gt;

&lt;p&gt;A full-screen pop-up that reminds you any time of the keyboard shortcuts you have set up.&lt;/p&gt;

&lt;h2&gt;
  
  
  🚢 Keep Calm and PowerToy On!
&lt;/h2&gt;

&lt;p&gt;Of course, I'm not doing any of these items justice in a single sentence. You can learn a bit more about each one &lt;a href="https://github.com/microsoft/powertoys#current-powertoy-utilities"&gt;here&lt;/a&gt;, and follow the links in each section to go deeper.&lt;/p&gt;

&lt;p&gt;Look out for my next post on using PowerToys to make your Mac keyboard work smoother Windows. See you soon!&lt;/p&gt;

&lt;p&gt;👋&lt;/p&gt;

</description>
      <category>windows</category>
      <category>devrel</category>
      <category>programming</category>
      <category>macos</category>
    </item>
    <item>
      <title>Crystal Fridays - May 16, 2020</title>
      <dc:creator>Aaron Schlesinger</dc:creator>
      <pubDate>Fri, 15 May 2020 02:40:34 +0000</pubDate>
      <link>https://forem.com/arschles/crystal-fridays-is-tomorrow-540j</link>
      <guid>https://forem.com/arschles/crystal-fridays-is-tomorrow-540j</guid>
      <description>&lt;p&gt;Hope to see you there! This week, we're gonna continue with our docs server.&lt;/p&gt;

&lt;p&gt;See you tomorrow at 10am US Pacific Time, twitch.tv/arschles&lt;/p&gt;

</description>
      <category>crystal</category>
      <category>programming</category>
      <category>livecoding</category>
    </item>
  </channel>
</rss>
