<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>Forem: Ivan Starkov</title>
    <description>The latest articles on Forem by Ivan Starkov (@istarkov).</description>
    <link>https://forem.com/istarkov</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F743530%2F0f4caa1c-ece8-4a61-83ae-698f5662bff0.jpeg</url>
      <title>Forem: Ivan Starkov</title>
      <link>https://forem.com/istarkov</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://forem.com/feed/istarkov"/>
    <language>en</language>
    <item>
      <title>Using Opencode as a Copy-Paste Backend for UI Prototyping</title>
      <dc:creator>Ivan Starkov</dc:creator>
      <pubDate>Sun, 23 Nov 2025 10:47:55 +0000</pubDate>
      <link>https://forem.com/istarkov/using-opencode-as-a-copy-paste-backend-for-ui-prototyping-4afc</link>
      <guid>https://forem.com/istarkov/using-opencode-as-a-copy-paste-backend-for-ui-prototyping-4afc</guid>
      <description>&lt;p&gt;&lt;a href="https://github.com/istarkov/ai-cli-edit" rel="noopener noreferrer"&gt;Link to repo&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;

  &lt;iframe src="https://www.youtube.com/embed/uPVvXCytOA8"&gt;
  &lt;/iframe&gt;


&lt;/p&gt;

&lt;p&gt;Sometimes I use OpenCode (and tools like Claude Code, Codex CLI, etc.) as a copy-paste backend: I prepare context in the browser, then paste it into an AI coding tool.&lt;/p&gt;

&lt;p&gt;At its core, most modern AI coding workflows boil down to two operations:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Selecting&lt;/strong&gt; the relevant context.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Chatting&lt;/strong&gt; about that selection.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In my setup, a tiny browser script handles the selection: it collects and shapes the context, writes a structured payload to the clipboard, and then I paste it into OpenCode, Claude Code, Codex CLI, or any other AI coding tool just to see how it behaves - without building a real AI backend.&lt;/p&gt;




&lt;h4&gt;
  
  
  How the selection script works
&lt;/h4&gt;

&lt;p&gt;When you hit &lt;code&gt;Escape&lt;/code&gt; in the browser, the &lt;a href="https://github.com/istarkov/ai-cli-edit/blob/main/client/selection.client.ts" rel="noopener noreferrer"&gt;selection script&lt;/a&gt; grabs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;your prompt (e.g. “Change color to red”)&lt;/li&gt;
&lt;li&gt;the current selection (code / text)&lt;/li&gt;
&lt;li&gt;some structure around it&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;and writes a structured payload to the clipboard, like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight xml"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;edit&amp;gt;&lt;/span&gt;
  prompt: Change color to red
  where:&lt;span class="nt"&gt;&amp;lt;selection&amp;gt;&lt;/span&gt;{context}&lt;span class="nt"&gt;&amp;lt;/selection&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;/edit&amp;gt;&lt;/span&gt;
...
{ADDITIONAL_PROMPT}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You can try this in the browser here: &lt;a href="https://istarkov.github.io/ai-cli-edit/" rel="noopener noreferrer"&gt;https://istarkov.github.io/ai-cli-edit/&lt;/a&gt; — press &lt;code&gt;Cmd + E&lt;/code&gt; or &lt;code&gt;Ctrl + E&lt;/code&gt; to enter editing mode and see the generated payload.&lt;/p&gt;




&lt;h4&gt;
  
  
  Making it work across different models
&lt;/h4&gt;

&lt;p&gt;Slow, higher-end reasoning models can usually consume this raw structure without extra help.&lt;/p&gt;

&lt;p&gt;Smaller or faster models often need the &lt;a href="https://github.com/istarkov/ai-cli-edit/blob/main/client/prompt.client.ts" rel="noopener noreferrer"&gt;&lt;code&gt;{ADDITIONAL_PROMPT}&lt;/code&gt;&lt;/a&gt; with more explicit instructions — for example:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;how to interpret &lt;code&gt;&amp;lt;selection&amp;gt;…&amp;lt;/selection&amp;gt;&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;what to edit and what to keep identical&lt;/li&gt;
&lt;li&gt;formatting rules&lt;/li&gt;
&lt;li&gt;which tools to call&lt;/li&gt;
&lt;/ul&gt;




&lt;h4&gt;
  
  
  Keep editing instructions isolated
&lt;/h4&gt;

&lt;p&gt;You &lt;em&gt;could&lt;/em&gt; dump all of this into &lt;code&gt;CLAUDE.md&lt;/code&gt; or &lt;code&gt;AGENTS.md&lt;/code&gt;, but those files are usually already full of generic rules and global guidelines.&lt;/p&gt;

&lt;p&gt;Better approach: keep editing-specific instructions in a separate, dedicated place.&lt;/p&gt;

&lt;p&gt;In OpenCode, this is done via &lt;a href="https://opencode.ai/docs/agents/#primary-agents" rel="noopener noreferrer"&gt;Primary Agents&lt;/a&gt;. Create a focused file like &lt;a href="https://github.com/istarkov/ai-cli-edit/blob/main/.opencode/agent/edit.md" rel="noopener noreferrer"&gt;&lt;code&gt;./.opencode/agent/edit.md&lt;/code&gt;&lt;/a&gt; that defines:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;your editing rules&lt;/li&gt;
&lt;li&gt;model parameters&lt;/li&gt;
&lt;li&gt;tools&lt;/li&gt;
&lt;li&gt;project-specific context (jargon, naming, edge cases, etc.)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In the Claude ecosystem, the same idea appears as &lt;a href="https://code.claude.com/docs/en/skills" rel="noopener noreferrer"&gt;Skills&lt;/a&gt;: small, targeted capabilities that encapsulate exactly this kind of task-specific behavior.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/istarkov/ai-cli-edit" rel="noopener noreferrer"&gt;Link to repo&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>tooling</category>
      <category>ui</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Local Development with Public and Private monorepos</title>
      <dc:creator>Ivan Starkov</dc:creator>
      <pubDate>Tue, 20 Jun 2023 21:41:32 +0000</pubDate>
      <link>https://forem.com/istarkov/local-development-with-public-and-private-monorepos-3554</link>
      <guid>https://forem.com/istarkov/local-development-with-public-and-private-monorepos-3554</guid>
      <description>&lt;p&gt;In the project I'm working on now &lt;a href="https://webstudio.is" rel="noopener noreferrer"&gt;webstudio.is&lt;/a&gt;, we have 2 mono repositories.&lt;/p&gt;

&lt;p&gt;One open and fully &lt;a href="https://github.com/webstudio-is/webstudio-builder" rel="noopener noreferrer"&gt;public&lt;/a&gt; and one closed source.&lt;/p&gt;

&lt;p&gt;Projects from the closed repository use libraries from the open one.&lt;/p&gt;

&lt;p&gt;So usually working on a closed repository looks like this&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;we publish libraries from public repository into npm&lt;/li&gt;
&lt;li&gt;we update libraries in a closed repository &lt;code&gt;pnpm up -r -L '@webstudio-is/*'&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Local development becomes terribly messy with this approach.&lt;br&gt;
Each iteration takes an awful lot of time &lt;em&gt;(15 minutes to 30 minutes on average)&lt;/em&gt;. &lt;br&gt;
Create PR wait for all checks to be done, wait for all checks to be done and publish.&lt;/p&gt;

&lt;p&gt;To avoid all this I tried.&lt;/p&gt;
&lt;h2&gt;
  
  
  Link protocol
&lt;/h2&gt;

&lt;p&gt;in package.json of the closed packages I replaced all&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="nl"&gt;"dependency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"@webstudio-is/form-handlers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"^0.75.0"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="err"&gt;...&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;to&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="nl"&gt;"dependency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"@webstudio-is/form-handlers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"link:/Users/User/webstudio-is/webstudio-builder/packages/form-handlers"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="err"&gt;...&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This worked for some of the packages in the closed repository. But some packages got runtime errors like &lt;code&gt;TypeError: Cannot read properties of null (reading 'useContext')&lt;/code&gt;. &lt;br&gt;
It seemed like some dependencies were duplicated and I didn't manage to get rid of this error.&lt;/p&gt;
&lt;h2&gt;
  
  
  Soft link packages from one monorepo into another
&lt;/h2&gt;

&lt;p&gt;I plugged the open repository as a workspace into a closed repo. &lt;code&gt;ln -s /Users/ice/webstudio-is/webstudio-designer/packages packages&lt;/code&gt; and added into workspaces. The result is the same as with 1.&lt;/p&gt;

&lt;p&gt;I guess it's possible to fix it but I failed.&lt;/p&gt;
&lt;h2&gt;
  
  
  Verdaccio
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://twitter.com/7rulnik?lang=en" rel="noopener noreferrer"&gt;Valentin Semirulnik&lt;/a&gt; gave me an idea to use &lt;a href="https://verdaccio.org/" rel="noopener noreferrer"&gt;verdaccio&lt;/a&gt;&lt;br&gt;
The idea to use private proxy registry for local development&lt;/p&gt;

&lt;p&gt;And it worked.&lt;/p&gt;

&lt;p&gt;Briefly how I use it. &lt;br&gt;
Everytime I need to check how an open library will work with a closed one, I do.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Run verdaccio locally&lt;/span&gt;
verdaccio
&lt;span class="c"&gt;# build all libs&lt;/span&gt;
pnpm run build
&lt;span class="c"&gt;# change version of all public libraries from current to exprimental like "0.75.0" =&amp;gt; "0.75.1-qawqisps.0" etc&lt;/span&gt;
pnpm &lt;span class="nt"&gt;-r&lt;/span&gt; &lt;span class="nb"&gt;exec &lt;/span&gt;pnpm version prepatch &lt;span class="nt"&gt;--preid&lt;/span&gt; &lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;cat&lt;/span&gt; /dev/urandom | &lt;span class="nv"&gt;LC_ALL&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;C &lt;span class="nb"&gt;tr&lt;/span&gt; &lt;span class="nt"&gt;-dc&lt;/span&gt; &lt;span class="s1"&gt;'a-z'&lt;/span&gt; | &lt;span class="nb"&gt;fold&lt;/span&gt; &lt;span class="nt"&gt;-w&lt;/span&gt; 8 | &lt;span class="nb"&gt;head&lt;/span&gt; &lt;span class="nt"&gt;-n&lt;/span&gt; 1&lt;span class="si"&gt;)&lt;/span&gt;
&lt;span class="c"&gt;# Publish all to verdaccio registry&lt;/span&gt;
pnpm &lt;span class="nt"&gt;-r&lt;/span&gt; publish &lt;span class="nt"&gt;--access&lt;/span&gt; public &lt;span class="nt"&gt;--no-git-checks&lt;/span&gt; &lt;span class="nt"&gt;--registry&lt;/span&gt; http://localhost:4873
&lt;span class="c"&gt;# Reset version changes I made to package.json (DANGEROUS! as can remove changes you made)&lt;/span&gt;
git restore &lt;span class="nt"&gt;--source&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;git branch &lt;span class="nt"&gt;--show-current&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt; &lt;span class="s1"&gt;'**/*/package.json
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And now in a closed project I can just run&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pnpm up &lt;span class="nt"&gt;-r&lt;/span&gt; &lt;span class="nt"&gt;-L&lt;/span&gt; &lt;span class="s1"&gt;'@webstudio-is/*'&lt;/span&gt; &lt;span class="nt"&gt;--registry&lt;/span&gt; http://localhost:4873
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now it takes me 1-2 minutes to check something. This is still a lot but I have not found a better solution.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>devops</category>
      <category>javascript</category>
      <category>npm</category>
    </item>
    <item>
      <title>JS defer or how to free resources at the end of code block (fun way).</title>
      <dc:creator>Ivan Starkov</dc:creator>
      <pubDate>Thu, 27 Oct 2022 20:40:26 +0000</pubDate>
      <link>https://forem.com/istarkov/js-defer-or-how-to-free-resources-at-the-end-of-code-block-fun-way-3h81</link>
      <guid>https://forem.com/istarkov/js-defer-or-how-to-free-resources-at-the-end-of-code-block-fun-way-3h81</guid>
      <description>&lt;p&gt;Interesting feature of the Go language is to delay the execution of a function until the current function returns - defer keyword.&lt;/p&gt;

&lt;p&gt;It allows to declare resource cleanup near the place it created i.e.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="n"&gt;file&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fileName&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;file&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="o"&gt;...&lt;/span&gt;
&lt;span class="n"&gt;file&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Close&lt;/span&gt; &lt;span class="n"&gt;will&lt;/span&gt; &lt;span class="n"&gt;be&lt;/span&gt; &lt;span class="n"&gt;called&lt;/span&gt; &lt;span class="n"&gt;at&lt;/span&gt; &lt;span class="n"&gt;the&lt;/span&gt; &lt;span class="n"&gt;end&lt;/span&gt; &lt;span class="n"&gt;of&lt;/span&gt; &lt;span class="n"&gt;function&lt;/span&gt;
&lt;span class="o"&gt;...&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The similar feature in js is &lt;code&gt;finally&lt;/code&gt; keyword. The issue with &lt;code&gt;finally&lt;/code&gt; that you open resource in one place but need to close it far far away from opening position. &lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;easy to make mistake&lt;/li&gt;
&lt;li&gt;in case of dependent resources easy to break order etc. &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In js we can have similar behavior like defer using generators, see example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;createResource&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;T&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;open&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;T&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;close&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;T&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;void&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;r&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;T&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nx"&gt;r&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;finally&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;r&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;r&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;open&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;open&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;somehandle&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;close&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;handle&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;handle&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;closed&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;await &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;x&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nf"&gt;createResource&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;open&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;close&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;x&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;setTimeout&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
  &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="c1"&gt;// throw new Error();&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Output will be: 'open', 1, 2, 'somehandle closed'&lt;/p&gt;

&lt;p&gt;Now our resource lifecycle is bounded to "for" block. We declared &lt;code&gt;open&lt;/code&gt;, &lt;code&gt;close&lt;/code&gt; in same place and it's somehow looks like &lt;code&gt;defer&lt;/code&gt;. &lt;/p&gt;

&lt;p&gt;PS: &lt;code&gt;createResource&lt;/code&gt; should rethrow Error so its just small simple version to show the idea.&lt;/p&gt;

&lt;p&gt;PPS: All above is just for fun to show generators power. &lt;/p&gt;

</description>
      <category>javascript</category>
      <category>typescript</category>
      <category>tutorial</category>
      <category>go</category>
    </item>
    <item>
      <title>Simple and fast way to run various devops workflows on your servers</title>
      <dc:creator>Ivan Starkov</dc:creator>
      <pubDate>Sun, 23 Oct 2022 15:29:08 +0000</pubDate>
      <link>https://forem.com/istarkov/simple-and-fast-way-to-run-various-devops-workflows-on-your-servers-37co</link>
      <guid>https://forem.com/istarkov/simple-and-fast-way-to-run-various-devops-workflows-on-your-servers-37co</guid>
      <description>&lt;p&gt;We periodically need to run different workflows on different servers. For example stop replication, do a vacuum or global database update, start replication. &lt;br&gt;
I don't want to give everyone ssh access to the servers. In addition some workflows need to be run by non-developers. &lt;br&gt;
So to solve this problem I need some simple UI to run workflows, the ability to see logs and the ability to easily and quickly create complex workflows.&lt;br&gt;
Recently I found what I think is the perfect and simple solution. This is &lt;a href="https://docs.github.com/en/actions" rel="noopener noreferrer"&gt;github actions&lt;/a&gt; + &lt;a href="https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners" rel="noopener noreferrer"&gt;self-hosted runners&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I roll out github action runner on each of my servers in addition to the usual set of software. (The code to install via terraform, namely the key generation is taken from here &lt;a href="https://github.com/myoung34/docker-github-actions-runner" rel="noopener noreferrer"&gt;https://github.com/myoung34/docker-github-actions-runner&lt;/a&gt;)&lt;/p&gt;

&lt;p&gt;Next I write the workflow:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Tasks&lt;/span&gt;

&lt;span class="na"&gt;on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;workflow_dispatch&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;inputs&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;DEPLOYMENT&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;choice&lt;/span&gt;
        &lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Database&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;is&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;production&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;or&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;test'&lt;/span&gt;
        &lt;span class="na"&gt;options&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;production&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;test&lt;/span&gt;
        &lt;span class="na"&gt;default&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;test&lt;/span&gt;
        &lt;span class="na"&gt;required&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;

      &lt;span class="na"&gt;task&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;choice&lt;/span&gt;
        &lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Select&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;task'&lt;/span&gt;
        &lt;span class="na"&gt;options&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;stop-replication&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;start-replication&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;ps&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;restart&lt;/span&gt;
        &lt;span class="na"&gt;default&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ps&lt;/span&gt;
        &lt;span class="na"&gt;required&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;

      &lt;span class="na"&gt;country&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;choice&lt;/span&gt;
        &lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Select&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;country'&lt;/span&gt;
        &lt;span class="na"&gt;options&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;ch&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;fr&lt;/span&gt;
        &lt;span class="na"&gt;required&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;

&lt;span class="na"&gt;jobs&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;main&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;runs-on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;self-hosted&lt;/span&gt;&lt;span class="pi"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;clickhouse-runner&lt;/span&gt;&lt;span class="pi"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;linux&lt;/span&gt;&lt;span class="pi"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;${{&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;github.event.inputs.DEPLOYMENT&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;}}"&lt;/span&gt;&lt;span class="pi"&gt;]&lt;/span&gt;
    &lt;span class="na"&gt;steps&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;if&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ github.event.inputs.task == 'restart' }}&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;docker compose restart&lt;/span&gt;

      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;if&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ github.event.inputs.task == 'ps' }}&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;docker compose ps&lt;/span&gt;

      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;if&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ github.event.inputs.task == 'stop-replication' }}&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
          &lt;span class="s"&gt;docker compose stop clickhouse-replication-updater clickhouse-init&lt;/span&gt;
          &lt;span class="s"&gt;docker compose exec -iT clickhouse clickhouse-client --query="DROP DATABASE IF EXISTS some_db"&lt;/span&gt;
          &lt;span class="s"&gt;echo Stopped&lt;/span&gt;

      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;if&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ github.event.inputs.task == 'start-replication' }}&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
          &lt;span class="s"&gt;docker compose start clickhouse-replication-updater clickhouse-init&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Here &lt;code&gt;workflow_dispatch.inputs&lt;/code&gt; gives nice UI at github&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fj8gdr5m17p6cwrtg6wkv.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fj8gdr5m17p6cwrtg6wkv.png" alt="Workflow run UI" width="668" height="516"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;And you can use steps/jobs etc to generate any workflow depending on inputs.&lt;/p&gt;

&lt;p&gt;With &lt;code&gt;runs-on: [self-hosted, clickhouse-runner, linux, "${{ github.event.inputs.DEPLOYMENT }}"]&lt;/code&gt; section you are able to select any server where you need to execute workflow.&lt;/p&gt;

&lt;p&gt;Since now everyone with access to repositary can execute server workflows with beautiful UI, logs etc.&lt;/p&gt;

</description>
      <category>github</category>
      <category>devops</category>
    </item>
    <item>
      <title>Use docker image tagging to speedup the builds.</title>
      <dc:creator>Ivan Starkov</dc:creator>
      <pubDate>Mon, 22 Nov 2021 12:19:15 +0000</pubDate>
      <link>https://forem.com/istarkov/use-docker-image-tagging-to-speedup-the-builds-3n4m</link>
      <guid>https://forem.com/istarkov/use-docker-image-tagging-to-speedup-the-builds-3n4m</guid>
      <description>&lt;p&gt;From the beginning, we did a review deployment of each commit of each pull requst.&lt;br&gt;
Initially it was two builds/deploys and accordingly you could not think about caching.&lt;br&gt;
Over 4 years this has grown to 15+ builds/deploys and only the build time has become 20+ minutes.&lt;/p&gt;

&lt;p&gt;After some reading we came across docker buildx and decided to use the features it provides,&lt;br&gt;
namely &lt;a href="https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-from" rel="noopener noreferrer"&gt;cache-from&lt;/a&gt; and &lt;a href="https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-to" rel="noopener noreferrer"&gt;cache-to&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Idea was to create some checksum for every subproject, then use that checksum for cache-from, cache-to source/destination.&lt;br&gt;
Like it's described &lt;a href="https://cloud.google.com/build/docs/speeding-up-builds" rel="noopener noreferrer"&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In our case it worked but did not give a very significant advantage. Given the number of different builds, we got that the time to download all the cache was comparable to the build time.&lt;/p&gt;

&lt;p&gt;Also having that we use yarn based monorepo to reuse docker layering we created internal &lt;code&gt;prune&lt;/code&gt; util which is not that simple and adds additional steps to the build. &lt;em&gt;(about prune you can read at amazing &lt;a href="https://turborepo.com/docs/reference/command-line-reference#turbo-prune---scopetarget" rel="noopener noreferrer"&gt;turborepo&lt;/a&gt;)&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Our builds are very simple, +-fast and consist of one step, there are a lot of them and so we did not get the significant advantage we expected.&lt;/p&gt;

&lt;p&gt;We decided not to use the cache, but to take advantage of the fact that a developer only works on one or two projects in one pull request. So we don't need to rebuild everything, but only the projects that have changed.&lt;/p&gt;

&lt;p&gt;And thanks to docker it allowed to do everything we need.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Single docker image can have mutiple tags.&lt;/li&gt;
&lt;li&gt;We can add tag to docker image without pulling it and gcloud even have ready command for this &lt;a href="https://cloud.google.com/sdk/gcloud/reference/container/images/add-tag" rel="noopener noreferrer"&gt;gcloud container images add-tag&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;We already had workspace checksum util from our previous work iteration with cache layers.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Now in pseudocode our build process for each subproject on every commit (yarn workspace) looks like.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;CHECKSUM&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;generate-workspace-checksum &lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;WORSPACE_NAME&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;if &lt;/span&gt;gcloud container images describe &lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;IMAGE_NAME&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;:&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CHECKSUM&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
  &lt;/span&gt;gcloud container images add-tag &lt;span class="nt"&gt;--quiet&lt;/span&gt; &lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;IMAGE_NAME&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;:&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CHECKSUM&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt; &lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;IMAGE_NAME&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;:&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;SHORT_SHA&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;else
  &lt;/span&gt;docker buildx build &lt;span class="nb"&gt;.&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;--file&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;universal.Dockerfile &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;--tag&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;IMAGE_NAME&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;:&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;SHORT_SHA&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;--tag&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;IMAGE_NAME&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;:&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CHECKSUM&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;--progress&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;plain &lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;BUILD_ARGS&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;--push&lt;/span&gt;
&lt;span class="k"&gt;fi&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We generate workspace checksum, check if image with tag equal to checksum exists, and if yes we add commit sha tag to existing image. This is really fast operation.&lt;br&gt;
In case if image is not exits we just rebuild workspace and tag image with 2 tags - checksum and commit sha.&lt;/p&gt;

&lt;p&gt;Deploys were already fast and executed in parallel so doing 2 or 15 deploys in our case doesnt matter a lot.&lt;/p&gt;

&lt;p&gt;Trick above allowed us to significantly reduce build times, sometimes to seconds (like documentation has changed) instead of 20+ minutes for every commit.&lt;/p&gt;

&lt;p&gt;For projects with mutibuild steps, various dependency installs etc above solution could not work, and caching would be the best solution. For us just image retagging works the best, removes the need of some external KV storage (sha =&amp;gt; checksum), significantly removed build times and simplified builds.&lt;/p&gt;

</description>
      <category>googlecloud</category>
      <category>devops</category>
    </item>
    <item>
      <title>Fast and easy way to setup web developer certificates</title>
      <dc:creator>Ivan Starkov</dc:creator>
      <pubDate>Sat, 13 Nov 2021 21:00:22 +0000</pubDate>
      <link>https://forem.com/istarkov/fast-and-easy-way-to-setup-web-developer-certificates-450e</link>
      <guid>https://forem.com/istarkov/fast-and-easy-way-to-setup-web-developer-certificates-450e</guid>
      <description>&lt;p&gt;Modern days having that cookies auth etc depends on https we need to have https local web environment.&lt;/p&gt;

&lt;p&gt;Before to generate local certificates I used &lt;a href="https://github.com/jsha/minica" rel="noopener noreferrer"&gt;minica&lt;/a&gt;. &lt;/p&gt;

&lt;p&gt;The main issue that you need a big readme for osx, linux and windows users, how to regenerate keys, &lt;br&gt;
how to add minica certificate to Keychain, how to change hosts file.&lt;/p&gt;

&lt;p&gt;Having that we use vscode remote for development it was 2x more work to register all that keys on local and remote machines.&lt;/p&gt;

&lt;p&gt;The solution below doesnt need any setup from developers. &lt;/p&gt;
&lt;h2&gt;
  
  
  Solution in short
&lt;/h2&gt;

&lt;p&gt;Register on DNS provider A records for development like:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;A blabla.devdomain.com 127.0.0.1&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Then using letsencrypt &lt;a href="https://certbot.eff.org/" rel="noopener noreferrer"&gt;certbot&lt;/a&gt; for your provider just generate needed certificates.&lt;/p&gt;

&lt;p&gt;They are already trusted and the only issue is 3 month expiration period, what can be easily fixed with cron.&lt;/p&gt;
&lt;h2&gt;
  
  
  Full solution.
&lt;/h2&gt;

&lt;p&gt;In our case we use cloudflare as DNS. &lt;br&gt;
Generation certificates for few domains on cloudflare looks:&lt;/p&gt;

&lt;p&gt;Create cloudflare API token &lt;a href="https://support.cloudflare.com/hc/en-us/articles/200167836-Managing-API-Tokens-and-Keys#12345680" rel="noopener noreferrer"&gt;https://support.cloudflare.com/hc/en-us/articles/200167836-Managing-API-Tokens-and-Keys#12345680&lt;/a&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;TF_VAR_CLOUDFLARE_API_KEY&lt;/span&gt;&lt;span class="o"&gt;={&lt;/span&gt;YOURAPITOKEN&lt;span class="o"&gt;}&lt;/span&gt;

&lt;span class="nb"&gt;mkdir&lt;/span&gt; &lt;span class="nt"&gt;-p&lt;/span&gt; /tmp/certbot/
&lt;span class="nb"&gt;mkdir&lt;/span&gt; &lt;span class="nt"&gt;-p&lt;/span&gt; /tmp/letsencrypt/

&lt;span class="nb"&gt;cat&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; /tmp/certbot/cloudflare.ini &lt;span class="o"&gt;&amp;lt;&amp;lt;-&lt;/span&gt;&lt;span class="no"&gt;DOCKERFILE&lt;/span&gt;&lt;span class="sh"&gt;
  dns_cloudflare_api_token = &lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;TF_VAR_CLOUDFLARE_API_KEY&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;
&lt;/span&gt;&lt;span class="no"&gt;DOCKERFILE

&lt;/span&gt;docker run &lt;span class="nt"&gt;-it&lt;/span&gt; &lt;span class="nt"&gt;--rm&lt;/span&gt; &lt;span class="nt"&gt;--name&lt;/span&gt; certbot  &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;-v&lt;/span&gt; &lt;span class="s2"&gt;"/tmp/letsencrypt/data:/etc/letsencrypt"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;-v&lt;/span&gt; &lt;span class="s2"&gt;"/tmp/certbot:/local/certbot"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
certbot/dns-cloudflare:v1.15.0 certonly &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;-m&lt;/span&gt; istarkov@gmail.com &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--dns-cloudflare&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--dns-cloudflare-credentials&lt;/span&gt; /local/certbot/cloudflare.ini &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--agree-tos&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--noninteractive&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;-d&lt;/span&gt; subdomain.mydomain.com &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;-d&lt;/span&gt; other.mydomain.com &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;-d&lt;/span&gt; blabla.hello.com

&lt;span class="c"&gt;# subdomain.mydomain.com, other.mydomain.com, blabla.hello.com must have A records on cloudflare pointing to 127.0.0.1&lt;/span&gt;

&lt;span class="nb"&gt;cp&lt;/span&gt; /tmp/letsencrypt/data/live/subdomain.mydomain.com/&lt;span class="k"&gt;*&lt;/span&gt; ./
&lt;span class="nb"&gt;cat&lt;/span&gt; ./fullchain.pem ./privkey.pem &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; ./haproxy.pem
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;thats all, now for nodejs apps use following https options&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;fs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;readFileSync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;./privkey.pem&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
&lt;span class="nx"&gt;cert&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;fs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;readFileSync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;./fullchain.pem&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;for haproxy use &lt;code&gt;haproxy.pem&lt;/code&gt; like in simple config below&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight ini"&gt;&lt;code&gt;&lt;span class="c"&gt;# haproxy -f ./playground/haproxy-http-2.cfg -db
&lt;/span&gt;
&lt;span class="err"&gt;frontend&lt;/span&gt; &lt;span class="err"&gt;rgw-https&lt;/span&gt;
  &lt;span class="err"&gt;bind&lt;/span&gt; &lt;span class="err"&gt;*:3009&lt;/span&gt; &lt;span class="err"&gt;ssl&lt;/span&gt; &lt;span class="err"&gt;crt&lt;/span&gt; &lt;span class="err"&gt;/root/realadvisor/https-dev-keys/haproxy.pem&lt;/span&gt; &lt;span class="err"&gt;alpn&lt;/span&gt; &lt;span class="err"&gt;h2,http/1.1&lt;/span&gt;
  &lt;span class="err"&gt;default_backend&lt;/span&gt; &lt;span class="err"&gt;rgw&lt;/span&gt;


&lt;span class="err"&gt;backend&lt;/span&gt; &lt;span class="err"&gt;rgw&lt;/span&gt;
  &lt;span class="err"&gt;balance&lt;/span&gt; &lt;span class="err"&gt;roundrobin&lt;/span&gt;
  &lt;span class="err"&gt;mode&lt;/span&gt; &lt;span class="err"&gt;http&lt;/span&gt;
  &lt;span class="err"&gt;server&lt;/span&gt;  &lt;span class="err"&gt;rgw1&lt;/span&gt; &lt;span class="err"&gt;127.0.0.1:3000&lt;/span&gt; &lt;span class="err"&gt;check&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is fast and simple way I prefer now to have development certificates, which doesnt need any additional documentation for developers.&lt;/p&gt;

</description>
      <category>devops</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Google cloud build webhooks on specific commit SHA.</title>
      <dc:creator>Ivan Starkov</dc:creator>
      <pubDate>Tue, 02 Nov 2021 23:03:06 +0000</pubDate>
      <link>https://forem.com/istarkov/google-cloud-build-webhooks-on-specific-commit-sha-44h2</link>
      <guid>https://forem.com/istarkov/google-cloud-build-webhooks-on-specific-commit-sha-44h2</guid>
      <description>&lt;p&gt;The gcloud build has webhook triggers that allow you to trigger the build using a simple http api.&lt;br&gt;
How to create such a trigger can be found &lt;a href="https://cloud.google.com/build/docs/automating-builds/create-webhook-triggers" rel="noopener noreferrer"&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The disadvantage for me was that the web trigger cannot be applied to a specific commit, but only to a branch such as master,&lt;br&gt;
moreover the branch name is fixed. &lt;/p&gt;

&lt;p&gt;Today I found an interesting way to trigger a build using a webhook trigger for any commit.&lt;/p&gt;

&lt;p&gt;Briefly, I create a normal push branch trigger using a branch name that can't exist. Then I create a webhook trigger which&lt;br&gt;
using the command &lt;code&gt;gcloud alpha builds triggers run --sha={COMMIT_SHA}&lt;/code&gt; to run the first trigger with the &lt;code&gt;commitSha&lt;/code&gt; of specific commit.&lt;/p&gt;

&lt;p&gt;Now the pictures.&lt;/p&gt;
&lt;h3&gt;
  
  
  This is push branch trigger.
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fpabqrf6x3wwbs5lrh5he.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fpabqrf6x3wwbs5lrh5he.png" alt="push trigger" width="630" height="1354"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h3&gt;
  
  
  This is the webhook trigger.
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fs18en4k2chltrlh4v4i3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fs18en4k2chltrlh4v4i3.png" alt="webhook trigger" width="608" height="1368"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;with following substitutions&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdnc4k4v3rot5vn20b6to.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdnc4k4v3rot5vn20b6to.png" alt="substitutions" width="800" height="314"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;and following inline source&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;steps&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;google/cloud-sdk'&lt;/span&gt;
    &lt;span class="na"&gt;args&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;-c'&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
        &lt;span class="s"&gt;set -eou pipefail&lt;/span&gt;
        &lt;span class="s"&gt;shopt -s inherit_errexit&lt;/span&gt;

        &lt;span class="s"&gt;if [[ "${COMMIT_SHA}" ]]; then&lt;/span&gt;
          &lt;span class="s"&gt;echo "HERE ${COMMIT_SHA}"&lt;/span&gt;
          &lt;span class="s"&gt;gcloud alpha builds triggers run ${_TRIGGER_NAME} --sha ${COMMIT_SHA}&lt;/span&gt;
        &lt;span class="s"&gt;else&lt;/span&gt;
          &lt;span class="s"&gt;echo "MASTER"&lt;/span&gt;
          &lt;span class="s"&gt;gcloud alpha builds triggers run ${_TRIGGER_NAME} --branch=master&lt;/span&gt;
        &lt;span class="s"&gt;fi&lt;/span&gt;
    &lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;aggregator-custom-build&lt;/span&gt;
    &lt;span class="na"&gt;waitFor&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;-'&lt;/span&gt;
    &lt;span class="na"&gt;entrypoint&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;bash&lt;/span&gt;
&lt;span class="na"&gt;tags&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;webhook-trigger&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Finally
&lt;/h3&gt;

&lt;p&gt;To invoke webhook trigger you need just.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;    curl &lt;span class="nt"&gt;-X&lt;/span&gt; POST &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"application/json"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="s2"&gt;"https://cloudbuild.googleapis.com/v1/projects/&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;PROJECT&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/triggers/webhook-invoke-trigger:webhook?key=&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;API_KEY&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"COMMIT_SHA": "${COMMIT_SHA}", "TRIGGER_NAME": "webhook-invoke-agg-custom-update"}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That allows me to run specific build steps like periodic data updates using "sha fixed" scripts.&lt;/p&gt;

</description>
      <category>googlecloud</category>
    </item>
  </channel>
</rss>
