Forem: Anton Golub

CVE-2024-29415: problem solution

Anton Golub — Tue, 16 Sep 2025 14:39:58 +0000

an illustrative note on the relationship logic between Enterprise and OSS based on a specific issue

If you've landed on this page, it's likely that your codebase relies on a widely deployed package that is currently struggling to maintain and is therefore vulnerable to certain scenarios. You have several strategies to follow:

Ask the author for a fix
Suggest a fix
Apply patches locally
Override internal registry versions
Consider migrating to other solutions
Just live with it

It's always a question of rational choice and cost of effort. In our case, the dependency tree profile required too much cascading fixes including external components, so we decided to take matters into our own hands — create a fully compatible alternative and apply it to our systems.

We found that the address classification error (private/public) was due to a lack of strictness in the parser implementation, and the format error was interpreted as a sign of being in (or out of) the range. An additional complication was that the logic was divided into several auxiliary functions, but was combined differently in the compositions of parsing addresses of variouus notations.

Algorithm

In the first attempt, we simply made the pattern conditions strict, but as expected, we encountered the problem of reduced performance. It became immediately clear that stricter conditions would require a different algorithmic approach.
Here we moved step by step: from split() to a string traverser, until we were satisfied with the ops/s metric.

The second significant change was related to the internal structure of stored addresses: Buffers have been replaced with bigints, which allowed us to greatly simplify and speed up the logic of comparisons and format translation. So it became easy to implement proper and full support for IPv6 for mask() and subnet() methods.

Address.from('0.0.0.0').big // 0n

The third major aspect of algorithmic improvements is related to the refactoring of special IP address ranges declarations. Instead of a set of magic conditions in helpers, a single source of truth is allocated now, and productive predicate functions are preinitialized.

https://en.wikipedia.org/wiki/Reserved_IP_addresses

const SPECIALS: Record<Special, string[]> = {
  loopback: [
    '127.0.0.0/8',    // IPv4 loopback
    '::1/128',        // IPv6 loopback
  ],
  // ...
]
const SPECIAL_MATCHERS: ((addr: Address) => Special | undefined)[] = []
for (const [cat, cidrs] of Object.entries(SPECIALS)) {
  for (const cidr of cidrs) {
    for (const x of ipv6fySubnet(cidr)) {
      const subnet = Address.cidrSubnet(x)
      SPECIAL_MATCHERS.push((addr: Address) => addr.family === subnet.family && subnet.contains(addr) ? (cat as Special) : undefined)
    }
  }
}

Compatibility

The key point was to keep backward compatibility. To ensure this, we completely borrowed the test suite from the upstream repo. For clarity, we also additionally investigated how similar other IP tools are in their operating logic. The conclusion is obvious: without additional modifications, it is impossible to achieve equivalent operating results in all scenarios. It was also important for us to make sure that the library functionality would not differ in different versions of runtimes and would be compatible with the browser environment. Polyfills and an extensive palette of smoke tests are right for this.

Drop-in

As we noted at the beginning, the process of cascading dependency updates is expensive, and some tools of the JS ecosystem provide workarounds for this. And the package should strive for seamlessness: so we've added an additional pkg entry point to support legacy IDE-hinted require('ip/lib/ip.js) statements.

- const ip = require('ip')
+ const ip = require('@webpod/ip')

Temporary workaround to avoid refactoring is using overrides / resolutions in your package.json:

{
  "overrides": {
    "ip": "@webpod/ip"
  }
}

Next steps

There is no discussion here. The only correct way is to push changes to upstream, reduce the heterogeneity of dependencies. I hope that maintainers, when they have enough time, will support the community's efforts in this direction. Until that happens, there is @webpod/ip

zx@lite: minimalistic shell scripting with TS/JS superpowers

Anton Golub — Thu, 19 Jun 2025 14:33:00 +0000

Ditch clunky bash syntax. Script smarter — with fewer dependencies and full TypeScript support.

⚡ What is `zx@lite`?

The original zx makes shell automations delightful in modern JS runtimes. The @lite version, introduced in v8.5.0, gives you optimal performance, devX without the extra API bloat:

✅ Core zx functionality ($, cd, ps)
✅ First-class TypeScript support
⚡ ~7× smaller than full zx (~124 KB vs ~875 kB)
🧱 Static build with zero dependencies.
🔌 Wide compat range: Node.js 12+, Deno 1.30+, Bun 1.0+, GraalVM 17+ on Win, Mac or Linux

Ideal for:

Fast script prototyping
DevOps and CI automations
Docker images
Small CLI tooling

🧾 Setup

npm install zx@lite

TypeScript is an option but highly recommended for safety and better IDE support.

npm install -D typescript @types/node

Attach tsconfig.json like this:

{
  "compilerOptions": {
    "target": "esnext",
    "module": "esnext",
    "moduleResolution": "nodenext",
    "strict": true
  }
}

🧪 Examples

Let’s say you want to clone several repos and list their directories. In bash, this would be a mess of &, wait, and error handling. In zx@lite, it’s much cleaner:

#!/usr/bin/env zx
import { $ } from 'zx'

// Run 3 commands in parallel
await Promise.all([
  $`git clone https://github.com/google/zx zx-clone`,
  $`git clone https://github.com/webpod/webpod webpod-clone`
])

const output = await $`ls -1`
console.log('Cloned directories:\n', output.lines())

Native parallelism with Promise.all()
Clean output handling via stdout, lines(), json() helpers
Controlled error flow: configurable exception throwing behavior

Combine zx@lite with runtime features. Need transport? Just bring in native fetch:

#!/usr/bin/env zx
import { $ } from 'zx'

const repo = 'google/zx'
const latest = await (await fetch(`https://api.github.com/repos/${repo}/releases/latest`)).json()
const v = latest.tag_name
const zip_file = `${repo.replace('/', '-')}-${v}.zip`

await $`curl -o ${zip_file} -L ${latest.zipball_url}`
await $`mkdir -p releases/${v}`
await $`unzip ${zip_file} -d releases/${v}`

console.log('✅ Done')

⚠️ What’s missing in `zx@lite`?

Feature	`zx`	`zx@lite`
`$`	✅	✅
`cd`	✅	✅
`ps`	✅	✅
`glob`	✅	❌
`YAML`	✅	❌
`fs-extra`	✅	❌
`minimist`	✅	❌

Check the full API comparison for details.

📦 Upgrade path

Switching between lite and full is trivial:

# Go full speed:
npm install zx

# Back to lean mode:
npm install zx@lite

📚 References

🧠 Final thoughts

zx@lite is everything you need — and nothing you don’t — for modern scripting with real programming ergonomics. Stop writing fragile bash. Start scripting like a developer.

💬 Got ideas, snippets, or zx@lite tips? Drop them below or feel free to share in the project repo: discussions, issues.

How and why do we bundle zx?

Anton Golub — Mon, 06 May 2024 08:12:21 +0000

When zx first appeared, it was a tiny esm script that just proposed a new idea for how child_process.spawn API could be enhanced with string template literals.

const result = await $`call smth --with any ${args}`

As we now know, this was exactly what JS folks had been looking for for many years :-). The main goal was to improve the developer experience, so we focused on finding the best combination of utils to solve everyday problems, expand the area of application.

Short history

The project went through 8 major releases, but in the context of the assembly issues, there are only 3 significant ones.

Packed as CommonJS

Despite the fact that the future belongs to ESM, many users have to remain in the notation of their legacy codebase (without any irony). So we have added compatibility with the require API for them.

https://github.com/google/zx/commit/303feea3cd0d95e8d733ee72b96b3316ea5827aa

Dropped CJS support

On the other hand, there were libraries on which zx depend: and only the latest modern major versions received patches, including vulnerabilities fixes. In fact we inherited CVE issues transitively, which was completely unacceptable. And we switched back to ESM.

https://github.com/google/zx/commit/5578d67ebae2049d430bf6489784f6a2aa47b0b1

Migrated to TypeScript

As the code base grew, it became harder for us to ensure consistency of internal contracts. Adding typing was the solution, but it also made us dependent on transpilation and we could no longer control the package artifact in detail.

https://github.com/google/zx/commit/8c2aea89fe7c65767d29c52bfb48f6a46af01864

Problems

Size

Filling the zx with new functionality, we continued to expand the dependency tree, which required their own dependencies, etc. Installation time increased significantly. And at some point it became impossible to ignore: zx#712, zx#669

1.15.2	2.1.0	3.1.0	4.3.0	5.3.0	6.2.5	7.2.3
1.38 MB	3.01 MB	3.01 MB	3.89 MB	10.5 MB	10.7 MB	16.3 MB

Compatibility

When we define node engine requirement we limit not users, but ourselves in what platform API we can use. But this in no way prevents our dependencies from pursuing their own policies which affects zx.

Stability & Security

Reproducible setups are important. Especially for utilities of our type. We can try to pin down versions, but if there is a caret (^, ~) or an asterisk (*) at the secondary level, this introduces node_modules variability. Lockfiles can mitigate the problem, but not for npx/yarn dlx execution case.

Bundle

At first we wanted to just get rid of all the helper utilities. Keep only the kernel, but this would mean a loss of backward compatibility. We needed some efficient code processing instead with recomposition and tree-shaking. We needed a bundler. But which one? Our testing approach relies on targets, not sources. We rebuilt the project frequently, speed was critical requirement. In essence, we chose a solution from a couple of among all available alternatives: esbuild and parcel. Esbuild won. Specifically in our case, it proved to be more productive and customizable.

js

Customizable means that we found a way to adapt the tool for our tasks. And a lot had to be improved, including writing our own plugins:

Esbuild provides code-splitting with dynamic imports to reuse common code chunks. But not is static form. The manual chunks API is missing.
esbuild-plugin-entry-chunks
By default, esbuild injects helpers into each cjs module. It's fine, when you have just a few, but definitely not when there are many.
esbuild-plugin-extract-helpers
We also wanted to bring back cjs support, but at the same time avoid duplicating code in two versions of the bundles.
esbuild-plugin-hybrid-export
Esbuild and its plugins mostly focus on sources processing, but sometimes additional modification is also necessary for dependencies or bundles:

Polyfill injects: esbuild#2840, esbuild#3517, esbuild#3099
Custom patches: esbuild#3360
Dynamic banners: esbuild#3291
Importable helpers: esbuild#1230.

esbuild-plugin-transform-hook

d.ts

While we were fighting against the modules, we forgot one small detail - their built-in typings. Esbuild can't do this at all yet. Unbelievable, but the tsc, native TS compiler, also does not provide a typings concat feature. Got around this problem: we've introduced a utility to combine typings of zx own code, and applied some monkey patches for external libdefs squashed via dts-bundle-generator.

Results

There is no silver bullet, all tools and approaches have limitations. Summary:

Install size reduced: 16Mb → 1Mb
Reproducible npx invocations
Hybrid package: ESM and CJS
Wide range of supported Node.js versions 12...22
Easy to audit: complete code in one place

Perhaps our efforts are not commensurate with the profit. But we gained invaluable experience. Now we understand the entire zx code in detail down to each char.

zx build setup

google / zx

A tool for writing better scripts

zx

#!/usr/bin/env zx

await $`cat package.json | grep name`

const branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`

await Promise.all([
  $`sleep 1; echo 1`,
  $`sleep 2; echo 2`,
  $`sleep 3; echo 3`,
])

const name = 'foo bar'
await $`mkdir /tmp/${name}`

Bash is great, but when it comes to writing more complex scripts, many people prefer a more convenient programming language. JavaScript is a perfect choice, but the Node.js standard library requires additional hassle before using. The zx package provides useful wrappers around child_process, escapes arguments and gives sensible defaults.

Install

npm install zx

Documentation

Read documentation on google.github.io/zx.

License

Apache-2.0

Disclaimer: This is not an officially supported Google product.

View on GitHub

config: apply extends

Anton Golub — Mon, 05 Feb 2024 08:00:02 +0000

As we know, the forms and approaches to describing configuration files are extremely varied. I discussed this in the previous note. But there are several techniques and practices that may one day become standards.

One of this – the extends directive. Many tools provide this feature, but it works a little differently in each place. For example, tsconfig applies deep merge to compilerOptions, while prettier concatenates overrides array sections, commitlint joins plugin, etc. So the authors have to implement these nuances on site every time, and it's tiring and annoying a bit. I believe we will come up with a more efficient way to handle this routine.

Proposal

import { populate, populateSync } from '@topoconfig/extends'

const tsconfig = await populate('tsconfig.json', {
  compilerOptions: 'merge'
})
const prettierCfg = populateSync('.prettierrc.json', {
  overrides: 'merge'
})

20 minutes adventure

At first glance, the extends may seem like a simple combination of Object.assign and fs.readFile. Well, technically it is, but not quite so. Let's see how the real scenario unfolds. To simplify the details, we will use pseudocode.

The starting point:

input = {
  foo: 'bar',
  extends: './base.json'
}

pick extends

resource = input.extends

load the referenced ./base.json

extra = JSON.parse(fs.readFileSync(resource, 'utf8'))

merge input and extra

result = {
  ...input,
  ...extra,
  extends: undefined,
}

How about ./base.js? Reasonable request.

provide js files support

input = { extends: './base.js' }

require hurries to the resque:

load = (ref) => isJs(ref) ? require(ref) : JSON.parse(fs.readFileSync(ref, 'utf8'))

./base.mjs?

Ooops. Here the API switches to async mode to handle mjs/esm inputs. Of course, we can patch require with esm package, or load module via a process.spawnSynced nodejs. We can, but we won't

load = isJs(ref)
  ? (await import(ref))?.default
  : fs.promises.readFile(ref, 'utf8')

What if the value is passed without an extension, like base?

add support for shared npm configs
provide configurable resolve

resolve = (ref, cwd) =>
  ref.startsWith('.') || ref.startsWith('/') || path.extname(ref)
    ? path.resolve(cwd, ref)
    : ref

But what if extends is a nested object? It definitely is.

apply everything recursively:

handle = input = ({
  ...input,
  ...handle(load(input.extends)),
  extends: undefined,
})

In this case, circular references are something to pay attention to.

add index to avoid infinite loops

memo = new Map()

memoizedLoad = ref => {
  if (memo.has(ref)) {
    return memo.get(ref)
  }
  memo.set(ref, load(ref))
  return load(ref)
}

Relative resource path should use the closest parent dirname as a base to work properly.

resolve cwd for each extends reference

base = path.resolve(process.cwd(), _opts.cwd ?? '.')
[cwd, _config] = typeof config === 'string'
  ? [path.resolve(base, path.dirname(config)), path.basename(config)]
  : [base, config]

Multiple extends maybe?

let extends hold several references

input = {
  foo: 'bar',
  extends: ['./extra.json', './mixin.js']
}
extras = [input.extends].flat().map(load)

How to read toml, yaml, ini, env? Obviously, custom formats are required too.

provide parse option

parse = (ref, contents) => {
  if (ref.endsWith('.yaml') || ref.endsWith('.yml')) 
      return parseYaml(contents)
  if (ref.endsWith('.json')) 
      return JSON.parse(contents)
  throw new Error(`Unsupported format: ${ref}`)
}

How to resolve value conflicts: merge / override?

provide rules to handle prop-specific join strategies

rules = {
  '*': 'merge',
  foo: 'override'
}
rule = getRule(propPath, rules)
result[key] = isObject(value) && rule === 'merge'
  ? extend({
    sources: [value],
    rules,
    prefix: p,
    index
  })
  : value

What about arrays?

implement array merge as concat

if (rule === 'merge') {
  arr.push(...sources.flat(1))
} else {
  arr.length = 0
  arr.push(...sources.slice(-1).flat(1))
}

It would be nice to avoid mutations.

introduce clone

safeLoad = compose(memoize, clone, load)

Conclusion

Summarizing:

search extends statements
load & parse referenced resources
- support async and sync flows
- use cloning to avoid mutations
- .js, .cjs, .mjs, .json
- provide support for custom formats
- resolve cwd for the nested resource by its closest parent resource
- support custom resolvers
walk recursively
- handle circular references
provide sync and async API
assemble the result
- handle arrays / plain objects
- apply prop-specific merge

Additionally:

build as cjs and esm
provide types
handle windows specific issues
add dozens of tests
handle different runtimes specifics (node, deno, bun)

There is an obvious benefit to implementing this algorithm on your own – it clarifies and makes controllable every aspect of config processing in place.
However, we do not always have enough time to do this exercise. Therefore, maybe someone will find this shortcut useful:

config: enhancing declarations with graphs

Anton Golub — Tue, 03 Oct 2023 11:14:44 +0000

Typically application configs are not a bottleneck or development issue. But everything can suddenly change when the context of expands with infrastructure domain, external sources interactions, environments.

Config mess

Many years ago configs were pretty simple. They looked more or less like .properties-files or INI-files, simple kv-maps with sections or composite keys to bring some kind of context:

# https://docs.oracle.com/cd/E23095_01/Platform.93/ATGProgGuide/html/s0204propertiesfileformat01.html

# You are reading a comment in ".properties" file.
! The exclamation mark can also be used for comments.
# Lines with "properties" contain a key and a value separated by a delimiting character.
# There are 3 delimiting characters: '=' (equal), ':' (colon) and whitespace (space, \t and \f).
website = https://en.wikipedia.org/
language : English
topic .properties files
# A word on a line will just create a key with no value.
empty

; last modified 1 April 2001 by John Doe
[owner]
name = John Doe
organization = Acme Widgets Inc.

[database]
; use IP address in case network name resolution is not working
server = 192.0.2.62     
port = 143
file = "payroll.dat"

At the same time, another part of the configuration was supplied from the environment variables or CLI parameters reflecting the idea of dynamic settings.

Now we use dotenv-files, ironic :

# https://hexdocs.pm/dotenvy/0.2.0/dotenv-file-format.html
S3_BUCKET=YOURS3BUCKET
SECRET_KEY=YOURSECRETKEYGOESHERE

Even then, the resolution logic began to penetrate into the app layer.

// Just an illustration. This problem existed before JS was invented

const config = require('config')
const logLevel = process.env.DEBUG ? 'trace' : config.get('log.level') || 'info'
//...
const dbConfig = config.get('Customer.dbConfig')
db.connect(dbConfig, ...)

if (config.has('optionalFeature.detail')) {
  const detail = config.get('optionalFeature.detail')
  //...
}

When centralized configuration management came, the settings has been moved partially to the remote storage. Local pre-config (entrypoints, db credentials) was used to get the rest. Configuration assembly itself has become multi-stage.

Later, specialized systems such as vault made new additions: now env holds an access token and defines an entrypoint by running mode to make a POST request to reveal credentials profile to mix this data to the entire config.

Here's how uniconfig obtains secrets from the vault storage:

{
  "data": {
    "secret": "$vault:data"
  },
  "sources": {
    "vault": {
      "data": {
        "data": {
          "method": "GET",
          "url": "$url:",
          "opts": {
            "headers": {
              "X-Vault-Token": "$token:auth.client_token"
            }
          }
        },
        "sources": {
          "url": {
            "data": {
              "data": {
                "data": {
                  "name": "$pkg:name",
                  "space": "openapi",
                  "env": "$env:ENVIRONMENT_PROFILE_NAME",
                  "vaultHost": "$env:VAULT_HOST",
                  "vaultPort": "$env:VAULT_PORT"
                },
                "template": "{{=it.env==='production' ? 'https': 'http'}}://{{=it.vaultHost}}:{{=it.vaultPort}}/v1/secret/applications/{{=it.space}}/{{=it.name}}"
              },
              "sources": {
                "env": {
                  "pipeline": "env"
                },
                "pkg": {
                  "pipeline": "pkg"
                }
              }
            },
            "pipeline": "datatree>dot"
          },
          "token": {
            "data": {
              "data": {
                "method": "POST",
                "url": "$url:",
                "opts": {
                  "json": {
                    "role": "$pkg:name",
                    "jwt": "$jwt:"
                  }
                }
              },
              "sources": {
                "pkg": {
                  "pipeline": "pkg"
                },
                "jwt": {
                  "data": {
                    "data": {
                      "data": {
                        "tokenPath": "$env:TOKEN_FILE",
                        "defaultTokenPath": "/var/run/secrets/kubernetes.io/serviceaccount/token"
                      },
                      "template": "{{=it.tokenPath || it.defaultTokenPath}}"
                    },
                    "sources": {
                      "env": {
                        "pipeline": "env"
                      }
                    }
                  },
                  "pipeline": "datatree>dot>file"
                },
                "url": {
                  "data": {
                    "data": {
                      "data": {
                        "env": "$env:ENVIRONMENT_PROFILE_NAME",
                        "vaultHost": "$env:VAULT_HOST",
                        "vaultPort": "$env:VAULT_PORT"
                      },
                      "template": "{{=it.env==='production' ? 'https': 'http'}}://{{=it.vaultHost}}:{{=it.vaultPort}}/v1/auth/kubernetes/login"
                    },
                    "sources": {
                      "env": {
                        "pipeline": "env"
                      },
                      "pkg": {
                        "pipeline": "pkg"
                      }
                    }
                  },
                  "pipeline": "datatree>dot"
                }
              }
            },
            "pipeline": "datatree>http>json"
          }
        }
      },
      "pipeline": "datatree>http>json"
    }
  }
}

Meanwhile, formats have been evolving (JSON5, YAML), config entry points are constantly changing. These fluctuations, fortunately, were covered by tools like the cosmiconfig.

[
  'package.json',
  `.${moduleName}rc`,
  `.${moduleName}rc.json`,
  `.${moduleName}rc.yaml`,
  `.${moduleName}rc.yml`,
  `.${moduleName}rc.js`,
  `.${moduleName}rc.ts`,
  `.${moduleName}rc.mjs`,
  `.${moduleName}rc.cjs`,
  `.config/${moduleName}rc`,
  `.config/${moduleName}rc.json`,
  `.config/${moduleName}rc.yaml`,
  `.config/${moduleName}rc.yml`,
  `.config/${moduleName}rc.js`,
  `.config/${moduleName}rc.ts`,
  `.config/${moduleName}rc.cjs`,
  `${moduleName}.config.js`,
  `${moduleName}.config.ts`,
  `${moduleName}.config.mjs`,
  `${moduleName}.config.cjs`,
]

Configs are still trying to be declarative, but they can't. Templates appeared first.

template:
    metadata:
      annotations:
        cni.projectcalico.org/ipv4pools: '["${APP_NAME}"]'
        vault.hashicorp.com/agent-init-first: "true"
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/secrets-injection-method: "env"
        vault.hashicorp.com/secrets-type: "static"
        vault.hashicorp.com/agent-inject-secret-${APP_NAME}: secret-v2/applications/${DEPLOYMENT_NAMESPACE}/${APP_NAME}
        vault.hashicorp.com/agent-inject-template-${APP_NAME}: |
          {{ with secret "secret-v2/applications/${DEPLOYMENT_NAMESPACE}/${APP_NAME}" }}
            {{- range $secret_key, $secret_value := .Data.data }}
            export {{ $secret_key }}={{ $secret_value }}
            {{- end }}
          {{ end }}
        vault.hashicorp.com/auth-path: ${AUTH_PATH}
        vault.hashicorp.com/role: ${APP_NAME}

Then templates inside templates. With commands and scripts invocations inside dynamic DSL wrapped into matrices.

      - uses: actions/cache@v3
        id: yarn-cache
        with:
          path: ${{ needs.init.outputs.yarn-cache-dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-

      - name: Restore artifact from cache (if exists)
        uses: actions/cache@v3
        with:
          path: artifact.tar
          key: artifact-${{ needs.init.outputs.checksum }}

      - name: Check artifact
        if: always()
        id: check-artifact
        run: echo "::set-output name=exists::$([ -e "artifact.tar" ] && echo true || echo false)"

As we can see, syntax complexity increases as the cost of declarativeness. It's still unclear how this problem can be mitigated. Perhaps new specialized formats will appear or more strict forms (schemas) of using existing ones will be introduced.

Budget loss

Anyway, ::$([ is definitely not an optimal solution. Сonfusing, fragile and overcomplicated for the most developers. For example, here is how Python Engineer was fighting against kube.yaml:

fix vault in kube yaml Jul 04   XS      
fix vault in kube yaml Jul 04   XS
fix vault in kube yaml Jul 04   XS
fix vault in kube yaml Jul 04   XS
fix vault in kube yaml Jul 04   XS
fix vault in kube yaml Jul 04   XS
fix vault in kube yaml Jul 04   XS
fix vault in kube yaml Jul 04   XS
fix vault in kube yaml Jul 03   XS
fix vault in kube yaml Jul 03   XS
fix vault in kube yaml Jul 03   XS
fix vault in kube yaml Jul 03   XS
fix vault in kube yaml Jul 03   XS
fix vault in kube yaml Jul 03   XS
...

This is definitely not configuring but more guessing. On a company scale, such exercises are a significant waste of resources. And this experience is almost one-time only, which cannot be formalized and transmitted except by copy-paste. Every time we see the same thing, with a different number of attempts.

What we need

The overcomplexity problem seems to have arisen from the fact that we combined resolving, processing and accessing data into one structure. Although the entire theory of programming / CS instructs us to do exactly the opposite. Separation of concerns: imagine a config which explicitly divides value resolutions, compositions and operations.

{
  "data": "<how to expose values>",
  "sources": "<how to resolve values>",
  "cmds": "<available cmds/ops/fns>"
}

Let data to represent how the result structure may be built if all the required transformations were made — like a mapping.

{
  "data": {
    "a": {
      "b": "$b.some.nested.prop.value.of.b",
      "c": "$external.prop.of.prop"
    }
  }
}

Templating bases on regular substring replacements:

String.format("foo %s", "bar")                   // gives 'foobar'
// But positional contract is enhanced with named refmap
String.format("foo $a $b $a", {"a": "A", "b": "B"}) // returns 'foo A B A'
//            ↑ data chunks ↑ sources map

Let sources to describe how to obtain and process values for referencing in data map. Like reducing pipelines.

{
  "sources": {
    "a": "<pipeline 1>",
    "b": "<pipeline 2>"
  }
}

Let pipeline to compose actions in natural ~~human~~ dev-readable format like CLI:

cmd param > cmd2 param param > ... > cmd3

Let intermediate values be referenced by lateral (bubbling concept) or nested contexts.

{
  "sources": {
    "a" : "cmd param",
    "b": "cmd $a" // b refers to a
  }
}

Apply DAG walker for consistency checks and processing.

Working draft

https://github.com/antongolub/misc/tree/master/packages/topoconfig

Install

yarn add topoconfig@draft

API

import {topoconfig} from 'topoconfig'

const config = await topoconfig({
  // define functions to use in pipelines: sync or async
  cmds: {
    foo: () => 'bar',
    baz: async (v) => v + 'qux'
  },
  // pipelines to resolve intermediate variables
  sources: {
    a: 'foo > baz', // pipeline returns 'barqux'
    b: {            // b refers to b.data
      data: {
        c: {
          d: 'e'
        }
      }
    }
  },
  // output value
  data: {
    // $name.inner.path populates var ref with its value
    x: '$b.c.d',  // 'e'
    y: {
      z: '$a'     // 'barqux'
    }
  }
})

Real-world usage example may look like:

import {topoconfig} from 'topoconfig'

const config = await topoconfig({
  data: {
    foo:      '$a',
    url:      'https://some.url',
    param:    'regular param value',
    num:      123,
    pwd:      '\\$to.prevent.value.inject.use.\.prefix',
    a: {
      b:      '$b.some.nested.prop.value.of.b',
      c:      '$external.prop.of.prop'
    },
    log: {
      level:  `$loglevel`
    }
  },
  sources: {
    a:        'file ./file.json utf8',
    b:        'json $a',
    c:        'get $b > assert type number',
    cwd:      'cwd',
    schema:   'file $cwd/schema.json utf8 > json',
    external: 'fetch http://foo.example.com > get .body > json > get .prop > ajv $schema',
    extended: 'extend $b $external',
    loglevel: 'find $env.LOG_LEVEL $argv.log-level $argv.log.level info',
    template: `dot {{? $name }}
<div>Oh, I love your name, {{=$name}}!</div>
{{?? $age === 0}}
<div>Guess nobody named you yet!</div>
{{??}}
You are {{=$age}} and still don't have a name?
{{?}} > assert $foo`,
  },
  cmds: {
    // http://olado.github.io/doT/index.html
    dot:      (...chunks) => dot.template(chunks.join(' '))({}),
    extend:   Object.assign,
    cwd:      () => process.cwd(),
    file:     (file, opts) => fs.readFile(file, opts),
    json:     JSON.parse,
    get:      lodash.get,
    argv:     () => minimist(process.argv.slice(2)),
    env:      () => process.env,
    find:     (...args) => args.find(Boolean),
    fetch:    async (url) => {
      const res = await fetch(url)
      const code = res.status
      const headers = Object.fromEntries(res.headers)
      const body = await res.body()

      return {
        res,
        headers,
        body,
        code
      }
    },
    //...
  }
})

Notes

export type TData = number | string | { [key: string]: TData } | { [key: number]: TData }
export type TCmd = (...opts: any[]) => any
export type TCmds = Record<string | symbol, TCmd>
export type TConfigDeclaration = {
  data: TData,
  sources: Record<string, string | TConfigDeclaration>
  cmds?: TCmds
}

TConfigDeclaration defines two sections: data and sources:

data describes how to build the result value based on the bound sources: it populates $-prefixed refs with their values in every place.
sources is a map, which declares the algorithm to resolve intermediate values through cmd calls composition. To fetch data from remote, to read from file, to convert, etc.

{
  "data": "$res",
  "sources": {
    "res": "fetch https://example.com > get .body > json"
  }
}

cmd is a provider that performs a specific action.

type TCmd = (...opts: any[]) => any

directive is a template for defining a value transformation pipeline

// fetch http://foo.example.com > get body > json > get .prop
// ↑ cmd ↑opts                  ↑ pipes delimiter

Next steps

Add ternaries: cmd ? cmd1 > cmd2 ... : cmd
Handle or statement: cmd > cmd || cmd > cmd
Expose commands presets: import {cmds} from 'topoconfig/cmds' or @topoconfig/cmds
Provide lazy-loading for cmds:

{
  cmds: {
    foo: 'some/package',
    bar: './local/plugin.js'
  }
}

Support a pipeline factory as cmd declaration.

{
  cmds: {
    readjson: 'path $0 resolve > file $1 > json'
  }
}

Use vars as cmd refs:

{
  sources: {
    files: 'glob ./*.*'
    reader: 'detect $files'
    foo: 'file $files.0 > $reader'
  }
}

Bring smth like watchers to trigger graph re-resolution from the specified vertex

antongolub / misc

Experiment on maintaining a multi-project monorepository

<misc>

An experiment on maintaining a multi-project monorepository

Statuses

Blueprint/B marks the project as an idea w/o any implementation provided. Just a contract proposal.
PoC/C — proof of concept that shows the declared behavior in action.
Working draft/W — the project work is in progress. Some known corner cases are not covered, but it's already mostly usable.
Production ready/R — the implementation is stable, documented, tested and ready for use.
Deprecated/D — the project is no longer maintained.

Package	Description	Status
@antongolub/blank	Blank TS project
@antongolub/infra	Repo infra assets
@topoconfig/cmds	Topoconfig basic cmds preset	W
@topoconfig/extends	Flexible config extender	W
depseek	Seeks for dependency references in JS/TS code	W
depshot	Gathers deps snapshot by analyzing sources	B
envader	Occupies env vars for data storage	C
envimist	Applies minimist to process.env	W
esbuild-c	Empowers esbuild with config processing	C
esbuild-plugin-entry-chunks	Esbuild plugin to compose entryPoints as chunks	C
lcov-utils

…

View on GitHub

Refs and Inspirations

NestJS + esbuild workarounds

Anton Golub — Tue, 13 Sep 2022 20:31:07 +0000

Q: Do we need ESM for backend apps?
A: Yes. More and more pkgs are moving to the ESM format, this is inevitable.

Q: Do we need bundles?
A: Probably, yes.

If your Nestjs app is distributed as a container with node_modules inside, bundling can greatly reduce its size.

du -hs ./node_modules
495M   ./node_modules
du -hs ./output.js   
10M    ./output.js

Bundle is the best point for a thorough ISEC audit: to search for redos, suspicious API usages, protestware, etc.
The fewer fs I/O, the faster the application starts.
(TS) bundle absorbs the all required contents, so there is no tsc-esm transpilation issues

Q: esbuild, swc, rollup, parcel or babel?
A: esbuild.

Stack

:-(

const config =  {
  platform: 'node',
  target: ['node18', 'es2021'],
  format: 'esm',
  bundle: true,
}

Problems

https://github.com/nestjs/nest-cli/issues/1157
https://github.com/nestjs/swagger/issues/1450
https://github.com/evanw/esbuild/pull/509
https://github.com/evanw/esbuild/issues/566

0. Some Nestjs internals are lazy-loaded, so sometimes they may be omitted, but in another cases they should be bundled. How to handle? https://esbuild.github.io/api/#external

const config =  {
  ...
  external: [
    'mqtt',
    'amqplib',
    'class-transformer/storage'
  ]
}

1. openapi is not defined. https://github.com/nestjs/swagger/issues/1450

__decorate([
  Post('event-unsafe-batch'),
  HttpCode(200),
  openapi.ApiResponse({ status: 200, type: String }),
  __param(0, Body()),
  __param(1, Req()),
  __metadata("design:type", Function),
  __metadata("design:paramtypes", [Object, Object]),
  __metadata("design:returntype", Promise)
], EventUnsafeController.prototype, "logEventBatch", null);

2. openapi / class-validator DTOs are referenced by require API. https://github.com/microsoft/TypeScript/issues/43329

export class CspReportDto {
  static _OPENAPI_METADATA_FACTORY() {
    return { timestamp: { required: false, type: () => Object }, 'csp-report': { required: true, type: () => require("./csp.dto.js").CspReport } };
  }
}

3. NodeJS builtins are referenced via require API.

var require_async4 = __commonJS({
  "node_modules/resolve/lib/async.js"(exports, module2) {
    var fs2 = require("fs");

4. esbuild-compiled ESM bundle cannot refer to views/redoc.handlebars

const redocFilePath = path_1.default.join(__dirname, "..", "views", "redoc.handlebars");

5. _OPENAPI_METADATA_FACTORY class fields may be empty, so the swagger declaration cannot be properly rendered.

var Meta = class {
};
// →
var Meta = class {
  static _OPENAPI_METADATA_FACTORY() {
    return { appName: { required: false,  type: () =>  String }, appHost: { required: false,  type: () =>  String }, appVersion: { required: false,  type: () =>  String }, appNamespace: { required: false,  type: () =>  String }, appConfig: { required: false,  type: () =>  typeof (_a3 = typeof Record !== "undefined" && Record) === "function" ? _a3 : Object }, deviceInfo: { required: false,  type: () =>  typeof (_b3 = typeof Record !== "undefined" && Record) === "function" ? _b3 : Object }, userAgent: { required: false,  type: () =>  String }, envProfile: { required: false,  enum:  typeof (_c = typeof import_substrate2.EnvironmentProfile !== "undefined" && import_substrate2.EnvironmentProfile) === "function" ? _c : Object } }
  }
};

6. Extra type wrappers cannot be processed by openapi / class-validator / class-transformer

  __metadata("design:type", typeof (_d = typeof Array !== "undefined" && Array) === "function" ? _d : Object)
  __metadata("design:type", typeof (_e = typeof import_substrate2.LogLevel !== "undefined" && import_substrate2.LogLevel) === "function" ? _e : Object)
  // →
  __metadata("design:type", Array)
  __metadata("design:type", import_substrate2.LogLevel)

How to fix?

The right solution is certainly to improve the tools: create issues, discuss, suggest PRs.

How to fix it right here and right now?

@anatine/esbuild-decorators + old good monkey patching.
Fragile. Wrong. Terrible.

// build.js
import { build } from 'esbuild'
import path from 'node:path'
import { esbuildDecorators } from '@anatine/esbuild-decorators'

const cwd = process.cwd()
const outfile = path.resolve(cwd, 'output.js')
const tsconfig = path.resolve(cwd, 'tsconfig.json')
const entryPoints = [path.resolve(cwd, 'src/main/ts/index.ts')]
const config =  {
  platform: 'node',
  target: ['node18', 'es2021'],
  format: 'esm',
  bundle: true,
  keepNames: true,
  plugins: [
    esbuildDecorators({
      tsconfig,
      cwd
    }),
  ],
  tsconfig,
  entryPoints,
  outfile,
  external: [
    'kafkajs',
    'mqtt',
    'amqplib',
    'amqp-connection-manager',
    'nats',
    '@grpc/grpc-js',
    '@grpc/proto-loader',
    '@nestjs/websockets/socket-module',
    'class-transformer/storage'
  ]
}

await build(config)

node build.js && nestjs-esm-fix --target=./output.js

But it works.

npm-registry-firewall

Anton Golub — Wed, 13 Apr 2022 10:44:15 +0000

I couldn't google it, so I wrote my own: 

antongolub / npm-registry-firewall

npm registry proxy with on-the-fly filtering

Motivation

To mitigate security and legal risks.

Open Source is essential for modern software development. According to various estimates, at least 60% of the resulting codebase is composed of open repositories, libraries and packages. And keeps growing. Synopsys OSSRA 2021 report found that 98% of applications have open source components.

But open does not mean free. The price is the risk that you take:

Availability
Security
Legality / license

Let's consider these problems in the context of the JS universe.

Availability risks

JS packages are distributed in various ways: git repos, cdns and package registries. Regardless of the method, there are only two entry types that are finally resolved by any pkg manager: git-commit pointers and tarball links.

"dependencies": {
  "yaf" : "git://github.com/antongolub/yarn-audit-fix.git#commit-hash",
  "yaf2": "antongolub/yarn-audit-fix",
  "yarn-audit-fix" : "*"
}

yaf2@antongolub/yarn-audit-fix:
  version "9.2.1"
  resolved "https://codeload.github.com/antongolub/yarn-audit-fix/tar.gz/706646bab3b4c7209596080127d90eab9a966be2"
  dependencies:
    "@types/find-cache-dir" "^3.2.1"
    "@types/fs-extra" "^9.0.13"

"node_modules/yaf": {
  "name": "yarn-audit-fix",
  "version": "9.2.1",
  "resolved": "git+ssh://git@github.com/antongolub/yarn-audit-fix.git#706646bab3b4c7209596080127d90eab9a966be2",
  "license": "MIT",

"node_modules/yarn-audit-fix": {
  "version": "9.2.1",
  "resolved": "https://registry.npmjs.org/yarn-audit-fix/-/yarn-audit-fix-9.2.1.tgz",
  "integrity": "sha512-4biFNP4ZLOHboB2cNVuhYyelTFR/twlfmGMQ2TgJgGRORMDM/rQdQqhJdVLuKvfdMLFEPJ832z6Ws5OoCnFcfA==",
  "dependencies": {

So the implementation of mirroring is fundamentally quite simple: we just need to save and expose these assets from an alternative ssh/https entry point. Luckily this has already happened. The main repository for JS code is registry.npmjs.org. And at least 5 public replicas are always available as alternatives:

If this reliability level is not enough, you can easily run one more registry:

Security risks

Any code may not work properly. Due to error or malice. Keep in mind that most OSS licenses exclude any liability for damages. It's also important to always remember that oss code is not verified before being published.
These two circumstances sometimes give rise to dangerous incidents like colors.js or node-ipc.

The independent audit process is expensive, time consuming, so only setting a delay before using new pkg version might be effective countermeasure.

Legal risks

License agreement is an attribute of the moment: it can suddenly change and affect the development process (for example, husky-5). Uncontrolled use of new versions may have legal and financial consequences. Therefore, automated license checks should be part of CI/CD pipeline or the registry's own feature.

Key Features

Restricts access to remote packages by predicate: name, org, version (semver range), license, dateRange, username, age.
Multi-configuration: define as many port/context-path/rules combinations as you need.
Expressjs-inspired server implementation.
Has no deps. Literally zero.

Requirements

Node.js >= 14

Install

# npm
npm i npm-registry-firewall

# yarn
yarn add npm-registry-firewall

Usage

CLI

npm-registry-firewall /path/to/config.json

JS API

import {createApp} from 'npm-registry-firewall'

const app = createApp({
  server: {
    host: 'localhost',
    port: 3001,
  },
  firewall: {
    registry: 'https://registry.npmmirror.com',
    rules: [
      {
        policy: 'allow',
        org: '@qiwi'
      },
      {
        policy: 'deny',
        name: '@babel/*'
      },
    ]
  }
})

await app.start()

Config

{
  "server": {
    "host": "localhost",        // Defaults to 127.0.0.1
    "port": 3000,               // 8080 by default
    "secure": {                 // Optional. If declared serves via https
      "cert": "ssl/cert.pem",
      "key": "ssl/key.pem"
    },
    "base": "/",                // Optional. Defaults to '/'
    "healthcheck": "/health",   // Optional. Defaults to '/healthcheck'. Pass null to disable
    "keepAliveTimeout": 15000,  // Optional. Defaults 61000
    "headersTimeout": 20000,    // Optional. Defaults 62000
    "requestTimeout": 10000     // Optional. Defaults 30000
  },
  "firewall": {
    "registry": "https://registry.npmmirror.com",  // Remote registry
    "entrypoint": "https://r.qiwi.com/npm",        // Optional. Defaults to `${server.secure ? 'https' : 'http'}://${server.host}:${server.port}${route.base}`
    "base": "/",                // Optional. Defaults to '/'
    "rules": [
      {
        "policy": "allow",
        "org": "@qiwi"
      },
      {
        "policy": "allow",
        "name": ["@babel/*", "@jest/*", "lodash"] // string[] or "comma,separated,list". * works as .+ in regexp
      },
      {
        "policy": "deny",
        "name": "colors",
        "version": ">= v1.4.0"  // Any semver range: https://github.com/npm/node-semver#ranges
      },
      {
        "policy": "deny",
        "license": "dbad"       // Comma-separated license types or string[]
      },
      {
        "policy": "allow",
        "username": ["sindresorhus", "isaacs"] // Trusted npm authors.
      },
      {
        "policy": "allow",
        "name": "d",
        // `allow` is upper, so it protects `< 1.0.0`-ranged versions that might be omitted on next steps
        "version": "< 1.0.0"
      },
      {
        "policy": "deny",
        // Checks pkg version publish date against the range
        "dateRange": ["2010-01-01T00:00:00.000Z", "2025-01-01T00:00:00.000Z"]
      },
      {
        "policy": "allow",
        "age": 5    // Check the package version is older than 5 days. Like quarantine
      }
    ]
  }
}

Multi-config

// Array at the top level
[
  // Two servers (for example, http and https) share the same preset
  {
    "server": [
      {"port": 3001},
      {"port": 3002},
    ],
    "firewall": {
      "registry": "https://registry.yarnpkg.com",
      "rules": {"policy": "deny", "org": "@qiwi"}
    }
  },
  // One server has a pair of separately configured endpoints
  {
    "server": {"port": 3003},
    "firewall": [
      {"base": "/foo", "registry": "https://registry.npmjs.org", "rules": {"policy": "deny", "org": "@qiwi"}},
      {"base": "/bar", "registry": "https://registry.yarnpkg.com", "rules": {"policy": "deny", "org": "@babel"}}
    ]
  }
]

.npmrc

registry=https://localhost:3000
strict-ssl=false

npm view

npm-registry-firewall % npm view d versions                          
[ '0.1.0', '0.1.1' ]

output

$ node src/main/js/cli.js config.json
{"level":"INFO","timestamp":"2022-04-11T20:56:47.031Z","message":"npm-registry-firewall is ready for connections: https://localhost:3000"}
{"level":"INFO","timestamp":"2022-04-11T20:56:49.568Z","traceId":"44f21c050d8c6","clientIp":"127.0.0.1","message":"GET /d"}
{"level":"INFO","timestamp":"2022-04-11T20:56:50.015Z","traceId":"44f21c050d8c6","clientIp":"127.0.0.1","message":"HTTP 200 446ms"}

Contributing

Feel free to open any issues: bug reports, feature requests or questions.
You're always welcome to suggest a PR. Just fork this repo, write some code, put some tests and push your changes.
Any feedback is appreciated.

License

MIT

The missing `yarn audit fix` for Yarn 2+ Berry

Anton Golub — Sun, 12 Dec 2021 16:12:44 +0000

As everybody knows, audit --fix feature is out of scope of Yarn 1 Classic yarn/7075 and it also has not been implemented yet (2021-12-12) for Yarn 2+ berry/3582.

Let's just fix it.

tldr

npm_config_yes=true npx yarn-audit-fix@latest

1.

First of all, we need a lib to read/write yarnlock v2 files. @yarnpkg/lockfile seems the best choice, but it works with v1 only. Maybe nodejs-lockfile-parser?
Missed the mark again. It swaps checksums and does not provide dump/format API what is expected for the parser :). It turns out that we are missing yet another one lockfile processor. No problem. If we look closely, the new shiny yarn.lock v2 is a regular yaml with a little strange formatting like extra empty line delimiters, extra quotes, and so on.

import yaml from 'js-yaml'

export const parse = (raw: string): TLockfileObject => {
  const data = yaml.load(raw)
  delete data.__metadata

  return Object.entries(data).reduce<Record<string, any>>(
    (m, [key, value]: [string, any]) => {
      key.split(', ').forEach((k) => {
        m[k] = value
      })
      return m
    },
    {},
  )
}

export const format = (lockfile: TLockfileObject): string => {
  const keymap = Object.entries(lockfile).reduce<Record<string, any>>(
    (m, [k, { resolution }]) => {
      const entry = m[resolution] || (m[resolution] = [])
      entry.push(k)

      return m
    },
    {},
  )

  const data = Object.values(lockfile).reduce<Record<string, any>>(
    (m, value) => {
      const key = keymap[value.resolution].join(', ')
      m[key] = value

      return m
    },
    {
      __metadata: {
        version: 5,
        cacheKey: 8,
      },
    },
  )

  return `# This file is generated by running "yarn install" inside your project.
# Manual changes might be lost - proceed with caution!

${yaml.dump(data, {
  quotingType: '"',
  flowLevel: -1,
  lineWidth: -1,
})
  .replace(/\n([^\s"].+):\n/g, '\n"$1":\n')
  .replace(/\n(\S)/g, '\n\n$1')
  .replace(/resolution: ([^\n"]+)/g, 'resolution: "$1"')}`
}

2.

We need to fetch audit data. Yarn Berry integrated audit API is much better than previous Classic, which returned the report in the form of chunks.
yarn npm audit --all --recursive --json gives exactly what we need:

{
  "actions": [],
  "advisories": {
    "1004946": {
      "findings": [
        {
          "version": "4.1.0",
          "paths": [
            "ts-patch>strip-ansi>ansi-regex",
            "lerna>npmlog>gauge>ansi-regex",
            "lerna>@lerna/bootstrap>npmlog>gauge>ansi-regex",
            ...
          ]
        }
      ],
      "metadata": null,
      "vulnerable_versions": ">2.1.1 <5.0.1",
      "module_name": "ansi-regex",
      "severity": "moderate",
      "github_advisory_id": "GHSA-93q8-gq69-wqmw",
      "cves": [
        "CVE-2021-3807"
      ],
      "access": "public",
      "patched_versions": ">=5.0.1",
      "updated": "2021-09-23T15:45:50.000Z",
      "recommendation": "Upgrade to version 5.0.1 or later",
      "cwe": "CWE-918",
      "found_by": null,
      "deleted": null,
      "id": 1004946,
      "references": "- https://nvd.nist.gov/vuln/detail/CVE-2021-3807\n- https://github.com/chalk/ansi-regex/commit/8d1d7cdb586269882c4bdc1b7325d0c58c8f76f9\n- https://huntr.dev/bounties/5b3cf33b-ede0-4398-9974-800876dfd994\n- https://github.com/chalk/ansi-regex/issues/38#issuecomment-924086311\n- https://app.snyk.io/vuln/SNYK-JS-ANSIREGEX-1583908\n- https://github.com/chalk/ansi-regex/issues/38#issuecomment-925924774\n- https://github.com/advisories/GHSA-93q8-gq69-wqmw",
      "created": "2021-11-18T16:00:48.472Z",
      "reported_by": null,
      "title": " Inefficient Regular Expression Complexity in chalk/ansi-regex",
      "npm_advisory_id": null,
      "overview": "ansi-regex is vulnerable to Inefficient Regular Expression Complexity",
      "url": "https://github.com/advisories/GHSA-93q8-gq69-wqmw"
    },

We take only significant fields: vulnerable_versions, module_name, patched_versions

export const parseAuditReport = (data: string): TAuditReport =>
  Object.values(JSON.parse(data).advisories).reduce<TAuditReport>(
    (m, { vulnerable_versions, module_name, patched_versions }: any) => {
      m[module_name] = {
        patched_versions,
        vulnerable_versions,
        module_name,
      }
      return m
    },
    {},
  )

3.

Almost done. Now we need to replace vulnerable packages versions in lockfile with the advisories, taking into account semver-compatibility and to remove the prev checksum fields. This brilliant idea was suggested by G. Kosev. Important note Seems like dependencies of the patched entries are not reloaded automatically by Yarn, so they need to be requested from the registry by hand: yarn npm info react --fields dependencies --json.

export const patchEntry = (
  entry: TLockfileEntry,
  name: string,
  newVersion: string,
  npmBin: string,
): TLockfileEntry => {
  entry.version = newVersion
  entry.resolution = `${name}@npm:${newVersion}`

  // NOTE seems like deps are not updated by `yarn mode='--update-lockfile'`, only checksums
  entry.dependencies =
    JSON.parse(
      invoke(
        npmBin,
        ['view', `${name}@${newVersion}`, 'dependencies', '--json'],
        process.cwd(),
        true,
        false,
      ) || 'null',
    ) || undefined

  delete entry.checksum

  return entry
}

export const _patch = (
  lockfile: TLockfileObject,
  report: TAuditReport,
  { flags, bins }: TContext,
  lockfileType: TLockfileType,
): TLockfileObject => {
  if (Object.keys(report).length === 0) {
    !flags.silent && console.log('Audit check found no issues')
    return lockfile
  }

  const upgraded: string[] = []

  for (const depSpec of Object.keys(lockfile)) {
    // @babel/code-frame@^7.0.0
    // @babel/code-frame@npm:^7.0.0

    const [, pkgName, desiredRange] =
      /^(@?[^@]+)@(?:\w+:)?(.+)$/.exec(depSpec) || []

    const pkgAudit = report[pkgName]
    if (!pkgAudit) continue
    const pkgSpec = lockfile[depSpec]
    if (sv.satisfies(pkgSpec.version, pkgAudit.vulnerable_versions)) {
      const fix = sv.minVersion(pkgAudit.patched_versions)?.format()
      if (fix === undefined) {
        console.error(
          "Can't find satisfactory version for",
          pkgAudit.module_name,
          pkgAudit.patched_versions,
        )
        continue
      }
      if (!sv.satisfies(fix, desiredRange) && !flags.force) {
        console.error(
          "Can't find patched version that satisfies",
          depSpec,
          'in',
          pkgAudit.patched_versions,
        )
        continue
      }
      upgraded.push(`${pkgName}@${fix}`)

      patchEntry(pkgSpec, pkgName, fix, bins.npm)
    }
  }

4.

The final step is to update the checksums.

yarn install mode='--update-lockfile'

Refs

Actually you don’t need 'semantic-release' for semantic release

Anton Golub — Sun, 23 May 2021 12:32:10 +0000

I’m a big fan of semantic-release since it appeared. I followed its development, studied its inners. I made in-house reports, held workshops and finally brought semrel to our build infrastructure. I wrote plugins, plugin-factories and testing-tools for it. For several years now, I've been trying to combine semantic releases and monorepositories in many OSS projects:

Etc, etc, so on. I’m just trying to say, that semrel had a significant impact on my professional life.

Semrel goal

The main purpose of semantic-release is to transform semantic (conventional) commits into build artifacts and deployments. With version bumping, changelogs, tagging, pkg publishing. “Fully-automated release” — is the true. There are also dozens on plugins, so you’ll most likely find a solution for any standard case. It really saves times.

But sometimes

You may need a minor tweak up. For example, push some pkg to both public and internal registries. Ooops. "...publishing to two different registry is not a good idea". In this case you can not rely on stable, convenient and tested in millions runs semrel/npm plugin, and you have to just write a pair of commands by hand with semantic-release/exec instead:

echo "//npm-registry.domain.com/:_authToken=${TOKEN}” >> .npmrc
echo "\`jq '.name="@scope/pkg-name”’ package.json\`" > package.json
npm config set registry https://npm-registry.domain.com
npm publish --no-git-tag-version

Another instance — disabling git notes fetching. "Afraid that won't be possible".

Of course, you may fork semrel and remove the mentioned line. Or create a plugin/hook, that will override loaded execa module with patched one version, than just skips git notes invocation (this is really frustrating, I did smth similar). Or… {{ another crazy workaround goes here }}.

This is a watershed moment. Once you start to fight against the tool, it's time to just pick another one. The new dilemma:

Spend days and days for searching, tuning and testing analogs.
Write your own semantic-release.

My opinionated suggestion: if your case is very simple or, conversely, very complex, the second option will be optimal. Release script — is not a rocket science!

140 lines alternative

Let's take a look at what exactly each release consists of, if we discard the high-level tool contracts. I use zx in the examples, but it could be execa or native child_process.exec too.

1. Git configuration

To make a commit you need a committer: just name and email that will be associated with author. Also PAT or SSH token is required to push the commit.

const {GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL, GITHUB_TOKEN} = process.env
if (!GITHUB_TOKEN || !GIT_COMMITTER_NAME || !GIT_COMMITTER_EMAIL) {
  throw new Error('env.GITHUB_TOKEN, env.GIT_COMMITTER_NAME & env.GIT_COMMITTER_EMAIL must be set')
}

const gitAuth = `${GIT_COMMITTER_NAME}:${GITHUB_TOKEN}`
const originUrl = (await $`git config --get remote.origin.url`).toString().trim()
const [,,repoHost, repoName] = originUrl.replace(':', '/').replace(/\.git/, '').match(/.+(@|\/\/)([^/]+)\/(.+)$/)
const repoPublicUrl = `https://${repoHost}/${repoName}`
const repoAuthedUrl = `https://${gitAuth}@${repoHost}/${repoName}`
await $`git config user.name ${GIT_COMMITTER_NAME}`
await $`git config user.email ${GIT_COMMITTER_EMAIL}`
await $`git remote set-url origin ${repoAuthedUrl}`

2. Commit analysis

Conventional commits are just a prefixed strings in git log. We should define some rules on how to associate messages substrings with corresponding release types:

const semanticTagPattern = /^(v?)(\d+)\.(\d+)\.(\d+)$/
const releaseSeverityOrder = ['major', 'minor', 'patch']
const semanticRules = [
{group: 'Features', releaseType: 'minor', prefixes: ['feat']},
{group: 'Fixes & improvements', releaseType: 'patch', prefixes: ['fix', 'perf', 'refactor', 'docs']},
{group: 'BREAKING CHANGES', releaseType: 'major', keywords: ['BREAKING CHANGE', 'BREAKING CHANGES']},
]

Then we search for the prev release tag, that satisfies semver pattern:

const tags = (await $`git tag -l --sort=-v:refname`).toString().split('\n').map(tag => tag.trim())
const lastTag = tags.find(tag => semanticTagPattern.test(tag))

And make commits cut from the found ref:

const newCommits = (lastTag
  ? await $`git log --format=+++%s__%b__%h__%H ${await $`git rev-list -1 ${lastTag}`}..HEAD`
  : await $`git log --format=+++%s__%b__%h__%H HEAD`)
  .toString()
  .split('+++')
  .filter(Boolean)
  .map(msg => {
    const [subj, body, short, hash] = msg.split('__').map(raw => raw.trim())
    return {subj, body, short, hash}
  })

Now we just need to parse them:

const semanticChanges = newCommits.reduce((acc, {subj, body, short, hash}) => {
  semanticRules.forEach(({group, releaseType, prefixes, keywords}) => {
    const prefixMatcher = prefixes && new RegExp(`^(${prefixes.join('|')})(\\(\\w+\\))?:\\s.+$`)
    const keywordsMatcher = keywords && new RegExp(`(${keywords.join('|')}):\\s(.+)`)
    const change = subj.match(prefixMatcher)?.[0] || body.match(keywordsMatcher)?.[2]

    if (change) {
      acc.push({
        group,
        releaseType,
        change,
        subj,
        body,
        short,
        hash
      })
    }
  })
  return acc
}, [])

Ta-da. Semantic changes:

semanticChanges= [
  {
    group: 'Fixes & improvements',
    releaseType: 'patch',
    change: 'perf: use git for tags sorting',
    subj: 'perf: use git for tags sorting',
    body: '',
    short: 'a1abdae',
    hash: 'a1abdaea801824d0392e69f9182daf4d5f4b97db'
  },
  {
    group: 'Fixes & improvements',
    releaseType: 'patch',
    change: 'refactor: minor simplifications',
    subj: 'refactor: minor simplifications',
    body: '',
    short: 'be847a2',
    hash: 'be847a26e2b0583e889403ec00db45f9f9555e30'
  },
  {
    group: 'Fixes & improvements',
    releaseType: 'patch',
    change: 'fix: fix commit url template',
    subj: 'fix: fix commit url template',
    body: '',
    short: '3669edd',
    hash: '3669edd7eb440e29dc0fcf493c76fbfc04271023'
  }
]

3. Resolve next version:

const nextReleaseType = releaseSeverityOrder.find(type => semanticChanges.find(({releaseType}) => type === releaseType))
if (!nextReleaseType) {
  console.log('No semantic changes - no semantic release.')
  return
}
const nextVersion = ((lastTag, releaseType) => {
  if (!releaseType) {
    return
  }
  if (!lastTag) {
    return '1.0.0'
  }

  const [, , c1, c2, c3] = semanticTagPattern.exec(lastTag)
  if (releaseType === 'major') {
    return `${-~c1}.0.0`
  }
  if (releaseType === 'minor') {
    return `${c1}.${-~c2}.0`
  }
  if (releaseType === 'patch') {
    return `${c1}.${c2}.${-~c3}`
  }
})(lastTag, nextReleaseType)

const nextTag = 'v' + nextVersion

4. Generate release notes

const releaseDiffRef = `## [${nextVersion}](${repoPublicUrl}/compare/${lastTag}...${nextTag}) (${new Date().toISOString().slice(0, 10)})`
const releaseDetails = Object.values(semanticChanges
.reduce((acc, {group, change, short, hash}) => {
const {commits} = acc[group] || (acc[group] = {commits: [], group})
const commitRef = `* ${change} ([${short}](${repoPublicUrl}/commit/${hash}))`

      commits.push(commitRef)

      return acc
    }, {}))
    .map(({group, commits}) => `
### ${group}
${commits.join('\n')}`).join('\n')

const releaseNotes = releaseDiffRef + '\n' + releaseDetails + '\n'

5. Update CHANGELOG.md

Attach releaseNotes to file. Just one string.

await $`echo ${releaseNotes}"\n$(cat ./CHANGELOG.md)" > ./CHANGELOG.md`

6. Update package version

await $`npm --no-git-tag-version version ${nextVersion}`

7. Git release.

Create commit. Create tag. Push them.

const releaseMessage = `chore(release): ${nextVersion} [skip ci]`
await $`git add -A .`
await $`git commit -am ${releaseMessage}`
await $`git tag -a ${nextTag} HEAD -m ${releaseMessage}`
await $`git push --follow-tags origin HEAD:refs/heads/master`

8. GitHub release

Just a single curl POST to gh rest api.

const releaseData = JSON.stringify({
  name: nextTag,
  tag_name: nextTag,
  body: releaseNotes
})
await $`curl -u ${GIT_COMMITTER_NAME}:${GITHUB_TOKEN} -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/${repoName}/releases -d ${releaseData}`

9. Npm publish

await $`npm publish --no-git-tag-version`

Need several registries? NP.

await $`npm config set registry https://registry.npmjs.org`
await $`npm publish --no-git-tag-version`
await $`echo "\`jq '.name="@${repoName}"' package.json\`" > package.json`
await $`npm config set registry https://npm.pkg.github.com`
await $`npm publish --no-git-tag-version`

Conclusions

This solution does not cover corner cases and has significant limitations of usage. Ultimately, you don't care if other tools have 99.99999% applicability until they ignore just one specific case — yours. But now you have completely taken back release flow control. You're able to improve and modify this snippet as you wish and whenever you like.

release.mjs
gh release.yaml
release log

semrel-extra / zx-semrel

`zx`-based release script as `semantic-release` alternative (PoC)

zx-semrel

zx -based release script as semantic-release alternative (PoC)

Sometimes bloody enterprise enforces you not to use any third-party solutions for sensitive operations (like release, deploy, so on) Old good script copy-paste hurries to the rescue!

Btw, here's an adaptation for monorepos: zx-bulk-release

Requirements

macOS / linux
Node.js >= 14.13.1
git >= 2.0
zx >= 1.6.0

Key features

Zero dependencies
Zero configuration
Pretty fast
Tiny, less than 140 lines with comments
Reliability, safety, simplicity and maintainability (sarcasm)

Functionality

Poor conventional commits analysis
CHANGELOG.md generation
package.json version bumping
Git release commit creation
GitHub Release
Package publishing to both npmjs and gh registries

🚀 Usage

Copy
Tweak up, inject tokens, etc
Run

curl https://raw.githubusercontent.com/semrel-extra/zx-semrel/master/release.mjs > ./release.mjs
zx ./release.mjs

or this like if zx is not installed:

# Just replace GIT* env values with your own
GIT_COMMITTER_NAME=antongolub GIT_COMMITER_EMAIL=mailbox@antongolub.ru GITHUB_TOKEN=token npx zx ./release.mjs

or just run it without any edits though npx:

…

View on GitHub

TS and ts-jest meet “type”: “module”

Anton Golub — Tue, 11 May 2021 15:02:57 +0000

To save anybody’s googling time

ES modules are becoming more widespread pkg format. So dependency updates break our es5-builds more often. This is expected. This is inevitable. This is the price of progress.

Trouble #1

Typescript code can be easily compiled into the latest versions of javascript. Almost. Imagine code snippet:

import {generate} from './license'

and tsconfig.json

{
  "compilerOptions": {
    "module": "es2020",
    "outDir": "target/es6"
  }
}

gives:

import { generate } from './license'; // ; ← is the diff

Everything seems ok until ”type”: “module” is not added to package.json:

Error [ERR_MODULE_NOT_FOUND]: Cannot find module ‘~/projects/license/target/es5/license' imported from /~/projects/license/target/es5/cli.js
    Did you mean to import ../../../license?
        at finalizeResolution (internal/modules/esm/resolve.js:276:11)
        at moduleResolve (internal/modules/esm/resolve.js:699:10)
        at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:810:11)
        at Loader.resolve (internal/modules/esm/loader.js:86:40)
        at Loader.getModuleJob (internal/modules/esm/loader.js:230:28)

TypeScript/issues/13422: TS should add .js extensions for rel paths as required by ECMA standard. But is doesn’t yet. Well, the dirty fix may be found in issue comments. 
find www/js -type f -name '*.js' -print0 | xargs -0 sed -i '' -E 's/from "([^"]+)";$/from "\1.js";/g' 

Restrictions: 

String replacer can not properly handle ‘./module’ and ./module/index loading cases.
Does not handle dynamic imports like import(‘./foo’).then(…)
sed -i '' -E works on Mac only, Linux sed should use just sed -i -e. And you need to handle this in your build script:   bash  if [[ "$OSTYPE" == "darwin"* ]]; then … else … fi    or maybe run perl instead: perl -pi -e.

Trouble #2

__dirname. And __filename too.

ReferenceError: __dirname is not defined
        at loadTemplate (file:///~/projects/license/target/es5/license.js:1:651)
        at render (file:///~/projects/license/target/es5/license.js:1:535)
        at generate (file:///~/projects/license/target/es5/license.js:1:800)
        at file:///~/projects/license/target/es5/cli.js:2:692
        at ModuleJob.run (internal/modules/esm/module_job.js:152:23)
        at async Loader.import (internal/modules/esm/loader.js:166:24)
        at async Object.loadESM (internal/process/esm_loader.js:68:5)

Now we have to use import.meta as the official NodeJS documentation says:

import { dirname } from 'path'
import { fileURLToPath } from 'url'

const __dirname = dirname(fileURLToPath(import.meta.url))

Unfortunately, ts-jest does not support this API now: ts-jest/issues/1174. Therefore, we have to keep __dirname/__filename in TS sources and perform the replacement in the bundles. Behold the glory of regex inside the regex replacer with escaped backslash escapes: 
 "build:fix-module-dirname": "find target/es5 ./target/es6 -type f -name '*.js' -print0 | xargs -0 perl -pi -e \"s/__dirname/\\/file:\\\\\\\\\\\\/\\\\\\\\\\\\/(.+)\\\\\\\\\\\\/\\[^\\/\\]\\/.exec(import.meta.url)[1]/g\"".
This piece of code just replaces all __dirname occurrences with /file:\/\/(.+)\/[^/]/.exec(import.meta.url)[1].

Restrictions:

Poor readability
Requires sed / perl

Fix 

Here’s an attempt to solve mentioned issues in a more convenient and maintainable form — as js util.  

antongolub / tsc-esm-fix

Make Typescript projects compatible with esm/mjs requirements

Features

Finds and replaces __dirname and __filename refs with import.meta.
Injects extentions to relative imports/re-exports statements.
- import {foo} from './foo' → import {foo} from './foo.js'
- Pays attention to index files: import {bar} from './bar' → import {bar} from './bar/index.js'
Follows outDir found in tsconfig.json.
Changes files extentions if specified by opts.
Supports Windows-based runtimes.

Install

yarn add tsc-esm-fix -D

CLI

tsc-es2020-fix [opts]

Option	Description	Default
`--tsconfig`	Path to project's ts-config(s)	`tsconfig.json`
`--target`	Entry points where compiled files are placed for modification	If not specified inherited from tsconfig.json compilerOptions.outDir
`--dirnameVar`	Replace `__dirname` usages with `import.meta`	true
`--filenameVar`	Replace `__filename` var references `import.meta`	true
`--ext`	Append extension to relative imports/re-exports	`.js`
`--cwd`	cwd	`process.cwd()`
`--out`	Output dir. Defaults to cwd, so files will be overridden

JS/TS

import { fix, IFixOptions } from 'tsc-esm-fix'

const fixOptions: IFixOptions = {
  tsconfig: 'tsconfig.build.json',
  dirnameVar: true,
  filenameVar: true,
  ext: true
}

await fix(fixOptions)

export interface IFixOptions {
  cwd: string
  out?: string,
  target?: string | string[]
  tsconfig: string | string[]
  dirnameVar: boolean
  filenameVar: boolean
  ext: boolean | string
}

UPD (2021-08-15) Alternatives

Refs

ERR_REQUIRE_ESM

Anton Golub — Wed, 31 Mar 2021 15:49:37 +0000

To save anybody's googling time

Once you update you package deps you may face with the error mentioned in this note title:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: ~/projects/semantic-release-toolkit/node_modules/zz/cjs.js
require() of ES modules is not supported.
require() of ~/projects/semantic-release-toolkit/node_modules/zz/cjs.js from ~/projects/semantic-release-toolkit/tt.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename cjs.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from ~/projects/semantic-release-toolkit/node_modules/zz/package.json.

Even though your code stopped working, this issue has positive context: the future is coming and some of your project's deps has been shipped as ES module.

What's wrong?

The short answer — es module brings top-level await, but require has no idea how to deal with it. Detailed explanation — announcing a new --experimental-modules.

How to fix?

There're several strategies:

1. Follow the trend

Migrate to ES modules. Replace require with import.

// before
const { sync } = require('read-pkg-up')
// after
import { readPackageUpSync } from 'read-pkg-up'

Publish as esm. Add a special instruction to package.json

{
  "type": "module"
}

2. Stay on ES5

Wrap ES6 module(s) with esm adapter. Just like the documentation says:

// read-pkg-up-wrapper.js
require = require('esm')(module, {
  mode: 'all',
  cjs: {
    ... //
  },
  force: false,
  cache: false,
  await: false
})
module.exports = require('read-pkg-up')

NOTE It's advisable to use patched esm that properly handles legacy (.js) extensions. See esm/issues/868 for details.

3. Take both.

Provide conditional export.
You're able to split your code into esm and cjs versions.

// package.json
{
   ...
  "exports": {
    "import": "./es6entry.mjs",
    "require": "./es5entry.cjs"
  }
}

NOTE .cjs/.mjs file extension has matter.

This approach requires that your code to be refactored in some way:

Rewrite by hand with require and import syntax.
Recompile (TypeScript) separately with CommonJS and ES6 module options.
Wrap one version inside the other.
- es6 in es5 (std/esm) as mentioned above
- reexport es5 from es6

import cjsModule from './index.cjs'
export const name = cjsModule.name

Conditional export is especially important for a situation when you're developing a package that needs ES6 deps, but its called by an external utility that uses legacy ES5 loading. For example, some-semrel-plugin depends from read-pkg-up@8 (es6), and is being invoked through require directive in semantic-release.

Related refs

Yarn audit fix: workaround

Anton Golub — Fri, 10 Jul 2020 17:32:11 +0000

tl;dr

npx yarn-audit-fix

Issue

Auditing and updating dependencies is an important part of the code lifecycle. There are different approaches to implement this task: services (renovate, snyk, dependabot and so on) and instruments (npm, yarn).

If you develop proprietary things in an isolated environment, external services are most likely unavailable. If you use monorepos, yarn is your only choice.

But there is a problem.

yarn audit fix

...does not quite what is expected. It just shows vulnerabilities, but doesn't fix them. And here is a long discussion of why this happens: https://github.com/yarnpkg/yarn/issues/7075

Most workarounds are based on different variations of semi-manual dependency patching.

{
  "resolutions": {
    "package-merge-lodash-4/*/lodash": "4.17.12"
  },
}
...
"scripts": {
  "preinstall": "npx npm-force-resolutions"
}

Fortunately, Gianfranco P. proposed another solution:

1. npm i --package-lock-only
2. npm audit fix
3. rm yarn.lock
4. yarn import
5. rm package-lock.json

It's simple, clear and it works. But this script ignores the case of monorepos, because npm doesn't support workspaces yet.

There must be another way to generate lock file. It exists! Synp converts yarn.lock to package-lock.json and vice versa. So

synp --source-file yarn.lock

gives package-lock.json, ~~that contains all the dependencies~~ (~~nope, but it will be fixed asap~~. Fixed) of the monorepository. Next npm audit fix works normally.
However, yarn import raises an error.

error An unexpected error occurred: "https://registry.yarnpkg.com/workspace-aggregator-bbb13589-b149-4858-b202-4f4e90c33e3f: Not found".
info If you think this is a bug, please open a bug report with the information provided in "/<root>/projects/foobar/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/import for documentation about this command.

One more side-effect of workspace field in package.json. Not a problem. It disappears if you temporarily remove/rename the field before synp invocation.

Performing these steps manually each time is slightly tedious. Let's automate.

import fs from 'fs-extra'
import synp from 'synp'
import {join} from 'path'
import findCacheDir from 'find-cache-dir'
import chalk from 'chalk'
import {invoke} from './invoke'

type TContext = { cwd: string, temp: string }

type TCallback = (cxt: TContext) => void

type TStage = [string, ...TCallback[]]

/**
 * Prepare temp assets.
 * @param {TContext} cxt
 * @return {void}
 */
const createTempAssets: TCallback = ({temp}) => {
  fs.copyFileSync('yarn.lock', join(temp, 'yarn.lock'))
  fs.copyFileSync('package.json', join(temp, 'package.json'))
  fs.createSymlinkSync('node_modules', join(temp, 'node_modules'), 'dir')
}

/**
 * Remove workspace field from package.json due to npm issue.
 * https://github.com/antongolub/yarn-audit-fix/issues/2
 * @param {TContext} cxt
 * @return {void}
 */
const fixWorkspaces: TCallback = ({temp}) => {
  const pkgJsonData = JSON.parse(fs.readFileSync(join(temp, 'package.json'), 'utf-8').trim())
  delete pkgJsonData.workspaces

  fs.writeFileSync(join(temp, 'package.json'), JSON.stringify(pkgJsonData, null, 2))
}

/**
 * Convert yarn.lock to package.json for further audit.
 * @param {TContext} cxt
 * @return {void}
 */
const yarnLockToPkgLock: TCallback = ({temp}) => {
  const pgkLockJsonData = synp.yarnToNpm(temp)

  fs.writeFileSync(join(temp, 'package-lock.json'), pgkLockJsonData)
  fs.removeSync(join(temp, 'yarn.lock'))
}

/**
 * Apply npm audit fix.
 * @param {TContext} cxt
 * @return {void}
 */
const npmAuditFix: TCallback = ({temp}) =>
  invoke('npm', ['audit', 'fix', '--package-lock-only'], temp)

/**
 * Generate yarn.lock by package-lock.json data.
 * @param {TContext} cxt
 * @return {void}
 */
const yarnImport: TCallback = ({temp}) => {
  invoke('yarn', ['import'], temp)
  fs.copyFileSync(join(temp, 'yarn.lock'), 'yarn.lock')
}

/**
 * Apply yarn install to fetch packages after yarn.lock update.
 * @param {TContext} cxt
 * @return {void}
 */
const yarnInstall: TCallback = ({cwd}) =>
  invoke('yarn', [], cwd)

/**
 * Clean up temporaries.
 * @param {TContext} cxt
 * @return {void}
 */
const clear: TCallback = ({temp}) =>
  fs.emptyDirSync(temp)

export const stages: TStage[] = [
  [
    'Preparing temp assets...',
    clear,
    createTempAssets,
    fixWorkspaces,
  ],
  [
    'Generating package-lock.json from yarn.lock...',
    yarnLockToPkgLock,
  ],
  [
    'Applying npm audit fix...',
    npmAuditFix,
  ],
  [
    'Updating yarn.lock from package-lock.json...',
    yarnImport,
    yarnInstall,
    clear,
  ],
  [
    'Done',
  ],
]

/**
 * Public static void main.
 */
export const run = async() => {
  const ctx = {
    cwd: process.cwd(),
    temp: findCacheDir({name: 'yarn-audit-fix', create: true}) + '',
  }

  for (const [description, ...steps] of stages) {
    console.log(chalk.bold(description))

    for (const step of steps) step(ctx)
  }
}

And finally it looks to be working.

antongolub / yarn-audit-fix

The missing `yarn audit fix`

Any feedback is welcome.

Forem: Anton Golub

CVE-2024-29415: problem solution

Algorithm

Compatibility

Drop-in

Next steps

zx@lite: minimalistic shell scripting with TS/JS superpowers

⚡ What is zx@lite?

🧾 Setup

🧪 Examples

⚠️ What’s missing in zx@lite?

📦 Upgrade path

📚 References

🧠 Final thoughts

How and why do we bundle zx?

Short history

Packed as CommonJS

Dropped CJS support

Migrated to TypeScript

Problems

Size

Compatibility

Stability & Security

Bundle

js

d.ts

Results

google / zx

A tool for writing better scripts

zx

Install

Documentation

License

config: apply extends

Proposal

20 minutes adventure

Conclusion

config: enhancing declarations with graphs

Config mess

Budget loss

What we need

Working draft

Install

API

Notes

Next steps

antongolub / misc

Experiment on maintaining a multi-project monorepository

<misc>

Statuses

Contents

Refs and Inspirations

NestJS + esbuild workarounds

Stack

:-(

Problems

How to fix?

How to fix it right here and right now?

npm-registry-firewall

antongolub / npm-registry-firewall

npm registry proxy with on-the-fly filtering

Motivation

Availability risks

Security risks

Legal risks

Key Features

Requirements

Install

Usage

CLI

JS API

Config

Multi-config

Contributing

License

The missing `yarn audit fix` for Yarn 2+ Berry

tldr

1.

2.

3.

⚡ What is `zx@lite`?

⚠️ What’s missing in `zx@lite`?

Fix 