Iterators and Generators - playing the prototpe

ZaneHannanAU — Mon, 03 Jun 2019 23:24:53 +0000

Playing the Prototype

Well, may as well.

Iterators and Generators are not new concepts. Nor is modifying the prototype. But rarely are both done.

So, let's get to it!

I'm using typescript here; but the way it's written it'd be a better shim in js.

let prot = ((function*(){}).prototype as unknown as any).__proto__ as any

It's ugly, but it gives us the global Generator prototype. Asserting this by other means can be annoying.

for (let fn of gens) Object.defineProperty(prot, fn.name, {value: fn})

… Yeah, using generators to generate generators. As if it were that easy…

let gens = [
  function* map<U, T>(this: IterableIterator<U>, cb: (value: U, iteration: number) => T): IterableIterator<T> {
    let iteration = 0
    for (const value of this) yield cb(value, iteration++)
  },
  function* take<T>(this: IterableIterator<T>, num: number): IterableIterator<T> {
    if (num < 1) return;
    for (const t of this) {
      yield t;
      if (--num < 1) break
    }
  },
  function* takeWhile<T>(this: IterableIterator<T>, condition: (value: T, iteration: number) => boolean): IterableIterator<T> {
    let iteration = 0
    for (const t of this) {
      if (condition(t, iteration++)) yield t
      else break
    }
  },
  function* filter<U, T = Exclude<"" | 0 | null | undefined, U>>(this: IterableIterator<U>, cb: (value: U, iteration: number) => boolean = v => !!v): IterableIterator<T> {
    let iteration = 0
    for (const value of this) if (cb(value, iteration++)) yield value as unknown as T
  },
  function fold<U, T>(this: IterableIterator<U>, init: T, cb: (acc: T, curr: U, iteration: number) => T) {
    let iteration = 0
    for (const c of this) init = cb(init, c, iteration++)
    return init
  },
  // Iterates until both are complete.
  function* zip<A, B>(this: IterableIterator<A>, other: IterableIterator<B>): IterableIterator<[A, B]> {
    try {
      let a = this.next(), b = other.next()
      while (!(a.done || b.done)) {
        yield [a.value, b.value]
        a = this.next()
        b = this.next()
      }
    } finally {
      this.return()
      other.return()
    }
  },
  function* enumerate<T>(this: IterableIterator<T>): IterableIterator<[number, T]> {
    let iteration = 0
    for (const t of this) yield [iteration++, t]
  },
  function unzip<A, B>(this: IterableIterator<[A | undefined, B | undefined]>): [A[], B[]] {
    let a = [], b = []
    for (const [aa, bb] of this) {
      if (aa !== undefined) a.push(aa);
      if (bb !== undefined) b.push(bb)
    }
    return [a, b]
  },
  function collect<T>(this: IterableIterator<T>): T[] {
    return Array.from(this)
  }
]

All of these are based on std::iter::Iterator from rust (13.2 MiB), and all use only the native js functions. Now you can write your infinite loop and have it go lazily … I guess.

function* idgen(i = 0) {while(1) yield i++}
for (let value of idgen().filter(v => v % 3 == 0).map(v => v * v).take(32)) console.log(value)
0
9
36
81
144
225
324
441
576
729
900
1089
1296
1521
1764
2025
2304
2601
2916
3249
3600
3969
4356
4761
5184
5625
6084
6561
7056
7569
8100
8649

bash - lazy completion evaluation.

ZaneHannanAU — Fri, 31 May 2019 10:08:04 +0000

lazy completion evaluation in bash

tl;dr: loading completions lazily can help with terminal init speed, reduce memory usage (if only slightly), and use expensive functions (eg generators) once only.

lazy evaluation comes with a lot of positives-- less memory, less parsing, less evaluation, and you only pay for what you use. So, what can we do with it in bash?

well, here's a stupid (well, requires some jumps) means of setting it up:

function __lazyfunc {
  if [[ -z "$(type -t $1)" || $LAZYFUNC_FORCE_REDEFINE ]]; then
    local fn="$1"
    shift
    eval "$fn () { unset -f $fn ; eval \"\$( $@ )\" ; $fn \$@ ; }"
  fi
}

so what does it do?

only define if not declared. If you rerun . .bashrc on this, it doesn't override the values.
(locally) take the first argument, shift it off the front
evaluate a string defining a function

that's it

of course, there are some downsides--

most declarations are not intended to be done like this
it takes some editing to rewrite the available source

for example, with this, rather than just

eval "$(pandoc --bash-completion)"

it now takes

__lazyfunc _pandoc pandoc --bash-completion

and it gets worse for others. eg, nvm:

export NVM_DIR="$HOME/.nvm"
__lazyfunc __nvm 'cat $NVM_DIR/bash_completion'
__lazyfunc nvm 'cat $NVM_DIR/nvm.sh'
complete -o default -F __nvm nvm

cargo and rustup?:

__lazyfunc _rustup rustup completions bash rustup
__lazyfunc _cargo 'cat $(rustc --print sysroot)/etc/bash_completion.d/cargo'
complete -F _rustup -o bashdefault -o default rustup
complete -F _cargo cargo

in use

using the above settings, with a few moments of loading (and adding time before eval call),

$ nvm a
real    0m0.007s
user    0m0.006s
sys 0m0.001s
lias ^C
$ cargo a
real    0m0.142s
user    0m0.089s
sys 0m0.042s
dd ^C
$ rustup 
real    0m0.023s
user    0m0.012s
sys 0m0.011s

completions     dump-testament  man             show            -v
component       -h              override        target          -V
default         --help          run             toolchain       --verbose
doc             help            self            uninstall       --version
docs            install         set             update          which
$ rustup ^C
$ pandoc 
real    0m1.221s
user    0m0.302s
sys 0m0.202s

$ pandoc
$ npm 
real    0m0.954s
user    0m0.535s
sys 0m0.351s
v12.2.0

real    0m2.186s
user    0m1.153s
sys 0m1.058s
$ node

real    0m0.993s
user    0m0.524s
sys 0m0.404s
Welcome to Node.js v12.2.0.
Type ".help" for more information.
>

(node and npm are loaded through nvm and have their own macros, so this takes a bit longer than usual)

function node {
        unset -f node npm
        local which=`nvm which | tail -n 1`
        NODE_ICU_DATA="$(dirname "$(dirname "$which")")/lib/node_modules/full-icu"
        PATH="$PATH:$(dirname "$which")"
        $which $@
}
function npm {
        >&2 node -v
        npm $@
}
__lazyfunc _npm_completion npm completion
complete -F _npm_completion npm

now, what good is it?

on a computer with slow disk access, this greatly decreases time between starting and having access to a terminal
on a phone (termux), this also decreases time between startup

of course, there are some downsides

initiation of a function can add time to initial completion
this initial setup can be annoying

now, I doubt very much that I'm the first to post about it; but it is worth the time.

investigation

as you can guess, this means it hot-swaps the functions. so, what's the difference?

$ type _cargo
_cargo is a function

_cargo () 
{ 
    unset -f _cargo;
    eval "$( cat $(rustc --print sysroot)/etc/bash_completion.d/cargo )";
    _cargo $@
}

$ type __nvm
__nvm is a function

__nvm () 
{ 
    unset -f __nvm;
    eval "$( cat $NVM_DIR/bash_completion )";
    __nvm $@
}

$ nvm alias ^C
$ type __nvm
__nvm is a function