Forem: Ash Wu

Use non-root user in scratch docker image

Ash Wu — Fri, 07 Jun 2024 02:00:05 +0000

It's considered best practice to use non-root user in docker images, even if it's built from scratch image.

But in scratch image it's really empty, you can't use commands like useradd to create a non-root user.

We can use multi stage builders to achieve this.

FROM ubuntu:latest
RUN useradd -u 10001 scratchuser
FROM scratch
COPY dosomething /dosomething
COPY --from=0 /etc/passwd /etc/passwd
USER scratchuser
ENTRYPOINT ["/dosomething"]

How can we verify it? In order to verify, we need id command to check if the user is set correctly. We can copy the commands from busybox.

FROM busybox:1.35.0-uclibc as busybox

COPY --from=busybox /bin/sh /bin/sh
COPY --from=busybox /bin/id /bin/id

And now we can use docker exec to run the id command to verify if it works.

Kamal Deploy on GCP

Ash Wu — Sat, 17 Feb 2024 04:48:56 +0000

Kamal (https://kamal-deploy.org/) serves as the Docker equivalent of Capistrano, presenting a familiar interface to those already acquainted with Capistrano.

In this article, I'll share insights gained from deploying a web application on Google Cloud Platform (GCP) using Kamal. Rather than offering a one-size-fits-all guide, I aim to provide a collection of useful snippets and references to facilitate your project's deployment.

Utilizing GCP's Artifact Registry

The integration of Kamal with GCP's Artifact Registry is streamlined by an ongoing PR (https://github.com/basecamp/kamal-site/pull/35). For seamless operation, configure the following in your Kamal settings, ensuring to replace the placeholders with your specific project details and incorporating the service account's JSON key.

image: <your gcp project id>/<artifact registry repo name>/<desired image name>
registry:
  server: <your registry region>-docker.pkg.dev
  username: _json_key_base64
  password:
    - KAMAL_REGISTRY_PASSWORD

Secure SSH Access via IAP

To securely SSH into GCP's Compute Engine VMs, the use of Identity-Aware Proxy (IAP) is advocated. Before proceeding, verify your ability to SSH via IAP by following GCP's official guide (https://cloud.google.com/compute/docs/connect/ssh-using-iap).

In the Kamal configuration file, define your server host as shown below:

servers:
  - "myhost.us-west1-a.my-gcp-project"

Setting up Proxy Command

For a smooth SSH connection, download and implement this script (https://gist.github.com/hSATAC/d72bd174f8845d8b9995f8921fe13b39) as your proxy_command. This script, compatible with both macOS and Linux, facilitates usage across various environments including CI runners.

Locate the script within the project at ./.kamal/scripts/:

ssh:
  proxy_command: sh ./.kamal/scripts/gcp-start-iap-tunnel-ssh-proxy-magic.sh gce_instance=%h sshuser=root

Enabling Root SSH Access

Though Kamal permits SSH access under any username, employing the root user simplifies processes, adhering to Kamal's default assumptions.

Enable root SSH access as per GCP's guidance (https://cloud.google.com/compute/docs/connect/root-ssh#gcloud), and in your SSH configuration (~/.ssh/config), include a reference to Google's Compute Engine private key:

Host *.my-gcp-project
  IdentityFile ~/.ssh/google_compute_engine

HowTo: Find egress traffic destination in Istio service mesh

Ash Wu — Fri, 18 Nov 2022 05:53:28 +0000

Before we enable the REGISTRY_ONLY options for Istio, we want to capture all the existing outbound traffic and add it as a ServiceEntry.

There are two metrics we can monitor:

istio_requests_total{destination_service_name="PassthroughCluster"}
istio_tcp_connections_closed_total{destination_service_name="PassthroughCluster"}

We can query these two metrics in prometheus to see if there’s any outbound traffic that was not registered yet.

The first one istio_requests_total is easy. It captures all the http requests. And because it’s a http request, the domain will be recorded in the destination_service field.

The second one istio_tcp_connections_closed_total is more complicated. It may be an HTTPS or TCP request. And for these requests the destination domain was not recorded. The only thing we can know is which workload generated these requests.

Envoy Debug Log

To find out where the request is going, we must first turn on the debug log of the sidecar proxy. There are two ways we can do that.

The first one is to add an annotation to the workload: sidecar.istio.io/logLevel: debug

If you don’t want to restart the pod, you can also use istioctl to enable debug log during runtime:

istioctl pc log <pod_name>.<namespace> --level debug

After the debug log was enabled, we then needed to trigger the application to make it send those requests. And then we can try to find PassthroughCluster in the logs.

k logs -n <namespace> <pod_name> -c istio-proxy | grep "Creating connection to cluster PassthroughCluster" -B2 -A2

After you enabled the REGISTRY_ONLY mode of the Istio service mesh, there will be no PassthroughCluster, instead, you should be monitoring BlackholeCluster.

Envoy Access Log

Another tool we can use is the access log of envoy proxy. We can enable the access log for specified namespace and workload. Here’s an example of enabling envoy access log for monitoring namespace.

apiVersion: telemetry.istio.io/v1alpha1
kind: Telemetry
metadata:
  name: envoy-access-log
  namespace: monitoring
spec:
  accessLogging:
    - providers:
      - name: envoy

After that, we can tail the log and pipe to engarde to view the logs.

k logs -n <namespace> <pod_name> -c istio-proxy -f | engarde --use-istio | jq 'select(.upstream_cluster=="BlackHoleCluster")'

Write a PHP library supports PHP 5.6 to 8.1

Ash Wu — Mon, 24 Oct 2022 04:02:18 +0000

Yes, you heard it right. We need to write a PHP library that supports from PHP 5.6 to PHP 8.1

There are still people out there using EOL PHP versions and we still want them to be able use our package.

Things that needs to be done needs to be done. Let's get to it.

The Code

Actually writing the code itself is probably the easiest part. You just don't use the fancy new stuff and stick to the old way.

I run two tools for this - php-cs-fixer on PHP 5.6 and the built-in syntax checker(php -l).

There's an existing Github Action prestashop/github-action-php-lint which handles this for you.

A full example of these two checks can be found here

The Tests

Writing the code is easy, but writing the tests is not. PHPUnit does not support old PHP versions and you can't use the latest PHP version with old PHPUnit version due to syntax issues.

Yaost/PHPUnit-Polyfills came to rescue. With just tiny adjustments to your tests, this can make your tests run across PHPUnit 4.8~9.x and PHP 5.4+

The CI

We also want to run tests for different versions automatically so we don't have to manually tests everything.

The official PHPUnit & composer Github Action didn't play well with old PHP versions.

I found shivammathur/setup-php did a better job on supporting old PHP versions, and you don't have to worry about the composer & extension compatibility, it handles those for you.

Here's a full example that you can run a matrix of PHP versions for your tests.

Wrap Up

Writing a PHP library that supports from PHP 5.6 to PHP 8.1 is not as hard as I imagine at the very beginning. PHP handles backward-compatibility very well. PHPUnit took me most of the time to figure out. Hopefully there're already a tool for that. I am not alone.

Hope this helps someone out there, trying to do the same thing.

Building A Ubuntu VirtualBox Image on AWS EC2 With Packer

Ash Wu — Fri, 16 Apr 2021 02:38:30 +0000

Building a virtualbox image on AWS EC2 with packer - this sounds straightforward, how hard could it be?

Indeed, it's relatively simple because there are existing tools and tutorials our there, but I still ran into a few hiccups during my experiment and I think it's worth nothing that down.

Why

Some background first. I need to build a VirtualBox image as a part of the release. There are few different approaches. In the past I did this with Vagrant and VirtualBox in my home-made Jenkins CI. So naturally I would dig in this way.

But now we're using CircleCI and it's hard to get VirtualBox running inside CircleCI environment.

I also checked AWS, AWS doesn't support nested virtualization unless you're using an expensive bare-metal machine. GCP supports nested virtualization but at that point I decided not to mess with it.

Chances are that we may need to build a AWS AMI too in the future, so after some digging I decide to use Packer with its amazon-ebs builder to build an AMI and then utilize aws-cli to export AMI to VMDK.

Packer

Using Packer to build an AMI is quite easy. Here's an example hcl file:

variable "ami_name" {
  type = string
  default = "${env("CUSTOM_AMI_NAME")}"
}
variable "region" {
  type    = string
  default = "ap-northeast-1"
}

source "amazon-ebs" "ubuntu" {
  ami_name      = "${var.ami_name}"
  instance_type = "c4.2xlarge"
  ssh_pty = true
  region        = "${var.region}"
  source_ami_filter {
    filters = {
      name                = "ubuntu/images/*ubuntu-bionic-18.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]
  }
  launch_block_device_mappings {
    device_name = "/dev/sda1"
    volume_size = 40
    volume_type = "gp2"
    delete_on_termination = true
  }
  ssh_username = "ubuntu"
}

build {
  sources = ["source.amazon-ebs.ubuntu"]

  provisioner "shell" {
    script = "./server_setup.sh"
  }
}

There are few things we need to pay attention here in the bootstrap script.

First, you'll want to wait until cloud-init is finished to proceed your script. Otherwise there will be random failures.

# Wait for cloud-init to finish
while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done

Second, if you build the AMI and export to VMDK, import it in your VirtualBox now, you'll find that there's no network interface in your virtual machine. Because those AMIs on AWS are optimized for cloud environments, we need to tweak a little bit after your bootstrap is finished.

# Remove cloud-init to speed up the boot in virtualbox
sudo touch /etc/cloud/cloud-init.disabled
sudo apt remove cloud-init --yes
sudo rm -rf /etc/cloud/; sudo rm -rf /var/lib/cloud/

# Setup network in virtualbox
sudo rm /etc/netplan/50-cloud-init.yaml
sudo bash -c "cat > /etc/netplan/config.yaml" <<'EOF'
network:
  version: 2
  renderer: networkd
  ethernets:
    id0:
      match:
        name: en*
      dhcp4: yes
EOF
sudo bash -c "echo '@reboot /usr/sbin/netplan apply' | crontab -"

Finally, you can convert AMI to VMDK on S3 when packer finished the building process.

export CUSTOM_AMI_NAME=your-ami-name
packer build example.pkr.hcl

AMI_ID=$(aws ec2 describe-images --owners self --filters "Name=name,Values=${CUSTOM_AMI_NAME}" | grep ImageId | cut -f4 -d'"')

aws ec2 export-image --image-id ${AMI_ID} --disk-image-format VMDK --s3-export-location S3Bucket=bucketname,S3Prefix=prefixname/

My IME setup for Windows 10

Ash Wu — Tue, 31 Dec 2019 01:10:28 +0000

It has been a huge pain for me to type Chinese in Windows 10. And it was the main reason I dislike Windows 10.

The problem is that I am used to switch between English & Chinese input methods by typing Ctrl + Space. This went way back from Windows 95, or even Windows 3.1

But suddenly it changed in Windows 10. You'll have to use Ctrl + Shift to switch between input methods, and Ctrl + Space became disable/enable the method.

What I'd like to have is the following:

English as the default input methods, no matter which window I go.
Use Ctrl + Space to switch between English & Chinese input methods.

With the default Windows 10 setting, I could only get one of these:

Default English, use Ctrl + Shift to switch.
Default Chinese, use Shift or Ctrl + Space to switch to English

None of these satisfied me and I don't really want to adopt myself to get use to it.

I used lots of workarounds before, but today I believe I found the perfect solution: autohotkey + regedit.

For autohotkey, I have these two settings in a ime.ahk file.

^Space::#Space
CapsLock::Ctrl

And put this into startup folder so it could be loaded on boot.

But this script sometimes failed, and my IME will be disabled. I'll have to suspend the autohotkey script and restore it, turn the script back up.

So here comes the second fix: disable the original Ctrl + Space.

Follow the instruction here, use regedit and disable this keybinding.

Finally I can type normally in Chinese on Windows 10.

Management GUI for Elasticsearch

Ash Wu — Tue, 18 Dec 2018 09:58:06 +0000

Preface

Elasticsearch is a developer-friendly, with minimal configuration & manual management. But sometimes we still want to know more about our cluster.

You can fetch these information from elasticsearch APIs, so if you're a elasticsearch API ninja you can skip this article.

Here are some tools that I prefer, to help me to quickly get the overview of the cluster.

Prerequisite

Usually our cluster is located in private VPC, so we have to port-forward to our local machine.

What I usually do is to forward the kubernetes API to my local machine, and use kubectl port-forward to handle the rest.



    # I modified the kubeconfig to use port 16443 for remote environments
    $ ssh -L 16443:127.0.0.1:6443 ssh-jumper

    # port-foward elasticsearch to localhost
    $ kubectl -n logging port-forward elasticsearch-data-0 9200:9200

    # check if it's ready
    $ curl localhost:9200

ElasticHQ

http://www.elastichq.org/

ElasticHQ is an open source application that offers a simplified interface for managing and monitoring Elasticsearch clusters.



    $ git clone https://github.com/ElasticHQ/elasticsearch-HQ.git
    $ cd elasticsearch-HQ

    # Python 3 is required.
    $ sudo pip3 install -r requirements.txt
    $ python3 application.py

    # Access HQ with: http://localhost:5000
    # If you're using docker version, access ES via host.docker.internal:9200

In this page we can check the cluster load, free space & heap usage. Also check the size of each indices, document count and total storage size.

Another goodie is the Diagnostics tab. HQ will highlight those metrics with potential risk and give you some advices. This can be a reference when you're troubleshooting or doing performance tuning on your elasticsearch cluster.

Cerebro

https://github.com/lmenezes/cerebro



    # Java 1.8 or newer is required. brew cask install java
    # Download the latest tarball from https://github.com/lmenezes/cerebro/releases/latest
    $ wget https://github.com/lmenezes/cerebro/releases/download/v0.8.1/cerebro-0.8.1.tgz
    $ tar zxvf cerebro-0.8.1.tgz
    $ cd cerebro-0.8.1
    $ ./bin/cerebro
    # Open cerebro with http://localhost:9000
    # If you're using docker version, access ES via host.docker.internal:9200

I usually use Cerebro to observe the index shards allocation. It's clear and intuitive.

And the cluster settings, aliases and index templates under more menu come handy when you need them.

sync.Pool

Ash Wu — Thu, 19 Apr 2018 06:33:27 +0000

Official document: [https://godoc.org/sync#Pool]

The usage of sync.Pool is a very common in high concurrency scenarios. You may create tons of goroutines and each of them allocates short-lived objects and later cause a slow GC.

With sync.Pool we can avoid this. Objects inside the pool will be cleaned after GC without any notification, so sync.Pool is not suitable for connection pool.

The objects being Put() back into the pool may be Get() and reused before GC happens. This means you have the chance to Get() a object with old values.

Example here:

package main

import (
    "fmt"
    "runtime"
    "runtime/debug"
    "sync"
)

func main() {
    newFunc := func() interface{} {
        return make([]byte, 32)
    }
    pool := sync.Pool{New: newFunc}

    v1 := pool.Get().([]byte)
    fmt.Printf("v1: %v\n", v1)
    v1[0] = 1
    fmt.Printf("modified v1: %v\n", v1)

    // modified v1, put back to pool, before GC
    pool.Put(v1)
    v2 := pool.Get().([]byte)
    fmt.Printf("v2: %v\n", v2)
    pool.Put(v2)

    // After GC
    debug.SetGCPercent(100)
    runtime.GC()
    v3 := pool.Get().([]byte)
    fmt.Printf("v3: %v\n", v3)
}

So do remember to clean the object before you Put() back, or init the object right after you Get() to prevent any accident.