Forem: flpslv

Provisioning MySQL/MariaDB Vault Database Secrets Engine with Terraform

flpslv — Tue, 30 Mar 2021 14:07:19 +0000

The goal of this post is to provide dynamic/temporary database credentials without having to manually create and manage them all.

I'm just going to start this by saying that this is just a proof of concept and best practices were not followed at all. (mainly security ones). All the procedures from this point are just simple tests with focus on easing all the side tasks just to see the whole process working.

Testing Containers

We'll be using MariaDB and Vault containers, (quick) launched as specified on both projects official DockerHub pages.

Vault

docker run --rm \
    --name vault \
    --cap-add=IPC_LOCK \
    -e 'VAULT_LOCAL_CONFIG={"backend": {"file": {"path": "/vault/file"}}, "default_lease_ttl": "168h", "max_lease_ttl": "720h"}' \
    -p 8200:8200 \
    -d vault

When vault finishes starting up, we can see from the container logs the following:

You may need to set the following environment variable:

$ export VAULT_ADDR='http://0.0.0.0:8200

The unseal key and root token are displayed below in case >you want to
seal/unseal the Vault or re-authenticate.

Unseal Key: kSUgoPDPyBrCGWc4s93CIlMUnDLZGcxdu4doYCkWSPs=
Root Token: s.I6TnqhrgYh8uET91FUsNvIwV

Development mode should NOT be used in production >installations!

So, we have our vault address, and the root token.

MariaDB

docker run --rm \
    --name mariadb \
    -p 3306:3306 \
    -e MYSQL_ROOT_PASSWORD=mysecretpw \
    -d mariadb:latest

And here we have our root user and password for MariaDB.

Because the root user shouldn't be used for anything, we're going to create a dedicated user for vault interactions.

and create the vault user

grant CREATE USER, SELECT, INSERT, UPDATE ON *.* TO 'vault'@'%' identified by 'myvaultsecretpw' WITH GRANT OPTION;

Terraform

With all the services running and configured, it's time to take care of the terraform part.

Once again, this is only a POC. All the hardcoded and weak passwords are meant for testing purposes.

provider "vault" {
    address = "http://localhost:8200"
        #Token provided from vault container log
    token = "s.I6TnqhrgYh8uET91FUsNvIwV" 
}

resource "vault_auth_backend" "userpass" {
  type = "userpass"
}

# USERS
resource "vault_generic_endpoint" "user_userro" {
  depends_on           = [vault_auth_backend.userpass]
  path                 = "auth/userpass/users/userro"
  ignore_absent_fields = true

  data_json = <<EOT
{
  "policies": ["db-ro"],
  "password": "userro"
}
EOT
}

resource "vault_generic_endpoint" "user_userrw" {
  depends_on           = [vault_auth_backend.userpass]
  path                 = "auth/userpass/users/userrw"
  ignore_absent_fields = true

  data_json = <<EOT
{
  "policies": ["db-all", "db-ro"],
  "password": "userrw"
}
EOT
}

# POLICIES
# Read-Only access policy
resource "vault_policy" "dbro" {
  name   = "db-ro"
  policy = file("policies/dbro.hcl")
}

# All permissions access policy
resource "vault_policy" "dball" {
  name   = "db-all"
  policy = file("policies/dball.hcl")
}


# DB
resource "vault_mount" "mariadb" {
  path = "mariadb"
  type = "database"
}

resource "vault_database_secret_backend_connection" "mariadb_connection" {
  backend       = vault_mount.mariadb.path
  name          = "mariadb"
  allowed_roles = ["db-ro", "db-all"]
  verify_connection = true

  mysql{
    connection_url = "{{username}}:{{password}}@tcp(192.168.11.71:3306)/"
  }

# note that I have my database address hardcoded and I'm using my lan IP, since I'm running the mysql client directly from my host and vault/mariadb are running inside containers with their ports exposed.

  data = {
    username = "vault"
    password = "myvaultsecretpw"
  } 

}

resource "vault_database_secret_backend_role" "role" {
  backend             = vault_mount.mariadb.path
  name                = "db-ro"
  db_name             = vault_database_secret_backend_connection.mariadb_connection.name
  creation_statements = ["GRANT SELECT ON *.* TO '{{name}}'@'%' IDENTIFIED BY '{{password}}';"]
}

resource "vault_database_secret_backend_role" "role-all" {
  backend             = vault_mount.mariadb.path
  name                = "db-all"
  db_name             = vault_database_secret_backend_connection.mariadb_connection.name
  creation_statements = ["GRANT ALL ON *.* TO '{{name}}'@'%' IDENTIFIED BY '{{password}}';"]
}

Policy files used in previous code listing:

policies/dball.hcl

path "mariadb/creds/db-all" {
  policy = "read"
  capabilities = ["list"]
}

policies/dbro.hcl

path "mariadb/creds/db-ro" {
  policy = "read"
  capabilities = ["list"]
}

After this, the usual process:
terraform init
terraform apply

And we have everything ready for some testing.

Testing

Testing the RO user

$ vault login -method userpass username=userro
Password (will be hidden): 
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
(...)

request DB credentials

$ vault read mariadb/creds/db-ro
Key                Value
---                -----
lease_id           mariadb/creds/db-ro/uGldyp0BAa2GhUlFyrEwuIbs
lease_duration     168h
lease_renewable    true
password           8vykdcZNHp-I0pajVtoN
username           v_userpass-d_db-ro_75wxnJaL69FW4

$ mysql -h 127.0.0.1 -u v_userpass-d_db-ro_75wxnJaL69FW4 -p'8vykdcZNHp-I0pajVtoN'

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 101
Server version: 10.3.13-MariaDB-1:10.3.13+maria~bionic mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]>

... try to do stuff

MariaDB [(none)]> create database my_database;
ERROR 1044 (42000): Access denied for user 'v_userpass-d_db-ro_75wxnJaL69FW4'@'%' to database 'my_database'

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| my_db              |
| mysql              |
| performance_schema |
+--------------------+
4 rows in set (0.00 sec)

MariaDB [(none)]> use my_db;

Database changed
MariaDB [my_db]> show tables;
+-----------------+
| Tables_in_my_db |
+-----------------+
| my_table        |
+-----------------+
1 row in set (0.00 sec)

MariaDB [my_db]> select * from my_table;
Empty set (0.00 sec)

MariaDB [my_db]>

Testing the RW user

$ vault login -method userpass username=userrw
Password (will be hidden): 
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
(...)

request DB credentials

$ vault read mariadb/creds/db-all
Key                Value
---                -----
lease_id           mariadb/creds/db-all/GHRHvpuqa2ITP9tX54YHEePl
lease_duration     168h
lease_renewable    true
password           L--8mPBoprFZcaItINKI
username           v_userpass-j_db-all_DMwlhs9nGxA8

$ mysql -h 127.0.0.1 -u v_userpass-j_db-all_DMwlhs9nGxA8 -p'L--8mPBoprFZcaItINKI'
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 109
Server version: 10.3.13-MariaDB-1:10.3.13+maria~bionic mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]>

... try to do stuff

MariaDB [(none)]> create database my_database;
Query OK, 1 row affected (0.01 sec)

MariaDB [(none)]> use my_db;
Database changed

MariaDB [my_db]> create table the_table (i integer);
Query OK, 0 rows affected (0.03 sec)

MariaDB [my_db]> show tables;
+-----------------+
| Tables_in_my_db |
+-----------------+
| my_table        |
| the_table       |
+-----------------+
2 rows in set (0.00 sec)

And that's it. We now have the base for our MariaDB dynamic/temporary users.

Encrypt your notes with GnuPG

flpslv — Sat, 06 Mar 2021 02:10:53 +0000

How many times did you want to put something on a text file, but didn't want to leave it there just waiting to be read by anyone?

How many times you were searching for a password manager and during that search you didn't have any place to store your secrets? (more on the password manager later)

There's a quick and easy way to do that: using GnuPG.

GnuPG is usually installed with every linux distribution by default. And if it isn't, in case you're using a minimal image, chances are you will need to install it even to perform some basic operations with your distribution like adding new repositories (for example).

Very often we come across the terms PGP, OpenPGP, GnuPG when searching for this subjects and it can be confusing sometimes.

Resuming:

PGP ( Pretty Good Privacy ) was the first implementation, by Phil Zimmermann, acquired by Network Associates Inc (NAI).
OpenPGP is the open standards version of PGP (which was by then owned by NAI.
GnuPG is an (complete and free) implementation of the OpenPGP standard.

More about PGP and OpenPGP can be read on the History of OpenPGP while GnuPG information can be found on The GNU Privacy Guard page

To take advantage of this, make sure you have gnupg installed on your system (yes, this is debian (/based) only but it exists on other distros):

sudo apt install gnupg2

Generate your key (if you already haven't got one). It will ask your Name, Email and a Passphrase. Don't forget that despite all of the security that GPG provides, if you choose a weak passphrase, you'll be weakening all the process.

gpg2 --generate-key

And put this little script (let's call it gvim) somewhere on your $PATH (like ~/bin/gvim for example and don't forget the chmod u+x ~/bin/gvim):

#!/bin/bash
srcfile=$1
tmpfile=$(mktemp -p ${HOME})
# remember the email you used during key generation?
email="_PUT_YOUR_Email_HERE" 
if [ $? -nq 0 ]; then
    echo "error creating tempfile"
    rm $tmpfile
    exit 1
fi
gpg2 -d $scrfile > $tmpfile
vim $tmpfile
#this will actually wait for vim to exit

#and overwrite the original file with the new encrypted version
gpg2 -e -r $email < ${tmpfile} > $srcfile

Next, let's setup an easy way to read those files.
It's simple to type gpg2 -d but it's simpler to create an alias (and put it on your .bashrc)

alias gcat='gpg2 -d  2>/dev/null'

So, now, we can just use
gvim to create a new encrypted note and gcat to read it anywhere.

That is just a basic script. It doesn't really backup or ensure you don't lose your original data. I started to do some backups of all my notes, but that's really up to you.

And last but not least, since I've mentioned the password manager, we can also use this awesome password manager which takes advantage of GPG and is very simple to use!

Terraform AWS - Imports, Key-Pairs and Broken States

flpslv — Thu, 04 Mar 2021 01:54:06 +0000

There is nothing better than starting a new week with a full terraform infra refactoring using a clean (and shared) remote state.

What do I have?

an old and functional terraform state
a new code folder better organized
an empty s3 bucket to store the new state

Starting with the clean and shared remote state:

variable "the_profile" {}
variable "region" {}

provider "aws" {
  region = var.region
  profile = var.the_profile
}

resource "aws_s3_bucket" "state_bucket" {
  bucket = "my-uniquely-named-state-bucket"
  acl = "private"
  versioning {
    enabled = "true" # this is a must
  }

  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        sse_algorithm = "AES256"
      }
    }
  }

resource "aws_dynamodb_table" "terraform_state_lock" {
  name = "my-lock-table"
  read_capacity  = 1
  write_capacity = 1
  hash_key       = "LockID"
  attribute {
    name = "LockID"
    type = "S"
  }
}

Creating this init structure will then allow us to properly configure our backend:

## Backend to keep status

provider "aws" {
  region  = var.region
  profile = var.the_profile
}

terraform {
  backend "s3" {
    bucket         = "my-uniquely-named-state-bucket"
    key            = "mystate.tfstate"
    region         = "INSERT_YOUR_REGION_HERE"
    encrypt        = "true"
    dynamodb_table = "my-lock-table"
  }
}

Things are starting to happen.

After some copy+paste oriented programing, I managed to place all my files from the old structure into the new and I'm ready do run my first plan.

And terraform said:

Plan: WAY TOO MANY to add, 0 to change, 0 to destroy.

( ok, it didn't say that, but that's what I read )

Adding the VPC, the Subnets, the Routing Tables, the Security Groups, the Instances and all the other "direct resources" was fairly easy. Nothing that a few queries here and there and some terraform import TERRAFORM_RESOURCE_NAME AWS_ID
didn't handle. And the easiest part was done.

Enter the Security Group Rules

This is where things started to get interesting.
Quoting directly from terraform aws_security_group_rule page,
Security Group Rules can be imported using the security_group_id, type, protocol, from_port, to_port, and source(s)/destination(s) (e.g. cidr_block) separated by underscores (_). All parts are required.

Luckily I had access to my previous functional terraform state. And with a little bit of python I could find and extract the required ID for each missing security group rule.

So, once more the
terraform import aws_security_group_rule.sg_allow_stuff sg-001122334455_ingress_tcp_8080_8080_sg-001122334455

Thing is, the plan continued to show things like:

-/+ resource "aws_security_group_rule" "sg_allow_stuff" {
      - cidr_blocks              = [] -> null
      ~ id                       = "sgrule-12345678" -> (known after apply)
      - ipv6_cidr_blocks         = [] -> null
      - prefix_list_ids          = [] -> null
      ~ self                     = false -> true # forces replacement
      ~ source_security_group_id = "sg-001122334455" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

when I noticed that line
self = false -> true # forces replacement
and the fact that the source and destiny security group IDs were the same.

So, the self was the issue. No problem, just add it at the end of the AWS_ID part on the import command:
terraform import aws_security_group_rule.sg_allow_stuff sg-001122334455_ingress_tcp_8080_8080_sg-001122334455_self

And it imported!

And the stubborn Key

Two minutes (to midnight) after and all the security group rules were added into the new state.

There was only this aws_key_pair missing.
Ok! Let's go!
terraform import aws_key_pair.auth thekeyname
Ok! It imported as all the other resources! Nothing new so far...

But, the next apply, THAT key was still:

  # aws_key_pair.auth will be created
  + resource "aws_key_pair" "auth" {
      + arn         = (known after apply)
      + fingerprint = (known after apply)
      + id          = (known after apply)
      + key_name    = "thekeyname"
      + key_pair_id = (known after apply)
      + public_key  = "#### EMPTY"
    }

Oh, that "#### EMPTY". I forgot to add the resource on my side.
A few keystrokes after (now with the correct key), the same:

  # aws_key_pair.auth will be created
  + resource "aws_key_pair" "auth" {
      + arn         = (known after apply)
      + fingerprint = (known after apply)
      + id          = (known after apply)
      + key_name    = "thekeyname"
      + key_pair_id = (known after apply)
      + public_key  = "ssh-rsa yaddayaddayadda..."
    }

Hello google my old friend :)
Turns out I wasn't the first to hit this wall

Ok, so let's try one of the suggested solutions there.

Download the new state from S3
Edit the state, search for the entry corresponding to the key
Enter the public key manually on the public_key field (not forgetting the quotes)
Save the state
Upload it and ...

!!!RUN TERRAFORM!!! ( !!! RUN FOR YOUR LIVES !!! )

Sadly, Terraform says NO! :(

Error: Error loading state: state data in S3 does not have the expected content.

This may be caused by unusually long delays in S3 processing a previous state
update.  Please wait for a minute or two and try again. If this problem
persists, and neither S3 nor DynamoDB are experiencing an outage, you may need
to manually verify the remote state and update the Digest value stored in the
DynamoDB table to the following value: 5d3a61408eb4ba89aeaf09818561d0a7

Like anyone in trouble, I just focused on the first part of the message. "Things were looking bad" I thought. On top of that, I was so confident that I didn't even backup the state before altering it.
BUT ... Bucket versioning to the rescue!

aws s3api get-object --bucket my-uniquely-named-state-bucket --key mystate.tfstate --version abcdversionreferencehere mystate.tfstate

Of course this didn't solve the problem (as I figured out after a second upload). I decided to calm down and read everything.

DynamoDB Digest? Interesting. So, there was no big problem after all. Turns out the tfstate md5 digest was stored on dynamoDB and there was a couple of missing steps on my procedure:

Download the new state from S3
Edit the state, search for the entry corresponding to the key
Enter the public key manually on the public_key field (not forgetting the quotes)
Save the state
get the md5 sum of the state
update the md5 digest on the DynamoDB terraform_state_lock table
Upload the modified state

And everything worked like it was supposed to!

What did I learn?

READ everything. Not just the begining
Always BACKUP (or Enable versioning when possible)
Google is your friend. (nothing really new here)

Running WineHQ inside a docker container

flpslv — Thu, 24 Sep 2020 10:00:17 +0000

(Before I waste any of your time, let me start by warning that this post was done using the nVidia gpu that's on my system. Unfortunately I don't have (yet) the means (nor time) to properly test this with an AMD or even Intel GPU.)

Well ...
Nowadays this isn't exactly true...
Nor this solution is the best.
But as I said in my profile, I'm just curious.

Things have evolved, tools like lutris or even steam's proton are providing easier and faster ways of playing games that do not have native linux ports.

I have tried both and things just work. It's a fact. And I was able to play the ( supported ) games I wanted without big effort:

With Lutris, I just had to go to the site and pull the pre-configured installer for the game I wanted and let it work it's magic.
With Steam Proton, I just had to download the game and play it.
With Wine itself, either installed by my distro package manager or compiled from source.

All of those, as I stated before just work perfectly (one way or another). But curiosity was still nagging: "Can I just have a container with all that stuff installed and not have all those dependencies dangling on my system?"

In fact, that question always comes back to me once in a while, when I think about how many stuff I have installed on my system and how many I really need on a daily basis. Even further, what will I miss when I change or need to reinstall my distro.

With that in mind, I started to experiment with containers. Even if only to test newer wine versions on a more or less clean way, because I've already upgraded to a newer version that unfortunately was broken, and had to reinstall the previous working version packages.

So, I decided to build things up from a Ubuntu Focal container and split in two containers:

My base container based on Ubuntu having only the nVidia drivers installed
My actual wine container with all the dependencies for the games.

I went for this approach since gpu drivers don't come out that often (unfortunately) and I can also have a base image to work with.

Before going into more detail, if you notice anything unusual with the Dockerfiles is because this build system is prepared to be used with Makefiles. But they also can be built normally with the traditional docker build cmd. The defaults can be overridden with --build-arg flags.

Also, make sure the driver version you have installed on your host are the same as the drivers installed inside the container.

Base container



ARG UBUNTU_VERSION=focal
ARG NVIDIA_DRIVER_VERSION=450
FROM ubuntu:$UBUNTU_VERSION

ARG UBUNTU_VERSION
ARG NVIDIA_DRIVER_VERSION

ENV DEBIAN_FRONTEND=noninteractive
ENV UBUNTU_VERSION=$UBUNTU_VERSION
ENV NVIDIA_DRIVER_VERSION=$NVIDIA_DRIVER_VERSION

ENV PKGNV="nvidia-driver-${NVIDIA_DRIVER_VERSION} libnvidia-gl-${NVIDIA_DRIVER_VERSION}:i386"
ENV PKGVK="libvulkan1 libvulkan1:i386 vulkan-tools vulkan-utils"


RUN     dpkg --add-architecture i386 && \
    apt update && \
    apt install -y gnupg2 apt-transport-https curl 

RUN apt install -y $PKGNV $PKGVK

So, this is the first container, based on a ubuntu focal version and nvidia drivers 450 (by default), vulkan and a few base tools to be used later. I will refer to it for the next steps as ubuntu-nvidia-vlk:focal.

Wine container



FROM ubuntu-nvidia-vlk:focal

ARG WVERSION="5.17"
ARG WTVERSION="20200412"

ENV WINE_VERSION="${WVERSION}"
ENV PKG_WINE_VERSION="${WVERSION}~${UBUNTU_VERSION}"
ENV WINE_TRICKS_VERSION="${WTVERSION}"

RUN curl -s https://dl.winehq.org/wine-builds/winehq.key | apt-key add - && \
    echo "deb https://dl.winehq.org/wine-builds/ubuntu/ ${UBUNTU_VERSION} main" | tee /etc/apt/sources.list.d/wine.list && \
    apt update && \
    apt install -y winbind cabextract wget fonts-wine ttf-mscorefonts-installer\
        winehq-staging=$PKG_WINE_VERSION \
        wine-staging=$PKG_WINE_VERSION \
        wine-staging-i386=$PKG_WINE_VERSION \
        wine-staging-amd64=$PKG_WINE_VERSION

ADD https://github.com/Winetricks/winetricks/archive/${WINE_TRICKS_VERSION}.zip /tmp/wt.zip 
RUN unzip /tmp/wt.zip -d /tmp/ && \
    cp /tmp/winetricks-${WINE_TRICKS_VERSION}/src/winetricks /usr/local/bin && \
    rm -Rf /tmp/*

And we can call this our winehq:5.17 container.

So, we have a base container and a wine container.
Since most of the games will ask for winbind and cabextract I've decided to put those together with wine. Also, some font packages since they're used a lot by launchers/loaders.
I guess I could have another stage just for those needed packages, but I didn't have the time yet to figure out the best stage to put them.
So now we only need to create a script to use the containers to avoid having to memorize a huge command line.

At this point, we could also add dxvk to our container by adding this lines to the dockerfile



ARG DXVKVERSION="1.7.1"
ENV DXVK_VERSION="${DXVK_VERSION}"
ADD https://github.com/doitsujin/dxvk/releases/download/v${DXVK_VERSION}/dxvk-${DXVK_VERSION}.tar.gz /tmp/dxvk.tar.gz

... but DXVK can also be installed by winetricks. So, I'll just go with winetricks method for now.

We can add this script to one of the PATH directories so we can start the container at any time.

Also, it's just a personal preference to have all the versions as environment variables on my container, so I can quickly check them any time I need.

It will create a temporary folder on your $HOME/tmp path and use it as $HOME inside the container. Also, the current path which will also be the WINEPREFIX will be placed inside the container on $HOME/game.

I had to do it this way, because the .Xauthority file will be mounted in the home dir and since it's a volume two things could happen:
1 - if I made the WINEPREFIX my home dir, the .Xauthority file would end up being copied to that location on the host.
2 - if the home dir didn't exist, it would be created by the user root (the container default user) which could lead to lack of permissions for some apps that use the home dir as default location to write theyr own files ( winetricks, for example).



#!/bin/bash

curdir="$(pwd)"
vhome="${HOME}/tmp/.winestorage"

if [ ! -d "$vhome" ]; then
    echo "creating $vhome"
    mkdir -p $vhome
fi

docker run --rm \
    --name wine \
    -u $(id -u):$(id -g) \
    -e DISPLAY \
    -e WINEPREFIX=/wine/game \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e PULSE_SERVER=unix:/pulse \
    -e HOME=/wine \
    -v $HOME/.Xauthority:/wine/.Xauthority \
    -v /run/user/$(id -u)/pulse/native:/pulse \
    --device /dev/nvidia0:/dev/nvidia0 \
    --device /dev/nvidiactl:/dev/nvidiactl \
    --device /dev/nvidia-uvm:/dev/nvidia-uvm \
    --device /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools \
    --device /dev/nvidia-modeset:/dev/nvidia-modeset \
    -v ${vhome}:/wine \
    -v ${curdir}:/wine/game \
    -ti \
    winehq:5.17 \
    bash

There are several considerations to this script:

it will be removed after termination --rm
it will map my user and group inside the container -u $(id -u):$(id -g)
it will set the WINEPREFIX env to /wine/game -e WINEPREFIX=/wine/game and set my HOME to /wine -e HOME=/wine
it will mount the directory containing the X11 sockets inside the container -v /tmp/.X11-unix:/tmp/.X11-unix
since I'm using pulesaudio it will define the socket path -e PULSE_SERVER=unix:/pulse and it will place my actual socket inside the container -v /run/user/$(id -u)/pulse/native:/pulse
it will mount my .Xauthority file on the user's home directory -v $HOME/.Xauthority:/wine/.Xauthority
add all my devices corresponding to the nVidia gpu
and finally mount the home and game paths in the container

We can also extend this script to pass extra wine variables to the container.

Now, inside the container we can just go directly to /wine/game and start installing or playing what we want to run.

When trying to install something that needs extra packages installed, there is also the possibility of opening a new shell, enter the container as root and install what we need just by running docker exec -u root -ti wine bash.

Using wine isn't really the target of this post, but there is enough information around the internet to help with that. And having winetricks installed is also a nice aid to configure prefixes.
One of the motives I had, using that directory structure (and the $vhome path) was also to try to maintain some cache for winetricks downloads.

I've tried this method to launch Battle.net and try a couple of games which worked surprisingly well.

I was also able to run Epic Games Store, but unfortunately had no luck when trying to play the few games I have there. I guess some things were missing on my prefix.

And finally, was able to install Wargaming's Game Center and play a few rounds of world of tanks.

At this point, most of the needed work is just figuring out which dependencies we need in our prefix and which wine libraries we need to override. Lutris installation scripts are nice to look for more information on a given game.
Some errors still occur, but the wine debug messages can also help to trace some problems and find the possible missing packages we might need to install.

Security Concerns

I'm not going to dive deep into this approach security flaws.
Ari Kalfus already did it very well on his post Running a Graphical Application from a Docker Container - Simply and Securely. It's worth the read even if you're not thinking on running any GUI applications.

So, let's just try to tweak a bit our launcher script to address some of those valid points:



#!/bin/bash

curdir="$(pwd)"
vhome="${HOME}/tmp/.winestorage"

if [ ! -d "$vhome" ]; then
    echo "creating $vhome"
    mkdir -p $vhome
fi

docker run --rm \
    --cap-drop=all \
    --name wine \
    -u $(id -u):$(id -g) \
    -e DISPLAY \
    -e WINEPREFIX=/wine/game \
    -v /tmp/.X11-unix:/tmp/.X11-unix:ro \
    -e PULSE_SERVER=unix:/pulse \
    -e HOME=/wine \
    -v $HOME/.Xauthority:/wine/.Xauthority:ro \
    -v /run/user/$(id -u)/pulse/native:/pulse \
    --device /dev/nvidia0:/dev/nvidia0 \
    --device /dev/nvidiactl:/dev/nvidiactl \
    --device /dev/nvidia-uvm:/dev/nvidia-uvm \
    --device /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools \
    --device /dev/nvidia-modeset:/dev/nvidia-modeset \
    -v ${vhome}:/wine \
    -v ${curdir}:/wine/game \
    -ti \
    winehq:5.17 \
    bash

Bear in mind that one of the things that stops working when we add the --cap-drop=all is the possibility to enter the container as root and change anything on the container system.

Of course there are easier ways to play games on linux, but having this working is a nice thing for me, so I can get rid of many unwanted installed packages on my host system.

I could improve this post on trying to get this working on AMD or Intel graphics, but it's something I won't be able to do for now.

What's your favorite minimal linux distro?

flpslv — Thu, 17 Sep 2020 21:45:47 +0000

What is your favorite minimal distro, so you can just install a base system (no DE, no X, nothing from start) and build it up from there?

Bypassing AWS Cli profile to use IAM Roles

flpslv — Fri, 28 Aug 2020 23:38:15 +0000

While trying to upgrade some legacy AWS instances which were already configured and working, I just needed to start configuring and using EC2 IAM Roles.

I just attached a simple (and permissive) EC2 role to my instance to see what I could do with it.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:List*",
                "s3:Get*"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

and started to furiously type my copy command
aws s3 cp s3://my-own-and-only-bucket/file .

Problem was that someone had already configured some profiles for the aws cli, even worst, the default profile was also configured and it was being used for some random operation I wasn't able to find out.

Checking AWS Documentation on configuration precedence

Command line options – Overrides settings in any other location. You can specify --region, --output, and --profile as parameters on the command line.

Environment variables

*CLI credentials file *( ~/.aws/credentials on Linux or macOS, or at C:\Users\USERNAME.aws\credentials on Windows.)

CLI configuration file ( ~/.aws/config on Linux or macOS, or at C:\Users\USERNAME.aws\config on Windows.)

Container credentials

Instance profile credentials – You can associate an IAM role with each of your Amazon Elastic Compute Cloud (Amazon EC2) instances.

And as the default profile didn't have all the needed S3 permissions I kept hitting the annoying 403 Forbidden.

It really crossed my mind first to delete the credentials file and second to rename the default profile to something else. I just had no way to know what process would break next.

So, to bypass the credentials file default profile and make the aws cli use the IAM Role, all I needed to do was to create a dummy almost empty profile setting the output ( for example) ...

vim ~/.aws/credentials

[profile dummy]
output = json

... and force my copy command to use that profile
aws s3 cp s3://my-own-and-only-bucket/file . --profile dummy

Turns out that without the access keys on that profile, it ended up using the next available credentials: the IAM role.

Now I could resume with the upgrade ... as soon as I found out what was using those credentials.

Chroming Incognito

flpslv — Wed, 26 Aug 2020 22:58:58 +0000

The subject has been brought up frequently from time to time, but I would like to ask "Do we really browse incognito?"

I mean, there are lots of warning guides teaching people about the Incognito mode and how to achieve it across several browsers, but is it really incognito? How far incognito does it go?

The other day I was trying to Incognito some browsing session due to my musical guilty pleasures (and not wanting to get recommendations based on those) when I thought:
Hey, How much Incognito is this, if:

It knows some of my history ?
It still shares the same data path as my regular mode browser ?
I open a second Incognito and it has my sessions from the first one ?

Ok, maybe Incognito isn't something we can fully achieve, there will be always some loophole but at least I wasn't going down that easily.

Just added this script to one of my path folders, and some of those thoughts just got some answers:

 #!/bin/bash

google-chrome --incognito --user-data-dir=$(mktemp -d /tmp/x.XXXXXXX) 2>/dev/null &

( and giving it execution permission of course )

It isn't perfect.
Along the day, my /tmp folder gets filled with these chrome data dirs, but it all goes away eventually (automatically with a reboot/power off or manually by simply removing them). Or I could just not run it in background and remove the folder after that. There's always room for improvement somewhere.

Now, this doesn't hold any guarantee that at some point, things stop being Incognito. But at least from what I've seen, it just creates another local degree of separation.

(I've only tested this with Chrome. I'm sure it must work with other browsers also)

Or you could just try to run the browser inside a container!!!

Using Makefiles to build and publish (docker) containers

flpslv — Tue, 25 Aug 2020 08:21:34 +0000

There are more things in make and dockerfiles, Horatio,
Than are explained of in your documentation.

Intro

First of all, let's start this brief train of thought by acknowledging that despite docker is a synonym of containers, the other way around isn't true.

We could dwell on the origins of containers, but there are tons of much more valid articles out there than anything I could begin to try to explain here.

I'm assuming that containers, docker and dockerfiles aren't new terms for anyone reading this, but if that's the case then you can probably start by checking containers, docker and [dockerfile].(https://docs.docker.com/engine/reference/builder/)

Motivation

Usually, when I'm writing a dockerfile to ensure I'll never forget the needed steps to successfully build this or that container I always find myself looking at all those hardcoded version numbers and other things that could really fit in the definition of variables but for some reason end up for (commit&push) posterity.

By now you must be thinking, using some nice words, that this guy should really learn how to work with docker build arguments and stop wasting my time, but there is no denying it always ends up on memorizing all the arguments, which never happens.

Dockerfile

First step: meet kubectl

What kubectl is or does is out of scope for this post. We're only using it out as an example, because it was the first thing that popped into my mind.
Let's build a small container for kubectl:

FROM alpine:3.12

ADD https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/linux/amd64/kubectl /usr/local/bin/kubectl

RUN chmod +x /usr/local/bin/kubectl

ENTRYPOINT ["kubectl"]

As you can see, in just a small dockerfile, 2 hardcoded versions just jump into eyesight:

the alpine version itself
the kubectl version

Second step: some adjustments

Luckily Dockerfile syntax (evolving or not through the years) allows us to do some nice modifications:

ARG ALP_VER=3.12
FROM alpine:$ALP_VER

ARG KCTL_VER=1.18.0

ADD https://storage.googleapis.com/kubernetes-release/release/v${KCTL_VER}/bin/linux/amd64/kubectl /usr/local/bin/kubectl

RUN chmod +x /usr/local/bin/kubectl

ENTRYPOINT ["kubectl"]

So now, not only I can modify those two versions as I please, as I can build the image without passing any argument (which will build it with the default versions).

Of course I could just go and issue a docker build command that simply works with that Dockerfile and shows how I can manipulate those arguments
docker build --build-arg ALP_VER=3.11 --build-arg KCTL_VER=1.18.0 -t my_kubectl:1.18.0 ..

But that just wasn't enough for me. It's not that hard, but then I'd have to remember the argument names for several dockerfiles.

The Makefile

I decided I'd only settle for a simpler syntax.
Not only something that didn't make me type those long commands but at the same time, something a little nicer than a simple (and ugly) bash script.

Now I can just build using whatever versions I want:

default make build
another alpine version make build alpver=3.11
another kubectl version make build kctlver=1.17.0
both modified versions make build alpver=3.11 kctlver=1.17.0

I can build and push into the configured registry

default make
another alpine version make alpver=3.11
... and so on ...

Or I can easily check ( without any file opening ) which things I can change on that image

make help

Surely there are other things that got hardcoded somewhere, like the repository name, but this is only the beginning.

Of course this isn't THE WAY nor THE ONLY WAY. I'm not that naive. I'm sure many other options are available to do it and I'd like to know about them in the comment area.