How to set up Vault in High Availability ( HA mode ) with MySQL as storage backend

In this tutorial we will look at how we can use MySQL as a backend to Vault. This setup will involve end to end TLS. We have already seen how to setup Vault with TLS frontend. We also saw how we can setup MySQL with TLS frontend. In this tutorial, we will look at how we can use TLS Enabled MySQL as a storage backend to Vault. This is a complete secure production setup.

Tutorial on how to setup Vault Dev & Production mode:

Hashicorp Vault | Dev and Prod server setup | Unseal | Policies | TLS setup | Tharun

Tharun Shiv ・ Jan 2 '22

#security #database #beginners #devops

Tutorial on how to setup TLS/SSL enabled MySQL/MariaDB:

Easiest way to setup MySQL/MariaDB with TLS/SSL in 10 minutes- v10.5 - Any OS - Ubuntu Focal | Developer Tharun

Tharun Shiv ・ Jan 15 '22

#mysql #beginners #tutorial #database

Create Vault user in MySQL

The Vault service needs credentials to login into MySQL server in order to store data and metadata in a backend. We will create this user in MySQL now.

mysql -uroot -p --ssl-ca=/etc/mysql/certs/ca.pem
<Enter password>

# create user
CREATE USER '<vault-mysql-username>'@'%' IDENTIFIED BY '<vault-mysql-password>';

# grant privileges
GRANT ALL PRIVILEGES ON vault.* TO '<vault-mysql-username>'@'%';

MySQL Bind Address

When setting up Vault, I came across difficulties bringing up the Vault server. I have listed the challenges and solution at the end of this post. One point I would like to address is the MySQL Bind address. This configuration of MySQL defines to which network interface the MySQL process binds to / listens on. The other clients such as Vault will be able to access MySQL by sending requests to this particular interface only.

We have seen where to set this in the below tutorial

Easiest way to setup MySQL/MariaDB with TLS/SSL in 10 minutes- v10.5 - Any OS - Ubuntu Focal | Developer Tharun

Tharun Shiv ・ Jan 15 '22

#mysql #beginners #tutorial #database

MySQL CA Pem file

Vault server needs the CA.pem of the MySQL server that we used in the MySQL TLS setup tutorial. Copy that to a directory that vault can access

cp ~/certs/ca.pem /opt/vault/tls/mysql-ca.pem

chown -R vault: /opt/vault/tls

Vault config

In the above tutorials we have setup Vault, now let us configure it to use MySQL Backend.

/etc/vault.d/vault.hcl:

ui = true ## or false

# MySQL backend config
storage "mysql" {
  ha_enabled = "true"
  address = "<mySQL-hostname>:3306"
  username = "<vault-mysql-username>"
  password = "<vault-mysql-password>"
  database = "<vault-mysql-database>"
  #plaintext_connection_allowed = "true" #non-TLS mysql
  #path to CA.pem to verify MySQL SSL
  tls_ca_file = "<path-to-mysql-ca-pem>" 
}

# Vault server listen configuration
listener "tcp" {
  address       = "<vault-hostname/IP>:8200"
  tls_cert_file = "<path-to-vault-tls-cert>"
  tls_key_file  = "<path-to-vault-tls-key>"
}

# the address to advertise for HA purpose
api_addr="https://<vault-hostname>:8200"

Restart Vault

Now we can go ahead export the Vault variables and restart the vault server

export VAULT_ADDR="https://<vault-server>:8200"
export VAULT_CACERT="<path-to-vault-tls-cert>"

# make sure MySQL is running and listening

# now restart / start Vault
service vault start 

# or
service vault restart

# check Vault server logs
journalctl -u vault.service 

# check Vault status
vault status

We have successfully setup Vault with TLS frontend, TLS MySQL backend, thereby securing Vault end to end making it a perfect Production setup.

Although there are 18 ways in which a Hashicorp Vault server can be attacked, and I have covered it in the below Video

Thank you for reading, This is Tharun Shiv a.k.a Developer Tharun

You can find more articles here: https://dev.to/developertharun

Roadrunners is a series that is aimed at delivering concepts as precisely as possible. Here, a roadrunner is referred to as a person who does things super fast & efficiently. Are you a roadrunner?

Thank you