Forem: Andres

Install Jenkins using Docker Compose

Andres — Tue, 05 May 2020 14:14:05 +0000

What is Jenkins?

Jenkins is an open source automation tool written in Java with plugins built for Continuous Integration purpose. It is used to build and test your software projects continuously making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. It also allows you to continuously deliver your software by integrating with a large number of testing and deployment technologies.

And Docker Compose?

Docker is a platform for running applications in an isolated environment called a "container" (or Docker container). Docker Compose is a tool for defining and running Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. It lets you take advantage of the benefits of Docker while abstracting the complexity of your stack.

Installing Jenkins directly in your OS can be tricky and expensive in terms of time and resources. You need to have Java installed in your local machine and at least 10 GB of drive space. On the other hand, using docker compose is really straightforward and offers a lot of advantages.

We're going through the steps to install Jenkins using Docker-Compose.

Install Docker Compose

Docker Desktop for Mac and Docker Toolbox already include Compose along with other Docker apps, so Mac users do not need to install Compose separately. Docker install instructions for these are here:

Install Docker Compose

You can verify the version of docker compose using:



> docker-compose --version
docker-compose version 1.25.4, build 8d51620a

Create docker-compose configuration

Inside your working directory create the docker-compose.yml file:



/jenkins-config

touch docker-compose.yml

Then copy the following configuration:



/jenkins-config/docker-compose.yml

version: '3.7'
services:
  jenkins:
    image: jenkins/jenkins:lts
    privileged: true
    user: root
    ports:
      - 8081:8080
      - 50000:50000
    container_name: jenkins
    volumes:
      - ~/jenkins:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/local/bin/docker:/usr/local/bin/docker

You need to ensure the directory ~/jenkins exists:



mkdir ~/jenkins

This volume will be used to persist all your data: configurations, plugins, pipelines, passwords, etc.

The remaining two volumes allow you to use docker inside the Jenkins server (Yes, you can create docker containers inside a docker container).

Run Docker Compose



/jenkins-config

> docker-compose up -d

Jenkins is running in localhost:8081.

Firs Log in

View the generated administrator password to log in the first time.



❯ docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword

Now, you're ready to install plugins and start creating pipelines. If you want to stop the Jenkins container you can do it with docker-compose down. When you restart it all your configuration, users, plugins previously installed will persist there.

Upgrading Versions

To upgrade to latest versions just modify the tag of the image in your docker-compose yml file, for example:



/jenkins-config/docker-compose.yml

version: '3.7'
services:
  jenkins:
    image: jenkins/jenkins:2.223.1
    ...

That's it! You can start using Jenkins to implement your CI/CD.

Multiple Playbooks in Ansible

Andres — Sun, 03 May 2020 15:34:38 +0000

Ansible is a simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs.

Ansible works by connecting to your nodes and pushing out small programs, called "Ansible modules" to them. These programs are written to be resource models of the desired state of the system. Ansible then executes these modules (over SSH by default), and removes them when finished.

TL;DR

Ansible has the option to import multiple playbooks into a single one using the method: import_playbook:

/playbooks/main.yml
---
- name: Playbook_1
    import_playbook: 00-playbook-1.yml
- name: Playbook_2
    import_playbook: 01-playbook-2.yml

When you execute ansible-playbook playbooks/main.yml Ansible will execute first the Playbook1, and then Playbook 2.

We're going through an example to see it in action

The first playbook executes a ping into your servers:

/playbooks/00-ping.yml
---
- hosts: all
  tasks:
    - name: Ping All Servers
      action: ping
    - debug: msg="Success"

The second playbook upgrades all your yum packages:

/playbooks/01-upgrade.yml
---
- hosts: all
  tasks:
    - name: upgrade all packages
      become: yes
      yum:
        name: '*'
        state: latest

The third playbook download a node binary distribution:

/playbooks/02-download-node.yml
---
- hosts: all
  tasks:
    - name: Download node binary distro
      shell: curl sL https://rpm.nodesource.com/setup_12.x | sudo bash -

The fourth playbook install node:

/playbooks/03-install-node.yml
---
- hosts: all
  tasks:
    - name: Install Node
      become: yes
      yum:
        name: nodejs
        state: present

The fifth playbook copy your app into the remote servers:

/playbooks/04-provisioning.yml
---
- hosts: all
  tasks:
    - name: Copy app files
      copy:
        src: ~/documents/my-app-folder/
        dest: /home/ec2-user/my-app
        force: no
    - debug: msg="Success"

And finally you need to install dependencies based on your package-json:

/playbooks/05-install-dependencies.yml
---
- hosts: all
  tasks:
    - name: Install packages based on package.json.
      npm:
        path: /home/ec2-user/my-app

Once you have all your playbooks ready, if you want to execute all of them in the desired order you need to create a main playbook, and import all of them as follow:

/playbooks/main.yml
---
- name: Ping
  import_playbook: 00-ping.yml

- name: Update
  import_playbook: 01-update.yml

- name: Download Node
  import_playbook: 02-download-node.yml

- name: Install Node
  import_playbook: 03-install-node.yml

- name: Provisioning
  import_playbook: 04-provisioning.yml

- name: Install Dependencies
  import_playbook: 05-install-dependencies.yml

That's it. You are ready to execute ansible-playbook playbooks/main.yml and you'll upgrade all your packages, download and install node, provision all your selected servers, and install your dependencies.

Remote Back-End in Terraform

Andres — Sun, 03 May 2020 02:47:07 +0000

A "backend" in Terraform determines how the state is loaded and how an operation such as apply is executed.

By default, Terraform uses the "local" backend, which is the normal behavior of Terraform you're used to, but, if you're working in a team, or you don't want to keep sensitive information in your local disk, or you're working remotely, it's highly recommended to store this 'state' in the cloud, and we're going to see in this article how it can be done storing the backend in an S3 bucket.

TL;DR

To configure your remote state in Terraform, you need two instances:

An s3 bucket where the terraform.state file is stored.
A dynamo table with a lock id to prevent two or more users perform changes to the state at the same time.

When you call terraform plan, or terraform apply it locks the access to the state using the Lock ID in the dynamo table and then uses the terraform.state file to apply the changes to your infrastructure. When the changes are done, the terraform.state file is updated and the table is unlocked to allow new operations.

Create S3 Bucket

The code to create the S3 bucket is the following:

provider "aws" {
  region = "us-east-1"
  version = "~>2.59"
}

resource "aws_s3_bucket" "enterprise_backend_state" {
  bucket = "my-backend-bucket"

  lifecycle {
    prevent_destroy = true
  }

  versioning {
    enabled = true
  }

  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        sse_algorithm = "AES256"
      }
    }
  }
}

Create the Dynamo Table

The code to create the dynamo table is the following:

resource "aws_dynamodb_table" "enterprise_backend_state" {
  hash_key = "LockID"
  name = "backend-lock-table"
  billing_mode = "PAY_PER_REQUEST"
  attribute {
    name = "LockID"
    type = "S"
  }
}

Once you have an s3 bucket and the dynamo table you're ready to tell Terraform to store the state in s3 bucket from now on:

terraform {
  backend "s3" {
    bucket         = "my-backend-bucket"
    key            = "dev/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "backend-lock-table"
    encrypt        = true
  }
}

That's it. You can do terraform init to execute the configuration. You'll notice Terraform will no continue storing the state on your disk. You´re ready to start collaborating with your team on the same infrastructure.