Forem: Prabhat Singh

Use keypair (.pem) to connect to Amazon EC2 Instance with PuTTy

Prabhat Singh — Sun, 17 Jul 2022 13:23:10 +0000

Good Day Everyone

Hope you all are doing well.

With this post, I'd like to share my learning on how to connect to your AWS EC2 instance using Windows SSH Client i.e. "Putty".
Though, there are several articles describing the steps, I'd like to share mine what I experienced during the task.

This experience arose from the exercise that I was about to execute for installing gitlab-runner rpm on AWS EC2 instance running with AWS AMI.
During the creation of the EC2 instance, I generated the keypair with the .pem extension whhich is to be used with Open SSH. There is already an option to generate the keypair with .ppk extension which is compatible to use with Putty. However, I chose the default option i.e. .pem

Once the EC2 instance came up, I tried connecting to EC2 instance using SSH client i.e. Putty installed on my Windows machine. However, when I configured the SSH connection in Putty with the key (.pem) download from AWS, it didn't work and prompted me to use a key with .ppk extension.

I didn't want to generate another keypair for my EC2 instance, hence I looked through the process to use the same .pem keypair to connect to the EC2 instance, which involved generating .ppk keypair using PuTTygen.

Below are steps executed in the exercise.

Steps Involved

Setup AWS EC2 instance with a public IP address and keypair (.pem)

For setting up the EC2 instance, following inputs were provided during the creation. I chose AWS AMI for the machine.

Name: example-vm
Application and OS Image: Amazon Linux

Instance type was set to t2.micro as it is available for free to use with free tier account.
Also, to connect the EC2 instance using external SSH client, I created a new keypair as I didn't have any to use from.

Instance Type: t2.micro (which is available for free in free tier account)
Key pair (login): Click on "create a new key pair"

During the keypair creation, following inputs were provided. Keypair type was set to RSA and private key file format was set to .pem which are default choice. However, one can change these based on the requirement.

Key pair name: example-vm-keypair
Key pair type: RSA
Private key file format: .pem

I made use of the default network settings ensuring that the EC2 instance is also assigned a public IP address to access it from the external machine.

Network Settings

Once all the settings of EC2 instance are finalized, click on Launch Instance. You can check the status by navigating to EC2 --> Instances
Once the EC2 instance is UP and running, it will be shown as below.

Instances

Install PuTTy and PuTTYgen

If Putty is not installed on your Windows machine, you can refer the link to download the executable.
Once the Putty is installed, you'll also notice that there is another utility installed with the name "PuTTygen".

Configure PuTTYgen

At this point, if you try to access the EC2 instance using PuTTy with keypair (.pem) generated earlier, you wouldn't be able to do that because PuTTy requires keypair with .ppk extension. Hence, it would show an error while loading the private key. For this purpose, we're going to generate the .ppk keypair with the help of .pem keypair.

Open PuTTygen and click on Load an existing private key file. Click on Load and browse to the keypair file with .pem extension

As soon as you select the .pem file, it gets loaded into PuTTygen.

Click on Save private key and select "yes" when prompted to save the file without passphrase. Save the file to a safe location on your machine.
This is keypair generated with .ppk extension which can further be used to configure PuTTy session to connect EC2 instance.

Configure PuTTy

It is time to configure PuTTy to be able to connect to EC2 instance using the private keypair with .ppk extension you just generated.

Open the AWS console and navigate to the running instance. Click on the instance.
Copy either Public IPv4 address or Public IPv4 DNS Name.

Open PuTTy on your Windows machine and configure new session for EC2 instance.
For this purpose, enter the public IPv4 address or public IPv4 DNS name in the Host Name field. By default, port is already set to 22 and connection type set to SSH.
If not, please do so.
To save the session, enter a name in the Saved Sessions field.

Host Name (or IP Address): Public IPv4 Address or Public IPv4 DNS Name
Port: 22
Connection Type: SSH
Saved Session: example-vm

Navigate to Connection --> SSH --> Auth --> Private key file for authentication --> Browse and select the keypair file with .ppk extension

Private key file for authentication: example-vm-keypair.ppk

Navigate to Session and click Save to save the newly created session. Click on Open.

A new PuTTy window opens up and asks for your confirmation to save the key in PuTTy cache. Click Yes.

It will ask the username. Once entered, you are logged into the AWS EC2 machine without entering any password.

login as: ec2-user

That is all Folks for now. I'll keep sharing my learnings. If my posts sound interesting to you, following are the places, where I can be reached.

Like, Share, Follow, Comment.

LinkedIn	Dev	Medium

Password management using ansible-vault

Prabhat Singh — Sun, 03 Jul 2022 09:19:36 +0000

Good Day Everyone!

Here is another post out of my learning these days on using ansible-vault.

I have been doing KodeKloud performance based tasks for a while and one of the tasks involved creating users and groups using ansible playbook. Passwords for the users have to be managed using ansible-vault.

I have not used ansible-vault earlier, so it's definitely a fun learning.

Task

Create an ansible playbook to perform the following task.

Create users and groups given in the file users.yml

# cat /home/centos/users.yml
admins:
  - rob
  - tim
developers:
  - mark
  - john

For users in "admins" group, home directory should be created in the "/home/" directory.
For users in "developers" group, home directory should be set to "/var/www". Also it should not create any home directory with username in "/var/www/".
Password of users of admins group should be set to "adminpassword".
Password of users of developers group should be set to "developerpassword".
To encrypt the password, make use of ansible-vault secret file which is located at "/home/centos/vault/password.txt".

Solution

In order to complete the task, we'll need to look and complete the task from the last to first.

Passwords for the users are supposed to be encrypted with the ansible-vault. Therefore, we need to make sure that the ansible.cfg file is configured with the vault password file.

Note: Password for ansible-vault must be kept at a safe location. I'm showing it for the sake of the exercise.

[centos@localhost vault]$ pwd
/home/centos/vault
[centos@localhost vault]$ cat password.txt 
redhat
[centos@localhost vault]$ grep vault_password_file /etc/ansible/ansible.cfg 
vault_password_file = /home/centos/vault/password.txt

There are two strings given in the tasks points 4 & 5 which are required to be encrypted with ansible-vault and those encrypted password would then be used during the user creation process.

For this purpose, we will use ansible-vault command to encrypt the string. We will also name the encrypted string for admins user password to a variable called "admin_pwd" and for developers user password to a variable called "developer_pwd".

Note: Important thing to ensure here is that the variable name must not contain the hyphen "-". Otherwise, the password won't work for the user. This is from my experience while attempting the task.

[centos@localhost ~]$ ansible-vault encrypt_string 'adminpassword' --name 'admin_pwd'
admin_pwd: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          30323636346564376638376364386335646463626662646535633863663265383563383238363236
          3863623764366533313162346335653736323438656331370a373263643733623766373264613530
          37333537663761383232313564343131663438303966386336663733633332343030633866613737
          3139373661623137650a303936363562383335613437653365323737373430363831633164366334
          3336
Encryption successful
[centos@localhost ~]$ ansible-vault encrypt_string 'developerpassword' --name 'developer_pwd'
developer_pwd: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          61656230636461653861356530346230386461646436636533366665346264366666623264383130
          3539343339626664383731363665306238386434306435620a653139353735366330336466346635
          39623832663063373738306238343433396661373566653232383236613662343437313832326562
          3166616232663137380a343133353439396261323939323365313339663464393266373437303335
          36323633306333386435616639653730396266616334643536643936623465383131
Encryption successful

As mentioned in the task point 3, users under developer group must have their home directory set to "/var/www". Let's ensure that directory is created into the system and appropriate permissions are applied to the directory.

[centos@localhost var]$ pwd
/var
[centos@localhost var]$ ls -ld www
drwxrwxrwx. 2 root root 27 Jul  2 17:53 www

Observe the permissions on the directory www. If it's not set, then use below command to update the permissions.

[centos@localhost var]$ sudo chmod 777 www

Now that we have configured the vault_password_file, created the variables containing password for the users under admins and developers group and created the home directory for developers users, let us start working on creating the playbook to create the groups and users

[centos@localhost ~]$ cat playbook.yml 
---
- hosts: localhost
  vars_files: /home/centos/users.yml
  vars:
    admin_pwd: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          30323636346564376638376364386335646463626662646535633863663265383563383238363236
          3863623764366533313162346335653736323438656331370a373263643733623766373264613530
          37333537663761383232313564343131663438303966386336663733633332343030633866613737
          3139373661623137650a303936363562383335613437653365323737373430363831633164366334
          3336
    developer_pwd: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          61656230636461653861356530346230386461646436636533366665346264366666623264383130
          3539343339626664383731363665306238386434306435620a653139353735366330336466346635
          39623832663063373738306238343433396661373566653232383236613662343437313832326562
          3166616232663137380a343133353439396261323939323365313339663464393266373437303335
          36323633306333386435616639653730396266616334643536643936623465383131
  tasks:
  - name: create group
    group:
      name: "{{ item }}"
      state: present
    loop:
      - admins
      - developers
  - name: create admins users
    user:
      name: "{{ item }}"
      create_home: yes
      groups: admins, wheel
      append: yes
      password: "{{ admin_pwd | string | password_hash('sha512') }}"
    loop: "{{ admins }}"
  - name: create developers users
    user:
      name: "{{ item }}"
      create_home: yes
      home: /var/www
      groups: developers
      append: yes
      password: "{{ developer_pwd | string | password_hash('sha512') }}"
    loop: "{{ developers }}"

vars_files is pointing to the file users.yml which contains the groups and users information.

vars is pointing to the variables which were created using the ansible-vault command

tasks list consists of the task first to create the groups given in the users.yml, then create the users using group and user modules of the ansible respectively.

create_home=yes -> home directory is supposed to be created in the /home

users in the admins group are also supposed to be sudo user, this is why groups: admins, wheels & append: yes have been added into the task. Similarly for users in the developers group, groups: developers

password for users in the admins and developers should be set to admin_pwd and developer_pwd variables created during encrypt string step. Password strings are also required to be hashed so as not to expose during the playbook execution or otherwise.

both user creation tasks consists of loop where values under the admins and developers list (array) are being used from users.yml

Now it's time for playbook execution. Execute the ansible-playbook command to implement changes from playbook.yml

[centos@localhost ~]$ ansible-playbook playbook.yml 

PLAY [localhost] *************************************************************************************************************

TASK [Gathering Facts] *******************************************************************************************************
ok: [localhost]

TASK [create group] **********************************************************************************************************
ok: [localhost] => (item=admins)
ok: [localhost] => (item=developers)

TASK [create admins users] ***************************************************************************************************
changed: [localhost] => (item=rob)
changed: [localhost] => (item=tim)

TASK [create developers users] ***********************************************************************************************
changed: [localhost] => (item=mark)
changed: [localhost] => (item=john)

PLAY RECAP *******************************************************************************************************************
localhost                  : ok=4    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

[centos@localhost ~]$

Note: If you'd like to see the detailed output of the playbook execution, use "-vv" option with the ansible-playbook command.

# ansible-playbook playbook.yml -vv

References

GitLab CD Pipeline for Linux package management

Prabhat Singh — Tue, 28 Jun 2022 08:43:32 +0000

Good Day Everyone!

Hope you’re all doing well.

I’m writing this article to share my learnings while working on different projects involving multiple skills in Telecommunications and Information Technology domains.

Recent project has enabled me with the skills such as GitLab, Ansible, Docker, and Shell script.

Use case

Package management on multiple Linux machines using GitLab pipeline

Tools

Following DevOps tools were used in implementing the CD pipeline

GitLab — Version control, project repository, CD pipeline
GitLab-Runner — Trigger the pipeline stages
Linux Machines — Centos VMs for controller node and target nodes
Ansible — Implements changes on the target nodes
Shell Scripting — Execute playbooks, used in the pipeline stages

Architecture

Below is a high level architecture of the setup

Prerequisites

GitLab account where you can store your project and configure CD pipeline.
Gitlab-runner package downloaded and installed on the controller node, which is basically a Linux machine.

Gitlab-runner for the project has been registered on the controller node. Navigate to GitLab → Project → Settings → CI/CD → Runners. Scroll down to the steps where it shows the steps to register gitlab-runner on the controller node.

Register the runner on the controller node with the details mentioned above. For this exercise, shell executor for gitlab-runner is used to trigger the gitlab jobs on the target machines.

Ansible installed on the controller node. For testing purpose, inventory file consists of the entry for the localhost

targetvm1 ansible_host=127.0.0.1 ansible_user=root

SSH password-less connectivity established between the controller node (gitlab-runner user) and target nodes (any user)(linux machines)

Git is installed on your local machine to clone, pull, & push the changes in your local repository

Implementation

Clone the project from the Github on your local machine

Change to the directory and edit the variables.yml and provide values against the parameters mentioned

Commit the changes to your local repository and push to the remote repository

As soon as you push the changes, a pipeline will be triggered on GitLab. Navigate to GitLab → Project → CI/CD → Pipelines.
Stages include in the pipeline: prechecks, prebackup, activity, postbackup, postchecks.

Ansible playbooks can further be modified for different operating systems, system checks, system backup before and after the installation, upgrade, & removal of the packages.