Forem: Efosa Oviawe

MID INTERNSHIP TASK: Building devopsfetch for Server Information Retrieval and Monitoring.

Efosa Oviawe — Tue, 23 Jul 2024 02:29:33 +0000

Introduction

Welcome to the documentation for DevOpsFetch, a powerful yet straightforward monitoring tool designed to meet the requirements of HNG Internship Task 5. At first glance, the task of creating a system to monitor various aspects of a server environment may seem simple. However, achieving this requires close attention to detail and a comprehensive understanding of different system components.

DevOpsFetch encapsulates this challenge, providing an intuitive solution that monitors:

Active network ports
Docker container statuses
Nginx configurations and domains
User login activities
System activities within specified time ranges

This documentation will guide you through the installation, configuration, and usage of DevOpsFetch. With clear instructions and practical examples, you'll find that what initially seemed complex can be managed efficiently with the right approach and tools. Let's dive in and simplify your monitoring tasks with DevOpsFetch.

Installation and Configuration

Prerequisites

Ensure your system has the necessary dependencies:

net-tools
nginx
docker.io

Installation Steps

Clone and Run the Installer Script:

git clone https://github.com/Oviawe007/HNG-Task-5.git
cd HNG-Task-5.git

Make the script executable:

chmod +x devopsfetch.sh
chmod +x install.sh

Run the script:

sudo ./install.sh

2. Check the Installation:

Verify that the devopsfetch.sh script is installed in > /opt/devopsfetch/ and is executable.
Confirm that a symbolic link to devopsfetch.sh is created at > /usr/local/bin/devopsfetch.
Ensure the devopsfetch.service file is created in > /etc/systemd/system/.

3. Check the Service:

sudo systemctl status devopsfetch.service

Configuration

The service is configured to run devopsfetch.sh every hour by default. You can modify the service configuration in

/etc/systemd/system/devopsfetch.service if needed.

Usage Examples

Run devopsfetch.sh as a root user with the appropriate flags to fetch the desired information.

Show Help:

sudo ./devopsfetch.sh -h

Get Active Ports:

sudo ./devopsfetch.sh -p

Get Specific Port Info:

sudo ./devopsfetch.sh -p [PORT]
# Example: sudo devopsfetch -p 4369

Get Docker Info:

sudo ./devopsfetch.sh -d

Get Specific Docker Container Info:

sudo ./devopsfetch.sh -d [CONTAINER_NAME]
# Example: sudo ./devopsfetch.sh -d my_container

Get Nginx Info:

sudo ./devopsfetch.sh -n

Get Specific Nginx Domain Info:

sudo ./devopsfetch.sh -n [DOMAIN]
# Example: sudo ./devopsfetch.sh -n example.com

Get User Logins:

sudo ./devopsfetch.sh -u

Get Specific User Info:

sudo ./devopsfetch.sh -u [USER]
# Example: sudo ./devopsfetch.sh -u ubuntu

Get Activities in Time Range:

sudo ./devopsfetch.sh -t [START_TIME] [END_TIME]
# Example: sudo ./devopsfetch.sh -t "2024-07-22 10:00:00" "2024-07-22 11:00:00"

Logging Mechanism

Log Rotation Configuration The installation script sets up log rotation for > /var/log/syslog to ensure logs do not consume excessive disk space. The log rotation configuration is defined in > /etc/logrotate.d/devopsfetch.

View Logs

To retrieve and view logs, you can use the tail -f command:

tail -f /var/log/devopsfetch.log

This command will display the logs for the DevOpsFetch service, allowing you to monitor its activities and debug if necessary.

Summary

DevOpsFetch provides a comprehensive solution for monitoring various system and application metrics, including active ports, Docker containers, Nginx configurations, and user logins. By following the installation steps and using the provided command-line flags, you can effectively gather and analyze the information needed to maintain and troubleshoot your system. The logging mechanism ensures that you have a record of all activities for further analysis and auditing.

Leave your questions and feedback in the comments.
Thank you.

Provision, Authentication, and Configuration on AWS Using Ansible

Efosa Oviawe — Sun, 21 Jul 2024 10:03:07 +0000

Introduction

Ansible, a powerful configuration management and deployment tool, offers robust capabilities for managing infrastructure. One critical aspect of server management is the ability to gracefully shut down systems based on specific criteria. This article delves into leveraging Ansible's conditional logic and powerful modules to automate the shutdown of Ubuntu instances. By combining Ansible's flexibility with precise conditions, we'll explore how to create tailored shutdown procedures that align with your infrastructure's need.

We'll examine the core concepts, provide practical code examples, and discuss best practices for implementing Ansible-based server shutdown automation.

Key areas we will cover include:

Understanding Ansible's conditional logic
Effectively utilizing the command module for shutdown operations
Incorporating shutdown criteria based on factors like operating system, hostname, and uptime
Best practices for ensuring reliable and efficient shutdown processes

By the end of this article, you'll gain a solid understanding of how to harness Ansible to automate server shutdown, enhancing your infrastructure management capabilities.

Requirements

The below requirements are needed on the host that executes this module as stated in Ansible docs.

python >= 3.6
boto3 >= 1.26.0
botocore >= 1.29.0

Tasks

Create three(3) EC2 instances on AWS using Ansible loops
- 2 Ubuntu Instances
- 1 Amazon Linux Instance
Set up passwordless authentication between Ansible control node and newly created instances.
Automate the shutdown of Ubuntu Instances only using Ansible Conditionals

Step One - Installations

Install Ansible on Mac

brew install ansible

Install boto3
pip install boto3

Step Two - Setup Vault

Whenever you are dealing with Ansible and you have API tokens, passwords, secret keys and access keys or any secured template, make sure you put them in your ansible-vault and protect them with your password.

Create a password for vault:
openssl rand -base64 2048 > vault.pass

Note: Ensure you have configured your AWS User credential (secret key and access key) that have AWSEC2FullAccess as the role

Add your AWS credentials using the below vault command
ansible-vault create group_vars/all/pass.yml --vault-password-file vault.pass

Step Three - Write Ansible Playbook

Ensure you are in the directory where you have initiated the project. create a file ec2_creation.yml, copy and paste the below code:

---
- name: Create EC2 Instances 
  hosts: localhost
  connection: local
  become: false

  vars:
    instance_types:
      - ami: ami-07c8c1b18ca66bb07 # Replace with your desired AMI
        instance_type: t3.micro
        count: 2
        distro: ubuntu
      - ami: ami-0249211c9916306f8 # Replace with your desired AMI
        instance_type: t3.micro
        count: 1
        distro: centos

  tasks:
    - name: Create EC2 instances
      amazon.aws.ec2_instance:
        key_name: ec2_mac_key  # Replace with your key pair name
        image_id: "{{ item.ami }}"
        instance_type: "{{ item.instance_type }}"
        region: eu-north-1
        count: "{{ item.count }}"  # Specify count directly
        ebs_optimized: true
        tags:
          Name: "{{ item.distro }}-server"
      loop: "{{ instance_types }}"
      loop_control:
        loop_var: item

Save and run the below command:
ansible-playbook ec2_creation.yml --vault-password-file vault.pass

Setup Passwordless Authentication Between Ansible Control Node And Newly Created Instances

Using Public Key
ssh-copy-id -f "-o IdentityFile <PATH TO PEM FILE>" ubuntu@<INSTANCE-PUBLIC-IP>

ssh-copy-id: This is the command used to copy your public key to a remote machine.
-f: This flag forces the copying of keys, which can be useful if you have keys already set up and want to overwrite them.
"-o IdentityFile ": This option specifies the identity file (private key) for the connection. The -o flag passes this option to the underlying ssh command.
ubuntu@: This is the username (ubuntu) and the IP address of the remote server you want to access.

You have to do it for the three IPs.

Automate The Shutdown of Ubuntu Instances Only Using Ansible Conditionals

Create another yml file called ec2_shutdown.yml and paste the below code:

---
- hosts: all
  become: true

  tasks:
    - name: Shutdown ubuntu instances only
      ansible.builtin.command: /sbin/shutdown -t now
      when:
       ansible_facts['os_family'] == "Debian"

Also, create an inventory file called inventory.ini in which all your servers' IP are listed.

Run the command
ansible-playbook -i inventory.ini ec2_shutdown.yml --vault-password-file vault.pass

Outputs

Additional Considerations:

Specific Conditions: You can customize the when condition based on your requirements.
Graceful Shutdown: Consider using shutdown -h +5 to allow processes to gracefully terminate before shutdown.
Error Handling: Implement error handling mechanisms to capture potential issues during shutdown.
Idempotency: Ensure the playbook is idempotent by using appropriate conditional logic or idempotent modules.
Alternative Shutdown Methods: Explore using the reboot or systemdmodules if needed.

Example with Multiple Conditions:

when:
  - ansible_os_family == "Debian"
  - ansible_hostname in ['ubuntu_instance1', 'ubuntu_instance2']
  - ansible_uptime > 3600  # Shutdown after 1 hour of uptime

Conclusion: Automating AWS Infrastructure with Confidence

In this article, we've explored leveraging Ansible's powerful capabilities to automate key infrastructure tasks on AWS. We covered:

Provisioning EC2 instances: We demonstrated efficient instance creation using Ansible loops, catering to distributions like Ubuntu and Amazon Linux.
Passwordless authentication: We explored securing access from your control node to the newly created instances using public key authentication.
Conditional shutdown automation: We implemented an Ansible playbook that selectively shuts down Ubuntu instances based on the operating system family.

By combining these techniques, you can significantly streamline your AWS infrastructure management. Customize the playbooks further to match your specific requirements and security protocols.

Ansible offers a vast range of modules and functionalities for managing your infrastructure. As you delve deeper, you can explore advanced features like:

Inventory management: Enhance inventory management to dynamically discover and target your AWS resources.
Configuration management: Automate server configuration and application deployment to ensure consistency across your infrastructure.
Cloud-specific modules: Utilize Ansible's extensive support for managing various cloud platforms like AWS, Azure, and GCP.

By embracing Ansible's automation capabilities, you can free yourself from repetitive tasks and focus on building and scaling your cloud infrastructure with greater efficiency and control.