Forem: sathish kumar

Learn the basics of git

sathish kumar — Mon, 10 Apr 2023 06:11:15 +0000

Before we dive deep into the technical details, let's list what we are going to learn from this article.

What exactly is Git
Visual representation of Git
Who uses git
Git commit workflow
Basics commands of git

What exactly is Git

Git is the version control software that tracks code changes and manages the history of changes. Some of the examples of version control software allow users to revisit earlier versions of the file changes, undo changes, compare changes between versions, merge the changes from different users, and much more.

One of the use cases where git version control software can be used, for instance: Is in an organization where many developers make code and they need a tool to integrate/collaborate the changes without overriding other developer code changes and release the software in a stable manner or unbroken code.

Visual representation of Git

Let's say you initialize a web application project and like to create an HTML page where you are adding a header, body, and footer by making commits in git repository in different point in time (commit generally means a snapshot of the code).

Assume your team lead says, let's remove the footer. And we can easily do that by going back to the previous commit in our repo.

Who uses Git

Software developers: Git is widely used by software developers to manage their source code and collaborate with other developers on projects.
System administrators: Git can be used by system administrators to manage configuration files and scripts across multiple servers.
Designers: Git can also be used by designers to manage design assets and collaborate with other designers on projects.
Project managers: Git can be used by project managers to track changes to project files and collaborate with team members on project development.
Researchers: Git is increasingly being used by researchers to manage code, data, and analysis scripts for their research projects.
Writers: Git can also be used by writers to track changes to written documents and collaborate with other writers on projects.

Git Commit Workflow

Let's try to understand how this git commit workflow works by breaking down these concepts working directory, staging area, and repository.

Working directory

Is a place where we add, delete, update, and save files.

Staging area

Select a group of files that we prepare to make a commit

Commit

Make a snapshot of the file changes that were added previously in the staging area which creates a commit id and store it in the git repository. Then later we can track the changes by referring to commit history.

Basic commands of Git

Git Init

Initializing a Git repository: To start using Git, you need to initialize a Git repository in your project directory. You can do this by running the command in your project directory.

git init

Make changes

Once you have initialized the Git repository, you can start working on files. Add some HTML and CSS files in the project working directory index.html, navbar.html, and sample.css

Git status

Checking the status of the repository: You can check the status of the repository at any time using the below command. This will show you which files have been modified or added to the repository.

git status

Adding files to the staging area:

You can add files using the below command to add specific files or all files in the current directory.

git add <filename> or git add .

Committing changes to git repo:

After you have added files to the repository, you can commit the changes using the command. This creates a snapshot of the changes you made to the files.

git commit -m "commit message"

Viewing commit history:

You can view the commit history of the repository using the below command. This will show you a list of all the commits that have been made, along with their commit messages and other details.

git log

Branching:

Git allows you to create branches in the repository, which are essentially copies of the code base that you can work on independently. You can create a new branch using the command.

git branch <branchname>

Merging:

Once you have made changes on a branch and want to merge those changes back into the main branch, you can use the command.

git merge <branchname>

These are some of the basic Git commands and concepts. Git is a powerful tool with many features, but these basics should be enough to get you started with version control.

Conclusion

These are just some of the basic Git commands and concepts that you will need to get started with version control. Git is a powerful tool with many features, and it can take some time to master all of them. However, by learning these basics, you will be well on your way to becoming a proficient Git user.

How to use Terraform in AWS multi-account via AzureDevOps pipelines

sathish kumar — Fri, 16 Sep 2022 05:48:18 +0000

In this session, we will see how to provision infrastructure using Terraform in AWS cloud multi-accounts, cross-account IAM roles, and AzureDevOps pipelines to automate the deployment.

Let's break down the topics below

Architecture diagram
AWS multi-account creation setup
Terraform infrastructure code
AzureDevOps pipelines as code to automate deployment

Architecture diagram

From the above architecture diagram, let's list down the workflow.

Workflow 1

We need to create AWS multi accounts, basically 3 accounts based on this example. The below steps need to be completed manually by creating via AWS console or AWS CLI commands.

- Management/DevOps account

In this account we will manage the IAM user(automation user for ex: vscode) and we will generate the secret key and access id for this user which will be used by Terraform
Assign an Administrator role or least privilege role by attaching only STS policy, so that the user 'vscode' can assume a role in trusting account like Development or Production account

- Development Account

Create a role for ex: ManageInDevAsAdmin with Administrator permission or least privilege that is required to manage this account and mention trusted account ARN number of Management/DevOps account

- Production Account

Create a role for ex: ManageInProdAsAdmin with Administrator permission or least privilege that is required to manage this account and mention trusted account ARN number of Management/DevOps account

Once these above steps are completed, Our terraform automation able to use those credentials for vscode and assume the IAM role to provision the resources in those different AWS accounts.

Workflow 2 and 3
Azure DevOps tool has a complete solution to manage source code repository, pipelines to automation deployment, and much more. In this use case, we use both repository and pipelines.

Azure DevOps Repository to keep the source code for Terraform infrastructure provisioning and Azure pipelines to execute the terraform commands.

Pre-requisites:

We need to upload a secure file in the Azure DevOps portal which will be used in pipelines for the Terraform to use the secret key and access id in the profile section.

Let's use azure-pipelines.yml pipelines as a code to automate the deployment using Terraform. The code has stages for Dev and Prod and it contains terraform commands.

The credentials file looks like below.

Secure file upload in AzureDevOps portal

azure-pipelines.yml file content.

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- main

pool:
  vmImage: ubuntu-latest

stages:    
  - stage: Dev
    displayName: Dev
    jobs:
      - job: plan
        pool:
          vmImage: ubuntu-latest
        displayName:  Terraform Dev Init and Apply
        steps:
        - task: DownloadSecureFile@1
          name: aws_credentials
          displayName: 'Download AWS Credentials'
          inputs:
            secureFile: 'credentials'
        - task: PowerShell@2
          displayName: 'New directory to keep the aws credentials'
          inputs:
            targetType: 'inline'
            script: |
              New-item -Path "~/" -Name ".aws" -ItemType "directory"
        - task: CopyFiles@2
          displayName: 'Copy aws credentials to home directory '
          inputs:
            SourceFolder: "$(Agent.TempDirectory)"
            contents: 'credentials'
            targetFolder: '~/.aws/'
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform init -reconfigure -backend-config="dev-backend.conf"'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: Terraform init for plan
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform plan --var-file="dev-app.tfvars" -var createdBy="dev-pipeline" -var tags="asd" -out main.tfplan'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: "Terraform Plan"
      - job: approve
        dependsOn: plan
        condition: and (succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
        pool: server
        displayName: Wait for approval
        steps:
        - task: ManualValidation@0
          timeoutInMinutes: 15 # task times out in 15 mins
          inputs:
            notifyUsers: saasproject86@outlook.com
            instructions: 'Please validate the build configuration and resume'
            onTimeout: 'reject'
      - job: apply
        dependsOn: approve
        pool:
          vmImage: ubuntu-latest
        displayName:  Terraform Dev Init and Apply
        steps:
        - task: DownloadSecureFile@1
          name: aws_credentials
          displayName: 'Download AWS Credentials'
          inputs:
            secureFile: 'credentials'
        - task: PowerShell@2
          displayName: 'New directory to keep the aws credentials'
          inputs:
            targetType: 'inline'
            script: |
              New-item -Path "~/" -Name ".aws" -ItemType "directory"
        - task: CopyFiles@2
          displayName: 'Copy aws credentials to home directory '
          inputs:
            SourceFolder: "$(Agent.TempDirectory)"
            contents: 'credentials'
            targetFolder: '~/.aws/'
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform init -reconfigure -backend-config="dev-backend.conf"'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: Terraform init for apply
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform apply --var-file="dev-app.tfvars" -var createdBy="dev-pipeline" -var tags="asd" --auto-approve'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: "Terraform Apply"
  - stage: Prod
    displayName: Prod
    jobs:
      - job: plan
        pool:
          vmImage: ubuntu-latest
        displayName: Terraform Prod Init and Apply
        steps:
        - task: DownloadSecureFile@1
          name: aws_credentials
          displayName: 'Download AWS Credentials'
          inputs:
            secureFile: 'credentials'
        - task: PowerShell@2
          displayName: 'New directory to keep the aws credentials'
          inputs:
            targetType: 'inline'
            script: |
              New-item -Path "~/" -Name ".aws" -ItemType "directory"
        - task: CopyFiles@2
          displayName: 'Copy aws credentials to home directory '
          inputs:
            SourceFolder: "$(Agent.TempDirectory)"
            contents: 'credentials'
            targetFolder: '~/.aws/'
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform init -reconfigure -backend-config="prod-backend.conf" -no-color'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: Terraform init for plan
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform plan --var-file="prod-app.tfvars" -var createdBy="prod-pipeline" -var tags="asd"  -out main_prod.tfplan -no-color '
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: "Terraform Plan"
      - job: approve
        dependsOn: plan
        condition: and (succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
        pool: server
        displayName: Wait for approval
        steps:
        - task: ManualValidation@0
          timeoutInMinutes: 15 # task times out in 15 mins
          inputs:
            notifyUsers: saasproject86@outlook.com
            instructions: 'Please validate the build configuration and resume'
            onTimeout: 'reject'
      - job: apply
        dependsOn: approve
        pool:
          vmImage: ubuntu-latest
        displayName: Terraform Prod Apply
        steps:
        - task: DownloadSecureFile@1
          name: aws_credentials
          displayName: 'Download AWS Credentials'
          inputs:
            secureFile: 'credentials'
        - task: PowerShell@2
          displayName: 'New directory to keep the aws credentials'
          inputs:
            targetType: 'inline'
            script: |
              New-item -Path "~/" -Name ".aws" -ItemType "directory"
        - task: CopyFiles@2
          displayName: 'Copy aws credentials to home directory '
          inputs:
            SourceFolder: "$(Agent.TempDirectory)"
            contents: 'credentials'
            targetFolder: '~/.aws/'
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform init -reconfigure -backend-config="prod-backend.conf" -no-color'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: Terraform init for apply
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform apply --var-file="prod-app.tfvars" -var createdBy="prod-pipeline" -var tags="asd" --auto-approve -no-color'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: "Terraform Apply"

Workflow 4 and 5
Let's walk through the stages in the pipeline.

Dev stage job tasks

Download a secure file

This task will download the secure file into the agent's temporary path

Copy secure file

This task will copy the file secure file from the temporary path to the agent home directory because we referred to this path in terraform provider section.

terraform providers.tf file content

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
  backend "s3" {}
}

provider "aws" {
  region                   = var.aws_region
  shared_credentials_files = ["~/.aws/credentials"]
  profile                  = "vscode"
  assume_role {
    role_arn = var.aws_assume_role_arn 
  }
}

3. Terraform init

For terraform initialization, we will use the S3 bucket as the backend state.

Pre-requisites:

Create an S3 bucket for Development and Production AWS account to maintain the terraform state for those accounts
Create dynamodb for Development and Production AWS account to maintain the terraform lock for those accounts
Role name should be present in backend configuration

dev-backend.conf file content


bucket         = "example-tf-dev-states"
key            = "example-dev-state/terraform.tfstate"
region         = "ap-southeast-2"
dynamodb_table = "example-tf-dev-locks"
encrypt        = true
profile        = "vscode"
role_arn = "arn:aws:iam::<AWS Development env account number >:role/ManagementInDEVAdminRole"

prod-backend.conf file content


bucket         = "example-tf-prod-states"
key            = "example-prod-state/terraform.tfstate"
region         = "ap-southeast-2"
dynamodb_table = "example-tf-prod-locks"
encrypt        = true
profile        = "vscode"
role_arn = "arn:aws:iam::<AWS Development env account number >:role/ManagementInProdAdminRole"

Terraform init command:
terraform init -reconfigure -backend-config="dev-backend.conf"

Here we are using -reconfigure with the backend config, this will correctly set the terraform state
s3 bucket for the corresponding AWS account based on the environment configuration file.

4. Terraform plan
Terraform plan is used to compare the state of the infrastructure and the current desired state from the code changes, It presents the changes that are going to happen in our infrastructure.

Terraform plan command:
terraform plan --var-file="dev-app.tfvars" -var createdBy="dev-pipeline" -var tags="asd" -out main.tfplan

Here we need to pass Development environment-specific values

dev-app.tfvars file content

# Common
project = "example-project"

# General 
aws_region  = "ap-southeast-2"
profile     = "vscode"
environment = "dev"
aws_assume_role_arn = "arn:aws:iam::<Development AWS account number>:role/ManagementInDEVAdminRole"

# Terraform Backend State
backend_s3_bucket      = "example-tf-dev-states"
backend_dynamodb_table = "example-tf-dev-locks"
backend_s3_key = "example-dev-state/terraform.tfstate"

# VPC
create_vpc           = true
cidr                 = "10.123.0.0/16"
instance_tenancy     = "default"
enable_dns_hostnames = true
enable_dns_support   = true
create_igw           = true

# Subnet
create_subnet         = true
subnet_cidr_block = "10.123.1.0/24"
subnet_map_public_ip_on_launch = true
subnet_availability_zone = "ap-southeast-2a"

Once the plan is completed, we can see the result.

4. Azure DevOps Terraform plan review and approve or reject
This approval gate helps us to review the terraform plan and understand the changes we are going to make in our infrastructure.

5. Terraform apply
In Azure DevOps pipeline, we need to define the download, copy a secure file, and terraform init task because we are using a new job definition.

Terraform apply command
terraform apply --var-file="dev-app.tfvars" -var createdBy="dev-pipeline" -var tags="asd" --auto-approve

Here we need to pass auto-approve because we already reviewed the terraform plan.

Once the terraform apply is completed, we can see the AzureDevOps pipeline status.

NOTE: For the Prod stage as well, we will be using the same concept and we need to pass different environment variable files.