Forem: Rafaf Tahsin

bash conditions

Rafaf Tahsin — Mon, 09 Oct 2023 04:25:55 +0000

If any command runs successfully inside if conditional expression then if treats it as true.

if print; then echo "foo"; else echo "bar"; fi

2 There's a command called test to evaluate conditional expression.

if test $a -ge $b; 
then 
    echo "a is bigger"; 
else 
    echo "b is bigger"; 
fi

see the test command ? It evaluates the conditional expression and return true / false base on the evaluation.

test is later replaced with [.

if [ $a -ge $b ]; 
then 
    echo "a is big"; 
else 
    echo "b is big"; 
fi

Yes, the command is [ and it starts evaluating the expression until it gets ]. You can check it yourself with which [ or even man [. [ is basically another representation of test command.

There's some limitations of using [ or test. For example, it can't evaluate &&, ||. So here comes [[ with improvements.

if [[ $a > $b || $a == $b ]]; then echo "a is big"; else echo "b is big"; fi

You can also read more about the differences between [ and [[ from here.

There's no ternary operator. But there's a hack ...

[[ $a == $b ]] && echo "Equal" || echo "Not equal"

How to import existing dynamodb in terrafrom

Rafaf Tahsin — Wed, 16 Aug 2023 11:28:44 +0000

Well I was having difficult time importing dynamodb and trying with arn, but it was db name that needs to be mentioned while importing.

terragrunt import aws_dynamodb_table.dynamodb_resource dynamodb_table_name

How to restrict AWS access from office Ips only

Rafaf Tahsin — Fri, 11 Aug 2023 17:24:10 +0000

Use Source IP condition in IAM policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Action": [
                "*"
            ],
            "Resource": [
                "*"
            ],
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "202.74.246.XXX/32"
                    ]
                }
            }
        }
    ]
}

Migrate an EC2 instance from one AWS account to another

Rafaf Tahsin — Sun, 23 Jul 2023 05:45:50 +0000

Create an AMI of the instance

aws ec2 create-image \
    --instance-id <i-0123456789abcdef> \
    --name "AMI-Image-Name" \
    --description "AMI Image of EC2 created from Source AWS Account to Destination AWS Account" \
    --region <region-id> \
    --profile <optional-profile>

Above command will output an Image ID. We will share this image with the destination AWS Account. Image creation might take some time. When the image is created run following command to give permission to destination AWS account.

Give AMI Permission to destination AWS Account

aws ec2 modify-image-attribute \
    --image-id <AMI-ID-retrieved-from-above-command> \
    --region <region-id> \
    --launch-permission "Add=[{UserId=<destination-aws-account-id>}]" \
    --profile infolytxdev

Reference:

DRY your terraform code using dynamic block

Rafaf Tahsin — Mon, 17 Jul 2023 05:49:19 +0000

Without dynamic block a security group resource looks like this

resource "aws_security_group" "instance_security_group" {
  ingress {
    # TLS (change to whatever ports you need)
    from_port   = 80
    to_port     = 80
    protocol    = "TCP"
    cidr_blocks = [ "0.0.0.0/0" ] # add a CIDR block here
  }

  ingress {
    # TLS (change to whatever ports you need)
    from_port   = 443
    to_port     = 443
    protocol    = "TCP"
    cidr_blocks = [ "0.0.0.0/0" ] # add a CIDR block here
  }

  ingress {
    # TLS (change to whatever ports you need)
    from_port   = 22
    to_port     = 22
    protocol    = "TCP"
    cidr_blocks = [ "${var.office_IP}/32" ]
  }
}

Above code can be dried with terraform dynamic block to look like this.

locals {
  ingress_rules = [
    { from_port = 80, to_port = 80, cidr_blocks = [ "0.0.0.0/0" ] },
    { from_port = 443, to_port = 443, cidr_blocks = [ "0.0.0.0/0" ] },
    { from_port = 22, to_port = 22, cidr_blocks = [ "${var.office_IP}/32" ] }
  ]
}

resource "aws_security_group" "instance_security_group" {
  dynamic "ingress" {
    for_each = local.ingress_rules
    iterator = i
    content {
      from_port   = i.value.from_port
      to_port     = i.value.to_port
      protocol    = "TCP"
      cidr_blocks = i.value.cidr_blocks
    }
  }
}

More about terraform dynamic block - https://developer.hashicorp.com/terraform/language/expressions/dynamic-blocks

aws cred to env

Rafaf Tahsin — Wed, 12 Jul 2023 06:17:38 +0000

Sometimes it's tidy to export aws credentials manually. It can be done by the following script

#!/bin/bash

echo "# Environment for AWS profile '$1'"
export AWS_ACCESS_KEY_ID=$(aws configure get aws_access_key_id --profile $1)
export AWS_SECRET_ACCESS_KEY=$(aws configure get aws_secret_access_key --profile $1)
export AWS_DEFAULT_REGION=$(aws configure get region --profile $1)

source the script and you are done.

Note:

Saving script as script.sh and calling it with ./script.sh won't work. Need to source it like source script.sh or . script.sh.

Inspired by https://gist.github.com/mjul/f93ee7d144c5090e6e3c463f5f312587

Any better way to calculate AWS CodeArtifact Usage?

Rafaf Tahsin — Mon, 03 Jul 2023 09:44:29 +0000

I needed to know consumed total storage in AWS CodeArtifact. I didn't get this from AWS Console or straight forward aws cli. So I came up with this dirty way ...

#!/bin/bash

export AWS_PROFILE=infolytxdev

domains=$(aws codeartifact list-domains | jq -r ".domains[].name")
total_artifactory_size=0

for d in $domains; do
  repos=$(aws codeartifact list-repositories-in-domain --domain $d | jq -r ".repositories[].name")
  for r in $repos; do
    packages=$(aws codeartifact list-packages --domain $d --repository $r | jq -r ".packages[].package")
    for p in $packages; do
      packageversions=$(aws codeartifact list-package-versions --package $p --domain $d --repository $r --format pypi | jq -r ".versions[].version")

      for pv in $packageversions; do
        sizes=$(aws codeartifact list-package-version-assets --domain $d --repo $r --format pypi --package $p --package-version $pv | jq -r ".assets[].size")
        echo $d" "$p" "$r" "$pv" "$sizes >>size.tb

        total=0

        for size in $sizes; do
          sum=$(($total + $size))
        done

        echo "Package Size: $sum" >>size.tb
        total_artifactory_size=$((total_artifactory_size + $sum))

      done
    done
  done
done

echo "Total Artifactory Size: $total_artifactory_size"

Any better way to accomplish this?

How to restrict a user to use single region in AWS

Rafaf Tahsin — Tue, 20 Jun 2023 08:51:35 +0000

IAM policy to restrict user to a single region

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Action": [
                "*"
            ],
            "Resource": [
                "*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": [
                        "us-west-1"
                    ]
                }
            }
        }
    ]
}

How to enforce MFA in AWS - Part I - Create User, Role & Policy

Rafaf Tahsin — Tue, 20 Jun 2023 08:34:13 +0000

In this tutorial I will demonstrate How to enfornce 2FA in both AWS CLI and AWS Console. Later in the tutorial I will share a script to quickly authenticate with MFA and use generated token.

1. At first we will create a user who has only access to sts:AssumeRole with MFA condition. We can create the user manually or with the following terraform script.

resource "aws_iam_user" "mfa_user" {
  # path = "/user_path_if_necessary/"
  name = "aws_cli_user"
}

data "aws_iam_policy_document" "sts_assume_role_policy_docuement" {
  statement {
    actions   = ["sts:AssumeRole"]
    resources = ["*"]
    effect    = "Allow"
    condition {
      test     = "Bool"
      values   = ["true"]
      variable = "aws:MultiFactorAuthPresent"
    }
  }
}

resource "aws_iam_policy" "sts_assume_role_policy" {
  name   = "sts_assume_role_policy"
  policy = data.aws_iam_policy_document.sts_assume_role_policy_docuement.json
}

resource "aws_iam_policy_attachment" "sts_assume_role_policy_attachment" {
  name       = "sts_assume_role_policy_attachment"
  policy_arn = aws_iam_policy.sts_assume_role_policy.arn
  users      = [aws_iam_user.mfa_user.name]
}

resource "aws_iam_access_key" "mfa_user_keys" {
  user = aws_iam_user.mfa_user.name
}

resource "aws_iam_user_login_profile" "mfa_user_console_login_profile" {
  user = aws_iam_user.mfa_user.name
}

2. You can get the credentials with the following output.tf output file

output "mfa_user_access_key" {
  value = aws_iam_access_key.mfa_user_keys.id
}

output "mfa_user_secret_key" {
  value     = aws_iam_access_key.mfa_user_keys.secret
  sensitive = true
}

output "mfa_user_console_password" {
  value = aws_iam_user_login_profile.mfa_user_console_login_profile.password
  sensitive = true
}

3. Now Create a Virtual device for mfa_user from AWS Console.

4. We will now create the role that will be assumed by the user. For the purpose of the tutorial I've attached an AWS managed Administrator policy with the role.

resource "aws_iam_role" "iam_admin_role_with_mfa_restriction" {
  name               = "admin_mfa"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Principal" : {
          "AWS" : "arn:aws:iam::${var.aws_account_id}:root"
        },
        "Action" : "sts:AssumeRole",
        "Condition" : {
          "Bool" : {
            "aws:MultiFactorAuthPresent" : "true"
          }
        }
      }
    ]
  })
  managed_policy_arns = ["arn:aws:iam::aws:policy/AdministratorAccess"]
}

Now the user is ready, you can have a look at the full terraform code here

Lets jump into Part II of the tutorial where we will discuss about how to access aws using MFA.

How to enforce MFA in AWS - Part II - Using `aws` CLI and terraform with MFA

Rafaf Tahsin — Tue, 20 Jun 2023 08:32:44 +0000

Part I => How to enforce MFA in AWS CLI - Part I

Here in Part II we will discuss how to access aws using MFA

`aws` cli

1. Use aws configure --profile mfa_user to configure mfa user in aws cli. Get the user credentials from terraform outputs. Sensitive outputs can be retrieved with terraform output <sensitive_output_data>. After configuration your ~/.aws/credentials should contains this

[mfa_user]
aws_access_key_id = ABCD$$$$$$$$
aws_secret_access_key = knklvdf093487jps/df\$$$$$$$$$$$$$$$$$$$$$$

2. Lets create role profile

To create a role profile add admin_role profile section in ~/.aws/config.

[profile admin_role]
role_arn = arn:aws:iam::<account_number>:role/admin_mfa
source_profile = mfa_user
mfa_serial = arn:aws:iam::<account_number>:mfa/mfa

You can get role_arn from terraform output. Get MFA Serial from AWS Console.

3. Now you can use aws cli with admin_role profile

aws s3 ls --profile admin_role

terraform

If you enable MFA, configuring terraform gets a bit hacky. You can use following script to automate this process.

#!/usr/bin/env bash

echo "totp is $1"

ROLE_ARN="arn:aws:iam::<account_number>:role/admin_role"
MFA_ARN="arn:aws:iam::<account_number>:mfa/mfa"

aws sts assume-role \
    --role-arn $ROLE_ARN \
    --role-session-name session-one \
    --serial-number $MFA_ARN \
    --token-code $1 > /tmp/sts.json

aws configure set aws_access_key_id $(cat /tmp/sts.json | jq -r '.Credentials.AccessKeyId') --profile terraform
aws configure set aws_secret_access_key $(cat /tmp/sts.json | jq -r '.Credentials.SecretAccessKey') --profile terraform
aws configure set aws_session_token $(cat /tmp/sts.json | jq -r '.Credentials.SessionToken') --profile terraform
aws configure set region "ap-southeast-1" --profile terraform

rm /tmp/sts.json

you can use this script with ./auth.sh <6 Digit MFA Code>. This will configure aws profile named terraform. You can configure terraform aws provider with with this profile.

provider "aws" {
  profile = "terraform"
}

AWS Console

Use Switch role button to switch to admin role after signing in to AWS Console.

You can't create aws virtual mfa with terrafrom

Rafaf Tahsin — Sun, 18 Jun 2023 12:42:40 +0000

It's just an FYI. Though theoretically you can create an aws virtual mfa device through terraform but you can't enable it or assign it to a user.

The process to create virtual mfa has two steps from aws-cli perspective.

Creating the mfa device

aws iam create-virtual-mfa-device --virtual-mfa-device-name BobsMFADevice --outfile C:/QRCode.png --bootstrap-method QRCodePNG

Enabling the mfa device with auth Code

aws iam enable-mfa-device \
    --user-name Bob \
    --serial-number arn:aws:iam::210987654321:mfa/BobsMFADevice \
    --authentication-code1 123456 \
    --authentication-code2 789012

You can complete both steps using aws-cli. But using terraform you only can create the mfa device, you can't enable it or assign it to a user.

Yes, I understand it's legit from terraform's perspective. As you need to put auth code every time to enable a virtual mfa device. That's not what we want to do with every terraform apply.

But the scope to create the virtual mfa doesn't mean anything without having the opportunity to assign it.

Looking forward how terraform solves this in future.