Forem: Bill Hegazy

Controlling Procrastination and Get Your AWS Security Specialty Certificate

Bill Hegazy — Sun, 19 Mar 2023 12:00:00 +0000

Hey there! You might remember my previous post on controlling procrastination while preparing for the AWS Solution Architect Professional Certificate.

Well, guess what? I'm back with more tips to help you tackle another awesome AWS: exam AWS Certified Security - Specialty.

Photo by Magnet.me on Unsplash

Just like you, I've been a victim of procrastination while preparing for my AWS exams. But, Do not fear Bill is here! 😆 I've got your back yet again! Let's dive in and explore some tips that can help you control procrastination and successfully achieve this certification!

Say Goodbye to Social Media (Temporarily)

During your study period, consider uninstalling social media apps like Instagram, TikTok, and Facebook. Trust me, your sanity and focus will thank you for it.

Social media can be a huge time sink and a major source of distractions, so by eliminating them temporarily, you'll be able to better concentrate on your studies.

Get Familiar and Hands-On with AWS Security Services

To prepare for the AWS Certified Security - Specialty exam, it's crucial to gain hands-on experience with various AWS security services. This will help you understand how these services work in real-world scenarios and enable you to apply the concepts you'll face during the exam. Focus your study more on the following services and really understand the differences between them:

IAM and SSO
KMS and CloudHSM
Cloudtrail, Cloudwatch, and S3
WAF and Shield
Config, Security Hub, and GuardDuty
Inspector, Macie, and Detective
Cloudfront and ACM

I highly recommend checking the exam guide for the full list of services.

Courses by Adrian Cantrill are super helpful for gaining practical experience.

Practice Tests, Study, Rinse, and Repeat

Practice tests are lifesavers. Start by taking one to figure out your strengths and weaknesses. Study the areas you struggle with, then take another practice test. Keep repeating this process to refine your understanding and boost your chances of acing the exam.

I recommend Tutorials Dojo Practice tests as the explanation is very clear and helpful.

Set a Deadline and Book Your Exam

As I mentioned in my previous post, and would like to emphasize here that it's so important, scheduling your exam upfront is a great strategy for overcoming procrastination. Although it's scary to set a date without knowing when you'll be ready, it actually helps you allocate time to study.

It's OK to Reschedule Your Exam

Hey, no need to stress if you're thinking about rescheduling your exam because you're not quite ready. It's totally cool to give yourself some extra time to prepare. After all, the main goal is to pass the exam, right? So, take the time you need to make sure you've got this!

And if you need someone's permission to reschedule, consider this your official go-ahead from me. Just remember, your success and well-being are what matter most, so focus on getting well-prepared for the exam 😄

Conclusion

And there you have it! By following these extra tips, I hope you control your procrastination temporarily until you achieve the AWS Certified Security - Specialty exam. Remember, the journey might be challenging, but the rewards are worth it. So, keep pushing forward and believe in yourself. You've got this!

Like this post? Consider following me on Medium billhegazy.

If you have questions or want to reach out, add me on LinkedIn.

Terraform EKS module upgrade from v17.x to v18.x

Bill Hegazy — Fri, 22 Jul 2022 03:20:11 +0000

If you’re like me and you are using the awesome terraform-aws-eks module to manage your EKS clusters, then you should know that there are many breaking changes when upgrading the module version from v17.x to v18.x, in this guide I will share the steps that I took to ease the upgrade a little bit (we have more than 12 clusters with similar config) and I hope that this helps someone.

I highly recommend that you also read the official guide UPGRADE-18.0.md document

1) Change module version and variables

Check the variables changes here, as everyone is using a different configuration, in my case I have ended up with the following:

module "eks" {
  source = "terraform-aws-modules/eks/aws"

  # check releases for the latest version
  # https://github.com/terraform-aws-modules/terraform-aws-eks/releases
  version = "18.26.2"

  # Needed for EKS module Upgrade
  # https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/docs/UPGRADE-18.0.md#upgrade-from-v17x-to-v18x
  prefix_separator                   = ""
  iam_role_name                      = var.cluster_name
  cluster_security_group_name        = var.cluster_name
  cluster_security_group_description = "EKS cluster security group."

  # Add this to avoid issues with AWS Load balancer controller
  # "error":"expect exactly one securityGroup tagged kubernetes.io/cluster/xxx
  # https://github.com/terraform-aws-modules/terraform-aws-eks/issues/1986
  node_security_group_tags = {
    "kubernetes.io/cluster/${var.cluster_name}" = null
  }

  cluster_name                    = local.name
  cluster_version                 = local.cluster_version
  cluster_endpoint_private_access = true
  cluster_endpoint_public_access  = true

  cluster_addons = {
    coredns = {
      resolve_conflicts = "OVERWRITE"
    }
    kube-proxy = {
      resolve_conflicts = "OVERWRITE"
    }
  }

  cluster_encryption_config = [
    {
      provider_key_arn = aws_kms_key.eks.arn
      resources        = ["secrets"]
    }
  ]

  vpc_id = module.vpc.vpc_id

  # Rename subnets to subnet_ids
  subnet_ids = module.vpc.private_subnets

  # Rename node_group_defaults to eks_managed_node_group_defaults
  eks_managed_node_group_defaults = {
    ami_type       = "AL2_x86_64"
    instance_types = ["m5.large", "m5a.large", "t3.large", "m5.xlarge"]

    iam_role_attach_cni_policy = true

    update_config = {
      max_unavailable_percentage = 50
    }

    block_device_mappings = {
      xvda = {
        device_name = "/dev/xvda"

        ebs = {
          volume_size = "100"
          volume_type = "gp3"
          encrypted   = true
          kms_key_id  = aws_kms_key.ebs.arn
        }

      }
    }
    metadata_options = {
      http_endpoint               = "enabled"
      http_tokens                 = "required"
      http_put_response_hop_limit = 2
      instance_metadata_tags      = "disabled"
    }
  }
  # Rename eks_managed_node_groups to eks_managed_node_groups
  # the variables in the sub module also changed, so be sure to rename them!
  eks_managed_node_groups = {
    default = {
      name            = "default"
      use_name_prefix = true
      subnet_ids      = module.vpc.private_subnets
      desired_size    = 3
      max_size        = 10
      min_size        = 3

      labels = {
        environment = local.environment
        capacity    = "on_demand"
      }

      tags = local.tags
    },
    spot = {
      name            = "spot"
      use_name_prefix = true
      capacity_type   = "SPOT"
      instance_types  = ["m5a.xlarge", "m5.xlarge", "r5.xlarge", "r5a.xlarge"]
      subnet_ids      = module.vpc.private_subnets
      desired_size    = 3
      max_size        = 15
      min_size        = 3
      cluster_version = local.cluster_version

      labels = {
        environment = local.environment
        capacity    = "spot"
      }

      tags = local.tags
    }
    tags = local.tags
  }
}

If your iam_role_name variable is NOT the cluster_name, the get the cluster iam role name using aws cli:

export CLUSTER_NAME=bill-eks-test
aws-vault exec aws-prod -- aws eks describe-cluster --name $CLUSTER_NAME --output json | jq -r .cluster.roleArn | cut -d/ -f2

2) Do a `terraform plan` to see what changes

You should see a lot of resources change including for example IAM, node groups, and even cluster control plane replacement (we will fix that next step)

Since there are a lot of changes and we wanted to make the upgrade simple for us, the best approach is to remove existing node groups and related resources from terraform state, this suggestion also came from users in the related Github issue

3) Terraform state migration and deletion

Everyone setup is different so be careful when playing with Terrafrom state file, as your tf state might get corrupted, always backup the state file first!

# Backup terraform state first 
terraform state pull > tf_backup.tfstate
# Remove node_groups from tf state
terraform state rm 'module.node_groups'
# Rename the cluster iam role tf resource to new name
terraform state mv 'aws_iam_role.cluster[0]' 'aws_iam_role.this[0]'
# Remove policy attachment for node groups from tf state
terraform state rm 'aws_iam_role_policy_attachment.workers_AmazonEC2ContainerRegistryReadOnly[0]' 'aws_iam_role_policy_attachment.workers_AmazonEKSWorkerNodePolicy[0]' 'aws_iam_role_policy_attachment.workers_AmazonEKS_CNI_Policy[0]'
# Remove node groups security group from state
terraform state rm 'aws_security_group.workers[0]'
# Remove node groups aws_security_group_rule resources from tf state
terraform state rm $(terraform state list | grep aws_security_group_rule.workers)
# Remove cluster aws_security_group_rule resources from tf state
terraform state rm $(terraform state list | grep aws_security_group_rule.cluster)
# Remove IAM Role of node groups from tf state
terraform state rm 'aws_iam_role.workers[0]'

# If you are managing the aws-auth configmap using EKS module
# Then remove aws-auth configmap from tf state as now the module dropped the support
terraform state rm 'kubernetes_config_map.aws_auth[0]'

# If you use addons then remove from state as well
terraform state rm 'aws_eks_addon.kube_proxy' 'aws_eks_addon.vpc_cni' 'aws_eks_addon.coredns'
# Then import the new add ons to state
terraform import 'aws_eks_addon.this["coredns"]' cluster:coredns
terraform import 'aws_eks_addon.this["vpc-cni"]' cluster:vpc-cni
terraform import 'aws_eks_addon.this["kube-proxy"]' cluster:kube-proxy

4) Test `terraform plan` again

Make sure you DO NOT see the following

The cluster Must be replaced
Old node groups must be destroyed
Node groups related resources (sg, sg rules, old node iam) must be destroyed

You should see the following:

Some cluster policy changes.
New node group adding.
New node groups related resources.

5) Proceed with `terraform apply`

You can now start by applying the changes with terraform apply .

After the apply is complete, you should see new nodes joining the cluster and working as expected

6) Manually delete the old node group and related resources

Now you can go ahead and delete the old node group manually and pods will restart on new node groups 👍 🎉.

Other related resources should be manually deleted as well such as old node group security groups.

Over to you

Like this post? Consider following me on Medium billhegazy.

If you have questions or want to reach out, add me on LinkedIn.

Controlling Procrastination and Get Your AWS Solution Architect Professional Certificate

Bill Hegazy — Mon, 18 Apr 2022 01:38:40 +0000

Photo by Annie Spratt on Unsplash

Yes, I admit it, I have been procrastinating for a few months while preparing for my AWS SA Pro exam, but I controlled it and finally took the exam last February, in fact, I have been procrastinating on writing this post for 2 months now😅.

I wanted to share a few tips, that helped me to control procrastination while studying for the AWS certification exam.

This post will also serve as a reminder for my future self 🙂

Set a goal

Procrastinator or not, you should set some kind of goal for yourself.

Trust me this helped me to have some kind of plan. I formed the goal using the OKR framework

The goal doesn’t have to be perfect. Here is an example, so you can get started:

Objective	Key Results
Become an AWS Certified Solution Architect Professional by the end of Q1 2022	KR1: Study a minimum of 1 hour daily
	KR2: Pass at least 2 practice exams with an 80% score or higher
	KR3: Buy a study material and complete it 100%

Schedule exam date upfront

If you’re a professional procrastinator (like myself), you have to do this.
I know it’s scary to schedule the exam date, and not know when you will be ready, but, trust me, this will help you to set aside some time to study.

You can cancel or reschedule your exam (2 times only) up to 24 hours before your scheduled appointment without an extra fee.

Before scheduling the exam:

If you’re going for an online proctor exam, then do read PSI / Pearson Vue FAQ.
Take advantage of the ESL +30, if English is not your first language. To get the extra 30 minutes, you must request this before scheduling your exam.
Depending on your situation and time, you will need to figure out (approximately) how long it will take for you to complete your studies. I recommend starting by going through the video course curriculum and practice exams.

Take notes along the way

Notes will be your best friend for the next few months to remind you about important AWS exam points, AWS services that you have never used before, and most importantly remind you to keep going.

You might be wondering, what notes should you take? I will give you an example of what my notes looks like:

I use notion as my note-taking app, I recommend note-taking app that can sync to multiple devices

Review: Note/screenshot of important points from the instructor or anything I find important to remember.
Weak services: Summary of the AWS services that I’m weak at or never used before.
Real life usage: Where I gather improvements that I want to apply to the current AWS Infrastructure that I manage.

Find a study partner (if possible)

The AWS SA Pro is tough and long, your study partner will remind you that, you are not alone and to keep going. A study partner can also give you tips that they found out while studying. I was lucky enough to have a partner at my work, who happened to be studying for the AWS SA Pro and we did a weekly sharing together.

Be gentle to yourself

“Have no fear of perfection - you'll never reach it.” — Salvador Dali

If you don't feel ready yet, don’t feel bad about it, and remember that It’s OK to re-schedule the exam date. I had to re-schedule 2 times myself.

Go through the study materials, go through the notes and review the parts that you're weak at.

The key is to remember that anyone can do it, you might need to study harder or need more time than others, but in the end, you will get there!

Study materials:

I understand that you might be wondering about the study materials that helped me prepare.Here is what I recommend:

Adrian Cantrill: AWS SA Pro video course, Adrian's course is very detailed and well organized.
Tutorials Dojo Practice exam: Practice exams currently include 4 exam sets, the practice exams will help you prepare for the real one and also identify your weak points.
AWS Exam Readiness: Official Exam Readiness from AWS.

Why get AWS Certified

I asked myself the exact same question. I have been managing AWS for 7+ years now, and I proved that I’m more than capable when it comes to AWS cloud.

There are many benefits listed here, for me, it's simple: I love learning!

Getting AWS certified, was an opportunity to learn more, learn what I have done wrong, and learn what can I improve in the current AWS architecture.

Conclusion

I have learned a lot from studying for such a tough exam like AWS SA Pro, learning new AWS skills, and controlling procrastination.

I hope that the tips can help you to get started on your AWS SA Pro exam journey.

If I could do it, then anyone can!

Over to you

Which AWS exams are you currently interested in? What stopping you from preparing for the AWS exam?

Like this post? Consider following me on Medium billhegazy.

If you have questions or want to reach out, add me on LinkedIn.

Terraform in AWS

Bill Hegazy — Sun, 17 Apr 2022 09:49:38 +0000

Originally published at https://medium.com on 19 June 2021

Tools and best practices, that makes your terraform life easier on AWS.

1) aws-vault

Although it's not exactly specific for Terraform aws-vault is a must-use tool for Terraform in AWS, especially when you have multiple AWS accounts.

aws-vault stores IAM credentials in your Os's secure Keystore and generates temporary credentials to be used in shell.

Using aws-vault with Terraform to easily switch between AWS accounts and avoid hard-coding AWS profile in Terraform backend state code.

Install aws-vault

brew install --cask aws-vault

Usage Example

# Run simple aws command
aws-vault exec aws_example_account -- aws s3 ls
# Login to aws console using temporary credentials
aws-vault login aws_example_account
# Use with terraform
aws-vault exec aws_example_account -- terraform apply

2) tfenv

tfenv is Terraform version manager similar to rbenv.

Install tfenv

brew install tfenv

Usage Example

# List tf remote versions
tfenv list-remote
# Install tf version
tfenv install 0.11.15
# Use tf version
tfenv use 0.11.15

tfenv automatic version switching

Add .terraform-version file to automatically switch between different Terraform versions and control versions between accounts.

Example:

├── .
├── aws_prodution_account
    ├── resource_1
    │   ├── main.tf
    |   ├── variables.tf
    |   └── ...
    ├── resource_2
    │   ├── main.tf
    |   ├── variables.tf
    |   └── ...
    ├── .terraform-version
├── aws_staging_account
    ├── resource_1
    │   ├── main.tf
    |   ├── variables.tf
    |   └── ...
    ├── resource_2
    │   ├── main.tf
    |   ├── variables.tf
    |   └── ...
    ├── .terraform-version
├── README.md
└── ...

3) pre-commit

Using pre-commit framework with terraform repository, will help your code to be kept clean, formated, updated document and checked for tf security issues (optional with tfsec) before committing and pushing the code to git source.

Install precommit and related tools

brew install pre-commit gawk terraform-docs tflint coreutils checkov terrascan

Install the pre-commit hook globally

DIR=~/.git-template
git config --global init.templateDir ${DIR}
pre-commit init-templatedir -t pre-commit ${DIR}

Initialize git repo with terraform hooks

cd your_terraform_git_repo
git init # if new repo
cat <<EOF > .pre-commit-config.yaml
repos:
- repo: git://github.com/antonbabenko/pre-commit-terraform
  rev: <VERSION> # Get the latest from: <https://github.com/antonbabenko/pre-commit-terraform/releases>
  hooks:
    - id: terraform_fmt
    - id: terraform_docs
EOF
pre-commit install
# Test pre commit
pre-commit run --all-

Now, whenever you run git commit on terraform repo, pre-commit will run the hooks

Auto generate Terraform docs with pre-commit

Using terraform-docs with terraform modules

cd terraform_example_module
cat <<EOF > README.md
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
lines here will be replaced by terraform_docs when pre-commit runs
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
EOF
pre-commit run --all-files

4) tfsec

Want static analysis for your terraform code to help spot potential security issues? then all you need is tfsec.

Install tfsec

brew install tfsec

Usage Example

cd terraform_folder
tfsec .

Add tfsec to your pre-commit config

Add terraform_tfsec hook to .pre-commit-config.yaml
Example:

repos:
- repo: git://github.com/antonbabenko/pre-commit-terraform
  hooks:
    - ...
    - id: terraform_tfsec

Ignoring some tfsec rules

You may wish to ignore some warnings from tfsec. you can simply add a comment containing tfsec:ignore:<RULE> to the offending line in your templates.

For example, to ignore an open security group rule:

resource "aws_security_group_rule" "my-rule" {
    type = "ingress"
    #tfsec:ignore:AWS006
    cidr_blocks = ["0.0.0.0/0"]
}

Other best practices:

Use official Terraform public module for AWS, official public module are well written and tested ( don't re-invent the wheel).
Limit access to Terraform state S3bucket access, encrypt it and enable versioning.
Avoid storing secrets when creating a resource as Terraform state will store secrets plain-text, at least create a temporary password and change it after the resource is created.

# Simple Example
resource "random_password" "db-password" {
  length  = 16
  special = false
}
resource "aws_db_instance" "default" {
  engine           = "mysql"
  engine_version = "5.7"
  instance_class       = "db.t3.micro"
  name                 = "mydb"
  username             = "foo"
  password             = random_password.db-password.result
  ...
}

New World Game Architecture

Bill Hegazy — Thu, 13 Jan 2022 07:47:21 +0000

AWS re:invent 2021 was fascinating, with a lot of interesting announcements. One of the most interesting parts for me was the last 10-15 minutes of Dr. Werner Vogels Keynote, when he shared the architecture of the online MMORPG Game New World.

New World is a Massively Multiplayer Online Role-Playing Game (MMORPG) developed by Amazon Games It’s an open-world online game where you create a custom character, level up, craft your weapons and armors, etc...

I have been playing New World since it was launched in September, 2021. Although there are some bugs in the game 😅, it is fun with much, much stuff to do.

This part of the keynote was interesting to me because I have been managing services in AWS for a few years now, so I wanted to share some points from the keynote.

New World lives fully in the cloud (AWS)

Aeternum, which is the name of the game’s world, is built out in 14 smaller regions (Windsward, Everfall, Brightwood, etc...).

REPS

The game has 4 EC2 instances in each world (each new world server). Amazon Games call those EC2s “remote entry points” or “REPS”.

Those EC2s act as Application routers (Nginx/Haproxy maybe?) and are the only public-facing instances. This is where security and resilience are handled.

There was no mention of any ALB or NLB, so my guess is:

Those EC2 instances are behind NLB, but why do the instances have to be public if they are behind a public NLB?
Those EC2s are behind Route53 using weighted, latency, or geo route. But why did Amazon Games not leverage AWS ALB/NLB in this case and make the instances private?

Hubs

Hubs handle the computing of a portion of the world. There are 7 Hubs (EC2s).

Hubs are basically where the core game backend servers are running.

In a single world (server), the hubs process:

2,500 Players
Around 7,000 A.I Entities
100,000s Objects

Overlaid on top of the Aeternum map is a series of grids. This is where the 7 hubs work together and spread the load.

Each hub picks up 2 pieces of the grid, but the 2 pieces are not in order, so if you move from grid 1 to grid 2 in the map, you will move from hub to hub (instance to instance).

All the hubs’ EC2 instances are stateless, which is good, meaning that if a few hubs fail, they can always be replaced quickly.

Shared Instance Pool

The instance pool is where all the single gameplay happens. For example, running an expedition or any other session-based mode in the game. Each session-based game mode will claim one or more EC2 instances, then when the session-based game is over (completed expedition), the EC2 instances return to the shared instance pool to be used by other players.

Database

The stateless hubs store the game state and write everything to AWS DynamoDB, which is around 800k writes every 30 seconds.

Data Analytics

New World logs 23M events/minute, pushed to AWS Kinesis into AWS S3 and then analyzed by AWS Athena, etc...

Multiple Amazon Games team members can use the data collected, the data analyst can discover which wolves have been followed the most, or which paths are most traveled. These data allow game designers to figure out how players enjoy the game and change the game in real-time based on the data.

Non-Core Gameplay

All the non-core gameplay services such as creating a character, creating a company, trading, are running as Serverless microservices using AWS Lambda and AWS API Gateway.

Conclusion

There are many MMORPG Games that live in AWS or other cloud providers, but I have never come across something similar to New World, which is a scalable and simple architecture. As Dr. Werner said.

“This truly a MMORPG game born in the cloud and it would only have been possible to actually run this in the cloud”

Reference

AWS re:Invent 2021 - Keynote with Dr. Werner Vogels

Over to you

What do you think of the New World Game Architecture? and what would you change if you were the one who maintained this massive game?
Like this post? Consider following me on Medium
billhegazy, If you have questions or would like to reach out, add me on LinkedIn.

Forem: Bill Hegazy

Controlling Procrastination and Get Your AWS Security Specialty Certificate

Say Goodbye to Social Media (Temporarily)

Get Familiar and Hands-On with AWS Security Services

Practice Tests, Study, Rinse, and Repeat

Set a Deadline and Book Your Exam

It's OK to Reschedule Your Exam

Conclusion

Terraform EKS module upgrade from v17.x to v18.x

1) Change module version and variables

2) Do a terraform plan to see what changes

3) Terraform state migration and deletion

4) Test terraform plan again

Make sure you DO NOT see the following

You should see the following:

5) Proceed with terraform apply

6) Manually delete the old node group and related resources

Over to you

Controlling Procrastination and Get Your AWS Solution Architect Professional Certificate

Set a goal

Schedule exam date upfront

Before scheduling the exam:

Take notes along the way

Find a study partner (if possible)

Be gentle to yourself

Study materials:

Why get AWS Certified

Conclusion

Over to you

Terraform in AWS

Tools and best practices, that makes your terraform life easier on AWS.

1) aws-vault

Install aws-vault

Usage Example

2) tfenv

Install tfenv

Usage Example

tfenv automatic version switching

3) pre-commit

Install precommit and related tools

Install the pre-commit hook globally

Initialize git repo with terraform hooks

Auto generate Terraform docs with pre-commit

4) tfsec

Install tfsec

Usage Example

Add tfsec to your pre-commit config

Ignoring some tfsec rules

Other best practices:

New World Game Architecture

New World lives fully in the cloud (AWS)

REPS

Hubs

Shared Instance Pool

Database

Data Analytics

Non-Core Gameplay

Conclusion

Reference

Over to you

2) Do a `terraform plan` to see what changes

4) Test `terraform plan` again

5) Proceed with `terraform apply`