Forem: Fumiaki Ueno

Highlights from AWS re:Invent 2022 Dr. Werner keynote

Fumiaki Ueno — Mon, 26 Dec 2022 14:31:52 +0000

re:Invent, the most exciting AWS event in a year has just ended in this December.

I was very impressed with the keynote speech of Dr. Werner Vogels. Let's summarize it and think about what the software architecture should look like.

Imagine a world that is ordered, synchronized, one by one

In a synchronized world you have to order one food at a time. when you want to eat a burger and fries, you say "Burger", and you have to wait for the waiter's "OK". You can't order like "Burger and Fries". The cook fries one potato stick at a time for fries.

AWS re:Invent 2022 - Keynote with Dr. Werner Vogels

The world is asynchronous

In a real world, you can order multiple foods at a time like "Burger and Fries". Of course the cook can fry multiple potato sticks at once. The world is also event-driven. In a restaurant, there are many events such as order, food is ready, finish eating, and waiters and cooks act triggered by these events.

Software Architecture

In synchronous systems, components are tightly coupled. One component fail leads a fail of overall system. Therefore, it's difficult to add a new component and evolve systems.

On the other hand, asynchronous systems make it easy to add new components. Everything is loosely coupled and you cann add a new component without changing other components. This makes it easy for the entire system to evolve.

How do we build asynchronous systems with AWS?

This ia an example of asynchronous, AWS serverless architecture.　Each component is simple but can look complex when combined. You also need AWS knowledge to take advantage AWS.

Learn more about Serverlesspresso

How do we start a serverless architecture? A new service, AWS Application Composer (Preview), will help with this. By drawing an architecture diagram visually, Application Composer generate infrastructure as code (IaC). The code follows best practices such as S3 Bucket encryption. Code and architecture are synchronized, making them easy to manage and share among members.

Many things to do when you build

When you build on AWS, you have to manage lot more than just building. For example, managing issues, building pipelines, setting up IDE, creating git repository etc. Amazon CodeCatalyst (Preview) launch new project and sample codes, CI/CD pipeline, issue management tool, member management tool, etc. You can manage all of these things from CodeCatalyst console.

The power of composing

In UNIX, pipes(|) make it easy to compose small components with a standard interface and format. you can build bigger application with composing many small components.

example of pipes

cat access.log | grep "/gateway/allapi" | grep "\" 500" | rev | cut -d " " -f4 | rev | sort | uniq

Can we do this with AWS Services? Yes, we can do this with Amazon EventBridge Pipes. It makes it easy to build event-driven applications. Before this service was available, you need to integrate code to connect AWS services. Now you can connect services without integration code.

Listen, learn, improve

AWS always develops services based on customer feedback. Some customers would like to do Map Reduce with Step Functions, not Kafka and EMR. AWS built AWS Step Functions Distributed Map This allow you to process very large amount of data and support a maximum concurrency of up to 10,000 executions in parallel.

AWS re:Invent 2022 - Keynote with Dr. Werner Vogels

Simulate everything

Simulations makes the impossible possible in a digital system. you can experiment safely and at low cost because simulations don't impact the real world. AWS SimSpace Weaver were announced in Keynote with Adam Selipsky. It make you easy to simulate spaces without managing infrastructure. Using simulation improve your system.

The world is asynchronous. The world and the universe are fragile and fault-tolerant, resilient, robust. We can learn a lot from the real world and apply the principles of nature to the system.

Summary

The world is asynchronous and the system should be asynchronous too
New services are announced
- AWS Application Composer
- Amazon CodeCatalyst
- Amazon EventBridge Pipes
- AWS Step Functions Distributed Map

When we build on AWS especially when using serverless services, we need to prepare a lot and there is a learning curve. Many new services will reduce our management and learning cost.

Easy setup for EC2 stop jobs with Amazon EventBridge Scheduler

Fumiaki Ueno — Mon, 14 Nov 2022 15:20:06 +0000

A great update Amazon EventBridge Scheduler announced!

https://aws.amazon.com/jp/blogs/compute/introducing-amazon-eventbridge-scheduler/

What is Amazon EventBridge Scheduler?

EventBridge Scheduler is a more powerful scheduling feature than the previous EventBridge rules.EventBridge Scheduler has the following characteristics.

You can create up to 1 million schedules per account
Support 1,000s of TPS
Support many AWS Service APIs(Over 270 services and over 6,000 API Actions)
You can create one-time schedules

Supporting many APIs means we can create many automated jobs without Lambda, code, etc, just with EventBridge Scheduler.

EC2 stop jobs with Amazon EventBridge Scheduler

Instance Scheduler was commonly used for stopping EC2 instances.It consists of Lambda and EventBridge rules.

Now it can be created only by EventBridge Scheduler. So let's create it.

A running instance was prepared in advance.

1. Create an IAM Role

We need to create an IAM Role for EventBridge Scheduler. In this case, Scheduler only stop EC2 instances, so we grant ec2:StopInstances action. Create the following IAM Role.

IAM Role Name: SchedulerEC2Stop
Trusted Policy:



{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "scheduler.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

IAM Policy: ```json

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ec2Stop",
"Effect": "Allow",
"Action": "ec2:StopInstances",
"Resource": "*"
}
]
}


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d54ho0tiwxpabpyn1ks5.png)

### 2. Create a Schedule

Moving to EventBridge Schedules page and create schedule. (Not a rules page.)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1gfojn6fsbve6af2vafr.png)

Set a schedule name and configure the schedule.If you want to stop instances daily 21:00, set as follows.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b95bjz134p0qch0u5z68.png)

You can optionally set Timezone, Start date, End date.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zh4a2yw2i2jmkz5lu0ng.png)

Select Amazon EC2 and select `stopInstances` action.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ejnlzz3jw6couacl3wmx.png)



![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vxaqbwk5pqm7il29mckb.png)


You only need to enter your instance ID. Multiple IDs can also be specified.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tesn79ik68zkimyhf2kq.png)

In the Settings phase, only an IAM role need to be set.　In this case, other settings are OK with default values.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kdjhi6mvfxo5eaike6pi.png)

Review settings and create schedule.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mvvomtppmt18zeb9u49y.png)

### 3. Check EC2 instance is stopped.

Now it's ready. When the scheduled time comes, you can confirm that EC2 instance is stopped.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2bv0rs08eoxk81x3z7eq.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jkqvrkc8lhe9noh5ssd5.png)

## conclusion

We could set up EC2 stopping job easily.　This feature can be applied to implement operational jobs such as ECS RunTask　and changing alarm configuration of CloudWatch without programming code.

If you want to know more detail of EventBridge Scheduler, Check the [AWS Documents](https://docs.aws.amazon.com/scheduler/latest/UserGuide/what-is-scheduler.html) and [AWS Blog post](https://aws.amazon.com/jp/blogs/compute/introducing-amazon-eventbridge-scheduler/).