Forem: luan vu

ArrayList và LinkedList trong Java

luan vu — Sat, 23 Apr 2022 09:38:59 +0000

List là một collection được sử dụng rất phổ biến trong Java, đặc biệt là ArrayList và LinkedList. Đã bao giờ bạn tự hỏi sự khác nhau giữa hai list này và các tình huống sử dụng của chúng. Cùng tìm hiểu dưới đây.

List trong Java

List trong Java, hay interface java.util.List được mô tả như là một

Danh sách có thứ tự
Có thể truy xuất phần tử ở một vị trí bất kỳ trong list

ArrayList

Ưu điểm của array là tiết kiệm bộ nhớ và tốc độ truy xuất ngẫu nhiên (tìm một phần tử ở vị trí bất kỳ) nhanh. Tuy nhiên, đổi lại thì array phải có độ dài cố định khi bạn khởi tạo. Nếu khai báo nhiều quá sẽ tốn bộ nhớ, ít quá thì không đủ để lưu trữ.
Để thuận tiện cho cả việc sử dụng và tốc độ truy xuất, ArrayList ra đời để khắc phục các nhược điểm đó nhưng vẫn kế thừa ưu điểm trong việc truy xuất dữ liệu của array.
ArrayList làm điều đó bằng cách sử dụng một dynamic array để lưu trữ các phần tử. Khi bạn thêm hoặc xóa phần tử, ArrayList sẽ tính toán để tạo một array mới có kích thước phù hợp và copy toàn bộ phần tử từ array cũ sang array mới.
Khởi điểm array sẽ rỗng và có một biến integer cho biết kích thước thực của ArrayList, gọi là size, bằng 0.

Array này là một static array rỗng và share cho tất cả các hàm khởi tạo ArrayList để giảm thiểu bộ nhớ sử dụng nếu bạn mới chỉ khởi tạo chứ chưa dùng.
Khi bắt đầu thêm mới một phần tử, elementData sẽ được gán bằng một mảng mới có độ dài thường là bằng 10.

Khi tiếp tục thêm phần tử, elementData sẽ được lấp đầy

Giả sử elementData đã được lấp đầy (10 phần tử)

Khi ta lại thêm phần tử, elementData sẽ lại được khởi tạo bằng một array mới có size tăng thêm 50% size gốc

Cách thức tăng của elementData là newSize = oldSize * 1.5
Tương tự:
10 -> 15
15 -> 22
22 -> 33
33 -> 49
49 -> 73

Trong trường hợp delete, giả sử delete tại index 5, thực hiện:

Copy item từ vị trí 5 + 1 tới cuối và đè vào index 5
Giá trị ở vị trí cuối sẽ bị thừa ra, đặt giá trị thành null
Giảm size của list đi 1

Giờ ta có một list chỉ còn 10 phần tử.
Nhận xét:

Với cách liên tục tạo mới array và copy array, ArrayList sẽ kém trong việc thêm mới hoặc xóa phần tử, sẽ ngày một tệ hơn khi số phần tử tăng lên.
Truy xuất ngẫu nhiên trong ArrayList chính là truy xuất theo index trong array nên tốc độ rất nhanh.
Cách delete của ArrayList sẽ hoạt động kém nếu bạn delete với những index nhỏ, tệ nhất sẽ là index = 0. Giả sử list của bạn có size = 100. Nếu bạn delete phần tử đầu tiên, ArrayList sẽ phải copy 99 phần tử.
array trong ArrayList sẽ ngày một to ra khi bạn thêm phần tử, thêm mỗi lần 50% khi thiếu chỗ lưu và sẽ không giảm đi khi bạn delete.

LinkedList

LinkedList là một danh sách liên kết kép ( Doubly-linked list), thay vì dùng array để lưu dữ liệu như ArrayList, LinkedList lại dùng cấu trúc danh sách liên kết để lưu trữ.
Cách thức hoạt động như sau:
LinkedList bao gồm:

size để lưu dung lượng thực của list, giống ArrayList
Node first để lưu phần tử đầu tiên của list. Ban đầu chưa có sẽ là null
Node last để lưu phần tử cuối cùng của list. Ban đầu chưa có sẽ là null
Mỗi node sẽ chứa data thực của node, gọi là item, một node kế tiếp của nó, gọi là next và một node đứng ngay trước nó, gọi là prev

Khởi điểm, LinkedList sẽ bao gồm 2 node first và last đều mang giá trị null

Khi thêm phần tử đầu tiên, thay thế node first bằng phần tử đó

Khi thêm phần tử tiếp theo, thay vào node last và liên kết giữa node first và node last. Nay list đã có tối thiểu 2 node

Cứ như vậy, khi tiếp tục insert, dù là ở cuối danh sách hay đầu danh sách, LinkedList chỉ việc liên kết các node lại với nhau

Tương tự khi thêm ở giữa list thì list cũng chỉ cần đơn giản là tạo ra node mới, thay đổi liên kết 2 node bên cạnh node mới thêm vào

Khi cần delete, LinkedList cũng thay đổi liên kết 2 node bên cạnh node vừa bị xóa

Khi cần lấy ra một phần tử ở index bất kỳ, không như ArrayList, LinkedList phải đi dần dần từng node một từ first hay last phụ thuộc vào index ở gần bên nào hơn.

Giả sử bạn có một list với 1000 phần tử. Khi bạn cần lấy ra phần tử có index = 500, LinkedList sẽ phải duyệt tới toàn bộ 500 phần tử để có thể lấy ra cái bạn muốn, điều đó là rất tệ.

Nhận xét:

Việc tìm phần tử ở vị trí ngẫu nhiên của LinkedList kém hơn ArrayList, đặc biệt với index ở trung tâm
Việc thêm hoặc xóa node trong LinkedList rất dễ dàng nhưng sẽ tệ dần nếu bạn insert/delete càng về trung tâm của list.

ArrayList vs LinkedList

Từ những nhận xét ở trên, các bạn hoàn toàn có thể rút ra so sánh giữa ArrayList và LinkedList.
Cá nhân mình, chưa bao giờ dùng LinkedList, vì các lý do sau:

1. Chúng ta đều sử dụng những list có kích thước nhỏ.

Khi list có kích thước nhỏ thì tốc độ mở rộng của dynamic array không hề thua kém so với LinkedList.
Dưới đây là benchmark tốc độ insert với các kích thước khác nhau.

Không có quá nhiều khác biệt

2. Chúng ta có xu hướng sử dụng list để duyệt và tìm kiếm nhiều hơn so với insert từng phần tử một.

Việc duyệt bằng array chắc chắn là vượt trội hơn so với cách duyệt qua từng phần tử của LinkedList

3. Có nhiều cách để tạo ra 1 ArrayList chứa nhiều phần tử mà vẫn khắc phục được nhược điểm của ArrayList

Tạo ArrayList từ Collection khác: new ArrayList<>(Collection<> e);
Thêm nhiều phần tử một lúc bằng list.addAll(Collection<> e);
Định số phần tử trước khi insert: new ArrayList<>(int initialCapacity)
Dùng Stream API (Stream API sử dụng list.addAll)

4. LinkedList tốn nhiều bộ nhớ để lưu trữ

Giả sử có một list các integer, với size = 1000, và size = 1000_000, dung lượng cần để lưu trữ như sau:

Cách tính một ArrayList khi add lần lượt 1000 integer sẽ bao gồm:

object header = 12
size = 4
modCount = 4
elementData reference = 4
elementData header = 12
elementData size = 4
elementData item = 16 * 1234

Total 19784 byte
Với LinkedList:

object header = 12
size = 4
modCount = 4
Node first reference = 4
Node last reference = 4
Object alignment gap = 4
Node size = 12 (object header) + 4 (item reference) + 16 (Integer) + 4 (next reference) + 4 (prev reference) = 40
All node= node size * 1000 = 40000

Total 40032 byte

Bạn có thể thấy là LinkedList tốn dung lượng gấp đôi so với ArrayList. Ngoài ra array luôn thân thiện với Garbage Collection hơn so với các Node của LinkedList.

5. LinkedList insert/delete tốt nhưng chưa chắc đã nhanh.

Giả sử bạn cần insert hoặc delete ở giữa list. Trước khi có thể insert/delete, nó phải duyệt qua một nửa số phần tử của list, điều mà LinkedList làm tệ nhất

Một số mẹo khi dùng với list

findFirst hay findAny

Có người thích dùng findFirst, có người lại findAny, cái nào đúng hơn?
Với sequential stream và List trong Java là một list có thứ tự thì findFirst hay findAny cho ra kết quả như nhau và tốc độ như nhau trong mọi trường hợp.
Nhưng đối với parallel stream, findAny có thể trả ra kết quả khác với findFirst. Hoặc bạn dùng nó với Set thì 2 hàm cũng cho ra kết quả khác nhau.

Tìm kiếm một phần tử trong list

Trước Java 8, tôi thường code như dưới đây

Nhưng đến nay, hãy code như sau:

Không if-else, không for-loop

Kiểm tra tất cả phần tử thỏa mãn điều kiện

Kiểm tra có ít nhất một phần tử thỏa mãn điều kiện

Sort list theo một field

Sort theo ascending

Sort theo descending

Delete các phần tử thỏa mãn điều kiện với removeIf

Gọn hơn so với cách dùng iterator.remove.

Reference

Java Object Size: Estimating, Measuring, and Verifying via Profiling - DZone Java
ArrayList (Java Platform SE 8 )
LinkedList (Java Platform SE 8 )

Originally published at https://luanvv.com.

From Zero to PSM I, less than 24 hours!

luan vu — Sat, 22 Jan 2022 10:07:05 +0000

All the knowledge you need to pass the certification exam, encapsulated in the so-called Scrum Guide, 19 pages, which you can read at https://www.scrumguides.org/scrum-guide.html. So, firstly, you need to read it.

Scrum Guide - 10 hours

I don't think reading 19 pages will make it easy to pass the exam unless you can memorize these 19 pages exactly every word. I think you should read it many times; some people even have to read up to 10 times. I recommend only reading up to 5 times because it's useless to read once you don't understand.

Reading often does not mean reading continuously without stopping; this is not chanting or mantras! It will help if you read for the first time to understand. Then watch the course (I will talk about it later), then go back to read concepts you don't understand. Then when doing the Practice test(talk later too), you should also go back to read the places you don't understand. And finally, you should make a note section that summarizes the guide's content, and of course, you have to re-read it. In short, I calculate that I read about five times, each time takes 1 hour (3 minutes/page is already relatively slow).

Fact: I only read it once; when I took notes, the other four times, I listened to it, taking advantage of the time to wash the dishes.

Online course - 2 hours

If you are a person who has no concept of Scrum, just reading the theory will only cause headaches. So, I recommend taking more online courses, in my opinion about $ 10, 3 Practice Test lessons and 2 hours of your life for the course is highly worth it.

The course is here https://www.udemy.com/course/scrum-master-certification-preparation-mock-exam-questions-psm-i/

The content of the course is not outside the Scrum Guide. However, the course will give you more visual examples and provide some tips & tricks for the exam. I consider this course the most critical key, especially for someone who has no idea about Scrum (like me).

Note: you can skip the live webinar, as it's not relevant to the test.

Fact: I also took advantage of the time to wash the dishes to watch the entire 2 hours of the course

Practice test - 10 hours

Practice tests are also significant. Knowledge is only 19 pages, so the question set of the real exam cannot be boundless. According to my experience, about 60% of the questions in the exam are pretty close to the Practice tests. In addition, doing many practice tests will help you get used to the speed of the test and practice fast thinking.

When I took the exam, it only took me 40 minutes for 80 questions; the remaining 20 minutes I spent reviewing 7--8 questions.

Some tests you can refer to:

https://mlapshin.com/index.php/scrum-quizzes/sm-learning-mode/ (Free). You can try Real Mode, but there is no need because the topic here is quite simple.
Open Assessments (Free). The owner's topic should ultimately be at the same level as the real test. However, they only gave 30 questions and did it in 30 minutes.
3 Practice tests in the course I have introduced above

After learning the above knowledge, the practice tests are not tricky; I usually only take about 40 minutes for each test with over 80 questions; the rest is to review. So I keep counting about 90 minutes at a time. When doing a particular sentence, you should practice looking at the keywords of the sentence and the method of exclusion. Also, your English shouldn't be too bad. For example, I have didn't know the word "prohibit" means "forbidden," so when I took the test, I got one wrong sentence. So, improve your vocabulary in practice tests.

You should score above 80% in Practice tests. In the course I mentioned above, there were three tests; I did it repeatedly, a total of about five times. I only did the article on https://mlapshin.com once because it was too easy. Open Assessments also only do it once because I'm pretty sure all of them.

When reviewing the test, never try to remember the question and the answer. Please read questions, answers, and explanations carefully; refer to the Scrum Guide for understanding. You will get stuck on puns during the real exam if you just try to memorize them. I WAS SHOCKED when I took the actual test because they had about 30% of the questions much more complicated than the Practice test. Those questions had slightly changed the wording, the answers also sounded different from the Scrum Guide, but they are never outside the Scrum Guide. Therefore, understanding is critical.

When you are confident, it's time to take the test.

Fact: I can't take advantage of this while washing the dishes because my hands are covered with soap. I usually take advantage before going to bed or free afternoon, even in some boring meetings.

Exam, 1 hour 10 minutes

Below are the details of the exam. To see the latest updates, you should visit the homepage at https://www.scrum.org/professional-scrum-master-i-certification#Details

Fee: $150 per attempt
Passing score: 85%
Time limit: 60 minutes
Number of Questions: 80
Format: Multiple Choice, Multiple Answer, and True/False
Difficulty: Intermediate
Language: English only
Scrum Resources
Required course: None
Recommended courses: Professional Scrum Foundations or Professional Scrum Master
PSM Subject Areas
Practice Assessment: Scrum Open
Ways to Learn More to help you prepare
Passwords have no expiration date but are valid for one attempt only

The exam is online testing; you can comfortably use everything at home. However, the number of questions is quite large (80 questions), the time is (too) short (60 minutes), and the score is very high (85%); every mistake, no matter how small, comes at a price, so keep focus.

A considerable disadvantage of online exams is the speed of the network. If your network is slow or your computer is dead, it's a disaster. Don't forget the notes I mentioned above; it will be convenient to look up. Another thing is that when you click Next, Previous, pay special attention; during the page load, you may click again and choose the wrong answer again. Or if your hands are shaking too much, it's easy to reload the page or close the tab/browser by accident, I don't know if it's okay, but it's going to be a problem. Be careful with each touch.

To register for the test is quite simple. You register an account on https://www.scrum.org/, buy Assessment, a password will be sent to your email, and you can start taking the test immediately with the password you just received.

Result

So I have summarized the PSM I am learning in just 23 hours and 10 minutes. It took me a week to study and take the exam because I didn't wash the dishes 24 hours in a row. My friend has taken only four days. With only two days of continuous learning, the total time could be less than 23 hours.

However, writing articles is how I do it, but $150 is yours to spend, and this is not a speed contest, so don't be too hasty to lose your money. Take the test when you are ready. Good luck.

How to Pass the AWS Solutions Architect Professional Exam

luan vu — Sat, 22 Jan 2022 09:53:23 +0000

You are looking to take the AWS Certified Solutions Architect — Professional certification exam. Today I will help you with some tips & tricks that I learned during my professional exam preparation two years ago.

How I PASSED the New AWS Solutions Architect Professional?

Another guide to the certification exam, probably reading the White Paper, watching videos on YouTube, taking that course, etc. No! Such articles are present on the internet a lot. I can get you some:

Those articles are already too many, and adding one of mine is nothing more. Are the other people correct? Right. But to do it is not as easy as it sounds. The exam depends on your ability to learn and your background knowledge. If you study for four months, you will die (I have looked for a whole year, what a shame), but you could still don't pass no matter an entire year's preparation.

What is the standard learning path?

In general, The path to passing the AWS certification exam includes (don't miss anything):

1. Read White papers

There are hundreds of White papers, some with thousands of pages. Can you read it all? No matter how many you read, how much are you able to remember?

I just checked, now there are 314 White Paper & Guides; you can check them at AWS Whitepapers & Guides.

You should read all the essential White Papers mentioned in those articles above.

2. Read FAQs

AWS | FAQs

FAQs are also important. You should read all.

3. Watch AWS re:Invent, This Is My Architecture in YouTube

If you don't have much chance to try services architectures (like me), these videos are beneficial. They show you how a service should be used in actual projects (the exam too, of course) and how a group of services interact in architecture to resolve problems.

Amazon Web Services

4. Take some online/offline courses

It is the easiest to access. Many (most) AWS courses are available, both paid and paid. AWS also has some Exam Readiness (Free) to guide you through the exam questions

Exam Readiness: AWS Certified Solutions Architect --- Professional

There are many types of courses, but they are not enough in general. Especially some of the default courses require you to have a good knowledge of AWS and the overview for you. If you stick to them only, failure is inevitable.

I found a precious course on Udemy. It could be more expensive than usual, but the author deserves it.

https://www.udemy.com/share/104sE03@v25ocWwp5mS6EczCcLEoR6xjbdtVk0GPohkBLMVwUaLf4HB5bd4GAH_1dxW_J10=/

Fun Fact: The first time I took the exam, It hit me in the face with a service called Amazon Mechanical Turk, which didn't appear in any of the White Papers, FAQs, or AWS Console. I think it is a question that has not been scored, so don't worry if AWS gives you such a surprise.

5. Practice

Those learning materials are certainly informative, but it's hard to remember them all if you just read them and watch them do it. Practicing helps you understand better reflect better. Never skip practice any service.

6. Learn English

It's serious. It depends on the certification level you take the exam. With Professional, I took the exam, and all questions are very long; printing must also be over 50 pages. As I read and thought of the answer, it was indeed a terrible sprint. For test takers whose native language is not English (non-English speakers or English as a Second Language), AWS allows you to request accommodations, whereby you get a bonus of 30 minutes each time, so take advantage of it.

Many of you are probably shocked and don't dare to study, but that's the truth. This test is challenging, requiring hundreds of hours of study and practice. You should have terrible determination to make it through. Fortunately, Amazon will only test you on a few essential services. If you know (to the point of mastering) them, you have achieved over 80%. And if you don't understand these services and lose points, then strange services you never touch will be a lifesaver, but it is boundless knowledge. That is why AWS Solutions Architect Professional is touched, it is difficult to understand something deeply, and if it is broad, it is too wide to understand it all.

Tip & Trick

I'm not a person who can remember well. I tried to learn by going from service to service and memorizing them. According to statistics, I have read at least 47 full books about AWS, about 10--20 White Papers. However, the more I try to remember, the more I forget, learning 10--20 services, overlooking the first ones, reading a lot of books also forgetting quite a bit. So, I chose to study on my own.

Service classification

AWS has already categorized it for you at https://aws.amazon.com/products/

Service classification is critical. It helps you know what you face and how you deal with it. If you encounter a storage service, you need to know security, capacity, availability, and reliability. If you use databases, remember it is RDBMS or NoSQL, cache, or Big data.

It also helps you to remember the service better. There are various service names you have never heard before, so classification helps you have some idea about it. For instance, EC2 Auto Scaling or Elastic Load Balancer sounds reminiscent, but what do you think Neptune's purpose? Universe? Snowball, Glacier, etc.

Based on Well-Architected Framework's Five Pillars

Well-Architected Framework provides guidelines to help customers deploy systems on AWS. The framework offers five main pillars:

Security
Reliability
Performance Efficiency
Cost Optimization
Operational Excellence

When encountering any service, you need to think about these pillars.

1. Security

AWS places great emphasis on security across all architects. There are two main types of security that you need to know:

Security operations
Data protection

Security operations

There are always employees who do wrong things they are not authorized to do or should not do. Those are why there are services like IAM, Organizations, Cognito, CloudTrail, etc. When learning a specific service, you must know how to grant access or limit it. Amazon S3, how to not allow others to publish it? How do I allow access to someone with another AWS account (and only them)? How to prevent someone from Terminating EC2 instances? Knowing what a service can do is important, but securing it is just as important.

Data protection

You must get used to the concept of Encrypt at rest and Encrypt in transit. Is data stored in Storage encrypted? Is it encrypted when transferring data to the internet or on-premises via VPN? Most services are encrypted in transit by default, but encrypt at rest is not (Encryption is not enabled by default on EBS). There are still a lot of demanding customers out there. They require even more security. Have to use CloudHSM, use their certificate, or use end-to-end encryption too, and it's tough as well. You will become confusing with such services as KMS, CloudHSM, Direct Connect, Security Hub, Transit Gateway, VPN, etc... Still, it will frequently appear in the exam because that's how enterprise applications are built.

2. Resiliency: Availability and Disaster Recovery (DR)

High Availability, Blue/Green Deployment, Zero Down Time, Uptime, service level agreement (SLA), recovery time objective (RTO), recovery point objective (RPO), Redundancy, Failback, etc. You must go into deep dives on it.

Maybe the projects you have done or are doing are at an elementary level: one server, one database, one data center, etc. Stop and start again. But when it came to AWS, the architecture standard took a new high level. Your system must be redundant for everything: hardware failure leading to server stop or data loss. Possibilities like hurricanes, earthquakes, and volcanoes destroy a data center. Even, you have to consider the case of the main error from AWS.

You need to know four ways to preserve the system

Backup and Restore
Pilot Light
Warm Standby
Multi-site Example

For example, EC2 only guarantees 95.0% availability, which means every 1 hour, EC2 can be down for 3 minutes. We need to combine with Auto Scaling Group, Load Balancer to increase availability. More advanced, use Cross-region Load Balancer, Multi-site between AWS infrastructure and on-premises. The Serverless option can also be considered as an alternative. What about data? EC2 stores data in EBS. How to ensure no data loss? By now, you will probably have to know things like RAID, Snapshot, Point-in-time-recovery (PITR).

Similarly, in RDS, we have RDS snapshot, backtrack, Replica, Multi-AZ, etc. With ElastiCache, there is also Multi A-Z, Failover, Append Only Files (AOF)...You also need to know how long it takes to recover (RTO) and how much data can be lost (RPO). If customers need to restore the system within 1 hour, storing it in Glacier is wrong.
Refer

3. Performance

The performance here is understood as using computing resources to meet system requirements and maintain performance when conditions change or technology evolves. Some of the issues raised could be:

Latency
IOPS
Throughput

When choosing a service to use for your architecture, there are four types of resources you need to consider: compute, storage, database, and network.

Compute

There are three main types of computing:

Instance: a virtual server, typically EC2 and Lightsail. You can use EC2 to solve everything if there is no other option. For example, launch a MySQL server inside an EC2 instance.
Container: this is a way of virtualizing the operating system, I can immediately think of Docker, Kubernetes. With AWS, think about ECS, Fargate, and EKS. When to use Container instead of Instance? Most of the answers (in the test) will be customers who already use on-premises containers and want to spend the least amount of effort if migrating to AWS.
Function: focus on code and run without having to manage instances. Functions have three essential benefits: minimal management effort, outstanding scalability and are particularly suitable for event-based programs.

Storage

There are three main types of storage:

Object Storage: just S3
Block Storage: just EBS
File Storage: EFS, FSx

In my opinion, Storage Gateway should also be included here. The storage classification doesn't make much sense either. You can save it in Object Storage, Block Storage, or File Storage with a file. However, there is a slight difference. E.g., S3 is limited to 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per prefix (directory name), and don't forget there's also a KMS limit if you choose to encrypt. And yet, accessing files from S3 will have extremely high latency. Think about them when you use them. With EBS, it is necessary to know the difference between the volume types, which are optimal for IOPS? Which are optimal for Throughput? If you need even higher performance, you can think of RAID, but what are the disadvantages of RAID? Data loss, Downtime. EBS still has certain limitations: IOPS, Throughput, Hybrid Storage, Limit storage size, Sharing, Availability, etc. Then you will need to use EFS.

Database

Databases can be divided into seven types:

Relational: RDS
Key-Value: DynamoDB, or maybe even Redis
Document: DocumentDB
In-Memory: ElastiCache, MemoryDB (new)
Graph: Neptune
Time-Series: Timestream
Ledger: QLDB

Each type of database will be used for different purposes, but sometimes it is not too noticeable.

E.g., Customers using MySQL on-premises does not mean that going to AWS is required to use RDS. That's also just an option. Another option could be MySQL on EC2 instance, converting MySQL to DynamoDB.

If you want a database with high access frequency low latency, it can be DynamoDB, but if the data is not too necessary, ElastiCache is also a good choice.

In general, the database selection problem is not too complex. Usually, the question will revolve around the issue of optimizing a particular type of database.

E.g., the customer is using MySQL on RDS and is noticing slow response as traffic increases. Think about multi-writer, multi-reader, and maybe even ElastiCache to reduce RDS load.

Refer:

AWS re:Invent 2019: Leadership session: AWS purpose-built databases (DAT209-L)

Network

All AWS components are connected through a network, AWS's private network, or the internet.

EC2 instances connect to EBS through the internal network; Route53, CloudFront, S3 in/out internet; the connection between AWS and on-premises... In this regard, there are several standard solutions:

CloudFront can reduce latency when accessing global
Deploying (adding a system) on a region near the user is also the best way to reduce latency
Upload/Download on S3 is slow: Use S3 transfer acceleration
Connecting to On-premises using VPN is slow: Spend more money to use Direct Connect
Latency-Based Routing in Amazon Route 53
Use VPC Endpoints to use AWS's internal network Refer
PERF 1: How do you select the best performing architecture?
PERF 2: How do you select your compute solution?
PERF 3: How do you select your storage solution?
PERF 4: How do you select your database solution?
PERF 5: How do you configure your networking solution?
PERF 6: How do you evolve your workload to take advantage of new releases?
PERF 7: How do you monitor your resources to ensure they are performing?
PERF 8: How do you use tradeoffs to improve performance?
Performance Efficiency

4. Cost

Usually, there are two scenarios:

Customers already have the system and want to reduce the price
Customers want to migrate to AWS for the lowest possible price.

So how to optimize the price:

Choose cheaper services. When to use Kinesis instead of SQS? When to use a VPN instead of Direct Connect? When to use Spot Instances?
Remove unnecessary resources.
Commitment: Savings Plans, Reserved Instances
Cost Management: Consolidated Billing, Cost Allocation Tags, Trust Advisor,...

And finally, remember, optimizing for price also comes at the expense of other things (performance, availability, reliability, security, etc.). The architecture you choose may not be perfect, but it is still the correct answer if it is feasible and the most cost-effective according to the customer's requirements.

5. Operational Excellence

"The Operational Excellence pillar includes the ability to support development and run workloads effectively, gain insight into their operations, and to continuously improve supporting processes and procedures to deliver business value"

It depends on services, but there are some common aspects:

Infrastructure as code: in short, CloudFormation, Opsworks. You don't need to understand what Ansible or Puppet is, but you should know what Opsworks is and what it is suitable for. You also don't need to master the CloudFormation template, but you must know some basic syntax and create some services by CloudFormation, such as VPC, Subnets, RDS...
Basic settings: some services have unique settings you must be familiar. E.g.: could I change Lambda CPU configuration? Restore an RDS snapshot, make a failover on an RDS instance? Change Launch Template, etc.
Monitoring, Incident response: logging, metric visualizer, alarms, incident management, and remediation
Some managed operator services: there are some services you should get to know, such as CloudTrail, Config, Personal Health Dashboard, Organizations, Service Catalog, Systems Manager. Refer
https://wa.aws.amazon.com/wellarchitected/2020-07-02T19-33-23/wat.pillar.operationalExcellence.en.html

Don't trust keywords

When taking the exam, many of you will have a trick that believes in the keyword, see this keyword, and immediately decide. But here, things are not so clear. If you insist on choosing the most appropriate service, it is easy to lead you to the wrong architecture because this is a collection of services and methods, not a story of just one service.

E.g., When a customer needs a Relational database, RDS comes to mind, but don't forget you could also use EC2 and launch the database on its instance. Why does it? Because of the cost, there are features of the database that RDS does not support.

When customers think of storing static data, you think of S3. But, saving in EBS, EFS is also away.

When facing a problem, the answers will gather a lot of services and methods to solve it. Your job is to choose the most suitable plan, only the most appropriate service is not enough.

Do what customers ask

There are widespread errors I have encountered. Each question will present many problems, and you must focus on only some of them: What the customer wants. These could be requests:

How to optimize price?
How to reduce latency?
How to shorten the processing time?

Why is it a problem? Because the content of the question will be very rambling and make you think of its request in the wrong way.

E.g., The customer has a website deployed in region us-east-1, using EC2 instance and Application Load Balancer, Auto Scaling Group. The website has about 1 million users worldwide. Customers find that users accessing from Asia will experience significant latency. Ask how to optimize the price?

You can think of CloudFront with global access, high latency, and optimal price. However, CloudFront is not price-optimized too much, sometimes even increasing costs, while customer requirements are price-optimized. So you have to think about Spot Instances, Reserved Instances. So, your job is to choose the answer to the question, don't let other things distract you.

Time management

You have 170 minutes (or 200 minutes if you request as ESL) to solve 75 questions. However, the topic is very long. Furthermore, to the pressure of the exam, it is easy to get immersed in some difficult questions and waste a lot of time. Try to practice solving each question within 2 or 2.5 minutes. I will review the sentences marked with review the rest of the time.

Also, do not mark the review too many. Should only keep the check under 20 sentences. For questions you don't know or think you can't answer, answer indiscriminately and don't look back; spend your precious time on questions that are more likely to earn points.

Conclude

The AWS Certified Solutions Architect --- Professional exam is challenging. You need a solid determination to overcome. But whether you pass or not, it doesn't matter; what matters is that you will gain various knowledge, which will help you in a real career path.

Spring Boot project using AWS RDS and Secrets Manager

luan vu — Sat, 22 Jan 2022 08:54:40 +0000

In this article, I will show you how to integrate Amazon RDS connection information into a Spring project using AWS Secrets Manager.

With traditional Spring projects, we tend to configure the data source directly inside the source code. In different environments (development, staging, production), it is common to use Spring profiles to change these parameters. This solution is convenient, but it lacks security (credentials explosion, inability to access manager), difficult to manage and change (password rotation policy).

AWS offers a solution using AWS Secrets Manager integration with RDS and Rotating Secrets, simplifying management and ensuring security. Refer to Security best practices for Amazon RDS

Here are 2 simple steps to integrate RDS data source and Spring Boot project using AWS Secrets Manager.

Step 1. Add this dependency in pom.xml file

<dependency>
  <groupId>com.amazonaws.secretsmanager</groupId>
  <artifactId>aws-secretsmanager-jdbc</artifactId>
  <version>1.0.5</version>
</dependency>

Step 2. Change data source configuration

spring:
  datasource:
    url: jdbc-secretsmanager:mysql://database-host:3306/rdsdb
    username: rdsrotation
    driver-class-name: com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver

url: it has a prefix jdbc-secretsmanager instead of just jdbc as usual
username: Secret Name which you’ve got in AWS Secrets Manager

The magic here is that driver-class-name is AWSSecretsManagerMySQLDriver. It is the AWS Secrets Manager Driver driver which wraps the original driver (for instance com.mysql.cj.jdbc.Driver).

Some databases and corresponding drivers:

MySQL: AWSSecretsManagerMySQLDriver
MariaDB: AWSSecretsManagerMariaDBDriver
Oracle DB: AWSSecretsManagerOracleDriver
PostgreSQL: AWSSecretsManagerPostgreSQLDriver
MSSQL: AWSSecretsManagerMSSQLServerDriver

Here is a GitHub link to this implementation.

So it’s done. Now you can use Spring data source normally. You can rotate your password automatically every day, every month in AWS Secrets Manager and your program will still work as you expected without restarting every time you change the password.

But How? If you need to understand in more detail how it works, continue to follow below.

How it work

When the program needs a JDBC connection, the java.sql.Driver#connect function is called. Instead of retrieving an existing username/password, the Driver wrapper (eg AWSSecretsManagerMySQLDriver) will call into the AWS Secrets Manager to retrieve the username/password.

The problem is that you have to calls the AWS Secrets Manager API every time you use the connection, it will take time and cost a lot(AWS Secrets Manager charges by the number of API calls). So, the better way is to cache and reuse username/password next time instead of calling AWS Secrets Manager.

If the cached has expired (default is 1 hour), the driver will continue to call into AWS Secrets Manager to retrieve the username/password (can be new or old).

So what if the cached has not expired but the username/password in AWS Secrets Manager(also in RDS of course) has been changed? Driver uses fallback mechanism. It doesn’t know credentials are changed, so it continuous connects to database using the existing username/password (in the cached), if the database returns an error indicating the wrong username/password, call AWS Secrets Manager to get the new username/password.

And that’s it. Above is the guide on how to integrate Amazon RDS and AWS Secrets Manager into Spring Boot project. This method is not limited to Spring but can be used with any Java project that uses JDBC.

Reference

=========

10 syntax in Java You should know

luan vu — Sat, 08 Jan 2022 09:11:47 +0000

1. Underscore in Numeric Literals

You have to write dozen of numbers to present a number of millions, billions and even more. You always must count it again and again to make sure the number is correct. Actually, you could place some underscore inside the number to make it clear, like you use comma in real life.

long noUnderscore = 100000000000l;  
long withUnderscore = 100_000_000_000l;

Refer: Underscores in Numeric Literals

2. Pattern.asPredicate

When use java.util.regex.Pattern with Stream API, I feel uncomfortable when I have to create a Matcher, and then use that matcher to test whether my text is matched the pattern.

var validEmails = emails.stream()  
        .filter(email -> EMAIL_REGEX.matcher(email).find())  
        .collect(Collectors.toList());

I have no idea why Java make it so complex for such a simple thing. Anyway, Java provides a way to shorten the code and make it more clear. That’s Pattern.asPredicate

var validEmails = emails.stream()  
        .filter(EMAIL_REGEX.asPredicate())  
        .collect(Collectors.toList());

Refer: Oracle docs Pattern.html#asPredicate

3. Named RegEx Capturing Group

When I need extract multiple information in texts with single Pattern. I have to use java.util.regex.Matcher#group and I usually pass the index of a capturing group in its pattern. The indexes always confuse me. So, you should place a named group to easily extract it by name, like this:

var p = Pattern.compile(  
        "^Today is (?<month>\\\\w+) (?<day>\\\\d+), (?<year>\\\\d+)"  
);  
var m = p.matcher("Today is August 4, 2026");  
System.out.println(m.group("month"));  
System.out.println(m.group("day"));  
System.out.println(m.group("year"));

Refer: Oracle docs Pattern.html#special

4. Arrays.asList

Need to create a List inline. The fastest way is use Arrays.asList

var numbers = Arrays.asList(1, 2, 3);  
var singletonList = Arrays.asList("Just me");

Beware: the list is returned is a fixed-size (but mutable) list.
Refer: Oracle docs Collections.html#singleton(T)

5. Collections.singleton*

Do you know an ArrayList, a HashSet or a HashMap instance wastes plenty of memory even when you just need to store a single item. They need an array to store data, and int value to store real size and a HashMap (lol)
If you need a collection to store the single item, you should use:

Collections.singleton to create a Set
Collections.singletonList to create a List
Collections.singletonMap to create a Map

var singletonSet = Collections.singleton(1);  
var singletonList = Collections.singletonList(1);  
var singletonMap = Collections.singletonMap("version", 1);

It’s a simple but very effective.

Refer: Oracle docs Collections#singleton)

6. Collections.empty*

Stop returning null value as a collection, return an empty collection instead.

public static List<String> getValidEmails(Collection<String> email) {  
    if (email == null) {  
        return Collections.emptyList();  
    }  
    ...  
}

We have

emptyList
emptyMap
emptySet
emptyIterator

And more, use them.

Refer: Oracle docs Collections emptyList)

7. Objects equality check

String gameA = "The Legend of Zelda: Breath of the Wild";  
String gameB = "Monster Hunter Rise";  
System.out.println(gameA.equals(gameB));

Am I seriously? Why don’t I use equals method to solve a simple thing. Well, what happened if either gameA or gameB is null? I would get an NullPointerException if I have a bad luck.

String gameA = null;  
String gameB = "Monster Hunter Rise";  
System.out.println(gameA.equals(gameB));  
// => Exception in thread "main" java.lang.NullPointerException

I can avoid it by using Objects.equals, it does work no matter either gameA or gameB is null.

String gameA = null;  
String gameB = "Monster Hunter Rise";  
System.out.println(Objects.equals(gameA, gameB));

Refer: Oracle docs Objects

8. List.of, Set.of and Map.of (since 9)

Do you prefer Functional Programming, and love immutable objects? Creating a immutable collection class by yourself could be challenging. Well, Java provides Unmodifiable Lists, Unmodifiable Sets, Unmodifiable Maps which basically are immutable classes.

For example, Unmodifiable Lists have some characteristics:

They are unmodifiable. Elements cannot be added, removed, or replaced. Calling any mutator method on the List will always cause UnsupportedOperationException to be thrown. However, if the contained elements are themselves mutable, this may cause the List's contents to appear to change.
They disallow null elements. Attempts to create them with null elements result in NullPointerException.
They are serializable if all elements are serializable.
The order of elements in the list is the same as the order of the provided arguments, or of the elements in the provided array.
They are value-based. Callers should make no assumptions about the identity of the returned instances. Factories are free to create new instances or reuse existing ones. Therefore, identity-sensitive operations on these instances (reference equality (==), identity hash code, and synchronization) are unreliable and should be avoided.
They are serialized as specified on the Serialized Form page.

Refer: Oracle docs List unmodifiable

To create an unmodifiable lists, use can use List.of

static List of()
static List of(E e1)
static List of(E… elements)
static List of(E e1, E e2)
static List of(E e1, E e2, E e3)
…

It sounds like Arrays.asList, but instead of a mutable list, List.of creates an immutable list and it doesn’t accept null values.

9. Comparator and null

In the past, I didn’t noticed that Comparator don’t work with null value by default.
For example, the code will throw a NullPointerException

var list = Arrays.asList(1, null, -1, null, 2);  
list.sort(Comparator.naturalOrder());  
// ==> Exception in thread "main" java.lang.NullPointerException

I used to code like this

var list = Arrays.asList(1, null, -1, null, 2);  
list.sort((a, b) -> {  
    if (a == null) {  
        return b == null ? 0 : -1;  
    }  
    if (b == null) {  
        return a == null ? 0 : 1;  
    }  
    return a.compareTo(b);  
});

Luckily, there are two simple methods resolve my problem. They are Comparator.nullsFirst and Comparator.nullsLast

var list = Arrays.asList(1, null, -1, null, 2);  
list.sort(Comparator.nullsFirst(Comparator.naturalOrder()));  
// [null, null, -1, 1, 2]

10. Predicate::not (since 11)

Stream API is good, filter is great and Predicate is awesome, but have you ever written something like that:

games.stream()  
   .filter(game -> !Strings.isNullOrEmpty(game))  
   .collect(Collectors.toList());

Fortunately, Java 11 save you from that ugly code by Predicate.not

games.stream()  
   .filter(Predicate.not(Strings::isNullOrEmpty))  
   .collect(Collectors.toList());