Forem: architec

AWS Industry Quest: Financial Services - Ingesting Exchange Data in Near Real Time Study Note

architec — Fri, 22 Dec 2023 02:01:09 +0000

DIY Steps:

Create two new materialized views

CREATE MATERIALIZED VIEW query_5 AS
select      ticker, round(sum(price),2) as total_negotiated
from        stock_lake.stock_data
where       status = 'complete'
and         order_type = 'buy'
group by    ticker;

CREATE MATERIALIZED VIEW query_6 AS
select      investor_id, round(sum(price),2) as total_negotiated
from        stock_lake.stock_data
where       status = 'complete'
and         order_type = 'buy'
group by    investor_id;

AWS Industry Quest: Financial Services - Investing Research Content Distribution Study Note

architec — Thu, 21 Dec 2023 02:02:39 +0000

DIY Steps:

Add Lambda function as a trigger to the Pre-Sign section in the Amazon Cognito user pool
In the Amazon CloudFront distribution, create a new behavior pattern with a TTL value

AWS Industry Quest: Financial Services - Loan Forms Processing Study Note

architec — Wed, 20 Dec 2023 08:40:47 +0000

DIY Steps:

Upload the .jpeg file into S3 ingestion-bucket
Validate by copy paste the new item id from loan_application DynamoDB table

AWS Cloud Quest: Image and Video Analysis Study Note

architec — Thu, 23 Mar 2023 23:43:18 +0000

This is the DIY challenge of the Image and Video Analysis in AWS Cloud Quest.

DIY Steps:

Update and Deploy the Lambda code

"""
The code is developed using reference from
https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/ex-lambda.html

"""

import json
import logging
import boto3

# It is good practice to use proper logging.
# Here we are using the logging module of python.
# https://docs.python.org/3/library/logging.html

logger = logging.getLogger()
logger.setLevel(logging.INFO)

# Declare the label that is used to identify
# if the image contains the label

LABEL = 'spaceship'

# Using boto3 S3 Client
# https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#client
s3_client = boto3.client('s3')

def lambda_handler(event, context):
    """
    This code gets the S3 attributes from the trigger event,
    then invokes the rekognition api to detect labels.
    If the label matches the one present on the LABELS list,
    response is written in the S3 bucket with "Status":"Label Found",
    else the response is written in the S3 bucket with "Status":"Label Not Found".
    """
    logger.info(event)
    bucket = event['Records'][0]['s3']['bucket']['name']
    image = event['Records'][0]['s3']['object']['key']
    output_key = 'output/rekognition_response.json'
    response = {'Status': 'Not Found', 'body': []}


    # Using rekognition boto3 client.
    # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/rekognition.html

    rekognition_client = boto3.client('rekognition')

    # Identify label
    # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/rekognition.html#Rekognition.Client.detect_labels

    try:
        response_rekognition = rekognition_client.detect_labels(   # You are calling detect_labels API 
            Image={                                                # to analyzing Images Stored in an Amazon S3 Bucket
                'S3Object': {
                    'Bucket': bucket,
                    'Name': image
                }
            },
            MinConfidence=70                                        # MinConfidence specifies the minimum confidence  
        )                                                           # level for the labels to return.


    # The below code section tries to find all labels in the detect_labels 
    # list. If the label is found in the list, the response JSON file is updated with 
    # "Status":"Success! <label_name> found" then updates the response body with all 
    # detected labels. If the label is not found, the response JSON file is updated with 
    # "Status":"Failed, <label_name> not found" then updates the response body with all 
    # detected labels.

        detected_labels = [] # Declaring empty label lists.

        if response_rekognition['Labels']:
            for label in response_rekognition['Labels']:
                detected_labels.append(label['Name'].lower())
            print(detected_labels)

            if LABEL in detected_labels:
                response['Status'] = f"Success! {LABEL} found"
                response['body'].append(response_rekognition['Labels'])
            else:
                response['Status'] = f"Failed! {LABEL} Not found"
                response['body'].append(response_rekognition['Labels'])

    except Exception as error:
        print(error)

# Finally the file will be written in the S3 bucket output folder.
    s3_client.put_object(
      Bucket=bucket,
      Key=output_key,
      Body=json.dumps(response, indent=4)
    )

    return response

'''
You can use below code to create test event to test
the Lambda function.
{
    "Records": [
                {
                "s3": {
                    "bucket": {
                    "name": "<Your_bucket_name>"
                    },
                    "object": {
                    "key": "input/ovni.png"
                    }
                }
                }
            ]
}
'''

# You can visit https://docs.aws.amazon.com/code-samples/latest/catalog/code-catalog-python-example_code-rekognition.html
# to get more sample codes.

AWS Cloud Quest: Analyzing Network Traffic Study Note

architec — Sat, 11 Mar 2023 00:07:27 +0000

This is the DIY challenge of the Analyzing Network Traffic in AWS Cloud Quest.

DIY Steps:

Add outbound rule similar to step 24-27

AWS Cloud Quest: Data Ingestion Methods Study Note

architec — Fri, 10 Mar 2023 22:08:23 +0000

This is the DIY challenge of the Data Ingestion Methods in AWS Cloud Quest.

DIY Steps:

Add destination for ClickStreamDataApp SQL applications(legacy)
- AWS Lambda function: AnalyticsDestinationFunction
- Existing IAM role: LabKinesisAnalyticsRole
- Existing in-application stream: DESTINATION_SQL_STREAM
- Output format: CSV

AWS Cloud Quest: Container Services Study Note

architec — Wed, 08 Mar 2023 21:57:48 +0000

This is the DIY challenge of the Container Services in AWS Cloud Quest.

DIY Steps:

Repeat step 28-42

AWS Cloud Quest: Decoupling Applications Study Note

architec — Fri, 03 Mar 2023 23:38:38 +0000

This is the DIY challenge of the Decoupling Applications in AWS Cloud Quest.

DIY Steps:

Repeat step 6-15 to create another SQS queue and subscribe

AWS Cloud Quest: Content Delivery Networks Study Note

architec — Fri, 24 Feb 2023 21:22:58 +0000

This is the DIY challenge of the Content Delivery Networks in AWS Cloud Quest.

DIY Steps:

Reconfigure CloudFront distribution to use S3 labstack-XXX
- In CloudFront distribution, select Origins -> Edit
- Change Origin domain to S3 labstack-XXX

AWS Cloud Quest: Backing Up Data Study Note

architec — Fri, 24 Feb 2023 04:51:29 +0000

This is the DIY challenge of the Backing Up Data in AWS Cloud Quest.

DIY Steps:

Configure an AWS Backup plan diy-backup-plan

Backup rules

Assign resources

AWS Cloud Quest: Automation with CloudFormation Study Note

architec — Wed, 22 Feb 2023 21:11:59 +0000

This is the DIY challenge of the Automation with CloudFormation in AWS Cloud Quest.

DIY Steps:

Change CloudFormation RoboticStack template to launch t2.small EC2

Resources:
  RobotAppServer:
    Type: 'AWS::EC2::Instance'
    Properties:
      InstanceType: t2.small
      ImageId: ami-087c17d1fe0178315
      SecurityGroups:
      - !Ref RobotAppSecurityGroup
  RobotAppSecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupDescription: Enable SSH access via port 22
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: '22'
        ToPort: '22'
        CidrIp: 0.0.0.0/0
  RobotS3Bucket:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Delete

AWS Cloud Quest: Resource Governance Study Note

architec — Wed, 22 Feb 2023 19:18:11 +0000

This is the DIY challenge of the Resource Governance in AWS Cloud Quest.

DIY Steps:

Add rule s3-bucket-versioning-enabled
Enable versioning on config-bucket-###