Forem: Ada Cheng

Write custom component for Home Assistant

Ada Cheng — Thu, 21 Apr 2022 08:33:23 +0000

This is a dummy component which integrates with binary sensor, lock, sensor and switch platforms for demonstrating the development of custom components used in Home Assistant.

Development

Setup development environment as described in the following URL:

https://developers.home-assistant.io/docs/development_environment/

Generate the skeleton codes for integration by executing the following command:

python3 -m script.scaffold integration

An interative Q&A session then starts. When it finishes, it will generate the skeleton codes.

   What is the domain?
   > dummy_garage

   What is the name of your integration?
   > Dummy Garage

   What is your GitHub handle?
   > @adafycheng

   What PyPI package and version do you depend on? Leave blank for none.
   >

   How will your integration gather data?

   Valid values are assumed_state, calculated, cloud_polling, cloud_push, local_polling, local_push

   More info @ https://developers.home-assistant.io/docs/creating_integration_manifest#iot-class

   > local_polling

   Does Home Assistant need the user to authenticate to control the device/service? (yes/no) [yes]
   > no

   Is the device/service discoverable on the local network? (yes/no) [no]
   > no

   Is this a helper integration? (yes/no) [no]
   > yes

   Can the user authenticate the device using OAuth2? (yes/no) [no]
   > no

   Scaffolding integration for the dummy_garage integration...
   Writing tests/components/dummy_garage/__init__.py
   Writing homeassistant/components/dummy_garage/const.py
   Writing homeassistant/components/dummy_garage/manifest.json
   Writing homeassistant/components/dummy_garage/__init__.py
   Updating dummy_garage manifest: {'codeowners': ['@adafycheng'], 'iot_class': 'local_polling'}

   Scaffolding config_flow_helper for the dummy_garage integration...
   Writing homeassistant/components/dummy_garage/const.py
   Writing homeassistant/components/dummy_garage/__init__.py
   Writing homeassistant/components/dummy_garage/sensor.py
   Writing homeassistant/components/dummy_garage/config_flow.py
   Writing tests/components/dummy_garage/test_config_flow.py
   Writing tests/components/dummy_garage/test_init.py
   Updating dummy_garage manifest: {'config_flow': True}
   Updating dummy_garage strings: ['config', 'options']

   Running hassfest to pick up new information.

   Running gen_requirements_all to pick up new information.

   Running script/translations_develop to pick up new translation strings.

   **************************

   Integration code has been generated

   Added the following files:
   - homeassistant/components/dummy_garage/config_flow.py
   - homeassistant/components/dummy_garage/manifest.json
   - homeassistant/components/dummy_garage/const.py
   - homeassistant/components/dummy_garage/sensor.py
   - homeassistant/components/dummy_garage/__init__.py

   Added the following tests:
   - tests/components/dummy_garage/test_init.py
   - tests/components/dummy_garage/test_config_flow.py

   The next step is to look at the files and deal with all areas marked as TODO.

In this integration, since we create the entities by configuration, therefore we clear most of the codes in __init__.py.
For each of the platforms,

i. Define the PLATFORM_SCHEMA (mandatoriness and data type) for getting the inputs from configuration.yaml.

   import voluptuous as vol

   from homeassistant.components.sensor import (
       PLATFORM_SCHEMA,
       SensorDeviceClass,
       SensorEntity,
       SensorStateClass,
   )

   DEFAULT_NAME = "Dummy Garage - Sensor"

   PLATFORM_SCHEMA = PLATFORM_SCHEMA.extend(
   {
       vol.Required(CONF_MAC): cv.string,
       vol.Optional(CONF_NAME, default=DEFAULT_NAME): cv.string,
   }
   )

ii. Add a async_setup_platform function to be called when the platform is being setup. In this function, the entity will be created and added to Home Assistant.

   async def async_setup_platform(
       hass: HomeAssistant,
       config: ConfigType,
       add_entities: AddEntitiesCallback,
       discovery_info: DiscoveryInfoType | None = None,
   ) -> None:
       """Set up the sensor platform."""
       name = config[CONF_NAME]
       mac_addr = config[CONF_MAC]

       add_entities([DummyGarageSensor(mac_addr, name)], True)

iii. Create a class with the Entity class corresponding to the platform as the input parameter.

The __init__ function is the constructor of the class.
The name function returns as the Name of the entity shown in the Home Assistant Web Application.
The unique_id function returns as the unique ID of the entity. I transform the input MAC address to a unique ID.

   class DummyGarageSensor(SensorEntity):
       """Representation of a Sensor."""

       def __init__(self, mac, name) -> None:
           super().__init__()
           self._mac = mac
           self._name = name
           self._attr_name = name
           self._attr_native_unit_of_measurement = TEMP_CELSIUS
           self._attr_device_class = SensorDeviceClass.TEMPERATURE
           self._attr_state_class = SensorStateClass.MEASUREMENT

       @property
       def name(self) -> str:
           """Return the name of the switch."""
           return self._name

       @property
       def unique_id(self) -> str:
           """Return a unique, Home Assistant friendly identifier for this entity."""
           return self._mac.replace(":", "")

       def update(self) -> None:
           """Fetch new state data for the sensor.

           This is the only method that should fetch new data for Home Assistant.
           """
           self._attr_native_value = 23

Deployment

Copy the directory homeassistant/components/dummy_garage/ to the Home Assistant Core /homeassistant/components/.
Add configuration for the desired entities in the config/configuration.yaml file.

   binary_sensor:
   - platform: dummy_garage
     name: "Dummy Car Presence"
     mac: "12:34:12:34:12:34"
     device_class: "presence"

   lock:
   - platform: dummy_garage
     name: "Dummy Garage Lock"
     mac: "EF:EF:EF:EF:EF:EF"

   switch:
   - platform: dummy_garage
     name: "Dummy Garage Switch"
     mac: "AB:AB:AB:CD:CD:CD"

   sensor:
   - platform: dummy_garage
     name: "Dummy Temperature"
     mac: "56:34:12:90:78:56"

For testing purpose, add automations for opening and closing the garage door in the config/automations.yaml file. Remember to change the values of entity_id if they are different from this example.

- id: open_garage
  alias: "Open Garage Door"
  trigger:
    - platform: time_pattern
      minutes: 5
    - platform: time_pattern
     minutes: 15
    - platform: time_pattern
      minutes: 25
    - platform: time_pattern
      minutes: 35
    - platform: time_pattern
      minutes: 45
    - platform: time_pattern
      minutes: 55
  action:
    - service: lock.unlock
      target:
        entity_id: lock.dummy_garage_lock
    - delay: 00:00:10
    - service: switch.turn_on
      target:
        entity_id: switch.dummy_garage_switch

- id: close_garage
  alias: "Close Garage Door"
  trigger:
    - platform: time_pattern
      minutes: 10
    - platform: time_pattern
      minutes: 20
    - platform: time_pattern
      minutes: 30
    - platform: time_pattern
      minutes: 40
    - platform: time_pattern
      minutes: 50
    - platform: time_pattern
      minutes: 0
  action:
    - service: switch.turn_off
      target:
        entity_id: switch.dummy_garage_switch
    - delay: 00:00:10
    - service: lock.lock
      target:
        entity_id: lock.dummy_garage_lock

Start Home Assistant Core:

hass -c config

Verification

Check that the entities are created.

Add the entities at the Overview tab.
Trigger the Open Garage Door action and observe the door lock and switch status.

Trigger the Close Garage Door action and observe the door lock and switch status.

Acknowledgements

Deploy a Docker image to Google Cloud using Cloud Run

Ada Cheng — Sun, 17 Apr 2022 15:20:29 +0000

In this article, I would like to document how I deploy a Docker image to Google Cloud using Cloud Run.

Build the docker image
Configure Google Cloud
Deploy to Google Cloud

Build the docker image

Refer to my article Dockerizing a Node.js web application for developing a Node.js application and building a docker image.

Configure Google Cloud

Create a Google Cloud Project

If you have already created a project, you can skip this section and go directly to Configure a Google Cloud Project.

In Google Cloud Shell, perform the following steps:
1. Lists credentialed accounts and make sure the desired account is set active.
```
gcloud auth list
```
2. Create a project.
```
gcloud projects create PROJECT_ID
```
```
* *Replace PROJECT_ID with your desired project ID.*
```
Enable Billing in Google Cloud Console.

Configure a Google Cloud Project

List all projects.
```
gcloud projects list
```
Set the default project.
```
gcloud config set project PROJECT_ID
```

* *Replace PROJECT_ID with your project ID.*

Print the project ID.
```
echo $GOOGLE_CLOUD_PROJECT
```

Set the default zone if it is not set:

gcloud config set compute/zone COMPUTE_ZONE

* *Replace COMPUTE_ZONE with your compute zone, such as europe-west2-c.*

Set the default region if it is not set:

gcloud config set compute/region COMPUTE_REGION

* *Replace COMPUTE_REGION with your compute region, such as europe-west2.*

Show the configuration.
```
gcloud config list
```
Enable APIs in Google Cloud Shell.
1. Enable Cloud Run.
```
run.googleapis.com
```

Deploy to Google Cloud

Deployment

Submit the docker image to Container Registry.

gcloud builds submit --tag gcr.io/$GOOGLE_CLOUD_PROJECT/website

* *Replace PROJECT_ID with your project ID.*

List all container images in this project to verify that the docker image has been submitted successfully.
```
gcloud container images list
```

Deploy the Container Image to Cloud Run.

gcloud run deploy CONTAINER_NAME \
--image gcr.io/$GOOGLE_CLOUD_PROJECT/DOCKER_IMAGE_NAME:DOCKER_IMAGE_VERSION \
--platform managed \
--region GCR_REGION \
--allow-unauthenticated

* *Replace CONTAINER_NAME with the desired Container Name.*

* *Replace DOCKER_IMAGE_NAME with your docker image name.*

* *Replace DOCKER_IMAGE_VERSION with the version of your docker image name.*

* *Replace PROJECT_ID with your project ID.*

* *Replace GCR_REGION with your region, such as europe-west2.*

Here below is a working example:

```shell
gcloud run deploy portfolio-website \
--image gcr.io/$GOOGLE_CLOUD_PROJECT/website:latest \
--platform managed \
--region europe-west1 \
--allow-unauthenticated
```

Verification

Check that the web application is running by opening URL of the Cloud Run Service in a web browser.

References

Deploy a Serverless Node.js application to Vercel in 5 minutes

Ada Cheng — Sat, 05 Mar 2022 17:13:44 +0000

Introduction
Add vercel.json file
Create a Vercel project
Conclusion
References

Introduction

Deploying a Serverless Node.js application to Vercel can be easily done by just adding a vercel.json configuration file to your project root directory.

Here I will show you how it can be achieved.

Please note that Vercel platform is serverless and designed for static frontends and Serverless Functions. There is no running server.

Add vercel.json file

First of all, add a vercel.json file in the root directory of your existing Node.js project. Add and commit the file to the Git repository.



{

   "version": 2,

   "name": "music-album",

   "builds": [

      { "src": "app.js", "use": "@vercel/node" }

   ],

   "routes": [

      { "src": "/(.*)", "dest": "/app.js" }

   ]

}

Create a Vercel project

Login Vercel.

If you don't have an account yet, you can sign up for a free account.
Click the "+ New Project" button on the Overview tab.
Import Git Repository by selecting the Git provider. GitHub, GitLab and Bitbucket are available.
Import Git Repository.
Configure the project.
1. Give a Project Name.
2. Click the "Deploy" button. The deployment should be triggered.
Check the deployment status.
Click the link under DOMAINS to view the application in browser.

Conclusion

Congratulations! Your Node.js application is now configured to be deployed to Vercel easily by adding a configuration file to the project.

References

Java Microservice on Google Kubernetes Engine (GKE) Cluster

Ada Cheng — Tue, 18 Jan 2022 13:10:51 +0000

Introduction
Prerequisites
- Install Google Cloud SDK
- Install Cloud Code Plugin for IntelliJ
Configure Google Cloud
- Create a Google Cloud Project
- Create an Artifact Registry
- Create a Google Kubernetes Engine (GKE) Cluster
Code the Java Microservice
- Write the application codes
- Test the application functions locally
Containerize the Microservice
Deploy the Microservice to GKE
- Configure a Run/Debug Configuration for Cloud Code: Kubernetes
- Deploy to GKE
- Test the application functions in GKE
Clean Up
Conclusion
References

Introduction

Java microservices remain as hot topic in 2022.

In this article, I would like to show you the steps to create a Java Microservice, and deploy it to Google Kubernetes Engine (GKE).

Prerequisites

Install Google Cloud SDK

Follow the instructions in Install Cloud SDK.

Install Cloud Code Plugin for IntelliJ

Install Cloud Code Plugin for IntelliJ from Preference -> Plugins. Search by "Cloud Code" in the "Marketplace" tab.

Configure Google Cloud

Create a Google Cloud Project

In Google Cloud Shell, perform the following steps:

i. Lists credentialed accounts and make sure the desired account is set active.
```
gcloud auth list
```
ii. Create a project.
```
gcloud projects create PROJECT_ID
```
Replace PROJECT_ID with your desired project ID.

iii. Set the default project.
```
gcloud config set project PROJECT_ID
```
Replace PROJECT_ID with your project ID.

iv. Print the project ID.
```
echo $GOOGLE_CLOUD_PROJECT
```
v. Set the default zone:
```
gcloud config set compute/zone COMPUTE_ZONE
```
Replace COMPUTE_ZONE with your compute zone, such as europe-west2-c.

vi. Set the default region:
```
gcloud config set compute/region COMPUTE_REGION
```
Replace COMPUTE_REGION with your compute region, such as europe-west2.

vii. Show the configuration.
```
gcloud config list
```
Enable Billing in Google Cloud Console.

Enable APIs in Google Cloud Shell.

i. Enable Artifact Registry API.

gcloud services enable artifactregistry.googleapis.com

ii. Enable Kubernetes Engine API.

gcloud services enable container.googleapis.com

iii. Enable Firestore.

gcloud services enable firestore.googleapis.com

Create an Artifact Registry

Google Cloud Artifact Registry is a successor of Container Registry, in where container images and language packages can be placed.

We will use Artifact Registry to store our container images.

Enable Artifact Registry, if it is not yet enabled.

gcloud services enable artifactregistry.googleapis.com

View a list of supported locations.
```
gcloud artifacts locations list
```

Create an artifact repository.

gcloud artifacts repositories create REPO_NAME \
--repository-format=docker \
--location=REPO_REGION \
--description=REPO_DESC \
--version-policy=REPO_POLICY

Replace REPO_NAME with the desired name for your artifact repository.

Replace REPO_REGION with your region, such as europe-west2.

Replace REPO_DESC with the description of your artifact repository.

Replace REPO_POLICY with keyword snapshot or release.

Here below is a working example:

gcloud artifacts repositories create ms-docker-image \
--repository-format=docker \
--location=europe-west2 \
--description="Snapshot repository for docker images" \
--version-policy=snapshot

The full name of the created repository will be

$(REPO_REGION)-$(REPO_FORMAT).pkg.dev/$(PROJECT_ID)/$(REPO_NAME)

List all artifact repositories created in the project.
```
gcloud artifacts repositories list
```

Create a Google Kubernetes Engine (GKE) Cluster

Enable Kubernetes Engine API, if it is not yet enabled.
```
gcloud services enable container.googleapis.com
```

Create a GKE Cluster.

gcloud container clusters create CLUSTER_NAME \
--num-nodes=1 \
--region europe-west2 \
--release-channel regular \
--scopes=datastore,storage-rw,compute-ro

Replace CLUSTER_NAME with the desired name for your GKE cluster.

It will take several minutes to create the cluster.

After the cluster is created, get authentication credentials to interact with the cluster.
```
gcloud container clusters get-credentials CLUSTER_NAME
```
Replace CLUSTER_NAME with your GKE cluster name.
List all clusters.
```
gcloud container clusters list
```
We can also check whether the cluster is created or not in Google Cloud Console -> Kubernetes Clusters.

Code the Java Microservice

Spring Boot enables quick development of Java microservices with various dependency and plugin support to Google Cloud technologies.

Complete Source Code can be found in my GitHub repositories.

Write the application codes

Generate a Spring Boot application.

To create a Spring Boot application, use the Spring Initializr to generate a simple Spring Boot . Add the following dependencies:
- Spring Web
- Spring Boot Actuator
- Lombok
Spring Boot Actuator provides end points for health check of the application.

Lombok is a Java annotation library that reduces boilerplate code by using annotations.

Click the "Generate" button to download the ZIP file.

Unzip the file to the project root directory.

Modify pom.xml to add dependencies for GCP.

i. Versions

<properties>
    <spring-cloud-gcp.version>1.2.8.RELEASE</spring-cloud-gcp.version>
    <spring-cloud.version>2021.0.0</spring-cloud.version>
</properties>

ii. Dependencies

<dependencies>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-gcp-data-datastore</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-gcp-starter-data-datastore</artifactId>
    </dependency>
</dependencies>

iii. Dependency Management

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-dependencies</artifactId>
            <version>${spring-cloud.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-gcp-dependencies</artifactId>
            <version>${spring-cloud-gcp.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Create a Java model class.

The Lombok @data annotation saves the time of writing the getter and setter functions.

The Lombok @AllArgsConstructor annotation saves the time of writing the constructor.

The @Entity annotation specifies that it is a Firestore entity in datastore mode.

import org.springframework.cloud.gcp.data.datastore.core.mapping.Entity;
import org.springframework.data.annotation.Id;

import lombok.AllArgsConstructor;
import lombok.Data;

@Entity
@Data
@AllArgsConstructor
public class CustomerAccount {

    @Id
    private Long accountId;

    private String lastName;

    private String firstName;

}

Create a Spring Repository interface for Google Datastore.

import org.springframework.cloud.gcp.data.datastore.repository.DatastoreRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface CustomerAccountRepository extends DatastoreRepository<CustomerAccount, Long> {

    List<CustomerAccount> findByLastName(String name);

}

Create a Java Controller class for REST endpoints.

The Lombok @RequiredArgsConstructor annotation saves the time of writing the constructor.

The Lombok @Slf4j annotation facilitates SLF4J for logging with the least configurations.

import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;

@RestController
@Slf4j
@RequiredArgsConstructor
public class CustomerAccountController {

    private final CustomerAccountRepository customerAccountRepository;

    @GetMapping("/api/customerAccounts")
    public Iterable<CustomerAccount> getAllCustomerAccounts() {
        log.debug("->getAllCustomerAccounts");
        return customerAccountRepository.findAll();
    }

    @GetMapping("/api/customerAccount/{lastName}")
    public List<CustomerAccount> findByLastName(@PathVariable("lastName") String name) {
        log.debug("->findByLastName");
        return customerAccountRepository.findByLastName(name);
    }

    @PostMapping("/api/customerAccounts")
    public CustomerAccount saveCustomerAccount(@RequestBody CustomerAccount customerAccount) {
        log.debug("->saveCustomerAccount {}", customerAccount);
        return customerAccountRepository.save(customerAccount);
    }
}

Enable Spring Boot Actuator in application.properties.
```
management.endpoint.health.enabled=true
```

Test the application functions locally

In IntelliJ Terminal,

Lists credentialed accounts and make sure the desired account is set active.
```
gcloud auth list
```
Set the default project.
```
gcloud config set project PROJECT_ID
```
Replace PROJECT_ID with your project ID.
Show the configuration.
```
gcloud config list
```
Start the Spring Boot application.
```
./mvnw spring-boot:run
```

Use a REST client, like Postman, to perform following tests:

Perform health check.
Test application function: Create new Customer Account.
Test application function: Find by Last Name.
Test application function: List all Customer Accounts.

Containerize the Microservice

In general, we need to write a Dockerfile to dockerize the microservice.

However, in this exercise, we will use Google Cloud's Jib Maven Plugin to build the container images without writing a Dockerfile.

Add Jib Maven plugin in pom.xml.

<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>jib-maven-plugin</artifactId>
    <version>3.1.4</version>
    <configuration>
        <to>
            <image>europe-west2-docker.pkg.dev/java-microservice-2022/ms-containers/${project.artifactId}</image>
        </to>
    </configuration>
</plugin>

Replace with your Full Container Name inside the <image> tag.

At the project root, execute the following command:
```
./mvnw install jib:build
```
This will build the docker image and publish it to the Artifact Registry.

Use the following command to list docker images in the Artifact Registry.
```
gcloud artifacts docker images list FULL_CONTAINER_NAME
```
Replace FULL_CONTAINER_NAME with your Full Container Name.

Or you can check the image in Google Cloud Console.

Deploy the Microservice to GKE

Configure a Run/Debug Configuration for Cloud Code: Kubernetes

In this process, we need to write manifest files in YAML format. One is deployment.yaml and the other is service.yaml.

Write deployment.yaml and put it in the project root directory.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: account-service-deployment
spec:
  selector:
    matchLabels:
      app: account-service-pod
  replicas: 1
  template:
    metadata:
      labels:
        app: account-service-pod
    spec:
      containers:
        - name: account-service-container
          image: europe-west2-docker.pkg.dev/java-microservice-2022/ms-containers/ms.client.account.gcp
          ports:
            - containerPort: 8080
          livenessProbe:
            initialDelaySeconds: 20
            httpGet:
              port: 8080
              path: /actuator/health
          readinessProbe:
            initialDelaySeconds: 30
            httpGet:
              port: 8080
              path: /actuator/health

Replace account-service-deployment with the name of your deployment.

Replace account-service-pod with the name of your pod.

Replace account-service-container with the name of your container.

Write server.yaml and put it in the project root directory.

apiVersion: v1
kind: Service
metadata:
  name: account-service
spec:
  type: NodePort
  selector:
    app: account-service-pod
  ports:
    - port: 8080
      targetPort: 8080

Replace account-service with the name of your service.

Replace account-service-pod with the name of your pod.

Add a new Run/Debug Configuration for Cloud Code: Kubernetes.
Change the name of the Configuration.
Click the "Initialize" button in the "Build/Deploy" tab.
Click the "pen" icon under Build Settings.
Select Jib Maven Pluin from the Builder dropdown list and click the "OK" button.
Click the "Initialize" button.
A skaffold.yaml file will be generated at the Project Root Directory.
Click the "Run" tab.

An alert message "Current Kubernetes context unknown. Please ensure you have Kubernetes clusters configured on your machine (such as minikube or GKE)." in red in shown.

To fix this problem, get authentication credentials to interact with the cluster by typing the following command in Terminal of InteglliJ.
```
gcloud container clusters get-credentials CLUSTER_NAME
```
Replace CLUSTER_NAME with your GKE cluster name.

The problem should then be solved.

Please note that the granted credentials are valid in this session only. If your session to Google Cloud is lost, you have to get the credentials again.

At last, make sure the option "Delete deployments when finished" is unchecked.

Click the "OK" button to finish the configuration.
In case of any changes of the deployment.yaml and service.yaml files, delete skaffold.yaml file generated and repeat steps 5-9 to generate the skaffold.yaml file again.

Deploy to GKE

Click the "Run" button of the Cloud Code: Kubernetes configuration just created.
The deployment process should be started.
Once the deployment is finished, test locally using the health check endpoint provided by Spring Actuator.

http://localhost:8080/actuator/health
In IntelligJ Terminal, type the following command to list the services.
```
kubectl get service
```
The following command lists the service details in yaml format.
```
kubectl get service account-service --output yaml
```
Expose with a target of 8080 using the following command.
```
kubectl expose deployment {deployment-name}  --name={load-balancer-service-name} --type=LoadBalancer --port 80 --target-port 8080
```
Here is the actual command I used in this exmaple.
```
kubectl expose deployment account-service-deployment --name=lb-account-service-gcp --type=LoadBalancer --port 80 --target-port 8080
```
Use the following command to check the progress:
```
kubectl get service
```
It may take several minutes to bind the external IP address.

Keep trying until an external IP address is bound.

With the external IP, we can now perform tests.

Test the application functions in GKE

Health Check using external IP:
Create new Customer Account using external IP:
List all Customer Accounts using external IP:
Find Customer Account by Last Name using external IP:

Clean up

Remember to clean up in order to avoid incurring cost.

kubectl delete service account-service
gcloud container clusters delete ms-cluster --region europe-west2-c

Conclusion

Hoorays!

We had created a Java Microservice, which interface with Google Cloud's NoSQL database. We then created a GKE cluster and deployed our microservice to it.

References

CI with Snyk using GitHub Actions

Ada Cheng — Fri, 24 Dec 2021 20:53:15 +0000

GitHub Actions can be used as a CI tool for building, testing and deploying our code. With the aid of Synk, it can also automate the process of checking vulnerabilities.

Introduction
Create Action Secrets in GitHub
Create a GitHub Action
Compose GitHub Action workflow file
Run the GitHub Action
Conclusion
References

Introduction

Snyk is a developer security platform for securing code, dependencies, containers, and infrastructure as code.

It can be used in IDE such as IntelliJ, Visual Studio Code and so on as a tool to scan vulnerabilities in the code and librabries.

It can also be used to secure containers by finding and fixing issues in containers and continuously monitor container images.

It can also be integrated with CI/CD tool to find and fix issues in application continuously.

In this article, we will integrate Synk with GitHub Action, a CI tool from GitHub, in order to secure our application continuously.

Create Action Secrets in GitHub

It is required to submit Access Tokens to Snyk and Npm Registry for authentication when performing security check and publishing respectively.

Action Secrets are environment variables that are encrypted. It is useful to store sensitive information such as access tokens. Action Secrets can be used in Action Workflow file.

Generate Access Token for publishing packages to npm Registry.

Login npm.
Click "Access Tokens" on the popup menu shown when the profile image is clicked.
Click the "Generate New Token" button on the "Access Tokens" page.
In the "New Access Token" page shown, select "Automation" from the "Select type" list, and click "Generate Token".
A new Token should then be generated. Copy the token for later use.

Generate Auth Token for authentication to Snyk.

Login Snyk.
Click Account Settings > API Token section.
In the KEY field, click "click to show", then select and copy your token.

Create Action Secrets

Create Actions Secrets NPM_TOKEN and SNYK_TOKEN for access to npm repository and Snyk respectively.

Login GitHub.
Click the target repository.
Select the "Settings" tab.
On the "Settings" page, select "Secrets" on the left navigation menu. "Action secrets" page should be shown.
Click the "New repository secret" button. A "New secret" page should be shown.
Input "Name" and "Value" of the tokens, and then click the "Add secret" button.

Name Value

NPM_TOKEN { Access Token for NPM }

SNYK_TOKEN { Auth Token for Sync }

Name	Value
NPM_TOKEN	{ Access Token for NPM }
SNYK_TOKEN	{ Auth Token for Sync }

Create a GitHub Action

Open the target GitHub repository in browser.
Click on the Actions tab.
Click the link on "set up a workflow yourself".
An editor should be shown for editing the workflow file main.yml.

Compose GitHub Action workflow file.

A basic GitHub Action workflow file consists of 3 secions:

name: Action Name
on: How the action will be triggered.
jobs: Jobs to be performed when the Action is triggered.

Update name section.

name: CI Publish, with security check using Snyk

Keep on section unchanged. By default, the action is triggered when a push or a pull request occurs.

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

Update jobs section.

There are 3 jobs that are set up in this Action:

i. security: Use Snyk to check for any vulnerability.

ii. build: This job is used to build the code. In this example, we build a Node.js application with various Node versions defined in an array. This allows us to test the application running on different Node versions in a very easy approach.

iii. publish: Publish the package to npm repository, allowing other developers to download and install the package, simple using the npm insall command.

To set up a job that depends on other job(s) to be run successfully, needs can be used. For example, needs: [security, build] means that the job publish requires the jobs security and build to be executed successfully before it can be run. If either of the jobs fails, the publish job will not be executed.

Here below list the entire workflow file:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@master
      - name: Run Snyk to check for vulnerabilities
        uses: snyk/actions/node@master
        env:
          SNYK_TOKEN&#58; &#36;&#123;&#123; secrets.SNYK_TOKEN &#125;&#125;
        with:
          command: monitor

  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
      node-version: [10.x, 12.x, 14.x, 15.x]

    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2
      - name: Use Node.js &#36;&#123;&#123; matrix.node-version &#125;&#125;
        uses: actions/setup-node@v2
        with:
          node-version&#58; &#36;&#123;&#123; matrix.node-version &#125;&#125;
      - name: Install dependencies
        run: npm ci
      - run: npm run build --if-present
      - run: npm test

  publish:
    needs: [security, build]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Use Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '15.x'
          registry-url: 'https://registry.npmjs.org'
      - name: Install dependencies
        run: npm ci
      - name: Publish
        run: npm publish
        env:
          NODE_AUTH_TOKEN&#58; &#36;&#123;&#123; secrets.NPM_TOKEN &#125;&#125;

Commit the workflow file.
- Click "Start Commit" button on the left.
- Input description. It is better to input the Action Name, since it will be shown in the Action History.
- Click "Commit changes" button.
- Once the "Commit changes" button is clicked, the Action will be triggered to execute.

Run the GitHub Action

Our GitHub Action action can be triggered when the workflow file is updated, push or pull request occurs.

Once the Action is triggered, the defined jobs will be run.
Once the Action is completed successfully, a green tick will be shown.
Upon completion, check the security job details by clicking the security link on the summary panel on the left.
```
security:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@master
    - name: Run Snyk to check for vulnerabilities
      uses: snyk/actions/node@master
      env:
        SNYK_TOKEN&#58; &#36;&#123;&#123; secrets.SNYK_TOKEN &#125;&#125;
      with:
       command: monitor
```
With monitor command, the scan is performed and report is generated, but the process will not be interrupted. In other words, even if vulnerabilities are found, the job is finished successfully without error and next job will not be interfered and will then start.

To view the report, open the link stated as "Explore this snapshot at" in the result of "Run Snyk to check for vulnerabilities" section of the security job in a browser.

It is recommended to use the monitor command at the beginning to find the vulnerabilities during development stage. After the vulnerabilities are fixed or ignored, in production stage, we can then set to "synk test" command. This command will fail the builds when vulnerabilities are found. In this case, we can decide whether to stop or continue the build for production deployment.
Check the build jobs details by clicking the build link on the summary panel on the left.
Check the publish job details by clicking the security link on the summary panel on the left.
Check whether the package is published at npm repository.

Conclusion

Security vulnerabilities are subject to attack and should be closely monitored. By applying security scan during CI/CD Integration, our applications can be continuously monitored against security threats.

References

Develop a React Component using create-react-library template

Ada Cheng — Thu, 23 Dec 2021 23:10:12 +0000

In this article, I would like to document how I created a React component and published it.

Prerequisites
Procedures
References

Prerequisites

Install required libraries

npm install react react-dom

npm install yarn

Procedures

Use the create-react-library to create a template
```
npm install -g create-react-library && create-react-library
```
Answer to questions:

Package Name reactjs-tabbedpane-component

Package Description A tabbed pane component using React.js

Author's GitHub Handle adafycheng

GitHub Repo Path adafycheng/reactjs-tabbedpane-component

License MIT

Package Manager yarn

Template default

Modify src/index.js

import React from 'react'
import styles from './styles.module.scss'
import $ from 'jquery'

const TabbedPaneComponent = ({ data }) => {
  $(function () {
  // read the input JSON content
    if (data !== undefined) {
      for (let i = 0; i < data.contents.length; i++) {
        const newDiv = $('<div class=' + styles.navbar + '></div>')
        const newAnchor = $('<a class="paneLink"></a>')
          .text(data.contents[i].subject)
          .attr('data-text', data.contents[i].text)
        newDiv.append(newAnchor)
        newAnchor.click(function () {
          $('#paneContentDiv').html($(this).data('text'))
        })
        $('#navbarDiv').append(newDiv)
      }
      if (data.contents.length > 0) {
        // Get the first link and click.
        $('.paneLink:first').click()
      }
    }
  })

  return (
    <div id='pane' className={styles.pane}>
      <div id='navbarDiv' />
      <div id='paneContentDiv' className={styles.paneContent} />
    </div>
  )
}

export default TabbedPaneComponent

Modify example/src/App.js for testing

import React from 'react'

import TabbedPaneComponent from 'reactjs-tabbedpane-component'
import 'reactjs-tabbedpane-component/dist/index.css'

const contentData = {
  contents: [
    {
      subject: 'Overview',
      text: 'This is content of <a href="#">Overview</a>.'
    },
    {
      subject: 'Assumptions',
      text: '<ul><li>Assumption 1</li><li>Assumption 2</li><li>Assumption 3</li><li>Assumption 4</li></ul>'
    },
    {
      subject: 'Technical Design',
      text: 'This is content of Technical Design.'
    },
    {
      subject: 'Data Design',
      text: 'This is content of Data Design.'
    },
    {
      subject: 'Conclusion',
      text: 'This is content of Conclusion.'
    }
  ]
}

const App = () => {
  return <TabbedPaneComponent data={contentData} />
}

export default App

To test,

In one terminal,

cd reactjs-tabbedpane-component && yarn start

In another terminal,

cd reactjs-tabbedpane-component/example && yarn start

View the component using browser at http://localhost:3000/.

Repeat steps 3 - 5 for any code changes.

To publish, update package.json for versions.

{
  "name": "reactjs-tabbedpane-component",
  "version": "1.0.8",
  "description": "A tabbed pane component built using React.js",
  "author": "adafycheng",
  "license": "MIT",
  "repository": "adafycheng/reactjs-tabbedpane-component",
  "main": "dist/index.js",
  "module": "dist/index.modern.js",
  "source": "src/index.js"
}

Build the component.
```
npm run build
```
Publish the component.
```
npm publish
```

References

Dockerizing a Node.js web application

Ada Cheng — Sat, 18 Dec 2021 15:11:40 +0000

In this article, I would like to document how I dockerize my portfolio web site, which is a Node.js application.

Build the Node.js application
Deployment
Verification

Build the Node.js application

Firstly, create Dockerfile for the Node.js application.

FROM node:14.18.2-alpine3.14
WORKDIR /app
COPY package.json ./
COPY package-lock.json ./
COPY ./ ./
RUN npm ci
CMD ["npm", "run", "start"]

Since I've used node-sass package for enabling SASS in my application and the package supports limited Node.js version, I need to find the right node.js docker image at docker hub.

At docker hub, search for the right image in the Tags tab. Once the desired image is found, put it on the first line (i.e. the FROM layer) of the Dockerfile.

Secondly, create .dockerignore file to exclude files and directories from the docker image.

node_modules
*.log

Thirdly, build the Node.js application.

yarn build

Fourthly, build the docker image.

*Make sure Docker Engine is running in your development environment.

docker build -f Dockerfile -t portfolio-website .

Deployment

Deploy by exposing port 5001 instead of default port 3000 (you can change to any port according to your environment).

docker run --name portfolio-website -it -p 5001:3000 portfolio-website

Verification

Check that the web application is running by opening http://localhost:5001/ in a web browser.

Running Java Microservices in Docker

Ada Cheng — Fri, 17 Dec 2021 23:51:10 +0000

Today, I managed to run my microservices in docker environment successfully.

Build the microservices
Deployment
Verification

Build the microservices

I have written four microservices using Spring Boot. One is Eureka Server and the other three are Eureka Clients.

Docker Tag Name	Description	Port	Source Code Path
ms-server-eureka	Spring Boot Application acting as Eureka Discovery Server.	8001	$MS_SRC_BASE/ms.server.eureka/
ms-client-user	Spring Boot Client Application connecting to MongoDB.	8002	$MS_SRC_BASE/ms.client.user/
ms-client-product	Spring Boot Client Application connecting to MongoDB.	8003	$MS_SRC_BASE/ms.client.product/
ms-client-cart	Spring Boot Client Application connecting to MongoDB.	8004	$MS_SRC_BASE/ms.client.cart/
ms-mongodb	MongoDB	27017	NIL

Firstly, create Dockerfile for each of the microservice.

Here below is the example for the User Microservice.

FROM openjdk:17-ea-22
COPY target/ms.client.user-0.0.1-SNAPSHOT.jar ms.client.user-SNAPSHOT.jar
EXPOSE 8001
ENTRYPOINT ["java","-jar","/ms.client.user-SNAPSHOT.jar"]

Secondly, build each of the Spring Boot Applications and package to JAR files.

./mvnw -Dmaven.test.skip=true clean package

Thirdly, build docker image of each of the Spring Boot Applications.
*Make sure Docker Engine is running on your development host.

cd $MS_SRC_BASE/ms.server.eureka/
docker build --tag=ms-server-eureka:latest .
cd $MS_SRC_BASE/ms.client.user/
docker build --tag=ms-client-user:latest .
cd $MS_SRC_BASE/ms.client.product/
docker build --tag=ms-client-product:latest .
cd $MS_SRC_BASE/ms.client.cart/
docker build --tag=ms-client-cart:latest .

Deployment

In order that the containers can find each other, use --network container:[containerName] to bind to a master container. In this case, I use ms-server-eureka as the master container.
All ports are exposed by publishing the ports on the master container.

docker run --name ms-server-eureka -p 8000-8003:8000-8003 -p 27017:27017 ms-server-eureka:latest
docker run --name ms-mongodb --network container:ms-server-eureka -d mongo
docker run --name ms-client-user --network container:ms-server-eureka ms-client-user:latest
docker run --name ms-client-product --network container:ms-server-eureka ms-client-product:latest
docker run --name ms-client-cart --network container:ms-server-eureka ms-client-cart:latest

Verification

Check the Eureka Server status by opening http://localhost:8000/ in a web browser.

Get data from the User Microservice by opening http://localhost:8001/user/search/findByLastName?name=Administrator in a web browser.

Write an Express server to send email

Ada Cheng — Sun, 12 Dec 2021 00:30:15 +0000

In this article, I would like to share how to write a Node.js program to send email via Gmail SMTP server using a Google account, and then deploy the program to Google Cloud.

Table of Contents

Generate App Password
Write Express.js Program
Deploy to Google Cloud

Generate App Password

First of all, create App Password to be used in the program. Due to security reason, Google does not allow non Google-app to sign in. It is required to explicitly generate an App Password to use.

In order to generate App Password:

Go to your Google Account.
Select Security.
Under "Signing in to Google," turn on 2-Step Verification.
Under "Signing in to Google," select App Passwords.
At the bottom,
- choose "Mail" from "Select app"
- choose "Other (Custom name)" from "Select device" A textbox will then be shown. Enter meaning description like "Send Email from application".
Follow the instructions to enter the App Password. The App Password is the 16-character code in the yellow bar on your device.
Tap Done.

Reference: Sign in with App Passwords

Write Express.js Program

Create package.json.

   {
     "name": "contact-form-server",
     "description": "This script serves as a server to accept and process request from a Contact Form, and then send to data to a destination email",
     "scripts": {
       "start": "node contact_form_server.js"
     },
     "version": "0.0.1",
     "dependencies": {
       "cors": "^2.8.5",
       "express": "^4.17.1",
       "nodemailer": "^6.7.1"
     }
   }

Create contact_form_server.js (you can have another js script filename, but make sure to change the script name of the "scripts:start" entry in package.json) as the Node.js script.
- express package is used for running a server.
- nodemailer package is used for sending email.
- cors package is used to allow Cross-Origin Resource Sharing.
Start the program locally for testing.

   npm start

Create a HTML form for testing.

Deploy to Google Cloud

In Google Cloud Console, create App Engine in standard environment using Node.js. The main advantage of running in Standard Environment is that "Application can scale to 0 instances when there is no traffic. Most cost-effective for applications that have significant periods where they are not serving traffic".
Create app.yaml for Google Cloud deployment.

   runtime: nodejs14

Enable Cloud Build API. Set the appropriate credential.
Deploy to Google Cloud.

   gcloud app deploy

Change the url value in the javascript in HTML form and test.