Forem: Cloud Tech

Best practices for ML lifecycle stages

Adit Modi — Tue, 08 Feb 2022 04:20:24 +0000

Building a machine learning model is an iterative process. For a successful deployment, most of the steps are repeated several times to achieve optimal results. The model must be maintained after deployment and adapted to changing environment. Let’s look at the details of the lifecycle of a machine learning model.

Data collection

The first step in the development of ML workloads is identification of data that is needed for training and performance evaluation of an ML model.
In the cloud environment, a data lake usually serves as a centralized repository that enables you to store all structured and unstructured data regardless of scale.
AWS provides a number of ways to ingest data, both in bulk and in real-time, from a wide variety of sources. You can use services such as AWS Direct Connect and AWS Storage Gateway to move data from on-premises environments, and tools like AWS Snowball and AWS Snowmobile for moving data at scale.
You can also use Amazon Kinesis to collect and ingest streaming data. You also have the option to use services such as AWS Lake Formation and Amazon HealthLake to quickly set up data lakes.
The following best practices are recommended for data collection and integration:
- Detail and document various sources and steps needed to extract the data. This can be achieved using AWS Glue Catalog, which automatically discovers and profiles your data, and generates ETL code to transform your source data to target schemas. AWS also recently announced a new feature named AWS Glue DataBrew, which provides a visual data preparation interface that makes it easy for data analysts and data scientists to clean and normalize data to prepare it for analytics and ML.
- Define data governance — Who owns the data, who has access, the appropriate usage of the data, and the ability to access and delete specific pieces of data on demand. Data governance and access management can be handled using AWS Lake Formation and AWS Glue Catalog.

Data integration and preparation

An ML model is only as good as the data being used to train it. Bad data is often referred to as “Garbage in, Garbage out”. Once the data has been collected, the next step is to integrate, prepare and annotate data. AWS provides a number of services that data engineers and data scientists can use to prepare their data for ML model training.
In addition to the services such as AWS Glue and Amazon EMR, which provide traditional ETL capabilities, AWS also provides tools as part of Amazon SageMaker, designed specifically for data scientists. These include:
- Amazon SageMaker Ground Truth, which can be used for data labeling
- SageMaker Data Wrangler, which simplifies the process of data preparation and feature engineering
- SageMaker Feature Store, which enables you to store, update, retrieve, and share ML features
Additionally, SageMaker Processing allows you to run your pre-processing, post- processing, and model evaluation workloads on a fully managed environment.
We recommend implementation of the following best practices for data integration and preparation:
- Track data lineage so that the location and data source is tracked and known during further processing. Using AWS Glue, you can visually map the lineage of their data to understand the various data sources and transformation steps that the data has been through. You can also use metadata provided by AWS Glue Catalog to establish data lineage. The SageMaker Data Wrangler Data Flow UI provides a visual map of the end-to-end data lineage.
- Versioning data sources and processing workflows — Versioning data sources processing workflows enables you to maintain an audit trail of the changes being made to your data integration processes over time, and recreate previous versions of your data pipelines. AWS Glue provides versioning capabilities as part of AWS Glue Catalog, and AWS Glue Schema Registry (for streaming data sources). AWS Glue and Amazon EMR jobs can be versioned using a version control system such as AWS CodeCommit.
- Automate data integration deployment pipelines — Minimize human touch points in deployment pipelines to ensure that the data integration workloads are consistently and repeatedly deployed, using a pipeline that defines how code is promoted from development to production. AWS Developer Tools allow you to build CI/CD pipelines to promote your code to a higher environment.

Feature engineering

Feature engineering involves the selection and transformation of data attributes or variables during the development of a predictive model. Amazon SageMaker Data Wrangler can be used for selection, extraction, and transformation of features.
You can export your data flow, designed in Data Wrangler, as a Data Wrangler Job, or export to SageMaker Pipelines.
ETL services like Amazon EMR and AWS Glue can be used for feature extraction and transformation. Finally, you can use Amazon SageMaker Feature Store to store, update, retrieve and share ML features.
The following best practices are recommended for feature engineering:
- Ensure feature standardization and consistency — It is common to see a different definition of similar features across a business. The use of Amazon SageMaker Feature Store allows for standardization of features, and helps to ensure consistency between model training and inference.
- If you are using SageMaker for feature engineering, you can use SageMaker Lineage Tracking to store and track information about the feature engineering steps (along with other ML workflow steps performed in SageMaker).

Model training

The model training step involves the selection of appropriate ML algorithms, and using the input features to train an ML model. Along with the training data (provided as input features prepared during the feature engineering stage), you generally provide model parameters to optimize the training process.
To measure how well a model is performing during training, AWS uses several metrics such as training error and prediction accuracy. Metrics reported by the algorithm depend on the business problem and the ML technique being used.
Certain model parameters, called hyperparameters, can be tuned to control the behavior of the model and the resulting model architecture. Model training typically involves an iterative process of training a model, evaluating its performance against relevant metrics, and tuning the hyperparameters in search for the most optimal model architecture.
This process is generally referred to as hyperparameter optimization. AWS recommends the application of the following best practices during the model training step:
- Follow a model testing plan and track your model experiments — Amazon SageMaker Experiments enables you to organize, track, compare, and evaluate ML experiments and model versions.
- Take advantage of managed services for model turning — SageMaker Automatic Model Tuning and SageMaker Autopilot help ML practitioners explore a large number of combinations to automatically and quickly zoom in on high- performance models.
- Monitor your training metrics to ensure your model training is achieving the desired results — SageMaker Debugger can be used for this purpose, which is designed to profile and debug your training jobs to improve the performance of ML models.
- Ensure traceability of model training as part of the ML lifecycle — SageMaker Lineage Tracking can be used for this purpose.

Model validation

After the model has been trained, evaluate it to determine if its performance and accuracy will enable you to achieve your business goals. Data scientists typically generate multiple models using different methods, and evaluate the effectiveness of each model.
The evaluation results inform the data scientists’ decision to fine-tune the data or algorithms to further improve the model performance.
During fine-tuning, data scientists might decide to repeat the data preparation, feature engineering, and model training steps. AWS recommends the following best practices for model validation:
- Keep track of the experiments performed to train models using different sets of features and algorithms — Amazon SageMaker Experiments, as discussed in the Model training section, can help keep track of different training iterations and evaluation results.
- Maintain different versions of the models and their associated metadata such as training and validation metrics in a model repository — SageMaker Model Registry enables you to catalog models for production, manage model versions, manage approval status of the models, and associate metadata, such as the training metrics of a model.
- Transparency about how a model arrives at their predictions is critical for regulators who require insights into how a model makes a decision — AWS recommends that you use model explainability tools, which can help explain how ML models make predictions. SageMaker Clarify provides the necessary tools for model explainability.
- Biases in the data can result in can introduce bias in ML algorithms, which can significantly limit the effectiveness of the models. This is of special significance in healthcare and life sciences, because poorly performing or biased ML models can have a significant negative impact in the real-world. SageMaker Clarify can be used to perform the post-training bias analysis against the ML models.

Additional considerations for AI/ML compliance

Additional considerations include:
- Auditability
- Traceability
- Reproducibility
- Model monitoring
- Model interpretability

Auditability

Another consideration for a well governed and secure ML environment is having a robust and transparent audit trail that logs all access and changes to the data and models, such as a change in the model configuration, or the hyperparameters.
AWS CloudTrail is one service that will log, nearly continuously monitor, and retain account activity related to actions across your AWS infrastructure. CloudTrail logs every AWS API call, and provides an event history of your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services.
Another service, AWS Config, enables you to nearly continuously monitor and record configuration changes of your AWS resources. More broadly, in addition to the logging and audit capabilities, AWS recommends a defense in depth approach to security, applying security at every level of your application and environment.
AWS CloudTrail and AWS Config can be used as Detective controls responsible for identifying potential security threats or incidents.
As the Detective controls identify potential threats, you can set up a corrective control to respond to and mitigate the potential impact of security incidents.
Amazon CloudWatch is a monitoring service for AWS resources, which can trigger CloudWatch Events to automate security responses. For details on setting up Detective and corrective controls, refer to Logging and Monitoring in AWS Glue.

Traceability

Effective model governance requires a detailed understanding of the data and data transformations used in the modeling process, in addition to nearly continuous tracking of all model development iterations.
It is important to keep track of which dataset was used, what transformations were applied to the data, where the dataset was stored, and what type of model was built.
Additional variables, such as hyperparameters, model file location, and model training metadata also need to be tracked. Any post-processing steps that have been applied to remove biases from predictions during batch inference also need to be recorded.
Finally, if a model is promoted to production for inference, there needs to be a record of model files/weights used in production, and model performance in production needs to be monitored.
One aspect of traceability that helps ensure you have visibility of what components or artifacts make their way into production, and how they evolve over time in the form of updates and patches, is the use of versioning.
There are three key components that provide versioning for different types of components involved in developing an ML solution:
- Using software version controls through tools such as GitHub to keep track of changes made to processing, training, and inference script. AWS provides a native version control system in the form of AWS CodeCommit that can be used for this purpose. Alternatively, you can use your own GitHub implementations.
- Using a model versioning capability to keep track of different iterations of models being created as part of iterative training runs. SageMaker Model Registry, which natively integrated with the wider SageMaker features, can be used for this purpose.
- Using a container repository to keep track of different container versions, which are used in SageMaker for processing, training, and inference. SageMaker natively integrates with Amazon ECR, which maintains a version of every container update.

Reproducibility

Reproducibility in ML is the ability to produce identical model artifacts and results by saving enough information about every phase in the ML workflow, including the dataset, so that it can be reproduced at a later date or by different stakeholders, with the least possible randomness in the process.
For GxP compliance, customers may need to reproduce and validate every stage of the ML workflow to reduce the risk of errors, and ensure the correctness and robustness of the ML solution.
Unlike traditional software engineering, ML is experimental, highly iterative, and consists of multiple phases that make reproducibility challenging. It all starts with the data. It’s important to ensure that the dataset is reproducible at each phase in the ML workflow.
Variability in the dataset could arise due to randomness in subsampling methods, creating train/validation/test splits and dataset shuffling.
Variability could also arise due to changes in the data processing, feature engineering, and post-processing scripts. Inconsistencies in any of these phases can lead to an irreproducible solution.
Methods that can help ensure reproducibility of the dataset as well as the data processing scripts include:
- Dataset versioning
- Using a fixed seed value across all the libraries in the code base
- Unit testing code to ensure that the outputs remain the same for a given set of inputs
- Version controlling the code base
The core components of the ML workflow are the ML models, which consist of a combination of model parameters and hyperparameters, which need to be tracked to ensure consistent and reproducible results.
In addition to these parameters, the stochastic (uncertain or random) nature of many ML algorithms adds a layer of complexity, because the same dataset along with the same code base could produce to different outputs.
This is more pronounced in deep learning algorithms, which make efficient approximations for complex computations. These results can be approximately reproduced with the same dataset, the same code base, and the same algorithm.
In addition to the algorithms, the underlying hardware and software environment configurations could impact reproducibility as well. Methods that can help ensure reproducibility and limit the number of sources of nondeterministic behavior in ML modeling include:
- Consistency in initializing model parameters
- Standardizing the infrastructure (CPUs and GPUs)
- Configuration management to ensure consistency in the runtimes, libraries and frameworks
When the solutions aren't fully deterministic, the need for quantifying the uncertainty in model prediction increases. Uncertainty quantification (UQ) plays a pivotal role in the reduction of uncertainties during optimization and decision making, and promotes transparency in the GxP compliance process.
A review of uncertainty quantification techniques, applications, and challenges in deep learning are presented in A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges.
Few methods for uncertainty quantification include:
- Ensemble learning techniques such as Deep Ensembles, which are generalizable across ML models and can be integrated into existing ML workflows.
- Temperature scaling, which is an effective post-processing technique to restore network calibration, such that the confidence of the predictions matches the true likelihood. Refer to a reference paper on calibrating neural networks.
- Bayesian neural networks with Monte Carlo dropout. For more information about these methods, refer to Methods for estimating uncertainty in deep learning.
Amazon SageMaker ML Lineage Tracking provides the ability to create and store information about each phase in the ML workflow. In the context of GxP compliance, this can help you establish model governance by tracking model lineage artifacts for auditing and compliance verification.
SageMaker ML Lineage Tracking tracks entities that are automatically created by SageMaker, or custom created by customers, to help maintain the representation of all elements in each phase of the ML workflow.

Model interpretability

Interpretability is the degree to which a human can understand the cause of a decision. The higher the interpretability of an ML model, the easier it is to comprehend the model’s predictions. Interpretability facilitates:
- Understanding
- Debugging and auditing ML model predictions
- Bias detection to ensure fair decision making
- Robustness checks to ensure that small changes in the input do not lead to large changes in the output
- Methods that provide recourse for those who have been adversely affected by model predictions
In the context of GxP compliance, model interpretability provides a mechanism to ensure the safety and effectiveness of ML solutions by increasing the transparency around model predictions, as well as the behavior of the underlying algorithm.
Promoting transparency is a key aspect of the patient-centered approach, and is especially important for AI/ML-based SaMD, which may learn and change over time.
There is a tradeoff between what the model has predicted (model performance) and why the model has made such a prediction (model interpretability).
For some solutions, a high model performance is sufficient; in others, the ability to interpret the decisions made by the model is key. The demand for interpretability increases when there is a large cost for incorrect predictions, especially in high-risk applications.

Trade-off between performance and model interpretability

Based on the model complexity, methods for model interpretability can be classified into intrinsic analysis and post hoc analysis.
- Intrinsic analysis can be applied to interpret models that have low complexity (simple relationships between the input variables and the predictions). These models are based on:
- Algorithms, such as linear regression, where the prediction is the weighted sum of the inputs
- Decision trees, where the prediction is based on a set of if-then rules
The simple relationship between the inputs and output results in high model interpretability, but often leads to lower model performance, because the algorithms are unable to capture complex non-linear interactions.
- Post hoc analysis can be applied to interpret simpler models, as described earlier, as well as more complex models, such as neural networks, which have the ability to capture non-linear interactions. These methods are often model- agnostic and provide mechanisms to interpret a trained model based on the inputs and output predictions. Post hoc analysis can be performed at a local level, or at a global level.
- Local methods enable you to zoom in on a single data point and observe the behavior of the model in that neighborhood. They are an essential component for debugging and auditing ML model predictions. Examples of local methods include:
  - Local Interpretable Model-Agnostic Explanations (LIME), which provides a sparse, linear approximation of the model behavior around a data point
  - SHapley Additive exPlanations (SHAP), a game theoretic approach based on Shapley values which computes the marginal contribution of each input variable towards the output
  - Counterfactual explanations, which describe the smallest change in the input variables that causes a change in the model’s prediction
  - Integrated gradients, which provide mechanisms to attribute the model’s prediction to specific input variables
  - Saliency maps, which are a pixel attribution method to highlight relevant pixels in an image
- Global methods enable you to zoom out and provide a holistic view that explains the overall behavior of the model. These methods are helpful for verifying that the model is robust and has the least possible bias to allow for fair decision making. Examples of global methods include:
- Aggregating local explanations, as defined previously, across multiple data points
- Permutation feature importance, which measures the importance of an input variable by computing the change in the model’s prediction due to permutations of the input variable
- Partial dependence plots, which plot the relationship and the marginal effect of an input variable on the model’s prediction
- Surrogate methods, which are simpler interpretable models that are trained to approximate the behavior of the original complex model
It is recommended to start the ML journey with a simple model that is both inherently interpretable and provides sufficient model performance.
In later iterations, if you need to improve the model performance, AWS recommends increasing the model complexity and leveraging post hoc analysis methods to interpret the results.
Selecting both a local method and a global method gives you the ability to interpret the behavior of the model for a single data point, as well as across all data points in the dataset.
It is also essential to validate the stability of model explanations, because methods in post-hoc analysis are susceptible to adversarial attacks, where small perturbations in the input could result in large changes in the output prediction and therefore in the model explanations as well.
Amazon SageMaker Clarify provides tools to detect bias in ML models and understand model predictions. SageMaker Clarify uses a model-agnostic feature attribution approach and provides a scalable and efficient implementation of SHAP.
To run a SageMaker Clarify processing job that creates explanations for ML model predictions, refer to Explainability and bias detection with Amazon SageMaker Clarify.

Model monitoring

After an ML model has been deployed to a production environment, it is important to monitor the model based on:
- Infrastructure — To ensure that the model has adequate compute resources to support inference workloads
- Performance — To ensure that the model predictions do not degrade over time
Monitoring model performance is more challenging, because the underlying patterns in the dataset are constantly evolving, which causes a static model to underperform over time.
In addition, obtaining ground truth labels for data in a production environment is expensive and time consuming. An alternative approach is to monitor the change in data and model entities with respect to a baseline.
Amazon SageMaker Model Monitor can help to nearly continuously monitor the quality of ML models in production, which may play a role in postmarket vigilance by manufacturers of Software as a Medical Device (SaMD).
SageMaker Model Monitor provides the ability to monitor drift in data quality, model quality, model bias, and feature attribution. A drift in data quality arises when the statistical distribution of data in production drifts away from the distribution of data during model training.
This primarily occurs when there is a bias in selecting the training dataset; for example, where the sample of data that the model is trained on has a different distribution than that during model inference, or in non-stationary environments when the data distribution varies over time.
A drift in model quality arises when there is a significant deviation between the predictions that the model makes and the actual ground truth labels.
SageMaker Model Monitor provides the ability to create a baseline to analyze the input entities, define metrics to track drift, and nearly continuously monitor both the data and model in production based on these metrics. Additionally, Model Monitor is integrated with SageMaker Clarify to identify bias in ML models.

Model deployment and monitoring for drift

For model monitoring, perform the following steps:

After the model has been deployed to a SageMaker endpoint, enable the endpoint to capture data from incoming requests to a trained ML model and the resulting model predictions.
Create a baseline from the dataset that was used to train the model. The baseline computes metrics and suggests constraints for these metrics. Real-time predictions from your model are compared to the constraints, and are reported as violations if they are outside the constrained values.
Create a monitoring schedule specifying what data to collect, how often to collect it, how to analyze it, and which reports to produce.
Inspect the reports, which compare the latest data with the baseline, and watch for any violations reported and for metrics and notifications from Amazon CloudWatch.

The drift in data or model performance can occur due to a variety of reasons, and it is essential for the technical, product, and business stakeholders to diagnose the root cause that led to the drift.

*Early and proactive detection of drift enables you to take corrective actions such as model retraining, auditing upstream data preparation workflows, and resolving any data quality issues.
If all else remains the same, then the decision to retrain the model is based on considerations such as:

Reevaluate target performance metrics based on the use-case
A tradeoff between the improvement in model performance vs. the time and cost to retrain the model
The availability of ground truth labeled data to support the desired retraining frequency
- After the model is retrained, you can evaluate the candidate model performance based on a champion/challenger setup, or with A/B testing, prior to redeployment.

Hope this guide helps you understand the Best practices for ML lifecycle stages.

Let me know your thoughts in the comment section 👇
And if you haven't yet, make sure to follow me on below handles:

👋 connect with me on LinkedIn
🤓 connect with me on Twitter
🐱‍💻 follow me on github
✍️ Do Checkout my blogs

Like, share and follow me 🚀 for more content.

Adit ModiFollow

👨‍💻 Join our Cloud Tech Slack Community
👋 Follow us on Linkedin / Twitter for latest news
💻 Take a Look at our Github Repos to know more about our projects
✍️ Our Website

Reference Notes

Machine Learning Lifecycle Process

Adit Modi — Mon, 31 Jan 2022 12:07:52 +0000

Building and operating a typical ML workload is an iterative process, and consists of multiple phases. We identify these phases loosely based on the open standard process model for Cross Industry Standard Process Data Mining (CRISP-DM) as a general guideline.
CRISP-DM is used as a baseline because it’s a proven tool in the industry and is application neutral, which makes it an easy-to-apply methodology that is applicable to a wide variety of ML pipelines and workloads.
The end-to-end ML process includes the following phases:
- Business goal identification
- ML problem framing
- Data collection
- Data integration and preparation
- Feature engineering
- Model training
- Model validation
- Business evaluation
- Production deployment (model deployment and model inference)
This article presents a high-level overview of the various phases of an end-to-end ML lifecycle, which helps frame our discussion around security, compliance, and operationalization of ML best practices which will be useful in our later blog posts.

The machine learning lifecycle process

The preceding figure describes the ML lifecycle process, along with the subject matter experts and business stakeholders involved through different stages of the process. It is also important to note that ML lifecycle is an interactive process.

The machine learning lifecycle process

Phase 1

Phase 1 is to define business problem and goals. Domain experts and business owners are most involved in this part, determining success metrics. KPIs and determining compliance and regulatory requirements also fall under this phase. Data scientists typically work with the SMEs to frame the business problem in a way that allows them to develop a viable ML solution.

Phase 2

Phase 2 involves gathering and preparing all relevant data from various data sources. This role is often performed by data engineers with expertise in big data tools for data extraction, transformation and loading (ETL). It is important to ensure that the data is versioned and the lineage of the data tracked for auditing and compliance purpose.
Once the raw datasets are available, data scientists perform data exploration, determine input features and target variables, outlier analysis, and necessary data transformations that may be needed. It is also important to ensure any transformations applied to training data can also be applied in production at inference time.

Phase 3

Next is the model development and model evaluation phase. Data scientists determine the framework they want to use, define out-of-sample, out-of-time datasets, and experiment with various ML algorithms, hyperparameters, in some cases, add additional training data.

Phase 4

Next, you take the trained models and run them on out of time and out of sample datasets, and pick the model or models that return the best results close to the metrics determined in Phase 1. Model artifacts and any corresponding code must be properly versioned and stored in a centralized code repository or in an artifact management system.
Note that this stage of the process is experimental, and data scientists may go back to the data collection or feature engineering stage if the model performance is consistently poor. More details on data and ML artifact lineage are available in the Traceability section of this document.
Data scientists are also required to provide reasons or explain feature/model influence on predictions. Model interpretability is discussed in later sections.

Phase 5

The next phase is to deploy the models into production. This is often the most impactful and difficult step because of the gap between technologies and skillsets used to build and deploy models in production.
A large part of making this successful requires intense collaboration among infrastructure professionals such as DevOps engineers, data scientists, data engineers, domain experts, end users, and business owners during the decision making process. There should be standardized metrics, and all decision makers should be able to interpret them directly.
In most organizations, lifecycles of ML models end with the deployment phase. There is a need for some form of shadow validation where models are deployed but not integrated in the production workflow to capture differences between training and live data.
This ensures that the model continues to perform as expected when receiving data from production systems. Once this validation proves successful, the model's predictions can be used in production workflows.
However, for ML models to be effective in the long run, continuously monitoring the model in real-time (if possible) to determine how well it is performing is necessary, as the accuracy of models can degrade over time.
If the performance of a model degrades below a certain threshold, you may need to retrain and redeploy your model.

Hope this guide gives you an high level overview of Different phrases of Machine Learning Lifecycle.

Let me know your thoughts in the comment section 👇
And if you haven't yet, make sure to follow me on below handles:

👋 connect with me on LinkedIn
🤓 connect with me on Twitter
🐱‍💻 follow me on github
✍️ Do Checkout my blogs

Like, share and follow me 🚀 for more content.

Adit ModiFollow

Reference Guide

Welcome to DEV, CLOUD TECH!

Adit Modi — Wed, 22 Dec 2021 10:23:59 +0000

Welcome to our Dev.to blog!

Everything about Cloud & Tech

At CloudTech, Our Goal is Built a community of self reliant Cloud Architects, Builders and Developers who are eager to help people get started with cloud along with building and developing applications which impact people around us.

We share latest news and articles related to Different Cloud Providers like AWS , Azure , GCP etc and much more.

We have decided to start this blog to help members in contributing to the community. Every member has a story to tell and by starting this blog, we aim to encourage each member to share their story and help inspire the next generation of cloud professionals and enthusiasts.

How do I contribute?

[1] Join our Cloud Tech Community

To contribute, you must be part of our Cloud Tech Community.
Join our Community here.

[2] Create an account here in Dev.to (if you haven't)

Dev.to is an awesome platform where developers are able to share their ideas. Sign up here for an account.

[3] Fill up the form

Fill up this form so we can properly contact you and add you to the group of Blog Contributors. We will also share with you the unique token that you will use to be able to post content in the Dev.to website.

[4] Let's get writing!

Once you are part of the community, you will be able to post to the Cloud Tech. We will strictly enforce our house rules below to ensure the community remains a safe space where members can share their ideas and start a discourse.

TLDR: Posts should be related to Cloud. It doesn't have to be technical in nature but it does need to somehow connect to above mentioned keywords. There should be no mention of competing products and it should not threaten or harass others.

Before you go, here are some house rules

The Cloud Tech is a safe space where people of any gender, nationality, association, sexual orientation can express their ideas and have a platform to share them with the world. But every community needs a set of rules to make sure everyone can take part in the community. Here are our house rules.

Each post should comply with the code of conduct of our platform, dev.to. You can view the document here: https://dev.to/code-of-conduct
On top of the platform code of conduct, our core team will also enforce:
- Each post should be related to Cloud. The community is a safe place. Any post that threatens, harass, causes harm to any member of the community or the dev.to user base in general will be taken down.
- Any post that blatantly sells products without regard to the community's objectives of sharing knowledge and promoting discourse will be taken down.
- We have no tolerance for plagiarized content. It will be taken down.
- The community is not a platform for users to reflect or assert their political and social views. Therefore, posts with political or social commentary will be taken down.

Let's get bloggin' 🥂

Introducing Karpenter – An Open-Source High-Performance Kubernetes Cluster Autoscaler

Adit Modi — Wed, 08 Dec 2021 08:25:29 +0000

At re:Invent 2021, AWS announced the v0.5.0 release of Karpenter, marking its open-source, Kubernetes node provisioning project as production ready. With Karpenter, Kubernetes users can now dynamically provision underlying compute nodes based on pod specifications more efficiently than the existing Kubernetes cluster-autoscaler project.

Karpenter is an open-source, flexible, high-performance Kubernetes cluster autoscaler built with AWS. It helps improve your application availability and cluster efficiency by rapidly launching right-sized compute resources in response to changing application load. Karpenter also provides just-in-time compute resources to meet your application’s needs and will soon automatically optimize a cluster’s compute resource footprint to reduce costs and improve performance.

Before Karpenter, Kubernetes users needed to dynamically adjust the compute capacity of their clusters to support applications using Amazon EC2 Auto Scaling groups and the Kubernetes Cluster Autoscaler. Nearly half of Kubernetes customers on AWS report that configuring cluster auto scaling using the Kubernetes Cluster Autoscaler is challenging and restrictive.

Kubernetes-native cluster autoscaler is now production-ready according to AWS.

So how does Karpenter work, and how is it different than cluster autoscaler?

Karpenter

If you are familiar with GKE Autopilot’s dynamic node provisioning process, you can view Karpenter as an open-source version of that tool, designed to work with any Kubernetes cluster (NOTE: currently AWS is the only officially supported cloud provider).
Similar to GKE Autopilot, Karpenter observes the pod specifications of unschedulable pods, calculates the aggregate resource requests, and sends a request to the underlying compute service (e.g. Amazon EC2) with capacity needed to run all the pods.
Underneath the hood, Karpenter defines a Custom Resource called Provisioner to specify the node provisioning configuration including instance size/type, topology (e.g. zone), architecture (e.g. arm64, amd64), and lifecycle type (e.g. spot, on-demand, preemptible).

On the flip-side, Karpenter can also deprovision nodes when they are no longer needed. This can be determined by node expiry config (ttlSecondsUntilExpired) or when the last workload running on Karpenter provisioned node is terminated.
Either of these two events triggers a finalization, which cordons the nodes, drains the pods, terminates the underlying compute resource, and deletes the node object. This deprovisioning feature can also be used to keep the nodes up to date with the latest AMI as well.

Karpenter vs. Cluster Autoscaler

At a cursory look, Karpenter works similarly to the existing Kubernetes cluster autoscaler project. After all, cluster autoscaler is also cloud-agnostic and can scale up or down based on pod resource requests.
Upon a closer look, however, Karpenter provides several advantages over cluster autoscaler:
- Kubernetes native scaling: Cluster autoscaler for AWS utilizes EC2 Auto Scaling groups to trigger scaling events. Since ASGs were designed before Kubernetes, this integration is clunky and slow. For example, managed node group users still can’t configure it to scale nodegroups to 0, making batch workload types more expensive to run on EKS.
- No need to pre-provision node groups: Cluster autoscaler can only provision nodes based on specifications provided by node groups, which require worker groups to have specific tags and work best with similar instance types. This meant that if you wanted to run performance tests, you needed to predefine a node group with beefier EC2 machine types for cluster autoscaler to trigger scaling events. With Karpenter, you can utilize all of AWS instance types on demand. Since Karpenter manages each instance directly without node groups, it is also much faster to request a new compute instance when capacity is unavailable.
- Faster scheduling: With cluster autoscaler, pods rely on kube-scheduler to create pods to new nodes once new resources become available. Since Karpenter manages nodes directly, it can immediately launch pods to new nodes without having to wait for the scheduler.

Conclusion

With Karpenter, we can offload node provisioning, autoscaling, and upgrades and focus on running our applications. Karpenter works with all kinds of Kubernetes applications, but it performs particularly well for use cases that require rapid provisioning and deprovisioning large numbers of diverse compute resources quickly. For example, this includes batch jobs to train machine learning models, run simulations, or perform complex financial calculations.

Let me know your thoughts in the comment section about the new karpenter service 👇
And if you haven't yet, make sure to follow me on below handles:

👋 connect with me on LinkedIn
🤓 connect with me on Twitter
🐱‍💻 follow me on github
✍️ Do Checkout my blogs

Like, share and follow me 🚀 for more content.

Adit ModiFollow

Amazon launches AWS RoboRunner to support robotics apps & much more

Adit Modi — Mon, 29 Nov 2021 09:10:48 +0000

At a keynote during its Amazon Web Services (AWS) re:Invent 2021 conference today, Amazon launched AWS IoT RoboRunner, a new robotics service designed to make it easier for enterprises to build and deploy apps that enable fleets of robots to work together.
Alongside IoT RoboRunner, Amazon announced the AWS Robotics Startup Accelerator, an incubator program in collaboration with nonprofit MassRobotics to tackle challenges in automation, robotics, and industrial internet of things (IoT) technologies.
As pandemics drive digital transformation, enterprises are accelerating the adoption of robotics and, more broadly, automation. recently report Automation World companies have found that most of the companies that have adopted robotics over the past year have adopted it to reduce labor costs, increase capacity, and overcome the shortage of available workers.
According to the same survey, 44.9% of companies now consider robots in assembly and manufacturing facilities to be an integral part of their day-to-day operations.
Amazon — a large investor in robotics itself — wasn’t shy about its intention to win most of the robotics software market. is expected It is worth more than $ 7.52 billion by 2022. In 2018, the company announced AWS RoboMaker, A product that assists developers in deploying robot applications with AI and machine learning capabilities. And Amazon earlier this year Rolled out SageMaker Reinforcement learning Kubeflow components, a toolkit that supports RoboMaker services for tuning robotics workflows.

IoT RoboRunner

Currently previewing IoT RoboRunner is built on technology already used in Amazon Warehouse for robotics management. This allows AWS customers to connect robots to existing automation software and combine each type of data to coordinate the work of the entire operation.
Fleet robot Standardize data types such as facilities, locations, and robot task data in the central repository.
the goal of IoT RoboRunner is to simplify the process of building a robot group management app. As companies become more and more dependent on robotics to automate their operations, they are choosing different types of robots, making it more difficult to organize robots efficiently.
Each robot vendor and work management system has its own, often incompatible control software, data formats, and data repositories. Also, as new robots are added to the fleet, they will need to be programmed to connect the control software to the workflow management system and program the logic of the management app.
Developers can use IoT RoboRunner to access the data needed to build robot management apps and leverage pre-built software libraries to create apps for tasks such as work assignments.
In addition to this, you can use IoT RoboRunner to deliver metrics and KPIs to the management dashboard via the API.
With AWS IoT RoboRunner, robot developers no longer have to manage their robots in silos, and centralized management can more effectively automate tasks across the facility.
AWS IoT RoboRunner lets you connect your robots and work management systems, thereby enabling you to orchestrate work across your operation through a single system view.

AWS Robotics Startup Accelerator

Amazon also announced the Robotics Startup Accelerator. The company says it will foster robotics by providing resources to develop, prototype, test, and commercialize products and services.
The AWS Robotics Startup Accelerator delivered by MassRobotics aims to help robotics startups adopt and use AWS to boost their robotics development, as well as get hands-on support from industry and AWS experts to rapidly scale their business.
As the trend towards automation continues, robotics start-ups, especially industrial robotics, are attracting the attention of venture capitalists. From March 2020 to March 2021, venture companies invested $ 6.3 billion in robotics companies, an increase of about 50% from March 2019 to March 2020. according to To the data from PitchBook.
In the long run, investment in robotics has more than quintupled over the past five years, rising from $ 1 billion in 2015 to $ 5.4 billion in 2020.
The accelerator is a four-week technical, business, and mentorship opportunity open to robotics hardware and software startups from around the globe.
Startups accepted into the four-week program will consult with AWS and MassRobotics industry experts on business models and with AWS robotics experts for help overcoming technological blockers.
Program benefits for startups include hands-on training about AWS solutions for robotics and up to $10,000 in promotional credits for use of AWS IoT, Robotics, and ML services to help guide them forward.
Participants will gain additional knowledge through mentoring from robotics domain experts and technical subject matter experts. To get ready for life after the accelerator, startups will also get business development and investment guidance from MassRobotics, and co-marketing opportunities with AWS via blogs and case studies.
Startups interested in applying to be part of the program can learn more here. Applications close on Sunday, January 16, 2022.

Let me know your thoughts in the comment section about the new aws services and accelerator program 👇
And if you haven't yet, make sure to follow me on below handles:

👋 connect with me on LinkedIn
🤓 connect with me on Twitter
🐱‍💻 follow me on github
✍️ Do Checkout my blogs

Like, share and follow me 🚀 for more content.

Adit ModiFollow

IaaS vs. PaaS vs. SaaS

Adit Modi — Mon, 21 Jun 2021 03:15:09 +0000

An increasing number of businesses are choosing cloud services. If you aren’t familiar with this topic, cloud computing is when hardware (servers, storage, etc.) and software are delivered over the internet.

Compared to on-premises hardware and software, cloud-based solutions such as IaaS, PaaS, and SaaS offer several major benefits. Let’s briefly mention these benefits in order to understand why cloud computing is so popular in 2019.

Scalability.

On-premises solutions are rather difficult to scale, as the type of hardware needed depends on your application’s demands. If your app experiences heavy traffic, you might need to significantly upgrade on-premises hardware. This problem doesn’t exist with a cloud service, which you can quickly scale up or down with a few clicks. Cloud services are a perfect solution for handling peak loads. With cloud-based services, businesses can use whatever computing resources they need.

Cost-effectiveness.

Cloud computing removes hardware expenses, as hardware is provided by a vendor. There’s no need to buy, install, configure, and maintain servers, databases, and other components of your runtime environment. Moreover, using cloud-based solutions, you pay only for what you use, so if you don’t need extra resources you can simply scale down and not pay for them.

Immediate availability.

Cloud solutions are available as soon as you’ve paid for them, so you can start using a cloud service right away. There’s no need to install and configure hardware.

Performance.

Cloud companies equip their data centers with high-performance computing infrastructure that guarantees low network latency for your applications.

Security.

Cloud infrastructure is kept in safe data centers to ensure a top level of security. Data is backed up and can easily be recovered. Moreover, cloud vendors ensure the security of your data by using networking firewalls, encryption, and sophisticated tools for detecting cybercrime and fraud.

The advantages of cloud solutions are huge, so it stands to reason that the cloud services market is booming. According to a forecast by Gartner, the global public cloud services market is expected to reach almost $247 billion this year and grow to over $383 billion by 2021.

Global Market of Public Cloud Services

Yet choosing the right cloud service can be rather challenging. Many people have no idea what SaaS, IaaS, and PaaS mean or which of these cloud solutions they need for their projects.

What do IaaS, PaaS, and SaaS mean?

There are three major types of cloud services: IaaS, PaaS, and SaaS. You’ve probably seen these abbreviations on the websites of cloud providers. Before going into details, let’s compare IaaS, PaaS, and SaaS to transportation:

Cloud Services Compared to Means of Transport

On-premises IT infrastructure is like owning a car. When you buy a car, you’re responsible for its maintenance, and upgrading means buying a new car.

IaaS is like leasing a car. When you lease a car, you choose the car you want and drive it wherever you wish, but the car isn’t yours. Want an upgrade? Just lease a different car!

PaaS is like taking a taxi. You don’t drive a taxi yourself, but simply tell the driver where you need to go and relax in the back seat.

SaaS is like going by bus. Buses have assigned routes, and you share the ride with other passengers.

These analogies will help you better understand our more detailed explanations. Let’s give a definition to each of these terms.

Software as a Service (SaaS)

SaaS allows people to use cloud-based web applications.

In fact, email services such as Gmail and Hotmail are examples of cloud-based SaaS services. Other examples of SaaS services are office tools (Office 365 and Google Docs), customer relationship management software (Salesforce), event management software (Planning Pod), and so on.

SaaS services are usually available with a pay-as-you-go (which means subscription) pricing model. All software and hardware are provided and managed by a vendor, so you don’t need to install or configure anything. The application is ready to go as soon as you get your login and password.

Software as a Service (SaaS)

Managed by you Managed by vendor

–Hosted applications
Development and management tools
Operating system
Servers and storage
Networking resources
Data center
Perfect for: end users

Platform as a Service (PaaS)

PaaS refers to cloud platforms that provide runtime environments for developing, testing, and managing applications.

Thanks to PaaS solutions, software developers can deploy applications, from simple to sophisticated, without needing all the related infrastructure (servers, databases, operating systems, development tools, etc). Examples of PaaS services are Heroku and Google App Engine.

PaaS vendors supply a complete infrastructure for application development, while developers are in charge of the code.

Just like SaaS, Platform as a Service solutions are available with a pay-as-you-go pricing model.

Platform as a Service (PaaS)

Managed by you Managed by vendor
Hosted applications
Development and management tools
Operating system
Servers and storage
Networking resources
Data center
Perfect for: software developers

Infrastructure as a Service (IaaS)

IaaS is a cloud service that provides basic computing infrastructure: servers, storage, and networking resources. In other words, IaaS is a virtual data center.

IaaS services can be used for a variety of purposes, from hosting websites to analyzing big data. Clients can install and use whatever operating systems and tools they like on the infrastructure they get. Major IaaS providers include Amazon Web Services, Microsoft Azure, and Google Compute Engine.

As with SaaS and PaaS, IaaS services are available on a pay-for-what-you-use model.

Infrastructure as a Service (IaaS)

Managed by you Managed by vendor
Hosted applications Servers and storage
Development and management tools Networking resources
Operating system Data center
Perfect for: IT administrators

As you can see, each cloud service (IaaS, PaaS, and SaaS) is tailored to the business needs of its target audience. From the technical point of view, IaaS gives you the most control but requires extensive expertise to manage the computing infrastructure, while SaaS allows you to use cloud-based applications without needing to manage the underlying infrastructure. Cloud services, thus, can be depicted as a pyramid:

IaaS, PaaS, SaaS Hierarchy Diagram

Now that you know what SaaS, PaaS, and IaaS mean, let’s be more specific about when each should be used and what their advantages and disadvantages are.

When and Why You Should Use SaaS

We’ve already mentioned some examples of SaaS solutions, so you have a general understanding of when they’re used. Let’s provide some more details.

SaaS solutions can be used for:

Personal purposes.

Millions of individuals all over the world use email services (Gmail, Hotmail, Yahoo), cloud storage services (Dropbox, Microsoft OneDrive), cloud-based file management services (Google Docs), and so on. People may not realize it, but all of these cloud services are actually SaaS services.

Business.

Companies of various sizes may use SaaS solutions such as corporate email services (Gmail is available for businesses, for example), collaboration tools (Trello), customer relationship management software (Salesforce, Zoho), event management software (EventPro, Cvent), and enterprise resource planning software (SAP S/4HANA Cloud ERP).

SaaS services offer plenty of advantages to individuals and businesses:

Access to applications from anywhere.

Unlike on-premises software, which can be accessed only from a computer (or a network) it’s installed on, SaaS solutions are cloud-based. Thus, you can access them from anywhere there’s internet access, be it your company’s office or a hotel room.
Can be used from any device. Cloud-based SaaS services can be accessed from any computer. You only need to sign in. Many SaaS solutions have mobile apps, so they can be accessed from mobile devices as well.

Automatic software updates.

You don’t need to bother updating your SaaS software, as updates are carried out by a cloud service vendor. If there are any bugs or technical troubles, the vendor will fix them while you focus on your work instead of on software maintenance.

Low cost.

Compared to on-premises software, SaaS services are rather affordable. There’s no need to pay for the whole IT infrastructure; you pay only for the service at the scale you need. If you need extra functionality, you can always update your subscription.

Simple adoption.

SaaS services are available out-of-the-box, so adopting them is a piece of cake. We’ve already mentioned what you need to do: just sign up. It’s as simple as that. There’s no need to install anything.

Of course, SaaS solutions have certain disadvantages as well, so let’s mention a couple of them:

You have no control over the hardware that handles your data.
Only a vendor can manage the parameters of the software you’re using.

When and Why You Should Use PaaS

PaaS solutions are used mostly by software developers. PaaS provides an environment for developing, testing, and managing applications. PaaS is therefore the perfect choice for software development companies.

No wonder that software developers use PaaS services such as Heroku, Elastic Beanstalk (offered by Amazon Web Services), and Google App Engine.

PaaS provides a number of benefits to developers:

Reduced development time.

PaaS services allow software developers to significantly reduce development time. Server-side components of the computing infrastructure (web servers, storage, networking resources, etc.) are provided by a vendor, so development teams don’t need to configure, maintain, or update them. Instead, developers can focus on delivering projects with top speed and quality.

Support for different programming languages.

PaaS cloud services usually support multiple programming languages, giving developers an opportunity to deliver various projects, from startup MVPs to enterprise solutions, on the same platform.

Easy collaboration for remote and distributed teams.

PaaS gives enormous collaboration capabilities to remote and distributed teams. Outsourcing and freelancing are common today, and many software development teams are comprised of specialists who live in different parts of the world. PaaS services allow them to access the same software architecture from anywhere and at any time.

High development capabilities without additional staff.

PaaS provides development companies with everything they need to create applications without the necessity of hiring additional staff. All hardware and middleware is provided, maintained, and upgraded by a PaaS vendor, which means businesses don’t need staff to configure servers and databases or deploy operating systems.
Of course, PaaS cloud services have certain disadvantages:

You have no control over the virtual machine that’s processing your data.
PaaS solutions are less flexible than IaaS. For example, you can’t create and delete several virtual machines at a time.

When and Why You Should Use IaaS

IaaS solutions can be used for multiple purposes. Unlike SaaS and PaaS, IaaS provides hardware infrastructure that you can use in a variety of ways. It’s like having a set of tools that you can use for constructing the item you need.

Here are several scenarios when you can use IaaS:

Website or application hosting.

You can run your website or application with the help of IaaS (for example, using Elastic Compute Cloud from Amazon Web Services).
Virtual data centers.

IaaS is the best solution for building virtual data centers for large-scale enterprises that need an effective, scalable, and safe server environment.

Data analysis.

Analyzing huge amounts of data requires incredible computing power, and IaaS is the most economical way to get it. Companies use Infrastructure as a Service for data mining and analysis.
Infrastructure as a Service provides the following major advantages for businesses:

No expenses on hardware infrastructure.

IaaS vendors provide and maintain hardware infrastructure: servers, storage, and networking resources. This means that businesses don’t need to invest in expensive hardware, which is a substantial cost savings as IT hardware infrastructure is rather pricey.

Perfect scalability.

Though all cloud-based solutions are scalable, this is particularly true of Infrastructure as a Service, as additional resources are available to your application in case of higher demand. Apps can also be scaled down if demand is low.

Reliability and security.

Ensuring the safety of your data is a IaaS vendor’s responsibility. Hardware infrastructure is usually kept in specially designed data centers, and a cloud provider guarantees security of your data.

Finally, let’s specify the disadvantages of IaaS cloud solutions:

IaaS is more expensive than SaaS or PaaS, as you in fact lease hardware infrastructure.

All issues related to the management of a virtual machine are your responsibility.

IaaS vs PaaS vs SaaS: Which Cloud Service Is Suitable for You?

It’s time to pick which cloud-based service you need. In fact, the choice totally depends on your business goals, so first of all consider what your company needs. Here are some common business needs that can easily be met with the appropriate cloud service:

If your business needs out-of-the-box software (CRM, email, collaboration tools, etc.), choose Software as a Service.
If your company requires a platform for building software products, pick Platform as a Service.
If your business needs a virtual machine, opt for Infrastructure as a Service.

Hope this guide helps you understand different Cloud Services like IaaS , PaaS and SaaS, feel free to connect with me on LinkedIn.
You can view my badges here.
If you are interested in learning more about AWS Services then follow me on github.
If you liked this content then do clap and share it . Thank You .

The Advantages of Adopting Cloud Computing Services

Adit Modi — Fri, 18 Jun 2021 06:35:30 +0000

As recently as just a few years ago, business leaders worried about the unknown factors of moving their core applications to the cloud. Factors like not knowing how a particular application might work in the cloud were deterrents to what would have been optimal cloud migration, according to Tech Target. With new attention to performance metrics and understanding how their applications will scale with cloud services, organizations are finding cloud computing companies more attractive all the time.

CMS Wire confirms the rapidly increasing adoption of cloud computing services, reporting that enterprises are more willing than ever to stretch beyond the boundaries of their on-premises data center systems to invest in IT infrastructure that supports deployment in cloud environments. This large-scale migration to the cloud—cutting across various industries and a range of business sizes—is set to result in a projected $266 billion annual spending on IT infrastructure and cloud services by 2021.

The Advantages of Adopting Cloud Computing Services

As you now seriously consider migrating your data and/or applications to one of the high-caliber cloud computing companies like Amazon Web Services, it may help you to learn just a few of the many ways your company can benefit from the advantages the cloud offers.

Automated Application Deployments and Data Backups -Managing the deployment of new code, security patches, data backup, and disaster recovery plans are crucial tasks for a software development company and should not be taken lightly. Even if your company is not a software development company, you still have to worry about security patches, data backup, and disaster recovery. Fortunately, these are all things that can be automated and managed more easily on the cloud. Cloud service providers like Amazon Web Services (AWS) provide the tools and automation necessary to make managing your deployments and data backups something you don't have to worry about.

Reduce and Streamline IT Infrastructure Costs -By adopting cloud services, you can say goodbye to the costly need to purchase servers that require expert installation, regular maintenance and full replacement every few years. You can also reduce the staffing costs you once spent to manage your detail-heavy infrastructure to focus your human resource allowance on tasks like tending to help desk issues and managing your system's internal controls.

Continually Available Infrastructure -The virtualized servers in cloud computing companies do not rely on specific hardware, so they are always available to your users via office computers, remote laptops or on mobile devices. Common hardware issues, such as hard-drive failures, no longer have any direct impact on your users' ability to access data and applications stored in the cloud.

Flexible, Scalable, and Resilient Services -Whether you own a small business and anticipate growth in the coming years, or you plan to further expand your large-scale operations geographically, cloud service providers like Amazon Web Services offer the flexibility, scalability, and resilience that your enterprise requires. No matter what changes you have in mind for your organization, this trio of advantages means that you can grow your company without worrying about impacting your users' needs. You can add more storage space as needed while your users—across town or around the world—can access your database or applications with ease and without interruption.

Trade capital expense for variable expense –Instead of having to invest heavily in data centers and servers before you know how you’re going to use them, you can pay only when you consume computing resources, and pay only for how much you consume.

Benefit from massive economies of scale –By using cloud computing, you can achieve a lower variable cost than you can get on your own. Because usage from hundreds of thousands of customers is aggregated in the cloud, providers such as AWS can achieve higher economies of scale, which translates into lower pay as-you-go prices.

Stop guessing capacity –Eliminate guessing on your infrastructure capacity needs. When you make a capacity decision prior to deploying an application, you often end up either sitting on expensive idle resources or dealing with limited capacity. With cloud computing, these problems go away. You can access as much or as little capacity as you need, and scale up and down as required with only a few minutes’ notice.

Increase speed and agility –In a cloud computing environment, new IT resources are only a click away, which means that you reduce the time to make those resources available to your developers from weeks to just minutes. This results in a dramatic increase in agility for the organization, since the cost and time it takes to experiment and develop is significantly lower.

Stop spending money running and maintaining data centers -Focus on projects that differentiate your business, not the infrastructure. Cloud computing lets you focus on your own customers, rather than on the heavy lifting of racking, stacking, and powering servers.

Go global in minutes –Easily deploy your application in multiple regions around the world with just a few clicks. This means you can provide lower latency and a better experience for your customers at minimal cost.

Conclusion

Cloud computing continues to grow and will likely continue to do so. The low-cost infrastructure for enterprise solutions combined with the high-value services has resulted in cloud services being consistently in-demand.

The added cloud computing benefits like mobility and improvement to business insights all contribute to pushing cloud tech-forward.

Numbers from Statista prove why cloud computing is such a big deal:

The size of the cloud computing market has exceeded $146 billion
Global cloud data center traffic topped 10.6 zettabytes in 2020
Demand for public cloud services is expected to grow more than 17 percent this year
Growth is expected among all types of cloud services, with businesses at all levels switching to cloud-based tools and storage.

Hope this guide helps you understand The Advantages of Adopting Cloud Computing Services, feel free to connect with me on LinkedIn.
You can view my badges here.
If you are interested in learning more about AWS then follow me on github.
If you liked this content then do clap and share it . Thank You .

Comparing Managed Kubernetes Services: EKS vs. AKS vs. GKE

Adit Modi — Fri, 28 May 2021 03:27:36 +0000

The way organizations are using Kubernetes has quickly evolved in the past years. All the giant cloud providers offer managed Kubernetes services for their customers so that they can easily automate the deployment, scale, and manage their containerized applications.

But how do these platforms perform? Do they live up to the hype? How well do they integrate? What’s it like maintaining and working with them? That’s why in this article, we reviewed the Managed Kubernetes solutions from the top cloud providers: Amazon Elastic Kubernetes Service (EKS) from Amazon, Google Kubernetes Engine (GKE) from Google Cloud Platform and Azure Kubernetes Service (AKS) from Microsoft Azure.

It is best to get a deep understanding and look beyond the price and consider factors like scalability, security, features before making a final decision. We’ve also decided to group the different features available for each managed Kubernetes services in this blog. So, let’s dig in.

General Overview

Amazon Elastic Kubernetes Service

Amazon Elastic Kubernetes Service (EKS) is a managed service publicly available in June 2018 to run Kubernetes on AWS. It can be integrated easily with all the services, apps, and protocols that run on a Kubernetes Environment.

EKS is designed entirely around Kubernetes, so everything you need to manage and deploy containers is included. Whether its seamless integration with the third-party tools for logs and performance metrics or advanced scaling capabilities.

EKS is an excellent option if you already have an established AWS cloud architecture and want to experiment with Kubernetes or looking forward to migrating workloads on different clouds

Azure Kubernetes Service

Azure Kubernetes Service (AKS) is a managed Kubernetes solution that was made available in 2018 by Microsoft. This is a fully managed service that makes containerized apps easy to deploy and manage in the Kubernetes environment.

AKS runs both on Azure Public Cloud, on-premises, which helps deliver mission-critical applications to customers. AWS also has Government Cloud support for Government and their partners to run sensitive workloads.

AKS is worth it when it comes to seamless integration with its tools, including Visual Studio and Active Directory and the rest of the Microsoft Cloud SaaS services. If you have an established enterprise agreement with Microsoft, and no preference for any other architecture, then AKS will perfectly suit your requirements.

Google Kubernetes Engine

Kubernetes itself started as a Google’s internal project, so it makes sense that they were the first to deliver managed Kubernetes solution in 2014, known as Google Kubernetes Engine(GKE).

Google Kubernetes Engine (GKE), as a managed production-grade container orchestration engine, is the most resilient and well-rounded Kubernetes offering when compared to AKS and EKS. It has support for the Istio service mesh out of the box, and gVisor for an extra layer of security between running containers. Also, one of the key benefits of GKE is that service upgrades and new versions are instantly available while other providers take time to provide update releases.

High availability of Clusters

High availability of clusters is crucial if you are running production critical applications on Kubernetes. It ensures your cluster will be available when something goes wrong. For instance, if your services rely on a single data center and it goes down, your services will not be interrupted.

GKE offers excellent support for highly available clusters in two modes: multi-zonal and regional. There is just one master node in multi-zonal mode, yet there can be many worker nodes across various zones. In regional mode, the master nodes are likewise spanned over all the regional zones to provide superior HA.

AKS doesn’t provide high availability for their master nodes, as of date. However, the nodes are deployed in Availability Zones, for greater availability.

EKS likewise provides HA across both workers, and master nodes spanned over different accessibility zones in the same way as GKE’s.

SLA

SLA (service level agreement) is a powerful acronym in all industries, and it is no different within the cloud community.

All cloud platform providers offer different SLA’s, which guarantees different uptimes according to their availability zones and their region of deployment. For example, Amazon EKS guarantees 99.95% uptime, AKS offers 99.95% when availability zones are enabled, and 99.9% when disabled, andGKE splits its managed Kubernetes clusters, offering 99.5% uptime for Zonal deployments and 99.95% for regional deployments.

Differences in SLA for Kubernetes control planes also present another area to compare. A Kubernetes control plane is a management infrastructure implemented by the cloud provider to efficiently perform all the essential processes for running your worker nodes.

It varies by different cloud providers. AKS Control Plane comes free of cost, and you do not pay anything for it. GKE was also free of cost initially, but they have announced they will start charging soon for the control plane. EKS initially charge for their control plan right from the beginning.

Resource Availability with Node Pools

For different types of workflows, different kinds of machines are allocated to clusters by node pools. For example, storage systems require better storage disks than workflows like visual data analysis, requiring a better CPU and GPU. With node pools, we can provide the best resource available for specific nodes and provide optimal performance on those nodes while not allocating resources to all the cluster nodes.

GKE and EKS are leading in this since they both provide functionality for node pooling. But AKS, on the other hand, does not provide node grouping and recommends different clusters in different scenarios.

Scalability

GKE, AKS, and EKS all provide you the ability to scale up nodes very quickly, just by using the UI. In autoscaling, GKE is leading as the most mature solution available on the interface. What a user needs to do is just specify the desired VM size and the range of nodes in the node pool. And the rest of the steps are managed by Google Cloud. EKS and AKS come after GKE in auto-scaling because they need some manual configurations to setup.

GKE and AKS also provides further customization in its ability to scale up. Unlike EKS. In GKE and AKS, you can configure to a cluster using the Cluster Autoscaler, which will scale your nodes up or down based on the required workload. That is especially helpful when you have to run short-lived processes. EKS can also implement auto-scaling policies, but you have to set them manually compared to GKE and AKS, which provides Cluster Autoscaler by default.

Bare Metal Clusters

As understandable by name, Bare metal clusters are deployed on a cloud architecture with no virtualization layer in between, in other words, no VMs. There are various benefits of this technique. The infrastructure overhead reduced drastically, which provide access to more computing and storage resources for application deployments. Access to more computing resources also increases the computing power, which helps reducing latency and downtimes for a particular application request.

Coming to GKE vs AKS vs EKS for bare-metal performance. EKS allows the use of bare metal nodes. GKE and AKS do not support bare metal nodes. Although EKS does not default to bare metal nodes as they are expensive to deploy.

Resource Limits

Resource limits are handled in a different way across providers — limits are per account with EKS, where AKS handle limits per subscription, and GKE balances limits on a per-project basis. EKS offers a maximum of 100 nodes per cluster, per account. AKS offers 500 nodes per cluster, and GKE offers 5000 nodes per cluster per project.

While most limits look clear on paper, some are not. In AKS, for example, the maximum number of nodes that you can have depends on whether the node is available in State Set or Availability Set. On the other hand, In EKS, the maximum number of nodes per cluster you can get varies on the node’s instance type. Whereas GKE provides you more highly available nodes without any location variables.

Resource monitoring

In terms of resource monitoring, all three cloud providers have offerings. GKE uses Stackdriver for resource monitoring within their Kubernetes cluster. It monitors the master and worker nodes, and all Kubernetes components inside the platform along with logging. AKS offers Azure Monitor to evaluate the health of a container and Application Insights to monitor the Kubernetes components. EKS requires the use of third-party tools and recommends Prometheus for resource monitoring.

Role-based access control (RBAC)

Role-based access control (RBAC) in Kubernetes allows admins to configure dynamic policies to deny unauthorized access. All the three hosted services providers provide RBAC implementations, but they set it differently. EKS has a slight advantage, with a tighter security policy overall, as it considers RBAC and pod security policies mandatory compared to GKE and AKS.

Availability as a Cloud Provider

All three providers have their offerings available in most regions globally. Google Cloud has the best availability among these three globally. It has services in almost every region following the lead is Azure. Azure comes above AWS after launching services in Latin America and Africa,

Also, EKS is not available in the AWS government cloud; AKS, however, has one Azure government cloud, whereas Google has no government clouds.

Secure Image Management

All three cloud providers offer container image registry services that provide secure image management and stable build creation. But the degree of control these cloud provider provides varies.

The image signing feature of Azure Container Registry (ACR) provides users with the ability to check their container images’ authenticity. In the same way, immutable image tags in Elastic Container Registry (ECR) allows users to create a secure container builds at all times.

Lastly, Binary Authorization in GKE prevents deployment of images conflicting with the set policies and triggers the automatic lock-down of those risky images.

Elastic Container Registry and Azure Container Registry also support resource-based permissions for access controls on a repository level to prevent unauthorized access, which Google container registry does not.

Pricing

Each vendor has its specific features, limitations, and pricing plans. Management and deployment of clusters that include master and worker machines running provided at no cost by GKE and AKS. You are charged only for services that you use on the go, such as bandwidth, storage, and virtual machines. In comparison, Amazon EKS costs you $0.10 per hour for each deployed cluster other than the instances and services you are using.

Concerning the price overall, here are some rough figures to help you determine costs when choosing a Kubernetes platform. This cost comparison assumes that you have 20 worker nodes, and each node has 80 CPU and 320GB of RAM. Billable hours: 14,600 hours per month.

Final Words

AWS, Microsoft Azure, and Google Cloud Platform all are claiming to be the best Managed Kubernetes solution from the past years. It’s on you. Whether you want the advantage of the most mature and budget product of Google or you want to leverage your Microsoft Enterprise Agreement to get better pricing and support on Azure, or you want to make your transition to the cloud easier with EKS on Amazon.

To find that, its always important to compare storage, network, and compute features for each provider before you decide for a managed Kubernetes service. It is also critical to compare the costs since services can vary between regions and are different for each configuration.

Completely testing service’s features and capabilities in your environment will ultimately provide the real-time pricing and performance metrics, which will help determine a Kubernetes offering that perfectly suits your business needs.

Hope this guide helps you understand the Different Managed Kubernetes Services: EKS vs. AKS vs. GKE, feel free to connect with me on LinkedIn.
You can view my badges here.
If you are interested in learning more about AWS then follow me on github.
If you liked this content then do clap and share it . Thank You .

Database Backup Scripts For MongoDB with Amazon S3

Adit Modi — Wed, 26 May 2021 07:23:11 +0000

This article will provide you with database backup scripts that not only allow you to create database backups, but also upload the backup dumps to Amazon S3 and automate the process daily.

->Why we need a database backup?
->Why Amazon S3 for backup?
->What is Cron?
->What is Chmod?
->Database Backup Script for MongoDB and Dumping to Amazon S3
->Generate a shell script which will dump the MongoDB database
->Create a shell script which sync the backups with Amazon S3
->Creating the folder in Amazon S3 for the database dumps
->How to configure the AWS CLI
->How to set up AWS key & Secret
->How to set up Cron (to automate the process)
->Conclusion

My Background: I am Cloud , DevOps & Big Data Enthusiast | 4x AWS Certified | 3x OCI Certified | 3x Azure Certified .

Why we need a database backup?

One might think why backup is necessary for my database? The answer is simple, backup creates a copy of your physical, logical, and operational data. Which you can store at any safe place such as Amazon S3. This copy comes into use if the running database gets corrupted. Database backup can include files like control files, datafiles, and archived redo logs.
Remove MongoDB backups from your to-do list.

Why Amazon S3 for backup?

For this tutorial, we have chosen Amazon S3 as it is a very common choice. You can do the same thing if you would like to use another cloud storage provider. The instructions won't differ a lot as long as the cloud provider is S3-compatible.
Below we defined some less known terms that we used in the article:

What is Cron?

Cron is a software utility that offers time-based job scheduling. It supports Unix computer operating systems. To set up software environments, the developer uses Cron. He/she schedules commands or shell scripts so that they run at chosen times. It could be daily, once a week, or any interval as desired.

What is Chmod?

The chmod a short command of 'change mode' enables the admin to set rules for file handling. In other words, with the help of a "chmod" system call. An administrator can change the access permissions of file system objects.

Database Backup Script for MongoDB and Dumping to Amazon S3

You can automate the creation of backup and storing it to Amazon S3 within a few minutes. Below bullets brief about what you are going to learn in this part of the article:
Create a script that automates the MongoDB backup directory creation
Upload/sync the backups with Amazon S3
Cron will run this command every day (to back up)

Generate a shell script which will dump the MongoDB database

cd ~ mkdir scripts cd scripts nano db_backup.sh

#!/bin/bash DIR=date +%d-%m-%yDEST=~/db_backups/$DIR mkdir $DEST mongodump -h localhost:27017 -d my_db_name -o $DEST

Now chmod the script to allow it to for execution
chmod +x ~/scripts/db_backup.sh

Create a shell script which sync the backups with Amazon S3

nano db_sync.sh

Copy and paste the script below to it
#!/bin/bash /usr/local/bin/aws s3 sync ~/db_backups s3://my-bucket-name

Now chmod the script to allow it for execution
chmod +x ~/scripts/db_sync.sh

Creating the folder in Amazon S3 for the database dumps

cd ~ mkdir db_backups

How to configure the AWS CLI

Before installing the AWS CLI you need to installpython-pi. Type the following commands:
apt-get update apt-get -y install python-pip curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"

Install the AWS CLI

Type the following command:
pip install awscli

How to set up AWS key & Secret
Configuration and credential file settings
cd ~ mkdir .aws nano ~/.aws/config

Paste in key_id and secret_access_key as shown below
[default] aws_access_key_id=AKIAIOSFODNN7EXAMPLE aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

How to set up Cron (to automate the process)
crontab -e

Paste the below commands at the bottom to automate the process

0 0 * * * ~/scripts/db_backup.sh # take a backup every midnight 0 2 * * * ~/scripts/db_sync.sh # upload the backup at 2am

This way the backup script will run and also sync with Amazon S3 daily.
Conclusion
Hence, by using these scripts you can achieve 3 goals:
Creating the database backup via a shell script
uploading the dump to Amazon S3
also automating this process using Cron.

Hope this guide helps you understand on how to use shell scripts to take daily backups of your database and push to s3 on a daily basis, feel free to connect with me on LinkedIn.
You can view my badges here.
If you are interested in learning more about AWS Services then follow me on github.
If you liked this content then do clap and share it . Thank You .

15 Years of Amazon S3 with 'Pi Week' Recap | Amazon S3 Object Lambda

Adit Modi — Tue, 04 May 2021 10:44:43 +0000

Amazon S3 was launched 15 years ago on Pi Day, March 14, 2006, and created the first generally available AWS service. Over that time, data storage and usage has exploded, and the world has never been the same.

Amazon S3 has virtually unlimited scalability, and unmatched availability, durability, security, and performance. Customers of all sizes and industries can use S3 to store and protect any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, and big data analytics.

My Background: I am Cloud , DevOps & Big Data Enthusiast | 4x AWS Certified | 3x OCI Certified | 3x Azure Certified .

When you store data in Amazon Simple Storage Service (S3), you can easily share it for use by multiple applications. However, each application has its own requirements and may need a different view of the data. For example, a dataset created by an e-commerce application may include personally identifiable information (PII) that is not needed when the same data is processed for analytics and should be redacted. On the other side, if the same dataset is used for a marketing campaign, you may need to enrich the data with additional details, such as information from the customer loyalty database.

To provide different views of data to multiple applications, there are currently two options. You either create, store, and maintain additional derivative copies of the data, so that each application has its own custom dataset, or you build and manage infrastructure as a proxy layer in front of S3 to intercept and process data as it is requested. Both options add complexity and costs, so the S3 team decided to build a better solution.

Watch the On-Demand AWS Pi Week 4-day , virtual event which was hosted from March 15-18, 2021 hosted on the AWS channel on Twitch as AWS celebrated the 15th birthday of the AWS Cloud. the event included talks from AWS leaders and experts as they took us back in time reviewing the history of AWS and the key decisions involved in the building and evolution of Amazon S3. they also dived into how you can leverage S3 to control costs and continuously optimize your spend, while building modern, scalable applications.

The event is ideal to watch for anyone ( on-demand ) who is eager to learn more about:

How S3 and other AWS services are architected for availability and durability inside AWS Regions and Availability Zones
How S3's strong consistency model works to support many different workloads
The history of and best practices for S3 data security
How AWS architects evolvable services that provide new features and greater scalability with no disruption to customers
Detailed ways to move data into and out of the AWS Cloud

pi-week

On-demand AWS Pi Week Twitch video streams

Day 1 - Amazon S3 origins - foundations of cloud infrastructure
Video 1 | Video 2
Day 2 - Building data lakes and enabling data movement
Video 1 | Video 2
Day 3 - Amazon S3 security framework and best practices

Day 4 - Amazon S3 and the foundations of a serverless infrastructure

What's New

S3 Object Lambda was announced by AWS, a new capability that allows you to add your own code to process data retrieved from S3 before returning it to an application. S3 Object Lambda works with your existing applications and uses AWS Lambda functions to automatically process and transform your data as it is being retrieved from S3. The Lambda function is invoked inline with a standard S3 GET request, so you don’t need to change your application code.

In this way, you can easily present multiple views from the same dataset, and you can update the Lambda functions to modify these views at any time.

There are many use cases that can be simplified by this approach, for example:

Redacting personally identifiable information for analytics or non-production environments.
Converting across data formats, such as converting XML to JSON.
Augmenting data with information from other services or databases.
Compressing or decompressing files as they are being downloaded.
Resizing and watermarking images on the fly using caller-specific details, such as the user who requested the object.
Implementing custom authorization rules to access data.
You can start using S3 Object Lambda with a few simple steps:

Create a Lambda Function to transform data for your use case.
Create an S3 Object Lambda Access Point from the S3 Management Console.
Select the Lambda function that you created above.
Provide a supporting S3 Access Point to give S3 Object Lambda access to the original object.
Update your application configuration to use the new S3 Object Lambda Access Point to retrieve data from S3.

Availability and Pricing

S3 Object Lambda is available today in all AWS Regions with the exception of the Asia Pacific (Osaka), AWS GovCloud (US-East), AWS GovCloud (US-West), China (Beijing), and China (Ningxia) Regions. You can use S3 Object Lambda with the AWS Management Console, AWS Command Line Interface (CLI), and AWS SDKs. Currently, the AWS CLI high-level S3 commands, such as aws s3 cp, don’t support objects from S3 Object Lambda Access Points, but you can use the low-level S3 API commands, such as aws s3api get-object.

With S3 Object Lambda, you pay for the AWS Lambda compute and request charges required to process the data, and for the data S3 Object Lambda returns to your application. You also pay for the S3 requests that are invoked by your Lambda function. For more pricing information, please see the Amazon S3 pricing page.

This new capability makes it much easier to share and convert data across multiple applications.

Amazon S3 Glacier announces a 40% price reduction for PUT and Lifecycle requests

Amazon S3 is reducing the cost to move data to Amazon S3 Glacier by lowering PUT and Lifecycle request charges by 40% for all AWS Regions. You can use the S3 PUT API to directly store compliance and backup data in S3 Glacier that does not require immediate access. You can also use S3 Lifecycle policies to move data from S3 Standard, S3 Standard-Infrequent Access, or S3 One Zone-Infrequent Access to S3 Glacier to save on storage costs when data becomes rarely accessed.

Amazon S3 Glacier is a secure, durable, and extremely low-cost Amazon S3 cloud storage class for data archiving and long-term backup. S3 Glacier provides a low-cost option for archiving data accessed once per quarter that needs to be accessible within minutes to a few hours.

In addition to being durable and secure, the S3 Glacier storage class is now even more cost-effective than before. Effective March 1, 2021, AWS is lowering the charges for PUT and Lifecycle requests to S3 Glacier by 40% for all AWS Regions. This includes the AWS GovCloud (US) Regions, the AWS China (Beijing) Region, operated by Sinnet, and the AWS China (Ningxia) Region, operated by NWCD. To learn more, see the S3 pricing page, and get started in the S3 console.

I hope this guide helps you understand all the new aws s3 features that were launched during the pi week, I know it's a little late to give a recap on "pi week" but I had like to do it anyways, feel free to contact me on LinkedIn.
You can view my badges here.
If you are interested in learning more about AWS then follow me on github.
If you liked this content then do clap and share it . Thank You .

Introduction to Containers with AWS

Adit Modi — Fri, 16 Apr 2021 05:01:37 +0000

Containerization—a virtualization method used to deploy and run distributed applications without the need to launch an entire virtual machine for each application—is changing the way businesses develop and deploy applications in cloud environments. Containers decompose applications into small, manageable packages containing everything the application needs to run: code, core data, configuration files, interfaces, and dependencies.

The container approach allows developers to focus on applications and not be concerned with deployment and infrastructure management. From a development perspective, there are numerous benefits to the container approach.

Accelerate the development pipeline, including testing and debugging.
Facilitate continuous integration (CI) **and **continuous deployment (CD) workflows, automatically rebuilding whenever a new code revision is committed.

Containers run locally on desktop or laptop and are easily uploaded directly to the Cloud.
Consistent results when moving code from development to test to production systems.

No need to rewrite code for each OS and cloud platform, making it easy to move containers from one cloud provider to another.
The advantages of containers extend beyond the development cycle. Containers utilize compute resources more efficiently by eliminating the need for a hypervisor.

They simply share OS kernel without impacting the performance of applications running inside the container. With a smaller footprint, more containers can run on a single host, resulting in better utilization of compute resources and lower costs.

Additionally, containers can be configured with only the desired binaries and components, eliminating potential vulnerabilities that might be found in a full fledged OS. Containers that can run on Amazon EC2 Spot Instances can obtain up to a 90% discount compared to On-Demand
prices.

My Background: I am Cloud , DevOps & Big Data Enthusiast | 4x AWS Certified | 3x OCI Certified | 3x Azure Certified .

The Introduction to AWS is a Series containing different articles that provide a basic introduction to different aws topics/categories. Each article covers the detailed guide on how to work with particular topic/category . This series aims at providing "A Getting Started Guide on Different aws topics / categories ."

There are a bunch of different ways to run your containerized workloads on AWS. This blog post compares the three most important ways to run Docker on AWS:

1.)Amazon Elastic Container Service (ECS) with AWS Fargate

2.)Amazon Elastic Container Service for Kubernetes (EKS)

3.)AWS Elastic Beanstalk (EB) with Single Container Docker

ECS with Fargate

First, let’s have a look at ECS, a fully-managed container orchestration service. ECS is a proprietary but free of charge solution offered by AWS. It is important to mention that ECS provides a high level of integration with the AWS infrastructure. For example, containers are 1st class citizens of the VPC with their network interface (ENI) and security groups.

ECS offers service discovery via a load balancer or DNS (Cloud Map).

Aside from that ECS is the only option to run Docker containers without running EC2 instances on AWS. Fargate is the compute engine for ECS. All the heavy lifting of scaling the number of EC2 instances and containers, rolling out updates to EC2 instances without affecting containers, and many more is gone.

ECS is free of charge. Fargate is billed per second based on CPU and memory allocated for your containers. A container with 1 vCPU and 4 GB is about USD 30 per month.

Keep in mind the following limitations of Fargate:

General purpose compute capacity only. Fargate does not support GPU, CPU/memory optimized configurations at the moment.
Persistent volumes are not supported out of the box (e.g., Docker volume driver).
No discounts for reserved capacity available.

EKS (Kubernetes)

The 2nd option to run Docker containers on AWS is Kubernetes (K8s). In summary, K8s is an open-source container orchestration solution. AWS offers the K8s master layer as a service. The master layer is responsible for storing the state of the container cluster and deciding on which machines new containers should be placed. On top of that, you are responsible for managing a fleet of EC2 instances used to run the containers.

The main selling point for K8s: it is open-source and runs on AWS, Azure, Google Cloud, on-premises, or even on your local machine. The resulting disadvantage is that Kubernetes is not that well integrated with the AWS infrastructure.

Kubernetes is designed for microservice architectures. For example, a built-in service discovery allows containers to talk to each other easily by using a local proxy.

EKS is about USD 144 per month for the master layer. Besides, you are paying for the EC2 instances powering your containers. A t3.medium instance provides 2 CPUs with 4 GiB of memory and costs around USD 30 USD per month.

You should not underestimate the complexity of operating EKS and EC2. For example, the way EKS integrates with the VPC comes with a few unexpected limitations (see EKS vs. ECS: orchestrating containers on AWS for more details).

Elastic Beanstalk

Another option to run Docker containers on AWS is Elastic Beanstalk. Some say Elastic Beanstalk is the PaaS (Platform-as-a-Service) offering from AWS. Nevertheless, Elastic Beanstalk is very easy to use. There are a bunch of environments to deploy your web application with Elastic Beanstalk. One of them is called Single Container Docker. This environment deploys a single Docker container to one or multiple EC2 instances.

Elastic Beanstalk is not only deploying your application; it is also creating the needed infrastructure consisting of a database, a load balancer, and EC2 instances. Important to note: Elastic Beanstalk creates EC2 instances automatically. But you are still responsible for these virtual machines they are not fully-managed by AWS.

Elastic Beanstalk is a proprietary but free of charge solution offered by AWS. You are only paying for the underlying infrastructure. For example, a t3.medium instance provides 2 CPUs with 4 GiB of memory and costs around USD 30 USD per month.

When Is Elastic Beanstalk The Best Method For Managing Docker Containers On AWS?

For businesses new to AWS or new to the containerization concept, just getting started with Docker, or developing new applications, Elastic Beanstalk may be the best approach to support Docker containers. Elastic Beanstalk offers a simple interface, allows Docker images to be pulled from public or private registries, and coordinates the deployment of multiple Docker containers to Amazon ECS clusters. Elastic Beanstalk gives you less control over application scaling and capacity but makes deploying Docker containers on AWS ever so straightforward.

When Is Elastic Container Service The Best Method For Managing Docker Containers On AWS?

In comparison to Elastic Beanstalk, Elastic Container Service provides greater control over application architectures and orchestration of Docker containers. You specify the size and number of cluster nodes and determine if auto-scaling should be used.

Elastic Container Service uses tasks to launch Docker containers. A task includes the container definition, providing the ability to group containers in sets that launch together then terminate simultaneously. ECS provides significantly greater flexibility and customization in scheduling and CPU and memory utilization. In addition, ECS does not require special integration efforts to work with many other AWS services.

Elastic Container Service is appropriate when you need to run microservices that require integration with other AWS services, or use custom or managed schedulers to run batch workloads on EC2 On-Demand, Reserved, or Spot Instances. Businesses wanting to containerize legacy code and migrate it to AWS without needing to rewrite code should take the ECS option. Applications or workflows comprised of loosely coupled, distributed services running on various platforms or accessing widely-distributed data source can also benefit by using Elastic Container Service.

When Is Elastic Kubernetes Service The Best Method For Managing Docker Containers On AWS?

If you want the flexibility to integrate externally with the open-source Kubernetes community, spending the additional effort on setting up EKS may be the better option. Kubernetes is preferred for legacy workloads. It allows you to build a dev/test/production environment on-premises, and then move it to the cloud if and when required. Kubernetes is best known for its true enterprise-level cluster and container management. It is extremely valuable when your containerized workloads begin to scale. If you are already running workloads on Kubernetes, EKS is going to be a familiar and simple route to moving to an AWS environment.

Hope this guide helps you understand containers on aws, feel free to connect with me on LinkedIn.
You can view my badges here.
If you are interested in learning more about AWS then follow me on github.
If you liked this content then do clap and share it . Thank You .

Deploying a Containerized App in Google GKE

Adit Modi — Mon, 12 Apr 2021 06:14:04 +0000

Because of its popularity and widespread adoption, Kubernetes has become the industry’s de facto for deploying a containerized app. Google Kubernetes Engine (GKE) is Google Cloud Products’ (GCP) managed Kubernetes service. It provides out-of-the-box features such as auto-scaling nodes, high-availability clusters, and automatic upgrades of masters and nodes. In addition, it offers the most convenient cluster setup workflow and the best overall developer experience.

this article will trace the life cycle of a containerized application in its most mature environment, GKE.

My Background: I am Cloud , DevOps & Big Data Enthusiast | 4x AWS Certified | 3x OCI Certified | 3x Azure Certified .

State-of-the-art Kubernetes in the Cloud

Each of the major cloud providers have a managed Kubernetes service (also known as Kubernetes-as-a-Service) which creates an isolated and supervised environment for Kubernetes clusters. These services provide the Kubernetes API setup, measure basic node health, autoscale or upgrade when needed, and maintain some security best practices.

There are more than ten listed providers in the official Kubernetes documentation, including AWS, GCP, and Azure. If you want a deep dive comparison of Google Kubernetes Engine, Azure Kubernetes Service, and Amazon Elastic Container Service for Kubernetes, check out my other blogs.

Google Cloud Platform provides the most reliable and easily managed Kubernetes clusters, as Google is the creator of Kubernetes and donated it to the open source community, hence we’ll be examining its service in this article.

Below, we’ll start our process by creating some containers for a microservice application in Google Cloud Platform. You’ll need a GCP account to continue. Keep in mind that, if you’re a new customer, GCP’s free tier gives you $300 credit.

CLI Environment Setup

The first step in deploying a containerized app is setting up a CLI environment in GCP. We will use Cloud Shell to do this, since it already has installed and configured gcloud, Docker, and kubectl. Cloud Shell will enable you to quickly start using the CLI tools with authentication and configuration in place. In addition, the CLI already runs inside the GCP console where you can access your resources and check their statuses.

Open the Google Cloud Console, and click “Activate Cloud Shell” on the top of the navigation bar:

A terminal with a command line prompt will open as follows:

Now that you have the environment set up, you are ready to create containers from microservices and prepare them for release.

Microservice to Containers

The life cycle of a containerized app starts with source code. In this tutorial, we’ll use an open source web server named hello-app, available on Github. Because Kubernetes is a container management system, we’ll need to create container images for our applications. The following steps will guide you through creating a Docker container for hello-app and uploading the container image to the registry. When you deploy the application, Kubernetes will download and run it from the registry.

In Cloud Shell, download the source code of the sample application:

git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples

cd kubernetes-engine-samples/hello-app

Then, build the container image with a tag that includes your GCP project ID:

docker build -t gcr.io/${DEVSHELL_PROJECT_ID}/hello-app:v1 .

GCP has a managed Container Registry service that can push and pull Docker container images. If you are using the registry for the first time, enable it in the API Library.

Authenticate to the registry with the following command, and then push the image:

gcloud auth configure-docker

docker push gcr.io/${DEVSHELL_PROJECT_ID}/hello-app:v1

The last step packaged and uploaded the container image to the registry. Now, we are ready to create a Kubernetes cluster and distribute the hello-app application over the cloud.

Creating a Kubernetes Cluster in GKE

Kubernetes originated in Google, and its cloud provider, GCP, has the most convenient and easy setup for creating new clusters. With a couple of clicks, you can create a managed Kubernetes cluster. Google Cloud Platform takes care of the cluster’s health, upgrades, and security. Open the Kubernetes Engine in the GCP control panel, and click on “Create cluster”:

In the cluster creation view, the cluster basics focus on names such as hello, node locations, and Kubernetes master version. You can go to the Automation, Networking, Security, Metadata, and Features tabs for the following additional options:

Automation: Configuration of automatic maintenance, autoscaling, and auto-provisioning.

Networking: Configuration of communication within the application and with the Kubernetes control plane and configuration of how clients reach the control plane. If you want to use a specific network, node subnet, or set network policy, you need to set it here.

Security: Configuration of cluster authentication handled by IAM and Google-managed encryption.

Metadata: Labels for and descriptions of the cluster.

Features: These are the “extra toppings” for the otherwise “vanilla” Kubernetes cluster— serverless, telemetry, Kubernetes dashboard, and Istio installation.

In this tutorial, we will go with the default setting, a single-zone three-node cluster, by simply clicking “Create”:

In a couple of minutes, your new cluster will be created. A green check will appear in the Kubernetes cluster list, as seen below:

Click the “Connect” button, and then click “Run in Cloud Shell” to continue in the terminal. In the terminal, run the following command to list the nodes:

kubectl get nodes

NAME STATUS ROLES AGE VERSION
gke-hello-default-pool-d30403be-94dc Ready 3m31s v1.14.10-gke.36
gke-hello-default-pool-d30403be-b2bv Ready 3m31s v1.14.10-gke.36
gke-hello-default-pool-d30403be-v9xz Ready 3m28s v1.14.10-gke.36

The default setup will create three nodes, as shown above. You can also run other kubectl commands and play around with your newly-created cluster.

Next, let’s deploy our containerized app to the cluster.

Containers to the Cloud

Kubernetes manages containers inside a logical grouping, called a pod, and encapsulates them with the network and storage to provide some level of isolation and segmentation. The pods are managed via Controllers which provide different capabilities to manage the life cycle of the pod, these include Deployments, StatefulSets, DaemonSets, and never other replication lifecycle types. Deployments focus on scalable and reliable applications. StatefulSets focus on stateful application management, such as databases. DaemonSets ensure that there is an instance of the application running on each node such as for observability needs.

Create a deployment with the following command:

kubectl create deployment hello-world --image=gcr.io/${DEVSHELL_PROJECT_ID}/hello-app:v1

deployment.apps/hello-world created

This command creates a 1-replica deployment from the image that we have already pushed. Now, scale it with the following command:

kubectl scale --replicas=5 deployment/hello-world

deployment.extensions/hello-world scaled

This command updates the deployment to have five replicas of the pods. Kubernetes assigns these pods to the nodes. Inside the nodes, the Docker images will be pulled, and containers will be started. Let’s check the status of the pods:

kubectl get pods

NAME READY STATUS RESTARTS AGE
hello-world-77cc5b59b6-4p5dc 1/1 Running 0 8m55s
hello-world-77cc5b59b6-bkph9 1/1 Running 0 8m54s
hello-world-77cc5b59b6-c8hc5 1/1 Running 0 8m52s
hello-world-77cc5b59b6-g59z7 1/1 Running 0 8m51s
hello-world-77cc5b59b6-qfnvr 1/1 Running 0 8m54s

Since all the pods are “Running,” we know that Kubernetes has distributed the pods to the nodes. Currently, we have five pods running over the three nodes. You can change the number of replicas based on your needs, metrics, and usage with the help of the kubectl scale command.

Monitoring the Applications

Kubernetes distributes the containers to the nodes, making it critical to collect logs and metrics in a central location.

There are currently two popular stacks used to collect logs from the clusters: Elasticsearch, Logstash, and Kibana (ELK) and Elasticsearch, Fluentd, and Kibana (EFK).

It’s also possible to check the Kubernetes metrics in GKE with the kubectl top command. Let’s use it to look at the usage of the pods:

kubectl top pods

NAME CPU(cores) MEMORY(bytes)
hello-world-77cc5b59b6-4p5dc 0m 1Mi
hello-world-77cc5b59b6-bkph9 0m 1Mi
hello-world-77cc5b59b6-c8hc5 0m 1Mi
hello-world-77cc5b59b6-g59z7 0m 1Mi
hello-world-77cc5b59b6-qfnvr 0m 1Mi

Similarly, you can use the kubectl top nodes command to retrieve aggregate data about the nodes:

kubectl top nodes

NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
gke-hello-default-pool-d30403be-94dc 50m 5% 714Mi 27%
gke-hello-default-pool-d30403be-b2bv 52m 5% 674Mi 25%
gke-hello-default-pool-d30403be-v9xz 52m 5% 660Mi 25%

Now, let’s open our application up to the internet and receive some Hello World responses.

Containers to the Internet

In addition to container management, Kubernetes provides resources to connect to applications from inside and outside the cluster. With the following command, you expose the deployment to the internet:

kubectl expose deployment hello-world --type=LoadBalancer --port 80 --target-port 8080

service/hello-world exposed

This command creates a service resource in Kubernetes, and it provides networking with an IP attached to the application instances. It can be checked from the menu item Kubernetes Engine > Services & Ingress:

The Service details page lists the networking configuration from the point of view of a Kubernetes cluster. In addition, there is an external IP assigned to the service enabling access from the internet. It’s created by GCP TCP Load Balancer by default for zonal and regional clusters. Let’s check the TCP load balancer in Network services > Load Balancer by clicking the load balancer in the previous view:

In this screenshot, load balancer instances for all three nodes are listed along with their health status. If you create a multi-region cluster, you will need an ingress controller and global load balancer deployed to your cluster for routing.

Check for the external IP with the following command:

kubectl get service

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-world LoadBalancer 10.0.2.152 35.232.168.243 80:30497/TCP 77s

kubernetes ClusterIP 10.0.0.1 443/TCP 54m

Now, open the external IP listed above for hello-world in your browser:

In the output, the hostname indicates the name of the pod. You can see all of your pod names as hostnames if you reload the browser tab a couple of times. You can expect a change of hostnames with each reload, since we have created a LoadBalancer type of service to expose the application.

Summary

This article has examined the life cycle of a containerized app in Google GKE, starting from the source code. We created a Docker container image which was pushed to the registry for future use in scaling. Then, we created a Kubernetes cluster in GKE and deployed our application into it. We scaled the app with many replicas and checked its status. We reviewed the metrics and logs with potential extensions. Finally, we exposed our application to the internet. With this hands-on knowledge, you should now be able to package, deploy, and manage containerized applications inside a Kubernetes cluster in GKE.

Hope this guide helps you understand how to Deploying a Containerized App in Google GKE, feel free to connect with me on LinkedIn.
You can view my badges here.
If you are interested in learning more about AWS / GCP then follow me on github.
If you liked this content then do clap and share it . Thank You .