Forem: Hannah

Deploying Microservices with GitOps

Hannah — Thu, 28 Apr 2022 20:50:52 +0000

Learn how to adopt GitOps for your microservices deployments.

This blog post will explain the highlights listed below and how it's possible to adopt GitOps and the benefits of deploying with Argo for your microservices.

What are microservices?
What are some of the challenges with microservices?
What is GitOps?
What is Argo CD?
How can GitOps and Argo CD help your deployments for your microservices?

Microservices

Microservices are a software architecture pattern for building scalable distributed applications. It's essentially a framework that helps break down monolithic architectures into small independent services that can communicate over common patterns like APIs, event-driven, or even messages. An organization might do this because monolithic architectures are tightly coupled and run as a single service. So, if a particular process of your application with a monolithic architecture experiences a spike in demand, this affects your entire architecture and must be scaled.

It's also important to note that any features or experiments made to this type of application become more complex and therefore can experience more issues and failures.

Breaking down a monolithic application with API services allows you to build and version each function of an application independently. Because of this, microservices enable an organization to run, modify, deploy and scale each service for an application much more quickly.

(include visual of monolithic architecture vs. microservice)

Challenges of Microservices

Microservices provide several benefits for organizations by enabling autonomy amongst development teams, allowing applications to scale, deploying those applications efficiently, and increasing flexibility and resilience for those applications. However, microservices include challenges that we will dive into and explain.

Those challenges are:

The exploding number of pipelines.

As the number of applications increases, the management of such pipelines becomes more complex. Often this can result in messy scripts and needing to leverage configuration management tools. With all of that comes maintenance, and operators spend a lot of time fixing pipelines. We have a blog post that dives deeper into this issue if you'd like to learn more.

Difficulty managing deployments (all at once, in sequence, and with dependencies).

While deploying a single microservice is more manageable than deploying a monolithic legacy application, there are still challenges. Suddenly, only a few pipelines for a monolithic application have turned into 20 or more pipelines with multiple microservices. These numbers will vary based on your organization, but it's no doubt that now managing multiple microservice repositories and pipelines is more complicated as more and more are created.

Difficult to monitor.

Now that there are multiple pipelines and multiple applications with an increase in deployments, those deployments and pipelines need to be monitored for any issues. Previously when using a monolithic architecture, these things could be monitored manually. However, the increased complexity of a distributed system can lead to chances of failure.

This is why it’s so important to ensure you’re monitoring for total failures, network failures, cascading failures caused by remote procedure calls between your microservices, and even ensuring that you’re honoring your SLAs (service level agreements). With a microservice architecture, we need to keep in mind that services can even be temporarily unavailable due to broken releases, configurations, and any changes due to human error. Clients of services should be architected in a resilient way so they do not suffer catastrophic failure when a service is down.

Difficult to debug (when things go wrong).

This topic itself could be a blog post. However, some of the critical challenges when debugging a microservice is fragmentation and ensuring that you create the same environment where the bug was identified. This means you need to ensure an identical setup to production, this may include specific cluster versions, serverless functions, the correct environment where the bug was reported, etc. In addition to the asynchronous call complications, the technicalities of running the microservice locally, and the cost it can take to execute all of these things in what you hope is a timely execution.

Even with the above example of the complexity that microservices can create when transitioning from a monolithic architecture to a microservice architecture, the benefits of moving to microservices can’t be understated. To gain the full benefits of microservices in scaling, reliability, and security we need to address these challenges head-on

So, what could help you with some of the challenges organizations face when managing, monitoring, deploying, and maintaining their microservices? Let me introduce you to GitOps, how you can apply it to your microservices, and how it might help!

GitOps

Before we dive into microservices and explain what they are and what sort of challenges they create for us. First, let's dive into what GitOps is so you can begin to percolate your thoughts as to WHY this might be helpful for your microservices as we explain them.

Perhaps you're familiar with Git and use it already for your source control and recognize how it enables a source of truth for your source code, but what if we expanded on that idea and applied it to our entire system? This is where GitOps comes into play.

GitOps is a paradigm that leverages a Git repository as the source of truth to manage continuous deployments and infrastructure management as code. This allows you to drive automation, bring repeatability, reduce downtime and eliminate human interventions in constant release cycles.

Now that we better understand GitOps let's explore a potential solution for deploying your microservices with GitOps that can be enabled by using Argo CD.

Argo CD

When determining the best deployment process and tools for GitOps, Argo CD is a great solution. Argo CD is a declarative CD (Continuous Delivery) tool that automatically synchronizes and deploys your applications whenever a change is made to your Git repository. This is why it’s one of the leading GitOps solutions.

It allows you to manage the lifecycle of your application and deployments while also providing version control, configuration management, and application definitions with a Kubernetes manifest. This allows you to organize your environments and complex infrastructure data with more ease.

Within this post, we will highlight ways that you can use Argo CD to deploy your microservices while applying GitOps to your workflow.

How GitOps and Argo can help

When trying to determine the best strategy to manage your Kubernetes applications, an increasingly popular approach is GitOps. Argo CD makes GitOps practical for you to use because Argo CD is a Kubernetes-native declarative continuous delivery tool that follows GitOps principles. Argo CD also enables you to accelerate the frequency of your application deployments and lifecycle management by maintaining your configurations and ensuring they’re in sync, applying your desired state directly from Git.

Essentially, Argo CD helps support your deployments while effectively providing several other benefits for your microservices which we’ll explain more below!

No more deployment pipelines -> Argo CD deploys them from Git

Previously we mentioned the explosion of pipelines when converting from a monolithic architecture to a microservice architecture. However, by leveraging Argo CD it can reduce the number of pipelines by simply eliminating them. This is because you can use Argo to deploy your microservice you rely on your Git repository instead of using a pipeline. When you add an Argo CD Application and an AppProject, your app's configuration settings can be defined using the Application Custom Resource (CRD) that your Kubernetes resource object represents.

Argo CD fully automates the deployment of an application by recognizing your configuration details from your manifest files and synchronizes any changes made to your system within your targeted namespace. Argo CD allows you to eliminate the use of a deployment pipeline.

Reduce files -> use an ApplicationSet

While microservices do make it easier to deploy more frequently, there’s also the increase in files for your configurations. Even when a service is deployed and managed using GitOps, this doesn’t reduce the number of files, such as your Kubernetes manifests, your services, secrets, and configmaps, all stored within your Git repository.

Luckily, Argo CD offers a controller known as an ApplicationSet to help us manage our deployments and files easier. The ApplicationSet allows you to create multiple Argo CD Applications and organize your microservices per Application. The controller also allows you to use a single manifest to target numerous clusters and a single manifest to deploy multiple applications, not just from one repository but multiple!

Always know their health -> Argo CD health

Whenever you make any change to your microservices or your system there’s a risk of a potential failure. For example, if you make a change and apply it to your manifest YAML file, Kubernetes does not wait until the resource reports back its status, nor do your automation tools typically report back, either. Instead, Kubernetes will apply the manifest and move on to the following manifest without waiting to know if the first one succeeded and was created or not. Something that can help alert you and provide observability for the state of your service is the Argo CD health status.

Argo CD offers a built-in health assessment that provides an overall Application health status for you. Argo CD checks the health for the following resources: services, ingress, deployment, replicaset, statefulset, daemonset, and a persistentvolumeclaim. You can also add custom health checks for custom resources or any persistent known issues affecting your Ingress or StatefulState. These custom health checks can be done two ways: within a ConfigMap or with a script and implemented with a YAML file. If you cannot find a health check for your resource, you can create one yourself for whatever use case you need it.

Above is an example of the Argo CD Application health status within the Argo CD Web UI. Since Git is our source of truth, this sort of observability and information about your service provides that source of truth for the actual state of your running system.

Deployment cadence -> Argo CD sync phases/waves

Typically when managing several microservices and making frequent deployments, it can be a challenge when needing to deploy a specific service in a particular order or, let's say, you want to deploy a service with a database, apart from deployment files. It would help if you had some solution for this. Whatever your use case may be, Argo CD provides a solution for this!

A Syncwave allows you to order how Argo CD applies the manifest files stored in your Git repository. By default, all manifests have a wave of zero, but you can set these by using an argocd.argoproj.io/sync-wave annotation.

For example, if you wanted to specify a wave, it would look something like this:

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "5"

When Argo CD starts a sync action, the manifest gets placed in the following order of their Phase.

When you apply the manifest in a Phase, you're using a resource hook to control the sync operation. Those operations are defined by the three primary phases known as the pre-sync, sync, and post-sync phases. To enable a sync, annotate the specific object manifest with argocd.argoproj.io/hook with the type of sync you want to use for that resource.

For example, if you wanted to use the PostSync hook, it would look something like this:

metadata:
  annotations:
    argocd.argoproj.io/hook: PostSync

Whenever Argo CD starts a sync, it orders the resources in the following precedence: the phase, the wave they are in, by kind (e.g., namespaces first and then other Kubernetes resources, followed by custom resources), and by name.

Dependencies -> See the graphical view of Argo CD

Now that you have containerized your microservices and wrapped your brain around the Kubernetes ecosystem. Suddenly an extra layer of complexity is added when trying to deploy all of those microservices and reference the never-ending Kubernetes documentation. This is where having some visualization for your deployments is helpful! Enabling some sort of topology for your application and infrastructure would allow you to better understand, monitor, and control your containerized microservice.

Argo CD does just that. The web UI provides a real-time view of application activity and dependencies and continuously monitors running applications when using Argo CD. Monitoring all deployed applications allows you to compare the current, live state against the desired target state (as specified within your Git repository). Any deployed application whose live state deviates from the target state is reported and allows you to view the differences while also providing facilities to automatically or manually sync the live state back to the desired target state of your system.

Image from Argo CD

The image above exemplifies an Argo CD Application with a topology view that shows all of the dependencies of that application.

Fast rollbacks/progressive delivery -> Argo Rollouts

When beginning to plan your deployment process for your new microservices, it's essential to determine which deployment strategy is best for you and your organization.

Potential strategies could include:

Rolling: Services that don't need to be available 24/7 can utilize a pattern of swapping in the new for old and essentially hitting the restart button. This allows minimal disruption should there be a service outage. The simplicity aspect is compelling, and the disruptions to end users are less so.

Progressive delivery (this includes blue-green deployments and canary): A blue-green deployment is simple, has excellent rollback, and has minimal downtime in best-case scenarios. Canary deployments tie together the best of all worlds for minimizing downtime, fast rollback, and minimal disruptions in the case of failed deployments.

However, while progressive delivery improves the reliability and stability of deployments, applying it within your complex microservice environments doesn't come easy. Kubernetes was built as a container orchestrator, not a deployment system, requiring manual work to execute progressive delivery. Kubernetes also has a default rolling update strategy that is simple to deploy yet provides little control over user traffic and rollback compared to the more advanced deployment methods like blue-green and canary.

This is where Argo Rollouts walks down the red carpet as an open source Kubernetes controller with a GitOps-based progressive delivery strategy.

Argo Rollouts introduces a new Kubernetes CRD for a rollout entity. It has the same functionality as a Kubernetes Deployment entity, but the Argo Rollouts will monitor for changes and can control the rollout. Users can define a specific rollout strategy configured with a manifest file. Argo Rollouts helps you simplify your release processes and provide full support of progressive delivery with automated deployment rollout and management in one!

Argo Rollout Image of UI Dashboard

This image above exemplifies a visualization of an individual Rollout using a Canary deployment strategy.

Conclusion

As you transition from a monolithic architecture to microservices, this process shares a common goal with GitOps: making it easier to deploy more frequently.

We have shared the benefits and possibilities of adopting GitOps with Argo for your microservices deployments within this blog post. To help you leverage these benefits and get started with your GitOps journey, make sure you understand what GitOps is and the fundamentals of this paradigm. To get started, you can check out our GitOps Fundamentals course for free, which allows you to learn about GitOps and get started with the basics of Argo, in tandem with receiving a certificate of completion to share with your employer and others!

Once you’ve completed this course, you should put your knowledge to the test and try Codefresh GitOps CD to help manage your Argo CD instances. This will provide you with a visualization of your services and any possible failures tied to a commit, a test, pipeline, etc. You can quickly pinpoint issues with access to a release timeline and functionality to execute rollbacks when needed.

Learn more about Codefresh GitOps CD and sign up here!

Best Practices for Argo CD

Hannah — Mon, 04 Apr 2022 15:20:18 +0000

What are some best practices when using Argo CD?

Within this blog post, we’ll be highlighting some best practices tied to Argo CD, that allow you to leverage GitOps easily within your deployment workflow.
Below we will explain the following:

What is Argo CD
What are some best practices with Argo CD?

What is Argo CD

Argo CD is the most popular and fastest-growing GitOps tool for Kubernetes. When following a GitOps deployment pattern, Argo CD makes it easy to define a set of applications with their desired state in a repository and where they should be deployed. After a deployment, Argo CD constantly monitors the state and can even catch configuration drift.

Argo CD's core component is the Applications Controller that executes continuous monitoring of applications and then compares them to the live application state against your target state defined within your Git repository.

The Application controller retrieves the desired resource manifest from the Git repository and compares it to the live resource from your Kubernetes cluster.

This approach enables GitOps for your deployment workflow by leveraging Git as your source of truth. Now, let's further explore how we can leverage Argo CD best for our deployments!

The image above shows the primary Argo CD dashboard with a single Argo application successfully deployed.

The dashboard above provides a detailed view of the same Argo application deployed in the image above. However, it also provides an understanding of the status of the Kubernetes resources.

Now, let's explore how we can apply best practices to our Argo CD deployments!

Argo CD Best Practices

Argo CD has several best practices; however, we will review some of the most important ones we've gathered from the Argo community and prioritized below.

1. Separate your Git repositories

It's super important to separate your configurations and source code into different Git repositories. Separating your configs in their repository limits commit access to avoid something being pushed to production environments. For example, if you accidentally push manifest changes to the config repo, it can trigger an infinite loop of build jobs and Git commit triggers. Using an automated CI pipeline, you can also prevent pushing manifest changes to the same repo.

Separating your repositories is also more secure and restricts commit access, so someone does not accidentally misconfigure an application.

Argo CD does not connect to your source code repository. However, if you make a change to your code and you want to deploy the change to a specific environment, you can do the following:

Build a new application code version and then merge a pull request (PR) in the configuration repo to use the new application version you created.
Utilize automation for the update you made to your source code by using a pipeline in the source code repository.
Argo CD Image Updater allows you to update your container images of the Kubernetes workload managed by Argo CD.

These approaches allow you to maintain logs for any auditing process. You can quickly identify development activity and reference your Git history for easy traceability.

2. Create a directory structure to enable a multi-application system for your Argo CD deployments

Once you've separated your source code and configs into separate repositories, it's essential to set up a directory structure that applies GitOps for Argo CD deployments. You can leverage several approaches for your Git repository structure that best serves your organization's needs.

However, we'll highlight some tips to best structure your directory when using Argo CD.

Do: We suggest modeling your environments or clusters using different folders instead of branches in your configuration repository (e.g., prod, staging, testing, etc.).
Do: Make sure your cluster and environment configurations repositories are separated (i.e., separate your prod configuration in a different repository from staging).
Do: Utilize some sort of manifest management, such as a raw Kubernetes YAML file, Kustomize, or Helm for your environment definitions for your apps.
Do: Create an 'argocd' folder in your configuration repository for each cluster and create an Argo CD Application manifest for each app in the cluster's repository.
- By creating the separate 'argocd' folder, you can also implement role-based access control for different clusters if you wish with Git repository permissions.
Do: Leverage a multi-folder or a multi-repo structure instead of a multi-branch approach. You should NOT have permanent branches for your clusters or environments.
Don't: Never put any independent applications or applications managed by different teams in the same repository.

Implementing these strategies above for your directory structure provides several advantages. Those advantages include improved security, easy rollbacks, audit log, easier testing, and automating your manifest updates. It also allows you to create and delete applications dynamically. If you'd like to see some examples of these directory structures, look here.

3. Determine a promotion strategy

Now that you've established a directory structure, you may face a challenge regarding the best promotion practice between clusters. When deploying multiple applications with Argo CD, it's best to pick one promotion strategy that suits your directory structure and stick with it. Below we'll highlight how you can do this based on your own needs.

Group your applications

ApplicationSets

First and foremost, once you've established a way to manage your applications and deployments, and there are too many apps to keep track of - the ApplicationSet is your best solution. We should also note that the ApplicationSet was previously an external controller that you had to install on your own, and now it's integrated and makes things much easier! However, you need some extra support when using the ApplicationSet instead of the App of Apps pattern that is easy enough to use right away without any additional help or a learning curve.

Yet, when deploying to production, an ApplicationSet is the best choice. This is because an ApplicationSet is a Kubernetes controller/custom resource definition (CRD) that enables automation and allows flexibility when managing multiple applications across several clusters.

The ApplicationSet consists of two main components:

You are essentially allowing the ApplicationSet to act as a templating agent.

The generators used with the ApplicationSet are responsible for generating params, utilizing the template to consume the variables, and then applying them to the cluster as Argo CD applications. So, any changes made to an ApplicationSet template and its fields will automatically be applied to each application created.

This way, you can simultaneously deploy your Argo apps to multiple Kubernetes clusters.

App of Apps

When managing ten or fewer applications, it's best to use the App of Apps pattern.

App of Apps was the pioneer before ApplicationSets that allowed us to deploy multiple applications at once with ease.

You are leveraging the actual app itself, aka a "Root App," to contain the other applications instead of individual Kubernetes objects. This, in turn, allows you to manage a group of applications that you can deploy declaratively. Essentially this pattern supports the declaration of children apps in a recursive way.

Image from the Argo CD documentation

Another possible approach to promoting changes is the Argo CD Image Updater. It's still a relatively new project, but it provides a manual way to update your manifest versions.

However, we should note that this approach is not GitOps friendly. Meaning it goes against the declarative nature of GitOps, but perhaps its functionality is what you and your team needs.

Which to choose?

One does not "need" a promotion strategy when beginning to deploy with Argo CD. The strategies listed above are needed when trying to solve "too many apps." This problem dramatically depends on the number of Argo CD applications you are managing and whether or not you already have a deployment process in place or not.

The Argo community has developed the strategies listed above to help you manage all your Argo CD application deployments, depending on your scale, directory structure, and manifest types, amongst other factors.

However, it's ultimately up to you and your workload, which is best for you and your team.

4. Manage your secrets securely

No one solution for secret management will work for all organizations. However, some common approaches for how to best handle your secrets with Argo CD can help based on these two significant factors:

Encrypt your secrets directly in your Git repository:

Bitnami Sealed Secrets

Sealed secrets are one way to encrypt secrets created by anyone but can only be decrypted by the controller running within the target cluster.

SOPS

SOPS (Secret OperationS) is an open source solution that allows you to encrypt and decrypt an entire file or field for your Kubernetes secrets. This approach will enable you to store your secrets, and other Kubernetes manifests directly in your Git repository.

Externalizing your secrets from your Git repository:

Argo CD Vault plugin

This solution creates debate about whether it's GitOps or not. Nonetheless, it's a solution using Kubernetes auth in Vault. The Argo CD repo server authorizes Vault to use the service account token in the secret manifest, substituting the required value and creating a secret for you.

Cloud Provider Secrets

Cloud provider secrets are created by your cloud service providers (CSP) depending on the cloud service. It depends on what your CSP offers and if their secret management solution meets your needs. Ensure you evaluate which strategy secures your data best.

Firstly, storing your secrets in a Git repository isn't easy! Perhaps your organization has a different policy for secrets, or you already have a system in place, so in this case, you may want to externalize your secrets.

However, if you want to store your secrets in your Git repository, plaintext secrets cannot be stored in your repo, and you never want to risk storing sensitive data and exposing it!

But, Git can truly be the source of truth, including your secrets, and you won't have to risk getting fired when they are encrypted asymmetrically. You can use Argo CD to manage your deployment state from Git into your cluster, including your secrets, by ensuring processes are put in place, and the tools above are used safely.

5. Confirm how your team accesses Argo CD

Essentially, we cannot say what the best practice for you and your team is to access Argo CD. However, we can advise you and your team to choose which approach works best for your organization and set a standard for your team.

Developers require access to Argo CD, so you need to set security measures and role-based access control (RBAC).

This approach allows your developers to access the Argo CD UI and ensure that each team has the appropriate access to their applications and their applications only.

Argo CD provides an RBAC configuration that includes two pre-defined roles:

Read-only
Admin

You can also implement a permissions definition for your applications and other resource types with Argo CD. You can then sync these roles and permissions together to create a policy to define access to your system.

No one within your organization requires access to Argo CD.

Some organizations do not want anyone to access the Argo CD UI or the API server. Perhaps the organization would like to use the automation but does not want to leverage the UI component. In this case, you can use a variant of Argo CD, also known as Argo CD Core. This allows developers to make a commit and ignore the UI aspects and reduce complexity. However, we should note that you lose multi-tenancy benefits by installing Argo CD Core.

So, whichever approach you choose above - as long as it benefits you and your team, you're making the right choice.

6. Increase automation for your system with the other Argo projects

If you're using Argo CD for Continuous Delivery (CD), you want to elevate the rest of your workflow to automate everything in Kubernetes to reduce downtime. Then, you should check out the other Argo projects and learn more about how they can help.

We don't encourage you to use any specific projects to leverage best practices, but instead, we encourage you to use those that will help you and your system best.

In the meantime, we can explore how we could use Argo Events, Workflows, and Rollouts to enhance your system if you choose to.

Progressive Delivery

Progressive delivery is a set of practices that roll out new features gradually instead of all at once.

Argo Rollouts

Rollouts provide advanced deployment capabilities and rolling updates for progressive delivery approaches you might already know, such as blue-green, canary, etc.

Advanced Deployment with CI

Argo Workflows and Events require you to have an existing Continuous Integration (CI) process to leverage these projects.

Argo Workflows

Workflows allow you to build and orchestrate parallel jobs and utilize a pipeline on Kubernetes.

Argo Events

Events is an event-driven workflow automation framework that is used with Kubernetes.

By utilizing these other Argo projects, you can easily manage your clusters, run workflows, and implement GitOps for your Kubernetes system! However, not all organizations are ready to implement automation for everything in Kubernetes. Combining all these tools requires time and understanding of how your system works before identifying which tool will best help your team's process.

Conclusion

Argo CD and the other Argo projects certainly make life easier by allowing you to automate each step for a production release, migration, and the operation workflows for your system!

By leveraging these best practices above, you're on the right track to empower your teams and apply GitOps for your infrastructure so you can deploy to prod more frequently.

I'd like to thank those who are part of the Argo community who participated and provided feedback for this blog. If you have any questions or think there are better ways to work with Argo CD, please reach out and let me know!

Using Codefresh with GKE Autopilot for native Kubernetes pipelines and GitOps deployment

Hannah — Mon, 06 Dec 2021 18:16:39 +0000

Several companies nowadays offer a cloud-native solution that manages Kubernetes applications and services. While these solutions seem easy at first glance, in reality, they still require manual maintenance.

As an example, an important decision for any Kubernetes cluster is the number of nodes and the autoscaling rules you define. You want to ensure that the size is just right and matches your workload, otherwise, this could become costly, unstable, and decrease the availability of your workload when you need it most!

GKE Autopilot Cluster

A solution to these problems is Google Kubernetes Engine (GKE) Autopilot from Google Cloud. GKE's cluster autoscaler adjusts the node pools based on whatever the demand is for your workload. For example, if the workload is high or low GKE Autopilot will scale depending on which and provide control on costs.

This allows you and your teams to no longer worry about maintaining your node pools or provisioning them - with Autopilot clusters these jobs are automatically done. This streamlined approach follows best practices for a cluster and workload setup, in addition to other best practices about security and observability. Allowing you to specify configurations needed for deployment to production and simplifying operations needed to manage an application’s infrastructure, nodes, and the control plane.

The Codefresh platform fully supports GKE Autopilot. By combining the two platforms you can easily:

Run CI/CD pipelines on your cluster to compile source code, run unit tests, perform security checks, etc.
Deploy your containers to your cluster by following the GitOps principles.

Deploying applications to your GKE Autopilot cluster following the GitOps principles

Codefresh GitOps Controller

Like any Kubernetes cluster integrations or the GKE Standard cluster, the GKE Autopilot Cluster integrates the same way with the Codefresh GitOps dashboard. The GitOps dashboard allows you to look at all your deployments in greater detail and instantly know the critical information such as:

What Git commit was deployed
Which cluster was affected
Who initiated the change
What features were shipped
Which pipelines took part in the deployment
What Kubernetes services were affected

To achieve this level of GitOps visibility, a GKE autopilot cluster can be the target of the Codefresh GitOps Controller, which is an agent installed in the cluster that monitors both the cluster and any defined Git repositories for changes. It installs an ArgoCD instance within the cluster and the associated GitOps Controller. This is done through the Codefresh CLI and can be authenticated with a Codefresh account, simply by creating an API token. This agent can also manage the runtime environment for a pipeline.

Once the GitOps controllers are installed, you can easily deploy any kind of application to your GKE Autopilot cluster using GitOps. This means that you can combine all the benefits of GitOps such as audibility, visibility, reduced downtime with the easy management of the GKE Autopilot capabilities.

Summary

GKE Autopilot cluster allows you to pursue a production-ready and scalable workflow for both your application and infrastructure systems. Its ability to configure cluster autoscaling and node auto-provisioning can improve resource utilization and reduce day-to-day challenges for organizations. These features reduce the operational cost of managing clusters and optimize them for production.

Google Cloud also provides a user-friendly experience, creating, configuring, and operating a Kubernetes cluster that you can integrate with the Codefresh DevOps Platform with a GitOps Controller, powered by ArgoCD.

Applied GitOps with Kustomize

Hannah — Wed, 01 Dec 2021 18:07:50 +0000

What is Kustomize?

Have you always wanted to have different settings between production and staging but never knew how? You can do this with Kustomize!

Kustomize is a CLI configuration manager for Kubernetes objects that leverage layering to preserve the base settings of the application. This is done by overlaying the declarative YAML artifacts to override default settings without actually making any changes to the original manifest.

Kustomize settings are defined in a kustomization.yaml file. Kustomize is also integrated with kubectl. With Kustomize, you can configure raw, template-free YAML files, which allows you to modify settings between deployment and production easily. This enables troubleshooting misconfigurations and keeps use-case-specific customization overrides intact.

Kustomize also allows you to scale easily by reusing a base file across all your environments (development, production, staging, etc.) and then overlay specifications for each.

Base Layer: This layer specifies the most common resources and original configuration.

Overlays Layer: This layer specifies use-case-specific resources by utilizing patches to override other kustomization files and Kubernetes manifests.

Overlays are what help us accomplish our goal by producing variants without templating.

Benefits of Kustomize

Kustomize offers some of the following benefits:

Reusability
With Kustomize you can reuse one of the base files across all environments (development, staging, production, etc.) and overlay specifications for each of those environments.
Quick Generation
Since Kustomize doesn't utilize templates, a standard YAML file can be used to declare configurations.
Debug Easily
Using a YAML file allows easy debugging, along with patches that isolate configurations, allowing you to pinpoint the root cause of performance issues quickly. You can also compare performance to the base configuration and other variations that are running.
Kubernetes Native Configuration
Kustomize understands Kubernetes resources and their fields and is not just a simple text templating solution like other tools.

See below an example of a Kustomize file structure including the base and overlays within an application. You can also reference a code-based demo project on GitHub.

The tree structure above is a simple example of how you can deploy a single application to 2 different environments (staging and production). Let's dig deeper into each directory and create an overlay.

base folder

The base folder holds common resources, such as the deployment.yaml, service.yaml, and configuration files. It contains the initial manifest and includes a namespace and label for the resources.

overlays folder

The overlays folder houses environment-specific overlays, which use patches to allow YAML files to be defined and overlaid on top of the base for any changes. Let's take a look at a couple of the environments within the overlays folder below that also includes the kustomization.yaml.

kustomization.yaml

Each directory contains a kustomization file, which is essentially a list of resources or manifests that describes how to generate or transform Kubernetes objects. There are multiple fields that can be added, and when this list is injected, the kustomization action can be referenced as an overlay that refers to the base.

Creating Overlays

Let's explore a simple example of how overlays work.
We'll make the following changes within our production and staging directories:

Within the staging overlay, we will enable a risky feature that is NOT enabled in production.

Within the production overlay, we'll assign a higher replica count.

We'll also ensure the web server from the cluster variants is different from one another.

overlays/staging/kustomization.yaml

In the staging directory, let's make a kustomization defining a new name prefix and different labels.

namePrefix: staging-
commonLabels:
 variant: staging
commonAnnotations:
  note: “Welcome to staging!”
bases:
- ../../base
patchesStrategicMerge:
- config-map.yaml

Staging patch

Add a configMap kustomization to change the server greeting from "Hello!" to "Kustomize rules!" We'll also enable the risky flag.

apiVersion: v1
kind: ConfigMap
metadata:
 name: the-map
data:
 altGreeting: “Kustomize rules!”
 enableRisky: “true”

overlays/production/kustomization.yaml

Within the production directory, we will make a kustomization with a different name prefix and label.

namePrefix: production-
commonLabels:
 variant: production
commonAnnotations:
  note: “Welcome to production!”
bases:
- ../../base
patchesStrategicMerge:
- deployment.yaml

Production patch

We'll make a production patch that will increase the replica count.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: the-deployment
spec:
 replicas: 10

Now, we can compare these overlays - the kustomizations and patches are required to create noticeable differences between staging and production variants within the Kubernetes cluster.

Based on the changes above, the output would look something like this:

<   altGreeting: Kustomize rules!
<   enableRisky: "true"
---
>   altGreeting: Hello!
>   enableRisky: "false"

<     note: Welcome, I am staging!
---
>     note: Welcome, I am production!

<     variant: staging
---
>     variant: production

You can now see the difference between the staging and production overlays.
Overlays contain a kustomization.yaml, and can also include manifests as new or additional resources, or to patch resources. The kustomization file is what defines how the overlays should be applied to the base and this is what we refer to as a variant.

Each time a change is made to an application, like the example above - it is the overlays that are doing the heavy lifting.

So, now that you have learned a bit about Kustomize, you might be wondering how you can apply the GitOps workflow to it. Below we'll explain more about GitOps and how you can apply it to your Kustomize project and deploy it.

GitOps works with all your existing tools

Now, let's learn more about how to apply GitOps to your Kustomize application deployment!

GitOps is a paradigm that incorporates best practices applied to both an application development workflow and the infrastructure of a system. This is done to empower organizations and developers to operate their systems from a single source of truth enabled by Git.
A couple of key aspects of GitOps that most aren't as familiar with and are crucial are the:

GitOps controller

The controller reads the declarative configuration and uses the reconciliation loops to converge the desired end state. This controller detects the differences between states and makes the necessary changes to maintain the desired state.

Automation reconciliation loop

The concept of the reconciliation loop is used to keep the state as defined within a manifest. This is such an important principle for the GitOps paradigm because these loops are what drive the entire cluster to the desired state in Git.

Now that you have a basic understanding of what both Kustomize and GitOps are and what some of their capabilities are, perhaps you've considered applying GitOps to your existing or future application and infrastructure workflow.

We'll explore 2 approaches on how to use Kustomize with or without GitOps:

Using only Kustomize

Using Kustomize with ArgoCD

What is ArgoCD?

ArgoCD is a GitOps controller specifically created for Kubernetes deployments. It supports a variety of configuration management tools like Helm (take a look at our documentation for more information), Ksonnet, Kustomize, etc. The core component is the Application Controller, which continuously monitors any running applications and compares the live state against the desired state defined in a Git repository.

If a deployed application whose live state drifts from the target state, ArgoCD is then considered OutOfSync. It then provides reports and visualizations to identify these changes and can provide automation when any modifications are made so that the target environments reflect the desired state of a system in Git.

If you'd like to explore these different deployment approaches with Kustomize from a code-based perspective, here is an example project that you can follow along with on GitHub. However, we will explore both of these approaches below with some more details.

Approach #1: Deploying only with Kustomize

If you want to use Kustomize, but your organization isn't ready to implement GitOps with your workflow, you can still use Kustomize on its own.

Install Kustomize

First, install Kustomize and this can be done either by utilizing kubectl version 1.14 or later, otherwise you can install based on your operating system and reference the Kustomize documentation.

Create a base directory

Next, in order to deploy an application with Kustomize, you need a kustomization.yaml file, and this is added to the base directory.

The kustomization.yaml file specifies what resources to manage due to the complexity of multiple resource types and different environments when handling configuration files for Kubernetes.

Within the base directory, there is also a service and deployment resource.

Create an Overlays directory

Next, you'll want to create an overlays directory, and this directory is what allows you to kustomize the base and apply any changes with a patch to modify a resource.

The overlays still use the same resources as the base but may vary in the number of replicas in a deployment, the CPU for a specific pod, or the data source used in the ConfigMap, etc.

Within our example, we have a staging and production overlay. These overlay directories contain the kustomizations and patches previously mentioned that are required to create distinct staging and production variants in a cluster. However, with Kustomize you can use the overlays for anything needed to organize environments, whether it's based on location: USA, Asia, Europe or internal/external, team A/team B, etc.

Kustomize is essentially an overlay-based engine that functions by finding and replacing specific sections in the manifest and replacing it with required fields and values. These values are then merged and deployed!

Create a namespace for specific environments

kubectl create ns staging

kubectl create ns production

Build and Deploy to environments

You can then apply the overlays to your cluster and deploy with the command:

kubectl apply -k overlays/staging

kubectl apply -k overlays/production

You are now successfully using Kustomize to manage your Kubernetes configurations and deploy your application to production and staging environments!

Approach #2: Deploying using Kustomize with GitOps

In the previous approach, you learned how to deploy an application using only Kustomize, so let's see how it fits into ArgoCD and how it can be used in a GitOps workflow.

ArgoCD supports Kustomize and has the ability to read a kustomization.yaml file to enable deployment with Kustomize and allow ArgoCD to manage the state of the YAML files.

ArgoCD monitors the resources within the git repository for any changes, ensuring that the live state of your system matches the desired state. Each time customization is made, ArgoCD detects those modifications and updates the deployment.

So, let's begin walking through the process to deploy a Kustomize project using ArgoCD!

Install ArgoCD and access a Kubernetes Cluster

First, we need to ensure the Kubernetes cluster is set up and you are logged into ArgoCD so that these resources are provided and can be deployed. You can use any Kubernetes cluster and install the argocd CLI.

Once you've accessed the argocd CLI you can access the ArgoCD server and log in to the ArgoCD UI. However, if you're more of a terminal fan, you can also deploy the application through the argocd CLI. Feel free to reference the demo application that will walk you through the deployment process through your terminal.

Create an ArgoCD application

Now, we can set up the Kustomize Project. Similar to using Helm with GitOps, we will approach this deployment the same way within the UI by creating an ArgoCD application.

Let's begin! First, click on the +NEW APP and include the name of the ArgoCD application, select the default project, and enable the Automatic SYNC POLICY.

Next, when adding your Kustomize project it helps to include a specific Path to segregate the application manifests inside the Git repository. We can then ask ArgoCD to only read the specific directories in the repository and read manifests within that path.

Then, within the Destination section, you need to provide the destination of the Kubernetes cluster details. Also, make sure to click on the checkbox for auto-create namespace when adding the input field value, or you can add a custom namespace you've created prior.

ArgoCD will read the kustomization.yaml file in the path you provided and prompt you to allow override with different values.

Synchronize Application and Deploy

Assuming you enabled auto-synchronization when creating your ArgoCD application, it will read the parameters and the Kubernetes manifests. Then, once the manifests are applied, you can review the application health and resources you deployed.

If your application has an error when trying to synchronize, you can execute the argocd history command, allowing you to view the application deployment history to identify a possible error:

argocd app history argocd_app_name

If you need to rollback you can do so by executing this command:

argocd app rollback argocd_app_name history_id

These commands leverage a faster and more secure deployment by enabling the tracking from the Git repository. This allows you to track the active Kubernetes resources and events. These actions can also be done within the UI.

Once the application is healthy and synchronized, each time you create a new kustomization.yaml file and the file changes, ArgoCD will be able to detect those changes and make updates to your deployment. You can even mention specific Kustomize tags and set up custom build options for your Kustomize build.

Summary

Kustomize allows you to use different configurations of a base Kubernetes manifest. Within this post, we've covered the Kustomize basics and how to deploy using just Kustomize and deploying with GitOps. This allows you to leverage the power of Kustomize to define the Kubernetes files without using a templating system.

To start deploying your Kustomize application, sign up for a Codefresh account today to apply GitOps to your deployment process!

Using Helm with GitOps

Hannah — Thu, 30 Sep 2021 15:15:45 +0000

This is the first of many posts highlighting GitOps topics that we’ll be exploring. Within this post, we will explore Helm, a tool used for Kubernetes package management, that also provides templating. Helm provides utilities that assist Kubernetes application deployment.

In order to better understand how Helm charts are mapped to Kubernetes manifests, we’ll explain more details below and how to use Helm with and without GitOps.

What is Kubernetes?

Kubernetes is an orchestrator for containers that allows you to automate scheduling, deployments, networking, scaling, and health monitoring for the containers.

Kubernetes was needed because we increased the usage of the following:

Microservices
Containers

This was difficult to manage with scripts and self-made tools and caused the need for orchestration technology like Kubernetes. It provides features like scalability, disaster recovery, and less downtime.

What is Helm?

Helm is a package manager for Kubernetes. It’s a convenient way for packaging collections of YAML files with a Helm chart for the Kubernetes application and allowing distribution with a Helm repository.

Maintaining Kubernetes manifests is time-consuming and tedious and this is why Helm is helpful!

Helm Charts

Helm Charts are deployable units for Kubernetes applications. These charts are a collection of files inside a directory. This directory is the name of the Helm chart and consists of a self-descriptor file, YAML file, and one or more Kubernetes manifests. Helm charts are typically written in the Go template language. Charts are created as such files, describing a related set of Kubernetes resources. Here’s an example of a Helm chart directory and it’s layout:

This particular directory contains a Chart.yaml file and this is where the global variables, versions, and descriptions are stored. Then, the templates directory is what contains the YAML files for Kubernetes, otherwise known as the Kubernetes manifests.

Files such as the deployment, service, and ingress files contain variables from the values.yaml file when the chart is deployed. The _helpers.tpl incorporates helpful functions for variable calculations.

You can then share the Helm chart to increase easy reusability for others to use. Sharing is done by storing the chart to a Helm repository. This repository can then be shared with others to deploy the application with the chart.

Helm Repositories

Helm supports a chart repository service that can be used to store the Helm charts. You can use any web server host or source code host for the Helm repository.

The repository has an index.yaml file that contains metadata about the package, including the Chart.yaml file. The index will contain information about each Helm chart in the chart repository. Then the server can serve your index and charts or the packages can be stored in the repository for shareable access.

Helm Release

Each install or upgrade will create a Helm release. A Helm release is a running instance of your Helm chart running within a Kubernetes cluster or a namespace. It’s essentially an instance of a versioned, templated chart. It’s also possible to have multiple releases of the same chart in a single cluster or namespace since the chart is self-contained. You can also roll back a Helm release to a previous version in case there are any failures.

Assuming you’re a developer with an existing cluster, perhaps you’d like to share your Helm application with an external vendor. Let's recap the Helm workflow and release process:

To share an application with others, you need to create a Helm chart. The chart is a package that contains templates for a set of resources necessary for the application. The template uses variables applied to the Values.yaml file when the manifest is created and describes how to configure the resources.
The Helm charts are then hosted within a repository that can then be downloaded or accessed from a server. This chart will contain all the necessary resource definitions needed for the developer to run an application.
Now that you have access to the cluster, you can add new features or bug fixes to the application and update the Helm chart. Helm offers useful tools to manage your releases. You can upgrade the chart and create a new deployment. In Helm a deployed instance of your application is referred to as the release.
You can now deploy your packaged application to the cluster.

Within this workflow we identified how Helm packages files and deploys them to a cluster. So, how do you apply GitOps to the workflow above? Let’s explore this below...

What is GitOps?

GitOps is a paradigm that incorporates best practices applied to the application development workflow all the way to the operating infrastructure of a system.

Some benefits of GitOps are:

Deploying faster and more often
Easier and quicker error handling and recovery
Self documenting deployments
Increased developer productivity and an enhanced experience for teams
Greater visibility on the lifecycle of developed features

These benefits make it easier to handle the applications and allow teams to deliver quality software faster.

GitOps relies on Git as the single source of truth for declarative configuration and active reconciliation. By adopting the GitOps methodology it provides transparency between the application configuration deployed in a cluster and the one residing in Git.

A GitOps tool that follows this approach of a Git-based workflow is Argo CD. It’s a continuous delivery tool for Kubernetes that is essentially a GitOps controller that does two-way synchronization. Argo continuously monitors running applications and compares the live state against the desired state in Git and applies it to the cluster. In addition, it also monitors the container registry for new images and updates the workload definitions based on the deployment policies.

We will mention Argo CD again in this post and can assume that it’s already been installed and implemented within our workflow!

GitOps works with all of your existing tools

Now that you have a good understanding of what Helm is and it’s capabilities, perhaps you’ve considered applying GitOps to your existing or future applications.

Let’s explore the various approaches of how to use GitOps with or without Helm:

Using only Helm (without GitOps)
Using GitOps (without Helm)
Using Helm with GitOps

If you’d like to explore these different approaches from a code-base perspective, here is an example project that you should check out on GitHub that digs deeper into how you can:

Install an application with Helm and deploy locally
Install an application with Argo CD and deploy locally both within the UI and command line

If not, then we’ll be sure to highlight these approaches below!

Approach #1: Using only Helm without GitOps

If you want to use Helm but your organization isn’t quite ready to implement the GitOps workflow, that’s perfectly fine. Helm was created before the GitOps methodology came to fruition, but can still work with GitOps (as we will see in the latter point).

The workflow allows you to search through Helm repositories for charts and install them to clusters, creating releases. This process enables execution of application deployments on a cluster. The maintenance of YAML manifests for Kubernetes objects is still the same as before. Helm releases are what keeps track of the version history of each chart installation and change and still allows rolling back to the previous cluster version. When installing a chart it creates a release of the new package and it’s the Helm release that deploys the Helm application.

When using Helm there are some powerful commands you will find useful, please see below.

To install a new package:

helm install <release_name> <name_of_chart_you_want_to_install>

To view currently deployed releases:

helm list

helm ls

To view revision numbers for a particular release:

helm history <release_name>

To learn more details about these commands and how to use them, reference the example project in GitHub.

Approach #2: Using GitOps without Helm

If you’re sold on the idea of GitOps, there are alternative tools that can be used if you’re unable to use Helm or choose not to.

You can use ArgoCD on its own. It is pretty flexible and can work with other templating solutions or even plain manifests. Another templating tool that can be used instead of Helm is Kustomize. This tool is built into kubectl and is native to Kubernetes. It allows you to customize Kubernetes configurations using only the Kubernetes API resource files.

Now, let’s explore our third approach about how to implement GitOps using Helm and Argo CD...

Approach #3: Using Helm with GitOps

Now that you’ve seen how GitOps can be applied without Helm, let’s explore how you can use it with Helm.

First, the Helm chart and any application changes must be committed in Git prior to being applied to the cluster. Meaning all of your workload definitions in a YAML format, the Helm charts, and any other Kubernetes custom resources that define the cluster’s desired state must be committed to the repo. This way rollbacks and logs are accessible and the state of production can be restored easily.

Within Argo CD you can connect the Git repository using HTTPS and add the URL directly. This then allows you to enable auto-synchronization to automatically synchronize the cluster to the desired state in the Git repository. Once synced successfully, the application status is recognized as Healthy. Otherwise, if the application is OutOfSync, you can rollback or view the release history to resolve the issue.

Here’s an example of a Healthy application in the Argo CD UI.

When the application is running you can view its resource components, logs, events, and the health status.

For good measure, here’s a visual of the Argo CD UI exemplifying an OutOfSync application when the live state deviates from the target state.

If your application had an error or was OutOfSync like the example above, you could execute the argo history command that allows you to view the application deployment history:

argocd app history <argocd_app_name>

Then, if you need to rollback the deployment you can do so by executing this command:

argocd app rollback <argocd_app_name> <history_id>

Essentially, these commands leverage a faster and more secure deployment, by enabling the tracking from the Git repository. It also allows you to track the active Kubernetes resources and events.

Therefore, using source control like this is what classifies this application deployment as GitOps!

Something important to note is that Argo CD provides native support for Helm, meaning you can directly connect a packaged Helm chart and Argo CD will monitor it for new versions. When this takes place the Helm chart will no longer function as a Helm chart and instead, is rendered with the Helm template when Argo is installed, using the Argo CD application manifest.

Argo CD then deploys and monitors the application components until both states are identical. The application is no longer a Helm application and is now recognized as an Argo app and can only operate by Argo CD. Hence if you execute the helm list command, you should no longer be able to view your helm release because the Helm metadata no longer exists.

Here’s an example of the command output. As you can see, the Argo CD application is NOT detected as a Helm application.

Summary

Helm is a powerful tool that is used often with Kubernetes deployments and provides tracking for each release. This helps ensure reliable and quick deployments for development teams.

So, if you’re completely new to Helm, I suggest referring to our documentation to learn more about it, here:

We'll discuss additional tools in our next post that can help you and your team continue implementing a GitOps process to your development and infrastructure systems.

How to Create Docker Images for ASP.NET Core

Hannah — Fri, 16 Jul 2021 19:25:33 +0000

Microsoft has begun working with the Docker team and community so Docker can be used for the following:

Build
Host
Scale .NET applications with ease

What will you learn in this tutorial?

If you would like to run an ASP.NET Core web app in a Docker container and learn how to create images, we will explain all the steps on how to do the following:

Build a custom image with Dockerfile
Use a prebuilt image hosted by Microsoft
Use a prebuilt image with HTTPS

A Docker container image is a standalone, lightweight package that can be executed and contains all the requirements you need to run an application, such as: code, runtime, libraries, and settings. The image can then be pushed to a container registry and pulled to your server to run as a container.

For this demo you can clone or download the ASP.NET Core sample app and run it in Docker containers. The sample app builds in a .NET SDK container that copies the result of the build into a new image based on the .NET Docker Runtime image. This demo also works with both Linux and Windows containers too!

Prerequisites:

Latest version of ASP.Net Core
Setup Docker:
Windows
macOS
Linux distributions:* CentOS
Debian
Fedora
GitHub

Build a custom Docker image with Dockerfile

Let’s start this tutorial by creating a Docker image from scratch using a Dockerfile for your app. Later on in this tutorial, we’ll explain how to use prebuilt Docker images with your ASP.NET Core web app.

Prerequisites:

ASP.NET Core web app:
Create your own app
DockerDemo app with Dockerfile
Note: The Dockerfile within the demo app uses a Docker multi-stage build feature that can be used to build and run in different containers. We’ll share more about this in a future blog post.

Create a Dockerfile in your project folder

The Dockerfile is essentially a recipe for creating your Docker image and is added to the project’s root. It includes the necessary commands for a user to build an image when executing docker build. For additional details to better understand the commands within the file, please refer to the Dockerfile reference.

Dockerfile

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base
WORKDIR /app
EXPOSE 80

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY \["DockerDemo.csproj", ""\]
RUN dotnet restore "./DockerDemo.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "DockerDemo.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "DockerDemo.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT \["dotnet", "DockerDemo.dll"\]

Create a .dockerignore file

Similar to a .gitignore file a .dockerignore file allows you to mention a list of files or directories that you want to ignore while building the Docker image. This is essential because it helps reduce the size of an image and speeds up the docker building process.

Note: We also have an excellent article reviewing the .dockerignore file that you might enjoy

.dockerignore

\*\*/.classpath
\*\*/.dockerignore
\*\*/.env
\*\*/.git
\*\*/.gitignore
\*\*/.project
\*\*/.settings
\*\*/.toolstarget
\*\*/.vs
\*\*/.vscode
\*\*/\*.\*proj.user
\*\*/\*.dbmdl
\*\*/\*.jfm
\*\*/azds.yaml
\*\*/bin
\*\*/charts
\*\*/docker-compose\*
\*\*/Dockerfile\*
\*\*/node\_modules
\*\*/npm-debug.log
\*\*/obj
\*\*/secrets.dev.yaml
\*\*/values.dev.yaml
LICENSE
README.md

Build Docker image and start container

The docker build builds a Docker image from the Dockerfile and a “context”. The context is a set of files located in a specified PATH or URL.

Open the terminal or command prompt and navigate to your project folder. Use the following command to build your Docker image:

docker build -t dockerdemo .

This returns the status of your build.

Create and run container

The docker runcommand creates a new container and runs the Docker image.

Open the terminal or command prompt and use the following command to run your Docker image:

docker run -d -p 8080:80 --name myapp dockerdemo

Check that the container was created and is running with the command:

docker ps

Lastly, go to http://localhost:8080/ to access your app in a web browser.

Note:

The Dockerfile and the .dockerignore files are generated for an ASP.NET Core Web app in Visual Studio 2019 when you select Create a new project and Docker is enabled. For more information, you can reference the Quickstart guide for Docker in Visual Studio.

You can also generate a Dockerfile and .dockerignore file in Visual Studio Code by opening the Command Palette and utilizing the Docker:Add Docker Files to Workspace command. For more information you can reference the Working with containers guide.

Fetch, Build, and Run a prebuilt Docker image

Now that you’re familiar with how to create a Docker image with a Dockerfile, let’s dive into using a prebuilt image for your ASP.NET Core app.

Prerequisites:

Download/Clone sample app https://github.com/dotnet/dotnet-docker

Assuming all your prerequisites are complete, you can now access existing prebuilt Docker images. You can reference existing Docker images at the public register, Docker Hub. Search for .NET Core and you will find several repos hosted by Microsoft.

Run the sample app and execute the following command in your terminal:

docker run -p 8000:80 --name my\_sample mcr.microsoft.com/dotnet/core/samples:aspnetapp

This command is fetching and running your container which is stored within:

mcr.microsoft.com/dotnet/core/samples

If you receive an error, it’s likely a port conflict, and you’ll need to select a port you’re not using.

However, your output should look like the following:

View all the images that you are currently running by simply running the command:

docker ps

This returns a lot of helpful information, including your container ID, the registry location, and even the command used to run it.

Launch your browser and navigate to:

http://localhost:8000

Note:

If you've changed your port, make sure you also change this url.

Wasn’t that easy? This demonstrates the simplicity and value of containerization and not having to worry about infrastructure complexities.

Next, run the command:

docker image list

The screenshot above is a list of all the installed images, with the latest at the top.

This demonstrates how to serve an image over HTTP locally; however, with containerization you should run your containers over HTTPS. HTTPS provides a secure channel within an insecure network by relying on certificates and encryption.

Build prebuilt Docker images with HTTPS

You’re on a roll! Let's review how to build prebuilt container images with HTTPS. ASP.NET Core uses HTTPS by default for security purposes because it encrypts the traffic between the client and server, so others can't see and prevents any modifications from malicious attackers.

Prerequisites:

.NET Core 3.1 or .NET 5

Certificates

For both Windows and macOS or Linux operating systems you will need a self-signed development certificate to host the image locally for non-production purposes. Using this certificate helps enable HTTPS on the server and protect your data using a Secure Socket Layer (SSL). This can be done multiple ways, and some of those ways are by utilizing dotnet dev-certs, Powershell, and OpenSSL. This way you can access your application from either port for demo purposes. There’s no right or wrong way to go about this. However, these instructions are similar to using a production certificate you can request within your team.

Windows using Windows containers

From your Windows terminal, use the following command to generate the cert and configure your local machine (make sure to replace “my password” with something unique):

dotnet dev-certs https -ep %USERPROFILE%\\.aspnet\\https\\aspnetapp.pfx -p mypassword
dotnet dev-certs https --trust

Afterwards, execute the docker pull/run commands to run the container image with ASP.NET Core configured for HTTPS (make sure your password matches the one you assigned for the certificate):

docker pull mcr.microsoft.com/dotnet/samples:aspnetapp

docker run --rm -it -p 8000:80 -p 8001:443 -e 
ASPNETCORE_URLS="https://+;http://+" -e ASPNETCORE_HTTPS_PORT=8001 -e 
ASPNETCORE_Kestrel__Certificates__Default__Password="mypassword" -e 
ASPNETCORE_Kestrel__Certificates__Default__Path=\https\aspnetapp.pfx -v 
%USERPROFILE%\.aspnet\https:C:\https\ 
mcr.microsoft.com/dotnet/samples:aspnetapp

If you want to learn more about configuring Windows using a Linux container, you can reference the Microsoft documentation for details.

MacOs/Linux

From your Mac/Linux terminal, you can execute this command to generate a certificate and configure your local machine (make sure to replace “my password” with something unique):

dotnet dev-certs https -ep ${HOME}/.aspnet/https/aspnetapp.pfx -p mypassword
dotnet dev-certs https –trust

We should note that depending on which Linux distribution you’re utilizing, the command: dotnet dev-certs https -trust may not work, and you may need to find an alternative way by researching the specific distribution’s documentation.

Then, execute the following command to run the container image with ASP.NET Core and configure with HTTPS (make sure your password matches the one you assigned for the certificate):

docker pull mcr.microsoft.com/dotnet/samples:aspnetapp

docker run --rm -it -p 8000:80 -p 8001:443 -e 
ASPNETCORE\_URLS="https://+;http://+" -e 
ASPNETCORE\_HTTPS\_PORT=8001 -e ASPNETCORE\_Kestrel\_\_Certificates\_\_Default\_\_Password="mypassword" -e ASPNETCORE\_Kestrel\_\_Certificates\_\_Default\_\_Path=/https/aspnetapp.pfx -v 
${HOME}/.aspnet/https:/https/ mcr.microsoft.com/dotnet/samples:aspnetapp

Summary

Within this demonstration, you were able to get familiar with Docker and fetch, build, and run a prebuilt ASP.NET Core container image and serve the container over to HTTPS. You also were able to touch on some of the core concepts such as the images and using a DockerFile and .dockerignore file. Nice work!

In a future blog post, we’ll review 3 different ways to create a Docker image for .NET, so stay tuned!

Cover photo by Pexels.