Forem: Cloudogu GmbH

Low-Ops Platform Cloudogu EcoSystem

flxebrt — Wed, 14 Jun 2023 10:14:59 +0000

Effectiveness with high efficiency is the key to success in our fast-paced and dynamic business world. Increasing digitalization requires quick responses to changing environments and requirements. To increase efficiency and the ability to respond while saving costs, it is important to optimize processes. Improving the integration of thematically related tools is one way to reduce administration and management work - which is becoming even more relevant in light of the current shortage of skilled workers.

In this blog post, we'll go into more detail about what makes the Cloudogu EcoSystem a low-ops platform and how it helps to support enterprises' digital transformation capabilities.

What is low-ops?

Low-Ops is based on the phases of the DevOps loop and is a concept that aims for little administration or operations, short "ops," by simplifying the deployment, operation, and monitoring of software tools through automation, standardization, and generalization.

SaaS solutions already remove the administration burden from users, but for many use cases, the use of SaaS tools is not desired, for example, due to issues such as dependence on the provider's service, data protection and data security. Operating on one's own (on-premises) or secure external infrastructures (private cloud) is common in these cases. This is exactly when the Cloudogu EcoSystem offers advantages by greatly simplifying the operation of a variety of tools on self-managed infrastructures.

The Cloudogu EcoSystem as a low-ops platform

Cloudogu's Cloudogu EcoSystem is the first scalable low-ops platform that enables diverse software tools from different vendors (packaged as Dogus) to be administered in unison, minimizing management overhead. The Cloudogu EcoSystem is a virtual platform that combines all these approaches:

Automation: It offers minimal administrative overhead, as it allows, for example, automatic data migration during upgrades. The Backup Dogu offers the possibility to automate a backup strategy, i.e. defined backup cycles according to a retention policy.
Standardization: Uniform containerization of all tools on the platform further standardizes and simplifies administration and operation, because the tools can be administered and operated in bundles rather than individually.
Generalization: The platform offers cross-tool services such as backup, user management, administration and restore for all tools operated on it. This enables, for example, cross-tool single sign-on or backup and restore. The holistic backup and restore solution enables simple administration.

The Cloudogu EcoSystem currently offers a selection of standard tools in the areas of project management, software development and wiki. Cloudogu regularly expands the toolstack of the Cloudogu EcoSystem and also responds to special customer requests. Each software tool (Dogu) of the toolchain runs in special containers. From the available tools, the appropriate Dogus for specific use cases can be selected as needed. The Dogus communicate with each other by means of plug-ins, and the interaction of the Dogus is continuously being expanded. Automated updates of the Dogus - including data migration if required - prevent manual interventions and reduce time spent. Thus, a multitude of tools can be administered in one step, even without special administration knowledge.

Low-ops improves transformability

By reducing administration, the low-ops idea supports transformability, i.e. the ability to successfully manage changes in one's own environment. An important characteristic of transformability is adaptability.

"The only constant in life is change" Heraclitus

This includes both the ability to adapt oneself to a dynamic environment and the ability to help shape the environment through proactive change. The Cloudogu EcoSystem gives companies the flexibility to use new tools or try them out first to take advantage of new helpful features. The processes of installing and operating Dogus are designed to be very simple and efficient. Companies also remain flexible in the type of operation (cloud; on-premises) and can tailor the environment to their own needs. This gives companies the opportunity for flexible adaptation to changes in the environment. This can promote continuous and iterative development.

Conclusion

The Cloudogu EcoSystem is a low-ops platform that significantly reduces the administration effort of the tools operated on it. The toolchain of the Cloudogu EcoSystem links different Dogus while enabling their central administration. The Cloudogu EcoSystem has an easy deployment of complete working environments and offers many possibilities for projects in a dynamic environment that demands digital mutability.

GitOps and Kubernetes – Secure Handling of Secrets

Daniel — Wed, 18 Jan 2023 16:11:28 +0000

Especially in well established companies, it may be the case that a certain way of handling secrets has become established, which is convenient, but not necessarily the safest solution. One example of this is the storage of secrets directly in the CI server. The use of a key management system (KMS) for storing secrets is a more secure solution, but it initially requires a lot of effort for the changeover. That is why KMS are not always used in the Kubernetes environment at present. GitOps with its declarative approach forces a rethink in the handling of secrets.

Kubernetes and Secrets

With Kubernetes, Secrets are basically stored unencrypted in the API server's underlying data store (etcd). This means that anyone who has access to the etcd can also obtain or modify secrets. Therefore, at least these measures should be taken:

Enable encryption of Secrets using Encryption at Rest,
implement a least-privilege approach for Secrets by configuring RBAC rules,
restrict access to Secrets to specific containers by mounting Secrets only in containers where they are needed,
finding a way to get Secrets into the cluster (for example, using the CI server), and
evaluating the use of an external provider for storing Secrets (KMS).

This shows that even without GitOps, secure and appropriate handling of Secrets in Kubernetes is not trivial. This is reinforced by the declarative approach of GitOps, since Secrets must also be stored or at least referenced outside the cluster, namely in the source code management. This enforces a secure handling of Secrets. Most of the solutions described in the following are not only exclusively applicable with GitOps, but are basically applicable when using Kubernetes.

GitOps enforces different handling of secrets

With GitOps, the state of a project is described declaratively in the source code management and the deployment environment continuously synchronizes with Git. The CI server is only used for the build, the deployment is done by the GitOps operator. This approach allows to take automation to a new level, but also leads to the need to find a new way of handling secrets. By storing the entire state in Git, a way must inevitably be found to store secrets in a secure manner in Git, because stored directly and unencrypted in Git offers a large attack surface. For the secure handling of secrets there are basically two approaches:

Encrypted storage of the Secrets in Git
Storing the secrets in a Key Management System (KMS) and referencing them in Git.

Encrypted storage in Git

Sealed Secrets (Operator)

An option that easily works with GitOps is the Operator Sealed Secrets from Bitnami. Secrets encrypted with it can only be decrypted by operators running inside the cluster, not even by the original author. For encryption, there is a CLI (and a third-party web UI) that requires a connection to the cluster. The disadvantage of this is that the key material is stored in the cluster, the secrets are bound to the cluster and one has to take care of backups and operation.

Use of Key Management System (KMS)

When using a KMS, the first step is to choose a suitable tool. Since the major operators of public clouds such as Google, Microsoft, Amazon, etc. usually offer a KMS that is easy to use and Hashicorp Vault has established itself as a solution for on-premises operated clusters, the first step is quickly completed.
The second step is then to integrate the KMS into the cluster. There are several ways to do this:

Running a special operator
Mounting secrets in the file system using Container Storage Interface (CSI)
Injecting a side car into pods
GitOps operator with either native support for KMS or via plugin

External Secrets (Operator)

External Secrets is an operator that integrates external KMS such as Hashicorp Vault or those of the major cloud providers. It reads secrets from the external APIs and injects them into Kubernetes secrets. The operator is a new implementation after the merge of similar projects from GoDaddy and ContainerSolutions.

Secrets Store CSI Driver

Mounting secrets into the file system of pods is another exciting way to go, as the CSI Driver is an official part of Kubernetes. It is developed by the Kubernetes Special Interest Group, making it potentially very durable. The CSI Driver provides providers for popular vendors, which in turn are developed by the vendors themselves.

Hashicorp Vault k8s (Sidecar Injector)

Hashicorp Vault k8s is an operator that modifies pods via a mutating webhook to connect between vault and pod via sidecars (additional containers) to provide secrets. This has the major advantage that no secret objects are created in Kubernetes here. The disadvantage is that this way only works with Vault.

SOPS

Mozilla Secret Ops (SOPS), already known from the time before Kubernetes, offers even more options – at the expense of a more complex configuration. Here, the key material can come from the key management systems (KMS) of the major cloud providers, an own HashiCorp Vault or from self-managed PGP keys. Since it is not Kubernetes-native, SOPS does not include an operator, but there are several different ways to use it with GitOps.

Flux v2 provides native support.
ArgoCD supports SOPS with the vault Plugin.
There is also the helm secrets plugin, which can also be used in ArgoCD with manual configuration.
There is also a third-party sops-secrets operator available.

Example: Hashicorp Vault with External Secrets Operator in the GitOps Playground

You can see first hand and try out the management of secrets with Hashicorp Vault and synchronization into the cluster with the External Secrets Operator in the GitOps Playground.
The GitOps Playground is an open source project for trying out GitOps including a sample application. To the GitOps Playground

Conclusion

The declarative approach of GitOps leads to the need to rethink the handling of secrets. The curse and the blessing is that there are many options that enable secure handling of secrets. Thanks to the different approaches there are suitable operators for different requirements. We would be happy if you post questions, feedback or suggestions for other operators, the GitOps Playground or GitOps in general in our GitOps community.

DevOps Toolchain Cloudogu EcoSystem DevStarter

Daniel — Thu, 18 Aug 2022 15:30:00 +0000

DevOps is a methodology or collection of approaches, practices and methods to deliver new software features faster. Depending on how the previous ways of working were in an organization, adopting DevOps can mean a fundamental change in ways of working and, consequently, can take a long time. It is important to note that DevOps cannot be introduced by simply using certain tools, because tools only support the implementation of the ways of working, the mindset however needs to be developed independently. Nevertheleess, choosing the right tools can help significantly with the introduction of DevOps.

Since working methods and processes are often changed when DevOps is introduced, new tools are usually also required to support these changes and support the automation of processes. That’s why it’s important to have both a flexible infrastructure that allows tools to be added easily and to use flexible tools per se that are intuitive and support cross-functional sharing.

That’s why we’re introducing the Cloudogu EcoSystem DevStarter, a platform that, even in its basic form, runs a variety of tools that users need for DevOps. The following graphic shows you which tools can be used in which DevOps phases.

Easy Redmine (Plan, Code and Operate)

The Easy Redmine project management tool offers, in addition to classic project management functions for Agile and Waterfall, other helpful features:

Issue tracker
Time tracking
Resource planning
Dashboards
Project planning via Agile board, Gantt chart and more.

Also, Easy Redmine offers a help desk extension, making it particularly helpful in the phases plan and code (task planning and processing) as well as operate (help desk).

BlueSpice MediaWiki (Plan and Code)

BlueSpice is an easy-to-use wiki that can be used in any department or task area in a company to document information or collect and discuss ideas. In software development, the wiki can be used to collect detailed requirements in the plan phase so that they can then be referred to during development (code phase). Of course, the wiki can also be used in other phases of the software lifecycle to record information.

SCM-Manager (Code)

The source code management tool SCM-Manager can be used to manage Git, Mercurial as well as Subversion repositories. This makes the tool much more flexible than other solutions that only support one kind of repositories, e,g, Git. This makes the tool particularly valuable for companies that still have Mercurial repositories, for example, but would like to switch to Git. But even if only one repository type is used, SCM-Manager has many advantages:

Variety of integrations with issue trackers like Easy Redmine available.
Easy integration into continuous development processes
Many functions of the tool can also be used via a GUI
Extensive code review process available
Simple operation

The tool is used by teams in the code phase to version the source code of software applications and then use it for builds.

Jenkins (Build and Deploy)

The continuous integration server Jenkins is the central building block of the development pipeline. The tool can be connected to a variety of other tools and used to automate steps. Automated builds as well as deployments of applications are only the most obvious functions. A development pipeline can consist of these steps:

Pulling the current state of the source code from the version management (SCM-Manager).
Performing the build and running unit tests using artifacts (Nexus Repository)
Start of a static code analysis (SonarQube)
Upon successful code analysis, storing the built version (Nexus Repository)
Deployment of the built version

SonarQube (Test)

SonarQube is an open source tool that can be used to perform static code analysis. The tool provides a variety of tests for many programming languages and allows quality gates to be set to enforce minimum code quality requirements. Examples of quality gates are

% coverage of code with unit tests
Rate of comments in the code
Number of (potential) bugs in the code

Nexus Lifecycle (Test)

Recently we have been shown several times the extent that security vulnerabilities in open source components can have. In the case of security vulnerabilities in very popular components, the news even makes it to the big media. For the majority of open source components, however, it can be very time-consuming to keep up to date on possible vulnerabilities. This is where Nexus Lifecycle comes in. The tool analyzes dependencies in your software projects and automatically informs you about known vulnerabilities. It also gives you information about possible licensing restrictions of open source software.

Nexus Repository (Build and Release)

The Nexus Repository tool is an artifact repository management tool that can be used, among other things, to manage and store build artifacts before they are deployed. It can also be used to manage binaries, providing a centralized source of information within the organization and speeding up builds.

Elasticsearch (Monitor)

Elasticsearch can be used to collect and process a wide variety of monitoring data and to use that data for alerting.

Further DevOps tools

As an addition to the tools included in the Cloudogu EcoSystem DevStarter, there are of course other tools widely used in the DevOps environment to enable even better collaboration between development and operations:

Kubernetes, a system for deploying, scaling and managing container applications.
Docker, a tool for isolating applications through container virtualization.
Terraform, an infrastructure-as-code tool that can be used to create, modify and improve infrastructure.
Selenium, a framework for creating automated tests for web applications.
Nagios, a tool for monitoring services in complex IT infrastructures.

Kubernetes least privilege implementation using the Google Cloud as an example

Daniel — Fri, 06 May 2022 14:43:58 +0000

Everyone knows it: granting privileges is always a balance between security, usability and maintenance effort. If permissions are granted very generously, the effort is very low and there are rarely any hurdles to use; however, security is compromised. If permissions are granted sparingly, security is higher, but there are costly processes and a lot of administrative overhead.

Kubernetes offers many possibilities with its Role-based access control (RBAC), which are also extensively documented (https://kubernetes.io/docs/reference/access-authn-authz/rbac/). Unfortunately, however, there are not many practical tips for actual implementation. To break out of this predicament, we have written plugins that allow you to use the Kubernetes-sudo context to get a simple but effective entry point for managing permissions. In doing so, this blog article illustrates how to install the plugins and configure the cluster using a managed Kubernetes from Google Cloud Platform as an example.

Permissions of developers in the cluster

The solution presented here refers to permissions of humans, since applications should basically only have read-only access to their own secrets and the configmap in the cluster. So this is straightforward from the point of view of assigning permissions. However, when it comes to permissions for developers, it becomes much more complex, because people’s tasks and roles can change over time and because permissions can be used to protect people from making careless mistakes. This results in the high maintenance effort mentioned at the beginning.

A solution with minimal maintenance effort is to give all developers the same, extensive authorizations. However, this creates the risk that harmful changes can be made accidentally at any time. Especially in productive environments, this can lead to critical downtimes and even data loss.

That’s why we decided to take an approach that briefly uses additional privileges to execute commands, similar to the sudo command in Linux.

Implementing the least-privilege approach using sudo-context

To implement sudo-style permissions, these things must be implemented:

Using the impersonate feature of Kubernetes
Setting up the sudo context
Granting permissions in the cluster

We will now describe these steps in detail.

Setting up the impersonate feature

The sudo function is Kubernetes’ internal “impersonate” feature of the Kubernetes API. It allows commands to be executed as a different user, group, or service account. The first step is to enable the sudo function in the cluster. There are different ways to do this depending on the use case. How these are finally installed and configured on client and cluster side follows in the following.

kubectl-sudo plugin

With the kubectl-sudo plugin, kubectl commands that require more extensive rights can be executed explicitly as a member of the admin group. This reduces the chance of accidentally modifying or deleting resources on the cluster, for example when running scripts or being in the wrong namespace.

The plugin only works for kubectl, but other tools that use kubeconfig (helmet, fluxctl, k9s, etc.) cannot use it. Here is a simple example of how to use the plugin kubectl sudo get pod.

helm-sudo plugin

In the Kubernetes environment, Helm charts are very important. So, to be able to use the functionality for Helm as well, we have developed a corresponding plugin that can be used analogously to kubectl-sudo. Analogous to the usage of the kubectl-sudo plugin, here is an example for the helm plugin helm sudo list.

sudo context for other tools

Alternatively and for all other tools like fluxctl or k9s there is the possibility to create a sudo context in kubeconfig. This can then be used as follows:

kubectl --context SUDO-mycontext # alternative to kubectl-sudo    
kgpo --context SUDO-mycontext # also works with aliases!     
helm --kube-context SUDO-mycontext   
fluxctl --context SUDO-mycontext     
k9s --context SUDO-mycontext # Changes also in k9s possible ":ctx"

If auto-completion features have been installed for certain tools, they will automatically detect the available contexts and can be selected with Tab.

Attention: When using the sudo context, always make sure to specify the namespace in which a command is to be executed. By default, only the current namespace is stored in kubeconfig when setting up the context. If a command is to be executed in a different namespace, this must be explicitly specified with a parameter: kubectl --context SUDO-mycontext --namespace mynamespace get secret

The sudo context should only ever be passed as a parameter. It should never be set as the active context, as this would grant permanent admin rights and thus undermine the protection against accidental changes. This is analogous to the sudo su command under Linux, with which a user has all permissions and is not stopped from performing risky actions.

However, both kubectl sudo and helm sudo do not require the namespace to be specified each time. Here the commands are always executed in the current namespace of the current context. Therefore, for helm and kubectl, the use of the sudo plugins is preferable.

Setting up the local tools

To create a sudo context this script is available: wget -P /tmp/ "https://raw.githubusercontent.com/cloudogu/sudo-kubeconfig/0.1.0/create-sudo-kubeconfig.sh"
With it, only these steps are necessary to interactively create a kubeconfig for the currently selected context:

chmod +x /tmp/create-sudo-kubeconfig.sh
/tmp/create-sudo-kubeconfig.sh

kubectl --context SUDO-mycontext get pod

If needed, the two plugins already mentioned, kubectl-sudo and helm-sudo, can be installed via bash:

Optional: install kubectl-sudo

sudo bash -c 'curl -fSL https://raw.githubusercontent.com/postfinance/kubectl-sudo/master/bash/kubectl-sudo -o /usr/bin/kubectl-sudo && chmod a+x /usr/bin/kubectl-sudo'    

kubectl sudo get pod

Optional: install helm-sudo

helm plugin install  https://github.com/cloudogu/helm-sudo --version=0.0.2

helm sudo list

Technical realization of the authorization

Now that the prerequisites for using the sudo function have been created on the local computer, the authorizations must be set up. We will show the steps for the technical realization as an example using a managed Kubernetes of the Google Cloud Platform.

RBAC

With the sudo function, we can now slip into the role of users, groups and service accounts to execute commands (impersonate). In order for us to get broader rights through the “impersonate”, they must be authorized using Kubernetes’ Role-based access control (RBAC).

The “impersonate” is implemented by:

kubectl-sudo: kubectl --as=$USER --as-group=system:masters "$@"
helm-sudo: helm --kube-as-user ${USER} --kube-as-group system:masters "$@"
and create-sudo-kubeconfig.sh: as-groups: [ system:masters ]

In an existing k8s cluster, two resources must be created to give users access to the impersonate feature:

A ClusterRole, which allows to use the impersonate feature.
A ClusterRoleBinding to allow individual users or groups to use the previously created ClusterRole.

# sudoer.yaml
# Creates a ClusterRole which allows to impersonate users,
# groups and serviceaccounts
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: sudoer
rules:
  - apiGroups: [""]
    verbs: ["impersonate"]
    resources: ["users", "groups", "serviceaccounts"]

# cluster-sudoers.yaml
# Allows users to use kubectl sudo on all resources in the cluster
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-sudoers
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: sudoer
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: admins@email.com
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: user1@email.com

In the current ClusterRoleBinding, everyone listed there has sudo permissions in all namespaces. It is also possible to use multiple ClusterRoleBindings and thus have permissions only for certain namespaces. This is a good approach if, for example, different teams have separate namespaces. To do this, another attribute namespace: namespace: namespace-name must be listed in the ClusterRoleBinding under metadata.

At first glance, it may look like anonymous changes to the cluster are now possible because a different role is assumed. However, in the audit logs in the Google Cloud Platform, the actual user principal who made a change can be seen. This means that it is possible to trace which user made a change. This works similarly in managed clusters of other cloud providers.

Google Cloud Platform (GCP)

In order for RBAC and the adjustments just described to be effective in the GCP at all, the original authorization per Google Cloud must first be bypassed. People who have the role Owner, Editor or Kubernetes Engine Admin in the GCP are normally allowed to execute everything in the cluster, even if they are not explicitly permitted by RBAC.

Under IAM, therefore, a custom role must be created once in the GCP that only allows authentication to the Kubernetes cluster. This role, which we call “Kubernetes Engine Authentication”, is assigned the following permissions:

container.apiServices.get
container.apiServices.list
container.clusters.get
container.clusters.getCredentials

This role now gets assigned to all users that need access to the cluster. The other permissions are then assigned by RBAC in the cluster. The role can also be assigned to entire groups. The groups are again managed via the GSuite groups. However, to do this, propagation of groups must be enabled when the cluster is created (Google Groups for RBAC). Unfortunately, this setting cannot be activated retrospectively for an already existing cluster. For this purpose, it must be rebuilt.

Conclusion

By using RBAC and sudo context, the effort for maintaining permissions and security are in a good balance. On the one hand, there is no need to maintain permissions for each person and on the other hand, the risk of unwanted changes, e.g. because one is in the wrong namespace, is significantly reduced.

Let’s take this scenario: A developer is testing changes to a deployment in his local dev cluster. During the course of the workday, minor work is done on the productive cluster in the GCP. Now the developer forgets to switch back to his local context and wants to delete the test deployment at the end of the day. Something like this has certainly happened to some people before. This can lead to downtimes or in the worst case to data loss.

However, if changes can only be applied to the production cluster using the sudo context or SUDO plugin, the accidental deletion will fail and the developer will notice his mistake. So, the vulnerability to accidental errors becomes less while maintaining ease of use, ease of implementation and high security. Since we have been using RBAC and the associated sudo context ourselves at Cloudogu, we have been working much more securely on our Kubernetes clusters.

GitLab DevSecOps Report 2021 - Proactively prevent vulnerabilities

Daniel — Mon, 20 Sep 2021 10:40:20 +0000

Web security or security-aware software development should no longer be a luxury. That's why terms like DevOps or DevSecOps have become an integral part of our industry. In other words, agile software development that is focused on security is one of the most important approaches to modern development. Or is it?

What does GitLab's DevSecOps Report 2021 have to say about this?

In it are some very interesting findings for the importance of security in software development:

99% of applications contain at least 4 vulnerabilities, 80% even have more than 20.
More than 90% of participants say that security scans run for more than 3 hours, with about a third running for more than 8 hours.
For more than two-thirds of participants, it takes more than 4 hours to fix a vulnerability.

These numbers show two things:

All applications contain vulnerabilities, although automated tests are already used to prevent them.
It is quite costly to fix vulnerabilities.

In addition, the report shows that a large percentage of companies have already been victims of successful attacks:

More than 70% have lost critical data.
Two-thirds have experienced operational disruptions and
More than 60% have seen negative impacts to their brand.

Based on these serious impacts, we might assume that security is becoming a higher priority. However, nearly 80% of DevOps teams reported just the opposite, saying they were under pressure to shorten release cycles. As a result, more than 50% of organizations reported sometimes skipping security scans to meet deadlines.

Preventing cyber-attacks and IT vulnerabilities

These results show that companies are in a dilemma: meeting deadlines or repercussions from successful cyber-attacks against themselves or their products. The simple solution to this would be to simply value security over new features. But nothing is simple when you constantly must innovate to succeed in today's fast-paced world.
Another solution to the dilemma is to equip development teams with the knowledge and tools to prevent security vulnerabilities from the start, when the code is first written. There are several ways to do this.

Continuing education in any form to proactively improve IT security.

There are a variety of different offerings in the area of in-person training: Training courses, eLearning, micro-learning, self-study, competitions, etc. Each of these forms of learning has a right to exist, as everyone has different preferences and strengths when it comes to learning. In addition, the different forms of learning have advantages with different levels of prior knowledge. Often a combination is very helpful. For example, in a classic training course, the basics can first be learned, which are then internalized through micro-learning or a competition. The important thing is to bring the training to the developers and not the other way round.

Classical training courses have, among other things, the advantages that they impart knowledge in a short period of time without distractions and that individual questions and requirements can be addressed. A disadvantage is that they often do not take place directly.
eLearning offers the freedom to work on the learning content at one's own pace, even in between. However, this often leads to the problem that continuing with lessons can easily be lost in the daily work routine alongside other tasks.
The situation is similar with micro-learning, in which learning content is broken down into small modules and ideally integrated into the daily work routine in a context-related manner. An example of this is the Secure Code Warrior plugin for SCM Manager (see below). The contextual integration of learning content has the advantage that the learning units are not in competition with other tasks because they are integrated into the tasks.
In self-study there is no fixed curriculum. This has the advantage that developers only acquire exactly the knowledge they really need. The disadvantage is, that all content must be researched independently.
At first glance, competitions are only suitable for deepening existing knowledge. However, they also offer the opportunity to gain new knowledge by working on problems that are new and have to be solved in a creative way.

Learn more about the free tournament here.

Micro-learning: improving safety through continuous and contextual learning

Contextual learning offers the opportunity to closely integrate practice and theory to improve learning outcomes. For this purpose, suitable learning content, e.g. in the form of micro-learning, is displayed during the processing of tasks. An example of this is the integration of videos and tasks on security vulnerabilities in the code review process.

Through such integrations, learning content is provided exactly when team members are working on tasks with potential security vulnerabilities. An example of this is the Secure Code Warrior plugin for SCM Manager mentioned earlier.

Conclusion

GitLab's DevSecOps Report 2021 shows that software security, while perceived as an important issue, is prioritized lower than the development of new features in many organizations. This prioritization is unlikely to change much in the future. Therefore, it is necessary to change from a reactive to a proactive approach in order to meet the security requirements while keeping release cycles short. This can be achieved through different types of training.

Learn to secure your app while coding it ...

jeromesch — Fri, 17 Sep 2021 10:05:29 +0000

Every Dev considers his Application as "safe" until he get's proven wrong.
Look up at the OWASP Top10 and tell me what you can check as "done" in your current project state:
-Broken Access Control
-Cryptographic Failures
-Injection
-Insecure Design
-Security Misconfiguration
-Vulnerable and Outdated Components
-Identification and Authentication Failures
-Software and Data Integrity Failures
-Security Logging and Monitoring Failures
-Server-Side Request Forgery

If you want to test your "secure coding skills", there's currently an tournament about exactly that:

https://community.cloudogu.com/t/secure-coding-tournament-how-to-take-part/189

SCW is reviewing your written Code automaticly against the (listed above) vulnerabilities, rates your overall score and shows where and how you can improve your skills.

Languages:
Kubernetes
Java
C# / MVC
JavaScript / React
Go
PHP
Python

Shorter release cycles through improved security

Daniel — Tue, 07 Sep 2021 15:55:43 +0000

The world’s reliance on software is already great and will continue to grow. That's why the security of applications will also become increasingly important. This development is further reinforced by the global pandemic, as more businesses and services have increased their online availability. The U.S. Federal Bureau of Investigation (FBI), for example, has reported a 300% increase in cybercrime since the beginning of the pandemic: This shows that with the growing reliance on software and applications, the risk of attacks is also increasing.

Software developing companies are therefore in a dilemma: On the one hand, they have to release new products and upgrades quickly in order to be successful on the market. On the other hand, these should also be secure in order to avoid successful attacks. However, security is often seen as a brake on rapid development - and therefore unfortunately neglected.
Note: Some tests, "penetration tests" for example, can take a very long time (2 weeks) - a contradiction to short development cycles.

This opinion comes from the fact that for a very long time development and security teams worked separately. Development teams write new code and get it ready for release. Before release, the security team looks at the code again and makes security requests before GoLive. This loop sometimes makes the development process extremely prolonged. It can also create tension between teams, as it is seen more as criticism than a security improvement.

DevSecOps for better integration of security in software development

The DevSecOps approach offers an answer to these challenges. It is an evolution of the DevOps approach that allows development and operations teams to work more closely together. In DevSecOps, security teams are integrated into DevOps teams.

From being reactive to being proactive in security.

No matter how teams are organized, in most cases (security) mistakes are corrected reactively. This is usually done in these ways:

Automated tests are written to check for known vulnerabilities.
A bug is reported that was discovered either in manual tests or in production.

The goal should be to detect bugs as early as possible so that the effort to fix it is as low as possible. The shift-left approach does exactly that and has been around for a very long time. However, no matter how early a bug is found in the development process, it always requires that someone from the team has to go back into the old code to correct the bug. Reducing the effort further only works by proactively preventing bugs. For more information about how expensive it is to fix security issues, see here.

Learn to write secure code and not discover vulnerabilities

In order to proactively prevent (security) bugs, developers must be provided with appropriate security guidelines and knowledge about security vulnerabilities. The Open Web Application Security Project (OWASP) has defined 10 measures for this purpose:

Define security requirements
Use security frameworks and libraries
Secure access to databases
Encrypt and escape data
Validate all input
Implement digital identities
Enforce authorization systems
Protect data everywhere
Log and monitor security-relevant events
Handle all errors and exceptions

Learning approaches to increase security in software development

In order for developers to be able to implement these measures, they must of course have the necessary knowledge. There are a wide variety of approaches for this, which can be used for learning depending on the topic and personal preference. The important thing is, that the learning material should not only be easily accessible, but also timely.

Security training in DevSecOps

According to traditional understanding, security is not one of the core tasks of developers, namely the implementation of new functions. That's why training on security topics needs to be fun, challenging, and engaging in order to impart critical knowledge. It is important that trainings are conducted using real code, in the respective language and according to prior knowledge in order to have a direct knowledge transfer.
Since the security awareness of the developers should be strengthened, it makes sense to conduct trainings regularly in small units and not, for example, to attend a training once a year.

Tournaments for more web security

Learning, as mentioned before, should be fun, challenging, and engaging for developers. All of this is achieved through contests where participants compete to win by fixing security vulnerabilities. Such contests can last as little as a few hours or as long as several days. And because the topic of web security is so important to us, we have planned a competition on the topic of security to coincide with the launch of the Secure Code Warrior plugin in our SCM-Manager.

Learn more about the free tournament here.

Automated real-time coaching

Meanwhile, there are extensions for development environments that check the code in real time against specified security policies. This way, developers get feedback on whether or not their code meets security specifications right as they are writing the code. Ideally, they are given direct suggestions for corrections, as with a spell checker. In addition, suitable exercises can also be suggested. This is very much in the spirit of gamification or e-learning.

Assessing the security leak

In order to be able to define measures to improve security, it is first necessary to determine what the current status is. This is the only way to identify weak points. Ideally, the status quo should not be determined only once, but the development should be monitored continuously.
One way to see the current state of knowledge of the team is to introduce badges or awards for developers who have completed certain trainings, for example.

Conclusion

To make software applications more secure without extending the development time, it is necessary to proactively write secure code. To achieve this, traditional structures must be broken down and more responsibility for the security of applications must be transferred directly to the developers. The basic prerequisite for this is that the developers have the necessary knowledge. Fortunately, there are now many interactive and engaging ways of imparting knowledge, such as training, tournaments, real-time coaching and assessments, which provide knowledge with a high level of practical relevance in small learning units.

Automation assistants: GitOps tools in comparison

schnatterer — Thu, 12 Aug 2021 13:20:06 +0000

If you want to switch from classic CI/CD environments to GitOps, then you can choose from any of a large number of available tools. However, it is not always easy to tell which features they support and how suitable they are for your project at first glance. This article provides help in making a decision.

The term GitOps is a combination of the name of the source code management system Git and the abbreviation Ops as in operations. The idea for adding this additional tool to the DevOps toolbox comes from the Kubernetes environment, and it promises a new level of IT automation. Like the continuous delivery approach, GitOps relies on maintaining all information in source code management. The difference, however, is that the deployment environment synchronizes its state directly from Git, and the CI server is not responsible for the roll-out. The configuration must therefore be versioned in Git. Since this is treated like code, it is referred to as "Infrastructure as Code".

It is no wonder then that there now is a growing number of GitOps tools to choose from. But what range of features do they offer? Is a single one sufficient, and can it automate "everything"? This article answers these and similar questions on the basis of specific examples. It lays out selection criteria and illustrates them by comparing the well-known GitOps tools ArgoCD and Flux v2.

A confusing market

Compiling an overview of GitOps tools available on the market is not as trivial a task as it might sound. On the one hand, this is due to the fact that there is a certain amount of hype surrounding the term, and vendors like to add the term GitOps to their products for marketing purposes. On the other hand, it is difficult to clearly define the term GitOps, and it is used on different levels of the stack (from physical infrastructure to the applications that are running in the cloud) in varying levels of maturity. The article “Hands off!” (published in iX 4/2021 - only available in German) delves into the topic in more detail.

Websites such as awesome-gitops, which was launched by Weaveworks, or gitops.tech, which was put together by INNOQ employees, provide an introductory overview of the available tools. When you take a closer look, you will see that the listed tools can be used to perform a wide variety of tasks related to implementing GitOps, and of course they also differ from one another in terms of their adoption, maturity, and how actively they are maintained. This article identifies three categories from the various use cases: Tools for Kubernetes, supplementary tools, and tools close to infrastructure. In addition, we compiled a table that summarizes the tools and their properties. The tables also contain various Git and GitHub-based metrics (current as of February 2021) that allow you to better assess their adoption, maturity, and how actively they are maintained.

Tools for Kubernetes

When it comes to GitOps tools, the first thing that usually comes up is the topic of operators for Kubernetes. In general, an operator (which is often also called a “custom controller”) is an application that runs in the Kubernetes cluster and automates operational tasks there (see Figure 1). This operator pattern is also used to implement GitOps. The GitOps operator is used to run the reconciliation loop, which synchronizes the target state declared in the Git repositories with the actual state of the cluster. In the event of differences (e.g., due to a new commit in Git), the operator takes care of convergence to the target state by applying Kubernetes resources to the API server. Experience has shown that additional features that go beyond the core feature set are required for efficient operation.
These include observability and a command line interface (CLI) or a user interface (UI). We will learn more about this later in the “Criteria for selecting the right tool” sidebar (at the end of the article).

The blog post by Weaveworks, which coined the term GitOps in 2017, also names the first GitOps operator: Flux. In the meantime, this has been completely rewritten as Flux v2. In addition to Flux and Flux v2, the associated project "Flux" develops other components. Weaveworks has now handed the project over to the Cloud Native Computing Foundation (CNCF). By now, the project is in the second maturity level: incubator phase.

ArgoCD offers an alternative to Flux. It belongs to the Argo project, which is also based at the CNCF, and which is, just like Flux, in the second maturity level (incubator phase). A comprehensive comparison of the two GitOps operators can be found later in the article.

A newer competitor is Fleet, which is developed by Rancher. Its special ability is that it is able to manage not just one, but a fleet of clusters. PipeCD is similarly young and has an even broader focus. Like Fleet, it promises the ability to manage multiple Kubernetes clusters, and it also offers a UI. In addition, it can handle Terraform and some services from the major cloud providers.

GitOps combined with CI

Jenkins X also offers a broader focus, but in a different area. Contrary to what the name suggests, it differs greatly from the well-known Jenkins server. It's not a monolithic tool, but rather it consists of different components, such as Tekton for running pipelines and Kaniko for building images. At the heart of Jenkins X is a CLI that the developers have rewritten for the current version 3 along with some fundamental architectural changes.

Overall, Jenkins X is more powerful than ArgoCD and Flux, and therefore it is more difficult to integrate into existing workflows. But it offers a significantly larger range of features. It is opinionated, i.e. it relieves the user of having to make many decisions, but it also reduces flexibility. Jenkins X is a complete continuous integration and continuous delivery (CI/CD) package. In contrast to this, the pure GitOps operators need an additional CI server for many use cases, which, for example, automates tests and builds images and makes them available in the registry.

Werf positions itself somewhere between a pure GitOps operator and a full CI/CD approach. The project was started under the name dapp, and then renamed werf in early 2019. Like an operator, it can apply Kubernetes resources from Git to a cluster. However, it runs outside of the cluster. This means that it does not utilize the pull principle, which is often associated with GitOps, in which the cluster itself pulls its target state from Git. Unlike ArgoCD and Flux, werf can also build images. An operator that runs in Kubernetes is planned (as of version v1.2 beta).

GitOps loves operators

A central point of GitOps is the complete declarative description of the state in Git. During the process of converting from the imperative, classic CI/CD (now referred to as CI-Ops in some places to differentiate it from GitOps) certain questions will naturally arise. How do you continue to use templating tools such as Helm or Kustomize that were previously executed by the CI server?

The answer to this is usually: by using additional operators. These expand the Kubernetes API server using so-called Custom Resource Definitions (CRDs) and then listen for changes to the associated Custom Resources (CRs). These CRs allow the desired state to be described declaratively, which is a perfect match for GitOps.

Flux brings with it Helm and Kustomize operators, which allow Helm releases and Kustomizations to be described declaratively via CR. If such a CR is applied to the cluster (typically using the GitOps operator), the Helm or Kustomize operator takes over the templating or overlay. As an alternative to the operator, the result of templating can be written to Git (for example, via a CI pipeline). The advantages include reduced efforts for maintaining infrastructure and more transparency in the Git repository. However, these advantages are counterbalanced by the need to create comprehensive and difficult-to-maintain YAML descriptions in Git.

By storing the entire state in Git, this inevitably leads to the creation of secrets there, too. However, if they are left unencrypted, these present a large attack surface. The answer to the question of how encryption and decryption can be combined with GitOps is yet again: through additional operators.

Components for better security

One simple option that works well together with GitOps is Bitnami's Sealed Secrets operator. It manages the key material in the cluster itself. There is a CLI for encryption that requires a connection to the cluster.

SOPS that was developed by Mozilla offers significantly more options, though at the expense of a more complex configuration. Here, the key material can come from the key management systems (KMS) of the major cloud providers, from your own HashiCorp Vault, or from configured PGP keys. SOPS itself does not contain an operator, but there are different ways to use it with GitOps. Flux v2 offers native support. There is also the helm-secrets plug-in, which can also be used in ArgoCD with the manual configuration. There is also a sops-secrets operator that has been developed by a third party.

Kamus may represent a compromise between Sealed Secrets and SOPS. It was created especially for the GitOps use case and includes an operator. It can either manage the key material itself or obtain it from the KMS of the cloud providers. Another special feature is that Kamus encrypts secrets directly for an application. They are then decrypted by the application itself or by an init container. This means that the unencrypted secret is never present on the API server and ideally also not in an environment variable with in the container.

If you are using an external KMS in any case, then there are other options, such as the kubernetes-external-secrets operator that was originally started by GoDaddy and the externalsecret-operator from Container Solutions. If you use HashiCorp Vault, you also have the option of using the Vault Secrets operator. This works similarly to the Sealed Secrets Operator, but instead of managing its own key material, it retrieves the secrets from Vault. The CNCF Technology Radar from January 2021 provides an overview of the types of tools that are available for secrets management.

Supplementary GitOps operators can also be used for deployment strategies, such as canary releases, A/B tests, and blue/green deployments, which have now been grouped under the term “progressive delivery”. The resources of most GitOps operators are not sufficient for this. One solution is Flagger. The tool that was launched by Weaveworks is now being developed as part of the Flux project. The Argo project also has an operator for this use case: Argo Rollouts. Both offer CRs for implementing progressive delivery strategies in interaction with various ingress controllers and service meshes.

Tools Close to Infrastructure

The term GitOps arose originally in the context of application deployments in Kubernetes. The tools for this use case are very mature. They are not limited to this use case, however. GitOps operators can also be used to roll out Kubernetes clusters. One scenario is to use the Kubernetes Cluster API, which was started as kube-deploy and renamed Cluster API (CAPI) in 2018. This can be implemented as follows: A GitOps operator runs in a management cluster and applies the CRs (defined by CAPI CRDs) stored in Git to the cluster. A infrastructure provider also running in the cluster reads these CRs and applies them to a target cluster.

As soon as this cluster is in place, the question arises of how applications can be rolled out there. Flux and ArgoCD can access the API server directly for this purpose. For other operators, such as Fleet and PipeCD, the architecture is specially designed for this case. They each offer one component for the management and target cluster: With Fleet, a manager operator connects to the agent operators, and with PipeCD, a control plane connects to daemons. So this does not require access to the API server from outside the cluster.

In addition to creating Kubernetes clusters, there is also an increasing number of opportunities to use various Infrastructure-as-Code (IaC) tools, such as Terraform, with GitOps. As was already mentioned, PipeCD offers support for Terraform. Terraform's vendor, HashiCorp, now also offers an official Terraform Kubernetes operator. However, it needs access to HashiCorp's Terraform Cloud. Alternatively, there are also third-party operators that can function without Terraform Cloud, such as the one developed by Rancher. However, it is still in alpha stage.

Another popular alternative for GitOps with Terraform is Atlantis: When generating a pull request, basing on the Terraform files that are found in the Git repository, it creates the Terraform plan and adds it to the pull request as a comment. After merging, it applies the Terraform plan. Atlantis is compatible with various Git providers. It can be flexibly hosted as either a binary, a Docker image, or as a Helm chart for Kubernetes. This makes it one of the few tools that will be of interest to those who want to implement GitOps without Kubernetes. Ansible Tower (and thus its open source upstream AWX) is also independent of Kubernetes. Red Hat considers the range of features to be comparable to a GitOps operator.

The example of Atlantis is, so to speak, symbolic of GitOps outside of Kubernetes. The pull principle of GitOps is more difficult to implement without a platform like Kubernetes that can run an operator. What remains often is the topic of "Operations by Pull Request”. However, with CI Ops the creation, modification, or merging of pull requests also triggers a CI/CD pipeline. This raises the question of whether the use of pull requests is sufficient for operations to be referred to as GitOps. It is more difficult to distinguish here. In fact, Atlantis does not refer to itself as a GitOps tool. But there are other examples, such as the Terraform alternative Pulumi. It works together with CI tools, such as GitHub Actions and GitLab CI, and it can be used to comment on pull requests on the associated platforms. Pulumi calls this GitOps.

Finally, there is one tool in particular that we should not leave out when creating a list of GitOps tools that are close to the infrastructure: Ignite, which was also launched by Weaveworks. It allows you to manage virtual machines (VMs) via GitOps. In order to do this, it runs a daemon on the physical host that can start and stop VMs in accordance with a description that is stored in a Git repository. Firecracker, which was originally launched by AWS, is used as virtualization technology.

So much and yet so little

Ironically, despite the abundance of tools, there are use cases that cannot yet be automated using GitOps. The selection of GitOps tools that are close to the infrastructure is significantly smaller than those for Kubernetes. The closer you get to the physical infrastructure, the less the tool support gets. Examples of tasks in operations that cannot yet be done via GitOps are starting physical machines, for example via the Preboot Execution Environment (PXE), or performing firmware upgrades in devices such as switches.

But even with the additional operators, not all use cases in operation can be fully automated with GitOps. One example is the Horizontal Pod Autoscaler in Kubernetes. To be compatible with GitOps, it would have to write the change in the number of replicas to a Git repository instead of sending it directly to the API server.

Another, even more complex example are the topics of persistence, backup, restore, and disaster recovery. Backup creation can be automated reliably using such operators as Velero. However, restoring a backup requires manual intervention. You can roll back to the state described in the GitOps repository, but this does not restore the state saved by the backup operator. Conversely, performing a manual restore using the backup operator does not reset the state in the GitOps repository.

Despite the abundant number of tools, there are still gaps in the feature set of the GitOps tool chain. Given the current dynamic of development, there is room for optimism that these gaps will be closed in the foreseeable future. In general, many of the available GitOps tools are based on Kubernetes. Even setting up the cluster or other infrastructure requires a Kubernetes cluster. Implementing GitOps without Kubernetes might requirea a bit of pioneering.

From the abundance of tools described above, it is now important to find the right one that satisfies your own requirements. The decision that you make will depend heavily on your use case. The "Criteria for selecting the right tool" sidebar (at the end of the article) will show you what to look out for.

Operators in comparison: ArgoCD vs. Flux v2

An example can better illustrate the criteria listed in the sidebar. So it makes sense to compare the two best-known GitOps operators, ArgoCD and Flux v2. As both projects continue to develop rapidly, so the comparison presented here can only be treated as a snapshot.

In order to gain an initial overview of similarities and differences, it is worth taking a look at the feature lists.

Both Flux v2 and ArgoCD have the following capabilities:

RBAC
Multi-tenancy
Observability (health status, notifications, and metrics)
CLI for automation and CI integration
Start of synchronization via a webhook
Calling webhooks using events (ArgoCD: hooks, Flux: notifications)
Rollback/roll-anywhere for certain commits in the Git repository
Support for Helm and Kustomize
Multi-cluster support
Execution of the container as an unprivileged user
Security context is configurable

Features only in ArgoCD:

Support for Ksonnet, Jsonnet, and others via the plug-in mechanism
Web interface to administer and monitor applications in real time
SSO integrations for UI and CLI

Features only in Flux v2:

Definition of dependencies in Helm and Kustomize applications
Support for SOPS
Automatic updating of the image version in the Git repository
Authentication of the CLI via Kubeconfig

Installation and configuration

ArgoCD can either be installed using simple Kubernetes resources and then patched or it can be rolled out using a configurable Helm Chart. All of the common settings are available, such as ingress and service accounts. Repositories, projects, applications, and various SSO integrations can also be preconfigured for deployment. The RBAC configuration is also extensive. You can define policies, roles, and groups yourself.

Another installation variant is the additional ArgoCD operator. This allows the actual ArgoCD components to be installed and configured via CRD. It is not documented how you can configure ArgoCD yourself via GitOps. This is conceivable, for example, using the ArgoCD operator. It remains to be determined whether this will work reliably and, above all, whether it supports continued operation via GitOps in the event of an error.

The Flux v2 operators are rolled out primarily via the CLI. However, it can also be used to generate all the necessary Kubernetes resources and to then deploy them to the cluster. There is currently no Helm Chart, which limits the usability of the common application mechanisms. During the initial installation, Flux sets up a Git repository. From this point forward, you can also configure the operator with GitOps. The CLI is extensive and allows you to control all aspects of the Flux v2 portfolio. You can use it to create, delete, read, start, and stop all Flux-v2-specific resources. This makes Flux v2 particularly suitable for scripting. This may seem contradictory for GitOps at first, but in practice there often is a gradual migration. The CLI can be used to create resources, which are then checked into a Git repository. This is useful for integratinh GitOps into existing CI/CD processes.

Application configuration

ArgoCD stores the target state internally through projects and applications. Repositories, clusters, and permissions can be defined in projects. In turn, any number of applications can be assigned to a project, where they will have access to the project's clusters and repositories. ArgoCD saves these applications and projects as CRs in the cluster. The repository and the path to a deployment can then be found in an application. In addition, you can also describe configurations for Helm, Kustomize, etc. in declarative form there as well. ArgoCD then applies this deployment, which is defined via the application, to the cluster. However, ArgoCD does not save Helm releases as such in the cluster, but converts them into simple Kubernetes resources using helm template and applies them to the cluster. In contrast to Flux, the Helm releases cannot be queried via helm ls and there is no history of the releases in the cluster.

All relevant elements are defined using CRDs, which can be configured using GitOps. Specifically, this means that you can maintain the projects, applications, etc., which are defined as CRs in your own Git repository.

With Flux v2, the repositories form the central point. They are used both for the deployment of workloads and the configuration of Flux itself. Flux can roll out Kubernetes resources, Helm Release CRs, and Kustomize CRs as workloads. The Helm Release CRD and Kustomize CRD also offer all of the known features of these tools in descriptive form. Since the Git repositories are also defined as CR, they can also be managed via GitOps.

UI

ArgoCD comes with a web interface for administering and monitoring applications. Since GitOps is an asynchronous process, you do not receive immediate feedback when changes are made to deployments. The included notification mechanisms provide a remedy for this issue (see the Observability section). They provide feedback through various channels, such as chat or e-mail. In addition, ArgoCD offers the opportunity to monitor deployments in real time via the UI (see Figure 2). If parts of the deployment fail, you can identify them via the UI and view error messages and logs.

This allows the developer to analyze their deployments and correct errors all without having to access the cluster. For authentication , there are interfaces for common protocols, such as LDAP and OIDC. Via configurable roles and groups, users can granted access the projects and applications for which they are responsible. The developers of Flux v2 are currently working on a web interface. However, it is still in an experimental state.

CLI

All relevant features can be performed from the CLI for ArgoCD. Users can create and delete objects, such as applications or projects, as well as change states, such as rolling back applications or triggering synchronizations. This makes the CLI suitable for all types of automation and integration in CI pipelines. The CLI communicates with the ArgoCD server, which makes exposing the Kubernetes API server unnecessary.

The Flux v2 CLI can also be used to access all relevant features. It also offers the option of rolling out additional tenants, which can be advantageous when automating a multi-tenancy environment. The CLI communicates directly with the Kubernetes API server, which must therefore be accessible from outside.

Authorization

ArgoCD relies on an organizational structure based on projects and applications. Applications, Git repositories, clusters and permissions (RBAC), as well as allow and deny lists for resource types can be assigned to a project. Applications offer the same options for restricting permissions. This allows users to assign permissions very easily and very accurately, including at both the project as well as the application levels.

Flux v2 currently only supports restricting the operators, Helm releases, and Kustomization CRs via RBAC. Simple Kubernetes resources, such as ConfigMaps and services, can only be controlled via the operator's permissions. At the project level, this might requires several instances of the Flux operator to assign project-specific permissions. Alternatively, access to the GitOps repository can also be implemented via a CI pipeline, which then only allows permitted resources to be pushed into the GitOps repository.

Observability

To send notifications about the synchronization status between Git and the cluster, ArgoCD utilizes the ArgoCD Notifications component, a notification system that is delivered with ArgoCD deployment. Events that arise when applications are rolled out can thus be forwarded to various channels. ArgoCD Notifications are currently supported on the SMTP, Slack, Opsgenie, Grafana, Telegram, and Webhooks channels.

As an alternative to ArgoCD Notifications, you could use Argo Kube Notifier and Kube Watch. However, these have to be operated in addition to ArgoCD. In any case, ArgoCD Notifications is best tailored to ArgoCD. It offers useful triggers and templates and can be installed and configured using the project's own Helm Chart.

To monitor the components and managed deployments, ArgoCD exposes two sets of Prometheus metrics: "Application metrics" for monitoring the status of the synchronization and the health status of deployments as well as "API Server Metrics" for monitoring the requests and responses to the API server. In addition, there are ready-made Grafana dashboards that are based on these metrics. This makes it very easy to implement a monitoring cockpit for the entire system.

Flux v2 has a special controller for Observability, the notification controller. As is true for all other Flux components, alerts and notifications are configured via corresponding CRDs. They can be used to set up providers, i.e., channels, such as chats and webhooks, alert rules and recipients. The associated CRs can also be maintained in a Git repository and deployed via Flux v2. The Notification Controller is currently suitable for the Slack, Discord, Microsoft Teams, RocketChat, and Webhooks channels.

There is also the option of attaching the status to a Git commit. This can be done with the following providers: GitHub, GitLab, Bitbucket, and Azure DevOps. At the moment, Flux v2 cannot connect a provider via the SMTP protocol in order to send e-mails in this way.

There are a number of Prometheus metrics and Grafana dashboards that are available for monitoring the controllers and deployments (see Figure 3). On the one hand, this allows you to monitor the error-free execution of the controller, providing you with statistics on CPU and memory consumption, for example. On the other hand, you can monitor the state of all Flux v2 CRDs, such as Helm releases and Kustomizations. This also makes it possible to send alerts by e-mail again. The alert rules can be defined in either Grafana or Prometheus Alert Manager. Prometheus and Grafana can also be installed and configured using the Flux CLI. In a multi-tenant environment, it can be used to automate deployment using scripts.

Conclusion

Both Flux v2 and ArgoCD offer many features that also reflect years of practical experience with GitOps. Both focus strongly on the GitOps core features, which makes them easy to integrate into existing CI/CD infrastructure. However, if you start out with a greenfield approach, you have to build it up separately.

Overall, ArgoCD offers more options for configuration (for example, for authorization) and provides a graphical user interface. However, this UI has to be configured and operated, and it also offers a larger attack surface. It depends on the use case as to whether a UI is needed at all, and this thus justifies the higher effort. Users of OpenShift might encounter less effort, since ArgoCD is integrated into the platform as OpenShift GitOps.

Flux has certain little features that ArgoCD doesn't have, such as support for SOPS and automatic updates for new image versions. However, the latter is the reasons that Flux v2 has not yet appeared in a stable version. It could be difficult to opt for a product with a version number 0.x when it is the central component in the supply chain. However, we do expect the release of a stable version here soon.
One possibility to see ArgoCD and Flux v2 in action and compare their features is the GitOps Playground project that was started by the authors.

Would you like some more?
There is a large number of tools that either bring themselves in connection with GitOps or that are often named in that context. The authors are available for discussions about the topic in the GitOps discussion forum at heise online. To meet the often-asked question after market overviews “Why is program X not mentioned?” head on, here are a few reasons why this article does not consider some candidates. Left out are tools that

Are only called imperatively or in the CI/CD process and therefore do not have a reconciliation loop (e.g. templating tools), even if they decorate themselves with the term GitOps;
Are no longer developed actively;
Are generally not recommended for use in production;
Have a highly limited use case, for example tools that are tailored for a specific cloud provider;
Are proprietary;
Are still very new and therefore not yet widely used.

Details about the mentioned standardization of the term GitOps can be found in the article “Hands Off”, published in iX 4/2021.

Criteria for selecting the right tool
Most of the GitOps tools are currently available in the Kubernetes environment. It is no wonder, because this is the oldest and most mature GitOps use case. This sidebar summarizes the important requirements based on the authors' own experience that you should consider when making a decision. However, many of these criteria apply not just to using Kubernetes. In this respect, they can also prove to be helpful for using GitOps in other environments.

Installation and configuration of the GitOps operator
As is well known, there are many ways to roll out applications in a Kubernetes cluster. This also applies to GitOps operators. Common options are rolling out via Helm Chart, CLI, or in the form of simple Kubernetes resources. Another aspect is changing the configuration of the operator once it is deployed. There is an interesting point to note here: Can the operator itself be configured via GitOps?

In addition, it is interesting to note what parts can be adapted at runtime (for example, using CRs – Custom Resources) and which parts require the operator to be restarted. Does the tool permit a multi-tenancy solution? An operator capable of managing multiple cluster can be helpful for implementing a multi-tenancy solution. Operators can support this using their own components in the target cluster, or they can communicate directly with the API server. Depending on the infrastructure, direct access to the API server may or may not be desired.

Application configuration
The Git repository must be defined and configured so that the GitOps operator can synchronize the cluster with a Git repository. It is important to map existing project structures with the operator used. It is also be interesting to find out whether the operator can handle multiple Git repositories and whether these can be added and configured without restart. In addition to the use of simple Kubernetes resources, there are other ways of rolling out applications in a Kubernetes cluster. These include Kustomize, Helm, Ksonnet, and Jsonnet.

The specific application types that an operator supports and whether they can be configured in accordance with the existing requirements will influence whether you decide to use it. Hooks can also play an additional role, i.e., they can react to events during the deployment in order to send messages, perform checks, or influence the deployment. Whether you need to map dependencies between resources may also be relevant. Examples of this include applying CRDs before associated CRs, or ensuring a database runs before the application is started. Occasionally, tools also offer additional opportunities for automation, such as, building images, writing new image versions to Git, or for executing entire pipelines.

UI
A graphical user interface allows easy access to the operator configuration and the application resources. It enables error analysis and handling without accessing the cluster by monitoring cluster objects and manipulating manifests. However, it should be noted here that the changes are not synchronized in the Git repository. Only the associated CRs in the cluster are changed. A single sign-on (SSO) is advantageous, since an existing user management can be used to enable users to access the UI.

CLI
A CLI can be used to develop scripts that automate GitOps processes or integrate them into CI pipelines. It is conceivable that new clusters will be rolled out, including operators, or the reconciliation loop will be triggered in existing workflows. The range of features that can be performed from the CLI will be a more important consideration depending on how extensively the CLI will be used.

Authorization
The assignment of permissions is of central importance for GitOps, since in principle everything that ends up in the Git repository is also applied to the cluster. Critical or security-relevant objects could be changed, or the cluster could be compromised or otherwise rendered unusable. One countermeasure is to restrict the operator's access to certain types of resources. Role-based access control (RBAC) is suitable for making sure that the operator is not rolled out with the permissions of a cluster administrator. Additional restrictions that you impose using allow and deny lists on resources and resource types can further improve security.

Observability
Due to the asynchronous nature of GitOps, feedback mechanisms play an important role. Because it is only after the operator has already deployed resources on the cluster that it is possible to know whether deployment was successful. Additional tools are required to receive these notifications (e.g., chat, e-mail, metrics, or commit status). Here you must consider which tools can be used and what effort integration into your own workflow is necessary.

Scrum vs. Kanban – How to select the best agile methodology for you

Daniel — Wed, 28 Jul 2021 11:56:23 +0000

During the last years it became common to use agile methods in software development. The most widespread ones are Scrum and Kanban. The “State of Agile” survey for 2020 for example found that a vast majority of companies (~75%) uses Scrum or Scrum hybrids. The second place is held by Kanban and “Scrumban” with about 15%. That is why we want to compare those two methodologies.

This post will be about Scrum and Kanban in general. For detailed information about the tools visit for example www.scrum.org or http://limitedwipsociety.ning.com/.

Why Agile?

What are the reasons to use agile approaches in projects? Let’s answer this question with some figures: The State of agile survey found that the greater majority of people (70%) using the agile approach improved their ability to manage changing priorities. Before people start working with agile tools they have expectations about the benefits, and after the projects completion they can say whether their expectations were met, or not. The following image shows the importance of several aspects and whether they improved by using agile methodologies or tools. (The data is from the 8th State of Agile survey, but the results stayed almost the same over the years.)

You can see that it is important for 75% of the participants of the survey to accelerate the “time to market” (blue bar). For 83% of projects this aspect improved by using agile methodologies (yellow bar). This behavior can be seen for all of the shown aspects: important aspects improved for the vast majority of projects. What the survey didn’t talk about, are the reasons why for some participants those aspects didn’t improve.

The Agile Manifesto

There are a lot of agile methodologies and tools that have similarities and differences. All are based on the same principles, which are written down in the “Manifesto for Agile Software Development”. The manifesto states that certain aspects are more important than others and that it is necessary to internalize those facts.

For example the manifesto says that responding to change is more important than following a plan. An important factor for the success of agile projects is that the participants have the right mindset to work in an agile project environment. Another important advice is that you shouldn’t limit yourself to just one tool. Combine aspects from different tools that fit your needs, but be aware of the fact that you are combining several tools.

There is a large number of agile methodologies. Many of them have quite similar approaches and use the same principles. Some of them use a lot of restrictions, others leave a lot of free space. The Agile Manifesto states the basic principles and each methodology uses it’s own set of tools and rules. The challenge is to find the methodology and tools that best fit your needs.

Comparison of Scrum and Kanban

After showing the most common reasons for using agile approaches and their basic principles, we can now compare the two methodologies that are being used the most: Scrum and Kanban. We want to provide a first insight to the mindset of Scrum and Kanban teams and show similarities and differences of the two tools.

It is said that working on projects with agile tools helps organizations to complete projects faster. This is because tools like Scrum or Kanban are process tools – they use transparency to show optimization potential and thereby help to work more effectively. An expectation to the users is to experiment by continuously adjusting to chaning circumstances and by customizing the environment. Another similarity of the two tools is, that they are both not very prescriptive – they provide a framework and leave space to adjust the methodology to your conditions. Nevertheless, Scrum is more prescriptive than Kanban.

Workload

One difference between the tools is, that Scrum limits the workflow by limiting the work in progress (WIP) indirectly whereas Kanban limits the WIP directly. In Scrum the limit of WIP is managed by the workload during one iteration, or sprint. In case of the following board the maximal workload is 4 (there can be a maximum of 4 cards in the WIP column), because there are only four cards. The Kanban board is different in one little detail, the 2 in the WIP column. This limits the WIP to two simultaneous tasks. Therefore it would be allowed to start with task C immediately, but task D can only be started after either B or C are finished. The Scrum team on the other hand is allowed to start the tasks C and D immediately.

Response Time for new Requirements

Another difference of the two tools is the way they respond to new requirements. Let’s say you have the following situation in your Scrum or Kanban project and someone turns up and wants to add task E to the board. In which way do the teams react to the new card?

The Scrum team typically says something like: “Sorry, but we have committed us to A, B, C and D for this sprint. Of course you can add E to the next sprint.”

The Kanban team would say something like: “Of course you can add E to the board. But to do that you have remove either C or D, because the limit of 2 tasks is reached right now. Or you wait until we finished A or B.”

Depending on the timing for the new requirement, in Scrum the response time to new requirements can be something between the full length of the sprint and one day (in case the requirement comes up one day before the new sprint starts). Therefore the average response time is half the sprint length. In Kanban the response time is as long as it takes to get capacity available. This can be instantly (by removing another task) or the time it takes to complete another task.

Comparison

The following tables will show you several similarities, differences and the basic work rules of the two methodologies to point out their strengths and weaknesses. The comparison should also help you to find the approach (or certain aspects) you can use in your projects.

Similarities

Both…
… are pull scheduling systems. This means that the team chooses when and how much work to commit to.

… are based on continuous and empirical process optimization.

… emphasize responding to change over following a plan.

… emphasize responding to change over following a plan.

… are Lean and Agile.

… limit WIP.

… use transparency to dive process improvement.

… focus on delivering releasable software early and often.

… are based on self-organizing teams.

… require breaking the work into pieces.

… help to continuously optimize the release plan based on empirical data.

Work Rules

The different approaches of the two tools can be shown best by showing their basic work rules.

Scrum	Kanban
Split your organization into small, cross-functional, self organizing teams. Split your work into a list of small, concrete deliverables. Sort the list by priority and estimate the relative effort of each item. Split time into short fixed-length iterations with potentially shippable code demonstrated after each iteration.	Visualize the workflow: Split the work into pieces, write each item on a card and put it on the wall. Use named columns to illustrate where each item is in the workflow.
Optimize the release plan and update priorities in collaboration with the customer, based on insights gained by inspecting the release after each iteration.	Limit Work in Progress (WIP) – assign explicit limits to how many items may be in progess at each workflow state.
Optimize the process by having a retrospective after each iteration.	Measure the lead time (average time to complete one item), optimize the process to make lead time as small and predictable as possible.

Differences

To emphasize the differences between the two tools a bit more the following table shows some aspects that are prescribed or optional in the tools.

Scrum	Kanban
Timeboxed iterations prescribed.	Timeboxed iterations optional. Can have separate cadences for planning, release, and process improvement. Can be eventdriven instead of timeboxed.
Team commits to a specific amount of work for each iteration.	Commitment optional.
Uses velocity as default metric for planning and process improvement.	Uses Lead Time as default metric for planning and process improvement.
Cross-functional teams prescribed.	Cross-functional teams optional. Specialist teams allowed.
Items must be broken down so they can be completed within 1 sprint.	No particular item size is prescribed.
Burndown chart prescribed.	No particular type of diagram is prescribed
WIP limited indirectly (per sprint)	WIP limited directly (per workflow state)
Estimation prescribed	Estimation optional
Cannot add items to ongoing iteration.	Can add new items whenever capacity is available.
A sprint backlog is owned by one specific team	A Kanban board may be shared by multiple teams or individuals
Prescribes 3 roles	Doesn't prescribe any roles
A Scrum board is reset between each sprint	A Kanban board is persistent
Prescribes a prioritized product backlog	Priority setting is optional

How to choose between Scrum and Kanban?

As you can see, the two tools have basic similarities, but they are very different in the details. If you want to use one of them, or certain aspects, you should be aware that both are more than just boards with a lot of cards and talking about those cards. Of course, using a board is a start, but there are so much more opportunities to improve a project. If you stick to the rules and tools of the methodologies they can help you a lot. An important thing you should keep in mind is that it is not important to stick to one methodology. You are free to combine aspects, tools and rules of different methodologies to set up your project management system.

So how do Scrum and Kanban look in real life? We will show an example of how projects proceed on the different boards. This again shows some advantages and disadvantages of the two methodologies and can help you to find the solution for your own project.

Examples of Scrum and Kanban in use

The following example represents a project that is less trivial than the one in the previous paragraph. There is a product backlog containing several tasks and a production process that consists of several steps.

In case of Scrum the team is in the first sprint, which consists of the tasks A, B, C, D, E and F. A is already done, B to E are at the moment in progress and F is still to do. During the upcoming sprints the team will commit to the tasks G to N.
The Kanban board consists of the backlog from which maximal 2 tasks can be selected for priorization. The development section allows max. 3 tasks at once (tasks from both columns – ongoing and done – count). After the development the features need to be tested and after that they will be in the production environment and are thereby done. You can see that the board represents the different states of the production process.

The Scrum board will be reset after each sprint, whereas the Kanban board is persistent.

After a few Scrum sprints or simply after some time the two boards could look like this:

Note: For Kanban it is not necessary that there is a maximal amount of tasks for each column.

The Kanban board, which contains more columns than the Scrum board, can help to optimize the process steps, because it is necessary to think about them in order to draw the board. It can also be very helpful to optimize the amount of allowed cards per column, because this can reveal bottle necks. The Scrum board and the partition of tasks into sprints helps to organize tasks and to think about reasonable packages. But remember, it is always possible to combine aspects of different methodologies and tools to find the board, rules and tools that fit your project best.

Other Agile Tools and Methodologies

Besides the two tools Scrum and Kanban there is a huge number of other agile tools and methodologies that serve as an inspiration for you. The more you read about them, the more you will see that they are based on the same principles (which are stated in the Agile Manifesto) and that some use the same approaches and methods. Here is a short list of some wide spread approaches that you could take a look into.

Extreme Programming (XP)
Lean Software Development
Feature Driven Development (FDD)
Crystal Family
Test Driven Development (TDD)

Bottom line

A Kanban board represents the workflow of a project whereas a Scrum board always consists of the same columns. Another big difference is that it is possible to limit the number of cards in certain columns when using Kanban while Scrum limits the WIP by restricting the number of cards in the Sprint Backlog.

Besides the two methodologies that were presented here there are numerous other agile approaches. Some of them are based on Scrum or Kanban. A very widespread approach is a combination of Extreme Programming and Scrum, because both supplement each other very well. It’s up to you to find the methodologies, tools and rules that you want to use to accomplish the aims of your projects. Since there is no “one fits all” for all projects you have to experiment and try variations.

How to define software requirements

Daniel — Mon, 26 Jul 2021 12:03:07 +0000

In order to prevent misunderstandings like this there are 2 important things you need to know:

how to define requirements clearly and
how to keep track of changes during the development.

How to Define Requirements

Depending on the way you work you need to describe requirements in different ways. In Scrum for example you create User Stories, in waterfall projects use cases or other forms of requirement documentation. Regardless of formal aspects you need to ensure that you have all information that are required for the implementation. To achieve this you only need to follow these 5 steps:

Identification of stakeholders: To assure that you get all relevant requirements you need to make sure that you know all stakeholders.
Collect requirements: There are different ways you can find out about requirements, e.g. you can use interviews, case scenarios or prototypes. No matter which method you’re using, you should always find answers to these questions:
- What is the product supposed to do?
- How well should it do it? (e.g. +/- limits, quality, measurable terms)
- Under what conditions? (e.g. environment, states)
Categorization of requirements: To get a good overview of the requirements you should categorize them. The two major categories are functional and non-functional requirements.
Interpretation and recording of requirements: Only well defined requirements can be considered adequately in the product. For each requirement you should…
- … define the requirement in detail.
- … prioritize the requirement.
- … analyze the impact of change.
- … resolve conflicting issues by talking to the stakeholders.
- … analyze the feasibility.
- … specify test cases.
Sign off: Before implementing a requirement you should get the ‘Go’ from the stakeholders.

After you talked to the stakeholders, identified requirements, defined and prioritized them, you have an evaluated list of aspects that need to be considered in the product.

Keep Track of Changes

It is one thing to capture all requirements for a project. The other thing is to ensure that all the requirements are met by the product. The Requirements Traceability Matrix can help you to keep track of all the requirements and associated test cases. The matrix links requirements with derived product specifications and test cases. This is what the matrix looks like:

ID	Priority	Description	Type	Risks	Specifications	Test Cases
Unique identifier	Priorization	A short description	Functional or non-functional	Associated risks	Link to the detailed description	List of associated test cases (unit, integration, system, user tests, etc.)

To make sure that a requirement is truly considered in the product, it is necessary that it is described in the product specifications and that it is associated with at least one test case.

If requirements change you have two different options:

Adjust the requirement specification accordingly and document the changes.
Add the changed requirement as a new item to the list and treat it like a new requirement.

No matter how you’re dealing with changes, the Requirements Traceability Matrix provides an overview of requirements and associated specifications. So if a specification changes, you can easily see which requirements are involved, and vice versa.

Requirements need to be tracked

Finding out about all requirements of a product and ensuring their implementation is the key to a happy customer. Therefore it is advisable to invest enough time into investigating and defining requirements. In addition to that it is necessary to ensure that the final product meets all requirements. This can only be achieved by considering them in the product specifications and by testing the specifications. This can be ensured by using a Requirements Traceability Matrix.

IT compliance in practice – correctly containing and deleting data and projects in B2B software development

Daniel — Fri, 23 Jul 2021 14:22:58 +0000

For many, compliance is an abstract concept that only managers For many, compliance is an abstract concept that only managers have to deal with. But it’s not. Compliance is everyone’s business.

What is compliance and why is it so important?

Briefly, compliance is the observance of laws, rules, standards, contracts and moral values, which can come from both outside and inside a company. The tricky thing about compliance is that non-compliance is not always immediately apparent. However, once it is discovered, things can quickly become unpleasant. Compliance affects not only management, but every employee, since anyone can knowingly or unknowingly violate it in the decisions they make. Examples of this include not using software under the terms of its license or failing to comply with data protection guidelines by not deleting data as required.

A special case: deletion or handing over of data

Since compliance is about adhering to regulations of any kind, this is very complex and individual issue. That’s why this post will be dealing with a very specific topic: the deletion of data when the contract ends.

In B2B software development, it is customary for contracts to contain a clause on how to hand over or destroy all records and documents related to the project.

At first glance, this does not seem to pose a problem. Yet, as is so often the case, the devil is in the details. Depending on how meticulous tidying up at the end of the project is, it can be very time-consuming to actually find and delete all of the information and data.

Inadequate isolation of projects can be a compliance issue

The clause referred to above means that at the end of a project all systems would have to be searched to find all of the data relating to the project. This ranges from obvious data such as repositories, artifacts, Wiki entries, documents such as conceptual designs or documentation to less obvious data such as tickets in the issue tracker or source code stored on secondary systems. How extensive this search is essentially depends on two factors:

The scope and duration of the project
Isolation of data from different projects

The scope and duration of projects

The longer and larger a project is, the more data is accumulated, increasing the likelihood that some of the stored data will be forgotten and that more and more systems will be used. This means that the “search radius” must be considerably expanded. It also means that there’s a lot more to do.

A wide dispersion of data can be reduced, for instance, by organizational rules on the storage of data. But rules like these often provide a lot of leeway and are hence only helpful to a limited extent.

Isolation of projects

In addition to the size and duration of a project, its isolation from other projects is also a factor influencing what has to be done to identify all of the relevant data. Here is a simple example of different levels of project isolation:

Low isolation: The data from different projects are all stored on one network drive. There is no prescribed folder structure.
Medium isolation: All of the data is on the same drive, but each project has its own folder.
High isolation: Data from different projects are stored on their own network drives with separate hard disks.

It is immediately apparent how much easier it is to identify the data in a highly contained project. If all of the data for one project is stored on a separate hard disk, it can easily be handed over to the customer or erased. In medium isolation, only a few folders need to be copied or deleted. The work involved in a low isolation situation is something everyone is free to imagine on their own.

It is also important to keep in mind that data may be present in any system used for the project. This quickly makes it relatively apparent why even small differences can have a big effect.

Backups – a story on its own

But that’s not all. Backups of systems are created to prevent data loss during a project. Depending on the level of project isolation, what needs to be done at the end of the project can be multiplied again. Because here too, the lower the level of isolation, the more complex the search will be. If the backups are still compressed or encrypted, the task becomes even more time-consuming, and because that is not enough, it must also be remembered that every change made to the backup puts its integrity at risk.

Separate infrastructure for each project

The good news is that it can also be very easy to abide by the contract: Using the Cloudogu EcoSystem, a completely independent instance can be used for each project. It contains all of the data such as repositories, issues, documentation, artifacts, etc., making it easy to delete all of the data at the end of the project or hand it over to the client. Backups are also easy to delete, since they are also created separately by project.

More security thanks to micro-learning and gamification – Secure Code Warrior plugin for SCM-Manager

Daniel — Fri, 09 Jul 2021 20:31:48 +0000

The regularity of media reports on cyberattacks shows that security is, or should be, a key issue for software development teams these days. Experience also shows that security vulnerabilities are usually not created by highly specialized functions. Rather, many successful attacks exploit well-known security vulnerabilities. For this reason, we are very pleased that the learning platform Secure Code Warrior is now integrated into our version management tool SCM-Manager.

An example of well-known security vulnerabilities is SQL injections, where arbitrary code is injected into database queries, allowing unauthorized information to be read (for more on this, see the Wikipedia article). Such attacks are very popular because they can be carried out very easily. That’s why SQL injections have consistently ranked first in the Open Web Application Security Project’s (OWASP) top 10 security risks since 2010.

These vulnerabilities are actually easy to close. Often, there just seems to be a lack of awareness, or the necessary time to perform appropriate security checks and design processes in such a way that security aspects are taken into account on an ongoing basis. Awareness can be created either classically through targeted training or through continuous learning, e.g. by means of microlearning or gamification. Secure Code Warrior is a very good example of the latter. By combining Secure Code Warrior with the version control management tool SCM-Manager, security aspects can be integrated into processes easily and in a time-saving manner.

Learning with Secure Code Warrior

The Secure Code Warrior platform makes it possible to use microlearning and gamification to gain knowledge about widespread security vulnerabilities and thus close them. The platform offers learning content on almost 150 security topics such as SQL Injection, Cross-Site Scripting (XSS), Memory Corruption or Client Side Injection for all common programming languages such as PHP, JSP, JavaScript, C++, Java Spring, .NET and many more. The content is taught in the form of videos (see example below) and programming exercises (challenges).

In combination with the plugin for the version control management tool SCM-Manager, the information is integrated directly into the software development process.

Open Source Version Management Tool SCM-Manager

SCM-Manager is an open source version management tool that Cloudogu took over in 2020 (to the official announcement of the acquisition). In the same year, we released the completely revised version 2 of the tool. SCM-Manager can be operated on-premises and offers, in addition to repository management, a complete review process for changes, the ability to edit files directly in the browser, and many other features.

The Integration of learning content about security vulnerabilities with the plugin for Secure Code Warrior is the latest enhancement of the tool.

Secure Code Warrior Plugin for SCM-Manager

With the free plugin, videos and links to security vulnerability challenges are displayed directly in pull requests. This way developers directly get all important information about the security vulnerability. For this purpose, the description of pull requests as well as comments and tasks from reviewers are searched for keywords.

For example, the pull request shown in figure 2 contains the keyword “SQL Injection” in the description. Therefore, the corresponding learning content is displayed.

This integration offers the possibility to use the information from Secure Code Warrior in different ways.

SCM-Manager makes the pull request a “security issue” with Secure Code Warrior

When a security vulnerability is found and fixed in the application, the pull request can be used to educate other team members on the topic – by performing the review. By mentioning the security vulnerability in the pull request’s description, information about the topic is displayed. This can be used to learn the theory. At the same time, the learned basics can be comprehended in the context of the expert’s changes in the own application. This approach spreads the knowledge of the topic over several people.

To have information on security topics displayed in pull requests, it is sufficient to mention the topic in the description or title of the pull request.

Security-related comments directly in the SCM-Manager review process

In SCM-Manager, reviewers can provide feedback on pull requests to point out potential security vulnerabilities. All that is required, is to mention a security topic in comments or in tasks. Thus, the corresponding Secure Code Warrior content is displayed in an automatically generated comment.

Note: Only “root” comments are searched for keywords, not replies to comments.

Take a look at the instructions if you want to try out the plugin.

Conclusion

The Secure Code Warrior plugin for SCM-Manager integrates information about vulnerabilities directly into the creation and approval process for changes. All that is required is that a person involved in the process mentions the security vulnerability. All the necessary information for implementation is then provided automatically. The advantage of this approach is that knowledge about security vulnerabilities is spread throughout the team without additional effort, and team members can educate themselves through self-study using micro-learning and gamification.