Forem: Nitin Naidu

How to Prevent Secret Leaks in Your Repositories

Nitin Naidu — Tue, 09 Jan 2024 08:38:06 +0000

Securing secrets exposed through Git repositories is crucial, considering it is one of the common ways of becoming vulnerable to security breaches. A survey by GitGuardian's State of Secrets Sprawl 2023 report states that 1 in 10 authors exposed a secret while pushing code in GitHub and 10Mn new secrets detected in public GitHub commits in just the year 2022 (an increase of 67% compared to 2021).

Git repositories can be secured by enabling secret scanning in repositories for any secrets already committed so that the exposed secrets can be identified and removed. While running scans can be a first step, one can also run checks for secrets getting committed during each commit so that such potential leaks are identified even before they happen, ultimately helping to follow the “Shift left” practice during code integration.

In this blog post, we will explore features in common Git hosting platforms that help avoid secret exposure. We will also explore an independent open source tool as an alternative.

How does secret scanning usually work?

Secret scanning relies on regular expressions (regex) to identify patterns associated with sensitive information in code repositories. Regular expressions are sequences of characters that define a search pattern and are widely used for text matching. The secret scanning regex patterns are crafted to recognize standard formats of sensitive data, such as API keys, passwords, and access tokens.

For instance, a regex pattern to catch any AWS access keys exposed might be something like AKIA[0-9A-Z]{16}. This regex pattern matches strings starting with "AKIA" followed by exactly 16 characters, each character being a digit or an uppercase letter. This could be used to identify AWS Access Key IDs, which are 20-character identifiers starting with "AKIA."

Securing secrets in Git hosting platforms with in-house features

With the recognition of the increasing number of secret exposures in the Git hosting platforms, many of the platforms have taken the initiative to address this issue by providing in-house features to avoid secret exposure on their platforms, and, if already exposed, ways to proactively flag it and take necessary action. Simple settings are available in these platforms to activate functionalities such as secret scanning, alerts for secrets pushed in pull requests, and blocking any incoming pull requests with exposed secrets.

Some of the platforms also have vendor partnerships to identify tool-specific vulnerabilities and actively maintain a list of patterns to recognize secrets, which are constantly updated. Utilizing AI (Artificial intelligence) is another option to extend the capabilities of secret scanning.

Let's have a brief look at what functionalities the major Git hosting platforms provide, which may or not fit your requirements for a secure repository.

1. Secret scanning with GitHub repository

GitHub provides the feature to scan for secrets existing in your repository as well as to put a restriction on pushing any secrets to the repository.

Enabling GitHub secret scanning

GitHub enables secret scanning for:

Owners of public repositories on GitHub.com.
Organizations with public repositories.
Organizations using GitHub Enterprise Cloud for public repositories (free) and private/internal repositories (with GitHub Advanced Security license).

When secret scanning identifies a secret in a commit, issue description, or comment, GitHub generates an alert. Alerts are sent to the registered email and can be viewed in the Security tab of the repository.

Also, GitHub allows you to enable secret push restriction at the repository setting and at the user level as a setting to avoid pushing to any repository, even if the setting is not enabled at the repository level. Check out the below links:

It is interesting to note that GitHub has a feature to use AI for generating regular expression which comes with an enterprise account. It is still in the beta stage but could help catch more secrets through regular expression. Further, you can read about troubleshooting secret scanning to understand why secret scanning might miss your secrets while scanning.

2. Secret scanning with Azure DevOps repository

Microsoft Azure provides secret scanning in the Azure DevOps repository under the GitHub Advanced Security license for the Azure DevOps repository. It provides secret scanning and push protection along with other features such as dependency scanning and code scanning.

Enabling Azure DevOps secret scanning

Secret scanning push protection and repository scanning are automatically enabled when you turn on Advanced Security. The scan details are displayed on the Azure DevOps Advanced Security page, where the user can take action on the alert.

Push protection works to stop any commits done in the command line and web interface as well. You can read the official documentation to learn more about Azure DevOps secret scanning.

3. Secret scanning with Bitbucket repository

Bitbucket scans your repositories for secrets and triggers notifications when leaked secrets are detected within new commits. Email notifications are sent to everyone involved in the commit history of the secret: the authors, committers, and the developer who pushed or merged the code containing secrets into the repositories.

Enabling Bitbucket secret scanning

Secret scanning is enabled by default in the Bitbucket instance, and both global and system admins can disable or enable secret scanning by modifying the configuration properties in the bitbucket.properties file. The functionality is provided without any additional cost. However, while writing this blog post, Bitbucket does not provide the functionality to proactively prevent any secrets from being committed to the repositories.

You can read the official website page to learn more about this feature of the Bitbucket repository.

4. Secret scanning with GitLab repository

GitLab proactively scans Git repositories to detect potential secrets (API keys, passwords, tokens, etc.) before they're accidentally committed and exposed. Secret scanning in GitLab runs a job to scan for secrets in the repository once it is enabled. It can be configured by enabling Auto DevOps or editing .gitlab-ci.yml.

GitLab provides the functionality as a combination of free and paid tier. In the free tier, one can run a scan, but certain features, such as security dashboards, are not available.

You can read official documentation to learn more about GitLab’s offering to secure secrets.

Open source tools

While we explored the in-built functionality provided by Git hosting platforms, there are scenarios where users may not be able to roll the end-to-end security in place to prevent any secret exposure in the Git repository due to certain constraints such as missing features, etc. End-to-end security means not only securing secrets already exposed in the Git repository but also adding a preventive step for users while doing commits locally. Some repository platforms provide all these features, such as GitHub and Azure DevOps repository, while others do not. At the same time, one also needs to buy an additional license to enable these features on some of the platforms. A crucial point to note is that if there is no step to stop the user from committing secrets locally, the secret is anyway compromised even if you have restrictions to disallow remote commits since the secrets may leave the user’s system.

To address these, we will explore open source tools as an alternative. We will check out how you can leverage Gitleaks in different ways to enforce security around secrets. The steps used for Gitleaks are similar to some of the other tools. While there are multiple tools available, one can select the tool based on the exact requirement. Below is a summary of the common tools and their features:

Feature	TruffleHog	Gitleaks	git-secrets	Infisical
Language	Python	Go	Shell	Go
Secret Detection Methods	Regex patterns, keyword lists, custom detectors	Regex patterns, custom patterns, machine learning	Regex patterns, custom patterns	AI-powered secret detection, pattern matching, anomaly detection
Ease of Use	Easy to set up and run	Easy to set up and run	Easy to set up and run	Requires minimal user configuration
Customization	Highly customizable with custom detectors and patterns	Moderately customizable with custom patterns	Limited customization options	Highly customizable with flexible policies and configuration options
Integrations	CI/CD pipelines, IDEs, security platforms	CI/CD pipelines, GitHub, GitLab	CI/CD pipelines, GitHub, GitLab	Extensive integrations with DevOps tools and platforms
Pricing	Open source (free)	Open source (free)	Open source (free)	Open source + paid (with additional features)
Overall Suitability	Suitable for developers and small teams	Suitable for developers and small teams	Suitable for developers and small teams	Suitable for enterprises and organizations with complex security needs
⭐ on Github

Secret scanning with Gitleaks

Gitleaks is an open source independent tool for detecting and preventing hardcoded secrets like passwords, API keys, and tokens in Git repositories. Gitleaks is an easy-to-use, all-in-one solution for detecting secrets, past or present, in your code. It can be a great option in case one is unable to implement steps to secure leaks of secrets into the Git repository end-to-end, starting from preventing any commits made by users in the local system to scanning for any secrets already pushed to repositories.

Gitleaks can be utilized to scan for secrets in:

GitHub action pipeline.
As a pre-commit file to scan for secrets every time a user runs a commit command.
To scan for secrets in the user's local repository before and after the Git commit with the Gitleaks CLI command.

1. Using Gitleaks in GitHub action pipeline

A simple example of using Gitleaks in your GitHub action is as follows:

name: gitleaks
on:
  pull_request:
  push:
  workflow_dispatch:
  schedule:
    - cron: "0 4 * * *" # run once a day at 4 AM
jobs:
  scan:
    name: gitleaks
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE}} # Only required for Organizations, not personal accounts.

The GitHub Actions workflow will automate the process of running Gitleaks to scan a repository for sensitive information, and it can be triggered by various events such as pull requests, pushes, manual requests, or on a scheduled basis.

2. Using Gitleaks as a pre-commit

Whenever a user clones the repository and tries to commit a file with a secret, it will give a warning to remove the secrets with Gitleaks as a pre-commit file. Follow the steps to implement it:

Install pre-commit.

Create a .pre-commit-config.yaml file at the root of your repository with the following content:
repos:

repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.16.1
    hooks:
      - id: gitleaks

Auto-update the config to the latest repository version by executing pre-commit autoupdate.
Install with pre-commit install.

Now you're all set!

3. Using Gitleaks in the user's local system

Pre-commit - The protect command can scan uncommitted changes in a Git repository. This command should be used on developer machines to follow shift left practice in security.

In your repository, run the below command, and it will let you know if any secrets are detected. You can use --staged along with the command to check for any secrets after adding new files to the staging area.
```
gitleaks protect -v
```

Post-commit - The detect command can be used to scan repositories, directories, and files for scanning any secrets committed

In your repository, run the below command, and it will let you know if any secrets are detected.
```
gitleaks detect -v
```

Gitleaks allows you to add custom configuration rules if you want to detect a particular pattern of secrets. It can be quite helpful to control false positives or to catch any particular pattern of secrets. For installation and to read more on the tool, check out the official GitHub documentation for Gitleaks.

Conclusion

Securing secrets can seem like a trivial task, but can be a crucial part of your step to have a secure repository and avoid unwanted security breaches. While some of the Git repository host platforms provide in-house features to scan your repository for secrets and also provide a barrier to any commits with secrets, some do not. You can leverage an independent open source tool like Gitleaks to prevent secret leaks in your repositories.

Furthermore, you might want to do some additional configuration work to ensure you can catch any secrets with particular patterns or ignore certain ones you do not want to flag, which are false positives. While securing the repository is one of the important measures to implement security, also check out implementing DevSecOps to secure your CI/CD pipeline for achieving adequate security in place for your organization from the integration of code to deployment and monitoring.

I hope you found this blog post informative and engaging. I’d love to hear your thoughts on this post. Let’s connect and start a conversation on LinkedIn.

Looking for help with securing your infrastructure or want to outsource DevSecOps to the experts? Learn why so many startups & enterprises consider us as one of the best DevSecOps consulting & services companies.

Mastering Platform Engineering with Kratix

Nitin Naidu — Sat, 25 Nov 2023 11:22:20 +0000

The evolution of DevOps to Platform Engineering is an interesting transformation occurring in the current technology landscape. Platform engineering includes the creation of innovative engineering solutions that help developers reduce the time spent on non-development activities such as infrastructure deployment so they can focus on application development and delivery.

The adoption of platform engineering is supported by various engineering solutions such as the IDP (Internal Development Platform) - for example Backstage. Some of these tools focus on creating a central dashboard for developers while some help platform engineers to build platform-as-a-service which could be as simple as helping to set up an environment for development by running a single command.

In this blog post, we will explore Kratix, a tool that enables you to build and deliver platform-as-a-service.

What is Kratix?

Kratix is an open source framework, offering platform engineers the capability to deliver platform as a product. Kratix empowers platform engineers to elevate their platform-building capabilities by harnessing the power of Kubernetes and GitOps at the same time providing means to include business logic as well in the service being offered.

Unlike other frameworks and tooling which focus exclusively on application-developer experience, Kratix focuses on empowering platform engineers to build better platforms. With teams adopting platform engineering technologies, Kratix could be an asset to your platform team if they are looking to deliver day-to-day platform engineering tasks as a self-service option for developers. Kratix could be an advantage if your platform team is looking to save up their time servicing individual requests from developers. While that said, Kratix is a flexible tool and can work alongside other tools as well and supports a wide range of use cases.

For visualizing how you may use Kratix, consider a situation where your QA team wants to test an application in a particular environment. This environment may include resources like a virtual network with a database and a Kubernetes cluster. The database and Kubernetes cluster may be required to be deployed with a particular version.

Without Kratix

In a typical business environment, it would involve raising a ticket and assigning it to the platform team who would then pick up the task and deliver it. This would entail platform engineers scrambling to put together the environment by firing up scripts and performing other activities while chasing the deadline given to deliver the task.

With Kratix

With Kratix, one would create a Promise which contains configuration to service a request such as environment creation. This Promise would entertain any request from the developer for an environment creation, with a couple of inputs from the developer, the Promise would run in the Kubernetes cluster and create the environment. In the background, the Promise would use Terraform CRD (Custom Resource Definition), and pipeline among others to deliver the task. We will discuss more about its working further down the blog post.

While this is just a simple use case, one could leverage this tool to deliver a comprehensive platform as a product with compliance, and business logic among others.

Features of Kratix

Promise

Promise is a transparent agreement between application and platform teams by shaping and implementing well-defined abstractions which are defined in a YAML document and created by the platform team. The Promise is installed on the Kubernetes cluster to fulfill the requirements asked by the developer or any stakeholder.

What’s in a Promise?

API

The API (in Kubernetes terms, the CRD) that application developers would use to request a Resource from the Kratix Promise. It defines the options that users can configure when they request the Promise.
Dependencies

A collection of prerequisites that enable the creation of a Resource that must be pre-installed on any destination. It could be Jenkins CRD (Custom Resource Definition) and the operator required to deploy Jenkins.
Workflows

Workflows allow you to define a pipeline that would run on on creation, maintenance, or update of a resource. The pipeline can constitute steps to convert user input to the operator’s expected document and much more. The pipeline can also help you to perform any tasks outside the Kubernetes cluster such as sending a Slack alert if the deployment has failed.

You can check out the Promise to deploy a PostgreSQL resource defined in YAML configuration to understand how the API, dependencies, and workflow come together to form a Promise.

Compound Promise

Compound Promises are Promises inside a Promise as a dependency which could be a way to deploy your entire stack starting from application to database packaged as one. Considering a business environment, a compound Promise could be your way of doing much more than deploying a single resource.

It could mean integrating a host of tasks starting from installing the Promise in a main cluster to installing dependencies such as CRDs in the worker cluster and also use cases like creating a golden path. It could involve creating a Promise that would include taking care of security, networking, storage, compliance, and deployment tasks while setting up an environment.

Kratix marketplace

Kratix marketplace is a place where you can find Promises that are created by the community and the Kratix team. This means you might not have to create a Promise from scratch, you could reuse existing Promises. If the need is not exactly in line with your requirement, you can go ahead and use it as a base Promise and build upon it to customize it as per your need. You can find multiple pre-created Promises for use cases such as deploying your monitoring stack - Prometheus and Grafana to setting up RabbitMQ or Kafka.

Integration of Kratix with other tools

Kratix complements many tools such as Backstage, Kubernetes operators, and Terraform among others. For instance, using Kratix you could create a Promise that could deploy Backstage views. This would be encoding the view as a configuration in Kratix. You could deploy these views as and when required any number of times.

Kratix could be used in tandem with the Terraform operator. Deploying cloud resources could be achieved through a Promise. A developer could simply request a virtual machine creation by requesting the resource and the Promise would service it by leveraging Terraform CRD to create the resource on the cloud.

How does Kratix work?

Kratix is built on top of Kubernetes. As mentioned earlier the working of Kratix revolves around a Promise. The Promise which contains the nuts and bolts to service the request would service any requests on a Kubernetes cluster. Kratix allows implementation in a single or multi-cluster architecture. In a single cluster, the Promise that fulfills requests for resource creation would be deployed in the same cluster where any incoming resource creation request would also happen. While in multi-cluster, you can have a platform cluster where Promise would be deployed and any incoming resource creation requests would be created in a separate worker cluster by the Promise. A simple visualization of Kratix in action is pictured below.

Implementing Kratix

To understand how Kratix can be implemented, we will install a Promise that fulfills a request for PostgreSQL resource creation. The Promise will be deployed on a single kind cluster called the platform cluster. In this example, you can think of the worker and platform cluster as one. This Promise consists of a base Promise from Kratix. This Promise accepts certain inputs from the user and creates the PostgreSQL resource based on the inputs. The base Promise has been updated to accept a new field ‘deployedBy’ which the user can also input during request. This field will be added as a label to the PostgreSQL resource which will be created.

The basic input parameters that the requester can pass include the name of the database, the name of the superuser who is deploying the resource, the namespace where it is getting deployed, and the label ‘deployedBy’.

The Promise for PostgreSQL is preconfigured with values such as CPU and memory limits for the PostgreSQL pod. A platform engineer could go a step beyond and also configure granular level access (reader, writer, and owner), node affinity, and many more which are specific to their business requirements. This helps the platform engineer to do away with intervention in every PostgreSQL deployment executed. At the same time, the developer has to just run a command with basic inputs to get the PostgreSQL up and running without having to worry about other database-related configurations which the developer does not have to decide upon. All of those could be pre-configured by a platform engineer in the Promise.

Below are the steps to perform the said use case:

Prerequisites

Docker should be installed
Kind should be installed (find how to install kind)

Steps

Kratix Installation

You can follow the official documentation to install Kratix as a single cluster setup.
Get the PostgreSQL Promise.

Clone the PostgreSQL Promise which is a base Promise from Kratix updated to have a new input parameter- ‘deployed by:’. Check out the promise.yaml and resource-request.yaml. The promise.yaml contains Promise configuration which is usually set by the platform team. The resource-request.yaml is the file that is used by the user to request PostgreSQL resource creation by adding the required input in the file.
```
$  git clone https://github.com/infracloudio/promise-postgresql.git  
```
Create pipeline image.

The pipeline in Kratix Promise for our use case creates manifest file based on user input any time user requests the resource. The manifest is required by Postgres Operator to create Postgres instance. This pipeline which consists of bash script that takes input and outputs minimal-postgres-manifest.yaml will be converted into a Docker image using a Dockerfile which we will use in subsequent steps. The rationale for this approach is the reusability of the pipeline once it's transformed into an image. The pipeline, once encapsulated in the form of a Docker image, can be easily reused across different Promise.

$ cd promise-postgresql/internal/configure-pipeline  

$ docker build . --tag kratix-workshop/postgres-configure-pipeline:dev  
[+] Building 0.2s (10/10) FINISHED  
docker:default  
=> [internal] load build definition from Dockerfile                    
=> => transferring dockerfile: 399B                                           
=> [internal] load .dockerignore                                             
=> => transferring context: 2B                                               
=> [internal] load metadata for docker.io/library/alpine:latest               
=> [1/5] FROM docker.io/library/alpine  
=> [internal] load build context                                              
=> => transferring context: 127B                                             
=> CACHED [2/5] RUN [ "mkdir", "/tmp/transfer" ]                              
=> CACHED [3/5] RUN apk update && apk add --no-cache yq                       
=> CACHED [4/5] ADD resources/* /tmp/transfer/                                
=> CACHED [5/5] ADD execute-pipeline execute-pipeline                         
=> exporting to image                                                        
=> => exporting layers                                                         
=> => writing image sha256:XX  
=> => naming to docker.io/kratix-workshop/postgres-configure-pipeline:dev

Give the kind cluster - ‘platform’ access to your pipeline image.

$ kind load docker-image kratix-workshop/postgres-configure-pipeline:dev --name platform   
Image: "kratix-workshop/postgres-configure-pipeline:dev" with ID "sha256:XXXXX"

Install Postgres Promise on the platform cluster and verify.

Make sure you are in the promise-postgresql folder while running the below command.

nitin@NITIN-NAIDU:~/kratix/Promise-postgresql$ kubectl apply   
--filename Promise.yaml  
Promise.platform.kratix.io/postgresql created  
nitin@NITIN-NAIDU:~/kratix/Promise-postgresql$ kubectl get crds  
NAME                                   CREATED AT
bucketstatestores.platform.kratix.io   2023-10-17T12:48:08Z  
certificaterequests.cert-manager.io    2023-10-17T12:47:14Z  
certificates.cert-manager.io           2023-10-17T12:47:14Z  
challenges.acme.cert-manager.io        2023-10-17T12:47:14Z  
clusterissuers.cert-manager.io         2023-10-17T12:47:14Z  
destinations.platform.kratix.io        2023-10-17T12:48:08Z  
gitstatestores.platform.kratix.io      2023-10-17T12:48:08Z  
issuers.cert-manager.io                2023-10-17T12:47:14Z  
orders.acme.cert-manager.io            2023-10-17T12:47:14Z  
postgresqls.marketplace.kratix.io      2023-10-17T12:50:31Z  
Promises.platform.kratix.io            2023-10-17T12:48:08Z  
workplacements.platform.kratix.io      2023-10-17T12:48:08Z  
works.platform.kratix.io               2023-10-17T12:48:08Z  

nitin@NITIN-NAIDU:~/kratix/Promise-postgresql$ kubectl --namespace default get pods  
NAME                                 READY   STATUS      RESTARTS   AGE        minio-create-bucket-pzjsr            0/1     Completed   0          22m      
postgres-operator-64cbcd6fdf-2dpkk   1/1     Running     0          14m

Verifying your Kratix Promise can be fulfilled.

nitin@NITIN-NAIDU:~/kratix/Promise-postgresql$ kubectl apply   
--filename resource-request.yaml  
postgresql.marketplace.kratix.io/example created  

nitin@NITIN-NAIDU:~/kratix/Promise-postgresql$ kubectl get pods  
NAME                                        READY   STATUS      RESTARTS   AGE  
acid-example-postgresql-0                   1/1     Running     0          29m  
configure-pipeline-postgresql-8b724-nx2ct   0/1     Completed   0          30m  
minio-create-bucket-pzjsr                   0/1     Completed   0          53m  
postgres-operator-64cbcd6fdf-2dpkk          1/1     Running     0          45m  

nitin@NITIN-NAIDU:~/kratix/Promise-postgresql$ kubectl get pods --selector deployedBy=InfraCloud  
NAME                        READY   STATUS    RESTARTS   AGE  
acid-example-postgresql-0   1/1     Running   0          30

There we go! We just deployed a customized Kratix Promise which helped to fulfill a request to install PostgreSQL in the Kubernetes cluster with the label - deployedBy and other inputs that the user provided during creation. We ran a command as a developer/user to request a resource which was successfully serviced.

Next, you can follow the official documentation to create a Kratix Promise of your own with business-specific requirements.

Conclusion

With platform engineering evolving, leveraging a tool such as Kratix would be beneficial to present your platform as a service, thereby alleviating the load on platform engineers and developers.

Kratix marketplace is a great place for you to use Promise developed by the community out-of-the-box. While some Promises may fit your requirements right away others may require customization to better cater to your specific needs.

In a nutshell, Kratix is a Kubernetes native flexible tool that could be used to create an abstraction for non-development activities for developers and serve as a handy tool for a platform engineer to create this abstraction for vivid platform engineering tasks.

Feel free to connect with our platform engineering consulting team to adopt tools such as Kratix and enable a shift to adopting platform engineering in your organization. You can reach out to me on LinkedIn to start a conversation regarding this blog post.

References

Kratix official documentation