How to Build a Robust CI/CD Process for a Bank: A Practical Guide

ToffGuy — Wed, 12 Mar 2025 10:29:56 +0000

Recently, someone from my network reached out with an intriguing question: "What are the best ways a bank can run its CI/CD process? How do we automate deployments and promote builds without unnecessary approvals?"

As someone with experience in infrastructure and DevOps, especially in highly regulated industries like banking, I decided to dive deep into this topic. The result is a comprehensive guide that outlines a CI/CD process tailored to the unique challenges of the banking sector.

In this post, I’ll share my thoughts, best practices, and lessons learned. Whether you’re working in banking or any other industry with strict compliance and security requirements, I hope this guide will serve as a practical resource to help you design or improve your CI/CD pipeline.

Understanding the Challenge

The banking industry operates under strict regulatory requirements, industry security standards, and internal policies. The challenge is to strike a balance between the speed of delivery and the need for rigorous security and quality controls.

The goal is to automate as much as possible while minimizing unnecessary approvals, without compromising compliance, security, or quality.

A Practical CI/CD Process for Banks

Here’s a step-by-step breakdown of the CI/CD process tailored for a banking organization:

1. Development

Use a unified code management system (e.g., GitLab) as the central hub for CI/CD processes.
Ensure all developer commits are signed with digital certificates to guarantee authorship and immutability.
Integrate automated security analysis into your CI pipelines:
- Scan code and dependencies for vulnerabilities (SAST, DAST, dependency scanning).
- Verify the SBOM (Software Bill of Materials) for prohibited licenses and vulnerable dependencies.
- Make these steps informative during development and blocking when merging into the main branch.
Leverage Merge Request Approval workflows:
- Implement the "four eyes" principle, requiring at least one peer to approve MRs.
- Add automated checks for code quality, linting, and unit tests with minimum coverage thresholds.

2. Build

Use a centralized artifact repository (e.g., Nexus or Artifactory) with clear zones of responsibility:
- Shared repositories for public proxies with access control and auditing.
- Team-specific repositories with restricted access based on roles.
Digitally sign artifacts and SBOM during the build phase to ensure immutability and traceability.
Standardize your CI pipeline to include:
- Unit, integration, and smoke tests.
- Code coverage analysis with minimum thresholds.
- Static code analysis and quality checks.
- Scanning containers and dependencies for vulnerabilities.
- Automatic versioning (semver) and changelog generation linked to task management systems (e.g., Jira).
Separate build and deploy containers:
- Minimize deploy containers to reduce the attack surface.
Regularly re-scan repository artifacts for newly discovered vulnerabilities.

3. Deploy

Differentiate approaches for test and production environments:
- Test environments should prioritize speed and informative checks.
- Production environments must pass all quality gates and security checks.
Major changes and infrastructure modifications require approval:
- Involve architectural and security teams to ensure compliance and consistency.
- Automate approvals with digital signatures and validations in CI pipelines.
Register and track changes using Change Management (ITSM):
- For manual CRQs (Change Requests), include:
  - Component and version.
  - Changelog of changes.
  - Update, verification, and rollback procedures.
  - Testing results.
  - Scheduled time and impact on other systems.
- Enable automatic CRQ (autoCRQ) registration under specific conditions:
  - No impact on service continuity or other components.
  - Automated testing is successfully completed.
  - Risk assessment is performed within the team.
  - In case of incidents caused by autoCRQ, temporarily suspend access until the issue is resolved.
Use modern deployment strategies:
- Canary deployments, Blue-Green deployments, and Feature Toggles.
- Monitor SLI/SLO and business metrics during deployment, with automatic rollbacks if performance degrades.

4. Monitoring and Tracking

Automatically update the IT Asset Management system (CMDB):
- Track all changes, configurations, and versions.
- Build a service-resource tree from business processes to infrastructure components.
- Support audits and regulatory reporting.
Maintain a unified internal Status Page for transparency and rapid response:
- Display the current state and SLA/SLI of IT systems.
- Monitor the health of critical business processes.
- Integrate with monitoring and alerting systems (e.g., Prometheus, Grafana, Zabbix).

5. Adjustments and Standardization

Use generators and templates for GitLab pipelines, Helm charts, and Kubernetes manifests:
- Developers provide a declarative application description (application manifest).
- Automatically generate necessary artifacts, including deploy containers and infrastructure code.
- This approach standardizes processes, minimizes errors, and ensures compliance with security and regulatory requirements.
Regularly audit and review processes for continuous improvement:
- Conduct retrospectives after major releases and incidents.
- Collect and analyze delivery metrics (e.g., DORA metrics, Lead Time, Deployment Frequency).

Reflections and Takeaways

Designing a CI/CD process for a bank is no small feat. It requires balancing speed, security, and compliance while ensuring transparency and control. The practices outlined above are not just theoretical—they’re grounded in real-world experience and best practices from the DevOps and ITSM domains.

However, no solution is perfect. Every organization has unique challenges, and there’s always room for improvement.

Let’s Collaborate!

I’d love to hear your thoughts and experiences!

What challenges have you faced when implementing CI/CD in highly regulated industries?
Are there additional practices or tools you’ve found invaluable?
How do you balance speed and compliance in your pipelines?

Let’s use this post as a starting point for a collective discussion. Together, we can refine and expand on these practices to create an even more comprehensive guide.

Feel free to share your insights in the comments or reach out directly. Let’s make CI/CD in banking (and beyond) as seamless, secure, and efficient as possible!

Thank you for reading, and I’m looking forward to your feedback! 😊

Forem: ToffGuy