Forem: Cloudsmith

13 KubeCon Europe 2026 sessions not to miss

Nigel Douglas — Mon, 23 Feb 2026 15:53:00 +0000

Thousands of platform engineers and security experts are about to gather for KubeCon + CloudNativeCon Europe 2026, making it the year’s definitive cloud-native event. To help you cut through the noise of a packed agenda, we’ve identified the standout sessions you can’t afford to miss. Whether you're looking for high-level strategy or interactive workshops, here is our curated roadmap to ensure you get the most out of your time on the ground.

📍 RAI Amsterdam (Europaplein 24, 1078 GZ Amsterdam, Netherlands)
🗓️ March 24 - 26, 2026

1. SBOOM: Making SBOMs Play Together

Date/Time: March 26, 14:30 - 15:00 CET
Location: Hall 8 | Room F
Theme: Security

In this session, research engineers Jacopo Bufalino (CNAM) and Agathe Blaise (Thales SIX GTS France) will dive into the messy reality of the Software Bill of Materials (SBOM). They plan to pull back the curtain on why current open-source and cloud-based tools (despite all their promises) frequently generate conflicting package lists and inconsistent vulnerability reports when tasked with scanning complex container images.

This talk is particularly electrifying for the security community because it tackles the compliance anxiety triggered by the EU’s Cyber Resilience Act (CRA). As the CRA shifts SBOMs from a nice-to-have transparency initiative to a legal necessity for software lifecycles, the industry is waking up to a major problem, which is inconsistent data. By dissecting the technical roots of these discrepancies, Bufalino and Blaise aren't just pointing out flaws, they will also be providing a roadmap for developers to ensure their tooling is actually CRA-compliant. For anyone navigating the shift toward a transparent and trustworthy software supply chain, this session offers the technical clarity needed to turn these metaphorical bombs into secure builds.

View session details

2. Modelpack: Standardising the packaging and distribution of AI/ML Models

Date/Time: March 23, 15:10 - 15:15 CET
Location: Elicium 2
Theme: AI + ML

In this high-impact lightning talk, Andrew Block, a Distinguished Architect at Red Hat, tackles one of the most pressing bottlenecks in modern infrastructure: the chaotic fragmentation of AI/ML integration within the cloud-native ecosystem. Block, a seasoned expert in helping organisations scale open-source solutions, will break down how the current wild west of competing formats and runtimes is actively stifling innovation. He will introduce ModelPack, a CNCF Sandbox project designed to act as the universal translator for AI/ML artifacts, allowing them to finally speak the same language as established tools like Kubernetes, ORAS, and Harbor.

The security and DevOps communities are closely watching this session because it addresses the Day 2 operations nightmare of managing AI at scale. As AI moves from experimental notebooks to production-grade Kubernetes clusters, the lack of standardisation creates massive technical debt and security blind spots. Block’s exploration of ModelPack and emerging artifact standards offers a glimpse into a future where AI models are managed with the same consistency and rigour as container images. For anyone struggling to bridge the gap between data science and cloud-native engineering, this talk provides the blueprint for a unified, scalable, and manageable AI supply chain.

View session details

3. Kubernetes security at Shopify scale: Automating security across an infrastructure monrepo

Date/Time: March 25, 16:00 - 16:30 CET
Location: Hall 8 | Room F
Theme: Security

In this session, Senior Infrastructure Security Engineers Jie Wu and Pulkit Garg will pull back the curtain on how one of the world’s largest e-commerce platforms secures its massive infrastructure monorepo. Drawing from their deep backgrounds in cloud defence and 5G network security, Wu and Garg will detail the high-stakes challenge of managing thousands of services where a single misconfiguration could impact millions of merchants. They will demonstrate how Shopify moved beyond manual checkbox security by building an automated pipeline that combines Semgrep for static analysis and Open Policy Agent (OPA) for real-time policy enforcement.

This talk is a must-attend for the security community because it provides a battle-tested blueprint for solving the velocity vs. security dilemma we’re all too familiar with. While many orgs struggle with friction between DevOps and Security teams, Shopify’s approach shows how to bake guardrails directly into the developer workflow without slowing down deployment. Attendees will gain rare insights into the so-called rough patches of scaling open-source security tools and leave with a practical framework for automating risk detection across complex, high-traffic Kubernetes environments.

View session details

4. To upstream or not? Why becoming the maintainer of your dependencies matters

Date/Time: March 26, 11:00 - 11:30 CET
Location: Hall 7 | Room B
Theme: Cloud Native Experience

In the session, Christos Markou, a Principal Software Engineer at Elastic and an OpenTelemetry code owner, dives into the strategic heart of modern software engineering. Markou moves beyond the typical "open source is good for the soul" narrative to present a pragmatic, business-first case for active maintenance. By sharing a high-stakes story of an OTel component saved from the brink of deprecation, he demonstrates how moving from a passive consumer to an active contributor transformed a potential technical debt nightmare into a rapid-response tool for solving critical customer issues.

This talk is a vital wake-up call for the security and observability communities because it addresses the growing crisis of software supply chain sustainability. In an era where a single abandoned dependency can lead to massive security vulnerabilities or operational outages, Markou provides a concrete example to share with your manager on why upstreaming isn't just altruism, it also can be treated as sensible risk management. For teams building on CNCF projects, this session offers a masterclass in how staying present in the ecosystem pays dividends in architectural control, security posture, and long-term engineering velocity.

View session details

5. How to add a new language feature to OPA

Date/Time: March 23, 10:20 - 10:25 CET
Location: Elicium 2
Theme: Open Policy Agent

Adding features to a widely adopted language like Rego involves more than just a simple pull request. In this lightning session, Apple's Charlie Egan explores the full-stack impact of introducing new syntax, from the initial parser modifications to updating the broader ecosystem of editor tools. Learn the specific engineering hurdles involved in upgrading a mature security project without disrupting the developer experience. At Cloudsmith, we’re huge fans of Open Policy Agent and Rego, so we can’t wait to hear what Charlie has in store for us.

View session details

6. Platform Engineering 2.0: Just-enough Kubernetes and AI-native DevOps

Date/Time: March 26, 15:15 - 15:45 CET
Location: Hall 11
Theme: Platform Engineering

In the session, Shweta Vohra, a Lead Architect at Booking.com and a seasoned authority on platform patterns, tackles the industry's growing complexity bloat. Drawing from 20 years of experience and the literal scars of re-architecting internal platforms at one of the world’s largest travel sites, Vohra argues that the future of infrastructure isn't about building more, it’s really about building just enough. She will break down how to transition from heavy, static systems to lean, right-sized architectures using k3s, Gateway API, and ambient mesh, while layering in AIOps via Kubeflow and Prometheus to transform raw automation into true system intelligence.

This talk is a critical touchstone for the DevOps and platform communities because it offers a pragmatic exit ramp from the scale at all costs mentality that leads to massive cloud waste and operational blindness. Vohra’s AI-native approach is a blueprint for creating self-optimising ecosystems that align infrastructure directly with business value. For engineering leaders and architects feeling the weight of over-engineered clusters, this session aims to be a masterclass in staying lean, staying smart, and evolving platforms to be adaptive rather than just additive.

View session details

7. AI Agents & Platform Engineering: Efficiency boost or new source of trouble?

Date/Time: March 26, 11:45 - 12:15 CET
Location: Auditorium
Theme: AI + ML

In the blockbuster panel, an elite group of industry leaders from Cisco, Red Hat, AWS, Solo.io, and the United Nations converges to debate the most disruptive shift in DevOps history. Featuring heavyweights like from the industry, this vendor-neutral discussion moves past the hype to ask the hard questions: Can non-deterministic AI agents actually keep pace with AI-accelerated developers, or are we just inviting unpredictable trouble into our production clusters? The panel will tackle the minimum viable platform foundation required for AI success, the shifting cost implications of agents in production, and how to build human trust in systems that don't always behave the same way twice.

This session is arguably the must-see event for the modern platform engineer because it addresses the looming intelligence gap in infrastructure. As developers pump out code at record speeds, the platform team is traditionally treated as the bottleneck. This panel explores whether AI agents are the ultimate scaling solution or a new category of technical debt. By bringing together perspectives from global tech giants and international organisations, attendees will walk away with a high-level framework for defining golden metrics for AI effectiveness and a roadmap for navigating the transition from static automation to dynamic, agent-led collaboration.

View session details

8. In Falco’s nest: The evolution of cloud native runtime security

Date/Time: March 24, 12:00 - 12:30 CET
Location: G102-103
Theme: Security

In the maintainer track session, Falco contributors Iacopo Rozzo (Sysdig) and Aldo Lacuku (Kong) provide a deep-dive update on the CNCF’s de facto standard for runtime threat detection. The core of their presentation focuses on the highly anticipated Falco Operator, a game-changer designed to automate the deployment and management of security sensors across massive, distributed clusters, effectively lowering the barrier to entry for enterprise-scale security.

For the security community, this talk is a vital look at the roadmap for high-throughput runtime defence. As cloud-native environments become more complex and data-heavy, traditional security tools often struggle with performance overhead; Rozzo and Lacuku will demonstrate the specific optimisations that allow Falco to maintain deep visibility without sacrificing system reliability. Beyond the code, this session also is a great opportunity to see how the Falco ecosystem is integrating with the broader CNCF landscape, offering attendees a first look at the features that will define Cloud Detection & Response (CDR) in the coming year.

View session details

9. What LLMs do, and don’t, know about securing Kubernetes

Date/Time: March 24, 14:30 - 15:00 CET
Location: Hall 11
Theme: Security

In the session, Rory McCune, a Senior Security Researcher at Datadog, explores the practical (and often perilous) intersection of Generative AI and cluster orchestration. McCune, a veteran KubeCon speaker and a foundational voice in container security, will present the results of his deep-dive research into whether Large Language Models (LLMs) can actually be trusted with the sensitive credentials. Rather than just discussing theory, Rory will aim to demonstrate how LLMs handle specific Kubernetes security tasks, revealing where they provide genuine architectural insight and where they tend to hallucinate dangerous misconfigurations that could leave an organisation exposed.

This talk is a critical reality check for the security community as AI-assisted DevOps moves from a trend to a standard operating procedure. McCune will break down how advanced techniques like improved prompting and chain-of-thought reasoning can significantly shift the safety of an LLM's output, while also highlighting the so-called no-go zones where human expertise remains non-negotiable. For security architects and engineers, this session provides a vital framework for auditing AI-generated IaC templates, ensuring that the speed of AI doesn't come at the cost of a catastrophic security breach.

View session details

10. Audit-ready Kubernetes: How Chase UK leveraged policy-as-code for continuous compliance

Date/Time: March 25, 14:15 - 14:45 CET
Location: Hall 7 | Room C
Theme: Security

In the session, Nischay Goyal, VP of Cloud Platform Engineering at JP Morgan Chase, and Jim Bugwadia, CEO of Nirmata and co-creator of Kyverno, provide a rare under the hood look at building a cloud platform within the high-stakes world of regulated retail banking. Goyal, who manages the end-to-end Kubernetes ecosystem for Chase UK, will detail how his team transformed a massive compliance undertaking into a streamlined, automated engine. By leveraging Kyverno, OpenReports, and Grafana, they successfully shifted security left, allowing their backend engineers to ship code at speed while maintaining a real-time policy-as-code safety net that satisfies stringent financial regulators.

This talk is a beacon for the security and compliance communities because it addresses the ultimate white whale of enterprise DevOps: reducing audit times from weeks to minutes. Bugwadia’s deep perspective as a Kubernetes Policy Working Group co-chair, combined with Goyal’s real-world production perspective from JP Morgan Chase, offers a definitive guide on how to empower security teams to write independent policies without bottlenecking development. For any platform team operating in a regulated sector (or simply running mission-critical workloads) this session provides a proven framework for turning compliance from a manual, fear-driven process into a better, more transparent, and scalable process.

View session details

11. Sandbox Operator: Enabling session-aware, efficient MCP tool execution in Kubernetes

Date/Time: March 25, 11:00 - 11:30 CET
Location: Auditorium
Theme: AI + ML

Alibaba Cloud engineers Mingshan Zhao and Zhen Zhang tackle the next massive infrastructure hurdle for AI: the Model Context Protocol (MCP). As maintainers of the OpenKruise project with experience managing Alibaba's million-container scheduling system, Zhao and Zhang are uniquely qualified to address the so-called “session explosion” problem. They will introduce the Sandbox Operator, a specialised Kubernetes controller designed to manage the lifecycle of AI tool executions without the massive resource waste or state loss typical of traditional Pod deployments.

This talk is a game-changer for the cloud-native community because it solves the sparse invocation dilemma, where these AI tools often sit idle, eating up expensive cluster resources while waiting for a user's next prompt. By integrating cutting-edge Checkpoint/Snapshot mechanisms, the Sandbox Operator allows tools to be paused and resumed without losing the memory of the conversation. For platform engineers and AI architects, this session shows how developers can efficiently scale AI agents to hundreds of thousands of concurrent users while keeping infrastructure costs low and user context intact.

View session details

12. Saxo service blueprint: Bridging legacy and modern world with Kubernetes operators

Date/Time: March 25, 16:00 - 16:30 CET
Location: Hall 11
Theme: Platform Engineering

In the session, Jinhong Brejnholt, Chief Cloud Architect at Saxo Bank, presents a sophisticated solution to the two-speed IT problem facing large enterprises. As a certified Kubestronaut and a leader in the Danish cloud-native community, Brejnholt details how Saxo Bank moved beyond the bottleneck of manual ticketing for DNS, certificates, and load balancing. She will showcase the Saxo Service Blueprint, a platform powered by Kubernetes Operators that exports the speed and reliability of GitOps to traditional VM-based applications, effectively unifying legacy infrastructure with modern cloud-native workflows.

This talk is particularly compelling for the platform engineering community because it addresses the last mile of digital transformation: managing the heavy, non-containerised dependencies that still power most financial institutions. Brejnholt will share how this architecture has saved thousands of developer hours and significantly bolstered disaster recovery capabilities for both cloud-native and legacy stacks. For architects struggling to reconcile modern Kubernetes automation with entrenched enterprise systems, this session offers some valuable real-world experiences and a roadmap for other banks to accelerate delivery across the entire organisational footprint.

View session details

13. Optimising error recovery for cost-efficient distributed AI model training with Kubernetes

Date/Time: March 26, 14:30 - 15:00 CET
Location: Elicium 2
Theme: AI + ML

For our final recommended talk, the academic precision of Radostin Stoyanov (undergoing his PhD at the University of Oxford) meets the enterprise-scale expertise of Andrey Velichkevich (AI Engineer at Apple & Kubeflow Steering Committee member). Together, they aim to address the GPU tax that plagues modern AI development: the massive costs incurred when long-running training jobs fail or when idle interactive workloads, like Jupyter notebooks, waste expensive compute resources. The duo will present a breakthrough approach using transparent GPU checkpointing integrated with Kubernetes-native APIs like Kueue, JobSet, and TrainJob to capture and restore the state of training jobs automatically, ensuring no progress is lost during failures or preemptions.

This talk is really exciting for the ML and Platform communities because it unlocks the holy grail of AI infrastructure: the ability to run reliable, high-stakes model training on preemptible spot instances. By moving checkpointing from the application layer to the infrastructure layer, Stoyanov and Velichkevich show how organisations can slash their cloud bills by up to 90% without risking their training timelines. For anyone tasked with scaling AI workloads while maintaining strict cost-efficiency and cluster utilisation, this session offers a sophisticated technical roadmap for turning unstable, expensive hardware into a resilient, high-performance training engine.

View session details

Secure software starts at Booth 570

Cloudsmith will be at booth 570, and we are bringing plenty of action to keep you busy. If you are ready to put your security skills to the test, join our capture the flag workshop: Stop the AGI apocalypse. You will hunt hidden malware, lock down LLM weights, and earn a shot at some great prizes. We also have a lineup of fun events to help you unwind, from canal cruises and kickoff drinks to beach-themed parties. Check out the full schedule of Cloudsmith activities here:
https://cloudsmith.com/events/in-person-events/kceu26

Track your Bandwidth & Storage limits with our Quota API

Kyle Harrison — Tue, 08 Dec 2020 14:14:26 +0000

At Cloudsmith, helping fledgeling startups grow from a single person operation to enterprise-level organisations is a constant joy! In those early stages, startups need all the help they can get to survive, and even veteran organisations experience similar challenges when scaling up rapidly.

Cloudsmith can help with our self-service approach to managing and defining storage and bandwidth limits to keep costs under control while allowing you to scale when needed. After all, no one wants to be caught between an unexpected bill for overages or any interruption to their business, no matter how minor.

The fantastic news is that now you have a new tool in your arsenal, as you can track your bandwidth and storage programmatically via the Quota API. It's easy to miss an email when your busy or a prompt within the Cloudsmith UI (if you haven't needed to log in recently); but you will always want to know if you're near/over a limit or even if you're over-provisioned.

Now you can automate a solution that works for you using either our API or CLI to always stay on top of your storage and bandwidth limits.

Getting started quickly

Using the Cloudsmith CLI, you can quickly and easily view the usage, limits, and maximum allocation of bandwidth and storage for your plan.

cloudsmith quota limits ORG-NAME

Alternatively, if you want to view the entire history for your organisation:

cloudsmith quota history ORG-NAME

Finally, to view the OSS usage for an organisation, you can add the -oss flag to any command to view only OSS limits and history.

cloudsmith quota limits ORG-NAME -oss
cloudsmith quota history ORG-NAME -oss

If you identify a limit that is over-provisioned or quickly approaching a threshold, then you can quickly and easily adjust your limits at any time within the Cloudsmith UI.

Additionally, if you wish to automate a solution, you can also check out the Quota API or the Cloudsmith API Bindings (available in multiple languages) and connect it to your favourite CI/CD or monitoring service to alert internally within your organisation.

Caching and Upstream Proxying for RedHat Packages

Dan McKinney — Wed, 25 Nov 2020 13:54:14 +0000

In keeping with our vision of offering a universal feature set across all the package formats we support, we are delighted to announce that we are now offering configurable upstream proxying and caching support for RedHat packages.

As we touched upon when announcing the same for Debian and Maven packages, there are a lot of reasons why this is a really good thing, so instead of going over those again, let’s jump straight into how you can set this up in you Cloudsmith repository.

Getting Started.

We don’t think that we could have made this much easier to configure (but, of course, we would love to hear your thoughts on this!)

In your Cloudsmith repository, you’ll see a menu item called “Upstream Proxying”. This is where we will configure our upstreams. Simply click the “Create Upstreams” button and select “RedHat” to create a new RedHat upstream:

You will then see the “Create RedHat Upstream” form:

You just need to add a Name for the upstream, the upstream URL, and a priority weighting – If you have multiple upstreams this will determine the order in which they are used.

We can choose to fetch and cache any requested package (instead of just fetching them), and to verify the SSL certificates provided by the upstream. Then, we select the individual distribution for this upstream.

Optionally - we can also add a GPG key, if required, for package signing, authentication headers for private repositories that require authentication and also some arbitrary headers if you wish to send something custom along with your request.

And that’s it, we have now added a new Redhat upstream to our Cloudsmith repository.

So now, the next request for a package that isn’t present in this repository will be passed through to the upstream and the package fetched it if it is available there. If you also enabled “fetch and cache”, the package will be cached in your Cloudsmith repo for any future requests.

Summary

By adding Redhat upstream proxying and caching, we are moving forwards towards our goal of being the most universal package management solution for your development, deployment and distribution workflows. We would love you to join us on this journey, and we are very excited about the platform we have built, the service that we provide, and where we are going to next. Sign up for free today, and experience Cloudsmith for yourself!

Did We Lose Something With The Adoption Of Containers? And Can We Get It Back?

Dan McKinney — Tue, 13 Oct 2020 10:36:05 +0000

The subject of containers probably doesn’t need much of an introduction. Since the launch of Docker in 2013 containers have become almost ubiquitous, with 89% of IT Professionals confirming their organizations used containers in some way during 2019.

That is no surprise. Docker itself calls the container “a standardized unit of software”, which is a nice way of saying that a typical container packages up all the relevant code, system tools, dependencies and libraries within an application, and effectively isolates that application from the rest of the environment and infrastructure.

As a result, containers have one great merit: they will reliably run, and run in the same way, wherever they are deployed. At a stroke, they eliminate the “well it’s working on my machine” problem that has caused endless heartache for both development and operations teams over the decades.

That benefit should not be under-estimated. For developers, in particular, it is rather wonderful to be able to put everything into a container and throw it over the wall knowing that it is going to deploy, and in turn knowing that it isn’t going to come back over the same wall for round 2.

It’s also - at least in theory - good for the operations team as well, for all of the same sorts of reasons.

But it isn’t all good news.

What Is A “Unit” Of Software, Really?

Prompted by the Docker definition of container I mentioned above, it might be instructive to ask ourselves whether the container is either the only, or indeed the best, ‘unit’ of software.

If we take a 10,000 ft view of the history of software development it probably looks something like this:

The entirety of the source code is the unit
The package is the unit
The container is the unit

In the old days of source code, you really had to be on top of things. The whole thing stood up or fell over as a whole, and things were pretty painful and expensive if (when) it did the latter.

When the package became the unit, things improved. Certainly, if we were following good package management and distribution best practices, then in most cases things fell over less and were easier to fix when they did. I’ll talk more about this later, but let’s also remember that packages also help speed up development full stop as they enable us to more easily re-use code and services.

In the move to containerization, the third stage in our evolution, the ‘unit’ of software has got bigger. As it has done so, it has concealed complexity in order to enable that crucial isolation from the environment it runs.

Essentially, the container runs as a kind of black box. The operations team just need to know what it does. They don’t need more detailed metadata and they don’t need to know what is inside it. After all, it’s all one self-contained package. So the unit of software has not only got bigger, it has also become less transparent.

The Problem With Being Big And Opaque

The problem with a large unit size is that within it are likely to be hundreds if not thousands of dependencies.

And the problem with being opaque is that we don’t know what they are.

What happens when something goes wrong? Let’s imagine (and this won’t take too much effort for veterans of LeftPad, HeartBleed, Event-Stream or one of any number of dependency related screw-ups) that a vulnerability is discovered within a certain package.

Where is that package currently in use in our ecosystem? We don’t know. Can we quickly roll that package back to a known ‘safe’ version wherever necessary? No.

Containers - as a ‘unit’ of software - live or die as a whole. If something is wrong or might be wrong, we have no option other than to spin up a new version and redeploy: not necessarily a simple task. And as we don’t necessarily know which containers are affected, we find ourselves struggling to get on top of an emerging security crisis based on limited data. Not a nice place to be.

The Lost Art Of Package Management

My last point is this: that containerization has led to a general decline in diligence and observance of best practices when it comes to handling packages and dependencies. After all, if I am going to throw them all into one container and check if it runs does it really matter where I get packages from?

The short answer is yes.

The longer answer is yes because if we aren’t going to be sure what is in any given container further down the line, we should be as careful as we possibly can be that nothing we can’t stand over sneaks in now.

Unfortunately, that doesn’t always happen. The path of least resistance is to get dependencies from pretty much anywhere and live with the consequences later. And as the developer cares about speed, and the ops team will be the ones dealing with those consequences, there is all the more reason to cut corners.

As developers, we need to get back to being diligent - and consistently diligent - about questions of security, provenance, reliability and availability when it comes to packages and dependencies. We need to be as sure as we can be that what we integrate into our projects (whether destined for containers or not) is what it says it is. And we need to know what we used where and when - so that we can fall back to a previous version quickly and easily if necessary.

It’s the least our colleagues in operations, and the users of our end product, deserve.

Restore authority with Token Bandwidth Controls!

Kyle Harrison — Tue, 06 Oct 2020 16:33:15 +0000

Now you can go beyond measuring your bandwidth usage and regain control via Cloudsmith's new bandwidth controls for Entitlement tokens. You can craft tokens with individual usage limits using the UI, API, and CLI, allowing you to decide the exact level of usage for each token.

Combining the new and existing limits for entitlement tokens, allowances are configurable to provide fine-grained control for any combination of properties. For example, the total amount of bandwidth, number of unique clients using a token, or the maximum number of downloads a token can perform on an individual token basis. Also, you can also scope your tokens by restrictions for advanced control of tokens.

If you are a vendor, you may want to have tiered levels of tokens for different users. Providing higher or even potentially unlimited allowances to your premium users is now possible, whilst maintaining control of your offering to free users with suitable limits. The best part is you can provide a lifetime limit for a token, or you can configure a refresh period to refresh the limits after the period has elapsed. For example, a token with a 1 GB daily limit, will allow up to 1 GB of daily usage and no more. After 1 day has passed, the token will be reset allowing another 1 GB of daily usage.

How it works

Within the Cloudsmith UI for Entitlement tokens, you can edit individual tokens to provide visibility restrictions, usage limits, or even provide additional metadata on the token for your own internal requirements.

To provide a bandwidth restriction with a refresh period. You will need to select a unit of Bandwidth e.g., GB, and enter a corresponding unit of bandwidth you would like to restrict by (e.g., 1GB of bandwidth). A "Monthly" refresh may be more than enough for most users but may prevent misuse of tokens by users accidentally using many TB's of data per month.

In this example, fine-grained bandwidth controls will be configured for the amount of bandwidth usage that token is allowed to consume every month. This is accomplished by selecting a few preset values and entering a bandwidth amount within the Edit Entitlement Token form.

To configure a 1GB bandwidth limit that resets monthly.

1. Select "Monthly" for "Refresh Token"

(Presets provide values from "Never Reset" to a range between "Daily" to "Annual")

2. Enter a number to Restrict by Bandwidth.

The number should be between 0 - 1000; you can go higher than 1000; however, the unit of bandwidth provides sensible defaults to keep values readable. (e.g. 1000000 bytes can instead be expressed as 1 Megabyte).

3. Select a Unit of Bandwidth.

Values range from a single Byte to an insanely large number of Petabytes.

Finally, select Edit to save your changes to the token. These restrictions will take effect almost immediately.

If you wish to set your limits programmatically, entitlement restrictions are configurable using the Entitlements API.

Finally, the easiest way to get started with making programmatic changes to entitlement tokens and applying restrictions is via the Cloudsmith CLI. Using the following command, you can set all visibility restrictions and usage limits for an entitlement token:

cloudsmith entitlements restrict OWNER/REPOSITORY/TOKEN_IDENTIFIER  --RESTRICTION

For Example:

cloudsmith entitlements restrict demo/example-repo/GYwg00eEElKs \
        --limit-bandwidth=1 \
        --limit-bandwidth-unit=gigabyte \
        --limit-num-clients=10 \
        --limit-num-downloads=1000 \
        --limit-package-query="package-darwin-amd64"  \
        --limit-path-query=tag:latest \
        --limit-date-range-from=2020-01-01T00:00:00Z \
        --limit-date-range-to=2077-01-01T00:00:00Z \
        --refresh-token=daily

It's that simple to get started!

Vendors Rejoice! Analyse your Bandwidth Usage

Kyle Harrison — Tue, 29 Sep 2020 15:13:29 +0000

As a vendor, understanding your bandwidth usage is an invaluable insight into how packages are distributed across your user base and how specific users have grown over a timeframe.

As a user base grows from 10s, 100s, and 1000s of users and package downloads are many multiples of your total number of users, it's essential to understand the distribution of your bandwidth utilisation by a user (or more precisely, entitlement token's) to aid in the identification of token's that make up a large percentage of your overall traffic.

When providing an entitlement token to a user under a specific license, you trust an entitlement token will be used to download packages upon the agreed-upon terms. However, an increase in your total bandwidth may start off slow and continuously grow over several months until it's suddenly become a massive increase in your overall bandwidth. Understanding this usage and precisely where this growth originates helps identify where/where change occurs and provides options to mitigate runaway bandwidth from specific entitlement tokens by changing how those specific users are managed.

How does it work?

Whenever a user downloads a package via an entitlement token, the exact number of bytes transmitted from our servers to the user host is stored for that specific token as an individual log entry. At the end of each day, we calculate the total bandwidth for every user across all of our repositories containing one or more packages and store the result as a daily aggregated value representing bandwidth usage.

The easiest way to get started exploring these metrics is to check out the metrics command within the Cloudsmith CLI. Alternatively, to get more fine-grained metrics, you can implement a programmatic solution using our API or one of the API binding libraries published in various languages.

Getting started quickly! Using the Cloudsmith CLI you can quickly and easily query the total bandwidth usage for your repository by running the following command with your organisation/repository:

cloudsmith metrics tokens OWNER/REPOSITORY

For Example:

cloudsmith metrics tokens demo/examples-repo

The following screenshot shows an example repository containing a number of active and inactive entitlement tokens alongside the total bandwidth used. The statistics table provides a simple insight into min/max/average token usage.

You can also retrieve usage metrics for one or more specific tokens by providing a comma-separated list of Entitlement token identifiers:

cloudsmith metrics tokens cloudsmith/example --tokens=ZGCV58VqT8Sl

If you wish to drill down further into a specific period of usage for a token, you can supply time and date for the start and finish parameters to filter for usage within that period. In this example, a single token is displayed for all of 2019:

cloudsmith metrics tokens cloudsmith/example --tokens=ZGCV58VqT8Sl --start=2019-01-01T00:00:00Z --finish=2019-12-31T00:00:00Z

It's that simple to get started!

You can find more information about Entitlement Tokens in our documentation.

Universal Package Tagging

Dan McKinney — Tue, 22 Sep 2020 10:55:53 +0000

A key factor in good package management is organization. How you organize and structure your repositories will help you to unlock the efficiencies and promises of modern DevOps processes.

There are, of course, a multitude of ways you can organize packages. You can group by version numbers, formats, architectures, filetype and more. At Cloudsmith we have supported this by extracting as much of this metadata as possible when you upload a package and making this metadata available for searching/filtering.

Sometimes, however, the metadata just wasn’t granular enough – or it didn’t provide quite the nomenclature that you may have wanted. You wanted more. And we are extremely pleased to say that we have now added a new feature that greatly expands upon this functionality. Presenting:

UNIVERSAL PACKAGE TAGGING

So, what is new about this? And why does it matter?

Well, in short, we now give you the ability to add ANY custom tags to ANY package or container, either during package upload or after the fact. And you can do this via the Cloudsmith CLI or the Cloudsmith API.

Let’s say for example that you were using Cloudsmith to distribute your packages to end-users, and you have “Free” and “Premium” editions of your package. Well now you can simply add “Free” and “Premium” tags to the respective packages, and then create Entitlement Tokens to give your users access to just the packages they should be allowed – without resorting to adding the edition into the filename or creating two separate repositories. You can manage it all from one central location. And of course, as Cloudsmith repositories are fully multi-tenant for package formats, you can apply these tags across all the package formats you use, all in one repository.

Or perhaps you would like to tag a package based on where it is deployed. You could tag a package as “rest-api”, or similar, to differentiate where in your production application the package is used.

Let’s look at an example

Let’s start off by listing the packages in our demo repo. The CLI command for this is:

cloudsmith list packages OWNER/REPOSITORY

So, we can see that we have a collection of RPM packages in this repo, with various different versions. Let’s check one of those packages to see if it has any custom tags attached. The CLI command for this is:

cloudsmith tags list OWNER/REPOSITORY/PACKAGE-IDENTIFIER

Example:

OK, this package does not have any custom tags. Let’s now add one, and the CLI command for adding a tag is:

cloudsmith tags add OWNER/REPOSITORY/PACKAGE-IDENTIFIER tag1, tag2, tag3

Example:

Great, we have now added a custom tag “free” to this package. We can repeat this for the other packages in the repository, varying the tags to suit our purpose. In addition, we can also specify that tags are “Immutable”, which means they can only be removed or altered by someone with Administrator permissions on the repository, or by the package owner. When we are finished adding tags, we can look at the repository on the Cloudsmith website:

You can see the new custom tags listed for each package alongside the tags that were automatically created from the metadata when the packages were processed after upload. We can now use these custom tags in any searching/filtering we need to do, or add them as a restriction on an Entitlement Tokens that we create for the repository:

The syntax for searching/filtering is the same syntax you use when creating a search-based restriction for an Entitlement Token, so it really is easy to create a set of access tokens that divide the repository up into subsets of packages.

Summary

Visibility of the attributes of your packages is important, it will help you not only structure your repositories but can also help you gain insight on what package is where or what package a group of customers can access. And it’s not just the basic attributes of a package that you need visibility on, it’s anything about a package that matters to your organization and workflows. With universal package tagging, you now have the ability to add your own searchable attributes to your packages, so you can define what is of importance.

Caching and Upstream Proxying for Debian Packages

Dan McKinney — Tue, 15 Sep 2020 19:57:32 +0000

At Cloudsmith, we want to be your “one central source of truth” for your dependencies and package management needs. And in keeping with this ideal, we are extremely pleased to announce that we have added fully configurable transparent Proxying and Caching support for Debian packages.

Why does this matter?

Well, in short, it means that you can now use your private Cloudsmith repository for all of your Debian package needs – whether that is your own private packages or packages that you need from public upstream sources. Your private Cloudsmith repository is all that you need to handle both.

If you request a package from your Cloudsmith repository, and that package isn’t present in the repo, then Cloudsmith will automatically check any upstream repos that you have configured and will then fetch (and optionally cache the package for future requests) from the upstream.

This brings you several important benefits:

Easier setup. Your Cloudsmith repository is the only repository you need to configure on your clients. No more need to configure multiple repos, and handle multiple authentication credentials etc. Configure the upstreams once in Cloudsmith, and that’s it done.
Isolation. If you have cached packages and dependencies that you require in your Cloudsmith repository, then if the upstream repo goes down, is otherwise unavailable, or if the packages are removed then you can still access your cached versions. No more breaking of build or deployment process due to an unreliable upstream.
Visibility. You can view details on what specific packages were requested from the upstreams. Gain insights into what you have, and what’s missing – or who and what else you are currently relying on.
Performance. Cloudsmith repositories are backed by a performant, global CDN. This means that your own packages and those cached from an upstream are delivered with the same low latencies and speeds. Going further than this, with edge nodes in almost all geographic regions, your users will experience this performance wherever they are located. Distributed teams all benefit equally.
Security and Control. All of your packages and dependencies in one place means it’s easier for you to implement the controls and security policies that you need. Multiple sources mean multiple management tasks. Keep everything in one place and keep a tighter hold on what you have.

Sounds Great!. How do we set this up?.

Well, it’s easy. In your Cloudsmith repository, you’ll see a menu item called “Upstream Proxying”. This is where we will configure our upstreams. Simply click the “Create Upstreams” button and select “Debian” to create a new Debian upstream:

You are then presented with the “Edit Debian Upstream” form. This is where we enter the details of the Debian upstream we wish to use.

We add a Name for the upstream, it’s URL and a priority weighting- in cases of multiple upstreams this will determine the order in which they are checked for a package.

We can then choose to fetch and cache any requested package (instead of just fetching them), and to verify the SSL certificates provided by the upstream. In addition, we can choose to enable this upstream for source packages too.

Next, we select the distributions and architectures that we wish to use this upstream for, and finally, we can add optional authentication headers (for private repositories that require authentication) and also optional arbitrary headers, if you wish to send something custom along with your request.

And that’s it, we have now added a new Debian upstream to our Cloudsmith repository.

Behind the scenes, Cloudsmith will now start to index the packages available in the upstream repository. The upstream will be ready for use as soon as the indexing is complete.

So now, for the next request for a package that isn’t present in this repository, Cloudsmith will check the upstream and fetch it if it is available there and also cache it in the Cloudsmith repo (if you enabled caching) for future requests. It’s that simple.

Summary

Debian upstream proxying and caching support is just another step on our path to providing you with the centralized controls, security, management and visibility that you need to enable a modern, high-velocity and DevOps-first workflow for your package management needs. It means fewer things to worry about, less exposure to change and most importantly....

Well, what's most important to you? Let us know!

Integrating a Cloudsmith Repository with a GitLab CI/CD Pipeline

Dan McKinney — Thu, 10 Sep 2020 08:44:38 +0000

What is GitLab CI/CD

GitLab CI/CD is a tool that is built into GitLab. It allows you to create automated tasks that you can use to form a Continuous Integration and Continuous Delivery / Deployment process.

You configure GitLab CI/CD by adding a yaml file (called .gitlab-ci.yml) to your source repository. This file creates a pipeline, which will then run when a code change is pushed to the repository. Pipelines are made up of a series of stages, and each stage can each contain a number of jobs or scripts. The GitLab Runner agent will then run these jobs.

For an on-premise instance of GitLab, you can install the GitLab runner agent on your own instances, and it supports many different operating systems (thereby creating your own fleet of instances to run your pipelines). But to keep things simple in the following examples, we will use gitlab.com and the default hosted GitLab runner environments provided.

Why use Cloudsmith with Gitlab CI/CD

So why would you want to use Cloudsmith with your Gitlab CI/CD pipelines? Well, there are a few reasons:

Universality – Cloudsmith supports over 20 package formats, so whatever artifacts your project produces, you can find a home for them in a private Cloudsmith repository
Control – Cloudsmith private repositories provide extensive security and access controls, that have been designed to accommodate workflows such as internal development, deployment or even distribution to external customers. The fine-grained permissions system available enables you to craft bespoke access control, and lock down or open up your repository as much or as little as you need.
Automation and Integration – Thanks to the Cloudsmith CLI and also native support for format-specific package managers, a Cloudsmith private repository can fit in seamlessly with other tools in your development or distribution processes. It provides you with a single source of truth across your packages and dependencies.
Performance and Reliability – As a cloud-native platform, Cloudsmith manages the availability and performance of your repositories. You don’t need to worry about managing a fleet of servers, containers or virtual machines. Global performance, backed by our ultra-fast CDN and multi-region infrastructure, ensures that your packages are delivered worldwide. Reliably, quickly and securely.

That’s not all. With features like custom domain support, configurable upstream proxying and caching, configurable edge caching rules, download logs/statistics and more, Cloudsmith aims to provide the best solution for all your package management needs. It really is the ideal tool in a high-velocity CI/CD workflow – precisely the type of workflow that GitLab CI/CD is intended to enable you to create.

Let’s work through an example.

OK, so let’s get started with a worked example. The very first thing you will need is some source code in a GitLab repository that you want to build. For this example, we will build Ruby source code into a Ruby Gem package.

Our project on GitLab has the following structure:

The important thing here, as previously mentioned, is the .gitlab-ci.yml file. This is where we define the GitLab CI/CD pipeline that will run when we push a change to the GitLab repo.

Let’s take a look at the .gitlab-ci.yml file:

image: "ruby:2.5"

variables:
  RUBYGEMS_HOST: "https://ruby.cloudsmith.io/demo/examples-repo"


stages:
  - build
  - push

build:
  stage: build
  script:
    - gem build cloudsmith-ruby-example.gemspec
  artifacts:
    paths:
      - cloudsmith-ruby-example-*

push:
  stage: push
  script:
    - mkdir -p ~/.gem
    - mv $CLOUDSMITH_API_KEY ~/.gem/credentials && chmod 0600 ~/.gem/credentials
    - gem push cloudsmith-ruby-example-1.0.1.gem

The first thing we have specified is the image that will be used when creating the Docker container for the GitLab runner that will execute this pipeline, in this case, Ruby 2.5.

Next, we add an environment variable, RUBYGEMS_HOST. This is where we define the URL of the Cloudsmith package repository that we will push the result of the build to.

We then define two stages in this pipeline, the build stage and the push stage.

The Build Stage

build:
  stage: build
  script:
    - gem build cloudsmith-ruby-example.gemspec
  artifacts:
    paths:
      - cloudsmith-ruby-example-*

This stage is pretty straightforward. We just run the gem build command to build our Ruby source as defined in our cloudsmith-ruby-example.gemspec file. Following this, we have defined an artifact job, as we need to temporarily store the output of the build so that the push stage next can use it.

This is because different stages in a pipeline will run on a new runner instance, so a subsequent stage wouldn’t have the access to the package built in a previous stage. You could also perform any jobs required to build and push within the same stage, and therefore on the same runner to avoid this, but for more complex pipelines you’ll likely have many stages.

The Push Stage

push:
  stage: push
  script:
    - mkdir -p ~/.gem
    - mv $CLOUDSMITH_API_KEY ~/.gem/credentials && chmod 0600 ~/.gem/credentials
    - gem push cloudsmith-ruby-example-1.0.1.gem

The push stage is a little bit more complex. This is due to the fact that we are going to push our package to a private Cloudsmith repo, which requires authentication. The Ruby package manager, gem, allows you to store your authentication credentials (in our case, the Cloudsmith API Key) in a credentials file, located at ~/.gem/credentials – But of course, we don’t want to check this credentials file into our GitLab repository along with our source!

So we can make use of GitLab's ability to add variables to the source code repository. We can create a file variable called CLOUDSMITH_API_KEY, and then as part of the push step, we add a job to move this variable into the required location before we run the gem push command.

You add this file variable in your GitLab repository settings:

Now, when our push step runs, the mkdir and mv jobs will create the required ~/.gem/credentials file (with our Cloudsmith API Key in it), and all without exposing our API Key in any logs on the runner instance, nor checked in with our source code.

The final job in our push step simply runs gem push to upload the package we have built to our private Cloudsmith repo, as Cloudsmith repositories offer full native support for the gem package manager.

Triggering the pipeline

All that is left to do is for us to make a change to our source, and then commit and push the change to our Gitlab repo. Let’s see what happens when we do that.

We make a change, commit it and push:

The Gitlab CI/CD pipeline starts, and the build stage executes:

The build stage runs gem build to build the package and then stores it in GitLab’s temporary artifact storage:

The Push stage then starts to execute once the Build stage completes:

The push stage downloads the package from the temporary artifact storage, creates the required ~/gem/credentials file, and runs gem push which uploads the package to our private Cloudsmith repository:

And that’s it, the pipeline is now complete and reports as "Passed":

If we now go and login to Cloudsmith, and check our "examples-repo" repository, we can see that the Ruby gem we just built is present:

Final Thoughts

Integrating a private Cloudsmith repository with GitLab CI/CD pipeline is easy, whether you are building a package that Cloudsmith provides native tooling support for (like Ruby) or even if you are using the Cloudsmith CLI to push a raw file, or other binary artifacts. You can add your own private Cloudsmith repository with just a couple of lines of configuration, and it just works.

Package management can be complex and difficult, and doubly so if you are trying to manage your own package management solution and infrastructure at the same time. Try it for yourself, and see the productivity and efficiency gains that you can get from a centralized, hosted, secure package management service.

Integrating a Cloudsmith Repository and a Buildkite pipeline

Dan McKinney — Tue, 01 Sep 2020 11:45:02 +0000

Cloudsmith and Buildkite

At Cloudsmith, you will often hear us refer to our mantra of “Automate Everything”. It's a quest that we never deviate from, and we believe that anything that can be automated, should be automated.

With that in mind, we would like to show you how simple it is to integrate a Cloudsmith repository with your Buildkite pipeline, and automate the pushing of your build artifacts into your own private repository for further CI/CD steps or even as a source for your global distribution needs.

What is Buildkite?

Buildkite is a platform for running fast, secure, and scalable continuous integration pipelines on your own infrastructure. That means you can use Buildkite to orchestrate and manage your own fleet of build hosts, and these can be anything from containers, to cloud instances, to bare metal servers.

Why would I want to integrate Cloudsmith with Buildkite?
Well, in short, a continuous integration pipeline is going to have an output, and you are going to need and want to put that output somewhere that is secure, controlled and integrates with all the other tooling that will form the next parts of your DevOps workflow - whether that is continuous deployment to production, distribution to an end-user or customer, or even consumption by another internal team as part of their development process.

This is where Cloudsmith fits in, and this is another phrase you might hear us use quite a bit - Cloudsmith offers a central source of truth for your packages and build artifacts. We provide a global platform that gives you the performance, scalability, security and visibility that you need to control and manage your software assets.

Using Buildkite and Cloudsmith – An Example:

So, you’re let’s assume that you’re new to Buildkite, how do you get started?

Step 1- Install the buildkite-agent

The first thing that you need to do is install the buildkite-agent so that you have a machine (remember, this can be a container, a VM, or even a real server) that will act as your build host. Buildkite provides install instructions for all major platforms and operating systems:

Once you have installed the agent, you will see a new build host in the Buildkite UI:

For this example, I have installed the buildkite-agent on a Debian instance.

Step 2(a) – Create a pipeline

The next thing you need to do is create your pipeline. A Buildkite pipeline is a series of steps that your build host(s) will execute in order to build your assets/artifacts. For this example, we will create a pipeline that will compile a simple C source program, package it into a deb package and then push that deb package to a Cloudsmith repository.

To get started, you give your pipeline a name, and then you specify a source repository for the pipeline. In this case, it is a GitHub repository (although Buildkite will also work with many other platforms as a source):

If using a private GitHub repository, you also need to remember to add an SSH key to your GitHub profile so that the host you have installed the buildkite-agent on can access the source repository.

2(b) – Add your pipeline steps.

Pipeline steps are where you specify the commands or scripts that you need to run in order to build your source / project / application. In Buildkite, you can define these steps via the Buildkite UI or as a pipeline.yaml file. To keep things simple, we will define two steps via the Buildkite UI.

PreBuild Step

In this step, we install the tools we need to build our source and package it into a deb package. We also install the Cloudsmith CLI:

BuildAndPushPackage Step

In this step, we use make to compile our source and then we use fpm to build the deb package. Finally, we use the cloudsmith push command to push the package to our Cloudsmith repository:

The equivalent pipeline.yaml file for these steps would look like:

steps:

  - label: "PreBuild"
  commands:
      - sudo apt update
      - sudo apt-get install ruby ruby-dev rubygems build-essential python-pip -y
      - sudo gem install --no-document fpm
      - pip install cloudsmith-cli 

  - label: "BuildAndPushPackage"
    commands:
      - make
      - fpm -f -s dir -t deb -v 1.0.1 -n cloudsmith-buildkite-test .
      - cloudsmith push deb demo/buildkite-demo/debian/buster cloudsmith-buildkite-test_1.0.1_amd64.deb

For more complex build pipelines, you’ll likely have a lot more steps and the advantage of using a pipeline.yaml file is that you can version it and check it in right alongside your source.

One thing to note is that pipeline steps in Buildkite are stateless. As a result, if you have a fleet of agents then each step is not guaranteed to run on the same agent. This means that if a subsequent step need / relies on the output from a previous step, the output from one step will need to be stored and then retrieved. Buildkite provides temporary artifact storage that you can use for this purpose (see here for more details), but to keep things simple in this example we performed the build and push in a single step.

Step 2(c) – Add a Github Webhook.

We want this pipeline to start when we push a new commit of our source to our GitHub repository and for that, we can configure a GitHub Webhook. Conveniently, Buildkite provide a webhook URL that we just need to copy and paste into our GitHub webhook configuration, and then select the event type we wish to fire this webhook on (in this case, all pushes):

That’s it, our pipeline is now built and will be ready to run.

Step 3 – Environment hooks

Buildkite supports several types of hooks that can run on a build host during a pipeline run and a final piece of configuration that we need to do is use an environment hook to configure our environment.

As we are using the Cloudsmith CLI to push the package to our Cloudsmith repository, we need to set up our Cloudsmith API Key as an environment variable. We do this because we don’t want to store a sensitive secret like an API-Key in our pipeline source, or within a build step as a plain text environment variable where it could be exposed in logs. There are other alternative methods of managing secrets in Buildkite, see here for more details.

Our environment hook is pretty simple:

set -euo pipefail

if [[ "$BUILDKITE_PIPELINE_SLUG" == "cloudsmith-buildkite-demo" ]]; then
    export CLOUDSMITH_API_KEY="abcdefghijklmnop1234567890"
    export PATH="$HOME/.local/bin:$PATH"

fi

It does two things:

Sets the CLOUDSMITH_API_KEY environment variable
Adds the location that the Cloudsmith CLI is installed to our PATH Additionally, it only runs when executed by a pipeline with the name “cloudsmith-buildkite-demo”

OK, at this stage our pipeline is built and ready, and the environment on our build host will be set up when the pipeline executes. It’s time to test it out.

Step 4 – Push a change to our source repository.

Now if we make a change our source, commit it and then push the change to our GitHub repository, our build will start:

And once the BuildAndPushPackage step has completed, we can view the output in the logs:

We can see that our deb package was successfully built and pushed to our Cloudsmith repository.

If we now go to the repository on the Cloudsmith website, we can see the built package:

To sum up

Buildkite is a very powerful and flexible CI management platform, and integrating your Cloudsmith repositories with your Buildkite pipelines is as simple as installing the Cloudsmith CLI and then using the Cloudsmith push command in your build steps. We would really encourage you to give it a go yourself, our trial is fully-featured and we are here to help you get started. No question is too big or too small.

Happy Build(kite)ing!

Cloudsmith Now Supports Conan!

Kyle Harrison — Wed, 26 Aug 2020 10:40:14 +0000

We’re delighted to announce that Cloudsmith now supports Conan!

As most of your know, Cloudsmith is universal. It is our aim to support all the languages and package formats our customers and prospective customers use. We think any organization benefits from being able to store, secure, manage and distribute ALL of their software assets in a single consistent manner.

That doesn’t necessarily mean multi-format repositories, but rather every member of the team knowing where to find the packages they need and being able to integrate them into build and deployment processes in the same way - no matter what format.

Of course, there are a lot of formats and languages out there. So we never stop working to ensure that we cover as many as possible. We listen and respond to our customers, all with the intention of building the only truly universal cloud-native package management platform.

Hence our support for Conan. Now on with the detail...

Introducing Conan

Conan is an open-source package manager for C/C++, including everything from its client to server implementation and even documentation.

It is actively developed on GitHub by an awesome community of contributors and a team of engineers working full time on the project. C++ and C popularity continues to hold a steady spot at 9th and 11th place in the "most popular programming, scripting and mark up languages" category of the 2019 Stack Overflow developer survey. Additionally, they hold the 6th and 9th place in the most popular programming languages on Github for 2019, demonstrating the C/C++ community's longevity.

Conan is an excellent choice as a package manager. It provides the flexibility developers crave in a developer tool. It uses Python-based package recipes for extensibility, customization and integration with other systems.

It also works on a multitude of systems; including Windows, Linux (Ubuntu, Debian, RedHat, ArchLinux, Raspbian), OSX, FreeBSD, and SunOS. It can target any existing platform, from bare metal to desktop, mobile, embedded, servers, cross-building and works with a range of build systems (Visual Studio MSBuild, CMake, Makefiles, SCons, etc), with extensibility to use any build system. When combined, these aspects of Conan make it an excellent choice as a multi-platform package manager.

Using Conan with Cloudsmith allows development teams to:

Develop packages internally and share them privately with other teams.
Distribute and deploy your packages in a pipeline at your organization.
Distribute packages as commercial software.
Make modifications to public packages, choosing how you wish to republish (open-source, public, private).
Capture the exact state of your dependencies at a particular version, release, user, and channel.
Control (allow list/deny list) at an organization, repository, and package level
In short, all the benefits of using Cloudsmith that are already enjoyed by development teams all over the world today, are now available for Conan.

Getting Started

Getting started with Cloudsmith and Conan couldn't be simpler. First, you'll need a Cloudsmith account and a repository to which you can upload your packages. If you need to install Conan you can find instructions on the Conan website.

Cloudsmith should work with all supported versions of Conan, but we recommend using at least Version 1.25.2 or later for the best experience. You can check your local version like so:

$ conan --version

Conan version 1.25.2

Creating a Conan Package

For the purpose of this demonstration, we will create a Conan package containing a single function that prints "Hello World" using the official example. Running the following example will create a new package called "hello" at version "0.0.1" without the optional user/channel.

The Conan create command is equivalent to running export, install, and test.

$ mkdir mypkg && cd mypkg

$ conan new hello/0.0.1 -t

$ conan create .

The conanfile.py generated as part of the above command will be used to by Conan to build packages however it will also be used by Cloudsmith to retrieve metadata related to a package such as the package name, version, license, etc which can be used for advanced filtering using the UI and Cloudsmith CLI.

If you wish to learn more about how Conan creates the Package Recipe and Test Packages, the official documentation provides a detailed breakdown for each command. You're now ready to upload your package to Cloudsmith.

Uploading your Conan Package

First, you need to add a remote for a specific namespace/repository to the list of Conan remotes. The below example uses Cloudsmith as the namespace but this could be your namespace or one of an organization in which you are a member.

$ conan remote add cloudsmith-testing-public

https://conan.cloudsmith.io/cloudsmith/testing-public/
Once a remote has been added, a user can then be configured using your Cloudsmith username and password in place of the substituted values:

$ conan user -p PASSWORD -r cloudsmith-testing-public USERNAME

Once you're remote and user has been configured within Conan your token will be cached in the client until it expires or becomes invalid. You're now ready to upload your package:

conan upload hello/0.0.1 --all -r cloudsmith-testing-public

Once uploaded, you can view your package in Cloudsmith.

It's that simple to get started with Conan on Cloudsmith

In Conclusion

Cloudsmith provides fully featured Conan package repositories on all plans, flexible enough for use whether you’re hosting public packages for a public or open-source project, or private packages for your company’s internal needs. We're extremely proud to be able to support the C/C++ ecosystem with this tooling.

You can find further, context-specific information, including detailed setup and integration instructions inside each Cloudsmith repository.

Why wait? Get your public and private Conan package repository hosting at Cloudsmith now.

Deploy packages from a Cloudsmith repository with Ansible

Dan McKinney — Tue, 18 Aug 2020 15:31:58 +0000

What is Ansible?

Ansible is an open source continuous configuration automation (CCA) tool. You can use it to automate the management of the configuration of host systems. For example: installing and configuring applications, services, security policies; or to perform a wide variety of other administration and configuration tasks.

You also can use Ansible with a provisioning tool (such as the excellent Terraform, from the awesome folks over at Hashicorp) to automate the entire build and deployment of your infrastructure, and take steps towards true Infrastructure-as-code DevOps practices.

Ansible And Cloudsmith

Ansible is also a perfect partner for your artifacts and assets stored in your private Cloudsmith repositories. As your Cloudsmith repositories can be your single source of truth, with all the access control and permission control that they provide, they give you another secure layer to your infrastructure build and management. By providing you with control of the sources of your packages that you use with your Ansible configuration automation, you insulate yourself further from any upstream changes.

Getting started – The Ansible Playbook.

An Ansible playbook is where you define the series of tasks that you wish to perform to ensure the configuration of your hosts is in the desired state. As an example, we have created an Ansible playbook that will install and configure a Cloudsmith repository for Debian packages, and then install a package from this repository

Here is our example Ansible playbook:

- hosts: DebianHosts
  remote_user: dmckinney

  tasks:
    - name: add Cloudsmith Repository GPG key
      apt_key:
        url: https://dl.cloudsmith.io/QOH0JBRgKQx5lE7S/demo/examples-repo/cfg/gpg/gpg.7D4D4CE49534374A.key
        state: present
      become: yes

    - name: add Cloudsmith Repository
      apt_repository:
        repo: 'deb https://dl.cloudsmith.io/QOH0JBRgKQx5lE7S/demo/examples-repo/deb/debian buster main'
        state: present
        update_cache: yes
      become: yes

    - name: install Cloudsmith Example package
      apt:
        name: cloudsmith-debian-example
        state: present
        update_cache: yes
      become: yes

Let’s break down what this playbook will do.

Hosts:

This is where we specify which of our hosts these tasks will run against. The hosts and their addresses are defined in a separate Ansible Inventory file and for this example, our inventory file is just a single host.

These hosts can be bare metal servers, cloud instances or virtual machines etc. You can define different groups of hosts within a playbook for different tasks and in this way a single playbook can manage a single machine or several hundred or even thousands of hosts.

Remote User:

This is where you specify the user on the host machine that the tasks will run as

Tasks:

This is where we specify the tasks themselves.

The first task is the “add Cloudsmith Repository GPG key” task. This task uses the apt_key ansible module to retrieve the GPG key from our Cloudsmith Repository and install in on our host system. We specify the URL for the GPG key and as this is a private Cloudsmith repository the URL contains an embedded entitlement token to authenticate for read only access:

- name: add Cloudsmith Repository GPG key
      apt_key:
        url: https://dl.cloudsmith.io/QOH0JBRgKQx5lE7S/demo/examples-repo/cfg/gpg/gpg.7D4D4CE49534374A.key
        state: present
      become: yes

We also add the state: present – which means Ansible will install the key if it is not already installed, and if it is already installed it will do nothing. This is important, as it means our task will be idempotent, meaning no matter how many times we run this task the outcome, and therefore the configuration will always be the same. We will see this state: present in all our ansible tasks in this playbook. Finally, as all apt operations on our host system require sudo permissions, we add become: yes which tells ansible to run this task with elevated permissions. Again, we will see this on all tasks in this playbook.

The second task is the “add Cloudsmith Repository” task. This task uses the apt_repository Ansible module to install our Cloudsmith repository on our host system. We specify the URL for the repository, again containing our embedded entitlement token for authentication:

    - name: add Cloudsmith Repository
      apt_repository:
        repo: 'deb https://dl.cloudsmith.io/QOH0JBRgKQx5lE7S/demo/examples-repo/deb/debian buster main'
        state: present
        update_cache: yes
      become: yes

The final task is the “install Cloudsmith Example package” task, and this task uses the apt Ansible module to install a deb package. We just need to specify the name of the package we wish to install:

- name: install Cloudsmith Example package
      apt:
        name: cloudsmith-debian-example
        state: present
        update_cache: yes
      become: yes

And that’s it, that’s all we need in this playbook to get this repository set up and our package installed.

Running the playbook.

To run a playbook, we use the ansible-playbook command:

ansible-playbook -i cloudsmith-demo-inventory cloudsmith-demo-playbook -K

the -K flag tells ansible to ask for our sudo password, but you can also store secrets and password encrypted using Ansible Vault – which means you could check the encrypted vault file into your version control system, alongside your playbooks themselves. But to keep things simple for this example, we will just have ansible ask us for the password:

When we run this playbook, the output was as follows:

You can see that the three tasks ran successfully, and report that they made a total of three changes. If we now check what packages we have installed our Debian host, we can see our cloudsmith-debian-example package:

And finally, if we were to run this playbook a second time (or any subsequent number of times), it reports that all tasks are OK, no changes need to be made:

In Closing

While this is a very simple example, we hope that it demonstrates the possibilities and the power of using a configuration automation tool like Ansible in association with a private Cloudsmith repository. It gives you the ability to automate not only common system configuration tasks, but deployment of your own packages, your own custom configurations and even the software build of your entire infrastructure. You can version these builds in your version control systems, track changes and provide an audit-trail-of-truth all the way back to your packages, stored securely in your own private repositories.

We hope you have very happy continuous configuration automation!