Forem: JFrog

Implementing the JFrog Xray “Summary View” in Slack

Alex Hung — Mon, 15 Nov 2021 17:06:41 +0000

Have you ever wanted to get your engineering teams real-time information about security issues happening during software development? As you may know, JFrog Xray already allows you to scan the entire composition of your binaries and enables you to send alerts to your teams using webhooks, but now with our new Slack integration we make it quite easy for entire channels to be updated in real time. The integration further allows teams to discuss new CVE's with developers on other teams as well. In order to make the JFrog Xray notifications super digestible in Slack, we built a whole new way to view your vulnerabilities and license compliance issues.

How It Works

The integration with Slack uses Jfrog Xray’s security and license compliance policies to trigger webhook events whenever a new violation is detected. Once configured, Xray sends a webhook event to our Slack integration which then transforms each issue in the event payload into UI cards that can be interacted with. In this blog, we’re going to talk about how we implemented one specific feature: the transformation of the payload to provide better usability to the end-user with our “Summary View” card.

Summary View - Why We Transform the Payload

When JFrog Xray scans your binaries and components, it uses a “watch” to tell it which repositories to scan artifacts from. The webhook will trigger a payload of vulnerability data based on how you setup your “policy” and set of rules in Xray. This payload will include every single vulnerability that has been introduced. We realized that when building a notifications app, the usability of this can be daunting. Imagine uploading a new artifact and realizing it has hundreds of vulnerabilities - getting hundreds of notifications in a Slack channel results in a lot of noise. This creates information overload and users can feel overwhelmed by the amount of messages in the channel resulting in them muting or ignoring the channel completely – which defeats the purpose of the information.

That is why we built what we call a “Summary view” of all the issues that come through the Xray payload. Christian Bongiorno (a senior software developer on the JFrog Partner team) created a transformed payload and we want to show you how it works.

Xray Notification Watches and Policies

Before Slack can receive messages from Xray, an Admin needs to assign your repositories inside Artifactory to a watch. This signifies that certain repositories should be monitored. You must also decide what policy and rules to apply that kickoff a notification to Slack. These rules can revolve around the level of severity you want to be notified of (low, medium, high), or if you want to be notified about specific license compliance issues.

Once you have setup policies and watches in Xray, you can then send notifications to Slack channels where your teams are monitoring these events.

How to Create Summary View Notifications in Slack

To create a notification, in the Slack app Home tab, click on the Create Notification button.

Select Xray Violation from the dropdown menu.

In the Watch text box, type in the name of the Xray watch you want to use for this notification. This box will respond to the character you start typing and should show all the Xray watches on your JFrog platform.

Next, select the channel you want the notification sent to.

The next screen will ask you if you want to get notifications by individual CVE or by Summary View.

Getting Notifications by Component - Summary

By default, the format type View by Component (Summary) is selected for you. This format type groups all the issues for an artifact into categories based on severity (High, Medium, Low, Unknown). Each category will include up to 5 violations. To see the full list of issues, you can use the Open in platform button which opens Xray in your browser and takes you to the full list of Xray issues. This view helps your teams understand the extent to which a specific component might be affected by vulnerabilities.

Here is an example of the summary view message:

Getting Notifications by Issue

Additionally, you can also get notification by each individual issue. This view is useful when you already have clean artifacts in production and just want to be notified anytime a new vulnerability pops up.

Here is an example of a individual security violation message:

To avoid flooding the channel, our integration automatically switches to Summary view mode if the webhook event contains more than 40 individual issues. We found that users can digest the summary view much faster when there are more than 40 issues.

How We Built the Transformation Code

As we started making this integration available, we also found that many current JFrog Xray customers wanted to know how we transformed the Xray event data into a ‘Summary view’ card. We’ve made the template code available in the rest of this document.

First, this is what the default Xray payload looks like:

{
  "created": "2021-05-28T19:37:50.075822379Z",
  "top_severity": "Medium",
  "watch_name": "slack_watch_test",
  "policy_name": "slack",
  "issues": [
    {
      "severity": "Medium",
      "type": "security",
      "provider": "JFrog",
      "created": "2021-04-08T04:02:38.999Z",
      "summary": "A flaw was found in the Nosy driver in the Linux kernel. This issue allows a device to be inserted twice into a doubly-linked list, leading to a use-after-free when one of these devices is removed. The highest threat from this vulnerability is to confidentiality, integrity, as well as system availability. Versions before kernel 5.12-rc6 are affected",
      "description": "A flaw was found in the Nosy driver in the Linux kernel. This issue allows a device to be inserted twice into a doubly-linked list, leading to a use-after-free when one of these devices is removed. The highest threat from this vulnerability is to confidentiality, integrity, as well as system availability. Versions before kernel 5.12-rc6 are affected",
      "impacted_artifacts": [
        {
          "name": "manifest.json",
          "display_name": "artifactory-fluentd:1.11.2",
          "path": "default/integrations/artifactory-fluentd/1.11.2/",
          "pkg_type": "Docker",
          "sha256": "10fd87ba58132673ac65ee8c11a01510509f93846bdb5f20300ba5981aa75eb0",
          "sha1": "",
          "depth": 2,
          "parent_sha": "10fd87ba58132673ac65ee8c11a01510509f93846bdb5f20300ba5981aa75eb0",
          "infected_files": [
            {
              "name": "linux-libc-dev:4.19.132-1",
              "path": "",
              "sha256": "391e2df82c21b15e12cd8207d3257baf60b10c824c400e94bb1bd6128c131d55",
              "depth": 0,
              "parent_sha": "c5b1980eb2a26b21e083b2930ec5cae78f473a19d8fc6affbe6b71792fbf6ae2",
              "display_name": "debian:buster:linux-libc-dev:4.19.132-1",
              "pkg_type": "Debian"
            }
          ]
        }
      ],
      "cve": "CVE-2021-3483"
    }
  ]
}

Then, when a Xray webhook event request arrives in our Slack integration, our transformer code extracts only the relevant information we want to use from the payload – then sorts the issues by severity.

const normalize = (violation) => violation.issues
  .map((issue) => issue.impacted_artifacts.map((artifact) => artifact.infected_files.map((file) => ({
    watch_name: violation.watch_name,
    severity: issue.severity,
    type: issue.type,
    pkg_type: artifact.pkg_type,
    summary: issue.summary,
    path: `${artifact.path.replace('default/', '')}`,
    file: file.name || artifact.name,
    description: issue.description,
    id: issue.cve || issue.summary,
  })))).flat(4);

const normalizedViolations = normalize(violation);
const reports = normalizedViolations.sort((a, b) => SEVERITY_MAPPING[a.severity] - SEVERITY_MAPPING[b.severity]);

It then checks if the number of issues is greater than the 40 issue limit, and switches the format to the summary view.

if (messageFormat === ISSUE_MESSAGE_FORMAT && reports.length > SLACK_APP_MAX_ISSUES_PER_ENTRY) {
  messageFormat = SUMMARY_MESSAGE_FORMAT;
  forcedSummaryFormat = true;
}

Afterwards it transforms the data into a Slack UI card using the corresponding format mapper module based on the format type.

const mapper = lookupFormatMapper(messageFormat);
return mapper(reports, jpdOrigin, policyOrWatchName, forcedSummaryFormat)?.map((r) => ({
  format,
  ...r,
}));

In the Slack Integration, we use the Slack Web API to send the message to the targeted channel. We take this transformer code (the examples above) and make it available to the Slack platform. That is how we turn normal Xray webhook events into a “Summary View” card.

Our next goal will be to make the summary view adjustable – giving users more options and ways to build the summary. For now, we’ve made the code available on GitHub so you can also create understand how to create a custom summary from the payload that comes from JFrog Xray webhooks: https://github.com/jfrog/partner-integrations/tree/main/Slack/Sample

Follow the steps in the README.md to try this out for yourself!

To learn more about the JFrog App for Slack, visit us: https://jfrog.com/integration/slack/

Time to make some order with GoCenter

Batel Zohar — Wed, 16 Dec 2020 12:42:46 +0000

Go is becoming one of the world’s fastest-growing software languages. To keep increasing my skill set as a developer I started learning Go a few months ago. Here is a snapshot of my journey and some insights I learned along the way.

Dependency Management

Learning a new language can be overwhelming so I decided to start with the basics - dependency management. So let’s start from the beginning the management of the dependencies, from version 1.11 Go supports modules, this feature makes dependency version information explicit and easier to maintain.

Go module

A module is a collection of Go packages stored in a file with a go.mod file at its root. The go.mod file defines the module’s module path, which is also the import path used for the root directory, and its dependency requirements, which are the other modules needed for a successful build. Each dependency requirement is written as a module path and a specific semantic version.

Let’s start with a simple example: hello world. In this example the go.mod file will look like the following:

module "rsc.io/hello"

require "rsc.io/quote" v1.5.1

After completing a simple go run and go build we now have a hello world example which is basic, but let’s try to make it a bit more complicated by adding yaml support. To do this we will use the following commands (I found that version 2.2.7 is recommended) so let’s give it a go:

gopkg.in/yaml.v2 v2.2.7

Then I figured that I used a vulnerable package and I found GoCenterthat provided me an amazing way to better understand Go packages. GoCenter has the following features:

Proxy my dependencies

First we can use GoCenter as a GOPROXY and we will redirect all module download requests to GoCenter which can be faster than directly from the VCS.

To change the GoProxy path just use the following commands:

For mac and linux:

export GOPROXY=https://gocenter.io

For Windows:

'''set GOPROXY=https://gocenter.io'''

For powershell:

'''$env:GOPROXY=https://gocenter.io'''

Protect your binaries

I’ve tried to learn a bit more about the yaml packages and this is how it looks on GoCenter:

First I found out that my version is vulnerable and contains CVE-2019-11254 like the following:

Also I noticed the feature that scans the dependencies in a go.mod file held by GoCenter and identifies every vulnerability. Under the dependencies tab we will get the detailed information about vulnerable components at every level of the dependency tree, once we will click on the orange triangle we will forward to the package and we can check the vulnerability page like the following example of

Learn more about your packages

So I clicked on the versions tab and saw that version 2.2.8 contains a fix and I upgraded to the latest version 2.4.0 now seems like they added some documentation and examples:

I love metrics. GoCenter’s metrics are colorful and provide a lot of information in a great visual way so I can easily see that there are a lot of downloads of the packages and 37 Contributors:

Advanced mode private GOPROXY

Another advantage for developers is the ability to improve our resolution tie by integrating our JFrog Artifactory server and create our Go private repository. We want to create a private Go repository to make sure that we are pulling directly from a virtual repository that contains a remote repository that points to GoCenter and our local repository with our project. A benefit of this method is that we don’t need to manage Artifactory we can just use the SaaS version which is free and limited.

Conclusion

To sum it all up, as I learn to write in Go I will continue to use GoCenter as a proxy for my dependencies, vulnerability scanning of my binaries, version control of my packages, beautiful metrics to give me a great visualization of the data

DevOps 101: Package Management

Kat Cosgrove — Wed, 25 Nov 2020 16:14:39 +0000

When you’re new to an industry, you encounter a lot of new concepts. This can make it really difficult to get your feet underneath you on an unfamiliar landscape, especially for junior engineers. What’s all this jargon? What does DevOps really mean? What’s all this software? Is DevOps a methodology, or a toolset? Is any of this actually going to make my life easier, or is it just a bunch of industry buzzwords? A lot of the documentation out there assumes you already have additional context and experience, or are proficient in some related tooling, and that doesn’t exactly make it easy to learn. DevOps has a ton of jargon in it, though. We're absolutely swimming in abbreviations and abstractions, and sometimes it's difficult to define a term satisfactorily without needing to define three more for context. It’s like running into a brick wall.

Here, I'll explain package managers.

What even is that?

From Wikipedia, a package manager or package-management system is a collection of software tools that automates the process of installing, upgrading, configuring, and removing computer programs for a computer's operating system in a consistent manner.

If you’re a developer, you have probably already used one of these. For instance, if you’re writing Node.js, you’re probably using NPM. That’s a package manager. If you’re using Linux and you’ve ever installed something with apt-get, APT is a package manager. In plain English, a package manager is what handles the heavy lifting involved in installing something (including that thing’s dependencies), validating that it is what it says it is (to an extent, by comparing checksums), keeping track of versions, upgrades, and uninstalling. This is accomplished in part through the use of a large amount of metadata, defining characteristics about a particular package, alongside the actual application binary. Modern package managers are why you can reliably run pip install Django=-1.3.3 or whatever and be pretty sure you are getting version 1.3.3 of Django, or just pip install Django and take whatever is the most recent version.

The Before Times

We didn’t always have this, though. The earliest versions of what you could call a package manager are from the mid to late 90s, with Debian introducing dpkg in 1994 and RedHat’s RPM in 1997. Before then, and until package management really took off, we had to do a lot of things manually and we had way less information about the things we were installing.

You would get a compressed directory, usually in the form of a .tar file. Decompress that, and you would find a readme file with instructions to follow. Some kind of config script would be there which, when run, would tell your C compiler what to do, where new binaries should go, what application dependencies to look for and where, etc. If anything went wrong, it would exit and you would have to go install some more dependencies. If it found everything it needed and the config script completed, it would spit out a Makefile. Run the make command and, if everything compiles correctly, run make install to finally install the application. Updates were just as involved, if not more so. Obviously, this is a time-consuming process. Imagine having to do that for every piece of software on your computer, and every piece of software required to run all of it. A lot of things could go wrong on your journey from downloading a .tar file to actually getting the thing installed.

Understandably, everyone thought this was exhausting and the release of the first package managers for Linux was A Big Deal. It immediately changed computing and application development for the better, forever. Those early package managers pretty much just gave you install, update, and uninstall, but over time, package managers have bundled all of that work and more into simpler commands and encoded extra information alongside the application binary and its dependencies, like version numbers, checksums, dependency graphs, and more. As the popularity of package managers for Linux grew, so did the demand for something similar for other languages. Thus, a whole series of package managers for other languages were born, from CPAN for Perl to PyPi for Python to Cargo for Rust, all with the goal of making it easier to distribute and use software.

This introduced a whole new problem, though. Applications are easier to install, update, manage, and remove, so we start building more complex applications, and with Continuous Integration becoming popular in the 90s, we start releasing more often, too. One organization might be using multiple languages, and we also start caring about security. This is where binary repository managers enter the playing field.

What’s a binary repository manager?

A binary repository manager is there to manage all of those binaries we now have into a system of repositories. Some of them support just one or two package types for very specialized applications, but the one I’ll talk about supports more. This isn’t the same thing as source control, which is where your code lives, but more of an extension of it. While your code might live in a repository on GitHub or BitBucket or whatever, the result of that code being built or compiled -- your artifacts -- live in a binary repository manager like JFrog Artifactory. For DevOps to really work, this is an important part of the equation. We’re delivering updates far more often now, and we need a way to organize our build artifacts in a sensible way so they’re easier for our other tools, like our CI/CD system, to interact with them and ultimately deploy the updates to the user. Without one, it’s much more difficult to track version numbers, control access, promote builds from testing to production, collect metadata, or detect security problems.

Try to imagine writing code and keeping it organized without tools like Git and GitHub. That sounds miserable, right? It’s the same for binaries and a binary repository manager, especially on teams that have multiple languages in one house. If you use a binary repository manager like Artifactory, you can store your Go, JavaScript, Docker images, Python, and Java binaries all in one tool, plus 22 other package types.

Artifactory breaks up the repositories for each package type into three different classes: Local, remote, and virtual. Local repositories are what they sound like: repositories for your build artifacts resulting from local code that exists on your machine. Remote repositories are also fairly self-explanatory; they contain remote build artifacts, like your project's dependencies. This functions sort of like a cache, so that after the first download, your project pulls its dependencies from the associated remote repository rather than from NPM or PyPi or whatever. If you are using Docker, this is particularly helpful, since it limits the number of pulls you need to make against Docker Hub and you won't hit their new limit for pulls from anonymous users as quickly. Virtual repositories are a little weirder -- they create a kind of envelope around the local and remote repositories for your project, and this is what you'll be interacting with most frequently.

A lot of headaches are saved here, from an organizational standpoint. Things get released faster because things are more organized and easier to integrate with a CI/CD solution, and there’s no jumping around between a dozen tools that all do the same thing but for different package types. This improvement alone decreases the likelihood of a bad build making it out into the wild, because we humans are really bad at repetitive tasks, and this takes over a lot of repetition for us.

Cool cool cool, how do I try using one of these things? Sounds enterprisey.

It's definitely a thing that's more beneficial to whole companies or dev teams than for an individual person on a side project, but it's still good to learn. There are a handful available, but Artifactory is free up to a certain amount of storage and data transfer. Try it here on the cloud provider of your choice. It also comes with Xray, a vulnerability detection tool, so throw in a CI/CD system and you're getting pretty close to an end-to-end DevOps solution to play with and learn on. If you don't quite understand CI/CD (or you just need some recommendations for where to start!), check out the previous article in this series: DevOps 101: CI/CD

Summarize this for me, high school essay style.

In conclusion, package managers are a set of tools that make it easier for you to install, use, update, and remove applications. They go further than just automating the steps we used to have to take manually, with config scripts and Makefiles, by also installing your dependencies and managing a bunch of extra metadata we didn’t have clear access to before. The next leap from there is the use of a binary repository as an extension of our source code repositories, to manage all of these binaries and build artifacts produced by our package managers. Doing so gives us more insight into what’s going on with our builds, a simpler way to control who has access to what, and a place to keep all of our build artifacts regardless of the languages or tools involved in our applications. This is a boon to your developers from a sanity and organization standpoint, to your users in the form of faster updates, and to your legal team in the form of faster detection of critical vulnerabilities. The invention of the package manager is possibly one of the most important innovations in computing in decades, and a universal binary repository manager is one of the most important parts of a functional DevOps pipeline.

I hope I’ve helped you understand what package management is and what it does for you. If you’re still confused, that’s okay too -- there's a lot going on in this space. Stay tuned for more!

Oh, and if you have specific requests, reach out to me on Twitter! My DMs are open.

De(v)lightful continuous benchmarks with Go

Omer Karjevsky — Thu, 05 Nov 2020 10:41:14 +0000

The following article is not going to tell you about the importance of benchmarking your Go applications.

Even if we understand the importance, many times benchmark tests are ignored when developing new application features, or when improving existing ones. This can be a result of many reasons, but chiefly it is a result of the hassle required to perform a meaningful benchmark.

Here, we will describe a method for making benchmarking an easy, low-friction step in the feature development lifecycle.

What we need

First, let’s describe the basic requirements when benchmarking an application feature:

The benchmark result should tell us something meaningful
The benchmark result should be reliable and reproducible
Benchmark tests should be isolated from one another

The benchmark tooling available in the Go testing package is a useful and accepted method to meet the aforementioned requirements, but in order to have the best workflow, we want to include additional nice-to-have ones as well:

The benchmark test should be (very) easy to write
The benchmark suite should run as fast as possible, so it can be integrated into CI
The benchmark results should be actionable

Easy, right?

At first glance, all of these requirements should also be covered by the existing tooling.
Looking at the example from Dave Cheney’s great blog post, writing a benchmark for Fib(int) function should be very easy and duplicated code between this test and another would be negligible.

func BenchmarkFib10(b *testing.B) {
    // run the Fib function b.N times
    for n := 0; n < b.N; n++ {
        Fib(10)
    }
}

Not In the real world

For the purpose of this article, let’s take a look at a more challenging use case. This time, we want to benchmark a function which performs a SELECT statement on an SQL database.
The stateful nature of this flow requires us to perform a setup step before running the actual test.

What would such a test look like? Probably something similar to this:

func Benchmark_dao_FindByName(b *testing.B) {
    ctx, ctxCancel := context.WithCancel(context.Background())
    defer ctxCancel()

    // when working with multiple DB types, or randomized ports, 
    // this can involve some logic, for example, 
    // resolving the connection info from env var / config.
    dbDriver := resolveDatabaseDriver()
    dbUrl := resolveDatabaseURL()

    db, err := sql.Open(driver, dbUrl)
    require.NoError(b, err)

    dao := domain.NewDAO(db)

    // populate the db with enough entries for the test to be meaningful
    populateDatabaseWithTestData(b, dao)

    // ignore setup time from benchmark
    b.ResetTimer()

    // run the FindByName function b.N times
    for n := 0; n < b.N; n++ {
        _, err = dao.FindByName(ctx, "last_inserted_name")
        require.NoError(b, err)
    }
}

Yikes, even with the most simplistic example, look at all that setup code. Now imagine writing that for each new benchmark test in a real application codebase!

Another thing to note here, is that if we would have forgotten (yes, mistakes happen) to reset the test timer where required, or running the test inside the required loop, the test would become invalid without any clear indication.

So, what can we do?

At JFrog, we want processes to be as pain-free as possible, so we can shift-left responsibilities to the dev teams without having it affect productivity and development times too much. An example of this mindset is writing benchmark tests as part of the application test suite, instead of being part of the end-to-end suite or the QA pipeline.

So, looking at the example above, we see that the experience of writing new benchmark tests would be painful for our developers - we need to fix that. Let’s look at how we can reduce the setup code as much as possible.

1. Wrap application setup and benchmark state

Most of the setup logic from the example above can be deferred to a wrapping utility.
Resolving the DB connection info, and creating the application object, so that it can be used in the test

type BenchSpec struct {
    Name string
    // Runs b.N times, after the benchmark timer is reset.
    // The provided context.Context and container.AppContainer 
    // are closed during b.Cleanup().
    Test func(t testing.TB, ctx context.Context, app myapp.Application)
}

func runBenchmark(b *testing.B, spec BenchSpec) {
    b.Helper()

    ctx, ctxCancel := context.WithCancel(context.Background())
    b.Cleanup(ctxCancel)

    // when working with multiple DB types, or randomized ports, 
    // this can involve some logic, for example, 
    // resolving the connection info from env var / config.
    dbDriver := resolveDatabaseDriver()
    dbUrl := resolveDatabaseURL()

    db, err := sql.Open(driver, dbUrl)
    require.NoError(b, err)

    dao := domain.NewDAO(db)
    // did you try https://github.com/google/wire yet? ;)
    app := myapp.NewApplication(dao)

    // populate the db with enough entries for the test to be meaningful
    populateDatabaseWithTestData(b, app)

    b.Run(spec.Name, func(b *testing.B) {
        for n := 0; n < b.N; n++ {
            spec.Test(b, ctx, app)
        }
    }
}

func Benchmark_dao_FindByName(b *testing.B) {
    runBenchmark(b, BenchSpec{
        Name: "case #1",
        Test: func(t testing.TB, ctx context.Context, app myapp.Application) {
            _, err = app.Dao.FindByName(ctx, "last_inserted_name")
            require.NoError(t, err)
        }
    })
}

Now, we can see the actual test code is trivial, all we need to do is pass the method we want to benchmark into the runBenchmark helper function. This function handles all the heavy lifting for us.

It also keeps us from making mistakes, note that the BenchSpec.Test argument t is scoped down to testing.TB instead of giving access to the full *testing.B object.

The wrapper utility can, of course, be extended to accept multiple test specifications at once, so that we can run multiple tests against a single database/application initialization.

2. Use pre-populated DB docker images

Populating the database in each test takes too much time. Using a pre-populated docker image will allow us to perform our tests against a well-known DB state, without having to wait.

As an added bonus, since the DB container does not require initialization actions in most cases, that means we can use an image that will start up much faster.

Actionability

Now that we have an easy way to add benchmarks for our new features, what do we do with them? The output will look something like this:

% go test -run="^$" -bench="." ./... -count=5 -benchmem

PASS
Benchmark_dao_FindByName   242    4331132 ns/op     231763 B/op      4447 allocs/op

ok      jfrog.com/omerk/benchleft       3.084s

Simply looking at the output and deciding if performance is good enough is nice, but can we get more actionable results from our benchmark suite?

When working on a change to our application, we rely on our CI to make sure we don’t break anything. Why not use the same flow to make sure we don’t degrade our application performance?

So, let’s run the benchmark suite as part of our CI. But getting the results on our working branch is not enough, we also need to compare these results to a stable state. In order to do that, we can run the benchmark suite on both our working branch and a release branch or the main branch of our git repository.

By using a tool like benchstat, we can compare the results of two separate benchmark runs. The output of such a comparison looks along the lines of:

% benchstat "stable.txt" "head.txt"

name                                                                   old time/op    new time/op    delta
pkg:jfrog.com/omerk/benchleft goos:darwin goarch:amd64 _dao_FindByName 4.04ms ± 8%    5.43ms ±40%  +34.61%  (p=0.008 n=5+5)

name                                                                   old time/op    new time/op    delta
pkg:jfrog.com/omerk/benchleft goos:darwin goarch:amd64 _dao_FindByName 10.9kB ± 0%    10.9kB ± 0%   +0.19%  (p=0.032 n=5+5)

name                                                                   old time/op    new time/op    delta
pkg:jfrog.com/omerk/benchleft goos:darwin goarch:amd64 _dao_FindByName 8.60k ± 0%     8.60k ± 0%   +0.01%  (p=0.016 n=4+5)

As can be seen above, we get a clear understanding of the performance differences (if any) for time, memory and number of allocations per operation.

Storing the stable benchmark results in a bucket or artifact repository (such as JFrog Artifactory) is one option, but to keep performance variance to a minimum, we would prefer to run the suite on both branches on the current CI agent if possible (using the tips above to keep our tests fast is crucial).

Here is an example of a (simplistic) bash script that will help us get actionable results from our benchmark suite:

#!/usr/bin/env bash
set -e -o pipefail

BENCH_OUTPUT_DIR="${BENCH_OUTPUT_DIR:-out}"
DEGRADATION_THRESHOLD="${DEGRADATION_THRESHOLD:-5.00}"
MAX_DEGRADATION=0
THRESHOLD_REACHED=0

function doBench() {
   local outFile="$1"
   go test -run="^$" -bench="." ./... -count=5 -benchmem | tee "${outFile}"
}

function calcBench() {
   local metricName="$1"
   MAX_DEGRADATION=$(cat "${BENCH_OUTPUT_DIR}/result.txt" | grep -A 2 "${metricName}" | head -n 3 | tail -n 1 | awk 'match($0,/(\+[0-9]+\.[0-9]+%)/) {print substr($0,RSTART,RLENGTH)}' | tr -d "+%")
   if [[ "${MAX_DEGRADATION}" == "" ]]; then
       echo "Benchmark ${metricName} - no degradation"
       return
   fi
   if [[ "${THRESHOLD_REACHED}" == "0" ]]; then
       THRESHOLD_REACHED=$(echo "${MAX_DEGRADATION} > ${DEGRADATION_THRESHOLD}" | bc -l)
   fi
   echo "Benchmark ${metricName} degradation: ${MAX_DEGRADATION}% | threshold: ${DEGRADATION_THRESHOLD}%"
}

mkdir -p "${BENCH_OUTPUT_DIR}"

git checkout stablebranch && git pull
doBench "${BENCH_OUTPUT_DIR}/stable.txt" || {git checkout - && exit 1; }

git checkout -
doBench "${BENCH_OUTPUT_DIR}/head.txt"

benchstat -sort delta "${BENCH_OUTPUT_DIR}/stable.txt" "${BENCH_OUTPUT_DIR}/head.txt" | tee "${BENCH_OUTPUT_DIR}/result.txt"

calcBench "time/op"
calcBench "alloc/op"
calcBench "allocs/op"

exit "${THRESHOLD_REACHED}"

doBench() will run the full benchmark suite and output the results into the specified file. calcBench() takes the comparison result output and calculates if our specified degradation threshold has been reached.
benchstat is used with the -sort delta flag, so we take the highest delta into our calculation.

The script performs the benchmarks against “stablebranch” and then against our working branch, compares the two results, and calculates against the threshold.

The exit code can be used to fail our CI pipeline in case of a large enough performance degradation.

Further Considerations

It’s important to remember that your CI pipeline (such as JFrog Pipelines) is a tool meant to increase productivity and observability. Adding the benchmark step can be very helpful but does not entirely alleviate the developer’s responsibility.

When following the examples above, you may notice that newly added benchmark tests are not considered when automating the degradation calculation, as there is no stable result to compare against. For this case, developer discretion is required to decide on a good base value.

There will also inevitably be cases where changes would cause expected and acceptable performance degradation. The CI pipeline should support these cases.

Lastly, regarding database-dependent benchmarks, it would be best to keep the tests isolated from the specific data stored in your snapshots - such tests should be based on the amount of data, and not the contents. This would allow updating the database snapshot without having to worry about breaking previously written tests.

Conclusion

Utilizing this methodology, we can reduce the cognitive load of writing benchmark tests, as well as reducing the time it takes, which ultimately allows us to include them as part of our feature development lifecycle with very little added effort. On top of that, having actionable benchmark results as part of our CI pipelines allows us to handle any issues much faster in the development lifecycle.

Taking full ownership and responsibility of feature delivery becomes easier, letting our developers focus on development instead of verification.

Interested in working on high-performance products with team members who care about improving workflows while keeping high standards? Come join us at JFrog R&D!

Docker new Download Rate Limits

Batel Zohar — Fri, 23 Oct 2020 10:31:44 +0000

The new Docker announcement could be a bit confusing, but in this blog post, I’ll try to summarize it and make it simpler to understand. On November 1st, Docker is planning to add a new subscription level, and here’s how this may affect us.

The Problem

There are two main issues Docker users will now be facing: new pull request limitations, and the image retention policy.

Pull request limitations

This new limitation means that we can create 100 pulls for anonymous users and 200 pulls for authorized users every 6 hours.

To better understand why rate limits were introduced, Docker found that most Docker users pulled images at a rate you would expect for normal workflows. However, there is an outsized impact from a small number of anonymous users. For example, roughly 30% of all downloads on Hub come from only 1% of our anonymous users.

The challenge is when pulling an existing image. Even if you don’t download the layers, this pull request will still be counted.

From Scaling Docker to Serve Millions More Developers: Network Egress

Image retention policy

Images stored in free Docker Hub repositories that have not had their manifest pushed or pulled in the last 6 months, will be removed at the mid of 2021. This policy does not apply to images stored by paid Docker Hub subscription accounts, Docker verified publishers or official Docker Images.

Let's take the following example of a free subscription user who pushed a tagged image called "batelt/froggy:v1" to Docker Hub on Oct 21, 2019. If this tagged image was never pulled since it was pushed, it will be considered inactive by mid 2021 when the new policy takes effect. The image and any tag pointing to it will be subject to deletion.

According to their wiki, Docker will also be providing tooling, in the form of a UI and APIs, that will allow users to easily manage their images.

The Solution

My favorite solution is to use JFrog Artifactory which is an artifact repository manager. This will allow us to store and protect our Docker images within a private Docker registry, keeping them in our cache using a remote repository,, reducing our requests to Docker hub and using our local images kept in cache as shown in the following diagram.

Why should we use Artifactory and how it works

JFrog provides us with the ability to host our own secure private Docker registries and proxy external Docker registries. It even provides us with a smart checksum-based storage.
storage, which utilizes storage to maximum potential.

Choose Artifactory version (Cloud or On-prem)
Create Docker repositories (local remote and virtual)
Configure repository advanced configuration

Artifactory version

If you don’t want to manage Artifactory you can just use the SaaS version which is free and limited. Or you can use the JFrog container registry that supports Docker and Helm repositories

Create a Docker repository

Configure repository advanced configuration

Now we can easily Configure repository advanced options like deciding how long before Artifactory checks for a newer version of a requested artifact in a remote repository and use our local caching so we will save our docker requests:

Learn more about Artifactory Pro that contains 27 different package types like Maven NPM and much more.

How to use Helm to Support Cloud-Native Development

Batel Zohar — Mon, 21 Sep 2020 14:55:42 +0000

One day my boss informed me that we are moving towards developing our software on the cloud. As a developer, this is one of my pet peeves as I hate spending time setting up complicated environments and prefer to focus on writing code and quickly releasing their software. And then I immediately got my first assignment to create a new software product using the CI/CD pipeline on the cloud. The questions that immediately arose were: When? How? And Why do we need it?

Since then I have learned many lessons and would like to share them with you on how to prepare and configure your environment to achieve DevOps automation in the cloud for your containerized secured applications using a CI/CD pipeline in Kubernetes.

Why go to the cloud?

Let’s first look at the main reasons for going to the cloud:

Achieve unlimited scalability:

Setting up your environment on the cloud provides unlimited scalability allowing you to grow according to your needs and is easily achieved by using cloud storage providers (Amazon S3, Google GCS or Microsoft Azure) in your environment.

Hardware costs are no longer an issue:

A huge advantage of going to cloud computing is the decrease in hardware costs whereby you pay only for exactly what you use. Instead of purchasing in-house equipment, hardware needs are now left to the cloud vendor. Adding new hardware every release or every quarter to meet your increasing needs can be very expensive and inconvenient, on the other hand, cloud computing alleviates these issues because resources can be acquired quickly and easily.

Gain redundancy:

And the most important thing you gain is redundancy. A number of cloud storage providers can ensure your data is stored on multiple machines and complies with any regulations that need to be applied (like keeping two copies of all the data in two different regions).

This blog post shows you how to prepare and configure a cloud-based environment to achieve an automated CI/CD pipeline for your containerized secured applications using Kubernetes.

So let’s get started.

Configuring a cloud-based automated CI/CD pipeline

Before you start, prepare the following:

A version control system (such as Git)
Kubernetes to deploy and manage your containerized applications
A CI/CD pipeline tool like Jenkins, TeamCity, Bamboo etc.
A Binary repository manager
A Compliance auditor

Step 1: Setting up your cloud environment

Let's start by creating a Kubernetes cluster. You can easily choose one of the available Kubernetes platforms in the cloud. Then isolate your environment by creating separate environments for development, staging, and production as displayed in the following diagram:

We recommend applying the "shift left” principle, whereby you move back tasks typically performed at later stages to earlier stages in the pipeline. For example, when running security testing or selecting the FOSS license. As mentioned earlier, the aim is to move faster to reduce delivery time while improving the quality of each release. At the same time, you are also faced with increasing pressure to reduce testing, so it means that the developer’s team needs to be integrated with the testing cycle earlier.

Step 2: Configuring the VCS server

When you finish developing the app, proceed to configure the VCS server VCS server in order to trigger the build after a successful pull request.

Step 3: Running the build

Run the build on your CI server (for example Jenkins), to create the application’s Docker image and proceed to run the unit test against the Docker container.

Step 4: Testing the build

After the builds tested, create a Docker image that will be uploaded to your private Docker registry and private helm repository, then run a number of tests against the running Docker container including integration tests.

Step 5: Scanning the build

Approximately three decades ago, Richard Stallman changed the developer’s world forever, by introducing the GNU Project, the first open-source coding project that included requirements for scanning external code. In that case, I recommend using the ChartCenter as our new source for our helm chart, when checking the chart view, in that case let’s talk about our DB we can easily get the chart information for the dependencies like the following:

Afterward, I can check the security report very easily and calculate the risk on this specific version

Step 6: Deploying to development

Now we can proceed to deploy your app to the staging environment and perform additional tests against this cluster.

Add the Chartcenter Helm repository

helm repo add center https://repo.chartcenter.io

Step 7: Deploying to staging

After running a full test cycle in the development environment, deploy the application to the isolated Kubernetes staging cluster, run the staging tests, and proceed to the next step.

Step 8: Deploying to production

Run a set of sanity tests and deploy them to the isolated production cluster. Be ready to perform a fast rollback, if necessary.

It’s time to have some fun!

So you see it ain’t that scary and the steps are out there, so go ahead and develop your next app in the cloud. To make it even easier, follow our 6-step CI/CD pipeline for a simple static website application based on of official nginx Docker image.

How Everyone Can Win in the New Era of Docker Hub Limits

Melissa McKay — Wed, 26 Aug 2020 16:34:58 +0000

AN OBSERVATION OF HUMAN BEHAVIOR

“Don't take anything for granted, because tomorrow is not promised to any of us.”
- Kirby Puckett

There is a behavior I’ve observed repeatedly over the years that I’m certain boils down to one of the basics of human psychology. The crux of it is this: when something is readily available to us in vast quantities, the less we appreciate said thing. Put a slightly different way, we tend to be less concerned with how much we use of something when we have plenty to spare. It’s an interesting phenomenon to watch play out, and I see it every single time I open a brand new tube of toothpaste. It’s the same pattern every time - the first glob out of the tube looks remarkably like the marketing images, full coverage in the nice shape of a wave. But as I get to the end of the tube, I become predictably stingy. It turns out that the small dabs of toothpaste that I use after obsessively smoothing out and rolling up the tube is all I really need, and yet, somehow a brand new tube equates to a full, beautiful wavy glob. Come to think of it, pretty much anything I use that comes in a tube or other similar container has the same fate.

Another real-life example of this behavior I regularly struggle with is how much ice cream I have access to. There is a direct relationship between my dieting success and the amount of ice cream in my freezer. I’m not saying ice cream is evil (it is), it’s the availability of too much ice cream that results in my repeated failures. My lack of self-control in this situation is a completely separate matter that I don’t wish to discuss.

You might not immediately relate to the toothpaste or ice cream scenarios (are you even human?), but there is a fairly long list of essential things we have all taken for granted at one point or another - the availability of running water in your home, the ease of flipping a switch to read in the evening, cool and breathable air! Of course, all of this is in varying degrees depending on our history and current access to these things. But that is exactly the point I’m making. We intrinsically know that these resources greatly improve our well-being and are of utmost importance (some essential to life!), and yet until we are faced with some kind of limiting factor, it’s difficult for us to appreciate them in the way we should.

To be clear, none of this is meant to shame or guilt anyone. This is all just an observation of something that is completely natural and probably even beneficial to us as human beings. If we spent our days worrying about everything that is essential to us and how our lives would be without them, we would be nothing but shriveling heaps of tears and angst at the end of the day. Living our lives is very much like spinning plates - it’s the wobbly plate that gets our immediate attention. The management of our resources is very much related to the quantity available to us and we are left to figure out how to deal with whatever crisis is at hand when we hit unexpected limits. We re-evaluate our needs and then we find clever solutions to subsist on what we have. And round and round we go.

This brings me to our current wobbly plate in our DevOps world.

Given the title of this article, you can probably tell where I’m going with this. Let’s set aside deep discussions of human behavior and life on this planet for another time, and instead, let’s figure out how to apply what we’ve learned from our observations so far to the latest happenings in DevOps tooling and resources.

WHAT IS THE PROBLEM? WHY IS THIS COMING UP NOW?

“For want is nexte to waste, and shame doeth synne ensue,” (waste not, want not)
- Richard Edwards

Docker Hub recently updated their terms of service, (see section 2.5), for their free service level accounts to include a rate limit (a limit on the number of pushes and pulls of Docker images over a span of six hours), as well as a retention policy for inactive images (images that are not pushed or pulled for the last six months are deleted). If you can imagine, these changes have lit quite the firestorm of discussion on social media. Everyone relying on these services is having to come to terms with these new limitations to be sure that their pipelines will not be adversely affected.

Let’s break this down.

Prior to these changes, developers and full-scale CI/CD systems were able to push and pull Docker images from Docker Hub without any limitations. On top of that, free storage! This is a pretty incredible service and frankly very easy to take advantage of. You know how when you have more storage, you store more things. This behavior permeates my own life across the board. My digital photo album is an excellent example. My house is another example. I moved from an apartment to a home and I magically have more stuff! Like goldfish, we tend to fill the space we’re in and then forget what we have.* Again, this is just a natural human behavior. But the moment that storage is assigned a price, (or a retention policy in the case of Docker images stored in Docker Hub), we now must take a step back and figure out how to manage our storage a little more thoughtfully. We must clean out our closets, so to speak.

The new limits imposed by Docker Hub is a bit of a call to action to define some netiquette around the use of these free services. This is both a jolt to re-evaluate and consider our use of the resources affected as well as an opportunity to save ourselves from some of the negative consequences of taking these high-valued resources for granted. For those DevOps professionals out there that are already following best practices, this announcement from Docker is far from a deal-breaker for the use of Docker Hub and will certainly not result in their software development and distribution pipelines grinding to a halt. We’ll talk in the next section about what those best practices are, but first, let’s discuss the real elephant in the room, and perhaps the real fear that the Docker terms of service update has unveiled.

There seems to be an unhealthy reliance on external resources when it comes to critical internal operations. Specifically, if my team of developers cannot access a Docker image when required in their personal development environments (and requests from developers could be multiple times a day depending on the circumstances), their progress on the next feature or bug fix is potentially blocked. In the same way, if my CI/CD system that is responsible for building my next software release cannot access the binaries it needs, my team may end up in a position where they cannot release. The same can be said for every intermediary step of the pipeline including initial integration and deployment to quality assurance test environments. By taking for granted the access to and storage of the most integral building blocks of our software, our software binaries, many find themselves completely at the mercy of an external service.

Docker Hub is not the only organization out there whose free service level offering is subject to limitations. It is not an uncommon occurrence that near the end of the month, Boost, (one of the most popular library projects in C++), reaches a point where the distributable is no longer accessible because the organization’s monthly download allowance has been exceeded. Docker and Boost have intentional limitations set. Some services will degrade or encounter downtime when demand is too high or because of any number of other reasons. For example, NuGet Gallery, the central repository for .NET packages, provides a status page to let stakeholders know what is going on when there is an outage. The most unfortunate scenario which has more to do with uncontrolled risk rather than free service limits is when a remote binary that your build relies upon just up and disappears, like what happened during the infamous NPM left-pad debacle of 2016. All of these examples call attention to the problems and potential productivity killers that teams face when relying on remote resources for software binaries. Another important point to make here... this is not a new problem!

HOW DO I DEAL? HOW DO I MITIGATE RISK FOR MY DEVOPS TEAM?

“There is a store of great value in the house of the wise, but it is wasted by the foolish man.”
- Proverbs 21:20 (BBE)

So now that I have a full understanding of the real value of my Docker images and other binaries and can better evaluate the methods I use to store and retrieve them, what can I do to help keep my builds and my whole software development pipeline alive and drama free? Obviously, taking the stance of never using free services like Docker Hub is unacceptable as this will put you at a disadvantage. These services are valuable and certainly have their place. But 100% reliance on them is clearly unhealthy. Expecting them to meet an unbounded need is also unrealistic.

Step 1: Take an inventory of your software project.
It’s important to know exactly what libraries and packages your software project is pulling in. Understand exactly where your binaries are coming from. For Docker images, make sure you understand thoroughly what is happening when you build your images. For example, are there any lines in your Dockerfile that pull from npm, perform a pip install, or update other software packages? All of these actions potentially reach out to remote service providers and will count against any download limits.

Step 2: Utilize multiple levels of caching.
Given that many remote offerings like Docker Hub, Boost, npm, NuGet Gallery, and many others have very real limitations and possibly unplanned downtime, it’s important to mitigate both the risk of not being able to access your binaries when needed as well as eliminate unnecessary polling for these resources. One of the most valuable things you can do is set up a caching proxy like JFrog's Artifactory, (a remote repository), for these remote resources. The next level of cache that will play an important role is a developer’s local environment. Developers should be set up to pull required resources from the caching proxy rather than repeatedly from the remote service.

Step 3: Modify CI/CD pipelines to pull from cache.
Even if your CI/CD processes involve building your code from scratch on brand-new, temporary instances, set them up to pull from your proxy setup in Step 2 rather than repeatedly pulling from remote sources. A misbehaving pipeline can easily meet up with throttling and other download limitations if left unchecked. It is better for your CI/CD pipelines to utilize internal resources that you have control over rather than leave them attempting to pull from remote sources that may be unavailable. If you set up your pipelines this way, you will be more empowered to troubleshoot and resolve any internal issues you experience in order to complete your pipeline processes successfully rather than be relegated to the priority queue of an external service.

I expect nothing less than a ton of buzz and discussion about this move by Docker and even the thoughts I’ve written here. This is a good thing. My hope is that this move will bring to light the realities of providing a service that so many in the industry have come to rely on and ultimately what it means to be a responsible user of community resources. I also hope that we come to fully appreciate the costs associated with access to and storage of our most valuable software building blocks - that we are more thoughtful about where we put them and how we get to them since they are fundamental to our organization’s software.

* This is actually an entirely untrue statement about goldfish, but you get my meaning. Blogs like this one perpetuate these falsehoods, so here is a resource to hopefully make up for it: https://www.tfhmagazine.com/articles/freshwater/goldfish-myths-debunked

Helm V3, Latest & Greatest of Kubernetes

shimib — Wed, 19 Aug 2020 17:53:39 +0000

Helm is becoming the de facto standard for managing Kubernetes deployments.
Although not the only tool in the landscape, it’s by far more popular than the alternatives.
The reason for using Helm is quite obvious: managing your K8S deployments by hand requires a lot of YAML manipulation which usually leads to high maintenance and duplication.

Recently, Helm v3 was released and I wanted to describe the changes and new features in detail.

Removal of Tiller

If you have worked with previous versions of Helm, one of the mandatory installation components was the Tiller Helm server that needed to be installed in your K8s cluster.
You might have asked, why is it needed? Can’t all the operations be performed from the client side?
Well, when Helm v2 was released in 2016, some of the K8s features that we are now used to (e.g., Custom Resource Definitions (CRDs)) weren’t available yet.
These days, there is really no need for Tiller.
In version 3, Tiller is no more ☺

With the removal of Tiller, there is no centralized namespace where all Releases’ information is stored (the namespace where the Tiller was installed). Now this information is stored in the namespace of the chart itself.
Your releases are now under their own namespace (yes you have to create the namespace).

Security is also now handled where it should, i.e., by K8s RBAC.

XDG-based Directory Structure

Starting with Helm v3, directory structure and its configuration are based on the XDG Base Directory Specification.
For those not familiar, the XDG specification defines standard environment variables for locating the home directory and various subfolders.

In version 3, $HELM_HOME is no more ☺

Also, the “helm init” and “helm home” commands no longer exist.

Library Charts

Starting with v3, a chart can have the type (meta-data chart property) of either “application” or “library” (“application” by default).
Library charts are common charts that are reusable and intended for use in a containing application.

Regarding chart dependencies, requirements and dependencies have been moved into the chart.yaml itself.
A Smooth Migration?

While experimenting with Helm v3, I ran into some issues where I had a chart deployed with v2 and tried to delete and replace it using a v3 client.
I got some weird errors when trying to reinstall the chart (e.g., already exists). Although looking at “helm ls” didn’t display anything about my chart.
This error occurred because the different versions of Helm store their catalog in different locations.
I had to revert to Helm v2 client to purge my chart.
So, keep that in mind and follow proper migration guides.

ChartCenter

Combined with the release of Helm v3, I was also excited to hear the announcement of ChartCenter.
ChartCenter (https://chartcenter.io/) provides you with all the information you need about the charts you depend on, including security vulnerabilities scanning information powered by JFrog Xray.
On the site’s UI you can dig deep into the subcomponents of the included containers and see the vulnerable components down to the application’s dependencies.
Not only do I now have a “go-to” place for fetching my infrastructure chart, I can also assure myself that my dependencies have no critical security vulnerabilities.

Best Practices for Onboarding Security & Compliance Scanning Tools

Eran Blumenthal — Sun, 02 Aug 2020 14:12:10 +0000

Introducing, adding, or replacing a new binary security and compliance analysis tool into your SDLC, if not handled correctly, can be very disruptive to the SDLC and organization.
This article will go over what I believe is a best practice for onboarding such tools in order to reduce disruption, improve adoption, and shift left to create a DevSecOps behaviour.

First, let’s begin by describing the scenario I think about when talking about onboarding security tools. What usually happens, especially when introducing a new tool, is that all screens go red and alerts appear from every direction. In such scenarios, it is a reasonable knee-jerk reaction to require a system lockdown; disqualify builds, reject dependencies, etc. However, such a reaction, even though it may seem valid, is counter productive. Such a behaviour, in theory, will bring production to a halt, cause frustration and conflicts. Naturally, in real life, in most cases, no one will really stop the business. Rather, the organisation may develop Alert Fatigue and will ignore the tool, which is definitely not something we want. Keep reading to find out how to avoid this scenario and address onboarding DevSecOps tools with minimal disruption and maximum gain.

The following is a list of 5 suggestions when onboarding SCA (Software Composition Analysis) tools:

Involve R&D

When you involve R&D in the process, you have a real chance at achieving a real DevSecOps process that is manageable and productive. There is approximately one security engineer per 100-200 developers. Reviewing and controlling all security issues by a single engineer is only possible when it is attached to development. Otherwise it will cause bottlenecks, delays, redundant work, and a lot of frustration.
A “watch” per application team

Each security tool uses a different name, but most of them have some notion of scope. By scope I mean ‘some way to group a set of resources (repositories, folders, builds, etc.) for the application of relevant security and compliance governing rules.
Creating a “watch” per application team allows giving every team their own “world” and responsibility. In turn, this provides the ability to “shift left” the responsibility of security governance.
Dev tools integration

Try to bring the information into the development tools (i.e. CI server, IDE), which will provide governance related information into the developmer’s environment.
Start small and work in cycles
- Choose one team to start with.
  - Considerations for choosing the right team:
    - A vanguard team - a team which is open to changes and testing new ways to improve (especially if they care about security).
    - A new app team - if you have the option to, choose a team which is starting a new project/service (i.e. starting “greenfield”)
    - Or a team with less “integrations”/dependency with other teams.
  - Work with the chosen team to establish and improve the process before expanding to other teams.
- Start with “Critical” issues - As mentioned in the introduction, introducing a new SCA tool will most likely generate tens or hundreds of alerts of all types and severities. Even in small scopes, trying to handle everything at once might not be reasonable. If you see that even the number of critical issues is too high and your tool supports it, try to be even more granular than Low/Medium/High/Critical. Try to go by CVSS score; start with 9.9-10, then 9.8-10, 9.7-10, etc. The granularity of this filtering should be based on your needs.
- Make a decision for each “critical” - Decide whether the issue needs and can be fixed, or whether to whitelist the issue. This might be a decision to temporarily, permanently whitelist the issue, or define a deadline for fixing the issue.
- Do not use “brute force actions” - Most tools allow you to take actions. Email notifications, webhooks, Slack, Jira, etc. Some tools provide additional capabilities such as blocking download, failing builds, or similar. Until you finish the cleanup of manageable existing issues (i.e. issues you can resolve or ignore) avoid any action that will disrupt the development lifecycle such as blocking download of a needed dependency or failing a build based on the new tool’s scan results.
  - Only when the build passes without violations consider a hard action future.
Add external notifications (Jira, Slack, etc.)
- You probably want to add external notifications only for new issues, otherwise you may be risking “Alert Fatigue”.

After handling criticals repeat the same process for major issues. For minor issues, you should consider if you want to handle them at this point in time, or rather move to the next project/team and only then deal with them.

This post provided a general best practice flow for introducing a new SCA tool into your organisation/environment. Hopefully this will help you remove any tension that you may have between Development and Security, as well as provide some DevSecOps foundation.

Note: A version of this blog post is also published here

JFrog Xray & Microsoft Teams

John Peterson — Thu, 30 Jul 2020 22:50:09 +0000

JFrog Xray & Microsoft Teams

More than ever we need to be made aware when security issues arise. JFrog Xray is a great product that brings operational awareness to your software development lifecycle combined together with Microsoft Teams we have the channel of communication we need to ensure our team is always on top of the latest security concerns.

Getting Started

Let's get started by opening up Microsoft Teams. We will need to make a new connector on the channel we want to deliver our messages into. Click on the more options "..." icon to bring up the menu of options for a channel. Select "Connector app" to bring up the popup.

Inside of the Connector popup search for "Incoming Webhooks" and click the "Add" button which will then bring you to the Configure screen shown below.

Enter the name "Xray Webhook" and download and save the Xray image to upload into the new webhook.

Scroll down in the popup after adding the webhook to grab the URL for the incoming webhook

Deploy the integration server

Download the Github repo here

Build the code using Golang

go build

Export environment variable MICROSOFT_TEAM_WEBHOOK with the URL of the incoming webhook

export MICROSOFT_TEAM_WEBHOOK=http://the-incoming-webhook-url

Run the integration server

./xray_msteam

Grab the hostname/ip address of the machine running the integration server we will use this to supply to Xray webhook in the below format

http://ip/host:8080/api/send
This is the endpoint to send the messages from Xray. Last stop let's configure Xray to send the outgoing webhook.

Xray Outgoing Webhook

As an admin user, open up JFrog Unified Platform and goto the administration setting shown below for Xray

Click on Webhooks and click on + New Webhook to open the new webhook screen. In this screen give a name and supply the URL of the integration server. Save the new webhook.

The next step is to add the new webhook to an Xray policy as a new rule. This is what will trigger the webhook when new violations are found in a watch associated to this policy. Click on Policies and create a new policy or update an existing one to add a new rule using the webhook as shown below.

That's it! Your done!

Congrats begin to watch the messages flow...

Hosting helm charts: A Maintainer’s Perspective

Prasanna Raghavendra — Tue, 07 Jul 2020 02:07:02 +0000

Helm charts is becoming one of the best ways to package any kubernetes application and indeed becoming popular among contributors and contributing companies. It provides a consistent management of any application on kubernetes and a very easy way to override default parameters.

With the release of ChartCenter by JFrog and given that I own hosting JFrog charts, it was important for me to look at this with a finer tooth.

I wanted to take a step back and see who are the providers of chart hosting and hit this nice blog. Let us take this a step further and try to compare them and see how we can leverage these options.

So, here goes my review of these options, against the following 8 criteria.

Host helm repo centrally

We need to ensure that our consumers can download these charts fast and effectively. We will need a way to host on a platform which provides a very good content caching so we are sure that this is available 24*7.

ChartCenter does provide this by running this on a well proven Artifactory, and an operation team who have managed bintray.

The other providers seem to be only working as a directory as they directly provide install commands to the maintainer’s repo. This leads maintainers to take the burden of hosting helm repo effectively, behind CDN.

Clarify chart dependencies

A maintainer would like to ensure that the consumers are clear of the dependencies that the chart has on both sides, where all it is referenced so the consumer can choose a larger chart if needed, and what it references further down, to ensure the consumer is aware of what consumer is picking before he/she installs without having to open the chart.

ChartCenter does provide this effectively.

Did not see them in any other.

Amount of charts hosted

This is important to ensure that as a maintainer you are hosting on a site that is popular and more consumers are on it. It is clear from the numbers (provided in the summary table below), all are in early stages and will surely evolve in the next few months.

Download metrics

Providing information on who is downloading which version of the application is very useful. I found all of them having gaps in this.

While ChartCenter does provide download information, (by of course allowing downloads directly from ChartCenter), it reports by chart version which is not that relevant in the world of applications. It is preferable to have this information by application version and also provide a trend, with download regions, something that was a given in bintray.

Others, being only a portal, cannot provide download functionality and hence no download information.

Related charts

When a consumer comes in a discovery mode, having related content makes a lot of sense to helm the consumer pick the right application.

Artifact Hub provides this well and felt it had relevant charts as well, when I tried wordpress.

ChartCenter, Kubeapps hub and Helm hub seem to be lacking in this area.

Deep curation

Having thoughts and reflection (provided by the host) on charts helps deepen one's understanding on how to compare, install and use applications.

Bitnami (Kubeapps Hub) includes their perspectives in a blog for each chart (of course popular ones). This gives a very good view-point on how one may end up using the chart.
The other three players have not been present in this space.

Make overall usage safe and secure

Having a security report at a chart level provides another nice perspective to know what is going on with the images bundled. This is an area where every chart maintainer is catching up and clearing up their internal security and also working on to depend on the cleanest and latest charts.

ChartCenter is the only one providing this perspective. They are extending this further to allow providers to provide their view-point on some of the open issues. This is an interesting direction ChartCenter has taken and I like this one very much.

Smarter set-me up for better on-boarding

Not all charts are completely independent, especially when you are looking at extension products where you expect some other service is already running, then charts would make a few mandatory values to be filled before getting them working.

Almost all of the providers seem to provide vanilla install commands ignoring some of the mandatory values to be passed, finally leading to a situation where they are actually incorrect.

Summary:

Criteria	HelmHub (CNCF)	Kubeapps Hub (Bitnami)	Artifact hub (OSS)	ChartCenter (JFrog)
Chart hosting	no	no	no	yes
Chart dependency	no	no	no	yes
Deep curation	no	yes	no	no
Chart numbers	Charts: 1355 Versions: NA	Charts: 1400 Versions: NA	Charts: 820 Versions:21k	Charts: 1521 Versions:29k
Download metrics (application level)	No (dependant on hosting)	No (dependant on hosting)	No (dependant on hosting)	Partial
Related charts	no	no	yes	no
Security	no	no	no	yes
Smart installs commands	no	no	no	no

As you can see in the table above, all charts are hosting a limited number of charts, while this ecosystem will start growing. Soon, you will see more and more on-boarding into many of these.

If your requirement is to have a completely hosted support, ChartCenter stands out as the only example. If you want to be aligned with the CNCF ecosystem, you could look at helm hub. If you would like to have a third-party validation along with hosting, Bitnami is a good solution.

However, you could also add ChartCenter for hosting and have the other three list your charts so you have the best of the breed.

Let me know, what do you guys think?

Go Big or Go Home: A Quick Review of Artifactory and CodeArtifact Repository Types and Capabilities

Melissa McKay — Sat, 20 Jun 2020 00:38:31 +0000

This article is part of a series written by JFrog's Developer Advocates. The index can be found here:

JFrog Artifactory vs AWS CodeArtifact: Comparison in 10-ish parts

JBaruch 🎩 ・ Jun 19 ・ 2 min read

#jfrog #artifactory #aws #codeartifact

I’m the first to admit that I fit well into the developer stereotype of someone that gets easily distracted by the new, shiny thing. It’s just part of my nature, and I accept it. Plus, I absolutely enjoy exploring new implementations, technologies, or strategies that promise to elicit that “Ah, much better!” feeling. From my perspective, the end result of that activity is always a success, because my effort can only go one of two ways: I either find something new that improves some aspect of my life and make changes, or I confirm that I've already made the best choice.

AWS just dropped CodeArtifact into their vast array of services within the AWS ecosystem. Curious, I couldn’t stop myself from checking it out and comparing it with the user experience I have had with JFrog Artifactory - both from my previous development experience and in my current position at JFrog. Here are three questions that I went in with and the answers I came out with.

Will it support my Go modules?

As far as package type support goes, CodeArtifact hits the big ones - Maven, PyPI & npm. My first venture was to find a way to support my Go modules, but I was unsuccessful. Other notable misses include NuGet, Bower, and Docker.

CodeArtifact Documentation

"Each repository contains three unique endpoints, one for each package format: npm, pypi, and maven."

Create Repository Screen in Artifactory

Support for more package types may very well be on the roadmap for the future, but Artifactory is way ahead of the game here, currently supporting 26 different package types including a Generic type for any other currently unsupported or even custom package types. AWS has a separate Docker registry solution, Amazon Elastic Container Registry (ECR), but Artifactory includes support for multiple Docker registries and the ability to manage my Docker images along with all of the other artifact repositories. Not to belabor the point, but I’ve gotten used to how nice it feels to have all of my artifacts managed in the same place. I liken it to having all of my clothes in my closet, right where I would expect them to be. In fact, this is actually a critical point! If I don't have my image management in the same tool, how can I correlate packages with the images they are contained in?

How do I resolve dependencies?

Organizing repositories is quite a bit different in CodeArtifact than in Artifactory. In Artifactory, there is a clear delineation between local repositories, (places where your internally built artifacts go), remote repositories, (caches of artifacts obtained from 3rd parties), and virtual repositories, (aggregated repositories consisting of local and remote repositories of your choosing). Virtual repositories are especially nice because you can add any repositories you like and specify a priority order for resolution.

CodeArtifact appears to have a similar capability. When creating a repository in CodeArtifact, there is an option to add any number of "upstream" repositories, the order of which determines the order of resolution. However, I was immediately disappointed that I was limited to a relatively small number of public repositories. Artifactory is ahead of the game again, by allowing users to define any external repository they need.

CodeArtifact Available Public Upstream Repos

Artifactory Basic Remote Repo Configuration

What statistics can I get from my artifacts and packages?

According to the documentation, CodeArtifact provides information about the artifacts stored in the repositories using CodeArtifact CLI and API commands. There's of course the basic stuff: name, version, license info, contents info, and some dependency information. I admit, at this point, I did not feel like going through the pains of actually getting an artifact uploaded to a CodeArtifact to see what the UI would look like. I spent a little time looking for an example screenshot in their documentation and didn't find any.

Part of my lackluster desire to pursue much further in CodeArtifact at this time, is that in contrast, the Artifactory package detail screens are, hands down, beautiful. You can search for a specific package and then view information (in addition to the basics that CodeArtifact provides) about the activity, the number of downloads, the number of versions available, and which repositories include the package. On top of that, if you have CI/CD integrated, you can drill down into specific package information further and determine which builds in the system used this package. Artifactory is undoubtedly a mature, tracing solution that is invaluable in troubleshooting any build or dependency issues.

Artifactory Package Detail Screen

Final Thoughts

This investigation is over for now, but I know there will be more to come. My exploratory efforts have fallen into that second outcome - confirmation that JFrog Artifactory is a better choice. I can't wait to see what's next!

Return to the index:

JFrog Artifactory vs AWS CodeArtifact: Comparison in 10-ish parts

JBaruch 🎩 ・ Jun 19 ・ 2 min read

#jfrog #artifactory #aws #codeartifact

Forem: JFrog

Implementing the JFrog Xray “Summary View” in Slack

How It Works

Summary View - Why We Transform the Payload

Xray Notification Watches and Policies

How to Create Summary View Notifications in Slack

Getting Notifications by Component - Summary

Getting Notifications by Issue

How We Built the Transformation Code

Time to make some order with GoCenter

Dependency Management

Go module

Proxy my dependencies

Protect your binaries

Learn more about your packages

Advanced mode private GOPROXY

Conclusion

DevOps 101: Package Management

What even is that?

The Before Times

What’s a binary repository manager?

Cool cool cool, how do I try using one of these things? Sounds enterprisey.

Summarize this for me, high school essay style.

De(v)lightful continuous benchmarks with Go

What we need

Easy, right?

Not In the real world

So, what can we do?

1. Wrap application setup and benchmark state

2. Use pre-populated DB docker images

Actionability

Further Considerations

Conclusion

Docker new Download Rate Limits

The Problem

Pull request limitations

Image retention policy

The Solution

Why should we use Artifactory and how it works

Artifactory version

Create a Docker repository

Configure repository advanced configuration

How to use Helm to Support Cloud-Native Development

Why go to the cloud?

Achieve unlimited scalability:

Hardware costs are no longer an issue:

Gain redundancy:

Configuring a cloud-based automated CI/CD pipeline

Step 1: Setting up your cloud environment

Step 2: Configuring the VCS server

Step 3: Running the build

Step 4: Testing the build

Step 5: Scanning the build

Step 6: Deploying to development

Step 7: Deploying to staging

Step 8: Deploying to production

It’s time to have some fun!

How Everyone Can Win in the New Era of Docker Hub Limits

AN OBSERVATION OF HUMAN BEHAVIOR

WHAT IS THE PROBLEM? WHY IS THIS COMING UP NOW?

HOW DO I DEAL? HOW DO I MITIGATE RISK FOR MY DEVOPS TEAM?

Helm V3, Latest & Greatest of Kubernetes

Removal of Tiller

XDG-based Directory Structure

Library Charts

ChartCenter

Best Practices for Onboarding Security & Compliance Scanning Tools

Involve R&D

A “watch” per application team

Dev tools integration

Start small and work in cycles

Add external notifications (Jira, Slack, etc.)

JFrog Xray & Microsoft Teams

JFrog Xray & Microsoft Teams

Getting Started

Deploy the integration server

Xray Outgoing Webhook

That's it! Your done!

Hosting helm charts: A Maintainer’s Perspective

Host helm repo centrally