CrowdSec v.1.0 is out: introduction of the local API

Jean Devaux — Tue, 08 Dec 2020 14:44:40 +0000

The team is happy to announce the official release of CrowdSec v.1.0 which introduces several improvements to the previous version, including a major architectural change: the introduction of a local REST API.

This local API allows all components to communicate together in a more efficient way, supporting more complex architectures, while keeping it simple for mono-machines users. It also makes the creation of bouncers (the remediation component) much simpler and renders them more resilient to upcoming changes, limiting the necessary maintenance time.

New architecture

In the new 1.0 release, the CrowdSec architecture has been deeply remodeled:

All CrowdSec components (the agent reading logs, cscli for humans, and bouncers to deter the bad guys) can now communicate together via a REST API, instead of reading or writing directly in the database. With this new version, only the local API service will interact with the database (supports SQLite, PostgreSQL and MySQL).

What it brings to the table

The benefits of having a REST API for bouncers are major:

Bouncers become truly stateless, making it possible to build new ones for serverless/lambda environments
Future changes in DB schemas won’t impact bouncers, and API versioning ensures smooth control over this
Bouncer implementation becomes really quick and easy: just one HTTP GET request with an API token is needed

Much more details about this new release benefits can be found in the product announcement here.

We would love to hear your feedback about this latest release. If you are interested in testing the software or would like to get in touch with the team, here are a few useful links:

Hope to hear from you soon!

The CrowdSec team

CrowdSec, an open-source, modernized & collaborative Fail2ban

Jean Devaux — Tue, 29 Sep 2020 15:19:41 +0000

Dear estimated community,

We would like to introduce a new security project, CrowdSec, and collect your feedback & comments.
The solution is available on GitHub and will remain open-source (MIT license) and free of charge.

TL;DR: CrowdSec parses logs from various data sources, normalizes and enriches them before applying heuristic scenarios to identify aggressive behaviors and protect you from most attack classes. Like with fail2ban, things like credential stuffing, web or port scans, ssh / ftp / telnet brute-force, and many others are really easy to defeat with the software, but CrowdSec modern grammar & architecture give the users more possibilities.

Target & goal: CrowdSec is designed to protect servers, services, containers or VMs exposed on the Internet with a server side agent. It currently runs on Linux (ports to MacOS & Windows are on the roadmap).

How it works: The software is written in Go-Lang and thought from day one to run on modern, complex architectures, like cloudified ones, lambdas, containers, etc. To achieve this, it’s “decoupled”. Meaning you can “detect here” (say in your database logs) and “remedy there” (say in your firewall or Rproxy). The tool internally uses leaky buckets to allow for tight event control. Scenarios are written in YAML to make them as simple and readable as possible, without sacrificing granularity. The inference engine lets you get insights from chain buckets or meta-buckets. (i.e. if several buckets (web scan, port scan and login attempt failed) overflow in a “meta bucket”, you can trigger a “targeted attack” remediation).

Aggressive IPs are dealt with by bouncers. The CrowdSec Hub offers ready to use data connectors, bouncers (Nginx, PHP, CloudFlare, Netfilter) and scenarios to deter various attack classes. Bouncers will be able to remedy threats in various ways. We work on bouncers like Captcha, limiting applicative rights, MFA, throttling queries, or activating Cloudflare attack mode just when needed, etc. You also already get a sense of what’s happening locally (and where from), with a lightweight visualisation interface and a strong prometheus observability.

While the software currently looks like a 2020 pimped fail2ban, the endgame is to leverage the power of the crowd to create a very accurate IP reputation database. When CrowdSec bounces a specific IP, the triggered scenario and the timestamp are sent to our API, to be checked and integrated in the global consensus of bad IPs. While we are already redistributing a block list to our community (you can see it with the CLI: cscli ban list --api), we plan to really improve this part as soon as we have dealt with other, prerequisite, code lines. The network already has sightings of 100K+ IPs (refreshed daily), and is able to redistribute ~10% (10K) of those to our community members. Also to be noted, the project has been designed to be GDPR compliant and privacy respectful, both in technical and legal terms.

Mid-term vision: When the CS community will be large enough, we will all generate, in real time, the most accurate IP reputation database. This global reputation engine coupled with the local behavior assessment and remediation should allow lots of businesses to get tighter security at a very low cost.

Current state: Setup is quick & easy, heavily assisted by the wizard, to allow the greatest number to use it. The project is production-grade and already runs in many places, including hosting companies. As a good example, one of the CrowdSec users was able to stop a botnet attack from 7,000 different IPs in 5 minutes last week thanks to the solution. We are looking for more users, contributors and ambassadors to take the project to the next level. As of today, community members come from 21 countries across 5 different continents.

We would love to hear your feedback and engage further discussions so don’t hesitate to comment, reach out through our website, GitHub, Discourse or give us a shout on Gitter.

Hope you will like it, use it and eventually contribute to improve it. Thanks in advance for sharing your thoughts.

The CrowdSec team

Forem: Jean Devaux

CrowdSec v.1.0 is out: introduction of the local API

CrowdSec, an open-source, modernized & collaborative Fail2ban