Forem: Alcide

New Kubernetes Node Vulnerability (CVE-2020-8558) bypasses localhost boundary

Gadi Naor — Wed, 22 Jul 2020 08:11:30 +0000

Vulnerability Description and Impact

A security issue was discovered in kube-proxy which allows adjacent nodes/hosts to reach TCP and UDP services bound to 127.0.0.1 running on the node or in the node's network namespace (host network). This breaks security assumptions made by services listening on localhost.

This security bug was originally raised in issue #90259 which details how the kube-proxy sets net.ipv4.conf.all.route_localnet=1 and causes the system not to reject traffic to localhost which originates on other hosts (martian traffic). Such traffic would look like packets on the wire with an IPv4 destination in the range 127.0.0.0/8 and a layer-2 destination MAC address of a target node may indicate that an attack is targeting this vulnerability.

For example, if a cluster administrator runs a TCP service on a node that listens on 127.0.0.1:1234, because of this security bug, that service would be potentially reachable by other hosts on the same LAN as the node, or by containers running on the same node as the service. If the example service on port 1234 required no additional authentication (because it assumed that only other localhost processes could reach it), then it could be vulnerable to attacks that make use of this security bug.

While many Kubernetes installers explicitly disable the API Server's insecure port, and Kubernetes v1.20 is planned to remove this insecure option, An API server that uses this insecure option and listens on 127.0.0.1:8080 will accept requests without authentication.
To mount such an attack on the API server, an attacker must have access to another system on the same LAN or with control of a container running on the master. Managed Kubernetes services such as EKS, AKS, GKE and others should be resilient attacks on the API server insecure port.

Are You Vulnerable?

The vulnerability affects kubelet & kube-proxy which are core Node components:

Affected Versions:

kubelet/kube-proxy v1.18.0-1.18.3
kubelet/kube-proxy v1.17.0-1.17.6
kubelet/kube-proxy <=1.16.10

Or if one or more of the following items are applicable to your environments:

Your cluster nodes run in an environment where untrusted hosts share the same layer 2 domain (i.e. same LAN) as the cluster nodes.
Your cluster allows untrusted pods to run containers with CAP_NET_RAW which is enabled by default by Kubernetes.
Your nodes (or hostnetwork pods) run any localhost-only services which do not require any further authentication. To list services that are potentially affected, run the following commands on nodes:

lsof +c 15 -P -n -i4TCP@127.0.0.1 -sTCP:LISTEN
lsof +c 15 -P -n -i4UDP@127.0.0.1

Risk:

Typical Clusters: medium (5.4) CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:N
In clusters where API server insecure port has not been disabled: high (8.8) CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H

Automatic Detection of adjacent Node attacks with Alcide Runtime

This new vulnerability has long been covered by Alcide Runtime without requiring any configuration by the user.
This detection has been part of Alcide Runtime for a long time, even prior to this vulnerability’s disclosure, and there was no need for us to add additional detection capabilities.

The traffic is flagged as spoofed traffic because localhost traffic should never cross network boundaries. Furthermore, if you explicitly define firewall policies using Alcide’s microservices firewall, then pods can’t access other resources in the network unless explicitly allowed. This is what zero-trust networking is all about!

Automatic Detection of API Server attacks with Alcide kAudit

Exploit attempts of this vulnerability via the unsecured port of the API Server would show up in its audit log as entries from the "system:unsecured" principal, similar to entries from K8s services on the Master node accessing the API Server locally.

Alcide kAudit analyzes the audit log of the Kubernetes API Server, continuously updates and compares behavioral activity profiles to actual observed activity and automatically detects anomalous access patterns from cluster principals (in this case, "system:unsecured" account) to various resource types, k8s APIs, namespaces, and specific resources. Thus, Alcide kAudit can automatically alert security teams to suspected attacks that rely on CVE-2020-8558 as they occur.

Furthermore, Alcide kAudit users can configure and customize it to alert on audit entries violating their specific policy. They can, for example, add alerts on API Server activity that reads or modifies sensitive namespaces or resources in the cluster, and use such alerts if they happen as an indicator that an exploit may be in progress and should be investigated.

Automatic Detection of Vulnerable Clusters with Alcide Advisor

Alcide Advisor is a Kubernetes multi-cluster vulnerability scanner that covers rich Kubernetes and Istio security best practices and compliance checks such as Kubernetes vulnerability scanning, hunting misplaced secrets, or excessive secret access, and many more security configuration and compliance checks.

With Alcide Advisor users can:

Identify the vulnerable clusters for this vulnerability (as well as other CVEs)
Identify and explicitly allow/deny which Pods can run with elevated privileges that enables CAP_NET_RAW
Identify and explicitly allow/deny which Pods can run on the host network.
On applicable environments, identify Pods that can run on master nodes that can potentially exploit the Kubernetes API Server.

Conclusion

Kubernetes, like any software, has bugs and vulnerabilities. Leveraging Kubernetes as cloud-native application infrastructure requires operators to monitor and secure all the moving parts, whether these are the application workloads or the platform and infrastructure components. CVE-2020-8558 joins other recent vulnerability disclosures (CVE-2020-8555 and CVE-2020-10749) and highlights the need for a purpose-built Kubernetes security solution that can drive cluster operators to run workloads, applications, and infrastructure while leveraging the best security practices of the native Kubernetes security controls, as well as security monitoring & prevention.

Start your 14-day trial or request a demo.

New Kubernetes Control Plane Vulnerability (CVE-2020-8555)

Nitzan Niv — Mon, 20 Jul 2020 13:40:33 +0000

Vulnerability Description and Impact

A security issue was discovered in Kubernetes and disclosed on June 1, 2020, as CVE-2020-8552.
The vulnerability enables an attacker to gain access to data from services that are connected to the host network of the cluster’s manager, and although the attack is not simple to execute, it can remotely bypass authorization controls and break confidentiality.

Are You Vulnerable?

The vulnerability affects kube-controller-manager which is part of Kubernetes control plane:

V1.18.0, v1.17.0 - v1.17.4, v1.16.0 - v1.16.8, and versions earlier than v1.15.11

The affected volume types that can be abused as part of the attack execution, as explained below, are:
GlusterFS, Quobyte, StorageFS, ScaleIO.

The vulnerability is patched in Kubernetes versions:

V1.18.1+, v1.17.5+, v1.16.9+, v1.15.12+

Remediation & Mitigation

Prior to upgrading, these vulnerabilities can be mitigated by adding endpoint protections on the master or restricting usage of the vulnerable volume types and restricting StorageClass write permissions through RBAC.

Technical Breakdown of the Exploit

According to Kubernetes’ GitHub issue, this vulnerability allows certain authorized users to access endpoints within the master's host network, such as link-local or loopback services.
By exploiting this vulnerability, these users can leak arbitrary information (up to 500 bytes per successful malicious request) from such unprotected endpoints.

The attack’s steps are:

An attacker with permissions to do so creates a pod
The attacker attaches a volume to the pod. This volume can be one with certain built-in Volume types (GlusterFS, Quobyte, StorageFS, ScaleIO). An attacker with permissions to create a StorageClass can use this capability to the same effect.
By using the volume attachment or storage class creation, the attacker makes the kube-controller-manager k8s component make GET requests or POST requests without an attacker-controlled request body from the master's host network.

Automatic Vulnerability and Attack Detection with Alcide kAudit

The Kubernetes API Server logs every request it receives in its audit log. Of specific relevance to the detection of attempts to exploit CVE-2020-8555, actions like creation of a new pod, creation of a StorageClass, and sending of any requests to the API server leave traces in the audit log.

Alcide kAudit automatically monitors and analyzes these audit logs. It creates and dynamically updates a profile of the normal behavior in the cluster. By comparing in real-time this profile to the audit log, kAudit can detect anomalous behavior that is associated with attempts to attack the k8s infrastructure of a cluster.

kAudit can detect anomalous behavior related to attempts to exploit CVE-2020-8555 at several points along the attack chain:

Attacker creates a pod
Attacker creates a StorageClass
kube-controller-manager sends unusual requests to API-Server
Unusual increase in unauthorized requests or other irregular status in responses, when an attacker attempts to execute the previous steps.

By combining correlated anomalies to an incident associated with the attacking user, which in this case is the one that creates the pod and StorageClass, Alcide kAudit can alert security teams that an attack was attempted and focus their attention on the relevant users, resources and actions to investigate.

Like other security tools, kAudit also enables the user to configure rules to filter specific entries in the audit log that are interesting for compliance and security investigation. However, in this case, even if the user happens to create rules that identify specific actions that are part of the attack, it will be difficult and time-consuming for a security expert to link these traces of the attack and to create a holistic understanding of it, as regular cluster activity also often creates pods, creates StorageClass, sends requests from the kube-controller component to the API server, or occasionally sends unauthorized requests.

On the other hand, the automated machine learning algorithm used by Alcide kAudit will sift through the log and focus the expert’s attention on the security incident and anomalous behavior that stands out against this noisy background.

Automatic Detection of Vulnerable Clusters with Alcide Advisor

With Alcide Advisor users can:

Identify the vulnerable clusters for this vulnerability (as well as other CVEs)
Define StorageClass whitelist and identify violations of this list.
Identify the components, as well as ServiceAccounts that have RBAC permissions to create pods.

Conclusion

Kubernetes, like any software, has bugs and vulnerabilities. Leveraging Kubernetes as a cloud-native application infrastructure requires operators to monitor and secure all the moving parts, whether these are the application workloads or the platform and infrastructure components.

Alcide Advisor can identify vulnerable clusters and understand the risk surface associated with CVE-2020-8555, as well as mitigate it. With Alcide kAudit, machine-learning driven dynamic profiling is used to detect actual attempts to exploit known weaknesses such as CVE-2020-8555, as well as detecting attacks targeting new and unknown vulnerabilities.

Try Alcide's security solution with a 14-day trial

Helm Scan With GitHub Actions & K8s Advisor

Gadi Naor — Tue, 21 Apr 2020 07:21:32 +0000

GitHub Actions is a recent continuous integration (CI) and continuous deployment (CD) service from GitHub. GitHub Actions powers GitHub's built-in continuous integration service. In its essence, GitHub Actions help developers automate software development workflows in the same place they store code and collaborate on pull requests and issues.

GitHub Actions enable developers to write individual tasks, called actions, and combine them to create a custom workflow. Workflows are custom automated processes that developers can set up in their repository to build, test, package, release, or deploy any code project on GitHub.

Create Kubernetes KIND Cluster

Kubernetes IN Docker, KIND, is a tool to create local clusters for testing Kubernetes using Docker containers. KIND was primarily designed for testing Kubernetes itself but may be used for local development or CI.

Multi-node clusters and other advanced features may be configured with a config file - the detailed usage information and documentation are here. GitHub Actions has a marketplace for reusable actions, and the folks behind Helm already managed to put together Kind Cluster, a reusable action that can be plugged into GitHub’s automation workflow.

Security Scanning of Helm Charts with GitHub Actions Workflow

Alcide Advisor, an API driven, Kubernetes security and hygiene scanner, has a wide integration surface into the continuous deployment (CD) platforms. However, GitHub's actions combined with KIND, introduce an interesting approach for scanning helm charts in continuous integration (CI) stage.
In the example below, a GitHub workflow has 3 sequential jobs:

Build
Test
Advisor Scan

The Advisor Scan Job performs the following steps:

Download Helm 3
Launch Kind Cluster using a GitHub Action
Install a chart (uswitch kiam in this example) into a specific namespace
Download Alcide Advisor scanner
Scan with Alcide Advisor the namespace into which kiam was installed
Publish the scan report into the pipeline artifacts

Conclusion

Helm is the de-facto tool for collaborating when creating, installing, and managing applications inside of Kubernetes. Rendering helm charts with configuration into a cluster that can be scanned by Alcide Advisor, opens the door for developers & DevOps to ‘get a handle‘ on the security and hygiene level of new helm charts as well as helm charts changes. To see the full pipeline example go to https://github.com/alcideio/pipeline

Container Networking - Explained

arikalcide — Mon, 20 Apr 2020 07:39:25 +0000

Container networking is one of the most critical concerns in production environments where scale, security, and availability are required to be as automated and as seamless as possible. In this blog post, I want to focus on the role that container networking plays in enterprises today.

In the past few years, containers have become the leading technology for implementing microservices applications. It is no longer deniable that containers have changed the way applications are developed and deployed, but not less neglected is their impact on how applications are connected to the network. While the majority of the discussion around containers focuses on developers aspects and orchestration, this blog comes to shed some light on container networking.

Container networking is one of the most critical concerns in production environments where scale, security and availability are required to be as automated and as seamless as possible.

Though they share similarities, there are some major differences between container networking and VM networking. Let’s name a few:

Containers share the same kernel. They can share the same NIC and network namespace with the host (‘host’ mode) OR they can be connected to an internal vNIC with their own network namespace (‘bridge’ mode - most used). VMs, on the other hand, simulate the entire hardware including a vNIC which is connected to the physical NIC.
Containers are also ephemeral. While VMs stay for long, containers are rapidly changing, rising and disappearing as their underlying application scales.
here are more containers than VMs. Multiple containers can run on the same host. More containers mean more NICs and more traffic. These require more resources including larger IP addresses space, more routing decisions, more firewall rules and more sockets in use. This means that efficient hardware is a must.

Over time and with containers becoming ubiquitous, running with multiple containers on multi-host networking has become a real connectivity issue. To address this problem, container projects adopted a model where networking was decoupled from the container runtime. In this model, the container network stack is handled by a ‘plugin’ or ‘driver’ that manages its network interfaces and defines how it connects to the network.

There are two main standards for container networking configuration on Linux containers: the CNI (Container Network Interface) and the CNM (Container Networking Model).

CNI project came up by CoreOS and created for writing network plugins. CNM, on the other hand, came up by Docker for the same purpose, each has its different ways for solving similar problems. Both help build modular networks along with a set of third-party vendors providing networking extended capabilities.

The basic model is composed of three major components:

Network - a group of endpoints that can communicate with each other directly, mostly implemented with Linux bridge.
Endpoint - a network interface that joins a Sandbox to a Network. Many endpoints can exist in a sandbox but only one can belong to the network. mostly implemented with a virtual Ethernet “veth” pair.
Sandbox - an isolated environment that contains the container’s network stack configuration. A sandbox can contain many endpoints from many networks, mostly implemented with Linux Network Namespace.

The Current State-of-the-Art in Container Networking

There are some areas that container networking handle fairly well:

Overlay networks allow to create and manage private multi-host networks for communication between containers and services, with isolation capabilities for the sense of a more secure network.
There are however some orchestration frameworks such as Kubernetes that automates and ease the operational containers network tasks
Monitoring is available with some of the great providers such as LogzIO and Datadog.
Third-party plugins support moving containerized applications between hosts with their state and storage.
Some CNIs support end-to-end encryption while others provide network policy capabilities for service mesh architecture.

OK, So What Now? Moving Towards Containerized Applications

Microservices architecture makes a lot of sense when dealing with scalability and the usage of containers helps to keep this architecture notion. Being a hybrid technology, containers can run (almost) everywhere. With easier and lighter deployment procedures it allows a quick duplication of microservices in runtime without having to provision new network resources. Microservices need to communicate with each other and are often required to be accessible to/from the outside world.

With the help of containers, it is possible to manage the internal communication between microservices by grouping all microservices of the same application under the same network. Moreover, containers network isolation provides segmentation capabilities at the level of a microservice and that serves both for security and compliance considerations.

Ensure Your Network is Architected to Handle Containers Effectively

When dealing with container networking, CNI and CNM fall short in meeting enterprise requirements. To apply for these requirements containers need to be agile, fast, and secure.

The perimeter has changed. What once was a monolithic application fully deployed on-premise is now spread and split on multiple cloud providers, whether is a private or public. The organization gateway is no longer dealing only with virtual and physical servers, but with multiple applications and microservices hiding behind a NAT, making the job of load balancing and security even harder. It is almost impossible to manage firewall rules for every microservice, and security groups are no longer efficient for applying a robust zero trust security approach.

Automated processes are one of the most crucial functionalities in an efficient, highly available and monitored data center. However, both container runtime and container networking plugin fall short in addressing these concerns. Auto-scale needs to be added (by coding it) to the cluster. In addition, running “everywhere” means that binaries still need to run on their compiled architecture and network policies need to be defined and written for each and every running container. Last but not least is the challenge of managing persistent storage for stateful applications.

Container Challenges For the Network Team

For those coming from legacy networking backgrounds, the adoption of containers can be a real challenge. The options are wide open when it comes to container networking implementation and yet, standardization efforts have started to take place.

Connectivity, availability and fast response times are the biggest concerns of any network team. These concerns become even greater when dealing with today’s complex networking stack. Containers’ network behavior acts differently from what we know in legacy networking: challenges such as maximizing the network performance and utilization are more complex with containerized applications. While data is usually going east-west, containers also add north-south traffic which may require some adjustments to the network architecture and load balancers.
It is important to keep network capacity neither under-utilized nor over-loaded and leading to a bottleneck in microservices environments.

GitOps - A Security Perspective (Part 1)

Gadi Naor — Mon, 13 Apr 2020 13:03:45 +0000

GitOps is a paradigm that puts Git at the heart of building and operating cloud native applications by using Git as the single source of truth and empowers developers to perform what used to fall under IT operations. This post is part a blog post series covering GitOps and Kubernetes security.

Kubernetes - A GitOps Companion

Kubernetes, as the new application server, leverages a “declarative” approach when it comes to building cloud native application, which means that application configuration is guaranteed by a set of facts instead of by a set of instructions. With application’s declarations versioned in Git, we have a single source of truth, our apps can be easily deployed and rolled back to and from Kubernetes, and when disaster strikes, your cluster’s infrastructure can also be reproduced.

With Git at the center of the delivery pipelines, developers can make pull requests to accelerate and simplify application deployments and operations tasks to Kubernetes.

Is GitOps right for you?

The fresh approach that GitOps + Kubernetes brings into the application delivery lifecycle is undeniably different, increasing engineering velocity, as well as simplifies building the CI+CD pipelines themselves. The question of whether a GitOps ‘Pull’ approach is a better fit than ‘Push’ approach is really a matter of engineering & operational culture of an organization, as well as almost a theological question of whether engineering are accountable for security and what visibility into this application server blackbox, security teams require.

GitOps & Security

GitOps changes are synched into the cluster only through the cluster git repository users. The repository is secured at the same level of the git user accounts. A compromised user account in with permissions to push into the cluster git repo can introduce changes that will result in a data breach, service disruption or anything in between. There must be additional guardrails that GitOps infrastructure must implement in the form of whitelisting in order to have a cluster side guardrails.
GitOps tools like Flux, ArgoCD and alike are practically running with cluster god permissions - and are persistent in the cluster. Kubernetes Dashboard, which is considered to be a high risk cluster, is often times removed from the cluster.
‘Push’ based CD pipelines, such as Spinnaker, Jenkins and alike are external to the cluster, invoke on-demand, and introduce automation-driven changes into the cluster.

Example: Flux RBAC Permission - Cluster God * * * * *
GitOps tools like Flux, ArgoCD and alike require cluster external access, represented as domain names (github.com, bitbucket.org, gitlab.com,..) which means that the Kubernetes native policies are not suitable to implement to segment those highly privileged in-cluster components.
Application secrets in a GitOps era requires a Kubernetes external secret provider. For example Hashicorp Vault, AWS KMS, Azure Vault and alike. Alternatively, teams can revert into Git Secrets which means committing secrets into git in their encrypted form, and decrypting the secrets before application consumption.

Conclusion

Continuous integration/continuous development (CI/CD) with the Kubernetes ecosystem does have a variety of tools to choose from and organizations should use the tools that are best suited for their specific use cases and culture. Glueing all the pieces together is not trivial. Integrating security and consuming security insights by various stakeholders is an equally challenging task to achieve. GitOps simplifies this in some aspects, but complicates in other aspects.

Stay tuned for Part 2.

Join our upcoming webinar on April 22nd: GitOps Best Practices for Continuous Deployment and Progressive Security.

Istio Service Mesh in 2020: Envoy In, Control Plane Simplified

Alon Berger — Mon, 13 Apr 2020 10:42:30 +0000

Since 2017, Kubernetes has soared and has played a key role within the cloud-native computing community. With this movement, more and more companies who already embraced microservices realized that a dedicated software layer for managing the service-to-service communication is required.

Enter the Service Mesh, and its leading contender as a preferred control plane manager - Istio, a platform built around an Envoy proxy to manage, control and monitor traffic flow and securing services and the connections between one another. Check out this page and Istio’s blog for more information and additional features to come.

According to the CNCF Survey 2019, Istio is at the top of the chart as the preferred service mesh project:

While Istio clearly made its mark as a powerful service mesh tool, it is still entwined with a relatively complex operation and integration requirements.

Istio’s roadmap for 2020 is all about supporting companies as they adopt microservices architectures for application development. The main focus of Istio’s latest release is simply making it faster and easier to use.

What Should We Expect?

Istio’s offering is a complete solution for enabling orchestration of a deployed services network with ease. It utilizes complex operational requirements like load-balancing, service-to-service authentication, monitoring, rate-limiting and more.

To achieve that, Istio provides its core features as key capabilities across a network of services:

Traffic management
Security
Observability
Platform support
Integration and customization

With its latest release, along with some most anticipated improvements, those features are getting buffed as well.

During 2019 Istio’s build and test infrastructure improved significantly, resulting in higher quality and easier release cycles. A big focus was around improving user experience, with many additional commands added to allow easier operations and smother troubleshooting experience.

Furthermore, Istio’s team reported exceptional growth in contributors within the product’s community.

Mixer Out, Envoy In

Extensibility with Istio was enabled by the Mixer, an entity responsible for providing policy controls and telemetry collection, which acts as an Intermediation layer that allows fine-grained control over all interactions between the mesh and infrastructure backends.

This entire model is now migrated directly in the proxies, in order to remove additional dependencies, resulting in a substantial reduction in latency and a significant improvement in overall performance. Eventually, the Mixer will be released as a separate add-on, as part of the Istio ecosystem.

The new model replacing Mixer will use Envoy’s extensions, which paves the path to even more capabilities and flexibility. There is already an ongoing implementation of a WebAssembly runtime in Envoy, which will potentially extend platform efficiency, This type of flexibility was a lot more challenging to achieve with Mixer.

Another key takeaway from this new model is the ability to avoid using a unique CRD for every integration with Istio.

Control Plane Simplified

The desire to have fewer moving parts during deployments drove the Istio team towards istiod, a new single binary, which now acts as a single daemon, responsible for the various microservices deployments.

This binary combines features from known key components such as the Pilot, Citadel, Galley and the sidecar.

This approach reduces complexity within domains across the board.

Installation, ongoing maintenance, and troubleshooting efforts will become much more straightforward while supporting all functionalities from previous releases.

Additionally, the node-agent’s functionality used to distribute certificates, moved to the istio-agent, which already runs in each pod, reducing even more dependencies.

Below is a “Before and After” of Istio’s high-level architecture.
Can you spot the differences?

Before:

After:

Securing All Fronts

Another major focus is on buffing up several security fundamentals like reliable workload identity, robust access policies, and comprehensive audit logging. The imperative nature of such requirements is what pushes the team to double down on stabilizing the API for these features.

Inevitably, network traffic will take up several security reinforcements, including implementation of the automated rollout of mutual TLS and leveraging of Secret Discovery Service, which will introduce a safer way of distributing certificates, thus reducing the risk of detection by other workloads running on the machine.

These upgrades will trim down both dependencies and requirements for cluster-wide security policies, leading to a much more robust system.

Here at Alcide, we offer Istio hygiene checks as part of the Alcide Advisor.
Check out our recent webinar on Security For Istio - an Incremental Approach to learn more.

Kubernetes RBAC Visualization

Gadi Naor — Sun, 05 Apr 2020 15:16:19 +0000

Role-based access control (RBAC) is a method of regulating access to a computer or network resources based on the roles of individual users within your organization. RBAC authorization uses the rbac.authorization.k8s.io API group to drive authorization decisions, allowing you to dynamically configure policies through the Kubernetes API.

Permissions are purely additive and there are no “deny” rules.

Define permissions on namespaced resources and be granted within individual namespaces
Define permissions on namespaced resources and be granted across all namespaces
Define permissions on cluster-scoped resources

Roles are used to define API access rules for resources within the namespace of the role, and ClusterRole is used to define API access across all cluster namespaces.

Alcide’s rbac-tool viz

Alcide’s rbac-tool, an open-source tool from Alcide, introduces a visualization functionality of the relationships between the resources that make your cluster RBAC configuration.

The diagram above captures the various relationship combinations between resources.

Roles - Defines the policy rules that constitute which API actions (read/create/update/delete) the subject (user/service) is allowed to perform on resources within the namespace resources.

ClusterRoles - Defines the policy rules that constitute which API actions (read/create/update/delete) the subject (user/service) is allowed to perform on resources cluster-wide.

Bindings, are the Kubernetes RBAC resources that define the link between principals (users of automated services).
Bindings can point to multiple Roles:

RoleBindings can point to ClusterRoles which grants the subject (user/service) cluster-wide access to the resources specified in the rules.

ClusterRoleBindings can point to ClusterRoles which grants the subject (user/service) cluster-wide access to the resources specified in the rules.

Nginx Ingress Controller RBAC

The following diagram shows the moving parts of the RBAC resources created by an Nginx Ingress Controller.

You can see 2 roles were created:

A role that defines the allowed resources access within the namespace
ClusterRole defines the cluster-wide access permissions

Note for example that the ClusterRole grants Nginx-ingress account to
Update the resource status of ingress within the extensions and networking.k8s.io API group.

The above visualization was generated by running the following command:

$ rbac-tool viz --include-subjects="nginx-ingress"

Under the hood, Alcide’s rbac-tool connects to the cluster context pointed by your kubeconfig , lists the various RBAC related resources, and visualize the resources based on the command line filters.

Example: API access for system:unauthenticated Group on GKE

$ rbac-tool viz --include-subjects="system:unauthenticated"

Example: GCP permission covered cloud-provider ServiceAccount for GKE

$ rbac-tool viz --include-subjects="^cloud-provider" --exclude-namespaces=""

Conclusion

Kubernetes RBAC is a critical component in your Kubernetes deployment, which is definitely something cluster operators and builders must master.
Alcide’s rbac-tool visualization and filtering capabilities helps to unfold and simplify Kubernetes RBAC.

Kubernetes RBAC | Moving from ‘It's Complicated’ to ‘In a Relationship’

Gadi Naor — Wed, 01 Apr 2020 11:09:48 +0000

Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within your organization. RBAC authorization uses the rbac.authorization.k8s.io API group to drive authorization decisions, allowing you to dynamically configure policies through the Kubernetes API.

Permissions are purely additive and there are no “deny” rules.

A Role always sets permissions within a particular namespace ; when you create a Role, you have to specify the namespace it belongs in. ClusterRole, by contrast, is a non-namespaced resource, and grants access at the cluster level. ClusterRoles have several uses.
You can use a ClusterRole to:
define permissions on namespaced resources and be granted within individual namespace(s)
define permissions on namespaced resources and be granted across all namespaces
define permissions on cluster-scoped resources

If you want to define a role within a namespace, use a Role; if you want to define a role cluster-wide, use a ClusterRole.

Default Cluster Roles
While Kubernetes RBAC is a complex topic, one would always want to implement RBAC in the cluster. For this purpose Kubernetes offers out-of-the-box default cluster roles that can be used as a starting point.
These are visible in the output of kubectl get clusterrole, and four cluster roles you can use right away are:

cluster-admin
admin
edit
view

With these roles, you can start to define who can interact with your cluster and in what way. It is highly recommended to follow the principle of least privilege, and grant additional privileges as necessary for work to proceed.
Kubernetes RBAC Resource Relationship

Example: Nginx Ingress Controller RBAC Policy

Explicitly Tuning RBAC Policies - ‘It's Complicated’
Components like Operators or highly privileged controllers that require Cluster wide ‘Read-Only’ but do not necessarily require to read secrets for example, may leave a user to take a shortcut and provision RBAC policy with excessive permissions.

Kubernetes RBAC additive model does not enable us to implement semantics that capture:
deny access to specific resource groups,
allow access to all other resources.

If we can have the explicit set of policy access rules of all cluster resources - and “subtract” from that group the resources we would like to deny access to - we achieve the above semantics.

Let’s how we can achieve that:
Run kubectl api-resources and get all of the cluster installed/supported resources and their respective api-groups
Derive the RBAC policy from the above list
Manually tune the policy and reduce any resource and/or api-groups that we wish to deny access to.

While this method will work, it’s manual, and takes significant effort.
An easier way to achieve this is with Alcide rbac-tool

rbac-tool - Simplify RBAC Policy Tuning
Alcide rbac-tool has the ability to address this exact use case.
Example: Generate a Role policy that allows create,update,get,list (read/write) everything except secrets, services, networkpolicies in core,apps & networking.k8s.io API groups
rbac-tool gen --generated-type=Role --deny-resources=secrets.,services.,networkpolicies.networking.k8s.io --allowed-verbs=* --allowed-groups=,extensions,apps,networking.k8s.io

The generated policy is cluster specific.
For a Kubernetes KIND cluster v1.16 the generated policy looks as follows:

Conclusion
Kubernetes RBAC helps us as users, to define and regulate API access to the Kubernetes cluster. In many cases where users wish to achieve even more granular controls, Validating Admission Controllers are the Kubernetes construct to achieve that. The Kubernetes ecosystem, and open-source tools such as Alcide’s rbac-tool help to unfold and simplify Kubernetes RBAC. get it here: https://github.com/alcideio/rbac-tool