Forem: Rodrigo Medina

Pyenv and Virtualenvs Quick-start

Rodrigo Medina — Sat, 29 Oct 2022 00:13:35 +0000

Image obtained from DALL-E: epic virtual pythons yellow and blue with a cyberpunk style

Whenever I get a new computer, I need to get through the python versions management, so this is the guide with the most common commands I need to execute to get up and running.

For this, I will use pyenv and the pyenv-virtualenv tools.

Initial help

This should be the most common first step for most of the tools we use.

# Get the available commands for pyenv
pyenv --help

Get The Installed Python Versions

This command will show us which versions we have already installed in our system. In the beginning, it should be only system, which is the default installation that comes with the macOS.

## List all the python versions available to pyenv
pyenv versions

Install A New Version

First, we should run the --help command, to get the available subcommands available in the install option.

# Generic help & usage
pyenv install --help

Then, we need to decide which version we want to install. For that, we may want to first see all the available versions. To achieve this, we need to execute:

# List all available versions
pyenv install --list

Finally, we need to execute the installation command with the version we want. As an example, I will install the 3.9.14 version.

# Install selected version
pyenv install 3.9.14

To verify our installation, we may execute pyenv versions again, and the recent installation should appear there.

Use The Recently Installed Python Version

To use the recently installed as the global, or default version, we need to execute:

# Sets the global Python version
pyenv global 3.9.14

Create a Virtual Environment

The best practice for python projects, is to use a virtualenv per project, so we can have isolated dependencies between them. To create a new virtualenv we need to execute:

# Create a new virtual env with the global python version
# pyenv virtualenv YOUR_VIRTUAL_ENV_NAME
# e.g.
pyenv virtualenv python-graphql-client

List All the Virtual Environments

A good way to verify that a virtual env has been created successfully is to list all the available virtual environments. To achieve this, we execute:

# Show available virtualenvs
pyenv virtualenvs

Activate The Virtual Environment

To activate, and start using your virtual environment, you need to execute:

# Activate a virtualenv
# pyenv activate YOUR_VIRTUAL_ENV_NAME
# e.g.
pyenv activate python-graphql-client

Deactivate The Virtual Environment

Once you're done using your virtual environment, you may want to deactivate it. To achieve this, you may run:

# The deactivation must be sourced
source deactivate

Extra

If you want to avoid activating and deactivating manually, you can use aactivator, which will activate and deactivate automatically based on the project you are placed.

Conclusions

This is the basic stuff that works for me. If you find it useful, consider giving a like. Happy hacking!

How To Setup Your CTFd Platform With HTTPS And SSL

Rodrigo Medina — Fri, 26 Nov 2021 05:28:22 +0000

If you want to organize and host a CTF event, one of the best and easiest options available for managing this is CTFd.

This open-source platform lets you manage users, challenges, and their categories in a very easy way, so the only thing we need to do is to clone the spin up a server, clone the repo, run the docker-compose, and set up the TLS certificate.

General requirements:

[ ] VPS/server/droplet (I'm using digital ocean but any server works)
[ ] A domain (I'm using a domain bought and configured in Google domains, but if you can modify the A registries, you're good to go)

1. Spin up your server

For this, I'm going to use Digital Ocean, but this can be done in practically any cloud provider.
I created a droplet (VPS) with the following features:

After connection to the new server (ssh root@YOUR_IP), the requirements are the following:

[ ] git
[ ] docker
[ ] docker-compose
[ ] some editor (I will use vim)

To install this, I first updated the packages with:



sudo apt update -y && sudo apt upgrade -y

1.1 [OPTIONAL] Installing tmux and nice terminal

I like to have tmux and some preconfigured vim, so I use this script which configures this for me.

You can have it by executing:



# Install zsh
sudo apt install zsh

# Execute bootstrap script
curl -L https://raw.githubusercontent.com/roeeyn/dotfiles/master/script/bootstrap_remote_server.sh | sh

After this, you need to log out and log in again to start with zsh.

1.2 Install Docker

Then executed the official commands from Docker documentation:



# Docker requirements
sudo apt install ca-certificates curl gnupg lsb-release

# Docker GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Docker setup of the stable repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/nul

# Docker installation
sudo apt update -y
sudo apt install docker-ce docker-ce-cli containerd.io

1.3 Install Docker Compose

Following the official docker-compose documentation, we can install this with:



# Get the docker-compose binary
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

# Make it executable
sudo chmod +x /usr/local/bin/docker-compose

2. Clone the CTFd repository

We need to clone the CTFd repository, which contains all the files that we need to set up our platform.



git clone https://github.com/CTFd/CTFd.git

3. Initialize the Platform

After cloning the repo, we can access the CTFd directory with cd CTFd and there we can see all the platform files.

To initialize the platform we can execute:



# If you would like to see the logs (recommended)
docker-compose up

# If you just trust that everything is going well (no logs)
docker-compose up -d

This will pull all the docker images and create all the needed containers. After finishing the startup process, the landing page of the platform should be available in the public ip.

Here you will need to set up the information about your CTF.

4. Add the IP to your DNS

When you already have your platform up and running, you need to register it to your domain. For this, I'm using Google domains, and you need to add an A register to the domain. In my case, it looks like the following:

If everything went well, at this point you should already be having the platform at your domain (with NO certificate or https yet).

5. Add a Certificate with Certbot (Let's Encrypt)

For adding a certificate we need to execute some more steps.

5.1 Generate a certificate

Before creating the certbot certificate, make sure to stop all the containers from the previous steps.
If you ran your containers from previous steps in normal mode (no -d after docker-compose), you can only press Ctrl-C in the running process.
If you ran your containers from previous steps in detached moded (with the -d flag after docker-compose) you should execute docker-compose down inside the CTFd directory.

The easiest way to generate a certificate is with the certbot docker image. Following the official documentation, this can be done with the following:



docker run -it --rm --name certbot \
          -v "/etc/letsencrypt:/etc/letsencrypt" \
          -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \
          certbot/certbot certonly

We should select the Spin up a temporary server option, enter your email, accept the agreements, and enter the domain we're planning to use.
This is similar to what you should see:



➜  ~ docker run -it --rm --name certbot \
            -v "/etc/letsencrypt:/etc/letsencrypt" \
            -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \
            -p 80:80 -p 443:443 certbot/certbot certonly

Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Spin up a temporary webserver (standalone)
2: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 1
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): rodrigo.medina.neri@gmail.com

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y
Account registered.
Please enter the domain name(s) you would like on your certificate (comma and/or
space separated) (Enter 'c' to cancel): examplectf.manguitoblue.io
Requesting a certificate for examplectf.manguitoblue.io

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/examplectf.manguitoblue.io/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/examplectf.manguitoblue.io/privkey.pem
This certificate expires on 2022-02-24.
These files will be updated when the certificate renews.

NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.
We were unable to subscribe you the EFF mailing list because your e-mail address appears to be invalid. You can try again later by visiting https://act.eff.org.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

5.2 Copy Your Recently Created Certificates

Your certificates should now be in /etc/letsencrypt/live/YOUR_DOMAIN/. In my case, it is /etc/letsencrypt/live/examplectf.manguitoblue.io/.
Let's copy fullchain.pem and privkey.pem into the CTFd folder, so we can use this inside our nginx container. For this, you can execute:



# Inside your CTFd repo directory
cp /etc/letsencrypt/live/YOUR_DOMAIN/fullchain.pem ./conf/nginx/fullchain.pem
cp /etc/letsencrypt/live/YOUR_DOMAIN/privkey.pem ./conf/nginx/privkey.pem

5.3 Modify Your docker-compose Configuration

After copying the certificates to the CTFd files, you need to add them to the nginx container volume, so they can be used inside the container. For this, you need to update the volumes section of the nginx service inside the docker-compose.yml with the following:



...
volumes:
      - ./conf/nginx/http.conf:/etc/nginx/nginx.conf
      - ./conf/nginx/fullchain.pem:/certificates/fullchain.pem:ro
      - ./conf/nginx/privkey.pem:/certificates/privkey.pem:ro
...

Also, you need to add the https port to the mapping, this is done by adding the following line in the file:



...
ports:
  - 80:80
  - 443:443
...

At the end the complete docker-compose.yml should look something like this:



version: '2'

services:
  ctfd:
    build: .
    user: root
    restart: always
    ports:
      - "8000:8000"
    environment:
      - UPLOAD_FOLDER=/var/uploads
      - DATABASE_URL=mysql+pymysql://ctfd:ctfd@db/ctfd
      - REDIS_URL=redis://cache:6379
      - WORKERS=1
      - LOG_FOLDER=/var/log/CTFd
      - ACCESS_LOG=-
      - ERROR_LOG=-
      - REVERSE_PROXY=true
    volumes:
      - .data/CTFd/logs:/var/log/CTFd
      - .data/CTFd/uploads:/var/uploads
      - .:/opt/CTFd:ro
    depends_on:
      - db
    networks:
        default:
        internal:

  nginx:
    image: nginx:1.17
    restart: always
    volumes:
      - ./conf/nginx/http.conf:/etc/nginx/nginx.conf
      - ./conf/nginx/fullchain.pem:/certificates/fullchain.pem:ro
      - ./conf/nginx/privkey.pem:/certificates/privkey.pem:ro
    ports:
      - 80:80
      - 443:443
    depends_on:
      - ctfd

  db:
    image: mariadb:10.4.12
    restart: always
    environment:
      - MYSQL_ROOT_PASSWORD=ctfd
      - MYSQL_USER=ctfd
      - MYSQL_PASSWORD=ctfd
      - MYSQL_DATABASE=ctfd
    volumes:
      - .data/mysql:/var/lib/mysql
    networks:
        internal:
    # This command is required to set important mariadb defaults
    command: [mysqld, --character-set-server=utf8mb4, --collation-server=utf8mb4_unicode_ci, --wait_timeout=28800, --log-warnings=0]

  cache:
    image: redis:4
    restart: always
    volumes:
    - .data/redis:/data
    networks:
        internal:

networks:
    default:
    internal:
        internal: true

5.4 Modify Your Nginx Configuration

Finally, we need to specify in the conf/nginx/http.conf that we want to use the certificates and the SSL, and also, redirect any traffic from http to https. For this, we need to do the following:

5.4.1 Add the http redirect



server {
  listen 80;
  return 301 https://$host$request_uri;
}

5.4.2 Define the SSL certificates



server {
  listen 443 ssl;

  ssl_certificate /certificates/fullchain.pem;
  ssl_certificate_key /certificates/privkey.pem;
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_ciphers HIGH:!aNULL:!MD5;
  ...

In the end, the complete conf/nginx/http.conf file should look something like this:



worker_processes 4;

events {

  worker_connections 1024;
}

http {

  # Configuration containing list of application servers
  upstream app_servers {

    server ctfd:8000;
  }

  server {
    listen 80;
    return 301 https://$host$request_uri;
  }

  server {

    listen 443 ssl;

    ssl_certificate /certificates/fullchain.pem;
    ssl_certificate_key /certificates/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!MD5;

    client_max_body_size 4G;

    # Handle Server Sent Events for Notifications
    location /events {

      proxy_pass http://app_servers;
      proxy_set_header Connection '';
      proxy_http_version 1.1;
      chunked_transfer_encoding off;
      proxy_buffering off;
      proxy_cache off;
      proxy_redirect off;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Host $server_name;
    }

    # Proxy connections to the application servers
    location / {

      proxy_pass http://app_servers;
      proxy_redirect off;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Host $server_name;
    }
  }
}

6. Restart docker-compose containers

As we needed to stop all the containers to create the certificate, we need to start them again. This time (and if the previous steps worked well) we may run it in detached mode, so they can be living there even when we log out from the remote server. For this, we can execute:



# Inside the CTFd repo directory
docker-compose up -d

# If you want to be extra cautious you can run:
# docker-compose up --force-recreate -d

If everything went well, you may now see that the page has a valid certificate.

Wrap up

Even if they seem to be lots of steps, they aren't as hard as they seem in the beginning. CTFd and certbot are great projects very well maintained and you shouldn't have any issues while using them.

If this guide helped you, please consider giving it a like.
Good luck in your CTF, and happy hacking!

Export from Org to Markdown in Spacemacs

Rodrigo Medina — Sat, 10 Oct 2020 22:26:40 +0000

Org-mode is one of the coolest features in Emacs and this case Spacemacs. It can be used to write documentation, personal notes, and also schedule your events in an awesome agenda.

In my case, I use Org files to write and export tutorials from one single file to three platforms, which are Medium,Dev.to, and my blog, using Spacemacs as my main editor.

To be able to export to a Markdown from an Org file, you need to do the following:

1. Make sure you have the markdown layer installed

Having the markdown layer installed will give you a better experience when editing markdown files.

2. Add the md export backend into Spacemacs

You need to add the md value into the export backends variable. The easiest way for me to do this is the following:

Inside Spacemacs type:

M-x customize-option

and then type org-export-backends.

It will bring a customization buffer in which you can move with the arrow keys, and select the md checkbox to enable it, and then press Enter in the Appy and save button.

3. Verify the exportation

After applying and saving the new export backend, you can do a new export doing the following:

With an org file buffer open, type:

, e e

This will open the export menu, and now we will see the new markdown option available. If you want to create a markdown file from your org file, inside the previous menu type:

m m

and this will create a new markdown file from your org file.

Conclusions

From now on, whenever you want to export an org file to a markdown file, you will only need to follow step 3.

If you find this useful, please consider giving it a like.
Happy hacking!

How to Install Spacemacs in MacOS Catalina

Rodrigo Medina — Sat, 10 Oct 2020 21:36:21 +0000

I am a fan of Spacemacs, and whenever I have to install it on a new computer, I have to go to the documentation and scroll for a while. This is why I created this quick guide with all the steps I usually follow.

1. Install Emacs

As Spacemacs is based in emacs, we need to install it first. As we're using macOS, the recommended way of doing this is using the brew cask:

brew cask install emacs

2. Install Source Code Pro font

This is the font suggested for the editor, and we can do it by using the cask/fonts tap from homebrew

brew tap homebrew/cask-fonts 
brew cask install font-source-code-pro

3. Clone the Spacemacs Project

Clone the Spacemacs project, it doesn't matter where, we're going to delete it at the end (after copying it).

git clone https://github.com/syl20bnr/spacemacs

4. Checkout develop branch

As the main or master branch is a little outdated, I always use the develop branch for a better experience.

So let's get into the folder

cd spacemacs

And then change the branch that we're going to copy

git checkout develop 
# Just to make sure everything is updated
git pull origin develop

5. Copy spacemacs

We need to copy the Spacemacs folder into the ~/.emacs.d. For this, we need to get outside the Spacemacs folder and execute

cp -a spacemacs ~/.emacs.d

6. Giving permissions

As MacOS Catalina has new security features, we need to give *Full Disk Access* to the /usr/bin/ruby script which is the one that launches Spacemacs.

For this, go to System Preferences, and then into Security and Privacy.

Then, go into Privacy -> Full Disk Access, and then unlock the lock to enable changes. After that, go and click the add button.

Then you need to search for the /usr/bin/ruby script. The easiest way to do this is with the KeyBoard shortcut

SHIFT + CMD + G

Which will open a search window and then there we can type /usr/bin/ruby.

We can click accept and lock again to prevent further changes.

7. Verify the access

If you open Spacemacs and the hit

SPC f f

you should be able to go now to the desktop and any folder you want.

If this is useful for you, please consider giving it a like.
Happy hacking!

Basic Security For Your Brand New Server

Rodrigo Medina — Tue, 06 Oct 2020 04:39:36 +0000

Whenever I get a brand new server for whatever reason, I always forget what to do as I only have in mind the project that this server will use. This is why I created a basic security checklist for me, to set up the basic stuff.

Note: Most of the times I work with Debian based servers, so this checklist will assume that.

1. Update and Upgrade

As you're receiving the server for the first time, it may not be necessary to update, but it is always recommended. Also, you may want to upgrade and remove the unused packages.

For this, you can throw a oneliner:

apt update -y && apt upgrade -y && apt dist-upgrade -y && apt autoremove -y

2. Change the root password

Most of the time the first interaction will be through ssh, using the default root password. This is mostly insecure as the server will be publicly available and some bored guy will have enough time to brute force the password.

Run this command to change the password (logged as root):

passwd

I will suggest to logout with exit and then login again to verify that everything is ok, but it is optional.

3. Add another user

Having your main user to be root is not the best of the ways to handle operations. I recommend adding another user that will carry all the usual operations that do not require root access. This will prevent that if the server is compromised, at least they will not have root access.

To create a new user, run this command:

# I will use pedrito, but you can use whatever username you want
adduser pedrito

4. Reconfigure SSH Server

It is a good idea to regenerate the ssh keys, even if they are new. This can be done by reconfiguring the open-ssh server.

For reconfiguring the ssh server run:

# Reconfigure
dpkg-reconfigure openssh-server
# Restart the server
systemctl restart ssh

5. Disable password authentication in sshd_config

Having password authentication is a bad idea because as I said before, attackers will have time to brute-force passwords and we don't want that. That's why we will disable the password auth for ssh and will make available only the pub keys.

For disabling password auth:

sudo vim /etc/ssh/sshd_config

Inside the file we need to change the following:

PubkeyAuthentication yes
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no
PermitRootLogin no
PasswordAuthentication no

Then, we need to copy our ssh pubkey in the authorized_keys file.

# inside local machine
cat ~/.ssh/id_rsa.pub | pbcopy

# inside server
mkdir -p /home/pedrito/.ssh
vim authorized_keys

Inside the authorized_keys file, you should paste whatever you copy from the id_rsa.pub JUST AS IT IS.

Then, you should change the ownership of the .ssh directory, to the normal user.

chown -R pedrito:pedrito /home/pedrito/.ssh

Before restarting the system, you may want to give your user some privileges, this can be done with visudo

visudo

And inside the file:

pedrito  ALL=(ALL) NOPASSWD:ALL

Finally, restart the ssh service executing:

systemctl restart ssh

6. Validate

Validate that you can log in with your ssh keys, by doing:

ssh pedrito@ipaddress

This time it will log in automatically because we have the ssh keys. If we try on another computer, it shouldn't let us log in. Also, the only user available for login should be pedrito, i.e. if we try to login as root it should throw another error and not proceed.

Conclusions

This is the basic stuff that works for me. If you find it useful consider giving a like. Happy hacking!

How to Get Ics Files From External Shared Calendars

Rodrigo Medina — Sat, 29 Aug 2020 02:35:03 +0000

There can be some scenarios where you require to have all your shared calendars in a .ics file. In my case, it was the need to have the MLH calendar, that the staff shared to us, shared with my agenda in Spacemacs.

For doing this, I had to follow the next steps.

Get the link

Inside the configuration of the shared calendar, you have to get the calendar public URL.

Get the Calendar Id

Now you have to get src part from the url, for example, if the original url was

https://calendar.google.com/calendar/embed?src=en-gb.christian%23holiday%40group.v.calendar.google.com&ctz=Europe%2FParis

en-gb.christian%23holiday%40group.v.calendar.google.com

Join the Calendar Id with the Calendar feed

The calendar feed usual structure is the following:

https://calendar.google.com/calendar/ical/**********/public/basic.ics

You need to replace the * with the url you got in the step 2. In this case, it will end as the following.

https://calendar.google.com/calendar/ical/en-gb.christian%23holiday%40group.v.calendar.google.com/public/basic.ics

Download the ics file

Now that you have the file URL, you can download it using wget.

wget -O my_cal.ics https://calendar.google.com/calendar/ical/en-gb.christian%23holiday%40group.v.calendar.google.com/public/basic.ics

This information was hard to find for me, and if you find it useful, leave a like please. The original answer that provided this solution is found here .