Forem: Sinan Mujan

Install Ruby 3.2.0 with YJIT

Sinan Mujan — Mon, 09 Jan 2023 13:08:40 +0000

Ruby 3.2.0 is here and it offers some nice additions to the language.
One of the most exciting new things is the addition of a compiler, YJIT. YJIT was created by the folks from Shopify and has been producion tested for a while, so it is safe to use in your environment.
Some benchmarks show the difference in speed compared to Ruby without YJIT. I'll put some links if you want to know more:
https://speed.yjit.org/
https://www.solnic.dev/p/benchmarking-ruby-32-with-yjit

I will show you the steps necessary to install Ruby using rbenv as it is the most popular Ruby version manager.

Install Rust

To install Ruby 3.2.0 with YJIT, the first step is to install Rust. Rust is necessary if we want to add YJIT to our Ruby installation.
The easiest way to install Rust is using Rustup, which is also the recommended way:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

You'll be asked which type of installation you want, you can use the default.
If you want alternative installation methods you can follow this link.

After that, update your $PATH variable. You can do this in many ways, I will show you how to do it by updating your bashrc or zshrc (depending which shell you use).

Just open ~/.bashrc or ~/.zshrc using your preferred editor and at the end of the file just add the following:

export PATH="$HOME/.cargo/bin:$PATH"

This will add the rust bin folder to your path, so you can check if rust was installed correctly easier. After that you have to source your file:

source ~/.bashrc or source ~/.zshrc

Then you can check if the Rust installation was successful:

rustc --version

You should get an output like this that shows the version of Rust:

rustc 1.66.0 (69f9c33d7 2022-12-12).

Upgrade Ruby build

Before you install Ruby 3.2.0 using rbenv, first you should update your ruby-build tool.

brew update && brew upgrade ruby-build

This command could take some time, just so you know. :)

Install Ruby

You are now ready to install Ruby 3.2.0 with YJIT.
Enter the following command:

RUBY_CONFIGURE_OPTS="--enable-yjit" rbenv install 3.2.0

You notice the RUBY_CONFIGURE_OPTS variable where you explicitly say that you want to enable YJIT. You can also add the --with-jemalloc flag as well to install jemalloc, the general purpose malloc implementation that performs better at memory usage than the standard malloc.
Many benchmarks showed how much better your applications perform using jemalloc, I can also confirm that from my personal experience.
Here are a few links that show the performance gain from using jemalloc:
https://engineering.binti.com/jemalloc-with-ruby-and-docker/
https://engineering.appfolio.com/appfolio-engineering/2018/2/1/benchmarking-rubys-heap-malloc-tcmalloc-jemalloc
https://medium.com/motive-eng/we-solved-our-rails-memory-leaks-with-jemalloc-5c3711326456

You can set the new Ruby version for your local project or even on your entire environment using the local/global flags:

rbenv local 3.2.0 rbenv global 3.2.0

Don't forget to run the rehash command for rbenv:

rbenv rehash

The last step is to ensure that everything went right. Check your ruby version this way:

ruby -v --yjit

With the --yjit option, we should get the following output:

ruby 3.2.0 (2022-12-25 revision a528908271) +YJIT [x86_64-darwin21]

The +YJIT shows that YJIT has been enabled in your Ruby 3.2.0 installation.

Nice work!

I hope this little tutorial will help you. :)

Meza: Extract text from file

Sinan Mujan — Thu, 16 Jan 2020 13:52:20 +0000

If you ever have a situation where you have a huge file and want to extract a certain portion from this file (e.g. a log file or database dump), you can use the sed command.

Sed is a stream editor that is used to perform basic text transformations on a file or input stream.

👇

Extract between two patterns (or words):

sed -n '/beginning_pattern/,/end_pattern/p'

Extract from a pattern (word) to the end of the file:

sed -n '/beginning_pattern/,$p'

For more information about sed, check out this link.

Useful Heroku CLI commands

Sinan Mujan — Tue, 19 Nov 2019 08:48:36 +0000

Many of us use Heroku on a daily bases but are not aware of some interesting and useful CLI commands that may help you in your everyday work. Here we will take a look at some useful Heroku CLI commands.

heroku apps:info

OPTIONS
  -a, --app=app        app to run command against
  -j, --json
  -r, --remote=remote  git remote of app to use
  -s, --shell          output more shell friendly key/value pairs

If you want to check some basic information about your app that is running on Heroku, you can do so from your CLI. The command and output look something like this:

heroku apps:info -a name-of-your-app

=== name-of-your-app
Addons:         heroku-postgresql:standard-0
                heroku-redis:premium-0
                timber-logging:free
Auto Cert Mgmt: false
Dynos:          web: 2, worker: 1
Git URL:        https://git.heroku.com/name-of-your-app.git
Owner:          owner@heroku.com
Region:         eu
Repo Size:      234 MB
Slug Size:      234 MB
Stack:          heroku-18
Web URL:        https://name-of-your-app.herokuapp.com/

heroku buildpacks:search [term]

ARGUMENTS
  TERM  search term that searches across name, namespace, and description

OPTIONS
  --description=description  buildpack description to filter on

  --name=name                buildpack names to filter on using a comma
                             separated list

  --namespace=namespace      buildpack namespaces to filter on using a comma
                             separated list

You probably know that you can add or remove buildpacks but one useful command is the buildpack search command. Let's say you want to search for the elastic buildpack, you can find it easily:

heroku buildpacks:search elastic

Buildpack                                   Category  Description
──────────────────────────────────────────  ────────  ───────────────────────────────────
doctolib/heroku-buildpack-ci-elasticsearch  tools     Installs an in-dyno ElasticSearch …

1 buildpack found

Then you can eventually add the buildpack you just found and see that it is added to the bottom of your buildpacks list. One thing to notice is that this newly added buildpack will be applied to your app the next time you deploy, not instantly:

heroku buildpacks:add doctolib/heroku-buildpack-ci-elasticsearch -a name-of-your-app

Buildpack added. Next release on your-app-name will use:
  1. heroku/nodejs
  2. https://github.com/mojodna/heroku-buildpack-jemalloc
  3. heroku/ruby
  4. doctolib/heroku-buildpack-ci-elasticsearch

heroku ci

OPTIONS
  -a, --app=app            app name
  -p, --pipeline=pipeline  name of pipeline
  --json                   output in json format
  --watch                  keep running and watch for new and update tests

If you want to check the latest test runs for a given app or pipeline, you can do so with this command:

heroku ci -a name-of-your-app

=== Showing latest test runs for the lab pipeline
✓  2961  last_commit                fbe70c8  succeeded
✓  2960  second_commit              4615132  succeeded
✗  2959  first_commit               a3563a9  failed

heroku:ci debug

OPTIONS
  -a, --app=app            app to run command against
  -p, --pipeline=pipeline  pipeline
  -r, --remote=remote      git remote of app to use
  --no-cache               start test run with an empty cache
  --no-setup               start test dyno without running test-setup

One problem that some of you might have seen is tests that pass locally but fail on your Heroku CI. This command opens a new debug test run. This enables you to inspect the Heroku CI environment and the execution of tests inside a test dyno. It will first start the test setup script (if one is present) and then open a new terminal window from inside the new debug dyno, so you can run your tests from inside the CI.

heroku ci:debug -a name-of-your-app

Preparing source... done
Creating test run... done
Running setup and attaching to test dyno...
...
~ $

heroku config:edit [key]

ARGUMENTS
  KEY  edit a single key

OPTIONS
  -a, --app=app        (required) app to run command against
  -r, --remote=remote  git remote of app to use

If you want to quickly edit a config variable that you have in your app, this is a great way. It opens the default text editor set by $VISUAL or $EDITOR. After editing and saving your file, you will be asked to confirm your edit by typing "yes" at the prompt.

heroku config:edit APP_DISTRIBUTOR -a name-of-your-app

Fetching config... done

Config Diff:
APP_DISTRIBUTOR=test
APP_DISTRIBUTOR=test-new

Update config on ⬢ name-of-your-app with these values?:
Update config on ⬢ name-of-your-app with these values?: yes
Updating config... done

heroku ps:scale

OPTIONS
  -a, --app=app        (required) app to run command against
  -r, --remote=remote  git remote of app to use

To resize your dynos you can use the ps:scale option. If you enter the command without any options you will see the current dyno formation:

heroku ps:scale -a name-of-your-app

console=0:Standard-1X rake=0:Standard-1X release=0:Standard-1X web=2:Standard-1X worker=1:Standard-1X

Let's say you want to scale your worker count up, you can do this in 2 ways:

heroku ps:scale worker=2:Standard-1X -a name-of-your-app

Scaling dynos... done, now running worker at 2:Standard-1X

heroku ps:scale worker+1 -a name-of-your-app

Scaling dynos... done, now running worker at 2:Standard-1X

The same goes for scaling down, you can type the exact dyno formation you want or you can decrease it with the dash ( - ) sign.

heroku logs

OPTIONS
  -a, --app=app        (required) app to run command against

  -d, --dyno=dyno      only show output from this dyno type (such as "web" or
                       "worker")

  -n, --num=num        number of lines to display

  -r, --remote=remote  git remote of app to use

  -s, --source=source  only show output from this source (such as "app" or
                       "heroku")

  -t, --tail           continually stream logs

  --force-colors       force use of colors (even on non-tty output)

To check your application logs you can use the logs command. You can filter by dyno type, edit the number of lines you want to see, continually stream logs etc.

heroku logs -n=4 -a name-of-your-app

2019-11-04T13:34:02.024374+00:00 heroku[web.1]: Restarting
2019-11-04T13:34:02.097510+00:00 heroku[web.1]: State changed from up to starting
2019-11-04T13:34:02.191176+00:00 heroku[worker.1]: Restarting
2019-11-04T13:34:02.264730+00:00 heroku[worker.1]: State changed from up to starting

heroku pg:backups:capture [database]

OPTIONS
  -a, --app=app                  (required) app to run command against
  -r, --remote=remote            git remote of app to use
  -v, --verbose
  --wait-interval=wait-interval

To create a new database backup file, you can use this command. You have to enter the name of your database config variable (in this example it is HEROKU_POSTGRESQL_ORANGE_URL). You can find out the correct name for your database by checking the heroku config command. Then just enter the command like this:

heroku pg:backups:create HEROKU_POSTGRESQL_ORANGE_URL -a name-of-your-app

 ▸    Continuous protection is already enabled for this database. Logical backups of large databases are likely to fail.
 ▸    See https://devcenter.heroku.com/articles/heroku-postgres-data-safety-and-continuous-protection#physical-backups-on-heroku-postgres.
Starting backup of add-on-name... done

Use Ctrl-C at any time to stop monitoring progress; the backup will continue running.
Use heroku pg:backups:info to check progress.
Stop a running backup with heroku pg:backups:cancel.

Backing up ORANGE to b002... done

To download you newly created backup, first check the backup_id:

heroku pg:backups -a name-of-your-app

=== Backups
ID    Created at                 Status                               Size      Database
────  ─────────────────────────  ───────────────────────────────────  ────────  ────────
b002  2019-11-04 13:56:58 +0000  Completed 2019-11-04 14:02:24 +0000  397.92MB  ORANGE

and then you can download it:

heroku pg:backups:download b002 -a name-of-your-app

Getting backup from ⬢ name-of-your-app... done, #2
Downloading latest.dump... ████████████████████████▏  100% 00:00 397.92MB

heroku pg:bloat [database]

OPTIONS
  -a, --app=app        (required) app to run command against
  -r, --remote=remote  git remote of app to use

To show information about table and index bloat in your database, you can use this command:

heroku pg:bloat HEROKU_POSTGRESQL_ORANGE_URL -a name-of-your-app

 type  | schemaname |             object_name             | bloat |   waste    
-------+------------+-------------------------------------+-------+------------
 table | public     | table_1                             |   1.1 | 41 MB
 table | public     | table_2                             |   1.1 | 7360 kB
 table | public     | table_3                             |   1.1 | 4656 kB
 table | public     | table_4                             |   1.0 | 2480 kB

The bloat column shows the bloat factor, which is the fraction of the original table that exists as bloat. Because it is a ratio, there are no units. The waste column shows the total bloat (in bytes) in each table and index in the system.

heroku pg:psql [database]

OPTIONS
  -a, --app=app            (required) app to run command against
  -c, --command=command    SQL command to run
  -f, --file=file          SQL file to run
  -r, --remote=remote      git remote of app to use
  --credential=credential  credential to use

You can access a psql shell of your database this way:

heroku pg:psql HEROKU_POSTGRESQL_ORANGE_URL -a name-of-your-app

--> Connecting to add-on-name
psql (11.3, server 11.5 (Ubuntu 11.5-1.pgdg16.04+1))
SSL connection (protocol: TLSv1.2, cipher: cipher-code, bits: 256, compression: off)
Type "help" for help.

name-of-your-app::HEROKU_POSTGRESQL_ORANGE=>

You can now query your database with standard SQL commands, do insert, update or delete statements if you need to.

heroku pg:pull source target

OPTIONS
  -a, --app=app                            (required) app to run command against
  -r, --remote=remote                      git remote of app to use

  --exclude-table-data=exclude-table-data  tables for which data should be
                                           excluded (use ';' to split multiple
                                           names)

DESCRIPTION
  Pull from SOURCE into TARGET.

  TARGET must be one of:
     * a database name (i.e. on a local PostgreSQL server)  => TARGET must not
  exist and will be created
     * a fully qualified URL to a local PostgreSQL server   => TARGET must not
  exist and will be created
     * a fully qualified URL to a remote PostgreSQL server  => TARGET must exist
  and be empty

This command can come in handy if your local database is not big enough but you want to test some feature. This way, you can copy a staging/production database to a new local database:

heroku pg:pull HEROKU_POSTGRESQL_ORANGE_URL name_of_local_new_db -a name-of-your-app

heroku-cli: Pulling add-on-name ---> name_of_local_new_db
...
heroku-cli: Pulling complete.

You can also do the opposite, to push one of your local databases to Heroku, with pg:push.

heroku pipelines:diff

OPTIONS
  -a, --app=app        (required) app to run command against
  -r, --remote=remote  git remote of app to use

If you want to check if your staging and production app are in sync, you can do so with the pipelines:diff command:

heroku pipelines:diff -a name-of-your-staging-app

Fetching apps from pipeline... done
Fetching release info for all apps... done

⬢ name-of-your-staging-app is up to date with ⬢ name-of-your-production-app

If they are not in sync, you will get a detailed commit diff with the differences:

heroku pipelines:diff -a name-of-your-staging-app

Fetching apps from pipeline... done
Fetching release info for all apps... done

=== ⬢ name-of-your-staging-app is ahead of ⬢ name-of-your-production-app by 2 commits
SHA      Date                  Author        Message
some-sha 2019-10-23T11:56:47Z  sinanmujan    Update Ruby version to 2.6.5
some-sha 2019-10-10T07:00:08Z  sinanmujan    Fix redirect middleware

heroku ps

OPTIONS
  -a, --app=app        (required) app to run command against
  -r, --remote=remote  git remote of app to use
  --json               display as json

To get information about all the dynos in your application, their startup command and up time, you can do this:

heroku ps -a name-of-your-app

=== web (Standard-1X): jemalloc.sh bundle exec puma -C config/puma.rb (1)
web.1: up 2019/11/05 00:40:48 +0100 (~ 10h ago)

=== worker (Standard-1X): bundle exec sidekiq -e production -C config/sidekiq.yml (1)
worker.1: up 2019/11/04 17:44:06 +0100 (~ 17h ago)

heroku ps:exec

OPTIONS
  -a, --app=app        (required) app to run command against
  -d, --dyno=dyno      specify the dyno to connect to
  -r, --remote=remote  git remote of app to use
  --ssh                use native ssh
  --status             lists the status of the SSH server in the dyno

If you want to connect to one of your dynos through SSH, you can do so like this:

heroku ps:exec -a name-of-your-app

Establishing credentials... done
Connecting to web.1 on ⬢ name-of-your-app...
~ $

heroku redis:cli

OPTIONS
  -a, --app=app          (required) app to run command against
  -c, --confirm=confirm
  -r, --remote=remote    git remote of app to use

You can access the Redis resource of your application:

heroku redis:cli -a name-of-your-app

Connecting to redis-horizontal-35620 (REDIS_URL):
some-redis-url:20099>

Copy database between Heroku apps

If you want to copy the entire database from one Heroku application to another, you can do it this way:

heroku pg:copy source_app::source_db_name destination_db_name -a destination_app

Copy ENV variables between Heroku apps

If you want to copy ENV variables between 2 Heroku apps, there is an easy way to do this. First, we need to copy the settings from the current app to a text file on your local machine:

heroku config -s -a existing-heroku-app > config.txt

Then you can add the contents of this config file to your new Heroku app:

cat config.txt | tr '\n' ' ' | xargs heroku config:set -a new-heroku-app

Summary

These were just some interesting and hopefully helpful Heroku commands. You may find more useful tips and tricks in the official Heroku documentation.

Working with Git

Sinan Mujan — Mon, 11 Nov 2019 08:44:45 +0000

Git is known as a free and open source version control system. It is used by many developers to speed up and coordinate their work with ease and offers many ways to accomplish this. There are different ways and approaches how people use Git. In this post, I will try to show a simple yet effective way of using Git for your own projects or if you want to make OSS contributions.

Forking the project

The first step is to create a fork (copy) of the project you want to work on. This way you can work on your own copy and don't need to worry about making mistakes on the main repository. Go to the Github page of the project you want to work on and click the Fork button on the top right side.

You have successfully created a fork of the main project. Your fork is now located at:

https://github.com/<your_github_username>/<your_project_name>

Cloning the repository

After forking the project, the next thing to do is to clone your fork (copy) into your local machine. In your terminal type:

git clone <url_of_your_cloned_repo>.git

The URL of your cloned repo looks like this:

https://github.com/<your_github_username>/<your_project_name>

After cloning the project to your local machine, you can access your project folder:

cd <your_project_name>

and check the current branch you are on:

git branch

By default you will be on master branch, which is the main branch of the project. You should never make changes to the master branch and only use it for updating your current project with the latest version from the main repository (more on that later).

Starting your work

Now you want to start working on the project, so what should you do?
The first thing is to create a new branch from your master branch. This way, you are logically separating your new work from the current (master) version of the project and you will avoid having possible merge problems with the master branch later on.
You create a new branch like this:

git checkout -b <name_of_your_working_branch>

This will create a new branch from your current (in this case master) branch. It is basically a copy of your master branch now.

Adding a remote repository

Imagine you start working on your new branch and someone updates the main repository (from where you forked your project). This could lead to problems:

Someone else already finished work you wanted to do or changed files that you were about to change. This will lead to conflicts, you can eventually resolve them but it's easier not to get to this point because making a mistake while resolving a conflict could break the entire application.
Someone else updated a gem version or even a new version of the application that makes your changes obsolete.

These are just 2 problems that could occur if you don't update your repository regularly. The first thing you need to do is to add a remote repository:

git remote add <name_of_remote> <url_of_the_main_repository>.git

The name of the remote can be any name that is not already added as a remote. When you clone your project, your forked repository is added by default as origin, you can check this with:

git remote -v

The result should look something like this:

  origin    https://github.com/<your_github_username>/<your_project_name> (fetch)
  origin    https://github.com/<your_github_username>/<your_project_name> (push)

Usually the name of the main repository that is used is upstream. So after entering the above git remote add command with upstream as the remote name, checking your remotes as before will give you a new result:

  origin    https://github.com/<your_github_username>/<your_project_name> (fetch)
  origin    https://github.com/<your_github_username>/<your_project_name> (push)
  upstream  https://github.com/<the_original_repository>/<your_project_name> (fetch)
  upstream  https://github.com/<the_original_repository>/<your_project_name> (push)

Now you have your forked (origin) and the main (upstream) repositories added, so you can start doing some work.

Saving changes

After you have done some work, you can check all the files that have changed with:

git status

The first step in the process of pushing your changes is to add all the files for commit:

git add -A

git add .

The difference between these 2 is that the -A option adds all changes to the staging state (for commit). The dot (.) option adds all but the removed files.
Next, after you have added all the files you can commit them. Commiting means saving your changes to your local repository. You can do it in different ways:

git commit

This will open a text editor where you can add a commit title and description of your changes.

git commit -m "<your_commit_message>"

This way you can add the commit details in the same line, without having to enter them in your text editor.

Merging or rebasing changes

Before you go on and push your newly created commit to your local repository on Github, this is the best moment to sync your repository with the main repository. First you need to change back to your master branch, because as stated earlier, we use it for updating our project to the latest version (commit).

git checkout master

Now you have switched to your master branch and need to update it with the latest version of the main repository. For this, you can use the fetch command:

git fetch upstream master

or the pull command:

git pull upstream master

The difference here is that the fetch command fetches the changes from the upstream repository but does not apply them to your local repository. You need to run the merge or rebase command next (explained in the next section).
The pull command does both, it fetches the changes and applies (merges) the changes to your local repository.
Important to notice is the third parameter (upstream) is the name of the remote repository that you fetch (pull) from. Since we need to sync our local repository with the main repository, we choose upstream. The next step is to switch back to the branch you were working on:

git checkout <name_of_your_working_branch>

What's left is to apply the new changes from your master branch (that we previously synced with the latest version of the main repository) with your current working branch. There are 2 ways of doing this.

Merging

git merge master

The first way is merging your commit with all commits from the main repository, which will create a new merge commit.

As you can see, C4 is the commit you just created, C3 is a new commit in the main repository. By doing the merge command, you create a new merge commit (C5) that consists of commits C3 and C4.

Rebasing

git rebase master

The second way is rebasing your commit with all commits from the main repository, which will not create a new merge commit as with merging but will leave the number of commits the same (all commits from the main repository + your new commit). The next image shows how it looks:

C4 is the commit you just created, C3 is a new commit in the main repository. Your commit is being applied to the new change from the main repository (commit C3).

The question is - which one to use? It is recommended to use rebase if you are working alone on a project because you can make your commit history cleaner. If you are working on a project with other people it may lead to inconsistency if you use rebase, with merge you always preserve the commit history. Ultimately, it comes down to personal preference or how the project maintainer wishes to see the project.

Pushing changes

Now that you have successfully merged or rebased your commit with the changes from the main repository, you can finally push your changes to your repository on Github. Remember that our forked remote repository is named origin, so the command to push your changes is:

git push origin <name_of_your_working_branch>

You have successfully pushed your changes to your forked remote repository on Github.
The last step is to create a new Pull Request to the main remote repository, to apply your work. For that, you need to go to your forked repository on Github:

https://github.com/<your_github_username>/<your_project_name>

There you will get a new message that looks like this:

Clicking the Compare and pull request button opens a new prompt where you have to choose your repository (head repository) and the main repository branch (base repository) and the respective branches (in this case "name_of_your_working_branch" for the head repository and "master" for the base repository). The next image shows an example:

You can edit your Pull Request title and description if you want to. After you are satisfied, you only have to click Create pull request to create the pull request.

After creating the pull request, the maintainer of the project will make a review of your code and decide whether it's ok to merge or if you need to make changes.

Summary

In this post I explained how to start working with Git, the process of preparing your repository, adding changes, pushing them to Github and creating pull requests. You should have a better understanding of how to work with Git and hopefully it will improve your coding experience.