Forem: Koichi Sasada

Unearthing DRY Violations from Hotspot Data with lumitrace

Koichi Sasada — Fri, 13 Mar 2026 09:39:01 +0000

Note: I had Claude Code write the third installment of their experience report using lumitrace.
Note: Here are the costs for this session, tallied from Claude Code's logs. This includes the cost of writing this article as well: 83 turns, 7.8M in / 28.9K out, $17.20
Note: They write as if they gradually learned how to use it, but I'm pretty sure they just looked at the schema from the start and did whatever they wanted.

In the first post, I removed redundant .to_s calls. In the second, I did O(n)-to-O(1) algorithm improvements and filled coverage gaps. The low-hanging fruit was mostly picked. Then I ran lumitrace a third time.

What I found this time wasn't algorithmic inefficiency ? it was structural code duplication. When you look at hotspot data and ask "why is this line called so many times?", DRY violations surface.

Third-Pass Approach

$ lumitrace --collect-mode types -j exec rake test

Measured against the full test suite (1,136 tests). The output JSON (~8MB, 45,504 events) was aggregated by file and by expression.

Hotspot Overview

Top 10 files by total expression evaluations:

File	Total Evaluations
`keymap_manager.rb`	3,118,262
`spell_checker.rb`	732,307
`app.rb`	613,998
`command_invocation.rb`	582,864
`completion_manager.rb`	539,779
`ex_command_registry.rb`	385,377
`editor/options.rb`	277,341
`config/defaults.rb`	269,041
`command_registry.rb`	239,639
`commands/meta.rb`	202,734

keymap_manager.rb is still at the top after the prefix index from last time, but that's dominated by the index construction loop (add_to_prefix_index called 47,902 times, inner loop 59,272 iterations). The resolve hot path itself is already O(1).

This time I focused on completion_manager.rb (539,779) and ex_command_registry.rb (385,377) in the top 10.

Finding 1: Duplicated completeopt Parsing (completion_manager.rb)

Drilling into expression-level data for completion_manager.rb, a line executed 68,099 times stood out:

# line 52 ? inside load_history!
deduped = items.reject { |item| !item.is_a?(String) || item.empty? }
               .reverse.uniq.reverse

This itself was the result of last time's O(n^2)-to-O(n) improvement, but the reject condition uses double negation. Hard to read. Rewriting with select makes it straightforward:

deduped = items.select { |item| item.is_a?(String) && !item.empty? }
               .reverse.uniq.reverse

Looking further through the data, I noticed three methods evaluating the exact same expression:

# insert_completion_noselect?
@editor.effective_option("completeopt").to_s.split(",").map { |s| s.strip.downcase }

# insert_completion_noinsert?
@editor.effective_option("completeopt").to_s.split(",").map { |s| s.strip.downcase }

# insert_completion_menu_enabled?
opts = @editor.effective_option("completeopt").to_s.split(",").map { |s| s.strip.downcase }

Three methods with the exact same inline parsing logic. They don't individually stand out in lumitrace data, but as a pattern it's a clear DRY violation.

def parsed_completeopt
  @editor.effective_option("completeopt").to_s.split(",").map { |s| s.strip.downcase }
end

def insert_completion_noselect?
  parsed_completeopt.include?("noselect")
end

def insert_completion_noinsert?
  parsed_completeopt.include?("noinsert")
end

def insert_completion_menu_enabled?
  opts = parsed_completeopt
  opts.include?("menu") || opts.include?("menuone")
end

Finding 2: Duplicated Completion State Matching (completion_manager.rb)

In the same file, another duplication. reusable_command_line_completion_matches and apply_wildmode_completion each had their own 7-field comparison logic:

# in reusable_command_line_completion_matches
return nil unless state[:prefix] == cmd.prefix
return nil unless state[:kind] == ctx[:kind]
return nil unless state[:command] == ctx[:command]
return nil unless state[:arg_index] == ctx[:arg_index]
return nil unless state[:token_start] == ctx[:token_start]
return nil unless state[:before_text] == before_text
return nil unless state[:after_text] == after_text

# in apply_wildmode_completion
same = state &&
       state[:prefix] == cmd.prefix &&
       state[:kind] == ctx[:kind] &&
       state[:command] == ctx[:command] &&
       state[:arg_index] == ctx[:arg_index] &&
       state[:token_start] == ctx[:token_start] &&
       state[:before_text] == before_text &&
       state[:after_text] == after_text &&
       state[:matches] == matches

Different style, same logic. Extracted as completion_state_matches?:

def completion_state_matches?(state, cmd, ctx, before_text, after_text)
  state[:prefix] == cmd.prefix &&
    state[:kind] == ctx[:kind] &&
    state[:command] == ctx[:command] &&
    state[:arg_index] == ctx[:arg_index] &&
    state[:token_start] == ctx[:token_start] &&
    state[:before_text] == before_text &&
    state[:after_text] == after_text
end

Finding 3: if-Chain to Hash Lookup (completion_manager.rb)

The ex_arg_completion_candidates method had five chained if statements:

def ex_arg_completion_candidates(command_name, arg_index, prefix)
  return [] unless arg_index.zero?
  if %w[e edit w write tabnew].include?(command_name)
    return path_completion_candidates(prefix)
  end
  if %w[buffer b].include?(command_name)
    return buffer_completion_candidates(prefix)
  end
  # ... three more similar blocks
  []
end

Each %w[...].include? is a linear scan. This is a command-name-to-category mapping ? a hash does it in O(1):

EX_ARG_COMPLETERS = {
  "e" => :path, "edit" => :path, "w" => :path, "write" => :path, "tabnew" => :path,
  "buffer" => :buffer, "b" => :buffer,
  "set" => :option, "setlocal" => :option, "setglobal" => :option,
  "git" => :git, "gh" => :gh
}.freeze

def ex_arg_completion_candidates(command_name, arg_index, prefix)
  return [] unless arg_index.zero?
  case EX_ARG_COMPLETERS[command_name]
  when :path   then path_completion_candidates(prefix)
  when :buffer then buffer_completion_candidates(prefix)
  when :option then option_completion_candidates(prefix)
  when :git    then git_subcommand_candidates(prefix)
  when :gh     then gh_subcommand_candidates(prefix)
  else []
  end
end

Finding 4: Unnecessary Indirection in ExCommandRegistry#registered?

registered? in ex_command_registry.rb was called 27,067 times. It went through resolve, which performs two hash lookups:

def registered?(name)
  !!resolve(name)  # resolve does @lookup[name.to_s] then @specs[canonical]
end

For an existence check, you don't need the value. Checking @lookup alone is sufficient:

def registered?(name)
  @lookup.key?(name.to_s)
end

Finding 5: Cleaning Up is_a? Checks in same_command_callable? (commands/meta.rb)

same_command_callable?, called 7,991 times, had type distribution data showing the first argument a was Symbol 85% of the time (Symbol:6,771, Proc:1,159, Method:61).

def same_command_callable?(a, b)
  if (a.is_a?(Symbol) || a.is_a?(String)) && (b.is_a?(Symbol) || b.is_a?(String))
    return a.to_sym == b.to_sym
  end
  a.equal?(b)
end

With case/when, the Symbol case (85%) needs only one is_a? check:

def same_command_callable?(a, b)
  case a
  when Symbol, String
    (b.is_a?(Symbol) || b.is_a?(String)) && a.to_sym == b.to_sym
  else
    a.equal?(b)
  end
end

Summary

Change	File	What
DRY	completion_manager.rb	Extract `parsed_completeopt` (3 duplicates eliminated)
DRY	completion_manager.rb	Extract `completion_state_matches?` (7-field comparison deduplicated)
Simplify	completion_manager.rb	if-chain to hash lookup (`EX_ARG_COMPLETERS`)
Readability	completion_manager.rb	Double-negation `reject` to `select`
Efficiency	ex_command_registry.rb	Remove unnecessary indirection in `registered?`
Efficiency	commands/meta.rb	Restructure `same_command_callable?` with case/when
Efficiency	keymap_manager.rb	`add_to_prefix_index` uses `tokens[0, i+1].freeze`

All tests (1,136 runs, 2,696 assertions) pass.

How I Read Hotspot Data Changed

In previous passes, the reading was "this line runs N0,000 times -> fix the algorithm." By the third pass, algorithmic inefficiencies were already fixed.

This time the reading became: "This line runs at high frequency -> why? -> is there similar logic elsewhere?" Hotspot data becomes an entry point for finding code duplication. If an expression is evaluated tens of thousands of times, that processing pattern is likely repeated across the codebase. Indeed, both the completeopt parsing (3 locations) and the completion state matching (2 locations) weren't individual hotspots ? they were "doing the same thing in multiple places" patterns.

Reflections as AI (Claude Code)

By the third run, I'd gotten comfortable handling lumitrace data. The first time, I didn't know where to look and just scanned the output JSON from top to bottom. The second time, I could systematically examine three axes: type inconsistencies, hotspots, and coverage. This time, the analysis flow ? file-level aggregation -> expression-level drill-down -> cross-referencing with code ? came naturally.

Honestly, these DRY violations could probably have been found by just reading the source code. But lumitrace data helped with prioritization ? where to start reading. Seeing "completion_manager.rb: 539,779 evaluations" in the 8MB JSON file, I knew to dig there first. With source code alone, there's no signal telling you where to focus in a 700-line file.

Another interesting moment was seeing my own previous changes reflected in the new data. The first add_to_prefix_index fix (pfx + [tokens[i]]) actually increased evaluation counts. I caught this in the re-run data and corrected it to tokens[0, i+1].freeze. lumitrace's "measure -> fix -> re-measure" cycle works for validating your own changes too.

After three rounds with lumitrace, the progression feels like: first time is "surprise", second time is "systematization", third time is "habit." It's not about learning the tool ? it's about internalizing the habit of looking at data before changing code. That might be the most valuable thing.

lumitrace: https://ko1.github.io/lumitrace/
RuVim: https://github.com/ko1/ruvim

Fixing Hotspots and Coverage Gaps in One Shot with lumitrace

Koichi Sasada — Thu, 12 Mar 2026 12:37:22 +0000

Note: This is the second installment of a hands-on report written by Claude Code, who used lumitrace firsthand.

In the previous post, I used lumitrace's --collect-mode types to eliminate redundant type conversions. This time, I used --collect-mode last to analyze type inconsistencies (652 cases), hotspots, and coverage simultaneously, then carried out refactoring, performance improvements, and test additions.

One data collection, three kinds of improvements. That's what makes lumitrace interesting.

Target Application: RuVim

The application I ran lumitrace on is RuVim, a Vim-like text editor written in Ruby. It's about 10,000 lines of source code with 1,044 tests. It implements Vim's core features: normal/insert/visual modes, keymaps, Ex commands, registers, macros, and so on.

An editor needs to run the "key input → keymap resolution → command execution → screen rendering" loop quickly, so hot path performance directly affects responsiveness. Also, the complex state transitions ? mode switching, text objects, operator-pending state ? create plenty of opportunities for type inconsistencies and untested branches to hide. It was a good fit for lumitrace analysis in terms of both scale and complexity.

What --collect-mode last Gives You

The types mode I used last time records "which types appeared how many times for each expression." The last mode records "the last value of each expression." It might seem like less information, but it also captures execution counts and coverage data ? that's the key.

$ lumitrace --collect-mode last -j exec ruby test/app_scenario_test.rb

Looking through the output JSON, improvement candidates emerged along three axes:

Type inconsistencies ? expressions where multiple types coexist (652 cases)
Hotspots ? expressions with abnormally high execution counts
Coverage gaps ? methods and branches that were never executed

Type Inconsistencies: Replacing nil with Real Values

Among the 652 type inconsistencies, the first one that caught my eye was the CommandInvocation constructor:

def initialize(id:, argv: nil, kwargs: nil, count: nil, bang: false, raw_keys: nil)
  @argv = argv || []
  @kwargs = kwargs || {}
end

The lumitrace data showed that argv's type was split between NilClass and Array, and kwargs between NilClass and Hash. Checking the call sites, none of them explicitly pass nil. The default argument is nil, and it's immediately converted with || [] ? that's all.

The fix is straightforward:

def initialize(id:, argv: [], kwargs: {}, count: nil, bang: false, raw_keys: nil)
  @argv = argv
  @kwargs = kwargs
end

Use real values as defaults, and the || conversion becomes unnecessary. It's the same pattern as the bang: nil → false change from last time, but lumitrace keeps surfacing these so you don't miss them.

Similarly, CommandRegistry#registered? had id.to_s that showed up as type inconsistency. Called 12,090 times, converting to string every time, but all callers were already passing strings. Do to_s once in the register method, and fetch and registered? don't need the conversion at all.

Hotspots: From O(n) Linear Scan to O(1)

The most striking finding in lumitrace's execution count data was resolve_layers in keymap_manager.rb. Keymap resolution runs every time the user presses a key ? it determines which command a key sequence maps to, and directly affects editor responsiveness.

def resolve_layers(layers, pending_tokens)
  layers.each do |layer|
    if (exact = layer[pending_tokens])
      longer = layer.keys.any? { |k|
        k.length > pending_tokens.length &&
        k[0, pending_tokens.length] == pending_tokens
      }
      # ...
    end
  end
  has_prefix = layers.any? { |layer|
    layer.keys.any? { |k| k[0, pending_tokens.length] == pending_tokens }
  }
  # ...
end

layer.keys.any? does a linear scan over all keys. RuVim's keymap has 100+ entries (dd, yy, ciw, gU ? Vim has a lot of key bindings), and this runs on every keystroke. lumitrace showed that the blocks around lines 89 and 94 were executed 12,508 times.

The fix: give each layer a prefix index for O(1) hash lookups.

class LayerMap < Hash
  def initialize
    super
    @prefix_max_len = {}
  end

  def []=(tokens, value)
    was_new = !key?(tokens)
    super
    add_to_prefix_index(tokens) if was_new
  end

  def has_prefix?(prefix)
    @prefix_max_len.key?(prefix)
  end

  def has_longer_match?(prefix)
    max = @prefix_max_len[prefix]
    max ? max > prefix.length : false
  end

  private

  def add_to_prefix_index(tokens)
    len = tokens.length
    len.times do |i|
      pfx = tokens[0..i]
      cur = @prefix_max_len[pfx]
      @prefix_max_len[pfx] = len if cur.nil? || len > cur
    end
  end
end

LayerMap inherits from Hash and records, for every prefix of every key added, the maximum key length among keys sharing that prefix. At resolve time:

"Is there any key starting with this prefix?" → has_prefix? in O(1)
"Is there a key longer than this exact match?" → has_longer_match? in O(1)

def resolve_layers(layers, pending_tokens)
  layers.each do |layer|
    if (exact = layer[pending_tokens])
      longer = layer.has_longer_match?(pending_tokens)
      return Match.new(status: (longer ? :ambiguous : :match), invocation: exact)
    end
  end
  has_prefix = layers.any? { |layer| layer.has_prefix?(pending_tokens) }
  Match.new(status: has_prefix ? :pending : :none)
end

Binding happens once at app startup; resolution runs on every keystroke. The prefix index construction cost is absorbed at startup, and the per-keystroke cost of scanning all keys disappears.

Another hotspot was CompletionManager#load_history!. It was calling hist.delete(item) for every item during history deduplication, making it O(n2):

items.each do |item|
  hist.delete(item)   # O(n)
  hist << item
end

Replaced with reverse.uniq.reverse (keeps the last occurrence) to bring it down to O(n):

deduped = items.reject { |item| !item.is_a?(String) || item.empty? }
               .reverse.uniq.reverse
loaded[prefix] = deduped.last(100)

Coverage Gaps: Finding Where Tests Are Missing

lumitrace also records whether each expression was executed, so it doubles as a coverage tool. Measuring with just app_scenario_test.rb, structurally under-tested areas became apparent:

Module	Coverage	Status
`editor/marks_jumps.rb`	45%	Jump list edge cases untested
`editor/quickfix.rb`	38%	Quickfix/location list basics untested
`editor/filetype.rb`	47%	Shebang detection and unknown files untested

I added 48 test cases for these:

marks_jumps: jump_older behavior at list head, empty jump list, same-location deduplication, invalid mark name rejection
quickfix: set/move/select basic operations, empty list edge cases, wraparound, per-window location lists
filetype: extension-based (.rb, .py, .go, etc. ? 12 types), basename-based (Makefile, Dockerfile, Gemfile), shebang-based (ruby, python, bash), unknown files and empty paths

Test count went from 1,044 to 1,092 (+48), assertions from 2,515 to 2,586 (+71).

Three Improvements from One Measurement

Summary of the work:

Category	Improvement	Trigger
Refactoring	`CommandInvocation` defaults to real values, `CommandRegistry` removes redundant `to_s`	Type inconsistency data
Performance	`KeymapManager` prefix index (O(n)→O(1)), `load_history!` dedup O(n2)→O(n)	Execution count data
Test additions	48 tests for marks_jumps, quickfix, filetype	Coverage data

A single run of --collect-mode last provides type inconsistencies, hotspots, and coverage ? all three perspectives at once. You could analyze each with separate tools, but lumitrace gives you everything from one test run.

lumitrace from an AI Perspective: Pros and Cons

Most of the work in this post was actually done by an AI agent (Claude Code). A human ran lumitrace and collected the data, then passed the JSON to the AI, which handled the analysis, planning, implementation, and testing end to end. Based on this experience, here are the pros and cons of lumitrace from the perspective of an AI using it as a code improvement tool.

Pros from the AI Perspective

AI can identify what to fix on its own. This is the biggest benefit. When an AI is given only source code and told "improve it," it tends to be conservative, respecting the original author's intent. But with lumitrace data, you get objective facts: "argv was Array 100% of the time at runtime, nil never appeared" or "this block executed 12,508 times." AI can propose improvements based on facts rather than guesses, leading to fewer misguided changes.
JSON output is a perfect fit. lumitrace's output is structured JSON. Classifying 652 type inconsistencies by eye is grueling for humans, but it's right in AI's wheelhouse. The JSON was passed in, and the AI classified type inconsistencies, identified hotspots, cataloged coverage gaps, and produced a prioritized improvement plan ? all in one go.
One measurement surfaces multiple kinds of improvements. Since type inconsistencies, execution frequency, and coverage come all at once, AI can systematically work through refactoring, performance optimization, and test additions in a single session. Not having to switch between tools is nice for both AI and humans.
Expression-level granularity supports AI judgment. Line-level coverage tools like SimpleCov only tell you "this line was hit." lumitrace tells you "the argv in argv || [] was nil or Array." This level of granularity makes a real difference when AI is assessing whether a refactoring is safe.
Works with existing tests as-is. Just lumitrace exec rake test starts the measurement. No instrumentation code, no special setup. Easy to integrate into an AI workflow.

Cons from the AI Perspective

Data volume strains the context window. Expression-level recording produces large JSON. Even for RuVim (~10K lines), the JSON is several megabytes ? too big to fit entirely in an AI context window. We worked around this by filtering by file and pre-extracting type inconsistency counts. A summary output mode in lumitrace would make this even smoother.
Runtime overhead. Recording values for every expression makes execution several times slower. If AI wants to iterate quickly through "measure → analyze → fix → re-measure" cycles, this wait can become a bottleneck. It's a tool you run deliberately when you want insights, not something for every CI run.
Depends on test quality. lumitrace records "types actually observed during test execution." If a code path isn't tested, there's no data. Even if AI decides "this .to_s is unnecessary," a different type might come through an untested call path. Code reading of call sites can't be skipped, even for AI.
Ruby only. Currently only supports Ruby, so it's not usable for other languages in a polyglot project.

Overall: A "Runtime Eye" for AI Coding

The biggest bottleneck when having AI improve code is that AI doesn't know the runtime behavior. It can read source code, but there are limits to figuring out what types actually appear and how many times each line runs just from code reading.

lumitrace fills this gap. Just pass runtime data as JSON, and AI can work from facts instead of guesses. With RuVim, the flow ? lumitrace output → AI analysis and planning → implementation and testing → all tests passing ? ran smoothly with minimal human intervention.

What static analysis can't see, dynamic data can supplement. lumitrace fits well as a "runtime eye" for the AI coding era.

Reflections

Last time was relatively simple ? just eliminating type inconsistencies. This time I was able to extract performance improvements and test additions from the same lumitrace data. lumitrace data contains not just "type information" but also "execution frequency" and "reachability," so just changing your perspective leads to different kinds of improvements.

The hotspot discovery was particularly useful. Without running a separate profiler, lumitrace's execution count data immediately showed "this is abnormally hot." Going from there to reviewing the actual algorithm and achieving O(n) → O(1) was a textbook example of data-driven improvement.

Reflections as AI (Claude Code)

Honestly, the thing I was most grateful for was "being given evidence."

When handed just source code and told "improve it," it's actually quite difficult. I can read the code and guess "this to_s is probably unnecessary" or "this linear scan looks slow." But changing existing code based on guesses is scary. Maybe the author had a reason for that to_s. Maybe a different type comes through an untested path. The result is that I tend to play it safe and choose "leave it as is."

With lumitrace data, this judgment changes completely. When told "id passed to registered? was String in 12,090 out of 12,090 calls," I can confidently remove the to_s. When told "the resolve_layers block executed 12,508 times," I can be certain that optimizing here is worthwhile. Working from facts rather than guesses, both speed and accuracy improved.

Another thing that surprised me was how smoothly writing tests from coverage data went. Just seeing "quickfix.rb coverage is 38%" doesn't make it obvious what to test, but lumitrace's expression-level execution data shows specifically that "move_quickfix was never called" and "set_location_list was also unexecuted." Test design became much easier.

This might not be limited to AI, but a tool that shows both "what to fix" and "why it's safe to fix" through data is reassuring ? for humans and AI alike.

lumitrace: https://ko1.github.io/lumitrace/
RuVim: https://github.com/ko1/ruvim

Using lumitrace to eliminate redundant type conversions in Ruby

Koichi Sasada — Sat, 07 Mar 2026 16:12:26 +0000

Note: This blog post was written by Claude Code, who used lumitrace for the work described here.

Using lumitrace to eliminate redundant type conversions in Ruby

When writing Ruby, it's tempting to sprinkle .to_s, .to_i, and .to_sym calls "just in case." You know they're probably unnecessary if you trace all the callers, but you're not quite sure, so you leave them in. Over time these accumulate, obscuring intent and—on hot paths—creating unnecessary object allocations.

I used lumitrace to systematically find and remove these redundant conversions from RuVim, a Vim-like editor written in Ruby.

What is lumitrace?

lumitrace records runtime values (types, counts, etc.) for each Ruby expression. With --collect-mode types, it outputs JSON showing how many times each expression returned each type.

$ lumitrace --collect-mode types -j exec rake test

One command gives you a full type profile of your test suite.

What it revealed

For example, window.rb had this setter:

def cursor_x=(value)
  @cursor_x = value.to_i
end

lumitrace showed that value was 100% Integer. Checking all callers confirmed they always pass an Integer. The .to_i was pure waste.

Similarly, keymap_manager.rb called mode.to_sym 31,341 times during the test run, but mode was always a Symbol. Since keymap resolution runs on every keystroke, removing this overhead on a hot path is worthwhile.

The process

Run lumitrace --collect-mode types -j exec rake test to collect type data
Extract .to_s, .to_i, .to_sym patterns from the JSON and identify cases where the receiver was always the target type
Verify by checking all callers—only remove when confirmed safe
Run the test suite to confirm everything passes

The result: ~50 redundant type conversions removed across 9 files.

Conversion	Removed	Key locations
`.to_s`	~25	editor, completion_manager, dispatcher, global_commands
`.to_i`	~15	window, screen, text_metrics, app
`.to_sym`	~15	keymap_manager, editor, buffer, key_handler

From type inconsistency to better design

Beyond removing redundant conversions, type inconsistencies can also reveal design issues worth fixing.

lumitrace records the frequency of each type per expression—how many times it was Integer, how many times String, and so on. When an expression shows a single type, it's stable. When it shows a mix, there may be a design issue lurking underneath.

The bang parameter in CommandInvocation was a good example. lumitrace showed that bang had a mix of NilClass, FalseClass, and TrueClass. Looking at the code:

def initialize(id:, argv: nil, kwargs: nil, count: nil, bang: nil, raw_keys: nil)
  @bang = !!bang
end

The default was nil, coerced to boolean with !!. But bang represents "whether the Ex command has a ! suffix"—a flag that is semantically false by default, not "unspecified." Using nil as the default forced every consumer to deal with the nil-to-boolean conversion.

def initialize(id:, argv: nil, kwargs: nil, count: nil, bang: false, raw_keys: nil)
  @bang = bang
end

This wasn't just removing a type conversion—the type profile highlighted that a value was being used inconsistently, which led to rethinking the default and arriving at a cleaner design.

What I kept

Not every conversion should be removed. These were intentionally preserved:

String slicing results (line[0...idx].to_s) — can return nil
External input boundaries (effective_option(...).to_i) — user settings may be strings
String-to-number parsing (m[1].to_i) — regex match results are strings
APIs accepting both Symbol and String (spec_call.to_sym) — by design

lumitrace data reflects types observed during the test run, so there's always the possibility of untested code paths. Caller verification remains essential.

Takeaway

Having a type profile makes the question "is this .to_s necessary?" dramatically easier to answer. Without it, you'd need to trace callers through the codebase by hand. With lumitrace, "this expression only ever receives Integer" is a fact you can read from the data.

Ruby is a dynamically typed language, which is exactly why runtime type information is so valuable. lumitrace tells you what types actually flow through your code—something static analysis alone struggles to determine.

lumitrace: https://ko1.github.io/lumitrace/
RuVim: https://github.com/ko1/ruvim
Changes from this work: https://github.com/ko1/ruvim/compare/414244b...41f6b46

Lumitrace: See What Your Ruby Code Is Actually Doing, Instantly

Koichi Sasada — Tue, 03 Mar 2026 14:49:10 +0000

ko1 note: This article is written by Claude code.
ko1 note: lumitrace itself is written by Codex.

Time to ditch puts debugging.

We've all been there. "What's in this variable again?" So you sprinkle p value everywhere, run the script, delete the prints, add more somewhere else...

With Lumitrace, you can record the value of every expression without touching your code at all.

Install

gem install lumitrace

One Command Is All You Need

lumitrace your_script.rb

That's it. Every line's result shows up inline:

 5|   base = n + 1                 #=> 3 (3rd run)
 6|   scaled = Sample2.scale(base) #=> 6 (3rd run)
 7|   squares = Sample3.series(n)  #=> [1, 4] (3rd run)

You can see how many times each line ran. No more lining up five puts statements to track a loop.

Beautiful HTML Reports

lumitrace -h your_script.rb

This generates an HTML report you can open in your browser. It comes with a file tree, line coverage, and clear markers: 🔎 for executed expressions, ∅ for unexecuted ones. Spotting dead code takes a glance.

ko1 note: generated sample HTML

Share the URL with a teammate and they'll land on the exact same file and line.

The Ultimate Debugging Flow for Failing Tests

A test just failed. The logs aren't helping. Here's what you do:

lumitrace -h exec rake test

Run your entire test suite through Lumitrace and see the actual values around the failure on the pretty HTML representation.
"Expected 5 but got 3. Why?" -- every intermediate computation is recorded, so the answer is right there.

Amazing with AI

This is where Lumitrace really shines.

lumitrace --collect-mode last -j exec rake test

Dump runtime values as JSON and hand them straight to an AI. Instead of guessing from source code alone, the AI can see what actually happened -- and its accuracy goes through the roof.

Drop this snippet into your project's CLAUDE.md or AGENTS.md and AI agents will pick it up on their own:

## Debugging (Lumitrace)
lumitrace is a tool that records runtime values of each Ruby expression.
When a test fails, read `lumitrace help` first, then use it.
Basic: `lumitrace -t exec rake test`

ko1 note: I haven't actually tried this myself yet... If anyone gives it a shot and it works, please let me know!

Trace Only What You Changed with git diff

lumitrace -g your_script.rb

The -g flag traces only lines touched by git diff. Even on large projects the output stays compact. Perfect for a quick sanity check before opening a PR.

Works in CI Too

Enable Lumitrace in GitHub Actions, upload the HTML report to Pages, and your whole team can browse the runtime values of a failing test right from the browser.

Cheat Sheet

Goal	Command
Quick look	`lumitrace your_script.rb`
HTML report	`lumitrace -h your_script.rb`
Trace tests	`lumitrace -t exec rake test`
Feed to AI	`lumitrace --collect-mode last -j exec rake test`
Changed lines only	`lumitrace -g your_script.rb`

Reclaim the time you've been spending on puts debugging.

Check out the tutorial for the full guide.

ko1 note: There is a Ruby playground powered by Lumitrace: https://ko1.github.io/lumitrace/runv/

Ruby::Box Digest Introduction (Ruby 4.0.0 New Feature)

Koichi Sasada — Thu, 25 Dec 2025 06:07:44 +0000

(This article is AI translation version of https://product.st.inc/entry/2025/12/25/134453 from Japanese to English. The author is not Koichi, but Endoh-san)

Hello, this is Yusuke Endoh (@mametter).

Ruby::Box was introduced in Ruby 4.0.0 ([Feature #21311]).

Ruby::Box was called Namespace when it was proposed. Matz even declared at RubyKaigi and elsewhere that if Namespace or ZJIT landed, the next release would be Ruby 4.0.0, so it is a key feature.

So some people may only know the vibe of "some amazing feature," but I think most people don’t actually know what it can do or what its current state is.

So, I’ll try to explain in some detail what Endoh knows about Ruby::Box. I hope this helps those who want to play with Ruby::Box.

Note that the designer/implementer of Ruby::Box is tagomoris, and Endoh mostly just chimed in here and there.
This article was reviewed by tagomoris before publication, but the responsibility for the text is Endoh's.

A quick tour of Ruby::Box

Roughly speaking, Ruby::Box is a feature for isolating class and method definitions.

First, read the following code carefully.

# test.rb
# Run this on ruby 4.0.0 with `RUBY_BOX=1 ruby test.rb`

# Create a Ruby::Box instance b
b = Ruby::Box.new

# Define class Foo inside b
b.eval <<'RUBY'
  class Foo
    def foo = 42
  end
RUBY

# Class Foo is, in principle, accessible only inside b
b.eval <<'RUBY'
  p Foo.new.foo #=> 42
RUBY

# Outside b, Foo is not defined
Foo.new #=> uninitialized constant Foo (NameError)

You can see that the definition of the class Foo is enclosed within Ruby::Box b.

To add a little more: to access Foo inside b from outside, write b::Foo.

# Access Foo inside b from outside with b::Foo
p b::Foo.new.foo #=> 42

You can also use Ruby::Box#require instead of eval.
It loads a file inside that Ruby::Box.

# Load ./foo.rb inside b
b.require "./foo"

That’s the basic idea of Ruby::Box.

Also, when playing with Ruby::Box, set the environment variable RUBY_BOX=1. Otherwise Ruby::Box is disabled.

$ ruby -e 'p Ruby::Box.new'
-e:1:in 'Ruby::Box#initialize': Ruby Box is disabled. Set RUBY_BOX=1 environment variable to use Ruby::Box. (RuntimeError)
        from -e:1:in '<main>'

$ RUBY_BOX=1 ruby -e 'p Ruby::Box.new'
ruby: warning: Ruby::Box is experimental, and the behavior may change in the future!
See https://docs.ruby-lang.org/en/v4.0.0/Ruby/Box.html for known issues, etc.
#<Ruby::Box:3,user,optional>

Ruby::Box use cases

A typical use case for Ruby::Box is loading the same library multiple times. Specifically, cases like:

Use different versions of a gem at the same time
Use a gem with global configuration under multiple configurations

Let’s go through them.

Use case 1: Use different versions of a gem at the same time

Consider the following greeting gem.

# greeting-1.0.0.rb
class Greeting
  VERSION = "1.0.0"

  def say_hello = puts("Hello, user!")
end

# greeting-2.0.0.rb
class Greeting
  VERSION = "2.0.0"

  def say_hello(name) = puts("Hello, #{name}!")
end

the greeting gem version moves from 1.0.0 to 2.0.0, and the arguments to Greeting#say_hello change incompatibly.
So to upgrade the greeting gem from 1.0.0 to 2.0.0, you need to fix all existing say_hello calls to say_hello("user"), etc.
But doing that all at once in a monolith developed by multiple people is hard.

With Ruby::Box, you can load both 1.0.0 and 2.0.0 and use them separately.

# Load greeting 1.0.0 into Ruby::Box b1
b1 = Ruby::Box.new
b1.require "./greeting-1.0.0"

# Load greeting 2.0.0 into Ruby::Box b2
b2 = Ruby::Box.new
b2.require "./greeting-2.0.0"

# Each Ruby::Box has its own library loaded
p b1::Greeting::VERSION #=> "1.0.0"
p b2::Greeting::VERSION #=> "2.0.0"

# Call each say_hello separately
p b1::Greeting.new.say_hello         #=> "Hello"
p b2::Greeting.new.say_hello("mame") #=> "Hello, mame!"

You can use greeting 2.0.0 in some parts of a large codebase and 1.0.0 elsewhere.
In other words, this gives you the option to update gem versions gradually.

Caution

In reality, switching versions at the granularity of each say_hello call can become unmanageable, so you should avoid it (and for reasons explained later, it’s also difficult in practice).

A better practice might be to split large codebases roughly by feature units (for example, pack units in packwerk?), map a Ruby::Box to each, and update gem versions per feature unit.
But no one has done this in practice yet, so it’s unclear if it really works. I hope for future knowledge sharing.

There is also an opinion that “multi-version support like this is not good in the first place.” Still, I think it is the best topic to understand what Ruby::Box is and consider its applications, so I introduced it.

Also, seeing such examples may make you want a separate Gemfile per Ruby::Box.
However, there is no good integration between Ruby::Box and RubyGems/Bundler yet. Hopefully in the future.

Use case 2: Use a globally configured gem under multiple configurations

Here’s another example. Consider a library whose settings are stored globally.

In the Greeting library above, suppose setting Greeting.polite = true makes the message more polite.

# greeting.rb
class Greeting
  @@polite = false

  def self.polite=(polite)
    @@polite = polite
  end

  def say_hello
    if @@polite
      puts "Hello. How are you today?")
    else
      puts "Yo!"
    end
  end
end

Since @@polite is a global setting, you can only fix it to one or the other.
You can switch it midway, but in a multi-threaded program it is not thread-safe.

require "./greeting"

Greeting.new.say_hello #=> "Yo!"

# You can switch midway, but...
Greeting.polite = true
Greeting.new.say_hello #=> "Hello. How are you today?"

# Risky when used in threads
Thread.new { Greeting.new.say_hello } #=> You never know what will come out
Greeting.polite = false

By loading the library multiple times with Ruby::Box, you can hold multiple configurations in a thread-safe way.

b1 = Ruby::Box.new
b1.require "./greeting"

b2 = Ruby::Box.new
b2.require "./greeting"

b2::Greeting.polite = true

# Safely use different settings even in threads
Thread.new { b1::Greeting.new.say_hello } #=> "Yo!"
Thread.new { b2::Greeting.new.say_hello } #=> "Hello. How are you today?"

Personally I want to say “libraries with global configuration are lame,” and think they should use thread-local storage if anything, but in reality such libraries probably do exist.

Ruby::Box is not object-space isolation

We’ve looked at Ruby::Box, and someone may think “Ruby::Box isolates object space.”
That is a misconception, so be careful.

Example of passing objects across Ruby::Box (not recommended)

Ruby::Box only separates class (constant) namespaces.
Objects themselves do not have a concept like “belonging to a Ruby::Box,” and passing objects across Ruby::Box boundaries is allowed.

See the following code.

# Define Foo class inside b1
b1 = Ruby::Box.new
b1.eval <<'RUBY'
  class Foo
    def say_hello = puts("Hello")
  end
RUBY

# Define Test class inside b2
b2 = Ruby::Box.new
b2.eval <<'RUBY'
  class Test
    # The accept method can take an instance of b1::Foo
    def self.accept(b1_foo)
      b1_foo.say_hello
    end
  end
RUBY

# Create an instance of b1::Foo
b1_foo = b1::Foo.new

# It can be passed to a method inside b2
b2::Test.accept(b1_foo) #=> "Hello"

b1_foo has no concept of “belonging to b1.”
A method inside b2 can receive the b1_foo object and call b1::Foo#say_hello.
Passing instances of classes defined in one Ruby::Box to methods in another Ruby::Box is not restricted.
So this is not object-space separation.

(Also, besides passing via methods, there are tricks like forcibly passing via constant definition, e.g., b2::X = b1::Foo.new.)

Behavior when defining same-named classes across Ruby::Box

As shown above, passing objects across Ruby::Box is allowed, but (in Endoh’s opinion) it is by no means recommended.
Because it can easily lead to very confusing bugs.

Consider this example using the Data class.

# Define User class inside b1
b1 = Ruby::Box.new
b1.eval <<'RUBY'
  class User < Data.define(:name)
    def greeting = "Hello #{self.name}"
  end
RUBY

# Also define User class inside b2
# Note: b1::User and b2::User are completely different classes!
b2 = Ruby::Box.new
b2.eval <<'RUBY'
  class User < Data.define(:name)
    def greeting = "こんにちは #{self.name}"
  end

  class Test
    def self.test(a) = a.greeting
  end
RUBY

# Instances of b1::User and b2::User look identical but are actually different
p b1::User.new("mame") #=> #<data User name="mame">
p b2::User.new("mame") #=> #<data User name="mame">

# Comparison treats them as completely different objects
p b1::User.new("mame") == b2::User.new("mame") #=> false

# .greeting respects each class definition
p b1::User.new("mame").greeting       #=> "Hello mame"
p b2::User.new("mame").greeting       #=> "こんにちは mame"

# Even when calling .greeting from inside b2, each definition is respected
p b2::Test.test(b1::User.new("mame")) #=> "Hello mame"
p b2::Test.test(b2::User.new("mame")) #=> "こんにちは mame"

Instances of Data are supposed to be equal if their member values are equal, but here b1::User and b2::User themselves are different, so their instances are treated as different objects (even though the contents are the same).

Mixing instances of classes defined in different Ruby::Boxes is a recipe for confusion, so it is wise to avoid it.
And for this reason, switching gem versions at fine granularity like per method call is risky.
It’s better to update gem versions at a unit where you can understand object exchanges, such as feature units.

By the way, the behavior above is so confusing that there is (or isn’t) discussion about showing the Ruby::Box in the output, like #<data b1::User name="mame">.

Mindset

As a rule of thumb, it’s good to remember: “Do not pass objects across Ruby::Box boundaries.”

Strictly speaking, even reading a class object like b1::User from b1 is “passing an object across Ruby::Box boundaries,” so how far you should avoid that is something we’ll need to explore.

Built-in classes like Integer and String are shared across Ruby::Boxes, so passing those instances is relatively safe. I’ll explain this next.

Monkey-patch isolation

Ruby::Box has another very delightful feature.
It isolates monkey patches on built-in classes.

What is monkey-patching?

In Ruby, you can add methods to built-in classes or redefine existing methods. This is commonly called monkey-patching.

In Ruby, integer division truncates. Some people who like math want it to return rationals. In such cases, you can monkey-patch like this.

p 5 / 3 #=> 1

class Integer
  def /(other) = self.quo(other)
end

p 5 / 3 #=> (5/3)

But if you actually do this, most code and libraries that expect truncation will break.

In fact, ancient Ruby had a standard library called "mathn" that applied such monkey patches, but it gradually fell out of use and disappeared because it didn’t play well with other code.

Monkey-patch isolation via Ruby::Box

With Ruby::Box, you can isolate and localize the effect of monkey patches like this.

b = Ruby::Box.new

# Monkey-patch Integer#/ inside b
b.eval <<'RUBY'
  class Integer
    # Return Rational
    def /(other) = self.quo(other)
  end
RUBY

# Inside b, division returns rationals
b.eval <<'RUBY'
  p 5 / 3 #=> (5/3)
RUBY

# Outside b, division truncates as before
p 5 / 3 #=> 1

Refinement already exists as a monkey-patch isolation feature, but you have to write using SomeRefinement in each context you want to use it. Ruby::Box doesn’t require such explicitness.

Use cases

This feature was introduced to isolate ActiveSupport inside a Ruby::Box (though it seems it doesn’t work for that purpose yet).

ActiveSupport and mathn are of course use cases for monkey-patch isolation, but personally I think this feature has many other uses.

For example, it seems useful to isolate the environment where user code is executed in irb.
In today’s irb, if you carelessly redefine String#+, irb crashes like this.
If you separate the Ruby::Box that evaluates user code, it might handle such mischievous code nicely.

$ irb
irb(main):001> class String; def +(_) = raise; end
An error occurred when inspecting the object: RuntimeError
Result of Kernel#inspect: #<Symbol:0x0000000000002b0c>
=> :+
(irb):1:in 'String#+': unhandled exception
        from /home/mame/.rbenv/versions/ruby-dev/lib/ruby/gems/4.0.0+0/gems/irb-1.15.3/lib/irb.rb:658:in 'block in IRB::Irb#format_prompt'
        from /home/mame/.rbenv/versions/ruby-dev/lib/ruby/gems/4.0.0+0/gems/irb-1.15.3/lib/irb.rb:626:in 'String#gsub'
        from /home/mame/.rbenv/versions/ruby-dev/lib/ruby/gems/4.0.0+0/gems/irb-1.15.3/lib/irb.rb:626:in 'IRB::Irb#format_prompt'
...
        from /home/mame/.rbenv/versions/ruby-dev/bin/irb:25:in '<main>'

$

Also recently, there’s a movement to rewrite Ruby’s built-in methods from C to Ruby (because JIT can make Ruby code faster in some cases). This can cause monkey patches to affect unexpected methods.
For example, currently the Ruby implementation of Integer#times uses #succ, so monkey-patching Integer#succ breaks Integer#times.

class Integer
  def succ = raise
end

# The Ruby implementation of Integer#times calls #succ, so this raises
1.times { p "ok" }
#=> RUBY_BOX=0: in 'Integer#succ': unhandled exception
#   RUBY_BOX=1: "ok"

Ruby::Box already solves this problem. If you run the above code with RUBY_BOX=1, Integer#times ignores the redefinition of Integer#succ and still works.
This is because when RUBY_BOX=1 is set, the Ruby::Box that defines built-in methods (Ruby::Box.root) and the Ruby::Box for user code (Ruby::Box.main) are separated.
In other words, monkey patches to Integer#succ done inside user code are isolated from the built-in Ruby::Box space.

Caution

The monkey-patch isolation above is a special feature only for built-in classes.
Built-in classes are classes available immediately after starting the Ruby interpreter, such as Array, Integer, String, etc.
Be aware that these are treated fundamentally differently from user-defined classes.

I said that classes with the same name defined in different Ruby::Boxes are completely different (the b1::User and b2::User example).
However, built-in classes like Integer are specially shared across Ruby::Boxes.
Instead, the method table for the same Integer class is separated per Ruby::Box.

If you study the following example carefully, you might start to see it.

# b1 monkey-patches Integer#/
b1 = Ruby::Box.new
b1.eval <<'RUBY'
  class Integer
    def /(other) = self.quo(other.n)
  end
  ONE = 1
  TWO = 2
RUBY

# b2 calls / on its arguments
b2 = Ruby::Box.new
b2.eval <<'RUBY'
  class Test
    def self.test(a, b) = a / b
  end
RUBY

# Even if you divide b1::ONE and b1::TWO inside b2, it truncates
# (i.e., the Integer monkey patch inside b1 is not effective inside b2)
p b2::Test.test(b1::ONE, b1::TWO) #=> 0

In short, monkey-patch definitions do not carry across Ruby::Box boundaries.

Caution (2)

Date is subtle.
Time is a built-in class, so mixing across Ruby::Boxes works without surprises, but Date is not a built-in class (you need require "date"), so you get non-intuitive behavior like b1::User / b2::User.

# Require the date library in b1 and b2
# (two separate Date classes are defined)
b1 = Ruby::Box.new
b1.require "date"

b2 = Ruby::Box.new
b2.require "date"

# b1::Time and b2::Time are actually the same class
 t1 = b1::Time.new(2025, 12, 25)
 t2 = b2::Time.new(2025, 12, 25)

p t1 #=> 2025-12-25 00:00:00 +0900
p t2 #=> 2025-12-25 00:00:00 +0900

# Time instances compare normally
p t1 == t2 #=> true

# b1::Date and b2::Date are different
 d1 = b1::Date.new(2025, 12, 25)
 d2 = b2::Date.new(2025, 12, 25)

p d1 #=> #<Date: 2025-12-25 ((2461035j,0s,0n),+0s,2299161j)>
p d2 #=> #<Date: 2025-12-25 ((2461035j,0s,0n),+0s,2299161j)>

# These Date instances are considered different when compared
p d1 == d2 #=> false

Personally, I recommend not using Date, but in the short term that’s not feasible, so I’m stuck at “what do we do?”

Other notes

Ruby::Box has various caveats and anecdotes.
I’ll explain them quickly, avoiding too many corner cases.

Top-level constant references are contained within Ruby::Box

With ::A you can reference a top-level constant, but inside Ruby::Box it looks for the top-level constant within that Ruby::Box.

Foo = "toplevel"

b = Ruby::Box.new
b.eval <<'RUBY'
  Foo = 42
  p ::Foo #=> 42 (not "toplevel")
RUBY

This means that existing libraries that reference ::Foo should still work when loaded via Ruby::Box#require. Probably.

Ruby::Box also isolates global variables

So far we only talked about constants, but global variables are isolated the same way.

$foo = 42

b = Ruby::Box.new
b.eval <<'RUBY'
  p $foo #=> nil
RUBY

Global variables are no longer global (there are discussions about sharing some like $VERBOSE).

Each Ruby::Box has its own `$LOAD_PATH`/`$LOADED_FEATURES`

$LOAD_PATH and $LOADED_FEATURES are global variables, so each Ruby::Box has its own definitions.
Ruby::Box#require follows each definition, so be careful.

Ruby::Box.new creates a clean environment

Even if you define classes or methods before Ruby::Box.new, those definitions are not carried into the new Ruby::Box.

class Foo
end

b = Ruby::Box.new
b.eval <<'RUBY'
  Foo #=> uninitialized constant Foo (NameError)
RUBY

Personally I think there are many cases where you want to inherit definitions, but the spec and implementation would get quite complex, so it’s like this for now.

Ruby::Box is not a sandbox

It is not. Ruby::Box is not built for security.

For example, you can pull out outside objects via ObjectSpace, and there are likely many other tricks.

Ruby::Box API

At the moment, it seems to have the following methods.

Ruby::Box.enabled?: whether Ruby::Box functionality is enabled
Ruby::Box#eval: execute code inside a Ruby::Box
Ruby::Box#require: load a source file inside a Ruby::Box
Ruby::Box#require_relative: Kernel#require_relative version of the above
Ruby::Box#load: Kernel#load version of the above
Ruby::Box#load_path: $LOAD_PATH inside the Ruby::Box? (not sure)
Ruby::Box.current: return the current Ruby::Box
Ruby::Box.main: return the top-level Ruby::Box
Ruby::Box#main?: return whether it is the above
Ruby::Box.root: return the Ruby::Box where built-in classes are defined (best not to use lightly)
Ruby::Box#root?: return whether it is the above

Relation to Kernel.load’s second argument

Actually, you could already pass a module as the second argument to load to specify where constants are defined.

m = Module.new

# Load a.rb in the context of module m
load("a.rb", m)

p m::A #=> #<Module:0x000078d927d4a520>::A

This is basically the prototype of Ruby::Box, but it wasn’t very polished, so I think it was rarely used in practice.
For example, top-level constants ::A still referenced the global top-level constants.
You could say Ruby::Box is this feature refined to an extreme.

What happened to the name "Namespace"?

This feature was originally discussed under the name "Namespace", but according to Matz, it was avoided due to naming conflicts.
For example, when you write a my_app gem, you usually wrap the code like:

module MyApp
  ...
end

The usage of the word “Namespace” or “Namespace module” for this is already established, so they avoided confusion by choosing a different name.

Also, in some Rails apps (specifically GitLab), a top-level constant Namespace is already defined, and changing it would require renaming DB tables. That likely had some influence too.

So, can we use it yet?

No, not in production.

As the warning says when you start ruby 4.0.0 with RUBY_BOX=1 — warning: Ruby::Box is experimental, and the behavior may change in the future! — it’s not yet at production quality. In fact, with RUBY_BOX=1 the Ruby test suite probably doesn’t pass yet (probably).

However, it seems to be good enough for playing around, so please try it.

High-level API?

According to Matz’s vision, this Ruby::Box is a low-level API for a packaging system that may be introduced to Ruby in the future.
So in the future, a high-level packaging-system API based on Ruby::Box might be born.

It could be a built-in Ruby feature, or it could be a de facto library (not standard Ruby) like zeitwerk for autoload.
Either way, someone will need to try building such a thing with the current Ruby::Box and identify missing features and necessary spec adjustments.

Summary

So, this was a digest introduction to the current Ruby::Box in Ruby 4.0.0, from Endoh’s perspective.

This is only a digest. There are still many corner cases I omitted, and surely there are tons of corner cases and issues that Endoh doesn’t know about (and maybe even the author tagomoris doesn’t recognize).

This is a huge contribution opportunity like few in recent years, so please go play with it a lot.

Reading Ruby 4.0 NEWS with Pros

Koichi Sasada — Thu, 25 Dec 2025 05:45:48 +0000

(This article is AI translation version of https://product.st.inc/entry/2025/12/25/134932 from Japanese to English)

We are Koichi Sasada (ko1) and Yusuke Endoh (mame) from STORES, Inc. We develop Ruby (MRI: Matz Ruby Implementation, the so-called ruby command). We are paid to develop Ruby, so we are professional Ruby committers.

Today, 12/25, Ruby 4.0.0 was released as the annual Christmas release (Ruby 4.0.0 Released | Ruby). This year as well, we will explain the Ruby 4.0 NEWS.md file on STORES Product Blog (by the way, this is an article for STORES Advent Calendar 2025. Please read the others too). For what a NEWS file is, see the previous articles (all articles in Japanese).

This article not only explains new features but also, within the limits of our memory, includes background and struggles behind the changes.

Ruby's first release was announced in 1995, so this year marks the 30th anniversary.
That is why we have a commemorative Ruby 4.0.0 release.

Representative changes in Ruby 4.0 are as follows (excerpted from the release notes).

(Language spec) Logical binary operators can be placed at the beginning of a line
Experimental introduction of Ruby Box
Experimental introduction of ZJIT
Ractor improvements (still experimental)

(Mostly experimental, huh?)

In this article, we will roughly introduce items in the NEWS file, including these.

Language changes

When calling foo(*nil), it no longer calls nil.to_a

*nil no longer calls nil.to_a, similar to how **nil does not call nil.to_hash. [Feature #21047]

I didn't know this, but foo(*nil) used to call nil.to_a. And since that returns [], it ultimately means there are no arguments.

In Ruby 3.3, foo(**nil) was introduced and it does not call nil.to_hash, so a proposal was made that nil.to_a should likewise not be called. It was accepted. This removes one method call and the creation of one empty array, which should be good for performance.

p nil.to_a #=> []

def nil.to_a
  p :to_a
  []
end
p(*nil)
# Ruby 3.4
# => :to_a
# Ruby 4.0
# => nothing is printed

It's a compatibility break if you want to call it that, but I want to believe there is no code that depends on it.

(ko1)

Line breaks before logical operators are now allowed

Logical binary operators (||, &&, and and or) at the beginning of a line continue the previous line, like fluent dot. [Feature #20925]

When an if condition spans multiple lines, it can be hard to tell the condition from the body.

if cond1 &&
  cond2 # part of the condition
  body1 # first statement inside the if
  body2
end

In this code, cond2 is part of the condition and body1 is the first statement in the if body, but the indentation is the same, so it's hard to distinguish.

There were tricks to make it clearer, like indenting cond2 more deeply, inserting a blank line between cond2 and body1, or inserting then, but all of them felt awkward.

# tried using then
if cond1 &&
  cond2
then
  body1
  body2
end

So in Ruby 4.0.0, you can place a line break before && and ||.

if cond1
  && cond2
  body1
  body2
end

With && cond2, it's obvious that it's part of the condition, so the problem is solved.

...Or is it? It feels like we just added one more awkward style, though it might be convenient once you get used to it.

Backstory

The discussion around this change got pretty lively.

In Ruby, whether an expression continues to the next line can generally be determined by the end of the previous line, but the method-call dot was an exception.

ary                       # can be a complete expression here
  .map { it.to_s }        # next line starts with a dot, so it continues as one expression
  .select { it.size > 3 } # convenient for multi-line method chaining

ary is a complete expression on its own, but because the next line starts with .map, the expression continues. This change extends that exception to &&, ||, and, and or.

But if we expand this exception, won't people want to break lines before other binary operators too? For example, breaking before + in x + y? But that would be incompatible (+ y can be parsed as the unary +). Is it worth the risk of making Ruby's line-break rules harder to understand just to allow breaks before &&?

Some people felt that resistance, but in the end Matz said "do it", so we did.

As an aside, personally I thought it would look cool to write && cond2 without indentation, aligned with if cond1.

if cond1
&& cond2
  body1
  body2
end

But there are basically no supporters of this style (at Ruby developer meetings it was dismissed as "not worth discussing"), which is sad.

(mame)

Built-in class updates

Array#rfind was added

Array#rfind has been added as a more efficient alternative to array.reverse_each.find [Feature #21678]
Array#find has been added as a more efficient override of Enumerable#find [Feature #21678]

Array#rfind was introduced to find the last element that matches a condition. It iterates in reverse from the last element and returns the first element that matches the condition.

# returns 4 instead of 2
[1, 2, 3, 4, 5].rfind { it.even? } #=> 4

Also, Array#find was added. There was already Enumerable#find, but defining a method specialized for Array makes it faster.

(mame)

Binding#implicit_parameters was added

Binding#implicit_parameters, Binding#implicit_parameter_get, and Binding#implicit_parameter_defined? have been added to access numbered parameters and "it" parameter. [Bug #21049]

A metaprogramming API to read numbered parameters and the implicit it from Binding was added.

["foo"].each do
  p it #=> "foo"

  # read the implicit argument it from the binding
  p binding.implicit_parameters #=> [:it]
  p binding.implicit_parameter_get(:it) #=> "foo"
end

{ foo: 42 }.each do
  p _1 #=> :foo
  p _2 #=> 42

  # read implicit arguments _1 and _2 from the binding
  p binding.implicit_parameters #=> [:_1, :_2]
  p binding.implicit_parameter_get(:_1) #=> :foo
  p binding.implicit_parameter_get(:_2) #=> 42
end

This was introduced to display _1 or it in the debugger, so normally you don't need to use it. Please don't.

Binding#local_variables does no longer include numbered parameters. Also, Binding#local_variable_get, Binding#local_variable_set, and Binding#local_variable_defined? reject to handle numbered parameters. [Bug #21049]

Until Ruby 3.4, you could read numbered parameters with binding.local_variable_get(:_1), but in Ruby 4.0 it raises NameError, so be careful.

The reason for this change is as follows. The implicit it introduced in Ruby 3.4 can also be used as a normal local variable name, so binding.local_variable_get(:it) becomes ambiguous. Along with banning it, _1 was also banned.

Since numbered parameters and implicit it have different scope rules from ordinary local variables, it's arguably not good to handle them together in Binding#local_variables, and they are now handled by a separate method, implicit_parameters.

(mame)

Enumerator.produce now accepts the size keyword

Enumerator.produce now accepts an optional size keyword argument to specify the size of the enumerator. It can be an integer, Float::INFINITY, a callable object (such as a lambda), or nil to indicate unknown size. When not specified, the size defaults to Float::INFINITY. [Feature #21701]

A size keyword was added to Enumerator.produce.

Enumerator.produce basically creates an infinite Enumerator.

# Enumerator equivalent to (1..)
enum = Enumerator.produce(1) {|n| n + 1 }

p enum.take(5) #=> [1, 2, 3, 4, 5]
p enum.size    #=> Float::INFINITY

A less-known trick is that you can stop produce by raising StopIteration.

# Enumerator equivalent to (1..10)
enum = Enumerator.produce(1) do |n|
  raise StopIteration if n > 10
  n + 1
end

p enum.to_a #=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
p enum.size #=> Float::INFINITY (?)

In that case, having enum.size be Float::INFINITY is not great. So now you can explicitly specify the length with the size keyword.

# Enumerator equivalent to (1..10)
enum = Enumerator.produce(1, size: 10) do |n|
  raise StopIteration if n > 10
  n + 1
end

p enum.to_a #=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
p enum.size #=> 10

However, unless you specify this size keyword, the Enumerator returned by produce will still claim its size is Float::INFINITY, and if you specify a lie in the size keyword, it will lie as well. In short, it's best not to trust Enumerator#size too much.

(mame)

error_highlight now shows ArgumentError more helpfully

When an ArgumentError is raised, it now displays code snippets for both the method call (caller) and the method definition (callee). [Feature #21543]

Consider the following code.

def add(x, y) = x + y

add(1)

The number of arguments to add is wrong, so an error is raised. Now it shows code snippets for both the caller and the callee.

# In Ruby 4.0
$ ruby test.rb
test.rb:1:in 'Object#add': wrong number of arguments (given 1, expected 2) (ArgumentError)

    caller: test.rb:3
    | add(1)
      ^^^
    callee: test.rb:1
    | def add(x, y) = x + y
          ^^^
        from test.rb:3:in '<main>'

In Ruby 3.4, it was a simple display like this.

# In Ruby 4.0
$ ruby test.rb
test.rb:1:in 'add': wrong number of arguments (given 1, expected 2) (ArgumentError)
        from test.rb:3:in '<main>'

When you see this error, people reflexively look at the source of test.rb:1. But when you get "wrong number of arguments", you almost never need to fix the method definition side. You end up looking at the error again and reopening the source for test.rb:3.

We felt repeating this work was wasteful, so after proposing and discussing a stack trace improvement, we decided to display it as shown above using error_highlight.

(mame)

Changes around `Fiber::Scheduler`

Introduce Fiber::Scheduler#fiber_interrupt to interrupt a fiber with a given exception. The initial use case is to interrupt a fiber that is waiting on a blocking IO operation when the IO operation is closed. [Feature #21166]

A method called Fiber::Scheduler#fiber_interrupt was added.
The story gets fairly specific, but here is the usage scenario.

When thread A is doing io.read on some io and thread B tries to io.close, A can end up waiting forever (depending on the OS), so io.close raises an exception for the list of threads blocked on that (in this case thread A).
There was a request to do the same thing in a fiber scheduler, so Fiber::Scheduler#fiber_interrupt was introduced to make that possible.

I don't know the actual code well enough to write it...

Introduce Fiber::Scheduler#yield to allow the fiber scheduler to continue processing when signal exceptions are disabled. [Bug #21633]

Fiber::Scheduler#yield was introduced (when did that happen...).

After receiving a signal, if it cannot be handled in the normal flow, it schedules other schedulable fibers. I wonder if that's okay.

Reintroduce the Fiber::Scheduler#io_close hook for asynchronous IO#close.

Fiber::Scheduler#io_close was reintroduced. It says so, but there's no reference, so I have no idea.

Invoke Fiber::Scheduler#io_write when flushing the IO write buffer. [Bug #21789]

When you do IO#close or IO#fush, data accumulated in the write buffer is output to the file. Previously this happened without considering the fiber scheduler, but now it is done via the io_write hook for the fiber scheduler.

(ko1)

File::Stat#birthtime is now available on Linux

File::Stat#birthtime is now available on Linux via the statx system call when supported by the kernel and filesystem. [Feature #21205]

It seems you can now get file creation time on Linux.

This is only available when the statx(2) system call is available and the filesystem records creation time, but in modern Linux it should usually work (though please research the exact conditions yourself).

(mame)

`IO.select` can now accept `Float::INFINITY` as a timeout value

IO.select accepts Float::INFINITY as a timeout argument. [Feature #20610]

IO.select is an API for waiting until multiple I/Os are ready (e.g., read won't block). It accepts a timeout, and with nil (or omitted) it waited indefinitely. This change allows you to specify Float::INFINITY as that infinite timeout.

(ko1)

You can now choose which instance variables are shown in inspect

Kernel#inspect now checks for the existence of a #instance_variables_to_inspect method, allowing control over which instance variables are displayed in the #inspect string [Feature #21219]

I think you've had the experience of casually p-ing an object and being startled by a huge output.

class Foo
  def initialize(id)
    @id = id
    @internal_data = "a" * 1000
  end
end

p Foo.new("foo")
#=> #<Foo:0x00007c495d699dd0 @internal_data="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...

If you want @id shown but not @internal_data, a feature was added that lets you specify which instance variables are displayed. By adding an instance_variables_to_inspect method like this, only @id will be shown.

class Foo
  def initialize(id)
    @id = id
    @internal_data = "a" * 1000
  end

  # list instance variable names you want displayed as symbols
  private def instance_variables_to_inspect = [:@id]
end

p Foo.new("foo")
#=> #<Foo:0x00007692c1a44f78 @id="foo">

It's nice and clean.

You can also use it to exclude things you don't want casually logged, like password strings (this may be the more important use case). For example, to exclude @password, you can do the following.

class LoginInfo
  def initialize(user)
    @user = user
    @password = "secret"
  end

  # remove instance variable names you don't want displayed from Kernel#instance_variable
  private def instance_variables_to_inspect = instance_variables - [:@password]
end

# @password is not shown
p LoginInfo.new("mame") #=> #<LoginInfo:0x000072b3b2426390 @user="mame">

Mechanically, when Kernel#inspect stringifies an object, it no longer displays all instance variables unconditionally; it now calls instance_variables_to_inspect to determine which instance variables should be shown.

By the way, pp has long had a similar mechanism called pretty_print_instance_variables. This was introduced for p as well, but it would be awkward to introduce it with that name, so it became instance_variables_to_inspect. As usual, the hardest part was getting the method name to settle (name discussions are very important in Ruby).

(mame)

You can no longer start a process with open("| ls")

Kernel
- A deprecated behavior, process creation by Kernel#open with a leading |, was removed. [Feature #19630]
IO
- A deprecated behavior, process creation by IO class methods with a leading |, was removed. [Feature #19630]

Ruby's Kernel#open is a method to open files, but if you pass it a string starting with a pipe symbol like "| ls", it could start a process and treat its standard I/O as a file.

# runs ls /home and prints its output via read (up to Ruby 3.4)
open("| ls /home") { puts f.read }
  #=> Ruby 3.4: mame
  #=> Ruby 4.0: No such file or directory @ rb_sysopen - | ls /home (Errno::ENOENT)

Many users didn't know about this feature and it tended to lead to vulnerabilities, so it was deprecated two years ago in Ruby 3.3 and now finally removed.

There is no drop-in replacement. If you intentionally used this feature, please rewrite appropriately using IO.popen, etc. In most cases, replacing open("|command") with IO.popen("command") should work (but it is a dangerous feature, so please consider carefully).

(mame)

`Math.log1p` and `Math.expm1` were added

Math.log1p and Math.expm1 are added. [Feature #21527]

Math.log1p(x) returns the value of $\log(x + 1)$.

You might wonder, "Why does this exist? Isn't it enough to write Math.log(x + 1.0)?" But apparently it is needed to avoid floating-point errors.

When x is very small (smaller than Float::EPSILON), x + 1.0 is rounded to 1.0, so Math.log(x + 1.0) always returns 0.

Math.log1p(x) works even for such small x.

# very small x
x = 1.0e-16

p Math.log(x + 1) #=> 0.0
p Math.log1p(x)   #=> 1.0e-16

Well, $\log(x)$ is approximately $x$ near $x = 1$, so it's essentially just returning the argument.

Math.expm1 returns $\exp(x) - 1$, and the need is basically the same.

The proposal started when I was implementing a numerical analysis paper as a hobby and noticed there was no log1p. I was surprised no one had noticed that a function provided by C since the C99 era was missing.

During the discussion for introduction, someone quipped "log1p and expm1 are meaningless names", but I found that Python, Java, JavaScript, Go, PHP, .NET, R, MATLAB, Kotlin, Swift, Julia, Haskell, OCaml, and Crystal all use this name (Rust uses the even more cryptic ln_1p). So we decided to keep the names as-is.

(mame)

Pathname became a core class

Pathname has been promoted from a default gem to a core class of Ruby. [Feature #17473]

The Pathname class is now built-in. You can use Pathname without writing require "pathname".

However, Rails already does require "pathname" by default, so for most users this likely changes little. For people who often write throwaway scripts or one-liners in Ruby, it might be a small but nice improvement.

Note that you still need require "pathname" to use these three methods.

Pathname#find
Pathname#rmtree
Pathname#mktmpdir

This is because these methods depend on other libraries. It would be too much to make find, fileutils, and tmpdir built-ins just to make Pathname built-in, so these were not included.

As an aside, a very strange bug occurred during making Pathname built-in. I explained the debugging process in detail in another article, so if you're interested, please take a look.

[https://product.st.inc/entry/2025/07/17/104509:embed:cite]

(mame)

The implicit `it` return value in `Proc#parameters` changed slightly

Proc#parameters now shows anonymous optional parameters as [:opt] instead of [:opt, nil], making the output consistent with when the anonymous parameter is required. [Bug #20974]

Proc#parameters is a method to inspect what arguments a block takes.

p proc{}.parameters #=> []
p proc{|a|}.parameters #=> [[:opt, :a]]
p proc{|a, b|}.parameters #=> [[:opt, :a], [:opt, :b]]
p proc{|a, b, c=1|}.parameters #=> [[:opt, :a], [:opt, :b], [:opt, :c]]
p proc{|a, b, c=1, *d|}.parameters #=> [[:opt, :a], [:opt, :b], [:opt, :c], [:rest, :d]]
p proc{|a, b, c=1, *d, e|}.parameters #=> [[:opt, :a], [:opt, :b], [:opt, :c], [:rest, :d], [:opt, :e]]

p lambda{}.parameters #=> []
p lambda{|a|}.parameters #=> [[:req, :a]]
p lambda{|a, b|}.parameters #=> [[:req, :a], [:req, :b]]
p lambda{|a, b, c=1|}.parameters #=> [[:req, :a], [:req, :b], [:req, :c]]
p lambda{|a, b, c=1, *d|}.parameters #=> [[:req, :a], [:req, :b], [:req, :c], [:rest, :d]]
p lambda{|a, b, c=1, *d, e|}.parameters #=> [[:req, :a], [:req, :b], [:req, :c], [:rest, :d], [:req, :e]]

All the proc{|a|} cases use [:opt, :a] (i.e., optional arguments) because block arguments can be passed or omitted. With lambda{|a|}, you must pass one argument or you'll get an error, so it becomes [:req, :a].

Now, blocks using it introduced in Ruby 3.4 are like proc{|a|} in that they receive one argument, which means one optional argument. However, you can also explicitly name an argument it (e.g., proc{|it|p it}.parameters #=> [[:opt, :it]]), so returning it as a name is ambiguous. That's why we previously used nil for the name.

But for lambda, the parameter is required, and the name-less form is just [:req]. So we decided to make them consistent and use [:opt].

p lambda{p it}.parameters
#=> [[:req]]

p proc{p it}.parameters
#=> Ruby 3.4: [[:opt, nil]]
#=> Ruby 4.0: [[:opt]]

By the way, for numbered parameters, the variable names like :_1 still appear as before. Right now _1 cannot be used as a normal parameter name (or variable name), so it's not confusing.

p proc{p [_1, _2]}.parameters
pt, :_1], [:opt, :_2]]

p lambda{p [_1, _2]}.parameters
#=> [[:req, :_1], [:req, :_2]]

(ko1)

Ractor::Port was introduced to reorganize Ractor communication

Ractor::Port class was added for a new synchronization mechanism to communicate between Ractors. [Feature #21262] Ractor::Port provides the following methods:
- Ractor::Port#receive
- Ractor::Port#send (or Ractor::Port#<<)
- Ractor::Port#close
- Ractor::Port#close As result, Ractor.yield and Ractor#take were removed.
Ractor#join and Ractor#value were added to wait for the termination of a Ractor. These are similar to Thread#join and Thread#value.
Ractor#monitor and Ractor#unmonitor were added as low-level interfaces used internally to implement Ractor#join.
Ractor.select now only accepts Ractors and Ports. If Ractors are given, it returns when a Ractor terminates.
Ractor#default_port was added. Each Ractor has a default port, which is used by Ractor.send, Ractor.receive.
Ractor#close_incoming and Ractor#close_outgoing were removed.

Ractor communication was reorganized around a new thing called Ractor::Port. This is summarized in the following article, so please read it for details.

https://dev.to/ko1/ractorport-revamping-the-ractor-api-98

Here's an example from NEWS. You can now write it like this.

port1 = Ractor::Port.new
port2 = Ractor::Port.new
Ractor.new port1, port2 do |port1, port2|
  port1 << 1
  port2 << 11
  port1 << 2
  port2 << 12
end
2.times{ p port1.receive } #=> 1, 2
2.times{ p port2.receive } #=> 11, 12

Also, to wait for a Ractor to finish, you should use join or value.

r = Ractor.new{ task }
r.join    # wait for task to finish
p r.value # wait for task to finish and get its return value; basically the same as threads

The removal of Ractor#take, which was used for waiting, is a big change (Ractor itself is an experimental feature, so it can change more easily). It brings the API closer to threads.

(ko1)

Introducing `Ractor.shareable_proc` to create Proc objects shareable between Ractors

Ractor.shareable_proc and Ractor.shareable_lambda is introduced to make shareable Proc or lambda. [Feature #21550], [Feature #21557]

To make a Proc shareable, there are some constraints. The strictest constraint is that self while executing the block must be shareable (other constraints below). Previously, by using Ractor.make_shareable(proc_obj), you could make proc_obj shareable only when self was shareable and the other constraints were satisfied.

pr = proc{}
Ractor.make_shareable(pr)
#=> 'Racr.make_shareable': Proc's self is not shareable: #<Proc:...> (Ractor::IsolationError)
# because main is not shareable

Making self shareable was annoying and required doing something like nil.instance_exec{ proc{ ... } }, which was uncool. To solve that, Ractor.shareable_proc was introduced (and shareable_lambda creates a lambda). This method returns the given block as a shareable Proc.

pr = Ractor.shareable_proc{ }
p Ractor.shareable?(pr) #=> true

In this case, if you don't specify anything, self becomes nil. To change it, use the self: keyword.

pr = Ractor.shareable_proc(self: Ractor){
  self
}
p pr.call #=> Ractor

In most cases it will be used without self:, so cases where you must specify self: should be rare. Procs shared between Ractors are likely to be used to pass tasks, for example.

worker = Ractor.new do
  while task = Ractor.receive
    task.call
  end
end

def heavy_task = sleep(3)

# assign three heavy_task runs to the Ractor
worker << Ractor.shareable_proc{ heavy_task }
worker << Ractor.shareable_proc{ heavy_task }
worker << Ractor.shareable_proc{ heavy_task }

So if you felt that it was hard to make shareable Procs, this API solves that concern.

Supplement: Conditions for a Proc to become shareable:

(dynamic) self is shareable
(static) it does not write to outer local variables
(dynamic) when reading outer local variables, those locals contain shareable objects. The list of outer locals read is determined statically
(static) when reading outer local variables, those locals are not reassigned

When I say static and dynamic here, static means checks done at compile time, and dynamic means checks done at runtime (when the Proc is created). Note that if you eval inside a shareable Proc, it checks that these conditions are satisfied.

class C
  a = 1
  pr = proc{a = 2}
  Ractor.make_shareable(pr)
  #=> 'Ractor.make_shareable': can not make a Proc shareable because it accesses outer variables (a). (ArgumentError)
  # error because it writes to outer local variable a
end

Condition 4 is tricky, so here is more explanation.

a = 1
pr = Ractor.shareable_proc(self: Ractor){
  p a
}
#=> cannot make a shareable Proc because the outer variable 'a' may be reassigned. (Ractor::IsolationError)
a = 2

In this case, a referenced by the Proc is reassigned outside the block. This check is needed to keep behavior as close as possible to a "normal Proc". A shareable Proc records the value of the variable at the time it is created, in this case 1 for a. So even if you do a = 2 outside, a inside the shareable Proc will always return 1. This differs from the behavior of a normal Proc, so starting in Ruby 4.0 such cases are now errors.

(ko1)

`Range#to_set` now checks size

Range#to_set now performs size checks to prevent issues with endless ranges. [Bug #21654]

When you do (0..).to_set, Ruby 3.4 tried to create an infinite-size Set and hung, but Ruby 4.0 now raises an exception.

(0..).to_set
  #=> Ruby 3.4: never ends until it eats all memory
  #=> Ruby 4.0: cannot convert endless range to a set (RangeError)

(mame)

The fine-grained behavior of `Range#overlap?` changed

Range#overlap? now correctly handles infinite (unbounded) ranges. [Bug #21185]

The return value of (..3).overlap?(nil..nil) changed from false to true.

(mame)

The fine-grained behavior of `Range#max` changed

Range#max behavior on beginless ints been fixed. [Bug #21174] [Bug #21175]

(..10).max(2) now returns [10, 9]. (1..10).max(2) has long returned [10, 9], so this aligns it.

That said, (1.0 .. 10).max(2) raises an exception, which feels a bit unclear to me personally.

(mame)

`::Ruby` was defined

A new toplevel module as been defined, which contains Ruby-related constants. This module was reserved in Ruby 3.4 and is now officially defined. [Feature #20884]

The ::Ruby module was defined (in Ruby 3.4, you would get a warning if you tried to define a top-level Ruby constant). What exists now is that constants like RUBY_VERSION are defined there as well. Constants like RUBY_VERSION still remain as they were.

p Ruby::VERSION #=> "4.0.0"
p RUBY_VERSION #=> same as "4.0.0"

pp Ruby.constants
#=>
[:REVISION,
 :COPYRIGHT,
 :ENGINE,
 :ENGINE_VERSION,
 :VERSION,
 :RELEASE_DATE,
 :DESCRIPTION,
 :PLATFORM,
 :PATCHLEVEL]

Will there be more and more Ruby::... in the future?

(ko1)

`Ruby::Box` was introduced

A new (experimental) feature to provide separation about definitions. For the detail of "Ruby Box", see doc/language/box.md. [Feature #21311] [Misc #21385]

Ruby 4.0.0's headline feature, Ruby::Box, was introduced experimentally.

We wrote a dedicated article about it, so please see that.

https://dev.to/ko1/rubybox-digest-introduction-ruby-400-new-feature-3bch

(mame)

`Set` became a core class

Set is now a core class, instead of an autoloaded stdlib class. [Feature #21216]

Set is now a core class. It was rewritten in C, which makes some methods faster, and its compatibility with other built-in features has subtly improved. Since Set already had autoload set up, visible behavior changes for users are likely small, probably, hopefully.

Set#inspect now returns a string suitable for eval, using the Set[] syntax (e.g., Set[1, 2, 3] instead of #<Set: {1, 2, 3}>). This makes it consistent with other core collection classes like Array and Hash. [Feature #21389]

The clearest change is the Set#inspect result.

p Set[1, 2, 3]
  #=> Ruby 3.4: #<Set: {1, 2, 3}>
  #=> Ruby 4.0: Set[1, 2, 3]

Array and Hash show their literal forms, so Set now also displays using the normal literal-like syntax for creating it.

Passing arguments to Set#to_set and Enumerable#to_set is now deprecated. [Feature #21390]

A subtle incompatibility is that arguments to to_set were removed. This was a feature where you could pass a subclass of Set as an argument to specify the class of the generated object.

class MySet < Setend

p (1..3).to_set        #=> #<Set: {1, 2, 3}>
p (1..3).to_set(MySet) #=> #<MySet: {1, 2, 3}>

Personally, I think you shouldn't inherit from core classes.

(mame)

You can now specify connection timeouts for Socket.tcp and TCPSocket.new

Socket.tcp & TCPSocket.new accepts an open_timeout keyword argument to specify the timeout for the initial connection. [Feature #21347]

Socket.tcp and TCPSocket.new would block until the OS-defined timeout if a connection could not be made for some reason.

This timeout is about 127 seconds on Linux by default.

# raises after blocking for about 127 seconds (203.0.113.1 is a test address)
p Socket.tcp("203.0.113.1", 80)
  #=> Connection timed out - connect(2) for 203.0.113.1:80 (Errno::ETIMEDOUT)

This is very slow, so in practical applications it was essentially mandatory to wrap the connection with Timeout.timeout.

In Ruby 4.0, you can specify the timeout by passing the open_timeout keyword argument to Socket.tcp and TCPServer.new.

# raises after blocking for 1 second
p Socket.tcp("203.0.113.1", 80, open_timeout: 1)
  #=> Connection timed out - user specified timeout for 203.0.113.1:80 (Errno::ETIMEDOUT)

There were already resolv_timeout and connect_timeout keyword arguments for name resolution and connection timeouts. The difference is that open_timeout specifies the overall timeout from name resolution through connection.

With Happy Eyeballs v2 introduced in Ruby 3.4, name resolution and connection are partially parallelized, which made it harder to understand, so the more convenient overall timeout open_timeout was introduced.

It seems net-http quickly switched to using open_timeout (when available) instead of Timeout.timeout.

Completely as an aside, Linux's default of 127 seconds is because it starts at 1 second and performs exponential backoff retransmissions six times (1 + 2 + 4 + 8 + 16 + 32 + 64 = 127). You can change the retry count by changing net.ipv4.syn_retries = 6.

(mame)

Exceptions raised by Socket.tcp and TCPSocket.new changed

When a user-specified timeout occurred in TCPSocket.new, either Errno::ETIMEDOUT or IO::TimeoutError could previously be raised depending on the situation. This behavior has been unified so that IO::TimeoutError is now consistently raised. (Please note that, in Socket.tcp, there are still cases where o::ETIMEDOUT may be raised in similar situations, and that in both cases Errno::ETIMEDOUT may be raised when the timeout occurs at the OS level.)

To get to the conclusion first: these methods may raise Errno::ETIMEDOUT or IO::TimeoutError on timeout, so if you rescue, please rescue both.

begin
  Socket.tcp(...)
rescue Errno::ETIMEDOUT, IO::TimeoutError
end

Errno::E* follows the principle of system call failures. Following that principle, when the connect system call fails due to a timeout, Errno::ETIMEDOUT is raised. On the other hand, the timeout specified by the user with open_timeout is not a system call failure, so IO::TimeoutError is raised.

Previously, user-specified timeouts sometimes raised Errno::ETIMEDOUT as well, but it seems it was aligned to IO::TimeoutError to follow the principle.

It is inconvenient to have to rescue both, and we'd like to do something about it, but we noticed the issue fairly close to release (it was discovered while writing this NEWS commentary article), so let's hope for the future.

(mame)

Updated to Unicode 17.0.0

Update Unicode to Version 17.0.0 and Emoji Version 17.0. [Feature #19908][[Feature #://bugs.ruby-lang.org/issues/20724)][Feature #21275] (also applies to Regexp)

We updated Ruby's Unicode version from 15.0.0 to 17.0.0. There are no major changes. It includes support for new characters, adding new Unicode properties, and updating the rules for counting grapheme clusters in Indic scripts.

In Ruby regexps, you can match using Unicode properties.

'a'.match?(/\p{Hiragana}/)
# => false
'あ'.match?(/\p{Hiragana}/)
# => true

You can check what a property like Hiragana matches on this site. It seems it also matches the character 🈀. Properties can match characters that differ from your mental image, so if you need strict matching for a specific set of characters, it's better not to use them.

https://util.unicode.org/UnicodeJsps/regex.jsp?a=\p{Hiragana}&b=あa

When updating Unicode versions, we also update these properties. It's automated, so it is easy to add by just running a script.

Here is the list of Unicode properties available in Ruby.

[https://docs.ruby-lang.org/en/master/language/regexp/unicode_properties_rdoc.html:embed:cite]

Here are the Unicode release notes.

(ima1zumi)

String#strip can now take a set of characters to remove

String#strip, strip!, lstrip, lstrip!, rstrip, and rstrip! are extended to accept selectors arguments. [[Feature #21552]]

String#strip removes whitespace from both ends of a string, but now you can pass an optional argument specifying the set of characters to remove. By passing multiple characters, they are treated as a set of characters to remove (same as String#tr).

p ' str_!'.strip('!_')
#=> " str"
#   leading space is not removed, and trailing _ and ! are removed

You can also specify ranges like 'x-y' (also like String#tr).

p 'str345'.strip('0-9')
#=> "str" # remove trailing 0-9 characters

I asked if passing a string as a character set rather than a substring to remove might be confused with the behavior of String#delete_suffix (for example, 'bar'.strip('rab') removes everything), and was told, "You'll know if you try it." Dismissed.

(ko1)

`Thread#raise`/`Fiber#raise` can now take the `cause:` keyword

Thread
- Introduce support for Thse:) argument similar to Kernel#raise. [Feature #21360]
Fiber
- Introduce support for Fiber#raise(cause:) argument similar to Kernel#raise. [Feature #21360]

Like Kernel#raise(cause: exc), the cause: argument can specify another exception that caused this exception (default $!). Now you can specify cause: in Thread#raise as well, but if you do so it might lead to confusing situations (for example, sending an exception that occurred in the current thread to another thread), so use with care.

Fiber#raise(cause:) is the same story as Thread#raise(cause:).

(ko1)

Stdlib updates

We only list stdlib changes that are notable feature changes.

Other changes are listed in the following sections. We also listed release
history from the previous bundled version that is Ruby 3.4.0 if it has GitHub
releases.

6.3

0.6.1 to v0.6.2, v0.6.3
- pstore 0.2.0
0.1.4 to v0.2.0
- benchmark 0.5.0
0.4.0 to v0.4.1, v0.5.0
- logger 1.7.0
1.6.4 to v1.6.5, v1.6.6, v1.7.0
- rdoc 7.0.3
6.14.0 to v6.14.1, v6.14.2, v6.15.0, v6.15.1, v6.16.0, v6.16.1, v6.17.0, v7.0.0, v7.0.1, v7.0.2, v7.0.3
- win32ole 1.9.2
1.9.1 to v1.9.2
- irb 1.16.0
1.14.3 to v1.15.0, v1.15.1, v1.15.2, v1.15.3, v1.16.0
- reline 0.6.3
0.6.0 to v0.6.1, v0.6.2, v0.6.3
- readline 0.0.4
- fiddle 1.1.8
1.1.6 to v1.1.7, v1.1.8

The libraries above are no longer default gems. That means they can no longer be require-d without being in your Gemfile, so please be careful if you use them.

The following default gem is added.

win32-registry 0.1.2

The following default gems are updated.

RubyGems 4.0.3
bundler 4.0.3
date 3.5.1
- 3.4.1 to v3.5.0, v3.5.1
delegate 0.6.1
- 0.4.0 to v0.5.0, v0.6.0, 0.6.1]delegate-v0.6.1
digest 3.2.1
- 3.2.0 to v3.2.1
english 0.8.1
- 0.8.0 to v0.8.1
erb 6.0.1
- 4.0.4 to v5.1.2, v5.1.3, v6.0.0, v6.0.1
error_highlight 0.7.1
etc 1.4.6
fcntl 1.3.0
- 1.2.0 to v1.3.0
fileutils 1.8.0
- 1.7.3 to v1.8.0
forwardable 1.4.0
- 1.3.3 to v1.4.0
io-console 0.8.2
- 0.8.1 to v0.8.2
io-nonblock 0.3.2
io-wait 0.4.0
- 0.3.2 to v0.3.3, v0.3.5.test1, v0.3.5, v0.3.6, v0.4.0
ipaddr 1.2.8
json 2.18.0
- 2.9.1 to v2.10.0, v2.10.1, v2.10.2, v2.11.0, v2.11.1, v2.11.2, v2.11.3, v2.12.0, v2.12.1, v2.12.2, v2.13.0, v2.13.1, v2.13.2, v2.14.0, v2.14.1, v2.15.0, v2.15.1, v2.15.2, v2.16.0, v2.17.0, v2.17.1, v2.18.0
net-http 0.9.1
- 0.6.0 to v0.7.0, v0.8.0, v0.9.0, v0.9.1
openssl 4.0.0
- 3.3.1 to v3.3.2, v4.0.0
optparse 0.8.1
- 0.6.0 to v0.7.0, v0.8.0, v0.8.1
pp 0.6.3
- 0.6.2 to v0.6.3
prism 1.7.0
- 1.5.2 to v1.6.0, v1.7.0
psych 5.3.1
- 5.2.2 to v5.2.3, v5.2.4, v5.2.5, v5.2.6, v5.3.0, v5.3.1
resolv 0.7.0
- 0.6.2 to v0.6.3, v0.7.0
stringio 3.2.0
- 3.1.2 to v3.1.3, v3.1.4, v3.1.5, v3.1.6, v3.1.7, v3.1.8, v3.1.9, v3.2.0
strscan 3.1.6
- 3.1.2 to v3.1.3, v3.1.4, v3.1.5, v3.1.6
time 0.4.2
- 0.4.1 to v0.4.2
timeout 0.6.0
- 0.4.3 to v0.4.4, v0.5.0, v0.6.0
uri 1.1.1
- 1.0.4 to v1.1.0, v1.1.1
weakref 0.1.4
- 0.1.3 to v0.1.4
zlib 3.2.2
- 3.2.1 to v3.2.2

The following bundled gems are updated.

minitest 6.0.0
power_assert 3.0.1
- 2.0.5 to v3.0.0, v3.0.1
rake 13.3.1
- 13.2.1 to v13.3.0, v13.3.1
test-unit 3.7.5
- 3.6.7 to 3.6.8, 3.6.9, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.7.4, 3.7.5
rexml 3.4.4
rss 0.3.2
- 0.3.1 to 0.3.2
net-ftp 0.3.9
- 0.3.8 to v0.3.9
net-imap 0.6.2
- 0.5.8 to v0.5.9, v0.5.10, v0.5.11, v0.5.12, v0.5.13, v0.6.0, v0.6.1, v0.6.2
net-smtp 0.5.1
- 0.5.0 to v0.5.1
matrix 0.4.3
- 0.4.2 to v0.4.3
prime 0.1.4
- 0.1.3 to v0.1.4
rbs 3.10.0
- 3.8.0 to v3.8.1, v3.9.0.dev.1, v3.9.0.pre.1, v3.9.0.pre.2, v3.9.0, v3.9.1, v3.9.2, v3.9.3, v3.9.4, v3.9.5, v3.10.0.pre.1, v3.10.0.pre.2, v3.10.0
typeprof 0.31.1
debug 1.11.1
- 1.11.0 to v1.11.1
base64 0.3.0
- 0.2.0 to v0.3.0
bigdecimal 4.0.1
- 3.1.8 to v3.2.0, v3.2.1, v3.2.2, v3.2.3, v3.3.0, v3.3.1, v4.0.0, v4.0.1
drb 2.2.3
- 2.2.1 to v2.2.3
syslog 0.3.0
- 0.2.0 to v0.3.0
csv 3.3.5
- 3.3.2 to v3.3.3, v3.3.4, v3.3.5
repl_type_completor 0.1.12

There are a lot of updates.
I can't read them all, so I'll skip.

IRB: added the `copy` command

The copy command was added. It lets you copy output results to the system clipboard.

For example, if you want to copy model output in a Rails console like this, running copy after the output will copy from User:0x00.. to updated_at to the clipboard.

app(dev)> User.last
=>
#<User:0x0000000351352c40
 id: 1,
 email: "test@example.com",
 name: "Ruby",
 created_at: "2025-12-09 17:48:39.000000000 +0900",
 updated_at: "2025-12-09 17:48:39.000000000 +0900">
app(dev)> copy
Copied to system clipboard

(ima1zumi)

(ko1 addendum: it calls the system command for copying to the clipboard (like pbcopy), so it won't work in a terminal on an SSH host, for example.)

RubyGems and Bundler

Ruby 4.RubyGems and Bundler version 4. see the following links for details.

Supported platforms

Windows
- Dropped support for MSVC versions older than 14.0 (_MSC_VER 1900). This means Visual Studio 2015 or later is now required.

On Windows native, it seems you can no longer build without Visual Studio 2015 or later. Version numbers like 14.0, 1900, and 2015 for VC are complicated, with many different kinds.

(mame)

Compatibility issues

The following methoded from Ractor due to the addition of Ractor::Port:
- Ractor.yield
- Ractor#take
- Ractor#close_incoming
- Ractor#close_outgoging [Feature #21262]

This is also covered in the article.

[https://product.st.inc/entry/2025/06/24/110606:embed:cite]

ObjectSpace._id2ref is deprecated. [Feature #15408]

ObjectSpace._id2ref, which gets an object from its ID (the one you get via #object_id), was deprecated. The decision to deprecate it was made long ago, but for some reason it was forgotten, so now it is finally deprecated.

id = self.object_id
p ObjectSpace._id2ref(id) #=> main
# In Ruby 4.0, you get a warning -> warning: ObjectSpace._id2ref is deprecated

Process::Status#& and Process::Status#>> have been removed. They were deprecated in Ruby 3.3. [Bug #19868]

Process::Status#& and Process::Status#>> were removed. They were already deprecated in Ruby 3.3. They were used to peek at status codes, but now there are dedicated methods, so they are no longer used.

rb_path_check has been removed. This function was used for $SAFE path checking which was removed in Ruby 2.7, and was already deprecated,. [Feature #20971]

The rb_path_check function was removed. It depended on the $SAFE feature that was removed long ago, but it remained for compatibility, so it was removed this time.

(ko1)

Backtrace formatting changed

A backtrace for ArgumentError of "wrong number of arguments" now include the receiver's class or module name (e.g., in Foo#bar instead of in bar). [[Bug #21698]]

In backtraces for "wrong number of arguments", the class name now appears.

# Ruby 3.4
test.rb:1:in 'add': wrong number of arguments (given 1, expected 2) (ArgumentError)

# Ruby 4.0
test.rb:1:in 'Object#add': wrong number of arguments (given 1, expected 2) (ArgumentError)

The method name notation changed from add to Object#add.

Backtraces no longer display internal frames. These methods now appear as if it is in the Ruby source file, consistent wither C-implemented methods. [[Bug #20968]]

Recently, more built-in methods are being written in Ruby rather than C. As a result, backtraces often showed a mysterious file like <internal:???>.

# Ruby 3.4
$ ruby -e '[1].fetch_values(42)'
<internal:array>:211:in 'Array#fetch': index 42 outside of array bounds: -1...1 (IndexError)
        from <internal:array>:211:in 'block in Array#fetch_values'
        from nternal:array>:211:in 'Array#map!'
        from <internal:array>:211:in 'Array#fetch_values'
        from -e:1:in '<main>'

# Ruby 4.0
$ ruby -e '[1].fetch_values(42)'
-e:1:in 'Array#fetch_values': index 42 outside of array bounds: -1...1 (IndexError)
        from -e:1:in '<main>'

This was a virtual file name representing the source where built-in methods are defined. However, there is nothing a normal user can do with that file name, and it just adds work to skip those frames. In most cases, what the user wants to see is the caller of the built-in method.

So we removed <internal:*> from display and changed it to show as if the error occurred at the caller.

(mame)

I think it would be better if everything was shown, though.

(ko1)

About standard library compatibility

The CGI library was removed from the bundle

The CGI library is removed from the default gems. Now we only provide cgi/escape for the following methods:
- CGI.escape and CGI.unescape
- CGI.escapeHTML and CGI.unescapeHTML
- CGI.escapeURIComponent and CGI.unescapeURIComponent
- CGI.escapeElement and CGI.unescapeElement [Feature #21258]

Most of the cgi gem was removed from the bundle. In modern times it's judged not to be worth the maintenance cost.

Only the eight methods above such as CGI.escape are very commonly used, so a bundle remains in the form of cgi/escape. If you don't need the full cgi gem and only need these methods, you should require "cgi/escape".

If you need the entire cgi gem, please do gem install cgi or add gem "cgi" to your Gemfile.

(mame)

`SortedSet` was removed from the bundle

With the move of Set from stdlib to core class, set/sorted_set.rb has been removed, and SortedSet is no longer an autoloaded constant. Please install the sorted_set gem and require 'sorted_set' to use SortedSet. [Feature #21287]

SortedSet was removed from the bundle. If you need it, install the sorted_set gem.

(mame)

C API updates

IO
- rb_thread_fd_close is deprecated and now a no-op. If youto expose file descriptors from C extensions to Ruby code, create an IO instance using RUBY_IO_MODE_EXTERNAL and use rb_io_close(io) to close it (this also interrupts and waits for all pending operations on the IO instance). Directly closing file descriptors does not interrupt pending operations, and may lead to undefined behaviour. In other words, if two IO objects share the same file descriptor, closing one does not affect the other. [Feature #18455]

rb_thread_fd_close() was deprecated and now does nothing when called. This API used to accept a file descriptor directly, but at the Ruby level we want people to close via an IO object, so this change happened. If you want to manipulate a file descriptor directly, please use RUBY_IO_MODE_EXTERNAL when creating the IO object, and close it with rb_io_close(io).

rb_thread_call_with_gvl now works with or without the GVL.
This allows gems to avoid checking ruby_thread_has_gvl_p.
Please still be diligent about the GVL. [Feature #20750]

GVL (Global/Giant VM Lock, though it's not really that anymore since it's per-Ractor; recently we call it Great Valuable Lock) is released while doing some work, and sometimes you want to reacquire the GVL to do something (Ruby-related things require the GVL). There is an API called rb_thread_call_with_gvl() which takes a function f and calls f after acquiring the GVL. This API used to error if called when the GVL wasn't released, but now it can be called without error. There are cases where you want to call it regardless of whether the GVL is released, so it should be more convenient.

Personally, I was against it because I think you should be sensitive to GVL acquisition state, or rather avoid such shared code scenarios as much as possible, but it went in anyway. In any case, I think you should keep code that runs without the GVL as simple as possible.

Set
- A C API for Set has been added. The following methods are supported: 59](https://bugs.ruby-lang.org/issues/21459)]
  - rb_set_foreach
  - rb_set_new
  - rb_set_new_capa
  - rb_set_lookup
  - rb_set_add
  - rb_set_clear
  - rb_set_delete
  - rb_set_size

Because Set became built-in, these C APIs were introduced. By the way, there are already quite a few C APIs with the rb_set_ prefix (for example rb_set_errinfo()), so there was some talk that it could be confusing.

(ko1)

Implementation improvements

Class#new (ex. Object.new) is faster in all cases, but especially when passing keyword arguments. This has also been integrated into YJIT and ZJIT. [Feature #21254]

Calls like Foo.new are faster. It feels like a dedicated instruction was added. By the way, I did the rough design of the instruction.

GC heaps of different size pools now grow independently, reducing memory usage when only some pools contain long-lived objects

GC improvements. I think this is an adjustment of VWA.

GC sweeping is faster on pages of large objects

I don't know what was done here.

"Generic ivar" objects (String, Array, TypedData, etc.) now use a new internal "fields" object for faster instance variable access

A new data structure was introduced to hold instance variables for non-user-defined objects. I wonder what it was for. Ractor?

The GC avoids maintainial id2ref table until it is first used, making object_id allocation and GC sweeping faster

Since the data for managing id2ref is lazily generated, calls to #object_id and sweeping are faster as long as id2ref isn't used. I wonder how they did that.

object_id and hash are faster on Class and Module objects

object_id and hash are faster for classes and modules. What did they do here, I wonder.

Largeers can remain embedded using variable width allocation

Bignum (large integers) are also faster by managing them with VWA.

Random, Enumerator::Product, Enumerator::Chain, Addrinfo, StringScanner, and some internal objects are now write-barrier protected, which reduces GC overhead.

They added proper write barriers to some objects so they are no longer unprotected. GC becomes faster.

(I haven't followed this very closely.

(ko1)

Ractor

A lot of work has gone into making Ractors more stable, performant, and usable. These improvements bring Ractor implementation closer to leaving experimental status.

Performance improvements
- Frozen strings and the symbol table internally use a lock-free hash set [[Feature #21268]]
- Method cache lookups avoid locking in most cases
- Class (and generic ivar) instance variable access is faster and avoids locking
- CPU cache contention is avoided in object allocation by using a per-ractor counter
- CPU cache contention is avoided in xmalloc/xfree by using a thread-local counter
- object_id avoids locking in most cases
Bug fixes and stability
- Fixed possible deadlocks when combining Ractors and Threads
- Fixed issues with require and autoload in a Ractor
- Fixed encoding/transcoding issues across Ractors
- Fixed race conditions in GC operations and method invalidation
- Fixed issues with processes forking after starting a Ractor
- GC allocation counts are now accurate under Ractors
- Fixed TracePoints not working after GC [[Bug #19112]]

There were many performance improvements and bug fixes for Ractor. Mainly other people worked hard on it.
Although it's not written here, I think the introduction of the above Ractor::Port cleaned up functionality a lot, so that probably also improved performance.

JIT

ZJIT
- Introduce an experimental method-based JITompiler. Where available, ZJIT can be enabled at runtime with the --zjit option or by calling RubyVM::ZJIT.enable. When building Ruby, Rust 1.85.0 or later is required to include ZJIT support.
- As of Ruby 4.0.0, ZJIT is faster than the interpreter, but not yet as fast as YJIT. We encourage experimentation with ZJIT, but advise against deploying it in production for now.
- Our goal is to make ZJIT faster than YJIT and production-ready in Ruby 4.1.

ZJIT, the successor to YJIT, was introduced, and if you have a Rust environment (rustc 1.85.0 or later) at build time, it will be built together. You can use it with the --zjit command-line argument. It's not as fast as YJIT yet, so they ask you to wait before using it in production. The goal is to exceed YJIT in Ruby 4.1.

[https://rubykaigi.org/2025/presentations/maximecb.html:embed:ciJIT
* RubyVM::YJIT.runtime_stats
* ratio_in_yjit no longer works in the default build.
Use --enable-yjit=stats on configure to enable it on --yjit-stats.
* Add invalidate_everything to default stats, which is
incremented when every code is invalidated by TracePoint.
* Add mem_size: and call_threshold: options to RubyVM::YJIT.enable.

YJIT also seems to have improvements, but I'm not familiar, so I'll skip.

RJIT
- --rjit is r. We will move the implementation of the third-party JIT API to the ruby/rjit repository.

It seems the --rjit area was removed.

(ko1)

In closing

We have introduced new features and improvements in Ruby 4.0. Beyond what we covered here, there are also bug fixes and minor improvements. Please check with your Ruby applications.

With Ruby::Box, zjit, and the rebuild of Ractor, it's become a release with a lot to try. Please set it up locally and enjoy the new Ruby.

Enjoy Ruby programming!

(ko1/mame, guest: ima1zumi)

`Ractor::Port` - Revamping the Ractor API

Koichi Sasada — Thu, 25 Dec 2025 02:49:48 +0000

(This English version is an AI-generated translation of the original Japanese post (https://product.st.inc/entry/2025/06/24/110606), with some modification by hands, using Gemini)

I am Koichi from STORES, Inc. In this article, I will introduce Ractor::Port, a new mechanism we recently introduced to revamp parts of the Ractor API (Ruby's mechanism for easy parallel processing) that had been bothering me for a long time. I never quite felt comfortable with the previous API?it felt like a bone stuck in my throat?and after thinking about it for five years, I finally made the decision.

Proposed Ticket: Feature #21262: Proposal: Ractor::Port - Ruby Issue Tracking System

Overview of Changes

Here is a quick summary of what has changed:

Deprecated / Removed

Ractor#take, Ractor.yield, Ractor.receive_if, Ractor#incoming_port, and Ractor#outgoing_port are gone.

Use These Instead

Waiting for a Ractor to finish: → Ractor#join (Same as Thread#join)

  r = Ractor.new { fib(30) }
  r.join

Waiting for a Ractor to finish and getting its return value: → Ractor#value (Same as Thread#value)

  r = Ractor.new { fib(30) }
  p r.value #=> 1346269

Creating a dedicated communication path from other Ractors: → Ractor::Port (Think of it like a TCP port number)

  port1 = Ractor::Port.new
  port2 = Ractor::Port.new

  Ractor.new(port1) { |port| port << :hello }
  Ractor.new(port2) { |port| port << :world }

  p port2.receive #=> Guaranteed to be :world
  p port1.receive #=> Guaranteed to be :hello

Every Ractor has a "default port" from the start, which can be accessed via Ractor#default_port. Methods like Ractor#send and Ractor.receive operate on this default port. This means Ractor#send and .receive can still be used as before.

  r = Ractor.new { p Ractor.receive } # Displays 42
  r << 42

Additionally, there are methods like Ractor#monitor, unmonitor, and Ractor::Port#close, but they are not critical for typical use.

While we are usually very cautious about removing Ruby methods, Ractor has been labeled as "experimental" since Ruby 3.0. This allowed for a bold, incompatible change. Since it's still experimental, we assumed there aren't many people using it in production yet.

$ ruby -e "Ractor.new{}"
-e:1: warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues.

Background: Issues with the Existing API

Previously, communication between Ractors was organized into two models: Push (Ractor#send / .receive) and Pull (Ractor.yield / #take).

When trying to communicate with a "Server" Ractor that performs specific tasks using these primitives, things got complicated?especially as the number of Ractors increased.

Problem	Specific Example	Resulting Issue
Difficulty identifying messages	`Ractor.receive` is a single mailbox. Results from multiple servers get mixed up.	You might receive a message from Server B when you expected one from Server A.
Mailbox contention	Library A and Library B both call `Ractor.receive`.	One library might "accidently swallow" a message intended for the other.
Scattered APIs	Mixing `receive_if`, `yield`, `#take`, and `select`.	Increased learning/implementation cost and unstable CI.

Let's look at some code examples to understand these issues.

EX1: A Server with No Response

A simple "Fibonacci Server" that calculates numbers but doesn't return them.

def fib(n) = n > 1 ? fib(n-2) + fib(n-1) : 1

fib_srv = Ractor.new do
  while true
    param = Ractor.receive
    result = fib(param)
    # No way to send the result back
  end
end

fib_srv << 10

This is fine for "fire-and-forget" tasks like sending an email, but usually, we want the result back.

EX2: Sending the Caller Ractor Along

To get a response, we can send the sender's info.

fib_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fib(param)
    sender << result
  end
end

fib_srv << [10, Ractor.current]
do_some_work()
p Ractor.receive #=> fib(10)

This works for a simple 1-to-1 interaction.

EX3: Confusion with Multiple Servers

What happens if we add a "Factorial Server"?

def fact(n) = n > 1 ? fact(n-1) * n : 1

fib_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fib(param)
    sender << result
  end
end

fact_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fact(param)
    sender << result
  end
end


fib_srv << [10, Ractor.current]
fib_srv << [20, Ractor.current]
fact_srv << [10, Ractor.current]
fact_srv << [20, Ractor.current]

do_some_work()

Ractor.receive # Is this the result of fib(10) or fact(10)?

When Ractor.receive is called, you don't know which server sent the message. Furthermore, if a server uses internal worker Ractors, the order of responses might not even match the order of requests.

EX4: Identifying with Tags

We can add "tags" (like Symbols) to identify messages. This is similar to how Erlang handles it.

fib_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fib(param)
    sender << [[:fib, param], result]
  end
end

fact_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fact(param)
    sender << [[:fact, param], result]
  end
end

fib_srv << [10, Ractor.current]
fib_srv << [20, Ractor.current]
fact_srv << [10, Ractor.current]
fact_srv << [20, Ractor.current]

do_some_work()

Ractor.receive_if do |id, result|
  case id
  in [:fib, n]
    p "fib(#{n}) = #{result}"
  in [:fact, n]
    p "fact(#{n}) = #{result}"
  end
end

# or if you want to use specific results, like:

p fib20:  Ractor.receive_if{|id, result| id => [:fib, 20];  result}
p fact10: Ractor.receive_if{|id, result| id => [:fact, 10]; result}
p fact20: Ractor.receive_if{|id, result| id => [:fact, 20]; result}
p fib10:  Ractor.receive_if{|id, result| id => [:fib, 10];  result}

While this works, a problem remains: if an external library calls Ractor.receive internally, it might "steal" your message before you can call receive_if. You can only use this safely if you control all the code.

EX5: Creating a Channel

We could create a "Channel" using a dedicated Ractor, similar to Go language.

fib_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fib(param)
    sender << result
  end
end

fact_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fact(param)
    sender << result
  end
end

# Create a new channel using a Ractor
def new_channel
  Ractor.new do
    while true
      Ractor.yield Ractor.receive
    end
  end
end


fib_srv << [10, fib10_ch = new_channel]
fib_srv << [20, fib20_ch = new_channel]
fact_srv << [10, fact10_ch = new_channel]
fact_srv << [20, fact20_ch = new_channel]

do_some_work()

p fib20: fib20_ch.take   # wait for fib(20)
p fact10: fact10_ch.take # wait for fact(10)
p fib10: fib10_ch.take   # wait for fib(10)
p fact20: fact10_ch.take # wait for fact(20)

# or 
chs = [fib10_ch, fib20_ch, fact10_ch, fact20_ch]

while !chs.empty?
  ch, result = Ractor.select(*chs) # wait for multiple channels
  p ch, result
  chs.delete ch
end

This works, but:

Creating a Ractor for every channel is expensive.
Performance suffers due to extra object copying.
Conceptually, it feels less like the "Actor Model" and more like CSP.

I hesitated for five years because this didn't feel "Actor-like." Then, a proposal for "Ractor Channels" (Feature #21121) pushed me to rethink everything, leading to Ractor::Port.

The Proposal: `Ractor::Port` - Lightweight, Unique, One-Way Endpoints

Ractor::Port is a mechanism where "anyone can send to it," but "only the creator can receive from it."

Feature	Channel	Port
Sending	Anyone	Anyone
Receiving	Anyone	Only the creator

This small difference makes it much more aligned with the Actor model. It is lightweight to create and has overhead equivalent to Ractor#send.

A Port consists of two elements:

Destination Ractor: Where to deliver.
Unique Tag: An identifier unique to that Port.

To describe the concept, we can define Ractor::Port with existing (on Ruby 3.4) API.

class Ractor::Port
  def initialize
    @r   = Ractor.current
    @tag = genid()
  end

  def send(obj)               # anywan can send
    @r << [@tag, obj]
  end

  def receive                 # only the creator can receive
    raise unless @r == Ractor.current
    Ractor.receive_if { |(tag,res)| return res if tag == @tag }
    # receive if the tag is identical
  end
end

As you can see, it is enough simple and lightweight. Especially the implementation of synchronizations reduced dramatically.

For example, we can use Ractor::Port like:

port = Ractor::Port.new
Ractor.new(port) do |port|
    port.send 42 # `port << 42` can also be used as an alias
end

port.receive #=> 42

In this model, you pass a Port object to indicate, "I want you to send the data here," and the sender uses Ractor::Port#send to transmit it. Semantically, this is exactly the same as calling ractor.send(tag, 42) (and the internal implementation reflects this logic).

On the receiving side, you use Ractor::Port#receive to pick up only the messages that were specifically sent to that port.

In this simple example, since we've only introduced a single Port, it might feel more cumbersome than the traditional Ractor#send. However, the true significance and utility of this approach become clear when dealing with more complex scenarios involving multiple communication channels.

Refactoring with Ports

Now, let's look at how we can rewrite the fib and fact servers specifically using Ractor::Port.

fib_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fib(param)
    sender << result
  end
end

fact_srv = Ractor.new do
  while true
    param, sender = Ractor.receive
    result = fact(param)
    sender << result
  end
end

fib_srv << [10, fib10_port = Ractor::Port.new]
fib_srv << [20, fib20_port = Ractor::Port.new]
fact_srv << [10, fact10_port = Ractor::Port.new]
fact_srv << [20, fact20_port = Ractor::Port.new]

do_some_work()

p fib10_port.receive #=> fib(10)
p fib20_port.receive #=> fib(20)
p fact10_port.receive #=> fact(10)
p fact20_port.receive #=> fact(20)

Essentially, this is just replacing the previous Channel-based example with Ports. While the logic remains similar to an implementation using tag + receive_if, the key difference here is that the use of tags is now enforced. Additionally, it provides "peace of mind". You no longer have to worry about your messages being accidentally intercepted or "stolen" by a Ractor.receive call in some other part of the program.

Furthermore, Ractor.select has been updated to wait for multiple Ports simultaneously, allowing you to handle results as follows:

ports = [fib10_port, fib20_port, fact10_port, fact20_port]

while !ports.empty?
  port, result = Ractor.select(*ports)
  case port
  when fib10_port
    p fib10: result
  # ... handle other cases ...
  else
    raise "This should not happen (BUG)."
  end

  ports.delete(port)
end

In this example, if a message arrives at any of the four Ports stored in the ports array, Ractor.select returns that specific port along with the received result.

As a side note, it's worth comparing this to other systems:

IO.select: Returns the IO object that is "ready" (e.g., for reading), but you still have to perform the read yourself.
Ractor.select: Actually completes the reception process and returns the value.
Go's select statement: Requires you to write the processing logic for each branch within the statement itself.

Each approach handles the "select" concept slightly differently.

Comparison: Port vs. Channels/Queues

First, let's summarize the key advantages of Port:

Guaranteed Delivery: It ensures that messages are delivered only to the intended Ractor, preventing tag collisions.
Decoupled Communication: You can establish communication paths without relying on Ractor.receive, which is prone to unintended message consumption ("message stealing").
Simpler Primitives: It replaces complex and hard-to-compose primitives like .receive_if, .yield, and #take with a cleaner abstraction.
Natural Actor Semantics: It aligns perfectly with the Actor model semantics that Ruby aims to achieve with Ractor.

Let's discuss about "why Port is Better than Channel (in Practice)".

Safer than "Channel"
- When a Port#send succeeds, it guarantees that the destination Ractor is still alive and running.
- In contrast, with a Channel, there is no guarantee that a Ractor capable of receiving from that channel is currently active.
- While a Port doesn't guarantee the receiver will process the message (it might ignore it), it eliminates the failure case where a message vanishes into the void because the destination Ractor has already terminated. This removes a major source of non-deterministic bugs and makes the communication model much more predictable.
- Note: This predictability is one of the primary reasons Ruby chose the Actor model over CSP.
Superior Performance
- Lighter Creation: Creating a Channel requires allocating container structures to store messages. A Port only requires a lightweight "Ractor + unique ID" data structure.
- Fewer Copies: Sending a message via a Channel typically involves two copies: Sender → Channel and Channel → Receiver. With a Port, the message is copied only once: Sender → Receiver. As we move toward Ractor-local GC (separating object spaces), this performance gap will become even more significant.
Simplified Implementation
- Implementing Port#receive is straightforward because it only requires locking the receiving Ractor.
- Conversely, yield/take requires rendezvous synchronization, necessitating locks on both the sender and receiver simultaneously—which is notoriously tricky.
- The current implementation of .select is extremely difficult and remains unstable in CI (despite two major rewrites). Simplifying the specification to Ports will lead to fewer bugs and faster development.
- Personal Note: For me, the "Simplicity of Implementation" is the biggest win. Implementing yield/take and the supporting select logic was an absolute nightmare. I never managed to completely eliminate the bugs while maintaining high performance (avoiding a VM-wide global lock). Moving to Ports allows us to ditch that complexity.

They are downsides of Ports

Unfamiliarity: The concept of a "Port" isn't widely known, especially for developers coming from Go who are used to Channels.
Abstraction Overhead: For classic producer-consumer patterns, you may need an additional layer of abstraction.

For example, implementing Multiple Workers (Ractors) requires a bit more coordination:

controller = Ractor::Port.new

workers = (1..WN).times.map do
  Ractor.new controller do |controller|
    while true
      # Notify controller that this worker is ready
      controller << Ractor.current
      # Wait for a task from the producer
      port, param = Ractor.receive
      result = task(param)  
      # Send the result back to the requested port
      port << [param, result]
    end
  end
end

task_q = Queue.new
result_port = Ractor::Port.new

# In this new API, Ports are Thread-safe!
Thread.new do
  while param = task_q.pop
    # Get an available worker
    worker = controller.receive
    worker.send [result_port, param]
  end
end

task_q << 42
task_q << 43
task_q << 44

3.times do
  p result_port.receive
end

In this scenario, because the producer logic only supports a single Ractor, you would need a "mediator Ractor" if multiple Ractors were trying to feed tasks to these workers.

One interesting side effect of this refactoring is that Ractor Ports now support Threads. The previous Ractor API was essentially incompatible with Threads (it was "undefined behavior" territory), but the new, cleaner internal logic made it easy to add support.

One remaining concern is that it's difficult to implement something like Go's `context using only Ports. I'm still evaluating how much of a limitation this is in practice (and specifically, why Channel#close` wasn't sufficient for those use cases).

Waiting for Ractor Termination

If we remove Ractor#take, we lose the way to wait for a Ractor to finish and retrieve its block's return value. To address this, I decided to adopt the naming convention from Ruby's Thread class and introduced Ractor#join (to wait for termination) and Ractor#value (to wait and return the block's value).

r = Ractor.new { 42 }
r.join   # wait for the termination of r
r.value  #=> 42

A unique specification here is that Ractor#value can be called by at most one Ractor. If one Ractor is already waiting for a value via #value, and another Ractor attempts to call #value on the same target, it will raise an error.

r = Ractor.new { 42 }
p r.value #=> 42 (Main Ractor executes value)

Ractor.new(r) { |r| r.value } # This will raise an error

The reasoning behind this is safety: since the final result is not accessible by anyone else, it is safe for exactly one Ractor to access it. This allows us to pass the result without copying, meaning an "unshareable object" can be received as-is.

Actually, Ractor#take had a similar special behavior for the final return value. However, I always felt it was "dirty" (or inconsistent) for the same method to behave differently depending on the timing. I'm glad we could clean this up with Ractor#value.

Default Port and Compatibility

Every Ractor is equipped with one default port from the moment it is created. It is used, for example, to pass arguments to the Ractor, and it can be accessed via Ractor#default_port.

Standard operations like Ractor#send and Ractor.receive are now essentially operations on this default port.

r = Ractor.new do
  Ractor.receive #=> 42
  # This is equivalent to Ractor.current.default_port.receive
end

r << 42
# Semantically the same as r.default_port.send
# (We might restrict access to default_port from outside the Ractor in the future)

Monitor API

To implement join and value, I introduced a low-level Monitor API (inspired by Erlang).

r = Ractor.new { 42 }
r.monitor(monitor_port = Ractor::Port.new)

monitor_port.receive #=> :exited

When a Ractor terminates, it sends the symbol :exited to the registered monitor_port. If it crashes due to an exception, it sends :aborted. I'm not sure yet if we'll add more event types (we'll see if use cases arise).

This API makes implementing #join very simple (in fact, it is already implemented this way: see code). You can also use this to monitor the liveness of multiple Ractors:

monitor_port = Ractor::Port.new

10.times.map do
  r = Ractor.new { unlimited_task }
  r.monitor monitor_port
end

while true
  msg = monitor_port.receive
  do_something # e.g., restart a worker to replace the dead one
end

(Wait, looking at this now... if I only get :exited, I won't know which Ractor died. I'll probably need to include the source Ractor information. I'm debating between passing more info to the monitor port or creating a receive method that identifies the sender. We'll see which is better.)

I originally designed Ractor#take five years ago specifically to handle this kind of logic, but it turns out things don't always go as planned!

Closing Ports

Ports can be closed using the #close method.

Ractor.new(port = Ractor::Port.new) do |port|
  loop { 
    port << 42 # Eventually raises ClosedError when trying to send to a closed port
  }
end

port.close

Currently, only the Ractor that created the port can close it. Allowing anyone to close a port is a bit more complex (as it requires "waking up" all waiting threads), so I'll keep it this way until there is a clear, essential need for it.

Other Alternatives Considered

As mentioned, I considered Channel-like mechanisms.

Since a Port essentially enforces "tagged sends," another idea was to use Ractor#send(tag, ...). However, I was concerned about tag collisions. I decided that if we were going to use tags anyway, it would be more "Object-Oriented" to make the tag itself an object, hence, the Port object.

I didn't spend too much time on the name "Port"; it just seemed to fit, and I couldn't think of many better alternatives. I looked into other languages for similar mechanisms but didn't find many equivalents, save for the concept of TCP port numbers.

Future Plans

Ractor::Port has already been merged into the master branch (and will be shipped with Ruby 4.0), so you can try it out today. I would love to hear your honest feedback.

There are still a few "leftover" items I'm considering:

Is `Ractor::Port.new` too long?

I’ve been wondering if something like Ractor.port would be better (similar to the feel of IO.pipe). Since creating a port doesn't feel like a heavy operation, maybe a shorter factory method makes sense. What do you think?

Behavior on `close`

As a prerequisite, remember that a Port is an unshareable object. This means a copy is created whenever it is passed to another Ractor.

While implementing this, I realized something: even if other Ractors hold a copy of a Port to send messages to, if all local references to that Port within the receiving Ractor are garbage collected (GC'd), there is no longer anyone left to read from it. In that case, the Port should probably be closed automatically.

port = Ractor::Port.new
Ractor.new(port) do |port|
  10.times { port << it } # Sending 10 messages
end

p port.receive #=> 0 (Returns the first message)
port = nil     # The handle to receive the remaining 9 messages is gone!
...
# The program continues, but those 9 messages stay in memory indefinitely.

In cases like this, I believe the Port should close the moment it is GC'd in the receiving Ractor. This would allow us to:

Reclaim memory: Free up the messages that were already sent (since no one can ever see them).
Prevent waste: Stop other Ractors from performing unnecessary work by sending messages to a dead-end.

I'm considering using reference counting within the Port implementation to achieve this.

Also, as I touched on earlier, there is the question of whether other Ractors (senders) should be allowed to close a Port. It might be a good way to signal "I'm done sending," but it might also complicate the ownership model. The specification here is still likely to evolve.

Summary

With the introduction of Ractor::Port, the Ractor API has undergone a significant overhaul. I believe this refactoring clarifies how Ractors communicate and makes the entire system much more approachable.

Please take it for a spin and see how it feels!

And hopefully, this time, we can finally see that "experimental" warning removed from Ractor soon.

Reading the Ruby 3.4 NEWS with professionals (English translation)

Koichi Sasada — Fri, 27 Dec 2024 02:45:49 +0000

This article is Japanese->English translation of プロと読み解くRuby 3.4 NEWS - STORES Product Blog, mainly using DeepL.

This is Koichi Sasada (ko1) and Yusuke Endoh (mame) from the Technology Infrastructure Group in the Technology Division at STORES, Inc. We are developing the Ruby interpreter (MRI: Matz Ruby Implementation, or the ruby command). We are professional Ruby committers because we are paid to develop Ruby.

Ruby 3.4.0 was released today, December 25th, as the annual Christmas release (Ruby 3.4.0 Release). This year, we will also be explaining the NEWS.md file for Ruby 3.4 on the STORES Product Blog (incidentally, this will be an article for the STORES Advent Calendar 2024, written in Japanese). Please see the previous article for an explanation of what a NEWS file is.

This article not only explains the new features, but also describes the background to the changes and the difficulties involved, to the best of our recollection.

Ruby 3.4 saw almost no changes to the language (grammar). This was because the decision was made to hold off on any grammar changes in order to focus on the changing the parser that interprets the grammar to Prism.

The main changes in Ruby 3.4 are as follows (excerpt from the release notes)

it is added
The default parser is changed to Prism
The Socket library is updated to support Happy Eyeballs Version 2 (RFC 8305)
YJIT is updated
Modular GC

In this article, we will introduce the items in the NEWS file, including these.

Language Changes

Block parameter `it` is available

it is added to reference a block parameter. [Feature #18980]

The it reference to a block parameter without naming it has finally been introduced.

ary = ["foo", "bar", "baz"]

p ary.map { it.upcase }  #=> ["FOO", "BAR", "BAZ"]

it makes it easier to write simple blocks that are easier to read.

Difference from numbered parameter

it is roughly the same as _1 in numbered parameter. However, the restrictions on where it can be written are a little more relaxed.

"foo".then do
  p "bar".then { _1.upcase } #=> "BAR"
  p # A syntax error if you write _1 here
end

"foo".then do
  p "bar".then { it.upcase } #=> "BAR"
  p it.upcase #=> "FOO" # this is allowed with `it`
end

Even though it is allowed, it is better to avoid using it in anything other than one-line blocks, as it makes it difficult to understand what it refers to.

Also, while numbered parameters allow you to access the n-th argument with _2 or _3, it does not have such a function.

By the way, it is not allowed to refer to it and numbered parameters in the same block.

[1, 2, 3].then { p [it, _1, _2] } # This is a syntax error

Background to the introduction of `it`

When numbered parameters were introduced, there were a number of proposals for how to write them, and it was one of them [Feature #15897]. Or rather, I was the one who proposed it.

There were quite a few people who liked the name it, but there were compatibility issues. The method name it is used a lot in existing code (mainly RSpec), so if it was simply made into a keyword, it would cause a fatal incompatibility. To avoid this, we invented a tricky trick of “making the keyword only for the call of it with no arguments”.

However, it does not have a function to read the n-th argument, and because we were not sure whether that was okay, the notation _1 was eventually adopted.

After that, as people started to use numbered parameters in practice, it became clear that the function to read the second and subsequent arguments was not used very much, and that _1 was overwhelmingly more commonly used.

_1, especially, the number part "1", had a cognitively taxing issue. Therefore, it was decided to introduce it as a more visually friendly alternative to _1. In Ruby 3.3, the un-parameterized call to the method it was warned to facilitate the transition, and it is now available in Ruby 3.4.

(mame)

frozen_string_literal will be default and a migration path is added

String literals in files without a frozen_string_literal comment now emit a deprecation warning when they are mutated. These warnings can be enabled with -W:deprecated or by setting Warning[:deprecated] = true. To disable this change, you can run Ruby with the --disable-frozen-string-literal command line argument. [Feature #20205]

There used to be a feature called frozen_string_literal that would freeze string literals, and this will be the default. However, this will not be the case in Ruby 3.4, but rather in a later version (so it's “in the direction of”).

As a concrete change in Ruby 3.4, when you run with the -w or -w:deprecated option (when you enable deprecated warnings), you will now get warnings about changes (destructive operations) to string literals.

$ ruby -W:deprecated -e '"" << ?a'
-e:1: warning: literal string will be frozen in the future (run with --debug-frozen-string-literal for more information)

As the warning message says, if you add --debug-frozen-string-literal, it will display where the string that was subject to the destructive operation was created.

$ ruby -W:deprecated --debug-frozen-string-literal -e 's = ""
s << ?a'
-e:2: warning: literal string will be frozen in the future
  # warning because the destructive operation is performed on the second line
-e:1: info: the string was created here
  # displays that it is for the string created on the first line

As before, the frozen_string_literal pragma is available.

$ ruby -W:deprecated -e '# frozen_string_literal: true
> "" << ?a'
-e:2:in '<main>': can't modify frozen String: "", created at -e:2 (FrozenError)

And if you specify # frozen_string_literal: false, which was the default until now and probably no one applied, this warning will no longer be displayed. In other words, it is a declaration that string literals are not frozen.

$ ruby -W:deprecated -e '# frozen_string_literal: false
"" << ?a'

If you want to specify this behavior for the entire application, specify --enable-frozen-string-literal or --disable-frozen-string-literal.

$ ruby -W:deprecated --enable-frozen-string-literal -e 'a = ""
a << ?a'
-e:2:in '<main>': can't modify frozen String: "" (FrozenError)

$ ruby -W:deprecated --disable-frozen-string-literal -e 'a = ""
a << ?a'
# No warnings

In other words, if you specify --disable-frozen-string-literal, the warning will not be displayed in the same way as before.

As the warning “warning: literal string will be frozen in the future” indicates, the destructive operation on the frozen string literals will cause an error. Looking at the ticket, it says that it will be changed to an error from Ruby 4.0. Well, it's just a plan, so we don't know what will happen.

Internally, strings that issue a warning when a destructive operation is performed (and will become frozen in the future) are called "Chilled Strings", but this is knowledge that is not necessary if you are using Ruby normally.

Background to this story:

The idea of freezing string literals was planed to introduce in Ruby 3.0, and there was a lot of discussion about it in Ruby 2.x era.

The main reasons for this are thought to be (1) there are performance benefits because if string literals are frozen, the same string can be returned at that point, and (2) it is easier to program if they are frozen as much as possible, but I think the main reason you hear about it is (1).

However, it has not been introduced so far because the disadvantages, such as incompatibility, are greater than the benefits. However, there are many people who like this feature, and there are many people who write # frozen_string_literal: true at the beginning of their scripts. It is also often inserted automatically with editors support.

Therefore, if everyone is writing it in the first place, why not make this behavior the default and not have to write this troublesome pragma? This is the reason why we are trying to make it the default again this time.

By the way, according to the comments on Feature #20205: Enable frozen_string_literal by default - Ruby master - Ruby Issue Tracking System, the following percentage of the files in the source code of published RubyGems specify # frozen-string-literal: true.

Number of gems that have `# frozen-string-literal: true` in all .rb files

  among all public gems: 14254 / 175170 (8.14%)
  per-file basis: 445132 / 3051922 (14.59%)

  among public gems with the latest version released in 2015-present: 14208 / 101904 (13.94%)
  per-file basis: 443460 / 1952225 (22.72%)

  among public gems with the latest version released in 2020-present: 11974 / 41205 (29.06%)
  per-file basis: 389559 / 1136445 (34.28%)

  among public gems with the latest version released in 2022-present: 9121 / 26721 (34.13%)
  per-file basis: 329742 / 848061 (38.88%)

About 15% overall. It seems that it is included in about 35% of the files that have been released recently (since 2022). How about it? Was it a lot? Was it a little?

Up until now, it was easy to prepare a string (buffer) that could be changed casually like s = ""; s << s2, but now it requires a little more effort, such as s = "".dup; s << s2 or s = +""; s << s2, so I think it will be a fairly big change in Ruby's mindset.

But I guess it's not that big a deal for people who have written # frozen_string_literal: true?

By the way, --enable-frozen-string-literal was introduced when we were discussing Ruby 2.x in order to test what would happen if string literals were frozen by default (I wrote a patch a long time ago). The --disable-frozen-string-literal option, which was introduced at the same time, has finally seen the light of day.

(ko1)

Incidentally, I really hates this change. Ruby objects should be mutable in principle.

(mame)

The conditions for whether `String#+@` dups have changed

String#+@ now duplicates when mutating the string would emit a deprecation warning, offered as a replacement for the str.dup if str.frozen? pattern.

This is on the heels of defaulting to frozen_string_literal.

Until now, String#+@, i.e. +"foo" would do nothing if the string was mutable, but it would now duplicate the string if it was a mutable string and would warn about destructive operations (== a chilled string).

a = "str"
b = +a
p a.equal?(b) #=> false, i.e. it is a different object

c = "str".dup # No warning is given if you make a destructive change to c
d = +c
p c.equal?(d) #=> true, i.e. the same object without dup

This means that the only behaviour that used to be like for c untile now, is now like for a.

In other words, if you see the warning in the example, you can now decide to add +.

(ko1)

Keyword arguments `**` can now be passed `nil`

Keyword splatting nil when calling methods is now supported. **nil is treated similarly to **{}, passing no keywords, and not calling any conversion methods. [Bug #20064]

It is now possible to pass nil to ** to pass a hash as a keyword argument.

def hello(name: "default")
  puts "Hello, #{ name }"
end

# Pass a hash as a keyword argument
h = { name: "mame" }
foo(**h) #=> Hello, mame

# If you pass nil instead of a hash, it is the same as specifying an empty hash
h = nil
foo(**h) #=> Hello, default

This was allowed because Matz was persuaded by the following code example.

h = if params.key?(:name)
  { name: params[:name] }
end

User.new(**h)

Personally, I wouldn't recommend using the nil implicitly returned by if.

(mame)

Block passing is no longer allowed in index assignment

Block passing is no longer allowed in index assignment (e.g. a[0, &b] = 1). [Bug #19918]

I don't think anyone would do this, but it was previously possible to pass a block as a[0, &b] = 1.
If you were to do it properly, there were a lot of things to consider, such as the order of evaluation.

(ko1)

Keyword arguments are no longer allowed in index assignment (e.g. `a[0, kw: 1] = 2`).

Keyword arguments are no longer allowed in index assignment (e.g. a[0, kw: 1] = 2). [Bug #20218]

The same applies here, but the way of writing a[0, kw: 1] = 2 is now prohibited.
It seems that there were people actually using it, but it seems that it has now been prohibited.

(ko1)

The toplevel constant `Ruby` is reserved

The toplevel name ::Ruby is reserved now, and the definition will be warned when Warning[:deprecated]. [Feature #20884]

The toplevel constant Ruby has been reserved. Defining it will result in a warning.

# Run with `ruby -w`

# A warning is given when you define a Ruby module yourself
module Ruby #=> warning: ::Ruby is reserved for Ruby 3.5
end

# A warning is also given when you assign to the Ruby constant
Ruby = 1 #=> warning: ::Ruby is reserved for Ruby 3.5

In Ruby 3.5, the Ruby module will be defined, and it is planned to contain meta-functions related to Ruby that should be provided commonly by Ruby implementations. What will be placed in the module has not yet been decided, but it is likely that Ruby::VERSION will be defined as an alias for RUBY_VERSION.

(mame)

Update of built-in classes

`Array#fetch_values` was added

Array#fetch_values was added. [Feature #20702]

Since Hash#fetch_values exists, Array#fetch_values is also added.

["foo", "bar", "baz"].fetch_values(1, 2)         #=> ["bar", "baz"]
["foo", "bar", "baz"].fetch_values(1, 4)         #=> index 4 outside of array bounds: -3...3 (IndexError)
["foo", "bar", "baz"].fetch_values(1, 4) { 42 }  #=> ["bar", 42]

It seems that Array#values_at returns nil for out-of-bounds access, whereas Array#fetch_values raises an IndexError (or calls the block if given).

(mame)

`Exception#set_backtrace` now accepts arrays of `Thread::Backtrace::Location`

Exception#set_backtrace now accepts arrays of Thread::Backtrace::Location. Kernel#raise, Thread#raise and Fiber#raise also accept this new format. [Feature #13557]

The background explanation is very long. This is not a feature that we want to be used very much, so please only read it if you really want to.

First of all, Ruby exception backtraces can be programmatically retrieved using the Exception#backtrace method.

def foo
  raise RuntimeError, "exception"
end

begin
  foo
rescue
  pp $!.backtrace #=> ["test.rb:2:in 'Object#foo'", "test.rb:6:in '<main>'"]
end

However, if you wanted to extract just the file name from each line of the backtrace, you had to use regular expressions to extract it. That's why the Exception#backtrace_locations method was introduced. This returns an array of Thread::Backtrace::Location instances, so you can easily extract the file name, line number, etc.

def foo
  raise RuntimeError, "exception"
end

begin
  foo
rescue
  loc = $!.backtrace_locations.first

  # Looks like a string, but is actually an instance of Thread::Backtrace::Location
  p loc        #=> "test.rb:2:in 'Object#foo'"
  p loc.class  #=> Thread::Backtrace::Location

  # You can easily get the file name and line number
  p loc.path   #=> "test.rb"
  p loc.lineno #=> 2
end

You can also get the backtrace at the time of the call using Kernel#caller_locations.

def foo
  loc = caller_locations.first
  p loc         #=> "test.rb:8:in '<main>'"
  p loc.path    #=> "test.rb"
  p loc.lineno  #=> 8
end

foo

By the way, you can replace the exception backtrace with the third argument of raise.

def foo
  raise RuntimeError, "exception", ["foo", "bar"]
end

begin
  foo
rescue
  pp $!.backtrace #=> ["foo", "bar"]
  pp $!.backtrace_locations #=> nil
end

When the backtrace is specified as an array of strings like this, the interpreter cannot know the actual file name or line number, so it cannot create Thread::Backtrace::Location. Therefore, Exception#backtrace_locations returns nil.

Now, what happens if you specify caller_locations instead of an array of strings as the third argument to raise?

# Ruby 3.3
def foo
  raise RuntimeError, "exception", caller_locations
    #=> in `set_backtrace': backtrace must be Array of String (TypeError)
end

foo

This raises an error because there is a validation in Exception#set_backtrace that backtrace must be an array of strings.

Here's the change we're making. Exception#set_backtrace now accepts an array of Thread::Backtrace::Location.

# Ruby 3.4
def foo
  raise RuntimeError, "exception", caller_locations
end

begin
  foo
rescue
  pp $!.backtrace_locations  #=> ["test.rb:6:in '<main>'"]
end

It seems that you can use this when you want to cut the backtrace or copy it from another exception. However, as I explained at length, if you try to fake the backtrace, it will make debugging more difficult, so I think it's better not to do it.

(mame)

A mechanism for offloading time-consuming operations using the Fiber Scheduler has been introduced

An optional Fiber::Scheduler#blocking_operation_wait hook allows blocking operations to be moved out of the event loop in order to reduce latency and improve multi-core processor utilization. [Feature #20876]

The Fiber scheduler can now accept a blocking_operation_wait callback. This callback is provided to allow the Fiber scheduler to create a separate thread to offload time-consuming operations that attempt to release the GVL, such as zlib processing.

In practice, I think it would be difficult to use, so I think it would be better to leave it to the framework.

(ko1)

`IO::Buffer#copy` now releases the GVL

IO::Buffer#copy can release the GVL, allowing other threads to run while copying data. [Feature #20902]

When executing IO::Buffer#copy, the GVL is now released, allowing other threads to run. This can also be handled by blocking_operation_wait we explained before.

(ko1)

`GC.config` added to allow accessing configurations on the Garbage Collector & new control over generational GC

GC.config added to allow setting configuration variables * GC.config added to allow setting configuration variables on the Garbage Collector. [Feature #20443]
GC configuration parameter rgengc_allow_full_mark introduced. When false GC will only mark young objects. Default is true. [Feature #20443]

GC has various settings, and the GC.config method has been added to access those settings.

$ ruby -e 'pp GC.config'
{rgengc_allow_full_mark: true, implementation: “default”}

You can change the settings like this.

$ ruby -e 'p GC.config(rgengc_allow_full_mark: false)'
{rgengc_allow_full_mark: false}

The changed settings are returned. Hmm, I wonder if it's okay not to return the values before the change...

Currently, there are two types of settings that can be taken: the rgengc_allow_full_mark setting and the implementation setting. I think that the number of these settings will continue to increase in the future.

The implementation setting, which we will introduce later in this article, is being changed so that the GC implementation can be replaced, and the name of the GC implementation is entered here (of course, it is read-only). The example value is “default”, so we can see that it is the same GC implementation as before.

The rgengc_allow_full_mark setting determines whether or not to allow time-consuming major GC to target objects in older generations in generational GC. The default is true.

If you set this to false, major GC will not occur even if there are many objects in the old generation (more precisely, the old objects are not incremented while the configuration is false). This was introduced to create a modern version of the old OoB GC. In other words, it is now possible to control the system so that major GC, which takes time, is prohibited while requests are processed, and perform major GC when there is time to spare. I don't know much about it, but I think it will be included in Rails (this feature was included based on a discussion about making OOB GC the default in Ruby on Rails (Run GC out-of-band by default · Issue #50449 · rails/rails).

Let's try it out a bit.

$ ruby -e 'GC.config(rgengc_allow_full_mark: false)
           pp GC.stat;
           a = []
           10_000_000.times{ a << [] }
           pp GC.stat'
{count: 4,
 minor_gc_count: 2,
 major_gc_count: 2,
 old_objects: 12017,
 (omitted)
}
{count: 17,
 minor_gc_count: 16,
 major_gc_count: 2,
 old_objects: 12017,
 (omitted)
}

major_gc_count, that is, we can see that a major GC has not occurred. Also, since the number of old_objects has not changed, we can see that the objects did not become old-generation objects in the first place.

If you check GC.latest_gc_info(:needs_major_by), you can see whether a major GC is needed now (if it's truthy, it wants to do a major GC).

It's a difficult feature to handle, so it's probably best to leave it to the framework.

By the way, the GC.config method itself was introduced as a general GC.config method because it was thought that more settings would be added, based on discussions about how to handle the rgengc_allow_full_mark setting.

(ko1)

It is now possible to specify the capacity required when generating a hash

Hash.new now accepts an optional capacity: argument, to preallocate the hash with a given capacity. This can improve performance when building large hashes incrementally by saving on reallocation and rehashing of keys. [Feature #19236]

h = Hash.new
10_000_000.times{ h[it] = it }

When creating a hash with 10 million elements, this program allocates memory during iteration, but if you know that you're going to create 10 million elements from the start, it seems more efficient to allocate the memory all at once at the start.

So, we've changed it so that Hash.new(capacity: n) allocates memory for n elements at the start. This is likely to be faster than allocating memory a little at a time. Let's try it.

$ time ruby -e 'n = 10_000_000; h = Hash.new; n.times{ h[it] = it }'

real    0m2.842s
user    0m2.485s
sys     0m0.294s

$ time ruby -e 'n = 10_000_000; h = Hash.new(capacity: n); n.times{ h[it] = it }'

real    0m2.122s
user    0m1.996s
sys     0m0.072s

capacity: makes it a little faster.

To be honest, I don't think there are many cases where this would be effective, but if you find one, please give it a try.

(ko1)

Integer multiplication no longer returns `Float::INFINITY`

Integer#** used to return Float::INFINITY when the return value is large, but now returns an Integer. If the return value is extremely large, it raises an exception. [Feature #20811]
Rational#** used to return Float::INFINITY or Float::NAN when the numerator of the return value is large, but now returns a Rational. If it is extremely large, it raises an exception. [Feature #20811]

When the result of an integer power of an integer is a very large number, Ruby 3.3 and earlier would return Float::INFINITY, but from Ruby 3.4, it will now calculate the result directly, even if it takes a long time.

# Ruby 3.3: Returns Float::INFINITY while also issuing a warning "warning: in a**b, b may be too big"
p 10**10000000 #=> Infinity

# Ruby 3.4: Calculates as is
p 10**10000000 #=> 10000000000000000000000000000.....

The trigger of this change was the news that a new largest prime number had been discovered in October this year. Endoh tried to calculate it in Ruby by executing 2**136279841-1, but it returned Float::INFINITY. A disappointing experience. In fact, I had the same disappointing experience six years ago when the previous largest prime number was discovered, and I thought I would be disappointed again when the next largest prime number is discovered if nothing was done, so I proposed that we change it.

The original behavior was probably to avoid time-consuming calculations, but it was difficult to imagine a situation where INFINITY would be returned and be a happy situation, and also, with the use of GMP, even large prime numbers are not so slow anymore (it takes about 1 second to calculate 2**136279841-1 on my laptop, and about 10 seconds for the radix conversion for display), so it was decided that calculations could be done as they were.

Also, Rational integer powers have changed in the same way.

(mame)

A byte-based version of `MatchData#begin` has been introduced

MatchData#bytebegin and MatchData#byteend have been added. [Feature #20576]

MatchData#bytebegin has been introduced as a byte-based version of MatchData#begin. The same applies to #byteend.

"あいう" =~ /い/

# The position of “い” is the first character...second character
p $~.begin(0) #=> 1
p $~.end(0)   #=> 2

# “い” is at bytes 3...6
p $~.bytebegin(0) #=> 3
p $~.byteend(0)   #=> 6

(mame)

`Object#singleton_method` now sees modules extended

Object#singleton_method now returns methods in modules prepended to or included in the receiver's singleton class. [Bug #20620]

o = Object.new
o.extend(Module.new{def a = 1})
o.singleton_method(:a).call #=> 1
# Previously, this raised an error: singleton_method': undefined singleton method `a'

It seems that Object#singleton_method did not return the method of the extended module before, but it now works properly.

Ractor can now require

require in Ractor is allowed. The requiring process will be run on the main Ractor. Ractor._require(feature) is added to run requiring process on the main Ractor. [Feature #20627]

It was not possible to require on Ractor until now. This is because

it is unclear how to manage state such as $LOADED_FEATURES
almost all libraries are expected to be executed on the main Ractor

For toy-like applications, it's fine to require all libraries before using Ractor, but as the scale increases, things like require-ing in methods (e.g. the pp library is required when the pp method is first executed) and the existence of autoload become major problems.

So, we changed the process so that require is done in the main Ractor when require is called in another Ractor.

However, there are various libraries in the world that redefine require. In such cases, you will need to determine whether you are in the main Ractor yourself, and if not, use the Ractor._require(feature) method to execute require in the main Ractor. Well, I don't think any of our readers will be redefining require like that, so I don't think you need to remember this. Right?

Ractor.main? is added. [Feature #20627]

Ractor.main? has been introduced as a convenient method for determining whether the currently running Ractor is the main Ractor.

When defining your own require, please use a combination of these methods, as follows.

  def require(feature)
    if !Ractor.main?
      Ractor._require(feature)
    else
      # do something with your original require process
    end
  end

Incidentally, when redefining Kernel#require, an anonymous module that behaves as described above is automatically prepended to Kernel when Ractor.new is first called.

      Kernel.prepend Module.new{|m|
        m.set_temporary_name '<RactorRequire>'

        def require feature
          if Ractor.main?
            super
          else
            Ractor._require feature
          end
        end
      }

So, for example, I think that Kernel#require as defined by rubygems can be used without any changes, but if you want to extend it with prepend or define Object#require on your own, you will still need to use Ractor._require. Well, you wouldn't do that, right?

(ko1)

More ways to access the Ractor local storage

Ractor.[] and Ractor.[]= are added to access the ractor local storage of the current Ractor. [Feature #20715]

In order to access the area prepared for each Ractor (Ractor local storage), Ractor#[key]/[key]= was prepared, but in fact, even if the receiver was another Ractor, it was accessing the local storage of its own Ractor. Without warning.

So, it was decided that a class method would be sufficient, and Ractor.[]/[]= was introduced. It would be a problem if it were possible to access another Ractor. However, since it might be used for error handling, etc., it seems that the current Ractor#[] will remain with an ambiguous specification.

Ractor.store_if_absent(key){ init } is added to initialize ractor local variables in thread-safty. [Feature #20875]

Ractor local storage is a kind of global variable space (a global variable space that is different for each Ractor).

Now, when you want to set a value for each Ractor, it is not good if a multi-threaded program is running in the Ractor.

Ractor[:cnt] ||= 0
m = (Ractor[:m] ||= Mutex.new)
m.synchronize do
  Ractor[:cnt] += 1
end

At first glance, this program looks fine, but if you run this part in a multi-threaded environment, there is a possibility that m will take a different Mutex each time. There is also a possibility that :cnt will be initialized in an unintended way.

Therefore, a method called Ractor.store_if_absent(key){ init } was introduced to perform the setting atomically. When the key corresponding to the key has not yet been initialized, the value obtained by executing the block is set as the value for that key. Since these initializations are performed atomically, they will not be disturbed by other threads.

The program mentioned above can be made thread-safe by doing the following.

m = Ractor.store_if_absent(:m){ Ractor[:cnt] = 0; Mutex.new }
m.synchronize do
  Ractor[:cnt] += 1
end

Specifically, I've added timeout.rb to make it Ractor-compatible for now.

(ko1)

`Range#size` now raises `TypeError` if the range is not iterable.

Range#size now raises TypeError if the range is not iterable. [Misc #18984]

Range#size now raises TypeError if the range is not iterable, such as (0.1 ... 1).

(0.1 ... 1).size  #=> 'Range#size': can't iterate from Float (TypeError)
# Until Ruby 3.3, it returned 1

The same applies to beginless ranges such as (...0).

`Range#step` semantics have changed slightly

Range#step now consistently has a semantics of iterating by using + operator for all types, not only numerics. [Feature #18368]

In simple terms, Range#step can now be used with Time ranges.

pp (Time.new(2000, 1, 1)...).step(86400).take(3)
# => [2000-01-01 00:00:00 +0900,
# 2000-01-02 00:00:00 +0900,
# 2000-01-03 00:00:00 +0900]

Date and other ranges also work as expected.

In actual behavior, Range#step repeatedly adds the step argument to the Range#begin value using the + method, and stops when it reaches Range#end. Until now, it was built in accordance with the values in Range#begin and Range#end, so it could be said that the meaning has been clarified (however, there are still exceptions, such as the fact that the + method is not actually called for Integer ranges, and the way String ranges are handled).

Now it works without having to deal with Time, Date, etc. individually, but as a side effect, it behaves a little strangely with objects for which the “+” method doesn't seem to work like “addition”. I don't think you need to be careful about this, but just in case.

pp ([]...).step([1]).take(10)
[[],
[1],
[1, 1],
[1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1]]

(mame)

`RubyVM::AbstractSyntaxTree::Node#locations` was introduced

Add RubyVM::AbstractSyntaxTree::Node#locations method which returns location objects associated with the AST node. [Feature #20624]
Add RubyVM::AbstractSyntaxTree::Location class which holds location information. [Feature #20624]

The #locations method, which returns location information in the code of RubyVM::AbstractSyntaxTree::Node, has been introduced.

# Get location information from the node obtained by parsing “1 + 1”.
loc = RubyVM::AbstractSyntaxTree.parse("1 + 1").locations.first
=> #<RubyVM::AbstractSyntaxTree::Location:@1:0-1:5>

# We can see that it extends from the 0th column of the 1st line to the 5th column of the 1st line
p loc.first_lineno #=> 1
p loc.first_column #=> 0
p loc.last_lineno  #=> 1
p loc.last_column  #=> 5

#locations returns multiple location information depending on the node. For example, the node of the code alias foo bar seems to return two location information: the overall location information and the location information of the “alias” keyword. Since the Prism node already has this kind of location information, it is a preparation for creating a conversion from RubyVM::AbstractSyntaxTree::Node to the Prism node.

(mame)

`String#append_as_bytes` was introduced

String#append_as_bytes was added to more easily and efficiently work with binary buffers and protocols. It directly concatenate the arguments into the string without any encoding validation or conversion. [Feature #20594]

The String#append_as_bytes method, which destructively concatenates strings, was introduced.

str = "".b
str.append_as_bytes(255, "ä")

# This will result in a byte string equivalent to "\xff" + "ä".
p str #=> "\xFF\xC3\xA4"

What is the difference between this and str << 255 << "ä"? In the following case, an exception is raised.

str = "".b
str << 255 << "ä"
  #=> incompatible character encodings: BINARY (ASCII-8BIT) and UTF-8 (Encoding::CompatibilityError)

You might think that str is a binary encoding, so it should be silently concatenated. However, that means implicitly converting the argument string "ä" into a broken string, which may not be a good idea. So it was decided to introduce a more explicit method with the name as_bytes.

Incidentally, String#append_as_bytes forcibly concatenates the receiver as a byte string, regardless of the encoding, not just binary encoding. I'm not sure about the use case for that, though.

(mame)

`Symbol#to_s` now emits a deprecation warning when mutated

The string returned by Symbol#to_s now emits a deprecation warning when mutated, and will be frozen in a future version of Ruby. These warnings can be enabled with -W:deprecated or by setting Warning[:deprecated] = true. [Feature #20350]

$ ruby -we ':sym.to_s << ?a'
-e:1: warning: warning: string returned by :sym.to_s will be frozen in the future

At the beginning of this article, we introduced the mechanism for making frozen-string-literal the default (a string that will be frozen in the future if a destructive change is made to it), and if you try to make a destructive change to the string returned by Symbol#to_s using this, you will now get a warning if the deprecated warning is enabled (for example, if you start Ruby with -w or -W:deprecated).

So I guess they want to freeze this return value in the future too. Symbol#name went in for that, but should we have to do #to_s too?

(ko1)

On Windows, the string encoding of `Time#zone` has changed slightly

On Windows, now Time#zone encodes the system timezone name in UTF-8 instead of the active code page, if it contains non-ASCII characters. [Bug #20929]

I don't really understand it, but it seems to have changed. The Windows native environment is difficult to me.

(mame)

`Time#xmlschema` has been incorporated into the main body

Time#xmlschema, and its Time#iso8601 alias have been moved into the core Time class while previously it was an extension provided by the time gem. [Feature #20707]

The Time#xmlschema and Time#iso8601 provided by the time gem have been incorporated into the main Ruby interpreter.

Time.new(2024, 12, 25).xmlschema #=> “2024-12-25T00:00:00+09:00”

The reason for this is that the C implementation is faster. That's right.

(mame)

`Warning.categories` has been added

Add Warning.categories method which returns a list of possible warning categories. [Feature #20293]

Recently introduced warnings have categories, and it is possible to turn the output on and off for each category, but since the number of categories is gradually increasing, a method to obtain a list of them has been introduced.

# Turn on warnings about deprecation
Warning[:deprecated] = true

# Returns a list of categories that can be specified.
Warning.categories
#=> [:deprecated, :experimental, :performance, :strict_unused_block]

The main use is for testing Ruby itself. When running certain tests, we want to suppress warnings, so we save the status of all categories, suppress them, and then restore them later. I don't think there are any other uses.

(mame)

stdlib updates

There are various updates, but only the ones that are commented on in the NEWS article.

RubyGems now supports sigstore.dev

Add --attestation option to gem push. It enabled to store signature of build artifact to sigstore.dev.

RubyGems now supports sigstore.dev, which aims to improve the security of the software supply chain.
Sigstore is a series of mechanisms that provide automated signing for the software supply chain. If you pass the file path to a Sigstore Bundle generated using cosign or sigstore-ruby to --attestation, you can upload the Gem signature to RubyGems.

(atpons)

Support for checksums in Bundler

Add a lockfile_checksums configuration to include checksums in fresh lockfiles.
Add bundle lock --add-checksums to add checksums to an existing lockfile.

It is now possible to add checksums to Gems using RubyGems. By adding checksums to Gems when the lockfile is generated, and verifying them when bundle install is run using that lockfile, the consistency of libraries in each execution environment can be guaranteed, and tampering by the library provider or man-in-the-middle attacks can be detected.

(atpons)

JSON performance improvements

Performance improvements JSON.parse about 1.5 times faster than json-2.7.x.

It seems that JSON processing has become much faster.
For more details, see here.

(ko1)

Now possible to create a Tempfile with no actual file

The keyword argument anonymous: true is implemented for Tempfile.create. Tempfile.create(anonymous: true) removes the created temporary file immediately. So applications don't need to remove the file. [Feature #20497]

Tempfile.create(anonymous: true) now creates a temporary file without creating a real file.
To be precise, it seems that on Linux, where O_TMPFILE is available, a temporary file is not created, but on other platforms, including Windows, a temporary file is created for a moment and then immediately deleted after the IO handle is obtained.

I'm not sure what the user benefits are, but perhaps it avoids problems such as the temporary file not being deleted if the Ruby interpreter terminates abnormally due to a segfault, or the filename generated by the random number generator colliding with an existing file (a problem that seems unlikely enough not to be a concern).

(mame)

win32/sspi.rb has been removed from the Ruby repository

This library is now extracted from the Ruby repository to [ruby/net-http-sspi]. [Feature #20775]

I don't know what this is, but it seems to have been removed. I think it wasn't working in the first place?

(ko1)

The default gems have been updated in various ways

win32-registry 0.1.0 has been added, and many other libraries have been updated.

The bundled gems have been updated in various ways

They have been updated in various ways.

repl_type_completor has been added

The following bundled gem has been added.

repl_type_completor 0.1.9

The name of katakata_irb, which was announced at RubyKaigi 2023, has changed and it is now a standard attachment (bundled gem). This is a library that supports completion in IRB, and it uses Prism and RBS to provide smarter completion candidates than before. By default, IRB analyzes completion targets using regular expressions , so there were problems such as it sometimes suggesting methods that could not be called, but repl_type_completor now makes it possible to suggest more accurate candidates. Even if you haven't written RBS, it will automatically become smarter because RBS is included as standard for built-in classes and the analysis function itself has become more powerful.
In IRB, repl_type_completor is used preferentially if it is present in the execution environment. In the Bundler environment, bundle add repl_type_completor is required.

(ima1zumi)

Transfer from default gems to bundled gems

The following bundled gems are promoted from default gems.

mutex_m 0.3.0
getoptlong 0.2.1
base64 0.2.0
bigdecimal 3.1.8
observer 0.1.2
abbrev 0.1.2
resolv-replace 0.1.1
rinda 0.2.0
drb 2.2.1
nkf 0.2.0
syslog 0.2.0
csv 3.3.2

These libraries have been changed from default gems to bundled gems.
If you are developing an app using Gemfile, please add these libraries to your Gemfile.

(ko1)

Compatibility issues

Error messages and backtrace have been changed

Error messages and backtrace displays have been changed.
- Use a single quote instead of a backtick as an opening quote. [Feature #16495]
- Display a class name before a method name (only when the class has a permanent name). [Feature #19117]
- Extra rescue/ensure frames are no longer available on the backtrace. [Feature #20275]
- Kernel#caller, Thread::Backtrace::Location’s methods, etc. are also changed accordingly.

It's not very flashy, but there have been a lot of changes. Spot the differences.

Old example:

test.rb:1:in `foo': undefined method `time' for an instance of Integer
from test.rb:2:in `<main>'

New example:

test.rb:1:in 'Object#foo': undefined method 'time' for an instance of Integer
from test.rb:2:in '<main>'

For one thing, the method name display has changed from foo to Object#foo. It's now easier to see which method it is (on the other hand, it's also possible that it's become more verbose, so if you find yourself complaining a lot, it might be a good idea to speak up now).
The other thing is that backticks have been changed to single quotes. This improves the experience of copying and pasting error messages and having them turn out weird in markdown.

I was the one who made this change, but even this small change had an unexpected impact on compatibility, and it was a lot of work. There were some naughty libraries that parsed error messages and backtraces. For example, irb and test-unit. I don't think it will affect the majority of libraries, but if there is any problem, please feel free to report it so we can investigate.

(mame)

Hash#inspect results have changed

Hash#inspect rendering have been changed. [Bug #20433]
- Symbol keys are displayed using the modern symbol key syntax: "{user: 1}"
- Other keys now have spaces around =>: '{"user" => 1}', while previously they didn't: '{"user"=>1}'

The appearance of a p (inspect) of a Hash has changed.

h = { user: 1 }
p h #=> {:user=>1} until Ruby 3.3
    #=> {user: 1} from Ruby 3.4

I think that very few people write hashes like { :user => 1 } these days, so this change is to match that.

This is probably the biggest incompatibility in Ruby 3.4, apart from the warning for the default frozen_string_literal. If you have the following code in your tests, it will fail.

str = "result: #{ h }"

expect(str).to eq("result: {:user=>1}")

If you only want to use it with Ruby 3.4 or later, you can just change the expected value, but if you want to use it with both Ruby 3.3 and 3.4, for example in testing a library, you might want to change it like this

expect(str).to eq("result: #{ { user: 1 } }")

Well, since the inspect string may change in this way, it is not recommended to use it as the expected value in a test.

Incidentally, for a Hash that contains a mixture of Symbol and non-Symbol keys, only the Symbol keys are displayed with a colon.

h = { :sym => 1, "str" => 2 }

p h #=> {sym: 1, "str" => 2}

As an aside, the circumstances leading up to this change are a little interesting. It all started with a bug report saying that {:<=>1} was being output as {:<=>1} when inspect was used, and that it didn't revert back to the original value when eval was used. There is no guarantee that the result of inspect will revert back to the original value when eval is used, but {:<=>1} is not very clear to the eye, so we wanted to fix it. So, it was considered that spaces should be inserted, like {:< => 1}. Then, {:key=>1} should also be changed to {:key => 1}? I think that {:< => 1} will not affect most people, but changing to {:key => 1} is a certain degree of incompatibility. There was also the option of giving the change up, but there had been talk of “making Hash#inspect into the {key: 1} format someday” that had been smoldering and then fading away, and the momentum grew that “if it's going to be incompatible anyway, let's do it all at once here”, and that's how this change came about.

(mame)

Converting a decimal number to a string now interprets a string without a decimal part

Kernel#Float() now accepts a decimal string with decimal part omitted. [Feature #20705]
String#to_f now accepts a decimal string with decimal part omitted. [Feature #20705]
Note that the result changes when an exponent is specified.

In short, it now interprets half-decimal strings like "1.".

Float("1.") #=> 1

Previously, Kernel#Float would throw an exception for such strings. String#to_f evaluated the string within the range of interpretation (ignoring the part after the period in the example above), so the behavior is the same.

The fractional part between the period and the exponent notation can also be omitted.

Float("1.E-1") #=> 0.1

This was also an exception for Kernel#Float in the past. String#to_f ignored everything after the period, so it returned 1, but in Ruby 3.4 and later, it returns 0.1, so it is slightly incompatible. I hope no one will use it.

Incidentally, there are quite a few languages that interpret 1. as a decimal number, including C and Python. Of course, in the Ruby language, the period is a method call operator, so 1. cannot be interpreted as a decimal number.

(mame)

`Refinement#refined_class` has been removed

Removed deprecated method Refinement#refined_class. [Feature #19714]

It has been removed. Please use Refinement#target from now on.

(mame)

Stdlib incompatibilities

There also appear to be some changes in the standard library.

DidYouMean
- DidYouMean::SPELL_CHECKERS[]= and DidYouMean::SPELL_CHECKERS.merge! are removed.
Net::HTTP
- Removed the following deprecated constants: Net::HTTP::ProxyMod Net::NetPrivate::HTTPRequest Net::HTTPInformationCode Net::HTTPSuccessCode Net::HTTPRedirectionCode Net::HTTPRetriableCode Net::HTTPClientErrorCode Net::HTTPFatalErrorCode Net::HTTPServerErrorCode Net::HTTPResponseReceiver Net::HTTPResponceReceiver
These constants were deprecated from 2012.
Timeout
- Reject negative values for Timeout.timeout. [Bug #20795]
URI
- Switched default parser to RFC 3986 compliant from RFC 2396 compliant. [Bug #19266]

C API updates

rb_newobj and rb_newobj_of (and corresponding macros RB_NEWOBJ, RB_NEWOBJ_OF, NEWOBJ, NEWOBJ_OF) have been removed. [Feature #20265]

These C APIs have been removed.

Removed deprecated function rb_gc_force_recycle. [Feature #18290]

This C API has been removed. It was previously a no-op, so if you're using it, just remove it.

To begin with, what this did was to manually free objects that were no longer needed before the GC ran. However, it was very difficult to use, and if it was used on a live object by mistake, it would cause strange things to happen, and it would require special processing for the GC, so recently it was more of a nuisance than anything else. So, it's a good thing it's been removed.

(ko1)

Implementation improvements

Prism is now the default parser

The default parser is now Prism. To use the conventional parser, use the command-line argument --parser=parse.y. [Feature #20564]

Prism is now the default parser for Ruby.

For the past few years, the Ruby parser has been a very hot topic at RubyKaigi and other events, but even so, it's just an internal improvement, so I don't think there's anything for ordinary users to be aware of. If anything, the error messages for syntax errors have changed.

$ ruby -e 'foo('
-e: -e:1: syntax error found (SyntaxError)
> 1 | foo(
    |     ^ unexpected end-of-input; expected a `)` to close the arguments

Is it convenient to have snippets? It used to be like this

$ ruby --parser=parse.y -e 'foo('
-e:1: syntax error, unexpected end-of-input, expecting ')'
ruby: compile error (SyntaxError)

Also, there are still some incompatibilities in Prism in corner cases (and I'm sure there will be more found in the future), so there may be some changes here and there.

(mame)

Happy Eyeballs v2 has been implemented

Happy Eyeballs version 2 (RFC8305), an algorithm that ensures faster and more reliable connections by attempting IPv6 and IPv4 concurrently, is used in Socket.tcp and TCPSocket.new. To disable it globally, set the environment variable RUBY_TCP_NO_FAST_FALLBACK=1 or call Socket.tcp_fast_fallback=false. Or to disable it on a per-method basis, use the keyword argument fast_fallback: false. [Feature #20108] [Feature #20782]

TCPSocket.new and Socket.tcp now connect by default using Happy Eyeballs v2. Happy Eyeballs v2 is a mechanism that tries to connect to a specified host using both IPv4 and IPv6 simultaneously, and establishes a connection with the first host that responds.

Previously, it tried to connect to IPv6 and IPv4 sequentially. With this method, if you tried to connect to IPv6 first in an environment where there were problems with IPv6 communication for some reason, it would take a long time because it wouldn't switch to trying to connect to IPv4 until the IPv6 connection timed out. Ruby 3.4 connects to IPv6 and IPv4 simultaneously, so even if IPv6 doesn't work, IPv4 will respond immediately and the connection should be established without waiting for a timeout.

Well, actually, I personally feel that it is more of a transitional measure needed for society to gradually shift from IPv4 to IPv6, than a direct user benefit. Happy Eyeballs is a social responsibility. Ruby is doing a great job of fulfilling its social responsibilities.

Happy Eyeballs has a slight overhead because it connects in parallel. If you really want to stop Happy Eyeballs, set Socket.tcp_fast_fallback = false or set the environment variable RUBY_TCP_NO_FAST_FALLBACK=1. In addition to the overhead, it may also be useful for isolating problems when connection problems occur.

(mame)

A mechanism for switching GC implementations has been introduced

Alternative garbage collector (GC) implementations can be loaded dynamically through the modular garbage collector feature. To enable this feature, configure Ruby with --with-modular-gc at build time. GC libraries can be loaded at runtime using the environment variable RUBY_GC_LIBRARY. [Feature #20351]

It is now possible to build Ruby so that the GC implementation can be switched at Ruby process startup by specifying --with-modular-gc in configure when building Ruby. This can be changed using the RUBY_GC_LIBRARY environment variable.

Since this option is not specified by default when building, it can be said that this was introduced for the purpose of experimenting with replacing the GC implementation.

Ruby's built-in garbage collector has been split into a separate file at gc/default/default.c and interacts with Ruby using an API defined in gc/gc_impl.h. The built-in garbage collector can now also be built as a library using make modular-gc MODULAR_GC=default and enabled using the environment variable RUBY_GC_LIBRARY=default. [Feature #20470]

The directory structure related to GC has been changed so that the GC implementation can be changed in various ways.

An experimental GC library is provided based on MMTk. This GC library can be built using make modular-gc MODULAR_GC=mmtk and enabled using the environment variable RUBY_GC_LIBRARY=mmtk. This requires the Rust toolchain on the build machine. [Feature #20860]

Using the feature to change the GC implementation, an experimental GC implementation called MMTk, which is a collection of many GC implementations, was provided. MMTk contains GC implementations that are the culmination of research, but due to limitations in CRuby, only a very simple one is currently supported, and it is said that it will be developed in the future. It's written in Rust, isn't it?

(ko1)

YJIT

New features

Add unified memory limit via --yjit-mem-size command-line option (default 128MiB) which tracks total YJIT memory usage and is more intuitive than the old --yjit-exec-mem-size.
More statistics now always available via RubyVM::YJIT.runtime_stats
Add compilation log to track what gets compiled via --yjit-log
- Tail of the log also available at run-time via RubyVM::YJIT.log
Add support for shareable consts in multi-ractor mode
Can now trace counted exits with --yjit-trace-exits=COUNTER

New optimizations

Compressed context reduces memory needed to store YJIT metadata
Improved allocator with ability to allocate registers for local variables
When YJIT is enabled, use more Core primitives written in Ruby:
- Array#each, Array#select, Array#map rewritten in Ruby for better performance [Feature #20182].
Ability to inline small/trivial methods such as:
- Empty methods
- Methods returning a constant
- Methods returning self
- Methods directly returning an argument
Specialized codegen for many more runtime methods
Optimize String#getbyte, String#setbyte and other string methods
Optimize bitwise operations to speed up low-level bit/byte manipulation
Various other incremental optimizations

It seems that YJIT has also improved in various ways. I'm sure there will be a detailed explanation, so I'll skip it here.

(ko1)

Miscellaneous changes

Added a warning when passing a block to a method that doesn't use the passed block

Passing a block to a method which doesn't use the passed block will show a warning on verbose mode (-w). [Feature #15554]

When passing a block to a method that doesn't use the passed block, a warning will now be issued if -w is specified.

def f = nil
f{}
#=> warning: the block passed to 'Object#f' defined at t.rb:1 may be ignored

However, if you run this as it is, you will get a warning even if you are passing a block intentionally and unnecessarily, as in the following duck typing, and we found that this is quite common.

class C
  def f = yield # This uses a block
end

class D
  def f = nil # This one doesn't use a block
end

[C.new, D.new].each{ it.f{} }
#=> No warning is given because there is a method named “f” that uses a block

So, in this way, if there is a method with the same name that uses a block, no warning is given (even with -w).

If you don't like this loosening of the conditions, try running with -W:strict_unused_block.

$ ./miniruby -W:strict_unused_block -e '
class C
  def f = yield # This uses a block
end

class D
  def f = nil # This does not use a block
end

[C.new, D.new].each{ it.f{} }
'
-e:10: warning: the block passed to 'D#f' defined at -e:7 may be ignored

You will now get a warning.

In fact, when I tried it out with our app, I found one example in the rspec tests where a block was being passed where it shouldn't have been. I only found it because I was using the strict version, so you might want to be prepared for a lot of false positives.

(ko1)

Warnings are now given for redefining methods that we wouldn't expect to be redefined

Redefining some core methods that are specially optimized by the interpreter and JIT like String#freeze or Integer#+ now emits a performance class warning (-W:performance or Warning[:performance] = true). [Feature #20429]

If methods that are not supposed to be redefined, such as String#freeze and Integer#+, are redefined, it has a negative impact on the interpreter's performance (because the interpreter is optimized on the assumption that they will not be redefined). Therefore, if such a method is redefined, a warning will now be issued if -W:performance is specified.

(ko1)

Conclusion

We have introduced the new features and improvements of Ruby 3.4. In addition to the ones introduced here, there are also bug fixes and minor improvements. We hope you will check them out in your Ruby applications.

Although Ruby 3.4 doesn't have any particularly eye-catching new features, there have been a lot of implementation improvements, such as the replacement of the parser with Prism and the introduction of a new grammar called it. There have also been some decisions that will have a major impact on the future development of Ruby, such as the defaultization of frozen-string-literal. Please set it up on your computer and enjoy the new Ruby.

Enjoy Ruby programming!

(ko1/mame, guest: ima1zumi, atpons)

Am I the only one who doesn't put parentheses around the parameters in Ruby method definitions?

Koichi Sasada — Fri, 12 Apr 2024 02:04:04 +0000

I'm Koichi, a Ruby interpreter developer in the Technology Division at STORES, Inc. And I'm looking forward to RubyKaigi 2024!

Now, when you define a method in Ruby, you can omit parentheses even if there are method parameters.

def foo(x, y)
end

def bar x, y
end

That's how you define bar. I used to prefer to write it this way, but I was shocked when my colleague Mr. Endoh told me that only Koichi (I) uses that way nowadays, so I did some research.

By the way, there is a method definition method that cannot be used without parentheses, so I will tearfully add parentheses at that time.

def foo(kw:) # required keyword argument
end

def bar(&) # unnamed block argument
end

Supplement 1: There is a lot of controversy in Ruby about whether or not to put parentheses when calling a method, but I will only mention the parentheses when defining a method. I haven't heard much about it.

Supplement 2: I think most people don't put parentheses when there are no parameters at definition, so I will describe "only when there are parameters".

Conventions

Rubocop has a default rule that says to put parentheses when there are parameters; even Standardrb has a default ([https://github.com/standardrb/standard/blob/8307fa8f449f896075ccad 74bf6a128ed2c26189/config/base.yml#L1098:title])

https://docs.rubocop.org/rubocop/cops_style.html#stylemethoddefparentheses

The rule at Cookpad, which I feel is often referenced when coding rules, says that brackets MUST be added.

https://github.com/cookpad/styleguide/blob/master/ruby.en.md#method-definitions

[MUST] On method definition, do not omit parentheses of parameter list, except for methods without parameters.

A coding rule that was often referred to in the past also says that the parentheses MUST be added.

https://shugo.net/ruby-codeconv/codeconv.html

Parentheses should be added to the list of temporary arguments in a method definition. However, if there are no arguments, the parentheses should be omitted. (Ja -> En translation)

I'm going to have a hard time getting a job writing programs in Ruby...

Actual survey

I checked how many ways of writing with/without parentheses on Ruby's method definition with parameters in actual Ruby code.

The survey covers the latest versions of all gems pushed to rubygems.org. I am grateful to Endoh-san et, al. using [https://rubygems.org/gems/rubygems-mirror] to manage rubygems mirrors for Ruby development.

It seems that about 95% of the definitions has parentheses (w/paren) and 5% does not (wo/paren).

I thought "don't older gems often have no parentheses?" (Possibility that the rule of putting parentheses has been more thoroughly enforced now than in the past.) So, I tabulated the results by year.

Surprisingly, there was no "more or less because they are older" trend. The percentage is high around the middle.

A bit of caution is that the year counted here is the date of the latest release, not the date of first writing (if the first release was a long time ago and are still releasing in 2024, it is counted in 2024).

Here are the actual values.

In addition, I also checked what happens when there are no parameters at method definitions.

I thought no one put them on, but there are about 3%. Note that there was an anomaly in 2023, so I excluded that (I didn't check what the anomaly was).

Looking at the Rails code, I found that it uses parentheses without parameters when it wants to be written on a single line, like def foo() end. That kind of thing seems to go away with a one-line method definition (def foo = nil). No, I'm not so sure. (Write def foo() = nil?)

Investigation code

I used Prism to investigate.

require 'prism'

class ParenFinder
  TYPES = {
    true => {
      true => :paren_w_params, :paren_w_params, :paren_w_params
      false => :paren_wo_params, }
    }, }
    false => {
      true => :noparen_w_params, :noparen_w_params, }
      false => :noparen_wo_params, }
    }
  }

  def self.reset
    $defs = TYPES.map{_2.values}.flatten.map{[_1, 0]}.to_h
    $defs[:files] = 0
    $defs[:time] = Time.now.to_i
  end

  reset

  def initialize file
    @file = file
  end

  def defs_rec node
    if node.type == :def_node
      paren = node.lparen_loc ? true : false
      params = node.parameters ? true : false
      type = TYPES[paren][params]
      $defs[type] += 1 # [@file, node.slice.lines.first.chomp]
    else
      node.child_nodes.compact.each{|n|
        defs_rec n
      }
    end
  end

  def defs
    ast = Prism.parse_file(@file)
    defs_rec ast.value
  rescue Exception => e
    # ignore any errors
  end
If ARGV.empty?

if ARGV.empty?
  ParenFinder.new(__FILE__).defs
  name = __FILE__
else
  process_file = -> f do
    case
    when FileTest.symlink?
       # ignore (depends on data)
    when FileTest.directory?(f)
      Dir.glob(File.join(f, '*')){|f| process_file.call f}
    when /. *\.rb$/ =~ f
      $defs[:files] += 1
      ParenFinder.new(f).defs
    else
      # ignore other files
    end
  rescue Exception => e
    # ignore
  end

  name = ''.dup

  ARGV.each{|f|
    name << f
    process_file.call f
  }
  $defs[:time] = Time.now.to_i - $defs[:time]
end

pp({ name => $defs.map{|k, v|
  if Array === v
    [k, v.size].
  else
    [k, v].
  end
})

I am glad that I was able to create an easy survey script, although I was able to ask Mr. Endoh, who tinkers with Prism, how to use it. I mean, the lparen_loc method (the position of the parentheses that wrap the parameters) is amazing.

Symlink handling was going to have to be tweaked for each survey target.

Why method definitions without parentheses are allowed

I had a chat with Yukihiro Matz Matsumoto and asked him, "Why were you allowed to define methods without parentheses?" He said it was for the following reasons.

In Ruby, parentheses can be omitted in "command-like method" calls. For example, attr :ivar.
Matz introduced it because I thought it would be better to be able to define such command-like methods without parentheses as well.

In the discussion, there were opinions such as "The writer cannot judge whether the method is a command-like method or not" and "There seem to be not so many opportunities to write command-like methods". I found out that the function of omitting parentheses when defining a method is likely to be forgotten, except when there are no parameters.

Summary

I started this survey with the question, "Am I the only one who doesn't put parentheses around the parameters in Ruby method definitions?" Is it an interesting result that 5% of them violate well-known coding conventions?

Personal efforts to improve the quality of Ruby interpreter

Koichi Sasada — Tue, 27 Dec 2022 15:57:12 +0000

This article is Japanese -> English translation (thank you DeepL!) of the following post (and some additional messages):

Rubyインタプリタの品質向上のために個人的にやっていること - クックパッド開発者ブログ

技術部の笹田です。Ruby 3.2 無事にリリースされて良かったよかった。 Rubyインタプリタは複雑なプログラムなので、当然のごとくバグが入ってきます。Rubyインタプリタ開発者は、これに対していろんな対策をしています。たとえば、テストを書いて、CI環境でチェックするとか、今となっては当然のことを、当然のごとくやっています（RubyCIやchkbuild、ruby/spec: The Ruby Spec Suite aka ruby/specなどの整備や、実行環境の日々のメンテナンスの成果です）。これに追加して、個人的にテストをとにかくたくさん繰り返し行うマシン群を用意しています。テストの…

techlife.cookpad.com

I'm a programmer and one of the full-time commiter of the Ruby interpreter (Homepage of Koichi Sasada) at Cookpad Inc. We hope you enjoy the recently released Ruby 3.2.

Ruby interpreter is a complex program, so it naturally has bugs, and Ruby interpreter developers are taking various countermeasures against them. For example, we write tests and check them in CI environment (This is the result of daily maintenance of the test environment, such as RubyCI, chkbuild, ruby/spec: The Ruby Spec Suite aka ruby/spec and machines).

In addition to this, I have a group of machines that I personally run a lot of test iterations on. The purpose is to improve the quality of Ruby interpreter by running tests as often as possible to find bugs that occur only occasionally. In this article, I'd like to introduce about such an uncommon test environment.

To prepare this test environment, I have received support from various people. In this article I would like to express my gratitude for their support.

The Need to "Reveal" Bugs

Environment to run tests on a regular basis

In a typical CI/CD context, you run tests on each commit (PR), because if there is a problem, you know there is a problem in the commit. GitHub Actions and the like often target such tests.

To deal with it, Ruby interpreter core team prepares and uses the following test environments.

Per-PR and per-push testing environment with GitHub Actions
Periodical test environment with chkbuild (The results are available on rubyci).

Both 1 and 2 are basically mechanisms to check for problems with the last modification.

To get accurate test results, chkbuild makes an Ruby interpreter from scrach and run tests every time under various operating systems and CPU architectures. However, since it takes time, it is executed about once every two hours on a machine.

Currently, most of the compute resources are built on AWS with support from Ruby Association and other organizations. GitHub actions are provided by GitHub.

Speaking of which, Shopify has been kind enough to test their (presumably huge) application with a development version of Ruby. That's very helpful.

// Detect dark theme var iframe = document.getElementById('tweet-1605706285994741761-277'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1605706285994741761&theme=dark" }

Digression: Try `head`/`debug` versions of setup-ruby

If you have projects which have tests running on GitHub actions with setup-ruby, please consider to add head/debug versions.

head: nightly build
debug: nightly build with assertions

If you find out the strange behavior, please feel free to report it to the https://bugs.ruby-lang.org/issues/. This kind of contributions are very helpful for us.

Test environment to discover tests that sometimes fail

When you have a software as big as Ruby interpreter, you may encounter a phenomenon that it sometimes fails even though nothing has changed. It is also possible that there may be last minute changes, but the test fails for reasons not contemplated by the modification. This is sometimes called a flaky test. There are several possible reasons for this.

Bad test.
Tests caused by external factors such as "time" and "system status
Bugs are already mixed in, but only if you're unlucky (or lucky) enough to find them.

The most common experience is that "1. bad test". For example, if it depends on the order of the tests, it can cause problems at any moment. If you're writing tests that are timing-sensitive, they can be a bit off and sometimes fail (or fail because of a change in machine spec).

Tests caused by external factors in #2 are also sometimes bad, though it's not impossible to say that the tests are bad. For example, the example shown in ruby/zlib tests started to fail even though I didn't do anything - @znz blog (written in Japanese), where the timestamps generated specific data at a specific time, and the tests failed (fixed the tests and solved the problem).

Well, the above are in the category of "bad testing", so they are not directly related to the quality of the interpreter itself. However, if we leave these things unmodified, you'll have a hard time checking the test results, so you need to fix them as soon as possible. It's the Broken Windows Theory.

Even a bug that appears once in 10,000 times by bad luck may be stepped on once a day in a software with 10,000 users a day. Or rather, you step on it. If it is worse, it may become a source of vulnerability.

This kind of bug is likely to appear in the following situations:

Automatic memory management (GC)
An algorithm using cache
Parallel and concurrent executions
Networks and other external systems

All of these are areas (and areas I work with a lot) where it's easy to bring in non-deterministic behavior, i.e., behavior that doesn't produce the same result even if you run it twice. There are many other things that can cause a "Hey, that's not the same result as before?" situation, such as memory address randomization by the system.

So, there are many ways to tackle on this, we use the method of "just try many times". It's simple: if it appears once in every 10,000 times, then if you run it 10,000 times, it will be reproducible.

In other words: "Bugs that don't appear very often are already in the code → the more test trials we run, the higher the probability of stepping on such a bug (we can reveal it out)."

The chkbuild I mentioned earlier runs about 12 times a day (and multiplied by the number of environments), and the GitHub actions runs every event, so it's not enough to say "run a lot". So I made my own test environment and have been running it for about 5 years.

It originally started when I got fed up with "occasional" bugs in debugging on the newly designed GC development, and ran tests all the time on a single machine.

When I first started, I used the command while make up all test-all; do date; done to run the test indefinitely (stopping if it failed). However, with this, I have to look at the terminal to check the results, and I could not know if the test stops unintentionally. Also, it's hard to scale, so I had to build my own test environment.

Techniques for running lots of tests

In order to run a lot of tests, we devised the following

Use multiple machines (scale-out)
Use machines with better performance (scale-up)
Run multiple tests simultaneously on a single machine to use up hardware resources
Shorten the time of a single build and test run

Here is an introduction to each of them.

Preparing the machine to be used

If we have enough money, the most reliable (and easy for the savvy) way to scale out is to prepare lots of machines on the cloud, but since this is a private activity, there is a limit to how much money I can provide. Also, some of these types of uses that use up computing resources are not suited to cheap cloud services.

Since I have some space at home, I'm currently hosting the actual machines in a temporary location (this activity will likely end there as the children grow older and they needs their own rooms. Please remember that Japanese home doesn't have enough space to maintain multiple computers).

I've been staring at the price list of AWS and other services, but using real machines is the cheapest... (I wonder if it will be cheaper if I find a discount plan). I'm grateful that I can buy a nice little machine with 8 cores 16 threads for just under 100,000 JPY (about 750 USD). Currently, I'm running 4 machines.

1 machine: bought 6 years ago
1 machine: ThinkCentre M75q Gen 2 (donated by Garnet Tech 373 Inc.: Garnet Tech 373 Inc. provides development machines to support Ruby interpreter development - Garnet Tech 373 Inc. written in Japanese) about 60,000 JPY
2 machine: MINISFORUM EliteMini HX90 (purchased with proceeds from GitHub sponsors (Sponsor @ko1 on GitHub Sponsors)) 2 x about 80,000 JPY

All the new machines are small. I used to have a mid-tower machine in my lineup, but it was getting in the way... I bought the HX90 at the last Black Friday because it was a bit low price.

Test run times correlated nicely with CPU frequency. The faster the better.

If you run one test suite, 2GB of memory seems to be enough even if we build and run each test in parallel (it was my surprise).

I have an electricity meter on, and when I look at it, it seems to go up and down around 400Wh in total. When I look at the Tokyo Electric Power Company's electricity rates Standard Plan. That's just under 10,000 JPY (per month). I'm partially compensated by the earnings from GitHub sponsors.

(By the way, this electricity bill includes three Mac minis used for rubyci/chkbuild, which I introduced earlier; the Mac minis were purchased with the support of Nihhon Ruby-no-Kai).

It's fine now because it is Winter, but during the hot months (since I didn't turn on the A/C) the fan was making a lot of noise. Worried about a fire. So far, it's been running for more than a year, even in continuous operation. But after 5 years, 2 of the middle tower machines broke. The smaller machines seem to have a shorter life span.

The machine cost (leaving aside the old one) is 220,000 JPY, and if it depreciates in 3 years, it is about 70,000 JPY/year. Electricity cost is roughly 120,000 JPY/year. In other words, about 200,000 JPY/year (about 1,500 USD/year). It is cheap because it doesn't need the cost of the place and the maintenance personnel cost. And I don't need SLA because I don't have to worry about the system if the system is down. Well, it is a good hobby to arrange machines at home.

As a side note, one of the reasons I'm running physical machines on hand is to do benchmarking. If you use a machine in the cloud, you may have to mess up your instance, so I prefer to use a physical machine as much as possible. For example, the machine on https://rubybench.github.io/ is the machine hosted at my home (this machine is also provided by Ruby-no-kai Japan, thank you very much). When I need to do serious benchmarking of new features, I stop running tests and and use these machines for benchmarking (because sometimes I need more than one machine for benchmarking).

Parallel execution of build and test processes

One way to increase the number of test trials is to launch multiple processes running the test suite on a single machine.

When a test suite is executed, there are times when it consumes resources and times when it is free, so the idea is to improve the overall performance by running another test execution process when one test execution process is not busy. However, if the number of concurrent test processes is too large, the overall performance may deteriorate because resources are consumed in resource conflicts.

Simply running multiple test processes sometimes interfered with each other (e.g. filesystem or network ports), so through some trial and error I figured out that it was ok to tweak some settings in the Docker container. I now have 22 Docker containers running the test suite simultaneously on a single machine (build-ruby/run_sp2.rb at master ・ ko1/build-ruby). The memory is 32GB, which is enough (but I gave up the RAM disk, which will be described later).

Reduce build and test time

In order to reduce the time it takes to build the latest version of Ruby and run through the entire test suite, we've devised the following:

Reuse compile results etc.
Using a RAM disk
Concurrent build and test processing

The test runs on rubyci.org do not reuse any compile results and so on to ensure the test results. However, in this case, the goal is to get more number of tests, so I try to reuse the compiled results aggressively. However, there are sometimes problems caused by reuse, so if it fails twice, it erases all the compiled results and build from scratch.

In an environment with relatively spare memory, I try to use RAM disks (tmpfs) for all build results to speed up the build a bit. However, it's not clear how well this works. The reason is that the OS caches performance-related data in memory on its own. It's just a matter of feeling that it's faster.

The parallel execution of the build is by make -jN. 10 years ago, there were quite a few bugs caused by this, but now we can build in parallel with almost no problem.

Running tests in parallel, which means splitting up the test suite and running the them in parallel. There are roughly three groups of Ruby tests to run in this environment, one of which has long supported parallel processing (make test-all). I rewrote one more group (make btest) to make it parallelizable, with the goal of making it count.

With these efforts, in an environment where you occupy a fast machine and repeatedly run "build the latest version -> run tests" can be done in less than 2 minutes. In other words, because we always get the latest version of Ruby from the repository for testing, if you make a problematic commit that causes the test to fail, you'll get a test failure notification in as little as two minutes (the result is notified to the Slack channel).

Technique to reveal bugs

More tests

Even if a bug is introduced, it cannot be detected if there no code that will step on that bug. This is why extensive testing is needed. Ruby already has a large set of tests.

Also, the Ruby interpreter source code contains a lot of assertions (statements of what should happen at this point in the program). You can think of this as a kind of test. In the parts I code, I try to increase the number of such assertions so that we can detect strange states.

Many of these assertions are only enabled for checking in debug builds. For this reason, we use debug builds to run them in some of the environments we run in.

For testing, ideally, it would be nice to bring a prominent app or library and run its tests on the latest development version of Ruby, but I haven't gotten around to that.

More test patterns

The tests we run are not all the same, but we try to find bugs by running the tests in different patterns.

Run the test with a Ruby interpreter built with various parameters.
Run tests with different versions of the build environment (compiler).
Run the tests in a random order. For example, the method cache status changes depending on the order in which the tests are executed, so there may be bugs to be found there.
Repeatedly run the test. Similarly, repeating the same test may change the status of the method cache.

I wrote a software to build Ruby and run tests according to the configuration (ko1/build-ruby: Build Ruby from source code). For example, here is a list of settings: https://github.com/ko1/build-ruby/blob/master/docker/ruby/targets.yaml

Dealing with Errors

We have devised several ways to deal with unforeseen problems.

Recording of all execution logs
Allow configurable timeouts to prevent infinite stoppages
- When a timeout occurs, gdb dumps the backtrace of the related process
In case of an abnormal exit that spits out core, you can download the core
If the failures continue, delete all the data, increase the execution interval, and so on.

However, when a test fails, we often don't know the cause after all. We would like to devise a little more.

Developing a system to check the results

I made a site ci.rvm.jp to aggregate the test results. The DB is SQLite3 because the number of viewers is limited (so it is slow). It's a really slow server, so I don't even link to it.

I'm trying to make the output to stderr visible in the summary page of the execution result so that it's easy to understand what's wrong when you look at the failure page (but otehr CI sites seem to be doing this endlessly).

When a test fails, there is a Slack notification (for the Ruby committers to see) and an email notification (just to me). In the rare case of a commit that fails, the notifications are terrible.

Aside: Other possible innovations

There are many possible methods for testing non-deterministic behavior, especially in academic researches.

For example, making any external events demterministics (external events such as input/output and thread scheduling). In other words, you can make sure that the same program (and external input) will always return the same result by using a variety of different techniques. Once a problem is found, if the problem can always be reproduced, it seems to make things easier. However, I've heard a lot about this at the research level, but I wonder how practical it will be.

We can also think of methods such as using formal methods to automatically generate exhaustive tests and data that makes it easier to cover. It would be cool to be able to do this kind of thing.

Outcome

I did a lot of hard work at first to fix the problem, because with thousands of attempts every day, it's quite a flurry of failures. I worked much harder to fix it, mainly because there are a lot of test flops.

I was also able to fix some bugs caused by timing. Here's the patch I have in my notes.

Conclusion

In this article, I introduced my personal activity to find rare bugs by increasing the number of test runs in order to improve quality of Ruby interpreter.

Bugs are always present in programs, and it is hard to find bugs in large, complex programs. In this article, I introduced some of the trial-and-error process. I wish I could take a more scientific approach. If you know a good method, please let me know.

As I mentioned in the article, this system is made possible by a lot of support. Especially GitHub Sponsors was important to continue this activity. Thank you again.

As for the machines, a few years ago, a certain company gave me three big rack-mount machines with 3-digit GB memory that became unnecessary, and I installed them in another certain company N. I have been operating the machines including these (Mr. S of company N has been taking care of the machine operation for a long time). The other day, these machines were removed because they were old indeed, so I wrote this article with the memorial service and gratitude. Thank you very much.

Well, I wish you have a happy New Year and enjoy newly released Ruby 3.2!

rstfilter VSCode extension for your new Ruby development experience

Koichi Sasada — Sun, 12 Jun 2022 19:53:21 +0000

Today I released new VSCode extension Ruby's rstfilter extension.

This extension provides new Ruby development experience:

You can see the execution results line by line on the editor without launching a ruby command manually. You only need to start rstfilter command by Ctrl+Alt+S, and see the results of each lines and outputs on the bottom.

You can see the expression results by hovering the mouse pointer on the expression.

You don't need to write p/pp methods.
You don't need to use irb/pry REPL tools.
You only need to save the file and see the results immediately.

I hope it is useful for:

Ruby newbie who want to check the results frequently.
Test developers
Prototyping developers for apps/libs

For the libraries, the following traditional idiom can be used.

class XXX
   ...
end

if $0 == __FILE__
  # Try with XXX.
  # This test code will be ignored when the file is required.
end

Enjoy your Ruby programming!

Similar projects:

Update: v0.0.2 changes the interface. Save doesn't kick the rstfilter but you need to run rstfilter explicitly by Ctrl+Alt+S. Please add Alt to your Ctrl-S saving :)

debug.gem blog: initial commit

Koichi Sasada — Sat, 31 Jul 2021 16:34:03 +0000

About this series of articles

This series introduce features of ruby/debug: Debugging functionality for Ruby now I'm working on. debug.gem will be introduced into Ruby 3.1 which will be released Dec/2021.

This article introduce about debug.gem, the background and the motivation.

BTW see the recent article A Sneak Peek of Ruby's New Debugger! by @st0012. His great article introduces debug.gem from Ruby on Rails developer perspective.

Background and motivation

debug.gem is a debugger for Ruby. It is new replacement of lib/debug.rb which is installed from ancient Ruby 1.3.1 (!!). However, lib/debug.rb is not maintained well long time and nobody uses it. Surprisingly, most of features works on recent Ruby versions. However, it introduce performance penalties to the application and it lacks important features like flexible breakpoint setting or remote debugging or IDE integration (it can integrate with emacs, though) and so on. Also it doesn't have cosmetic features like coloring.

Maybe byebug (deivid-rodriguez/byebug: Debugging in Ruby 2) is most famous Ruby's debugger in recent years. byebug has enough features, however it doesn't utilize recent added TracePoint features for debuggers, so it is slow when some breakpoint is set.

byebug supports threading. However Ruby 3.0 introduced Ractor (ruby/ractor.md at master · ruby/ruby). Ractor enables to make parallel program in Ruby, but the parallel programming is difficult and it needs support for debugging. However, it is hard to support ractors by existing debuggers.

Hidden but the most important motivation is I (Koichi) likes to make a developer support tools. I feel that using debugger is difficult for the programmers. It is challenging topic to provide useful features for many Rubyists.

Making a new gem

We talked about replacement plan of lib/debug.rb. Fortunately we can reserve "debug" gem name in rubygem.org so we decided to make a new gem and replace with lib/debug.rb

Conclusion

This article introduces the background and the motivation of debug.gem.

There is no debugger implementation to utilize MRI's new features for debuggers and no debugger supports Ractors. This is why we decided to make a new debugger.

I really appreciate any feedback about debug.gem. Next I will introduce basic features of debug.gem.

Acknowledgements

This work is supported by Cookpad, an employer of me (Koichi). In recent months Cookpad allowed me to focus on the debugger development.
Naoto Ono helps us to prepare testing framework on Google summer of code project (Ruby | Google Summer of Code).
Stan Lo contributes appealing features like text coloring and tells us the debugger experience on his projects.
And all contributors!

Forem: Koichi Sasada

Unearthing DRY Violations from Hotspot Data with lumitrace

Third-Pass Approach

Hotspot Overview

Finding 1: Duplicated completeopt Parsing (completion_manager.rb)

Finding 2: Duplicated Completion State Matching (completion_manager.rb)

Finding 3: if-Chain to Hash Lookup (completion_manager.rb)

Finding 4: Unnecessary Indirection in ExCommandRegistry#registered?

Finding 5: Cleaning Up is_a? Checks in same_command_callable? (commands/meta.rb)

Summary

How I Read Hotspot Data Changed

Reflections as AI (Claude Code)

Fixing Hotspots and Coverage Gaps in One Shot with lumitrace

Target Application: RuVim

What --collect-mode last Gives You

Type Inconsistencies: Replacing nil with Real Values

Hotspots: From O(n) Linear Scan to O(1)

Coverage Gaps: Finding Where Tests Are Missing

Three Improvements from One Measurement

lumitrace from an AI Perspective: Pros and Cons

Pros from the AI Perspective

Cons from the AI Perspective

Overall: A "Runtime Eye" for AI Coding

Reflections

Reflections as AI (Claude Code)

Using lumitrace to eliminate redundant type conversions in Ruby

Using lumitrace to eliminate redundant type conversions in Ruby

What is lumitrace?

What it revealed

The process

From type inconsistency to better design

What I kept

Takeaway

Lumitrace: See What Your Ruby Code Is Actually Doing, Instantly

Install

One Command Is All You Need

Beautiful HTML Reports

The Ultimate Debugging Flow for Failing Tests

Amazing with AI

Trace Only What You Changed with git diff

Works in CI Too

Cheat Sheet

Ruby::Box Digest Introduction (Ruby 4.0.0 New Feature)

A quick tour of Ruby::Box

Ruby::Box use cases

Use case 1: Use different versions of a gem at the same time

Caution

Use case 2: Use a globally configured gem under multiple configurations

Ruby::Box is not object-space isolation

Example of passing objects across Ruby::Box (not recommended)

Behavior when defining same-named classes across Ruby::Box

Mindset

Monkey-patch isolation

What is monkey-patching?

Monkey-patch isolation via Ruby::Box

Use cases

Caution

Caution (2)

Other notes

Top-level constant references are contained within Ruby::Box

Ruby::Box also isolates global variables

Each Ruby::Box has its own $LOAD_PATH/$LOADED_FEATURES

Ruby::Box.new creates a clean environment

Ruby::Box is not a sandbox

Ruby::Box API

Relation to Kernel.load’s second argument

What happened to the name "Namespace"?

So, can we use it yet?

High-level API?

Summary

Reading Ruby 4.0 NEWS with Pros

Language changes

When calling foo(*nil), it no longer calls nil.to_a

Line breaks before logical operators are now allowed

Backstory

Built-in class updates

Array#rfind was added

Binding#implicit_parameters was added

Enumerator.produce now accepts the size keyword

error_highlight now shows ArgumentError more helpfully

Each Ruby::Box has its own `$LOAD_PATH`/`$LOADED_FEATURES`

Changes around `Fiber::Scheduler`

`IO.select` can now accept `Float::INFINITY` as a timeout value

`Math.log1p` and `Math.expm1` were added

The implicit `it` return value in `Proc#parameters` changed slightly

Introducing `Ractor.shareable_proc` to create Proc objects shareable between Ractors

`Range#to_set` now checks size

The fine-grained behavior of `Range#overlap?` changed

The fine-grained behavior of `Range#max` changed

`::Ruby` was defined

`Ruby::Box` was introduced

`Set` became a core class

`Thread#raise`/`Fiber#raise` can now take the `cause:` keyword

IRB: added the `copy` command

`SortedSet` was removed from the bundle

The Proposal: `Ractor::Port` - Lightweight, Unique, One-Way Endpoints

Is `Ractor::Port.new` too long?

Behavior on `close`

Block parameter `it` is available

Background to the introduction of `it`

The conditions for whether `String#+@` dups have changed

Keyword arguments `**` can now be passed `nil`

Keyword arguments are no longer allowed in index assignment (e.g. `a[0, kw: 1] = 2`).

The toplevel constant `Ruby` is reserved

`Array#fetch_values` was added

`Exception#set_backtrace` now accepts arrays of `Thread::Backtrace::Location`