Skip to content

Ruby 3.1 Support in TruffleRuby #2733

New issue
New issue
Closed
Closed
Ruby 3.1 Support in TruffleRuby#2733
Assignees
eregonandrykonchin
Labels
compatibilitygood first issue
Milestone
 
23.0.0 Release (June 13, 2023)
@fniephaus

Description

@fniephaus
fniephaus
opened on Sep 22, 2022

We are working on Ruby 3.1 support for the next release.

Any help is appreciated whether it is:

To make it easier to find good getting-started issues, I annotated items with easy, medium and hard.
There is also pure ruby for changes which should only affect Ruby code and not require any change to Java sources or any Java knowledge.
If there is no pure ruby then some Java code is involved, but it should still be fairly straightforward if easy or medium.
hard typically requires more knowledge about TruffleRuby implementation details.

If you'd like to start on some task please mention it here or edit the description directly for committers (to avoid duplicated work).

Notes

  • Fiber scheduler changes are not implemented because it seems not worth it until Truffle supports VirtualThread on both Native Image and HotSpot.
  • Ractor is not implemented because it seems of little value. Thread already run in parallel and are far more compatible.

Full list of changes for Ruby 3.1.2

NOTE: https://rubyreferences.github.io/rubychanges/3.1.html gives more details for many features and changes.

From https://github.com/ruby/ruby/blob/master/doc/NEWS/NEWS-3.1.0.md:

This document is a list of user-visible feature changes
since the 3.0.0 release, except for bug fixes.

Note that each entry is kept to a minimum, see links for details.

Language changes

  • The block argument can now be anonymous if the block will
    only be passed to another method. [Feature #11256]

    def foo(&)
  bar(&)
end

  • [hard, need pattern matching first] Pin operator now takes an expression. [Feature #17411]

    Prime.each_cons(2).lazy.find_all{_1 in [n, ^(n + 2)]}.take(3).to_a
#=> [[3, 5], [5, 7], [11, 13]]

  • [hard, need pattern matching first] Pin operator now supports instance, class, and global variables.
    [Feature #17724]

    @n = 5
Prime.each_cons(2).lazy.find{_1 in [n, ^@n]}
#=> [3, 5]

  • [hard, need pattern matching first] One-line pattern matching is no longer experimental.

  • [hard, need pattern matching first] Parentheses can be omitted in one-line pattern matching.
    [Feature #16182]

    [0, 1] => _, x
{y: 2} => y:
x #=> 1
y #=> 2

  • @andrykonchin [hard, java] Multiple assignment evaluation order has been made consistent with
    single assignment evaluation order. With single assignment, Ruby
    uses a left-to-right evaluation order. With this code:

    foo[0] = bar

    The following evaluation order is used:

    1. foo
    2. bar
    3. []= called on the result of foo

    In Ruby before 3.1.0, multiple assignment did not follow this
    evaluation order. With this code:

    foo[0], bar.baz = a, b

    Versions of Ruby before 3.1.0 would evaluate in the following
    order

    1. a
    2. b
    3. foo
    4. []= called on the result of foo
    5. bar
    6. baz= called on the result of bar

    Starting in Ruby 3.1.0, the evaluation order is now consistent with
    single assignment, with the left-hand side being evaluated before
    the right-hand side:

    1. foo
    2. bar
    3. a
    4. b
    5. []= called on the result of foo
    6. baz= called on the result of bar

    [Bug #4443]

  • Values in Hash literals and keyword arguments can be omitted.
    [Feature #14579]

    For example,

    • {x:, y:} is a syntax sugar of {x: x, y: y}.
    • foo(x:, y:) is a syntax sugar of foo(x: x, y: y).

    Constant names, local variable names, and method names are allowed as
    key names. Note that a reserved word is considered as a local
    variable or method name even if it's a pseudo variable name such as
    self.

    • [hard, need Ractor first] Non main-Ractors can get instance variables (ivars) of classes/modules
      if ivars refer to shareable objects.
      [Feature #17592]

  • Support endless method definitions #3038 [hard, parser] A command syntax is allowed in endless method definitions, i.e.,
    you can now write def foo = puts "Hello".
    Note that private def foo = puts "Hello" does not parse.
    [Feature #17398], depends on [Feature #16746]

Command line options

  • --disable-gems is now explicitly declared as "just for debugging".
    Never use it in any real-world codebase.
    [Feature #17684]

Core classes updates

Note: We're only listing outstanding class updates.

Array

Class

  • (@andrykonchin) Class#subclasses, which returns an array of classes
    directly inheriting from the receiver, not
    including singleton classes. [Feature #18273]

    class A; end
class B < A; end
class C < B; end
class D < A; end
A.subclasses    #=> [D, B]
B.subclasses    #=> [C]
C.subclasses    #=> []

Enumerable

  • Enumerable#compact is added. [Feature #17312]

  • Enumerable#tally now accepts an optional hash to count. [Feature #17744]

  • @horakivo [easy, pure ruby] Enumerable#each_cons and each_slice to return a receiver. [GH-1509]

    [1, 2, 3].each_cons(2){}
# 3.0 => nil
# 3.1 => [1, 2, 3]

[1, 2, 3].each_slice(2){}
# 3.0 => nil
# 3.1 => [1, 2, 3]

Enumerator::Lazy

File

  • [easy, pure ruby] File.dirname now accepts an optional argument for the level to
    strip path components. [Feature #12194]

GC

  • "GC.measure_total_time = true" enables the measurement of GC.
    Measurement can introduce overhead. It is enabled by default.
    GC.measure_total_time returns the current setting.
    GC.stat[:time] or GC.stat(:time) returns measured time
    in milli-seconds. [[Feature #10917]]

  • GC.total_time returns measured time in nano-seconds. [[Feature #10917]]

Integer

Kernel

  • (@andrykonchin) Kernel#load now accepts a module as the second argument,
    and will load the file using the given module as the
    top-level module. [Feature #6210]

Marshal

  • [medium, pure ruby] Marshal.load now accepts a freeze: true option.
    All returned objects are frozen except for Class and
    Module instances. Strings are deduplicated. [Feature #18148]

MatchData

Method / UnboundMethod

  • Method#public?, Method#private?, Method#protected?,
    UnboundMethod#public?, UnboundMethod#private?,
    UnboundMethod#protected? have been added. [Feature #11689]

Module

  • [hard, java, to do after fixing super] Module#prepend now modifies the ancestor chain if the receiver
    already includes the argument. Module#prepend still does not
    modify the ancestor chain if the receiver has already prepended
    the argument. [Bug #17423]

  • @horakivo [medium, use SingleValueCastNode] Module#private, #public, #protected, and #module_function will
    now return their arguments. If a single argument is given, it
    is returned. If no arguments are given, nil is returned. If
    multiple arguments are given, they are returned as an array.
    [Feature #12495]

Process

  • @horakivo [easy, pure ruby, should be an "undefined" method] Process._fork is added. This is a core method for fork(2).
    Do not call this method directly; it is called by existing
    fork methods: Kernel.#fork, Process.fork, and IO.popen("-").
    Application monitoring libraries can overwrite this method to
    hook fork events. [Feature #17795]

Struct

  • (won't do, would makes things slower, more complex, and the code be removed in 3.2) @eregon [easy, pure ruby] Passing only keyword arguments to Struct#initialize is warned.
    You need to use a Hash literal to set a Hash to a first member.
    [Feature #16806]

  • (@moste00) [easy, pure ruby] StructClass#keyword_init? is added [Feature #18008]

String

  • @horakivo [easy, pure ruby] Update Unicode version to 13.0.0 [Feature #17750]
    and Emoji version to 13.0 [Feature #18029]

  • (@andrykonchin) String#unpack and String#unpack1 now accept an offset: keyword
    argument to start the unpacking after an arbitrary number of bytes
    have been skipped. If offset is outside of the string bounds
    ArgumentError is raised. [Feature #18254]

Thread

Thread::Backtrace

  • [easy, pure ruby] Thread::Backtrace.limit, which returns the value to limit backtrace
    length set by --backtrace-limit command line option, is added.
    [Feature #17479]

Thread::Queue

  • Thread::Queue.new now accepts an Enumerable of initial values.
    [Feature #17327]

Time

  • [easy, pure ruby] Time.new now accepts optional in: keyword argument for the
    timezone, as well as Time.at and Time.now, so that is now
    you can omit minor arguments to Time.new. [Feature #17485]

    Time.new(2021, 12, 25, in: "+07:00")
#=> 2021-12-25 00:00:00 +0700

    At the same time, time component strings are converted to
    integers more strictly now.

    Time.new(2021, 12, 25, "+07:30")
#=> invalid value for Integer(): "+07:30" (ArgumentError)

    Ruby 3.0 or earlier returned probably unexpected result
    2021-12-25 07:00:00, not 2021-12-25 07:30:00 nor
    2021-12-25 00:00:00 +07:30.

  • (@andrykonchin) Time#strftime supports RFC 3339 UTC for unknown offset local
    time, -0000, as %-z. [Feature #17544]

TracePoint

  • [medium, java] TracePoint.allow_reentry is added to allow reenter while TracePoint
    callback.
    [Feature #15912]

$LOAD_PATH

  • [medium, pure ruby, check if not already the case] $LOAD_PATH.resolve_feature_path does not raise. [Feature #16043]

Fiber Scheduler

    • [hard, need Fiber scheduler core first] Add support for Addrinfo.getaddrinfo using address_resolve hook.
      [Feature #17370]
    • [hard, need Fiber scheduler core first] Introduce non-blocking Timeout.timeout using timeout_after hook.
      [Feature #17470]
    • [hard, need Fiber scheduler core first] Introduce new scheduler hooks io_read and io_write along with a
      low level IO::Buffer for zero-copy read/write. [Feature #18020]
    • [hard, need Fiber scheduler core first] IO hooks io_wait, io_read, io_write, receive the original IO object
      where possible. [Bug #18003]

  • Make Monitor fiber-safe. [Bug #17827]

  • Replace copy coroutine with pthread implementation. [Feature #18015]

Refinement

  • New class which represents a module created by Module#refine.
  • @horakivo [medium, pure ruby] include and prepend are deprecated, and import_methods is added
    instead. [Bug #17429]

Stdlib updates

  • The following default gem are updated.

    • RubyGems 3.3.3
    • base64 0.1.1
    • benchmark 0.2.0
    • bigdecimal 3.1.1
    • bundler 2.3.3
    • cgi 0.3.1
    • csv 3.2.2
    • date 3.2.2
    • did_you_mean 1.6.1
    • digest 3.1.0
    • drb 2.1.0
    • erb 2.2.3
    • error_highlight 0.3.0
    • etc 1.3.0
    • fcntl 1.0.1
    • fiddle 1.1.0
    • fileutils 1.6.0
    • find 0.1.1
    • io-console 0.5.10
    • io-wait 0.2.1
    • ipaddr 1.2.3
    • irb 1.4.1
    • json 2.6.1
    • logger 1.5.0
    • net-http 0.2.0
    • net-protocol 0.1.2
    • nkf 0.1.1
    • open-uri 0.2.0
    • openssl 3.0.0
    • optparse 0.2.0
    • ostruct 0.5.2
    • pathname 0.2.0
    • pp 0.3.0
    • prettyprint 0.1.1
    • psych 4.0.3
    • racc 1.6.0
    • rdoc 6.4.0
    • readline 0.0.3
    • readline-ext 0.1.4
    • reline 0.3.0
    • resolv 0.2.1
    • rinda 0.1.1
    • ruby2_keywords 0.0.5
    • securerandom 0.1.1
    • set 1.0.2
    • stringio 3.0.1
    • strscan 3.0.1
    • tempfile 0.1.2
    • time 0.2.0
    • timeout 0.2.0
    • tmpdir 0.1.2
    • un 0.2.0
    • uri 0.11.0
    • yaml 0.2.0
    • zlib 2.1.1

  • The following bundled gems are updated.

    • minitest 5.15.0
    • power_assert 2.0.1
    • rake 13.0.6
    • test-unit 3.5.3
    • rexml 3.2.5
    • rbs 2.0.0
    • typeprof 0.21.1

  • The following default gems are now bundled gems.

    • net-ftp 0.1.3
    • net-imap 0.2.2
    • net-pop 0.1.1
    • net-smtp 0.3.1
    • matrix 0.4.2
    • prime 0.1.2
    • debug 1.4.0

  • The following gems has been removed from the Ruby standard library.

    • dbm
    • gdbm
    • tracer

  • [hard, java] Coverage measurement now supports suspension. You can use Coverage.suspend
    to stop the measurement temporarily, and Coverage.resume to restart it.
    See [Feature #18176] in detail.

  • Random::Formatter is moved to random/formatter.rb, so that you can
    use Random#hex, Random#base64, and so on without SecureRandom.
    [Feature #18190]

Compatibility issues

Note: Excluding feature bug fixes.

  • [note: they are not actually deprecated in headers] rb_io_wait_readable, rb_io_wait_writable and rb_wait_for_single_fd are
    deprecated in favour of rb_io_maybe_wait_readable,
    rb_io_maybe_wait_writable and rb_io_maybe_wait respectively.
    rb_thread_wait_fd and rb_thread_fd_writable are deprecated. [Bug #18003]
  • [easy, C] rb_io_maybe_wait_readable, rb_io_maybe_wait_writable and rb_io_maybe_wait are added.

Stdlib compatibility issues

  • ERB#initialize warns safe_level and later arguments even without -w.
    [Feature #14256]

  • lib/debug.rb is replaced with debug.gem

  • Kernel#pp in lib/pp.rb uses the width of IO#winsize by default.
    This means that the output width is automatically changed depending on
    your terminal size. [Feature #12913]

  • Psych 4.0 changes Psych.load as safe_load by the default.
    You may need to use Psych 3.3.2 for migrating to this behavior.
    [Bug #17866]

C API updates

Implementation improvements

  • Inline cache mechanism is introduced for reading class variables.
    [Feature #17763]

  • instance_eval and instance_exec now only allocate a singleton class when
    required, avoiding extra objects and improving performance. [GH-5146]

  • The performance of Struct accessors is improved. [GH-5131]

  • mandatory_only? builtin special form to improve performance on
    builtin methods. [GH-5112]

  • Experimental feature Variable Width Allocation in the garbage collector.
    This feature is turned off by default and can be enabled by compiling Ruby
    with flag USE_RVARGC=1 set. [Feature #18045] [Feature #18239]

Debugger

  • A new debugger debug.gem is bundled.
    debug.gem is a fast debugger implementation, and it provides many features
    like remote debugging, colorful REPL, IDE (VSCode) integration, and more.
    It replaces lib/debug.rb standard library.

  • rdbg command is also installed into bin/ directory to start and control
    debugging execution.

error_highlight

A built-in gem called error_highlight has been introduced.
It shows fine-grained error locations in the backtrace.

Example: title = json[:article][:title]

If json is nil, it shows:

$ ruby test.rb
test.rb:2:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError)

title = json[:article][:title]
            ^^^^^^^^^^

If json[:article] returns nil, it shows:

$ ruby test.rb
test.rb:2:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError)

title = json[:article][:title]
                      ^^^^^^^^

This feature is enabled by default.
You can disable it by using a command-line option --disable-error_highlight.
See the repository in detail.

  • [hard, won't do while error_highlight depends on RubyVM, we can do this once error_highlight switches to Prism] Investigate how to make it work, currently it's not loaded by default and requiring seems to have no effect.

Miscellaneous changes

  • [easy, pure Ruby, need to update tool/import-mri-files.sh to copy this file] lib/objspace/trace.rb is added, which is a tool for tracing the object
    allocation. Just by requiring this file, tracing is started immediately.
    Just by Kernel#p, you can investigate where an object was created.
    Note that just requiring this file brings a large performance overhead.
    This is only for debugging purposes. Do not use this in production.
    [Feature #17762]

  • [easy, pure Ruby, need to update tool/import-mri-files.sh to copy this file] Reuse ext/objspace/lib/objspace.rb from CRuby

  • @eregon [easy, java] Now exceptions raised in finalizers will be printed to STDERR, unless
    $VERBOSE is nil. [Feature #17798]

  • ruby -run -e httpd displays URLs to access. [Feature #17847]

  • Add ruby -run -e colorize to colorize Ruby code using
    IRB::Color.colorize_code.

  • @eregon fiber.so is now required by default, so Fiber.current, transfer available by default.

Metadata

Metadata

Assignees

Labels

compatibilitygood first issue

Type

No type

Projects

TruffleRuby Roadmap

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions