ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
* [ruby-core:120637] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
@ 2025-01-13 17:35 byroot (Jean Boussier) via ruby-core
  2025-01-13 19:47 ` [ruby-core:120639] " Eregon (Benoit Daloze) via ruby-core
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: byroot (Jean Boussier) via ruby-core @ 2025-01-13 17:35 UTC (permalink / raw)
  To: ruby-core; +Cc: byroot (Jean Boussier)

Issue #21032 has been reported by byroot (Jean Boussier).

----------------------------------------
Bug #21032: `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
https://bugs.ruby-lang.org/issues/21032

* Author: byroot (Jean Boussier)
* Status: Open
* Backport: 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED
----------------------------------------
Reproduction script:

```ruby
require 'benchmark'

$LOAD_PATH << 'relative-path'

autoload :FOO, '/tmp/foo.rb'

puts Benchmark.realtime {
  500_000.times do
    Object.autoload?(:FOO)
  end
}
```

The above takes 2.5 to 3 seconds on my machine, but just removing `$LOAD_PATH << 'relative-path'` make it complete in 50ms.
It's such a stark difference that I think it is a bug, and it cause Zeitwerk, a very popular gem, to be way slower than it should
when the load path contains relative paths.

I have a patch for it, that passes all tests, but I'd appreciate some eyes on it: https://github.com/ruby/ruby/pull/12562

cc @fxn 



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [ruby-core:120639] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
  2025-01-13 17:35 [ruby-core:120637] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path byroot (Jean Boussier) via ruby-core
@ 2025-01-13 19:47 ` Eregon (Benoit Daloze) via ruby-core
  2025-01-13 19:49 ` [ruby-core:120640] " byroot (Jean Boussier) via ruby-core
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Eregon (Benoit Daloze) via ruby-core @ 2025-01-13 19:47 UTC (permalink / raw)
  To: ruby-core; +Cc: Eregon (Benoit Daloze)

Issue #21032 has been updated by Eregon (Benoit Daloze).


For curiosity, what's adding a relative path in `$LOAD_PATH`? That sounds rather dubious/wrong.

----------------------------------------
Bug #21032: `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
https://bugs.ruby-lang.org/issues/21032#change-111468

* Author: byroot (Jean Boussier)
* Status: Open
* Backport: 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED
----------------------------------------
Reproduction script:

```ruby
require 'benchmark'

$LOAD_PATH << 'relative-path'

autoload :FOO, '/tmp/foo.rb'

puts Benchmark.realtime {
  500_000.times do
    Object.autoload?(:FOO)
  end
}
```

The above takes 2.5 to 3 seconds on my machine, but just removing `$LOAD_PATH << 'relative-path'` make it complete in 50ms.
It's such a stark difference that I think it is a bug, and it cause Zeitwerk, a very popular gem, to be way slower than it should
when the load path contains relative paths.

I have a patch for it, that passes all tests, but I'd appreciate some eyes on it: https://github.com/ruby/ruby/pull/12562

cc @fxn 



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [ruby-core:120640] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
  2025-01-13 17:35 [ruby-core:120637] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path byroot (Jean Boussier) via ruby-core
  2025-01-13 19:47 ` [ruby-core:120639] " Eregon (Benoit Daloze) via ruby-core
@ 2025-01-13 19:49 ` byroot (Jean Boussier) via ruby-core
  2025-01-14  8:25 ` [ruby-core:120658] " byroot (Jean Boussier) via ruby-core
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: byroot (Jean Boussier) via ruby-core @ 2025-01-13 19:49 UTC (permalink / raw)
  To: ruby-core; +Cc: byroot (Jean Boussier)

Issue #21032 has been updated by byroot (Jean Boussier).


@fxn tracked it down to https://github.com/emailage/Emailage_Ruby/blob/64b9762cda7608ac1eeced2a85ad5f4b7919f4f0/lib/emailage.rb#L1, and yes that's an anti-pattern in my opinion too.



----------------------------------------
Bug #21032: `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
https://bugs.ruby-lang.org/issues/21032#change-111469

* Author: byroot (Jean Boussier)
* Status: Open
* Backport: 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED
----------------------------------------
Reproduction script:

```ruby
require 'benchmark'

$LOAD_PATH << 'relative-path'

autoload :FOO, '/tmp/foo.rb'

puts Benchmark.realtime {
  500_000.times do
    Object.autoload?(:FOO)
  end
}
```

The above takes 2.5 to 3 seconds on my machine, but just removing `$LOAD_PATH << 'relative-path'` make it complete in 50ms.
It's such a stark difference that I think it is a bug, and it cause Zeitwerk, a very popular gem, to be way slower than it should
when the load path contains relative paths.

I have a patch for it, that passes all tests, but I'd appreciate some eyes on it: https://github.com/ruby/ruby/pull/12562

cc @fxn 



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [ruby-core:120658] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
  2025-01-13 17:35 [ruby-core:120637] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path byroot (Jean Boussier) via ruby-core
  2025-01-13 19:47 ` [ruby-core:120639] " Eregon (Benoit Daloze) via ruby-core
  2025-01-13 19:49 ` [ruby-core:120640] " byroot (Jean Boussier) via ruby-core
@ 2025-01-14  8:25 ` byroot (Jean Boussier) via ruby-core
  2025-01-14 13:15 ` [ruby-core:120661] " Eregon (Benoit Daloze) via ruby-core
  2025-01-27  9:34 ` [ruby-core:120801] " byroot (Jean Boussier) via ruby-core
  4 siblings, 0 replies; 6+ messages in thread
From: byroot (Jean Boussier) via ruby-core @ 2025-01-14  8:25 UTC (permalink / raw)
  To: ruby-core; +Cc: byroot (Jean Boussier)

Issue #21032 has been updated by byroot (Jean Boussier).


I dug more into this today, based on @nobu's review. `autoload?` isn't the only thing slowed down in such case.

Perhaps we should try to emit a performance warning when a relative path or non-string object is appended to `$LOAD_PATH`. That would at least make it easier to notice this issue.

The problem of course is that `$LOAD_PATH` is just a regular array, so there isn't a clean hook where to check for this.

----------------------------------------
Bug #21032: `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
https://bugs.ruby-lang.org/issues/21032#change-111484

* Author: byroot (Jean Boussier)
* Status: Open
* Backport: 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED
----------------------------------------
Reproduction script:

```ruby
require 'benchmark'

$LOAD_PATH << 'relative-path'

autoload :FOO, '/tmp/foo.rb'

puts Benchmark.realtime {
  500_000.times do
    Object.autoload?(:FOO)
  end
}
```

The above takes 2.5 to 3 seconds on my machine, but just removing `$LOAD_PATH << 'relative-path'` make it complete in 50ms.
It's such a stark difference that I think it is a bug, and it cause Zeitwerk, a very popular gem, to be way slower than it should
when the load path contains relative paths.

I have a patch for it, that passes all tests, but I'd appreciate some eyes on it: https://github.com/ruby/ruby/pull/12562

cc @fxn 



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [ruby-core:120661] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
  2025-01-13 17:35 [ruby-core:120637] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path byroot (Jean Boussier) via ruby-core
                   ` (2 preceding siblings ...)
  2025-01-14  8:25 ` [ruby-core:120658] " byroot (Jean Boussier) via ruby-core
@ 2025-01-14 13:15 ` Eregon (Benoit Daloze) via ruby-core
  2025-01-27  9:34 ` [ruby-core:120801] " byroot (Jean Boussier) via ruby-core
  4 siblings, 0 replies; 6+ messages in thread
From: Eregon (Benoit Daloze) via ruby-core @ 2025-01-14 13:15 UTC (permalink / raw)
  To: ruby-core; +Cc: Eregon (Benoit Daloze)

Issue #21032 has been updated by Eregon (Benoit Daloze).


byroot (Jean Boussier) wrote in #note-2:
> @fxn tracked it down to https://github.com/emailage/Emailage_Ruby/blob/64b9762cda7608ac1eeced2a85ad5f4b7919f4f0/lib/emailage.rb#L1, and yes that's an anti-pattern in my opinion too.

Right, that's just broken code. It would consider files under `lib` in $PWD for require which is wrong.

> Perhaps we should try to emit a performance warning when a relative path or non-string object is appended to `$LOAD_PATH`. That would at least make it easier to notice this issue.
> 
> The problem of course is that `$LOAD_PATH` is just a regular array, so there isn't a clean hook where to check for this.

Maybe just warn on `require` or so, i.e. when noticing the path is relative, if it's too hard to notice when it's added?
I think there is also a cache for the expanded load path, maybe that could be a reasonable place to check.
It wouldn't immediately point at the culprit, but printing the $LOADED_FEATURES should help find it (maybe there should be a CLI flag/ENV var to print files as they are loaded, TruffleRuby has that).

----------------------------------------
Bug #21032: `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
https://bugs.ruby-lang.org/issues/21032#change-111487

* Author: byroot (Jean Boussier)
* Status: Open
* Backport: 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED
----------------------------------------
Reproduction script:

```ruby
require 'benchmark'

$LOAD_PATH << 'relative-path'

autoload :FOO, '/tmp/foo.rb'

puts Benchmark.realtime {
  500_000.times do
    Object.autoload?(:FOO)
  end
}
```

The above takes 2.5 to 3 seconds on my machine, but just removing `$LOAD_PATH << 'relative-path'` make it complete in 50ms.
It's such a stark difference that I think it is a bug, and it cause Zeitwerk, a very popular gem, to be way slower than it should
when the load path contains relative paths.

I have a patch for it, that passes all tests, but I'd appreciate some eyes on it: https://github.com/ruby/ruby/pull/12562

cc @fxn 



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [ruby-core:120801] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
  2025-01-13 17:35 [ruby-core:120637] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path byroot (Jean Boussier) via ruby-core
                   ` (3 preceding siblings ...)
  2025-01-14 13:15 ` [ruby-core:120661] " Eregon (Benoit Daloze) via ruby-core
@ 2025-01-27  9:34 ` byroot (Jean Boussier) via ruby-core
  4 siblings, 0 replies; 6+ messages in thread
From: byroot (Jean Boussier) via ruby-core @ 2025-01-27  9:34 UTC (permalink / raw)
  To: ruby-core; +Cc: byroot (Jean Boussier)

Issue #21032 has been updated by byroot (Jean Boussier).

Status changed from Open to Closed

I merged https://github.com/ruby/ruby/pull/12562 / d4a1a2780c39bc648496ac92fc6e6ce2eb38ab47 because I couldn't find any case where this would change behavior.

----------------------------------------
Bug #21032: `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path
https://bugs.ruby-lang.org/issues/21032#change-111668

* Author: byroot (Jean Boussier)
* Status: Closed
* Backport: 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED
----------------------------------------
Reproduction script:

```ruby
require 'benchmark'

$LOAD_PATH << 'relative-path'

autoload :FOO, '/tmp/foo.rb'

puts Benchmark.realtime {
  500_000.times do
    Object.autoload?(:FOO)
  end
}
```

The above takes 2.5 to 3 seconds on my machine, but just removing `$LOAD_PATH << 'relative-path'` make it complete in 50ms.
It's such a stark difference that I think it is a bug, and it cause Zeitwerk, a very popular gem, to be way slower than it should
when the load path contains relative paths.

I have a patch for it, that passes all tests, but I'd appreciate some eyes on it: https://github.com/ruby/ruby/pull/12562

cc @fxn 



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2025-01-27  9:50 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-01-13 17:35 [ruby-core:120637] [Ruby master Bug#21032] `Module#autoload?` is slow when `$LOAD_PATH` contains a relative path byroot (Jean Boussier) via ruby-core
2025-01-13 19:47 ` [ruby-core:120639] " Eregon (Benoit Daloze) via ruby-core
2025-01-13 19:49 ` [ruby-core:120640] " byroot (Jean Boussier) via ruby-core
2025-01-14  8:25 ` [ruby-core:120658] " byroot (Jean Boussier) via ruby-core
2025-01-14 13:15 ` [ruby-core:120661] " Eregon (Benoit Daloze) via ruby-core
2025-01-27  9:34 ` [ruby-core:120801] " byroot (Jean Boussier) via ruby-core

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).