[ruby-dev:52069] [Ruby master Bug#20311] Struct.new("A") memory leak?

[ruby-dev:52069] [Ruby master Bug#20311] Struct.new("A") memory leak?

[MaxLap (Maxime Lapointe) via ruby-dev]

Issue #20311 has been reported by MaxLap (Maxime Lapointe).

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The following code gives the impression of a memory leak.

```
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52070] [Ruby master Bug#20311] Struct.new("A") memory leak?

[byroot (Jean Boussier) via ruby-dev]

Issue #20311 has been updated by byroot (Jean Boussier).


I had a quick look at this using `ObjectSpace.dump_all`, and it appears that these classes are being retained in the VM root:

```ruby
require 'objspace'

3.times do
  puts ObjectSpace.dump(Struct.new("A"))
  Struct.send(:remove_const, :A)
end

GC.start
ObjectSpace.dump_all(output: File.open("/tmp/heap.json", "w+"))
```

```
{"address":"0x102bc33f8", "type":"CLASS", ....
{"address":"0x102bc3218", "type":"CLASS", ....
{"address":"0x102bc3038", "type":"CLASS", ....
```

```
$ rg -F '"0x102bc3218"' /tmp/heap.json 
1:{"type":"ROOT", "root":"vm", "references":[..., "0x102bc3218"
$ rg -F '"0x102bc33f8"' /tmp/heap.json 
1:{"type":"ROOT", "root":"vm", "references":[..., "0x102bc33f8"
```



----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107049

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The following code gives the impression of a memory leak.

```
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52071] [Ruby master Bug#20311] Struct.new("A") memory leak?

[nobu (Nobuyoshi Nakada) via ruby-dev]

Issue #20311 has been updated by nobu (Nobuyoshi Nakada).


Seems referred from `defined_module_hash`.
`rb_const_remove` needs to take care of it when the removed constant is a class or module?

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107052

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The following code gives the impression of a memory leak.

```
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52072] [Ruby master Bug#20311] Struct.new("A") memory leak?

[byroot (Jean Boussier) via ruby-dev]

Issue #20311 has been updated by byroot (Jean Boussier).


> rb_const_remove needs to take care of it when the removed constant is a class or module?

The problem is that it's a set:

```c
st_insert(vm->defined_module_hash, (st_data_t)module, (st_data_t)module);
```

So if a module is added twice, you need to remove it from the set if it's removed twice... I'll see if I can do that.

But I also wonder why we have this root in the first place, I'd think these get marked from being constants in `::Object`.

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107073

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The following code gives the impression of a memory leak.

```
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52073] [Ruby master Bug#20311] Struct.new("A") memory leak?

[byroot (Jean Boussier) via ruby-dev]

Issue #20311 has been updated by byroot (Jean Boussier).


> But I also wonder why we have this root in the first place, I'd think these get marked from being constants in ::Object.

Nevermind, if I understand correctly, it's not to mark them, it's to pin them.

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107074

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The following code gives the impression of a memory leak.

```
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52074] [Ruby master Bug#20311] Struct.new("A") memory leak?

[byroot (Jean Boussier) via ruby-dev]

Issue #20311 has been updated by byroot (Jean Boussier).


I think https://github.com/ruby/ruby/pull/10143 will do. But a good followup would also be to not pin `Struct.new("A")`.

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107075

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The following code gives the impression of a memory leak.

```
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52075] [Ruby master Bug#20311] Struct.new("A") memory leak?

[byroot (Jean Boussier) via ruby-dev]

Issue #20311 has been updated by byroot (Jean Boussier).


So https://github.com/ruby/ruby/pull/10143  is turning into a huge yak-shave and will be hard to backport. The reason is a lot of code end up relying on classes defined in C becoming immortal (in addition to being pinned).

So maybe a better short term solution that is easy to backport is to just not use these API for naming structs: https://github.com/ruby/ruby/pull/10144

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107077

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: WONTFIX, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED
----------------------------------------
The following code gives the impression of a memory leak.

```
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52077] [Ruby master Bug#20311] Struct.new("A") memory leak?

[nobu (Nobuyoshi Nakada) via ruby-dev]

Issue #20311 has been updated by nobu (Nobuyoshi Nakada).


byroot (Jean Boussier) wrote in #note-3:
> So if a module is added twice, you need to remove it from the set if it's removed twice... I'll see if I can do that.

Is there any chance for the same module to be added twice?


----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107087

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: WONTFIX, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED
----------------------------------------
The following code gives the impression of a memory leak.

```
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52078] [Ruby master Bug#20311] Struct.new("A") memory leak?

[nobu (Nobuyoshi Nakada) via ruby-dev]

Issue #20311 has been updated by nobu (Nobuyoshi Nakada).


byroot (Jean Boussier) wrote in #note-7:
> So maybe a better short term solution that is easy to backport is to just not use these API for naming structs: https://github.com/ruby/ruby/pull/10144

Seems fine, but I'm afraid what can happen when an extension library stores `rb_path2class("Struct::A")` result then remove that constant?
Probably this is same for any Ruby-level defined classes/modules.

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107089

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: WONTFIX, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED
----------------------------------------
The following code gives the impression of a memory leak.

```ruby
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52079] [Ruby master Bug#20311] Struct.new("A") memory leak?

[byroot (Jean Boussier) via ruby-dev]

Issue #20311 has been updated by byroot (Jean Boussier).


> Is there any chance for the same module to be added twice?

Yes, aliasing isn't uncommon.

> Seems fine, but I'm afraid what can happen when an extension library stores rb_path2class("Struct::A") result then remove that constant?
Probably this is same for any Ruby-level defined classes/modules.

Yes, my reasoning is that it's not reasonable to expect classes created in Ruby to be immortal and immovable like the ones created in C.

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107090

* Author: MaxLap (Maxime Lapointe)
* Status: Open
* ruby -v: 3.3.0
* Backport: 3.0: WONTFIX, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED
----------------------------------------
The following code gives the impression of a memory leak.

```ruby
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52081] [Ruby master Bug#20311] Struct.new("A") memory leak?

[naruse (Yui NARUSE) via ruby-dev]

Issue #20311 has been updated by naruse (Yui NARUSE).

Backport changed from 3.0: WONTFIX, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED to 3.0: WONTFIX, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: DONE

ruby_3_3 f79b1d1ef1f7aa64d20f0eadbb3b0f8f7084deb3 merged revision(s) e626da82eae3d437b84d4f9ead0164d436b08e1a,f3af5ae7e6c1c096bbfe46d69de825a02b1696cf.

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-107404

* Author: MaxLap (Maxime Lapointe)
* Status: Closed
* ruby -v: 3.3.0
* Backport: 3.0: WONTFIX, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: DONE
----------------------------------------
The following code gives the impression of a memory leak.

```ruby
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/

[ruby-dev:52092] [Ruby master Bug#20311] Struct.new("A") memory leak?

[nagachika (Tomoyuki Chikanaga) via ruby-dev]

Issue #20311 has been updated by nagachika (Tomoyuki Chikanaga).

Backport changed from 3.0: WONTFIX, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: DONE to 3.0: WONTFIX, 3.1: REQUIRED, 3.2: DONE, 3.3: DONE

ruby_3_2 commit:bd5df1693c89d389471d145fc19b487c708912b1 merged revision(s) commit:e626da82eae3d437b84d4f9ead0164d436b08e1a, commit:f3af5ae7e6c1c096bbfe46d69de825a02b1696cf.

----------------------------------------
Bug #20311: Struct.new("A") memory leak?
https://bugs.ruby-lang.org/issues/20311#change-108988

* Author: MaxLap (Maxime Lapointe)
* Status: Closed
* ruby -v: 3.3.0
* Backport: 3.0: WONTFIX, 3.1: REQUIRED, 3.2: DONE, 3.3: DONE
----------------------------------------
The following code gives the impression of a memory leak.

```ruby
10.times do
  5000.times do
    Struct.new("A")
    Struct.send(:remove_const, :A)
  end

  GC.start
  puts `ps -o rss= -p #{$$}`.to_i
end
```

```
27868
35324
43400
51472
58676
66144
73764
81196
88512
95752
```

Is there another location where the struct gets set that I need to clear up for the GC free the memory?

Happens in 3.2.2, 3.2.3, 3.3.0, 3.3-head, ruby-head.



-- 
https://bugs.ruby-lang.org/
_______________________________________________
ruby-dev mailing list -- ruby-dev@ml.ruby-lang.org
To unsubscribe send an email to ruby-dev-leave@ml.ruby-lang.org