LuaTeX: Weird GC behaviour

LuaTeX: Weird GC behaviour

[Henri]

Dear devs,

This was asked on TeX.SX: https://tex.stackexchange.com/questions/404617
The example below runs perfectly fine in the normal Lua interpreter, but crashes in texlua because
the maximum number of open file handles is exceeded.  This seems to be a peculiarity of io.lines in
texlua.  Is there anything special to the implementation of it in LuaTeX?

Cheers, Henri



-- Create the file (once)
local filename = 'luatexfilelimit.txt'
local file = io.open(filename, "w")
file:write("hello world\n")
file:close()

-- Print its contents (many times)
for i = 1, 3000000 do
   print(string.format('Attempt %d', i))
   for line in io.lines(filename) do
      print(line)
   end
end

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: LuaTeX: Weird GC behaviour

[Hans Hagen]

On 12/5/2017 10:47 PM, Henri wrote:
> Dear devs,
> 
> This was asked on TeX.SX: https://tex.stackexchange.com/questions/404617
> The example below runs perfectly fine in the normal Lua interpreter, but crashes in texlua because
> the maximum number of open file handles is exceeded.  This seems to be a peculiarity of io.lines in
> texlua.  Is there anything special to the implementation of it in LuaTeX?
yes: it's wrapped in code that does some checking wrr to what is permitted

no: lua should collect the open handles but in your example there has 
not been a reason for collecting (hardly a memory demand)

i'll see if i can hack a solution

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: LuaTeX: Weird GC behaviour

[Hans Hagen]

On 12/7/2017 12:40 AM, Shreevatsa R wrote:
> (Hope this gets threaded with the previous two messages...)
> 
> I was the one who asked the question
> (https://tex.stackexchange.com/questions/404617). Another thing I'd
> like to mention is that after a while `io.lines` simply returned nil,
> and the second and third return values (error message and error code)
> that io.lines is supposed to return in case of error were also nil.
> Even if this issue is not perfectly fixed (i.e. even if io.lines
> cannot be made to close the file descriptor when it reaches the end of
> file), I think it would be preferable to throw a clear error when
> there's a failure to open a new file, instead of silently (and in a
> not 100% reproducible way) returning nil.
Ok, we can quit with an error but the message will be a bit different 
(as we have different code). The file gets now closed explicitly which 
doesn't guarantee collection but it looks ok so far. I never use 
io.lines so I can't really test in practice.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: LuaTeX: Weird GC behaviour

[Shreevatsa R]

(Hope this gets threaded with the previous two messages...)

I was the one who asked the question
(https://tex.stackexchange.com/questions/404617). Another thing I'd
like to mention is that after a while `io.lines` simply returned nil,
and the second and third return values (error message and error code)
that io.lines is supposed to return in case of error were also nil.
Even if this issue is not perfectly fixed (i.e. even if io.lines
cannot be made to close the file descriptor when it reaches the end of
file), I think it would be preferable to throw a clear error when
there's a failure to open a new file, instead of silently (and in a
not 100% reproducible way) returning nil.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________