Fwd: lua-widow-control module error in LMTX

[Eduardo Bohoyo via ntg-context <ntg-context@ntg.nl>]

Hello, Max:

As I promised you this morning, I had time this evening to read more 
carefully all your remarks in the last mail. And now I can answer you 
with more basis.

In my particular case, it is not necessary to consider any of the seven 
possible problems you describe from highest to lowest probability. The 
question you asked me in the first paragraph about "interesting" 
features in my ConTeXt code is the key.

When I uncomment grid=yes in my \setuplayout, lwc makes its real 
appearance on the scene. I suppose that in other documents than mine, 
i.e. less complex, this performance would be a success. But in my file 
it is, considering my current aesthetic requirements, a failure. This is 
because for me grid=yes is a non-negotiable part of my code.

As I told you this morning when I sent you my pdf without the module 
activated, but with the layout my way, I understand that lwc can have 
problems with a text whose chapters have capital letters and small caps 
on the first page with less text than on the following pages; with long 
quotations that involve paragraphs of different layout and separated 
from the main text; with more than one footnote in a row, with a 
bibliography at the end with French indentation; with vertical spaces 
separating two lines within the same chapter when in the dummy text 
there is a supposed change of scene within the narrative; with etc. 
etc., etc., etc...

Anyway, after uncommenting grid=yes again, I will send you the log file, 
as you asked me. By the way, I do get some "Widow/Orphan NOT removed on 
page..." in spite of the rest of "successes" that move the final lines 
of my pages away from the result I would like, and that you have seen in 
the pdf I sent you this morning.

Do you want the new pdf with lwc actually acting on my file? Then you 
can get a better idea of what I mean by the end of some pages when I 
comment grid=yes (apart from the mischief that happens with the 
horizontal spaces in some paragraphs of the bibliographic section).

Or, if you prefer, and you need more feedback, I can send you my current 
code to use it as a test bench for " daring " texts.

Greetings,

edu


El 27/4/22 a las 9:14, Max Chernoff escribió:
> Quick question before I begin: are you using any especially "interesting"
> ConTeXt features? By "interesting" I mean things like grid typesetting,
> pagecolumns, bidirectional text, etc. I haven't tested lwc with every
> possible ConTeXt feature, so there may be some adverse interaction. If
> you are using something like this, try disabling it and see if that solves
> anything (then let me know so that I can fix it!)
>
> On 2022-04-26 3:45 a.m., Eduardo Bohoyo wrote:
>
>> I can see "modules         > 'lua-widow-control' is loaded".
>>
>> But, luckily, I can also see this:
>>
>> open source     > level 2, order 4, name 
>> '/opt/luametatex/texmf-modules/tex/context/third/lua-widow-control/t-lua-widow-control.mkxl' 
>>
>> resolvers       > lua > loading file 
>> '/opt/luametatex/texmf-modules/tex/luatex/lua-widow-control/lua-widow-control.lua' 
>> succeeded
>> close source    > level 2, order 4, name 
>> '/opt/luametatex/texmf-modules/tex/context/third/lua-widow-control/t-lua-widow-control.mkxl' 
>>
>> module          > lua-widow-control > Already enabled
>
> Ok, so this is good; lwc is for sure being loaded successfully.
>
>> No line such as "Widow/orphan detected. Attempting to delete".
>>
>> I see interleaved new groups with the same line always repeating a 
>> warning message throughout the whole file. In short, there are 613 
>> new lines with the message "luatex warning > tex: left parfill skip 
>> is gone".
>> But I didn't give it any importance, because I interpreted that they 
>> could be inherent to the module.
>
> Well this at least narrows the issue down quite a bit. Lwc runs in pretty
> much two stages: when a paragraph has finished being broken by TeX, lwc
> saves the paragraph. The second stage is ran just before each time the
> output routine is triggered so that lwc can remove the widows and orphans.
>
> Due to an lwc bug, the first stage results in the "left parfill skip"
> warning being printed twice for each paragraph. Normally this is quite
> annoying, but here it is good -- we know for sure that the first stage
> is running just fine.
>
> It is the second stage where you should get the "Widow/orphan detected"
> message, but this isn't happening. The code here is at lwc.lua:362-388.
>
> Here is a list of all possible reasons, in order of likelihood, why
> "Widow/orphan detected" wouldn't be printed when there is actually
> a widow or orphan:
>
> (Just listing all of these to make sure that *I* don't forget 
> anything. I'd
>  say that 1 and 2 are the only ones that are actually likely -- you can
>  probably ignore all of the others)
>
>  1. "\clubpenalty" and/or "\widowpenalty" are either zero or infinite
>
>     In these cases, lwc can't distinguish a widow or orphan from other
>     penalties, so it doesn't do anything. You can check for this with
>
>         \showthe\widowpenalty
>         \showthe\clubpenalty
>
>     some time after "\starttext". This will print the values to the log
>     file. The values should probably be "1", but things like "150" and
>     "5000" are fine too. If you see negative values, "0", or values
>     greater than ten thousand, then something is wrong here.
>
>  2. Something is "hiding" the widow/orphan penalties from lwc.
>
>     Something like "\vadjust{\penalty X}" or the e-TeX "\-penalties"
>     commands could potentially do this.
>
>     One potential solution for (a subset of) this problem would be to
>     modify lwc.mkxl to insert "\directsetup{*reset}" at line 52, to
>     get:
>
>         \startsetups[*default]
>             \directsetup{*reset}
>             \clubpenalty=\lwcparameter{orphanpenalty}
>             \widowpenalty=\lwcparameter{widowpenalty}
>             \displaywidowpenalty=\lwcparameter{widowpenalty}
>             \brokenpenalty=\lwcparameter{brokenpenalty}
>         \stopsetups
>
>     If that doesn't do anything, keep the above modifications but also 
> insert
>
>         \setups[*default]
>
>     after "\starttext" in your test document.
>
>  3. Lwc hasn't saved any paragraphs since the last output routine.
>
>     Since we know that the first stage is running, this shouldn't be
>     possible unless there is a weird bug somewhere else.
>
>  4. A weird engine/format bug.
>
>     The LMTX test file was successful on Debian x86_64 three days ago
>
> https://github.com/gucci-on-fleek/lua-widow-control/runs/6144354147?check_suite_focus=true
>
>     so something quite weird would have to be going on here.
>
>  5. Something is overwriting the "pre_output_filter" callback.
>
>     Now this is an interesting one. ConTeXt has frozen most of the core
>     engine callbacks, but "pre_output_filter" is unfrozen and unassigned.
>     There is a ConTeXt interface for most of the other callbacks, but here
>     I need to fallback to the low-level interface. With this interface,
>     you can only register a single callback, so it is possible that some
>     other package is overwriting the callback. To test this, the very
>     beginning of your document should look like:
>             \usemodule[lua-widow-control]
>         \startluacode
>             lwc.callbacks.disable_box_warnings.enable = function () end
>             lwc.callbacks.disable_box_warnings.disable = function () end
>             callback = nil
>         \stopluacode
>
>     If you get a really strange error from some other package/module (not
>     lwc), then this may be the issue. Fairly unlikely.
>
>  6. The values of "\club/widowpenalty" change between the paragraph
>     being typeset and the output routine being triggered.
>
>     I can't imagine this happening for *every* page in a document
>     except unless you are specifically trying to do this. Fairly
>     unlikely.
>
>  7. You have the exact Lua code
>             lwc.callbacks.remove_widows.disable()
>         written in your document. Exceedingly unlikely.
>
>> On the other hand, the distribution and size of my paragraphs take 
>> great care that their "design" optimises the module's goodness, 
>> except, of course, for the first pages of each of the nine dummy text 
>> chapters (they start at a third of a page).
>
> Well as long as you aren't seeing
>
>     Widow/Orphan NOT removed
>
> in your log file then your text should be fine.
>
>> I will go over the lua-widow-control.pdf document once more in case 
>> there is a tiny detail I am missing, but I think, if I keep going at 
>> this pace, I will end up learning it by heart.
>
> There's probably not too much else in there -- and the documentation
> is a bit of a mess right now anyways.
>
> ---
>
> If none of the above solutions solved anything, then what I'll need to
> debug further will be for you to insert
>
>     \setuplwc[debug=true]
>
> into your document right after you load lwc. Then, send me the full
> log file -- probably as a link since it will be quite large.
>
> Thanks.
>
> -- Max
>
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________