* ConTeXt Minimals parsing of .tui broken with ruby 1.9.1 in Windows
@ 2009-07-13 17:38 Jose Augusto
2009-07-14 8:02 ` Hans Hagen
0 siblings, 1 reply; 4+ messages in thread
From: Jose Augusto @ 2009-07-13 17:38 UTC (permalink / raw)
To: ntg-context
[-- Attachment #1.1: Type: text/plain, Size: 5397 bytes --]
Hello all,
I want to report a problem that is either in ConTeXt, or in ruby 1.9.1
(last version of ruby). More probably, the problem has to do with ruby
handling non-ASCII characters. I have no means of trying Linux, Solaris,
etc...
Anyone using ConTeXt with ruby 1.9.1 will face it probably (at least in
Windows :-)
The problem happens with all files, even with the simple "Hello":
\starttext
Hello World
\stoptext
After installing ConTeXt Minimals (the devel version) yesterday,
I ran the above example with ruby 1.9.1-p129 in Windows
(both Win 2000 and XP show the problem).
Meanwhile I compiled and tried several versions of Ruby, and found the
following pattern of problems:
ruby 1.9.1p129 (2009-05-12 revision 23412) [i386-mingw32] PROBLEM
ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-mingw32] PROBLEM
ruby 1.9.0 (2008-10-04 revision 19669) [i386-mingw32] No problem
ruby 1.8.7 (2009-06-12 patchlevel 174) [i386-mingw32] No problem
ruby 1.8.6 (2009-06-08 patchlevel 369) [i386-mingw32] No problem
So, whatever it is, it is broken with ruby 1.9.1.
All the versions of ruby were compiled in Windows using the mingw toolchain,
with GCC 3.4.5.
>>>> Here is the description of what happens <<<<
When texutil parses the .tui file, I get the following (see comments after
this text output):
---------------------------------------------
...........................
Output written on con-hello1.pdf (1 page, 21759 bytes).
Transcript written on con-hello1.log.
TeXUtil | parsing file con-hello1.tui
TeXUtil | debug 1 jasa #<File:0x1271d18>
xxx c \thisissectionseparator{-}
xxx c \thisisutilityversion{2008.10.14}
xxx c \thisisbytesequence{?+Ç}
TeXUtil | fatal error in parsing con-hello1.tui
TeXUtil | shortcuts : 0
TeXUtil | expansions: 0
TeXUtil | reductions: 0
TeXUtil | divisions : 0
TeXUtil | loaded files: 0
TeXUtil | temporary files: 0
TeXUtil | commands: 2
TeXUtil | programs: 0
TeXUtil | tuo file saved
TeXExec | runtime: 2.703125
--------------------------------------------
The lines with "debug 1 jasa" and starting with "xxx" result from
the simple "debug code" I inserted in the file texutil.rb to find the
problematic line. The error happens when the following ruby code
is executed: (the extra debug lines have a mark # jasa )
---------------------- texutil.rb (snippet, around line 1025)
----------------------
def loaded(filename)
begin
tuifile = File.suffixed(filename,'tui')
if FileTest.file?(tuifile) then
report("parsing file #{tuifile}")
if f = open(tuifile) then
report("debug 1 jasa #{f}") # jasa
f.each do |line|
print "xxx #{line}" # jasa
case line.chomp
when /^f (.*)$/o then
@plugins.reader('MyFiles', $1.splitdata)
when /^c (.*)$/o then
@plugins.reader('MyCommands', [$1])
when /^e (.*)$/o then
@plugins.reader('MyExtras', $1.splitdata)
when /^s (.*)$/o then
@plugins.reader('MySynonyms', $1.splitdata)
when /^r (.*)$/o then
@plugins.reader('MyRegisters',$1.splitdata)
when /^p (.*)$/o then
@plugins.reader('MyPlugins', $1.splitdata)
when /^x (.*)$/o then
@plugins.reader('MyKeys', $1.splitdata)
when /^r (.*)$/o then # nothing, not handled
here
else
# report("unknown entry #{line[0,1]} in line
#{line.chomp}")
end
end
f.close
end
else
report("unable to locate #{tuifile}")
end
rescue
report("fatal error in parsing #{tuifile}")
@filename = 'texutil'
else
@filename = filename
end
end
---------------------------------------------------------------
>From the debugging lines that are expelled, it is clear that the line in the
.tui file that triggers the problem is:
c \thisisbytesequence{ ...non-ASCII codes... }
and, precisely, it s the second line of the 'case':
when /^c (.*)$/o then @plugins.reader('MyCommands', [$1])
which processes the .tui line and triggers the 'rescue' clause. So I think
the problem lies in the "digestion"
of non-ASCII characters by the last version of Ruby.
I don't know what is the meaning of the \thisisbytesequence line in ConTeXt
and the maening of those non-ASCII chars. I followed the
@plugins.reader('MyCommands', [$1])
and figured out that what raises the exception happens before the
@plugins.reader method,
since it is never reached when the "\thisisbytesequence" line is processed.
To finish, I think this can be of interest for Ruby guys, but is also
breaking ConTeXt for anyone with ruby 1.9.1 installed in Windows.
Thanks,
Jose Soares Augusto
jasaugusto@gmail.com
jasa@fisica.fc.ul.pt
[-- Attachment #1.2: Type: text/html, Size: 6035 bytes --]
[-- Attachment #2: Type: text/plain, Size: 487 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ConTeXt Minimals parsing of .tui broken with ruby 1.9.1 in Windows
2009-07-13 17:38 ConTeXt Minimals parsing of .tui broken with ruby 1.9.1 in Windows Jose Augusto
@ 2009-07-14 8:02 ` Hans Hagen
2009-07-14 12:56 ` Mojca Miklavec
0 siblings, 1 reply; 4+ messages in thread
From: Hans Hagen @ 2009-07-14 8:02 UTC (permalink / raw)
To: mailing list for ConTeXt users
Jose Augusto wrote:
> Hello all,
>
> I want to report a problem that is either in ConTeXt, or in ruby 1.9.1
> (last version of ruby). More probably, the problem has to do with ruby
> handling non-ASCII characters. I have no means of trying Linux, Solaris,
> etc...
> Anyone using ConTeXt with ruby 1.9.1 will face it probably (at least in
> Windows :-)
>
> The problem happens with all files, even with the simple "Hello":
>
> \starttext
> Hello World
> \stoptext
>
> After installing ConTeXt Minimals (the devel version) yesterday,
> I ran the above example with ruby 1.9.1-p129 in Windows
> (both Win 2000 and XP show the problem).
(maybe mojca can patch this in core-uti.mkii: ):
% \appendtoks
% \immediatewriteutilitycommand{\thisisbytesequence{\testbytesequence}}%
% \to \everyopenutilities
\let\testbytesequence \empty % keep this
\let\thisisbytesequence\gobbleoneargument % keep this
The reason for this test is that in the past there were engines around
that were 8 bit but configured to be 7 bit. Especially tetex was a
problem but as that is now obsolete we can remove this test.
> Meanwhile I compiled and tried several versions of Ruby, and found the
> following pattern of problems:
>
> ruby 1.9.1p129 (2009-05-12 revision 23412) [i386-mingw32] PROBLEM
> ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-mingw32] PROBLEM
> ruby 1.9.0 (2008-10-04 revision 19669) [i386-mingw32] No problem
> ruby 1.8.7 (2009-06-12 patchlevel 174) [i386-mingw32] No problem
> ruby 1.8.6 (2009-06-08 patchlevel 369) [i386-mingw32] No problem
>
> So, whatever it is, it is broken with ruby 1.9.1.
> All the versions of ruby were compiled in Windows using the mingw toolchain,
> with GCC 3.4.5.
actually, ruby 1.9+ broke the scripts anyway, as some file related
functionality was no longer available (and moved to modules) so i
already adapted all the ruby scripts
>>From the debugging lines that are expelled, it is clear that the line in the
> ..tui file that triggers the problem is:
>
> c \thisisbytesequence{ ...non-ASCII codes... }
>
> and, precisely, it s the second line of the 'case':
>
> when /^c (.*)$/o then @plugins.reader('MyCommands', [$1])
>
> which processes the .tui line and triggers the 'rescue' clause. So I think
> the problem lies in the "digestion"
> of non-ASCII characters by the last version of Ruby.
looks that way ... quite disturbing if they changed the default; maybe
they move to utf8 but then i'd expect that to happen in 2+ versions
thanks for looking into it
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
| www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ConTeXt Minimals parsing of .tui broken with ruby 1.9.1 in Windows
2009-07-14 8:02 ` Hans Hagen
@ 2009-07-14 12:56 ` Mojca Miklavec
2009-07-15 23:48 ` Jose Augusto
0 siblings, 1 reply; 4+ messages in thread
From: Mojca Miklavec @ 2009-07-14 12:56 UTC (permalink / raw)
To: mailing list for ConTeXt users
>> After installing ConTeXt Minimals (the devel version) yesterday,
>> I ran the above example with ruby 1.9.1-p129 in Windows
>> (both Win 2000 and XP show the problem).
>
> (maybe mojca can patch this in core-uti.mkii: ):
>
>
> % \appendtoks
> % \immediatewriteutilitycommand{\thisisbytesequence{\testbytesequence}}%
> % \to \everyopenutilities
>
> \let\testbytesequence \empty % keep this
> \let\thisisbytesequence\gobbleoneargument % keep this
Done, but untested.
Mojca
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ConTeXt Minimals parsing of .tui broken with ruby 1.9.1 in Windows
2009-07-14 12:56 ` Mojca Miklavec
@ 2009-07-15 23:48 ` Jose Augusto
0 siblings, 0 replies; 4+ messages in thread
From: Jose Augusto @ 2009-07-15 23:48 UTC (permalink / raw)
To: mailing list for ConTeXt users
[-- Attachment #1.1: Type: text/plain, Size: 2138 bytes --]
Hi all,
Thanks for the patch. I just updated ConTeXt Minimals and re-tried.
Here is the GOOD result, now its working:
-----------------------------------------
F:\ANOS\TeXes>ruby -v
ruby 1.9.1p129 (2009-05-12 revision 23412) [i386-mingw32]
F:\ANOS\TeXes>texexec con-hello1.tex
TeXExec | processing document 'con-hello1.tex'
....................................................................
Output written on con-hello1.pdf (1 page, 21759 bytes).
Transcript written on con-hello1.log.
TeXUtil | parsing file con-hello1.tui
TeXUtil | shortcuts : 169
TeXUtil | expansions: 308
TeXUtil | reductions: 0
TeXUtil | divisions : 0
TeXUtil | loaded files: 1
TeXUtil | temporary files: 0
TeXUtil | commands: 20
TeXUtil | programs: 0
TeXUtil | tuo file saved
TeXExec | runtime: 4.578125
-----------------------------------------------------------------------------
Meanwhile, the "evil line" with "ctrl chars" is not anymore in the .tui
file.
I want to thank Hans and Mojca for the patching and the kindness.
Jose.
On Tue, Jul 14, 2009 at 1:56 PM, Mojca Miklavec <
mojca.miklavec.lists@gmail.com> wrote:
> >> After installing ConTeXt Minimals (the devel version) yesterday,
> >> I ran the above example with ruby 1.9.1-p129 in Windows
> >> (both Win 2000 and XP show the problem).
> >
> > (maybe mojca can patch this in core-uti.mkii: ):
> >
> >
> > % \appendtoks
> > %
> \immediatewriteutilitycommand{\thisisbytesequence{\testbytesequence}}%
> > % \to \everyopenutilities
> >
> > \let\testbytesequence \empty % keep this
> > \let\thisisbytesequence\gobbleoneargument % keep this
>
> Done, but untested.
>
> Mojca
>
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to
> the Wiki!
>
> maillist : ntg-context@ntg.nl /
> http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive : https://foundry.supelec.fr/projects/contextrev/
> wiki : http://contextgarden.net
>
> ___________________________________________________________________________________
>
[-- Attachment #1.2: Type: text/html, Size: 3113 bytes --]
[-- Attachment #2: Type: text/plain, Size: 487 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2009-07-15 23:48 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-07-13 17:38 ConTeXt Minimals parsing of .tui broken with ruby 1.9.1 in Windows Jose Augusto
2009-07-14 8:02 ` Hans Hagen
2009-07-14 12:56 ` Mojca Miklavec
2009-07-15 23:48 ` Jose Augusto
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).