Problem with rfc4180splitter in last beta?

Problem with rfc4180splitter in last beta?

[Jaroslav Hajtmar]

Hi Hans,
I encountered while experimenting with the latest beta version of 
standalone context to a problem when using the splitter. When I process 
without header file then compilation crashed when you compile the latest 
beta version. Conversely, the processing of a header file is no problem. 
ConTEXt while compiling the 2014 TL is fine in both cases. I do not know 
from which beta version was a problem, I discovered it by accident until 
now. Send minimal example. When gCSVHeader = true everything is OK when 
it is gCSVHeader = false and compilation fails.


Thanx for any advice
Jaroslav Hajtmar

Here is my minimal version of problematic example:



% csvfile.txt content:
% first,second,third,fourth
% 1,"2","3","4"
% "a","b","c","d"
% "foo","bar""baz","boogie","xyzzy"
%



\starttext

\startluacode
      --gCSVHeader=true -- works fine when compiled any version of ConTeXt
      gCSVHeader=false -- crashed, when compiled by last beta version of 
standalone
      gColNames={}
      gColumnNames={}
      local inpcsvfile='csvfile.txt'
      local currentlyprocessedcsvfile = io.loaddata(inpcsvfile)
      local mycsvsplitter = utilities.parsers.rfc4180splitter{
         separator = ',',
         quote = '"',
         }
     if gCSVHeader then
      gTableRows, gColumnNames = 
mycsvsplitter(currentlyprocessedcsvfile,true)
      inspect(gTableRows)
      inspect(gColumnNames)
          for i=1,#gTableRows[1] do
             gColNames[tostring(gColumnNames[i])] = i -- for indexing use
         end
     else
       gTableRows = mycsvsplitter(currentlyprocessedcsvfile)
       inspect(gTableRows)
       -- ad now set column names for withoutheader situation:
         for i=1,#gTableRows[1] do
         context(gTableRows[i])
          gColumnNames[i]='c'..(i)
         end
     end
      gNumRows=#gTableRows -- Getting number of rows
       gNumCols=#gTableRows[1] -- Getting number of columns

      context.setgvalue("numrows",tostring(gNumRows))
      context.setgvalue("numcols",tostring(gNumCols))
      context.setgvalue("filename",tostring(inpcsvfile))
\stopluacode


File \filename\ has \numrows\ lines and \numcols\ columns.

\stoptext
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Problem with rfc4180splitter in last beta?

[Jaroslav Hajtmar]

After I read my e-mail the I cut more my source code and I am sending 
now more minimalized version of minimal example

Sorry, Jaroslav Hajtmar


Here is another minimal version:

% csvfile.txt content:
% first,second,third,fourth
% 1,"2","3","4"
% "a","b","c","d"
% "foo","bar""baz","boogie","xyzzy"
%



\starttext

\startluacode
      gCSVHeader=true -- works fine when compiled any version of ConTeXt
      --gCSVHeader=false -- crashed, when compiled by last version of 
standalone
      gColNames={}
      gColumnNames={}
      local inpcsvfile='csvfile.txt'
      local currentlyprocessedcsvfile = io.loaddata(inpcsvfile)
      local mycsvsplitter = utilities.parsers.rfc4180splitter{
         separator = ',',
         quote = '"',
         }
     if gCSVHeader then
      gTableRows, gColumnNames = 
mycsvsplitter(currentlyprocessedcsvfile,true)
      inspect(gTableRows)
      inspect(gColumnNames)
     else
       gTableRows = mycsvsplitter(currentlyprocessedcsvfile)
       inspect(gTableRows)
     end
      gNumRows=#gTableRows -- Getting number of rows
       gNumCols=#gTableRows[1] -- Getting number of columns

      context.setgvalue("numrows",tostring(gNumRows))
      context.setgvalue("numcols",tostring(gNumCols))
      context.setgvalue("filename",tostring(inpcsvfile))
\stopluacode


File \filename\ has \numrows\ lines and \numcols\ columns.

\stoptext




Dne 15. 6. 2015 v 10:45 Jaroslav Hajtmar napsal(a):
> Hi Hans,
> I encountered while experimenting with the latest beta version of 
> standalone context to a problem when using the splitter. When I 
> process without header file then compilation crashed when you compile 
> the latest beta version. Conversely, the processing of a header file 
> is no problem. ConTEXt while compiling the 2014 TL is fine in both 
> cases. I do not know from which beta version was a problem, I 
> discovered it by accident until now. Send minimal example. When 
> gCSVHeader = true everything is OK when it is gCSVHeader = false and 
> compilation fails.
>
>
> Thanx for any advice
> Jaroslav Hajtmar
>
> Here is my minimal version of problematic example:
>
>
>
> % csvfile.txt content:
> % first,second,third,fourth
> % 1,"2","3","4"
> % "a","b","c","d"
> % "foo","bar""baz","boogie","xyzzy"
> %
>
>
>
> \starttext
>
> \startluacode
>      --gCSVHeader=true -- works fine when compiled any version of ConTeXt
>      gCSVHeader=false -- crashed, when compiled by last beta version 
> of standalone
>      gColNames={}
>      gColumnNames={}
>      local inpcsvfile='csvfile.txt'
>      local currentlyprocessedcsvfile = io.loaddata(inpcsvfile)
>      local mycsvsplitter = utilities.parsers.rfc4180splitter{
>         separator = ',',
>         quote = '"',
>         }
>     if gCSVHeader then
>      gTableRows, gColumnNames = 
> mycsvsplitter(currentlyprocessedcsvfile,true)
>      inspect(gTableRows)
>      inspect(gColumnNames)
>          for i=1,#gTableRows[1] do
>             gColNames[tostring(gColumnNames[i])] = i -- for indexing use
>         end
>     else
>       gTableRows = mycsvsplitter(currentlyprocessedcsvfile)
>       inspect(gTableRows)
>       -- ad now set column names for withoutheader situation:
>         for i=1,#gTableRows[1] do
>         context(gTableRows[i])
>          gColumnNames[i]='c'..(i)
>         end
>     end
>      gNumRows=#gTableRows -- Getting number of rows
>       gNumCols=#gTableRows[1] -- Getting number of columns
>
>      context.setgvalue("numrows",tostring(gNumRows))
>      context.setgvalue("numcols",tostring(gNumCols))
>      context.setgvalue("filename",tostring(inpcsvfile))
> \stopluacode
>
>
> File \filename\ has \numrows\ lines and \numcols\ columns.
>
> \stoptext
> ___________________________________________________________________________________ 
>
> If your question is of interest to others as well, please add an entry 
> to the Wiki!
>
> maillist : ntg-context@ntg.nl / 
> http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : http://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Problem with rfc4180splitter in last beta?

[Jaroslav Hajtmar]

Hi Hans.
Once again. It occurred to me to send a minimum Hans's example. When I 
compile with new beta version are the results the same for header and 
noheader versions, while a compilation of TL 2014 Contex proceeds 
differently.


Jaroslav Hajtmar

\starttext

% csvfile.txt content:
% first,second,third,fourth
% 1,"2","3","4"
% "a","b","c","d"
% "foo","bar""baz","boogie","xyzzy"
%


\startluacode

local mycsvsplitter = utilities.parsers.rfc4180splitter()
local crap = io.loaddata("csvfile.txt")

local list, names = mycsvsplitter(crap,true)   inspect(list) inspect(names)
-- local list, names = mycsvsplitter(crap)        inspect(list) 
inspect(names)

context.starttabulate { "|l|r|" }
for i=1,#list do
     local l = list[i]
     context.NC() context(l[1])
     context.NC() context(l[3])
     context.NC()
     context.NR()
end
context.stoptabulate()

\stopluacode

\stoptext







Dne 15. 6. 2015 v 11:00 Jaroslav Hajtmar napsal(a):
> After I read my e-mail the I cut more my source code and I am sending 
> now more minimalized version of minimal example
>
> Sorry, Jaroslav Hajtmar
>
>
> Here is another minimal version:
>
> % csvfile.txt content:
> % first,second,third,fourth
> % 1,"2","3","4"
> % "a","b","c","d"
> % "foo","bar""baz","boogie","xyzzy"
> %
>
>
>
> \starttext
>
> \startluacode
>      gCSVHeader=true -- works fine when compiled any version of ConTeXt
>      --gCSVHeader=false -- crashed, when compiled by last version of 
> standalone
>      gColNames={}
>      gColumnNames={}
>      local inpcsvfile='csvfile.txt'
>      local currentlyprocessedcsvfile = io.loaddata(inpcsvfile)
>      local mycsvsplitter = utilities.parsers.rfc4180splitter{
>         separator = ',',
>         quote = '"',
>         }
>     if gCSVHeader then
>      gTableRows, gColumnNames = 
> mycsvsplitter(currentlyprocessedcsvfile,true)
>      inspect(gTableRows)
>      inspect(gColumnNames)
>     else
>       gTableRows = mycsvsplitter(currentlyprocessedcsvfile)
>       inspect(gTableRows)
>     end
>      gNumRows=#gTableRows -- Getting number of rows
>       gNumCols=#gTableRows[1] -- Getting number of columns
>
>      context.setgvalue("numrows",tostring(gNumRows))
>      context.setgvalue("numcols",tostring(gNumCols))
>      context.setgvalue("filename",tostring(inpcsvfile))
> \stopluacode
>
>
> File \filename\ has \numrows\ lines and \numcols\ columns.
>
> \stoptext
>
>
>
>
> Dne 15. 6. 2015 v 10:45 Jaroslav Hajtmar napsal(a):
>> Hi Hans,
>> I encountered while experimenting with the latest beta version of 
>> standalone context to a problem when using the splitter. When I 
>> process without header file then compilation crashed when you compile 
>> the latest beta version. Conversely, the processing of a header file 
>> is no problem. ConTEXt while compiling the 2014 TL is fine in both 
>> cases. I do not know from which beta version was a problem, I 
>> discovered it by accident until now. Send minimal example. When 
>> gCSVHeader = true everything is OK when it is gCSVHeader = false and 
>> compilation fails.
>>
>>
>> Thanx for any advice
>> Jaroslav Hajtmar
>>
>> Here is my minimal version of problematic example:
>>
>>
>>
>> % csvfile.txt content:
>> % first,second,third,fourth
>> % 1,"2","3","4"
>> % "a","b","c","d"
>> % "foo","bar""baz","boogie","xyzzy"
>> %
>>
>>
>>
>> \starttext
>>
>> \startluacode
>>      --gCSVHeader=true -- works fine when compiled any version of 
>> ConTeXt
>>      gCSVHeader=false -- crashed, when compiled by last beta version 
>> of standalone
>>      gColNames={}
>>      gColumnNames={}
>>      local inpcsvfile='csvfile.txt'
>>      local currentlyprocessedcsvfile = io.loaddata(inpcsvfile)
>>      local mycsvsplitter = utilities.parsers.rfc4180splitter{
>>         separator = ',',
>>         quote = '"',
>>         }
>>     if gCSVHeader then
>>      gTableRows, gColumnNames = 
>> mycsvsplitter(currentlyprocessedcsvfile,true)
>>      inspect(gTableRows)
>>      inspect(gColumnNames)
>>          for i=1,#gTableRows[1] do
>>             gColNames[tostring(gColumnNames[i])] = i -- for indexing use
>>         end
>>     else
>>       gTableRows = mycsvsplitter(currentlyprocessedcsvfile)
>>       inspect(gTableRows)
>>       -- ad now set column names for withoutheader situation:
>>         for i=1,#gTableRows[1] do
>>         context(gTableRows[i])
>>          gColumnNames[i]='c'..(i)
>>         end
>>     end
>>      gNumRows=#gTableRows -- Getting number of rows
>>       gNumCols=#gTableRows[1] -- Getting number of columns
>>
>>      context.setgvalue("numrows",tostring(gNumRows))
>>      context.setgvalue("numcols",tostring(gNumCols))
>>      context.setgvalue("filename",tostring(inpcsvfile))
>> \stopluacode
>>
>>
>> File \filename\ has \numrows\ lines and \numcols\ columns.
>>
>> \stoptext
>> ___________________________________________________________________________________ 
>>
>> If your question is of interest to others as well, please add an 
>> entry to the Wiki!
>>
>> maillist : ntg-context@ntg.nl / 
>> http://www.ntg.nl/mailman/listinfo/ntg-context
>> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
>> archive  : http://foundry.supelec.fr/projects/contextrev/
>> wiki     : http://contextgarden.net
>> ___________________________________________________________________________________ 
>>
>
> ___________________________________________________________________________________ 
>
> If your question is of interest to others as well, please add an entry 
> to the Wiki!
>
> maillist : ntg-context@ntg.nl / 
> http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : http://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Problem with rfc4180splitter in last beta (sticking point was found... maybe)

[Jaroslav Hajtmar]

Hello Hans.
I tried to find differences in beta compared to a fully functional 
version of the TL 2014 and I found two changes in the file-utils prs.lua 
compared to a fully functional version of utils-prs.lua in the TL 2014 
version of ConTeXt:

Changes in file utils-prs.lua (beta version) compared to TL 2014 (fully 
functional version):

line no 520 of beta standalone:
specification     = specification and 
setmetatableindex(specification,defaultspecification) or 
defaultspecification

while the TL 2014 version line no 478:
specification     = specification and 
table.setmetatableindex(specification,defaultspecification) or 
defaultspecification

and next diference:

line no 533 of beta standalone:
local wholeblob   = Ct((newline^(specification.strict and -1 or 1) * 
record)^0)

while the TL 2014 version line no 491:
local wholeblob   = Ct((newline^-1 * record)^0)


When I  change these lines and regenerate ConTeXt format then suddenly 
it works as expected. Of course, you will probably need to make some 
adjustments depending on your idea of a line 533rd.

I hope this information helps a little in search of a problem.


Jaroslav Hajtmar





Dne 15. 6. 2015 v 10:45 Jaroslav Hajtmar napsal(a):
> Hi Hans,
> I encountered while experimenting with the latest beta version of 
> standalone context to a problem when using the splitter. When I 
> process without header file then compilation crashed when you compile 
> the latest beta version. Conversely, the processing of a header file 
> is no problem. ConTEXt while compiling the 2014 TL is fine in both 
> cases. I do not know from which beta version was a problem, I 
> discovered it by accident until now. Send minimal example. When 
> gCSVHeader = true everything is OK when it is gCSVHeader = false and 
> compilation fails.
>
>
> Thanx for any advice
> Jaroslav Hajtmar
>
> Here is my minimal version of problematic example:
>
>
>
> % csvfile.txt content:
> % first,second,third,fourth
> % 1,"2","3","4"
> % "a","b","c","d"
> % "foo","bar""baz","boogie","xyzzy"
> %
>
>
>
> \starttext
>
> \startluacode
>      --gCSVHeader=true -- works fine when compiled any version of ConTeXt
>      gCSVHeader=false -- crashed, when compiled by last beta version 
> of standalone
>      gColNames={}
>      gColumnNames={}
>      local inpcsvfile='csvfile.txt'
>      local currentlyprocessedcsvfile = io.loaddata(inpcsvfile)
>      local mycsvsplitter = utilities.parsers.rfc4180splitter{
>         separator = ',',
>         quote = '"',
>         }
>     if gCSVHeader then
>      gTableRows, gColumnNames = 
> mycsvsplitter(currentlyprocessedcsvfile,true)
>      inspect(gTableRows)
>      inspect(gColumnNames)
>          for i=1,#gTableRows[1] do
>             gColNames[tostring(gColumnNames[i])] = i -- for indexing use
>         end
>     else
>       gTableRows = mycsvsplitter(currentlyprocessedcsvfile)
>       inspect(gTableRows)
>       -- ad now set column names for withoutheader situation:
>         for i=1,#gTableRows[1] do
>         context(gTableRows[i])
>          gColumnNames[i]='c'..(i)
>         end
>     end
>      gNumRows=#gTableRows -- Getting number of rows
>       gNumCols=#gTableRows[1] -- Getting number of columns
>
>      context.setgvalue("numrows",tostring(gNumRows))
>      context.setgvalue("numcols",tostring(gNumCols))
>      context.setgvalue("filename",tostring(inpcsvfile))
> \stopluacode
>
>
> File \filename\ has \numrows\ lines and \numcols\ columns.
>
> \stoptext
> ___________________________________________________________________________________ 
>
> If your question is of interest to others as well, please add an entry 
> to the Wiki!
>
> maillist : ntg-context@ntg.nl / 
> http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : http://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Problem with rfc4180splitter in last beta (sticking point was found... maybe)

[Hans Hagen]

On 6/15/2015 8:22 PM, Jaroslav Hajtmar wrote:
> Hello Hans.
> I tried to find differences in beta compared to a fully functional
> version of the TL 2014 and I found two changes in the file-utils prs.lua
> compared to a fully functional version of utils-prs.lua in the TL 2014
> version of ConTeXt:
>
> Changes in file utils-prs.lua (beta version) compared to TL 2014 (fully
> functional version):
>
> line no 520 of beta standalone:
> specification     = specification and
> setmetatableindex(specification,defaultspecification) or
> defaultspecification
>
> while the TL 2014 version line no 478:
> specification     = specification and
> table.setmetatableindex(specification,defaultspecification) or
> defaultspecification

that ons is irrelevant

> and next diference:
>
> line no 533 of beta standalone:
> local wholeblob   = Ct((newline^(specification.strict and -1 or 1) *
> record)^0)
>
> while the TL 2014 version line no 491:
> local wholeblob   = Ct((newline^-1 * record)^0)
>
>
> When I  change these lines and regenerate ConTeXt format then suddenly
> it works as expected. Of course, you will probably need to make some
> adjustments depending on your idea of a line 533rd.
>
> I hope this information helps a little in search of a problem.

it was on user request that it was changed

the two lines look nearly the same, apart from:

specification.strict and -1 or 1

so, if you pass a specification with strict = true then you get the 
behaviour you want

Hans



-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Problem with rfc4180splitter in last beta (sticking point was found... maybe)

[Jaroslav Hajtmar]

Thanx Hans very much. It works fine now.

Jaroslav Hajtmar




Dne 15. 6. 2015 v 21:40 Hans Hagen napsal(a):
> On 6/15/2015 8:22 PM, Jaroslav Hajtmar wrote:
>> Hello Hans.
>> I tried to find differences in beta compared to a fully functional
>> version of the TL 2014 and I found two changes in the file-utils prs.lua
>> compared to a fully functional version of utils-prs.lua in the TL 2014
>> version of ConTeXt:
>>
>> Changes in file utils-prs.lua (beta version) compared to TL 2014 (fully
>> functional version):
>>
>> line no 520 of beta standalone:
>> specification     = specification and
>> setmetatableindex(specification,defaultspecification) or
>> defaultspecification
>>
>> while the TL 2014 version line no 478:
>> specification     = specification and
>> table.setmetatableindex(specification,defaultspecification) or
>> defaultspecification
>
> that ons is irrelevant
>
>> and next diference:
>>
>> line no 533 of beta standalone:
>> local wholeblob   = Ct((newline^(specification.strict and -1 or 1) *
>> record)^0)
>>
>> while the TL 2014 version line no 491:
>> local wholeblob   = Ct((newline^-1 * record)^0)
>>
>>
>> When I  change these lines and regenerate ConTeXt format then suddenly
>> it works as expected. Of course, you will probably need to make some
>> adjustments depending on your idea of a line 533rd.
>>
>> I hope this information helps a little in search of a problem.
>
> it was on user request that it was changed
>
> the two lines look nearly the same, apart from:
>
> specification.strict and -1 or 1
>
> so, if you pass a specification with strict = true then you get the 
> behaviour you want
>
> Hans
>
>
>
> -----------------------------------------------------------------
>                                           Hans Hagen | PRAGMA ADE
>               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
>     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
>                                              | www.pragma-pod.nl
> -----------------------------------------------------------------
> ___________________________________________________________________________________ 
>
> If your question is of interest to others as well, please add an entry 
> to the Wiki!
>
> maillist : ntg-context@ntg.nl / 
> http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : http://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________