bug with numeral conversion function

bug with numeral conversion function

[Khaled Hosny]

[-- Attachment #1.1: Type: text/plain, Size: 462 bytes --]


It seems that there is a bug in converters.alphabetic function,
converters.alphabetic(0,"arabic") returns the western 0 (no matter what
is the selected language) while converters.alphabetic(1,"arabic") gives
the Arabic 0 not 1 and so one i.e. it looks like as if it starts
counting from zero.

This has been reported few weeks ago, but I think it got missed.

Regards,
 Khaled

-- 
 Khaled Hosny
 Arabic localizer and member of Arabeyes.org team

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bug with numeral conversion function

[Khaled Hosny]

[-- Attachment #1.1.1: Type: text/plain, Size: 1211 bytes --]

On Sat, Aug 23, 2008 at 12:46:47AM +0300, Khaled Hosny wrote:
> 
> It seems that there is a bug in converters.alphabetic function,
> converters.alphabetic(0,"arabic") returns the western 0 (no matter what
> is the selected language) while converters.alphabetic(1,"arabic") gives
> the Arabic 0 not 1 and so one i.e. it looks like as if it starts
> counting from zero.

I finally got myself to understand some lua code.

I think the problem is that lua tables start counting from 1 not 0,
getting the value of key 0 from code table will give nil while 1 will
give the value of 0 (first key), so the solution would be incrementing n
by 1 in:

local function do_alphabetic(n,max,chr)
    n = n + 1 -- to get the correct key
    if n > max then
        do_alphabetic(floor((n-1)/max),max,chr)
        n = (n-1)%max+1
    end
    characters.flush(chr(n))
end

This does work for the case of Arabic, but I don't know about others
(especially greek and slovenian which look different)

Now, I think I discovered another bug (or feature?), the function will
ignore any zeros at the left which isn't what one expects.

-- 
 Khaled Hosny
 Arabic localizer and member of Arabeyes.org team

[-- Attachment #1.1.2: core-con.lua.diff --]
[-- Type: text/x-diff, Size: 321 bytes --]

--- core-con.lua~	2008-09-01 19:24:44.000000000 +0200
+++ core-con.lua	2008-09-01 19:23:31.000000000 +0200
@@ -103,6 +103,7 @@
 end
 
 local function do_alphabetic(n,max,chr)
+    n = n + 1 -- to get the correct key
     if n > max then
         do_alphabetic(floor((n-1)/max),max,chr)
         n = (n-1)%max+1

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bug with numeral conversion function

[Khaled Hosny]

[-- Attachment #1.1.1: Type: text/plain, Size: 927 bytes --]

On Mon, Sep 01, 2008 at 07:26:43PM +0200, Khaled Hosny wrote:
> Now, I think I discovered another bug (or feature?), the function will
> ignore any zeros at the left which isn't what one expects.

This happen to be some thing in Lua itself:
s = 000123 print(s)
will give 123, so it have to be a string to keep the to the left zeros.

I rewrote the converters.alphabetic() and converters.Alphabetic() so
that it will not expect a number and will just iterate through the given
string, and now \arabicnumerals and its brothers will pass strings to
it.

Also I found that \abjadnumerals were referring to
converters.arabicnumerals witch doesn't exist, I changed it to 
converters.abjadnumerals but there is no converters.abjadnaivenumerals
and I've no idea what it is supposed to do.

See the attached patch and tell me what you think.


-- 
 Khaled Hosny
 Arabic localizer and member of Arabeyes.org team

[-- Attachment #1.1.2: numerals.diff --]
[-- Type: text/x-diff, Size: 5355 bytes --]

diff -Naur cont-tmf/tex/context/base/core-con.lua cont-tmf.local/tex/context/base/core-con.lua
--- cont-tmf/tex/context/base/core-con.lua	2008-06-24 23:02:50.000000000 +0300
+++ cont-tmf.local/tex/context/base/core-con.lua	2008-09-02 12:01:17.000000000 +0200
@@ -102,22 +102,22 @@
     texsprint(utfchar(n+m))
 end
 
-local function do_alphabetic(n,max,chr)
-    if n > max then
-        do_alphabetic(floor((n-1)/max),max,chr)
-        n = (n-1)%max+1
-    end
-    characters.flush(chr(n))
-end
-
 function converters.alphabetic(n,code)
     local code = counters[code] or counters['**']
-    do_alphabetic(n,#code,function(n) return code[n] or fallback end)
+    for c in string.characters(n) do
+        local c = c + 1
+	local chr = function(n) return code[n] or fallback end
+	characters.flush(chr(c))
+    end
 end
 
 function converters.Alphabetic(n,code)
     local code = counters[code] or counters['**']
-    do_alphabetic(n,#code,function(n) return characters.uccode(code[n] or fallback) end)
+    for c in string.characters(n) do
+        local c = c + 1
+	local chr = function(n) return characters.uccode(code[n] or fallback) end
+	characters.flush(chr(c))
+    end
 end
 
 function converters.character(n)  converters.chr (n,96) end
diff -Naur cont-tmf/tex/context/base/core-con.mkiv cont-tmf.local/tex/context/base/core-con.mkiv
--- cont-tmf/tex/context/base/core-con.mkiv	2008-06-24 22:55:44.000000000 +0300
+++ cont-tmf.local/tex/context/base/core-con.mkiv	2008-09-02 12:17:49.000000000 +0200
@@ -17,8 +17,8 @@
 
 \def\romannumerals       #1{\ctxlua{converters.romannumerals(\number#1)}}
 \def\Romannumerals       #1{\ctxlua{converters.Romannumerals(\number#1)}}
-\def\abjadnumerals      #1{\ctxlua{converters.arabicnumerals(\number#1)}}
-\def\abjadnodotnumerals #1{\ctxlua{converters.arabicnodotnumerals(\number#1)}}
+\def\abjadnumerals      #1{\ctxlua{converters.abjadnumerals(\number#1)}}
+\def\abjadnodotnumerals #1{\ctxlua{converters.abjadnodotnumerals(\number#1)}}
 \def\abjadnaivenumerals #1{\ctxlua{converters.arabicnaivenumerals(\number#1)}}
 
 \defineconversion [romannumerals]      [\romannumerals]
@@ -32,8 +32,8 @@
 \def\characters#1{\ctxlua{converters.characters(\number#1)}}
 \def\Characters#1{\ctxlua{converters.Characters(\number#1)}}
 
-\def\languagecharacters#1{\ctxlua{converters.alphabetic(\number#1,"\currentlanguage")}} % new
-\def\languageCharacters#1{\ctxlua{converters.Alphabetic(\number#1,"\currentlanguage")}} % new
+\def\languagecharacters#1{\ctxlua{converters.alphabetic("#1","\currentlanguage")}} % new
+\def\languageCharacters#1{\ctxlua{converters.Alphabetic("#1","\currentlanguage")}} % new
 
 \def\getdayoftheweek#1#2#3{\normalweekday\ctxlua{converters.weekday(\number#1,\number#2,\number#3)}}
 \def\dayoftheweek   #1#2#3{\doconvertday{\ctxlua{converters.weekday(\number#1,\number#2,\number#3)}}}
@@ -73,19 +73,19 @@
 
 % we could use an auxiliary macro to save some bytes in the format
 %
-% \def\dolanguagecharacters#1#2{\ctxlua{converters.alphabetic(\number#2,"#1")}}
+% \def\dolanguagecharacters#1#2{\ctxlua{converters.alphabetic("#2","#1")}}
 
 % this does not belong here, but in a lang-module
 
-\def\thainumerals      #1{\ctxlua{converters.alphabetic(\number#1,"thai")}}
-\def\devanagarinumerals#1{\ctxlua{converters.alphabetic(\number#1,"devanagari")}}
-\def\gurmurkhinumerals #1{\ctxlua{converters.alphabetic(\number#1,"gurmurkhi")}}
-\def\gujaratinumerals  #1{\ctxlua{converters.alphabetic(\number#1,"gujarati")}}
-\def\tibetannumerals   #1{\ctxlua{converters.alphabetic(\number#1,"tibetan")}}
-\def\greeknumerals     #1{\ctxlua{converters.alphabetic(\number#1,"greek")}}
-\def\Greeknumerals     #1{\ctxlua{converters.Alphabetic(\number#1,"greek")}}
-\def\arabicnumerals    #1{\ctxlua{converters.alphabetic(\number#1,"arabic")}}
-\def\persiannumerals   #1{\ctxlua{converters.alphabetic(\number#1,"persian")}}
+\def\thainumerals      #1{\ctxlua{converters.alphabetic("#1","thai")}}
+\def\devanagarinumerals#1{\ctxlua{converters.alphabetic("#1","devanagari")}}
+\def\gurmurkhinumerals #1{\ctxlua{converters.alphabetic("#1","gurmurkhi")}}
+\def\gujaratinumerals  #1{\ctxlua{converters.alphabetic("#1","gujarati")}}
+\def\tibetannumerals   #1{\ctxlua{converters.alphabetic("#1","tibetan")}}
+\def\greeknumerals     #1{\ctxlua{converters.alphabetic("#1","greek")}}
+\def\Greeknumerals     #1{\ctxlua{converters.Alphabetic("#1","greek")}}
+\def\arabicnumerals    #1{\ctxlua{converters.alphabetic("#1","arabic")}}
+\def\persiannumerals   #1{\ctxlua{converters.alphabetic("#1","persian")}}
 
 \let\arabicexnumerals  \persiannumerals
 
diff -Naur cont-tmf/tex/context/base/lang-sla.mkiv cont-tmf.local/tex/context/base/lang-sla.mkiv
--- cont-tmf/tex/context/base/lang-sla.mkiv	2007-08-15 20:38:03.000000000 +0300
+++ cont-tmf.local/tex/context/base/lang-sla.mkiv	2008-09-02 12:07:45.000000000 +0200
@@ -11,7 +11,7 @@
 %C therefore copyrighted by \PRAGMA. See mreadme.pdf for
 %C details.
 
-\def\sloveniancharacters#1{\ctxlua{converters.alphabetic(\number#1,"sl")}}
-\def\slovenianCharacters#1{\ctxlua{converters.Alphabetic(\number#1,"sl")}}
+\def\sloveniancharacters#1{\ctxlua{converters.alphabetic("#1","sl")}}
+\def\slovenianCharacters#1{\ctxlua{converters.Alphabetic("#1","sl")}}
 
 \endinput

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bug with numeral conversion function

[Hans Hagen]

Khaled Hosny wrote:
> On Sat, Aug 23, 2008 at 12:46:47AM +0300, Khaled Hosny wrote:
>> It seems that there is a bug in converters.alphabetic function,
>> converters.alphabetic(0,"arabic") returns the western 0 (no matter what
>> is the selected language) while converters.alphabetic(1,"arabic") gives
>> the Arabic 0 not 1 and so one i.e. it looks like as if it starts
>> counting from zero.
> 
> I finally got myself to understand some lua code.
> 
> I think the problem is that lua tables start counting from 1 not 0,
> getting the value of key 0 from code table will give nil while 1 will
> give the value of 0 (first key), so the solution would be incrementing n
> by 1 in:
> 
> local function do_alphabetic(n,max,chr)
>     n = n + 1 -- to get the correct key
>     if n > max then
>         do_alphabetic(floor((n-1)/max),max,chr)
>         n = (n-1)%max+1
>     end
>     characters.flush(chr(n))
> end
> 
> This does work for the case of Arabic, but I don't know about others
> (especially greek and slovenian which look different)

we cannot change that function without breaking other things

there are several ways to convert, one is using the languages.counters 
table and i don't know if the arabic entries in it are right (i just 
found out that the greek vector is uppercase while it should be 
lowercase but i'll make a fix for that)


 >rgrep arabicn *.tex

core-con.tex 671: \defineconversion [arabicnumerals]   [\arabicnumerals]
core-con.tex 672: \defineconversion [persiannumerals]  [\arabicnumerals]
 >rgrep arabicn *.mkiv

core-con.mkiv 20: \def\abjadnumerals 
#1{\ctxlua{converters.arabicnumerals(\number#1)}}
core-con.mkiv 21: \def\abjadnodotnumerals 
#1{\ctxlua{converters.arabicnodotnumerals(\number#1)}}
core-con.mkiv 22: \def\abjadnaivenumerals 
#1{\ctxlua{converters.arabicnaivenumerals(\number#1)}}
core-con.mkiv 87: \def\arabicnumerals 
#1{\ctxlua{converters.alphabetic(\number#1,"arabic")}}
core-con.mkiv 99: \defineconversion [arabicnumerals]     [\arabicnumerals]
 >

the abjad number converters are definied differently (based on specs by 
idris)

so, if the 0/1 offset is a problem then we need to fix the tables, not 
the function (unless arabic follows a different logic, as chinese does)

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bug with numeral conversion function

[Khaled Hosny]

[-- Attachment #1.1: Type: text/plain, Size: 1367 bytes --]

On Tue, Sep 02, 2008 at 08:25:38PM +0200, Hans Hagen wrote:
> Khaled Hosny wrote:
> core-con.mkiv 20: \def\abjadnumerals 
> #1{\ctxlua{converters.arabicnumerals(\number#1)}}
> core-con.mkiv 21: \def\abjadnodotnumerals 
> #1{\ctxlua{converters.arabicnodotnumerals(\number#1)}}
> core-con.mkiv 22: \def\abjadnaivenumerals 
> #1{\ctxlua{converters.arabicnaivenumerals(\number#1)}}
> core-con.mkiv 87: \def\arabicnumerals 
> #1{\ctxlua{converters.alphabetic(\number#1,"arabic")}}
> core-con.mkiv 99: \defineconversion [arabicnumerals]     [\arabicnumerals]
>  >
> 
> the abjad number converters are definied differently (based on specs by 
> idris)

True, but in:
\def\abjadnumerals#1{\ctxlua{converters.arabicnumerals(\number#1)}}
There is actually no converters.arabicnumerals, only
converters.abjadnumerals, so I think this is a typo.

> 
> so, if the 0/1 offset is a problem then we need to fix the tables, not 
> the function (unless arabic follows a different logic, as chinese does)

That is OK as far as it works :) I was thrilled with how easy is lua and
thought I'd try to write some code (like a kid playing with this new toy
:p), and I don't think we have any special logic, it is essentially the
same system with different numerals.


Regards,
 Khaled

-- 
 Khaled Hosny
 Arabic localizer and member of Arabeyes.org team

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________