Context using LuaTeX - distribution packaging question

Context using LuaTeX - distribution packaging question

[Tobias Burnus]

Hello,

for openSUSE there are now beta TeX Live 2008 packages available
(http://download.opensuse.org/repositories/Publishing/). [By Werner
Fink, who does also the normal openSUSE packages.]

The question is how the ConTeXt with LuaTeX should be best handled. By
default, there is no luatex-cache and no (luatex) format.

Of cause a user can do:

  export TEXMFCNF=/usr/share/texmf/web2c
  # export TEXMFCACHE=somewhere
  luatools --generate --verbose
  luatools --ini --compile cont-en

but in principle I would expect that the distribution does this already,
which saves the TEXMFCACHE per user and makes "texexec --lua" working
without requiring to run first all the other commands.


Questions:

a) Is the TEXMFCNF environment variable necessary or can this also be
read by a config file, which is overwritten by the environment variable?
Which scripts/executables make use of that environemt variable?

b) Is is possible -- as with kpsewhich -- to have a global cache for the
format + global directories - and local directories which have then a
higher precedence? I mean like: A global directory cache incl. the
format files plus if the user creates a $HOME/texmf tree a user
$TEXMFCACHE which contains the hashes of the user directory but such
that luatex still finds the formats in the global directory?

Tobias
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Tobias Burnus]

Tobias Burnus wrote:
> a) Is the TEXMFCNF environment variable necessary or can this also be
> read by a config file, which is overwritten by the environment variable?
> Which scripts/executables make use of that environemt variable?
>   
I just saw that this might be possible in theory, but it fails due to
the missing symbolic link resolving:

        if input.env(instance,'TEXMFCNF') == "" then
            local ownpath = environment.ownpath() or "."

ownpath is here "/usr/bin" and obvious finding "texmf" by moving up in
that path does not work. In

function environment.ownpath()

one need to add in

            if lfs.isfile(b..".exe") or lfs.isfile(b) then
                ownpath = p
                break

some check whether "b" is a symbolic link and if it is, one follows the
symbolic link until one finds the real binary; then one can use that
path for ownpath. I tried to find some Lua routine which does so, but I
couldn't find one ad hoc.  (I have also the vague feeling that the
algorithm fails for /some/directory/which/is/not/in/PATH/luatools.)

I tried lfs.symlinkattributes() (whose mode should be "link") plus
posix.readlink, but I failed to get this work. (lfs.symlinkattributes
and posix are not found.)

Tobias
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Hans Hagen]

Tobias Burnus wrote:
> Hello,
> 
> for openSUSE there are now beta TeX Live 2008 packages available
> (http://download.opensuse.org/repositories/Publishing/). [By Werner
> Fink, who does also the normal openSUSE packages.]
> 
> The question is how the ConTeXt with LuaTeX should be best handled. By
> default, there is no luatex-cache and no (luatex) format.
> 
> Of cause a user can do:
> 
>   export TEXMFCNF=/usr/share/texmf/web2c
>   # export TEXMFCACHE=somewhere
>   luatools --generate --verbose
>   luatools --ini --compile cont-en
> 
> but in principle I would expect that the distribution does this already,
> which saves the TEXMFCACHE per user and makes "texexec --lua" working
> without requiring to run first all the other commands.
> 
> 
> Questions:
> 
> a) Is the TEXMFCNF environment variable necessary or can this also be
> read by a config file, which is overwritten by the environment variable?
> Which scripts/executables make use of that environemt variable?

there is this automess stuff implemented which assumes that the 
distribution kind of follows the de-facto standards of having teh bin 
relative to the tree; in al other cases one needs to set env vars

btw, there is a file texmfcnf.lua alongside the cnf file whioch can be 
tweaked; so you can leave the original cnf untouched

> b) Is is possible -- as with kpsewhich -- to have a global cache for the
> format + global directories - and local directories which have then a
> higher precedence? I mean like: A global directory cache incl. the
> format files plus if the user creates a $HOME/texmf tree a user
> $TEXMFCACHE which contains the hashes of the user directory but such
> that luatex still finds the formats in the global directory?

not yet .. i'm still pondering over a robust system (one side effect is 
for instance that when there are multiple cached copies we need to get 
not only the one first in the chain, but also a valid one; i.e. what to 
do if the latest one has an older version number than demanded and such; 
no big deal, but it means that the cache loader needs to be made more 
clever which is something that i will not do in a current or beta but 
first on my own system when we have reached some stability)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Hans Hagen]

Tobias Burnus wrote:

> some check whether "b" is a symbolic link and if it is, one follows the
> symbolic link until one finds the real binary; then one can use that
> path for ownpath. I tried to find some Lua routine which does so, but I
> couldn't find one ad hoc.  (I have also the vague feeling that the
> algorithm fails for /some/directory/which/is/not/in/PATH/luatools.)
> 
> I tried lfs.symlinkattributes() (whose mode should be "link") plus
> posix.readlink, but I failed to get this work. (lfs.symlinkattributes
> and posix are not found.)

i remember that there has been some discussion about this following link 
  issue but i forgot what was the outcome

do you test the latest binaries or the already outdated texlive ones?


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Tobias Burnus]

Hans Hagen wrote:
> i remember that there has been some discussion about this following link 
>   issue but i forgot what was the outcome
>
> do you test the latest binaries or the already outdated texlive ones?
>   

That was TeX Live, though I don't know whether it is the SVN version or
something else. However, I compared TeX Live's luatools.lua with the one
of the latest cont-tmf.zip and they are identical.

Tobias
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Tobias Burnus]

Hi Hans,

Hans Hagen wrote:
> i remember that there has been some discussion about this following link 
>   issue but i forgot what was the outcome
>   

Can you incorporate the following patch in your cont-tmf.zip? It should
work with most Linux systems and it should not harm on other systems.

Tobias

--- /dev/shm/luatools.lua       2008-10-29 12:46:21.261006248 +0100
+++ luatools.lua                2008-10-29 15:56:04.000000000 +0100
@@ -3053,9 +3053,16 @@
                     if not lfs.isdir(file.join(ownroot,"texmf")) then
                         ownroot = input.normalize_name(file.join(ownpath,".."))
                         if not lfs.isdir(file.join(ownroot,"texmf")) then
-                            input.verbose = true
-                            input.report("error", "unable to identify cnf file")
-                            return
+                            -- On Linux installations, the file might be under $PREFIX/bin
+                            -- while the texmf tree is under $PREFIX/share/texmf
+                            if string.gmatch(ownpath, "/bin$") then
+                                ownroot = input.normalize_name(file.join(ownpath,"../share"))
+                            end
+                            if not lfs.isdir(file.join(ownroot,"texmf")) then
+                               input.verbose = true
+                               input.report("error", "unable to identify cnf file")
+                               return
+                            end
                         end
                     end
                     local texmfcnf = file.join(ownroot,"texmf-local/web2c",filename) -- for minimals and myself


___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Hans Hagen]

Tobias Burnus wrote:
> Hi Hans,
> 
> Hans Hagen wrote:
>> i remember that there has been some discussion about this following link 
>>   issue but i forgot what was the outcome
>>   
> 
> Can you incorporate the following patch in your cont-tmf.zip? It should
> work with most Linux systems and it should not harm on other systems.

you use an old luatools / luat-inp.lua; if a patch is needed, then it 
should go into

function input.identify_own()
     local instance = input.instance
     local ownpath = input.getownpath() or lfs.currentdir()
     local ie = instance.environment
     if ownpath then
         if input.env('SELFAUTOLOC')    == "" then os.env['SELFAUTOLOC'] 
    = file.collapse_path(ownpath) end
         if input.env('SELFAUTODIR')    == "" then os.env['SELFAUTODIR'] 
    = file.collapse_path(ownpath .. "/..") end
         if input.env('SELFAUTOPARENT') == "" then 
os.env['SELFAUTOPARENT'] = file.collapse_path(ownpath .. "/../..") end
     else
         input.verbose = true
         input.report("error: unable to locate ownpath")
         os.exit()
     end
     if input.env('TEXMFCNF') == "" then os.env['TEXMFCNF'] = 
input.cnfdefault end
     if input.env('TEXOS')    == "" then os.env['TEXOS']    = 
input.env('SELFAUTODIR') end
     if input.env('TEXROOT')  == "" then os.env['TEXROOT']  = 
input.env('SELFAUTOPARENT') end
     if input.verbose then
         for _,v in 
ipairs({"SELFAUTOLOC","SELFAUTODIR","SELFAUTOPARENT","TEXMFCNF"}) do
             input.report("variable %s set to %s",v,input.env(v) or 
"unknown")
         end
     end
     function input.identify_own() end
end

function input.identify_cnf()
     local instance = input.instance
     if #instance.cnffiles == 0 then
         -- fallback
         input.identify_own()
         -- the real search
         input.expand_variables()
         local t = input.split_path(input.env('TEXMFCNF'))
         t = input.aux.expanded_path(t)
         input.aux.expand_vars(t) -- redundant
         local function locate(filename,list)
             for _,v in ipairs(t) do
                 local texmfcnf = 
input.normalize_name(file.join(v,filename))
                 if lfs.isfile(texmfcnf) then
                     table.insert(list,texmfcnf)
                 end
             end
         end
         locate(input.luaname,instance.luafiles)
         locate(input.cnfname,instance.cnffiles)
     end
end


> Tobias
> 
> --- /dev/shm/luatools.lua       2008-10-29 12:46:21.261006248 +0100
> +++ luatools.lua                2008-10-29 15:56:04.000000000 +0100
> @@ -3053,9 +3053,16 @@
>                      if not lfs.isdir(file.join(ownroot,"texmf")) then
>                          ownroot = input.normalize_name(file.join(ownpath,".."))
>                          if not lfs.isdir(file.join(ownroot,"texmf")) then
> -                            input.verbose = true
> -                            input.report("error", "unable to identify cnf file")
> -                            return
> +                            -- On Linux installations, the file might be under $PREFIX/bin
> +                            -- while the texmf tree is under $PREFIX/share/texmf
> +                            if string.gmatch(ownpath, "/bin$") then
> +                                ownroot = input.normalize_name(file.join(ownpath,"../share"))
> +                            end
> +                            if not lfs.isdir(file.join(ownroot,"texmf")) then
> +                               input.verbose = true
> +                               input.report("error", "unable to identify cnf file")
> +                               return
> +                            end
>                          end
>                      end
>                      local texmfcnf = file.join(ownroot,"texmf-local/web2c",filename) -- for minimals and myself
> 
> 
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : https://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________


-- 

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Tobias Burnus]

Hi Hans,

Hans Hagen wrote:
> you use an old luatools / luat-inp.lua;
I updated now luat-inp.lua from the latest experimental cont-tmf.zip,
however, that does not help.

a) luatools.lua does not use luat-inp.lua (and luatools.lua is up to
date in TeX Live) Thus updating does not help. (Besides, without the
texmfcnf file, luatools.lua cannot use any other file as one cannot
search for it.)

b) Following UNIX symbolic links does not seem to work under Linux. I
played around with luat-inp.lua and it does not work.

c) Werner asks: Why is the KPSE library not used? I have to agree - KPSE
uses the compile-time defaults for the path and thus should provide the
correct default TEXMFCNF directory. Afterwards, I'm fine using a
non-KPSE method. How about using KPSE as backup method if it doesn't
find a config file otherwise?


For "luatex" itself (contrary to "luatools") there is no problem as
luatex uses the compile-time path and finds the texmfcnf file.


Back to (b): The symbolic links support in luat-inp.lua seems only to
work if your "directory" is a symbolic link but not if the file is a
symbolic link, e.g. "foo/bin -> /usr/local/bar-bin" then for
"foo/bin/file" doing "chdir(foo/bin)" makes sense as then "currentdir()"
prints "/usr/local/bar-bin" but this does not work for /usr/bin/luatools
-> ../share/texmf/..../luatools.lua as the chdir(/usr/bin) makes no
difference.

One really needs "posix.readlink" support, which is not in my texlua.
Or, alternative 2, if path ends in bin, try "../share/texmf" as my old
patch does. Or one indeed uses KPSE:
            kpse.set_program_name('luatex')
            local kpse_result1 = kpse.find_file(input.luaname,'cnf',true)
            local kpse_result2 = kpse.find_file(input.cnfname,'cnf')

I'm also open for other suggestions.

Tobias
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Tobias Burnus]

Tobias Burnus wrote:
> c) Werner asks: Why is the KPSE library not used?
>   
I have a scenario which fails with the current mechanism - even if
following the sym links worked. Assume that a user only puts a texmf.cnf
under "$HOME/texmf/web2c/texmf.cnf". That file will be used by "luatex",
"pdftex" etc. but "luatools" will still resolve
/usr/share/texmf/.../luatools.lua if it follows $SELF.

Thus I'm in growingly in favour of using KPSE (just) for finding
"texmf.cnf" and "texmfcnf.lua"; for all other purposes including
seaching files whatever other method can be used.

Tobias
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Hans Hagen]

Tobias Burnus wrote:
> Hi Hans,
> 
> Hans Hagen wrote:
>> you use an old luatools / luat-inp.lua;
> I updated now luat-inp.lua from the latest experimental cont-tmf.zip,
> however, that does not help.
> 
> a) luatools.lua does not use luat-inp.lua (and luatools.lua is up to
> date in TeX Live) Thus updating does not help. (Besides, without the
> texmfcnf file, luatools.lua cannot use any other file as one cannot
> search for it.)

luat-inp.lua is merged into luatools in order to make in 'stand-alone'

luatools --selfupdate

could do that depending on the fact if your system works

i really cannot find the code in my luatools (i'll send it to you)

> b) Following UNIX symbolic links does not seem to work under Linux. I
> played around with luat-inp.lua and it does not work.

hm, maybe taco knows ..

> c) Werner asks: Why is the KPSE library not used? I have to agree - KPSE
> uses the compile-time defaults for the path and thus should provide the
> correct default TEXMFCNF directory. Afterwards, I'm fine using a
> non-KPSE method. How about using KPSE as backup method if it doesn't
> find a config file otherwise?

several reasons:

(1) in spite of promises, kpse was never extended to support other input 
methods
(2) the lua library can handle things like zip files, url's, soon 
sockets, etc. i don't want to be limited in context
(3) the lua library is (once the database is there) faster than kpse
(4) when used in scripts we only load the databsse once so in the case 
of context luatools (or mtxrun) can run scripts that themselves need the 
database without a reload (ok, to some extend that can also be done with 
the kpse lib)
(5) i have control over the lua based library -)

> For "luatex" itself (contrary to "luatools") there is no problem as
> luatex uses the compile-time path and finds the texmfcnf file.

hm, but since luatools is ran using texlus (which is luatex) the same 
should apply

- make sure that luatools is run by texlua and not by lua itself
- don't use a stub, just rename luatools.lua to luatools and mtxrun.lua 
to mtxrun (so, no texmfstart like extra stubs which would slow down 
things considerably)

> Back to (b): The symbolic links support in luat-inp.lua seems only to
> work if your "directory" is a symbolic link but not if the file is a
> symbolic link, e.g. "foo/bin -> /usr/local/bar-bin" then for
> "foo/bin/file" doing "chdir(foo/bin)" makes sense as then "currentdir()"
> prints "/usr/local/bar-bin" but this does not work for /usr/bin/luatools
> -> ../share/texmf/..../luatools.lua as the chdir(/usr/bin) makes no
> difference.
> 
> One really needs "posix.readlink" support, which is not in my texlua.
> Or, alternative 2, if path ends in bin, try "../share/texmf" as my old
> patch does. Or one indeed uses KPSE:
>             kpse.set_program_name('luatex')
>             local kpse_result1 = kpse.find_file(input.luaname,'cnf',true)
>             local kpse_result2 = kpse.find_file(input.cnfname,'cnf')
> 
> I'm also open for other suggestions.

once we have the same luatools we can look into the share thing

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Hans Hagen]

Tobias Burnus wrote:
> Hi Hans,
> 
> Hans Hagen wrote:
>> you use an old luatools / luat-inp.lua;
> I updated now luat-inp.lua from the latest experimental cont-tmf.zip,
> however, that does not help.

did you copy that file to 'luatools' in the bin path?

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Tobias Burnus]

Hi Hans,

Hans Hagen wrote:
> luat-inp.lua is merged into luatools in order to make in 'stand-alone'
>   
OK, I missed that.

By the way, unless I screwed up completely, the new luatools you have
sent me work:

LuaTools | variable SELFAUTOLOC set to /usr/bin
LuaTools | variable SELFAUTODIR set to /usr
LuaTools | variable SELFAUTOPARENT set to /
LuaTools | variable TEXMFCNF set to
{$SELFAUTODIR,$SELFAUTOPARENT}{,{/share,}/texmf{-local,.local,}/web2c}
LuaTools | skipping configuration for /usr/share/texmf/web2c from
/home/tob/luatex-cache/context/c67bf06401a0e1a39f3ee04c60209dc5/trees/e027248d6557d124c703335e8a95ecd5

The symbolic links are not resolved, but
"$SELFAUTODIR/share/texmf/web2c/" is the correct directory.


>> b) Following UNIX symbolic links does not seem to work under Linux. I
>> played around with luat-inp.lua and it does not work.
>>     
> hm, maybe taco knows ..
>   
I think there is no way round but posix.readlink. However, it might be
not needed as the new, just sent "luatools" works.


>> c) Werner asks: Why is the KPSE library not used?
>>     
>
> several reasons:
>   
I agree about the KPSE shortcomings, however, just for finding
"texmf.cnf" KPSE should be good enough. (OK, finding "texmfcnf.lua" does
not seem to work well as the "lua" extension means that "web2c" is not
searched.)

>> For "luatex" itself (contrary to "luatools") there is no problem as
>> luatex uses the compile-time path and finds the texmfcnf file.
>>     
> hm, but since luatools is ran using texlus (which is luatex) the same 
> should apply
>   

I think the algorithm used to find the texmf.cnf file is different and
thus a different file is found. Your algorithm never checks
"$HOME/texmf/web2c" where a user could have put his file. For pdftex
$HOME/texmf/web2c has a higher precedence than /usr/share/texmf/web2c. I
assume that the same thing is true for "luatex". In any case the luatex
binary contains the following string:

/etc/texmf:/etc/texmf/web2c:/var/lib/texmf/web2c:{~/texmf,!!/etc/texmf,!!/var/lib/texmf,!!/usr/lib/texmf,!!/usr/local/share/texmf,!!/usr/share/texmf}/web2c

Using KPSE the same string seems to be used to check for the config
file. But "luatools" uses a completely different method. As long as the
user does not have its own texmf.cnf there is no problem. (Should be the
case most of the time.) However, as soon as (s)he has, luatex and
"luatools" access different files!


> - make sure that luatools is run by texlua and not by lua itself
>   
That seems to be the case - my "lua" does not know about "lfs" and that
works with texlua.


> - don't use a stub, just rename luatools.lua to luatools and mtxrun.lua 
> to mtxrun (so, no texmfstart like extra stubs which would slow down 
> things considerably)
>   
That is done differently: luatools.lua remains like that, but
"/usr/bin/luatools" is a symbolic link to "luatools.lua".

Tobias
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Hans Hagen]

Tobias Burnus wrote:

> I agree about the KPSE shortcomings, however, just for finding
> "texmf.cnf" KPSE should be good enough. (OK, finding "texmfcnf.lua" does
> not seem to work well as the "lua" extension means that "web2c" is not
> searched.)

but using kpse would mean initializing it and that means loading the 
file database which can be pretty large for a non minimal tex tree so 
that's a no-go

> I think the algorithm used to find the texmf.cnf file is different and
> thus a different file is found. Your algorithm never checks
> "$HOME/texmf/web2c" where a user could have put his file. For pdftex
> $HOME/texmf/web2c has a higher precedence than /usr/share/texmf/web2c. I
> assume that the same thing is true for "luatex". In any case the luatex
> binary contains the following string:

what files are looked for depends also on env settings; but anyhow, at 
some point mkiv/luatex might drop the cnf file

> /etc/texmf:/etc/texmf/web2c:/var/lib/texmf/web2c:{~/texmf,!!/etc/texmf,!!/var/lib/texmf,!!/usr/lib/texmf,!!/usr/local/share/texmf,!!/usr/share/texmf}/web2c

hm, where is that set? hard coded in the binary?

but concerning the cnf file, luatex/mkiv does not use that many paths 
(less fonts for instance)

> Using KPSE the same string seems to be used to check for the config
> file. But "luatools" uses a completely different method. As long as the
> user does not have its own texmf.cnf there is no problem. (Should be the
> case most of the time.) However, as soon as (s)he has, luatex and
> "luatools" access different files!

well, if something is hard coded in the binary then i'm not going to 
bother about it too much; if something is determined by env vars, we can 
see what makes sense to support; also, an installer that does things 
differently from the standard tex live setup can also take care of 
setting a few variables

> That is done differently: luatools.lua remains like that, but
> "/usr/bin/luatools" is a symbolic link to "luatools.lua".

ok

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Hans Hagen]

Tobias Burnus wrote:
> Tobias Burnus wrote:
>> c) Werner asks: Why is the KPSE library not used?
>>   
> I have a scenario which fails with the current mechanism - even if
> following the sym links worked. Assume that a user only puts a texmf.cnf
> under "$HOME/texmf/web2c/texmf.cnf". That file will be used by "luatex",
> "pdftex" etc. but "luatools" will still resolve
> /usr/share/texmf/.../luatools.lua if it follows $SELF.

well, users can always force a file

if there is logic we can try to follow it, if there is hard coded 
behaviour, well, we can forget about it; also, keep in mind that 
luatools collapses all cnf files that it uses (it does use more if 
specified) into one file in the cache, so any change to a cnf must be 
followed by luatools --generate

> Thus I'm in growingly in favour of using KPSE (just) for finding
> "texmf.cnf" and "texmfcnf.lua"; for all other purposes including
> seaching files whatever other method can be used.

no option

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Peter Münster]

On Wed, Oct 29 2008, Tobias Burnus wrote:

> I have a scenario which fails with the current mechanism - even if
> following the sym links worked. Assume that a user only puts a texmf.cnf
> under "$HOME/texmf/web2c/texmf.cnf". That file will be used by "luatex",
> "pdftex" etc. but "luatools" will still resolve
> /usr/share/texmf/.../luatools.lua if it follows $SELF.
> 
> Thus I'm in growingly in favour of using KPSE (just) for finding
> "texmf.cnf" and "texmfcnf.lua"; for all other purposes including
> seaching files whatever other method can be used.

Would KPSE find texmfcnf.lua in $HOME/texmf/web2c ?
>From one of your previous messages, I understand that it would *not* find
it.

Cheers, Peter

-- 
http://pmrb.free.fr/contact/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Tobias Burnus]

Hans,

Hans Hagen wrote:
> what files are looked for depends also on env settings; but anyhow, at 
> some point mkiv/luatex might drop the cnf file
>   
I think some kind of cnf file will still be necessary to tell TeX where
to look for files, though with reasonable defaults, the file would only
be created (by the user) if needed.


>> /etc/texmf:/etc/texmf/web2c:/var/lib/texmf/web2c:{~/texmf,!!/etc/texmf,!!/var/lib/texmf,!!/usr/lib/texmf,!!/usr/local/share/texmf,!!/usr/share/texmf}/web2c
>>     
> hm, where is that set? hard coded in the binary?
>   
Yes, though the exact value depends on the distribution. In openSUSE, if
I found the correct variable, it is

WEB2C = ${TEXMFHOME}/web2c;${TEXMFSYSCONFIG}/web2c;${TEXMFSYSVAR}/web2c;$TEXMF/web2c  


How about keeping the current searching algorithm (of the luatools.lua
you have sent me) plus look additionally in $HOME/texmf/web2c/ ? I think
this is the only reasonable other place. The $HOME/texmf should have
higher precedence to allow the user to overwrite the system defaults.
(That seems to be in line with all teTeX/TeX Live distributions on
Linux, which I know.)

(Another possibility would be to access the stored strings in "texlua"
and use that search path; I don't know whether this is possible.)

Tobias


PS:
Peter Münster wrote:
> Would KPSE find texmfcnf.lua in $HOME/texmf/web2c ?
> >From one of your previous messages, I understand that it would *not* find
> it.
>   
It does - I messed up the "web2c" string search in the luatex binary;
there are two strings. The right one seems to be:

~/texmf/web2c:/etc/texmf/web2c:/var/lib/texmf/web2c:{~/texmf,!!/etc/texmf,!!/var/lib/texmf,!!/usr/lib/texmf,!!/usr/local/share/texmf,!!/usr/share/texmf}/web2c

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[luigi scarso]

[-- Attachment #1.1: Type: text/plain, Size: 694 bytes --]

>
>
>
> PS:
> Peter Münster wrote:
> > Would KPSE find texmfcnf.lua in $HOME/texmf/web2c ?
> > >From one of your previous messages, I understand that it would *not*
> find
> > it.
> >
> It does - I messed up the "web2c" string search in the luatex binary;
> there are two strings. The right one seems to be:
>
>
> ~/texmf/web2c:/etc/texmf/web2c:/var/lib/texmf/web2c:{~/texmf,!!/etc/texmf,!!/var/lib/texmf,!!/usr/lib/texmf,!!/usr/local/share/texmf,!!/usr/share/texmf}/web2c
>
>
I'm on a linux box,
and minimal, minimal-beta works ok
(I also have tl2008 in a separate folder, and it also works , but I don't
mix installation) .
Maybe you can have a look at tex/setuptex in minimals .

-- 
luigi

[-- Attachment #1.2: Type: text/html, Size: 1027 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Context using LuaTeX - distribution packaging question

[Hans Hagen]

Tobias Burnus wrote:
> Hans,
> 
> Hans Hagen wrote:
>> what files are looked for depends also on env settings; but anyhow, at 
>> some point mkiv/luatex might drop the cnf file
>>   
> I think some kind of cnf file will still be necessary to tell TeX where
> to look for files, though with reasonable defaults, the file would only
> be created (by the user) if needed.
> 
> 
>>> /etc/texmf:/etc/texmf/web2c:/var/lib/texmf/web2c:{~/texmf,!!/etc/texmf,!!/var/lib/texmf,!!/usr/lib/texmf,!!/usr/local/share/texmf,!!/usr/share/texmf}/web2c
>>>     
>> hm, where is that set? hard coded in the binary?
>>   
> Yes, though the exact value depends on the distribution. In openSUSE, if
> I found the correct variable, it is
> 
> WEB2C = ${TEXMFHOME}/web2c;${TEXMFSYSCONFIG}/web2c;${TEXMFSYSVAR}/web2c;$TEXMF/web2c  
> 
> 
> How about keeping the current searching algorithm (of the luatools.lua
> you have sent me) plus look additionally in $HOME/texmf/web2c/ ? I think
> this is the only reasonable other place. The $HOME/texmf should have
> higher precedence to allow the user to overwrite the system defaults.
> (That seems to be in line with all teTeX/TeX Live distributions on
> Linux, which I know.)

well, i remember these kind of exploding path definitions in tetex with 
about every permutation of local,share,opt,sys,... no fun

> (Another possibility would be to access the stored strings in "texlua"
> and use that search path; I don't know whether this is possible.)

could be an option (given that it is exposed)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________