ntg-context - mailing list for ConTeXt users
 help / color / mirror / Atom feed
* [NTG-context] How to load external lua library
@ 2025-01-23 18:25 krulis.tomas
  2025-01-23 19:32 ` [NTG-context] " Hans Hagen
  0 siblings, 1 reply; 5+ messages in thread
From: krulis.tomas @ 2025-01-23 18:25 UTC (permalink / raw)
  To: ntg-context

Dear ConTeXt mailing list,

I've been struggling with loading the sqlite external lua library. I tried the following:

1. Instaling the system package and running ConTeXt with the `--permitloadlib` option (I am running Manjaro, an arch-linux derivative: https://archlinux.org/packages/core/x86_64/sqlite/)
2. Installing the luarocks package (https://luarocks.org/modules/dougcurrie/lsqlite3) for the 5.1, 5.3 and 5.4 lua versions. I also tried to symlink the files in $HOME/.luarocks/lib/lua/{version}/lsqlite3.so to the project directory
3. I tried also setting the relevant lua paths with `eval $(luarocks path)`.

The compilation always fails with an error that context can't locate the sqlite3 library.

How can I load data from sqlite database during context compilation? As an MWE, I am adding the example in the documentation for the interaction with sqlite from the context garden (https://www.pragma-ade.nl/general/manuals/sql-mkiv.pdf):

```tex
\starttext

\startluacode
require("util-sql")
utilities.sql.setmethod("sqlite")
require("util-sql-loggers")

local loggers = utilities.sql.loggers
local presets = {
  -- method = "sqlite",
  database = "loggertest",
  datatable = "loggers",
  id = "loggers",
}

os.remove("loggertest.db") -- start fresh
local db = loggers.createdb(presets)
loggers.save(db, { -- variant 1: data subtable
  type = "error",
  action = "process",
  data = { filename = "test-1", message = "whatever a" }
} )

loggers.save(db, { -- variant 2: flat table
  type = "warning",
  action = "process",
  filename = "test-2",
  message = "whatever b"
} )

local result = loggers.collect(db, {
  start = {
  day = 1,
  month = 1,
  year = 2016,
},
  stop = {
  day = 31,
  month = 12,
  year = 2116,
},
  limit = 1000000,
  -- type = "error",
  action = "process"
})

context.starttabulate { "||||||" }

for i=1,#result do
  local r = result[i]
  context.NC() context(r.time)
  context.NC() context(r.type)
  context.NC() context(r.action)

  if r.data then
    context.NC() context(r.data.filename)
    context.NC() context(r.data.message)
  else
    context.NC()
    context.NC()
  end

  context.NC() context.NR()
end

context.stoptabulate()

-- local result = loggers.cleanup(db, {
  -- before = {
  -- day = 1,
  -- month = 1,
  -- year = 2117,
  -- },
-- })
\stopluacode

\stoptext
```

I am sorry for the poor quality of the example, but I don't know how to add an attachment to a message in a mailing list.

I have found a few references on the web and on this mailing list (https://mailman.ntg.nl/archives/list/ntg-context@ntg.nl/message/IDZBJQIXG3JIG5O5CNUNXUIPGR4UFTWE/), but I wasn't able to figure out what I could be doing wrong. I found a reference that the issue might be a version mismatch, so I tested multiple lua versions.

My project is about using multiple data sources in multiple formats and creating reports from them. I know I will definitely need to load YAML as well (with the `lyaml` luarocks package I guess). There's always the alternative of trying to use other programs to get the data, but I was wondering if I could leverage ConTeXt to do that directly.

Thank you for any guidance on this, and I hope this message is not too long ...
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [NTG-context] Re: How to load external lua library
  2025-01-23 18:25 [NTG-context] How to load external lua library krulis.tomas
@ 2025-01-23 19:32 ` Hans Hagen
  2025-01-24 13:21   ` krulis.tomas
  0 siblings, 1 reply; 5+ messages in thread
From: Hans Hagen @ 2025-01-23 19:32 UTC (permalink / raw)
  To: ntg-context

On 1/23/2025 7:25 PM, krulis.tomas@seznam.cz wrote:
> Dear ConTeXt mailing list,
> 
> I've been struggling with loading the sqlite external lua library. I tried the following:
> 
> 1. Instaling the system package and running ConTeXt with the `--permitloadlib` option (I am running Manjaro, an arch-linux derivative: https://archlinux.org/packages/core/x86_64/sqlite/)
> 2. Installing the luarocks package (https://luarocks.org/modules/dougcurrie/lsqlite3) for the 5.1, 5.3 and 5.4 lua versions. I also tried to symlink the files in $HOME/.luarocks/lib/lua/{version}/lsqlite3.so to the project directory
> 3. I tried also setting the relevant lua paths with `eval $(luarocks path)`.
> 
> The compilation always fails with an error that context can't locate the sqlite3 library.
> 
> How can I load data from sqlite database during context compilation? As an MWE, I am adding the example in the documentation for the interaction with sqlite from the context garden (https://www.pragma-ade.nl/general/manuals/sql-mkiv.pdf):
> 
> ```tex
> \starttext
> 
> \startluacode
> require("util-sql")
> utilities.sql.setmethod("sqlite")
> require("util-sql-loggers")
> 
> local loggers = utilities.sql.loggers
> local presets = {
>    -- method = "sqlite",
>    database = "loggertest",
>    datatable = "loggers",
>    id = "loggers",
> }
> 
> os.remove("loggertest.db") -- start fresh
> local db = loggers.createdb(presets)
> loggers.save(db, { -- variant 1: data subtable
>    type = "error",
>    action = "process",
>    data = { filename = "test-1", message = "whatever a" }
> } )
> 
> loggers.save(db, { -- variant 2: flat table
>    type = "warning",
>    action = "process",
>    filename = "test-2",
>    message = "whatever b"
> } )
> 
> local result = loggers.collect(db, {
>    start = {
>    day = 1,
>    month = 1,
>    year = 2016,
> },
>    stop = {
>    day = 31,
>    month = 12,
>    year = 2116,
> },
>    limit = 1000000,
>    -- type = "error",
>    action = "process"
> })
> 
> context.starttabulate { "||||||" }
> 
> for i=1,#result do
>    local r = result[i]
>    context.NC() context(r.time)
>    context.NC() context(r.type)
>    context.NC() context(r.action)
> 
>    if r.data then
>      context.NC() context(r.data.filename)
>      context.NC() context(r.data.message)
>    else
>      context.NC()
>      context.NC()
>    end
> 
>    context.NC() context.NR()
> end
> 
> context.stoptabulate()
> 
> -- local result = loggers.cleanup(db, {
>    -- before = {
>    -- day = 1,
>    -- month = 1,
>    -- year = 2117,
>    -- },
> -- })
> \stopluacode
> 
> \stoptext
> ```
> 
> I am sorry for the poor quality of the example, but I don't know how to add an attachment to a message in a mailing list.
> 
> I have found a few references on the web and on this mailing list (https://mailman.ntg.nl/archives/list/ntg-context@ntg.nl/message/IDZBJQIXG3JIG5O5CNUNXUIPGR4UFTWE/), but I wasn't able to figure out what I could be doing wrong. I found a reference that the issue might be a version mismatch, so I tested multiple lua versions.
> 
> My project is about using multiple data sources in multiple formats and creating reports from them. I know I will definitely need to load YAML as well (with the `lyaml` luarocks package I guess). There's always the alternative of trying to use other programs to get the data, but I was wondering if I could leverage ConTeXt to do that directly.
> 
> Thank you for any guidance on this, and I hope this message is not too long ...

On my machine I have:

  Directory of e:\tex-context\tex\texmf-win64\bin\lib\luametatex\sqlite

06/16/2017  10:24 AM         1,730,048 sqlite3.dll

so, you should put sqlite3.so in a similar place in your texmf-linux tree

just the regular lib, nothing needs to be compiled for lua

\usewmodule[sql]

should load the right thing then

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [NTG-context] Re: How to load external lua library
  2025-01-23 19:32 ` [NTG-context] " Hans Hagen
@ 2025-01-24 13:21   ` krulis.tomas
  2025-01-24 14:27     ` Hans Hagen
  0 siblings, 1 reply; 5+ messages in thread
From: krulis.tomas @ 2025-01-24 13:21 UTC (permalink / raw)
  To: ntg-context

Hello Mr. Hagen,

I am actually using ConTeXt bundled with TeXlive, and I struggled with finding the file tree you mentioned there, so I went into installing the ConTeXt distribution separately. With the current installation (ConTeXt  ver: 2024.12.30 16:14 LMTX  fmt: 2025.1.24  int: english/english), I am hiting this error:

```
token call, execute: ...ext/tex/texmf-context/tex/context/base/mkiv/util-sql.lua:91: attempt to index a boolean value (local 'json')
stack traceback:
        ...ext/tex/texmf-context/tex/context/base/mkiv/util-sql.lua:91: in main chunk
        [C]: in upvalue 'requiem'
        ...xt/tex/texmf-context/tex/context/base/mkiv/l-sandbox.lua:180: in function <...xt/tex/texmf-context/tex/context/base/mkiv/l-sandbox.lua:165>
        (...tail calls...)
        [ctxlua]:2: in main chunk
```

which is rather confusing to me, and doesn't happen with TeXlive. Am I missing a part of the installation, or can I fix this in some other way?
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [NTG-context] Re: How to load external lua library
  2025-01-24 13:21   ` krulis.tomas
@ 2025-01-24 14:27     ` Hans Hagen
  2025-01-25 15:37       ` krulis.tomas
  0 siblings, 1 reply; 5+ messages in thread
From: Hans Hagen @ 2025-01-24 14:27 UTC (permalink / raw)
  To: ntg-context

On 1/24/2025 2:21 PM, krulis.tomas@seznam.cz wrote:
> Hello Mr. Hagen,
> 
> I am actually using ConTeXt bundled with TeXlive, and I struggled with finding the file tree you mentioned there, so I went into installing the ConTeXt distribution separately. With the current installation (ConTeXt  ver: 2024.12.30 16:14 LMTX  fmt: 2025.1.24  int: english/english), I am hiting this error:
> 
> ```
> token call, execute: ...ext/tex/texmf-context/tex/context/base/mkiv/util-sql.lua:91: attempt to index a boolean value (local 'json')
> stack traceback:
>          ...ext/tex/texmf-context/tex/context/base/mkiv/util-sql.lua:91: in main chunk
>          [C]: in upvalue 'requiem'
>          ...xt/tex/texmf-context/tex/context/base/mkiv/l-sandbox.lua:180: in function <...xt/tex/texmf-context/tex/context/base/mkiv/l-sandbox.lua:165>
>          (...tail calls...)
>          [ctxlua]:2: in main chunk
> ```
> 
> which is rather confusing to me, and doesn't happen with TeXlive. Am I missing a part of the installation, or can I fix this in some other way?

can you change this at the end of util-json

package.loaded["util-jsn"] = json
package.loaded.json        = json

return json

and remake the format?

(the json module is now always preloaded)

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [NTG-context] Re: How to load external lua library
  2025-01-24 14:27     ` Hans Hagen
@ 2025-01-25 15:37       ` krulis.tomas
  0 siblings, 0 replies; 5+ messages in thread
From: krulis.tomas @ 2025-01-25 15:37 UTC (permalink / raw)
  To: ntg-context

In my (non-TeXlive) ConTeXt installation, I've found the file:

$HOME/context/tex/texmf-context/tex/context/base/mkiv/util-jsn.lua

and added the changes you mentioned (so the end of the file now is):

package.loaded["util-jsn"] = json
package.loaded.json        = json

return json

(around the line 460)

and I ran context --make

But I am still getting the same error.

Are these steps correct, or did I miss anything? I am not familiar with ConTeXt codebase (or not well enough) ... Thank you for guiding me through this!
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2025-01-25 15:41 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-01-23 18:25 [NTG-context] How to load external lua library krulis.tomas
2025-01-23 19:32 ` [NTG-context] " Hans Hagen
2025-01-24 13:21   ` krulis.tomas
2025-01-24 14:27     ` Hans Hagen
2025-01-25 15:37       ` krulis.tomas

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).