Re: Files on a server

[Hans Hagen <j.hagen@xs4all.nl>]

On 5/16/2018 2:58 PM, Hans van der Meer wrote:
> Beste Taco,
> 
> Ik probeer jouw oplossing maar er gebeurt nog iets raars. Ik krijg 
> inderdaad 200 resp. 404 terug bij bestaande niet-bestaande Uri’s. Maar 
> ik krijg wel een error die aan het eind van de run gegenereerd wordt. 
> Heb jij enig idee waar ik dat zou moeten zoeken? Als ik de call naar 
> \verbose uitschakel komt er geen foutmelding. Blijft er na de socket 
> call iets achter wat nog opgeruimd had moeten worden?


how about adding local ...

\startluacode
local content, status, authinfo = socket.http.request {
     method = "HEAD",
     url    = 
"http://hansvandermeer.myqnapcloud.com/archive/denhaag/hga-dtb-1869-6040.pdf",
}

print (status)

\stopluacode


> Hier een deel van de context-log en de code fragmenten in kwestie. 
> <takepersons> is de root node van het bewerkte bestand.
> 
> HVDM-PEV-TEST   > return code: 200
> HVDM-PEV        > number of events processed: 1
> HVDM-PEV        > ============= end of takepersons =============
> backend         > xmp > using file 
> '/Users/hansm/TeX/context-45/tex/texmf-context/tex/context/base/mkiv/lpdf-pdx.xml'
> pages           > flushing realpage 1, userpage 1, subpage 1
> close source    > level 2, order 13, name 
> '/Users/hansm/Documents/Genealogie/familie-event.tex'
> close source    > level 1, order 13, name 
> '/Users/hansm/TeX/context-45/tex/texmf-context/tex/context/base/mkiv/cont-yes.mkiv'
> 
> lua error       > lua error on line 0 in file :
> 
> ...-45/tex/texmf-context/tex/context/base/mkiv/core-uti.lua:92: attempt 
> to index a number value (global 'status')
> 
> <empty file>
> 
> 
> ?
> Process aborted
> 
> Code in tex:
> \def\URIReturnCode#1{\ctxlua{tex.print(hvdm.urireturncode("#1"))}}
> \verbose{HVDM-PEV-TEST}{return code: \URIReturnCode{\theurl}}
> Code in lua:
> hvdm.urireturncode = function (theuri)
> content, status, authinfo = socket.http.request{
> method = "HEAD",
> url = theuri,
> }
> return tostring(status)
> end
> 
> Groet Hans
> 
> 
> 
>> On 16 May 2018, at 14:23, Taco Hoekwater <taco@elvenkind.com 
>> <mailto:taco@elvenkind.com>> wrote:
>>
>>
>> Or use luasocket, which is included in the luatex binary:
>>
>> \startluacode
>> content, status, authinfo = socket.http.request{
>>    method = "HEAD",
>>    url = 
>> "http://hansvandermeer.myqnapcloud.com/archive/denhaag/hga-dtb-1869-6040.pdf” 
>>
>> }
>>
>> print (status)
>>
>> \stopluacode
>>
>> prints ‘404’ in this case.
>>
>> Taco
>>
>>> On 16 May 2018, at 13:45, Henri Menke <henrimenke@gmail.com 
>>> <mailto:henrimenke@gmail.com>> wrote:
>>>
>>> On 05/16/2018 11:23 PM, Hans van der Meer wrote:
>>>> I would be satisfied when a returned 404 error code will be handled
>>>> within a reasonable (configurable) time delay. As for redirection, there
>>>> I would not mind if it is not included.
>>>> Restrictions like that are not a problem for me, because this is for
>>>> building a pdf that accesses many internet located files. The check on
>>>> existence is therefore especially meant for spotting missing or misnamed
>>>> files.
>>>
>>> You can use libcurl via ffi: https://curl.haxx.se/libcurl/
>>>
>>> \startluacode
>>> local ffi = require("ffi")
>>>
>>> ffi.cdef[[
>>> typedef struct Curl_easy CURL;
>>> typedef int CURLcode;
>>> typedef int CURLoption;
>>> typedef int CURLINFO;
>>> typedef size_t(*callback)(void *buffer, size_t size, size_t nmemb, void
>>> *userp);
>>>
>>> CURLcode curl_global_init(long flags);
>>> CURL *curl_easy_init(void);
>>> CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);
>>> CURLcode curl_easy_perform(CURL *curl);
>>> CURLcode curl_easy_getinfo(CURL *curl, CURLINFO info, ...);
>>> void curl_easy_cleanup(CURL *curl);
>>> void curl_global_cleanup(void);
>>> ]]
>>>
>>> -- magic numbers
>>> local CURL_GLOBAL_DEFAULT = 3
>>> local CURLOPT_URL = 10002
>>> local CURLOPT_NOBODY = 44
>>> local CURLOPT_HEADER = 42
>>> local CURLOPT_WRITEFUNCTION = 20011
>>> local CURLINFO_RESPONSE_CODE = 0x200000 + 2
>>>
>>> local lcurl = ffi.load("curl")
>>>
>>> local function discard(buffer, size, nmemb, userp)
>>>   return size * nmemb
>>> end
>>>
>>> function check_status(url)
>>>   lcurl.curl_global_init(CURL_GLOBAL_DEFAULT)
>>>
>>>   local curl = lcurl.curl_easy_init()
>>>
>>>   lcurl.curl_easy_setopt(curl, CURLOPT_URL, url)
>>>   lcurl.curl_easy_setopt(curl, CURLOPT_NOBODY, ffi.cast("long",1))
>>>   lcurl.curl_easy_setopt(curl, CURLOPT_HEADER, ffi.cast("long",1))
>>>   lcurl.curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION,
>>> ffi.cast("callback",discard));
>>>
>>>   local res = lcurl.curl_easy_perform(curl)
>>>   local c_http_code = ffi.new("long[1]")
>>>   lcurl.curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, c_http_code);
>>>
>>>   lcurl.curl_easy_cleanup(curl)
>>>   lcurl.curl_global_cleanup()
>>>
>>>   return tonumber(c_http_code[0])
>>> end
>>> \stopluacode
>>>
>>> \starttext
>>>
>>> \ctxlua{context(check_status("http://example.com/"))}
>>>
>>> \ctxlua{context(check_status("https://example.com/"))}
>>>
>>> \ctxlua{context(check_status("https://example.com/xyz"))}
>>>
>>> \stoptext
>>>
>>>>
>>>> Hans van der Meer
>>>>
>>>>> On 16 May 2018, at 12:41, Henri Menke <henrimenke@gmail.com 
>>>>> <mailto:henrimenke@gmail.com>
>>>>> <mailto:henrimenke@gmail.com>> wrote:
>>>>>
>>>>> On 05/16/2018 09:31 PM, Hans van der Meer wrote:
>>>>>> I tried to determine the existence of a file on the internet. See the
>>>>>> following macro call:
>>>>>>
>>>>>>        % Test if file exists.
>>>>>>        \edef\theurl{\linkprotocol://\urlbase\xmlatt{#1}{link}\thesuffix}
>>>>>>        \doiffileelse
>>>>>>        {\theurl}
>>>>>>        {\verbose{HVDM-PEV-TEST}{file exists: \theurl}}
>>>>>>        {\verbose{HVDM-PEV-TEST}{file does not exist: \theurl}}
>>>>>>
>>>>>> It seems that \doiffileelse always results in true except for local
>>>>>> files where nonexistence is correctly handled.
>>>>>>
>>>>>> For example the following url definitely does not exist but 
>>>>>> nevertheless
>>>>>> \doiffileelse reports its existence:
>>>>>> HVDM-PEV-TEST   > file exists:
>>>>>> http://hansvandermeer.myqnapcloud.com/archive/denhaag/hga-dtb-1869-6040.pdf
>>>>>>
>>>>>> Is it possible that the \doiffile macros are extended for files on the
>>>>>> internet or a correponding set \doifurl is developed?
>>>>>
>>>>> I'm not speaking for the devs here, but I don't think that this is 
>>>>> going
>>>>> to be included.  It would require that all webpages send proper 404
>>>>> codes and timeout after reasonable delay.  Also how would you handle
>>>>> redirects?  Should the ConTeXt run be stalled because checking for a
>>>>> files existence on the web is caught in a redirect loop?
>>>>>
>>>>>>
>>>>>> Hans van der Meer
>>>>>>
>>>>>>
>>>>>>
>>>>>> ___________________________________________________________________________________
>>>>>> If your question is of interest to others as well, please add an
>>>>>> entry to the Wiki!
>>>>>>
>>>>>> maillist : ntg-context@ntg.nl <mailto:ntg-context@ntg.nl> 
>>>>>> <mailto:ntg-context@ntg.nl> /
>>>>>> http://www.ntg.nl/mailman/listinfo/ntg-context
>>>>>> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
>>>>>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>>>>>> wiki     : http://contextgarden.net
>>>>>> ___________________________________________________________________________________
>>>>>>
>>>>>
>>>>> ___________________________________________________________________________________
>>>>> If your question is of interest to others as well, please add an entry
>>>>> to the Wiki!
>>>>>
>>>>> maillist : ntg-context@ntg.nl <mailto:ntg-context@ntg.nl>
>>>>> <mailto:ntg-context@ntg.nl> / 
>>>>> http://www.ntg.nl/mailman/listinfo/ntg-context
>>>>> webpage  : http://www.pragma-ade.nl
>>>>> <http://www.pragma-ade.nl/> / http://context.aanhet.net
>>>>> <http://context.aanhet.net/>
>>>>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>>>>> wiki     : http://contextgarden.net <http://contextgarden.net/>
>>>>> ___________________________________________________________________________________
>>>>
>>>>
>>>>
>>>> ___________________________________________________________________________________
>>>> If your question is of interest to others as well, please add an 
>>>> entry to the Wiki!
>>>>
>>>> maillist : ntg-context@ntg.nl <mailto:ntg-context@ntg.nl> / 
>>>> http://www.ntg.nl/mailman/listinfo/ntg-context
>>>> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
>>>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>>>> wiki     : http://contextgarden.net
>>>> ___________________________________________________________________________________
>>>>
>>>
>>> ___________________________________________________________________________________
>>> If your question is of interest to others as well, please add an 
>>> entry to the Wiki!
>>>
>>> maillist : ntg-context@ntg.nl <mailto:ntg-context@ntg.nl> / 
>>> http://www.ntg.nl/mailman/listinfo/ntg-context
>>> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
>>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>>> wiki     : http://contextgarden.net
>>> ___________________________________________________________________________________
>>
>> Taco Hoekwater
>> Elvenkind BV
>>
>>
>>
>>
>> ___________________________________________________________________________________
>> If your question is of interest to others as well, please add an entry 
>> to the Wiki!
>>
>> maillist : ntg-context@ntg.nl <mailto:ntg-context@ntg.nl> / 
>> http://www.ntg.nl/mailman/listinfo/ntg-context
>> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>> wiki     : http://contextgarden.net
>> ___________________________________________________________________________________
> 
> 
> 
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> archive  : https://bitbucket.org/phg/context-mirror/commits/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________
> 


-- 

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________