Re: Files on a server

[Hans van der Meer <havdmeer@ziggo.nl>]

[-- Attachment #1.1: Type: text/plain, Size: 9238 bytes --]

Beste Taco,

Ik probeer jouw oplossing maar er gebeurt nog iets raars. Ik krijg inderdaad 200 resp. 404 terug bij bestaande niet-bestaande Uri’s. Maar ik krijg wel een error die aan het eind van de run gegenereerd wordt. Heb jij enig idee waar ik dat zou moeten zoeken? Als ik de call naar \verbose uitschakel komt er geen foutmelding. Blijft er na de socket call iets achter wat nog opgeruimd had moeten worden?

Hier een deel van de context-log en de code fragmenten in kwestie. <takepersons> is de root node van het bewerkte bestand.

HVDM-PEV-TEST   > return code: 200
HVDM-PEV        > number of events processed: 1
HVDM-PEV        > ============= end of takepersons =============
backend         > xmp > using file '/Users/hansm/TeX/context-45/tex/texmf-context/tex/context/base/mkiv/lpdf-pdx.xml'
pages           > flushing realpage 1, userpage 1, subpage 1
close source    > level 2, order 13, name '/Users/hansm/Documents/Genealogie/familie-event.tex'
close source    > level 1, order 13, name '/Users/hansm/TeX/context-45/tex/texmf-context/tex/context/base/mkiv/cont-yes.mkiv'

lua error       > lua error on line 0 in file :

...-45/tex/texmf-context/tex/context/base/mkiv/core-uti.lua:92: attempt to index a number value (global 'status')

<empty file>


? 
Process aborted

Code in tex:
	\def\URIReturnCode#1{\ctxlua{tex.print(hvdm.urireturncode("#1"))}} 
	\verbose{HVDM-PEV-TEST}{return code: \URIReturnCode{\theurl}}
Code in lua:
	hvdm.urireturncode = function (theuri)
		content, status, authinfo = socket.http.request{
			method = "HEAD",
			url = theuri,
			}
		return tostring(status)
	end

Groet Hans



> On 16 May 2018, at 14:23, Taco Hoekwater <taco@elvenkind.com> wrote:
> 
> 
> Or use luasocket, which is included in the luatex binary:
> 
> \startluacode
> content, status, authinfo = socket.http.request{
>    method = "HEAD",
>    url = "http://hansvandermeer.myqnapcloud.com/archive/denhaag/hga-dtb-1869-6040.pdf” 
> }
> 
> print (status)
> 
> \stopluacode
> 
> prints ‘404’ in this case.
> 
> Taco
> 
>> On 16 May 2018, at 13:45, Henri Menke <henrimenke@gmail.com> wrote:
>> 
>> On 05/16/2018 11:23 PM, Hans van der Meer wrote:
>>> I would be satisfied when a returned 404 error code will be handled
>>> within a reasonable (configurable) time delay. As for redirection, there
>>> I would not mind if it is not included.
>>> Restrictions like that are not a problem for me, because this is for
>>> building a pdf that accesses many internet located files. The check on
>>> existence is therefore especially meant for spotting missing or misnamed
>>> files.
>> 
>> You can use libcurl via ffi: https://curl.haxx.se/libcurl/
>> 
>> \startluacode
>> local ffi = require("ffi")
>> 
>> ffi.cdef[[
>> typedef struct Curl_easy CURL;
>> typedef int CURLcode;
>> typedef int CURLoption;
>> typedef int CURLINFO;
>> typedef size_t(*callback)(void *buffer, size_t size, size_t nmemb, void
>> *userp);
>> 
>> CURLcode curl_global_init(long flags);
>> CURL *curl_easy_init(void);
>> CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);
>> CURLcode curl_easy_perform(CURL *curl);
>> CURLcode curl_easy_getinfo(CURL *curl, CURLINFO info, ...);
>> void curl_easy_cleanup(CURL *curl);
>> void curl_global_cleanup(void);
>> ]]
>> 
>> -- magic numbers
>> local CURL_GLOBAL_DEFAULT = 3
>> local CURLOPT_URL = 10002
>> local CURLOPT_NOBODY = 44
>> local CURLOPT_HEADER = 42
>> local CURLOPT_WRITEFUNCTION = 20011
>> local CURLINFO_RESPONSE_CODE = 0x200000 + 2
>> 
>> local lcurl = ffi.load("curl")
>> 
>> local function discard(buffer, size, nmemb, userp)
>>   return size * nmemb
>> end
>> 
>> function check_status(url)
>>   lcurl.curl_global_init(CURL_GLOBAL_DEFAULT)
>> 
>>   local curl = lcurl.curl_easy_init()
>> 
>>   lcurl.curl_easy_setopt(curl, CURLOPT_URL, url)
>>   lcurl.curl_easy_setopt(curl, CURLOPT_NOBODY, ffi.cast("long",1))
>>   lcurl.curl_easy_setopt(curl, CURLOPT_HEADER, ffi.cast("long",1))
>>   lcurl.curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION,
>> ffi.cast("callback",discard));
>> 
>>   local res = lcurl.curl_easy_perform(curl)
>>   local c_http_code = ffi.new("long[1]")
>>   lcurl.curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, c_http_code);
>> 
>>   lcurl.curl_easy_cleanup(curl)
>>   lcurl.curl_global_cleanup()
>> 
>>   return tonumber(c_http_code[0])
>> end
>> \stopluacode
>> 
>> \starttext
>> 
>> \ctxlua{context(check_status("http://example.com/"))}
>> 
>> \ctxlua{context(check_status("https://example.com/"))}
>> 
>> \ctxlua{context(check_status("https://example.com/xyz"))}
>> 
>> \stoptext
>> 
>>> 
>>> Hans van der Meer
>>> 
>>>> On 16 May 2018, at 12:41, Henri Menke <henrimenke@gmail.com
>>>> <mailto:henrimenke@gmail.com>> wrote:
>>>> 
>>>> On 05/16/2018 09:31 PM, Hans van der Meer wrote:
>>>>> I tried to determine the existence of a file on the internet. See the
>>>>> following macro call:
>>>>> 
>>>>>        % Test if file exists.
>>>>>        \edef\theurl{\linkprotocol://\urlbase\xmlatt{#1}{link}\thesuffix}
>>>>>        \doiffileelse
>>>>>        {\theurl}
>>>>>        {\verbose{HVDM-PEV-TEST}{file exists: \theurl}}
>>>>>        {\verbose{HVDM-PEV-TEST}{file does not exist: \theurl}}
>>>>> 
>>>>> It seems that \doiffileelse always results in true except for local
>>>>> files where nonexistence is correctly handled.
>>>>> 
>>>>> For example the following url definitely does not exist but nevertheless
>>>>> \doiffileelse reports its existence:
>>>>> HVDM-PEV-TEST   > file exists:
>>>>> http://hansvandermeer.myqnapcloud.com/archive/denhaag/hga-dtb-1869-6040.pdf
>>>>> 
>>>>> Is it possible that the \doiffile macros are extended for files on the
>>>>> internet or a correponding set \doifurl is developed?
>>>> 
>>>> I'm not speaking for the devs here, but I don't think that this is going
>>>> to be included.  It would require that all webpages send proper 404
>>>> codes and timeout after reasonable delay.  Also how would you handle
>>>> redirects?  Should the ConTeXt run be stalled because checking for a
>>>> files existence on the web is caught in a redirect loop?
>>>> 
>>>>> 
>>>>> Hans van der Meer
>>>>> 
>>>>> 
>>>>> 
>>>>> ___________________________________________________________________________________
>>>>> If your question is of interest to others as well, please add an
>>>>> entry to the Wiki!
>>>>> 
>>>>> maillist : ntg-context@ntg.nl <mailto:ntg-context@ntg.nl> /
>>>>> http://www.ntg.nl/mailman/listinfo/ntg-context
>>>>> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
>>>>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>>>>> wiki     : http://contextgarden.net
>>>>> ___________________________________________________________________________________
>>>>> 
>>>> 
>>>> ___________________________________________________________________________________
>>>> If your question is of interest to others as well, please add an entry
>>>> to the Wiki!
>>>> 
>>>> maillist : ntg-context@ntg.nl
>>>> <mailto:ntg-context@ntg.nl> / http://www.ntg.nl/mailman/listinfo/ntg-context
>>>> webpage  : http://www.pragma-ade.nl
>>>> <http://www.pragma-ade.nl/> / http://context.aanhet.net
>>>> <http://context.aanhet.net/>
>>>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>>>> wiki     : http://contextgarden.net <http://contextgarden.net/>
>>>> ___________________________________________________________________________________
>>> 
>>> 
>>> 
>>> ___________________________________________________________________________________
>>> If your question is of interest to others as well, please add an entry to the Wiki!
>>> 
>>> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
>>> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
>>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>>> wiki     : http://contextgarden.net
>>> ___________________________________________________________________________________
>>> 
>> 
>> ___________________________________________________________________________________
>> If your question is of interest to others as well, please add an entry to the Wiki!
>> 
>> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
>> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
>> archive  : https://bitbucket.org/phg/context-mirror/commits/
>> wiki     : http://contextgarden.net
>> ___________________________________________________________________________________
> 
> Taco Hoekwater
> Elvenkind BV
> 
> 
> 
> 
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> archive  : https://bitbucket.org/phg/context-mirror/commits/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________


[-- Attachment #1.2: Type: text/html, Size: 18102 bytes --]

[-- Attachment #2: Type: text/plain, Size: 492 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________