Proxy support for zftp functions

Proxy support for zftp functions

[Andrej Borsenkow]

I'd like to add proxy support to these. Looking around, it looks, like
it enough to add a couple of lines to zfopen - but I'm not sure, if
there can be any side effects (e.g. we set ZFTP_HOST but open real
connection to another host).

Peter, do you envision some problems here? Currently I am interested
only in user@host type of proxy, that is tranparent and should be easy
enough. Other types (with login) may require some more changes
(requesting/keeping proxy password is one).

Of course, some way to set "no proxy" domains is needed.

-andrej

Have a nice DOS!
B >>

Re: Proxy support for zftp functions

[Peter Stephenson]

> I'd like to add proxy support to these. Looking around, it looks, like
> it enough to add a couple of lines to zfopen - but I'm not sure, if
> there can be any side effects (e.g. we set ZFTP_HOST but open real
> connection to another host).

This was on my original plan, but not only didn't I need it myself I didn't
even have a proxy host to try on, so I never thought much about it.
I was actually imagining doing it in the C code.  But if the server is
configured suitably, you ought to get away with it.  The parameters
ZFTP_HOST, ZFTP_USER etc. are readonly, but not special, and are not
checked by the C code, only set by it, so running typeset +r then changing
the variable after the connection is open and a user logged in should be
OK.  I don't actually know how proxies work, having never used one, but if
logging into a remote server involves sending a special username to the
proxy server there shouldn't be a problem.

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

RE: Proxy support for zftp functions

[Andrej Borsenkow]

> This was on my original plan, but not only didn't I need it
> myself I didn't
> even have a proxy host to try on, so I never thought much about it.
> I was actually imagining doing it in the C code.

Yes, it makes more sense. Proxy is relevant only for a logon - then it
is completely transparent (hopefully :-).

Actually, much more interesting (needed) feature is ls parsing.
Currently, zftp relies on MDTM/SIZE to exist - and, unfortunately, they
are missing in "common" Unix servers. Have you ever thought about it?

-andrej

Re: Proxy support for zftp functions

[Peter Stephenson]

> > This was on my original plan, but not only didn't I need it
> > myself I didn't
> > even have a proxy host to try on, so I never thought much about it.
> > I was actually imagining doing it in the C code.
> 
> Yes, it makes more sense. Proxy is relevant only for a logon - then it
> is completely transparent (hopefully :-).

Actually, for just that reason, your idea of doing it in the functions is
probably better.  The less hidden code the more I like it.

> Actually, much more interesting (needed) feature is ls parsing.
> Currently, zftp relies on MDTM/SIZE to exist - and, unfortunately, they
> are missing in "common" Unix servers. Have you ever thought about it?

There's a nice project for someone with a bit of time.  This can probably
be done in shell functions too --- it's system-specific, so not good to
hard code, and the only place this is used internally is for supplying to
functions doing progress reports, which can be rewritten to use information
returned by ls where necessary.

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

RE: Proxy support for zftp functions

[Andrej Borsenkow]

> >
> > Yes, it makes more sense. Proxy is relevant only for a
> logon - then it
> > is completely transparent (hopefully :-).
>
> Actually, for just that reason, your idea of doing it in the
> functions is
> probably better.  The less hidden code the more I like it.
>

May be. For a start function is easier, of course.

> > Actually, much more interesting (needed) feature is ls parsing.
> > Currently, zftp relies on MDTM/SIZE to exist - and,
> unfortunately, they
> > are missing in "common" Unix servers. Have you ever thought
> about it?
>
> There's a nice project for someone with a bit of time.  This
> can probably
> be done in shell functions too --- it's system-specific, so
> not good to
> hard code, and the only place this is used internally is for
> supplying to
> functions doing progress reports, which can be rewritten to
> use information
> returned by ls where necessary.
>

The main use for it is to be able to compare times on remote/local
files. Without it no real directory sync (and even reliable reget) is
possible. Is it possible to pass file date/size from function to zftp.c
code? O.K., file mtime can be set by function as well ...

There are some possibilities:

- use external Perl code. The excellent piece of work is Mirror, that
understands a dozen remote systems (inluding DOS-based FTP server! Wow!
:-) That may do for a start, but thinking about calling Perl every time
I need to parse ls line makes me shudder. What is the status of Perl
module for Zsh? It would make life much easier.

- rewrite Perl modules (system detection + parsing routines) in Zsh.
They are based on regexps, so it is doable (but needs some time, of
course).

If we had working Perl module, I would favour the first approach. No
need to reinvent the wheel.

-andrej

Re: Proxy support for zftp functions

[Peter Stephenson]

> The main use for it is to be able to compare times on remote/local
> files. Without it no real directory sync (and even reliable reget) is
> possible. Is it possible to pass file date/size from function to zftp.c
> code? O.K., file mtime can be set by function as well ...

Checking the time is is only ever done in the shell code; checking the size
is only done by C for passing down to zftp_progress for info, so the only
reason for using C code at all would be to make parsing easier.

But it's certainly not a trivial problem to compare two dates from shell
code, because the information about time zones is hard to get at --- for
exactly that reason, the zftp functions call perl to do it for them (see
the function zfrtime).  I don't have a simple answer to this, but a date
and time parsing and conversion module might be a nice extension --- or, of
course, a piece of shell code that does it neatly, if that's possible.

Come to think of it, there's a hack using the stat module for local files.
% touch foo; stat -s +mtime foo; stat -g +mtime foo
Mon May 22 17:59:38
Mon May 22 16:59:38

(Unfortunately you only get it as text, since the time in seconds since the
epoch is always in GMT.)

And how do you get the GMT modification time for a remote file reliably if
you can't rely on MDTM, anyway?
-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

Re: Proxy support for zftp functions

[Bart Schaefer]

On May 22,  6:04pm, Peter Stephenson wrote:
> Subject: Re: Proxy support for zftp functions
> 
> [...] a date
> and time parsing and conversion module might be a nice extension --- or, of
> course, a piece of shell code that does it neatly, if that's possible.

Here's at least a start at it.  The formats recognized are those typically
found in email Date: headers or as output of `date`; e.g. it makes no
attempt to figure out whether 3/5/7 is March 5, 1907 or May 3, 2007 etc.,
so it simply ignores dates that don't spell out the name of the month.
It also doesn't parse year-month-day (the day is assumed to be the first
number outside an HH:MM[:SS] form) but maybe there's a clever way to fit
that in unambiguously.

This expects to find one or both of the associative arrays $date and $time,
which it fills in the obvious way.  If you don't pre-declare either of them,
you can only check the return value to see whether the parse succeeded.

---- 8< ---- snip ----8< ----
[[ -z "$*" || $ZSH_VERSION < 3.1.6 ]] && return 1

setopt localoptions extendedglob noshwordsplit noksharrays

local HHMMSS='(<->):(<->):#(<->)#'
local DDmmYY='(<->)[-[:space:]]([[:alpha:]]##)[-[:space:]](<->)'
local DDmm='(<->)[[:space:]]([[:alpha:]]##)'
local ZONE='[[:space:]]#([^:[:space:]]##)'

[[ ${(t)date} == association ]] || typeset -A date ; date=()
[[ ${(t)time} == association ]] || typeset -A time ; time=()

set $=*		# Canonicalize whitespace

if [[ "$*" == (#i)(#b)${~DDmmYY}[[:space:]]${~HHMMSS}[[:space:]]#([ap]m)#${~ZONE} ]]
then
    date=(day "$match[1]" month "$match[2]" year "$match[3]")
    time=(hour "$match[4]" minute "$match[5]" second "$match[6]"
	  ampm "$match[7]" zone "$match[8]")
elif [[ "$*" == (#i)(#b)${~DDmm}[[:space:]]${~HHMMSS}[[:space:]]#([ap]m)# ]]
then
    date=(day "$match[1]" month "$match[2]" year "")
    time=(hour "$match[3]" minute "$match[4]" second "$match[5]"
	  ampm "$match[6]" zone "")
elif [[ "$*" == (#i)(#b)${~HHMMSS}[[:space:]]([ap]m)#${~ZONE}[[:space:]]${~DDmmYY} ]]
then
    time=(hour "$match[1]" minute "$match[2]" second "$match[3]"
	  ampm "$match[4]" zone "$match[5]")
    date=(day "$match[6]" month "$match[7]" year "$match[8]")
elif [[ "$*" == (#i)(#b)${~HHMMSS}[[:space:]]([ap]m)#${~ZONE}[[:space:]]${~DDmm}[[:space:]](<->) ]]
then
    time=(hour "$match[1]" minute "$match[2]" second "$match[3]"
	  ampm "$match[4]" zone "$match[5]")
    date=(day "$match[6]" month "$match[7]" year "$match[8]")
elif [[ "$*" == (#i)(#b)([[:alpha:]]##),[[:space:]]#${~DDmmYY}[[:space:]]${~HHMMSS}[[:space:]]#([ap]m)#${~ZONE} ]]
then
    # Day of the week (Mon, Tue, ...) is $match[1]
    date=(day "$match[2]" month "$match[3]" year "$match[4]")
    time=(hour "$match[5]" minute "$match[6]" second "$match[7]"
	  ampm "$match[8]" zone "$match[9]")
elif [[ "$*" == (#i)(#b)([[:alpha:]]##),[[:space:]]#${~DDmm}[[:space:]](<->)[[:space:]]${~HHMMSS}[[:space:]]#([ap]m)#${~ZONE} ]]
then
    # Day of the week (Mon, Tue, ...) is $match[1]
    date=(day "$match[2]" month "$match[3]" year "$match[4]")
    time=(hour "$match[5]" minute "$match[6]" second "$match[7]"
	  ampm "$match[8]" zone "$match[9]")
elif [[ "$*" == (#i)(#b)([[:alpha:]]##)[[:space:]]([[:alpha:]]##)[[:space:]](<->)[[:space:]]${~HHMMSS}[[:space:]]#([ap]m)#${~ZONE}[[:space:]](<->) ]]
then
    # Day of the week (Mon, Tue, ...) is $match[1]
    date=(day "$match[3]" month "$match[2]" year "$match[9]")
    time=(hour "$match[4]" minute "$match[5]" second "$match[6]"
	  ampm "$match[7]" zone "$match[8]")
elif [[ "$*" == (#i)(#b)([[:alpha:]]##)[[:space:]](<->)[[:space:]]${~HHMMSS}[[:space:]]([ap]m)# ]]
then
    date=(day "$match[2]" month "$match[1]" year "")
    time=(hour "$match[3]" minute "$match[4]" second "$match[5]"
	  ampm "$match[6]" zone "")
elif [[ "$*" == ${~DDmmYY} || "$*" == ${~DDmm}[[:space:]](<->) ]]
then
    date=(day "$match[1]" month "$match[2]" year "$match[3]")
    time=(hour "" minute "" second "" ampm "" zone "")
elif [[ "$*" == ${~HHMMSS}[[:space:]]#([ap]m)# ]]
then
    date=(day "" month "" year "")
    time=(hour "$match[1]" minute "$match[2]" second "$match[3]"
	  ampm "$match[4]" zone "")
fi

(( $#date || $#time ))
---- 8< ---- snip ----8< ----

Re: Proxy support for zftp functions

[Bart Schaefer]

Oops, I pasted in the wrong version of this.  Hand-edit to fix the following:

On May 22,  5:19pm, Bart Schaefer wrote:
> 
[...]
> 
> local HHMMSS='(<->):(<->):#(<->)#'
> local DDmmYY='(<->)[-[:space:]]([[:alpha:]]##)[-[:space:]](<->)'
> local DDmm='(<->)[[:space:]]([[:alpha:]]##)'
> local ZONE='[[:space:]]#([^:[:space:]]##)'

local -a match

> [[ ${(t)date} == association ]] || typeset -A date ; date=()
> [[ ${(t)time} == association ]] || typeset -A time ; time=()
> 
[...]
> elif [[ "$*" == ${~DDmmYY} || "$*" == ${~DDmm}[[:space:]](<->) ]]
                 ^                     ^
               (#i)(#b)              (#i)(#b)
> then
>     date=(day "$match[1]" month "$match[2]" year "$match[3]")
>     time=(hour "" minute "" second "" ampm "" zone "")
> elif [[ "$*" == ${~HHMMSS}[[:space:]]#([ap]m)# ]]
                 ^
               (#i)(#b)
> then
>     date=(day "" month "" year "")
>     time=(hour "$match[1]" minute "$match[2]" second "$match[3]"
> 	  ampm "$match[4]" zone "")
> fi
> 
> (( $#date || $#time ))
> ---- 8< ---- snip ----8< ----
>-- End of excerpt from Bart Schaefer