base64 coding for zsh ?

base64 coding for zsh ?

[Marc Chantreux]

Hi all, 

I wrote a function that read ldif entry from a stream and store
attributes in an associative array.

so i can write :

ldapsearch uid=mc | while { ldif_entry_read } {
     for oc (  ${(P)u[objectClass]} ) { print $u[uid] is a $oc }
}

ldif_entry_free u

Now i want my function to deal with base64. The fact is, i don't know
how to do it.

Someone can help ? 

regards,
mc

-- 
téléphone : 03.90.24.00.19
courriel  : marc.chantreux@ulpmm.u-strasbg.fr
---------------------------------------

Re: base64 coding for zsh ?

[Bart Schaefer]

On Sep 12,  5:08pm, Marc Chantreux wrote:
}
} Now i want my function to deal with base64. The fact is, i don't know
} how to do it.
} 
} Someone can help ? 

It's about 50 lines of readably-formatted C code to write a base64
decoder, slightly more than that to write the encoder, not counting the
lookup tables. It could be done in shell script, but it would be very
slow and probably not worth the effort.

alias enB64="perl -MMIME::Base64 -e 'print encode_base64(shift @ARGV)'"
alias deB64="perl -MMIME::Base64 -e 'print decode_base64(shift @ARGV)'"

enB64 abcdef
deB64 YWJjZGVm

Re: base64 coding for zsh ?

[Marc Chantreux]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 886 bytes --]

le 12/09/2006,
Bart Schaefer nous écrivait :
> It's about 50 lines of readably-formatted C code to write a base64
> decoder,

so i'll write my first zsh module. It's added to my todo list. 

> slightly more than that to write the encoder, not counting the
> lookup tables. It could be done in shell script, but it would be very
> slow and probably not worth the effort.

ok, i give this solution away.

> alias enB64="perl -MMIME::Base64 -e 'print encode_base64(shift @ARGV)'"
> alias deB64="perl -MMIME::Base64 -e 'print decode_base64(shift @ARGV)'"

well ... i think perl is quite huge for this problem. I'll use openssl
as temporary hack.

alias B64enc='openssl base64'
alias B64dec='openssl base64 -d'

thanks for help.

regards.

-- 
téléphone : 03.90.24.00.19
courriel  : marc.chantreux@ulpmm.u-strasbg.fr
---------------------------------------

Re: base64 coding for zsh ?

[Brian K. White]

----- Original Message ----- 
From: "Marc Chantreux" <marc.chantreux@ulpmm.u-strasbg.fr>
To: "zsh-users" <zsh-users@sunsite.dk>
Sent: Wednesday, September 13, 2006 11:08 AM
Subject: Re: base64 coding for zsh ?


le 12/09/2006,
Bart Schaefer nous crivait :
> It's about 50 lines of readably-formatted C code to write a base64
> decoder,

so i'll write my first zsh module. It's added to my todo list.

> slightly more than that to write the encoder, not counting the
> lookup tables. It could be done in shell script, but it would be very
> slow and probably not worth the effort.

ok, i give this solution away.

> alias enB64="perl -MMIME::Base64 -e 'print encode_base64(shift @ARGV)'"
> alias deB64="perl -MMIME::Base64 -e 'print decode_base64(shift @ARGV)'"

well ... i think perl is quite huge for this problem. I'll use openssl
as temporary hack.

alias B64enc='openssl base64'
alias B64dec='openssl base64 -d'

thanks for help.

regards.

------------

I've been using the stand-alone:
http://www.fourmilab.ch/webtools/base64/

I never realised I already had it in openssl.
Thanks for the tip.

This led me to see what else I might already have or could get in a ready 
made package rather than having to use a source.
(I do need this util in some form on every box and previously all my boxes 
were SCO OpenServer and I had been using the fourmilab source for years.)

Then I decided to run a few simple time tests, since this gets used a lot in 
various system() commands in application code and cgi scripts etc...
I expected the simple plain base64 util to be the fastest but I was wrong:
In each case I repeated the same command many times and always got almost 
exactly the same results, that is, +- 0m0.004 of the numbers shown so these 
are representative not just a fluke or caching effects or effects of the os 
being busy elsewhere.

the fourmilab source:
nj4:~ # time base64 </boot/vmlinuz >/dev/null

real    0m0.074s
user    0m0.072s
sys     0m0.000s

openssl:
nj4:~ # time opsnssl base64 </boot/vmlinuz >/dev/null
-bash: opsnssl: command not found

real    0m0.001s
user    0m0.000s
sys     0m0.000s

mimencode comes in the metamail package:
nj4:~ # time mimencode </boot/vmlinuz >/dev/null

real    0m0.078s
user    0m0.076s
sys     0m0.004s


gmime-uuencode:
It actually does base64 not uuencode if you give it the -m or --base64 
option.
It comes in the gmime package.
It prepends and appends some junk to the actual base64 output so it's 
inconvenient for me to actually use:
    nj4:~ # echo this is a test |gmime-uuencode -m -
    begin-base64 600 -
    dGhpcyBpcyBhIHRlc3QK
    ====
    nj4:~ #
As for the speed:
nj4:~ # time gmime-uuencode -m - </boot/vmlinuz >/dev/null

real    0m0.009s
user    0m0.008s
sys     0m0.000s

Ooenssl destroys the rest.
So even though I have the dedicated util it's actually better to use 
openssl.
As a side benefit, thats one less special thing to maintain on all my boxes.


On a related note. I had/have a need for a standalone url encoder/decoder 
and made one myself, then receive the help of someone else to make it into a 
proper util with man page and getopts() and basic sanity checking etc... The 
source is here. I never bothered to make linux binaries except for myself 
yet but it's such basic code it builds with no problems anywhere.
http://www.aljex.com/bkw/sco/#urldec

Does anyone know of an already existing util to do that?
I can't imagine how something so basic can be made 78 times faster but I 
would have said the same thing about base64 until now also.
Note: I'd like it to work as a filter like cat, like all the utils above.
In fact, my util has options to do both, work as a filter or take a string 
on the command line.
Thanks.

btw, compared to above, ick...
nj4:~ # time urlenc </boot/vmlinuz >/dev/null

real    0m0.332s
user    0m0.332s
sys     0m0.000s


Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!

Re: base64 coding for zsh ?

[Brian K. White]

----- Original Message ----- 
From: "Danek Duvall" <duvall@comfychair.org>
To: "Brian K. White" <brian@aljex.com>
Sent: Wednesday, September 13, 2006 8:33 PM
Subject: Re: base64 coding for zsh ?


>> openssl:
>> nj4:~ # time opsnssl base64 </boot/vmlinuz >/dev/null
>> -bash: opsnssl: command not found
>>
>> real    0m0.001s
>> user    0m0.000s
>> sys     0m0.000s
>>
>> Ooenssl destroys the rest.
>
> Probably because "opsnssl" is not found, so it returns really quickly. 
> You
> might fix the typo first.  :)
>
> Danek


wow. um. yeah ok then.

.... and it still wins

nj4:~ # time openssl base64 </boot/vmlinuz >/dev/null

real    0m0.014s
user    0m0.016s
sys     0m0.000s

Yes its really there really doing it's thing:
nj4:~ # echo this is a test |openssl base64
dGhpcyBpcyBhIHRlc3QK
nj4:~ #

So it's only 5 x faster instead of 78 x :)

Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!

Re: base64 coding for zsh ?

[Marc Chantreux]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 602 bytes --]

le 13/09/2006,
Brian K. White nous écrivait :
> Sent: Wednesday, September 13, 2006 11:08 AM
> alias B64enc='openssl base64'
> alias B64dec='openssl base64 -d'
> 
> I've been using the stand-alone:
> http://www.fourmilab.ch/webtools/base64/
> 
> I never realised I already had it in openssl.
> Thanks for the tip.

I didn't know that before yesterday but it was a bad idea in fact :
openssl base64 doesn't handle utf8 encoded content. So i found recode
command. 

finaly :

alias b64dec='recode b64..data'
alias b64enc='recode data..b64'

benches are welcome ;)

regards,
mc

Re: base64 coding for zsh ?

[Bart Schaefer]

On Sep 14,  4:50pm, Marc Chantreux wrote:
}
} openssl base64 doesn't handle utf8 encoded content.

That sentence *shouldn't* make any sense.  The input to base64 encoding
is a raw byte stream, and the output is ascii.  Conversely, the input
to base64 decoding is ascii, and the output is raw bytes.  Any other
encoding shouldn't enter into the process at all.

In what sense does "handle utf8" matter?

Re: base64 coding for zsh ?

[Brian K. White]

----- Original Message ----- 
From: "Marc Chantreux" <marc.chantreux@ulpmm.u-strasbg.fr>
To: <zsh-users@sunsite.dk>
Sent: Thursday, September 14, 2006 10:50 AM
Subject: Re: base64 coding for zsh ?


le 13/09/2006,
Brian K. White nous crivait :
> Sent: Wednesday, September 13, 2006 11:08 AM
> alias B64enc='openssl base64'
> alias B64dec='openssl base64 -d'
> 
> I've been using the stand-alone:
> http://www.fourmilab.ch/webtools/base64/
> 
> I never realised I already had it in openssl.
> Thanks for the tip.

I didn't know that before yesterday but it was a bad idea in fact :
openssl base64 doesn't handle utf8 encoded content. So i found recode
command. 

finaly :

alias b64dec='recode b64..data'
alias b64enc='recode data..b64'

benches are welcome ;)

---------

Not good but not horrible.
Consistently slower than everything else but only microscopically.
Even though it's the slowest of all, you still have to call it "fast".

nj4:~ # time recode data..b64 </boot/vmlinuz >/dev/null

real    0m0.086s
user    0m0.080s
sys     0m0.008s
nj4:~ #

Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!