RFT: Request for a trick :O)

RFT: Request for a trick :O)

[Meino Christian Cramer]

Hi,

 I read about the magic of MULTIOS in the zshall manpage and I am
 wondering whether it is possible to achieve the following without
 using "the standard way" (i.e. visible temporyry files)...

 In my directory there are two files:

	a.7z

 and

    b.7z

 with both the same length. Both are packed with the p7zip-packer (by
 the way: The compression ratio of this tool is even better as that of
 bzip2 and it is available for Mi*ft Wi*ows and Linux/UNIX!). 

 A md5sum shows, that both files are "different", but it seems this is
 due to the packer, which (may be) include the packing date. So md5sum
 returns different hash value even if the contents of the archives is
 identical.

 The standard way to check both archives for identity is to unpack
 both into temporary files, md5sum both files, compare the hash
 values and at last remove both temporary files.

 In the zshall I saw constructions (if MULTIOS is set) like

	sort < foo < bar

 which have the same result as:

    cat foo bar | sort

 .

 Hmmmm.....

 May be something like

	 cmp < `7z x -so foo.7z` < `7z x -so bar.7z`

 may work I thought...but it doesnt (the assumption was totally
 wrong)...  ( "7z x -so <file>.7z" means "unpack <file>.7z to stdout)

 But...may be someone else knows a tricky way to prevent temporary
 files to compare the (single file) contents of two archives in one
 go???

 Keep zshing!
 Meino


 

Re: RFT: Request for a trick :O)

[Michal Politowski]

On Sat,  6 Aug 2005 06:56:38 +0200, Meino Christian Cramer wrote:
[...]
> 	 cmp < `7z x -so foo.7z` < `7z x -so bar.7z`
> 
>  may work I thought...but it doesnt (the assumption was totally
>  wrong)...  ( "7z x -so <file>.7z" means "unpack <file>.7z to stdout)
> 
>  But...may be someone else knows a tricky way to prevent temporary
>  files to compare the (single file) contents of two archives in one
>  go???

Almost there. Read about process substitution in the documentation.
It looks like <(command), >(command) and =(command)
so in your case
cmp <(7z x -so foo.7z) <(7z x -so bar.7z)
should work.

-- 
Michał Politowski
Talking has been known to lead to communication if practiced carelessly.

Re: RFT: Request for a trick :O)

[Meino Christian Cramer]

From: Michal Politowski <mpol@charybda.icm.edu.pl>
Subject: Re: RFT: Request for a trick :O)
Date: Sat, 6 Aug 2005 09:50:40 +0200

> On Sat,  6 Aug 2005 06:56:38 +0200, Meino Christian Cramer wrote:
> [...]
> > 	 cmp < `7z x -so foo.7z` < `7z x -so bar.7z`
> > 
> >  may work I thought...but it doesnt (the assumption was totally
> >  wrong)...  ( "7z x -so <file>.7z" means "unpack <file>.7z to stdout)
> > 
> >  But...may be someone else knows a tricky way to prevent temporary
> >  files to compare the (single file) contents of two archives in one
> >  go???
> 
> Almost there. Read about process substitution in the documentation.
> It looks like <(command), >(command) and =(command)
> so in your case
> cmp <(7z x -so foo.7z) <(7z x -so bar.7z)
> should work.
> 
> -- 
> Michał Politowski
> Talking has been known to lead to communication if practiced carelessly.
> 

Hi Michael !

 Thanks for your reply :O)

 The following line:

     cmp <(7z x -so render/GlassBowl001.blend.7z ) <(7z x -so  render2/GlassBowl001.blend.7z )

 gaves:


    7-Zip 4.20  Copyright (c) 1999-2005 Igor Pavlov  2005-05-30
    p7zip Version 4.20 (locale=C,Utf16=off,HugeFiles=on)
    
    Processing archive: render/GlassBowl001.blend.7z
    
    
    7-Zip 4.20  Copyright (c) 1999-2005 Igor Pavlov  2005-05-30
    p7zip Version 4.20 (locale=C,Utf16=off,HugeFiles=on)
    
    Processing archive: render2/GlassBowl001.blend.7z
    cmp (GNU diffutils) 2.8.1
    Copyright (C) 2002 Free Software Foundation, Inc.
    
    This program comes with NO WARRANTY, to the extent permitted by law.
    You may redistribute copies of this program
    under the terms of the GNU General Public License.
    For more information about these matters, see the file named COPYING.
    
    Written by Torbjorn Granlund and David MacKenzie.
    mccramer@solfire:.process> 
    Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5488 Broken pipe             /usr/lib/p7zip/7z "$@"
    Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5492 Broken pipe             /usr/lib/p7zip/7z "$@"

 

 Both files exists, the command was submitted being in the correct cwd
 and the files are processable by 7z (no corruption etc...)...

 What's going wrong here ?

 Keep zshing!
 Meino

Re: RFT: Request for a trick :O)

[Peter Stephenson]

Meino Christian Cramer wrote:
>  The following line:
> 
>      cmp <(7z x -so render/GlassBowl001.blend.7z ) <(7z x -so  render2/Glas=
> sBowl001.blend.7z )
> 
>  gaves:
>     Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5488 Broken pipe
>           /usr/lib/p7zip/7z "$@"
>     Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5492 Broken pipe
>           /usr/lib/p7zip/7z "$@"

Not sure this is the error, but: it's quite likely cmp needs to be able
to seek backwards. <(...) is often implemented using a pipe on which you
can't do that.  (Results may vary, so you can't assume it will always
work from the behaviour on one system.)

The =(...) substitution exists for this very reason.

  cmp =(output1...) =(output2...)

I'm not sure this is the error because I would expect a slightly
different error message, but it's worth trying.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************

Re: RFT: Request for a trick :O)

[Stephane Chazelas]

On Mon, Aug 08, 2005 at 11:04:14AM +0100, Peter Stephenson wrote:
> Meino Christian Cramer wrote:
> >  The following line:
> > 
> >      cmp <(7z x -so render/GlassBowl001.blend.7z ) <(7z x -so  render2/Glas=
> > sBowl001.blend.7z )
> > 
> >  gaves:
> >     Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5488 Broken pipe
> >           /usr/lib/p7zip/7z "$@"
> >     Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5492 Broken pipe
> >           /usr/lib/p7zip/7z "$@"
> 
> Not sure this is the error, but: it's quite likely cmp needs to be able
> to seek backwards. <(...) is often implemented using a pipe on which you
> can't do that.  (Results may vary, so you can't assume it will always
> work from the behaviour on one system.)
[...]

I think it's rather because cmp stops reading at the first
difference -> so exit -> so close the pipes -> so the feeders
get a SIGPIPE.

That's normal and expected, you'd get the same with:

7z ... | cmp - other-file

-- 
Stephane

Re: RFT: Request for a trick :O)

[Peter Stephenson]

Stephane Chazelas wrote:
> > >     Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5488 Broken pipe
> > >           /usr/lib/p7zip/7z "$@"
> > >     Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5492 Broken pipe
> > >           /usr/lib/p7zip/7z "$@"
> > 
> 
> I think it's rather because cmp stops reading at the first
> difference -> so exit -> so close the pipes -> so the feeders
> get a SIGPIPE.
> 
> That's normal and expected, you'd get the same with:
> 
> 7z ... | cmp - other-file

You're right.  I'd assumed the files were actually identical, but reading
back I see Meino did say they weren't.  The "broken pipe" is simply a
detail of the implementation of the <( ... ).

Meino, if you want a full comparison to the end of the file, you can
use "cmp -l", which will give lots of output.  Otherwise, you can redirect
2>/dev/null in the p7zip command.

pws


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************

Re: RFT: Request for a trick :O)

[Meino Christian Cramer]

From: Peter Stephenson <pws@csr.com>
Subject: Re: RFT: Request for a trick :O) 
Date: Mon, 08 Aug 2005 11:41:29 +0100

Hi,

 thanks for all the help ! :)

 I did not need the exact position and byte counts of differing bytes
 at all. I only need a "yes" or "no" whether I have identical file
 (and can delete the doublettes).

 I saw this "broken pipe"-thingy and didnt dare to delete files on the
 base of this...I thought I did something very wrong (I amstill a very
 new zsher ;).

 "Broken pipe" means: different files.
 
 Ok...if my memory serves me right...the return status of the last
 command in a pipe is the return status of the whole pipe, isn't it ?

 But unfortunately, the cmp is the first command in that construction... 

 I think with something like  

   function zipcmp ()
   {
	  	cmp <(7z x -so $1 2>/dev/null ) <(7z x -so $2 2>/dev/null )		
   }
   
   if [ ! zipcmp $1 $2 ]
   then
      rm -f $1
   fi

 I mostly would get an empty harddisc.... ;)

 How can I get the return code of cmp, if <( is implemented as a pipe
 ?

 Keep zshing!
 Meino
 

> Stephane Chazelas wrote:
> > > >     Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5488 Broken pipe
> > > >           /usr/lib/p7zip/7z "$@"
> > > >     Extracting  GlassBowl001.blend/usr/bin/7z: line 2:  5492 Broken pipe
> > > >           /usr/lib/p7zip/7z "$@"
> > > 
> > 
> > I think it's rather because cmp stops reading at the first
> > difference -> so exit -> so close the pipes -> so the feeders
> > get a SIGPIPE.
> > 
> > That's normal and expected, you'd get the same with:
> > 
> > 7z ... | cmp - other-file
> 
> You're right.  I'd assumed the files were actually identical, but reading
> back I see Meino did say they weren't.  The "broken pipe" is simply a
> detail of the implementation of the <( ... ).
> 
> Meino, if you want a full comparison to the end of the file, you can
> use "cmp -l", which will give lots of output.  Otherwise, you can redirect
> 2>/dev/null in the p7zip command.
> 
> pws
> 
> 
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
> 
> **********************************************************************
> 

Re: RFT: Request for a trick :O)

[Peter Stephenson]

Meino Christian Cramer wrote:
>  How can I get the return code of cmp, if <( is implemented as a pipe
>  ?

You may be running across a problem that someone else did recently,
that if a program running in the shell gets a SIGPIPE the shell pretends
it did when it's running interactively, and so aborts.  This isn't
particularly useful and we changed it on the head of the main line.
However, I can't get it to happen at all at the moment.

If that's not the problem (the shell is not aborting at the cmp), please
say what is.

pws


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************

Re: RFT: Request for a trick :O)

[Meino Christian Cramer]

From: Peter Stephenson <pws@csr.com>
Subject: Re: RFT: Request for a trick :O) 
Date: Mon, 08 Aug 2005 17:02:29 +0100

Hi,

 ...oh sorry..I think I have caused more confusion as I had
 intended...:)

 The script was speculativ (and buggy)...just a thought written down
 in an email.

 The corrected script is:

    #!/bin/zsh
    function zipcmp ()
    {
     cmp -s <(7z x -so $1 2>/dev/null ) <(7z x -so $2 2>/dev/null )    
    }
       
    
    if zipcmp $1 $2
    then
     echo "$1 and $2 are identical..."
    fi

 which works fine for me.

 When I will struggle again about similiar things, an email will be on
 the way... :O)

 If you find any bug or things to improve in the above script, please
 email me...I will use this script as a base for a
 compare-files-and-remove-doubles-script...


 Happy zshing!
 Meino



 

> Meino Christian Cramer wrote:
> >  How can I get the return code of cmp, if <( is implemented as a pipe
> >  ?
> 
> You may be running across a problem that someone else did recently,
> that if a program running in the shell gets a SIGPIPE the shell pretends
> it did when it's running interactively, and so aborts.  This isn't
> particularly useful and we changed it on the head of the main line.
> However, I can't get it to happen at all at the moment.
> 
> If that's not the problem (the shell is not aborting at the cmp), please
> say what is.
> 
> pws
> 
> 
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
> 
> **********************************************************************
>