From mboxrd@z Thu Jan  1 00:00:00 1970
From: wkt@tuhs.org (Warren Toomey)
Date: Thu, 18 Sep 2003 21:45:26 +1000
Subject: [TUHS] Lexical comparator
In-Reply-To: <200309181041.h8IAfAWe000686@skeeve.com>
References: <200309181041.h8IAfAWe000686@skeeve.com>
Message-ID: <20030918114526.GA54312@minnie.tuhs.org>

On Thu, Sep 18, 2003 at 01:41:10PM +0300, Aharon Robbins wrote:
> > If anybody has Unix kernel trees which they cannot divulge due to licensing
> > restrictions, I'd appreciate you creating tokenised files of the kernel
> > source and e-mailing them to me.
> 
> Hmmm.  Just between us chickens, given tokenized versions of an entire tree,
> how hard would it be to recreate a functional kernel?

Pretty damn hard. All identifiers, (variable names) are reduced to
a single token. Actually, that's not true. The meaning of the names
is removed an replaced with numeric identifiers that are unique to
each file. Here's a tokenised portion of 32V (bio.c):

   56:   struct id10 * 
   57:   id13 ( id14 , id15 ) 
   58:   id16 id14 ; 
   59:   id17 id15 ; 
   60:   { 
   61:   register struct id10 * id18 ; 
   62:   
   63:   id18 = id19 ( id14 , id15 ) ; 
   64:   if ( id18 ->id20 & id21 ) { 
   65:   #ifdef id1 
   66:   id9 . id5 ++ ; 
   67:   #endif 
   68:   return( id18 ) ; 
   69:   } 
   70:   id18 ->id20 |= id22 ; 
   71:   id18 ->id23 = id24 ; 
   72:   ( * id25 [ id26 ( id14 ) ] . id27 ) ( id18 ) ; 
   73:   #ifdef id1 
   74:   id9 . id3 ++ ; 
   75:   #endif 
   76:   id28 ( id18 ) ; 
   77:   return( id18 ) ; 
   78:   } 

Now go and check the actual source and work out which function it is!
[ see http://minnie.tuhs.org/UnixTree/32VKern/usr/src/sys/sys/bio.c.html ]

	Warren