[9fans] source code as data not text

[9fans] source code as data not text

[Matt]

Maybe I'm not hard-core enough but one thing I miss is syntax highlighting.

And if you go so far as to parse the source code in the editor then you can
use the opportunity to develop the idea.

There are some good ideas in a paper here http://mindprod.com/scid.html
about source code living in a database rather than a plain text file.

The basic premise is that when an editor knows about what source code is and
does it can offer new views of it etc.

Anyone got any views of their own?

matt

Re: [9fans] source code as data not text

[Laura Creighton]

The problem with putting source code in a database is that you then need
complicated database tools to read your source code.  Then it is harder
to use pipes and shellscripts.  When David Slocum, Aaron Markus, somebody
whose name I forget did a study of bit-map fonts and their effect
on the comprehension of C code for a DARPA grant we discovered that it
was much better to build a parser in a displaying tool that you could
make display plain ascii any way you like by some sort of parsing rule,
as opposed to making the source code itself complicated.

In our efforts to learn this we discovered that it is very tough on your
experiemental method to have different versions of source code that are
supposed to be identical except for formatting details.  They will vary.
We wasted a fair bit of time proving conclusively that students who
are new to C have a hard time understanding it when you have left out
the odd line or character by mistake, and the like.

Things worked better when the person whose name I can't remember and
I hacked the C pre-processor and the compiler to produce, instead of
machine code, bit maps (which was radical new technology at the time)
which we could then baffle more students with.

There is also the helpless factor.  On occasion I have had to write
C code in MS word.  It goes very badly.  I very much feel a helpless
prisoner while this is going on; locked inside layers and layers of
gorp which contribute absolutely nothing towards my main goal, making
the machine do what I want it to do.  It makes writing code hard for me.

Syntax highlighting does catch errors.  Does it make you lazier about not
making errors in the first place?  How can we test this hypothesis?  I
know people who never used line editors like ed extensively are 
surprised when old ed hackers sit down and write 20, 50, 100 lines of
code, one line right after each other, without going back and fiddling with
bits.  If you started writing programs when there was no cursor addressing,
or screen editors (or they were banned because of the load they 
put on those old machines) this is no trick at all.

Laura

ps. we are all safe and sound here.  We should all be cleaned up soon.
But I never made it to the used computer store; Saturday's revised
demonstration path made to miss the trashed out area went right down the
street where I live.  Thanks for all the support.

Re: [9fans] source code as data not text

[Matt]

bah, my fault but I wasn't really thinking about the database side of
things. Changing the file format is definately a no-no.

What sparked my interest was making the editor do more work.
Maybe it's because I write code that writes code and having string literals
in a different colour from the source code helps me see.

I did like VB's auto parsing if input code. Highlighting a sections that are
stynax errors etc. must reduce the cycle?

I'm glad I asked here though. Can't think of a better bunch of people to run
it past.

matt

RE: [9fans] source code as data not text

[Richard Elberger]

In my opinion, you're on the right track.

As for databases storing code, I think it is good.  I disagree with Laura's
shell script (etc) theory -- I have a hard time believing she works in
anything else but an academic environment -- I have been doing release
management for a long time and I would rather have code stored in a
clustered rdbms that is accessed through its own shell [that is
cross-platform like limbo] than relying on shell scripts, etc, that aren't
cross-platform.  In fact, you'd be surprised how many big software companies
are already moving that way with internal teams building such change
management systems.

2p.

-- rich

>-----Original Message-----
>From: 9fans-admin@cse.psu.edu [mailto:9fans-admin@cse.psu.edu]On Behalf
>Of Matt
>Sent: Monday, 18 June 2001 9:14 p.m.
>To: 9fans@cse.psu.edu
>Subject: Re: [9fans] source code as data not text
>
>
>bah, my fault but I wasn't really thinking about the database side of
>things. Changing the file format is definately a no-no.
>
>What sparked my interest was making the editor do more work.
>Maybe it's because I write code that writes code and having string literals
>in a different colour from the source code helps me see.
>
>I did like VB's auto parsing if input code. Highlighting a
>sections that are
>stynax errors etc. must reduce the cycle?
>
>I'm glad I asked here though. Can't think of a better bunch of
>people to run
>it past.
>
>matt
>
>
>
>
>
>

Re: [9fans] source code as data not text

[Dan Cross]

In article <200106180852.KAA04199@boris.cd.chalmers.se> you write:
>
>  [...]
>
>Syntax highlighting does catch errors.  Does it make you lazier about not
>making errors in the first place?  How can we test this hypothesis?  I
>know people who never used line editors like ed extensively are 
>surprised when old ed hackers sit down and write 20, 50, 100 lines of
>code, one line right after each other, without going back and fiddling with
>bits.  If you started writing programs when there was no cursor addressing,
>or screen editors (or they were banned because of the load they 
>put on those old machines) this is no trick at all.

I think that there has to be a break-even point.  Before that point,
the tools are too difficult, thus too restricting.  Afterwards, the
tools are too easy and make one lazy.  Does syntax highlighting help?
I don't think so; in my experience, programmers who use syntax
highlighting editors don't turn out code as good as that produced by
programmers who use a non-syntax highlighting editor, and they
certainly don't do it as quickly.  On the other hand, I've programmed
using ed, and I don't particularly want to go back (though it's really
handy for some things!).

Regarding using a database to store code, please don't confuse the term
``database'' with ``relational database.''  Storing code in an RDBMS
certainly doesn't buy you anything, as code doesn't map well to the
relational model (though I suppose one could argue that the relational
model could implicitly encapsulate the functionality of mk and its ilk
without additional code).  An object database with versioning, on the
other hand, might be used to house a repository of code, but that's not
the issue at hand; the issue is a programmer editting within the
context of a database at all times.  I don't think this is a good idea,
if for no other reason than yes, it defies the tool-based approach.
Specifically, as Nigel pointed out, I'm forced into using a vendors
proprietary tools to edit my source.  Bye bye acme?  No thanks.

Besides, tools like CVS, Perforce, etc, already do what the object
database would do for me, but do so using normal text files (okay;
perforce uses a database on the ``back end'').  I use these things
in industry all the time (despite my email address :-), and they
can come in really handy.  However, I almost never use a syntax
highlighting editor (unless I'm doing something really repeditive),
and I never use a database to store code in my directory.

I do use shell scripts and text-based tools all the time.  Good God,
I'd cut off my hands and stop programming forever if I couldn't.

	- Dan C.

Re: [9fans] source code as data not text

[Boyd Roberts]

> Maybe I'm not hard-core enough but one thing I miss is syntax highlighting.

i think this is nonsense, but it makes me think that maybe you could
have a database of buggy code that you had written and it could be used
to warn you of your folies.

Re: [9fans] source code as data not text

[nigel]

[-- Attachment #1: Type: text/plain, Size: 2439 bytes --]

>> The basic premise is that when an editor knows about what source code is and
>> does it can offer new views of it etc.
>> 
>> Anyone got any views of their own?

Yes.

It's not a matter of being hard-core. The fundamental premise is that
text is the only universal format. If you keep the 'data' in a 'database',
then how do other tools reason on it? Only if they can read the format,
of course. This tends to mean, "only if they're from the same vendor".
Keeping your 'data' in a universally agreed format is important. I would
contend that text is the only one for source code.

Having structural information is important, but the common format
must not thrown away in pursuit of it. The concept is in direct
opposition to the toolset principle.

But, in this case, the guy is actually promoting strict syntax
directed editing, a well-worn-out concept. Under the section "Real
World SCID Implementations", he says

	The usual reaction I get from programmers when I mention SCIDs is that
	they have tried them and they hate them.  What they have tried are
	coding templates where you fill in the blanks.  These stop you from
	coding in the old way, yet offer almost no payback.  Granted SCIDs
	will force you to rethink how you compose programs.  Code must at all
	times be 100% syntactically correct.  However, a good SCID will pay
	back 100 fold for this inconvenience.  If you try to import or paste
	code that is not correct, you will find much of it being turned into a
	special kind of comment

'Nuff said. You really don't want it. The last thing you want when in the
middle of creating something is to have an editor forcing you to dot
the I's and cross the t's.

Composition of programs is not linear, in exactly the same way as a
novelist might be writing the introduction, the guts, and the end all
at the same time.  So even if syntactically correct entry is an
improvement over coding templates, they both force an uncomfortable
approach which strangles creativity.  The programmers aren't wrong,
there is plenty of research to support this, and he shouldn't be so
dismissive.

Every so often someone ressurects this idea.  In this case, despite
his claim to have been promoting it since the 70's, he doesn't mention
any of the classic research on the subject, which started at least as
early as the 70's.

Try reading papers on the Gandalf project as a starting point.


[-- Attachment #2: Type: message/rfc822, Size: 2009 bytes --]

From: "Matt" <matt@proweb.co.uk>
To: <9fans@cse.psu.edu>
Subject: [9fans] source code as data not text
Date: Mon, 18 Jun 2001 01:31:13 +0100
Message-ID: <00d701c0f78d$fb26d750$6401a8c0@freeze2k>

Maybe I'm not hard-core enough but one thing I miss is syntax highlighting.

And if you go so far as to parse the source code in the editor then you can
use the opportunity to develop the idea.

There are some good ideas in a paper here http://mindprod.com/scid.html
about source code living in a database rather than a plain text file.

The basic premise is that when an editor knows about what source code is and
does it can offer new views of it etc.

Anyone got any views of their own?

matt

Re: [9fans] source code as data not text

[Laura Creighton]

Cutting and pasting code is plain evil.  If you need it twice, you
need a library function, or a template, I suppose, though I 
dislike templates for other reasons.  Any program that attempts to
help you by making sure that you cannot cut and paste things in that
are not syntactically correct is doing you no favour.  As you change
things to make the program happier, you are producing two slightly
different versions of essentially the same code -- precisely the evil
you need most to avoid.  I have no idea why close to every human
being on the planet feels the compelling need to write their own 
strcmp, with their own unique bugs in them, but the problem is that
everybody is out reinventing the square wheel, not that the syntax
in some of them is questionable.

Laura

Re: [9fans] source code as data not text

[Boyd Roberts]

> Cutting and pasting code is plain evil.

you said it.

back in paris and i have eurocave catalogue.

that stuff is is _way cool_.

Re: [9fans] source code as data not text

[Boyd Roberts]

one of the problems is when it's so easy to snarf
paste, coding it into a loop doesn't enter into it.

Re: [9fans] source code as data not text

[nigel]

[-- Attachment #1: Type: text/plain, Size: 46 bytes --]

Oh, and the link to SeeSoft is broken too.


[-- Attachment #2: Type: message/rfc822, Size: 2009 bytes --]

From: "Matt" <matt@proweb.co.uk>
To: <9fans@cse.psu.edu>
Subject: [9fans] source code as data not text
Date: Mon, 18 Jun 2001 01:31:13 +0100
Message-ID: <00d701c0f78d$fb26d750$6401a8c0@freeze2k>

Maybe I'm not hard-core enough but one thing I miss is syntax highlighting.

And if you go so far as to parse the source code in the editor then you can
use the opportunity to develop the idea.

There are some good ideas in a paper here http://mindprod.com/scid.html
about source code living in a database rather than a plain text file.

The basic premise is that when an editor knows about what source code is and
does it can offer new views of it etc.

Anyone got any views of their own?

matt

Re: [9fans] source code as data not text

[anothy]

//I have no idea why close to every human being on the planet
//feels the compelling need to write their own  strcmp...

i think i could dig up a _few_ people who've never expressed any
desire to write a strcmp. ☺
-α.

Re: [9fans] source code as data not text

[Barry]

>//I have no idea why close to every human being on the planet
>//feels the compelling need to write their own  strcmp...
>
>i think i could dig up a _few_ people who've never expressed any
>desire to write a strcmp. 
>-.

... But I HAD to write my own strcmp(). No one understands me.
"A word means exactly what I say it means." (misquoted from
_Through the Looking Glass_ (?) by Lewis Carrol.


==================================================================
  Once a proud programmer of Apple II computers, he now spends his days
     and nights in cheap dives fraternizing with exotic dancers.

RE: [9fans] source code as data not text

[anothy]

//In my opinion, you're on the right track.

so how do you deal with the concerns raised? for one, the probability that a
database-based source code repository would lock you into a specific vendor?
and while i'm not entirely clear on what exactly lac was refering to, can you
deny the usefulness of being able to "grep ^foo(" in code, or suggest that a
databasse system would make that easier somehow? again, as noted earlier,
db-based source code systems are another failure to use the little-tools
model appropriatly.

and as dan points out, SCCS/CVS style stuff is not what's at issue here.

//In fact, you'd be surprised how many big software companies are already
//moving that way...

much to my dismay, i believe you. but then, i'm often suprised how many big
companies are doing _lots_ of things i consider stupid. PeopleSoft, anyone?

the most involved syntax highlighting i find useful is paren/bracket/quote/&c
matching. i could imagine an editor that did that well (acounting for the
language-specific escapes and such) being quite nice. more than that i've not
found useful, and it's distracting and damaging to people who're learning.
and what acme's got now is good 95 times out of 100.
-α.

RE: [9fans] source code as data not text

[Richard Elberger]

>and while i'm not entirely clear on what exactly lac was refering 
>to, can you
>deny the usefulness of being able to "grep ^foo(" in code, or 
>suggest that a
>databasse system would make that easier somehow? again, as noted earlier,
>db-based source code systems are another failure to use the little-tools
>model appropriatly.

select .. from .. where ..

What's the difference?  The only difference I see is every time you grep you do an fopen/fclose.  With databases, information can even be cached, and can perhaps be "smarter" through cross-referencing.  While this is a generalization on a massive scale, I think it is comparable.

>
>and as dan points out, SCCS/CVS style stuff is not what's at issue here.
>

But it is -- editing, checking in/out, collaborating, etc.  To just have another editor is short-sighted.  Imo, Rational Software almost has it 100% right -- they have process, tools, and repositories integrated with 3rd party products.  They just don't have the full-blown integration/collaboration that's needed.  

>//In fact, you'd be surprised how many big software companies are already
>//moving that way...
>
>much to my dismay, i believe you. but then, i'm often suprised how many big
>companies are doing _lots_ of things i consider stupid. PeopleSoft, anyone?

But people still buy their software ... why?  Perhaps to the customers it adds value.  In any case, internal process is different than how products realize the customer's need.  Maybe your comment is about internal-Peoplesoft, which I know nothing about...

>
>the most involved syntax highlighting i find useful is 
>paren/bracket/quote/&c
>matching. i could imagine an editor that did that well (acounting for the
>language-specific escapes and such) being quite nice. more than 
>that i've not
>found useful, and it's distracting and damaging to people who're learning.
>and what acme's got now is good 95 times out of 100.
>-α.
>
>

In your particular case... acme still tires me out. When using Inferno, I use that plain-text editor instead of acme -- with the syntax highlighting for limbo.  I think it's nice to have the option.  I just wish there was a nicely integrated source management system so I wouldn't have to worry.

-- rich

RE: [9fans] source code as data not text

[David Gordon Hogan]

> I would rather have code stored in a
> clustered rdbms that is accessed through its own shell [that is
> cross-platform like limbo]

The last thing we need is another shell.

What if the code were accessed through a file system interface
which is cross-platform?

Re: [9fans] source code as data not text

[Steve Kilbane]

dhog:
> What if the code were accessed through a file system interface
> which is cross-platform?

As has been pointed out, how something is stored isn't that relevant
here; the issue is *what*, and the principle issue is whether your
program is stored as ascii source, or as some other representation
(that presumably adds additional structuring, else why bother?)

If you're storing an ascii file, then whether you store it in
a cvs-like system, an RDBMS or something entirely different isn't
that much more important than whether it's stored in a file system
that's on a local disk or a networked one: you may need to use
special means to retrieve it, but that's your call.

If it's plain ascii, and you've got an editor that can show pretty
colours, then fine. I'm happy for you, and it doesn't mean that anything
special has to be done to the file that'll stop it working on any other
editor.

If your source file is saved as something that represents additional
structure, then I'd suggest you're using the wrong terminology: that's
an object file, that's been compiled into another representation. From
here on in, you need a special set of tools to interpret and manipulate
the contents of that file. Regardless of the means by which you retrieved
the file, it's effectively a black box to most of the surrounding
system.

As I've said for many a year, "standardise on file formats and protocols,
and the applications take care of themselves."

steve

RE: [9fans] source code as data not text

[Richard Elberger]

>
>As has been pointed out, how something is stored isn't that relevant
>here; the issue is *what*, and the principle issue is whether your
>program is stored as ascii source, or as some other representation
>(that presumably adds additional structuring, else why bother?)
>
>If you're storing an ascii file, then whether you store it in
>a cvs-like system, an RDBMS or something entirely different isn't
>that much more important than whether it's stored in a file system
>that's on a local disk or a networked one: you may need to use
>special means to retrieve it, but that's your call.
>

How something is stored is very relevant to source management -- you'd
better ask the operations folks who have to run the hardware and are
responsible for maintaining data integrity.

-- rich

Re: [9fans] source code as data not text

[Steve Kilbane]

> How something is stored is very relevant to source management

Sigh. I was trying to clarify the issues, but obviously failed.
I'll try a different tack.

The discussion started off with syntax-directed editors, picked
up issues concerned with proprietary file formats, and then moved
into proprietary repositories versus things with a file-system.
My apologies for over-simplifying.

My point was merely this: given a file that contains some
representation of source, the issues of the file's internal
structure, the structure of the repository that holds the
file, and the how the file's contents are rendered
to the user are distinct issues and should not be confused.

The storage system is not relevant in all cases, but in the
discussion of what you do with a file _once you've retrieved
it from the storage system_, it is.

steve

RE: [9fans] source code as data not text

[Richard Elberger]

Yes, I think my wording was too general.  Yes, too many shells like sh and
ksh, but I wasn't thinking a cmd line shell -- just would like to see a
standard wrapper for the handling of code, how it maps to business (or
other) process -- like if a "feature module" will be plugged in or not to
the build, and how that is managed/operated, knowing where the code was
coming from, what environment it was written in, what scenarios/environment
it was unit tested in, etc ... a lot of things to track and imo it is all
important (though varying in degrees of importance).  Kind of like an app,
but it is basically a software team's creative bucket -- a system that is
self-enclosed and shares namespaces (be it source, devices, or workspaces).
So much stuff and syntax checking is just part of an app that is part of
what should be a consistent system to minimize risk.

I already thought of styx, and how it would be excellent to leverage this
for a full-on collaborative development environment.  It would be a huge
undertaking to create something like this, and it would be very exciting.

-- rich

>-----Original Message-----
>From: 9fans-admin@cse.psu.edu [mailto:9fans-admin@cse.psu.edu]On Behalf
>Of David Gordon Hogan
>Sent: Tuesday, 19 June 2001 6:49 a.m.
>To: 9fans@cse.psu.edu
>Subject: RE: [9fans] source code as data not text
>
>
>> I would rather have code stored in a
>> clustered rdbms that is accessed through its own shell [that is
>> cross-platform like limbo]
>
>The last thing we need is another shell.
>
>What if the code were accessed through a file system interface
>which is cross-platform?
>
>

Re: [9fans] source code as data not text

[Boyd Roberts]

my man dhog speaks ths truth:

> The last thing we need is another shell.

how many does vita's inferno have?  n too many.

Re: [9fans] source code as data not text

[Boyd Roberts]

merde, this message was meant for laura.

----- Original Message ----- 
From: "Boyd Roberts" <boyd@fr.inter.net>
To: <9fans@cse.psu.edu>
Sent: Thursday, June 28, 2001 11:00 PM
Subject: Re: [9fans] source code as data not text 


> > Cutting and pasting code is plain evil.
> 
> you said it.
> 
> back in paris and i have eurocave catalogue.
> 
> that stuff is is _way cool_.