[9fans] code complexity

[9fans] code complexity

[Richard Uhtenwoldt]

Most of the people on this list value "elegance" in source code.  To
them, a solution that requires 1000 lines of code is almost always
better than one that requires 10,000 lines as long as both solutions
solve the same set of problems.  Let us use the phrase "code complexity"
or "software complexity" to refer to the mental effort needed to
understand a piece of code.  Number of lines of source code, excluding
comments, is the easiest measure for estimating code complexity.

Whereas the people on this list tend to view code complexity as a cost
to be minimized/pessimized, many people --including probably most
programmers-- do not.  They may object to "code bloat" as a waste of
memory or cpu cycles or diskspace, but that is a different thing than
objecting to large source code sizes because they makes the code
unnecessarily difficult to understand, maintain and modify.  Some people
actually welcome code complexity, eg, some managers and economists even
today measures programmer "productivity" in lines of source code
produced.

Few on this list are going to disagree with the statement that the
popular desktop and server platforms are significantly more complex than
necessary.  but some numbers might be fun.

The Debian distribution of Linux contained 55,000,000 lines of code as
of 18 months ago.  (Includes all software including very specialized
stuff, eg, specialized for plane pilots, doctors, etc.)  Probably double
that now.

http://people.debian.org/~jgb/debian-counting/

the Mozilla web browser over 2,000,000 lines.  gcc 984,000.  I did not
inquire whether that included the g++ code to compile C++.

http://www.dwheeler.com/sloc/redhat71-v1/summary

The idea that complexity is a cost to be minimized is not new.  Dijkstra
devoted his Turing Award lecture to it in 1972 ("The Humble
Programmer").  And yet, it is still decidedly a minority view, even
among programmers.  Why?

Re: [9fans] code complexity

[Matt H]

> And yet, it is still decidedly a minority view, even
> among programmers.  Why?
 
I reckon it's from not having to fit everything into 1k, 16k, 32k, 64k, 640k 

or 24 lines
or over 9600 baud

there won't be too many more Mels

http://tuxedo.org/~esr/jargon/html/The-Story-of-Mel.html

Matt

Re: [9fans] code complexity

[Andrew Simmons]

>With a DLL all share its data; you do not get a copy.

I didn't think this had been true since the days of 16 bit Windows?

>However, they are both evil.
>
I'm afraid I've been turned to the dark side here. I've found them very
useful to add functionality to applications written by others -
particularly MS Excel, where I've written DLL add-ins to do various things
such as NZ bond pricing & pulling in data from ISAM files. There may be
other & better ways of doing this kind of thing, but I don't know what they
are. I believe the Oberon system has some way of adding functionality at
run time.

Re: [9fans] code complexity

[Boyd Roberts]

George Michaelson wrote:
> I've been told DLLs ...

We have a terminology problem:

    shared library = unix thing

    DLL = windows thing

They are not the same things.

With a DLL all share its data; you do not get a copy.

However, they are both evil.

Re: [9fans] code complexity

[ozan s yigit]

philw@entrisphere.com (philw) writes:

> Programming is an art - few people in industry have the luxury
> of enjoying it on the macro scale because they work with a large
> group that has unreasonable deadlines and business constraints
> beyond the engineering.

exactly. ever since i left yorku in 94 (these days i work for sun)
i've been trying to clean up crappy code written to meet deadlines at any
cost. it is very very depressing. even if it started great, it eventually
got hacked to the ground. the only chance is to get as many good programmers
you can for the initial product, and after that, hope for the best. :)

hope your venture is going well.

oz
-- 
www.cs.yorku.ca/~oz	 | don't count your chickens in glass houses
york u. computer science | until the cows come home. -- david vestal

Re: [9fans] code complexity

[George Michaelson]

I've been told DLLs include some smarts about code introspection so
you can find out whats in the damn things sensibly, runtime and know
how to interact with them.

I would have thought there were simple ways to do that, and Its not
such a bad idea to include a call which tells you what to pass in, to
talk to the external bindings made visible inside the package.

-George

--
George Michaelson       |  APNIC
Email: ggm@apnic.net    |  PO Box 2131 Milton QLD 4064
Phone: +61 7 3367 0490  |  Australia
  Fax: +61 7 3367 0482  |  http://www.apnic.net

Re: [9fans] code complexity

[Steve Kilbane]

I agree with presotto in that the real world induces complexity, but
for a slightly different reason. Plan 9 is an example of a system
where a problem can be solved at the "right" place, but most programmers
don't have that luxury. They're playing in a very small corner of the
field, and no matter how they complain, the rest of the field is not
going to shift itself around to accommodate them. It doesn't matter that
the change might produce a better system, because there's too much
inertia, and too much already invested in the status quo. So programmers
bung on their own little bit that solves their current problem, and move
on.

None of which excuses the ones who write isolated code and still make it
complex.

Code bloat, on the other hand, has another driving factor: it's often
easier for someone to write something from scratch than learn their system
well enough to realise that there's something already there that does it.
That's just laziness.

steve

Re: [9fans] code complexity

[Dan Cross]

> [...]  I helped to teach an
> introductory computer science theory course for three years,
> and we had the same problems there.  Students didn't
> see any point to distilling a correct yet awkward proof
> down to its essence; after all, it was correct, wasn't it?
> (I don't know if this was natural or they got it from
> taking intro to programming the year before.)
> Even so, we didn't give much opportunity to help
> them rewrite their proofs.  We did hand out the best
> most elegant proofs we had on the solution sets,
> and I think that helped somewhat, but I'm still not
> sure how many internalized the message.

Hmm, interesting.  Programming is more or less isomorphic to
mathematics in this respect, as you point out; it's all about
generalization and abstraction, but too often individuals fail to
perceive the value of either.  I think that the problems students
encounter in both come from the same place; essentially a fear of math
and math-related subjects.  People don't want to understand *how*
something works, they just want to see *what* to do with it.

A large part of that comes from the public school system (at least,
this is true in the USA).  People get so convinced so early on that
math is something that they can't get their heads around because
they're `taught' by people who are inept and don't have their heads
around math to start with and teach it the wrong way.  The
self-defeating idea that ``I can't do it'' follows a person, and
perpetuates itself through higher education.  The result is that
students don't understand what's behind the symbols in math, and just
memorize equations and theorems to pass the tests.  They might have
some understanding of how the numbers and symbols fit together, but
they're still eluded by the higher mathematical ``truth,'' and even
worse, they don't think of themselves as capable of understanding it.

When they get out of school they collectively breath a sigh of relief
and say, ``Ahh, I never have to care about that ever again.'' When
something math related comes up later on, they mostly go into dummy
mode and just slog through it, all the while beset my flashbacks to
less happy days in which they were forced to slog through similar,
seemingly incomprehensible equations and so on to get a decent grade.

This is especially true amongst scientists and engineers.  For
instance, does anyone really know *why* the result of the cross-product
is called a pseudovector?  Hmm, perhaps present company should be
excluded from my sweeping generalizations, but you get the picture.

Anyway, back to programming; it's just too close to math for a lot of
people.  Probably because, in a sense, it is math.  But again, it's
something they feel very despondent about, and don't want to get into
other than at a superficial level.

Anyway, this post is much too long and rambling.  The point is that the
average programmer doesn't see elegance or generality as something to
strive for, because he or she hasn't seen that as something useful
since the age of 5.

	- Dan C.

RE: [9fans] code complexity

[philw]

[-- Attachment #1: Type: text/plain, Size: 3248 bytes --]

Russ's rant is academic. Many programmers write code
to make revenue. I care the code is simple because that
makes it maintainable. Program complexity has a lot to do
with style. It can be taught. I live with bad code often because
I have more work than I can possibly do in a given time and
I must buy lots of code that I integrate with my product. I am
rarely pleased with code from stack vendors. Good
programmers are expensive and my hiring follows a gausian:
some very good, some very bad most in the middle. I have few
chances to revisit anything since I could be writing code
which generates more revenue. That means I clean up what absolutley
must be maintainable then test and forget the rest.

Programming is an art - few people in industry have the luxury
of enjoying it on the macro scale because they work with a large
group that has unreasonable deadlines and business constraints
beyond the engineering.

Sometimes I pine for the good old days ....

phil

-----Original Message-----
From: Russ Cox [mailto:rsc@plan9.bell-labs.com]
Sent: Tuesday, February 05, 2002 8:35 AM
To: 9fans@cse.psu.edu
Subject: Re: [9fans] code complexity


I don't really feel like most of those line counts
are comparable, since it's not clear how much is 
there.  If you want something else that is not
really comparable, note that the TeX add-on package
for Plan 9 is almost the same size as the "everything else"
base package.

As for why, there certainly exists a small class of programmers
that actively believes writing complex programs is
just macho.  To them, I admit: I am not man enough to
keep up with your layers upon layers of complexity.
When I run into code they have written, I inevitably
complain to whoever will listen until I get fed up
enough to rewrite it.  To some extent, Mel is in this
class, although he understands programs as craft too.

At the same time, I think most programmers just don't
understand that programs are intended to be read by humans.
They fiddle until it works and then don't see the point
to doing it over again in a cleaner, simpler, more elegant
manner.  Especially if someone is being measured by some
bogus productivity metric like like lines of code written,
papers published, or problem sets handed in.

Good programming is like good writing in any subject;
it happens only by much revision and practice.  The sad
thing is that at least in my experience, introductory
programming courses just don't get that across.  (In fact,
in my experience most writing courses state that but then
do a poor job of backing it up.)  I helped to teach an
introductory computer science theory course for three years,
and we had the same problems there.  Students didn't
see any point to distilling a correct yet awkward proof
down to its essence; after all, it was correct, wasn't it?
(I don't know if this was natural or they got it from
taking intro to programming the year before.)
Even so, we didn't give much opportunity to help
them rewrite their proofs.  We did hand out the best
most elegant proofs we had on the solution sets,
and I think that helped somewhat, but I'm still not
sure how many internalized the message.

End of rant.
Russ

[-- Attachment #2: Type: text/html, Size: 5360 bytes --]

Re: [9fans] code complexity

[david presotto]

Simplicity pretty much goes out the window when
you try to keep up with the world since the world
isn't simple.  Try to build a browser that supports
just enough to get along with the pages that are
out there and you still end up with something 
bigger than Plan 9 no matter how hard you work
at it.

The question however is not the total size of the code
but the complexity of the internal interactions.  If
one added as many device drivers to Plan 9 as one
has in Windows or Linux, our code base would be
truly huge also.  However, the complexity of any
instance of plan 9 wouldn't be anygreater because the
vast body of device drivers do not interact with
each other and any particular instance will have a
small selection of them in use.  They all follow
pretty much the same API talking to the kernel
and to programs and that API is not very large.

It would be nice if such an approach could be
applied to a browser.  I haven't seen the inards
of Opera, so perhaps it has.  It definitely isn't
true of netscape or exploder.

One of the problems of uncontrolled excess is
that API's tend to get very wide to cover all possible
corners.  (The same is true of  in other arenas,
hence the current cross over cars that are SUV's,
compacts, luxury cars, and trucks all built into
one.)  The wider the API, the larger the number
of interactions with that piece of code and everything
that interfaces with it.  We've tried to keep API's
small to avoid that, i.e., put in what's absolutely
necessary and drag our heels on everything else.
However, that means saying no to a lot of
people with real problems, people we ignore
in order to keep our playland comprehensible.
To that extent Linux is much truer to the ideals
of an open market than Plan 9 or BSD.  If
something it wanted by enough people, it appears
no matter what it does to the complexity or
supportability of the code.

This leads back into the DLL discussion because,
once someone has DLL's, they tend to feel that
the size of the API no longer matters, everyone
will be sharing the code bloat so why bother
keeping things small.  There's nothing inherent in
the existence of DLL's that causes this, it's just
human nature.

Re: [9fans] code complexity

[Ronald G Minnich]

On Tue, 5 Feb 2002, Boyd Roberts wrote:

> Russ Cox wrote:
> > As for why, there certainly exists a small class of programmers
> > that actively believes writing complex programs is
> > just macho.
>
> Bad programmers build complex stuff because they can and they get
> away with it most of the time, until it all comes crashing down.

or they just start from a design point that seems simple, and they don't
know enough to know that because they picked the wrong starting point it's
going to grow into a large complicated ugly ball of tar. Like Linux.

ron

Re: [9fans] code complexity

[Russ Cox]

> Bad programmers build complex stuff because they can and they get
> away with it most of the time, until it all comes crashing down.

That was the second paragraph.  I assert the existence of good
programmers who just believe that making complex things is cool.

Re: [9fans] code complexity

[Boyd Roberts]

Russ Cox wrote:
> As for why, there certainly exists a small class of programmers
> that actively believes writing complex programs is
> just macho.

Bad programmers build complex stuff because they can and they get
away with it most of the time, until it all comes crashing down.

Re: [9fans] code complexity

[Russ Cox]

I don't really feel like most of those line counts
are comparable, since it's not clear how much is 
there.  If you want something else that is not
really comparable, note that the TeX add-on package
for Plan 9 is almost the same size as the "everything else"
base package.

As for why, there certainly exists a small class of programmers
that actively believes writing complex programs is
just macho.  To them, I admit: I am not man enough to
keep up with your layers upon layers of complexity.
When I run into code they have written, I inevitably
complain to whoever will listen until I get fed up
enough to rewrite it.  To some extent, Mel is in this
class, although he understands programs as craft too.

At the same time, I think most programmers just don't
understand that programs are intended to be read by humans.
They fiddle until it works and then don't see the point
to doing it over again in a cleaner, simpler, more elegant
manner.  Especially if someone is being measured by some
bogus productivity metric like like lines of code written,
papers published, or problem sets handed in.

Good programming is like good writing in any subject;
it happens only by much revision and practice.  The sad
thing is that at least in my experience, introductory
programming courses just don't get that across.  (In fact,
in my experience most writing courses state that but then
do a poor job of backing it up.)  I helped to teach an
introductory computer science theory course for three years,
and we had the same problems there.  Students didn't
see any point to distilling a correct yet awkward proof
down to its essence; after all, it was correct, wasn't it?
(I don't know if this was natural or they got it from
taking intro to programming the year before.)
Even so, we didn't give much opportunity to help
them rewrite their proofs.  We did hand out the best
most elegant proofs we had on the solution sets,
and I think that helped somewhat, but I'm still not
sure how many internalized the message.

End of rant.
Russ