Re: [9fans] Ports tree for Plan 9

Re: [9fans] Ports tree for Plan 9

[mveety]

I tried out the sbase zip on my system. That seems to work fine. Heres the log:

cpu% lc
sbase-master.zip
cpu% unzip -f sbase-master.zip
cpu% lc
sbase-master/		sbase-master.zip
cpu% cd sbase-master/
cpu% lc
LICENSE		comm.1		find.1		md5.h		readlink.c	split.c		uname.c
Makefile	comm.c		find.c		md5sum.1	renice.1	sponge.1	unexpand.1
README		compat.h	fold.1		md5sum.c	renice.c	sponge.c	unexpand.c
TODO		config.mk	fold.c		mkdir.1		rm.1		strings.1	uniq.1
arg.h		cp.1		fs.h		mkdir.c		rm.c		strings.c	uniq.c
basename.1	cp.c		grep.1		mkfifo.1	rmdir.1		sync.1		unlink.1
basename.c	cron.1		grep.c		mkfifo.c	rmdir.c		sync.c		unlink.c
cal.1		cron.c		head.1		mktemp.1	sed.1		tail.1		utf.h
cal.c		crypt.h		head.c		mktemp.c	sed.c		tail.c		util.h
cat.1		cut.1		hostname.1	mv.1		seq.1		tar.1		uudecode.1
cat.c		cut.c		hostname.c	mv.c		seq.c		tar.c		uudecode.c
chgrp.1		date.1		join.1		nice.1		setsid.1	tee.1		uuencode.1
chgrp.c		date.c		join.c		nice.c		setsid.c	tee.c		uuencode.c
chmod.1		dirname.1	kill.1		nl.1		sha1.h		test.1		wc.1
chmod.c		dirname.c	kill.c		nl.c		sha1sum.1	test.c		wc.c
chown.1		du.1		libutf/		nohup.1		sha1sum.c	text.h		which.1
chown.c		du.c		libutil/	nohup.c		sha256.h	time.1		which.c
chroot.1	echo.1		link.1		paste.1		sha256sum.1	time.c		xargs.1
chroot.c	echo.c		link.c		paste.c		sha256sum.c	touch.1		xargs.c
cksum.1		env.1		ln.1		printenv.1	sha512.h	touch.c		yes.1
cksum.c		env.c		ln.c		printenv.c	sha512sum.1	tr.1		yes.c
cmp.1		expand.1	logger.1	printf.1	sha512sum.c	tr.c
cmp.c		expand.c	logger.c	printf.c	sleep.1		true.1
col.1		expr.1		logname.1	pwd.1		sleep.c		true.c
col.c		expr.c		logname.c	pwd.c		sort.1		tty.1
cols.1		false.1		ls.1		queue.h		sort.c		tty.c
cols.c		false.c		ls.c		readlink.1	split.1		uname.1

--
Veety

Re: [9fans] Ports tree for Plan 9

[erik quanstrom]

> There have been many fixes to BWK's code since then. If you're going to
> start over, it should be done from his current code, available from
> his Princeton home page.

9atom's awk has been updated with bwk's recent source.  it also has a fix for the problem
sometimes seen with plan 9 installs where a score looks enough like floating point that
awk tries to treat it as such and gets a fp exception in the process.

i would consider adding pure extensions, such as %C, or the ability to deal with hex.

- erik

Re: [9fans] Ports tree for Plan 9

[arnold]

> Have the MKS sources ever been released?

I don't believe so. I think they were the basis for z/OS, which is
a POSIX environment on top of IBM's MVS.

The MKS awk made its way out into the world via Solaris, which for some
reason chose that code base, instead of a more recent version of BWK's
awk, for their POSIX awk.  Quite some time ago I ported it to Linux; it
took an hour or so. It'd take more work to make it generally portable, which
I never bothered to do.

Arnold

Re: [9fans] Ports tree for Plan 9

[arnold]

Kurt H Maier <khm@sciops.net> wrote:

> I mangled some webshit earlier today on that server (bad timing I guess).
> Correct link to the hg repo is https://code.9front.org/hg/awk

This appears to be based on Brian Kernighan's awk from sometime in 1999
and not written from scratch.

There have been many fixes to BWK's code since then. If you're going to
start over, it should be done from his current code, available from
his Princeton home page.

Arnold

Re: [9fans] Ports tree for Plan 9

[Kurt H Maier]

Quoting erik quanstrom <quanstro@quanstro.net>:

> instead of guessing, you could see if the pool library's checks are
> really a bottleneck.
> it is straightforward to add header and tail magic and the callerpc
> stuff to ape
> malloc and run the comparsion again.
>
> otherwise, it seems far more likely that the problem is that quicklicks are
> faster than tree allocators.

I'm not a programmer.

>> I'm reasonably certain Moore's Law has fixed this issue for all
>> practical applications, however...
>
> i'm resonablly certain that plan 9 malloc's poor performance has
> cost me quite
> a bit of work.

I was talking about awk, not malloc in all applications.  Sorry if I was
insufficiently precise.  So far I have not been displeased with the
performance of native awk, but I'd be interested in seeing use cases
where it becomes a real-world problem.

khm

Re: [9fans] Ports tree for Plan 9

[lucio]

> and the
> MKS awk as found in the various OpenSolaris derivatives

MKS was my introduction to Unix, I was a PCDOS user back then :-)

It's interesting to hear about that port.  I still tread carefully in
vi because of a minor nit (which my fingers remember better than my
brain) with backspace in replace mode (it will come to me, I seriously
still suffer from it :-).

Have the MKS sources ever been released?

Lucio.

Re: [9fans] Ports tree for Plan 9

[erik quanstrom]

On Sat May 30 22:02:11 PDT 2015, quanstro@quanstro.net wrote:
> On Sat May 30 21:43:03 PDT 2015, khm@sciops.net wrote:
> > Quoting arnold@skeeve.com:
> >
> > > BWK has said that malloc affects the performance of his awk; I think
> > > it's in his README file.
> >
> > Yes, it was explained to me that plan 9 malloc does useful things instead
> > of just shoving things into the first available hole like APE malloc.
>
> instead of guessing, you could see if the pool library's checks are really a bottleneck.
> it is straightforward to add header and tail magic and the callerpc stuff to ape
> malloc and run the comparsion again.
>
> otherwise, it seems far more likely that the problem is that quicklicks are
> faster than tree allocators.

also, ape %f/%g are much faster than the native version.

- erik

Re: [9fans] Ports tree for Plan 9

[erik quanstrom]

On Sat May 30 21:43:03 PDT 2015, khm@sciops.net wrote:
> Quoting arnold@skeeve.com:
>
> > BWK has said that malloc affects the performance of his awk; I think
> > it's in his README file.
>
> Yes, it was explained to me that plan 9 malloc does useful things instead
> of just shoving things into the first available hole like APE malloc.

instead of guessing, you could see if the pool library's checks are really a bottleneck.
it is straightforward to add header and tail magic and the callerpc stuff to ape
malloc and run the comparsion again.

otherwise, it seems far more likely that the problem is that quicklicks are
faster than tree allocators.

> I'm reasonably certain Moore's Law has fixed this issue for all
> practical applications, however...

i'm resonablly certain that plan 9 malloc's poor performance has cost me quite
a bit of work.

- erik

Re: [9fans] Ports tree for Plan 9

[Kurt H Maier]

Quoting arnold@skeeve.com:

> BWK has said that malloc affects the performance of his awk; I think
> it's in his README file.

Yes, it was explained to me that plan 9 malloc does useful things instead
of just shoving things into the first available hole like APE malloc.

I'm reasonably certain Moore's Law has fixed this issue for all
practical applications, however...

> I can't get to this with a browser. How does one get the source?  Is it
> intended to run on *nix also? It'd be nice to have since it's always
> fun to compare multiple implementations when trying to figure out a
> corner case.  Besides BWK's awk, there is mawk, BusyBox awk, and the
> MKS awk as found in the various OpenSolaris derivatives.

I mangled some webshit earlier today on that server (bad timing I guess).
Correct link to the hg repo is https://code.9front.org/hg/awk

And I think I have a tarball at http://intma.in/downloads/awk.tgz but
that webserver Content-type is broken so you should hget it instead of
using a browser to download.

khm

Re: [9fans] Ports tree for Plan 9

[arnold]

lucio@proxima.alt.za:

> It may be worth twisting Aaron's arm, he may well have a test suite
> for GAWK that can be used here?

The gawk test suite is part of the dist. See test/Makefile.am for the
list of tests that are general and those that are gawk specific.
I've tried to keep the separation clean.

Kurt H Maier <khm@sciops.net> wrote:
> Current status:  the only failures are bizarre corner cases, but
> presumably they're in the testsuite for a reason?

Yes - people will do anything they can. Experience has taught me that
even bizarre corner cases need to be handled.

> Native awk is slower than APE awk, and Paul mentioned he thinks it's
> because of the malloc implementation.

BWK has said that malloc affects the performance of his awk; I think
it's in his README file.

> [1] http://code.9front.org/awk

I can't get to this with a browser. How does one get the source?  Is it
intended to run on *nix also? It'd be nice to have since it's always
fun to compare multiple implementations when trying to figure out a
corner case.  Besides BWK's awk, there is mawk, BusyBox awk, and the
MKS awk as found in the various OpenSolaris derivatives.

Thanks,

Arnold

Re: [9fans] Ports tree for Plan 9

[sl]

> this doesn't seem like motiviation to rewrite awk.  there must be another reason?

I think "rewrite" is a mischaracterization (nobody is talking about re-
implementing the awk interpreter), so arguing against that seems to be beside
the point. Probably, "port awk to Plan 9 without using APE" is more accurate.

>From memory, the "awk is not a native Plan 9 program" problem has been discussed
a few times on 9fans. Googling, I found the following message, which describes
some of the issues raised:

	https://www.mail-archive.com/9fans@cse.psu.edu/msg17798.html


> by the way, thinking a bit bigger, what i'd like to see is x, where x is to awk as rc is to the
> bourne shell.

The ssam (stream interface to sam) script from plan9port is heavier than
awk by itself, but can be useful for a lot of the same tasks.

sl

Re: [9fans] Ports tree for Plan 9

[erik quanstrom]

On Sat May 30 13:36:14 PDT 2015, sl@9front.org wrote:
> On May 30, 2015, at 11:54 AM, erik quanstrom <quanstro@quanstro.net> wrote:
>
> >> I would very much like to see this fast and conformant, so that APE
> >> awk can be thrown in the trash.
> >
> > i don't understand this.  awk is bwk's ota source, with some minor tweaks to fit the
> > environment.  it works well, and allows portable awk to be written.  can you
> > explain what is to be gained by a re do?  i don't think "doesn't use ape" per ce
> > is a good argument.  it would have to be explained what this enables.  i can't see
> > that part.
> >
> > - erik
>
> if i understood correctly, the major reasons were better unicode handling and not using sh for system().

using rc instead of /bin/ape/sh is a a bind away.  similarly, adding %C to awk is also trivial.
but there are compatability tradeoffs.  (i used rune/uconv (see rune(1),
http://sources.9atom.org/magic/man2html/1/rune)

this doesn't seem like motiviation to rewrite awk.  there must be another reason?

by the way, thinking a bit bigger, what i'd like to see is x, where x is to awk as rc is to the
bourne shell.

- erik

Re: [9fans] Ports tree for Plan 9

[Stanley Lieber]

On May 30, 2015, at 11:54 AM, erik quanstrom <quanstro@quanstro.net> wrote:

>> I would very much like to see this fast and conformant, so that APE
>> awk can be thrown in the trash.
> 
> i don't understand this.  awk is bwk's ota source, with some minor tweaks to fit the
> environment.  it works well, and allows portable awk to be written.  can you
> explain what is to be gained by a re do?  i don't think "doesn't use ape" per ce
> is a good argument.  it would have to be explained what this enables.  i can't see
> that part.
> 
> - erik

if i understood correctly, the major reasons were better unicode handling and not using sh for system().

sl

Re: [9fans] Ports tree for Plan 9

[Stanley Lieber]

> On May 30, 2015, at 12:27 PM, Jeff Sickel <jas@corpus-callosum.com> wrote:
> 
> 
>>> On May 30, 2015, at 11:17 AM, Kurt H Maier <khm@sciops.net> wrote:
>>> 
>>> pretty difficult to do if there is a desire to use git or hg.
>> 
>> does hgfs use APE?  I haven't investigated too closely.
> 
> hgfs is a read-only Hg tool written in Limbo.  You still need hg running
> on your host to pull/commit/push changes.

he was referring to the c program hgfs that was written for 9front. currently, yes, it is read-only.

sl

Re: [9fans] Ports tree for Plan 9

[Jeff Sickel]

> On May 30, 2015, at 11:17 AM, Kurt H Maier <khm@sciops.net> wrote:
>
>> pretty difficult to do if there is a desire to use git or hg.
>
> does hgfs use APE?  I haven't investigated too closely.

hgfs is a read-only Hg tool written in Limbo.  You still need hg running
on your host to pull/commit/push changes.

Re: [9fans] Ports tree for Plan 9

[Kurt H Maier]

Quoting erik quanstrom <quanstro@quanstro.net>:

> i don't understand this.

It is a personal preference not rooted in any technological excuses.


> pretty difficult to do if there is a desire to use git or hg.

does hgfs use APE?  I haven't investigated too closely.

khm

Re: [9fans] Ports tree for Plan 9

[erik quanstrom]

> Personally, it's just one more reason to reduce our nation's dependence on
> foreign code -- does anyone want to help test pap's native awk?

pretty difficult to do if there is a desire to use git or hg.

- erik

Re: [9fans] Ports tree for Plan 9

[erik quanstrom]

> I would very much like to see this fast and conformant, so that APE
> awk can be thrown in the trash.

i don't understand this.  awk is bwk's ota source, with some minor tweaks to fit the
environment.  it works well, and allows portable awk to be written.  can you
explain what is to be gained by a re do?  i don't think "doesn't use ape" per ce
is a good argument.  it would have to be explained what this enables.  i can't see
that part.

- erik

Re: [9fans] Ports tree for Plan 9

[lucio]

> I am not saying that they are the ideal or good tools - just that most 3rd
> party source expect certain behavior and a "compatibility environment"
> (like APE) has as first priority to deal with 3rd party stuff. Enabling as
> much as possible without judgement is at least to me desirable.

Remember that APE was developed to determine the portability to Posix of software developed natively for Plan 9.  That it turned on its master and took on a totally different mantle is understandable, but it is also not surprising that some shortcomings may never be uovercome.

Lucio.

Re: [9fans] Ports tree for Plan 9

[lucio]

> I would very much like to see this fast and conformant, so that APE
> awk can be thrown in the trash.

In my wild dreams I wish for a native version of ghostscript (the only
justified use of APE, if you believe in fairness (or fairy tales :-)).

But maybe Go will eventually stimulate development of a sane text
processing suite.  One is allowed to dream, isn't one?  Donald, where
are you when one needs you?

Lucio.

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

Den 30 maj 2015 10:23 skrev "Charles Forsyth" <charles.forsyth@gmail.com>:
>
>
> On 30 May 2015 at 08:21, Jens Staal <staal1978@gmail.com> wrote:
>>
>> am also interested in seeing how compatible the ported m4 is with GNU m4
if there are good tests
>
>
> GNU m4 is insane, and completely missed the point about GPM (and thus m4).
>
> My m4 port is based on Ritchie's m4, although I might re-do a few things
to make it a Plan 9 program
> and account for a few changes in the C environment. You could put gnu m4
in APE I suppose, but
> since it's mainly used for autotools which won't work anyway because they
aren't portable, I'm not sure what's the point.

I was talking about "quasar m4" in ports

https://bitbucket.org/mveety/9front-ports/src/devel/m4/

Which apparently is a modified BSD m4 specifically aiming for GNU
compatibility.

I am not saying that they are the ideal or good tools - just that most 3rd
party source expect certain behavior and a "compatibility environment"
(like APE) has as first priority to deal with 3rd party stuff. Enabling as
much as possible without judgement is at least to me desirable.

All the ports are optional so nobody needs to feel "violated" by my heresy
;)

[-- Attachment #2: Type: text/html, Size: 1648 bytes --]

Re: [9fans] Ports tree for Plan 9

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]

On 30 May 2015 at 08:21, Jens Staal <staal1978@gmail.com> wrote:

> am also interested in seeing how compatible the ported m4 is with GNU m4
> if there are good tests


GNU m4 is insane, and completely missed the point about GPM (and thus m4).

My m4 port is based on Ritchie's m4, although I might re-do a few things to
make it a Plan 9 program
and account for a few changes in the C environment. You could put gnu m4 in
APE I suppose, but
since it's mainly used for autotools which won't work anyway because they
aren't portable, I'm not sure what's the point.

[-- Attachment #2: Type: text/html, Size: 1000 bytes --]

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

[-- Attachment #1: Type: text/plain, Size: 544 bytes --]

Den 30 maj 2015 08:41 skrev <lucio@proxima.alt.za>:
>
> > does anyone want to help test pap's native awk?
>
> Build it and they'll come :-)
>
> URL?  Is it portable?  How carefully was it ported?
>
> It may be worth twisting Aaron's arm, he may well have a test suite
> for GAWK that can be used here?
>
> Lucio.
>
>

I was going to attempt a new gawk port sometime later.

I am also interested in seeing how compatible the ported m4 is with GNU m4
if there are good tests. A plan is to attempt bison/flex porting some time.

[-- Attachment #2: Type: text/html, Size: 759 bytes --]

Re: [9fans] Ports tree for Plan 9

[Kurt H Maier]

Quoting Kurt H Maier <khm@sciops.net>:

> Paul wrote it from scratch.

No he didn't; he started with Boyd's awk.  Been a while since
I looked at the commit history.  Sorry.

khm

Re: [9fans] Ports tree for Plan 9

[Kurt H Maier]

Quoting lucio@proxima.alt.za:

>> does anyone want to help test pap's native awk?
>
> Build it and they'll come :-)
>
> URL?  Is it portable?  How carefully was it ported?
>
> It may be worth twisting Aaron's arm, he may well have a test suite
> for GAWK that can be used here?
>
> Lucio.

Paul wrote it from scratch.  I have a copy[1] but I'm unsure it's his
latest version -- he no longer has it, so I guess maybe it is.

Paul also asked bwk for his awk test suite, which is still online[2],
thankfully.

Current status:  the only failures are bizarre corner cases, but
presumably they're in the testsuite for a reason?  Native awk is
slower than APE awk, and Paul mentioned he thinks it's because of the
malloc implementation.

I would very much like to see this fast and conformant, so that APE
awk can be thrown in the trash.

[1] http://code.9front.org/awk
[2] http://www.cs.princeton.edu/~bwk/btl.mirror/awktest.a

khm

Re: [9fans] Ports tree for Plan 9

[lucio]

> does anyone want to help test pap's native awk?

Build it and they'll come :-)

URL?  Is it portable?  How carefully was it ported?

It may be worth twisting Aaron's arm, he may well have a test suite
for GAWK that can be used here?

Lucio.

Re: [9fans] Ports tree for Plan 9

[lucio]

>> It is the Plan 9 Way (TM) to avoid > nested inclusion of header files,
>
> $ arch/dat.h includes port/portdat.h in kernel. Exempted too?

That's out of necessity, the alternative(s) would be considerably less
practical.  If memory serves, port/portdat.h is not strictly a header
file in the connventional "C" sense.  If memory serves, it would also
be possible to arrange the 9 kernel sources in a more elegant manner,
but the cost to benefit ratio would be too low to bother.

Note that no one has ever prohibited nesting includes where it could
have been possible to do it in the compiler and pre=processor.  It is
not a crime, it is a bad practice.  I'm of the opinion that it is as
avoidable as the GOTO command in programming languages (I don't use
GOTOs and don't really cope well reading code that does), but there is
enough code out there to suggest my opinion needs a formal proof.

So, yes, I think there are exceptions and the one you quote is not the
only one, mostly for historical reasons; it is also my belief that
such exceptions could be eliminated, but with the advancing of time,
it becomes less and less practical to do so.  Further, I mentioned
exemption for APE because it is an uncomfortable reality we don't need
to encourage: like the kernel stuff that dates back to the 1980s, once
adopted, it becomes very hard to eliminate.

Lucio.

Re: [9fans] Ports tree for Plan 9

[Kurt H Maier]

Quoting lucio@proxima.alt.za:

> It is the Plan 9 Way (TM) to avoid nested inclusion of header files,
> although I guess the APE may be exempted.

while I agree it's not very plan-9-like, the posix standard is horrible and
broken and nobody should be surprised that the easy way to implement it
involves horrible brokenness.

I guess it's implementation-defined whether you prefer to preserve the purity
of essence by redefining the types in wait.h, or join the mutiny of preverts
with a nested include.

Personally, it's just one more reason to reduce our nation's dependence on
foreign code -- does anyone want to help test pap's native awk?

khm

Re: [9fans] Ports tree for Plan 9

[Álvaro Jurado]

[-- Attachment #1: Type: text/plain, Size: 1136 bytes --]

> It is the Plan 9 Way (TM) to avoid > nested inclusion of header files,

$ arch/dat.h includes port/portdat.h in kernel. Exempted too?

Álvaro Jurado Cuevas
http://colmenar.biz.tm
El 30/05/2015 07:11, <lucio@proxima.alt.za> escribió:

> > Which version?
> >
> > "The id_t and pid_t types shall be defined as described in
> > <sys/types.h>." in issue 6
> >
> > "The <sys/wait.h> header shall define the id_t and pid_t types as
> > described in <sys/types.h>." in issue 7
> >
> > in the sys/wait.h part of the headers section of base definitions
>
> I haven't looked at cinap's work, but...
>
> It is the Plan 9 Way (TM) to avoid nested inclusion of header files,
> although I guess the APE may be exempted.  I also appreciate that
> adding conditional definitions of id_t and pid_t in <sys/wait.h> that
> match those in <sys/types.h> could lead to eventual inconsistencies,
> but I would still prefer to follow the Plan 9 guidelines.
>
> But without a more formal code review structure and the apparent
> absence of guidance from Bell Labs, I suppose I'm just farting in the
> wind.
>
> Lucio.
>
>
>

[-- Attachment #2: Type: text/html, Size: 1589 bytes --]

Re: [9fans] Ports tree for Plan 9

[lucio]

> Which version?
>
> "The id_t and pid_t types shall be defined as described in
> <sys/types.h>." in issue 6
>
> "The <sys/wait.h> header shall define the id_t and pid_t types as
> described in <sys/types.h>." in issue 7
>
> in the sys/wait.h part of the headers section of base definitions

I haven't looked at cinap's work, but...

It is the Plan 9 Way (TM) to avoid nested inclusion of header files,
although I guess the APE may be exempted.  I also appreciate that
adding conditional definitions of id_t and pid_t in <sys/wait.h> that
match those in <sys/types.h> could lead to eventual inconsistencies,
but I would still prefer to follow the Plan 9 guidelines.

But without a more formal code review structure and the apparent
absence of guidance from Bell Labs, I suppose I'm just farting in the
wind.

Lucio.

Re: [9fans] Ports tree for Plan 9

[erik quanstrom]

On Fri May 29 14:26:54 PDT 2015, cinap_lenrek@felloff.net wrote:
> i did a google search for it and found this:
>
> http://pubs.opengroup.org/onlinepubs/007904975/basedefs/sys/wait.h.html
>
> which stated:
>
> "The id_t and pid_t types shall be defined as described in <sys/types.h>."
>
> and also looked in openbsd's <sys/wait.h> which did #include <sys/types.h>
> which was good enougth for me, tho i'm not a unix expert.

excellent.  thanks.

- erik

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

i did a google search for it and found this:

http://pubs.opengroup.org/onlinepubs/007904975/basedefs/sys/wait.h.html

which stated:

"The id_t and pid_t types shall be defined as described in <sys/types.h>."

and also looked in openbsd's <sys/wait.h> which did #include <sys/types.h>
which was good enougth for me, tho i'm not a unix expert.

--
cinap

Re: [9fans] Ports tree for Plan 9

[Kurt H Maier]

Quoting erik quanstrom <quanstro@quanstro.net>:

> On Wed May 27 12:51:19 PDT 2015, cinap_lenrek@felloff.net wrote:
>> > A potential bug in APE sys/wait.h : the header does not make sure
>> that pid_t
>> > has been defined.
>> > Compiling sbase on Plan9/APE ended up in situations where there
>> were lots of
>> > compilation faliures simply because <sys/wait.h> was included without
>> > <sys/types.h> being included before.
>>
>> fixed.
>
> uh, i just read the open group spec.  i did not see the requirement
> that wait.h
> define pid_t.  if someone could site chapter and verse, i would be
> happy to add
> the include, but it would seem hasty without a standards citation.
>
> - erik

Which version?

"The id_t and pid_t types shall be defined as described in
<sys/types.h>." in issue 6

"The <sys/wait.h> header shall define the id_t and pid_t types as
described in <sys/types.h>." in issue 7

in the sys/wait.h part of the headers section of base definitions


khm

Re: [9fans] Ports tree for Plan 9

[erik quanstrom]

On Wed May 27 12:51:19 PDT 2015, cinap_lenrek@felloff.net wrote:
> > A potential bug in APE sys/wait.h : the header does not make sure that pid_t
> > has been defined.
> > Compiling sbase on Plan9/APE ended up in situations where there were lots of
> > compilation faliures simply because <sys/wait.h> was included without
> > <sys/types.h> being included before.
>
> fixed.

uh, i just read the open group spec.  i did not see the requirement that wait.h
define pid_t.  if someone could site chapter and verse, i would be happy to add
the include, but it would seem hasty without a standards citation.

- erik

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

> A potential bug in APE sys/wait.h : the header does not make sure that pid_t
> has been defined.
> Compiling sbase on Plan9/APE ended up in situations where there were lots of
> compilation faliures simply because <sys/wait.h> was included without
> <sys/types.h> being included before.

fixed.

--
cinap

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

On Friday 15 May 2015 07:53:39 cinap_lenrek@felloff.net wrote:
> commited the fix.

Playing with the ports has so far uncovered 3 bugs (hget, zip and the one
below) so rather fruitful playing :)

A potential bug in APE sys/wait.h : the header does not make sure that pid_t
has been defined.
Compiling sbase on Plan9/APE ended up in situations where there were lots of
compilation faliures simply because <sys/wait.h> was included without
<sys/types.h> being included before.

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

On Friday 15 May 2015 07:53:39 cinap_lenrek@felloff.net wrote:
> fixed, its a bug in gunzip. the extra-len field in the gzip header has two
> byte length field instead of one byte.
>
> commited the fix.

awesome!
now it works

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

fixed, its a bug in gunzip. the extra-len field in the gzip header has two
byte length field instead of one byte.

commited the fix.

--
cinap

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

the sbase works fine for me, tho i can reproduce the mksh-R50f.tgz gzip
problem.

lemme do some debugging on that.

--
cinap

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

On Thursday 14 May 2015 16:37:05 cinap_lenrek@felloff.net wrote:
> pretty sure this is apache bug/misconfiguration. googled for it
> and the issue seems to be known problem.

The zlib archive works fine now :)
by the way, did you try the mksh archive after hget was fixed? I still get the
same error as before

"gunzip: <stdin>: inflate failed: corrupted data
/bin/tar: EOF reading archive ..."

https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R50f.tgz

but now the md5sums match between the linux-downloaded and the 9front-
downloaded archive.

Also sbase archives (.zip, tar.gz and tar.bz2) suffer from the same issue.
http://git.suckless.org/sbase/snapshot/sbase-master.zip
http://git.suckless.org/sbase/snapshot/sbase-master.tar.gz
http://git.suckless.org/sbase/snapshot/sbase-master.tar.bz2

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

pretty sure this is apache bug/misconfiguration. googled for it
and the issue seems to be known problem.

--
cinap

Re: [9fans] Ports tree for Plan 9

[Steffen Nurpmeso]

cinap_lenrek@felloff.net wrote:
 |found it. the server sends Content-Encoding header which causes hget
 |to add a decompression filter, so you get as output a tarball.
 |
 |<- Content-Type: application/x-gzip
 |<- Content-Encoding: gzip

 |

 |this is clearly silly, as the file is already compressed, \
 |and decompressing it
 |will not yield the indicated content-type: application/x-gzip, \
 |but a tarball.
 |
 |maybe the w3c is wrong, or is ignored in practice or we need to handle gzip
 |specially. the problem is that some webservers compress the \

The problem is that IANA doesn't support a tar-gz MIME type, so
that mime.types(5) (tika [1] for Apache) will return "silly"
values, as in

  application/gzip              tgz gz emz
  application/x-bzip2           bz2 tbz2 boz
  # EXTENSION .tbz
  application/x-xz              xz tbz
  application/x-tar             tar

  [1] http://svn.apache.org/viewvc/tika/trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml

 |data, like you request
 |a html file and it gives you gzip back, thats why hget uncompresses.

mime.types(5) (re-)evaluating expanded content seems what IANA has
in mind with its decision (it would be all too simple if it would
just work (tm)).

--steffen

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

nope, apache is wired.

--
cinap

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

commited a work arround for this now.

--
cinap

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

haha, this appears to be an apache bug, and mozilla has work arrounds
for this. we might need todo similar thing and check for content type
as well.

--
cinap

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

found it. the server sends Content-Encoding header which causes hget
to add a decompression filter, so you get as output a tarball.

<- Content-Type: application/x-gzip
<- Content-Encoding: gzip

from the w3c:

The Content-Encoding entity-header field is used as a modifier to the media-type.
When presented, its value indicates what additional content codings have been applied
to the entity-body, and thus what decoding mechanisms must be applied in order to
obtail the media-type referenced by the Conent-Type header field. Content-Encoding
is primarily used to allow a document to be compressed without losing the
identity of its underlying media type.

this is clearly silly, as the file is already compressed, and decompressing it
will not yield the indicated content-type: application/x-gzip, but a tarball.

maybe the w3c is wrong, or is ignored in practice or we need to handle gzip
specially. the problem is that some webservers compress the data, like you request
a html file and it gives you gzip back, thats why hget uncompresses.

--
cinap

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

On Thursday 14 May 2015 13:10:20 Jens Staal wrote:
>  This might be a vbox bug (known for flaky network? I use the recommended
> settings from 9front wiki), so I will try in qemu instead.

I just tried with the archives in qemu too and got the same error (and the
same deviant md5sum) - so hget is weird?

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

On Thursday 14 May 2015 12:52:48 cinap_lenrek@felloff.net wrote:
> i just tried this and it works all fine:
>
> hget http://bitbucket.org/9front/plan9front/get/tip.tar.gz | gunzip | tar t
>
> so please give a example command with a url that gives you issues.

Hi
sorry about the lack of details before (I was on my phone so no urls handy)

I tried 2 "low hanging fruit" (=no source changes) packages for the ports -
zlib and mksh.

http://zlib.net/zlib-1.2.8.tar.gz
https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R50f.tgz

I remembered wrong... both were gzipped. I had other archives that were bz2

now when I was at my computer I took the opportunity to compare md5sums
between the archives downloaded on 9front and on Linux and they differed.
Before I just assumed that there was an issue with tar.

 This might be a vbox bug (known for flaky network? I use the recommended
settings from 9front wiki), so I will try in qemu instead.

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

i just tried this and it works all fine:

hget http://bitbucket.org/9front/plan9front/get/tip.tar.gz | gunzip | tar t

so please give a example command with a url that gives you issues.

--
cinap

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

why can't you just give a url so someone can try to reproduce it?

--
cinap

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

[-- Attachment #1: Type: text/plain, Size: 315 bytes --]

Both tar.gz (zlib official site) and tar.bz2 (mksh official site). I just
wonder if they get corrupted during transfer with hget or if there is a
different issue.
Den 14 maj 2015 10:49 skrev <cinap_lenrek@felloff.net>:

> could you be more specific what files fail to unpack with tar?
>
> --
> cinap
>
>

[-- Attachment #2: Type: text/html, Size: 559 bytes --]

Re: [9fans] Ports tree for Plan 9

[cinap_lenrek]

could you be more specific what files fail to unpack with tar?

--
cinap

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

Den 12 maj 2015 07:47 skrev <mveety@mveety.com>:
>
> Thanks Jens!  I can add you to the bitbucket if you wish so you can
> contribute at your leisure. Also, if anyone else wants commit access,
> just ask.  :) (I think bitbucket has some dumb limited commit bit
> thing though.  Hopefully I'll get off it soon.)
>
> --
> Veety
>
>

I just set up a clean 9front VM (vbox, will see if the same issue exists on
qemu/kvm) and I have an issue where tar fails on every archive downloaded
with hget.

Any ideas what the issue may be? Need to get that resolved to validate my
ports before uploading them :)

[-- Attachment #2: Type: text/html, Size: 788 bytes --]

Re: [9fans] Ports tree for Plan 9

[Adrian Regenfuss]

[-- Attachment #1: Type: text/html, Size: 1292 bytes --]

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

On Tuesday 12 May 2015 01:45:13 mveety@mveety.com wrote:
> Thanks Jens!  I can add you to the bitbucket if you wish so you can
> contribute at your leisure. Also, if anyone else wants commit access,
> just ask.  :) (I think bitbucket has some dumb limited commit bit
> thing though.  Hopefully I'll get off it soon.)
>
> --
> Veety


cool my e-mail adress used for these things is staal1978@gmail.com
I just started watching the repo.

Some form questions:
* should categories be listed according to groups similar to OpenBSD ports?
(got some stuff for "archivers", "shell", "graphics" and "lang" ... and
perhaps a few more)
* for APE packages (for example libraries that depend on other APE libraries),
should this be a specific sub-category? for example ape/graphics/[package] ?
some binaries, libraries and headers should probably be installed in
/rc/bin/ape, $objtype/bin/ape/, $objtype/lib/ape and /sys/include/ape/
* Config/ports.conf
is "catagory" intentional or a typo?

I will see when I have time to play with this... I am eager to though :)

Re: [9fans] Ports tree for Plan 9

[Ori Bernstein]

What I've wanted is less a ports tree and more a server that
takes images of some sort (tarballs?) in $home/lib/pkg and binds
the namespace in the right place. So, eg, I have "foo.tgz", which
may contain, for example:

    fortune.tgz:lib/fortunes.txt
    fortune.tgz:bin/fortune

Which would get mounted and bound in:

    /lib/fortunes.txt
    /bin/fortune

I've been thinking that it may even be possible to interpose a bit, and
provide /bin/fortune as a short stub that binds the data files into
a private namespace so that different packages can't stomp each other's
data files. Although that may be overthinking things.

On Mon, 11 May 2015 22:02:32 -0400, mveety@mveety.com wrote:

> Hey 9fans,
> 	I wrote a ports tree for 9front, but it should work fine on
> labs Plan 9. It's a bit light on software and probably has bugs,
> so I would really love comments on it and mkfiles for new software.
> Take a look at the code, try it out, tell me what you think!
>
> Ports repo: https://bitbucket.org/mveety/9front-ports , install it by
> running install.rc . It installs to /sys/ports/ .
>
> --
> Veety
>


--
    Ori Bernstein

Re: [9fans] Ports tree for Plan 9

[mveety]

Thanks Jens!  I can add you to the bitbucket if you wish so you can
contribute at your leisure. Also, if anyone else wants commit access,
just ask.  :) (I think bitbucket has some dumb limited commit bit
thing though.  Hopefully I'll get off it soon.)

--
Veety

Re: [9fans] Ports tree for Plan 9

[Jens Staal]

[-- Attachment #1: Type: text/plain, Size: 751 bytes --]

Den 12 maj 2015 04:13 skrev <mveety@mveety.com>:
>
> Hey 9fans,
>         I wrote a ports tree for 9front, but it should work fine on
> labs Plan 9. It's a bit light on software and probably has bugs,
> so I would really love comments on it and mkfiles for new software.
> Take a look at the code, try it out, tell me what you think!
>
> Ports repo: https://bitbucket.org/mveety/9front-ports , install it by
> running install.rc . It installs to /sys/ports/ .
>
> --
> Veety
>
Awesome!

Just before I had to stop playing with porting to Plan9 (family/kids taking
time), I tried to bootstrap pkgsrc. That was probably doomed to fail. This
got a better chance of success.

As soon as time allows, I will try to migrate my old ports

[-- Attachment #2: Type: text/html, Size: 1001 bytes --]

[9fans] Ports tree for Plan 9

[mveety]

Hey 9fans,
	I wrote a ports tree for 9front, but it should work fine on
labs Plan 9. It's a bit light on software and probably has bugs,
so I would really love comments on it and mkfiles for new software.
Take a look at the code, try it out, tell me what you think!

Ports repo: https://bitbucket.org/mveety/9front-ports , install it by
running install.rc . It installs to /sys/ports/ .

--
Veety