Re: [9fans] file server page faulting

Re: [9fans] file server page faulting

[nigel]

[-- Attachment #1: Type: text/plain, Size: 1003 bytes --]

It really ought not to page fault for a bad block. The file server
driver certainly recovers from bad block accesses, as I saw
quite a few on my server this summer. Unfortunately, as my
fake worm filled up, I have pulled the scsi drives from the server,
replacing them with IDE as the prices make it prohibitive to
work any other way. 70Gb SCSI drives are £450 +VAT here,
whereas 80GB IDE are <£80 +VAT. You can mirror the drives
and still spend less than half for the same capacity.

I still have SCSI in my terminal.

However, there are compatibility problems between 53c8xx
chips and PCI busses, which is nothing to do with Plan 9. I
certainly found that old controllers (53c810s) did not
work reliabuly in PCI 2.1 busses; bus faults I think. I guess
that the same issues could exist with a new controller and
an old motherboard, but I don't have any old motherboards
any more.

It is certainly supported hardware, but only if the hardware
is plugged into a compatible socket.

[-- Attachment #2: Type: message/rfc822, Size: 3615 bytes --]

From: Andrew <afrayedknot@thefrayedknot.armory.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] file server page faulting
Date: Fri, 13 Sep 2002 08:23:48 -0700
Message-ID: <20020913152348.GA19419@thefrayedknot.armory.com>

On Fri, Sep 13, 2002 at 04:14:04PM +0100, nigel@9fs.org wrote:
> > the hardware is an older pentium (90?) it has 40 mb of ram with integrated
> > s3 video and uses a symbios scsi adapter with a 2.5gb hard drive. No
> > dump media is attached. The ethernet card is a i82557 mini-nic. I can
> > make this happen by usually accessing lots and lotsof data. For example
> > if i cat /dev/zero into a file then cat it back out a couple of times
> > it usually breaks. I swapped the scsi setup into another system (my
> > terminal actually), which does not have an integrated video controller
> > and the same type of nic. It had the same problem. The crash is known
> > to happen seemingly sporadically also.
> >
> > is it worth reinstalling onto ide drives? I can backup all my changes
> > fairly easily.
>
> It may be. When it breaks, what is the address it happens at?
> When you say integrated S3 do you mean in the chip set? Does it used
> shared memory?
i will try and get the address.
>
> The possible issues here are
>
> 1. the S3 is sharing the memory; there is no allowance for this, and
>     when the buffer cache meets the frame buffer horrible things happen

the reason i doubt this is that i get the same results on a different
machine that does not have an integrated controller, it has an isa video
card of trident genre so i doubt that the memory is shared between those two.

>
> 2. which symbios controller? check that the PCI bus specs. match, as
>     older symbios controllers don't work on newer PCI busses
it is a fairly new, working, symbios controller I am 99% certain it is
the one specified in the supported hardware section. i will check when
i get home from work. but i believe its listed as scsi0=type=ncr53c8xx
in the plan9.ini file.
>
> If it is the scsi controller, and the bus specs. match, then I get more
> interested, having written the symbios driver.
>
Im planning on putting the scsi setup in one of my linux machines and
seeing what happens if i cat the whole drive out. That way if theres
any blocks that its having trouble with i can find out.

thanks for your help.

Re: [9fans] file server page faulting

[Andrew]

On Fri, Sep 13, 2002 at 04:37:34PM +0100, nigel@9fs.org wrote:
> It really ought not to page fault for a bad block. The file server
> driver certainly recovers from bad block accesses, as I saw
> quite a few on my server this summer. Unfortunately, as my
> fake worm filled up, I have pulled the scsi drives from the server,
> replacing them with IDE as the prices make it prohibitive to
> work any other way. 70Gb SCSI drives are ??450 +VAT here,
> whereas 80GB IDE are <??80 +VAT. You can mirror the drives
> and still spend less than half for the same capacity.
>
> I still have SCSI in my terminal.
>
> However, there are compatibility problems between 53c8xx
> chips and PCI busses, which is nothing to do with Plan 9. I
> certainly found that old controllers (53c810s) did not
> work reliabuly in PCI 2.1 busses; bus faults I think. I guess
> that the same issues could exist with a new controller and
> an old motherboard, but I don't have any old motherboards
> any more.
>
> It is certainly supported hardware, but only if the hardware
> is plugged into a compatible socket.

one more point of interest: when i first went to setup the file server i
had two drives on the scsi bus and i tried to have them both interleave
as the main drive (i followed the wiki's instructions). When I went to
do a replica it would always crash.

if you are interested I can send send the hardware to you for you to
work on, i have no shortage of machines (just scsi interfaces).

Re: [9fans] file server page faulting

[nigel]

[-- Attachment #1: Type: text/plain, Size: 168 bytes --]

What does the kernel say when it detects the card? I'm not aware
of a 53c810ae; there's no manual for it on the lsi logic website, so
wonder if the 'e' is relevant.

[-- Attachment #2: Type: message/rfc822, Size: 2336 bytes --]

From: Andrew <afrayedknot@thefrayedknot.armory.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] file server page faulting
Date: Mon, 16 Sep 2002 08:30:00 -0700
Message-ID: <20020916153000.GB8246@thefrayedknot.armory.com>

On Mon, Sep 16, 2002 at 08:13:51AM -0700, Jack Johnson wrote:
> Douglas A. Gwyn wrote:
> >like this, but others are fine.  I'm using an old ASUS SC200 and a
> >cheaper, newer generic 810-based PCI controller on my Pentium IIIs
> >under several different OSes.
>
> Could you tell us which card that is?
symbios 53c810ae

it doesnt matter though since the problem was actually a bad nic (i hope
i hope i hope!)and I believe i was getting the same problems with ide
disks (which is what im using in my 'new' file server).

> I'm always on the lookout for cheaper.  ;)
>
cheaper gave me a motherboard with onboard sound and ethernet neither
of which i can use because certain sequences of DMA accesses will lock
the the motherboard.

the only kind of cheap hardware i get now is of the older and free variety.

Re: [9fans] file server page faulting

[Andrew]

On Mon, Sep 16, 2002 at 08:13:51AM -0700, Jack Johnson wrote:
> Douglas A. Gwyn wrote:
> >like this, but others are fine.  I'm using an old ASUS SC200 and a
> >cheaper, newer generic 810-based PCI controller on my Pentium IIIs
> >under several different OSes.
>
> Could you tell us which card that is?
symbios 53c810ae

it doesnt matter though since the problem was actually a bad nic (i hope
i hope i hope!)and I believe i was getting the same problems with ide
disks (which is what im using in my 'new' file server).

> I'm always on the lookout for cheaper.  ;)
>
cheaper gave me a motherboard with onboard sound and ethernet neither
of which i can use because certain sequences of DMA accesses will lock
the the motherboard.

the only kind of cheap hardware i get now is of the older and free variety.

Re: [9fans] file server page faulting

[Jack Johnson]

Douglas A. Gwyn wrote:
> like this, but others are fine.  I'm using an old ASUS SC200 and a
> cheaper, newer generic 810-based PCI controller on my Pentium IIIs
> under several different OSes.

Could you tell us which card that is?

I'm always on the lookout for cheaper.  ;)

-Jack

Re: [9fans] file server page faulting

[Douglas A. Gwyn]

Andrew wrote:
> >     older symbios controllers don't work on newer PCI busses
> it is a symbios 53c810ae is that indicative of anything?

I don't think it's a matter of age of the controller as much as
to whether it was built for a restrictive subset of PCI, such as
INTA-only.  Some of the cheaper 810-based controllers were indeed
like this, but others are fine.  I'm using an old ASUS SC200 and a
cheaper, newer generic 810-based PCI controller on my Pentium IIIs
under several different OSes.

Re: [9fans] file server page faulting

[Andrew]

for all interested parties the problem has turned out (apparently) to
be a bad nic. I have replaced the nic in the auth server and all the
normal things i did before to crash it have worked just fine. Thanks to
all those who helped.

Andrew

Re: [9fans] file server page faulting

[nigel]

> maybe this will further confuse matters but provide more information on
> this problem:
>

Yes, I think so. The problem seems more motherboard related.

Re: [9fans] file server page faulting

[Geoff Collyer]

> 1. the S3 is sharing the memory; there is no allowance for this, and
>    when the buffer cache meets the frame buffer horrible things happen

My main file server has the SiS 630E chipset in it, which includes a
built-in VGA chipset that shares host memory.  The motherboard manual
says it's the SiS 300 AGP VGA.  I added a trivial "pcihinv" command
that just calls the otherwise-unused pcihinv function
(/sys/src/fs/pc/pci.c:/^pcihinv).  Here's what pcihinv had to say:

 bus dev type vid  did intl memory
 [ ... ]
 1   0/0 0300 1039 6300 11  0:f0000008 134217728 1:eb000000 131072 2:0000a801 128

Aperture size and shared memory size may be embedded there somewhere.
According to vendor and device ids, this is an SiS 630 integrated AGP
VGA at 0xa800.  Who knows?  The SiS 300 has a different device id
(0x300) but some software (e.g., XFree86) treats the 300 and 630 (and
540) similarly.  SiS's numbering seems to be intended to obfuscate.

I'd been worried about the shared memory too and when I first set up
the file server, had odd crashes shortly after boot, so I inserted a
PCI VGA card into one of the whopping two (002) PCI slots, and told
the BIOS to use it, not the built-in VGA.  Eventually, the odd crashes
turned out to be an uninitialised FPU, but even after fixing that, I
was uneasy about the built-in VGA and so left the PCI VGA card in.
However, I have gradually come to the conclusion that, at least for my
VGA chipset, it's not a problem for these reasons:

- the file server never enables VGA mode but runs the console in CGA
mode, which doesn't need much memory anyway (it can only display 2,000
characters),

- my BIOS reports that the system has 510MB of normal memory + 2MB of
shared video memory (it has 512MB of RAM), now that I've cut the
shared memory size down to 2MB in the BIOS (and the aperture down to
4MB), and

- prints that I added report that memory bank1 runs up to 0x1fe00000
(534773760 = 510MB exactly), so I assume that the BIOS is claiming
only 510MB of RAM or the memory sizing code detects that the memory at
510MB through 512MB behaves strangely and so excludes it.

I then built an fs kernel with devwrite() stubbed out, removed the PCI
VGA card, booted the no-writes fs kernel, read well over 512MB of
distinct data, checked all the file systems (they are roughly 20GB or
40GB each), and everything was still behaving.  The checks are
significant only because they consume RAM.  After running for a while
longer, I then booted my normal fs kernel and things are still okay.

	last boot Thu Sep 12 02:18:44 2002

It's now been over two days, and it's still behaving.

Re: [9fans] file server page faulting

[Andrew]

maybe this will further confuse matters but provide more information on
this problem:

I swapped the file server and cpu/auth server. Now the former auth
server has the scsi interface and bootup disk, and the file server has
the cpu/auth boot disk. Ive yet to make the new file server page fault
the same way, but the cpu/auth server seems to just stop responding if i
put any sort of load on it. Once that happens the file server no longer
lets anything get files off of it. You reboot the auth server and it
gets to version... and sits there. I end up having to reboot the file
server. Ive tested out another machine as the auth server in this setup,
it had also page faulted as the file server. The only visible errors
were something about a lock being held on the file server.


and to confuse things /more/ prior to trying this out, in my original
configuration i attempted to use an ide disk. It page faulted or more
often stopped responding in the same fashion.

thanks for any help.
Andrew

Re: [9fans] file server page faulting

[Andrew]

just a thought, does parity vs non-parity have anything to do with
this? I noticed in the bios that the memory is non-parity.  perhaps its
just old slow (cheap) ram?

Re: [9fans] file server page faulting

[Andrew]

> 1. the S3 is sharing the memory; there is no allowance for this, and
>     when the buffer cache meets the frame buffer horrible things happen
>
> 2. which symbios controller? check that the PCI bus specs. match, as
>     older symbios controllers don't work on newer PCI busses

it is a symbios 53c810ae is that indicative of anything?

Re: [9fans] file server page faulting

[Fco.J.Ballesteros]

Unfortunatelly, the problem dissapeared here before I could trace
it, and I've not been able to reproduce it. In may case, I ended up
thinking that the problem was due to a race condition in the ether driver
that did show up for arp frames that were *s l o w* to be replied.
I'd say you could try just replacing the fs card and see what happens.
If the problem terminates, then it would mean that it was the race and that
it's still there (and I could try to fix it, with your help).

Re: [9fans] file server page faulting

[Andrew]

On Fri, Sep 13, 2002 at 05:06:49PM +0200, Fco. J. Ballesteros wrote:
> Have you tried using a different ether card?

i will, the unfortunate thing is, i am short on 100mbit ethernet cards,
except for those type or nics (of which i have 25-30). but as a debugging
exercise I will see if things change.

should i change the nics in the cpu/auth server as well? ive been able
to load it pretty heavily and not have any problems.

Re: [9fans] file server page faulting

[Andrew]

On Fri, Sep 13, 2002 at 04:14:04PM +0100, nigel@9fs.org wrote:
> > the hardware is an older pentium (90?) it has 40 mb of ram with integrated
> > s3 video and uses a symbios scsi adapter with a 2.5gb hard drive. No
> > dump media is attached. The ethernet card is a i82557 mini-nic. I can
> > make this happen by usually accessing lots and lotsof data. For example
> > if i cat /dev/zero into a file then cat it back out a couple of times
> > it usually breaks. I swapped the scsi setup into another system (my
> > terminal actually), which does not have an integrated video controller
> > and the same type of nic. It had the same problem. The crash is known
> > to happen seemingly sporadically also.
> >
> > is it worth reinstalling onto ide drives? I can backup all my changes
> > fairly easily.
>
> It may be. When it breaks, what is the address it happens at?
> When you say integrated S3 do you mean in the chip set? Does it used
> shared memory?
i will try and get the address.
>
> The possible issues here are
>
> 1. the S3 is sharing the memory; there is no allowance for this, and
>     when the buffer cache meets the frame buffer horrible things happen

the reason i doubt this is that i get the same results on a different
machine that does not have an integrated controller, it has an isa video
card of trident genre so i doubt that the memory is shared between those two.

>
> 2. which symbios controller? check that the PCI bus specs. match, as
>     older symbios controllers don't work on newer PCI busses
it is a fairly new, working, symbios controller I am 99% certain it is
the one specified in the supported hardware section. i will check when
i get home from work. but i believe its listed as scsi0=type=ncr53c8xx
in the plan9.ini file.
>
> If it is the scsi controller, and the bus specs. match, then I get more
> interested, having written the symbios driver.
>
Im planning on putting the scsi setup in one of my linux machines and
seeing what happens if i cat the whole drive out. That way if theres
any blocks that its having trouble with i can find out.

thanks for your help.

Re: [9fans] file server page faulting

[nigel]

> the hardware is an older pentium (90?) it has 40 mb of ram with integrated
> s3 video and uses a symbios scsi adapter with a 2.5gb hard drive. No
> dump media is attached. The ethernet card is a i82557 mini-nic. I can
> make this happen by usually accessing lots and lotsof data. For example
> if i cat /dev/zero into a file then cat it back out a couple of times
> it usually breaks. I swapped the scsi setup into another system (my
> terminal actually), which does not have an integrated video controller
> and the same type of nic. It had the same problem. The crash is known
> to happen seemingly sporadically also.
>
> is it worth reinstalling onto ide drives? I can backup all my changes
> fairly easily.

It may be. When it breaks, what is the address it happens at?
When you say integrated S3 do you mean in the chip set? Does it used
shared memory?

The possible issues here are

1. the S3 is sharing the memory; there is no allowance for this, and
    when the buffer cache meets the frame buffer horrible things happen

2. which symbios controller? check that the PCI bus specs. match, as
    older symbios controllers don't work on newer PCI busses

If it is the scsi controller, and the bus specs. match, then I get more
interested, having written the symbios driver.

Re: [9fans] file server page faulting

[Fco.J.Ballesteros]

Have you tried using a different ether card?

Re: [9fans] file server page faulting

[Andrew]

On Thu, Sep 12, 2002 at 07:57:55AM +0100, nigel@9fs.org wrote:
> Can't say I've ever got a page fault out of 9fs. Plenty of other
> things.
>
> As the kernel does not demand page, page faults should not happen.
> You are down to hardware or software bugs. A first step would be
> to reveal the hardware involved. If you have a motherboard with
> a shared memory vga, for example, then you could be in deep trouble.
> You can also check where the page fault occurs in case it is at a
> consistent point in the kernel, which would suggest a software bug.
> If it wanders around randomly, then it's probably hardware.

the hardware is an older pentium (90?) it has 40 mb of ram with integrated
s3 video and uses a symbios scsi adapter with a 2.5gb hard drive. No
dump media is attached. The ethernet card is a i82557 mini-nic. I can
make this happen by usually accessing lots and lotsof data. For example
if i cat /dev/zero into a file then cat it back out a couple of times
it usually breaks. I swapped the scsi setup into another system (my
terminal actually), which does not have an integrated video controller
and the same type of nic. It had the same problem. The crash is known
to happen seemingly sporadically also.

is it worth reinstalling onto ide drives? I can backup all my changes
fairly easily.

Re: [9fans] file server page faulting

[FJ Ballesteros]

That happen to me with sargazos before some changes prior to 4th ed
fixed(?)
my problem. The problem had to do with a very long delay between arp
requests and replies, that I think put the ether driver into throuble.



Andrew ha escrito:
>
> I posted a few days ago with one of my concerns about how my network
> loses its connections. well the problem is that the file server page
> faults, crashes and then reboots. So now the question is, why does it
> page fault? I have tried out the emelie kernel and saragazo. Same problem
> in both cases. what has been known to do this?
>
> Thanks in advance.
>
>         Andrew

Re: [9fans] file server page faulting

[nigel]

[-- Attachment #1: Type: text/plain, Size: 543 bytes --]

Can't say I've ever got a page fault out of 9fs. Plenty of other
things.

As the kernel does not demand page, page faults should not happen.
You are down to hardware or software bugs. A first step would be
to reveal the hardware involved. If you have a motherboard with
a shared memory vga, for example, then you could be in deep trouble.
You can also check where the page fault occurs in case it is at a
consistent point in the kernel, which would suggest a software bug.
If it wanders around randomly, then it's probably hardware.

[-- Attachment #2: Type: message/rfc822, Size: 1723 bytes --]

From: Andrew <afrayedknot@thefrayedknot.armory.com>
To: 9fans@cse.psu.edu
Subject: [9fans] file server page faulting
Date: Wed, 11 Sep 2002 21:50:39 -0700
Message-ID: <20020912045039.GA26537@thefrayedknot.armory.com>

I posted a few days ago with one of my concerns about how my network
loses its connections. well the problem is that the file server page
faults, crashes and then reboots. So now the question is, why does it
page fault? I have tried out the emelie kernel and saragazo. Same problem
in both cases. what has been known to do this?

Thanks in advance.

        Andrew

[9fans] file server page faulting

[Andrew]

I posted a few days ago with one of my concerns about how my network
loses its connections. well the problem is that the file server page
faults, crashes and then reboots. So now the question is, why does it
page fault? I have tried out the emelie kernel and saragazo. Same problem
in both cases. what has been known to do this?

Thanks in advance.

        Andrew