* venti survery results
@ 2008-07-04 4:18 Russ Cox
2008-07-04 9:07 ` [9fans] " Richard Miller
0 siblings, 1 reply; 8+ messages in thread
From: Russ Cox @ 2008-07-04 4:18 UTC (permalink / raw)
To: 9fans
The venti surveys have stopped trickling in. As promised,
here is a summary.
Below is a table summarizing the venti data.
Each line is one server that someone submitted.
I am surprised that there are people out there
with 4+ year old servers. You take very good care
of your disks. I suspect that most of the large but new
servers replaced older ones.
Most people who replied had less than 2 million blocks,
and I suspect most people who didn't reply have servers like
the bottom half of the table rather than the top half.
You could keep a million-block index in 50 MB of memory
and do away with the index hash table completely.
It might be worth having venti detect when the index
cache can fit the entire index and operate entirely out
of the cache, without ever needing to read from the
hash table. You could get much better performance
out of venti doing that, and not even need a bloom filter.
In fact, in that mode you wouldn't even need to configure
an index disk.
Then there are the power users, with their tens or hundreds
of millions of blocks. One GB of index cache only gets you
thirty million index entries, so these people have essentially
no choice but to maintain the hash table index with its
heavy seek penalties. Maybe in a few years flash storage will
save the day.
Russ
GB GB days
clumps cclumps raw compr age
338708170 279714997 1158.6 593.4 681 most clumps
243894713 207350734 5614.7 1657.2 69 most bytes
148582718 105977246 447.2 186.5 561
45255915 23277726 135.0 87.0 169
27395797 15624078 95.9 62.9 7
22095841 12085340 78.9 53.4 66
10194765 6088883 38.8 22.7 -
7991230 6622335 34.4 17.3 1768
6618670 2068276 46.2 39.5 60
6158824 4209478 37.7 21.2 1780
5971129 4325778 40.4 23.8 361
5805550 2246660 40.2 30.4 41
4900543 3749792 23.4 12.2 132
4151139 274551 31.3 30.4 214
2465171 1065357 16.3 12.4 1044
1756289 324119 12.6 11.4 1984 oldest
1544611 17057 23.4 23.3 359
1353686 901896 2.4 1.1 -
1308981 981760 7.3 4.0 1413
1279725 1073807 7.2 3.1 1544
1197798 961554 6.8 3.3 1436
1136953 609235 6.7 4.8 220
1082686 971796 6.9 1.9 1119
837292 347263 5.5 4.2 239
834808 549286 5.3 3.2 1164
742362 609642 4.1 1.8 490
641818 457784 3.8 2.2 1631 slowest growth
279319 222898 1.6 0.8 -
250992 168172 1.4 0.8 547
238121 215040 1.4 0.5 268
213772 183795 1.2 0.5 42
151431 134091 0.9 0.4 108
141257 121809 0.8 0.3 208
122807 105275 0.6 0.3 32
70 67 0.0 0.0 0 newest
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti survery results
2008-07-04 4:18 venti survery results Russ Cox
@ 2008-07-04 9:07 ` Richard Miller
2008-07-04 9:46 ` Kernel Panic
0 siblings, 1 reply; 8+ messages in thread
From: Richard Miller @ 2008-07-04 9:07 UTC (permalink / raw)
To: 9fans
> I am surprised that there are people out there
> with 4+ year old servers. You take very good care
> of your disks.
An arena can be older than the disk it's on -- you can copy
arenas verbatim from an old disk to a new one without changing
the timestamps.
> 641818 457784 3.8 2.2 1631 slowest growth
That's my 7-year-old thinkpad t21, with a 4-year-old venti
store on a 2-year-old disk. It's my out-and-about machine,
with most of the /usr subtree encrypted and marked '+t', which
is why the arenas grow slowly. For backup, I use replica/push
to sync with my main fossil/venti machine.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti survery results
2008-07-04 9:07 ` [9fans] " Richard Miller
@ 2008-07-04 9:46 ` Kernel Panic
2008-07-04 9:54 ` Richard Miller
0 siblings, 1 reply; 8+ messages in thread
From: Kernel Panic @ 2008-07-04 9:46 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
Richard Miller wrote:
>> I am surprised that there are people out there
>> with 4+ year old servers. You take very good care
>> of your disks.
>>
>
> An arena can be older than the disk it's on -- you can copy
> arenas verbatim from an old disk to a new one without changing
> the timestamps.
>
1756289 324119 12.6 11.4 1984 oldest
That is the case... I wanted to save some power and I migrated from my old
P-II 4x IDE disk server to a tiny VIA C7 2x SATA disk one. They are
always running and are backed up by UPS.
I use always fs mirror for the arena partitions with 2 disks from different
brands. but no luck yet... no disks got bad on these machines so far.
By the way... the VIA machine is the one that got me lots of trouble
with the
SATA until i hacked a bruteforce retry-n-reset-loop in the ide driver,
now works
perfectly stable (84 days uptime for now)
:-)
cinap
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti survery results
2008-07-04 9:46 ` Kernel Panic
@ 2008-07-04 9:54 ` Richard Miller
2008-07-04 11:03 ` Kernel Panic
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Richard Miller @ 2008-07-04 9:54 UTC (permalink / raw)
To: 9fans
> the VIA machine is the one that got me lots of trouble
> with the
> SATA until i hacked a bruteforce retry-n-reset-loop in the ide driver,
Me too. Maybe one of us should submit a patch?
-- Richard
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti survery results
2008-07-04 9:54 ` Richard Miller
@ 2008-07-04 11:03 ` Kernel Panic
2008-07-04 14:04 ` erik quanstrom
2008-07-05 20:36 ` [9fans] via vt8237 erik quanstrom
2 siblings, 0 replies; 8+ messages in thread
From: Kernel Panic @ 2008-07-04 11:03 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
Richard Miller wrote:
>> the VIA machine is the one that got me lots of trouble
>> with the
>> SATA until i hacked a bruteforce retry-n-reset-loop in the ide driver,
>>
>
> Me too. Maybe one of us should submit a patch?
>
this is the driver that works for me:
http://9hal.ath.cx/usr/cinap_lenrek/viamod/sdata.c
> -- Richard
>
I'm lazy as hell
cinap
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti survery results
2008-07-04 9:54 ` Richard Miller
2008-07-04 11:03 ` Kernel Panic
@ 2008-07-04 14:04 ` erik quanstrom
2008-07-05 21:42 ` Richard Miller
2008-07-05 20:36 ` [9fans] via vt8237 erik quanstrom
2 siblings, 1 reply; 8+ messages in thread
From: erik quanstrom @ 2008-07-04 14:04 UTC (permalink / raw)
To: 9fans
>> the VIA machine is the one that got me lots of trouble
>> with the
>> SATA until i hacked a bruteforce retry-n-reset-loop in the ide driver,
>
> Me too. Maybe one of us should submit a patch?
is this one of the via chipsets that may also be driven as ahci?
ahci should show up with class/subclass/ccrp as 01.06.01. as in:
x.y.z: disk 01.06.01 vid/did 11 0:x 16 1:x 16 2:x 16 3:x 16 4:x 16 5:y != 0 1024 (or bigger)
i was just taking a look at the linux driver and it looks like
nvidia and sis should just work. (anybody with such hardware care to
test?) there were just a couple of tweeks that the linux guys felt
necessary with via, but i didn't look carefully enough to see of they
apply to the plan 9 driver, as i have no hardware.
- erik
^ permalink raw reply [flat|nested] 8+ messages in thread
* [9fans] via vt8237
2008-07-04 9:54 ` Richard Miller
2008-07-04 11:03 ` Kernel Panic
2008-07-04 14:04 ` erik quanstrom
@ 2008-07-05 20:36 ` erik quanstrom
2 siblings, 0 replies; 8+ messages in thread
From: erik quanstrom @ 2008-07-05 20:36 UTC (permalink / raw)
To: 9fans, 9fans
i think the posted patch might not be
appropriate for controllers not having
the via controllers' problem as it is quite
quick to reset the controller if a command
takes a long time. with a properly
functioning controller, it is generally a
bad idea to do a reset with a command in
flight.
i haven't seen any other patches so i don't
know how they work.
i also took a look at the linux driver for
the same hardware and didn't find the same
reset loop. i did however find some pci
setup code specific to this chipset which
i posted some time ago
http://9fans.net/archive/2008/02/277
this code was translated from the linux.
i don't know if this was ever tried. but
it tries to attack the root of the problem,
missed interrupts, by setting some pcie
magic.
i can't try this myself as i don't have
appropriately buggy hardware.
if this doesn't work, it might make sense
to have a controller setting to optionally
check for missed interrupts.
this will allow us to still tell the difference
between an interrupt that got lost and a
controller that's gone off into the weeds.
i've put up a driver that may work if
this is the case but it's not tested due to
lack of hardware. sources is down, so it
can be accessed here:
hget http://www.quanstro.net/vtsdata.c
- erik
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti survery results
2008-07-04 14:04 ` erik quanstrom
@ 2008-07-05 21:42 ` Richard Miller
0 siblings, 0 replies; 8+ messages in thread
From: Richard Miller @ 2008-07-05 21:42 UTC (permalink / raw)
To: 9fans
> is this one of the via chipsets that may also be driven as ahci?
>
> ahci should show up with class/subclass/ccrp as 01.06.01. as in:
> x.y.z: disk 01.06.01 vid/did 11 0:x 16 1:x 16 2:x 16 3:x 16 4:x 16 5:y != 0 1024 (or bigger)
Depending on whether I select SATA mode = RAID or IDE in the bios,
I get one of these:
0.15.0: disk 01.04.00 1106/3149 11 0:0000ea01 16 1:0000e001 16 2:0000e101 16 3:0000e201 16 4:0000e301 16 5:0000d801 256
0.15.0: disk 01.01.8f 1106/3149 11 0:0000ea01 16 1:0000e001 16 2:0000e101 16 3:0000e201 16 4:0000e301 16 5:0000d801 256
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2008-07-05 21:42 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-07-04 4:18 venti survery results Russ Cox
2008-07-04 9:07 ` [9fans] " Richard Miller
2008-07-04 9:46 ` Kernel Panic
2008-07-04 9:54 ` Richard Miller
2008-07-04 11:03 ` Kernel Panic
2008-07-04 14:04 ` erik quanstrom
2008-07-05 21:42 ` Richard Miller
2008-07-05 20:36 ` [9fans] via vt8237 erik quanstrom
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).