* [9fans] 9bench: lmbench for Plan 9 @ 2008-03-21 17:21 john 2008-03-21 17:51 ` erik quanstrom 0 siblings, 1 reply; 9+ messages in thread From: john @ 2008-03-21 17:21 UTC (permalink / raw) To: 9fans I've just finished porting what we thought were the 8 most useful benchmarks from the lmbench v.2 suite to run natively in Plan 9. The benchmarks are: bw_file_rd: Measure file read bandwidth bw_mem: Measure memory bandwidth for a several different operations bw_pipe: Measure pipe bandwidth bw_tcp: Measure tcp bandwidth on the local machine lat_ctx: Measure context switch latency lat_syscall: Measure system call latency lat_tcp: Measure tcp latency on local system lat_udp: Measure udp latency on local system As I note in the README, lmbench's license carries the additional restriction that you cannot publish results from modified benchmarks, so keep your results to yourself. The source is in /n/sources/contrib/john/9bench.tgz John ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [9fans] 9bench: lmbench for Plan 9 2008-03-21 17:21 [9fans] 9bench: lmbench for Plan 9 john @ 2008-03-21 17:51 ` erik quanstrom 2008-03-21 18:55 ` ron minnich 0 siblings, 1 reply; 9+ messages in thread From: erik quanstrom @ 2008-03-21 17:51 UTC (permalink / raw) To: 9fans > As I note in the README, lmbench's license carries the additional > restriction that you cannot publish results from modified benchmarks, > so keep your results to yourself. doesn't generating unpublishable results defeat the purpose of a benchmark? - erik ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [9fans] 9bench: lmbench for Plan 9 2008-03-21 17:51 ` erik quanstrom @ 2008-03-21 18:55 ` ron minnich 2008-03-21 21:28 ` erik quanstrom 0 siblings, 1 reply; 9+ messages in thread From: ron minnich @ 2008-03-21 18:55 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Fri, Mar 21, 2008 at 10:51 AM, erik quanstrom <quanstro@coraid.com> wrote: > > As I note in the README, lmbench's license carries the additional > > restriction that you cannot publish results from modified benchmarks, > > so keep your results to yourself. > > doesn't generating unpublishable results defeat the purpose of a benchmark? what is the purpose? ron ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [9fans] 9bench: lmbench for Plan 9 2008-03-21 18:55 ` ron minnich @ 2008-03-21 21:28 ` erik quanstrom 2008-03-22 11:18 ` hiro 0 siblings, 1 reply; 9+ messages in thread From: erik quanstrom @ 2008-03-21 21:28 UTC (permalink / raw) To: 9fans > On Fri, Mar 21, 2008 at 10:51 AM, erik quanstrom <quanstro@coraid.com> wrote: > > > As I note in the README, lmbench's license carries the additional > > > restriction that you cannot publish results from modified benchmarks, > > > so keep your results to yourself. > > > > doesn't generating unpublishable results defeat the purpose of a benchmark? > > what is the purpose? i would assume that the main reason to go with an official benchmark is to compare results. if you don't need to compare results and you want to tell if something you've changed has made things faster or slower, i would think rolling one's own would be simple and effective. "time dd ..." is often a great relative measure. - erik ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [9fans] 9bench: lmbench for Plan 9 2008-03-21 21:28 ` erik quanstrom @ 2008-03-22 11:18 ` hiro 2008-03-22 13:53 ` Francisco J Ballesteros 0 siblings, 1 reply; 9+ messages in thread From: hiro @ 2008-03-22 11:18 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Perhaps the copyright owner just wanted to make fun of intellectual property;) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [9fans] 9bench: lmbench for Plan 9 2008-03-22 11:18 ` hiro @ 2008-03-22 13:53 ` Francisco J Ballesteros 2008-03-22 13:56 ` erik quanstrom 0 siblings, 1 reply; 9+ messages in thread From: Francisco J Ballesteros @ 2008-03-22 13:53 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Is a port of lmbench considered a modified benchmark? I understand you cannot publish results saying it´s the output of lmbench if you changed the bechmark, but only because you used a ported one...? Funny in any case. On Sat, Mar 22, 2008 at 12:18 PM, hiro <23hiro@googlemail.com> wrote: > Perhaps the copyright owner just wanted to make fun of intellectual property;) > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [9fans] 9bench: lmbench for Plan 9 2008-03-22 13:53 ` Francisco J Ballesteros @ 2008-03-22 13:56 ` erik quanstrom 2008-03-22 23:27 ` john 0 siblings, 1 reply; 9+ messages in thread From: erik quanstrom @ 2008-03-22 13:56 UTC (permalink / raw) To: 9fans > Is a port of lmbench considered a modified benchmark? > I understand you cannot publish results saying it´s the output > of lmbench if you changed the bechmark, but only because you > used a ported one...? > > Funny in any case. you're also not allowed to publish partial results. they have really overthought things. they are worried that someone will try to look better than they are by publishing partial results. - erik ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [9fans] 9bench: lmbench for Plan 9 2008-03-22 13:56 ` erik quanstrom @ 2008-03-22 23:27 ` john 2008-03-23 1:48 ` erik quanstrom 0 siblings, 1 reply; 9+ messages in thread From: john @ 2008-03-22 23:27 UTC (permalink / raw) To: 9fans >> Is a port of lmbench considered a modified benchmark? >> I understand you cannot publish results saying it´s the output >> of lmbench if you changed the bechmark, but only because you >> used a ported one...? >> >> Funny in any case. > > you're also not allowed to publish partial results. they have > really overthought things. they are worried that someone > will try to look better than they are by publishing partial results. > > - erik Apparently he also didn't want companies (read: Sun) rewriting portions of the benchmark to make their software/hardware look better. Since Plan 9 doesn't have sockets or the bzero call or a number of other things that you'll find on Unix, I had to do some rewriting. I made sure to accomplish the same tasks as on Unix, but in some cases more than others I had to do a lot of changing. The timing code should still be pretty much the same, although I added a little to include counting of clock cycles (approximate). The pipe bandwidth test, if I remember correctly, took basically no changes to port (I think I had to change sprintf to sprint or something like that), but others, like the networking ones, required that I essentially cut out the majority of the tests and replace them with a Plan 9 equivalent. If anyone is running these and finds a test that has the wrong behavior, please let me know so I can fix it. As for the utility of the benchmarks, come on. I know there are people on this list who write kernel code (hmm, I seem to recall somebody on here who does a lot of work on networking drivers...) and they may find it useful to get these kinds of measurements to see if changes have improved performance. Besides, there's nothing to stop you from running the benchmarks on Linux and Plan 9 and comparing (for yourself) the difference in performance--you just can't try for e-cred by posting your results on comp.os.linux (hint: the comparisons don't come out very favorably for Plan 9 anyway). Oh, and if anybody ever wants to see the Macros From Hell, just take a look at bench.h for BENCH0, BENCH, BENCH1, and BENCH-INNER. (disclaimer: I didn't write those) John ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [9fans] 9bench: lmbench for Plan 9 2008-03-22 23:27 ` john @ 2008-03-23 1:48 ` erik quanstrom 0 siblings, 0 replies; 9+ messages in thread From: erik quanstrom @ 2008-03-23 1:48 UTC (permalink / raw) To: 9fans > As for the utility of the benchmarks, come on. I know there are people > on this list who write kernel code (hmm, I seem to recall somebody > on here who does a lot of work on networking drivers...) and they > may find it useful to get these kinds of measurements to see if > changes have improved performance. Besides, there's nothing to stop you > from running the benchmarks on Linux and Plan 9 and comparing (for > yourself) the difference in performance--you just can't try for > e-cred by posting your results on comp.os.linux (hint: the > comparisons don't come out very favorably for Plan 9 anyway). i tend to use very simple timing runs in my testing. i use dd for AoE and disk drivers and a small program named zerohose for (other) network protcols. time(1) seems good enough for timing given an appropriate amount of i/o. the reason for the simple timing tests is because good hardware is hard to get wrong but marginal hardware (bad design, undersized buffers, unsufficient pcie lanes) is typically very sensitve to cache pecularities. even with simple tests some hardware can have >20% variance between runs. it's very hard to reason about this sort of thing. i'd hate to have to also have to understand what the benchmark is doing for^wto me, too. oh, plan 9 can push packets faster on the nics i've tested than linux, but for some reason, plan 9 tcp performance hasn't been very good, especially on 10gbe hardware. il and aoe fair much better. - erik ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2008-03-23 1:48 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2008-03-21 17:21 [9fans] 9bench: lmbench for Plan 9 john 2008-03-21 17:51 ` erik quanstrom 2008-03-21 18:55 ` ron minnich 2008-03-21 21:28 ` erik quanstrom 2008-03-22 11:18 ` hiro 2008-03-22 13:53 ` Francisco J Ballesteros 2008-03-22 13:56 ` erik quanstrom 2008-03-22 23:27 ` john 2008-03-23 1:48 ` erik quanstrom
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).