Tech Insider					     Technology and Trends


			      USENET Archives

From: groud...@iplus.fr (Gerard Roudier)
Subject: Unices are created equal, but ...
Date: 1996/04/13
Message-ID: <Pine.LNX.3.91.960413224027.150A-100000@gerard>
X-Deja-AN: 147397816
sender: n...@camelot.de
content-type: TEXT/PLAIN; charset=US-ASCII
organization: Mail2News Gateway at nasim
mime-version: 1.0
newsgroups: muc.lists.freebsd.hackers


Hi all,

I was implementing some performances enhancement for "Unix A" kernel.
Seems to work fine.
I had a look for another Unix in order to compare performances.
I had luck, since "Unix B" is installed on my machine on the same
hard disk.
Unix B is installed at the beginning of the disk media and Unix A at the end.
Unix B should have better IO throughput (see below if that's ok or not ok).

Then I run the first benchmark I found to prepare the tests.

I get the following results:

P90/Plato/24MB/NCR53C810/IBMS12.

  BYTE UNIX Benchmarks (Version 3.11)
  System -- Unix A gerard 1.3.87 #31 Sat Apr 13 18:34:46 GMT 1996 i586
  Start Benchmark Run: Sat Apr 13 21:25:08 GMT 1996
   1 interactive users.
Dhrystone 2 without register variables   121950.7 lps   (10 secs, 1 samples)
Dhrystone 2 using register variables     121973.7 lps   (10 secs, 1 samples)
Arithmetic Test (type = arithoh)         415167.1 lps   (10 secs, 1 samples)
Arithmetic Test (type = register)         12996.9 lps   (10 secs, 1 samples)
Arithmetic Test (type = short)            12121.0 lps   (10 secs, 1 samples)
Arithmetic Test (type = int)              12998.6 lps   (10 secs, 1 samples)
Arithmetic Test (type = long)             12993.7 lps   (10 secs, 1 samples)
Arithmetic Test (type = float)            15954.7 lps   (10 secs, 1 samples)
Arithmetic Test (type = double)           15946.0 lps   (10 secs, 1 samples)
System Call Overhead Test                 65139.8 lps   (10 secs, 1 samples)
Pipe Throughput Test                      68105.2 lps   (10 secs, 1 samples)
Pipe-based Context Switching Test         22788.7 lps   (10 secs, 1 samples)
Process Creation Test                       774.4 lps   (10 secs, 1 samples)
Execl Throughput Test                       267.6 lps   (9 secs, 1 samples)
File Read  (10 seconds)                  209771.0 KBps  (10 secs, 1 samples)
File Write (10 seconds)                   18000.0 KBps  (10 secs, 1 samples)
File Copy  (10 seconds)                    4116.0 KBps  (10 secs, 1 samples)
File Read  (30 seconds)                  212303.0 KBps  (30 secs, 1 samples)
File Write (30 seconds)                   18400.0 KBps  (30 secs, 1 samples)
File Copy  (30 seconds)                    3441.0 KBps  (30 secs, 1 samples)
C Compiler Test                             120.4 lpm   (60 secs, 1 samples)
Shell scripts (1 concurrent)                265.0 lpm   (60 secs, 1 samples)
Shell scripts (2 concurrent)                139.0 lpm   (60 secs, 1 samples)
Shell scripts (4 concurrent)                 71.0 lpm   (60 secs, 1 samples)
Shell scripts (8 concurrent)                 36.0 lpm   (60 secs, 1 samples)
Dc: sqrt(2) to 99 decimal places           9098.9 lpm   (60 secs, 1 samples)
Recursion Test--Tower of Hanoi             2140.5 lps   (10 secs, 1 samples)


                     INDEX VALUES            
TEST                                        BASELINE     RESULT      INDEX

Arithmetic Test (type = double)               2541.7    15946.0        6.3
Dhrystone 2 without register variables       22366.3   121950.7        5.5
Execl Throughput Test                           16.5      267.6       16.2
File Copy  (30 seconds)                        179.0     3441.0       19.2
Pipe-based Context Switching Test             1318.5    22788.7       17.3
Shell scripts (8 concurrent)                     4.0       36.0        9.0
                                                                 =========
     SUM of  6 items                                                  73.5
     AVERAGE                                                          12.2

  BYTE UNIX Benchmarks (Version 3.11)
  System -- Unix B gerard 2.0.5-RELEASE XXXXXXXXXXXX: 
  Fri Oct 20 00:30:52 1995 gerard:/usr/src/sys/compile/GERARD i386
  Start Benchmark Run: Sat Apr 13 21:48:12  1996
   1 interactive users.
Dhrystone 2 without register variables   130585.3 lps   (10 secs, 1 samples)
Dhrystone 2 using register variables     130526.3 lps   (10 secs, 1 samples)
Arithmetic Test (type = arithoh)         413311.1 lps   (10 secs, 1 samples)
Arithmetic Test (type = register)         12753.0 lps   (10 secs, 1 samples)
Arithmetic Test (type = short)            12066.6 lps   (10 secs, 1 samples)
Arithmetic Test (type = int)              12950.6 lps   (10 secs, 1 samples)
Arithmetic Test (type = long)             12956.4 lps   (10 secs, 1 samples)
Arithmetic Test (type = float)            17777.0 lps   (10 secs, 1 samples)
Arithmetic Test (type = double)           17775.0 lps   (10 secs, 1 samples)
System Call Overhead Test                 45727.8 lps   (10 secs, 1 samples)
Pipe Throughput Test                      22094.0 lps   (10 secs, 1 samples)
Pipe-based Context Switching Test          6304.2 lps   (10 secs, 1 samples)
Process Creation Test                       240.3 lps   (10 secs, 1 samples)
Execl Throughput Test                        68.1 lps   (10 secs, 1 samples)
File Read  (10 seconds)                  115117.0 KBps  (10 secs, 1 samples)
File Write (10 seconds)                    3600.0 KBps  (10 secs, 1 samples)
File Copy  (10 seconds)                    3457.0 KBps  (10 secs, 1 samples)
File Read  (30 seconds)                  115920.0 KBps  (30 secs, 1 samples)
File Write (30 seconds)                    3533.0 KBps  (30 secs, 1 samples)
File Copy  (30 seconds)                    3431.0 KBps  (30 secs, 1 samples)
C Compiler Test                              81.8 lpm   (60 secs, 1 samples)
Shell scripts (1 concurrent)                118.0 lpm   (60 secs, 1 samples)
Shell scripts (2 concurrent)                 60.0 lpm   (60 secs, 1 samples)
Shell scripts (4 concurrent)                 30.0 lpm   (60 secs, 1 samples)
Shell scripts (8 concurrent)                 15.0 lpm   (60 secs, 1 samples)
Dc: sqrt(2) to 99 decimal places           2518.2 lpm   (60 secs, 1 samples)
Recursion Test--Tower of Hanoi             2147.1 lps   (10 secs, 1 samples)


                     INDEX VALUES            
TEST                                        BASELINE     RESULT      INDEX

Arithmetic Test (type = double)               2541.7    17775.0        7.0
Dhrystone 2 without register variables       22366.3   130585.3        5.8
Execl Throughput Test                           16.5       68.1        4.1
File Copy  (30 seconds)                        179.0     3431.0       19.2
Pipe-based Context Switching Test             1318.5     6304.2        4.8
Shell scripts (8 concurrent)                     4.0       15.0        3.8
                                                                 =========
     SUM of  6 items                                                  44.7
     AVERAGE                                                           7.4


Even if this benchmark is a little questionnable, I invite people who say or
write that Unix B is FASTER than Unix A to stop, or to say or write 
the OPPOSITE.

Regards, Gerard.

From: j...@time.cdrom.com (Jordan K. Hubbard)
Subject: Re: Unices are created equal, but ...
Date: 1996/04/14
Message-ID: <5777.829437102@time.cdrom.com>#1/1
X-Deja-AN: 147776935
sender: n...@camelot.de
references: <Pine.LNX.3.91.960413224027.150A-100000@gerard>
content-type: text/plain; charset=ISO-8859-1
organization: Mail2News Gateway at nasim
mime-version: 1.0
newsgroups: muc.lists.freebsd.hackers

Considering that you did not even pick the latest release versions of
either OS (2.0.5 is close to a year old and 2.1.0-RELEASE has been out
for 4 months), these results are not even germain and you have no
excuse, except perhaps laziness, for not running these benchmarks on
more recent releases.

I could run benchmarks all day on aged Linux kernels, but what would
that prove?  My stupidity, perhaps, but nothing more.

					Jordan

From: t...@dyson.iquest.net (John S. Dyson)
Subject: Re: Unices are created equal, but ...
Date: 1996/04/14
Message-ID: <199604132351.SAA04861@dyson.iquest.net>#1/1
X-Deja-AN: 147831557
sender: n...@camelot.de
references: <Pine.LNX.3.91.960413224027.150A-100000@gerard>
content-type: text/plain; charset=US-ASCII
organization: Mail2News Gateway at nasim
mime-version: 1.0
reply-to: dy...@FreeBSD.org
newsgroups: muc.lists.freebsd.hackers

> 
> 
> Even if this benchmark is a little questionnable, I invite people who say or
> write that Unix B is FASTER than Unix A to stop, or to say or write 
> the OPPOSITE.
> 
I suggest that you benchmark recent versions of both (say FreeBSD-current
vs. Linux-current).  FreeBSD fork/exec perf has gone up significantly,
among other things.  Please compare equivalent vintages.

Note also that the Byte/Lmbench benchmarks DO NOT measure systems under
significant VM load.  Remember, simple algorithms work quickly until
you actually use them.

John
dy...@freebsd.org

From: j...@time.cdrom.com (Jordan K. Hubbard)
Subject: Re: Unices are created equal, but ...
Date: 1996/04/15
Message-ID: <16036.829530040@time.cdrom.com>#1/1
X-Deja-AN: 147670020
sender: n...@camelot.de
references: <199604142207.RAA00341@dyson.iquest.net>
content-type: text/plain; charset=ISO-8859-1
organization: Mail2News Gateway at nasim
mime-version: 1.0
newsgroups: muc.lists.freebsd.hackers

> How's about maintaining competition, and working to make the respective
> U**X clone better competitively.  Competition helps keep each party
> honest!!!

Just to make a general point here, and *not* to continue the flame war,
I think John's absolutely right.

Consider, for a moment, if FreeBSD or Linux were the only free
operating system.  Who would drive us?  The commercial vendors?
Unlikely, since we could simply escape that comparison by saying "hey,
we're free and they're not!"  With the current situation, there is
_significant_ motivation to not fall too far behind in contrast with
the other group - sort of like two brothers growing up where one can
run just a little faster than the other.  Each will be constantly
motivated to try and win the next foot race by pushing himself that
much harder, the loser being motivated in turn to redouble his
efforts.

I honestly cannot imagine a world where there was only one free OS to
choose from, and I think things have actually worked out far better
than I'd have even dared hope.

					Jordan

From: j...@time.cdrom.com (Jordan K. Hubbard)
Subject: Re: Unices are created equal, but ...
Date: 1996/04/15
Message-ID: <16058.829530125@time.cdrom.com>#1/1
X-Deja-AN: 147684922
sender: n...@camelot.de
references: <9604142231.AA22327@gte.esi.us.es>
content-type: text/plain; charset=ISO-8859-1
organization: Mail2News Gateway at nasim
mime-version: 1.0
newsgroups: muc.lists.freebsd.hackers

> Who cares? linux is such a better name compared with *BSD*.

But we have a nicer looking mascot.. :-)

					Jordan

From: a...@cymru.net (Alan Cox)
Subject: Re: Unices are created equal, but ...
Date: 1996/04/15
Message-ID: <199604150332.EAA05825@snowcrash.cymru.net>#1/1
X-Deja-AN: 147694966
sender: n...@camelot.de
references: <199604141524.RAA03026@x14.mi.uni-koeln.de>
content-type: text/plain; charset=US-ASCII
organization: Mail2News Gateway at nasim
mime-version: 1.0
newsgroups: muc.lists.freebsd.hackers

> The (old) BYTE benchmark is not a suitable performance
> indicator at all! I've made it a port under BSD, since
> it can show misconfiguration and problem areas, and I'd
> like to know, whether you at least used that version 
> (available as a "port" or "package" for easy installation).

Can we flush this whole thread down the bitbucket. lmbench is just
a very low level analysis, the byte benchmarks are near enough broken beyond
usefulness. Until people are doing formal SPEC and transaction benchmarks
we wont have good answers (and some people maintain those are garbage too
;)). As various people have observed nobody can agree what to compare, what
benchmark is right and what it means anyway.



Alan

From: torva...@cs.Helsinki.FI (Linus Torvalds)
Subject: Re: Unices are created equal, but ...
Date: 1996/04/15
Message-ID: <Pine.LNX.3.91.960415083932.30002I-100000@linux.cs.Helsinki.FI>
X-Deja-AN: 147701404
sender: n...@camelot.de
references: <199604142055.NAA05263@ref.tfs.com>
content-type: TEXT/PLAIN; charset=US-ASCII
organization: Mail2News Gateway at nasim
mime-version: 1.0
newsgroups: muc.lists.freebsd.hackers



On Sun, 14 Apr 1996, JULIAN Elischer wrote:
> 
> Ok, here is my attempt at the missing information....

This is probably approaching being closer, but it's not really as 
clear-cut as even this.

Personally, I _want_ linux to be faster on everything, but when it comes 
to real life, I'd say that _any_ of the Free UNIXen are "fast enough". 
There is always going to be some differences in different areas, but with 
both groups being pretty performance-minded I don't really think we'll 
ever have any really noticeable differences in any "normal" circumstances.

> I would expect the following to be true between Linix 1.3.x and FreeBSD 2.0.5.
> category 1:--Linux faster in:
> 		context switch, including some system calls.
> 		possibly some program startup.
> 		possibly pipe code.
> 		possibly FP emulation.
> 		possibly FP exception handling.
> 		Any test that does a lot of filesystem meta-data manipulation.
> 		  e.g. file creation and deletion.

I can't really say much about it: I can't even claim to have benchmarked 
any *BSD machine. All the things you mention are "good" under linux, 
though, so I'd suspect that linux is at least on par with BSD in any of 
the above and may be faster.

On the other hand, Dyson tells me that the FreeBSD team has been 
optimizing lots of those things too, and I wouldn't be surprised if they 
are more or less at the same level as linux is..

Again, I don't think the speed makes a huge difference (a 10% difference 
looks huge in benchmarks, but has little meaning in real life unless 
either is 10% faster at _everything_)

> category 2:--FreeBSD 2.0.5 faster in:
>		Anything to do with networking

No longer true. These days it's "some things to do with networking", and 
while I suspect that BSD may have the edge here still, these days it's 
really the _edge_, not the whole blade ;-)

> 		Anything using a raw tape or disk device.

This may well be true. I'll be the first to admit that I don't really 
care for "raw devices". The only time I ever use the raw device is when 
fsck runs, and I couldn't care less about performance there (well, I 
could, but you get the idea).

Tapes I have no idea about. You may or may not be right.

> 		Any benchmark that loaded the system very heavily,
> 		  especially if it produced swapping.

The differences shouldn't be that large any more. The asynchronous 
swapping code and the swap deamon in newer linux kernels help performance 
under load. Again, I'm more inclined to think that one system may be 
better on some hardware, while the other might be better at something 
else. 

> 		Any benchmark that tested high-speed large sustained 
> 		  IO to files.

This has actually changed, and I suspect that's why Gerard did the 
benchmark in the first place. He's been working on that part of the code, 
and we're doing very well.

Unlike raw device accesses, I consider filesystem access speed very 
important, and I've mostly concentrated on getting good performance 
through good caching. Linux traditionally hasn't been as good in things 
where caches don't help (ie the cases you mention above), but that has 
improved a lot lately. Maybe somebody has access to FreeBSD-current and 
Linux-current on the same machine and can compare bonnie?

> category 3:--Linux and FreeBSD 2.0.5 about equivalent in:
> 		Anything that relies mostly on plain CPU
> 		with no or little OS involvement.
> 		(as both use the same cpu.)

I'd like to put a lot more here. In practice I don't think the 
differences are so large.

HOWEVER, despite the fact that I don't think it matters in real life, I'm 
all for people doing benchmarks, and crying out when one or the other is 
slower in something. Not because I think it makes much of a difference for 
normal users, but because it's good for development. It's a great way to 
motivate people to do better (show that the competition can do better at 
something, and you force us to try to improve ourselves).

I think _that_ is why benchmarks are important, not so much for testing 
which one is more "worthy"..

> >     If you are a FreeBSD-current user and if you have about the same 
> >     configuration as mine, can you run the old BYTE benchmark 
> >     and send to me your results?
> 
> I don't think it would be useful unless we had EXACTLY the same hardware..
> I have seen small differences make up to 50% difference..

Indeed. Even on the same machine the placement of the partitions can make 
a noticeable difference for disk tests, so benchmarking is not a trivial 
thing.. It might still be interesting to see some kind of benchmarking 
done, just for "some data" as opposed to "THE data".

If somebody wants to do benchmarking,I'd suggest using at least
 - lmbench (nice microbenchmark)
 - bonnie (reasonable disk performance benchmark)
 - webstone (or something similar. But use "apache" as the server, not 
   some braindead horror like NCSA).
 - ???

(the three mentioned should cover different areas, all very reasonable, 
but have I missed some important area?)

		Linus

From: t...@dyson.iquest.net (John S. Dyson)
Subject: Re: Unices are created equal, but ...
Date: 1996/04/15
Message-ID: <199604151339.IAA08978@dyson.iquest.net>#1/1
X-Deja-AN: 148246133
sender: n...@camelot.de
references: <Pine.LNX.3.91.960415091252.30002K-100000@linux.cs.Helsinki.FI>
content-type: text/plain; charset=US-ASCII
organization: Mail2News Gateway at nasim
mime-version: 1.0
reply-to: dy...@freebsd.org
newsgroups: muc.lists.freebsd.hackers

> 
> 
> On Sun, 14 Apr 1996, John S. Dyson wrote:
> > 
> > How's about maintaining competition, and working to make the respective
> > U**X clone better competitively.  Competition helps keep each party
> > honest!!!
> 
> I'd be nervous about the benchmarks used. It's too easy to optimize for a 
> specific set of circumstances, and getting blind to the "larger picture". 
> But with a reasonable selection of benchmarks, this might not be a bad 
> idea.
> 
> 		Linus

I agree with your sentiments.  You have stated EXACTLY the concerns that
I have been expressing about depending solely upon benchmarks like lmbench,
iozone, bonnie, etc...  I have some that I use that does show *significant*
differences because of loading issues.  My benchmarks are in rough shape
and are not for public use (for reasons other than the simplistic nature
of the above mentioned benchmarks -- I am not comfortable with the
prettyness of the code.)

It would be nice if we could somehow improve on the quality of the synthetic
benchmarks available in the free realm.

John
dy...@freebsd.org

			        About USENET

USENET (Users’ Network) was a bulletin board shared among many computer
systems around the world. USENET was a logical network, sitting on top
of several physical networks, among them UUCP, BLICN, BERKNET, X.25, and
the ARPANET. Sites on USENET included many universities, private companies
and research organizations. See USENET Archives.

		       SCO Files Lawsuit Against IBM

March 7, 2003 - The SCO Group filed legal action against IBM in the State 
Court of Utah for trade secrets misappropriation, tortious interference, 
unfair competition and breach of contract. The complaint alleges that IBM 
made concentrated efforts to improperly destroy the economic value of 
UNIX, particularly UNIX on Intel, to benefit IBM's Linux services 
business. See SCO v IBM.

The materials and information included in this website may only be used
for purposes such as criticism, review, private study, scholarship, or
research.

Electronic mail:			       WorldWideWeb:
   tech-insider@outlook.com			  http://tech-insider.org/