Path: sparky!uunet!zaphod.mps.ohio-state.edu!magnus.acs.ohio-state.edu!
usenet.ins.cwru.edu!agate!darkstar!osr
From: a...@cs.vu.nl (Andy Tanenbaum)
Newsgroups: comp.os.research
Subject: A challenge
Message-ID: <32507@darkstar.ucsc.edu>
Date: 9 Apr 92 19:09:12 GMT
Sender: use...@darkstar.ucsc.edu
Organization: Fac. Wiskunde & Informatica, Vrije Universiteit, Amsterdam
Lines: 106
Approved: comp-os-resea...@ftp.cse.ucsc.edu
Originator: osr@ftp
Gee, reading this newsgroup is more fun than whacking a hornet's nest with
a baseball bat.
I am quite unconvinced. While I could go after each of the comments posted
here in the past couple of days, I will resist the temptation. Instead,
for the second act, I hereby challenge all the people who have commented
on my list to do better. List what YOU consider the five most important things
we have learned about distributed operating systems in the past 20 years.
If you don't think we have learned five things, a shorter list will have to do.
A short statement followed by a short explanation will do nicely.
If you don't think we have learned anything at all, be brave enough to
state that publicly, and hope NSF and DARPA don't read this newsgroup.
The challengees are:
Arindam Banerji
Ken Birman
Steve Chapin
Geoff Collyer
Crispin Cowan
Gary Craig
Mark Day
Fred Douglis
Dick Dunn
Amr El-Kadi
Martin Fouts
Glenn Kasten
Husam Kinawi
J.P.Knight
Ari Lemmke
Jacob Levy
Andy Lowry
Richard M. Mathews
Larry McVoy
Ronald G Minnich
Peter Moylan
Andrew Mullhaupt
Michael O'Dell
Thomas Page
Jehan-Francois Paris
Craig Partridge
Rob Pike
Boris Putanec
Marcus J. Ranum
Dave Safford
Nirad Sharma
Niranjan G Shivarat
Charles Shub
Charles Bennett Smith
Stephen P Spackman
Henry Spencer
Dave Stow
Werner Vogels
Brent Welch
Phil Winterbottom
Brian Witt
Victor Yodaiken
(And of course, everyone else who has not yet put in his or her 2 cents worth.)
I'll be a sport and go first.
THE TOP FIVE
1. Microkernels are the way to go.
For the system to work, the kernel has to be reliable. Putting the
file system and other user services inside makes it harder to get right
and less flexible. The performance loss is (guess) less than a factor
of two. If you worry about factors of two, I humbly suggest you write
all your code in assembler henceforth.
2. Synchronous (blocking) communication is easier to use than asynchronous.
Starting a send, then going off to do something, followed by getting
an interrupt when the send is done is a nightmare to program. If you
must do something while waiting for the reply, put that code in another
(synchronous) thread. The biggest problem in the computer industry is
not slow machines, but hairy, unreliable software. Using asynchronous
communication just makes the problem worse.
3. RPC is a good idea to base your system on.
SEND/RECEIVE primitives make I/O your central paradigm. RPC makes
procedure call, an abstraction mechanism, your central paradigm.
Abstraction is a key tool for mastering complexity. We need all
the help we can get.
4. Atomic group communication (broadcast or multicast) is highly useful.
Not all communication is two party. For replicated servers and many
other applications, being able to send a (synchronous) message to
everyone and know they all get it in the same order without loss makes
programming easier and programs more reliable. A real win.
5. Distributed shared memory in one form or another is a convenient model.
Even better than communicating synchronously is not communicating
explicitly at all. Just let the programmer think there is shared
memory, be it page-based, object-based, or other. Let the system
do the work. There is an obvious analogy with virtual memory.
As you can see, all of my points relate to one theme: making it easier to
build and use distributed systems. I think this is the real issue of the coming
decade, not squeezing another factor of two out of the hardware. Improvements
in technology will take care of that.
Andy Tanenbaum (a...@cs.vu.nl)
Path: sparky!uunet!elroy.jpl.nasa.gov!swrinde!zaphod.mps.ohio-state.edu!
magnus.acs.ohio-state.edu!usenet.ins.cwru.edu!agate!darkstar!osr
From: yodai...@chelm.cs.umass.edu (victor yodaiken)
Newsgroups: comp.os.research
Subject: Re: A challenge
Message-ID: <32548@darkstar.ucsc.edu>
Date: 10 Apr 92 14:25:27 GMT
Sender: use...@darkstar.ucsc.edu
Organization: University of Massachusetts, Amherst
Lines: 39
Approved: comp-os-resea...@ftp.cse.ucsc.edu
Originator: osr@ftp
In article <32...@darkstar.ucsc.edu> a...@cs.vu.nl (Andy Tanenbaum) writes:
>
>Gee, reading this newsgroup is more fun than whacking a hornet's nest with
>a baseball bat.
>
>I am quite unconvinced. While I could go after each of the comments posted
>here in the past couple of days, I will resist the temptation. Instead,
>for the second act, I hereby challenge all the people who have commented
>on my list to do better. List what YOU consider the five most important things
>we have learned about distributed operating systems in the past 20 years.
Here's four. Two are negative, but that's the breaks.
1. The "fast, clean, communication system" as a foundation for
constructing a distributed o.s. is a mythical object. Distributed o.s's
require different communication mechanisms for different purposes and there
is no single abstraction which can be used as a building block for all others
without a significant cost in performance and ease of use.
2. The UNIX "everything is a file" paradigm extends to networks pretty
well.
3. It is possible to perform significant, on line, "compilation" of
multi-layered o.s. code. The use of this method by Massalin, seems to me to
be one of the most significant advances in the state of the art in
recent years.
4. Current system design methodologies are serious impediments to advance.
Our inability to quickly prototype ideas, to concisely describe (in detail)
the workings of distributed algorithms, and to perform any kind of
verification of algorithms short of testing make o.s. research overly
tedious.
--
yodai...@chelm.cs.umass.edu
Path: sparky!uunet!think.com!ames!agate!darkstar!osr
From: w...@rodos1.adfa.oz.au (Warren Toomey)
Newsgroups: comp.os.research
Subject: Re: A challenge
Message-ID: <32644@darkstar.ucsc.edu>
Date: 11 Apr 92 01:56:21 GMT
Sender: use...@darkstar.ucsc.edu
Organization: Australian Defence Force Academy, Canberra, Australia
Lines: 89
Approved: comp-os-resea...@ftp.cse.ucsc.edu
Originator: osr@ftp
In article <32...@darkstar.ucsc.edu>, by a...@cs.vu.nl (Andy Tanenbaum):
[ who asks us what have we learned in the last 20 yrs ]
My 2 cents worth, but first background: young programmer who has spent the
last 3 years helping design and build a distributed OS frm the ground up.
Young enough not to have played on Version 7 Unix.
> 1. Microkernels are the way to go.
Very much agreed. Putting as much hardware/dependent code in one layer
make the whole thing easier to port, and gives the OS proper an abstract
machine to support.
> The performance loss is (guess) less than a factor
> of two. If you worry about factors of two, I humbly suggest you write
> all your code in assembler henceforth.
Well, if you `hand tune' your ukernel, then you can achieve good performance
on all platforms and still keep the same abstract machine for the real OS.
[ And a question from a naive OS weenie. Why does Mach call itself a
ukernel when it's HUGE? It should really be a millikernel or something]
> 3. RPC is a good idea to base your system on.
> SEND/RECEIVE primitives make I/O your central paradigm.
Hell, make RPC your central paradigm for everything. The OS I've been working
on (RHODOS BTW) has exactly _seven_ system calls: send,call,receive plus
comms endpoint manipulation ones. Everything else is done by server processes.
Makes the OS more modular and flexible, and easier to debug.
However, make your message passing FAST & EFFICIENT. Copying more than twice
is right out. BTW, with shared memory, message passing can be used for
token passing/semaphores while the bulk of the information is passed through
shared buffers.
> 2. Synchronous (blocking) communication is easier to use than asynchronous.
Sometimes avoiding a bottleneck by using async comms is more important. But
as Andy points out, it isn't as clean, so the reasons have to be good. We
provide both types in RHODOS, at least for flexibility.
> 4. Atomic group communication (broadcast or multicast) is highly useful.
The provision of several types of multicast is even better, because each
has a different purpose:
0 or more receive - non-blocking, no guarantee anyone will receive
it. Useful for transport protocols that want to
avoid state & can afford to retransmit a bit more.
At least 1 receive - blocking. Useful where there may be N servers,
and a client only needs to get one answer back.
All receive - as per the reasons Andy stated.
> 5. Distributed shared memory in one form or another is a convenient model.
I'll agree on the shared bit. Synchronising memory updates & access over
a network is a real headache.
In RHODOS we decided that threads were just too troublesome to do in the
kernel. Let the user process do the work itself, by using a library of
thread routines that form part of the user process.
Instead, we use a MEDIUMWEIGHT process model, where processes have their
own stack spaces but share a common data space read/write. This means that
we do normal context scheduling on all processes, but they can access each
other's memory - a flag in our fork() equivalent makes the parent/child a
mediumweight process pair.
I'll add another lesson to the list: file mapping. This is great! It gets
rid of multiple data copying (disk->buffer->user buffer), and means the
whole of physical memory can be used as a disk cache. Moreover, if you
use the paradigm of file-memory object equivalence, you can use the whole
of the disk for paging, instead of just one partition. And the migration
of processes and their memory can be done through a common file server,
allowing the moving of the dirty pages only.
I expect flames from the elders. Go ahead, educate me :-)
Warren Toomey VK1XWT, on the path to insanity
No longer in the bowels of ADFA Comp Science.
`I wants virtual memory! I wants it now, precious.'
Path: sparky!uunet!spool.mu.edu!agate!darkstar!osr
From: he...@zoo.toronto.edu (Henry Spencer)
Newsgroups: comp.os.research
Subject: Re: A challenge
Message-ID: <32708@darkstar.ucsc.edu>
Date: 12 Apr 92 00:18:19 GMT
Article-I.D.: darkstar.32708
Sender: use...@darkstar.ucsc.edu
Organization: U of Toronto Zoology
Lines: 34
Approved: comp-os-resea...@ftp.cse.ucsc.edu
Originator: osr@ftp
In article <32...@darkstar.ucsc.edu> a...@cs.vu.nl (Andy Tanenbaum) writes:
>for the second act, I hereby challenge all the people who have commented
>on my list to do better. List what YOU consider the five most important
>things we have learned about distributed operating systems in the past
>20 years.
1. A distributed operating system should look, to the customer, just like
a non-distributed operating system, apart from the possibility of more
creative failure modes. Cleverness should go into hiding its distributed
nature, not exposing it.
2. Unix filesystem semantics -- resources organized in a tree structure
named by ASCII names, and operated on by open/close/read/write -- work
just as well in a distributed system as in a non-distributed one, unlike
most of their competitors.
3. Most resources are used by only one customer at a time, so it is
important to streamline the *un*shared case while handling the shared
case gracefully.
4. Pipes are an excellent interprocess-communication method, superior
to most of their competitors in flexibility and distributability (is
that a word? :-))... provided you have some way of establishing them
between unrelated processes, and preferably some way of passing them
between processes dynamically.
5. Most distributed-OS designers would rather design something new and
weird, in hopes that everyone will say "oh, how wonderful, we must
rewrite all our programs to use this at once", than observe point #1
above and build a distributed implementation of the same old thing,
on which all that software will run without change.
--
GCC 2.0 is to C as SVR4 is to Unix. | Henry Spencer @ U of Toronto Zoology
-Dick Dunn | he...@zoo.toronto.edu utzoo!henry