From: Paul Barton-Davis <p...@op.net>
Subject: a joint letter on low latency and Linux
Date: 2000/06/28
Message-ID: <linux.kernel.200006290021.e5T0Lso29954@op.net>
X-Deja-AN: 640235880
Approved: n...@nntp-server.caltech.edu
X-To: linux-ker...@vger.rutgers.edu
Newsgroups: mlist.linux.kernel
[ This letter was sent privately to Linus as well. In this version,
all email addresses of the signees except for p...@op.net have
been removed for privacy/spam reasons. ]
Dear Linus,
we are a group of programmers designing, writing and extending
audio+MIDI (and in some cases, video) applications for Linux. As you
could tell from the list of signees below, we represent the developers
of many significant and substantial audio+MIDI applications for Linux,
as well as members/employees of several well known music and audio
research institutions and companies.
[
If you want to get an overview of the current state for audio+MIDI under
Linux, there is no better source than Dave Phillip's magnificently
comprehensive web site (http://www.bright.net/~dlphilp/linuxsound/).
]
One member of our group, Benno Senoner, did a lot of work last year in
investigating and documenting problems involved in using Linux for real-time
audio applications. Others, such as Juhana Sadeharju, had long noted problems
using Linux for hard disk recording of audio data. Partly as a result of
Benno's work, Ingo Molnar did a fantastic job of coming up with a patch for
the 2.2 series that dramatically improved the latencies that could be obtained
from Linux.
How good? Well, good enough that members of our community who were convinced
that RTLinux was needed to do a professional job with audio changed their
minds. Good enough that we were close to (or sometimes better than) BeOS, an
OS that has had a lot of excellent press in the audio world as a replacement
for the known-to-be-problematic Windows and MacOS systems. Good enough that in
some cases, Linux is as good as dedicated hardware solutions, offering
latencies in the realm of 2-3ms for our applications.
There was a lot of excitement that 2.4 might include a version of the low
latency patches. The excitement came from the possibility that the next
release of the various distributions of Linux would represent a set of
"desktops" that were ready for excellent, "real-time" audio and MIDI
applications. CPU and disk performance has improved to the point where we are
on the threshold of a revolution in the way that sound synthesis and
processing is done, and many of us want to ride Linux into the heart of that
revolution.
However, it turns out, as best we can gather, that you were not happy
with the basic structure of some or all of Ingo's low latency
work. Our impression is that you want to see more careful code design
that avoids interrupt disabling or holding high level locks for too
long, rather than using preemption points. So, as far as we can tell
right now, 2.4 will represent more of the same as far as low latency
limitations, and for us, more of the same means performance much worse
than Windows or MacOS present.
How much worse ? Linux currently offers worst-cases latencies that are *10-15*
times worse than Windows or MacOS. Developers for those platforms still
complain about their performance with respect to latency - imagine their
response to the situation with Linux as it stands today!
We understand that we could try to maintain a version of Ingo's low latency
patches in parallel to the current kernel. But this is not a good situation
for us. We would like to persuade several companies that produce applications
and API's for audio+MIDI work to make their code, designs and programs
available for Linux. We would like to be able to produce our own "real-time"
applications and not have to tell users (who will likely know nothing about
Linux, or computers in general) that they need to patch their kernel before
using them. Neither of these goals are realistic given the current state of
low latency support in the emerging 2.4.
We would like to know:
* what are your general feelings on modifying Linux to support
the kind of applications we are concerned with ?
* what kinds of compromises, if any, you might accept in order
to get good low latency performance into the kernel sooner,
rather than later ?
* what design goals you have in mind when you talk about
doing low latency "right", rather than "wrong, as in
Ingo's approach" ?
* to what extent are you willing to enter into a debate about the
merits of a preemption-point-based approach to lowering kernel
latency ?
Above all, we are all interested in having fun with Linux and
audio/music/MIDI. We would invite you to come tour a professional studio,
watch the cool and wonderful stuff that can be done right now (with lots of
annoying problems) on machines that runs Windows and MacOS, and get a sense of
the extent to which general purpose computers are about to replace a whole
bunch of expensive stuff sitting around in a typical studio. We want to see
penguins there, and soon!
Several of the signees below have noted that lowering latency improves the
quality of games, most kinds of multimedia and many non-hard-real-time
automated systems. Incorporating a low-latency patch could be seen as extremely
helpful in providing competitive performance in areas outside of audio as well.
Thank you for your consideration, for Linux and your benign dictatorship.
Name Project/Org. Affiliations(*) Email
---------------------------------------------------------------------------------
Paul Barton-Davis ALSA,Quasimodo,Ardour,SoftWerk <p...@op.net>
Benno Senoner Low-latency audio+HDR benchmarking
Dave Phillips "The Book of Linux Music+Sound"
John Littler Linux MusicStation
Dustin Barlow
Joe Miklojcik GESM, Quasimodo
Erik Steffl
Scott Wilson SoundWire, CCRMA(Stanford)
Andy Lo A Foe AlsaPlayer
Gad Berger LSDVD
Dave O'Toole GNU Octal
Ivar Vasara
John Lazzaro Sfront (MPEG 4-SA decoder)
Ben Crowder
Scott McNab ALSA
Robert Jonsson
Reine Jonsson
Matthias Pfisterer Tritonus (open-src Java Sound API)
Jay Ts
Stephane Letz Centre National de Creation Musicale
Iain Sandoe
Pedro Kroger
Niklas Werner tonwerk.de
Francois Dechelle jMax,IRCAM Real-time Systems Team
Maarten de Boer Tapiir, AV Institute
Erik de Castro Lopo libsndfile,xmms plugins,vsound
Steve Morphet
Maurizio De Cecco jMax,Mandrakesoft
Alessandro Fogar
Kai Vehmanen ecasound
Yann Orlarey Centre National de Creation Musicale
Robert Murray
Sebastian Niller
Juhana Sadeharju
Bill Gribble Ardour,Quasimodo
Stefan Westerfeld aRts (KDE2.0 sound server)
Kirk Reiser
Chris Baugher
Ned Judy
Jorn Nettingsmeier linux-audio-dev website
Jason Clouse GNUstep/NSSound (MusicKit API)
Reynald Hoskinson
John Beppu Lineo
Nolan
Brian Redfern
Tobias Ulbricht
Jussi Laako HydroAcoustic Signal Analysis
Jarno Seppanen
Garth Brantley
Eli Brandt
Paul Winkler Pysco
Edmund Humenberger
Mico Leao
Andrew Clausen
Miriam Rainsford
Alexander Konig
Fernando Lopez-Lezcano CCRMA(Stanford)
Tijs van Bakel
David Olofson Audiality,MuCoS
Anders Torger nwfiir,ALSA
Joe Grisso E-Mu/Ensoniq
John Watson
Jaroslav Kysela ALSA,SuSE
Paul Wagland SDL
Ben Campbell SDL
Abramo Bagnara ALSA,SuSE
Pablo Silva
Nicolas Justin
Mike Andrews Shiman Associates
Dave Andruczyk
John Thaden
Frank Barknecht
Jaymz Julian
Peter Enderborg
Frank van de Pol ALSA sequencer
Rizzuti Luca
Jeremy Hall
(*) As usual, organizational affiliation is for informational purposes
only, and does not imply official approval or sanction of this letter
by the named organizations.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar <mi...@elte.hu>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.Pine.LNX.4.10.10006291453290.2745-100000@elte.hu>#1/1
X-Deja-AN: 640411044
Approved: n...@nntp-server.caltech.edu
X-To: Paul Barton-Davis <p...@op.net>
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-Cc: linux-ker...@vger.rutgers.edu, Linus Torvalds <torva...@transmeta.com>
Reply-To: mi...@elte.hu
Newsgroups: mlist.linux.kernel
On Wed, 28 Jun 2000, Paul Barton-Davis wrote:
> However, it turns out, as best we can gather, that you were not happy
> with the basic structure of some or all of Ingo's low latency work.
> [...]
*I* was unhappy with the structure of that patch to begin with. The patch
is ugly and unacceptable (read: a kludge) for inclusion into the
mainstream kernel, period. I also said that i'll send a similar patch for
2.4 as well, once the 2.4 codebase stabilizes. (right now we still have a
high flux of fixes coming in - but i'll soon port the patch to 2.4)
so please, do not make this appear as some 'fault' of Linus. Linus is
rightfully (and thankfully) watching the quality of the mainstream kernel,
and ugly patches are simply not accepted, regardless of the usefulness of
a given patch. In fact it's my fault of not submitting those patches in a
saner way. I'll fix this in the coming weeks.
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Paul Barton-Davis <p...@Op.Net>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.200006291507.LAA20313@renoir.op.net>#1/1
X-Deja-AN: 640581321
Approved: n...@nntp-server.caltech.edu
X-To: mi...@elte.hu
X-Cc: linux-ker...@vger.rutgers.edu, Linus Torvalds <torva...@transmeta.com>
Newsgroups: mlist.linux.kernel
First:
I need to apologize for Ingo for not contacting you before we sent
this letter. I intended to do so, but when it came time to actually
mail it out, I forgot that this was a prior step I had mentally
committed to taking. It would have been the right thing to do, and I
am sincerely sorry that it was not done.
That said, Ingo writes:
>> However, it turns out, as best we can gather, that you were not happy
>> with the basic structure of some or all of Ingo's low latency work.
>> [...]
>
>*I* was unhappy with the structure of that patch to begin with. The patch
>is ugly and unacceptable (read: a kludge) for inclusion into the
>mainstream kernel, period. I also said that i'll send a similar patch for
>2.4 as well, once the 2.4 codebase stabilizes. (right now we still have a
>high flux of fixes coming in - but i'll soon port the patch to 2.4)
>
>so please, do not make this appear as some 'fault' of Linus. Linus is
>rightfully (and thankfully) watching the quality of the mainstream kernel,
>and ugly patches are simply not accepted, regardless of the usefulness of
>a given patch. In fact it's my fault of not submitting those patches in a
>saner way. I'll fix this in the coming weeks.
I am a little confused, Ingo. There were several exchanges that you
had with Linus about the patches in which you put forward several
arguments in favor of their inclusion. These culminated in Linus
saying point blank that he would not do so.
Although there is an argument that some of the later patches went too
far (e.g. adding code to spinlock functions), I think that the basic
structure of your code (not the source, but the notion of adding
preemption points at locations that were identifiable problems in
linux' latency behaviour) was thought by several people to be a good
solution. Everything I have read from Linus has suggested to me
that he thinks that this is a bad way to solve the latency issues.
I don't have much to say about the specific nature of the code in the
patches, but my understanding was that this wasn't the reason for
Linus' not liking them.
--p
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: l...@bitmover.com (Larry McVoy)
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <fa.h6r6t7v.1aheeq0@ifi.uio.no>#1/1
X-Deja-AN: 640447414
Original-Date: Thu, 29 Jun 2000 08:41:48 -0700
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <200006291541.IAA09529@work.bitmover.com>
To: Paul Barton-Davis <p...@Op.Net>
Organization: Internet mailing list
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu
> Although there is an argument that some of the later patches went too
> far (e.g. adding code to spinlock functions), I think that the basic
> structure of your code (not the source, but the notion of adding
> preemption points at locations that were identifiable problems in
> linux' latency behaviour) was thought by several people to be a good
> solution. Everything I have read from Linus has suggested to me
> that he thinks that this is a bad way to solve the latency issues.
I'm coming into this a little late but here's my take on all of this.
I have a little background on this, I was asked to support this letter
by using LMbench to "show" that there were no problems with the Ingo
patches and declined. I felt that what I was asked to do was misleading
and I didn't agree with what was in the letter.
Paul and the others in favor of the low latency fixes are fond of pointing
out that they must be good because this is how IRIX and BeOS solve the
problem. That's never a good reason to do anything - fortunately for
all of us, Linus evaluates things on their technical merits. And "IRIX
does it" has no technical merit whatsoever.
I and others have pointed out that if you need hard real time then
what you do is use RT Linux and you can get exactly what you need.
I've heard two arguments against this:
a) ``Other operating systems offer "soft realtime" and we don't want to
port our code from that to the RT Linux model.'' Translation:
``other operating systems have made poor design decisions, let's try
and pressure Linus into doing the same thing with Linux.'' That's a
poor approach to the problem. Just saying that NT does it does not
make it right. Linux has a large, quickly growing, market share.
In my opinion, it is better to do the right thing, wait until Linux is
the market leader, and then laugh at the screwed up systems that made
the wrong choices. Screwing up Linux so it has the same problems that
other systems have in the name of portablility is not the Linux way.
b) ``If we take the RT Linux path then Linus will never add the low latency
features we want.'' I love this argument. It implies knowledge that there
is a better way, that if you use the better way then there is no need to do
what Linus doesn't really want to do. Exactly.
In fairness to the low latency application people, I think that the
RT Linux folks need to provide skeleton "apps" that show how to solve
problems in the RT Linux space. Whether that happens or not, the RT Linux
approach is really brilliant and it is the right approach to the problem
space, and I'm more than a little sick of hearing people avoid using it.
Get with the program, folks, use the best tool for the job. No matter
what Ingo does, it is a _fact_ that you can not get both good performance
for a multi user time sharing operating system and real time applications.
Please understand that it might be that some of Ingo's work is a good
thing and that it will go into the kernel. I am not sure about that
one way or the other. What I am sure about is that putting features
into the kernel for the benefit of real time applications is a slippery
slope that just leads to bad change on top of bad change. It effects
the scheduler, the I/O paths, the interrupt handling, and probably a
bunch of other places I'm forgetting. The changes are at direct odds
with time sharing performance and the proponents of the changes will
argue each one in isolation, rather than looking at the effects of all
of them.
My advice, heeded or not, is to just say no. We have an excellent answer
for realtime, it's called RT Linux. Go use it - it's better than any
other answer.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.Pine.LNX.4.10.10006290931560.2815-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 640581530
Approved: n...@nntp-server.caltech.edu
X-To: Larry McVoy <l...@bitmover.com>
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-Cc: Paul Barton-Davis <p...@Op.Net>, mi...@elte.hu, linux-ker...@vger.rutgers.edu
Newsgroups: mlist.linux.kernel
Just to clear up my opinion a bit:
- I'm definitely not against low latency. I think latency is hugely
important, mostly more so than throughput (within reason, of course:
all this is very much a balancing issue).
- I'm not even against well-defined and well-thought-out "scheduling
points". They are hackish, but if there is one or two of them to cover
some specific behaviour then that's ok. Not a big deal.
- I _am_ against patches whose only purpose in life is to run some
arbitrary benchmark, and try to make that benchmark look good.
Most of the low-latency patches I've seen have been of the third kind, I'm
afraid.
For example, I would probably accept a patch that adds a simple
if (current->need_resched) schedule();
to the case of generic_file_write/generic_file_read. Why? Because that
case is a real-world case where it's obvious that with huge caches a
normal user can quite simply cause bad latencies, and this is not an issue
that needs all that much discussion - it's an obvious hack, but is't also
equally obvious why it's there, and what the point of it is. We've had a
few of these before: I think the /dev/null driver already does exactly
this.
In fact, because we should look at the "UP threading code" for 2.5.x
anyway (ie using the spinlocks on UP to generate a fairly well-threaded UP
kernel without any source modifications), the one-liner if-statement
should probably be a #define with a simple-to-grep-for name so that it can
be easily removed.
Why just "probably"? A few reasons:
- I suspect that especially if we generate such a macro, people will
start sprinkling it around at various random places. And I do not want
such a simple hack to spread out any more than necessary. One or two
places are fine. So is four. Numerous places in the networking code,
device drivers and filesystems doing it is _not_. At that point it has
gone from a very specific and slightly tasteless hack to an ugly
architecture.
This could probably be avoided by giving it a clear comment and a
discouraging name.
- I do think the "copy_to/from_user()" case is the cleaner one, but I'd
hate to do the test in-line, and I'd hate to do it all over the place.
And if we do it in copy_to/from_user(), we don't do it _anywhere_ else:
again the difference is one between a slightly tasteless hack, and a
ugly rule of life.
This is the more complex case, and requires more careful code in
<asm/uaccess.h>. The "generic" arbitrarily-sized functions should
probably be moved out-of-line into arch/xxx/lib/uaccess.c, because that
test is no longer worth doing inline. Together with benchmark runs.
See? "slightly tasteless" and "strictly localized" are ok. Anything more
hackish than that is not.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar <mi...@elte.hu>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.Pine.LNX.4.10.10006291903550.9487-100000@elte.hu>#1/1
X-Deja-AN: 640581568
Approved: n...@nntp-server.caltech.edu
X-To: Paul Barton-Davis <p...@op.net>
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-Cc: linux-ker...@vger.rutgers.edu, Linus Torvalds <torva...@transmeta.com>
Reply-To: mi...@elte.hu
Newsgroups: mlist.linux.kernel
On Thu, 29 Jun 2000, Paul Barton-Davis wrote:
> I am a little confused, Ingo. There were several exchanges that you
> had with Linus about the patches in which you put forward several
> arguments in favor of their inclusion. These culminated in Linus
> saying point blank that he would not do so.
that was a completely different patch. It was a different approach than
the 'low latency patch', it was about making the kernel preemptable.
Making a kernel preemptable is not an easy area, so disagreement was IMO
natural.
I'm going to do the same lowlatency patch for 2.4 what i did for 2.2. Note
that 2.4 already has a couple of lowlatency-related enhancements. (i
remember Andrea has added a better swapinfo/meminfo algorithm some time
ago for exactly this reason - and other areas improved as well.)
Btw., the lowlatency patch was written via looking at latency-profiling
output, where i used Benno's soundcard-latency-testing utility on a real
soundcard. That is a latency area we really care about. I didnt use
lmbench for _this_ patch btw., because the 'did the soundcard skip a
buffer, and how many times' metric is that i think is the most RL-relevant
number.
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Benno Senoner <sbe...@gardena.net>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/30
Message-ID: <linux.kernel.00063002113301.00815@linuxhost.localdomain>
X-Deja-AN: 640612726
Approved: n...@nntp-server.caltech.edu
X-To: l...@bitmover.com
Content-Type: text/plain; charset=US-ASCII
MIME-Version: 1.0
X-Cc: linux-ker...@vger.rutgers.edu, torva...@transmeta.com, Paul Barton-Davis <p...@op.net>
Newsgroups: mlist.linux.kernel
Larry McVoy wrote:
> a) ``Other operating systems offer "soft realtime" and we don't want to
> port our code from that to the RT Linux model.'' Translation:
> ``other operating systems have made poor design decisions, let's try
> and pressure Linus into doing the same thing with Linux.'' ...
You should investigate the problem more deeply before saying
"we do not want to port our code" ..
Do you really think that the multimedia driver folks are willing to port
hundreds of thousand of lines to the RTLinux model and then mantain 2 sets of
APIs to satisfy the realtime multimedia folks ?
please read the attached mails from Paul and me CAREFULLY, to get an idea WHY
RTLinux is not the best solution from multiple points of view:
some quick examples:
- target audience : how many of the of the millions of linux desktops have
RTLinux installed , how many distros ship it/install it by default, when will
Linus accept RTLinux into the mainstream kernel, when will Joe Sixpack
be able to buy Redhat 12.0 and run his smooth playing software DVD-decoder
or run his cool software-audio-synth ?
The frustrating thing is that Ingo's patch although ugly, can cover
ALL realtime multimedia needs NOW, and this better than Windows and Mac with a
simple userspace programming model. (opposed to win & mac developers forced
to stay at kernel level and/or talking directly to the hardware in order to get
good latencies because the OS is broken)
- too restrictive : problems with large memory needs,
working in kernelspace (do we really want to run COMPLEX apps within the
kernel space (a physical modelling synth) ? ,
debugging disadvantages (in userspace code add a few printf()s to debug
stuff, remove SCHED_FIFO so that you can press CTRL-C if something goes wrong),
no interaction with multimedia drivers like audio, MIDI, framebuffer and
video4linux
- the perceivable "latencies" of ear and eye are limited: ( ear <5ms latencies
are perceived as realtime, eye: more than 25-30 frames/sec ( = 30ms ) are
perceived as fluid animation)
So why should I use RTLinux which can do 20-50usec when 99.99% of cases can be
covered using and userspace app running SCHED_FIFO / mlock() because most
apps do not need <5msec latencies.
Victor Yodaiken asked if some of the linux-audio-dev folks consider
using RTLinux for audio, and most of the responsed were negative,
see the attached text below
( do not forget that as soon you try to drive audio latencies under a certain
value, you will get so many context switches / sec that it will trash the
CPU's cache plus assuming you are running chained plugins,
the CPU will spend more time doing procedure calls , stack ping-pong
and DSP-algorithm setup, than doing actual USEFUL DSP stuff.
PS: every few days I get a mail from people asking NON audio-related stuff
like:
"I have an application which polls an external controlling device with a
frequency of 200HZ. Do I need RTLinux for this ?"
Ingo's kernel can easily meet this criteria thus the app above becomes a simple:
set SCHED_FIFO
mlockall()
while(1)
{
sleep_5ms() (using RTC for example)
poll_device()
record_your_data()
}
I think this what Montavista thinks:
many "realtime" applications (even many of the embedded ones) do not require
usec accuracy but only 1-5msec reaction time, and this can be done comfortably
staying in userspace , provided that the kernel can deliver good latencies.
PS: have you thought about the speedup that a low-latency kernel could give
to a beowulf cluster when heavy disk I/O occurs:
assume a 200 node beowulf where all machines wait on a barrier on every
calculation iteration: if one machine stalls for 100msecs (happens during heavy
disk IO), 200 machines will have to wait for the stalled node, thus 200 CPUs
will lose 100msec of processing time.
Since these 100ms stalls can potentially happen (current kernels) every 30-60
secs on each box, assuming uniform distribution a stall could happen every
60/200 secs = 0.3 secs (worst case) , so losing 100msecs every 300msecs
means a performance drop of up to 20-30% .
NOT NICE.
Benno.
Paul Barton-Davis wrote:
>I don't see why it would require a "complete rewrite".
the ALSA drivers currently represent a minimum of 64000 C statements
(i counted semicolons in the files; there are 174,000 lines in total,
including comments).
although i strongly suspect that most of the work involved in porting
them to RTLinux could be done with a portability "wrapper", i also
strongly suspect that we'd have to go over every single line of every
file and check it for various assumptions.
the doesn't include the video/tv interfaces needed by actual
MULTI-media apps (as opposed to multi-media).
no, its not a "complete rewrite", but its about as close to it as you
can get without it actually becoming one.
>> i think, however, that all of this is moot. For almost all of us, the
>> performance offered by Ingo's patches to 2.2.{10,11,12,13,14,15} is
>> more than adequate. Using RTLinux involves, at the absolute minimum,
>> some real work, a definite learning experience, and possibly a
>> redesign of already operational applications. To gain what ? Better
>> performance than we actually need. victor, its a tough sell ;)
>
>Someone told me that about "C" in 1980. How's that for a stunningly
>irrelevant response?
>I think that the motivations of performance and intellectual
>curiosity are strong.
take a look at the hardware.
44.1kHz -> 1 sample = 22usec
48kHz -> 1 sample = 20usec
96kHz -> 1 sample = 10usec
192kHz -> 1 sample = 5usec
so RTLinux would could our performance down to roughly the same level
as the converters.
alas, its nots so simple. there are always buffers on the audio
interface which accumulate sample data in both directions. plus, there
is the PCI bus burst transfer size (64 bytes). so, lets assume a
minimum transfer size of 64 bytes:
16 32 bit samples @ 48kHz: 320usec
32 16 bit samples @ 48kHz: 640usec
so, clearly, RTLinux would get us down to the h/w limits of today's
audio interfaces.
this is an attractive goal in some senses, except for the fact that as
Benno has pointed out, doing a single pass through our audio algorithm
is probably going to cost *at least* the same time *just for the
overhead* (i.e forget any actual computation).
so, its back to the age-old (well, as old as the Music N language
family) tradeoff between a "blocksize" (number of samples processed
per iteration) that favors rapid response and one that favors high
efficiency.
realistically, people have found with programs like Csound that the
best compromise value is probably on the order of a 100 to 1000
samples depending on the application. in some cases, a sample size of
1 is necessary (e.g. to properly implement some kinds of
filters/delays).
Lets pick 256 as a nice power of 2 number:
256 samples @48kHz: 5ms
Presto: we can do 2.5ms with Ingo's code. So we're in the clear here
for almost all applications.
There are a few cases where the ability to do "better" than the h/w
can is a win. But these are very rare - I don't know of any current
pro-audio gear that even attempts such a thing. Single-sample transfer
is common for thru routing, but its rare when doing signal processing.
My take-home point; RTLinux is a fabulous piece of work. But it lets
us do better than we need to at the cost of a bunch of driver porting,
app redesign, and some learning, contrasting with the low latency
patches which involve no change in app design (i.e code runs on
dog-slow latency systems too, albeit poorly), no "porting" of drivers
to a new "platform", and the chance to carry on as usual. ah, what a
pathetic specimen i am.
--p
Benno Senoner wrote:
Paul's summary is quite complete, (blocksize tradeoffs vs efficiency)
(plus do not forget cache trashing when doing 5000 context
switches per second)
What I want to add is although Victor's offer is nice,
besides the fact that the ALSA , video4linux, framebuffer folks
(eg the multimedia driver writers) are not willing and have not the time
to port the code to RTLinux and mantain two sets of drivers,
I have another big fear:
(and 1ms latencies is not a big deal on CPUs which can do
500 million operations/sec = 500K ops per msec.)
And as Paul pointed out audio latency limitations like
PCI burst size = 64 bytes , blocksizes have to be bigger than a certain
amount ( eg >1-2ms worth of audio) if you don't want to burn
80% of the available CPU cycles doing useless stuff instead
of actual DSP work ecc.
make look RTLinux like using a Ferrari when driving on a mountain road:
expensive and you will always be forced to drive at low speeds (1st,2nd gear).
BTW: one advantage of debugging userspace realtime audio apps:
during debug phase do not set SCHED_FIFO and use big blocksizes
(there will be a delay due to bigger buffersizes, but audio will play
correclty).
Using bigger buffersize , you can even use printf() within the main loop
to display for example buffer pointers and other internal variables.
As soon as you app feels bugfree,
remove the printf()s , decrease the buffersize to the desired amount,
and enable SCHED_FIFO , mlock().
I use this technique ALL the time and has been very helpful to debug my
stuff and speed up development.
I don't think that this is easily possible under RTLinux.
I think that this userspace vs RTLinux programming model is one of the key
points of Montavista. ( ease of use, does not require the learning of new
interfaces and speedup in developement time)
Even if they do not aim at achieving 20-50usec latencies as RTLinux does,
they say many applications (embedded and non-embedded) can live
with 1ms worst-case latencies.
Video is similar to audio, a 200Hz frame update rate doesn't buy us much
over a 60Hz update frequency since the reaction speed of the retina is limited.
( cinema is 24FPS , TV 50/60 Hz interlaced)
and 60HZ = 16msec = PLENTY of time for a lowlatency kernel which has
worst case latencies around 1-2msecs.
So as you can see video is even less demanding than audio.
Audio is probably the most demanding multimedia field in terms of
latency , so my conclusion is that the lowlatency way is adequate to cover
all needs of a modern multimedia desktop.
Of course driver writers are required to meet certain criteria,
like not turning IRQs off for long time (eg more than 1ms) and to keep
the codepaths short (in the case of design-constrained long codepaths,
check for rescheduling)
of course we should not drop efforts to research methods to achieve even
better performance than now , therefore I will keep an eye on
RTLinx + audio , but what the linux audio community wants is to have a
solid platform sooner than later.
(eg: load up a plain Redhat 7.0, run you RT audio app (userspace app which runs
SCHED_FIFO) , and get rock solid 3ms latencies, without the need of installing
low-latency patches, RTLinux drivers etc)
I know a few studio folks, and they see the computers only as a tool just as
dedicated racks, they do not care about technical details, their only goal
is to make music and if the PC is not easy to use enough, they will look for
other alternatives.
----------
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: l...@bitmover.com (Larry McVoy)
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <fa.h5r2t7v.1cg2fqf@ifi.uio.no>#1/1
X-Deja-AN: 640604476
Original-Date: Thu, 29 Jun 2000 16:29:06 -0700
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <200006292329.QAA01756@work.bitmover.com>
To: Benno Senoner <sbe...@gardena.net>
Organization: Internet mailing list
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu
> You should investigate the problem more deeply before saying
> "we do not want to port our code" ..
>
> Do you really think that the multimedia driver folks are willing to port
> hundreds of thousand of lines to the RTLinux model and then mantain 2 sets of
> APIs to satisfy the realtime multimedia folks ?
What "hundreds of thousand of lines"? I think your problem is that
you think that if you use RTLinux then you have to do everything inside
RTLinux. That's not the case at all. You do the part that can't have
jitter there but the data processing is all down under Linux as it
is today.
It's true that I'm not an expert in this area so my I'm completely wrong,
educate me. My understanding is pretty basic - there are input channels
and output channels and these have to be sampled at a constant rate, with
no delays, right? And the basic issue is that if we are doing disk I/O
(or whatever) then the deadline is not met, right?
And since you are insisting that ypu do not want to run in kernel mode,
then I can assume that most of the work happens in user space, but the
sampling happens in the drivers, right?
If that's not right then please tell me how it works. If that is more or
less right, then it seems to me to be trivial to open up a pipe to a real
time process which does the sampling and away you go.
On the other hand, if you are doing all the work in the kernel or in the
drivers, then the portability argument goes away because you have to port
all that glop to Windows/whatever.
My guess is that it is close to what I think it is and that you don't realize
that the point of the RTLinux APIs is to give you a small amount of code that
does exactly what you want when you want it and then passes the results out to
exactly the same user level processing you would use today.
So educate me, tell me what I don't understand.
> - target audience : how many of the of the millions of linux desktops have
> RTLinux installed
This is the same crock of sh*t that MIS directors used against Linux
itself: "How many machines from Compaq come with Linux installed?"
Of course the standard distributions don't have it, there aren't apps that
need it yet. But you get busy and make some whizzy apps that work flawlessly
under RTLinux to give you great sound and video and watch how fast Red Hat
ships with RTLinux. Using the "it isn't shipping yet" argument is wacko -
the same can be said for every new idea. If we used your logic, nothing
new could _ever_ happen.
> The frustrating thing is that Ingo's patch although ugly, can cover
> ALL realtime multimedia needs NOW
Really? So you've done the analysis and testing to show that this is true
for "ALL realtime multimedia needs NOW"? You know what, I don't believe that.
Show me.
> - too restrictive : problems with large memory needs,
> working in kernelspace (do we really want to run COMPLEX apps within the
> kernel space (a physical modelling synth) ? ,
No, you don't, and nobody is telling you to do so. Just gather the data
you need in the RT part and send it out to Linux just as you do today.
[other complaints about young technology deleted]
> So why should I use RTLinux which can do 20-50usec when 99.99% of cases can be
> covered using and userspace app running SCHED_FIFO / mlock() because most
> apps do not need <5msec latencies.
Because RTLinux can give you what you need.
Because it doesn't require you to hack up the kernel.
Because it is far more scalable and reliable.
Because it is cool.
Because your approach is the same tired bag of mistakes that helped make
other operating systems piles of sh*t.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Benno Senoner <sbe...@gardena.net>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/30
Message-ID: <linux.kernel.00063002345402.00815@linuxhost.localdomain>#1/1
X-Deja-AN: 640629474
Approved: n...@nntp-server.caltech.edu
X-To: linux-ker...@vger.rutgers.edu
Content-Type: text/plain; charset=US-ASCII
MIME-Version: 1.0
X-Cc: torva...@transmeta.com
Newsgroups: mlist.linux.kernel
Linus Torvals wrote:
> - I do think the "copy_to/from_user()" case is the cleaner one, but I'd
> hate to do the test in-line, and I'd hate to do it all over the place.
> And if we do it in copy_to/from_user(), we don't do it _anywhere_ else:
> again the difference is one between a slightly tasteless hack, and a
> ugly rule of life.
I may miss something, but Ingo's initial low latency patch, did just
the schedule checks in the uaccess macros, but that was not enough to
cover all cases and ensure flat latency diagrams under all kinds of load.
So I am curious if the "we don't do it anywhere else" thing is possible
in practice, without requiring a radical change within the kernel.
Question: From a pure overhead point of view: does the preemption-points
approach score better (how much) than other approaches (eg pre-emptable kernel)
?
Benno.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.Pine.LNX.4.10.10006291634170.11793-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 640629476
Approved: n...@nntp-server.caltech.edu
X-To: Benno Senoner <sbe...@gardena.net>
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-Cc: linux-ker...@vger.rutgers.edu
Newsgroups: mlist.linux.kernel
On Fri, 30 Jun 2000, Benno Senoner wrote:
>
> I may miss something, but Ingo's initial low latency patch, did just
> the schedule checks in the uaccess macros, but that was not enough to
> cover all cases and ensure flat latency diagrams under all kinds of load.
And that is exactly my point.
I refuse to have a kernel that is bogged down with random crap all over
the place. It's wrong. It's distasteful. And it leads to more and more
crap over time. That's how you get a BAD operating system.
Your assumption that "Why can't we just do this ugly thing" is inherently
flawed. The reason we can't do that ugly thing is also the reason that
people want to use Linux in the first place. Otherwise you might as well
run Windows or something else.
If it cannot be handled with just a few well-placed things, then it
shouldn't be handled at all, and you should be working on trying to make
the UP-threaded kernel work ok, so that for the 2.5.x timeframe we won't
have this issue any more.
Live with it.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: l...@bitmover.com (Larry McVoy)
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.200006300019.RAA02300@work.bitmover.com>#1/1
X-Deja-AN: 640629490
Approved: n...@nntp-server.caltech.edu
X-To: David Lang <david.l...@digitalinsight.com>
X-Cc: l...@bitmover.com, Benno Senoner <sbe...@gardena.net>,
linux-ker...@vger.rutgers.edu, torva...@transmeta.com, Paul Barton-Davis <p...@op.net>
Newsgroups: mlist.linux.kernel
> > If that's not right then please tell me how it works. If that is more or
> > less right, then it seems to me to be trivial to open up a pipe to a real
> > time process which does the sampling and away you go.
>
> it's more then just sampling and storing the data. they are currently
> using the approach that you describe (drivers do the IO, processing in
> user space. the trouble is that the user space processing can be delayed
> to the point that you get a buffer underrun so if you use RTLinux to fix
> this then your userspace code needs to be converted to RTLinux code. Thus
> the "thousands of lines of code" that need to be converted.
OK, now we are getting somewhere. So I have a kernel level source, user
level processing, and a kernel level sink, right? And the problem is that
we can have long delays which mess that up. Here is how you handle that
with RTLinux:
There are three processes:
RTLinux producer
for (;;) {
sample();
put_data_in_shared_mem();
up(producer_sema);
}
Linux processor
for ( ;; ) {
down(producer_sema);
get_data_from_shared_mem();
process();
put_processed_data_in_shared_mem();
up(consumer_sema);
}
RTLinux consumer
for ( ;; ) {
down(consumer_sema);
move_shared_mem_to_device();
}
I think that 99,000 out of the 100,000 lines of code still lives in the
Linux process just like it always did. The only difference is that the
RT processes get scheduled when they need to get scheduled - the Linux
kernel and all of it's processes get preempted any time a RT process
wants to run.
Think about this. You already have a mechanism that gives you exactly
what you need. And it is _way_ better than any of the proposed hacks.
The only complaint that is valid is that it is "different". Yup, that's
true, the wheel was different too. Such is life. You have to give a
little to get a lot, that's a better tradeoff than you normally get.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.Pine.LNX.4.10.10006291723380.1083-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 640643580
Approved: n...@nntp-server.caltech.edu
X-To: Larry McVoy <l...@bitmover.com>
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-Cc: David Lang <david.l...@digitalinsight.com>, Benno Senoner
<sbe...@gardena.net>, linux-ker...@vger.rutgers.edu, Paul Barton-Davis <p...@op.net>
Newsgroups: mlist.linux.kernel
On Thu, 29 Jun 2000, Larry McVoy wrote:
>
> OK, now we are getting somewhere. So I have a kernel level source, user
> level processing, and a kernel level sink, right? And the problem is that
> we can have long delays which mess that up. Here is how you handle that
> with RTLinux:
Well, I personally would rather see that nobody ever needed RTlinux at
all. I think hard realtime is a waste of time, myself, and only to be used
for the case where the CPU speed is not overwhelmingly fast enough (and
these days, for most problems the CPU _is_ so overwhelmingly "fast enough"
that hard realtime should be a non-issue).
I definitely agree with low-latency requirements even in a standard Linux.
I just disagree violently with doing them with horrible cludges instead of
working on doing it right.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: l...@bitmover.com (Larry McVoy)
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.200006300044.RAA02367@work.bitmover.com>#1/1
X-Deja-AN: 640643560
Approved: n...@nntp-server.caltech.edu
X-To: Linus Torvalds <torva...@transmeta.com>
X-Cc: Larry McVoy <l...@bitmover.com>, David Lang <david.l...@digitalinsight.com>,
Benno Senoner <sbe...@gardena.net>, linux-ker...@vger.rutgers.edu, Paul Barton-Davis <p...@op.net>
Newsgroups: mlist.linux.kernel
> On Thu, 29 Jun 2000, Larry McVoy wrote:
> > OK, now we are getting somewhere. So I have a kernel level source, user
> > level processing, and a kernel level sink, right? And the problem is that
> > we can have long delays which mess that up. Here is how you handle that
> > with RTLinux:
>
> Well, I personally would rather see that nobody ever needed RTlinux at
> all. I think hard realtime is a waste of time, myself, and only to be used
> for the case where the CPU speed is not overwhelmingly fast enough (and
> these days, for most problems the CPU _is_ so overwhelmingly "fast enough"
> that hard realtime should be a non-issue).
>
> I definitely agree with low-latency requirements even in a standard Linux.
> I just disagree violently with doing them with horrible cludges instead of
> working on doing it right.
I guess the problem I have is that I don't see a clean way to make sure
that no high latency events ever creep into the kernel. I'm 100% in
agreement with the idea that all code paths through the kernel should
be short and sweet, but that isn't always the case. All it takes is
one misbehaving driver that hangs onto the CPU too long and you missed
your deadline.
I'm not a fan of realtime either but I hate half assed realtime creeping
into a time sharing kernel - everything I know personally or have read
about says that this is a bad idea.
Given all that, if you want to take a Linux box and use it to drive a
pile of devices with hard guarentees (think factory floor, CNC devices,
mixers, lots of stuff that is currently done in ASICs), then you need a
better answer than Linux gives, even with the Ingo patches.
Do we just ignore that space?
Seems like we are close to having a decent answer, why not support it
for those applications?
I'd rather support RTLinux than see lots of kludges being slipped into
the generic kernel.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.Pine.LNX.4.10.10006291809130.856-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 640643579
Approved: n...@nntp-server.caltech.edu
X-To: Larry McVoy <l...@bitmover.com>
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-Cc: David Lang <david.l...@digitalinsight.com>, Benno Senoner
<sbe...@gardena.net>, linux-ker...@vger.rutgers.edu, Paul Barton-Davis <p...@op.net>
Newsgroups: mlist.linux.kernel
On Thu, 29 Jun 2000, Larry McVoy wrote:
>
> I guess the problem I have is that I don't see a clean way to make sure
> that no high latency events ever creep into the kernel. I'm 100% in
> agreement with the idea that all code paths through the kernel should
> be short and sweet, but that isn't always the case. All it takes is
> one misbehaving driver that hangs onto the CPU too long and you missed
> your deadline.
I agree. Badly written code will be a problem. The approach that the
patches so far have taken is to just add scheduling points all over the
map.
> I'd rather support RTLinux than see lots of kludges being slipped into
> the generic kernel.
In many cases I just think that RTLinux is a worse fix than the disease. I
think RTLinux is perfect for those things that truly need latency
guarantees: no OS at _all_ in the way. But using it for "normal" stuff
like just streaming audio and video is overkill. They don't have
microsecond latency requirements.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Paul Barton-Davis <p...@Op.Net>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.200006300238.WAA15945@renoir.op.net>#1/1
X-Deja-AN: 640671025
Approved: n...@nntp-server.caltech.edu
X-To: torva...@transmeta.com
X-Cc: linux-ker...@vger.rutgers.edu
Newsgroups: mlist.linux.kernel
Linus writes:
>I refuse to have a kernel that is bogged down with random crap all over
>the place. It's wrong. It's distasteful. And it leads to more and more
>crap over time. That's how you get a BAD operating system.
and later (in a different message) wrote:
>Maybe people in the end can convince me that every single scheduling
>point makes 100% sense, and is not a hack at all but a natural
>thing. Even if that were the case, I'd like to get the thing in
>smaller and explainable pieces..
I'm having a hard time reconciling both of these points of view. Maybe
its just that I'm too philosophical and you're too pragmatic. I can
see 2 possibilities from here:
1) your revulsion at "random" scheduling points is a really strong
belief that would likely make convincing you of the value of
each particular point impossible,
2) you you accept the idea that there may need to be a bunch of
"random" scheduling points for this to work, and whilst you
consider this ugly, you accept that there isn't much of an
alternative. people will have to have a lot of good numbers
to convince you to apply a patch that adds a scheduling point.
Do either of these sound like a reasonable summary of your position,
or is there some other precis ?
>flawed. The reason we can't do that ugly thing is also the reason that
>people want to use Linux in the first place. Otherwise you might as well
>run Windows or something else.
Windows is not an option. It doesn't get the job done. If it stays up,
its better than the current Linux kernels for this stuff, but thats
not saying much.
>If it cannot be handled with just a few well-placed things, then it
>shouldn't be handled at all, and you should be working on trying to make
>the UP-threaded kernel work ok, so that for the 2.5.x timeframe we won't
>have this issue any more.
Can someone explain what is meant by getting the UP-threaded kernel to
work ok, and how this would impact scheduling jitter ? If a kernel
thread blocks interrupts for a long time, how can it help to have a
multi-threaded kernel ? My very rough impression is that about 40% of
the required reschedules come from added or pre-existing scheduling
points, and 60% come from a reschedule on return from an interrupt. If
interrupts are disabled, it seems to me as though we are still hosed.
>Live with it.
We can't. *Nobody* will run our apps if the scheduling jitter causes
audio dropouts and lousy live performance response. We can "die" with
it, however.
--p
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/29
Message-ID: <linux.kernel.Pine.LNX.4.10.10006291953110.1043-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 640671045
Approved: n...@nntp-server.caltech.edu
X-To: Paul Barton-Davis <p...@Op.Net>
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-Cc: linux-ker...@vger.rutgers.edu
Newsgroups: mlist.linux.kernel
On Thu, 29 Jun 2000, Paul Barton-Davis wrote:
>
> I'm having a hard time reconciling both of these points of view. Maybe
> its just that I'm too philosophical and you're too pragmatic.
I _am_ pragmatic. That which works, works, and theory can go screw itself.
However, my pragmatism also extends to maintainability, which is why I
also want it done well.
> I can
> see 2 possibilities from here:
>
> 1) your revulsion at "random" scheduling points is a really strong
> belief that would likely make convincing you of the value of
> each particular point impossible,
I want more than an explanation like "I ran our latency tester, and this
point seemed to be really bad, so I added a scheduling point here".
For example, for the specific read/write user-copy code, I don't even need
to get numbers. It's clear that with a machine that has tons of memory,
and where the data is cached, we can generate a "read()" system call that
spends quite a lot of time without needing any active re-scheduling. This
is not a random point: it's something that can be clearly explained from
the sources, and a case where there clearly is no better solution unless
you fully thread the thing.
> 2) you you accept the idea that there may need to be a bunch of
> "random" scheduling points for this to work, and whilst you
> consider this ugly, you accept that there isn't much of an
> alternative. people will have to have a lot of good numbers
> to convince you to apply a patch that adds a scheduling point.
More than just numbers, but yes. I'd like to know that the code isn't just
crap. For example, let's say that something uses an O(n^3) algorithm, and
to "overcome" the expense of this thing we add scheduling points in it.
That's the easy way to do it. But maybe the right thing to do is to
realize that the code may be badly structured in the first place?
> Can someone explain what is meant by getting the UP-threaded kernel to
> work ok, and how this would impact scheduling jitter ? If a kernel
> thread blocks interrupts for a long time, how can it help to have a
> multi-threaded kernel ?
If the kernel blocks interrupts for a long time, adding re-scheduling
points won't help. You must not schedule with interrupts blocked anyway.
The multi-threaded UP-kernel depends on the fine-grained locks that we
_already_ have, as a result of SMP threading. We don't use them at all on
UP: which is quite nice from a performance perspective, because a no-op is
always faster than threading something. But we could use them
intelligently, and get a UP kernel that can be pre-empted 90% of the time.
Making the explicit scheduling points much less of an issue..
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Benno Senoner <sbe...@gardena.net>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/30
Message-ID: <linux.kernel.00063004352105.00815@linuxhost.localdomain>#1/1
X-Deja-AN: 640657561
Approved: n...@nntp-server.caltech.edu
X-To: Linus Torvalds <torva...@transmeta.com>, Larry McVoy <l...@bitmover.com>
Content-Type: text/plain; charset=US-ASCII
MIME-Version: 1.0
X-Cc: David Lang <david.l...@digitalinsight.com>, linux-ker...@vger.rutgers.edu,
Paul Barton-Davis <p...@op.net>
Newsgroups: mlist.linux.kernel
On Fri, 30 Jun 2000, Linus Torvalds wrote:
> On Thu, 29 Jun 2000, Larry McVoy wrote:
> >
> > I guess the problem I have is that I don't see a clean way to make sure
> > that no high latency events ever creep into the kernel. I'm 100% in
> > agreement with the idea that all code paths through the kernel should
> > be short and sweet, but that isn't always the case. All it takes is
> > one misbehaving driver that hangs onto the CPU too long and you missed
> > your deadline.
>
> I agree. Badly written code will be a problem. The approach that the
> patches so far have taken is to just add scheduling points all over the
> map.
But do you think the way to avoid that bad code (accidentally) gets into the
kernel, is to run extensive latency benchmarks (which should uncover the bad
code) each time a new kernel release comes out ?
Just like we run bonnie on each new kernel release in order to ensure that
we still get the full disk speed.
>
> > I'd rather support RTLinux than see lots of kludges being slipped into
> > the generic kernel.
>
> In many cases I just think that RTLinux is a worse fix than the disease. I
> think RTLinux is perfect for those things that truly need latency
> guarantees: no OS at _all_ in the way. But using it for "normal" stuff
> like just streaming audio and video is overkill. They don't have
> microsecond latency requirements.
Amen.
Linus, to remove my last doubts:
(I think I will write a FAQ out of all stuff I collected today)
assume someone rolls out a clean low latency approach for 2.4 ,
would you accept the patches or would you delay the stuff to 2.5 ?
(I have no idea if it is a big amount of work)
there are people that want to use low latency applications now:
should we encourage the use of Ingo's kernels
(which said he will do a 2.4 version of his 2.2 patches too),
perhaps by providing RPMs (or DEBs) with the patched kernels for the various
distros, so that it will me more easy for beginners to use it ?
(adding a statement saying that the current lowlatency kernels achieve their
performance by rescheduling points (which is a bit of a hack), but that future
linux kernels will deliver similar performance using the clean design , meaning
that future linux distos will include the "well realtime-performing" kernel by
default.
>
> Linus
cheers,
Benno.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: "Theodore Y. Ts'o" <ty...@MIT.EDU>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/30
Message-ID: <linux.kernel.200006301310.JAA06222@tsx-prime.MIT.EDU>#1/1
X-Deja-AN: 640849374
Approved: n...@nntp-server.caltech.edu
X-To: Benno Senoner <sbe...@gardena.net>
X-Cc: Linus Torvalds <torva...@transmeta.com>, Larry McVoy <l...@bitmover.com>,
David Lang <david.l...@digitalinsight.com>, linux-ker...@vger.rutgers.edu,
Paul Barton-Davis <p...@op.net>
Newsgroups: mlist.linux.kernel
From: Benno Senoner <sbe...@gardena.net>
Date: Fri, 30 Jun 2000 04:16:14 +0200
assume someone rolls out a clean low latency approach for 2.4 ,
would you accept the patches or would you delay the stuff to 2.5 ?
(I have no idea if it is a big amount of work)
I'd like to remind people that we're at Linux 2.4.0test2, and **test**,
plus Linus's earlier declaration of a code (really feature) freeze imples
this is probably not the time to be making changes to core kernel
internals?
I'd really like a stable Linux kernel released sometime before the end
of this century, thank you very much.
Date: Thu, 29 Jun 2000 20:02:02 -0700 (PDT)
From: Linus Torvalds <torva...@transmeta.com>
More than just numbers, but yes. I'd like to know that the code isn't just
crap. For example, let's say that something uses an O(n^3) algorithm, and
to "overcome" the expense of this thing we add scheduling points in it.
That's the easy way to do it. But maybe the right thing to do is to
realize that the code may be badly structured in the first place?
Ah, you've looked at the POSIX byte-range locking code recently, have
you? The last time Al Viro, Ben LaHaise, and I looked at it, we think
we figured out that it was O(n^3), but having just eaten, we didn't
spend more time looking at it than was necessary. :-)
From: David Schleef <d...@stm.lbl.gov>
Date: Fri, 30 Jun 2000 04:06:29 -0700
How about these?
- Winmodems
For $30 dollars, you can buy yourself a real modem. (And be much happier)
(We don't have any real drivers for them anyway; well, there are two GPL
violations, which I need work on....)
- crappy UARTs running at high speeds
For $10 dollars, you can buy yourself a real UART. (And be much happier)
- High-accuracy stratum-1 NTP servers
- Almost any data acquisition application with a board that
costs less than USD 1000.
And these are common applications? More importantly, both are ones
which can *easily* be handled by RT Linux.
Multimedia applications are the only ones people have listed that for
which I'm even remotely sympathetic, and there I agree that 100-300ms
latency in the kernel is bad, and we need to get that down. But please,
make that a 2.5 priority.
(I'll note that on my SMP machine, I'm running MP3 decoders much of the
time, and in practice very rarely hear any dropouts. So the folks who
are painting such a dire picture of the state of Multimedia are
overstating their case a little, I think.)
- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: Paul Barton-Davis <p...@Op.Net>
Subject: Re: a joint letter on low latency and Linux
Date: 2000/06/30
Message-ID: <linux.kernel.200006301506.LAA12457@renoir.op.net>#1/1
X-Deja-AN: 640904888
Approved: n...@nntp-server.caltech.edu
X-To: "Theodore Y. Ts'o" <ty...@MIT.EDU>
X-Cc: linux-ker...@vger.rutgers.edu
Newsgroups: mlist.linux.kernel
>I'd like to remind people that we're at Linux 2.4.0test2, and **test**,
>plus Linus's earlier declaration of a code (really feature) freeze imples
>this is probably not the time to be making changes to core kernel
>internals?
Well, I'm sympathetic to this. But just yesterday, I saw Stephen
Tweedie saying that the VM system needed another fairly significant
redesign begore it could be considered ready for 2.4.0. Of course, I
was completely mystified just what Linus was thinking with this
numbering choice. Switching to 2.4-anything when the VM system and
streaming disk I/O are still clearly not working correctly seemed
incredible to me.
>Multimedia applications are the only ones people have listed that for
>which I'm even remotely sympathetic, and there I agree that 100-300ms
>latency in the kernel is bad, and we need to get that down. But please,
>make that a 2.5 priority.
>
>(I'll note that on my SMP machine, I'm running MP3 decoders much of the
>time, and in practice very rarely hear any dropouts. So the folks who
>are painting such a dire picture of the state of Multimedia are
>overstating their case a little, I think.)
I purposely did *not* use the term multimedia to describe what most of
the signees are up to. As I've stated many times, simple playback and
record of audio+MIDI (+video? don't know) under Linux works fine. One
might even say "very well".
But the real-time synthesis and processing stuff is a totally
different story, and the situation really *is* dire.
--p
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/