Tech Insider					     Technology and Trends


			      USENET Archives

From: Benno Senoner <sbe...@gardena.net>
Subject: Scheduling latencies news: less RAM = less latency 
Date: 1999/07/31
Message-ID: <fa.k412ekv.dmuigt@ifi.uio.no>#1/1
X-Deja-AN: 507397611
Original-Date: Sat, 31 Jul 1999 23:34:10 +0200
Sender: owner-linux-ker...@vger.rutgers.edu
Content-Transfer-Encoding: 7BIT
Original-Message-Id: <99073123400600.00720@linuxhost.localdomain>
To: linux-ker...@vger.rutgers.edu, alsa-de...@alsa-project.org
Content-Type: text/plain; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Hi folks, again news 

as Roger Larrson suspected, there are some parts in the kernel
which have a too long execution path:
On his PPro with 512MB RAM, d_lookup takes up to 80ms to execute !

(I put Roger's latency profiling patch on my page)

I tested this on my PII400 (UP box) + 256MB RAM:

The test-kernel was 2.2.10 + a buffer.c patch (from Andrea) + uaccess.h 
(from Mingo) patch, which insert additional schedule() calls. 

I made 3 tests, booting with mem=64m ,  mem=128m and mem=256m
the audio buffer was 4.35ms ( 3x256 bytes)

the results are very interesting: (look at the diagrams !)

mem=64m  :

 /proc stress  3.5ms latency  0 overruns  (this is VERY good !)
 disk write:  12.5ms latency 18 overruns
 disk copy:   11.1ms latency 14 overruns
 disk read:    6.3ms latency 16 overruns  (quite nice :-) )

http://www.gardena.net/benno/linux/audio/2.2.10+patch+64mb/3x256.html


mem=128m  :
 /proc stress  5.4ms latency  102 overruns  
 disk write:  14.5ms latency   20 overruns
 disk copy:   11.4ms latency   43 overruns
 disk read:   10.5ms latency    7 overruns

http://www.gardena.net/benno/linux/audio/2.2.10+patch+128mb/3x256.html

mem=256m  :
 /proc stress  9.2ms latency  101 overruns  
 disk write:  48.7ms latency   13 overruns
 disk copy:   31.6ms latency   24 overruns
 disk read:   11.9ms latency    4 overruns

http://www.gardena.net/benno/linux/audio/2.2.10+patch+256mb/3x256.html


IMHO having less memory , leads to  less mem for disk buffering, 
and less pages to process, so the kernel routines take less time
go through lists, inodes etc.

I hope that the delay of the release of Mingo's patch is related to
the fact that he tries to shorten these paths, which is not always a trivial
task.
Mingo, can you confirm this ?

comments ?

ciao,
Benno.


--
Benno Senoner
E-Mail: sbe...@gardena.net
Linux scheduling latency benchmarks
http://www.gardena.net/benno/linux/audio


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: Scheduling latencies news: less RAM = less latency 
Date: 1999/07/31
Message-ID: <fa.o7qbhnv.g4o7r9@ifi.uio.no>#1/1
X-Deja-AN: 507397612
Original-Date: Sat, 31 Jul 1999 12:59:13 -0700 (PDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.4.10.9907311246150.1878-100000@penguin.transmeta.com>
References: <fa.k412ekv.dmuigt@ifi.uio.no>
To: Benno Senoner <sbe...@gardena.net>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu



On Sat, 31 Jul 1999, Benno Senoner wrote:
> 
> as Roger Larrson suspected, there are some parts in the kernel
> which have a too long execution path:
> On his PPro with 512MB RAM, d_lookup takes up to 80ms to execute !

Somebody has bogus profiling information. There's no way it's going to
take that long unless you have a broken filesystem that doesn't do the
hashes right and hashes everything to the same entry. Even if all of
memory is just dcache information, which is basically impossible to do.

If you get numbers like the above, then ALL your numbers are extremely
suspect, and I would seriously look into your methodology if I were you.

		Linus



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Ingo Molnar <mi...@chiara.csoma.elte.hu>
Subject: Re: Scheduling latencies news: less RAM = less latency 
Date: 1999/07/31
Message-ID: <fa.nlpl0uv.1h4iigl@ifi.uio.no>#1/1
X-Deja-AN: 507400002
Original-Date: Sat, 31 Jul 1999 22:10:03 +0200 (CEST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.3.96.990731220052.25534F-100000@chiara.csoma.elte.hu>
References: <fa.o7qbhnv.g4o7r9@ifi.uio.no>
To: Linus Torvalds <torva...@transmeta.com>
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu


On Sat, 31 Jul 1999, Linus Torvalds wrote:

> Somebody has bogus profiling information. There's no way it's going to

no, it really happens. With 512M RAM and a 4-way Xeon i easily got 20ms+
latencies. These latencies are rare because it's caused by prune_dcache(),
but they do happen. 

some other latencies: si_meminfo() and si_swapinfo() caused a ~5msec delay
even on this box. Console switching and scrolling (when at the end of
video memory) caused ~2msec latencies. big-file truncating caused up to
20msec latencies here, and i havent even tried too hard. exit() on a big
process caused 8msec latency. fork() on a big process likewise. looking at
a big process in /proc caused similar latencies. sync_old_buffers() 
introduced big latencies as well (scanning several thousands of buffers). 
Some 'bad' drivers like the PS2 keyboard code caused 1-3 msec latencies as
well. All these things are fixed by my previous patch - but i'm still
working on the dcache stuff though. 

the conceptual reason of these problems is that on big memory boxes there
is nothing that forces processes to sleep. This means we have to put
'scheduling controls' more explicitly to various points in the kernel. I
do not claim my patch is the correct approach, but something like that has
to be done i'm quite sure. I've been working on this in the last 3 days,
and i'm starting to reach the end of the (quite long) 'latency source'
list. The box feels much more responsive interactively even under high and
complex load.

-- mingo


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: Scheduling latencies news: less RAM = less latency 
Date: 1999/07/31
Message-ID: <fa.o7q3gvv.g4g6je@ifi.uio.no>#1/1
X-Deja-AN: 507405151
Original-Date: Sat, 31 Jul 1999 13:16:01 -0700 (PDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.4.10.9907311311100.1878-100000@penguin.transmeta.com>
References: <fa.nlpl0uv.1h4iigl@ifi.uio.no>
To: Ingo Molnar <mi...@chiara.csoma.elte.hu>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu



On Sat, 31 Jul 1999, Ingo Molnar wrote:
> 
> no, it really happens. With 512M RAM and a 4-way Xeon i easily got
> 20ms+ latencies. These latencies are rare because it's caused by
> prune_dcache(), but they do happen.

prune_dcache() I can believe. But the report was about d_lookup(). So
somebody is using bad profiling information, and that's dangerous.

Also, the si_meminfo() etc stuff is just ridiculous. It's not a question
of latency: it's a question of CPU usage. We need to just get rid of those
functions instead of hacking around them - regardless of whether you add
"reschedule" calls in them, they just eat too much CPU, plain and simple.
Again, please don't treat the symptoms - I will not accept patches that
just say "oh, this is crap, so let's reschedule a bit here". They need to
be fixed properly or not at all.

		Linus


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Ingo Molnar <mi...@chiara.csoma.elte.hu>
Subject: Re: Scheduling latencies news: less RAM = less latency 
Date: 1999/07/31
Message-ID: <fa.nmpn16v.1m4gioi@ifi.uio.no>#1/1
X-Deja-AN: 507405153
Original-Date: Sat, 31 Jul 1999 22:22:57 +0200 (CEST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.3.96.990731221743.25534I-100000@chiara.csoma.elte.hu>
References: <fa.o7q3gvv.g4g6je@ifi.uio.no>
To: Linus Torvalds <torva...@transmeta.com>
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu


On Sat, 31 Jul 1999, Linus Torvalds wrote:

> prune_dcache() I can believe. But the report was about d_lookup(). So
> somebody is using bad profiling information, and that's dangerous.

sure - d_lookup() is definitely bogus.

> Also, the si_meminfo() etc stuff is just ridiculous. It's not a question
> of latency: it's a question of CPU usage. We need to just get rid of
> those functions instead of hacking around them - regardless of whether
> you add "reschedule" calls in them, they just eat too much CPU, plain
> and simple.  Again, please don't treat the symptoms - I will not accept
> patches that just say "oh, this is crap, so let's reschedule a bit
> here". They need to be fixed properly or not at all. 

i agree - I wanted to have something for 2.2 for people to test - i wanted
to cure symptoms first, we have way too many latency sources right now and
i first wanted to get something that can be tested. I think that the
uaccess.h changes are too intrusive as well - i'll rework these things.

	Ingo


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Benno Senoner <sbe...@gardena.net>
Subject: Re: Scheduling latencies news: less RAM = less latency
Date: 1999/07/31
Message-ID: <fa.k316esv.cmuiov@ifi.uio.no>#1/1
X-Deja-AN: 507412150
Original-Date: Sun, 1 Aug 1999 00:23:43 +0200
Sender: owner-linux-ker...@vger.rutgers.edu
Content-Transfer-Encoding: 7BIT
Original-Message-Id: <99080100513801.00720@linuxhost.localdomain>
References: <fa.o7q3gvv.g4g6je@ifi.uio.no>
To: torva...@transmeta.com, Ingo Molnar <mi...@chiara.csoma.elte.hu>
Original-References: <Pine.LNX.4.10.9907311311100.1878-100...@penguin.transmeta.com>
Content-Type: text/plain; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

On Sat, 31 Jul 1999, Linus Torvalds wrote:
> On Sat, 31 Jul 1999, Ingo Molnar wrote:
> > 
> > no, it really happens. With 512M RAM and a 4-way Xeon i easily got
> > 20ms+ latencies. These latencies are rare because it's caused by
> > prune_dcache(), but they do happen.
> 
> prune_dcache() I can believe. But the report was about d_lookup(). So
> somebody is using bad profiling information, and that's dangerous.
> 
> Also, the si_meminfo() etc stuff is just ridiculous. It's not a question
> of latency: it's a question of CPU usage. We need to just get rid of those
> functions instead of hacking around them - regardless of whether you add
> "reschedule" calls in them, they just eat too much CPU, plain and simple.
> Again, please don't treat the symptoms - I will not accept patches that
> just say "oh, this is crap, so let's reschedule a bit here". They need to
> be fixed properly or not at all.
> 
> 		Linus

Linus, I agree that this solution is perhaps not the cleanest, 
one thing to check, is how much slower this kernel would be
compared to a standard kernel.
I think "reschedules" during lenghty kernel operations is not a so bad
idea, the important thing is not to reschedule too often, to avoid wasting
too much CPU time in the scheduler.
I'm easily willing to trade 1-5% of the CPU in exchange of a responsive <5ms
latency system.
If the performance drop worries you, we could add this as a compile time
option, "kernel optimized for server", or "kernel optimized for multimedia" .

If's ridiculous to get up to 150ms latencies on a powerful machine like the
PII400 on Linux.
Even a Windows user (with a properly tuned machine) laughs at these values.

Simple reschedules in uaccess.h + buffer.c lowered the latency *DRAMATICALLY*
on my box, about an order of magnitude.  ( /proc down to 3ms , disk read down
to 6ms)


Linus, what are your proposal for making the kernel "low-latency" in a CLEAN
way ?

I think making the kernel fully preemptable is not an easy task and will
not happen very soon.

Plus what disturbs me is the busy-waiting for RT processes for sleeps <2ms
2ms is PLENTY of time on modern CPUs,and I call THIS wasitng CPU time,
 therefore we should change this approach.

comments ?

P.S: the profiling patch and infos are from Roger Larrson 
(nra02...@norran.net).
The profiling-patch is on my page if you want. 

 Benno.


--
Benno Senoner
E-Mail: sbe...@gardena.net
Linux scheduling latency benchmarks
http://www.gardena.net/benno/linux/audio


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Ingo Molnar <mi...@chiara.csoma.elte.hu>
Subject: Re: Scheduling latencies news: less RAM = less latency
Date: 1999/07/31
Message-ID: <fa.nj9r2ev.1jk8i0m@ifi.uio.no>#1/1
X-Deja-AN: 507416708
Original-Date: Sat, 31 Jul 1999 23:00:08 +0200 (CEST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.3.96.990731225242.26845A-100000@chiara.csoma.elte.hu>
References: <fa.k316esv.cmuiov@ifi.uio.no>
To: Benno Senoner <sbe...@gardena.net>
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu


On Sun, 1 Aug 1999, Benno Senoner wrote:

> I think "reschedules" during lenghty kernel operations is not a so bad
> idea, [...]

i think you have a misconception - the concept of decreasing latencies in
a mandatory way is nothing new to Linux, we've been doing it for quite
some time. (mem.c, lp.c, etc.) We just have to export it to all affected
places - now that bigger memory boxes became widespread. We already (of
course!) do 'reschedules during lengthy kernel operations', when the need
arises, we just dont do it explicitly when _another_ process wants us to
reschedule. The patch i've sent fills this hole - more coming.

> I'm easily willing to trade 1-5% of the CPU in exchange of a responsive
> <5ms latency system. 

my guesstimate is that we will not see any numbers getting worse. 
Actually, some RL numbers will get much better. (eg. i can already log in
much faster during heavy kernel compiles) The thing you missed i think is
that we do not have to do a reschedule, we just have to check the (local) 
->need_resched flag at a couple of well-chosen places. (see the patch)

> If the performance drop worries you, we could add this as a compile time
> option, "kernel optimized for server", or "kernel optimized for
> multimedia" . 

you are arguing the wrong point ... Latency _is_ top priority in Linux. 
And it's not hard to get it, it's hard to _measure_ it reliably. (and this
is why your tool is so cool.)

> If's ridiculous to get up to 150ms latencies on a powerful machine like
> the PII400 on Linux. 

i can get excellent latencies on my box, without giving up speed - good
enough?

-- mingo


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: Scheduling latencies news: less RAM = less latency 
Date: 1999/08/01
Message-ID: <fa.o8a1g7v.hke7b0@ifi.uio.no>#1/1
X-Deja-AN: 507468791
Original-Date: Sat, 31 Jul 1999 17:34:32 -0700 (PDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.4.10.9907311721130.2036-100000@penguin.transmeta.com>
References: <fa.nmpn16v.1m4gioi@ifi.uio.no>
To: Ingo Molnar <mi...@chiara.csoma.elte.hu>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu



On Sat, 31 Jul 1999, Ingo Molnar wrote:
> 
> On Sat, 31 Jul 1999, Linus Torvalds wrote:
> 
> > prune_dcache() I can believe. But the report was about d_lookup(). So
> > somebody is using bad profiling information, and that's dangerous.
> 
> sure - d_lookup() is definitely bogus.

Note that I suspect that prune_dcache() being a problem spot is probably
entirely valid, but for all the wrong reasons.

For example, I suspect that you can find "free_inodes()" very high on your
profiles if you start really looking. We have a few bad behaviour corner
cases right now that aren't actually all that hard to trigger: because the
new code (rightly) refuses to free inodes when there are pages associated
with them (2.2.x used to just force-free the pages), the logic to handle
"max_inodes" realy is fairly broken.

That brokenness probably feeds over into prune_dcache(). We try
desperately to free inodes, but we can't because a lot of them have pages
associated with them, and that in turn makes us do bad things with the
dcache too (the inode free logic thinks that the reason the inodes are
locked down is a dcache reference - something that _used_ to be true).

So you end up trying to prune the dcache even when the dcache really
doesn't need to be pruned at all, and in situtations where it might not
actually be a good idea (again, the dcache pruning refuses to prune an
entry with an inode that has active pages...)

I've considered just getting rid of the max_inodes thing. The only reason
it exists is because we currently don't free the inodes due to memory
fragmentation, so we need to artifically cut off the inode supply at some
point. We might want to try just using kmalloc/kfree (or a specialized
slab thing), but I suspect that will waste much more memory than we do
now. It may be worth trying, though.

If you're really doing tests in this area, that's something worth looking
into, I suspect. I bet you'll have a much harder time triggering dcache
problems.

> i agree - I wanted to have something for 2.2 for people to test - i wanted
> to cure symptoms first, we have way too many latency sources right now and
> i first wanted to get something that can be tested. I think that the
> uaccess.h changes are too intrusive as well - i'll rework these things.

Note that the "si_meminfo()" thing really started its life as a debugging
check: verifying that the free pages count actually matched what you got
if you counted the pages by hand.

It was later dis-used from that capacity, as it's not even correct:
si_meminfo gets the wrong results for pages that are part of multi-page
allocations (but we didn't use to support multi-page allocations
originally). But in the meantime it had been hooked into sys_getinfo(),
and thus it stayed on, even though it really is a completely broken piece
of code.

It's really ridiculous. The only thing si_meminfo really gets us is the
number of shared pages, and to calculate that it often uses about 10% of
the CPUtime when you have "xosview" active or something like that. We
could trivially keep track of the number of shared pages on the fly, but
I've been lazy.

But getting rid of some of these problems might be valid on 2.2.x too.

		Linus


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: Scheduling latencies news: less RAM = less latency
Date: 1999/08/01
Message-ID: <fa.o7qbgfv.h4o736@ifi.uio.no>#1/1
X-Deja-AN: 507470963
Original-Date: Sat, 31 Jul 1999 17:44:30 -0700 (PDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.4.10.9907311737080.2036-100000@penguin.transmeta.com>
References: <fa.k316esv.cmuiov@ifi.uio.no>
To: Benno Senoner <sbe...@gardena.net>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu



On Sun, 1 Aug 1999, Benno Senoner wrote:
>
> I think "reschedules" during lenghty kernel operations is not a so bad
> idea, the important thing is not to reschedule too often, to avoid wasting
> too much CPU time in the scheduler.

Oh, I agree that "reschedule()" can be a very nice thing, and I'm not
against adding them where necessary.

The thing that makes me reall ynervous about adding them is that it's a
classic case of "easy solution", and once we start going down that path
code quality will start suffering if people aren't REALLY careful.

I suspect that in just about _all_ of the really bad cases you can
fundamentally trace the badness to Linux being stupid rather than any
inherently "lengthy" operation. And then the right fix is to fix the
stupidity rather than add a reschedule.

There are cases where this isn't true. A classic such case is a large
"read()" system call - where we really may end up doing a really lengthy
operation simply because the user asks us to. That's a case where it's not
about being stupid, and where a explicit re-schedule makes 100% sense.

But 

> I'm easily willing to trade 1-5% of the CPU in exchange of a responsive <5ms
> latency system.

It shouldn't be necessary. In most cases bad latency should be traceable
to bad performance - you can improve latency by _improving_ performance
rather than making it worse.

> If the performance drop worries you, we could add this as a compile time
> option, "kernel optimized for server", or "kernel optimized for multimedia" .

That is the Windows NT approach, and it is _wrong_. 

> If's ridiculous to get up to 150ms latencies on a powerful machine like the
> PII400 on Linux.

Agreed. And it is even more ridiculous to add stupid code to "fix" it.

> Linus, what are your proposal for making the kernel "low-latency" in a CLEAN
> way ?

I sent out two of them. "si_meminfo()" should just go away. Same goes for
"prune_dcache()", although the latter probably doesn't need to go away, it
just needs to have fewer bad corner cases.

> I think making the kernel fully preemptable is not an easy task and will
> not happen very soon.

It will not happen EVER. 

Pre-emptable kernels are stupid, and anybody who thinks otherwise needs to
get his head examined. 

But trying to work around bad programming by adding MORE bad programming
is also extremely stupid.

			Linus


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Benno Senoner <sbe...@gardena.net>
Subject: Low-latency patches working GREAT (<2.9ms audio latency), 
see testresults ,but ISDN troubles
Date: 1999/08/28
Message-ID: <fa.k21ohcv.fm4h8u@ifi.uio.no>#1/1
X-Deja-AN: 518278955
Original-Date: Sat, 28 Aug 1999 22:40:57 +0200
Sender: owner-linux-ker...@vger.rutgers.edu
Content-Transfer-Encoding: 7BIT
Original-Message-Id: <99082823144300.00799@linuxhost.localdomain>
To: linux-ker...@vger.rutgers.edu, linux-so...@vger.rutgers.edu, 
alsa-de...@alsa-project.org
Content-Type: text/plain; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Hi,

I benchmarked Mingo's latest low-latency patches (2.2.10-N6 a bit modified)

The patches give me excellent results with sporadic very 2.9ms peaks !

See the testresults here:
http://www.gardena.net/benno/linux/audio/2.2.10-n6b/index.html

You can get the patch here:
http://www.gardena.net/benno/linux/audio/patches/lowlatency-2.2.10-N6B.patch

More details on my audio page.

Unfortunately the patch has still some problems , not latency-related:

- The ISDN hisax driver (my card is a Fritz classic) crashes the kernel at
modprobe hisax.o
(does not happen on an unpatched 2.2.10 kernel)   
Can someone of the ISDN maintainers please reproduce/fix this ?
(maybe a race at module initialization ?) 

- The disk performance decreases by 10-25% when I increase the CPU load in
the "latencytest" bench.
(On light CPU load there are no disk performance differences,
maybe this is related to higher scheduling overhead)

I think most of us want to have these "low-latency" features in the upcoming
2.4 kernel since it will make Linux a very good _MULTIMEDIA_OS_.

With Mingo's patches the Linux low-latency performance comes very close
to BEOS, and is much much better (3-4 times) Windows on the same hardware.
It's now time to stress audio-software vendors to port their cool apps to Linux.

comments ?


PS: To Microsoft's Anti-Linux team: just download the patch and compare the
performance with your crappy DirectX API :-)

regards,
Benno.


--
Benno Senoner
E-Mail: sbe...@gardena.net
Linux scheduling latency benchmarks
http://www.gardena.net/benno/linux/audio


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: yodai...@fsmlabs.com
Subject: Re: [rtl] Low-latency patches working GREAT (<2.9ms audio latency), 
see testresults ,but ISDN troubles
Date: 1999/08/28
Message-ID: <fa.f85ke2v.1f24tb8@ifi.uio.no>#1/1
X-Deja-AN: 518348561
Original-Date: Sat, 28 Aug 1999 17:55:10 -0600
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <19990828175510.A4356@hq.fsmlabs.com>
References: <fa.k21ohcv.fm4h8u@ifi.uio.no>
To: Benno Senoner <sbe...@gardena.net>
Original-References: <99082823144300.00...@linuxhost.localdomain>
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
Mime-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

On Sat, Aug 28, 1999 at 10:40:57PM +0200, Benno Senoner wrote:
> - The disk performance decreases by 10-25% when I increase the CPU load in
> the "latencytest" bench.
> (On light CPU load there are no disk performance differences,
> maybe this is related to higher scheduling overhead)
> 
> I think most of us want to have these "low-latency" features in the upcoming
> 2.4 kernel since it will make Linux a very good _MULTIMEDIA_OS_.


A 25% disk i/o decrease is very serious. Lets get some serious feedback
from people running internet and database servers before we blow off
the server users in order to compete with BEOS.



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Linus Torvalds <torva...@transmeta.com>
Subject: Re: [rtl] Low-latency patches working GREAT (<2.9ms audio latency), 
see testresults ,but ISDN troubles
Date: 1999/08/29
Message-ID: <fa.o9a3g7v.hkg6ra@ifi.uio.no>#1/1
X-Deja-AN: 518422414
Original-Date: Sat, 28 Aug 1999 23:21:42 -0700 (PDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.4.10.9908282320260.2144-100000@penguin.transmeta.com>
References: <fa.f85ke2v.1f24tb8@ifi.uio.no>
To: yodai...@fsmlabs.com
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu



On Sat, 28 Aug 1999 yodai...@fsmlabs.com wrote:
> 
> A 25% disk i/o decrease is very serious. Lets get some serious feedback
> from people running internet and database servers before we blow off
> the server users in order to compete with BEOS.

Guys, if anybody thinks we're competing with BeOS, then wake up. BeOS is a
niche OS that isn't worth competing against, and at most we can try to
find out what it's good at and see if we can emulate some of it. But 25%
disk IO decrease is definitely not something we want to even consider.

		Linus


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Ingo Molnar <mi...@chiara.csoma.elte.hu>
Subject: Re: [rtl] Low-latency patches working GREAT (<2.9ms audio latency), 
see testresults ,but ISDN troubles
Date: 1999/08/29
Message-ID: <fa.lgoji0v.f5s0g6@ifi.uio.no>#1/1
X-Deja-AN: 518430209
Original-Date: Sun, 29 Aug 1999 09:17:31 +0200 (CEST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.3.96.990829091531.4857C-100000@chiara.csoma.elte.hu>
References: <fa.gq5evnv.1p5un9c@ifi.uio.no>
To: Alan Cox <a...@lxorguk.ukuu.org.uk>
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu


On Sun, 29 Aug 1999, Alan Cox wrote:

> We also have tons of time. Its not a 2.2. candidate

yes. between 2.2 and 2.3 there were so many fundamental changes that i
will not even try to clean the 2.2 patch up - it would be too fundamental
anyway. If anyone wants a solution for 2.2, the current patch can be used
(it's 100% stable), but i'm developing the 'clean patch' only for 2.3.

-- mingo


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Ingo Molnar <mi...@chiara.csoma.elte.hu>
Subject: Re: [rtl] Low-latency patches working GREAT (<2.9ms audio latency), 
see testresults ,but ISDN troubles
Date: 1999/08/29
Message-ID: <fa.lgonj0v.f5k1g9@ifi.uio.no>#1/1
X-Deja-AN: 518432634
Original-Date: Sun, 29 Aug 1999 09:13:26 +0200 (CEST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.3.96.990829085834.4857A-100000@chiara.csoma.elte.hu>
References: <fa.o9a3g7v.hkg6ra@ifi.uio.no>
To: Linus Torvalds <torva...@transmeta.com>
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu


On Sat, 28 Aug 1999, Linus Torvalds wrote: 

> Guys, if anybody thinks we're competing with BeOS, then wake up. BeOS is
> a niche OS that isn't worth competing against, and at most we can try to
> find out what it's good at and see if we can emulate some of it. But 25%
> disk IO decrease is definitely not something we want to even consider. 

definitely. The patches are 'work in progress', and i know they are only
acceptable for inclusion if they: 

	1) cause no measureable slowdown _anywhere_

	2) fixes 'buggy' latencies by redesigning the latency source, not
	   by intruding the latency core with conditional reschedule
	   points.

i'm quite certain both 1) and 2) are very much possible, Benno's reported
25% slowdown (which btw. i think is not a pure bandwith degradation, but a
slowdown in certain, not necesserily well understood circumstances) is
simply a bug in my patch.

the patch in it's current form is just a 'demo' that shows that good
in-kernel latencies (the example to follow is not BeOS but QNX i think)
are indeed possible without architectural impact. On my box i've killed
all latencies that are bigger than 0.5 msec. (i'll try to fix the 2.9msec
peak reported as well) I'll send an updated and fixed patch (in smaller
pieces) for 2.3.

-- mingo


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Ingo Molnar <mi...@chiara.csoma.elte.hu>
Subject: Re: Low-latency patches working GREAT (<2.9ms audio latency), 
see testresults ,but ISDN troubles
Date: 1999/08/29
Message-ID: <fa.lfovi0v.e5g0gc@ifi.uio.no>#1/1
X-Deja-AN: 518432635
Original-Date: Sun, 29 Aug 1999 09:22:24 +0200 (CEST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.3.96.990829091817.4857D-100000@chiara.csoma.elte.hu>
References: <fa.k21ohcv.fm4h8u@ifi.uio.no>
To: Benno Senoner <sbe...@gardena.net>
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu


On Sat, 28 Aug 1999, Benno Senoner wrote:

> - The disk performance decreases by 10-25% when I increase the CPU load
> in the "latencytest" bench.  (On light CPU load there are no disk
> performance differences, maybe this is related to higher scheduling
> overhead) 

just in case anyone misunderstood the above result (as i think many did). 
The 'CPU load' mentioned above is a _realtime process_ (unless Benno you
changed the benchmark). This is a _good_ result.  'increase CPU load'
simply means 'generate more RT load' - which in _this case_ might mean
less disk performance - but the RT task asked for it in the first place.
All in one: i can see no problem here.

if a 'simple' CPU-user (non-RT) is getting more active there is no
slowdown.

-- mingo


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Paul Barton-Davis <p...@Op.Net>
Subject: Re: [alsa-devel] Re: [rtl] Low-latency patches working GREAT 
(<2.9ms audio latency), see testresults ,but ISDN troubles 
Date: 1999/08/29
Message-ID: <fa.ehkmisv.1qmu0hl@ifi.uio.no>#1/1
X-Deja-AN: 518432636
Original-Date: Sun, 29 Aug 1999 03:34:41 -0400
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <199908290728.DAA29729@renoir.op.net>
References: <fa.o9a3g7v.hkg6ra@ifi.uio.no>
To: linux-ker...@vger.rutgers.edu, linux-so...@vger.rutgers.edu, 
alsa-de...@alsa-project.org
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Linus wrote:

>Guys, if anybody thinks we're competing with BeOS, then wake up. BeOS is a

last i heard, we weren't "competing" with anyone. we were trying to
write a really really good OS. 

BeOS is unique in that its the first vaguely stable OS to be designed
with multimedia in mind from the ground up. there are those of us who
see computing technology as either currently or very soon delivering
the power to do things with audio and video that have been hard to do
without special purpose hardware as little as two years ago. right
now, linux is not the right platform for doing this, and BeOS is, and
I don't like that.

there is persistent denial on l-k and elsewhere that an OS designed to
effectively support "traditional" computer usage (compilation, number
crunching, data serving) has any fundamental differences from one
based around, say, real-time audio generation & manipulation.

i doubt that this is true, but burying our heads in the sand won't
change the answer one way or another. i personally suspect that the
only reason BeOS has a niche *at all* is (1) because its true that there
is a difference and (2) because there are companies out there who
would like to develop for something better than Win/Mac and are scared
to do the linux^H^H^H^H^Hright thing.

it would be very nice to be able to compile linux with options that
say, in essence: "this is a mostly single-user box with little or no
network use, not much disk I/O that really benefits from a smart
filesystem, and regular use of soft-real-time tasks that want to take
over the machine if at all possible."

this is a profile that has a very different emphasis from many of the
things that kernel folks are working on supporting. to dismiss this
kind of system as a "niche" is to potentially miss a broad wave in
computing usage over the next decade or so. IMHO.

>niche OS that isn't worth competing against, and at most we can try to
>find out what it's good at and see if we can emulate some of it. But 25%

"emulate" ? i'd rather we copy it and/or do it better.

>disk IO decrease is definitely not something we want to even consider.

oh come on linus - nobody has gotten close to suggesting that its an
acceptable change! benno was merely observing the current status of
things. i strongly suspect that victor's "tunable parameter"
suggestion could solve quite a bit of the difference.

--p

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Benno Senoner <sbe...@gardena.net>
Subject: Re: Low-latency patches working GREAT (<2.9ms audio latency), 
see testresults ,but ISDN troubles
Date: 1999/08/29
Message-ID: <fa.k4hke4v.d68igj@ifi.uio.no>#1/1
X-Deja-AN: 518640661
Original-Date: Mon, 30 Aug 1999 01:41:58 +0200
Sender: owner-linux-ker...@vger.rutgers.edu
Content-Transfer-Encoding: 7BIT
Original-Message-Id: <99083001561703.00812@linuxhost.localdomain>
References: <fa.lfovi0v.e5g0gc@ifi.uio.no>
To: Ingo Molnar <mi...@chiara.csoma.elte.hu>
Original-References: <Pine.LNX.3.96.990829091817.4857D-100...@chiara.csoma.elte.hu>
Content-Type: text/plain; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

On Sun, 29 Aug 1999, Ingo Molnar wrote:
> On Sat, 28 Aug 1999, Benno Senoner wrote:
> 
> > - The disk performance decreases by 10-25% when I increase the CPU load
> > in the "latencytest" bench.  (On light CPU load there are no disk
> > performance differences, maybe this is related to higher scheduling
> > overhead) 
> 
> just in case anyone misunderstood the above result (as i think many did). 
> The 'CPU load' mentioned above is a _realtime process_ (unless Benno you
> changed the benchmark). This is a _good_ result.  'increase CPU load'
> simply means 'generate more RT load' - which in _this case_ might mean
> less disk performance - but the RT task asked for it in the first place.
> All in one: i can see no problem here.

Ingo, you are right: I just ran the unmodified latencytest and was measuring 
disk performace.
In my example I used 1.45ms fragments, this means that my realtime thread
has to be rescheduled about 600times/sec ,
and during each 1.45ms cycle the rt thread wastes about 1.1ms (= 80% of 1.45ms)
of the CPU.


If you lower the CPU load to low values ( 10% for example) , there is not a big
substantial disk WRITE performance loss compared to an unpatched kernel.

The interesting thing is that the disk READ performance is unaffected by your
patches, on CPU loads of 80% I get the same disk READ performance on 
your lowlatency-N6 kernel as on a standard kernel.
Perhaps the problem lies in your modifications in the disk write code ?


> 
> if a 'simple' CPU-user (non-RT) is getting more active there is no
> slowdown.

I don't tested this yes, but I think you are right.
(Will report back some numbers when I get the benchmarking done).

You said your patch was 100% stable: are we talking about the same patches ?
The latest I got was -N6 + shm.c backed out + Roger's conditional_schedule()
wrapping in filemap.c.

The kernel is rock solid , except it crashes when I do a modprobe of the ISDN
hisax.o module.

regards,
Benno.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

			        About USENET

USENET (Users’ Network) was a bulletin board shared among many computer
systems around the world. USENET was a logical network, sitting on top
of several physical networks, among them UUCP, BLICN, BERKNET, X.25, and
the ARPANET. Sites on USENET included many universities, private companies
and research organizations. See USENET Archives.

		       SCO Files Lawsuit Against IBM

March 7, 2003 - The SCO Group filed legal action against IBM in the State 
Court of Utah for trade secrets misappropriation, tortious interference, 
unfair competition and breach of contract. The complaint alleges that IBM 
made concentrated efforts to improperly destroy the economic value of 
UNIX, particularly UNIX on Intel, to benefit IBM's Linux services 
business. See SCO vs IBM.

The materials and information included in this website may only be used
for purposes such as criticism, review, private study, scholarship, or
research.

Electronic mail:			       WorldWideWeb:
   tech-insider@outlook.com			  http://tech-insider.org/