Tech Insider					   Technology and Trends


			   USENET Archives


Electronic mail:			      WorldWideWeb:
   tech-insider@outlook.com		         http://tech-insider.org/

Subject: 2.4.6-pre2, pre3 VM Behavior
Message-ID: <992460707.3b27bfa31aa98@eargle.com>
Date: Wed, 13 Jun 2001 21:40:07 +0200
From: Tom Sightler <tt...@tuxyturvy.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7BIT
User-Agent: IMP/PHP IMAP webmail program 2.2.5
Sender: robo...@news.nic.it
X-Mailing-List: linux-kernel@vger.kernel.org
Approved: robo...@news.nic.it (1.20)
NNTP-Posting-Host: 98980.anti-phl.bofh.it
Newsgroups: linux.kernel
Organization: linux.*_mail_to_news_unidirectional_gateway
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
newsfeeds.belnet.be!news.belnet.be!newsfeed00.sul.t-online.de!t-online.de!
bofh.it!robomod
X-Original-Date: Wed, 13 Jun 2001 15:31:47 -0400 (EDT)
X-Original-Sender: linux-kernel-ow...@vger.kernel.org
X-Original-To: Linux-Kernel <linux-ker...@vger.kernel.org>
Lines: 52

Hi All,

I have been using the 2.4.x kernels since the 2.4.0-test days on my Dell 5000e
laptop with 320MB of RAM and have experienced first hand many of the problems
other users have reported with the VM system in 2.4.  Most of these problems
have been only minor anoyances and I have continued testing kernels as the 2.4
series has continued, mostly without noticing much change.

With 2.4.6-pre2, and -pre3 I can say that I have seen a marked improvement on my
machine, especially in interactive response, for my day to day workstation uses.
 However, I do have one observation that seems rather strange, or at least wrong.

I, on occasion, have the need to transfer relatively large files (750MB-1GB)
from our larger Linux servers to my machine.  I usually use ftp to transfer
these files and this is where I notice the following:

1.  Transfer of the first 100-150MB is very fast (9.8MB/sec via 100Mb Ethernet,
close to wire speed).  At this point Linux has yet to write the first byte to
disk.  OK, this might be an exaggerated, but very little disk activity has
occured on my laptop.

2.  Suddenly it's as if Linux says, "Damn, I've got a lot of data to flush,
maybe I should do that" then the hard drive light comes on solid for several
seconds.  During this time the ftp transfer drops to about 1/5 of the original
speed.

3.  After the initial burst of data is written things seem much more reasonable,
and data streams to the disk almost continually while the rest of the transfer
completes at near full speed again.

Basically, it seems the kernel buffers all of the incoming file up to nearly
available memory before it begins to panic and starts flushing the file to disk.
 It seems it should start to lazy write somewhat ealier.  Perhaps some of this
is tuneable from userland and I just don't know how.

This was much less noticeable on a server with a much faster SCSI hard disk
subsystem as it took significantly less time to flush the information to the
disk once it finally starterd, but laptop hard drives are traditionally poor
performers and at 15MB/s it take 10-15 seconds before things stable out, just
from transferring a file.

Anyway, things are still much better, with older kernels things would almost
seem locked up during those 10-15 seconds but now my apps stay fairly responsive
(I can still type in AbiWord, browse in Mozilla, etc).

Later,
Tom
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Date: Wed, 13 Jun 2001 22:30:12 +0200
From: Rik van Riel <r...@conectiva.com.br>
X-X-Sender: <r...@duckman.distro.conectiva>
Subject: Re: 2.4.6-pre2, pre3 VM Behavior
In-Reply-To: <992460707.3b27bfa31aa98@eargle.com>
Message-ID: <Pine.LNX.4.33.0106131716510.1742-100000@duckman.distro.conectiva>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: robo...@news.nic.it
X-Mailing-List: linux-kernel@vger.kernel.org
Approved: robo...@news.nic.it (1.20)
NNTP-Posting-Host: 38179.anti-phl.bofh.it
Newsgroups: linux.kernel
Organization: linux.*_mail_to_news_unidirectional_gateway
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!nntp.infostrada.it!bofh.it!robomod
References: <992460707.3b27bfa31aa98@eargle.com>
X-Original-Cc: Linux-Kernel <linux-ker...@vger.kernel.org>
X-Original-Date: Wed, 13 Jun 2001 17:21:10 -0300 (BRST)
X-Original-Sender: linux-kernel-ow...@vger.kernel.org
X-Original-To: Tom Sightler <tt...@tuxyturvy.com>
Lines: 68

On Wed, 13 Jun 2001, Tom Sightler wrote:

> 1.  Transfer of the first 100-150MB is very fast (9.8MB/sec via 100Mb Ethernet,
> close to wire speed).  At this point Linux has yet to write the first byte to
> disk.  OK, this might be an exaggerated, but very little disk activity has
> occured on my laptop.
>
> 2.  Suddenly it's as if Linux says, "Damn, I've got a lot of data to flush,
> maybe I should do that" then the hard drive light comes on solid for several
> seconds.  During this time the ftp transfer drops to about 1/5 of the original
> speed.
>
> 3.  After the initial burst of data is written things seem much more reasonable,
> and data streams to the disk almost continually while the rest of the transfer
> completes at near full speed again.
>
> Basically, it seems the kernel buffers all of the incoming file up to nearly
> available memory before it begins to panic and starts flushing the file to disk.
>  It seems it should start to lazy write somewhat ealier.
> Perhaps some of this is tuneable from userland and I just don't
> know how.

Actually, it already does the lazy write earlier.

The page reclaim code scans up to 1/4th of the inactive_dirty
pages on the first loop, where it does NOT write things to
disk.

On the second loop, we start asynchronous writeout of data
to disk and and scan up to 1/2 of the inactive_dirty pages,
trying to find clean pages to free.

Only when there simply are no clean pages we resort to
synchronous IO and the system will wait for pages to be
cleaned.

After the initial burst, the system should stabilise,
starting the writeout of pages before we run low on
memory. How to handle the initial burst is something
I haven't figured out yet ... ;)

> Anyway, things are still much better, with older kernels things
> would almost seem locked up during those 10-15 seconds but now
> my apps stay fairly responsive (I can still type in AbiWord,
> browse in Mozilla, etc).

This is due to this smarter handling of the flushing of
dirty pages and due to a more subtle bug where the system
ended up doing synchronous IO on too many pages, whereas
now it only does synchronous IO on _1_ page per scan ;)

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

			   USENET Archives


Notice
******

The materials and information included in this website may only be used
for purposes such as criticism, review, private study, scholarship, or 
research.


Electronic mail:			      WorldWideWeb:
   tech-insider@outlook.com		         http://tech-insider.org/