Subject: 2.4.6-pre2, pre3 VM Behavior Message-ID: <992460707.3b27bfa31aa98@eargle.com> Date: Wed, 13 Jun 2001 21:40:07 +0200 From: Tom Sightler <tt...@tuxyturvy.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7BIT User-Agent: IMP/PHP IMAP webmail program 2.2.5 Sender: robo...@news.nic.it X-Mailing-List: linux-kernel@vger.kernel.org Approved: robo...@news.nic.it (1.20) NNTP-Posting-Host: 98980.anti-phl.bofh.it Newsgroups: linux.kernel Organization: linux.*_mail_to_news_unidirectional_gateway Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu! newsfeeds.belnet.be!news.belnet.be!newsfeed00.sul.t-online.de!t-online.de! bofh.it!robomod X-Original-Date: Wed, 13 Jun 2001 15:31:47 -0400 (EDT) X-Original-Sender: linux-kernel-ow...@vger.kernel.org X-Original-To: Linux-Kernel <linux-ker...@vger.kernel.org> Lines: 52 Hi All, I have been using the 2.4.x kernels since the 2.4.0-test days on my Dell 5000e laptop with 320MB of RAM and have experienced first hand many of the problems other users have reported with the VM system in 2.4. Most of these problems have been only minor anoyances and I have continued testing kernels as the 2.4 series has continued, mostly without noticing much change. With 2.4.6-pre2, and -pre3 I can say that I have seen a marked improvement on my machine, especially in interactive response, for my day to day workstation uses. However, I do have one observation that seems rather strange, or at least wrong. I, on occasion, have the need to transfer relatively large files (750MB-1GB) from our larger Linux servers to my machine. I usually use ftp to transfer these files and this is where I notice the following: 1. Transfer of the first 100-150MB is very fast (9.8MB/sec via 100Mb Ethernet, close to wire speed). At this point Linux has yet to write the first byte to disk. OK, this might be an exaggerated, but very little disk activity has occured on my laptop. 2. Suddenly it's as if Linux says, "Damn, I've got a lot of data to flush, maybe I should do that" then the hard drive light comes on solid for several seconds. During this time the ftp transfer drops to about 1/5 of the original speed. 3. After the initial burst of data is written things seem much more reasonable, and data streams to the disk almost continually while the rest of the transfer completes at near full speed again. Basically, it seems the kernel buffers all of the incoming file up to nearly available memory before it begins to panic and starts flushing the file to disk. It seems it should start to lazy write somewhat ealier. Perhaps some of this is tuneable from userland and I just don't know how. This was much less noticeable on a server with a much faster SCSI hard disk subsystem as it took significantly less time to flush the information to the disk once it finally starterd, but laptop hard drives are traditionally poor performers and at 15MB/s it take 10-15 seconds before things stable out, just from transferring a file. Anyway, things are still much better, with older kernels things would almost seem locked up during those 10-15 seconds but now my apps stay fairly responsive (I can still type in AbiWord, browse in Mozilla, etc). Later, Tom - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Date: Wed, 13 Jun 2001 22:30:12 +0200 From: Rik van Riel <r...@conectiva.com.br> X-X-Sender: <r...@duckman.distro.conectiva> Subject: Re: 2.4.6-pre2, pre3 VM Behavior In-Reply-To: <992460707.3b27bfa31aa98@eargle.com> Message-ID: <Pine.LNX.4.33.0106131716510.1742-100000@duckman.distro.conectiva> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: robo...@news.nic.it X-Mailing-List: linux-kernel@vger.kernel.org Approved: robo...@news.nic.it (1.20) NNTP-Posting-Host: 38179.anti-phl.bofh.it Newsgroups: linux.kernel Organization: linux.*_mail_to_news_unidirectional_gateway Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu! news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!nntp.infostrada.it!bofh.it!robomod References: <992460707.3b27bfa31aa98@eargle.com> X-Original-Cc: Linux-Kernel <linux-ker...@vger.kernel.org> X-Original-Date: Wed, 13 Jun 2001 17:21:10 -0300 (BRST) X-Original-Sender: linux-kernel-ow...@vger.kernel.org X-Original-To: Tom Sightler <tt...@tuxyturvy.com> Lines: 68 On Wed, 13 Jun 2001, Tom Sightler wrote: > 1. Transfer of the first 100-150MB is very fast (9.8MB/sec via 100Mb Ethernet, > close to wire speed). At this point Linux has yet to write the first byte to > disk. OK, this might be an exaggerated, but very little disk activity has > occured on my laptop. > > 2. Suddenly it's as if Linux says, "Damn, I've got a lot of data to flush, > maybe I should do that" then the hard drive light comes on solid for several > seconds. During this time the ftp transfer drops to about 1/5 of the original > speed. > > 3. After the initial burst of data is written things seem much more reasonable, > and data streams to the disk almost continually while the rest of the transfer > completes at near full speed again. > > Basically, it seems the kernel buffers all of the incoming file up to nearly > available memory before it begins to panic and starts flushing the file to disk. > It seems it should start to lazy write somewhat ealier. > Perhaps some of this is tuneable from userland and I just don't > know how. Actually, it already does the lazy write earlier. The page reclaim code scans up to 1/4th of the inactive_dirty pages on the first loop, where it does NOT write things to disk. On the second loop, we start asynchronous writeout of data to disk and and scan up to 1/2 of the inactive_dirty pages, trying to find clean pages to free. Only when there simply are no clean pages we resort to synchronous IO and the system will wait for pages to be cleaned. After the initial burst, the system should stabilise, starting the writeout of pages before we run low on memory. How to handle the initial burst is something I haven't figured out yet ... ;) > Anyway, things are still much better, with older kernels things > would almost seem locked up during those 10-15 seconds but now > my apps stay fairly responsive (I can still type in AbiWord, > browse in Mozilla, etc). This is due to this smarter handling of the flushing of dirty pages and due to a more subtle bug where the system ended up doing synchronous IO on too many pages, whereas now it only does synchronous IO on _1_ page per scan ;) regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/