Tech Insider					     Technology and Trends


			      USENET Archives

Newsgroups: comp.os.linux.development.system
Path: pad-thai.cam.ov.com!bloom-beacon.mit.edu!gatech!newsxfer.itd.umich.edu!
agate!hpg30a.csc.cuhk.hk!news.ust.hk!usenet
From: cc...@cs.ust.hk (Chi-Chung Hui)
Subject: Suggestion on the memory management of Linux
Message-ID: <1995Jan16.085103.17348@uxmail.ust.hk>
Keywords: Linux, Memory Management, Suggestion
Sender: use...@uxmail.ust.hk (usenet account)
Nntp-Posting-Host: cssu135.cs.ust.hk
Reply-To: cc...@cs.ust.hk
Organization: Computer Science Department, HKUST
Date: Mon, 16 Jan 1995 08:51:03 GMT
Lines: 111

I would like to share my experience on using Linux with you, and I hope I can
give some useful suggestions about its memory management.

First of all, I am running Linux v1.1.73 with 16M RAM and 8M swap space.

When I am using GCC under X (with olvwm + 2 textedits + 2 xterms + 2 tcshs),
the system begins to thrash. This does not happen when X is not running.
When I sum up the memory usage (got by 'ps -aux'), I found out that the
total memory utilization is 104%, and every process (no matter it is idle
for a long time or not) utilized certain amount of memory:
-------------------------
linux:cchui:1>ps -aux
USER       PID %CPU %MEM SIZE  RSS TTY STAT START   TIME COMMAND
...
cchui      599 13.2  3.6  379  560 pp0 S    11:46   0:00 -tcsh
cchui      610  2.0  9.6  286 1488 pp0 S    12:31   0:00 textedit
cchui      612 13.0  8.3  222 1288 pp0 S    12:31   0:00 textedit
cchui      602  0.0  1.5   80  236 pp0 R    11:46   0:00 ps -aux
root         1  0.0  1.4   48  216  ?  S   Dec 20   0:03 init auto
root         5  0.0  0.8   24  124  ?  S   Dec 20   0:00 bdflush (daemon)
root         6  0.0  0.8   24  128  ?  S   Dec 20   0:00 update (bdflush)
root        24  0.0  0.8   24  128  ?  S   Dec 20   0:00 update (bdflush)
root        38  0.0  1.5   57  232  ?  S   Dec 20   0:02 /usr/sbin/syslogd
root        40  0.0  1.2   36  200  ?  S   Dec 20   0:00 /usr/sbin/klogd
root        42  0.0  1.4   68  220  ?  S   Dec 20   0:00 /usr/sbin/inetd
root        44  0.0  1.3   64  204  ?  S   Dec 20   0:00 /usr/sbin/lpd
root        49  0.0  2.5  251  396  ?  S   Dec 20   0:00 sendmail: accepting c
root        54  0.0  1.5   88  232 v04 S   Dec 20   0:00 /sbin/getty tty4 3840
root        55  0.0  1.5   88  232 v05 S   Dec 20   0:00 /sbin/getty tty5 3840
root        56  0.0  1.5   88  232 v06 S   Dec 20   0:00 /sbin/getty tty6 3840
root       201  0.0  1.5   88  232 v02 S   Dec 20   0:00 /sbin/getty tty2 3840
root       205  0.0  1.5   88  232 v03 S   Dec 20   0:00 /sbin/getty tty3 3840
root       224  0.0  1.5   88  232 v01 S   Dec 20   0:00 /sbin/getty tty1 3840
root       598  3.4  1.7   78  264  ?  S    11:46   0:00 in.telnetd
...
-------------------------
All the processes that should be idle (e.g., sendmail and getty's) are
still occupying memories (I am running under X, the getty's must be idle.
Sendmail should be idle also since I have no network connection). These
processes should be swapped out so that an extra 11.5% of memories can
be allocated to GCC. If I do not use textedit during compilation, the
memories occupied by the two textedits can be swapped out temporarily
to free an extra 17.9% of memory. I am sure that no swapping will occur
anymore when GCC has an extra 29.4% of memory (~= 4.7Mbyte).

Unfortunatly, I found out that these processes are not swapped out and the
disk I/O is very heavy.

I would like to compare this with SunOS 4.1.3. I got the following ps output
from a SPARCstation:
-------------------------
sparc:home:15>ps -aux
USER       PID %CPU %MEM   SZ  RSS TT STAT START  TIME COMMAND
...
cchui    28792  0.0  4.9  120 1384 co S    11:44   0:00 clock -Wp 240 240 -Ws 13
cchui    29074  0.0  1.7  248  488 p1 R    12:45   0:00 ps -aux
cchui    28744  0.0 14.1 2980 3956 co S    11:44   0:29 /usr/openwin/bin/xnews :
cchui    28816  0.0  0.0  284    0 p2 IW   11:45   0:00 -bin/tcsh (tcsh)
cchui    27849  0.0  0.0  268    0 co IW   17:34   0:00 -tcsh (tcsh)
cchui    28852  0.0  0.0  220    0 p5 IW   11:45   0:00 cmdtool -Wp 0 0 -Ws 539 
cchui    28780  0.0  1.4  276  384 p1 S    11:44   0:00 -bin/tcsh (tcsh)
root     27448  0.0  0.0  356    0 ?  IW   Nov 15  6:04 automount
root     29030  0.0  0.0   36    0 ?  IW   12:37   0:00 in.telnetd
cchui    28832  0.0  0.0  192    0 co IW   11:45   0:00 textedit -Wp 549 79 -Ws 
cchui    28825  0.0  0.0  192    0 co IW   11:45   0:00 textedit -Wp 0 79 -Ws 53
csrico   28706  0.0  0.0   48    0 p7 IW   11:14   0:00 rlogin csl3su10
cchui    28760  0.0  0.9  216  252 co S    11:44   0:00 olwm -3
cchui    28779  0.0  2.3  228  644 p1 S    11:44   0:02 cmdtool -Wp 548 408 -Ws 
cchui    28739  0.0  0.0   32    0 co IW   11:44   0:00 /bin/sh /usr/openwin/bin
cchui    28772  0.0  3.5  692  992 co S    11:44   0:04 mailtool -Wp 549 79 -Ws 
cchui    28773  0.0  0.0   92    0 co IW   11:44   0:00 olwmslave
...
-------------------------
Only the activating processes are occupying memories (i.e., I am sending
email to you <mailtool>. The clocking is running continuously <clock>.
And I am running olwm <xnews and olwm>, and <ps> under <tcsh>). All the
idle processes, including the textedit's, cmdtool, and the deamons are
swapped out. The total memory utilization is only 36%, although there
are three persons logging in.

If I guess correctly, Linux uses part of the memory in storing the shared
libraries, and the remaining for the process and disk buffer. When all
the buffers are used, and *only* when there are new processes coming, the
kernel begins to thrash in order to make free memories.
And after that, every new process introduce some levels of thrashing
which degrade the system performance (response time) dramatically.

What I want to suggest is that, is it possible to introduce a machanism
to ensure that there are sufficient free memory pages at any time? There
are several mechanisms to achieve this:

1) Add a swapper (by "swapper" I mean "the process with
   pid 0", which function is to swap the idle processes to the swap space)
   to swap out the idle processes (e.g., not activate for
   more than 1 minute), and the corresponding pages are put at the end
   of the free list. However, the content of these pages are not cleared
   so that it can be reused by the idle processes immediately when they
   are activated.

2) Begin to flush the disk buffer onto the disk as long as the disk is idle.
   In this way, the disk buffer can be reused without flushing anymore.

The effect of the above mechanisms is to make sure that at least x% of
the memory are ready to be used (x should be at least 20). In this way,
we can ensure the response time at a satisfactory level.

-------
C.C.Hui
The Hong Kong University of Science and Technology

Path: nntp.gmd.de!Germany.EU.net!EU.net!nova.puug.pt!news.ist.utl.pt!
ci.ist.utl.pt!l39801
From: l39...@ci.ist.utl.pt (Highlander C H B M)
Newsgroups: comp.os.linux.development.system
Subject: Re: Suggestion on the memory management of Linux
Date: 2 Feb 1995 12:54:11 GMT
Organization: Instituto Superior Tecnico 
Lines: 46
Message-ID: <3gqklj$qb3@alfa.ist.utl.pt>
References: <1995Jan16.085103.17348@uxmail.ust.hk> <D2KLqn.29J@pe1chl.ampr.org> 
<RAJU.95Feb1182047@darbari.toi.ernet.in>
NNTP-Posting-Host: ci.ist.utl.pt
X-Newsreader: TIN [version 1.2 PL2]

Raj Mathur (r...@darbari.toi.ernet.in) wrote:
: >>>>> "Rob" == Rob Janssen <r...@pe1chl.ampr.org> writes:
: In article <D2KLqn....@pe1chl.ampr.org> r...@pe1chl.ampr.org (Rob Janssen) 
writes:

:     [snip]
:     Rob> A hint to improve things a bit: after you have booted the
:     Rob> system and started X, run a program that uses a lot of
:     Rob> memory.  It will force out the idle processes, and when it
:     Rob> terminates there is a lot of room.  The system will be much
:     Rob> faster after that.  Indeed it would be nice when this is
:     Rob> somehow done automatically, like you propose.  Sounds like
:     Rob> you have a nice project to try!

:     Rob> Rob

:     Rob> example program:
:     [snip]

: I just give the command

: 	swapoff -a ; swapon -a

: from time to time, which clears up the swap and in the process the
: buffers (BTW, this is while in X). If my luck is bad, it also kills my
: Emacs. If my luck absolutely sux, it hangs the system :-) However, on
: the whole it does beat having to reboot the system in order to free up
: RAM.

: So am I doing the right thing? What advantage (if any) would I gain by
: running your program instead of the swap{off,on} combination?

[snip]
	Why not just write a little C like :
while(1) { if ( malloc(2048) == NULL ) return(0) } 
this would hog all the space it could and then exit netly returning it .
( MHO )
--

+---------------------------------+-------------------------------------------+
|  Carlos Hugo Baptista Morgado   |                                           |
| e-mail: l39...@alfa.ist.utl.pt  |   "God is real unless declered integer"   |
|         l39...@ci.ist.utl.pt    |                                           |
|        PGP key on finger        +-------------------------------------------+
|   Instituto Superior Tecnico de Lisboa -- Universidade Tecnica de Lisboa    |
|           ( Technical Superior Institute of Lisbon - Portugal )             |
+-----------------------------------------------------------------------------+

Path: nntp.gmd.de!dearn!esoc!linuxed1!peernews.demon.co.uk!btnet!bt!pipex!
howland.reston.ans.net!
 lamarck.sura.net!hookup!swrinde!sgiblab!sgigate.sgi.com!fido.asd.sgi.com!
slovax!lm
From: l...@slovax.engr.sgi.com (Larry McVoy)
Newsgroups: comp.os.linux.development.system
Subject: Re: Suggestion on the memory management of Linux
Date: 3 Feb 1995 04:58:01 GMT
Organization: Silicon Graphics Inc., Mountain View, CA
Lines: 39
Message-ID: <3gsd4p$asb@fido.asd.sgi.com>
References: <1995Jan16.085103.17348@uxmail.ust.hk> <D2KLqn.29J@pe1chl.ampr.org> 
<RAJU.95Feb1182047@darbari.toi.ernet.in> <3gqklj$qb3@alfa.ist.utl.pt>
Reply-To: l...@slovax.engr.sgi.com
NNTP-Posting-Host: slovax.engr.sgi.com
X-Newsreader: TIN [version 1.2 PL2]

Highlander C H B M (l39...@ci.ist.utl.pt) wrote:
: 	Why not just write a little C like :
: while(1) { if ( malloc(2048) == NULL ) return(0) }
: this would hog all the space it could and then exit netly returning it .

I have added some system calls that do this in my version of Linux.
They work like so

	swapout(pid_t pid) - swapout the process completely

	shrink_buffers() - sync all buffers and free those that can be freed

They aren't ready for prime time yet, but I can give you early
results:  a 16MB machine is a far more pleasant place to work.  When I
start up and get into X things get kinda full (and I like to have the
Linux logo and xfishtank going too - things got real tight).  Top looks
like:

	Mem:  15128K av, 12732K used,  2396K free,  5648K shrd,  2904K buff
	Swap: 30036K av,  2316K used, 27720K free

After running swapout on all the processes

	Mem:  15128K av,  7048K used,  8080K free,   732K shrd,  3168K buff
	Swap: 30036K av,  9564K used, 20472K free

And after running shrink

	Mem:  15128K av,  4848K used, 10280K free,   788K shrd,   860K buff
	Swap: 30036K av,  9556K used, 20480K free

Going from 2.3MB free to 10.2MB free, yeah, I like that.  I'm using 5MB,
running X and xfishtank and top.  I can live with that.

As soon as I get these a bit polished, I'll ship them to Linus and see if
he'll put them in the base kernel.
--
---
Larry McVoy			(415) 390-1804			 l...@sgi.com

Newsgroups: comp.os.linux.development.system
From: ja...@purplet.demon.co.uk (Mike Jagdis)
Path: nntp.gmd.de!dearn!esoc!linuxed1!peernews.demon.co.uk!purplet!jaggy
Subject: Re: Suggestion on the memory management of Linux
Organization: FidoNet node 2:252/305 - The Purple Tentacle, Reading
X-Posting-Host: purplet.demon.co.uk
Date: Sat, 4 Feb 1995 00:24:00 +0000
Message-ID: <809.2F33FD61@purplet.demon.co.uk>
Sender: use...@demon.co.uk
Lines: 21

* In message <3gsd4p$...@fido.asd.sgi.com>, Larry McVoy said:

LM> I have added some system calls that do this in my version of
LM> Linux.
LM> They work like so
LM>         swapout(pid_t pid) - swapout the process completely
LM>         shrink_buffers() - sync all buffers and free those
LM> that can be freed

Hmmm... I took the approach of a bdflush() like syscall that wakes up when
pages start running low and pages things out until a high water mark is
reached again. When the swap daemon is running processes don't swap
themselves - they just hang on a semaphore and are woken up when a page
becomes available. If the swap daemon isn't running the normal eveyone
thrash at once rule applies :-).

  I haven't done a lot of tuning with this either. I was waiting for 1.3
before I do too much...

                                Mike

			        About USENET

USENET (Users’ Network) was a bulletin board shared among many computer
systems around the world. USENET was a logical network, sitting on top
of several physical networks, among them UUCP, BLICN, BERKNET, X.25, and
the ARPANET. Sites on USENET included many universities, private companies
and research organizations. See USENET Archives.

		       SCO Files Lawsuit Against IBM

March 7, 2003 - The SCO Group filed legal action against IBM in the State 
Court of Utah for trade secrets misappropriation, tortious interference, 
unfair competition and breach of contract. The complaint alleges that IBM 
made concentrated efforts to improperly destroy the economic value of 
UNIX, particularly UNIX on Intel, to benefit IBM's Linux services 
business. See SCO vs IBM.

The materials and information included in this website may only be used
for purposes such as criticism, review, private study, scholarship, or
research.

Electronic mail:			       WorldWideWeb:
   tech-insider@outlook.com			  http://tech-insider.org/