Path: gmdzi!unido!mcsun!uunet!ogicse!ucsd!ucbvax!koko.UUCP!shoshana
From: shosh...@koko.UUCP (Shoshana Abrass)
Newsgroups: comp.sys.sgi
Subject: Swap questions
Message-ID: <9011150232.AA00704@koko.pdi.com>
Date: 15 Nov 90 02:32:04 GMT
Sender: dae...@ucbvax.BERKELEY.EDU
Organization: The Internet
Lines: 35
Posted: Thu Nov 15 03:32:04 1990
I've been playing around with osview, trying to determine the optimal
swap space for a PI with 48 to 64 Mbytes of memory. I noticed a few
oddities, and I'm hoping someone at SGI can explain them.
First, the reason we have so much memory is that we manipulate large
image files (1-16 Mb), several at a time. But even on a machine that's
manipulating 16 Mb files, swap activity seems very low. "swap -l"
claims that only 5 Mb of swap are being used. "osview -i1" shows lots
of page faults, but hardly any swapping activity. So my questions are:
1. Are the image files ever paged out to the swap area? it seems not.
It seems like the executable code is the only thing getting paged
to virtual memory.
2. Are the image files 'swapin'ed (according to osview) when they're
first read into memory? What's the difference between the "Virtual
Memory" and "Swap" in osview?
In a kernel class I took, the instructor claimed that whenever a process
was read into memory, swap space was reserved/allocated for it, before
it ever got paged out. Is this true under Irix? (is it ever true, for
that matter). If this is true, it implies that one must have at least
as much swap as real memory... even though "swap -l" claims that no
swap is being used. Also, if it is true, is it only true for the
executable code, and not the data?
What's the real scoop here? Any help much appreciated.
-shoshana
pdi!shosh...@sgi.com
================== Disclaimer necessitated by mailpath: ==================
I don't work for sgi, I just work downstream.
==========================================================================
Path: gmdzi!unido!mcsun!uunet!munnari.oz.au!uhccux!ames!decwrl!sgi!
shinobu!odin!patton.wpd.sgi.com!jmb
From: j...@patton.wpd.sgi.com (Doctor Software)
Newsgroups: comp.sys.sgi
Subject: Re: Swap questions
Message-ID: <1990Nov16.155730.2670@odin.corp.sgi.com>
Date: 16 Nov 90 15:57:30 GMT
References: <9011150232.AA00704@koko.pdi.com>
Sender: n...@odin.corp.sgi.com (Net News)
Reply-To: j...@patton.wpd.sgi.com (Doctor Software)
Organization: Silicon Graphics Inc.
Lines: 75
Posted: Fri Nov 16 16:57:30 1990
In article <9011150232.AA00...@koko.pdi.com>, shosh...@koko.UUCP
(Shoshana Abrass) writes:
>
> 1. Are the image files ever paged out to the swap area? it seems not.
> It seems like the executable code is the only thing getting paged
> to virtual memory.
This is mostly correct. Only pieces of executable programs get swapped
out, whether they are text or data. It would make little sense to swap a
data file - it's already on the disk anyway!
Files are managed through the system "buffer cache", which is an area of
memory used to cache file pages. In IRIX 3.3 and later, the system
dynamically sizes the buffer cache, and will attempt to keep a balance
between executable code and file data in memory to minimize disk traffic
and maximize performance.
>
> 2. Are the image files 'swapin'ed (according to osview) when they're
> first read into memory? What's the difference between the "Virtual
> Memory" and "Swap" in osview?
Image file traffic can be examined in the buffer traffic section of
osview, or the "bdev" bar of gr_osview. Also, the real memory section of
osview or the "rmem" and "rmemc" bars of gr_osview show memory usage,
and what is assigned to the buffer cache versus executable code, etc.
The man page for gr_osview describes what all this stuff means in quite
a bit of detail.
In essence, your file is only read into memory as you access it, not all
at once, so to see the traffic effect look at the block device (buffer
cache) information. Virtual memory is paging activity on program text
and data. Paging activity does not necessarily imply disk activity! Page
faults occur for copy-on-write, modifications, zero-on-fill, double TLB
faults, etc. Swap only occurs when, as the "last resort", the system
must push some pages out to disk to make room for other pages. A swapin
only occurs as a "last resort" if a user accesses a page that only
resides on the disk. Going to the disk is expensive - the system avoids
it whenever possible. Again, the man page for gr_osview has a lot more
information on all this stuff.
> In a kernel class I took, the instructor claimed that whenever a process
> was read into memory, swap space was reserved/allocated for it, before
> it ever got paged out. Is this true under Irix? (is it ever true, for
> that matter). If this is true, it implies that one must have at least
> as much swap as real memory... even though "swap -l" claims that no
> swap is being used. Also, if it is true, is it only true for the
> executable code, and not the data?
Swap space is allocated (it's cheap - just a counter) whenever a new
page of program memory is created. This means that when a program is
started, the text, initial stack and data segments have backing store
allocated in swap. This avoids deadlock of the system. Sar, osview and
gr_osview only report the number of swap pages that actually have an
active copy of a memory page on them - this is different than just
making sure that pages will be available in case swapping is necessary.
To imagine the deadlock, consider what would happen if I started a
program without making sure that swap space was available. I could
already have filled up my swap space with other program data. Then, the
new program allocates a new page of data, and I run out of memory. I
need to swap something to get memory for the new page, but swap space is
full so I can't swap, so ... deadlock.
>
> What's the real scoop here? Any help much appreciated.
I think all your questions have been answered!
> -shoshana
> pdi!shosh...@sgi.com
-- Jim Barton
Silicon Graphics Computer Systems
j...@sgi.com
Path: gmdzi!unido!mcsun!sunic!uupsi!rpi!zaphod.mps.ohio-state.edu!
think.com!yale!umich!umeecs!msi.umn.edu!cs.umn.edu!poincare.geom.umn.edu!slevy
From: sl...@poincare.geom.umn.edu (Stuart Levy)
Newsgroups: comp.sys.sgi
Subject: Re: Swap questions
Message-ID: <1990Nov17.225159.13857@cs.umn.edu>
Date: 17 Nov 90 22:51:59 GMT
References: <9011150232.AA00704@koko.pdi.com> <1990Nov16.155730.2670@odin.corp.sgi.com>
Sender: n...@cs.umn.edu (News administrator)
Organization: Geometry Group, University of Minnesota
Lines: 30
Posted: Sat Nov 17 23:51:59 1990
Nntp-Posting-Host: poincare.geom.umn.edu
In article <1990Nov16.155730.2...@odin.corp.sgi.com> j...@patton.wpd.sgi.com
(Doctor Software) writes:
>In article <9011150232.AA00...@koko.pdi.com>, shosh...@koko.UUCP
>(Shoshana Abrass) writes:
>> ...In a kernel class I took, the instructor claimed that whenever a process
>> was read into memory, swap space was reserved/allocated for it, before
>> it ever got paged out. Is this true under Irix? (is it ever true, for
>> that matter). If this is true, it implies that one must have at least
>> as much swap as real memory...
>
>Swap space is allocated (it's cheap - just a counter) whenever a new
>page of program memory is created. This means that when a program is
>started, the text, initial stack and data segments have backing store
>allocated in swap. ...
I'm still puzzled. Doesn't that mean that the answer to Shoshana's question
is "yes" -- if you have more main memory than disk-based swap space,
you could never use all the main memory for user programs, because a disk-based
swap page would need to be allocated for each user page?
I know other UNIX implementations behave this way but thought SGI
was among those who had fixed this. The answer actually affects us -- our main
Iris has 64MB RAM but only about 50 MB swap area, yet we certainly seem to be
able to get > 50MB and in fact > 64MB virtual space allocated to running
processes (even accounting for sharable stuff). I haven't checked that we
can reach 50 + 64 - size of kernel data before the kernel starts killing
processes, but it seems plausible.
So like Shoshana I ask, what *is* the real scoop here?
Stuart Levy, Geometry Group, University of Minnesota
sl...@geom.umn.edu
Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!
sdd.hp.com!ucsd!ames!sgi!shinobu!odin!patton.wpd.sgi.com!jmb
From: j...@patton.wpd.sgi.com (Doctor Software)
Newsgroups: comp.sys.sgi
Subject: Re: Swap questions
Message-ID: <1990Nov28.163415.14317@odin.corp.sgi.com>
Date: 28 Nov 90 16:34:15 GMT
References: <9011150232.AA00704@koko.pdi.com>
<1990Nov16.155730.2670@odin.corp.sgi.com> <1990Nov17.225159.13857@cs.umn.edu>
Sender: n...@odin.corp.sgi.com (Net News)
Reply-To: j...@patton.wpd.sgi.com (Doctor Software)
Organization: Silicon Graphics Inc.
Lines: 18
Apparently, I was wrong. I've been perusing the code for 3.3, and if I can
still read C right, your total virtual memory is (real memory) + (swap
memory). If swap is too small, and a process has pages which need to be
swapped, the OS kills the process in order to avoid deadlock. This only
happens in the case that memory is filled, the OS needs to page
something out to keep going, and there are no pages of swap left free.
This seems to make the lesson be that you can run quite fine without
swap until you actually need to use it - but then watch out. There's no
garuntee of which process will actually be killed in this case, its
just whomever has the page that can't go out.
-- jmb
-- Jim Barton
Silicon Graphics Computer Systems
j...@sgi.com