From: v...@math.psu.edu (Alexander Viro)
Subject: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <Pine.GSO.4.10.10007130033490.14674-100000@weyl.math.psu.edu>#1/1
X-Deja-AN: 645773725
Sender: owner-linux-ker...@vger.rutgers.edu
X-Authentication-Warning: weyl.math.psu.edu: viro owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
OK, folks - current bunch of changes looks rather interesting from
the scalability point of view, but it means changes in 3rd-party code.
Doing it during the 2.4 is obviously not an option and postponing to
2.5 looks like an overkill since changes are massive but very simple.
What had been changed recently:
* ->mmap() in file_operations, ->revalidate(), ->readlink() and
->follow_link() in inode_operations and all methods in dentry_operations
are called without the BKL. Take yourself if you need it.
* some instances of ->open() are called without BKL. In near future all of
them will. Again, if you need it - you get it.
* Documentation on locking warranties provided by VFS added. See
Documentation/filesystems/Locking. And watch the changes - this sucker is
supposed to be kept in sync with the kernel changes.
What is going to happen:
* ->release() will be called without BKL. You need it - you take it.
I've added lock_kernel()/unlock_kernel() in all dubious instances of
->release(). In many cases removal will be trivial (switching a variable
or two from int to atomic_t, etc.), so when it will go into the tree -
check your drivers. Folks with 3rd-party code will have to add BKL
grabbing themselves - sorry, I'm not clairvoyant. Patch submitted to
Linus.
* Probably inode methods for directories are going to be called without
BKL. Again, I'll do the work for in-kernel filesystems, but 3rd-party ones
will have to be fixed by their maintainers.
* Probably ->fasync() is going to be called without BKL - in most cases
BKL is not be needed at all.
BTW, while you are at it - check your ->read() and ->write(). In many
drivers they do not look SMP-safe, AFAICS.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: a...@lxorguk.ukuu.org.uk (Alan Cox)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <E13CdZ2-0002xY-00@the-village.bc.nu>#1/1
X-Deja-AN: 645807616
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <Pine.GSO.4.10.10007130033490.14674-100000@weyl.math.psu.edu>
Content-Type: text/plain; charset=us-ascii
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
> * ->mmap() in file_operations, ->revalidate(), ->readlink() and
> ->follow_link() in inode_operations and all methods in dentry_operations
> are called without the BKL. Take yourself if you need it.
Is this true for drivers. If so the sound stuff will probably need work. Work
Im not doing.
> * ->release() will be called without BKL. You need it - you take it.
You want it for 2.4 you fix the drivers.
This continual VFS redesign creep is going to far now. We'lll never get a
2.4 if we keep moving all the locking around.
Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: a...@lxorguk.ukuu.org.uk (Alan Cox)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <E13Cm7x-0003tG-00@the-village.bc.nu>#1/1
X-Deja-AN: 645970579
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <396FE6C4.A0168052@reiser.to>
Content-Type: text/plain; charset=us-ascii
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
> I hope you understand why I think that adding reiserfs is a small disturbance
> compared to continuing revisions of VFS.
This kind of lock shifting is a major upheaval. It invalidates any device
driver testing done in the past weeks when we have been slowly moving towards
more stability.
Im just hoping Linus refuses the changes
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: torva...@transmeta.com (Linus Torvalds)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <Pine.LNX.4.10.10007131011100.1239-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 645980683
Sender: owner-linux-ker...@vger.rutgers.edu
References: <E13Cm7x-0003tG-00@the-village.bc.nu>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Alan Cox wrote:
>
> This kind of lock shifting is a major upheaval. It invalidates any device
> driver testing done in the past weeks when we have been slowly moving towards
> more stability.
I much rather do it now than later. Later means either: (a) during 2.4.x
or (b) early in 2.5.x.
(a) is obviously impossiblme. 'Nuff said.
(b) means that we'll have these subtle incomptibilities between the
release kernel and the development kernel, and backporting of fixes etc
suddenly becomes much harder.
The (b) kind of inconsistency is unavoidable, but I'd rather avoid it for
the "mindless" kind of lock movement that affects a lot of drivers and
filesystems, yet is basically very simple ("mindless" != "easy").
Right now all the critical file_operations are without the kernel lock.
There's still stuff like "llseek" etc, but they're not used that much and
few filesystems/drivers need their own versions, so I think we can ignore
the ones that are left. We've reached the stage where the whole
open->read->write->close
sequence basically works without the kernel lock, and that is important.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: rgo...@ras.ucalgary.ca (Richard Gooch)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <200007131818.e6DIIEh07166@vindaloo.ras.ucalgary.ca>#1/1
X-Deja-AN: 646009095
Sender: owner-linux-ker...@vger.rutgers.edu
References: <E13Cm7x-0003tG-00@the-village.bc.nu>
<Pine.LNX.4.10.10007131011100.1239-100000@penguin.transmeta.com>
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Linus Torvalds writes:
>
>
> On Thu, 13 Jul 2000, Alan Cox wrote:
> >
> > This kind of lock shifting is a major upheaval. It invalidates any device
> > driver testing done in the past weeks when we have been slowly moving towards
> > more stability.
>
> I much rather do it now than later. Later means either: (a) during 2.4.x
> or (b) early in 2.5.x.
>
> (a) is obviously impossiblme. 'Nuff said.
>
> (b) means that we'll have these subtle incomptibilities between the
> release kernel and the development kernel, and backporting of fixes
> etc suddenly becomes much harder.
But at some point you need to cut a new kernel version and live with
it.
> The (b) kind of inconsistency is unavoidable, but I'd rather avoid
> it for the "mindless" kind of lock movement that affects a lot of
> drivers and filesystems, yet is basically very simple ("mindless" !=
> "easy").
Except it will cost us time and stability. Where do you draw the line?
Or even better: when will 2.4 be really, really, *really* released?
Regards,
Richard....
Permanent: rgo...@atnf.csiro.au
Current: rgo...@ras.ucalgary.ca
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: torva...@transmeta.com (Linus Torvalds)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <Pine.LNX.4.10.10007131211150.1554-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 646034543
Sender: owner-linux-ker...@vger.rutgers.edu
References: <200007131818.e6DIIEh07166@vindaloo.ras.ucalgary.ca>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Richard Gooch wrote:
> >
> > (b) means that we'll have these subtle incomptibilities between the
> > release kernel and the development kernel, and backporting of fixes
> > etc suddenly becomes much harder.
>
> But at some point you need to cut a new kernel version and live with
> it.
Yes.
The thing I hate, though, is code like
#if LINUX_VERSION_CODE >= VERSION(2,5,10)
lock_kernel();
#endif
in drivers. _Particularly_ in drivers. We'll end up living with 2.4.x for
two years or more, judging by past performance, and we'll have especially
driver writers that concentrate on the 2.4.x stuff for a long time.
Having driver interface inconsistencies like that is nasty, and the ones
we _know_ that we'll have should be minimized.
This is another reason why the open/read/write/close series is
particularly important to get done first: it's the stuff every driver
tends to do. Things like "fasync" is less critical because even if it is
used a lot by drivers, it tends to use the helper routines (so drivers
often do not need to worry over-much about locking).
This shoul dhave been the last locking issue, though. We scale too well
for words.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: a...@lxorguk.ukuu.org.uk (Alan Cox)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <E13CpqF-0004Lc-00@the-village.bc.nu>#1/1
X-Deja-AN: 646059732
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <396FE6C4.A0168052@reiser.to>
Content-Type: text/plain; charset=us-ascii
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
> freeze will be in three months, and VFS/VM code freeze will be in 6 weeks.
> Sorry, I'd like to see 2.4 ship in three months, but I am hearing people guess
> that it won't ship this year, and if they are right maybe we will be best off
> admitting it.
6 weeks to fix the VM is too short. But then I think we may have to accept
that we dont get the VM perfect until 2.5. We just need to get it back working
as well as 2.0 did
Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: jmer...@timpanogas.com (Jeff V. Merkey)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <396E2E4B.4EA7E5E9@timpanogas.com>#1/1
X-Deja-AN: 646066441
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <E13CpqF-0004Lc-00@the-village.bc.nu>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
Organization: TRG, Inc.
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Alan Cox wrote:
>
> > freeze will be in three months, and VFS/VM code freeze will be in 6 weeks.
> > Sorry, I'd like to see 2.4 ship in three months, but I am hearing people guess
> > that it won't ship this year, and if they are right maybe we will be best off
> > admitting it.
>
> 6 weeks to fix the VM is too short. But then I think we may have to accept
> that we dont get the VM perfect until 2.5. We just need to get it back working
> as well as 2.0 did
>
> Alan
>
Alan,
I think the soak time for 2.4 will probably continue past this since
there's still tons of stuff still pending (including me getting my code
tuned). I've been working on auto-repair from the driver so the FS can
auto repair itself just like NetWare does when it mounts volumes. The
biggest issue is that 2.4 still needs Linus and you to get everyone out
of the code for a couple of months, and you guys do some performance
tuning work without it chaning beneath you. 6 weeks is very agressive.
I think you could make this date, but the performance work still will
need to get done, and I don't see how with all the stuff going on how
you guys will get this done in 6 weeks. To date 2.4 is still noticeably
slower than the 2.2 tree in some areas and overall, at least from what
we are seeing, but this is just IMHO.
:-)
Jeff
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majord...@vger.rutgers.edu
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: rgo...@ras.ucalgary.ca (Richard Gooch)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <200007132108.e6DL8ZL08547@vindaloo.ras.ucalgary.ca>#1/1
X-Deja-AN: 646067364
Sender: owner-linux-ker...@vger.rutgers.edu
References: <200007131818.e6DIIEh07166@vindaloo.ras.ucalgary.ca>
<Pine.LNX.4.10.10007131211150.1554-100000@penguin.transmeta.com>
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Linus Torvalds writes:
>
>
> On Thu, 13 Jul 2000, Richard Gooch wrote:
> > >
> > > (b) means that we'll have these subtle incomptibilities between the
> > > release kernel and the development kernel, and backporting of fixes
> > > etc suddenly becomes much harder.
> >
> > But at some point you need to cut a new kernel version and live with
> > it.
>
> Yes.
>
> The thing I hate, though, is code like
>
> #if LINUX_VERSION_CODE >= VERSION(2,5,10)
> lock_kernel();
> #endif
>
> in drivers. _Particularly_ in drivers. We'll end up living with
> 2.4.x for two years or more, judging by past performance, and we'll
> have especially driver writers that concentrate on the 2.4.x stuff
> for a long time.
Perhaps the solution is to have shorter development cycles. Say 6
months before feature freeze, then a planned 3 month shakedown
(extended only if there are reliability problems). Part of the problem
is that so many new things get piled into each development series. Cut
back on the number of significant things per cycle (say just one or
two).
> Having driver interface inconsistencies like that is nasty, and the
> ones we _know_ that we'll have should be minimized.
Yeah, but things will always be changing, so we have to live with
it. Otherwise we can end up waiting for the perfect kernel before it
gets released. Next we'll be investing in an organ.
Regards,
Richard....
Permanent: rgo...@atnf.csiro.au
Current: rgo...@ras.ucalgary.ca
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: torva...@transmeta.com (Linus Torvalds)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <Pine.LNX.4.10.10007131410140.3044-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 646070684
Sender: owner-linux-ker...@vger.rutgers.edu
References: <200007132108.e6DL8ZL08547@vindaloo.ras.ucalgary.ca>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Richard Gooch wrote:
>
> Perhaps the solution is to have shorter development cycles. Say 6
> months before feature freeze, then a planned 3 month shakedown
> (extended only if there are reliability problems). Part of the problem
> is that so many new things get piled into each development series. Cut
> back on the number of significant things per cycle (say just one or
> two).
I agree.
This was what I wanted for the 2.2 -> 2.4 change too, and a much shorter
development cycle would be wonderful. It obviously was not to be: 2.4 is
already about 18 months out. We're not as bad as the 2.0 -> 2.2 cycle was,
but it's painful.
It seems to be a lot harder than it should be to keep short development
releases. I certainly haven't found the magic combination yet. 2.4.x is
definitely all I hoped for (the big goal for me was to make sure that the
mindcraft-like web-performance scalability problems would be gone, and we
definitely fixed that), but it took much too long.
What would help would be more modular releases: smaller pieces of the
kernel getting released independently, allowing for smaller and shorter
release cycles. In Linux at least we don't have the "whole world" release
issue that most other OS people have (including the free ones: I think the
BSD "world" approach is horrible partly because of the release issues),
but even just the kernel is so big that it would be nice to be able to see
it as multiple independent projects.
At the same time it's obviously not true that there are independent
projects, and especially drivers (which _sound_ independent) are very
likely to be impacted quite a lot by infrastructure changes. There are no
really clear lines to split development up by, and the cures are worse
than the disease ("microkernels", I hear somebody shout. But that approach
would just make it impossible to do the kinds of improvements we _have_
done and probably will continue to do).
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: a...@lxorguk.ukuu.org.uk (Alan Cox)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <E13CqEp-0004Pp-00@the-village.bc.nu>#1/1
X-Deja-AN: 646074505
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <Pine.LNX.4.21.0007131808000.23729-100000@duckman.distro.conectiva>
Content-Type: text/plain; charset=us-ascii
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
> Since VM is basically a performance issue, I don't think
> we should let it hold up 2.4 release. Sure there are some
> minor stability issues in extreme cases, but no more than
> with 2.2 VM.
Except we are _still_ fixing the 2.2 VM and paying a stability penalty
along the way for it. That is -bad-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: r...@conectiva.com.br (Rik van Riel)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <Pine.LNX.4.21.0007131808000.23729-100000@duckman.distro.conectiva>#1/1
X-Deja-AN: 646075119
Sender: owner-linux-ker...@vger.rutgers.edu
References: <E13CpqF-0004Lc-00@the-village.bc.nu>
X-Sender: r...@duckman.distro.conectiva
X-Authentication-Warning: duckman.distro.conectiva: riel owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Alan Cox wrote:
> > freeze will be in three months, and VFS/VM code freeze will be in 6 weeks.
> > Sorry, I'd like to see 2.4 ship in three months, but I am hearing people guess
> > that it won't ship this year, and if they are right maybe we will be best off
> > admitting it.
>
> 6 weeks to fix the VM is too short. But then I think we may have
> to accept that we dont get the VM perfect until 2.5. We just
> need to get it back working as well as 2.0 did
The new VM subsystem will be available as a series of
"new VM sneak preview" patches. I'll make sure to give
you one which is stable and works well enough that we
can integrate it into 2.4 later on (eg at 2.4.10).
Since VM is basically a performance issue, I don't think
we should let it hold up 2.4 release. Sure there are some
minor stability issues in extreme cases, but no more than
with 2.2 VM.
regards,
Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000
http://www.conectiva.com/ http://www.surriel.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: jmer...@timpanogas.com (Jeff V. Merkey)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <396E362E.8B2B9DF0@timpanogas.com>#1/1
X-Deja-AN: 646077707
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <Pine.LNX.4.10.10007131410140.3044-100000@penguin.transmeta.com>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
Organization: TRG, Inc.
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Linus Torvalds wrote:
>
>
>
> What would help would be more modular releases: smaller pieces of the
> kernel getting released independently, allowing for smaller and shorter
> release cycles. In Linux at least we don't have the "whole world" release
> issue that most other OS people have (including the free ones: I think the
> BSD "world" approach is horrible partly because of the release issues),
> but even just the kernel is so big that it would be nice to be able to see
> it as multiple independent projects.
>
> At the same time it's obviously not true that there are independent
> projects, and especially drivers (which _sound_ independent) are very
> likely to be impacted quite a lot by infrastructure changes. There are no
> really clear lines to split development up by, and the cures are worse
> than the disease ("microkernels", I hear somebody shout. But that approach
> would just make it impossible to do the kinds of improvements we _have_
> done and probably will continue to do).
>
> Linus
>
Breaking the device drivers, protocol stacks, disk drivers, file
systems, etc. out of the kernel proper and forcing them to always be
loadable modules would achieve this end with the added benefit of
allowing folks to plug in their own optimized binary versions -- Like
NetWare and NT are today. The only nasty place I can think of where
this won't work well is the incestuous hooking of block_read/block_write
functions which seens prevalant in the drivers to always force drivers
into the buffer cache. Having an alternate debugger interface in the
kernel that's loadable would also be really nice since it would allow me
to plug in the MANOS kernel debugger into the kernel proper (and speed
up our development by a factor of 20) rather than forcing us to use code
reviews and that putrid hacked up gdb variant to debug the kernel (which
is useless for SMP drivers).
Jeff
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majord...@vger.rutgers.edu
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: a...@lxorguk.ukuu.org.uk (Alan Cox)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <E13CqiC-0004Uo-00@the-village.bc.nu>#1/1
X-Deja-AN: 646080477
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <396E362E.8B2B9DF0@timpanogas.com>
Content-Type: text/plain; charset=us-ascii
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
> Breaking the device drivers, protocol stacks, disk drivers, file
> systems, etc. out of the kernel proper and forcing them to always be
> loadable modules would achieve this end with the added benefit of
> allowing folks to plug in their own optimized binary versions -- Like
It breaks if you distribute them seperately. We have this problem in the way
the model works. It suprised me until I understood what is going on
I claim:
Free software development is about minimising information exchange and knowledge
of external modules.
The practical upshot of this is we generate a fair amount of duplicated code.
Worse because of the 'minimising information exchange' aspect most developers
do not aggressively track the modules they initially took their skeleton code
from - thus we get bug replication.
One thing I have to do with many bug fixes is to grep the tree to see who
else copied the bug. That needs a single tree.
Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: torva...@transmeta.com (Linus Torvalds)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <Pine.LNX.4.10.10007131441410.4354-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 646081644
Sender: owner-linux-ker...@vger.rutgers.edu
References: <396E362E.8B2B9DF0@timpanogas.com>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Jeff V. Merkey wrote:
> >
>
> Breaking the device drivers, protocol stacks, disk drivers, file
> systems, etc. out of the kernel proper and forcing them to always be
> loadable modules would achieve this end with the added benefit of
> allowing folks to plug in their own optimized binary versions -- Like
> NetWare and NT are today.
No, it would be a complete disaster.
You'd get the same kind of disaster that all binary-only modules have: you
cannot change fundamental interfaces.
In another five or ten years, we may be at the point where the fundamental
interfaces _really_ don't change, and that we've handled all the
scalability issues and that we have no need to add new interfaces. At THAT
point we can just say "ok, drivers are truly independent".
It's not true today.
Basically, the thing that allows 2.4.x to scale as well as it does (and it
does really well: look at the current SpecWeb99 world record numbers, and
compare it to the also-ran second place), is exactly because we had all
the source in one place, and we _could_ make fundamental changes. Claiming
anything else is silly - if we had broken-up device drivers, we'd have
been up shit creek without a paddle. End of story.
This is the thing that people don't understand. In theory it is wonderful
to have modularization. It's the best thing on earth. But if that
modularization means that you can't fix the module interfaces, then you're
going to remain broken for all time.
This is why I rather fix module interfaces early and often. Make sure that
we (a) have good interfaces that matches what the different parts of the
kernel want to have and (b) make people used to the fact that a driver or
a filesystem is not a static thing, and keep them aware of the fact that
it depends on the kernel underneath it.
We're certainly getting closer to a good interface in many areas. The
current VFS interfaces, for example, are pretty good - although many of
the less important ones still depend on the kernel lock etc. But we're
_not_ at the stage yet where we could just say "ok, a driver is a driver,
and we don't need to worry about it".
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: jmer...@timpanogas.com (Jeff V. Merkey)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <396E3BCF.1C850FA5@timpanogas.com>#1/1
X-Deja-AN: 646086297
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <Pine.LNX.4.10.10007131441410.4354-100000@penguin.transmeta.com>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
Organization: TRG, Inc.
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Linus Torvalds wrote:
>
> On Thu, 13 Jul 2000, Jeff V. Merkey wrote:
> > >
> >
> > Breaking the device drivers, protocol stacks, disk drivers, file
> > systems, etc. out of the kernel proper and forcing them to always be
> > loadable modules would achieve this end with the added benefit of
> > allowing folks to plug in their own optimized binary versions -- Like
> > NetWare and NT are today.
>
> No, it would be a complete disaster.
>
> You'd get the same kind of disaster that all binary-only modules have: you
> cannot change fundamental interfaces.
>
> In another five or ten years, we may be at the point where the fundamental
> interfaces _really_ don't change, and that we've handled all the
> scalability issues and that we have no need to add new interfaces. At THAT
> point we can just say "ok, drivers are truly independent".
>
> It's not true today.
>
> Basically, the thing that allows 2.4.x to scale as well as it does (and it
> does really well: look at the current SpecWeb99 world record numbers, and
> compare it to the also-ran second place), is exactly because we had all
> the source in one place, and we _could_ make fundamental changes. Claiming
> anything else is silly - if we had broken-up device drivers, we'd have
> been up shit creek without a paddle. End of story.
>
> This is the thing that people don't understand. In theory it is wonderful
> to have modularization. It's the best thing on earth. But if that
> modularization means that you can't fix the module interfaces, then you're
> going to remain broken for all time.
>
> This is why I rather fix module interfaces early and often. Make sure that
> we (a) have good interfaces that matches what the different parts of the
> kernel want to have and (b) make people used to the fact that a driver or
> a filesystem is not a static thing, and keep them aware of the fact that
> it depends on the kernel underneath it.
>
> We're certainly getting closer to a good interface in many areas. The
> current VFS interfaces, for example, are pretty good - although many of
> the less important ones still depend on the kernel lock etc. But we're
> _not_ at the stage yet where we could just say "ok, a driver is a driver,
> and we don't need to worry about it".
>
> Linus
Linus,
We already have versioning with modver to enforce version mismatches.
This sounds like a control and freedom issue relative to being able to
efficiently make changes, which I agree is very important. Novell
requires all driver writers to provide source code for driver
certification for the very reasons you cite. I think this could be
made a policy issue since it relates more to the human element than any
technical arguments (other than the freedom to fixed busted and shitty
drivers, and BTW I agree completely with your position here).
If folks were always required by licensing to post the code for binary
drivers they want you to consider for inclusion in Linux (the Linus
Torvalds driver certifiation program), I think this could be easily
dealt with. Also, having an NLM-like "kernel only" interface much like
modutils provides that would allow you to segregate specific code pieces
for Linux kernel mode only would allow you the freedom to control these
issues since you could easily create a special binary interface just for
this stuff -- and you'd be the one making the rules the rest of us live
by.
Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: torva...@transmeta.com (Linus Torvalds)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <Pine.LNX.4.10.10007131501550.4354-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 646093621
Sender: owner-linux-ker...@vger.rutgers.edu
References: <396E3BCF.1C850FA5@timpanogas.com>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Jeff V. Merkey wrote:
>
> We already have versioning with modver to enforce version mismatches.
It's not about version mismatches.
It's about getting at all the drivers easily, so that one person (or a
small team) can do a change that updates the interfaces for _everything_
in one fell swoop.
Which is actually how most of these things get done.
This is how people like Jeff Garzik can update the PCI initialization and
finding code for hundreds of drivers. This is how Al Viro can get at the
locking rules, and just update everything he can get his grubby hands on
in one go. This is, in short, how we can make big changes without having
to synchronize with every single maintainer of every single piece of the
system..
Version mismatches protect against mis-use, at least to some degree, and
at least of the accidental kind. But they don't allow for updating of
external drivers.
Note, for example, how ReiserFS has been mentioned a number of times in
the last week on Linux-kernel. That's because reiserfs has reached the
point where it's used by a lot of people, but it hasn't reached the point
where it has gotten integrated yet, so while the other filesystems were
updated to new VFS layer changes, ReiserFS missed out. So did the other
external ones, but they tend to have a more controlled user base so far:
ReiserFS has apparently already gotten "out".
Don't get me wrong. To some degree I would _love_ to not have a large
kernel archive. It's big, and it makes releases harder. No question about
that. But the monolithic approach definitely has a lot of advantages.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: jmer...@timpanogas.com (Jeff V. Merkey)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <396E4100.F7F954F4@timpanogas.com>#1/1
X-Deja-AN: 646095489
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <Pine.LNX.4.10.10007131501550.4354-100000@penguin.transmeta.com>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
Organization: TRG, Inc.
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Linus Torvalds wrote:
>
> On Thu, 13 Jul 2000, Jeff V. Merkey wrote:
> >
> > We already have versioning with modver to enforce version mismatches.
>
> It's not about version mismatches.
>
> It's about getting at all the drivers easily, so that one person (or a
> small team) can do a change that updates the interfaces for _everything_
> in one fell swoop.
>
> Which is actually how most of these things get done.
>
> This is how people like Jeff Garzik can update the PCI initialization and
> finding code for hundreds of drivers. This is how Al Viro can get at the
> locking rules, and just update everything he can get his grubby hands on
> in one go. This is, in short, how we can make big changes without having
> to synchronize with every single maintainer of every single piece of the
> system..
>
> Version mismatches protect against mis-use, at least to some degree, and
> at least of the accidental kind. But they don't allow for updating of
> external drivers.
>
> Note, for example, how ReiserFS has been mentioned a number of times in
> the last week on Linux-kernel. That's because reiserfs has reached the
> point where it's used by a lot of people, but it hasn't reached the point
> where it has gotten integrated yet, so while the other filesystems were
> updated to new VFS layer changes, ReiserFS missed out. So did the other
> external ones, but they tend to have a more controlled user base so far:
> ReiserFS has apparently already gotten "out".
>
> Don't get me wrong. To some degree I would _love_ to not have a large
> kernel archive. It's big, and it makes releases harder. No question about
> that. But the monolithic approach definitely has a lot of advantages.
>
> Linus
Linus,
I'm posting MANOS today -- there's an NT flavor DLL loader in the
sources if you want to look at it that's more like Microsoft's than the
one in Linux. NT uses something called a forwarder chain in the DLL
loader to handle this problem for dynanmically linked modules, though I
agree that the way they do it is an absolute hack. They allow the
loader to detect changed APIs and insert a "forwarder" function when the
external reference is resolved by the DLL loader -- this function
evenually calls the real function in kernel but if it's added a
parameter or something, it lets you slip in a forwarder function to
fixup the stack and munge the parms if someone altered an interface.
Something like this would allow us to go modular, and give you the
flexibility you want, but I agree it's still going to be a f_cking
nightmare to get this thing coordinated properly. You may want to
consider diverging after 2.4 and create a "modular" linux tree and see
how many of us bite on it (I certainly would).
:-)
Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: torva...@transmeta.com (Linus Torvalds)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <Pine.LNX.4.10.10007131450370.4354-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 646155421
Sender: owner-linux-ker...@vger.rutgers.edu
References: <E13CqiC-0004Uo-00@the-village.bc.nu>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Alan Cox wrote:
>
> I claim:
> Free software development is about minimising information exchange and knowledge
> of external modules.
Indeed.
Note that when it comes to the ->release() BKL semantics, what people
don't apparently don't appreciate is the fact that Al Viro actually
grepped for all release functions, and updated all the ones in the
standard kernel. His announcement and warning was about _external_
modules, exactly because external modules have this problem where they
cannot be updated automatically when something changes.
This is why a central kernel repository is so nice: most of the time when
something changes, we can fix everything in one go, and people don't have
to be all that aware of the changes. It's not always true: some of the VFS
changes (namely the page cache write-through etc) were _so_ intrusive that
it was hard to make the fix-ups available, and as a result a number of
filesystems were left in a broken state.
And quite often the "grep for places to change" approach misses a few
(this, btw, is why I've grown to love the new syntax for structure
initializers: it's a h*ll of a lot easier to do a "grep 'release:' *.c"
than it is to try to figure out where the different "release" entries are
initialized).
But the fact that we need a big kernel repository right now does not
necessarily mean that we'll need one forever. With good enough interfaces
that people can truly feel happy about, it would be possible to split
stuff up one day. That is, after all, how the system call interfaces work,
and is what allows us to split the kernel from everything else.
Of course, usually what is "good enough" one day ends up being "really
bad" the next when some clever bastard came up with the really good way of
doing something..
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: rgo...@ras.ucalgary.ca (Richard Gooch)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <200007132150.e6DLoZA08931@vindaloo.ras.ucalgary.ca>#1/1
X-Deja-AN: 646157718
Sender: owner-linux-ker...@vger.rutgers.edu
References: <200007132108.e6DL8ZL08547@vindaloo.ras.ucalgary.ca>
<Pine.LNX.4.10.10007131410140.3044-100000@penguin.transmeta.com>
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Linus Torvalds writes:
>
>
> On Thu, 13 Jul 2000, Richard Gooch wrote:
> >
> > Perhaps the solution is to have shorter development cycles. Say 6
> > months before feature freeze, then a planned 3 month shakedown
> > (extended only if there are reliability problems). Part of the problem
> > is that so many new things get piled into each development series. Cut
> > back on the number of significant things per cycle (say just one or
> > two).
>
> I agree.
>
> This was what I wanted for the 2.2 -> 2.4 change too, and a much shorter
> development cycle would be wonderful. It obviously was not to be: 2.4 is
> already about 18 months out. We're not as bad as the 2.0 -> 2.2 cycle was,
> but it's painful.
>
> It seems to be a lot harder than it should be to keep short
> development releases. I certainly haven't found the magic
> combination yet. 2.4.x is definitely all I hoped for (the big goal
> for me was to make sure that the mindcraft-like web-performance
> scalability problems would be gone, and we definitely fixed that),
> but it took much too long.
Why not just shut the door on new features after 6 months? Even if
they are things you personally would really like to see (like the I/O
and networking scaling changes), if they're not ready in time, they
have to wait for the next cycle.
Looking back at 2.3.x, we see that Ingo's I/O scalability stuff went
in really early, because it was "ready" ("ready" doesn't mean safe;-).
But the networking scalability stuff (aka softnet) didn't go in until
this year. Either it wasn't ready, so should have waited for 2.5 when
the patch was more mature, or it was close enough and it should have
gone in regardless. If it's OK to break the kernel for several
patchlevels for I/O changes, it's also OK for networking changes.
If you just want to manage one significant change at a time,
particularly if it's a bit rough around the edges, wait at least a
month before applying the next major patch, to make sure the previous
one has settled down. Or wait until the next cycle.
Even after the softnet patch went in, we kept being hit by more and
more scalability patches. And multiple VFS redesigns. We didn't
*really* need all these changes to go in before 2.4.
> What would help would be more modular releases: smaller pieces of
> the kernel getting released independently, allowing for smaller and
> shorter release cycles. In Linux at least we don't have the "whole
> world" release issue that most other OS people have (including the
> free ones: I think the BSD "world" approach is horrible partly
> because of the release issues), but even just the kernel is so big
> that it would be nice to be able to see it as multiple independent
> projects.
>
> At the same time it's obviously not true that there are independent
> projects, and especially drivers (which _sound_ independent) are
> very likely to be impacted quite a lot by infrastructure
> changes. There are no really clear lines to split development up by,
> and the cures are worse than the disease ("microkernels", I hear
> somebody shout. But that approach would just make it impossible to
> do the kinds of improvements we _have_ done and probably will
> continue to do).
I don't think splitting the kernel development is the answer. Having
stuff separate is just too frustrating. I was so glad to see PCMCIA
drivers go into the kernel.
As for "microkernel". Ug. I see another flamewar coming.
PEOPLE: read the FAQ before you so much as breath that word on the
list, let alone post "well, why not?". <shudder>
Maybe you just need to get tougher with the cabal. You're pretty
hard-nosed when it comes to rejecting patches which so much as offend
your tastes (let alone those that suck). How about extending that to
timing? No matter how lovely the patch, if it's past feature-freeze,
reject it. Of course, come next cycle, said patch should go in
promptly.
Also, if there is a reliability problem somewhere, only accept patches
during feature freeze which do the minimum to fix the problem. Don't
let "bugfixes" be a back door to doing wholesale redesigns which fix
the problem along the way. Go for a surgical fix, not chemotherapy.
Too much chemo and the patient dies.
It's getting embarrasing when people ask me when 2.4 will be out. Not
only don't I know, AFAIK, nobody knows. And when of course they say
"but wasn't 2.4 supposed to come out last year, and didn't Linus want
a shorter cycle than 2.2?", I have to look away and mumble something
and hope they'll go away. Or try and change the subject. I can't keep
getting away with the brush-off. People are beginning to suspect.
Regards,
Richard....
Permanent: rgo...@atnf.csiro.au
Current: rgo...@ras.ucalgary.ca
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: jmer...@timpanogas.com (Jeff V. Merkey)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <396E39A6.F5729F5D@timpanogas.com>#1/1
X-Deja-AN: 646171163
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <E13CqiC-0004Uo-00@the-village.bc.nu>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
Organization: TRG, Inc.
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Alan,
Than I'll do my part to help further linux by giving you guys an example
of something that gets around these problems. Darren is going to string
me up for this, but we were planning to release it anyway. We'eve
developed a NetWare clone that we were going to open source at some
point anyway, but the debugger in it is exquisite compared to what's in
Linux. I give it freely. Let me know who wants to help. I'm open
sourcing MANOS today. I am not including and disk or LAN support, since
we got hit with a court order halting this work in 1998 (though the
court order is now expired). it's got a full source level KERNEL
debugger that Linux really needs. Let's get some folks to convert it
over to gcc, and we'll have it for Linux. I'll have it up on the server
in about 20 minutes. It's a full SMP OS that kicks the snot out of NT
and Linux on Raw SMP performance. TRG is the maintainer, so post your
fixes here and ignore the copyright notices. Post bugs to this list
since I was planning to merge it with Linux anyway, so I might as well
let you and Linus take a shot at using some of it, and the work will get
done faster with you guys help.
It uses Borland tools and loaded PE and DLL exes (the OS is really just
a big DLL).
:-)
Jeff
Alan Cox wrote:
>
> > Breaking the device drivers, protocol stacks, disk drivers, file
> > systems, etc. out of the kernel proper and forcing them to always be
> > loadable modules would achieve this end with the added benefit of
> > allowing folks to plug in their own optimized binary versions -- Like
>
> It breaks if you distribute them seperately. We have this problem in the way
> the model works. It suprised me until I understood what is going on
>
> I claim:
> Free software development is about minimising information exchange and knowledge
> of external modules.
>
> The practical upshot of this is we generate a fair amount of duplicated code.
> Worse because of the 'minimising information exchange' aspect most developers
> do not aggressively track the modules they initially took their skeleton code
> from - thus we get bug replication.
>
> One thing I have to do with many bug fixes is to grep the tree to see who
> else copied the bug. That needs a single tree.
>
> Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: a...@lxorguk.ukuu.org.uk (Alan Cox)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <E13Cq1w-0004NV-00@the-village.bc.nu>#1/1
X-Deja-AN: 646201567
Content-Transfer-Encoding: 7bit
Sender: owner-linux-ker...@vger.rutgers.edu
References: <396FE6C4.A0168052@reiser.to>
Content-Type: text/plain; charset=us-ascii
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
> I hope you understand why I think that adding reiserfs is a small disturbance
> compared to continuing revisions of VFS.
The reiserfs concerns I have are more about doing it the right way for all
fs's. Linus call anyway.
"I sense a disturbance in the source"
Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: ty...@MIT.EDU (Theodore Y. Ts'o)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/13
Message-ID: <200007131948.PAA16486@tsx-prime.MIT.EDU>#1/1
X-Deja-AN: 646215198
Sender: owner-linux-ker...@vger.rutgers.edu
References: <200007131818.e6DIIEh07166@vindaloo.ras.ucalgary.ca>
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
Date: Thu, 13 Jul 2000 12:18:14 -0600
From: Richard Gooch <rgo...@ras.ucalgary.ca>
> The (b) kind of inconsistency is unavoidable, but I'd rather avoid
> it for the "mindless" kind of lock movement that affects a lot of
> drivers and filesystems, yet is basically very simple ("mindless" !=
> "easy").
Except it will cost us time and stability. Where do you draw the line?
Or even better: when will 2.4 be really, really, *really* released?
I have to agree with Richard here. If we do this, it'll set back 2.4 by
at least month (and I may be conservative here). There will *always* be
more places where we can do more/better fine-grained locking. Where
indeed do we draw the line, and do we really want to be doing this while
we're at 2.4.0-test*?!?
If we were still accepting stuff like this, we shouldn't have moved the
verison number to 2.3.99 or 2.4.0testn.
- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: and...@suse.de (Andrea Arcangeli)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/14
Message-ID: <Pine.LNX.4.21.0007140323080.894-100000@inspiron.random>#1/1
X-Deja-AN: 646144964
Sender: owner-linux-ker...@vger.rutgers.edu
X-PGP-Key-URL: http://e-mind.com/~andrea/aa.asc
References: <E13CqEp-0004Pp-00@the-village.bc.nu>
X-Sender: and...@inspiron.random
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-GnuPG-Key-URL: http://e-mind.com/~andrea/aa.gnupg.asc
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Alan Cox wrote:
>Except we are _still_ fixing the 2.2 VM and paying a stability penalty
>along the way for it. That is -bad-
It should be fixed in VM-global-patch-2. Main issue was kpiod.
I released a few minutes ago 2.2.17pre11aa1 that also have some more VM
stuff into it. I'd like if people could try it and tell me what they
thinks about the VM behaviour. It merges VM-global-patch-2 and all the
other few stuff is completly orthogonal with VM-global-patch-2.
Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: dw...@infradead.org (David Woodhouse)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/14
Message-ID: <21595.963564218@cygnus.co.uk>#1/1
X-Deja-AN: 646243037
Sender: owner-linux-ker...@vger.rutgers.edu
References: <Pine.LNX.4.10.10007131011100.1239-100000@penguin.transmeta.com>
Content-Type: text/plain; charset=us-ascii
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
torva...@transmeta.com said:
> On Thu, 13 Jul 2000, Alan Cox wrote:
> > This kind of lock shifting is a major upheaval. It invalidates any
> > device driver testing done in the past weeks when we have been slowly
> > moving towards more stability.
> I much rather do it now than later. Later means either: (a) during
> 2.4.x or (b) early in 2.5.x.
If we're going to do it, we're removing the last cases where it was valid
to use sleep_on() and friends (with a few exceptions).
Can we now remove sleep_on() completely, or at least apply something like
the following:
Index: sched.c
===================================================================
RCS file: /inst/cvs/linux/kernel/sched.c,v
retrieving revision 1.4.2.31
diff -u -w -r1.4.2.31 sched.c
--- sched.c 2000/07/13 10:13:52 1.4.2.31
+++ sched.c 2000/07/14 08:38:51
@@ -786,10 +786,18 @@
__remove_wait_queue(q, &wait); \
wq_write_unlock_irqrestore(&q->lock,flags);
+#define SLEEP_ON_LOCKCHECK \
+ if (current->lock_depth == -1 /* ? */) { \
+ printk(KERN_WARNING __FUNCTION__ " called without BKL held!\n"); \
+ BUG(); \
+ }
+
void interruptible_sleep_on(wait_queue_head_t *q)
{
SLEEP_ON_VAR
+ SLEEP_ON_LOCKCHECK
+
current->state = TASK_INTERRUPTIBLE;
SLEEP_ON_HEAD
@@ -801,6 +809,8 @@
{
SLEEP_ON_VAR
+ SLEEP_ON_LOCKCHECK
+
current->state = TASK_INTERRUPTIBLE;
SLEEP_ON_HEAD
@@ -814,6 +824,8 @@
{
SLEEP_ON_VAR
+ SLEEP_ON_LOCKCHECK
+
current->state = TASK_UNINTERRUPTIBLE;
SLEEP_ON_HEAD
@@ -825,6 +837,8 @@
{
SLEEP_ON_VAR
+ SLEEP_ON_LOCKCHECK
+
current->state = TASK_UNINTERRUPTIBLE;
SLEEP_ON_HEAD
--
dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: r...@conectiva.com.br (Rik van Riel)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/14
Message-ID: <Pine.LNX.4.21.0007141216370.1692-100000@duckman.distro.conectiva>#1/1
X-Deja-AN: 646356250
Sender: owner-linux-ker...@vger.rutgers.edu
References: <E13Cq1w-0004NV-00@the-village.bc.nu>
X-Sender: r...@duckman.distro.conectiva
X-Authentication-Warning: duckman.distro.conectiva: riel owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Alan Cox wrote:
> > I hope you understand why I think that adding reiserfs is a small disturbance
> > compared to continuing revisions of VFS.
>
> The reiserfs concerns I have are more about doing it the right
> way for all fs's. Linus call anyway.
I think it should be possible to integrate reiserfs once the
new VM code is ready. The VM plans we have for 2.4 are all in
the "obviously right" category and will be more a code and
mechanism cleanup than new mechanisms, with the exception of
the page->mapping->flush(page) callback which is needed for
journaling filesystems.
> "I sense a disturbance in the source"
Nothing a few jedi can't solve...
(is there a jedi in the room? anyone?)
regards,
Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000
http://www.conectiva.com/ http://www.surriel.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: r...@conectiva.com.br (Rik van Riel)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/14
Message-ID: <Pine.LNX.4.21.0007141244170.1692-100000@duckman.distro.conectiva>#1/1
X-Deja-AN: 646371246
Sender: owner-linux-ker...@vger.rutgers.edu
References: <Pine.LNX.4.21.0007141631580.324-100000@inspiron.random>
X-Sender: r...@duckman.distro.conectiva
X-Authentication-Warning: duckman.distro.conectiva: riel owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Fri, 14 Jul 2000, Andrea Arcangeli wrote:
> On Fri, 14 Jul 2000, Andrea Arcangeli wrote:
>
> >It should be fixed in VM-global-patch-2. Main issue was kpiod.
>
> I just noticed VM-global-patch-2 could do I/O even if __GFP_IO
> wasn't set in the gfp_mask
Guess why code readability and maintainability is at the absolute
top of my priority list for the new VM code? ;)
regards,
Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000
http://www.conectiva.com/ http://www.surriel.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: and...@suse.de (Andrea Arcangeli)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/14
Message-ID: <Pine.LNX.4.21.0007140330490.894-100000@inspiron.random>#1/1
X-Deja-AN: 646400094
Sender: owner-linux-ker...@vger.rutgers.edu
X-PGP-Key-URL: http://e-mind.com/~andrea/aa.asc
References: <E13CpqF-0004Lc-00@the-village.bc.nu>
X-Sender: and...@inspiron.random
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
X-GnuPG-Key-URL: http://e-mind.com/~andrea/aa.gnupg.asc
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Thu, 13 Jul 2000, Alan Cox wrote:
>6 weeks to fix the VM is too short. But then I think we may have to accept
>that we dont get the VM perfect until 2.5. We just need to get it back working
>as well as 2.0 did
IMHO the fact the memory balancing doesn't know about the _class_zone
where the user want to allocate memory is one major problem that we have
in 2.4.x. Without that chance Juan's kernel will keep trying to take some
ISA DMA memory free even if he doesn't use floppy or ISA soundcard and it
won't provide LRU behaviour within the classzone. 2.0.x wasn't wasting
time this way.
This is one of the things that is fixed since _ages_ ago (I trapped it as
soon as it seen the light) in classzone.
What I mean is that if shrink_mmap doesn't get a zone_t pointer as
parameter, the VM will always end doing something wrong. There's no way to
fix that except reinserting the parameter that gives the information about
the allocation as I did in classzone.
Also in the long term we'll pass much more than the zone_t information to
the memory balancing code (we'll add the order of the allocation for
keeping care of fragmentation and the wanted colour of the page for
example).
I'll be glad to extract such part of classzone as a clean patch against
latest Linus's tree if somebody is interested.
Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
From: torva...@transmeta.com (Linus Torvalds)
Subject: Re: [Announce] BKL shifting into drivers and filesystems - beware
Date: 2000/07/14
Message-ID: <Pine.LNX.4.10.10007141222230.1553-100000@penguin.transmeta.com>#1/1
X-Deja-AN: 646445834
Sender: owner-linux-ker...@vger.rutgers.edu
References: <21595.963564218@cygnus.co.uk>
X-Authentication-Warning: penguin.transmeta.com: torvalds owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Newsgroups: linux.dev.kernel
X-Loop: majord...@vger.rutgers.edu
On Fri, 14 Jul 2000, David Woodhouse wrote:
>
> torva...@transmeta.com said:
>
> > I much rather do it now than later. Later means either: (a) during
> > 2.4.x or (b) early in 2.5.x.
>
> If we're going to do it, we're removing the last cases where it was valid
> to use sleep_on() and friends (with a few exceptions).
People _still_ don't understand.
AL VIRO DID NOT REMOVE THE BKL.
He MOVED it. He moved it into the "release()" function, so that we can
pick them off one-by-one. Some of them were never needed (like the
networking code that actually dropped it on purpose immediately), and some
of them will be trivial to fix up.
But no behaviour was changed. sleep_on() is still valid, because code that
used to have the kernel lock _still_ has the kernel lock. It's just that
they got it themselves, instead of depending on the VFS layer to get it
for them.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/