To: video4linux list <video4linux-list@xxxxxxxxxx>
Subject: v4l2 + kernel
From: Gerd Knorr <kraxel@xxxxxxx>
Date: Tue, 10 Apr 2001 19:32:03 +0200
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.3.12i
Hi,
I'm just browsing the v4l and v4l2 code. I'd like to see v4l2 early
in 2.5.x in the standard kernel and looking for a clever way to get
this done (and fix some videodev problems).
I've seen my videodev patch from bttv 0.8.x made it into the videodevX
code. Now we have *three* methods to register a driver:
(1) The very old one (video_device + function pointers)
(2) The bttv 0.8.x one (video_device + fops)
(3) The v4l2 one (v4l2_device).
This is one to much. We have to keep (1) for backward compatibility
with the existing drivers in the kernel. I'd like to drop (3) and keep
(2) for the following reasons:
* I think it's clever to simply use a struct file_operations directly.
Most of the fops functions in videodevX.c simply pass down the call
to the driver (the only exception is v4l2_video_ioctl), so why not
call the driver directly in the first place? Also we don't have to
fiddle with videodev if struct file_operations changes again for
some reason.
* I'd like to have videodev.c just register drivers and hand out the
minor numbers and do _nothing_ else, i.e. do what soundcore does
for the sound drivers. videodev.c shouldn't care which API a driver
uses. All the v4l1 compatibility helper code which is in videodevX
right now should go to some other module (v4l1-compat.c or something
like that).
Plan as follows:
* fix videodev.c. A patch is below. It does add register method (2)
to the kernel's videodev.c. It adds a lock to the register function
to fix a race. It allows drivers to ask for a specific device number.
This becomes important these days as people start using different
devices (bttv+usbcam) at the same time. Fixed module load order
being the only way to get a fixed special file -> device mapping
is not very nice [longer version of the patch which fixes all kernel
drivers is available too].
IMHO this should go into 2.4.x.
* Fixes/cleanups in the v4l2 capture API. Major item on the TODO
list is the dma-to-userspace issue discussed here some time ago.
Also some minor byte order problems (15/16 bpp rgb is available
in little endian only, ...). Put the v4l2 capture API largely
as-is into the 2.5.x kernel then.
The vbi stuff can be dropped from v4l2 I think. The current vbi API
in 2.4.x is cloned from v4l2, and we certainly don't need exactly the
same API twice in the kernel.
Comments?
What is the status of the other v4l2 API's? codec? effects? Any
existing drivers which use these?
Gerd
Code
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Justin Schoeman < justin@xxxxxxxxxxxxxxxxxxxx>
Date: Wed, 11 Apr 2001 10:52:04 +0200
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
Organization: University of Pretoria
Gerd Knorr wrote:
>
> Hi,
>
> I'm just browsing the v4l and v4l2 code. I'd like to see v4l2 early
> in 2.5.x in the standard kernel and looking for a clever way to get
> this done (and fix some videodev problems).
>
> I've seen my videodev patch from bttv 0.8.x made it into the videodevX
> code. Now we have *three* methods to register a driver:
>
> (1) The very old one (video_device + function pointers)
> (2) The bttv 0.8.x one (video_device + fops)
> (3) The v4l2 one (v4l2_device).
>
> This is one to much. We have to keep (1) for backward compatibility
> with the existing drivers in the kernel. I'd like to drop (3) and keep
> (2) for the following reasons:
>
> * I think it's clever to simply use a struct file_operations directly.
> Most of the fops functions in videodevX.c simply pass down the call
> to the driver (the only exception is v4l2_video_ioctl), so why not
> call the driver directly in the first place? Also we don't have to
> fiddle with videodev if struct file_operations changes again for
> some reason.
>
> * I'd like to have videodev.c just register drivers and hand out the
> minor numbers and do _nothing_ else, i.e. do what soundcore does
> for the sound drivers. videodev.c shouldn't care which API a driver
> uses. All the v4l1 compatibility helper code which is in videodevX
> right now should go to some other module (v4l1-compat.c or something
> like that).
Agreed. The only provisio being that there must be some way for a
driver registering by method 2 to say if it is v4l1/v4l2 so that the
compat layer can initialise for that device. We will also need a clean
method for "alternative" names to id the compatible devices (the
compatibility layer and actual driver will no longer be able to share
the same minor number.
> Plan as follows:
>
> * fix videodev.c. A patch is below. It does add register method (2)
> to the kernel's videodev.c. It adds a lock to the register function
> to fix a race. It allows drivers to ask for a specific device number.
> This becomes important these days as people start using different
> devices (bttv+usbcam) at the same time. Fixed module load order
> being the only way to get a fixed special file -> device mapping
> is not very nice [longer version of the patch which fixes all kernel
> drivers is available too].
> IMHO this should go into 2.4.x.
I think the base name for the device should also be passed in the
"register" call - this will make it possible to also free the v4l1
drivers from arbitrary restrictions on minor numbers, and allow for more
sensible devfs names.
Will also have to look at the "type" parameter, as this changes meaning
for v4l1/v4l2 - perhaps offset them, and use this to id v4l1/v4l2
devices?
Also, fix the procfs stuff to be v4l2-aware.
> * Fixes/cleanups in the v4l2 capture API. Major item on the TODO
> list is the dma-to-userspace issue discussed here some time ago.
> Also some minor byte order problems (15/16 bpp rgb is available
> in little endian only, ...). Put the v4l2 capture API largely
> as-is into the 2.5.x kernel then.
The DMA issue is still a major question - the discussion ended on two
completely divergent solutions, each with major advantages an
disadvantages, and no definitive solution... This will need a bit more
discussion (or possibly a decent all out brawl ;->).
The byte order problems should be resolved shortly with the definition
of more video formats for the big endian machines.
> The vbi stuff can be dropped from v4l2 I think. The current vbi API
> in 2.4.x is cloned from v4l2, and we certainly don't need exactly the
> same API twice in the kernel.
No. The only thing that is cloned is "struct vbi_format", and there
will be no saving removing this - just namespace confusion.
> What is the status of the other v4l2 API's? codec? effects? Any
> existing drivers which use these?
There was some work in this direction, but I am not sure what happened
to it...
-justin
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Gerd Knorr <kraxel@xxxxxxx>
Date: Wed, 11 Apr 2001 16:42:40 +0200
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
In-reply-to: <3AD41B34.F38CDB00@xxxxxxxxxxxxxxxxxxxx>;
from justin@xxxxxxxxxxxxxxxxxxxx on Wed, Apr 11, 2001 at 10:52:04AM +0200
User-agent: Mutt/1.3.12i
> Agreed. The only provisio being that there must be some way for a
> driver registering by method 2 to say if it is v4l1/v4l2 so that the
> compat layer can initialise for that device.
IMHO the compatibility layer should'nt be mandatory. I think the best
way to handle that is to have a set of helper functions for code which
likely can be useful for multiple drivers (like translating formats
between v4l1 <=> v4l2) in a compat module. The v4l2 driver can use
all / some / none of these functions. If it wants to use these
functions, it simply calls them.
Not sure how to handle the ioctl translation best. The problem is where
to do the arg copying from/to userspace. Right now the v4l2 ioctl
translation functions calls v4l2_driver->ioctl(), which is expected to
receive a kernel pointer. With file_operations->ioctl() this obviously
doesn't work any more, because that function is expected to handle
userspace copying itself. We'll have to pass a callback function to
translate_ioctl() somehow. Either directly as argument, or somewhere
in a struct.
> We will also need a clean
> method for "alternative" names to id the compatible devices (the
> compatibility layer and actual driver will no longer be able to share
> the same minor number.
Why? The driver can simply call translate_ioctl() for all ioctls it
doesn't understand. I don't see a need to use another minor for
the v4l1 backward compatibility.
> I think the base name for the device should also be passed in the
> "register" call - this will make it possible to also free the v4l1
> drivers from arbitrary restrictions on minor numbers, and allow for more
> sensible devfs names.
We can't get rid of the minor numbers (yet?). devfs is optional and
still marked experimental, so it must work with the traditional
major/minor scheme too.
> Will also have to look at the "type" parameter, as this changes meaning
> for v4l1/v4l2 - perhaps offset them, and use this to id v4l1/v4l2
> devices?
Add a type2 field?
> The DMA issue is still a major question - the discussion ended on two
> completely divergent solutions, each with major advantages an
> disadvantages, and no definitive solution... This will need a bit more
> discussion (or possibly a decent all out brawl ;->).
That's why this is the big todo list item.
> The byte order problems should be resolved shortly with the definition
> of more video formats for the big endian machines.
Ack.
> > The vbi stuff can be dropped from v4l2 I think. The current vbi API
> > in 2.4.x is cloned from v4l2, and we certainly don't need exactly the
> > same API twice in the kernel.
>
> No. The only thing that is cloned is "struct vbi_format",
And two ioctl's to get/set that. So you have VIDIOCGVBIFMT(struct
vbi_format) in v4l1 and VIDIOC_G_FMT(struct v4l2_format) with
type = V4L2_TYPE_VBI in v4l2. I can't see any difference here,
and these are the only vbi-specific ioctls in v4l1/2 ...
> > What is the status of the other v4l2 API's? codec? effects? Any
> > existing drivers which use these?
>
> There was some work in this direction, but I am not sure what happened
> to it...
Then these should left out too IMHO. There is no point in having
a unused API in the kernel. And in case someone finds a design bug
while writing a driver it is easier to fix if it is not yet in the
standard kernel.
Gerd
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Alan Cox <alan@xxxxxxxxxx>
Date: Wed, 11 Apr 2001 12:02:43 -0400 (EDT)
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
In-reply-to: <20010411164240.B3572@xxxxxxxxxxxxxxxxx>
from "Gerd Knorr" at Apr 11, 2001 04:42:40 PM
> Not sure how to handle the ioctl translation best. The problem is where
> to do the arg copying from/to userspace. Right now the v4l2 ioctl
> translation functions calls v4l2_driver->ioctl(), which is expected to
> receive a kernel pointer. With file_operations->ioctl() this obviously
I speciically didnt do this with v4l1 because of driver private ioctls which
you cannot move into kernel space. The networking code takes a different
tack and has a seperate pointer for device private ioctl handling
> We can't get rid of the minor numbers (yet?). devfs is optional and
> still marked experimental, so it must work with the traditional
> major/minor scheme too.
You need major/minor numbers for ever. You can swap them for text strings
and break compatibility massively but it doesnt change anything in truth.
> vbi_format) in v4l1 and VIDIOC_G_FMT(struct v4l2_format) with
> type = V4L2_TYPE_VBI in v4l2. I can't see any difference here,
> and these are the only vbi-specific ioctls in v4l1/2 ...
Where they are the same keep the old one. Similarly you can map formats
by making most of the numbers line up.
Alan
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: M.Hunold@xxxxxxxxxxx (Michael Hunold)
Date: Wed, 11 Apr 2001 22:37:19 +0200
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
Hello Gerd,
> IMHO the compatibility layer should'nt be mandatory. I think the best
> way to handle that is to have a set of helper functions for code which
> likely can be useful for multiple drivers (like translating formats
> between v4l1 <=> v4l2) in a compat module. The v4l2 driver can use
> all / some / none of these functions. If it wants to use these
> functions, it simply calls them.
I have no problems with your idea of using the "video_device + fops"
device handling, but I don't understand the idea of removing
the "v4l2-awareness" out of the videodevX-module.
Just to get things right here: do you want to modify the original
videodev1-module to let the v4l2 driver fit into there, or do
you want to modify the videodev2/videodevX-module to let the
v4l drivers fit into there?
I always thought that it would be better to drop videodev1
and modify videodev2, so that v4l1 drivers and apps still work.
At the moment (e.g. with videodevX/videodev2)
there is minimum v4l-backward compatibility code
at least in my v4l2 driver -- and that's good. All important
stuff (ioctl translation, format translation, etc.) is done
outside the v4l2 driver.
Do you really mean to move the responsibility of being
v4l-backward compatible deep down to a v4l2 driver, e.g.
looking at the ioctls, formats and do the translation
by calling helper functions?
At the moment
videodevX adds backward compatibility to any v4l2 without
special help of the driver. With your proposal, we would
have code duplication for v4l backward compatibility
in every v4l2 driver around.
Even good and robust v4l2 drivers, who are v4l1 aware
through videodevX, would need to get tweaked to become
v4l1 aware the way you want it.
I don't think that's a wise idea.
Please correct my if I am terribly wrong on something.
> Gerd
CU
Michael.
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Alan Cox <alan@xxxxxxxxxx>
Date: Wed, 11 Apr 2001 18:32:27 -0400 (EDT)
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
In-reply-to: <3AD4C07F.A9A85D79@xxxxxxxxxxx>
from "Michael Hunold" at Apr 11, 2001 10:37:19 PM
> Do you really mean to move the responsibility of being
> v4l-backward compatible deep down to a v4l2 driver, e.g.
> looking at the ioctls, formats and do the translation
> by calling helper functions?
It would to think much more about putting the bits of V4L2 that are missing
or better into V4L not about two interfaces. That way lies pain.
Which reminds me. Advance notice that in 2.5.0 I will ask Linus to delete
i2c-old.h. I will also be deleting all mail complaining about it at that
point 8)
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Bill Dirks < bdirks@xxxxxxxxxx>
Date: Wed, 11 Apr 2001 20:09:53 -0700
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
Organization: Rendition
I agree with most of what all of you are saying.
Where possible, I designed the V4L2 videodev to insulate the V4L2
drivers from kernel details like struct file and struct file_operations
that can change with new versions. When your driver is not in the kernel
and needs to work with many kernel versions, that is a plus. If V4L2 is
in the kernel, then the need diminishes. Still, if videodev can hold
code that would otherwise need to be duplicated in the drivers, then why
not?
I'm not sure what all the issues are with driver registration, but it
seems to me a good idea to keep separate registration functions for V4L2
and v4l drivers so videodev knows which is which in case it becomes
necessary. We can tweak the v4l and V4L2 registration if need be.
Moving the v4l compatibility out to a separate, optional module is a
great idea. To make it optional, all you have to do is, when the compat.
module loads, have it pass pointers to its helper functions to videodev
by a registration function in videodev.
Michael Hunold wrote:
> Do you really mean to move the responsibility of being
> v4l-backward compatible deep down to a v4l2 driver, e.g.
> looking at the ioctls, formats and do the translation
> by calling helper functions?
The burden seems small. As Gerd pointed out, it reduces to:
v4l2_ioctl(...) {
switch (cmd) {
default: return v4l2_translate_ioctl(...);
...
}
}
Maybe some code in mmap() too.
But I don't think that would be necessary, would it? Control always has
to pass through the videodev ioctl() hook, so it could be done there?
Alan Cox wrote:
> It would to think much more about putting the bits of V4L2
> that are missing or better into V4L not about two interfaces.
> That way lies pain.
I don't think one can "put bits of V4L2 that are better into V4L" and
end up with a decent, coherent interface. V4L2 is arranged differently.
The main reason applications are still being written for and supporting
v4l is because it's in the standard kernel and V4L2 is not. If V4L2 is
in the kernel, that reason goes away. Once V4L2 is in the kernel, v4l
support can later be made optonal, and eventually be phased out.
> > What is the status of the other v4l2 API's? codec? effects? ...
>
> Then these should left out too IMHO. There is no point in having
> a unused API in the kernel.
Agree.
Bill.
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Gerd Knorr <kraxel@xxxxxxx>
Date: Thu, 12 Apr 2001 08:51:19 +0200
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
In-reply-to: <3AD51C81.FC6CEC45@xxxxxxxxxx>;
from bdirks@xxxxxxxxxx on Wed, Apr 11, 2001 at 08:09:53PM -0700
User-agent: Mutt/1.3.16i
On Wed, Apr 11, 2001 at 08:09:53PM -0700, Bill Dirks wrote:
> I agree with most of what all of you are saying.
>
> Where possible, I designed the V4L2 videodev to insulate the V4L2
> drivers from kernel details like struct file and struct file_operations
> that can change with new versions.
If they change, they usually change for a reason. That change might be
important for the driver, so I don't think it is good to hide that within
the videodev layer.
> in the kernel, then the need diminishes. Still, if videodev can hold
> code that would otherwise need to be duplicated in the drivers, then why
> not?
That code I'd like to see in the compatibility/helper/... module.
> Moving the v4l compatibility out to a separate, optional module is a
> great idea. To make it optional, all you have to do is, when the compat.
> module loads, have it pass pointers to its helper functions to videodev
> by a registration function in videodev.
Why register? The driver can simply call the functions in the helper
module.
> Michael Hunold wrote:
> > Do you really mean to move the responsibility of being
> > v4l-backward compatible deep down to a v4l2 driver, e.g.
> > looking at the ioctls, formats and do the translation
> > by calling helper functions?
>
> The burden seems small. As Gerd pointed out, it reduces to:
>
> v4l2_ioctl(...) {
> switch (cmd) {
> default: return v4l2_translate_ioctl(...);
> ...
> }
> }
>
> Maybe some code in mmap() too.
>
> But I don't think that would be necessary, would it? Control always has
> to pass through the videodev ioctl() hook, so it could be done there?
It has not. If video_open() swaps the struct file_operations pointer in
the file handle (look at the patch I've mailed), videodev will _not_ see
ioctl calls from the application.
> Alan Cox wrote:
> > It would to think much more about putting the bits of V4L2
> > that are missing or better into V4L not about two interfaces.
> > That way lies pain.
>
> I don't think one can "put bits of V4L2 that are better into V4L" and
> end up with a decent, coherent interface. V4L2 is arranged differently.
I agree. Nearly all ioctls work different. I think the only ones which
are completely unchanged are VIDIOC_G_FREQ/VIDIOC_S_FREQ. Anything else
has changed alot (like the mmap() capture interface) or at least a little
bit (missing fields added / moved some to the right place / ...).
> in the kernel, that reason goes away. Once V4L2 is in the kernel, v4l
> support can later be made optonal, and eventually be phased out.
v4l has been _the_ interface for years. And it probably takes at least
one more year until v4l2 shows up in a stable kernel (2.6). I don't
think we can phase out v4l1 ...
Gerd
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Alan Cox <alan@xxxxxxxxxx>
Date: Thu, 12 Apr 2001 06:31:59 -0400 (EDT)
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
In-reply-to: <3AD51C81.FC6CEC45@xxxxxxxxxx>
from "Bill Dirks" at Apr 11, 2001 08:09:53 PM
> I'm not sure what all the issues are with driver registration, but it
> seems to me a good idea to keep separate registration functions for V4L2
> and v4l drivers so videodev knows which is which in case it becomes
> necessary. We can tweak the v4l and V4L2 registration if need be.
Or just break pure V4L1 drivers. Most of those are extremely simple so fixing
the ioctl side of stuff will be easy and with the copying done by the v4l2_ioctl
actually clean some of them up.
The hard bit will be read/write/mmap
Alan
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Bill Dirks <bdirks@xxxxxxxxxx>
Date: Mon, 16 Apr 2001 12:25:52 -0700
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
Organization: Rendition
I've made a first swing at separating out the compatibility layer in
videodevX.
ftp://ftp.thedirks.org/pub/v4l2/kernel2.4/videodevX/
videodevX-20010416.tgz
The compatibility layer is in a separate module called v4l_compat. When
loaded it calls a new compatibility layer registration function,
v4l2_v4l_compat_register(), in videodevX with its interface. The
interface has two methods, translate_ioctl() and fix_offset(). The
latter is used in the v4l2_mmap() wrapper.
Note: If you install the new one, you will not have v4l->V4L2
compatibility unless you do an explicit modprobe v4l_compat. That can be
fixed with either a post-install line in /etc/modules.conf or a
request_module() in videodevX, but I haven't done it yet.
Gerd Knorr wrote:
> Why register? The driver can simply call the functions in the helper
> module.
More flexible, cleaner, I guess. It can be loaded and unloaded; doesn't
have to be made a compile-time option in to the drivers. It can be
called from videodevX transparent to the driver.
Gerd Knorr wrote:
> Bill Dirks wrote:
> > I don't think one can "put bits of V4L2 that are better
> > into V4L" and end up with a decent, coherent interface. V4L2
> > is arranged differently.
>
> I agree. Nearly all ioctls work different. I think the only
> ones which are completely unchanged are VIDIOC_G_FREQ/VIDIOC_S_FREQ.
And even those will probably have to change. The set-top box guys are
after me to support multiple tuners. Overlay (preview) support needs
much expansion for the set-top box market too. I started working on it
here: http://www.thedirks.org/v4l2/proposed/overlay.htm
(I've got to tweak the terminology, where you see "plane" think
"overlay" or "visual".)
> > Once V4L2 is in the kernel, v4l support can later be made
> > optonal, and eventually be phased out.
>
> v4l has been _the_ interface for years. And it probably takes
> at least one more year until v4l2 shows up in a stable kernel
> (2.6). I don't think we can phase out v4l1 ...
Well, I meant phase it out three years later, or somesuch.
Bill.
To: video4linux-list@xxxxxxxxxx
Subject: Re: v4l2 + kernel
From: Gerd Knorr <kraxel@xxxxxxx>
Date: Tue, 17 Apr 2001 15:27:41 +0200
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx
In-reply-to: <3ADB4740.F737C88C@xxxxxxxxxx>;
from bdirks@xxxxxxxxxx on Mon, Apr 16, 2001 at 12:25:52PM -0700
User-agent: Mutt/1.3.16i
On Mon, Apr 16, 2001 at 12:25:52PM -0700, Bill Dirks wrote:
> I've made a first swing at separating out the compatibility layer in
> videodevX.
> ftp://ftp.thedirks.org/pub/v4l2/kernel2.4/videodevX/
> videodevX-20010416.tgz
IŽll have a look.
> > Why register? The driver can simply call the functions in the helper
> > module.
>
> More flexible, cleaner, I guess. It can be loaded and unloaded; doesn't
> have to be made a compile-time option in to the drivers. It can be
> called from videodevX transparent to the driver.
I'd like to avoid passing everything throuth videodevX. And I don't think
it is that good to try to hide v4l1 completely from the driver, it doesn't
work completely anyway. Look for example at the mmap issue: The driver
has to handle v4l1 and v4l2 mappings in different ways (by looking at
the V4L2_BUF_REQ_CONTIG flag).
IMHO a driver should be able to handle some or all v4l1 compatibility
issues itself.
> Overlay (preview) support needs
> much expansion for the set-top box market too. I started working on it
> here: http://www.thedirks.org/v4l2/proposed/overlay.htm
> (I've got to tweak the terminology, where you see "plane" think
> "overlay" or "visual".)
why a v4l2 interface for that? Setting up a overlay IMHO does belong
into the X-Server (useable throuth the Xvideo Extention) or into the
framebuffer driver.
> > v4l has been _the_ interface for years. And it probably takes
> > at least one more year until v4l2 shows up in a stable kernel
> > (2.6). I don't think we can phase out v4l1 ...
>
> Well, I meant phase it out three years later, or somesuch.
Maybe, we will see.
Gerd