Tech Insider					     Technology and Trends


		   Video for Linux Mailing List Archives

To: livid-gatos@xxxxxxxxxxxxxx, video4linux-list@xxxxxxxxxx, 
linux-kernel@xxxxxxxxxxxxxxx 
Subject: [RFC] alternative kernel multimedia API 
From: volodya@xxxxxxxxxxxxxx 
Date: Thu, 25 Oct 2001 13:06:12 -0400 (EDT) 
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx 

 After looking at and working with Xv, v4l and v4l2 I became somewhat
dissatisfied with the current state of affairs. I have attached a
description of the API that would make (at least) me much happier. 

 I would very much appreciate comments from interested people..

                           thanks !

                               Vladimir Dergachev




              Kernel multimedia architecture

        **     the latest version can be obtained from     **
	
	http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/gatos/km/km.rfc.txt

0) Motivation
    v4l, v4l2 and Xv are all suffering from the same problem: attempt to fit 
    existing multimedia devices into a fixed scheme. The use of pre-defined
    structs to describe parameters of device is inherently wrong because
    these parameters vary widely. This leads to either bloating of the control
    structures with parameters used only by few devices, proliferation of
    device-specific ioctl and/or struct versioning. This also makes it increasingly
    hard to implement support for new parameters.

    The solution, IMO, is to move away from hard-coded models of multimedia
    devices and instead allow greater flexibility to driver developers by
    providing _symbolic_ interface. 
    
    The specific details follow, but the core idea is as follows:
    
    a) for each hardware device/subsystem there is a "control" device
     which user application can use to set/query parameters. For example,
     to set hue it can issue a command like:
      HUE=7\n
     to be replied with
     +\n
     (indicating that the command succeeded)
     or
     -HUE=INVALID
     if HUE attribute is not available.
     To query picture size it can issue:
     ?WIDTH,HEIGHT\n
     To be replied with
     +WIDTH=640,HEIGHT=480\n
     In case the device supports it.
     
    b) for some devices (all capture or output devices) there is one (or more ?)
       "data" device. 
       
       The data device is meant for high-speed transfer of data. The data is 
       accessed by memory mapping the data device (the size is found out via
       control device). 
       
       The beginning of the memory mapping is a control/status struct whose
       primary use is exchange of information and synchronization between
       application and kernel driver. 
       
       For example, for a frame grabber the control struct will have (at least)
       three fields for each buffer: timestamp, number,  and busy. 
       "busy" will be set when the buffer is acquiring new data,
       "timestamp"  will be an integer value monotonically increasing with time 
       and "number" will be the serial number of the captured frame.
       
       When capturing video the application will read data from the buffers
       until it encounters one with smaller serial number and/or marked as busy.
       It then has an option to do something else and/or issue a request
       on control interface to notify when more data is available.
       
       The advantage of this scheme is elimination of context switches
       due to ioctl and selects, but, more importantly, elimination of the
       numereous ioctls whose sole purpose is to transfer data between 
       application and the kernel.
       
    c) So far we replaced numerous ioctl with a control interface and a control struct.
       However the question of the format of the control struct still remains.
       The answer is that the format itself is quieried via control interface:
       
       ?GLOBAL_PARAM_OFFSET           returns the offset of the global parameter area
       ?GLOBAL_PARAM_FORMAT           returns the interface class of the global
                                      parameter area followed by the lists of fields
				      and offsets within that area
	?BUFFERS_COUNT                returns the number of buffers
	?BUFFER_OFFSET,BUFFER=X       returns the offset of the buffer
	?BUFFER_PARAM_OFFSET,BUFFER=X returns the offset of the buffer-specific
	                              parameter area
	?BUFFER_PARAM_FORMAT,BUFFER=X returns the interface class of the 
	                               buffer specific parameter area
				       followed by list of fields and offsets
				       within that parameter area

     d) Questions that occured to me while I was writing this:
     
              Q1) Should something that complex be in the kernel?
	      A1) The control language is very simple. An example of
	         something similar would be software modems and ISDN
		 which must be able to parse AT command set.

	      Q2) Is it really necessary to bloat video drivers with
	          parsing functions
	      A2) the idea is that there will be a library of functions
	          used by all drivers simultaneously. The driver code
		  will only need to declare it structures and let the
		  library know about them - the actual interface code
		  will be common to all drivers

       
1) User-space interface basics

     /devfs/multimedia/devices   - a text file that enumerates known devices
     /devfs/multimedia/control/deviceX - control devices
     /devfs/multimedia/data/dataY    - high-speed data devices
       (X and Y are numbers)     
     
     Protocol:
     
     a) user application reads /etc/fstab (/etc/mtab) to find out where /devfs
      is mounted.
      
     b) user application reads /devfs/multimedia/devices to find out about
        devices present
	
     c) to open a device it opens /devfs/multimeda/control/* and negotiates
        parameters/performs control functions
	
     d) when data transfer is needed it queries /devfs/multimedia/control/deviceX
        about port number for the data interface and opens
	/devfs/multimedia/data/portY for R/W or R depending on intended purpose
	
	It then queries /devfs/multimedia/control/deviceX about the size of
	data interface region of portY and memory maps the area with R/W or R
	privileges. 
	
	Next it queries control deviceX about the size and format of control
	section of data interface region of portY which is always located in
	the beginning of data interface region.
	
	It obtains buffer information from the control section of data interface
	region.
	
     e) to make actual transfer it uses interface specific protocol
     
2) User-space interface: control interface protocol

control interfaces and /devfs/multimedia/devices are character based devices.
The negotiation, command issue and query is conducted in the following fashion:
Each request/query takes one line and is terminated by '\n'. The request
consists of status character and several fields. Examples:

a)
?PCI_ID\n

b)
 REPORT_VSYNC=1\n
(space)

c)
?FRAME_RATE,WIDTH=640,HEIGHT=480\n

d)
+\n

e)
-PCI_ID=INVALID\n

f)
:VSYNC=4\n


       Status character        Description
------------------------------------------------------------------------------
             ?                 Perform query, no device state is changed
	     ' ' (space)       Execute. Parameters are set and/or action is 
	                       performed.
	     +                 Reply: query/command succeeded. Addition information
	     			may follow (esp. in the case of query)
	     -                 Reply: query/command failed. In particular this 
	                       is returned if certain fields are not recognized
			       by the driver. In this case the fields are always
			       appended as in e) (for example this could be
			       returned for a USB device).
	     :                 Indicates out-of-sequence reply. Normally each
	      			query/command is responded to with a + or -
				reply. However, sometimes that driver needs to
				report a state change (i.e. in example f it
				reports that 4 interrupts occured since
				last report) without propmting of user-space
				application.
				
/devfs/multimedia/devices format

Each line represents one device or subsystem of a device. The following fields
are required:

     DEVICE_ID                  an identifier specific to this device
                                it should be the same if the device is
				plugged into a different port or a different
				computer
     LOCATION_ID                an identifier specific to the location of the
                                device. All devices plugged into the same
				port should have the same LOCATION_ID.
				Example: PCI_ID for PCI cards
     INSTANCE_ID                an identifier specific to the device
     			        it is required to be unique for each device
				in the system
All *_ID fields should be composed by attaching different strings with "-"
so that the user can use pattern matching ala X fonts.

     The following fields are optional:
     
      CONTROL                    specifies the number of the control 
                                 interface used for this device
      SUBSYSTEM_OF               specifies INSTANCE_ID of the device this one
                                 is a subsystem of. Useful for devices that
				 provide more then one hardware interface.

To: video4linux-list@xxxxxxxxxx 
Subject: Re: [RFC] alternative kernel multimedia API 
From: Justin Schoeman < justin@xxxxxxxxxxxxxxxxxxxx> 
Date: Fri, 26 Oct 2001 08:54:14 +0200 
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx 
Organization: University of Pretoria 
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.2) Gecko/20010704 

volodya@xxxxxxxxxxxxxx wrote:


> After looking at and working with Xv, v4l and v4l2 I became somewhat
> dissatisfied with the current state of affairs. I have attached a

> description of the API that would make (at least) me much happier. 
> I would very much appreciate comments from interested people..

>                           thanks !

>                               Vladimir Dergachev





> ------------------------------------------------------------------------

>               Kernel multimedia architecture

>         **     the latest version can be obtained from     **
	
> 	http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/gatos/km/km.rfc.txt

> 0) Motivation

>     v4l, v4l2 and Xv are all suffering from the same problem: attempt to fit 
>     existing multimedia devices into a fixed scheme. The use of pre-defined 
>     structs to describe parameters of device is inherently wrong because
>     these parameters vary widely. This leads to either bloating of the control
>     structures with parameters used only by few devices, proliferation of
>     device-specific ioctl and/or struct versioning. This also makes it increasingly
>     hard to implement support for new parameters.

>     The solution, IMO, is to move away from hard-coded models of multimedia
>     devices and instead allow greater flexibility to driver developers by

> providing _symbolic_ interface. 

...


I don't know - I don't see anything in the interface design that can do anything more 
than v4l2. In fact, for high speed capturing I can see a number of possible catches. 


I would like to see a detailed explanation of how the capabilities differ from those 
of v4l2. 


(I also doubt that an interface this complex would ever make it into the kernel.) 

-justin

To: video4linux-list@xxxxxxxxxx 
Subject: Re: [RFC] alternative kernel multimedia API 
From: Gerd Knorr <kraxel@xxxxxxxxxxx> 
Date: 26 Oct 2001 10:17:05 GMT 
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx 
Newsgroups: lists.linux.video4linux 
Organization: SuSE Labs, Außenstelle Berlin 
User-agent: slrn/0.9.7.1 (Linux) 

>  I don't know - I don't see anything in the interface design that can do 
>  anything more than v4l2.  In fact, for high speed capturing I can see a 
>  number of possible catches.

I agree.  The contingous capture basically removes all syncronisation
between driver and application and thus leads to plenty of race
conditions and other headage.  Applications can't stall capture if they
can't keep up, instead old (maybe not processed yet) data gets
overwritten.

And if you want something generic to control devices, the v4l2 controls
allow you to do that.

Also note that devfs is still experimental, I don't like the idea to
build something on top of devfs.  /me stopped using devfs recently btw.

  Gerd

-- 
Netscape is unable to locate the server localhost:8000.
Please check the server name and try again.

To: video4linux-list@xxxxxxxxxx 
Subject: Re: [RFC] alternative kernel multimedia API 
From: volodya@xxxxxxxxxxxxxx 
Date: Fri, 26 Oct 2001 07:56:16 -0400 (EDT) 
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx 
In-reply-to: <3BD90896.6090909@xxxxxxxxxxxxxxxxxxxx> 

On Fri, 26 Oct 2001, Justin Schoeman wrote:

> volodya@xxxxxxxxxxxxxx wrote:
> 
> >  After looking at and working with Xv, v4l and v4l2 I became somewhat
> > dissatisfied with the current state of affairs. I have attached a
> > description of the API that would make (at least) me much happier. 
> > 
> >  I would very much appreciate comments from interested people..
> > 
> >                            thanks !
> > 
> >                                Vladimir Dergachev
> > 
> > 
> > 
> > 
> > 
> > ------------------------------------------------------------------------
> > 
> >               Kernel multimedia architecture
> > 
> >         **     the latest version can be obtained from     **
> > 	
> > 	http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/gatos/km/km.rfc.txt
> > 
> > 0) Motivation
> >     v4l, v4l2 and Xv are all suffering from the same problem: attempt to fit 
> >     existing multimedia devices into a fixed scheme. The use of pre-defined
> >     structs to describe parameters of device is inherently wrong because
> >     these parameters vary widely. This leads to either bloating of the control
> >     structures with parameters used only by few devices, proliferation of
> >     device-specific ioctl and/or struct versioning. This also makes it increasingly
> >     hard to implement support for new parameters.
> > 
> >     The solution, IMO, is to move away from hard-coded models of multimedia
> >     devices and instead allow greater flexibility to driver developers by
> >     providing _symbolic_ interface. 
> 
> ...
> 
> I don't know - I don't see anything in the interface design that can do 
> anything more than v4l2.  In fact, for high speed capturing I can see a 
> number of possible catches.
> 
> I would like to see a detailed explanation of how the capabilities 
> differ from those of v4l2.

The major difference is in increased flexibility and stability of
kernel-user space interface.

On the application side:
       * ability to utilize advanced driver features without explicit
         knowledge of their behaviour
       * wider compatibility without the need for recompile
On the kernel side:
       * elimination of interfaced specific headers
On the driver side:
       * be compatible with a wide range of applications
       * introduce support for new features without the need to modify
         kernel interfaces

The last point is the very important in my opinion. For example, in
current specification, v4l2 has no support for TV-out in graphics cards.
It has no support for setting complex parameters like gamma tables. 
Using memory-mapped buffers requires device specific ioctls. 

The goal is to create an interface that does not rely on structures
defined in kernel headers for communication. 

> 
> (I also doubt that an interface this complex would ever make it into the 
> kernel.)

It is not very complex. It is  much simpler than software modem driver,
for example.

Ultimately these questions are best answered with a sample implementation.
Would you have any other concerns/suggestions ?

                            Vladimir Dergachev

> 
> -justin
> 
> 
> 
> _______________________________________________
> Video4linux-list mailing list
> Video4linux-list@xxxxxxxxxx
> https://listman.redhat.com/mailman/listinfo/video4linux-list
> 

To: video4linux-list@xxxxxxxxxx 
Subject: Re: [RFC] alternative kernel multimedia API 
From: Alan Cox <alan@xxxxxxxxxx> 
Date: Fri, 26 Oct 2001 08:22:27 -0400 (EDT) 
Delivered-to: video4linux-list@xxxxxxxxxxxxxxxxxx 
In-reply-to: <3BD90896.6090909@xxxxxxxxxxxxxxxxxxxx> 
from "Justin Schoeman" at Oct 26, 2001 08:54:14 AM 

I do it when I get time - it really is erratic updates not a caching bug 8)

			        About USENET

USENET (Users’ Network) was a bulletin board shared among many computer
systems around the world. USENET was a logical network, sitting on top
of several physical networks, among them UUCP, BLICN, BERKNET, X.25, and
the ARPANET. Sites on USENET included many universities, private companies
and research organizations. See USENET Archives.

		       SCO Files Lawsuit Against IBM

March 7, 2003 - The SCO Group filed legal action against IBM in the State 
Court of Utah for trade secrets misappropriation, tortious interference, 
unfair competition and breach of contract. The complaint alleges that IBM 
made concentrated efforts to improperly destroy the economic value of 
UNIX, particularly UNIX on Intel, to benefit IBM's Linux services 
business. See SCO vs IBM.

The materials and information included in this website may only be used
for purposes such as criticism, review, private study, scholarship, or
research.

Electronic mail:			       WorldWideWeb:
   tech-insider@outlook.com			  http://tech-insider.org/