Difference between revisions of "CM-T3730: Linux: Camera"

From Compulab Mediawiki
Jump to: navigation, search
(Camera ISP data paths)
(Hardware Setup)
Line 12: Line 12:
 
OMAP ISP subsystem is supported by Linux OMAP3 Camera-ISP driver.
 
OMAP ISP subsystem is supported by Linux OMAP3 Camera-ISP driver.
 
In addition, any camera sensor requires appropriate sensor driver to be available.
 
In addition, any camera sensor requires appropriate sensor driver to be available.
The OMAP3 Camera-ISP driver is a V4L2 framework driver that makes use of the Media Controller framework to configure the media hardware (camera).
+
The OMAP3 Camera-ISP driver is a V4L2 framework driver that makes use of the Media Controller framework to configure the media hardware.
Examples in this article are using an Aptina (Micron) MT9T031 3MP sensor.
+
Examples in this article are using the Aptina (Micron) MT9T031 3MP sensor and an NTSC analog composite camera.
  
 
== Hardware Setup ==
 
== Hardware Setup ==
 +
USB camera does not require any special hardware setup. It can be connected to any SB-T35 USB Host port.<br>
 +
MT9T031 camera sensor (Micron MT9T031C12STCH demo head-board) is connected through the VIP-ADP to the OMAP camera ISP module.
 +
Analog composite NTSC camera is connected with TVP5151 Video Decoder on the VIP-ADP board through the RCA composite input port.<br>
 +
VIP-ADP information and connection details are available in the [[Video Input Port Adapter Board]] article.
  
USB camera does not require any special hardware setup. It can be connected to any SB-T35 USB port.
+
In this acticle MT9T031 sensor and TVP5151 decoder are referenced as '''ISP camera devices'''.
  
ISP camera sensor (MT9T031) is connected with the Micron MT9T031C12STCH ES demo head-board.
+
Video streaming to display examples provided in this article were tested with 1024x768 DVI monitor.
Follow the instruction in [[Video Input Port Adapter Board]] article to connect the sensor to CM-T3730.
 
  
 
== Linux kernel ==
 
== Linux kernel ==
Line 68: Line 71:
 
=== USB Camera configuration ===
 
=== USB Camera configuration ===
  
* Enable "OMAP2/OMAP3 V4L2-Display driver" and "V4L USB devices" in the "Video capture adapters" submenu:
+
* Enable "V4L USB devices" in the "Video capture adapters" submenu:
 
   ┌──────────────────────────── Video capture adapters ─────────────────────────────┐
 
   ┌──────────────────────────── Video capture adapters ─────────────────────────────┐
 
   │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │   
 
   │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │   
Line 77: Line 80:
 
   │ │    < >  Virtual Video Driver                                              │ │   
 
   │ │    < >  Virtual Video Driver                                              │ │   
 
   │ │    < >  VPFE Video Capture Driver                                          │ │   
 
   │ │    < >  VPFE Video Capture Driver                                          │ │   
   │ │    '''<*>  OMAP2/OMAP3 V4L2-Display driver'''                                   │ │   
+
   │ │    < >  OMAP2/OMAP3 V4L2-Display driver                                    │ │   
 
   │ │    < >  CPiA2 Video For Linux                                              │ │   
 
   │ │    < >  CPiA2 Video For Linux                                              │ │   
 
   │ │    < >  Support for timberdale Video In/LogiWIN                            │ │   
 
   │ │    < >  Support for timberdale Video In/LogiWIN                            │ │   
Line 147: Line 150:
 
   └─────────────────────────────────────────────────────────────────────────────────┘   
 
   └─────────────────────────────────────────────────────────────────────────────────┘   
  
* Enable "Aptina MT9T001 support" in the "Encoders, decoders, sensors and other helper chips" submenu:
+
* Enable "Texas Instruments TVP5150 video decoder" and "Aptina MT9T001 support" in the "Encoders, decoders, sensors and other helper chips" submenu:
 
   ┌────────────── Encoders, decoders, sensors and other helper chips ───────────────┐
 
   ┌────────────── Encoders, decoders, sensors and other helper chips ───────────────┐
 
   │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │   
 
   │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │   
 +
  │ │    < > Texas Instruments TVP514x video decoder                              │ │ 
 +
  │ │    '''<*> Texas Instruments TVP5150 video decoder'''                              │ │ 
 +
  │ │    < > Texas Instruments TVP7002 video decoder                              │ │ 
 +
  │ │    < > vpx3220a, vpx3216b & vpx3214c video decoders                        │ │ 
 +
  │ │        *** Video and audio decoders ***                                    │ │ 
 +
  │ │    < > Philips SAA7171/3/4 audio/video decoders                            │ │ 
 +
  │ │    < > Conexant CX2584x audio/video decoders                                │ │ 
 +
  │ │        *** MPEG video encoders ***                                          │ │ 
 +
  │ │    < > Conexant CX2341x MPEG encoders                                      │ │ 
 +
  │ │        *** Video encoders ***                                              │ │ 
 +
  │ │    < > Philips SAA7127/9 digital video encoders                            │ │ 
 +
  │ │    < > Philips SAA7185 video encoder                                        │ │ 
 +
  │ │    < > Analog Devices ADV7170 video encoder                                │ │ 
 +
  │ │    < > Analog Devices ADV7175 video encoder                                │ │ 
 
   │ │    < > ADV7343 video encoder                                                │ │   
 
   │ │    < > ADV7343 video encoder                                                │ │   
 
   │ │    < > AK8813/AK8814 video encoders                                        │ │   
 
   │ │    < > AK8813/AK8814 video encoders                                        │ │   
Line 157: Line 174:
 
   │ │    '''<*> Aptina MT9T001 support'''                                              │ │   
 
   │ │    '''<*> Aptina MT9T001 support'''                                              │ │   
 
   │ │    < > Micron mt9v011 sensor support                                        │ │   
 
   │ │    < > Micron mt9v011 sensor support                                        │ │   
  │ │    < > Micron MT9V032 sensor support                                        │ │ 
 
  │ │    < > TCM825x camera sensor support                                        │ │ 
 
  │ │        *** Video improvement chips ***                                      │ │ 
 
  │ │    < > NEC Electronics uPD64031A Ghost Reduction                            │ │ 
 
  │ │    < > NEC Electronics uPD64083 3-Dimensional Y/C separation                │ │ 
 
 
   │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │   
 
   │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │   
 
   ├─────────────────────────────────────────────────────────────────────────────────┤   
 
   ├─────────────────────────────────────────────────────────────────────────────────┤   
 
   │                        <Select>    < Exit >    < Help >                        │   
 
   │                        <Select>    < Exit >    < Help >                        │   
 
   └─────────────────────────────────────────────────────────────────────────────────┘
 
   └─────────────────────────────────────────────────────────────────────────────────┘
 +
 +
 +
{{Note|While using the TVP5151 decoder, the {{parameter|nohlt}} kernel command line parameter must be used (appended to U-Boot bootargs variable). This prevents the WFI instruction from halting the processor in idle state.}}
  
 
== Setting up ISP Camera ==
 
== Setting up ISP Camera ==
Line 183: Line 198:
  
 
The {{cmd|media-ctl}} command line tool can be used to manipulate the Media Device through the Media Controller API.
 
The {{cmd|media-ctl}} command line tool can be used to manipulate the Media Device through the Media Controller API.
The {{cmd|media-ctl}} tool can be found on [http://www.angstrom-distribution.org/ The Ångström Distribution] official website.
 
 
This tool allows to enumerate media entities and their pads, set/get and enumerate links between pads of different entities, define pads data format.
 
This tool allows to enumerate media entities and their pads, set/get and enumerate links between pads of different entities, define pads data format.
 
+
{{cmd|media-ctl}} package is included in the CM-T3730 Angstrom Linux root filesystem image.
==== Installation ====
 
Download and install required packages:
 
<pre>
 
opkg install libc6_2.12-r28_armv7a.ipk media-ctl_0.0.1-r0_armv7a.ipk
 
</pre>
 
  
 
==== Explore Media Device ====
 
==== Explore Media Device ====
Line 199: Line 208:
 
</pre>
 
</pre>
  
The following output corresponds to entity number 16 which is the Camera sensor.
+
The following output corresponds to entity number 16 which is MT9T031 camera sensor in this example.
  
 
<pre>
 
<pre>
Line 227: Line 236:
 
Data paths (pipelines) inside the camera ISP hardware depend on the image format sourced by the sensor (RAW Bayer, YUV4:2:2, JPEG,...).
 
Data paths (pipelines) inside the camera ISP hardware depend on the image format sourced by the sensor (RAW Bayer, YUV4:2:2, JPEG,...).
 
ISP driver supports different pipelines. To setup the Media Device links and formats properly, please refer to the Camera ISP section in the DM3730 technical reference manual.
 
ISP driver supports different pipelines. To setup the Media Device links and formats properly, please refer to the Camera ISP section in the DM3730 technical reference manual.
 +
 +
===== RAW Data Path =====
 
The following chart shows various pipelines relevant to MT9T031 sensor that produces RAW Bayer 10bit image data.
 
The following chart shows various pipelines relevant to MT9T031 sensor that produces RAW Bayer 10bit image data.
  
[[Image:Isp_data_path.jpg|450px|center|Camera ISP Data Path]]
+
[[Image:Isp_data_path.jpg|450px|center|Camera ISP RAW Data Path]]
RAW data are processed by the CCDC module and are directly pipelined to the Preview engine(1). Another way is to output directly from the CCDC to memory (C). In the Preview block, the format is converted from RAW data to YUV4:2:2. The data can be output to memory (4) or pipelined to the Resizer (2). The rescaled YUV4:2:2 image is finally stored in memory (3).
+
RAW data are processed by the CCDC module and are directly pipelined to the Preview engine (1). Another way is to output directly from the CCDC to memory (C). In the Preview block, the format is converted from RAW data to YUV4:2:2. The data can be output to memory (4) or pipelined to the Resizer (2). The rescaled YUV4:2:2 image is finally stored in memory (3).
  
Following pipelines for image capture are valid with MT9T031 sensor:
+
Following pipelines for image capturing are valid with MT9T031 sensor:
  
 
* Sensor->CCDC->Memory - (C)
 
* Sensor->CCDC->Memory - (C)
 
* Sensor->CCDC->Preview->Memory - (1), (4)
 
* Sensor->CCDC->Preview->Memory - (1), (4)
 
* Sensor->CCDC->Preview->Resizer->Memory - (1), (2), (3)
 
* Sensor->CCDC->Preview->Resizer->Memory - (1), (2), (3)
 +
 +
===== YUV4:2:2 Data Path =====
 +
 +
The following chart shows various pipelines relevant to TVP5151 decoder.
 +
The TVP5151 decoder can be configured for output the data in ''8-bit YUV4:2:2 with discrete sync output'' format or ''8-bit ITU−R BT.656 with embedded sync output'' format (which also implements 4:2:2 sampling).
 +
The modules in red in the chart below are not used when the format is YUV4:2:2.
 +
 +
[[Image:Isp_data_path_yuv.jpg|450px|center|Camera ISP YUV4:2:2 Data Path]]
 +
 +
When the sensor-output format is YUV4:2:2, the CCDC block is pipelined to the resizer (1) or directly to memory (C). The rescaled YUV4:2:2 images are finally stored in memory (2).
 +
 +
Following pipelines for image capturing are valid with TVP5151 decoder:
 +
 +
* Sensor->CCDC->Memory - (C)
 +
* Sensor->CCDC->Resizer->Memory - (1), (2)
  
 
==== Setting up ISP pipeline ====
 
==== Setting up ISP pipeline ====
Line 242: Line 268:
 
This section describes how to translate valid ISP pipelines into links between Media Entities using {{cmd|media-ctl}} utility.
 
This section describes how to translate valid ISP pipelines into links between Media Entities using {{cmd|media-ctl}} utility.
  
Following example demonstrates setting ISP pipeline to '''Sensor->CCDC->Preview->Memory''' and configuring entity pads formats.
+
Following example demonstrates setting ISP pipeline for MT9T031 sensor to '''Sensor->CCDC->Preview->Memory''' and configuring entity pads formats.
  
 
Set up ISP pipeline:
 
Set up ISP pipeline:
Line 303: Line 329:
  
 
=== ISP Camera ===
 
=== ISP Camera ===
Check that MT9T0031 camera driver and ISP module has been recognized correctly through the kernel debug messages log (using the {{cmd|dmesg}} command):
+
 
 +
VIP-ADP can be configured to communicate with a single camera device at a time, either digital camera sensor or analog composite camera.
 +
 
 +
Depending on VIP-ADP configuration settings check that camera device and ISP module have been recognized correctly through the kernel debug messages log (using the {{cmd|dmesg}} command):
 +
 
 +
* '''OMAP ISP'''
 
<pre>
 
<pre>
 
Linux media interface: v0.10
 
Linux media interface: v0.10
Line 309: Line 340:
 
omap3isp omap3isp: Revision 15.0 found
 
omap3isp omap3isp: Revision 15.0 found
 
omap-iommu omap-iommu.0: isp: version 1.1
 
omap-iommu omap-iommu.0: isp: version 1.1
 +
...
 +
</pre>
 +
 +
* '''TVP5151'''
 +
<pre>
 +
...
 +
tvp5150 3-005c: chip found at address 0x5c (OMAP I2C adapter)
 +
tvp5150 3-005c: *** tvp5151 chip detected.
 +
tvp5150 3-005c: *** Rom ver is 1.0
 +
</pre>
 +
 +
* '''MT9T0031'''
 +
<pre>
 +
...
 
mt9t001 3-005d: Probing MT9T001 at address 0x5d
 
mt9t001 3-005d: Probing MT9T001 at address 0x5d
 
mt9t001 3-005d: MT9T001 detected at address 0x5d
 
mt9t001 3-005d: MT9T001 detected at address 0x5d
Line 320: Line 365:
 
</pre>
 
</pre>
  
Each listed Media Entity should have an associated V4L2 video device. Pay special attention to the camera sensor entity. It must be present in the list.
+
Each listed Media Entity should have an associated V4L2 video device. Pay special attention to the camera device entity. It must be present in the list.
  
 
== Playback and Streaming tools ==
 
== Playback and Streaming tools ==
Line 331: Line 376:
 
=== yavta ===
 
=== yavta ===
  
{{cmd|yavta}} (“Yet Another V4L2 Test Application”) is a data capture tool that allows to enumerate and set V4L2 device controls like brightness, exposure, autofocus etc. This is a generic tool that can be applied to any V4L2 compliant device.  
+
{{cmd|yavta}} (“Yet Another V4L2 Test Application”) is a data capture tool that allows to enumerate and set V4L2 device controls like brightness, exposure, autofocus etc. This is a generic tool that can be applied to any V4L2 compliant device. {{cmd|yavta}} package is included in the CM-T3730 Angstrom Linux root filesystem image.  
 
 
==== Installation ====
 
 
 
{{cmd|yavta}} package can be found  on [http://www.angstrom-distribution.org/ The Ångström Distribution] official website.
 
 
 
Download and install required packages:
 
<pre>
 
opkg install libc6_2.12-r28_armv7a.ipk yavta_0.0-r0_armv7a.ipk
 
</pre>
 
  
 
==== Configure V4L2 device ====
 
==== Configure V4L2 device ====
Line 346: Line 382:
 
V4L2 device typically has a number of configurable controls. {{cmd|yavta}} allows the controls enumeration and configuration.
 
V4L2 device typically has a number of configurable controls. {{cmd|yavta}} allows the controls enumeration and configuration.
  
Run the following command to print the list of all available controls of the Camera sensor exposed through {{filename|/dev/v4l-subdev8}}.
+
Run the following command to print the list of all available controls of the Camera sensor exposed through {{filename|/dev/v4l-subdev8}}. Output below corresponds to MT9T031 sensor.
  
 
<pre>
 
<pre>
Line 395: Line 431:
  
 
Capturing stills images from ISP Camera can be done using several ISP pipelines. Examples below show how to setup appropriate pipeline, set pads format and capture an image using {{cmd|yavta}}.
 
Capturing stills images from ISP Camera can be done using several ISP pipelines. Examples below show how to setup appropriate pipeline, set pads format and capture an image using {{cmd|yavta}}.
 +
 +
===== MT9T031 =====
  
 
Run the following set of commands to capture an image from '''Sensor->CCDC->Memory''' pipeline (capture RAW Bayer 10bit image from CCDC):
 
Run the following set of commands to capture an image from '''Sensor->CCDC->Memory''' pipeline (capture RAW Bayer 10bit image from CCDC):
Line 424: Line 462:
 
</pre>
 
</pre>
 
<pre>
 
<pre>
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024], \
+
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024], \
 
               "OMAP3 ISP resizer":1 [UYVY 1024x768]'
 
               "OMAP3 ISP resizer":1 [UYVY 1024x768]'
 
</pre>
 
</pre>
 
<pre>
 
<pre>
 
yavta -f UYVY -s 1024x768 --capture=1 --file=image /dev/video6
 
yavta -f UYVY -s 1024x768 --capture=1 --file=image /dev/video6
 +
</pre>
 +
 +
===== TVP5151 =====
 +
 +
Run the following set of commands to capture an image from '''Sensor->CCDC->Memory''' pipeline (capture YUV4:2:2 image from CCDC):
 +
 +
<pre>
 +
media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
 +
</pre>
 +
<pre>
 +
yavta -f UYVY -s 720x524 --capture=1 --file=image /dev/video2
 
</pre>
 
</pre>
  
Line 446: Line 498:
  
 
==== Displaying captured images ====
 
==== Displaying captured images ====
[[Image:Isp_cam_snapshoot_scaled.jpg|thumb|right|250px|2048x1536 ISP Camera RAW Bayer 10bit image snapshot, processed by UFRaw, scaled to 1024x768 and converted to JPG]]
+
[[Image:Isp_cam_snapshoot_scaled.jpg|thumb|right|250px|2048x1536 ISP Camera RAW Bayer 10bit image snapshot, processed by UFRaw, scaled to 640x480 and converted to JPG]]
  
 
MT9T031 camera sensor produces RAW Bayer 10bit image data. The raw data can be captured directly from ISP CCDC or after processing by Preview/Resizer in YUV4:2:2 format.
 
MT9T031 camera sensor produces RAW Bayer 10bit image data. The raw data can be captured directly from ISP CCDC or after processing by Preview/Resizer in YUV4:2:2 format.
Line 455: Line 507:
 
</pre>
 
</pre>
  
YUV4:2:2 image can be viewed by {{cmd|display}} tool from the ImageMagic application package. For instance, in order to view the image captured from Preview module in the example above run the following set of commands:
+
YUV4:2:2 image can be viewed by {{cmd|display}} tool from the ImageMagic application package. For instance, in order to view the image captured from MT9T031 sensor through Preview module in the example above run the following set of commands:
 
<pre>
 
<pre>
 
mv image-000000.bin image-000000.uyvy
 
mv image-000000.bin image-000000.uyvy
Line 462: Line 514:
 
display -size 2031x1527 -colorspace rgb image-000000.uyvy
 
display -size 2031x1527 -colorspace rgb image-000000.uyvy
 
</pre>
 
</pre>
 +
 +
{{Note| Image captured through TVP5151 decoder from camera connected to composite input of VIP-ADP module requires deinterlacing.}}
 
<br style="clear: both" />
 
<br style="clear: both" />
  
Line 470: Line 524:
  
 
==== Output to display ====
 
==== Output to display ====
 
+
===== USB Camera =====
 
The following command can be used to stream the USB camera output to the display using default configuration settings.
 
The following command can be used to stream the USB camera output to the display using default configuration settings.
 
<pre>
 
<pre>
Line 530: Line 584:
 
|
 
|
 
{|cellpadding="0" cellspacing="0" width="100%"
 
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=screenshot}}
+
|width="30px"| || {{parameter|screenshot}}
 
|}
 
|}
 
|| Allows acquiring screenshots of the movie. Make sure that the {{parameter|screenshot}} filter is the last filter, so the exact picture on the display will be saved.
 
|| Allows acquiring screenshots of the movie. Make sure that the {{parameter|screenshot}} filter is the last filter, so the exact picture on the display will be saved.
 +
|-
 +
|
 +
{|cellpadding="0" cellspacing="0" width="100%"
 +
|width="30px"| || {{parameter|1=crop=720:480:0:44}}
 +
|}
 +
|| Crops the 720x480 part of the image from position (0,44) and discards the rest.
 +
|-
 +
|
 +
{|cellpadding="0" cellspacing="0" width="100%"
 +
|width="30px"| || {{parameter|1=pp=li}}
 +
|}
 +
|| Enables linear interpolating deinterlacing sub-filter.
 
|-
 
|-
 
|{{parameter|-vo}} || Specifies MPlayer video output driver.
 
|{{parameter|-vo}} || Specifies MPlayer video output driver.
Line 551: Line 617:
 
{{Note| To start X Window application (e.g. MPlayer) from your serial console, the {{parameter|DISPLAY}} environment variable must be set.}}
 
{{Note| To start X Window application (e.g. MPlayer) from your serial console, the {{parameter|DISPLAY}} environment variable must be set.}}
  
Additional example for streaming USB camera output to a separate window in the X Window system:
+
*Additional example for streaming USB camera output to a separate window in the X Window system:
 
<pre>
 
<pre>
 
mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -vo xv
 
mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -vo xv
 
</pre>
 
</pre>
  
Following set of commands will set up ISP pipeline to '''Sensor->Preview->Resizer->Memory''' and stream 1024x768 video to OMAP framebuffer:
+
===== MT9T031 =====
 +
Following set of commands will set up ISP pipeline to '''Sensor->CCDC->Preview->Resizer->Memory''' and stream 1024x768 video from MT9T031 sensor to OMAP framebuffer:
 
<pre>
 
<pre>
 
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
 
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
Line 566: Line 633:
 
<pre>
 
<pre>
 
mplayer tv:// -tv driver=v4l2:device=/dev/video6:outfmt=uyvy:width=1024:height=768 -vo omapfb
 
mplayer tv:// -tv driver=v4l2:device=/dev/video6:outfmt=uyvy:width=1024:height=768 -vo omapfb
 +
</pre>
 +
 +
===== TVP5151 =====
 +
Following set of commands will set up ISP pipeline to '''Sensor->CCDC->Memory''' and stream 720x524 video from TVP5151 decoder to OMAP framebuffer:
 +
<pre>
 +
media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
 +
</pre>
 +
<pre>
 +
mplayer tv:// -tv driver=v4l2:device=/dev/video2:outfmt=uyvy:width=720:height=524 -vo omapfb -vf crop=720:480:0:44,pp=li
 
</pre>
 
</pre>
  
Line 571: Line 650:
  
 
[[Image:Usb_camera_screenshoot.jpg|thumb|right|200px|640x480 USB Camera screenshot converted to jpg]]
 
[[Image:Usb_camera_screenshoot.jpg|thumb|right|200px|640x480 USB Camera screenshot converted to jpg]]
 +
[[Image:Tvp_snapshot_dinter.jpg|thumb|right|200px|720x480 NTSC car reverse camera screenshot converted to jpg]]
 +
 +
===== USB Camera =====
 
To take snapshots from USB Camera, run the MPlayer with the following command line:
 
To take snapshots from USB Camera, run the MPlayer with the following command line:
 
<pre>
 
<pre>
 
mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -vf screenshot -vo omapfb
 
mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -vf screenshot -vo omapfb
 
</pre>
 
</pre>
Once there is an output streamed on the display, press {{parameter|s}} to take a snapshot. Each snapshot will be saved in your current directory as {{filename|shotXXXX.png}}.
 
  
 +
===== MT9T031 =====
 +
ISP Camera snapshoot from MT9T031 sensor could be obtained by the following set of commands:
 +
<pre>
 +
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
 +
                "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024],  "OMAP3 ISP resizer":1 [UYVY 1024x768]'
 +
</pre>
 +
<pre>
 +
mplayer tv:// -tv driver=v4l2:device=/dev/video6:outfmt=uyvy:width=1024:height=768 -vf screenshot -vo omapfb
 +
</pre>
  
ISP Camera snapshoot could be obtained by setting ISP pipeline as described in the [[#Output to display|previous section]] and running the following command:  
+
===== TVP5151 =====
 +
ISP Camera snapshoot from TVP5151 decoder could be obtained by the following set of commands:  
 
<pre>
 
<pre>
mplayer tv:// -tv driver=v4l2:device=/dev/video6:outfmt=uyvy:width=1024:height=768 -vf screenshot -vo omapfb
+
media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
 +
</pre>
 +
<pre>
 +
mplayer tv:// -tv driver=v4l2:device=/dev/video2:outfmt=uyvy:width=720:height=524 -vf crop=720:480:0:44,pp=li,screenshot -vo omapfb
 
</pre>
 
</pre>
 +
 +
Once there is an output streamed on the display, press {{parameter|s}} to take a snapshot. Each snapshot will be saved in your current directory as {{filename|shotXXXX.png}}.
 
<br style="clear: both" />
 
<br style="clear: both" />
  
Line 593: Line 695:
 
GStreamer is a pipeline-based multimedia framework that allows you to create a variety of media-handling software components.
 
GStreamer is a pipeline-based multimedia framework that allows you to create a variety of media-handling software components.
 
This includes simple audio playback, audio and video playback, recording, streaming, and editing.
 
This includes simple audio playback, audio and video playback, recording, streaming, and editing.
The pipeline design allows you to easily write any type of streaming multimedia application.
+
The pipeline design allows you to easily write any type of streaming multimedia application.
 
GStreamers capabilities are extended through plugins. Several simple media manipulations are demonstrated below.
 
GStreamers capabilities are extended through plugins. Several simple media manipulations are demonstrated below.
 
+
GStreamer package and required plugins are included in the CM-T3730 Angstrom Linux root filesystem image.
==== Installation ====
 
For the GStreamer to function properly, there are several packages that need to be installed. Those can be found on the [http://www.angstrom-distribution.org/ The Ångström Distribution] official website.
 
 
 
* To install the basic GStreamer set, use the following command:
 
<pre>
 
opkg install libxml2_2.7.8-r9.1.6_armv7a.ipk libgcc1_4.5-r47+svnr184907_armv7a.ipk libc6_2.12-r28_armv7a.ipk libffi5_3.0.10-r0_armv7a.ipk \
 
            libz1_1.2.6-r1_armv7a.ipk libglib-2.0-0_2.30.3-r0_armv7a.ipk gstreamer_0.10.36-r0_armv7a.ipk libxcb1_1.8.1-r0_armv7a.ipk \
 
            libxv1_1.0.7-r0_armv7a.ipk libuuid1_2.21-r5_armv7a.ipk libudev0_182-r0_armv7a.ipk libgudev-1.0-0_182-r0_armv7a.ipk \
 
            libgstinterfaces-0.10-0_0.10.36-r1_armv7a.ipk libgstvideo-0.10-0_0.10.36-r1_armv7a.ipk --force-overwrite
 
</pre>
 
 
 
* For GStreamer to support Video4Linux2 and several plugins for manipulating the output stream, the following needs to be installed:
 
<pre>
 
opkg install gst-plugins-good-video4linux2_0.10.31-r0_armv7a.ipk gst-plugin-videorate_0.10.35-r1_armv7a.ipk \
 
            gst-plugin-ffmpegcolorspace_0.10.35-r1_armv7a.ipk gst-plugin-ximagesink_0.10.35-r1_armv7a.ipk
 
</pre>
 
 
 
* To support encoding with Theora codec (required if you want to save the encoded video to a file), the following needs to be installed:
 
<pre>
 
opkg install libogg0_1.3.0-r0_armv7a.ipk libtheora_1.1.1-r1_armv7a.ipk libgsttag-0.10-0_0.10.36-r1_armv7a.ipk gst-plugin-theora_0.10.35-r1_armv7a.ipk \
 
            libgstpbutils-0.10-0_0.10.36-r1_armv7a.ipk libgstaudio-0.10-0_0.10.36-r1_armv7a.ipk libgstriff-0.10-0_0.10.36-r1_armv7a.ipk \
 
            gst-plugin-ogg_0.10.35-r1_armv7a.ipk --force-overwrite
 
</pre>
 
  
 
==== Validation ====
 
==== Validation ====
Line 631: Line 710:
 
</pre>
 
</pre>
 
<br style="clear: both" />
 
<br style="clear: both" />
 +
 
==== Streaming to display ====
 
==== Streaming to display ====
  
Line 637: Line 717:
 
The below examples stream the video from the V4L2 video capture device to the OMAP frame buffer (sink) with several stream manipulations applied.
 
The below examples stream the video from the V4L2 video capture device to the OMAP frame buffer (sink) with several stream manipulations applied.
  
 +
===== USB Camera =====
 
Run the following command to stream the USB camera to display
 
Run the following command to stream the USB camera to display
 
<pre>
 
<pre>
Line 642: Line 723:
 
</pre>
 
</pre>
  
Set up ISP pipeline as described in [[CM-T3730: Linux: Camera#MPlayer|MPlayer]] section and run the following command to stream ISP Camera to display
+
===== MT9T031 =====
 +
Run the following set of commands to stream video from MT9T031 sensor to display
 +
<pre>
 +
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
 +
                "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024],  "OMAP3 ISP resizer":1 [UYVY 1024x768]'
 +
</pre>
 
<pre>
 
<pre>
 
gst-launch v4l2src device=/dev/video6 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=1024,height=768' ! omapdmaifbsink
 
gst-launch v4l2src device=/dev/video6 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=1024,height=768' ! omapdmaifbsink
 +
</pre>
 +
 +
===== TVP5151 =====
 +
Run the following set of commands to stream video from TVP5151 decoder to display
 +
<pre>
 +
media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
 +
</pre>
 +
<pre>
 +
gst-launch v4l2src device=/dev/video2 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=524' ! videocrop top=44 ! deinterlace mode=1 fields=1 ! omapdmaifbsink
 
</pre>
 
</pre>
  
Line 665: Line 766:
 
|-
 
|-
 
|{{parameter|1=video/x-raw-yuv,format=(fourcc)YUY2,width=640,height=480}} || Link the source to a ''capability'', which is defined by a mime type and a few optional properties. The properties are: video and resolution.  
 
|{{parameter|1=video/x-raw-yuv,format=(fourcc)YUY2,width=640,height=480}} || Link the source to a ''capability'', which is defined by a mime type and a few optional properties. The properties are: video and resolution.  
 +
|-
 +
|{{parameter|videocrop}} || Crops video frames.
 +
|-
 +
|
 +
{|cellpadding="0" cellspacing="0" width="100%"
 +
|width="30px"| || {{parameter|1=top=44}}
 +
|}
 +
|| Remove 44 pixels from the top of the image.
 +
|-
 +
|{{parameter|deinterlace}} || Deinterlace the video.
 +
|-
 +
|
 +
{|cellpadding="0" cellspacing="0" width="100%"
 +
|width="30px"| || {{parameter|1=mode=1}}
 +
|}
 +
|| Force deinterlacing.
 +
|-
 +
|
 +
{|cellpadding="0" cellspacing="0" width="100%"
 +
|width="30px"| || {{parameter|1=fields=1}}
 +
|}
 +
|| Use top fields only for deinterlacing.
 
|-
 
|-
 
|{{parameter|videorate}} || Use each frame of the source stream and feed it to the next element at the requested framerate.
 
|{{parameter|videorate}} || Use each frame of the source stream and feed it to the next element at the requested framerate.
Line 677: Line 800:
 
This section examples demonstrate how to encode and save the output stream to a file, using the Theora codec.
 
This section examples demonstrate how to encode and save the output stream to a file, using the Theora codec.
  
 +
===== USB Camera =====
 
Run the following command to save the USB Camera output stream to a file:
 
Run the following command to save the USB Camera output stream to a file:
 
<pre>
 
<pre>
Line 683: Line 807:
 
</pre>
 
</pre>
  
Run following set of commands to save the ISP Camera output stream to a file:
+
===== MT9T031 =====
 +
Run following set of commands to save the MT9T031 sensor output stream to a file:
  
 
<pre>
 
<pre>
Line 696: Line 821:
 
gst-launch v4l2src device=/dev/video6 ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! \
 
gst-launch v4l2src device=/dev/video6 ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! \
 
           queue ! ffmpegcolorspace ! theoraenc ! queue ! oggmux !  filesink location=media_file.ogg
 
           queue ! ffmpegcolorspace ! theoraenc ! queue ! oggmux !  filesink location=media_file.ogg
 +
</pre>
 +
 +
===== TVP5151 =====
 +
Run following set of commands to save the TVP5151 decoder output stream to a file:
 +
 +
<pre>
 +
media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
 +
</pre>
 +
<pre>
 +
gst-launch v4l2src device=/dev/video2 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=524' ! videocrop top=44 ! \
 +
          deinterlace mode=1 fields=1 ! videoscale ! 'video/x-raw-yuv,width=310,height=240' ! queue ! videorate ! \
 +
          'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! theoraenc ! queue ! oggmux ! filesink location=media_file.ogg
 
</pre>
 
</pre>
  
Line 706: Line 846:
 
|-
 
|-
 
|{{parameter|oggmux}} || Muxes a stream into an ogg container.
 
|{{parameter|oggmux}} || Muxes a stream into an ogg container.
 +
|-
 +
|{{parameter|videoscale}} || Resizes video frames.
 
|-
 
|-
 
|{{parameter|filesink}} || Writes the incoming data to a file.
 
|{{parameter|filesink}} || Writes the incoming data to a file.
Line 712: Line 854:
 
==== Streaming to a file and display ====
 
==== Streaming to a file and display ====
  
 +
===== USB Camera =====
 
Following command splits the source stream from USB Camera and links to the frame buffer and to an encoded file:
 
Following command splits the source stream from USB Camera and links to the frame buffer and to an encoded file:
  
Line 720: Line 863:
 
</pre>
 
</pre>
  
The same for ISP Camera using ISP pipeline settings from the [[#Streaming to a file|previous section]]:
+
===== MT9T031 =====
 +
<pre>
 +
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
 +
                "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 640x480], "OMAP3 ISP CCDC":2 [SGRBG10 640x480], "OMAP3 ISP preview":1 [UYVY 640x480],  "OMAP3 ISP resizer":1 [UYVY 320x240]'
 +
</pre>
 
<pre>
 
<pre>
 
gst-launch v4l2src device=/dev/video6 always-copy=false ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=320,height=240' ! tee name=t_video ! \
 
gst-launch v4l2src device=/dev/video6 always-copy=false ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=320,height=240' ! tee name=t_video ! \
 
           queue ! videorate ! ffmpegcolorspace ! omapdmaifbsink t_video. ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! \  
 
           queue ! videorate ! ffmpegcolorspace ! omapdmaifbsink t_video. ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! \  
 
           theoraenc ! queue ! oggmux ! filesink location=media_file.ogg
 
           theoraenc ! queue ! oggmux ! filesink location=media_file.ogg
 +
</pre>
 +
 +
===== TVP5151 =====
 +
<pre>
 +
media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
 +
</pre>
 +
<pre>
 +
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
 +
</pre>
 +
<pre>
 +
gst-launch v4l2src device=/dev/video2 always-copy=false ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=524' ! videocrop top=44 ! \
 +
          deinterlace mode=1 fields=1 ! videoscale ! 'video/x-raw-yuv, width=310,height=240' ! tee name=t_video ! queue ! omapdmaifbsink \
 +
          t_video. ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! queue ! ffmpegcolorspace ! theoraenc ! queue ! oggmux !  \
 +
          filesink location=media_file.ogg
 
</pre>
 
</pre>
  

Revision as of 06:16, 5 June 2012

Overview

CM-T3730 Linux supports various camera devices connected either to the USB subsystem or to the camera interface (CIF). This article describes both camera types setup, configuration and provides several usage examples.

Linux USB camera support is provided by USB Video Class (UVC) driver. All UVC compliant camera peripherals can be managed by the UVC driver. Linux also supports various non UVC cameras, but with some limitations. Therefore non UVC cameras are not covered in this article. Linux UVC driver implements the Video for Linux Two (V4L2) API. V4L2 is a suite of driver specifications for different types of video devices and video-related data. Examples in this article are using a AV301W-based camera.

The CM-T3730 module features camera image signal processor (ISP) that supports multiple formats and interfacing options for a wide variety of image sensors. OMAP ISP subsystem is supported by Linux OMAP3 Camera-ISP driver. In addition, any camera sensor requires appropriate sensor driver to be available. The OMAP3 Camera-ISP driver is a V4L2 framework driver that makes use of the Media Controller framework to configure the media hardware. Examples in this article are using the Aptina (Micron) MT9T031 3MP sensor and an NTSC analog composite camera.

Hardware Setup

USB camera does not require any special hardware setup. It can be connected to any SB-T35 USB Host port.
MT9T031 camera sensor (Micron MT9T031C12STCH demo head-board) is connected through the VIP-ADP to the OMAP camera ISP module. Analog composite NTSC camera is connected with TVP5151 Video Decoder on the VIP-ADP board through the RCA composite input port.
VIP-ADP information and connection details are available in the Video Input Port Adapter Board article.

In this acticle MT9T031 sensor and TVP5151 decoder are referenced as ISP camera devices.

Video streaming to display examples provided in this article were tested with 1024x768 DVI monitor.

Linux kernel

All the use cases and examples in this article were tested with the CM-T3730 Linux package (currently based on kernel v3.0) that can be obtained from CM-T3730 product webpage.

The following kernel configuration procedure is required:

Common media configuration

  • Enable "Multimedia support" in the "Device Drivers" submenu:
 ┌──────────────────────────────── Device Drivers ─────────────────────────────────┐
 │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │  
 │ │    [*] Watchdog Timer Support  --->                                         │ │  
 │ │        Sonics Silicon Backplane  --->                                       │ │  
 │ │        Broadcom specific AMBA  --->                                         │ │  
 │ │    -*- Multifunction device drivers  --->                                   │ │  
 │ │    -*- Voltage and Current Regulator Support  --->                          │ │  
 │ │    <*> Multimedia support  --->                                             │ │  
 │ │        Graphics support  --->                                               │ │  
 │ │    <*> Sound card support  --->                                             │ │  
 │ │    [*] HID Devices  --->                                                    │ │  
 │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │  
 ├─────────────────────────────────────────────────────────────────────────────────┤  
 │                        <Select>    < Exit >    < Help >                         │  
 └─────────────────────────────────────────────────────────────────────────────────┘  
  • Enable "Video For Linux" and "Video capture adapters" in the "Multimedia support" submenu:
 ┌────────────────────────────── Multimedia support ───────────────────────────────┐
 │ ┌─────────────────────────────────────────────────────────────────────────────┐ │  
 │ │    --- Multimedia support                                                   │ │  
 │ │          *** Multimedia core support ***                                    │ │  
 │ │    [ ]   Media Controller API (EXPERIMENTAL)                                │ │  
 │ │    <*>   Video For Linux                                                    │ │  
 │ │    < >   DVB for Linux                                                      │ │  
 │ │          *** Multimedia drivers ***                                         │ │  
 │ │    <*>   Remote Controller adapters  --->                                   │ │  
 │ │    [ ]   Load and attach frontend and tuner driver modules as needed        │ │  
 │ │    [*]   Customize analog and hybrid tuner modules to build                 │ │  
 │ │          Customize TV tuners  --->                                          │ │  
 │ │    [*]   Video capture adapters  --->                                       │ │  
 │ │    [ ]   Memory-to-memory multimedia devices  --->                          │ │  
 │ │    [*]   Radio Adapters  --->                                               │ │  
 │ └─────────────────────────────────────────────────────────────────────────────┘ │  
 ├─────────────────────────────────────────────────────────────────────────────────┤  
 │                        <Select>    < Exit >    < Help >                         │  
 └─────────────────────────────────────────────────────────────────────────────────┘

USB Camera configuration

  • Enable "V4L USB devices" in the "Video capture adapters" submenu:
 ┌──────────────────────────── Video capture adapters ─────────────────────────────┐
 │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │  
 │ │    [ ]   Enable old-style fixed minor ranges for video devices              │ │  
 │ │    [ ]   Autoselect pertinent encoders/decoders and other helper chips      │ │  
 │ │    <*>     I2C module for IR                                                │ │  
 │ │          Encoders, decoders, sensors and other helper chips  --->           │ │  
 │ │    < >   Virtual Video Driver                                               │ │  
 │ │    < >   VPFE Video Capture Driver                                          │ │  
 │ │    < >   OMAP2/OMAP3 V4L2-Display driver                                    │ │  
 │ │    < >   CPiA2 Video For Linux                                              │ │  
 │ │    < >   Support for timberdale Video In/LogiWIN                            │ │  
 │ │    < >   SR030PC30 VGA camera sensor support                                │ │  
 │ │    < >   NOON010PC30 CIF camera sensor support                              │ │  
 │ │    < >   SoC camera support                                                 │ │  
 │ │    [*]   V4L USB devices  --->                                              │ │  
 │ └─────────────────────────────────────────────────────────────────────────────┘ │  
 ├─────────────────────────────────────────────────────────────────────────────────┤  
 │                        <Select>    < Exit >    < Help >                         │  
 └─────────────────────────────────────────────────────────────────────────────────┘  
   
  • Enable "USB Video Class (UVC)" in "V4L USB devices" submenu:
 ┌──────────────────────────────── V4L USB devices ────────────────────────────────┐  
 │ ┌─────────────────────────────────────────────────────────────────────────────┐ │  
 │ │    --- V4L USB devices                                                      │ │  
 │ │    <*>   USB Video Class (UVC)                                              │ │  
 │ │    [*]     UVC input events device support                                  │ │  
 │ │    <M>   GSPCA based webcams  --->                                          │ │  
 │ │    < >   Hauppauge WinTV-PVR USB2 support                                   │ │  
 │ │    < >   Hauppauge HD PVR support                                           │ │  
 │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │  
 ├─────────────────────────────────────────────────────────────────────────────────┤  
 │                        <Select>    < Exit >    < Help >                         │  
 └─────────────────────────────────────────────────────────────────────────────────┘  

UVC camera and the driver do not need any special kernel command line parameters option.

ISP Camera configuration

  • Enable "Media Controller API" in the "Multimedia support" submenu:
 ┌────────────────────────────── Multimedia support ───────────────────────────────┐
 │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │  
 │ │          *** Multimedia core support ***                                    │ │  
 │ │    [*]   Media Controller API (EXPERIMENTAL)                                │ │  
 │ │    <*>   Video For Linux                                                    │ │  
 │ │    [*]     V4L2 sub-device userspace API (EXPERIMENTAL)                     │ │  
 │ │    < >   DVB for Linux                                                      │ │  
 │ │          *** Multimedia drivers ***                                         │ │  
 │ │    < >   Remote Controller adapters  --->                                   │ │  
 │ │    [ ]   Load and attach frontend and tuner driver modules as needed        │ │  
 │ │    [ ]   Customize analog and hybrid tuner modules to build                 │ │  
 │ │    [*]   Video capture adapters  --->                                       │ │  
 │ │    [ ]   Memory-to-memory multimedia devices  --->                          │ │  
 │ │    [ ]   Radio Adapters  --->                                               │ │  
 │ └─────────────────────────────────────────────────────────────────────────────┘ │  
 ├─────────────────────────────────────────────────────────────────────────────────┤  
 │                        <Select>    < Exit >    < Help >                         │  
 └─────────────────────────────────────────────────────────────────────────────────┘
  • Enable "OMAP 3 Camera support" in the "Video capture adapters" submenu:
 ┌──────────────────────────── Video capture adapters ─────────────────────────────┐
 │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │  
 │ │          Encoders, decoders, sensors and other helper chips  --->           │ │  
 │ │    < >   Virtual Video Driver                                               │ │  
 │ │    < >   VPFE Video Capture Driver                                          │ │  
 │ │    <*>   OMAP2/OMAP3 V4L2-Display driver                                    │ │  
 │ │    < >   CPiA2 Video For Linux                                              │ │  
 │ │    < >   Support for timberdale Video In/LogiWIN                            │ │  
 │ │    < >   SR030PC30 VGA camera sensor support                                │ │  
 │ │    < >   NOON010PC30 CIF camera sensor support                              │ │  
 │ │    < >   Fujitsu M-5MOLS 8MP sensor support                                 │ │  
 │ │    <*>   OMAP 3 Camera support (EXPERIMENTAL)                               │ │  
 │ │    [ ]     OMAP 3 Camera debug messages                                     │ │  
 │ │    < >   SoC camera support                                                 │ │  
 │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │  
 ├─────────────────────────────────────────────────────────────────────────────────┤  
 │                        <Select>    < Exit >    < Help >                         │  
 └─────────────────────────────────────────────────────────────────────────────────┘  
  • Enable "Texas Instruments TVP5150 video decoder" and "Aptina MT9T001 support" in the "Encoders, decoders, sensors and other helper chips" submenu:
 ┌────────────── Encoders, decoders, sensors and other helper chips ───────────────┐
 │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │  
 │ │    < > Texas Instruments TVP514x video decoder                              │ │  
 │ │    <*> Texas Instruments TVP5150 video decoder                              │ │  
 │ │    < > Texas Instruments TVP7002 video decoder                              │ │  
 │ │    < > vpx3220a, vpx3216b & vpx3214c video decoders                         │ │  
 │ │        *** Video and audio decoders ***                                     │ │  
 │ │    < > Philips SAA7171/3/4 audio/video decoders                             │ │  
 │ │    < > Conexant CX2584x audio/video decoders                                │ │  
 │ │        *** MPEG video encoders ***                                          │ │  
 │ │    < > Conexant CX2341x MPEG encoders                                       │ │  
 │ │        *** Video encoders ***                                               │ │  
 │ │    < > Philips SAA7127/9 digital video encoders                             │ │  
 │ │    < > Philips SAA7185 video encoder                                        │ │  
 │ │    < > Analog Devices ADV7170 video encoder                                 │ │  
 │ │    < > Analog Devices ADV7175 video encoder                                 │ │  
 │ │    < > ADV7343 video encoder                                                │ │  
 │ │    < > AK8813/AK8814 video encoders                                         │ │  
 │ │        *** Camera sensor devices ***                                        │ │  
 │ │    < > OmniVision OV7670 sensor support                                     │ │  
 │ │    < > Aptina MT9P031 support                                               │ │  
 │ │    <*> Aptina MT9T001 support                                               │ │  
 │ │    < > Micron mt9v011 sensor support                                        │ │  
 │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │  
 ├─────────────────────────────────────────────────────────────────────────────────┤  
 │                        <Select>    < Exit >    < Help >                         │  
 └─────────────────────────────────────────────────────────────────────────────────┘


Admolition note.png While using the TVP5151 decoder, the nohlt kernel command line parameter must be used (appended to U-Boot bootargs variable). This prevents the WFI instruction from halting the processor in idle state.

Setting up ISP Camera

Typical image grabbing and display operations are managed by several sub-blocks responsible for data processing like resizing, format conversion etc. For instance, the MT9T031 sensor produces RAW Bayer 10bit image data that could be captured without any processing or redirected to OMAP ISP Preview Engine to obtain image in YUV4:2:2 format.

The video input path can consist of various low level components (such as previewers, resizers, color space converters, etc). The components are a part of the video input path topology, that can be explored and modified by the Media Controller framework.

The OMAP3 Camera-ISP driver implements Media Controller API and supports the following hardware blocks: Sensor, CSI2, CCP2, CCDC, Preview, Resizer, H3A AEWB, H3A AF and Histogram. The hardware blocks are called Media Entities and can be accessed through Media Device. Each Media Device is exposed to user space through a device file (e.g. /dev/media0). Each Media Entity has one or more input and output Pads and is connectable to another entity through a Link.

media-ctl utility

The media-ctl command line tool can be used to manipulate the Media Device through the Media Controller API. This tool allows to enumerate media entities and their pads, set/get and enumerate links between pads of different entities, define pads data format. media-ctl package is included in the CM-T3730 Angstrom Linux root filesystem image.

Explore Media Device

Enumerate Media Entities and print the Media Device topology (/dev/media0 by default):

media-ctl -p 

The following output corresponds to entity number 16 which is MT9T031 camera sensor in this example.

...
- entity 16: mt9t001 3-005d (1 pad, 1 link)
             type V4L2 subdev subtype Unknown
             device node name /dev/v4l-subdev8
	pad0: Output [SGRBG10 2048x1536 (32,20)/2048x1536]
		-> 'OMAP3 ISP CCDC':pad0 []

Camera sensor Media Entity properties in details:

mt9t001 3-005d Name of the camera device registered in CM-T3730 board support file where 3-005d is the I2C bus number and device address.
/dev/v4l-subdev8 V4L2 Video device node associated with this entity which is independently configurable through its own set of file operations (if any registered)
pad0: Output pad0 is an output pad
[SGRBG10 2048x1536 (32,20)/2048x1536] pad0 is configured for - RAW Bayer 10bit image format with dimensions 2048x1536. The image window is cropped. The crop rectangle corner is defined by raw and column coordinates (32,20) and the crop window width and height 2048x1536. Default crop settings define a maximum MT9T031 sensor window size. Crop settings are used to provide desired resolution, establish electronic panning and control the frame rate. For more details see camera documentation.
-> 'OMAP3 ISP CCDC':pad0 [] Output pad0 of the Sensor is linked to the input pad0 of "OMAP3 ISP CCDC" entity. Empty brackets specify non active link.

Camera ISP data paths

Data paths (pipelines) inside the camera ISP hardware depend on the image format sourced by the sensor (RAW Bayer, YUV4:2:2, JPEG,...). ISP driver supports different pipelines. To setup the Media Device links and formats properly, please refer to the Camera ISP section in the DM3730 technical reference manual.

RAW Data Path

The following chart shows various pipelines relevant to MT9T031 sensor that produces RAW Bayer 10bit image data.

Camera ISP RAW Data Path

RAW data are processed by the CCDC module and are directly pipelined to the Preview engine (1). Another way is to output directly from the CCDC to memory (C). In the Preview block, the format is converted from RAW data to YUV4:2:2. The data can be output to memory (4) or pipelined to the Resizer (2). The rescaled YUV4:2:2 image is finally stored in memory (3).

Following pipelines for image capturing are valid with MT9T031 sensor:

  • Sensor->CCDC->Memory - (C)
  • Sensor->CCDC->Preview->Memory - (1), (4)
  • Sensor->CCDC->Preview->Resizer->Memory - (1), (2), (3)
YUV4:2:2 Data Path

The following chart shows various pipelines relevant to TVP5151 decoder. The TVP5151 decoder can be configured for output the data in 8-bit YUV4:2:2 with discrete sync output format or 8-bit ITU−R BT.656 with embedded sync output format (which also implements 4:2:2 sampling). The modules in red in the chart below are not used when the format is YUV4:2:2.

Camera ISP YUV4:2:2 Data Path

When the sensor-output format is YUV4:2:2, the CCDC block is pipelined to the resizer (1) or directly to memory (C). The rescaled YUV4:2:2 images are finally stored in memory (2).

Following pipelines for image capturing are valid with TVP5151 decoder:

  • Sensor->CCDC->Memory - (C)
  • Sensor->CCDC->Resizer->Memory - (1), (2)

Setting up ISP pipeline

This section describes how to translate valid ISP pipelines into links between Media Entities using media-ctl utility.

Following example demonstrates setting ISP pipeline for MT9T031 sensor to Sensor->CCDC->Preview->Memory and configuring entity pads formats.

Set up ISP pipeline:

media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP preview output":0[1]'
-r Reset all links to inactive
-l Set up links by comma-separated list of links descriptors
"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1] Link output pad number 0 of Camera sensor to CCDC input pad number 1 and set this link active
"OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1] Link output pad number 2 of CCDC to Preview input pad number 0 and set this link active
"OMAP3 ISP preview":1->"OMAP3 ISP preview output":0[1] Link output pad number 1 of Preview to Preview Output input pad number 0 and set this link active

Configure pads formats:

media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 2048x1536 (32,20)/2048x1536], "OMAP3 ISP CCDC":2 [SGRBG10 2048x1536], "OMAP3 ISP preview":1 [UYVY 2048x1536]'
-f Set up pads formats by comma-separated list of formats descriptors
"mt9t001 3-005d":0 [SGRBG10 2048x1536 (32,20)/2048x1536] Set up Camera sensor pad number 0 format to RAW Bayer 10bit image with resolution 2048x1536. Set maximum allowed sensor window width by specifying crop rectangle.
"OMAP3 ISP CCDC":2 [SGRBG10 2048x1536] Set up CCDC pad number 2 format to RAW Bayer 10bit image with resolution 2048x1536.
"OMAP3 ISP preview":1 [UYVY 2048x1536] Set up Preview pad number 1 format to YUV4:2:2 image with resolution 2048x1536.


Admolition note.png Image dimensions specified in format are not necessarily supported by respective pad. For instance the CCDC always crops one (H) line when outputting data to the Preview engine, the Preview engine crops 18 columns (W) and 8 lines (H). Consequently, the actual output image dimensions in the last example above are 2030x1527. The image can be scaled by modifying the resizer output size. Please always track media-ctl command output log to ensure what formats are actually set.

Video support validation

USB Camera

Check that your camera has been recognized correctly through the kernel debug messages log (using the dmesg command):

usb 1-2.2: New USB device found, idVendor=1871, idProduct=0f01
usb 1-2.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 1-2.2: Product: USB2.0 Camera
usb 1-2.2: Manufacturer: AVEO Technology Corp.
usb 1-2.2: usb_probe_device
usb 1-2.2: configuration #1 chosen from 1 choice
usb 1-2.2: adding 1-2.2:1.0 (config #1, interface 0)
uvcvideo 1-2.2:1.0: usb_probe_interface
uvcvideo 1-2.2:1.0: usb_probe_interface - got id
uvcvideo: Found UVC 1.00 device USB2.0 Camera (1871:0f01)
input: USB2.0 Camera as /devices/platform/usbhs-omap.0/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input7
usb 1-2.2: adding 1-2.2:1.1 (config #1, interface 1)

Also, the /dev/video0 device special file should be created.

ISP Camera

VIP-ADP can be configured to communicate with a single camera device at a time, either digital camera sensor or analog composite camera.

Depending on VIP-ADP configuration settings check that camera device and ISP module have been recognized correctly through the kernel debug messages log (using the dmesg command):

  • OMAP ISP
Linux media interface: v0.10
Linux video capture interface: v2.00
omap3isp omap3isp: Revision 15.0 found
omap-iommu omap-iommu.0: isp: version 1.1
...
  • TVP5151
...
tvp5150 3-005c: chip found at address 0x5c (OMAP I2C adapter)
tvp5150 3-005c: *** tvp5151 chip detected.
tvp5150 3-005c: *** Rom ver is 1.0
  • MT9T0031
...
mt9t001 3-005d: Probing MT9T001 at address 0x5d
mt9t001 3-005d: MT9T001 detected at address 0x5d

Also, the /dev/media0 device special file should be created.

To ensure proper Media Device initialization, print its topology by running

media-ctl -p

Each listed Media Entity should have an associated V4L2 video device. Pay special attention to the camera device entity. It must be present in the list.

Playback and Streaming tools

Linux provides a variety of playback and streaming tools for the V4L2 compliant devices. We provide several examples that do not necessarily require an X Window System to operate your camera and capture images.


Admolition important.png In this article we assume that a single camera (USB or ISP) is connected to CM-T3730 board. It is possible to connect several cameras, but then corresponding video devices nodes will not match their referenced names.

yavta

yavta (“Yet Another V4L2 Test Application”) is a data capture tool that allows to enumerate and set V4L2 device controls like brightness, exposure, autofocus etc. This is a generic tool that can be applied to any V4L2 compliant device. yavta package is included in the CM-T3730 Angstrom Linux root filesystem image.

Configure V4L2 device

V4L2 device typically has a number of configurable controls. yavta allows the controls enumeration and configuration.

Run the following command to print the list of all available controls of the Camera sensor exposed through /dev/v4l-subdev8. Output below corresponds to MT9T031 sensor.

root@cm-t35:~# yavta --list-controls /dev/v4l-subdev8
Device /dev/v4l-subdev8 opened.
--- User Controls (class 0x00980001) ---
control 0x0098090b `Black Level' min 1 max 1 step 1 default 1 current 1.
control 0x00980911 `Exposure' min 1 max 1048576 step 1 default 1561 current 1700.
control 0x00981901 `Test pattern' min 0 max 1023 step 1 default 0 current 0.
control 0x00981902 `Black Level, Auto' min 0 max 1 step 1 default 1 current 1.
control 0x00981903 `Black Level, Offset' min -256 max 255 step 1 default 32 current 32.
control 0x00981904 `Black Level, Calibrate' min 0 max 0 step 0 default 0 current n/a.
--- Camera Controls (class 0x009a0001) ---
control 0x009a1001 `Gain, Red' min 8 max 1024 step 1 default 8 current 8.
control 0x009a1002 `Gain, Green (R)' min 8 max 1024 step 1 default 8 current 8.
control 0x009a1003 `Gain, Green (B)' min 8 max 1024 step 1 default 8 current 8.
control 0x009a1004 `Gain, Blue' min 8 max 1024 step 1 default 8 current 8.

Following command will change the sensor exposure to 1000:

yavta --set-control '0x00980911 1000' /dev/v4l-subdev8

ISP Media Entity can be referred by name using media-ctl -e option. For example, OMAP3 ISP Preview entity V4L2 device controls can be explored by the following command:

root@cm-t35:~# yavta --list-controls `media-ctl -e "OMAP3 ISP preview"`
Device /dev/v4l-subdev3 opened.
--- User Controls (class 0x00980001) ---
control 0x00980900 `Brightness' min 0 max 255 step 1 default 0 current 0.
control 0x00980901 `Contrast' min 0 max 255 step 1 default 16 current 16.
2 controls found.

yavta parameters for configuring V4L2 device

-l, --list-controls List available controls.
-w, --set-control 'ctrl value' Set control 'ctrl' to 'value'.
--enum-formats Enumerate formats supported by video device.
--enum-inputs Enumerate inputs of video device. Useful to find out video format details of specific input. That is video format name and ID, frame dimensions and buffer size.

Capture stills images

Capturing stills images from ISP Camera can be done using several ISP pipelines. Examples below show how to setup appropriate pipeline, set pads format and capture an image using yavta.

MT9T031

Run the following set of commands to capture an image from Sensor->CCDC->Memory pipeline (capture RAW Bayer 10bit image from CCDC):

media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 2048x1536 (32,20)/2048x1536], "OMAP3 ISP CCDC":1 [SGRBG10 2048x1536]'
yavta -f SGRBG10 -s 2048x1536 --capture=1 --file=image  /dev/video2

Run the following set of commands to capture an image from Sensor->CCDC->Preview->Memory pipeline (capture YUV4:2:2 image from the Previewer):

media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP preview output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 2048x1536 (32,20)/2048x1536], "OMAP3 ISP CCDC":2 [SGRBG10 2048x1536], "OMAP3 ISP preview":1 [UYVY 2048x1536]'
yavta -f UYVY -s 2030x1527 --capture=1 --file=image /dev/video4

Run the following set of commands to capture an image from Sensor->CCDC->Preview->Resizer->Memory pipeline (capture YUV4:2:2 image from the resizer):

media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
                 "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024], \
              "OMAP3 ISP resizer":1 [UYVY 1024x768]'
yavta -f UYVY -s 1024x768 --capture=1 --file=image /dev/video6
TVP5151

Run the following set of commands to capture an image from Sensor->CCDC->Memory pipeline (capture YUV4:2:2 image from CCDC):

media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
yavta -f UYVY -s 720x524 --capture=1 --file=image /dev/video2

yavta parameters for capturing images

-f UYVY Set the video format to UYVY.
-s 1024x768 Set the frame size 1024x768.
--capture=1 Capture one frame.
--file=image Write frame to a file image-000000.bin.
/dev/video6 Specify video capture device.

Displaying captured images

2048x1536 ISP Camera RAW Bayer 10bit image snapshot, processed by UFRaw, scaled to 640x480 and converted to JPG

MT9T031 camera sensor produces RAW Bayer 10bit image data. The raw data can be captured directly from ISP CCDC or after processing by Preview/Resizer in YUV4:2:2 format.

RAW Bayer array requires demosaicing, white balance setting and other processing steps. It can be done by UFRaw tool:

ufraw image-000000.bin

YUV4:2:2 image can be viewed by display tool from the ImageMagic application package. For instance, in order to view the image captured from MT9T031 sensor through Preview module in the example above run the following set of commands:

mv image-000000.bin image-000000.uyvy
display -size 2031x1527 -colorspace rgb image-000000.uyvy


Admolition note.png Image captured through TVP5151 decoder from camera connected to composite input of VIP-ADP module requires deinterlacing.


MPlayer

MPlayer has support for most modern video and audio formats. MPlayer is highly configurable, and can be controlled by keyboard, mouse, joystick or remote control (with LIRC). MPlayer package is included in the CM-T3730 Angstrom Linux root filesystem image.

Output to display

USB Camera

The following command can be used to stream the USB camera output to the display using default configuration settings.

mplayer tv:// -vo omapfb

Press q or ESC to stop playing and quit program.
Adding more parameters will do some customization as shown below.

mplayer tv:// -tv driver=v4l2:width=800:height=600:device=/dev/video0:brightness=80:contrast=10:saturation=-30 -fs -vf mirror -vo omapfb

Parameters description:

mplayer tv:// Stream data using TV capture module
-tv This option tunes various properties of the TV capture module (see below).
driver=v4l2
Set TV input driver.
width=800:height=600
Output window width and height.
device=/dev/video0
Specify TV device.
outfmt=uyvy
Specify the output format of the TV device.
brightness=80:contrast=10:saturation=-30
Set the image equalizer.
-fs Fullscreen playback. Note: -fs is not necessarily supported by all MPlayer video output drivers.
-vf Video filters allow modification of the video stream and its properties.
mirror
Mirrors the image on the Y axis.
screenshot
Allows acquiring screenshots of the movie. Make sure that the screenshot filter is the last filter, so the exact picture on the display will be saved.
crop=720:480:0:44
Crops the 720x480 part of the image from position (0,44) and discards the rest.
pp=li
Enables linear interpolating deinterlacing sub-filter.
-vo Specifies MPlayer video output driver.
omapfb
TI OMAP video frame buffer output.
xv
Separate output window if using X Window System.


Admolition note.png To start X Window application (e.g. MPlayer) from your serial console, the DISPLAY environment variable must be set.
  • Additional example for streaming USB camera output to a separate window in the X Window system:
mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -vo xv
MT9T031

Following set of commands will set up ISP pipeline to Sensor->CCDC->Preview->Resizer->Memory and stream 1024x768 video from MT9T031 sensor to OMAP framebuffer:

media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
                 "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024],  "OMAP3 ISP resizer":1 [UYVY 1024x768]'
mplayer tv:// -tv driver=v4l2:device=/dev/video6:outfmt=uyvy:width=1024:height=768 -vo omapfb
TVP5151

Following set of commands will set up ISP pipeline to Sensor->CCDC->Memory and stream 720x524 video from TVP5151 decoder to OMAP framebuffer:

media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
mplayer tv:// -tv driver=v4l2:device=/dev/video2:outfmt=uyvy:width=720:height=524 -vo omapfb -vf crop=720:480:0:44,pp=li

Snapshots

640x480 USB Camera screenshot converted to jpg
720x480 NTSC car reverse camera screenshot converted to jpg
USB Camera

To take snapshots from USB Camera, run the MPlayer with the following command line:

mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -vf screenshot -vo omapfb
MT9T031

ISP Camera snapshoot from MT9T031 sensor could be obtained by the following set of commands:

media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
                 "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024],  "OMAP3 ISP resizer":1 [UYVY 1024x768]'
mplayer tv:// -tv driver=v4l2:device=/dev/video6:outfmt=uyvy:width=1024:height=768 -vf screenshot -vo omapfb
TVP5151

ISP Camera snapshoot from TVP5151 decoder could be obtained by the following set of commands:

media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
mplayer tv:// -tv driver=v4l2:device=/dev/video2:outfmt=uyvy:width=720:height=524 -vf crop=720:480:0:44,pp=li,screenshot -vo omapfb

Once there is an output streamed on the display, press s to take a snapshot. Each snapshot will be saved in your current directory as shotXXXX.png.

Playback

To play a movie, run the MPlayer with the following command line:

mplayer media_file.ogg -vo omapfb

GStreamer

GStreamer is a pipeline-based multimedia framework that allows you to create a variety of media-handling software components. This includes simple audio playback, audio and video playback, recording, streaming, and editing. The pipeline design allows you to easily write any type of streaming multimedia application. GStreamers capabilities are extended through plugins. Several simple media manipulations are demonstrated below. GStreamer package and required plugins are included in the CM-T3730 Angstrom Linux root filesystem image.

Validation

GStreamer source test output

To validate the installation of the GStreamer components, launch gst-inspect GStreamer utility.

Testing the basic streaming to OMAP frame buffer:

gst-launch videotestsrc ! omapdmaifbsink


Streaming to display

To start streaming the video to display the simple GStreamer pipeline can be used. The basic GStreamer pipeline should have at least the source of the stream and the output sink specified.
The below examples stream the video from the V4L2 video capture device to the OMAP frame buffer (sink) with several stream manipulations applied.

USB Camera

Run the following command to stream the USB camera to display

gst-launch v4l2src device=/dev/video0 ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=640,height=480' ! videorate ! ffmpegcolorspace ! omapdmaifbsink
MT9T031

Run the following set of commands to stream video from MT9T031 sensor to display

media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
                 "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024],  "OMAP3 ISP resizer":1 [UYVY 1024x768]'
gst-launch v4l2src device=/dev/video6 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=1024,height=768' ! omapdmaifbsink
TVP5151

Run the following set of commands to stream video from TVP5151 decoder to display

media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
gst-launch v4l2src device=/dev/video2 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=524' ! videocrop top=44 ! deinterlace mode=1 fields=1 ! omapdmaifbsink

GStreamer pipeline parameters in details:

v4l2src Element for reading frames from a Video4Linux2 device.
device=/dev/video0
Specify the V4L2 /dev/video0 device as a stream source.
always-copy=false
Disable copying of the stream buffers. This can slightly improve the GStreamer performance.
video/x-raw-yuv,format=(fourcc)YUY2,width=640,height=480 Link the source to a capability, which is defined by a mime type and a few optional properties. The properties are: video and resolution.
videocrop Crops video frames.
top=44
Remove 44 pixels from the top of the image.
deinterlace Deinterlace the video.
mode=1
Force deinterlacing.
fields=1
Use top fields only for deinterlacing.
videorate Use each frame of the source stream and feed it to the next element at the requested framerate.
ffmpegcolorspace Make the color space conversion, required by the sink element.
omapdmaifbsink Define the OMAP frame buffer as the output sink. Sink can be: a file, screen or a network. For X Window System, the ximagesink parameter can be used.

Streaming to a file

This section examples demonstrate how to encode and save the output stream to a file, using the Theora codec.

USB Camera

Run the following command to save the USB Camera output stream to a file:

gst-launch v4l2src ! 'video/x-raw-yuv,width=320,height=240' ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! \
           theoraenc ! queue ! oggmux !  filesink location=media_file.ogg
MT9T031

Run following set of commands to save the MT9T031 sensor output stream to a file:

media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], \
                 "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 320x240], "OMAP3 ISP CCDC":2 [SGRBG10 320x240], "OMAP3 ISP preview":1 [UYVY 320x240], \
              "OMAP3 ISP resizer":1 [UYVY 320x240]'
gst-launch v4l2src device=/dev/video6 ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! \
           queue ! ffmpegcolorspace ! theoraenc ! queue ! oggmux !  filesink location=media_file.ogg
TVP5151

Run following set of commands to save the TVP5151 decoder output stream to a file:

media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
gst-launch v4l2src device=/dev/video2 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=524' ! videocrop top=44 ! \
           deinterlace mode=1 fields=1 ! videoscale ! 'video/x-raw-yuv,width=310,height=240' ! queue ! videorate ! \
           'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! theoraenc ! queue ! oggmux ! filesink location=media_file.ogg

GStreamer parameters in details:

queue Provides a buffer for the next element in the pipeline.
theoraenc Encodes raw video into theora stream.
oggmux Muxes a stream into an ogg container.
videoscale Resizes video frames.
filesink Writes the incoming data to a file.

Streaming to a file and display

USB Camera

Following command splits the source stream from USB Camera and links to the frame buffer and to an encoded file:

gst-launch v4l2src always-copy=false ! 'video/x-raw-yuv,width=320,height=240' ! tee name=t_video ! queue ! videorate ! ffmpegcolorspace ! \
           omapdmaifbsink t_video. ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! theoraenc ! \
           queue ! oggmux ! filesink location=media_file.ogg
MT9T031
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
                 "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 640x480], "OMAP3 ISP CCDC":2 [SGRBG10 640x480], "OMAP3 ISP preview":1 [UYVY 640x480],  "OMAP3 ISP resizer":1 [UYVY 320x240]'
gst-launch v4l2src device=/dev/video6 always-copy=false ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=320,height=240' ! tee name=t_video ! \
           queue ! videorate ! ffmpegcolorspace ! omapdmaifbsink t_video. ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! \ 
           theoraenc ! queue ! oggmux ! filesink location=media_file.ogg
TVP5151
media-ctl -r -l '"tvp5150 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
media-ctl -f '"tvp5150 3-005c":0 [UYVY2X8 720x524], "OMAP3 ISP CCDC":0 [UYVY2X8 720x524]'
gst-launch v4l2src device=/dev/video2 always-copy=false ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=524' ! videocrop top=44 ! \
           deinterlace mode=1 fields=1 ! videoscale ! 'video/x-raw-yuv, width=310,height=240' ! tee name=t_video ! queue ! omapdmaifbsink \
           t_video. ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! queue ! ffmpegcolorspace ! theoraenc ! queue ! oggmux !  \
           filesink location=media_file.ogg
tee Split data to multiple streams (element pads).
name=t_video
Names the element's pad. t_video is then linked to theora encoder.

File playback

To play a movie encoded by Theora codec and saved in ogg contaner, use the following command line:

gst-launch filesrc location=media_file.ogg ! oggdemux ! theoradec ! ffmpegcolorspace  ! omapdmaifbsink

See also