Difference between revisions of "CM-T3730: Linux: Camera"
(→Camera ISP data paths) |
|||
| Line 24: | Line 24: | ||
== Linux kernel == | == Linux kernel == | ||
| − | All the use cases and examples in this article were tested with the CM-T3730 Linux package (currently based on kernel | + | All the use cases and examples in this article were tested with the CM-T3730 Linux package (currently based on kernel v3.0) that can be obtained from [http://compulab.co.il/products/computer-on-modules/cm-t3730/ CM-T3730 product webpage]. |
The following kernel configuration procedure is required: | The following kernel configuration procedure is required: | ||
| Line 229: | Line 229: | ||
The following chart shows various pipelines relevant to MT9T031 sensor that produces RAW Bayer 10bit image data. | The following chart shows various pipelines relevant to MT9T031 sensor that produces RAW Bayer 10bit image data. | ||
| − | [[Image:Isp_data_path. | + | [[Image:Isp_data_path.jpg|450px|center|Camera ISP Data Path]] |
RAW data are processed by the CCDC module and are directly pipelined to the Preview engine(1). Another way is to output directly from the CCDC to memory (C). In the Preview block, the format is converted from RAW data to YUV4:2:2. The data can be output to memory (4) or pipelined to the Resizer (2). The rescaled YUV4:2:2 image is finally stored in memory (3). | RAW data are processed by the CCDC module and are directly pipelined to the Preview engine(1). Another way is to output directly from the CCDC to memory (C). In the Preview block, the format is converted from RAW data to YUV4:2:2. The data can be output to memory (4) or pipelined to the Resizer (2). The rescaled YUV4:2:2 image is finally stored in memory (3). | ||
Revision as of 12:30, 10 May 2012
Contents
Overview
CM-T3730 Linux supports various camera devices connected either to the USB subsystem or to the camera interface (CIF). This article describes both camera types setup, configuration and provides several usage examples.
Linux USB camera support is provided by USB Video Class (UVC) driver. All UVC compliant camera peripherals can be managed by the UVC driver. Linux also supports various non UVC cameras, but with some limitations. Therefore non UVC cameras are not covered in this article. Linux UVC driver implements the Video for Linux Two (V4L2) API. V4L2 is a suite of driver specifications for different types of video devices and video-related data. Examples in this article are using a AV301W-based camera.
The CM-T3730 module features camera image signal processor (ISP) that supports multiple formats and interfacing options for a wide variety of image sensors. OMAP ISP subsystem is supported by Linux OMAP3 Camera-ISP driver. In addition, any camera sensor requires appropriate sensor driver to be available. The OMAP3 Camera-ISP driver is a V4L2 framework driver that makes use of the Media Controller framework to configure the media hardware (camera). Examples in this article are using an Aptina (Micron) MT9T031 3MP sensor.
Hardware Setup
USB camera does not require any special hardware setup. It can be connected to any SB-T35 USB port.
ISP camera sensor (MT9T031) is connected with the Micron MT9T031C12STCH ES demo head-board. Follow the instruction in Video Input Port Adapter Board article to connect the sensor to CM-T3730.
Linux kernel
All the use cases and examples in this article were tested with the CM-T3730 Linux package (currently based on kernel v3.0) that can be obtained from CM-T3730 product webpage.
The following kernel configuration procedure is required:
Common media configuration
- Enable "Multimedia support" in the "Device Drivers" submenu:
┌──────────────────────────────── Device Drivers ─────────────────────────────────┐ │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │ │ │ [*] Watchdog Timer Support ---> │ │ │ │ Sonics Silicon Backplane ---> │ │ │ │ Broadcom specific AMBA ---> │ │ │ │ -*- Multifunction device drivers ---> │ │ │ │ -*- Voltage and Current Regulator Support ---> │ │ │ │ <*> Multimedia support ---> │ │ │ │ Graphics support ---> │ │ │ │ <*> Sound card support ---> │ │ │ │ [*] HID Devices ---> │ │ │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────────────┤ │ <Select> < Exit > < Help > │ └─────────────────────────────────────────────────────────────────────────────────┘
- Enable "Video For Linux" and "Video capture adapters" in the "Multimedia support" submenu:
┌────────────────────────────── Multimedia support ───────────────────────────────┐ │ ┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ --- Multimedia support │ │ │ │ *** Multimedia core support *** │ │ │ │ [ ] Media Controller API (EXPERIMENTAL) │ │ │ │ <*> Video For Linux │ │ │ │ < > DVB for Linux │ │ │ │ *** Multimedia drivers *** │ │ │ │ <*> Remote Controller adapters ---> │ │ │ │ [ ] Load and attach frontend and tuner driver modules as needed │ │ │ │ [*] Customize analog and hybrid tuner modules to build │ │ │ │ Customize TV tuners ---> │ │ │ │ [*] Video capture adapters ---> │ │ │ │ [ ] Memory-to-memory multimedia devices ---> │ │ │ │ [*] Radio Adapters ---> │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────────────┤ │ <Select> < Exit > < Help > │ └─────────────────────────────────────────────────────────────────────────────────┘
USB Camera configuration
- Enable "OMAP2/OMAP3 V4L2-Display driver" and "V4L USB devices" in the "Video capture adapters" submenu:
┌──────────────────────────── Video capture adapters ─────────────────────────────┐ │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │ │ │ [ ] Enable old-style fixed minor ranges for video devices │ │ │ │ [ ] Autoselect pertinent encoders/decoders and other helper chips │ │ │ │ <*> I2C module for IR │ │ │ │ Encoders, decoders, sensors and other helper chips ---> │ │ │ │ < > Virtual Video Driver │ │ │ │ < > VPFE Video Capture Driver │ │ │ │ <*> OMAP2/OMAP3 V4L2-Display driver │ │ │ │ < > CPiA2 Video For Linux │ │ │ │ < > Support for timberdale Video In/LogiWIN │ │ │ │ < > SR030PC30 VGA camera sensor support │ │ │ │ < > NOON010PC30 CIF camera sensor support │ │ │ │ < > SoC camera support │ │ │ │ [*] V4L USB devices ---> │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────────────┤ │ <Select> < Exit > < Help > │ └─────────────────────────────────────────────────────────────────────────────────┘
- Enable "USB Video Class (UVC)" in "V4L USB devices" submenu:
┌──────────────────────────────── V4L USB devices ────────────────────────────────┐ │ ┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ --- V4L USB devices │ │ │ │ <*> USB Video Class (UVC) │ │ │ │ [*] UVC input events device support │ │ │ │ <M> GSPCA based webcams ---> │ │ │ │ < > Hauppauge WinTV-PVR USB2 support │ │ │ │ < > Hauppauge HD PVR support │ │ │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────────────┤ │ <Select> < Exit > < Help > │ └─────────────────────────────────────────────────────────────────────────────────┘
UVC camera and the driver do not need any special kernel command line parameters option.
ISP Camera configuration
- Enable "Media Controller API" in the "Multimedia support" submenu:
┌────────────────────────────── Multimedia support ───────────────────────────────┐ │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │ │ │ *** Multimedia core support *** │ │ │ │ [*] Media Controller API (EXPERIMENTAL) │ │ │ │ <*> Video For Linux │ │ │ │ [*] V4L2 sub-device userspace API (EXPERIMENTAL) │ │ │ │ < > DVB for Linux │ │ │ │ *** Multimedia drivers *** │ │ │ │ < > Remote Controller adapters ---> │ │ │ │ [ ] Load and attach frontend and tuner driver modules as needed │ │ │ │ [ ] Customize analog and hybrid tuner modules to build │ │ │ │ [*] Video capture adapters ---> │ │ │ │ [ ] Memory-to-memory multimedia devices ---> │ │ │ │ [ ] Radio Adapters ---> │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────────────┤ │ <Select> < Exit > < Help > │ └─────────────────────────────────────────────────────────────────────────────────┘
- Enable "OMAP 3 Camera support" in the "Video capture adapters" submenu:
┌──────────────────────────── Video capture adapters ─────────────────────────────┐ │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │ │ │ Encoders, decoders, sensors and other helper chips ---> │ │ │ │ < > Virtual Video Driver │ │ │ │ < > VPFE Video Capture Driver │ │ │ │ <*> OMAP2/OMAP3 V4L2-Display driver │ │ │ │ < > CPiA2 Video For Linux │ │ │ │ < > Support for timberdale Video In/LogiWIN │ │ │ │ < > SR030PC30 VGA camera sensor support │ │ │ │ < > NOON010PC30 CIF camera sensor support │ │ │ │ < > Fujitsu M-5MOLS 8MP sensor support │ │ │ │ <*> OMAP 3 Camera support (EXPERIMENTAL) │ │ │ │ [ ] OMAP 3 Camera debug messages │ │ │ │ < > SoC camera support │ │ │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────────────┤ │ <Select> < Exit > < Help > │ └─────────────────────────────────────────────────────────────────────────────────┘
- Enable "Aptina MT9T001 support" in the "Encoders, decoders, sensors and other helper chips" submenu:
┌────────────── Encoders, decoders, sensors and other helper chips ───────────────┐ │ ┌────^(-)─────────────────────────────────────────────────────────────────────┐ │ │ │ < > ADV7343 video encoder │ │ │ │ < > AK8813/AK8814 video encoders │ │ │ │ *** Camera sensor devices *** │ │ │ │ < > OmniVision OV7670 sensor support │ │ │ │ < > Aptina MT9P031 support │ │ │ │ <*> Aptina MT9T001 support │ │ │ │ < > Micron mt9v011 sensor support │ │ │ │ < > Micron MT9V032 sensor support │ │ │ │ < > TCM825x camera sensor support │ │ │ │ *** Video improvement chips *** │ │ │ │ < > NEC Electronics uPD64031A Ghost Reduction │ │ │ │ < > NEC Electronics uPD64083 3-Dimensional Y/C separation │ │ │ └────v(+)─────────────────────────────────────────────────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────────────┤ │ <Select> < Exit > < Help > │ └─────────────────────────────────────────────────────────────────────────────────┘
Setting up ISP Camera
Typical image grabbing and display operations are managed by several sub-blocks responsible for data processing like resizing, format conversion etc. For instance, the MT9T031 sensor produces RAW Bayer 10bit image data that could be captured without any processing or redirected to OMAP ISP Preview Engine to obtain image in YUV4:2:2 format.
The video input path can consist of various low level components (such as previewers, resizers, color space converters, etc). The components are a part of the video input path topology, that can be explored and modified by the Media Controller framework.
The OMAP3 Camera-ISP driver implements Media Controller API and supports the following hardware blocks: Sensor, CSI2, CCP2, CCDC, Preview, Resizer, H3A AEWB, H3A AF and Histogram. The hardware blocks are called Media Entities and can be accessed through Media Device. Each Media Device is exposed to user space through a device file (e.g. /dev/media0). Each Media Entity has one or more input and output Pads and is connectable to another entity through a Link.
media-ctl utility
The media-ctl command line tool can be used to manipulate the Media Device through the Media Controller API. The media-ctl tool can be found on The Ångström Distribution official website. This tool allows to enumerate media entities and their pads, set/get and enumerate links between pads of different entities, define pads data format.
Installation
Download and install required packages:
opkg install libc6_2.12-r28_armv7a.ipk media-ctl_0.0.1-r0_armv7a.ipk
Explore Media Device
Enumerate Media Entities and print the Media Device topology (/dev/media0 by default):
media-ctl -p
The following output corresponds to entity number 16 which is the Camera sensor.
...
- entity 16: mt9t001 3-005d (1 pad, 1 link)
type V4L2 subdev subtype Unknown
device node name /dev/v4l-subdev8
pad0: Output [SGRBG10 2048x1536 (32,20)/2048x1536]
-> 'OMAP3 ISP CCDC':pad0 []
Camera sensor Media Entity properties in details:
| mt9t001 3-005d | Name of the camera device registered in CM-T3730 board support file where 3-005d is the I2C bus number and device address. |
| /dev/v4l-subdev8 | V4L2 Video device node associated with this entity which is independently configurable through its own set of file operations (if any registered) |
| pad0: Output | pad0 is an output pad |
| [SGRBG10 2048x1536 (32,20)/2048x1536] | pad0 is configured for - RAW Bayer 10bit image format with dimensions 2048x1536. The image window is cropped. The crop rectangle corner is defined by raw and column coordinates (32,20) and the crop window width and height 2048x1536. Default crop settings define a maximum MT9T031 sensor window size. Crop settings are used to provide desired resolution, establish electronic panning and control the frame rate. For more details see camera documentation. |
| -> 'OMAP3 ISP CCDC':pad0 [] | Output pad0 of the Sensor is linked to the input pad0 of "OMAP3 ISP CCDC" entity. Empty brackets specify non active link. |
Camera ISP data paths
Data paths (pipelines) inside the camera ISP hardware depend on the image format sourced by the sensor (RAW Bayer, YUV4:2:2, JPEG,...). ISP driver supports different pipelines. To setup the Media Device links and formats properly, please refer to the Camera ISP section in the DM3730 technical reference manual. The following chart shows various pipelines relevant to MT9T031 sensor that produces RAW Bayer 10bit image data.
RAW data are processed by the CCDC module and are directly pipelined to the Preview engine(1). Another way is to output directly from the CCDC to memory (C). In the Preview block, the format is converted from RAW data to YUV4:2:2. The data can be output to memory (4) or pipelined to the Resizer (2). The rescaled YUV4:2:2 image is finally stored in memory (3).
Following pipelines for image capture are valid with MT9T031 sensor:
- Sensor->CCDC->Memory - (C)
- Sensor->CCDC->Preview->Memory - (1), (4)
- Sensor->CCDC->Preview->Resizer->Memory - (1), (2), (3)
Setting up ISP pipeline
This section describes how to translate valid ISP pipelines into links between Media Entities using media-ctl utility.
Following example demonstrates setting ISP pipeline to Sensor->CCDC->Preview->Memory and configuring entity pads formats.
Set up ISP pipeline:
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP preview output":0[1]'
| -r | Reset all links to inactive |
| -l | Set up links by comma-separated list of links descriptors |
| "mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1] | Link output pad number 0 of Camera sensor to CCDC input pad number 1 and set this link active |
| "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1] | Link output pad number 2 of CCDC to Preview input pad number 0 and set this link active |
| "OMAP3 ISP preview":1->"OMAP3 ISP preview output":0[1] | Link output pad number 1 of Preview to Preview Output input pad number 0 and set this link active |
Configure pads formats:
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 2048x1536 (32,20)/2048x1536], "OMAP3 ISP CCDC":2 [SGRBG10 2048x1536], "OMAP3 ISP preview":1 [UYVY 2048x1536]'
| -f | Set up pads formats by comma-separated list of formats descriptors |
| "mt9t001 3-005d":0 [SGRBG10 2048x1536 (32,20)/2048x1536] | Set up Camera sensor pad number 0 format to RAW Bayer 10bit image with resolution 2048x1536. Set maximum allowed sensor window width by specifying crop rectangle. |
| "OMAP3 ISP CCDC":2 [SGRBG10 2048x1536] | Set up CCDC pad number 2 format to RAW Bayer 10bit image with resolution 2048x1536. |
| "OMAP3 ISP preview":1 [UYVY 2048x1536] | Set up Preview pad number 1 format to YUV4:2:2 image with resolution 2048x1536. |
|
Image dimensions specified in format are not necessarily supported by respective pad. For instance the CCDC always crops one (H) line when outputting data to the Preview engine, the Preview engine crops 18 columns (W) and 8 lines (H). Consequently, the actual output image dimensions in the last example above are 2030x1527. The image can be scaled by modifying the resizer output size. Please always track media-ctl command output log to ensure what formats are actually set. |
Video support validation
USB Camera
Check that your camera has been recognized correctly through the kernel debug messages log (using the dmesg command):
usb 1-2.2: New USB device found, idVendor=1871, idProduct=0f01 usb 1-2.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0 usb 1-2.2: Product: USB2.0 Camera usb 1-2.2: Manufacturer: AVEO Technology Corp. usb 1-2.2: usb_probe_device usb 1-2.2: configuration #1 chosen from 1 choice usb 1-2.2: adding 1-2.2:1.0 (config #1, interface 0) uvcvideo 1-2.2:1.0: usb_probe_interface uvcvideo 1-2.2:1.0: usb_probe_interface - got id uvcvideo: Found UVC 1.00 device USB2.0 Camera (1871:0f01) input: USB2.0 Camera as /devices/platform/usbhs-omap.0/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input7 usb 1-2.2: adding 1-2.2:1.1 (config #1, interface 1)
Also, the /dev/video0 device special file should be created.
ISP Camera
Check that MT9T0031 camera driver and ISP module has been recognized correctly through the kernel debug messages log (using the dmesg command):
Linux media interface: v0.10 Linux video capture interface: v2.00 omap3isp omap3isp: Revision 15.0 found omap-iommu omap-iommu.0: isp: version 1.1 mt9t001 3-005d: Probing MT9T001 at address 0x5d mt9t001 3-005d: MT9T001 detected at address 0x5d
Also, the /dev/media0 device special file should be created.
To ensure proper Media Device initialization, print its topology by running
media-ctl -p
Each listed Media Entity should have an associated V4L2 video device. Pay special attention to the camera sensor entity. It must be present in the list.
Playback and Streaming tools
Linux provides a variety of playback and streaming tools for the V4L2 compliant devices. We provide several examples that do not necessarily require an X Window System to operate your camera and capture images.
|
In this article we assume that a single camera (USB or ISP) is connected to CM-T3730 board. It is possible to connect several cameras, but then corresponding video devices nodes will not match their referenced names. |
yavta
yavta (“Yet Another V4L2 Test Application”) is a data capture tool that allows to enumerate and set V4L2 device controls like brightness, exposure, autofocus etc. This is a generic tool that can be applied to any V4L2 compliant device.
Installation
yavta package can be found on The Ångström Distribution official website.
Download and install required packages:
opkg install libc6_2.12-r28_armv7a.ipk yavta_0.0-r0_armv7a.ipk
Configure V4L2 device
V4L2 device typically has a number of configurable controls. yavta allows the controls enumeration and configuration.
Run the following command to print the list of all available controls of the Camera sensor exposed through /dev/v4l-subdev8.
root@cm-t35:~# yavta --list-controls /dev/v4l-subdev8 Device /dev/v4l-subdev8 opened. --- User Controls (class 0x00980001) --- control 0x0098090b `Black Level' min 1 max 1 step 1 default 1 current 1. control 0x00980911 `Exposure' min 1 max 1048576 step 1 default 1561 current 1700. control 0x00981901 `Test pattern' min 0 max 1023 step 1 default 0 current 0. control 0x00981902 `Black Level, Auto' min 0 max 1 step 1 default 1 current 1. control 0x00981903 `Black Level, Offset' min -256 max 255 step 1 default 32 current 32. control 0x00981904 `Black Level, Calibrate' min 0 max 0 step 0 default 0 current n/a. --- Camera Controls (class 0x009a0001) --- control 0x009a1001 `Gain, Red' min 8 max 1024 step 1 default 8 current 8. control 0x009a1002 `Gain, Green (R)' min 8 max 1024 step 1 default 8 current 8. control 0x009a1003 `Gain, Green (B)' min 8 max 1024 step 1 default 8 current 8. control 0x009a1004 `Gain, Blue' min 8 max 1024 step 1 default 8 current 8.
Following command will change the sensor exposure to 1000:
yavta --set-control '0x00980911 1000' /dev/v4l-subdev8
ISP Media Entity can be referred by name using media-ctl -e option. For example, OMAP3 ISP Preview entity V4L2 device controls can be explored by the following command:
root@cm-t35:~# yavta --list-controls `media-ctl -e "OMAP3 ISP preview"` Device /dev/v4l-subdev3 opened. --- User Controls (class 0x00980001) --- control 0x00980900 `Brightness' min 0 max 255 step 1 default 0 current 0. control 0x00980901 `Contrast' min 0 max 255 step 1 default 16 current 16. 2 controls found.
yavta parameters for configuring V4L2 device
| -l, --list-controls | List available controls. |
| -w, --set-control 'ctrl value' | Set control 'ctrl' to 'value'. |
| --enum-formats | Enumerate formats supported by video device. |
| --enum-inputs | Enumerate inputs of video device. Useful to find out video format details of specific input. That is video format name and ID, frame dimensions and buffer size. |
Capture stills images
Capturing stills images from ISP Camera can be done using several ISP pipelines. Examples below show how to setup appropriate pipeline, set pads format and capture an image using yavta.
Run the following set of commands to capture an image from Sensor->CCDC->Memory pipeline (capture RAW Bayer 10bit image from CCDC):
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 2048x1536 (32,20)/2048x1536], "OMAP3 ISP CCDC":1 [SGRBG10 2048x1536]'
yavta -f SGRBG10 -s 2048x1536 --capture=1 --file=image /dev/video2
Run the following set of commands to capture an image from Sensor->CCDC->Preview->Memory pipeline (capture YUV4:2:2 image from the Previewer):
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP preview output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 2048x1536 (32,20)/2048x1536], "OMAP3 ISP CCDC":2 [SGRBG10 2048x1536], "OMAP3 ISP preview":1 [UYVY 2048x1536]'
yavta -f UYVY -s 2030x1527 --capture=1 --file=image /dev/video4
Run the following set of commands to capture an image from Sensor->CCDC->Preview->Resizer->Memory pipeline (capture YUV4:2:2 image from the resizer):
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
"OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024], \
"OMAP3 ISP resizer":1 [UYVY 1024x768]'
yavta -f UYVY -s 1024x768 --capture=1 --file=image /dev/video6
yavta parameters for capturing images
| -f UYVY | Set the video format to UYVY. |
| -s 1024x768 | Set the frame size 1024x768. |
| --capture=1 | Capture one frame. |
| --file=image | Write frame to a file image-000000.bin. |
| /dev/video6 | Specify video capture device. |
Displaying captured images
MT9T031 camera sensor produces RAW Bayer 10bit image data. The raw data can be captured directly from ISP CCDC or after processing by Preview/Resizer in YUV4:2:2 format.
RAW Bayer array requires demosaicing, white balance setting and other processing steps. It can be done by UFRaw tool:
ufraw image-000000.bin
YUV4:2:2 image can be viewed by display tool from the ImageMagic application package. For instance, in order to view the image captured from Preview module in the example above run the following set of commands:
mv image-000000.bin image-000000.uyvy
display -size 2031x1527 -colorspace rgb image-000000.uyvy
MPlayer
MPlayer has support for most modern video and audio formats. MPlayer is highly configurable, and can be controlled by keyboard, mouse, joystick or remote control (with LIRC). MPlayer package is included in the CM-T3730 Angstrom Linux root filesystem image.
Output to display
The following command can be used to stream the USB camera output to the display using default configuration settings.
mplayer tv:// -vo omapfb
Press q or ESC to stop playing and quit program.
Adding more parameters will do some customization as shown below.
mplayer tv:// -tv driver=v4l2:width=800:height=600:device=/dev/video0:brightness=80:contrast=10:saturation=-30 -fs -vf mirror -vo omapfb
Parameters description:
| mplayer tv:// | Stream data using TV capture module | ||
| -tv | This option tunes various properties of the TV capture module (see below). | ||
|
Set TV input driver. | ||
|
Output window width and height. | ||
|
Specify TV device. | ||
|
Specify the output format of the TV device. | ||
|
Set the image equalizer. | ||
| -fs | Fullscreen playback. Note: -fs is not necessarily supported by all MPlayer video output drivers. | ||
| -vf | Video filters allow modification of the video stream and its properties. | ||
|
Mirrors the image on the Y axis. | ||
|
Allows acquiring screenshots of the movie. Make sure that the screenshot filter is the last filter, so the exact picture on the display will be saved. | ||
| -vo | Specifies MPlayer video output driver. | ||
|
TI OMAP video frame buffer output. | ||
|
Separate output window if using X Window System. |
|
|
To start X Window application (e.g. MPlayer) from your serial console, the DISPLAY environment variable must be set. |
Additional example for streaming USB camera output to a separate window in the X Window system:
mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -vo xv
Following set of commands will set up ISP pipeline to Sensor->Preview->Resizer->Memory and stream 1024x768 video to OMAP framebuffer:
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], \
"OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 1280x1024], "OMAP3 ISP CCDC":2 [SGRBG10 1280x1024], "OMAP3 ISP preview":1 [UYVY 1280x1024], "OMAP3 ISP resizer":1 [UYVY 1024x768]'
mplayer tv:// -tv driver=v4l2:device=/dev/video6:outfmt=uyvy:width=1024:height=768 -vo omapfb
Snapshots
To take snapshots from USB Camera, run the MPlayer with the following command line:
mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -vf screenshot -vo omapfb
Once there is an output streamed on the display, press s to take a snapshot. Each snapshot will be saved in your current directory as shotXXXX.png.
ISP Camera snapshoot could be obtained by setting ISP pipeline as described in the previous section and running the following command:
mplayer tv:// -tv driver=v4l2:device=/dev/video6:outfmt=uyvy:width=1024:height=768 -vf screenshot -vo omapfb
Playback
To play a movie, run the MPlayer with the following command line:
mplayer media_file.ogg -vo omapfb
GStreamer
GStreamer is a pipeline-based multimedia framework that allows you to create a variety of media-handling software components. This includes simple audio playback, audio and video playback, recording, streaming, and editing. The pipeline design allows you to easily write any type of streaming multimedia application. GStreamers capabilities are extended through plugins. Several simple media manipulations are demonstrated below.
Installation
For the GStreamer to function properly, there are several packages that need to be installed. Those can be found on the The Ångström Distribution official website.
- To install the basic GStreamer set, use the following command:
opkg install libxml2_2.7.8-r9.1.6_armv7a.ipk libgcc1_4.5-r47+svnr184907_armv7a.ipk libc6_2.12-r28_armv7a.ipk libffi5_3.0.10-r0_armv7a.ipk \
libz1_1.2.6-r1_armv7a.ipk libglib-2.0-0_2.30.3-r0_armv7a.ipk gstreamer_0.10.36-r0_armv7a.ipk libxcb1_1.8.1-r0_armv7a.ipk \
libxv1_1.0.7-r0_armv7a.ipk libuuid1_2.21-r5_armv7a.ipk libudev0_182-r0_armv7a.ipk libgudev-1.0-0_182-r0_armv7a.ipk \
libgstinterfaces-0.10-0_0.10.36-r1_armv7a.ipk libgstvideo-0.10-0_0.10.36-r1_armv7a.ipk --force-overwrite
- For GStreamer to support Video4Linux2 and several plugins for manipulating the output stream, the following needs to be installed:
opkg install gst-plugins-good-video4linux2_0.10.31-r0_armv7a.ipk gst-plugin-videorate_0.10.35-r1_armv7a.ipk \
gst-plugin-ffmpegcolorspace_0.10.35-r1_armv7a.ipk gst-plugin-ximagesink_0.10.35-r1_armv7a.ipk
- To support encoding with Theora codec (required if you want to save the encoded video to a file), the following needs to be installed:
opkg install libogg0_1.3.0-r0_armv7a.ipk libtheora_1.1.1-r1_armv7a.ipk libgsttag-0.10-0_0.10.36-r1_armv7a.ipk gst-plugin-theora_0.10.35-r1_armv7a.ipk \
libgstpbutils-0.10-0_0.10.36-r1_armv7a.ipk libgstaudio-0.10-0_0.10.36-r1_armv7a.ipk libgstriff-0.10-0_0.10.36-r1_armv7a.ipk \
gst-plugin-ogg_0.10.35-r1_armv7a.ipk --force-overwrite
Validation
To validate the installation of the GStreamer components, launch gst-inspect GStreamer utility.
Testing the basic streaming to OMAP frame buffer:
gst-launch videotestsrc ! omapdmaifbsink
Streaming to display
To start streaming the video to display the simple GStreamer pipeline can be used.
The basic GStreamer pipeline should have at least the source of the stream and the output sink specified.
The below examples stream the video from the V4L2 video capture device to the OMAP frame buffer (sink) with several stream manipulations applied.
Run the following command to stream the USB camera to display
gst-launch v4l2src device=/dev/video0 ! 'video/x-raw-yuv,format=(fourcc)YUY2,width=640,height=480' ! videorate ! ffmpegcolorspace ! omapdmaifbsink
Set up ISP pipeline as described in MPlayer section and run the following command to stream ISP Camera to display
gst-launch v4l2src device=/dev/video6 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=1024,height=768' ! omapdmaifbsink
GStreamer pipeline parameters in details:
| v4l2src | Element for reading frames from a Video4Linux2 device. | ||
|
Specify the V4L2 /dev/video0 device as a stream source. | ||
|
Disable copying of the stream buffers. This can slightly improve the GStreamer performance. | ||
| video/x-raw-yuv,format=(fourcc)YUY2,width=640,height=480 | Link the source to a capability, which is defined by a mime type and a few optional properties. The properties are: video and resolution. | ||
| videorate | Use each frame of the source stream and feed it to the next element at the requested framerate. | ||
| ffmpegcolorspace | Make the color space conversion, required by the sink element. | ||
| omapdmaifbsink | Define the OMAP frame buffer as the output sink. Sink can be: a file, screen or a network. For X Window System, the ximagesink parameter can be used. |
Streaming to a file
This section examples demonstrate how to encode and save the output stream to a file, using the Theora codec.
Run the following command to save the USB Camera output stream to a file:
gst-launch v4l2src ! 'video/x-raw-yuv,width=320,height=240' ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! \
theoraenc ! queue ! oggmux ! filesink location=media_file.ogg
Run following set of commands to save the ISP Camera output stream to a file:
media-ctl -r -l '"mt9t001 3-005d":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], \
"OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
media-ctl -f '"mt9t001 3-005d":0 [SGRBG10 320x240], "OMAP3 ISP CCDC":2 [SGRBG10 320x240], "OMAP3 ISP preview":1 [UYVY 320x240], \
"OMAP3 ISP resizer":1 [UYVY 320x240]'
gst-launch v4l2src device=/dev/video6 ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! \
queue ! ffmpegcolorspace ! theoraenc ! queue ! oggmux ! filesink location=media_file.ogg
GStreamer parameters in details:
| queue | Provides a buffer for the next element in the pipeline. |
| theoraenc | Encodes raw video into theora stream. |
| oggmux | Muxes a stream into an ogg container. |
| filesink | Writes the incoming data to a file. |
Streaming to a file and display
Following command splits the source stream from USB Camera and links to the frame buffer and to an encoded file:
gst-launch v4l2src always-copy=false ! 'video/x-raw-yuv,width=320,height=240' ! tee name=t_video ! queue ! videorate ! ffmpegcolorspace ! \
omapdmaifbsink t_video. ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! theoraenc ! \
queue ! oggmux ! filesink location=media_file.ogg
The same for ISP Camera using ISP pipeline settings from the previous section:
gst-launch v4l2src device=/dev/video6 always-copy=false ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=320,height=240' ! tee name=t_video ! \
queue ! videorate ! ffmpegcolorspace ! omapdmaifbsink t_video. ! queue ! videorate ! 'video/x-raw-yuv,framerate=10/1' ! ffmpegcolorspace ! \
theoraenc ! queue ! oggmux ! filesink location=media_file.ogg
| tee | Split data to multiple streams (element pads). | ||
|
Names the element's pad. t_video is then linked to theora encoder. |
File playback
To play a movie encoded by Theora codec and saved in ogg contaner, use the following command line:
gst-launch filesrc location=media_file.ogg ! oggdemux ! theoradec ! ffmpegcolorspace ! omapdmaifbsink
See also
- Linux UVC driver and tools
- Camera ISP Driver
- MPlayer official resource
- GStreamer official resource
- Introducing GStreamer
- GStreamer TI Plugin
- Ångström Distribution Feed browser
- CM-T3730: Linux: Getting started
- CM-T3730: Linux: Angstrom
- CM-T3730: Linux: Kernel
- Video Input Port Adapter Board
- CM-T3530: Linux: Video playback
- Linux Development for ARM modules
