CM-T3730: Linux: Video Playback

Overview

The CM-T3730 features DM3730 Digital Media System on Chip (SoC) with High Performance Image, Video, Audio (IVA) Accelerator Subsystem. It includes TMS320C64x+™ DSP Core (up to 800-MHz), Enhanced Direct Memory Access (EDMA) Controller and Video Hardware Accelerators.

DM3730 supports hardware acceleration of the following Video/Audio codecs:

Video (up to 720p) : H.264 - Baseline, Medium, and High profiles, JPEG, MPEG2 - Medium profile, MPEG4 - Simple profile, VC1

Audio : AAC - High-Efficiency and Low-Complexity profiles, G.711, MP3, WMA

CM-T3730 Linux package supports DaVinci Multimedia Application Interface (DMAI). DMAI is a utility layer on top of the operating system and the Codec Engine (CE). It is able to interact with the OMAP DSP or acceleration modules for encoding and decoding tasks. It allows offloading the processing of the applications running on MPU to OMAP hardware accelerators. The TI plugin for GStreamer multimedia framework is built on top of DMAI. In this article we describe sample application of GStreamer to demonstrate hardware acceleration features for video and audio stream playback.

We also provide an overview of the FFmpeg application to prepare media files compatible with CM-T3730 hardware decoding capabilities.

Boot Loader and Linux Kernel

To set the maximum permitted frequency of the DSP processor core, please run the following command in U-Boot before booting the kernel:

mw 48004040 0x0009900c

Linux kernel boot options require adjustment to enable DSP support. Following example modifies default CM-T3730 boot options by designating additional memory and OMAP frame-buffer related parameters:

omapfb.mode=dvi:1024x768-24@60 omapdss.def_disp=dvi vram=18M omapfb.vram=0:6M,1:6M,2:6M mem=55M@0x80000000 mem=128M@0x88000000

Please refer to CM-T3730: Linux: Angstrom and CM-T3730: Linux: Debian articles for Linux DMAI support on CM-T3730.

All provided playback examples were tested with 1024x768 DVI Monitor. If you want to test the playback of 720p video, please change kernel command line options accordingly, for example by setting display resolution to 1280x1024:

omapfb.mode=dvi:1280x1024-24@60

GStreamer

GStreamer is a pipeline-based multimedia framework that allows creating a wide variety of media-handling software components. This includes simple audio playback, audio and video playback, recording, streaming, and editing. The pipeline design allows easy creation of any type of streaming multimedia application. GStreamer capabilities can be extended through plugins. For instance, TI GStreamer plugin uses DMAI to make access to the underlying hardware and DSP to perform tasks, such as video encoding/decoding, using hardware acceleration. This article demonstrates media manipulations by using of hardware acceleration. GStreamer package and required plugins are included in the CM-T3730 Angstrom and Debian Linux root filesystem image.

Validation

GStreamer test video stream output

To validate the installation of the GStreamer components, launch gst-inspect GStreamer utility.

Testing the basic streaming to OMAP frame buffer:

gst-launch videotestsrc ! omapdmaifbsink

GStreamer pipeline parameters in details:

videotestsrc	Creates a test video stream.
omapdmaifbsink	Define the OMAP frame buffer as the output sink.

Test basic encoding/decoding capabilities by using synthetic video test pattern and measure MPU/DPS performance. The same applies to any playback example in this article.

Following pipeline encodes 640x480 test video stream using H.264 codec:

gst-launch videotestsrc ! 'video/x-raw-yuv, width=640, height=480' ! TIVidenc1 codecName=h264enc engineName=codecServer ! \
           dmaiperf print-arm-load=true engine-name=codecServer ! filesink location=testsrc_sample.264

Press Ctrl+C to stop streaming.
The following pipeline plays the recorded video:

gst-launch filesrc location=testsrc_sample.264 ! typefind ! TIViddec2 ! \
           dmaiperf print-arm-load=true engine-name=codecServer ! omapdmaifbsink

GStreamer pipeline parameters in detail:

video/x-raw-yuv, width=640, height=480	Link the source to a capability, which is defined by a mime type and a few optional properties. The properties are: video and resolution.
typefind	Find the media type of a stream
TIVidenc1	Encode video stream.
codecName=h264enc	Specify H.264 codec. Another possible values are: mpeg2enc, mpeg4enc.
engineName=codecServer	Specify codec server.
TIViddec2	Decode video stream.
codecName=h264dec	Specify H.264 decoder. Another possible values are: mpeg2dec, mpeg4dec. If the upstream element (i.e. demuxer or typefind element) is able to negotiate with the decoder element and expose its capabilities, the engineName and codecName properties will be auto-detected based on the mime type requested.
engineName=codecServer	Specify codec server.
dmaiperf	Capture pipeline performance data.
print-arm-load=true	Print the ARM CPU load information.
engine-name=codecServer	Specify codec server.
filesrc	Read data from a file.
location=testsrc_sample.264	Specify input file name.
filesink	Write incoming data to a file.
location=testsrc_sample.264	Specify output file name.

To output a video in a separate window in X windows environment, the DISPLAY environment variable must be set:

export DISPLAY=:0

Playback

720x480 H.264/AAC movie playback.

This section provides several examples of GStreamer video playback pipelines. More TI DMAI GStreamer Plugin applications and use cases can be found in See also section. Encoding and network streaming of live camera video can be found in the Camera article.

• Auto-playback of all supported video / audio formats.

gst-launch playbin2 uri=file:///home/root/test.avi video-sink=omapdmaifbsink

• Playback of plain (not muxed into container) MPEG-2/MPEG-4/H.264 video file.

gst-launch filesrc location=test.bin ! typefind ! TIViddec2 ! omapdmaifbsink

• Playback of plain MPEG-4 video file

gst-launch filesrc location=test.bin ! TIViddec2 codecName=mpeg4dec engineName=codecServer  ! omapdmaifbsink

• Playback of all supported video muxed into AVI container

gst-launch filesrc location=test.avi ! avidemux name=demux demux. ! TIViddec2  ! omapdmaifbsink

• Playback of all supported video / audio muxed into MOV container

gst-launch filesrc location=test.mov ! qtdemux name=demux demux. ! queue ! TIViddec2  ! omapdmaifbsink demux. ! \
           queue  max-size-time=0 ! TIAuddec1 ! alsasink

GStreamer pipeline parameters in detail:

playbin2	Auto-plug and play media from an uri. playbin2 builds a playback pipeline using automatic file type recognition and automatic selection of the right audio/video/subtitle demuxers/decoders. The resulting pipeline is a bit more complicated than explicit custom pipelines and can increase the MPU load. This element is very simple to use and supports HW accelerated playback.
uri=file:///home/root/test.avi	Specify a media file for playback.
queue	Provide a buffer for the next element in the pipeline. It also required to force the use of separate threads for audio and video decoding.
max-size-time=0	Max. amount of data in the queue in nano seconds.
qtdemux	Demux a .mov file into audio and video streams. This element can be replaced by avidemux, oggdemux, mpegtsdemux etc. depending on the input file container format.
name=demux	Specify demuxer name. Respective pads of demux. are then linked to video and audio decoders.
TIAuddec1	Decode audio. Due to the licensing restriction DVSDK does not come with mp3 support integrated in the default codec server.
alsasink	Output to a sound card via ALSA subsystem.

If a video or an audio decoder element doesn't support input media stream type, GStreamer will output a failure report.

FFmpeg

FFmpeg is a very fast video and audio converter application. It can be installed on the desktop computer to prepare media files compatible with CM-3730 hardware accelerated playback capabilities. This article describes FFmpeg installation and usage for Ubuntu 11.10 Linux distribution.

Installation

This section describes the compilation and installation of the FFmpeg from the packages upstream repositories. Installation from sources allows to benefit from bug-fixes, enhancements and most recent FFmpeg version. For example, Ubutnu repository FFmpeg package application is precompiled with restriction to use of proprietary or commercial encoders like MP3, AAC, MPEG4 etc.

• Remove previously installed applications and libraries

sudo apt-get remove ffmpeg x264 libx264-dev libva-dev

• Install dependancies

sudo apt-get -y install build-essential checkinstall git libfaac-dev libgpac-dev libjack-jackd2-dev libmp3lame-dev \
                        libopencore-amrnb-dev libopencore-amrwb-dev librtmp-dev libsdl1.2-dev libtheora-dev libva-dev \
                        libvdpau-dev libvorbis-dev libx11-dev libxfixes-dev texi2html yasm zlib1g-dev

• x264 library

git clone git://git.videolan.org/x264
cd x264
./configure --prefix=/usr --enable-static && make
sudo checkinstall --pkgname=x264 --pkgversion="3:$(./version.sh | awk -F'[" ]' '/POINT/{print $4"+git"$5}')" \
                  --backup=no --deldoc=yes --fstrans=no --default

• FFmpeg

git clone --depth 1 git://source.ffmpeg.org/ffmpeg
cd ffmpeg
./configure --prefix=/usr --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb \ 
            --enable-libopencore-amrwb --enable-librtmp --enable-libtheora --enable-libvorbis \
            --enable-libx264 --enable-nonfree --enable-version3 --enable-x11grab && make
sudo checkinstall --pkgname=ffmpeg --pkgversion="5:$(date +%Y%m%d%H%M)-git" --backup=no --deldoc=yes \
                  --fstrans=no --default

hash x264 ffmpeg ffplay ffprobe

Usage

• Convert to H.264 (High Profile) 720x480 video

ffmpeg -i sample_1080p.mov -vcodec libx264 -vprofile high -preset slow -r 30 -s 720:480 -threads 0 -an -f h264 test.bin

• Convert to MPEG4 (Simple Profile) 720x480 video

ffmpeg -i sample_1080p.mov -vcodec mpeg4 -b:v 1024k -r 30 -s 720:480 -threads 0 -an -f m4v test.bin

• Convert to H.264 (High Profile) 720x480 video, AAC 128Kbit/s audio muxed into MOV container

ffmpeg -i sample_1080p.mov -vcodec libx264 -vprofile high -preset slow -s 720:480 -threads 0 \
       -acodec libfaac -ab 128k test.mov

FFmpeg command line parameters in detail:

-i sample_1080p.mov	Specify input file name.
-vcodec libx264	Encode video to H.264 format using libx264 library.
-vprofile high	Force the limits of the H.264 profile to high. Other valid options are baseline and main (supported by CM-T3730).
-preset slow	Set encoding preset for x264 - slower presets give more quality at the same bitrate, but take more time to encode. slow is a good balance between encoding time and quality. Other valid options are: ultrafast, superfast, veryfast, faster, fast, medium, slower, veryslow, placebo.
-vcodec mpeg4	Encode video to MPEG4.
-b:v 1024k	Set video bitrate to 1024 kbits/s.
-r 30	Set video framerate to 30 fps.
-s 720:480	Specify output frame size - 720x480.
-acodec libfaac	Encode audio to AAC format using libfaac library.
-ab 128k	Set audio bitrate to 128 kbits/s.
-an	Disable audio.
-f m4v	Force setting of output file format to MPEG4.
-threads 0	Use an optimal number of threads when encoding.
test.mov	Specify output file and the mov muxer (autodetected from extension). Anything found on the command line which cannot be interpreted as an option is considered to be an output filename.

See also

• DMAI GStreamer Plug-In Elements
• DMAI GStreamer Plug-In
• GStreamer official resource
• FFmpeg official resource

• CM-T3730: Linux: Angstrom - Digital Video SDK
• CM-T3730: Linux: Debian - Digital Video SDK
• CM-T3730: Linux: Camera
• CM-T3730: Linux: Getting started
• CM-T3730: Linux: Angstrom
• CM-T3730: Linux: Debian
• CM-T3730: Linux: Kernel