CM-T3730: Linux: Video Playback

From Compulab Mediawiki
Jump to: navigation, search

Overview

The CM-T3730 features DM3730 Digital Media System on Chip (SoC) with High Performance Image, Video, Audio (IVA) Accelerator Subsystem. It includes TMS320C64x+™ DSP Core (up to 800-MHz), Enhanced Direct Memory Access (EDMA) Controller and Video Hardware Accelerators.

DM3730 supports hardware acceleration of the following Video/Audio codecs:

Video (up to 720p) : H.264 - Baseline, Medium, and High profiles, JPEG, MPEG2 - Medium profile, MPEG4 - Simple profile, VC1
Audio : AAC - High-Efficiency and Low-Complexity profiles, G.711, MP3, WMA

CM-T3730 Linux package supports DaVinci Multimedia Application Interface (DMAI). DMAI is a utility layer on top of the operating system and the Codec Engine (CE). It is able to interact with the OMAP DSP or acceleration modules for encoding and decoding tasks. It allows offloading the processing of the applications running on MPU to OMAP hardware accelerators. The TI plugin for GStreamer multimedia framework is built on top of DMAI. In this article we describe sample application of GStreamer to demonstrate hardware acceleration features for video and audio stream playback.

We also provide an overview of the FFmpeg application to prepare media files compatible with CM-T3730 hardware decoding capabilities.

Boot Loader and Linux Kernel

To set the maximum permitted frequency of the DSP processor core, please run the following command in U-Boot before booting the kernel:

mw 48004040 0x0009900c

Linux kernel boot options require adjustment to enable DSP support. Following example modifies default CM-T3730 boot options by designating additional memory and OMAP frame-buffer related parameters:

omapfb.mode=dvi:1024x768-24@60 omapdss.def_disp=dvi vram=18M omapfb.vram=0:6M,1:6M,2:6M mem=55M@0x80000000 mem=128M@0x88000000

Please refer to CM-T3730: Linux: Angstrom and CM-T3730: Linux: Debian articles for Linux DMAI support on CM-T3730.

All provided playback examples were tested with 1024x768 DVI Monitor. If you want to test the playback of 720p video, please change kernel command line options accordingly, for example by setting display resolution to 1280x1024:

omapfb.mode=dvi:1280x1024-24@60

GStreamer

GStreamer is a pipeline-based multimedia framework that allows creating a wide variety of media-handling software components. This includes simple audio playback, audio and video playback, recording, streaming, and editing. The pipeline design allows easy creation of any type of streaming multimedia application. GStreamer capabilities can be extended through plugins. For instance, TI GStreamer plugin uses DMAI to make access to the underlying hardware and DSP to perform tasks, such as video encoding/decoding, using hardware acceleration. This article demonstrates media manipulations by using of hardware acceleration. GStreamer package and required plugins are included in the CM-T3730 Angstrom and Debian Linux root filesystem image.

Validation

GStreamer test video stream output

To validate the installation of the GStreamer components, launch gst-inspect GStreamer utility.

Testing the basic streaming to OMAP frame buffer:

gst-launch videotestsrc ! omapdmaifbsink

GStreamer pipeline parameters in details:

videotestsrc Creates a test video stream.
omapdmaifbsink Define the OMAP frame buffer as the output sink.


Test basic encoding/decoding capabilities by using synthetic video test pattern and measure MPU/DPS performance. The same applies to any playback example in this article.

Following pipeline encodes 640x480 test video stream using H.264 codec:

gst-launch videotestsrc ! 'video/x-raw-yuv, width=640, height=480' ! TIVidenc1 codecName=h264enc engineName=codecServer ! \
           dmaiperf print-arm-load=true engine-name=codecServer ! filesink location=testsrc_sample.264

Press Ctrl+C to stop streaming.
The following pipeline plays the recorded video:

gst-launch filesrc location=testsrc_sample.264 ! typefind ! TIViddec2 ! \
           dmaiperf print-arm-load=true engine-name=codecServer ! omapdmaifbsink

GStreamer pipeline parameters in detail:

video/x-raw-yuv, width=640, height=480 Link the source to a capability, which is defined by a mime type and a few optional properties. The properties are: video and resolution.
typefind Find the media type of a stream
TIVidenc1 Encode video stream.
codecName=h264enc
Specify H.264 codec. Another possible values are: mpeg2enc, mpeg4enc.
engineName=codecServer
Specify codec server.
TIViddec2 Decode video stream.
codecName=h264dec
Specify H.264 decoder. Another possible values are: mpeg2dec, mpeg4dec. If the upstream element (i.e. demuxer or typefind element) is able to negotiate with the decoder element and expose its capabilities, the engineName and codecName properties will be auto-detected based on the mime type requested.
engineName=codecServer
Specify codec server.
dmaiperf Capture pipeline performance data.
print-arm-load=true
Print the ARM CPU load information.
engine-name=codecServer
Specify codec server.
filesrc Read data from a file.
location=testsrc_sample.264
Specify input file name.
filesink Write incoming data to a file.
location=testsrc_sample.264
Specify output file name.

To output a video in a separate window in X windows environment, the DISPLAY environment variable must be set:

export DISPLAY=:0

Playback

720x480 H.264/AAC movie playback.

This section provides several examples of GStreamer video playback pipelines. More TI DMAI GStreamer Plugin applications and use cases can be found in See also section. Encoding and network streaming of live camera video can be found in the Camera article.

  • Auto-playback of all supported video / audio formats.
gst-launch playbin2 uri=file:///home/root/test.avi video-sink=omapdmaifbsink
  • Playback of plain (not muxed into container) MPEG-2/MPEG-4/H.264 video file.
gst-launch filesrc location=test.bin ! typefind ! TIViddec2 ! omapdmaifbsink
  • Playback of plain MPEG-4 video file
gst-launch filesrc location=test.bin ! TIViddec2 codecName=mpeg4dec engineName=codecServer  ! omapdmaifbsink
  • Playback of all supported video muxed into AVI container
gst-launch filesrc location=test.avi ! avidemux name=demux demux. ! TIViddec2  ! omapdmaifbsink
  • Playback of all supported video / audio muxed into MOV container
gst-launch filesrc location=test.mov ! qtdemux name=demux demux. ! queue ! TIViddec2  ! omapdmaifbsink demux. ! \
           queue  max-size-time=0 ! TIAuddec1 ! alsasink

GStreamer pipeline parameters in detail:

playbin2 Auto-plug and play media from an uri. playbin2 builds a playback pipeline using automatic file type recognition and automatic selection of the right audio/video/subtitle demuxers/decoders. The resulting pipeline is a bit more complicated than explicit custom pipelines and can increase the MPU load. This element is very simple to use and supports HW accelerated playback.
uri=file:///home/root/test.avi
Specify a media file for playback.
queue Provide a buffer for the next element in the pipeline. It also required to force the use of separate threads for audio and video decoding.
max-size-time=0
Max. amount of data in the queue in nano seconds.
qtdemux Demux a .mov file into audio and video streams. This element can be replaced by avidemux, oggdemux, mpegtsdemux etc. depending on the input file container format.
name=demux
Specify demuxer name. Respective pads of demux. are then linked to video and audio decoders.
TIAuddec1 Decode audio. Due to the licensing restriction DVSDK does not come with mp3 support integrated in the default codec server.
alsasink Output to a sound card via ALSA subsystem.


Admolition note.png If a video or an audio decoder element doesn't support input media stream type, GStreamer will output a failure report.

FFmpeg

FFmpeg is a very fast video and audio converter application. It can be installed on the desktop computer to prepare media files compatible with CM-3730 hardware accelerated playback capabilities. This article describes FFmpeg installation and usage for Ubuntu 11.10 Linux distribution.

Installation

This section describes the compilation and installation of the FFmpeg from the packages upstream repositories. Installation from sources allows to benefit from bug-fixes, enhancements and most recent FFmpeg version. For example, Ubutnu repository FFmpeg package application is precompiled with restriction to use of proprietary or commercial encoders like MP3, AAC, MPEG4 etc.

  • Remove previously installed applications and libraries
sudo apt-get remove ffmpeg x264 libx264-dev libva-dev
  • Install dependancies
sudo apt-get -y install build-essential checkinstall git libfaac-dev libgpac-dev libjack-jackd2-dev libmp3lame-dev \
                        libopencore-amrnb-dev libopencore-amrwb-dev librtmp-dev libsdl1.2-dev libtheora-dev libva-dev \
                        libvdpau-dev libvorbis-dev libx11-dev libxfixes-dev texi2html yasm zlib1g-dev
  • x264 library
git clone git://git.videolan.org/x264
cd x264
./configure --prefix=/usr --enable-static && make
sudo checkinstall --pkgname=x264 --pkgversion="3:$(./version.sh | awk -F'[" ]' '/POINT/{print $4"+git"$5}')" \
                  --backup=no --deldoc=yes --fstrans=no --default
  • FFmpeg
git clone --depth 1 git://source.ffmpeg.org/ffmpeg
cd ffmpeg
./configure --prefix=/usr --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb \ 
            --enable-libopencore-amrwb --enable-librtmp --enable-libtheora --enable-libvorbis \
            --enable-libx264 --enable-nonfree --enable-version3 --enable-x11grab && make
sudo checkinstall --pkgname=ffmpeg --pkgversion="5:$(date +%Y%m%d%H%M)-git" --backup=no --deldoc=yes \
                  --fstrans=no --default
hash x264 ffmpeg ffplay ffprobe

Usage

  • Convert to H.264 (High Profile) 720x480 video
ffmpeg -i sample_1080p.mov -vcodec libx264 -vprofile high -preset slow -r 30 -s 720:480 -threads 0 -an -f h264 test.bin
  • Convert to MPEG4 (Simple Profile) 720x480 video
ffmpeg -i sample_1080p.mov -vcodec mpeg4 -b:v 1024k -r 30 -s 720:480 -threads 0 -an -f m4v test.bin
  • Convert to H.264 (High Profile) 720x480 video, AAC 128Kbit/s audio muxed into MOV container
ffmpeg -i sample_1080p.mov -vcodec libx264 -vprofile high -preset slow -s 720:480 -threads 0 \
       -acodec libfaac -ab 128k test.mov

FFmpeg command line parameters in detail:

-i sample_1080p.mov Specify input file name.
-vcodec libx264 Encode video to H.264 format using libx264 library.
-vprofile high
Force the limits of the H.264 profile to high. Other valid options are baseline and main (supported by CM-T3730).
-preset slow
Set encoding preset for x264 - slower presets give more quality at the same bitrate, but take more time to encode. slow is a good balance between encoding time and quality. Other valid options are: ultrafast, superfast, veryfast, faster, fast, medium, slower, veryslow, placebo.
-vcodec mpeg4 Encode video to MPEG4.
-b:v 1024k Set video bitrate to 1024 kbits/s.
-r 30 Set video framerate to 30 fps.
-s 720:480 Specify output frame size - 720x480.
-acodec libfaac Encode audio to AAC format using libfaac library.
-ab 128k Set audio bitrate to 128 kbits/s.
-an Disable audio.
-f m4v Force setting of output file format to MPEG4.
-threads 0 Use an optimal number of threads when encoding.
test.mov Specify output file and the mov muxer (autodetected from extension). Anything found on the command line which cannot be interpreted as an option is considered to be an output filename.

See also