Linux: Setting up a development environment

Overview

Development for an embedded device presents some unique challenges:

• Software development is almost always done on a system other than the one on which the software will eventually run. It is common nowadays to use x86 desktop workstations to develop software for ARM-based embedded products. Such an approach requires cross-compilation tools and environment.
• Communication with the embedded device is almost always done remotely. This requires the use of communication tools such as ssh or serial communication programs.
• Software deployment entails a lot of overhead: each change to the kernel version requires that all the kernel modules be replaced on the device as well. Filesystem updates can take a long time.

This article will guide you through setting up a usable development environment to tackle the unique challenges of embedded software development.

Cross-Compilation

A cross compiler makes it possible to compile software for a different system than the one on which development is done. In cross-compiler terminology, the host system is the system on which development and compilation is performed (usually an x86 workstation), while the target system is the architecture on which the software will run (such as ARM).

There are several options for cross-compilation toolchain setup. The cross-compiler should support the ARM embedded-application binary interface ("EABI").
Recommended tools:

• Linaro GNU soft floating point
• Linaro GNU hard floating point
• ELDK

Communicating with the embedded device

During advanced stages of development it is often possible to connect a keyboard (and mouse) to the embedded device and control it directly. However, this is usually not an option during the early stages of development, and it is not the most convenient option since it forces the developer to get away from their development workstation to engage the device. A more convenient option is connecting to the device using ssh, or via a serial port.

Serial port communication

The two most popular serial communication programs are minicom and kermit. Both programs also support scripting for the communication session, which can be used to automate software tests and deployment tasks.

Working with kermit

The following kermit settings should work with the vast majority of CompuLab SoM products.

• Create a kermit.conf file with the following content:

workstation-pc # cat << eof > /path/to/kermit.conf
set carrier-watch off
set handshake none
set flow-control none 
set key \127 \008
robust
eof

• Run kermit (set TTY to the appropriate device file, e.g. ttyS0 or ttyUSB0):

workstation-pc # sudo kermit -l /dev/<TTY> -b 115200 -y /path/to/kermit.conf

• You should now see the kermit prompt. Type "connect" to start communicating with the device:

C-Kermit 9.0.302 OPEN SOURCE:, 20 Aug 2011, for Linux+SSL+KRB5 (64-bit)
 Copyright (C) 1985, 2011,
  Trustees of Columbia University in the City of New York.
Type ? or HELP for help.
(/home/) C-Kermit> connect

ssh

ssh stands for "Secure SHell". It is an encrypted network protocol that facilitates remote login over an unsecured network. ssh can be used in advanced stages of development when the device OS has support for network communication and a running user space. Using ssh requires knowing the IP address of the embedded device, so if it is not known in advance you will have to access the device in some other way and query it using the ifconfig command. For this reason ssh is best used as an alternative means of communication with the device.

• Make sure your workstation is in the same network as the device, and initiate the ssh connection:

workstation-pc # ssh root@<device IP address>

Software deployment

Two tools that significantly simplify software deployment are TFTP (Trivial File Transfer Protocol) and NFS (Network File System).

TFTP is a simplified version of FTP. Its main advantage is that it is basic enough to be easily supported by bootloaders like U-Boot. TFTP can be used to quickly load a new bootloader or Linux kernel binaries via network once your bootloader has network support.

With NFS you can store the embedded device's filesystem on your development machine instead of storing it on the embedded device itself (the embedded device will connect to your workstation as part of Linux boot and will look for the filesystem in your NFS folder). Working this way has multiple adventages:

• Managing kernel modules is easier: instead of manually copying new kernel modules to the embedded device, the Linux build script can be setup to install new modules into the filesystem in your NFS folder.
• Sending files to the remote device is easier (just copy them to the filesystem in your NFS folder).
• It avoids filesystem divergence between embedded devices by using the exact same filesystem for all the devices.

Setting up tftp

• Install the following packages:

workstation-pc # sudo apt-get install xinetd tftpd tftp

• Create /etc/xinetd.d/tftp:

workstation-pc # cat << eof > /etc/xinetd.d/tftp
service tftp
{
protocol        = udp
port            = 69
socket_type     = dgram
wait            = yes
user            = nobody
server          = /usr/sbin/in.tftpd
server_args     = /tftpboot
disable         = no
}
eof

• Create the folder described in server_args and provide it with the appropriate permissions:

workstation-pc # sudo mkdir /tftpboot
workstation-pc # sudo chmod -R 777 /tftpboot

• Restart the xinetd service

workstation-pc # sudo /etc/init.d/xinetd restart

• You can test the tftp setup by dropping a file into your /tftpboot folder, booting the embedded device into U-Boot bootloader, and attempt to download the file to memory:

U-Boot# dhcp
link up on port 0, speed 1000, full duplex
BOOTP broadcast 1
BOOTP broadcast 2
DHCP client bound to address 192.168.50.219 (834 ms)
U-Boot# setenv serverip <your ip address>
U-Boot# tftpboot 80a00000 test.img
link up on port 0, speed 1000, full duplex
Using cpsw device
TFTP from server <your ip address>; our IP address is 192.168.50.219
Filename 'test.img'.
Load address: 0x80a00000
Loading: ##
	 2.4 MiB/s
done
Bytes transferred = 10091 (276b hex)

Setting up nfs

• Install the nfs server

workstation-pc # sudo apt-get install nfs-kernel-server

• Edit /etc/exports file and add the following entry:

/path/to/nfs/export	*(rw,no_subtree_check,sync,no_root_squash)

• Start the nfs server

workstation-pc # sudo service nfs-kernel-server start

• You can now use kernel bootargs to point the kernel to the filesystem located in your nfs exported file. For example:

U-Boot# setenv bootargs 'console=ttyO0,115200n8 root=/dev/nfs nfsroot=<serverip>:/path/to/nfsroot'

Chroot with QEMU

QEMU is a hardware emulator and hypervisor. One of its features is providing a translation layer that dynamically translates computer instructions from one architecture to another. This makes it possible to host and chroot into a filesystem that was compiled for a different architecture than the workstation architecture. You can use this feature to interact with a filesystem that is intended to be installed on the embedded target as if you were actually running an ARM device, but your session will be backed by the x86 hardware.

Binfmt_misc

The binfmt_misc Linux kernel feature allows running of different binary formats provided the existence of an appropriate interpreter. In conjunction with statically built qemu-arm or qemu-armeb the binfmt_misc provides ability to run ARM binaries on x86 systems.

Most modern Linux distributions enable the binfmt_misc support by default

The binfmt_misc should be enabled in the Linux kernel configuration in "Userspace binary formats" menu:

 .config - Linux/arm 4.2.0-cm-t43-4 Kernel Configuration
 > Userspace binary formats ───────────────────────────────────────────────────
  ┌─────────────────────── Userspace binary formats ────────────────────────┐
  │  Arrow keys navigate the menu.  <Enter> selects submenus ---> (or empty │  
  │  submenus ----).  Highlighted letters are hotkeys.  Pressing <Y>        │  
  │  includes, <N> excludes, <M> modularizes features.  Press <Esc><Esc> to │  
  │  exit, <?> for Help, </> for Search.  Legend: [*] built-in  [ ]         │  
  │ ┌─────────────────────────────────────────────────────────────────────┐ │  
  │ │    [*] Kernel support for ELF binaries                              │ │  
  │ │    [*] Write ELF core dumps with partial segments                   │ │  
  │ │    <*> Kernel support for scripts starting with #!                  │ │  
  │ │    <*> Kernel support for MISC binaries                             │ │  
  │ │                                                                     │ │  
  │ └─────────────────────────────────────────────────────────────────────┘ │  
  ├─────────────────────────────────────────────────────────────────────────┤  
  │        <Select>    < Exit >    < Help >    < Save >    < Load >         │  
  └─────────────────────────────────────────────────────────────────────────┘

QEMU

Statically build QEMU ARM emulator should reside in the target (ARM) filesystem to allow invocation of ARM binaries on x86 system using binfmt_misc.

Ubuntu Karmic Koala, Debian Sid and Squeeze and probably other Linux distributions provide pre-built packages for static QEMU

If the binary package is not available in your distribution, building static version of QEMU is straightforward:

wget http://download.savannah.gnu.org/releases/qemu/qemu-0.11.1.tar.gz
tar xzf qemu-0.11.1.tar.gz
cd qemu-0.11.1
./configure --static --target-list="armeb-linux-user arm-linux-user"
make
cp arm-linux-user/qemu-arm armeb-linux-user/qemu-armeb /path/to/target/rootfs/user/local/bin

qemu-arm and qemu-armeb have to be registered with binfmt_misc as ARM binary interpreters:

echo ":qemu-arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:\
/usr/local/bin/qemu-arm:" > /proc/sys/fs/binfmt_misc/register
echo ":qemu-armeb:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28:\
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:\
/usr/local/bin/qemu-armeb:" > /proc/sys/fs/binfmt_misc/register

After the above setup is complete it is possible to execute

workstation-pc # chroot /path/to/target/rootfs /bin/sh

and use the chroot'ed filesystem for application development