Difference between revisions of "CM-QS600: Android: Building from source code"

From Compulab Mediawiki
Jump to: navigation, search
(Flash ''boot.img'' onto eMMC)
 
(2 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
= Overview =
 +
The following outlines the procedure necessary to acquire the Android source code and build it for CM-QS600 System-on-Module / Computer-on-Module.<br/>
 +
CompuLab Android package is obtained by applying CompuLab patches on top of Code Aurora Forum's Android package for APQ8064 CDP platform.
 +
 
= Prerequisites =
 
= Prerequisites =
The required steps, as outlined below, were tested on Ubuntu 14.04 (Trusty).<br/>
+
The required steps, as described below, were tested on Ubuntu 14.04 (Trusty).<br/>
 
Install required packages, as outlined on [http://source.android.com/source/initializing.html#installing-required-packages-ubuntu-1404  AOSP web-page].<br/>
 
Install required packages, as outlined on [http://source.android.com/source/initializing.html#installing-required-packages-ubuntu-1404  AOSP web-page].<br/>
 
Install additional packages (not listed on AOSP web-page):
 
Install additional packages (not listed on AOSP web-page):
  $ sudo apt-get install xz-utils make flex lib32z1 zip
+
  $ sudo apt-get install xz-utils make flex lib32z1 zip curl
 +
Install {{filename|repo}} tool for work with multiple {{filename|git}} repositories in the context of Android:
 +
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
 +
$ chmod a+x ~/bin/repo
  
 
= Getting Android Source Code =
 
= Getting Android Source Code =
== Overview ==
+
== Deploy Code Aurora Forum base ==
CompuLab Android package is obtained by applying CompuLab patches on top of Code Aurora Forum's Android package for MSM APQ8064 CDP platform.
+
Download the code base from Code Aurora Forum (CAF):
 
 
== Download software packages ==
 
Proceed to CM-QS600 [http://compulab.co.il/products/computer-on-modules/cm-qs600/#devres Downloads] page in order to acquire the following software packages:
 
* '''Qualcomm proprietary libraries'''
 
: Prebuilt Qualcomm proprietary libraries.
 
* '''CompuLab patches on top of Android for MSM APQ8064'''
 
: CompuLab patches on top of Android source code tree include, among other, Linux kernel patches, Android device configuration files and firmware binary blobs.
 
  
== Deploy Android source code repository ==
+
  $ mkdir ~/android
=== Code Aurora Forum base ===
+
  $ cd ~/android
Download the code base from Code Aurora Forum (CAF):
 
  $ mkdir myandroid
 
  $ cd myandroid
 
 
  $ repo init -u git://codeaurora.org/platform/manifest.git -b release -m '''LNX.LA.2.7-06310-8064.0.xml''' --repo-url=git://codeaurora.org/tools/repo.git --repo-branch=caf-stable
 
  $ repo init -u git://codeaurora.org/platform/manifest.git -b release -m '''LNX.LA.2.7-06310-8064.0.xml''' --repo-url=git://codeaurora.org/tools/repo.git --repo-branch=caf-stable
 
  $ repo sync -j4
 
  $ repo sync -j4
Line 27: Line 24:
 
{{Note | The last instruction maintains git branch naming convention, which is useful for clarity, but is not strictly necessary.}}
 
{{Note | The last instruction maintains git branch naming convention, which is useful for clarity, but is not strictly necessary.}}
  
=== Qualcomm proprietary prebuilt libraries ===
+
== Deploy Wireless kernel driver ==
 +
Clone out of tree wireless kernel driver:
 +
<pre>
 +
$ cd ~/android/external
 +
$ git clone git://codeaurora.org/platform/external/compat-wireless.git compat-wireless
 +
$ git checkout -b codeaurora LNX.LF.3.2-27500-8x10.1
 +
</pre>
 +
 
 +
== Deploy CompuLab Android package ==
 +
=== Download software package ===
 +
Download CM-QS600 [http://www.compulab.co.il/products/computer-on-modules/cm-qs600/#devres  Android software package] in order to access the development resources:
 +
* Prebuilt Qualcomm proprietary libraries for bluetooth, audio, camera, etc.
 +
* CompuLab patches on top of Android for APQ8064 including Linux kernel patches, Android device configuration files and firmware binary blobs.
 +
 
 +
Uncompress the software package to {{filename|~/tmp}} directory.
 +
 
 +
=== Unpack Qualcomm proprietary prebuilt libraries ===
 
Extract prebuilt Qualcomm proprietary libraries:
 
Extract prebuilt Qualcomm proprietary libraries:
$ mkdir -p myandroid/vendor/qcom/proprietary
+
<pre>
$ tar -xJf cm-qs600-qcom-proprietary.tar.xz -C myandroid/vendor/qcom/proprietary
+
$ tar -xJf ~/tmp/cm-qs600-android/development/cm-qs600-qcom-proprietary.tar.xz -C ~/android/vendor/qcom/
 +
</pre>
  
=== CompuLab patch ===
+
=== Apply CompuLab patch ===
Apply CompuLab patch:
+
Apply CompuLab patch with the aid of the provided utility:
$ mkdir -p /tmp/sandbox
+
<pre>
$ tar -xJf cm-qs600-compulab-patch.tar.xz -C /tmp/sandbox
+
$ cd ~/android
$ cd myandroid
+
$ ~/tmp/cm-qs600-android/development/compulab-patch-apply.sh ~/tmp/cm-qs600-android/development/patches
$ /tmp/sandbox/compulab-patch-apply /tmp/sandbox
+
</pre>
  
=== Wireless kernel driver ===
 
Clone out of tree wireless kernel driver:
 
$ cd myandroid/external
 
$ git clone --branch upstream https://github.com/compulab/cm-qs600-compat-wireless.git compat-wireless
 
  
== Building Android image ==
 
$ cd myandroid
 
$ export USE_CCACHE=1
 
$ source build/envsetup.sh
 
$ choosecombo 1 bellatrix 3
 
$ make -j4 BUILD_ID=CM-QS600 BUILD_NUMBER=20140901
 
  
In case Java version different from the system default, is required by the build system, download the required JDK version, and export path to its root directory:
+
= Building Android image =
$ export JAVA_HOME=<full path to JDK>
+
<pre>
 +
$ cd ~/android
 +
$ export USE_CCACHE=1
 +
$ source build/envsetup.sh
 +
$ choosecombo 1 bellatrix 3
 +
$ make -j4 BUILD_ID=CM-QS600 BUILD_NUMBER=20140901
 +
</pre>
  
While the '''make''' instruction above builds the whole Android image, including all its partitions, it is possible to build '''boot''' partition only, which is useful for most Android hacking as the '''boot''' partition contains the '''kernel''' and the '''ramfs''' with most configuration files:
+
{{note|In case Java version required by the build system is different from the system default, download and install the required JDK version, and export path to its root directory:<pre>$ export JAVA_HOME=/path/to/JDK</pre>}}
==== Bring up kernel configuration menu ====
 
$ make kernelconfig
 
  
==== Build boot image (kernel + initramfs) ====
+
While the {{cmd|make}} instruction above builds the whole Android image, including all its partitions, it is possible to build {{filename|boot}} partition alone, which is useful for most Android hacking use cases as the {{filename|boot}} partition contains the '''kernel''' and the '''ramfs''' with most of configuration files:
$ make bootimage -j4
+
* Bring up kernel configuration menu for adjusting the kernel configuration:<pre>$ make kernelconfig</pre>
 +
* Build boot image (kernel + initramfs):<pre>$ make bootimage -j4</pre>
  
== Summary ==
+
== Build Summary ==
Upon successful build, the output directory (<code> myandroid/out/target/product/bellatrix </code>) contains ready to deploy OS partitions represented both as directories with appropriate files and compressed image files ready to deploy on eMMC:
+
Upon successful build, the output directory ({{filename|~/android/out/target/product/bellatrix}}) contains ready to deploy OS partitions represented both as directories with appropriate files and compressed image files ready to deploy onto the eMMC:
 
{| class="wikitable" border="1" cellspacing="0"
 
{| class="wikitable" border="1" cellspacing="0"
 
! partition<br/>name
 
! partition<br/>name
Line 80: Line 89:
 
| read only
 
| read only
 
| style="text-align:center;" | -
 
| style="text-align:center;" | -
| 20
+
| style="text-align:center;" | 20
 
| kernel, ramdisk
 
| kernel, ramdisk
 
| style="text-align:center;" | -
 
| style="text-align:center;" | -
Line 90: Line 99:
 
| read only
 
| read only
 
| style="text-align:center;" | -
 
| style="text-align:center;" | -
| 20
+
| style="text-align:center;" | 20
 
| alternative to ''boot'' for recovery and upgrade
 
| alternative to ''boot'' for recovery and upgrade
 
| currently not in use
 
| currently not in use
Line 100: Line 109:
 
| read only
 
| read only
 
| /system
 
| /system
| 512
+
| style="text-align:center;" | 512
 
| the entire OS:<br/>
 
| the entire OS:<br/>
 
Android UI, system applications
 
Android UI, system applications
Line 111: Line 120:
 
| read / write
 
| read / write
 
| /cache
 
| /cache
| 64
+
| style="text-align:center;" | 64
 
| frequently accessed data cache
 
| frequently accessed data cache
 
| style="text-align:center;" | -
 
| style="text-align:center;" | -
Line 121: Line 130:
 
| read / write
 
| read / write
 
| /data
 
| /data
| 2048
+
| style="text-align:center;" | 2048
 
| user applications and content
 
| user applications and content
 
| style="text-align:center;" | -
 
| style="text-align:center;" | -
 
|-
 
|-
 
|}
 
|}
 +
<br/>
  
 
= Deploying Android Image on Target Medium =
 
= Deploying Android Image on Target Medium =
== Overview ==
+
The root of Android file system is located on a RAM-disk. Physical storage device partitions, such as {{filename|system}}, {{filename|cache}}, {{filename|userdata}}, are mounted under the root.
In normal boot mode, the root of the file system is mounted from RAM-disk, whereas storage device partitions containing system utilities, applications and data are mounted under it.<br/>
+
These partitions mounting rules are specified in {{filename|/fstab}} file.<br/>
Root on a physical device, or file system mounted via NFS are also viable options, but are used mainly throughout development stage.<br/>
+
Root on a physical device or on NFS are also viable options. Those options are useful throughout development stage.<br/>
LK bootloader expects a valid boot partition ('''boot.img''') on the eMMC. ''System'', ''userdata'' and ''cache'' partitions mounting is ruled by ''fstab'' file - they can reside on any available storage device.<br/>
+
On CM-QS600, LK bootloader expects a valid boot partition ({{filename|boot.img}}) to be present on the eMMC.
In case LK does not identify a valid boot partition on the eMMC, it falls back to [[Android: Fastboot setup | ''fastboot mode'']], where it is possible, along with other options, to load ''boot.img'' dynamically from PC workstation.
+
In case LK cannot identify a valid boot partition, it falls back to [[Android: Fastboot setup|fastboot mode]].
  
=== [[CM-QS600: Android: Getting started#Installation onto eMMC | Deploy file system on eMMC]] ===
+
=== Deploy file system on eMMC ===
 
* Enter ''fastboot mode''.
 
* Enter ''fastboot mode''.
 
* Flash each partition image file to the eMMC with the aid of ''fastboot'' utility:  
 
* Flash each partition image file to the eMMC with the aid of ''fastboot'' utility:  
Line 143: Line 153:
 
$ fastboot flash userdata userdata.img
 
$ fastboot flash userdata userdata.img
 
</pre>
 
</pre>
 +
 +
{{note|More detailed description of the file system deployment on the eMMC is available on the [[CM-QS600: Android: Getting started#Installation onto eMMC | Getting started]] page.}}
  
 
=== Deploy file system on SD-card ===
 
=== Deploy file system on SD-card ===
 
==== Prepare SD-card ====
 
==== Prepare SD-card ====
* [http://linux.die.net/man/8/gdisk  GPT-partition] SD-card, creating ''system'', ''cache'', ''userdata'' partitions.
+
* [http://linux.die.net/man/8/gdisk  GPT-partition] SD-card, creating ''system'', ''cache'', ''userdata'' partitions, and setting GPT names of the partitions appropriately.
 
* Format each partition with ''ext4'' file system.
 
* Format each partition with ''ext4'' file system.
* Copy ''system'' partition contents, maintaining proper file permissions:
+
* Copy {{filename|system}} partition contents, maintaining proper file permissions:
$ cp -a ${OUT}/system/* ${SYSTEM}/.
+
<pre>
$ for f in `find ${SYSTEM}`; do [ -f ${f} ] && chmod go-w ${f}; [ `echo ${f} | awk /\.so$/` ] && chmod a-x ${f}; done
+
$ export OUT=~/android/out/target/product/bellatrix
 +
$ export SYSTEM=/media/user/system
 +
$ cp -a ${OUT}/system/* ${SYSTEM}/.
 +
$ for f in `find ${SYSTEM}`; do [ -f ${f} ] && chmod go-w ${f}; [ `echo ${f} | awk /\.so$/` ] && chmod a-x ${f}; done
 +
</pre>
 +
 
 +
{{note|The {{parameter|SYSTEM}} location above is specified as it appears in Ubuntu 14.04. It may differ in other Ubuntu versions or distributions.}}
  
 
==== Re-build ''boot.img'' with patched ''fstab'' ====
 
==== Re-build ''boot.img'' with patched ''fstab'' ====
* Edit ''device/qcom/bellatrix/fstab.qcom'' file, in order to override default destination medium device:
+
* Edit {{filename|device/qcom/bellatrix/fstab.qcom}} file, in order to override default destination storage medium:
 
<pre>
 
<pre>
  #boot time mount
+
--- HEAD.fstab.qcom 2015-09-22 11:48:27.958235845 +0300
- /dev/block/platform/msm_sdcc.1/by-name/system        /system        ext4    ro,barrier=1                                      wait
+
+++ fstab.qcom 2015-09-22 11:48:36.962236014 +0300
+ /dev/block/platform/msm_sdcc.3/by-name/system        /system        ext4    ro,barrier=1                                     wait
+
@@ -9,5 +9,5 @@
- /dev/block/platform/msm_sdcc.1/by-name/cache          /cache            ext4    nosuid,nodev,noatime,barrier=1                   wait,check
+
#boot time mount
+ /dev/block/platform/msm_sdcc.3/by-name/cache          /cache            ext4    nosuid,nodev,noatime,barrier=1                   wait,check
+
-/dev/block/platform/msm_sdcc.1/by-name/system        /system        ext4    ro,barrier=1                                      wait
- /dev/block/platform/msm_sdcc.1/by-name/userdata      /data          ext4    nosuid,nodev,noatime,barrier=1,noauto_da_alloc    wait,check,length=1073741824,encryptable=footer
+
-/dev/block/platform/msm_sdcc.1/by-name/cache          /cache              ext4    nosuid,nodev,noatime,barrier=1                   wait,check
+ /dev/block/platform/msm_sdcc.3/by-name/userdata      /data          ext4    nosuid,nodev,noatime,barrier=1,noauto_da_alloc    wait,check,length=1073741824,encryptable=footer
+
-/dev/block/platform/msm_sdcc.1/by-name/userdata      /data          ext4    nosuid,nodev,noatime,barrier=1,noauto_da_alloc    wait,check,length=1073741824,encryptable=footer
 +
+/dev/block/platform/msm_sdcc.3/by-name/system        /system        ext4    ro,barrier=1                                     wait
 +
+/dev/block/platform/msm_sdcc.3/by-name/cache          /cache              ext4    nosuid,nodev,noatime,barrier=1                   wait,check
 +
+/dev/block/platform/msm_sdcc.3/by-name/userdata      /data          ext4    nosuid,nodev,noatime,barrier=1,noauto_da_alloc    wait,check,length=1073741824,encryptable=footer
 
</pre>
 
</pre>
  
* Rebuild ''boot.img'':
+
* Rebuild {{filename|boot.img}}:
$ make bootimage -j4
+
<pre>$ make bootimage -j4</pre>
  
 
==== Flash ''boot.img'' onto eMMC ====
 
==== Flash ''boot.img'' onto eMMC ====
As pointed out above, ''boot.img'' has to reside on the eMMC, flash it via ''fastboot'' utility:
+
As pointed out above, ''boot.img'' has to reside on the eMMC.
$ fastboot flash boot boot.img
+
Flash it via ''fastboot'' utility:
 +
<pre>
 +
$ fastboot flash boot boot.img
 +
</pre>
 +
 
 +
{{Note|The same considerations are applicable to file system deployment on mSATA drive.}}
  
 
= See Also =
 
= See Also =
 +
* [[CM-QS600: Android: Software package | Software package contents]]
 
* [https://www.codeaurora.org  CodeAurora Forum website]
 
* [https://www.codeaurora.org  CodeAurora Forum website]
 
* [http://source.android.com/source/building.html  Android|Downloading and Building]
 
* [http://source.android.com/source/building.html  Android|Downloading and Building]
Line 177: Line 204:
 
* [http://source.android.com/devices/index.html  Android|Subsystems explained]
 
* [http://source.android.com/devices/index.html  Android|Subsystems explained]
 
* [http://www.kandroid.org/online-pdk/guide/build_system.html  AOSP|Android Build System]
 
* [http://www.kandroid.org/online-pdk/guide/build_system.html  AOSP|Android Build System]
 +
* [http://blogofterje.wordpress.com/2012/01/14/optimizing-fs-on-sd-card  Some thoughts about partitioning flash storage devices]
  
 
[[Category:Android]]
 
[[Category:Android]]
 
[[Category:CM-QS600]]
 
[[Category:CM-QS600]]

Latest revision as of 13:23, 1 November 2015

Overview

The following outlines the procedure necessary to acquire the Android source code and build it for CM-QS600 System-on-Module / Computer-on-Module.
CompuLab Android package is obtained by applying CompuLab patches on top of Code Aurora Forum's Android package for APQ8064 CDP platform.

Prerequisites

The required steps, as described below, were tested on Ubuntu 14.04 (Trusty).
Install required packages, as outlined on AOSP web-page.
Install additional packages (not listed on AOSP web-page):

$ sudo apt-get install xz-utils make flex lib32z1 zip curl

Install repo tool for work with multiple git repositories in the context of Android:

$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo

Getting Android Source Code

Deploy Code Aurora Forum base

Download the code base from Code Aurora Forum (CAF):

$ mkdir ~/android
$ cd ~/android
$ repo init -u git://codeaurora.org/platform/manifest.git -b release -m LNX.LA.2.7-06310-8064.0.xml --repo-url=git://codeaurora.org/tools/repo.git --repo-branch=caf-stable
$ repo sync -j4
$ repo forall -c "git checkout -b codeaurora"


Admolition note.png The last instruction maintains git branch naming convention, which is useful for clarity, but is not strictly necessary.

Deploy Wireless kernel driver

Clone out of tree wireless kernel driver:

$ cd ~/android/external
$ git clone git://codeaurora.org/platform/external/compat-wireless.git compat-wireless
$ git checkout -b codeaurora LNX.LF.3.2-27500-8x10.1

Deploy CompuLab Android package

Download software package

Download CM-QS600 Android software package in order to access the development resources:

  • Prebuilt Qualcomm proprietary libraries for bluetooth, audio, camera, etc.
  • CompuLab patches on top of Android for APQ8064 including Linux kernel patches, Android device configuration files and firmware binary blobs.

Uncompress the software package to ~/tmp directory.

Unpack Qualcomm proprietary prebuilt libraries

Extract prebuilt Qualcomm proprietary libraries:

$ tar -xJf ~/tmp/cm-qs600-android/development/cm-qs600-qcom-proprietary.tar.xz -C ~/android/vendor/qcom/

Apply CompuLab patch

Apply CompuLab patch with the aid of the provided utility:

$ cd ~/android
$ ~/tmp/cm-qs600-android/development/compulab-patch-apply.sh ~/tmp/cm-qs600-android/development/patches


Building Android image

$ cd ~/android
$ export USE_CCACHE=1
$ source build/envsetup.sh
$ choosecombo 1 bellatrix 3
$ make -j4 BUILD_ID=CM-QS600 BUILD_NUMBER=20140901


Admolition note.png In case Java version required by the build system is different from the system default, download and install the required JDK version, and export path to its root directory:
$ export JAVA_HOME=/path/to/JDK

While the make instruction above builds the whole Android image, including all its partitions, it is possible to build boot partition alone, which is useful for most Android hacking use cases as the boot partition contains the kernel and the ramfs with most of configuration files:

  • Bring up kernel configuration menu for adjusting the kernel configuration:
    $ make kernelconfig
  • Build boot image (kernel + initramfs):
    $ make bootimage -j4

Build Summary

Upon successful build, the output directory (~/android/out/target/product/bellatrix) contains ready to deploy OS partitions represented both as directories with appropriate files and compressed image files ready to deploy onto the eMMC:

partition
name
directory image file file system
type
access mount point default
partition
size [MB]
designation notes
boot root boot.img - read only - 20 kernel, ramdisk -
recovery recovery recovery.img - read only - 20 alternative to boot for recovery and upgrade currently not in use
system system system.img ext4 read only /system 512 the entire OS:

Android UI, system applications

-
cache cache cache.img ext4 read / write /cache 64 frequently accessed data cache -
userdata data userdata.img ext4 read / write /data 2048 user applications and content -


Deploying Android Image on Target Medium

The root of Android file system is located on a RAM-disk. Physical storage device partitions, such as system, cache, userdata, are mounted under the root. These partitions mounting rules are specified in /fstab file.
Root on a physical device or on NFS are also viable options. Those options are useful throughout development stage.
On CM-QS600, LK bootloader expects a valid boot partition (boot.img) to be present on the eMMC. In case LK cannot identify a valid boot partition, it falls back to fastboot mode.

Deploy file system on eMMC

  • Enter fastboot mode.
  • Flash each partition image file to the eMMC with the aid of fastboot utility:
$ fastboot flash boot     boot.img
$ fastboot flash system   system.img
$ fastboot flash cache    cache.img
$ fastboot flash userdata userdata.img


Admolition note.png More detailed description of the file system deployment on the eMMC is available on the Getting started page.

Deploy file system on SD-card

Prepare SD-card

  • GPT-partition SD-card, creating system, cache, userdata partitions, and setting GPT names of the partitions appropriately.
  • Format each partition with ext4 file system.
  • Copy system partition contents, maintaining proper file permissions:
$ export OUT=~/android/out/target/product/bellatrix
$ export SYSTEM=/media/user/system
$ cp -a ${OUT}/system/* ${SYSTEM}/.
$ for f in `find ${SYSTEM}`; do [ -f ${f} ] && chmod go-w ${f}; [ `echo ${f} | awk /\.so$/` ] && chmod a-x ${f}; done


Admolition note.png The SYSTEM location above is specified as it appears in Ubuntu 14.04. It may differ in other Ubuntu versions or distributions.

Re-build boot.img with patched fstab

  • Edit device/qcom/bellatrix/fstab.qcom file, in order to override default destination storage medium:
--- HEAD.fstab.qcom	2015-09-22 11:48:27.958235845 +0300
+++ fstab.qcom	2015-09-22 11:48:36.962236014 +0300
@@ -9,5 +9,5 @@
 #boot time mount
-/dev/block/platform/msm_sdcc.1/by-name/system         /system      	   ext4    ro,barrier=1                                      wait
-/dev/block/platform/msm_sdcc.1/by-name/cache          /cache              ext4    nosuid,nodev,noatime,barrier=1                    wait,check
-/dev/block/platform/msm_sdcc.1/by-name/userdata       /data        	   ext4    nosuid,nodev,noatime,barrier=1,noauto_da_alloc    wait,check,length=1073741824,encryptable=footer
+/dev/block/platform/msm_sdcc.3/by-name/system         /system      	   ext4    ro,barrier=1                                      wait
+/dev/block/platform/msm_sdcc.3/by-name/cache          /cache              ext4    nosuid,nodev,noatime,barrier=1                    wait,check
+/dev/block/platform/msm_sdcc.3/by-name/userdata       /data        	   ext4    nosuid,nodev,noatime,barrier=1,noauto_da_alloc    wait,check,length=1073741824,encryptable=footer
  • Rebuild boot.img:
$ make bootimage -j4

Flash boot.img onto eMMC

As pointed out above, boot.img has to reside on the eMMC. Flash it via fastboot utility:

$ fastboot flash boot boot.img


Admolition note.png The same considerations are applicable to file system deployment on mSATA drive.

See Also