OpentestHowTo

From AragoWiki

Jump to: navigation, search

Contents

How to setup new TEE

Install and Configure optional debian packages

 Set proxy settings (i.e. ~/.bashrc and ubuntu network proxy)
 Set git-proxy.sh (see below)
 sudo apt-get install nfs-kernel-server
 sudo apt-get install iperf
 sudo apt-get install git
 sudo apt-get install u-boot-tools (required for mkimage) 
 Make sure /etc/exports is exporting NFS mount point, e.g.
 /home/tigtfarm01/NFS_exports *(rw,nohide,insecure,no_subtree_check,async,no_root_squash)
 restart service, e.g. sudo service nfs-kernel-server restart
 sudo apt-get install cifs-utils (previously known as smbfs)
 sudo apt-get install openssh-server
 sudo apt-get install tftpd-hpa and set /tftpboot dir in /etc/default/tftpd-hpa
 sudo apt-get install isc-dhcp-server (if need to provide dhcp service on private network)
 It is recommended to isolate DUTs from corporate NW.
 The TEE host PC must have at least 2 ethernet interfaces, one will be connected to the corporate nw and the other to the private test nw.
 TEE will use corporate iface to register itself with Opentest TMC
 TEE will run nfs-server, tftp and dhcp services on private iface only.
 Configure:
 /etc/default/isc-dhcp-server
 /etc/dhcp/dhcpd.conf. After configuring the dhcp server setting for the private network, make sure set this option with global scope "one-lease-per-client true;" . This ensures that only one address is assigned per MAC which reduces the risk of running out of IP addresses.
 Might need rule to restart tftp service after iface is up. For example add following to /
 etc/network/if-up.d/dut-services:
 restart tftpd-hpa
 restart nfs-kernel-server
 restart isc-dhcp-server
 mount gtautoftp and gtsnowball
 Download & Run latest opentest installer from http://arago-project.org/files/releases/opentest/
 IMPORTANT: Make sure you add '-x 1' to the downwardTranslator.xsl command element to force a reboot on first failure.
 * Sample proxy env vars:
 http_proxy=http://webproxy.ext.ti.com:80
 ftp_proxy=http://webproxy.ext.ti.com:80
 GIT_PROXY_COMMAND=/home/tigtfarm01/git-proxy.sh
 https_proxy=http://webproxy.ext.ti.com:80
 * Sample git-proxy.sh
 #! /bin/bash
 PROXYHOST=webproxy.ext.ti.com
 PROXYPORT=80
 exec env corkscrew ${PROXYHOST} ${PROXYPORT} $*

Create udev rules to have fix names to tty nodes associated with each equipment

 1) Create udev rules file. e.g. sudo vim /etc/udev/rules.d/opentest.rules
 2) For each equipment on which permanent rule is desired do the following:
 udevadm info --attribute-walk --name /dev/tty<NAME ASSOCIATED WITH EQUIPMENT>
 Determine unique attribute for the equipment, for example serial number
 add rule to opentest.rules file. eg: SUBSYSTEM=="tty" ATTRS{serial}=="ST168498" SYMLINK+="vatf@am335x-evm
 It is recommended to use the tee name as the symlink name for DUT serial ports.
 Warning: Some boards like am335x-sk exposes two interfaces with the same serial number because there is only one ftdi chip in the evm,
 so on those you can not use serial attribute. One possibility is to use KERNELS atribute which is unique and won't change as long as you don't
 change the USB port where the board connects to the host pc,
 For a good article on udev rules, check http://www.reactivated.net/writing_udev_rules.html


Setting up Host PC so DUT can access Internet/intranet

Setting up IP forwarding and NAT to access Internet from private NW

 *** Enable IP forwarding ***
 $ sudo sysctl -w net.ipv4.ip_forward=1
 *** Setup iptables *** 
 $ sudo iptables --flush
 $ sudo iptables --table nat --flush
 $ sudo iptables --delete-chain
 $ sudo iptables --table nat --delete-chain
 # adjust eth2 below as appropriate. This is the interface with Internet connectivity
 $ sudo iptables --table nat --append POSTROUTING --out-interface eth2 -j MASQUERADE
 # adjust eth0 below as appropriate. This is the interface on the private LAN
 $ sudo iptables --append FORWARD --in-interface eth0 -j ACCEPT
 $ sudo sh -c 'iptables-save > /etc/iptables.conf'
 
 Add the following command to /etc/rc.local
 iptables-restore < /etc/iptables.conf


Setup DNS servers

 * Open /etc/dhcp/dhcpd.conf and setup following entries:
   option domain-name "ti.com";
   option domain-name-servers 192.0.2.2, 192.0.2.3;

 * Restart dhcp server:
   sudo service isc-dhcp-server restart

Note: http_proxy must be defined in the DUT environment. By default wget uses proxy if environment variable is defined, so when accessing internal URLs it is required to disabled proxy by passing '--proxy off' option in wget command.


Fixing issues caused by upowerd blocking access to ttyUSB ports

Disable WattsUpPro

 sudo vim /etc/UPower/UPower.conf
 Change:
    EnableWattsUpPro=true
 To:
    EnableWattsUpPro=false


Restart upower daemon

* Get process ID, e.g. ps -ef | grep -i power
* Kill daemon, e.g.  sudo kill -9 <PID>
* Daemon should start automatically after few seconds, if not you can start it, e.g. sudo /usr/lib/upower/upowerd &

Installing ubuntu 12.04 with SoftwareRAID RAID 1 (You need 2 Hard Drive, preferably same size)

 1) Create CD with ubuntu alternate image (e.g. ubuntu12.04.3-alternate-amd64.iso)
 2) Create 32GB (logical), and rest-of-space (primary) partitions on each drive.
    Make sure to set 'use as physical volumne for RAID' on all the partitions
    Also make sure you set boot flag on primary partition
 3) Configure software RAID
 4) Create MD device
 5) Choose RAID1 and select 2 swap partitions
 6) Repeat 4) and 5) for 2 primary partitions
 7) Now go to RAID1 device #0. Press enter and set use as 'swap area'
 8) Now go to RAID1 device #1. Press enter and set use as 'ext4 journal file system'
 9) Compleye ubuntu installation
 Sample video at http://www.youtube.com/watch?v=z84oBqOxsD0


Installing ubuntu 12.04 with FakeRAID RAID 1 (You need 2 Hard Drives, preferably same size)

 1) Create CD with ubuntu alternate image (e.g. ubuntu12.04.3-alternate-amd64.iso)
 2) Following the system's motherboard documentation, use bios to setup the hard drive in RAID mode
 3) Using the BIOS option ROM utility, create the RAID 1 arrays with the two HD
 4) Boot the system with ubuntu CD created in step 1
 5) Select "yes" when asked if you want the installer to activate the RAID 1 array
 7) Select "yes" when asked if you want to write the changes to disk
 8) Select "use complete <raid array info> disk for installation" when asked where to install
 9) Enter the required information for system name, user name, password, etc when asked
 10)IMPORTANT: Before the installer performs the last step, it will displayed a message saying that it did not detect another OS
    and it will ask if it should go ahead and install the GRUB (the bootloader) to the array. Select "No", and when prompt for the install location
    enter the dev node of the array, typically /dev/dm-0 (if not sure what the dev node is do Ctrl+F1 to get to a shell and do ls -ltr /dev/mapper/<RAID link>,
    and use the target of the link, then Ctrl+F7 to return to the installer), then enter.
 11) When the installer is done it will ask you to remove the disk and reboot


Configuring static ip address Ubuntu 12.04 (Optional, it is better to use dhcp which is the default configuration)

 NOTE: This section is not required in the majority of the cases, the prefer method is to
 use dhcp for the system to obtain an ip address. This section is here for completeness,
 and for the special cases in which a static ip address is required.
 1) Open Network Connections. Dash home -> type network -> click on "Network Connections" Application.
 2) Select the network interface you which to configure and click the "Edit" button.
 3) Click on the IPv4 settings tab
 4) Select "Manual" from the drop-down box of the "Method:" field
 5) Click the "Add" button on the adresses section
 6) Enter the IP address, netowork mask and gateway on the editable sections of the address just created
 7) Enter the DNS server's adresses (comma separated) in the "DNS servers:" text field.
 For example 1.2.3.4, 1.2.3.5
 8) Enter the comma separated list of search domains (if any) in the "Search Domains:" text field.
 For example gt.design.ti.com, am.dhcp.ti.com, telogy.design.ti.com, sanb.design.ti.com, toro.design.ti.com, dal.design.ti.com
 9) Change the fqdn (fully qualified domain name) so that it matches your new name in /etc/hosts.
 10) Reset your machine to make sure the changes take effect.
 11) Contact IT so that they can set the address to in use and add the system to their dns servers.


Send notifications when something happens SoftwareRAID

1) Setup email service

* sudo dpkg-reconfigure exim4-config 
  - select mail sent by smarthost, no local mail
  - leave defaults for most question except for 'IP address or host name of the outgoing smarthost':
    Use: smtp.mail.ti.com

2) Configure email address

 sudo vim /etc/mdadm/mdadm.conf and then set/change
 MAILADDR lcpd_systest@list.ti.com

3) restart the PC to make sure mdadm contains right info


Send notifications when something happens FakeRAID

1) Setup email service, sudo apt-get install exim4

* sudo dpkg-reconfigure exim4-config 
  - select mail sent by smarthost, no local mail
  - leave defaults for most question except for 'IP address or host name of the outgoing smarthost':
    Use: smtp.mail.ti.com

2) Create a bash script in the root file system (i.e sudo vim /sbin/raid_status.sh) with the following content:

   #!/bin/sh
   # check raid status and email if not ok
   STATUS=`/sbin/dmraid -s | grep "status"`
   if [ "$STATUS" != "status : ok" ]
   then
      /sbin/dmraid -s | mail -s "RAID ERROR ON `hostname`: $STATUS" lcpd_systest@list.ti.com
   fi

3) Change the mode of the script to 755, i.e. "sudo chmod 755 /sbin/raid_status.sh" 4) setup a cron job for the root user. Run "sudo crontab -e" and add the following line to the crontab

   0 1 * * * <path to the script created in step 2>. For example to run script /sbin/raid_status.sh every day at 1 am
   0 1 * * * /sbin/raid_status.sh


Test notifications when something happens mdadm (SoftwareRAID)

 sudo mdadm --detail /dev/md1 (or whatever md node you want to simulate a failure on)
 sudo mdadm --manage --set-faulty /dev/md1 /dev/sda2
 # EMAIL notification should go out at this point
 # Now remove and add back the node so things go back to normal.
 # It will take a bit for the recovery process to complete
 sudo mdadm /dev/md1 -r /dev/sda2
 sudo mdadm /dev/md1 -a /dev/sda2


Recovering malfunctioning HD with fakeRAID

 1) Shutdown the system, replace the bad HD and reboot
 2) Use OROM to add the new HD to the RAID array
 3) The Volume in OROM will show a Rebuild status
 4) Exit OROM and let the system boot.
 5)  Use the dmraid command to rebuild "sudo dmraid -R <raid_set>" where <raid_set> is the name of the raid array created.
      To obtain the name of the raid set run "sudo dmraid -s" , and copy the value of the name: field that is printed out, i.e. isw_....
 6) watch “dmsetup status” for the ratio to be 1.


Upgrading from 12.04 LTS to 14.04 LTS fakeRAID

 1) When the "New Ubuntu release xx.xx LTS" alert appears in the Update manager click the "Upgrade" button
 2) The system will take some time to download and upgrade the software and eventually an error message
 will appear stating that there was an error installing GRUB. At this point select continue without installing but
 DO NOT REBOOT THE SYSTEM AFTER THE UPGRADE COMPLETES.
 3) Open a terminal console and do "ls -l /dev/mapper". In the listing you will noticed that the link to ../dm-0 is missing
 which is why the GRUB installation failed
 4) Change directory to /dev/mapper
 5) Create a symbolic link to ../dm-0, with "sudo ln -sf ../dm-0 <name of raid volume>", where <name of raid volume> can
 be obtained with "sudo dmraid -s" , and using the value of the name: field that is printed out, i.e. isw_..... Which also happens
 to be the base name of the other links found for the RAID array in /dev/mapper
 6) Run sudo apt-get purge grub-pc
 7) Run sudo apt-get install grub-pc:
      a) When asked to remove the files under /boot/grub select no
      b) When the grub configuration menu appears select the options containing dm-0 for the
 install location
 8) Once GRUB is installed and configure, run "sudo update-grub"
 9) Reboot the system
 10) if a message with grub appears after reboot, open a terminal window and run "sudo update-grub" again.
 11) A problem with cups may appear after reboot, select send report and this message should not appear again



How-To Install Opentest Client tools

  • Find a Ubuntu system (10.04 or later should be fine)
  • Download and untar latest opentest_installer*.tar.gz (recommended 13.09 or later) from http://arago-project.org/files/releases/opentest/
  • Run the installer (e.g. ./install_opentest.sh) and select option 8 (Command Line Tools). This option will also install the STAF utility. For installation details see the list below:
    • The installer will install Java if required
    • Then it will install STAF (you can leave all the default install values)
    • Then it will install opentest_client components. You can leave default values for most options, however subnets must be entered. If you are in the US we recommend entering at least following 3 subnets (separated by spaces): 158.218.*.* 10.218.*.* 128.247.*.*
    • When prompted for the Tools installation menu select option 1 to Install All tools.
    • When prompted where to install the files you can enter . to install in the current directory. You should see this directory in the next prompt.
  • Finally you can select option 9 to quit.

How-To Start STAF

  • Check if STAF is running
$ staf local help
Usage: STAF [-verbose] <Endpoint | LOCAL> <Service> <Request>
  • If you don't get an error while running staf local help command, then STAF is running and you are good to go. However if you do get an error then you need to open /usr/local/staf/STAFEnv.sh and verify that STAF_INSTANCE_NAME is set to a value as shown below
if [ $# = 0 ]
then
   STAF_INSTANCE_NAME=STAF
else
   if [ $1 != "start" ]
   then
       STAF_INSTANCE_NAME=$1
   else
       # Ignore "start" STAF instance name
       STAF_INSTANCE_NAME=STAF
   fi
fi
export PATH LD_LIBRARY_PATH CLASSPATH STAFCONVDIR STAF_INSTANCE_NAME
  • Source the /usr/local/staf/STAFEnv.sh script to properly initialize your environment for STAF (assuming that STAF was installed in the default location).
    . /usr/local/staf/STAFEnv.sh
  • Finally run /usr/local/staf/startSTAFProc.sh to start STAF. Cat nohup.out file to check correct initialization
$ ./startSTAFProc.sh 
$ nohup: appending output to `/home/charliebrown/nohup.out'
$ cat /home/charliebrown/nohup.out
Machine          : charliebrown
Machine nickname : charliebrown
Startup time     : 20130905-18:06:02
  • You should now be able to run the staf local help command and verify your STAF installation.

How-To easily Run a boot test on multiple boards

./CI.rb -m CI_sample_boottest.rb --machine-adapter-type none --input-adapter-type none --build-name chtest --do-not-publish-results
IMPORTANT

--build-name should not have spaces. You probably want to use a unique ID (e.g. your employee ID). One can control the set of boards to test by removing entries from @machines list at the bottom of CI_sample_boottest.rb


How-To Run your own tests on the boards Farm

 # Simple test to boot and echo 'hello world' upon booting.
./linux-devtest.py -s "echo 'hello world'" <options>
IMPORTANT

The command above will not run by default without some options being passed. At a minimum you should specify the board farm to use, but likely you will also add your kernel image to test

How-To Run LTP-DDT tests on the boards Farm

./linux-devtest.py -t <LTP-DDT testplan name> <options>
IMPORTANT

The command above will not run by default without some options being passed. At a minimum you should specify the board farm to use, but likely you will also add your kernel image to test

Creating your own LTP-DDT testplan

  • Update default tests. This will clone ltp-ddt project if required.
./linux-devtest.py -U
  • Create your own test plan
cp tests/default.py tests/mytests.py
  • Modify tests_to_run list in tests/mytests.py to suit your needs by copying entries from tests_available list into it.

Each entry in tests_available lists points to a ltp-ddt test scenario (aka test suite). The syntax is:
<test scenario name>:<setup requirements>:<test cases filter>

How to create new LTP-DDT tests

IMPORTANT

This step is more involved. It requires cloning/modifying/building/installing ltp-ddt in the rootfs
  • Export NFS filesystem in your host.

LTP-DDT expects certain test binaries to be installed in the filesystem. If you use a tisdk filesystem then all the required test tools have been already installed in the filesystem. One can get a recent tisdk rootfs image from http://lcpd.dal.design.ti.com/core-sdk/nightly/deploy/images/
Select appropriate tisdk-rootfs-image-<MACHINE>*.tar.gz

  • Make sure ltp-ddt sources are up to date. This step will clone latest ltp-ddt project into ltp-ddt directory
./linux-devtest.py -U
  • Build Kernel, modules and headers and install them in the filesystem
cd <your kernel source root>
make ARCH=arm CROSS_COMPILE=... zImage modules
sudo make INSTALL_MOD_PATH=<nfs root> modules_install 
sudo make INSTALL_HDR_PATH=<nfs root>/usr headers_install
IMPORTANT

Make sure you use recommended toolchain. See toolchain installation instructions
  • Build and install latest LTP-DDT sources
cd <ltp-ddt root dir>.  # This is the directory created by Step 2 above 
# Change cross-compiler, kernel include and kernel usr include and platform accordingly in the command below
# KERNEL_INC should point to you kernel sources include directory
# KERNEL_USR_INC should point to the filesystem/usr/include directory
make CROSS_COMPILE=arm-linux-gnueabihf- KERNEL_INC=<kernel sources root>/include KERNEL_USR_INC=<nfs root>/usr/include SKIP_IDCHECK=1 PLATFORM=omap5-evm
# Finally install ltp-ddt in the filesystem
sudo make DESTDIR=<nfs root>/opt/myltp SKIP_IDCHECK=1 PLATFORM=omap5-evm install



how to contribute tests back to ltp-ddt

Please submit your LTP-DDT patches to opentest@arago-project.org.
You may want to check the README-DDT file in the ltp-ddt directory for more information about how to create new tests.

How-To Run Testlink tests plans on the boards Farm

  • Use this option to run a Testlink test plan. If you use this option results will be stored in the Testlink server.
  • Make sure you have already install Opentest Client tools
  • Also verify that STAF is running
  • Run linux-devtest.py -T from the location where you installed Opentest client tools
./linux-devtest.py -T <Testlink testplan> <options>
IMPORTANT

The command above will not run by default without some options being passed. At a minimum you should specify the board farm to use, but likely you will also add your kernel image to test

The Testlink testplan name syntax is: <project name>:<testplan name>

To get the list of available projects, use following command:

./linux-devtest.py --list-projects

To get the list of available testplans on a project, use --list-testplans <project ID> where project ID can be obtained from --list-projects command. For example:

./linux-devtest.py --list-testplans 1140921

The above command will return the list of testplans for LCPD linux. Project linux_psp2(id=1140921) hosts LCPD's linux test plans.

Please note that --list-testplans uses project ID, while linux-devtest.py -T uses project and testplans names.

linux-devtest.py arguments help

command help

# use -h flag to show command help
./linux-devtest.py -h

-hw argument

The -hw argument selects the type of hardware (aka DUT) to run the tests on. It is also possible to specify capabilities or requirements that selected DUT must have. For instance, capabilities could be used to ask for a board that has an usb mass storage device attached to it. The argument syntax is -hw evm,capabilities where capabilities is an optional underscore-separated list of strings Valid evm names are the same used by OE/Yocto: am335x-evm, omap5-evm, beaglebone, dra7xx-evm, etc. For valid capabilities see Capabilities cheat sheet

linux-devtest.py ... -hw omap5-evm,linux

-p, -b, -k, -m, -d, -r, -n Software arguments

Set software images (i.e. bootloader, kernel, filesystem, etc.) to used for the test. It is not mandatory to provide all of them. For example if bootloader (-p and -b) are not provided then the system will try to boot the DUT using whatever bootloader image is already installed on it. Most software image values can take either a local path or a http/ftp url. Use ./linux-devtest.py -h for more details


-u, --user-bins argument

Executable or tarball of executables that should be installed on FS. This is useful for cases where user wants to run a binary that is not available in the default filesystem The value provided can be either a local file or a public http/ftp url

linux-devtest.py ... -u /home/tmp/myapp

-s, --script argument

Use this option to run any arbitrary commands or a shell script in the DUT. The syntax is either a string of semicolon-separated commands (e.g. "echo 'hello'; echo 'bye'") or a path to a shell script. In either case the test will pass if the return value of the commands or script is zero.

linux-devtest.py ... -s /home/tmp/mytest.sh

-t, --tests argument

Use this option to run your own ltp-ddt test plan. Please note that -s, -t and -T are mutually exclusive options

 linux-devtest.py ... -t mytests

To create your own ltp-ddt test plan follow this steps:

  • Update default tests. This will clone ltp-ddt project if required.
./linux-devtest.py -U
  • Create a copy of default test plan
cp tests/default.py tests/mytests.py
  • Modify tests_to_run list in tests/mytests.py to suit your needs by copying entries from tests_available list into it.

Each entry in tests_available lists points to a ltp-ddt test scenario (aka test suite). The syntax is:
<test scenario name>:<setup requirements>:<test cases filter>


-T, --testplan argument

  • Use this option to run a Testlink test plan. If you use this option results will be stored in the Testlink server.

The Testlink Testplan name syntax is: <project name>:<testplan name>. For example:

./linux-devtest.py -T linux_psp2:ch_sandbox
  • To get the list of available projects, use following command:
./linux-devtest.py --list-projects
  • To get the list of available testplans on a project, use --list-testplans <project ID> where project ID can be obtained from --list-projects command. For example:
./linux-devtest.py --list-testplans 1140921

The above command will return the list of testplans for LCPD linux. linux_psp2 is the project name that host LCPD's linux test plans.

Please note that --list-testplans uses project ID, while linux-devtest.py -T uses project and testplans names.

-f, --farm argument

forces usage of a board in a farm.

-f tigt_farm
-f tid_farm

--advanced-params

advanced-params are variables that advanced users can set to tweak execution. At the moment the only advanced-params supported by linux-devtest.py is var_use_default_env. Usually linux-devtest.py will try to boot the DUT using provided software images (i.e. kernel, filesystem, etc.), but advanced users may change that default behavior.

  • var_use_default_env=1 means Boot using default environment (i.e. env default -a -f; boot )
  • var_use_default_env=2 means Do not touch uboot env, just power cycle and let it go

Errors

  • If you get ImportError: No module named argparse while running linux-devtest.py then you need to install python-argparse
$ sudo apt-get install python-argparse
Personal tools