OpentestOtherSrvLinuxDevtest

From AragoWiki

(Difference between revisions)
Jump to: navigation, search
(How to use it)
Line 42: Line 42:
== Executing ad-hoc commands ==
== Executing ad-hoc commands ==
-
'''linux-devtest.py -s <commands or script>'''
+
linux-devtest.py -s <commands or script>
Use this executing method to run any arbitrary commands or even a shell script that can reside in your host pc.
Use this executing method to run any arbitrary commands or even a shell script that can reside in your host pc.
=== Setting EVM type ===
=== Setting EVM type ===
-
'''linux-devtest.py -s <commands or script> -hw omap5-evm,linux'''
+
linux-devtest.py ... -hw omap5-evm,linux
The -hw syntax is ''evm,capabilities'' where ''capabilities'' is underscore-separated list of strings
The -hw syntax is ''evm,capabilities'' where ''capabilities'' is underscore-separated list of strings
-
 
=== Setting SW images ===
=== Setting SW images ===
-
 
+
linux-devtest.py ... -p /my/path/MLO -b /my/path/uboot -k /my/path/uImage -d /my/path/dtb -n 1.2.3.4:/my/nfs/root
 +
All SW images are optional. If you don't specify them, the test script assumes that by default they are available in the MMC card.
 +
It is possible to tweak the default behavior by defining arguments using --advance-params option. For more information see [[Link advanced-params]]
 +
=== Forcing execution on a specific Test Execution Engine (TEE) ===
 +
linux-devtest.py ... -a "vatf on 158.218.103.10"
 +
It is typical that users will want the test to run on their TEE, by default Opentest will assign the test job to any capable idle TEE. however, it is possible to force execution on a particular TEE by using -a option. The assignment string pass to -a is "<TEE> on <Machine>", where TEE can be either just the TEE type or the type@id and Machine is the IP address.
=== Examples ===
=== Examples ===
1) Running arbitrary commands
1) Running arbitrary commands

Revision as of 14:28, 13 May 2013

Contents

What's linux-devtest

It is a python script to run automated tests on Linux using Opentest framework.

How to install it

Download latest opentest_installer*.tar.gz from http://arago-project.org/files/releases/opentest/ untar it, run it and then:

  • If you plan to run tests using TI's board farm
Select options 2 (STAF) and 8 (Command Line Tools)
  • If you plan to run tests on your own desktop
Select options  2 (STAF), 4 (TMC), 5 (VATF), 6 (TEE) , 7 (BEE) and 8 (Command Line Tools) 

NOTE: Depending on your linux distribution, you may also need to install python module 'argparse'.

There are many files that contain setup configuration. The installer will preset most values during the installation process, however the most important configuration parameters are documented here:

  • Test Master Controller (TMC) IP address
 Defines Test Master Controller to send test requests to.
 Location: config/site_info.xml <tmc_machine> xml element
 Default: 158.218.108.109 (which is TI Germantown TMC server address)
  • Local FTP Server URL
 Defines ftp server directory url where local files (i.e. kernel, dtb, etc.) should be copy to.
 This URL should be accessible from TMC defined above.
 Location: config/site_info.xml <local_ftp_server_url> xml element
 Default: ftp://anonymous:anonymous@gtautoftp.gt.design.ti.com/anonymous/opentest/

Configuration parameters applicable to installation case 2) only (running tests on your own desktop):

  • VATF TEE's bench file
 Describes the equipment (DUT and test) available in the VATF Test Execution Engine
 and its properties
 Location: <Opentest root>/bench/
  • NFS export root directory
 Root directory where NFS tar balls (if provided) can be untared and exported to DUT
 Location: VATF bench file

How to use it

linux-devtest.py allows users to load a specified evm type with specified software images and then run a set of tests.
The tool supports 3 different types of test executions as described in the following sections:

Executing ad-hoc commands

linux-devtest.py -s <commands or script>

Use this executing method to run any arbitrary commands or even a shell script that can reside in your host pc.

Setting EVM type

linux-devtest.py ... -hw omap5-evm,linux

The -hw syntax is evm,capabilities where capabilities is underscore-separated list of strings

Setting SW images

linux-devtest.py ... -p /my/path/MLO -b /my/path/uboot -k /my/path/uImage -d /my/path/dtb -n 1.2.3.4:/my/nfs/root

All SW images are optional. If you don't specify them, the test script assumes that by default they are available in the MMC card. It is possible to tweak the default behavior by defining arguments using --advance-params option. For more information see Link advanced-params

Forcing execution on a specific Test Execution Engine (TEE)

linux-devtest.py ... -a "vatf on 158.218.103.10"

It is typical that users will want the test to run on their TEE, by default Opentest will assign the test job to any capable idle TEE. however, it is possible to force execution on a particular TEE by using -a option. The assignment string pass to -a is "<TEE> on <Machine>", where TEE can be either just the TEE type or the type@id and Machine is the IP address.

Examples

1) Running arbitrary commands

./linux-devtest.py -hw omap5-evm,linux -a "vatf on 158.218.103.10" --advanced-params var_use_default_env=yes -s "echo 'hello world'; echo 'goodbye world'"

2) Running shell script from host machine

./linux-devtest.py -hw omap5-evm,linux -a "vatf on 158.218.103.10" --advanced-params var_use_default_env=yes -s /home/a0850405/tmp/mytest.sh

Executing LTP-DDT Tests

linux-devtest.py -s <commands or script> Use this executing method to run any arbitrary commands or even a shell script that can reside in your host pc.

Setting EVM type

Setting SW images

Examples

Executing Testlink Testplans

linux-devtest.py -s <commands or script> Use this executing method to run any arbitrary commands or even a shell script that can reside in your host pc.

Setting EVM type

Setting SW images

Examples

Detailed arguments list

usage: linux-devtest.py [-h] [-hw HW] [-p PRI_BOOT] [-b SEC_BOOT] [-k KERNEL]
                       [-m KERNEL_MODULES] [-d DTB] [-r RAMFS] [-n NFS]
                       [-s SCRIPT | -t TESTS | -T TESTPLAN | --list-projects | --list-testplans LIST_TESTPLANS | --list-builds LIST_BUILDS]
                       [-w] [-a ASSIGNED_TO_TEE] [-c]
                       [--advanced-params ADVANCED_PARAMS] [-R REPORT] [-U]
                       [-f FARM]
Run ltp-ddt tests using Opentest
optional arguments:
 -h, --help            show this help message and exit
 -hw HW                DUT type and optional capabilities. i.e.
                       beaglebone,linux
 -p PRI_BOOT, --pri-boot PRI_BOOT
                       Primary bootloader. Can be local file or http/ftp url
 -b SEC_BOOT, --sec-boot SEC_BOOT
                       Secondary bootloader (i.e. u-boot). Can be local file
                       or http/ftp url
 -k KERNEL, --kernel KERNEL
                       kernel image to load on DUT. Can be local file or
                       http/ftp url
 -m KERNEL_MODULES, --kernel-modules KERNEL_MODULES
                       kernel modules to install in the FS. Can be local file
                       or http/ftp url
 -d DTB, --dtb DTB     device tree blob to pass to kernel. Can be local file
                       or http/ftp url
 -r RAMFS, --ramfs RAMFS
                       RAMFS image to load on DUT (i.e. rootfs.ext2.gz). Can
                       be local file or http/ftp url
 -n NFS, --nfs NFS     Use NFS. Value can be local file, http/ftp url or nfs
                       url (i.e. server:/path/to/nfs/root). Recent filesystem
                       tar ball is always available at http://gtautoftp.gt.de
                       sign.ti.com/anonymous/linux/latest/arago-test-
                       image.tar.gz
 -s SCRIPT, --script SCRIPT
                       Shell test script to run in the DUT
 -t TESTS, --tests TESTS
                       File with list of ltp-ddt tests to run
 -T TESTPLAN, --testplan TESTPLAN
                       Run Testlink named testplan on named testproject
                       optionally using named build.
                       Syntax:'<testproject>:<testplan>[:buildname]'
 --list-projects       Get list of Testlink test projects
 --list-testplans LIST_TESTPLANS
                       Get list of Testlink test plans for named project ID.
                       Value must be valid testlink project ID. Used --list-
                       projects to get valid project IDs
 --list-builds LIST_BUILDS
                       Get list of Testlink test builds for named testplan
                       ID. Value must be valid testlink testplan ID. Used
                       --list-testplans to get valid testplan IDs
 -w, --distribute-workload
                       Allow running test on multiple DUTs in parallel
 -a ASSIGNED_TO_TEE, --assigned-to-tee ASSIGNED_TO_TEE
                       Force tests to run on this Test Execution Engine
 -c, --force-test-scripts-clone
                       Clone latest TEE test scripts
 --advanced-params ADVANCED_PARAMS
                       extra tweaks for advanced users. Separate multiple
                       values with ~
 -R REPORT, --report REPORT
                       Get Testlink testplan results in this format
 -U, --update-default-tests
                       Get latest version of test cases from ltp-ddt
 -f FARM, --farm FARM  Force test to run in a board at this farm (tigt_farm,
                       tid_farm, tii_farm). To run on any farm then just use
                       'farm'

how to create new tests (ltp-ddt)

   ./linux-devtest.py -U, cd ltp-ddt, make, make install to nfs
  it is possible to get a core-sdk if no -n is passed

how to contribute tests back to ltp-ddt

More information

  • Check the README file in the Opentest Client root directory
  • Ask for help at opentest@arago-project.org
Personal tools