OpentestOtherSrvLinuxDevtest

From AragoWiki

(Difference between revisions)
Jump to: navigation, search
(--advanced-params)
m
Line 168: Line 168:
=== --advanced-params ===
=== --advanced-params ===
-
* var_use_default_env=yes will boot using whatever is the platform's default boot method. The test script will just do ''env default -a -f'' followed by ''boot''
+
* var_use_default_env=yes will boot using whatever is the platform's default boot method. The test script will just do ''env default -a -f '' followed by ''boot''
= how to create new tests (ltp-ddt) =
= how to create new tests (ltp-ddt) =
 +
The easier way to create new ltp-ddt tests is to run tests using nfs and then to build/install your local ltp-ddt into this filesystem.
     ./linux-devtest.py -U, cd ltp-ddt, make, make install to nfs
     ./linux-devtest.py -U, cd ltp-ddt, make, make install to nfs
   it is possible to get a core-sdk if no -n is passed
   it is possible to get a core-sdk if no -n is passed

Revision as of 18:53, 13 May 2013

Contents

What's linux-devtest

It is a python script to run automated tests on Linux using Opentest framework.

How to install it

Download latest opentest_installer*.tar.gz from http://arago-project.org/files/releases/opentest/ untar it, run it and then:

  • If you plan to run tests using TI's board farm
Select options 2 (STAF) and 8 (Command Line Tools)
  • If you plan to run tests on your own desktop
Select options  2 (STAF), 4 (TMC), 5 (VATF), 6 (TEE) , 7 (BEE) and 8 (Command Line Tools) 

NOTE: Depending on your linux distribution, you may also need to install python module 'argparse'.

There are many files that contain setup configuration. The installer will preset most values during the installation process, however the most important configuration parameters are documented here:

  • Test Master Controller (TMC) IP address
 Defines Test Master Controller to send test requests to.
 Location: config/site_info.xml <tmc_machine> xml element
 Default: 158.218.108.109 (which is TI Germantown TMC server address)
  • Local FTP Server URL
 Defines ftp server directory url where local files (i.e. kernel, dtb, etc.) should be copy to.
 This URL should be accessible from TMC defined above.
 Location: config/site_info.xml <local_ftp_server_url> xml element
 Default: ftp://anonymous:anonymous@gtautoftp.gt.design.ti.com/anonymous/opentest/

Configuration parameters applicable to installation case 2) only (running tests on your own desktop):

  • VATF TEE's bench file
 Describes the equipment (DUT and test) available in the VATF Test Execution Engine
 and its properties
 Location: <Opentest root>/bench/
  • NFS export root directory
 Root directory where NFS tar balls (if provided) can be untared and exported to DUT
 Location: VATF bench file

How to use it

linux-devtest.py allows users to load a specified evm type with specified software images and then run a set of tests.
The tool supports 3 different types of test executions as described in the following sections:

Executing ad-hoc commands

linux-devtest.py -s <commands or script>

Use this option to run any arbitrary commands or even a shell script that can reside in your host pc.

Setting EVM type

linux-devtest.py ... -hw omap5-evm,linux

The -hw syntax is evm,capabilities where capabilities is underscore-separated list of strings

Setting SW images

linux-devtest.py ... -p /my/path/MLO -b /my/path/uboot -k /my/path/uImage -d /my/path/dtb -n 1.2.3.4:/my/nfs/root

All SW images are optional. If you don't specify them, the test script assumes that by default they are available in the MMC card. It is possible to tweak the default behavior by defining arguments using --advance-params option. For more information see advanced-params

Forcing execution on a specific Test Execution Engine (TEE)

linux-devtest.py ... -a "vatf on 158.218.103.10"

It is typical that users will want the test to run on their TEE, by default Opentest will assign the test job to any capable idle TEE. however, it is possible to force execution on a particular TEE by using -a option. The assignment string pass to -a is "<TEE> on <Machine>", where TEE can be either just the TEE type or the type@id and Machine is the IP address.

Examples

1) Running arbitrary commands

./linux-devtest.py -hw omap5-evm,linux -a "vatf on 158.218.103.10" --advanced-params var_use_default_env=yes -s "echo 'hello world'; echo 'goodbye world'"

2) Running shell script from host machine

./linux-devtest.py -hw omap5-evm,linux -a "vatf on 158.218.103.10" --advanced-params var_use_default_env=yes -s /home/a0850405/tmp/mytest.sh




Executing LTP-DDT Tests

linux-devtest.py -t <LTP-DDT testplan name>

Use this option to run your own LTP-DDT test plans.

Setting EVM type

linux-devtest.py ... -hw beaglebone

The -hw syntax in this case just identifies the platform name.

Setting SW images

See setting SW images

Creating your own LTP-DDT testplan

1) Update default tests. This will clone ltp-ddt project if required.

./linux-devtest.py -U

2) Create your own test plan

cp tests/default.py tests/mytests.py

3) Modify tests_to_run list in tests/mytests.py to suit your needs by copying entries from tests_available list into it.
Each entry in tests_available lists points to a ltp-ddt test scenario (aka test suite). The syntax is:
<test scenario name>:<setup requirements>:<test cases filter>

Examples

1) Running mytests testplan using specified kernel, devicetree blob and network filesystem. Execution is forced to tee vatf@2 on machine 158.218.103.10

./linux-devtest.py beaglebone -n 158.218.103.10:/home/a0850405/NFS_exports/linux/6c0214dbe934fc0c6c2b8dcd2245d31e -a "vatf@2 on 158.218.103.10" -k /mnt/gtautoftp/tmp/carlos/nightly-images/ti-linux-3.8-rc5/uImage-beaglebone.bin -d /mnt/gtautoftp/tmp/carlos/nightly-images/ti-linux-3.8-rc5/uImage-beaglebone.dtb -w -t mytests

2) Running default ltp-ddt tests (i.e. no -t | -T | -s options) on am335x-evm

./linux-devtest.py -hw am335x-evm -a "vatf@4 on 158.218.103.10"  -n 158.218.103.10:/home/a0850405/NFS_exports/linux/arago-test/autofs/4bd521fbc31c8e728ac5d778b0648fcd





Executing Testlink Testplans

linux-devtest.py -T <Testlink testplan>

Use this option to run Testlink testplans.

Setting EVM type

linux-devtest.py ... -hw beaglebone

The -hw syntax in this case just identifies the platform name.

Setting SW images

See setting SW images

Examples

1) Running Testlink lcpd_functional_sanity_tests that it is part of linux_psp2 project

./linux-devtest.py -hw am335x-evm -n 158.218.103.10:/home/a0850405/NFS_exports/linux/6c0214dbe934fc0c6c2b8dcd2245d31e -a "vatf@4 on 158.218.103.10"  -k /mnt/gtautoftp/tmp/carlos/nightly-images/ti-linux-3.8-rc5/uImage-beaglebone.bin -d /mnt/gtautoftp/tmp/carlos/nightly-images/ti-linux-3.8-rc5/uImage-beaglebone.dtb -T linux_psp2:lcpd_functional_sanity_tests

See arguments list for more information about how to get the list of projects and testplans available in Testlink




Detailed arguments list

usage: linux-devtest.py [-h] [-hw HW] [-p PRI_BOOT] [-b SEC_BOOT] [-k KERNEL]
                       [-m KERNEL_MODULES] [-d DTB] [-r RAMFS] [-n NFS]
                       [-s SCRIPT | -t TESTS | -T TESTPLAN | --list-projects | --list-testplans LIST_TESTPLANS | --list-builds LIST_BUILDS]
                       [-w] [-a ASSIGNED_TO_TEE] [-c]
                       [--advanced-params ADVANCED_PARAMS] [-R REPORT] [-U]
                       [-f FARM]
Run ltp-ddt tests using Opentest
optional arguments:
 -h, --help            show this help message and exit
 -hw HW                DUT type and optional capabilities. i.e.
                       beaglebone,linux
 -p PRI_BOOT, --pri-boot PRI_BOOT
                       Primary bootloader. Can be local file or http/ftp url
 -b SEC_BOOT, --sec-boot SEC_BOOT
                       Secondary bootloader (i.e. u-boot). Can be local file
                       or http/ftp url
 -k KERNEL, --kernel KERNEL
                       kernel image to load on DUT. Can be local file or
                       http/ftp url
 -m KERNEL_MODULES, --kernel-modules KERNEL_MODULES
                       kernel modules to install in the FS. Can be local file
                       or http/ftp url
 -d DTB, --dtb DTB     device tree blob to pass to kernel. Can be local file
                       or http/ftp url
 -r RAMFS, --ramfs RAMFS
                       RAMFS image to load on DUT (i.e. rootfs.ext2.gz). Can
                       be local file or http/ftp url
 -n NFS, --nfs NFS     Use NFS. Value can be local file, http/ftp url or nfs
                       url (i.e. server:/path/to/nfs/root). Recent filesystem
                       tar ball is always available at http://gtautoftp.gt.de
                       sign.ti.com/anonymous/linux/latest/arago-test-
                       image.tar.gz
 -s SCRIPT, --script SCRIPT
                       Shell test script to run in the DUT
 -t TESTS, --tests TESTS
                       File with list of ltp-ddt tests to run
 -T TESTPLAN, --testplan TESTPLAN
                       Run Testlink named testplan on named testproject
                       optionally using named build.
                       Syntax:'<testproject>:<testplan>[:buildname]'
 --list-projects       Get list of Testlink test projects
 --list-testplans LIST_TESTPLANS
                       Get list of Testlink test plans for named project ID.
                       Value must be valid testlink project ID. Used --list-
                       projects to get valid project IDs
 --list-builds LIST_BUILDS
                       Get list of Testlink test builds for named testplan
                       ID. Value must be valid testlink testplan ID. Used
                       --list-testplans to get valid testplan IDs
 -w, --distribute-workload
                       Allow running test on multiple DUTs in parallel
 -a ASSIGNED_TO_TEE, --assigned-to-tee ASSIGNED_TO_TEE
                       Force tests to run on this Test Execution Engine
 -c, --force-test-scripts-clone
                       Clone latest TEE test scripts
 --advanced-params ADVANCED_PARAMS
                       extra tweaks for advanced users. Separate multiple
                       values with ~
 -R REPORT, --report REPORT
                       Get Testlink testplan results in this format
 -U, --update-default-tests
                       Get latest version of test cases from ltp-ddt
 -f FARM, --farm FARM  Force test to run in a board at this farm (tigt_farm,
                       tid_farm, tii_farm). To run on any farm then just use
                       'farm'

--advanced-params

  • var_use_default_env=yes will boot using whatever is the platform's default boot method. The test script will just do env default -a -f followed by boot

how to create new tests (ltp-ddt)

The easier way to create new ltp-ddt tests is to run tests using nfs and then to build/install your local ltp-ddt into this filesystem.

   ./linux-devtest.py -U, cd ltp-ddt, make, make install to nfs
  it is possible to get a core-sdk if no -n is passed

how to contribute tests back to ltp-ddt

More information

  • Check the README file in the Opentest Client root directory
  • Ask for help at opentest@arago-project.org
Personal tools