OpentestHowTo

From AragoWiki

(Difference between revisions)
Jump to: navigation, search
(Created page with '= How-To Install Opentest Client tools = * Find a Ubuntu system (10.04 or later should be fine) * Download and untar latest opentest_installer*.tar.gz from http://arago-project.o…')
Line 38: Line 38:
* Also verify that [[#How-To_Start_STAF | STAF is running ]]
* Also verify that [[#How-To_Start_STAF | STAF is running ]]
* Run linux-devtest.py -s from the location where you installed Opentest client tools
* Run linux-devtest.py -s from the location where you installed Opentest client tools
-
   # Simple test to boot am335x-evm using given kernel, dtb, modules and filesystem and echo 'hello world' upon booting.
+
   # Simple test to boot and echo 'hello world' upon booting.
  ./linux-devtest.py '''-s "echo 'hello world'"''' ...
  ./linux-devtest.py '''-s "echo 'hello world'"''' ...
-
* If you need check [[#linux-devtest.py_arguments_help | linux-devtest.py help]]
+
* Please check [[#linux-devtest.py_arguments_help | linux-devtest.py help]] for more details
Line 48: Line 48:
* Run linux-devtest.py -t from the location where you installed Opentest client tools
* Run linux-devtest.py -t from the location where you installed Opentest client tools
  ./linux-devtest.py '''-t <LTP-DDT testplan name>''' ...
  ./linux-devtest.py '''-t <LTP-DDT testplan name>''' ...
-
* If you need check [[#linux-devtest.py_arguments_help | linux-devtest.py help]]
+
* Please check [[#linux-devtest.py_arguments_help | linux-devtest.py help]] for more details
= How-To Run Testlink tests plans on the boards Farm =
= How-To Run Testlink tests plans on the boards Farm =
Line 55: Line 55:
* Run linux-devtest.py -T from the location where you installed Opentest client tools
* Run linux-devtest.py -T from the location where you installed Opentest client tools
  ./linux-devtest.py '''-T <Testlink testplan>''' ...
  ./linux-devtest.py '''-T <Testlink testplan>''' ...
-
* If you need check [[#linux-devtest.py_arguments_help | linux-devtest.py help]]
+
* Please check [[#linux-devtest.py_arguments_help | linux-devtest.py help]] for more details
= linux-devtest.py arguments help =
= linux-devtest.py arguments help =
Line 72: Line 72:
</div>
</div>
  linux-devtest.py ... -hw omap5-evm,linux
  linux-devtest.py ... -hw omap5-evm,linux
 +
 +
== -p, -b, -k, -m, -d, -r, -n Software arguments ==
 +
Set software images (i.e. bootloader, kernel, filesystem, etc.) to used for the test.
 +
It is not mandatory to provide all of them. For example if bootloader (-p and -b) are not provided then the system will try to boot the DUT using whatever bootloader image is already installed on it.
 +
Most software image values can take either a local path or a http/ftp url. Use ''./linux-devtest.py'' -h for more details
 +
 +
== -u, --user-bins argument ==
== -u, --user-bins argument ==
Line 118: Line 125:
== --advanced-params ==
== --advanced-params ==
 +
advanced-params are variables that advanced users can set to tweak execution.
 +
At the moment the only advanced-params supported by linux-devtest.py is ''var_use_default_env''.
 +
Usually linux-devtest.py will try to boot the DUT using provided software images (i.e. kernel, filesystem, etc.), but adavanced users may change that default behavior.
* var_use_default_env=1 means Boot using default environment (i.e. env default -a -f; boot )
* var_use_default_env=1 means Boot using default environment (i.e. env default -a -f; boot )
* var_use_default_env=2 means Do not touch uboot env, just power cycle and let it go  
* var_use_default_env=2 means Do not touch uboot env, just power cycle and let it go  

Revision as of 15:36, 6 September 2013

Contents

How-To Install Opentest Client tools

  • Find a Ubuntu system (10.04 or later should be fine)
  • Download and untar latest opentest_installer*.tar.gz from http://arago-project.org/files/releases/opentest/
  • Run the installer (e.g. ./install_opentest.sh) and Select options 2 (STAF) and 8 (Command Line Tools)
    • The installer will install Java if required
    • Then it will install STAF (you can leave all the default install values)
    • Finally it will install opentest_client components. You can leave default values for most options, however subnets must be entered. If you are in the US we recommend entering at least following 3 subnets: 158.218.*.* 10.218.*.* 128.247.*.*

How-To Start STAF

  • Check if STAF is running
$ staf local help
Usage: STAF [-verbose] <Endpoint | LOCAL> <Service> <Request>
  • If you don't get an error while running staf local help command, then STAF is running and you are good to go. However if you do get an error then you need to open /usr/local/staf/STAFEnv.sh and verify that STAF_INSTANCE_NAME is set to a value as shown below
if [ $# = 0 ]
then
   STAF_INSTANCE_NAME=STAF
else
   if [ $1 != "start" ]
   then
       STAF_INSTANCE_NAME=$1
   else
       # Ignore "start" STAF instance name
       STAF_INSTANCE_NAME=STAF
   fi
fi
export PATH LD_LIBRARY_PATH CLASSPATH STAFCONVDIR STAF_INSTANCE_NAME
  • Finally run /usr/local/staf/startSTAFProc.sh to start STAF. Cat nohup.out file to check correct initialization
$ ./startSTAFProc.sh 
$ nohup: appending output to `/home/charliebrown/nohup.out'
$ cat /home/a0850405/nohup.out
Machine          : charliebrown
Machine nickname : charliebrown
Startup time     : 20130905-18:06:02


How-To Run your own tests on the boards Farm

 # Simple test to boot and echo 'hello world' upon booting.
./linux-devtest.py -s "echo 'hello world'" ...


How-To Run ltp-ddt tests on the boards Farm

./linux-devtest.py -t <LTP-DDT testplan name> ...

How-To Run Testlink tests plans on the boards Farm

./linux-devtest.py -T <Testlink testplan> ...

linux-devtest.py arguments help

command help

# use -h flag to show command help
./linux-devtest.py -h

-hw argument

The -hw argument selects the type of hardware (aka DUT) to run the tests on. It is also possible to specify capabilities or requirements that selected DUT must have. For instance, capabilities could be used to ask for a board that has an usb mass storage device attached to it. The argument syntax is -hw evm,capabilities where capabilities is an optional underscore-separated list of strings Valid evm names are the same used by OE/Yocto: am335x-evm, omap5-evm, beaglebone, dra7xx-evm, etc. For valid capabilities see Capabilities cheat sheet

linux-devtest.py ... -hw omap5-evm,linux

-p, -b, -k, -m, -d, -r, -n Software arguments

Set software images (i.e. bootloader, kernel, filesystem, etc.) to used for the test. It is not mandatory to provide all of them. For example if bootloader (-p and -b) are not provided then the system will try to boot the DUT using whatever bootloader image is already installed on it. Most software image values can take either a local path or a http/ftp url. Use ./linux-devtest.py -h for more details


-u, --user-bins argument

Executable or tarball of executables that should be installed on FS This is useful for cases where user wants to run a binary that is not available in the default filesystem The value provided can be either a local file or a public http/ftp url

linux-devtest.py ... -u /home/tmp/myapp

-s, --script argument

Use this option to run any arbitrary commands or a shell script in the DUT. The syntax is either a string of semicolon-separated commands (e.g. "echo 'hello'; echo 'bye'") or a path to a shell script. In either case the test will pass if the return value of the commands or script is zero.

linux-devtest.py ... -s /home/tmp/mytest.sh

-t, --tests argument

Use this option to run your own ltp-ddt test plan. Please note that -s, -t and -T are mutually exclusive options

 linux-devtest.py ... -t mytests

To create your own ltp-ddt test plan follow this steps:

  • Update default tests. This will clone ltp-ddt project if required.
./linux-devtest.py -U
  • Create a copy of default test plan
cp tests/default.py tests/mytests.py
  • Modify tests_to_run list in tests/mytests.py to suit your needs by copying entries from tests_available list into it.

Each entry in tests_available lists points to a ltp-ddt test scenario (aka test suite). The syntax is:
<test scenario name>:<setup requirements>:<test cases filter>


-T, --testplan argument

Use this option to run a Testlink test plan. If you use this option results will be stored in the Testlink server. The Testlink Testplan name syntax is: <project name>:<testplan name>

./linux-devtest.py -T linux_psp2:ch_sandbox

This command will return the list of projects

./linux-devtest.py --list-projects

and this one will return the testplans available for linux_psp2 project. linux_psp2 is the project name for most Linux-related work

./linux-devtest.py --list-testplans 1140921

-f, --farm argument

forces usage of a board in a farm.

-c, --force-test-scripts-clone argument

Makes sure that latest version of Opentest's scripts are used by forcing a pull of test scripts project at the Test Execution Engine.

--advanced-params

advanced-params are variables that advanced users can set to tweak execution. At the moment the only advanced-params supported by linux-devtest.py is var_use_default_env. Usually linux-devtest.py will try to boot the DUT using provided software images (i.e. kernel, filesystem, etc.), but adavanced users may change that default behavior.

  • var_use_default_env=1 means Boot using default environment (i.e. env default -a -f; boot )
  • var_use_default_env=2 means Do not touch uboot env, just power cycle and let it go

Errors

  • If you get ImportError: No module named argparse while running linux-devtest.py then you need to install python-argparse
$ sudo apt-get install python-argparse
Personal tools