OpentestTeeVatf

From AragoWiki

Jump to: navigation, search

Contents

Background

VATF was originally develop in 1Q 2007. It was developed to automate Digital Video Software Development Kit (DVSDK) System Test cycle, hence the name Video Automation Test Framework (VATF). However, VATF is generic enough and can/has been used to automate test suites on other products such as Linux, Android and Wince Platform Support Packages or Development kits
VATF was written using the Ruby programming language.

Why Ruby?
Test automation involves a lot of string handling operations. Scripting languages typically supports string operations much better than other languages such as Java and C.
Ruby is an object-oriented language that combines some of the best features from Perl and Java and adds some on its own like module mixin and code closures. 
Ruby is a very dynamic language and has strong reflection capabilities.
From Ruby's main site "Ruby is A dynamic, open source programming language with a focus on simplicity and productivity. 
It has an elegant syntax that is natural to read and easy to write"

High-Level Design

Services provided by VATF

The VATF Core engine provides the following services:

1) Read test parameters from a Test Suite Database.

2) Initialize all equipment required to run a test

3) Calls the appropriate test script and pass to it:

a) a representation of the test parameter that abstract the Testlink DB (@test_params object),
b) a generic representation of the test equipment (@equipment object), 
c) a generic results form that can be used in the script to provide detailed results data (@html_result_form object), and 
d) a method to set the test result back in the test suite database (set_result() method)

4) Save results back to the Test Suite Database and create an HTML test execution report


VATF tries to be as generic and open as possible. It encourages test script reuse because the equipment and database details are hidden from the user. As an example see the test script snippet in the figure below

Image:vatf_testScript_interface.jpg

Hardware Management

VATF abstracts the hardware details from the test script by relying on 2 user-specified pieces of information:

  • The Bench file: The user defines the equipment he/she has on his/her bench (aka test station) by creating a bench file.

The Bench file describes all the equipment available in a given test station.
All the equipment parameters, including the driver class that controls it, are specified in the bench file exclusively.
This provides a clean separation or interface between the equipment and VATF.
The bench file shown below defines the following equipment: one multimeter, one AM335x-evm, one power controller, one linux PC and one USB switch.

# Sample Bench File w/ five pieces of equipment:
minfo = EquipmentInfo.new("multimeter")
minfo.serial_port = '/dev/ttyUSB0'
minfo.serial_params = {"baud" => 19200, "data_bits" => 8, "stop_bits" => 1, "parity" => SerialPort::NONE}
minfo.driver_class_name = 'KeithleyMultiMeterDriver'
minfo.params = {'number_of_channels' => 40}
dut = EquipmentInfo.new("am335x-evm","linux_sd_usbhostmsc_usbhosthid")
dut.driver_class_name = "LinuxAm335xDriver"
dut.serial_port = '/dev/ttyS0'
dut.serial_params = {"baud" => 115200, "data_bits" => 8, "stop_bits" => 1, "parity" => SerialPort::NONE}
dut.prompt = /[\w\d]+@.+[@:#]+/
dut.login = 'root'
dut.telnet_login = 'root'
dut.prompt = /#/
dut.boot_prompt = /U-Boot#/
dut.login_prompt = /login:/
dut.board_id='20100720'
dut.power_port = {'apc.158.218.103.33' => 8}
dut.params = {'usb_port' => 1, 'multimeter1' => minfo}
pwr = EquipmentInfo.new("power_controller", "apc.158.218.103.33")
pwr.telnet_ip = '158.218.103.33'
pwr.telnet_port = 23
pwr.driver_class_name = 'StafApcPowerController'
pwr.params = {'staf_ip' => 'local'}
svr = EquipmentInfo.new("linux_server")
svr.tftp_path = '/tftpboot'
svr.driver_class_name = 'LinuxLocalHostDriver'
svr.telnet_login = 'a0850405local'
svr.prompt = /@@/
usb = EquipmentInfo.new("usb_switch", 0)
usb.serial_port = '/dev/ttyUSB1'
usb.serial_params = {"baud" => 9600, "data_bits" => 8, "stop_bits" => 1, "parity" => SerialPort::NONE}
usb.driver_class_name = 'ExtronUsbSwitch'


  • The HW Assets field in TestLink: Identifies the equipment required to run a test case and provides a mapping between the generic equipment names used in the test script and the real hardware available in the test station.

Sample test setup file or HW Assets:

dut1 = ["am335x-evm",linux_sd]; server1 = ["linux_server]

For more information about HW assets field information, please see HW Resource Manager


The HW assets field and the bench file described above allow writing test scripts in a generic hardware-independent way. For instance in the above example the test scripts will use syntax like @equipment['dut1'].do_something or @equipment['server1'].do_something without knowing or caring about the actual platform type (i.e. am335x, am37x, dm365x, etc.)


Steps required to automate a new area

To automate a given area, the user needs to create the following components:

  1. Test Script: This the actual script that contains the logic to run the test.
    The test script MUST implement 3 methods: setup(), run() and clean().
    These methods are the interface or entry points between VATF and the test script.
  2. Test Equipment Driver: Optionally the user might need to write a new Test Equipment driver class if one, for the hardware he/she wants to use, doesn't exist

List of available DUT/Test Equipment Drivers

Drivers for the following Hardware has already been written and validated:

  1. Generic Linux host driver
  2. APC Power Controller
  3. Extron Video/Audio Switches
  4. Tascam DVD players
  5. Pionner DVD players
  6. Q-Master's Video Quality System
  7. Video Clarity's Video Quality System
  8. Extron USB Switch
  9. Keithley Multimeter
Personal tools