Department of Commerce ISSI Evaluation and Test Tool

DIETS User Guide


M. Ranganathan

      NIST Advanced Networking Technologies Division

1. Introduction

The Department of Commerce ISSI Evaluation and Test System ( DIETS ) is a reference implementation and test framework to test ISSI implementations. It can be used to generate conformance test cases for the ISSI specification. In its current state, the DoC ISSI Evaluation & Test System is a set of tools to evaluate and test implementations of the Project 25 ISSI M&P specification (TIA-102.BACA). The DEITS tool set includes the following:

2. Intended Audience

The DIETS tool is intended as an aid to developers of the ISSI Specification. It is a tool that can allow vendors to establish compliance with the ISSI M&P document and to develop additional tests that are not currently part of the document. The reader of this document is assumed to have a familiarity with the ISSI M&P Document ( BACA 1020) from the ISSI Task group. The reader is also assumed to have some level of familiarity with the java programming language and XML.


3. Architectural Overview

The approach followed in building DIETS is to provide an instrumented Reference Implementation (RI) embedded in a test framework. DIETS emulates the ISSI G interfaces as shown below. DIETS will appear as a live ISSI/RFSS interface to the Implementation Under Test (IUT). RFSS/SU behavior behind DIETS will be emulated. The emulated reference implementation generates test messages to interact with unit(s) under test and reacts to incoming messages. In its most general configuration, the ISSI Emulator software is a distributed tester that allows you to test one or more actual RFSS implementations against emulated RFSS implementations. The diagram below shows DIETS configured in such a fashion:







The diagram above shows a single IUT surrounded by emulated components, which in the general case can be distributed.

Note that the tester instantiation consists of a front end (DIETS GUI) and one or more back end processes that emulate the RFSS’s. The role of the font end is to provide a Graphical User Interface ( GUI) to allow the user to easily pick a test and instruct the back end daemons to run the test. The GUI and back end communicate via HTTP. In distributed mode, DIETS starts a web server on each location where it is started to allow for communication with the GUI. In standalone mode ( where the GUI and emulated components are integrated into a single client), there is no web server started and everything runs as a single process.

For conformance testing, assuming that the IUT can handle non 5060 ports for SIP signaling, the emulator can be run as a single process and there is a single unit under test. Further, the trace capture facility is not necessary when RFSSs are tested in isolation because packets will be logged in application memory at the emulated nodes.


4. Prerequisites

The emulator is written in the Java Programming language and requires Java 1.6 or above to run. Java is available from java.sun.com. Please download and install the java development kit (JDK) and set your JAVA_HOME environment variable. The emulator also uses the Ant platform independent build tool. Ant is available from the Apache software foundation ( www.apache.org). Please download and install ant. Add junit support to ant so ant can be used to execute junit test cases.

5. Emulator Configuration information

In this section and all following sections where we describe XML tags and attributes, we follow the convention that xml tags are specified in boldface and attributes are specified in italics. Rather than present the complete DTD for the xml, we refer the reader to the files located in the DTD Documentation directory and just give an overview here.


Here are the major configuration files for the DIETS emulator:

  1. An emulator tester configuration file (e.g. see testerconfig/distributed-configuration.xml) that indicates the nodes on which the tester daemons run. This file is provided to the GUI and to each emulator process.A tester can be run on multiple IP addresses. Each IP address on which the tester can run should be specified in this file using a diets-daemon tag. A daemon node can host a conformance tester and a packet monitor. It can be referred to using the nested refid tags. The purpose of this file is to provide information to the GUI and the back end about where to listen for incoming HTTP requests. This file also specifies whether or not to start a packet monitor. The packet monitor also communicates via http with the front end and its HTTP address and port are specified in this file.

  1. There is a system topology file ( e.g. testsuites/conformance/systemtopology.xml ) that specifies how the RFSSs are configured and system IDs, which is provided to each emulator node and on startup. The IP addresses and ports of the RFSSs ( both emulated and real ), the WACN ID, RFSS ID and System ID of each emulated and real RFSS are specified in this file.

  2. There is a global topology file for a given test suite (e.g. testsuites/conformance/testscripts/globaltopology.xml in the distribution) which is provided to the emulator and the GUI on startup. The purpose of this global topology file is to specify assignment of SUs to RFSSs (specification of home and initial serving RFSS), group home, group membership and other details. These are common across all tests. The details of this file are elaborated below.

  3. Each test includes one or more topology files which specializes the global topology file. The topology file can specify attributes such as the time it takes to process INVITE (i.e. SIP Call setup ). Additional symbolic identifiers can be specified in this file to refer to RFSSs and SUs symbolically in test scripts. The details of this file are elaborated below

  4. Each test includes a testscript.xml file which is the event driven script corresponding to a given test. Events can include message arrivals and global time events.

  5. The catalog of test cases that is loaded by the emulator is in a file called testsuites/test-suite-name/testscripts/testregistry.xml.



NAME

Example

Purpose

Documentation

tester-configuration

testerconfig/distributed-configuration.xml

Tells the front end controller where the other tester daemons are located.

issi-tester-configuration.dtd.html

systemtopology

testsuites/conformance/systemtopology.xml

Provides the IP address and ports of the RFSSs. A flag indicates whether a given RFSS is real or emulated.



systemtopology.dtd.html

globaltopology

testsuites/conformance/testscripts/globaltopology.xml

Provides test specific topology information ( inherits from systemtopology.xml)



globaltopology.dtd.html

testtopology

testsuites/conformance/su_registration_failed/topology1.xml

Inherits from globaltopology.xml and specializes RFSS settings and group assignments for a specific test.


testscript

testsuits/conformance/su_registration_failed/testscript

A single test script per test case that indicates the event-trigger asynchronous action for the script.



testscript.dtd.html




The emulator comes bundled with a set of tests which are the conformance tests specified by the ISSI Working group. There is a directory for each test. The tests are cataloged in a test registry. The test suites are organized by test registry. For example testsuites/conformance is a test registry for the conformance test cases.



6. Starting the emulator


You can start the DIETS tools using the DIETS console or you can start individual tools using ant. The latter option gives you more control and is good for debugging.


The emulator can be started in distributed or standalone mode. In distributed mode, the emulator GUI is separated from the back ends. The back ends emulate the components under test. In standalone mode, the back and front ends are combined into a single process. The standalone mode can be used if your IUT can communicate with emulated components that do not run at port 5060 (for SIP Signaling). The instructions provided in README.txt should be followed in running the emulator.

Here is a screen shot of what you should see when you start the GUI. The GUI has all the test cases cataloged in the left column. You can select and run a conformance test case by clicking on an item in this menu.










The important XML configuration files and scripts that are associated with a given test are visible in the GUI tabs. The steps involved in running a test are explained next.


7. Running Conformance Test Cases

Conformance testing is a two step process. In the first step, you have to generate a reference trace for a given scenario. In the second step, you run against your Implementation Under Test (IUT). Reference traces are not included in the distribution because they depend upon the System ID and WACN ID that you specify in the global topology file. Each DIETS node generates its own reference trace and hence the file systems should not be shared for the different emulator machines.


7.1 Edit the System Topology file


Every test suite contains a systemtopology.xml that is global to the entire test suite. You need edit this only once for a given test suite. The information contained here has been previously explained. You have to edit this file and set up the IP addresses and ports to where your emulated and real RFSS components reside. Note that you should use the @node_name notation for the IP address of emulated RFSSs to assign these to the tester daemons in a distributed network. Set the emulated flag to false for the RFSS under test. See systemtopology.dtd documentation for additional information the contents of this file.


7.2 Edit the Global Topology


The global topology file (see testscript/globaltopology.xml for a the conformance suite for example) specifies the global group and SU settings and assignments for the entire test suite. Note that it does not specify group membership. This is specified on an individual test basis in the topology.xml files for that test case The contents of this file are documented in detail in globaltopology.dtd.html



8.0 The Details



This section presents some of XML files associated with a test case that you do not necessarily need to edit unless you plan to write your own test cases for the emulator.

8.1 The Test Topology Specialization


Each test includes one or more topology.xml files. The function of these files is to specify conditions that must be setup for an instantiation of the global topology in order to run the test. This is where you specify SU and SG profile information for the test. If no profile is specified then a default profile is used. To override the profile values, you can use the attributes defined for this purpose. It is essentially a specialization of the global topology and includes references to the global topology file. In addition, the purpose of this file is to specify resource conditions and group membership. The tags and attributes for the topology file are specified in the documentation topologyconfig.dtd


8.2 The event driven test script (testscript.xml)


Each test case is contained in its own directory. A test case contains a test topology (topology.xml) which inherits tags and attributes from the globaltopology.xml file and specifies some additional attributes for emulated RFSSs. It also contains a test script which specifies the event triggered scripting actions. testscript.xml defines an event triggered set of actions. Event triggers are events (such as real-clock time, SIP or PTT message arrivals at an emulated RFSS) that trigger a scripting action. Scripting actions are specified using xml tags where ever possible so that parameters may be checked at the time the script is parsed. This reduces runtime errors. Following this design approach, all call setup actions are specified using xml tags. In addition we support actions to specify fragments of jython code (a Java implementation of Python). These fragments of jython code may invoke a set of functions specific to the RFSS or SU for which they are specified. The xml event tags and their meanings are elaborated upon below in the DTD Section [testscript.dtd.html].




9 Trace Capture Tool


DIETS includes a packet monitor or trace capture tool called ITAT ( ISSI Trace Analysis Tool). The purpose of ITAT is to capture packets for a call flow and show the results as a nicely formatted sequence diagram. ITAT works using JPCap ( wrapper over libpcap ) on Windows XP or the LINUX platform. You launch one ITAT trace capture daemon for every subnet where you will run an emulated or real RFSS and you launch a front end to retrieve captured packets from ITAT and display them as a sequence diagram. Here is a conceptual view of how ITAT works:

The topology file specifies the IP addresses and ports for SIP signaling ( systemtopology.xml). The packet monitor picks up the relevant packets headed for the IP addresses specified in the systemtopology.xml file using Ethernet in promiscuous mode to sniff packets. Note that the capture daemon needs to run on a hub. It cannot run on a switch. It uses jpcap ( a Java wrapper on top of libpcap) to do thisPackets are logged in memory for the duration of the call flow. When the call flow is done, the front end can retrieve packets for display. At this time packets are sorted and analyzed.


You can start the packet capture daemon using the instructions in README.txt. After the back ends are started, the front end is started. Packet capture starts when you press the button on the front end GUI. The front end retrieves traces and displays them at the end of a run. Note that the test tool is not required in order to run ITAT. You can use ITAT for interoperability testing without needing to run DIETS.