SimpleTester Release Notes

These notes describe the new enhancements made to the SimpleTester since Version 12.0.

General Testing

A communication diagnostic window has been added to allow a user to send ping packets, run a traceroute or send SNMP requests to the agent under test. This will help in testing connectivity and in establishing correct parameters to communicate with the agent.

Added support for SNMPv3 username based parameter profiles. Now when selecting a different username in a drop-down list, all the username's associated SNMPv3 security parameters will get loaded.

Added SimpleTester Tcl procedure call tracing. The trace output is included in the test results file, just like the SNMP debug and packet trace feature. This will show all the SimpleSoft Tcl procedure calls along with the parameters included with each call.

3DES

The enhanced privacy version of SimpleTester now includes support for 3DES in addition to various types of AES.

Script Generation and Execution

A test suite script runner, similar to the Tcl interactive script interpreter, has been added. This allows the user to interactively run and modify a script for easy debugging and development.

Other Fixes and Enhancements

The About box has been enhanced to include licensing information.

SimpleTester Ver 12.0 Release Notes (April 17th, 2009)

These notes describe the new enhancements made to the SimpleTester since Version 11.5.

General Testing

The CMD file, used in running the SimpleTester in batch mode, supports two new parameters. You can specify a Settings profile name and all the parameters associated with the profile will be used during the current invocation. Variables can now be passed into the scripts running in the Scr/Tcl Script Runner for the CMD file.

The name of the current Settings profile is now included in the header of each test results file.

SimpleTester for Workgroups license server now supports USB dongles.

SimpleTesterPro has been enhanced to support the license server.

The MIB Browser has been updated.

New Tests

The vulnerability tests in the SimpleTesterPro have been updated to include checks based on a recent CERT advisory (#878044). This advisory identifies an SNMPv3 authentication vulnerability in some implementations.

Script Generation and Execution

The Script Generator and the Row Status Template generators now support creating Tcl scripts in addition to the older .scr scripts.

A new interactive window has been added to edit, run, and debug Tcl scripts all in one place. Modifying and testing Tcl scripts will be faster and easier with this new window.

Other Fixes and Enhancements

A "Remember Me" checkbox has been added to the login prompt for the SimpleTester For Workgroups. In addition to streamlining the login process, with this enabled, the user will no longer have to specify the username and password when running in batch mode.

The name of the current Settings profile is now included in the header of each test results file.

A new command line utility is now included to do SNMP requests.

SimpleTester Ver 11.5 Release Notes (April 7th, 2008)

These notes describe the new enhancements made to the SimpleTester since Version 11.0.

General Testing

The test engineer can now include an interpacket delay in between Set requests sent in the Syntax test suites. The feature was added to support agents that first respond to a Set request, and then affect the change.

A new command was added to the .scr script interpreter to control the output settings with in the test script.

The Learn Additional Info feature has been enhanced to automatically determine the "LASTONE" value, based on the contents of the walk file.

SNMPv3 testing

New semantic scripts were added to the SNMPv3 test suite to verify the key change feature of USM User Table in the agent.

ST_md5digest and ST_shadigest has been enhanced to support variable length keys and random numbers.

Tcl Interpretor

The Tcl Interpretor inside SimpleTester has been upgraded to 8.4.18 version.

Other Fixes and Enhancements

Fixed Gentest5 to handle missing columns from a MIB group. It now checks to make sure that the new OID is indeed beyond the current MIB group.

Fixed a problem with removing multiple MIBs in the MIB compiler window.

Fixed crash after a User Test Suite was modified by the User Test Suite Builder, when a get suite is not selected and the view menu is clicked, and in GetNTest2 when LASTONE is specified in additional info file.

SimpleTester Ver 11.0 Release Notes (March 29th, 2007)

These notes describe the new enhancements made to the SimpleTester since Version 10.0.

MIB Compiler Interface

The SimpleTester has an updated MIB compilation interface that has enhanced support to resolve external references from imported MIBs. The MIB selection and invocation interface has also been improved.

Script Editor Tool

A Graphical Script Editor tool has been added to allow you to more easily create Tcl based test scripts. The tool provides templates of all the SimpleTester Tcl extensions added to the Tcl interpretor and also includes templates for common procedures like testing counters and strings. Using this, you can more easily create User Defined Test Suites for testing manufacturer specific device capabilities.

RowStatus Script Tempates

Support has now been extended to SNMPv3 agents for testing specific tables that support RowStatus. Now, a trilingual agent supporting dynamic row creation using RowStatus can be tested using SNMPv1, SNMPv2c and SNMPv3 protocols.

Other Fixes and Enhancements

Documentation has been updated to reflect the current behavior of time outs. Better support is now added for large values returned during Parallel testing.

SimpleTester Ver 10.0 Release Notes (March 16th, 2006)

These notes describe the new enhancements made to the SimpleTester since Version 9.5.

Posix Test Results

The SimpleTester now supports the Posix test results as defined in IEEE std 2003-1997.

MIB Browser

The MIB browser has been enhanced to support performance graphs.

New Syntax Tests

Two new syntax tests were added: one to check and make sure the varbind list in a response matches what was sent in the corresponding Set request and another to make sure the community string in the response is the same as the one in the corresponding request.

SimpleTester Ver 9.5 Release Notes (March 30th, 2005)

These notes describe the new enhancements made to the SimpleTester since Version 9.0.

IPv6 Test Suite

The SimpleTester now includes a IPv6 semantic test suite. It tests some of the variables in the following RFCs: RFC 2452, RFC 2454, RFC 2465, and RFC 2466.
MIB Browser

The MIB Browser has been enhanced to include new features such as setting multiple objects based on what was retrieved, debug packet tracing and Get and GetBulk support.
MIB Compiler

A new, more robust MIB compiler is included with SimpleTester.

SimpleTester Ver 9.0 Release Notes (July 16th, 2004)

These notes describe the new enhancements made to the SimpleTester since Version 8.0.

IPv6

The SimpleTester now supports sending of SNMP requets to agents addressable by IPv6. Several types of IPv6 unicast addresses are supported.
AES

Support for AES is now available in the SimpleTester upon request within US. Key lengths of 128, 192 and 256 are supported as per draft-blumenthal-aes-usm-04.txt
Command Line Invocation

The path and name of the log file can now be specified on the command line using the -l option.

Support for X parameters in the the UNIX version
GUI/Scripting enhancements

Support is now added to remember last selection in the test tree

Trap Receiver contains fixes for Report/Response PDU generation.

Script Runner now allows embedded commas and close parenthesis.

SimpleTester Ver 8.0 Release Notes (November 10th, 2003)

These notes describe the new enhancements made to the SimpleTester since Version 7.0.

SimpleTester Tcl Package

The SimpleTester now contains a tcl package that allows you access to its syntax and semantic test suites as well as its SNMP commands within your own tcl interpretor.

This allows simpler integration of the extensive SNMP testing capabilities of the SimpleTester within your own test framework.
Command Recorder

With the Command Recorder feature, test sequences carried out using the graphical user interface get recorded into a command file. This command file can then be used to carry out the same tests in command line mode, without the GUI.

The command recorder automatically creates the corresponding command file making it easy to use the SimpleTester in an unattended, automated manner.
HTML Test Summary Reporting

The Test Summary file can now also be created in HTML format where the test results are displayed in an easy to read tabular manner.

Links are also provided to the actual test results so that you can easily drill down to get additional information.
Tips/Suggestions for fixing Errors.

Based on years of agent development experience, the syntax tests now include suggestions and tips regarding the possible cause of the errors detected.

These tips can help in quickly fixing the problems identified.
Eliminate Error Duplication.

Sometimes, the same error gets caught and reported in multiple test results. Support is now added to optionally, detect some of the error duplication cases, and report them only once.

SimpleTester Ver 7.0 Release Notes (November 25th, 2002)

These notes describe the new enhancements made to the SimpleTester since Version 6.5.

Test Suite Builder

This application eases the burden of creating and organizing a test suite structure in the SimpleTester to support your own semantic tests.

There are two methods provided to create a test suite. One is to create a directory structure based on a MIB. This the way the current semantic test suites are generally organized. The application will create the structure, based on the MIB, in a matter of seconds. The second method is to create the directory and scripts one at a time.

Organizing and maintaining a test suite becomes easier, also, with the ability to rename, create, and delete directories and files, and move scripts from one directory to the next.
Raw Trap Receiver

The Trap Receiver window now has the option to handle traps in a much more efficient manner. Selecting the "Save trap PDUs to file" option will save the trap as it comes over the network and then post process it to create a more readable output.
Miscellaneous Enhancements

The last line in a file can now be recognized properly when it does not contain the standard Windows end of line termination sequence of characters. A file like this can be created with the Windows notepad application if an Enter key is not pressed at the end of the last line.

The Status window now indicates when the SimpleTester is skipping MIB objects during MIB test 1 and MIB test 2.

The ST_pause procedure has been enhanced to support more variations. You can now change the icon to EXCLAMATION", "INFORMATION", "QUESTION", or "STOP." The buttons can be Abort/Retry/Ignore, OK, OK/Cancel, Retry/Cancel, Yes/No, or Yes/No/Cancel.
Fixes

Fixed the SNMP version reported for the response PDUs in the debug trace.

SimpleTester Ver 6.5 Release Notes (August 30th, 2002)

These notes describe the new enhancements made to the SimpleTester since Version 6.0.

Performance Test Suite

SimpleTester now includes a performance test suite that allows you to check the performance of the SNMP Agent. Two types of tests are supported: Sequential and Parallel.

In the Sequential tests, the number of specified requests are sent one after the other, after getting the response for the previous request. The SNMP operation, the number of variable bindings, and SNMP version can all be specified. The minimum, average and maximum times are reported.

In the Parallel tests, a number of simultaneous requests are sent and the response time measured. The test suite automatically increments the number of simultaneous requests sent from the specified starting value to the max. This gives you an idea of the receive queue lengths of the agent and its ability to talk with mutliple managers.
Enhancements/Fixes

Various enhancements/fixes required for DOCSIS related testing have been incorporated in this release from previous minor releases.

Support has been added for additional IP addresses in the command generator part of the application.

Support has also been added for additional IP addresses in the trap receiver.
Miscellaneous

Added check to ensure that the IP address used during the "Get variables to test" phase is the same one on which we are conducting the tests.

Added references to RFCs in the syntax tests for additional validation of the test procedures.

SimpleTester Ver 6.0 Release Notes (November 20th, 2001)

These notes describe the new enhancements made to the SimpleTester since Version 5.5.

Ease of Use

The Semantic Test Suite is now the first/main window for the SimpleTester. It has also been reimplemented to be resizable. All the panels are also resizable to give more flexibility as to what is shown.

The Setting window has been updated to include all the stester.ini options that are available. Now options such as IGNORELEXERRORS, PACKETTRACE, or SNMPPORT can be set via the Settings window. With this changes the separate windows to set SNMPv3 options have all been merged into the one Settings window and are accessible by merely clicking on a Tab.

Each test rating in the Test Suite window is now depicted with an icon in the test list. As each test is completed, the rating icon is displayed on the line corresponding to the test that was executed.

Addressing the agent is no longer limited to the IP address. You can now enter a host name and the SimpleTester will do the name to IP address translation.

The ability to edit/view a MIB file from the Compile MIB window has now been added.
Result Files

A new option to add the node name to the results file name has been added so that the test results of various devices are kept separate. For example, once test 4 is run, with this new option, the file name will have the node name or IP Address prepended. So result file used to be called "test4.res." With this new feature the file name might look like 192.168.40.22.test4.res. This naming convention has also been added to the summary file name.

A new option to concatinate all the result files with errors has been added. In the Test Suite when, any test that has a rating of "Error Level" or below is append to this file. So for example, if the Error Level is Good, then any tests with a rating of Good, Fair, Poor, or UNKNOWN is appended to this file.
Miscellaneous

A new capability in running semantic tests in batch mode has been added. Now the list is not just limited to script names. If you list a node in the test tree, all the tests that are subordinate to that node will be executed.

The capability in the additional info file to ignore MIB objects has now been included. These objects will not be tested by the syntax tests.

A new Tcl extension has been added to prompt the user for input. The command is ST_prompt and what this command will do is display a window with an edit box. Any text typed into the edit box is then returned to the script.

Support for SNMPv3 discovery when receiving Inform Requests has been added.

The Trap Receiver erroneously displayed the last trap defined in the MIBINFO.OUT file when receiving a trap with no corresponding trap definition. It now displays the numeric value for the trap.

Tests 45 and 47 now clear the Reportable flag when running in SNMPv3 mode.

SimpleTester Ver 5.5 Release Notes (June 18th, 2001)

These notes describe the new enhancements made to the SimpleTester since Version 5.0.

Ease of Use

The Semantic Test window has been enhanced to be an all-in-one test suite runner. The test tree now includes Semantic and Syntactic, as well as a branch for User defined test suites.

When the compiler must split a file containing multiple MIB modules, the temporary files now use the name of the original file as their base name. This makes finding an error in a multi-module file much easier.

Added code to ignore "mib" as an extension on the MIBEXTLIST option in the stester.ini file. This would cause files to show up twice in the Compile MIBs dialog.

Added snmpModule(3) under snmpv2 in the MIB Browser tree.

Increased support for SNMPv3 profile names from 10 to 50.

Increased input buffer size to handle longer OctetString values

The status window will now automatically scroll in the Script Runner dialog.
Packet Tracing
An option has been added to dump a packet trace in the test results file. This allows for looking at the ASN.1 encodings as they were sent over the wire. Adding the keyword PACKETTRACE=YES will cause the SimpleTester to include a hex dump of each and every PDU that is sent and received.
Max PDU Size Limits
The maximum PDU size is no longer hard coded to 1500 bytes. An option was added to specify the maximum size. Adding the keyword MAXPDULEN and the size in number of octets will allow the SimpleTester to acommondate longer messages: both sending and receiving.

Changed test 42 to accommodate larger PDU sizes.

Fixed Get Variables to Test to create lines longer than 1024 in the walk.out file.
Miscellaneous

Added support for the RECORDFORMAT directive in command file. This controls the output format to either be normal or comma separated.

Fixed the ST_setsession to properly recognize the -snmpversion parameter value to match the documentation.

Added fix to remove bad characters at beginning of the line in result file when TimeStamping is turned off.

Fixed problem with MIB Browser when a new node is placed below internet node.

Fixed problem with Script Runner when the summary files is not present.

Fixed problem when receiving a trap in the Trap Receiver.

Added support for "Write-Only" MIB Objects

Fixed problem when the Stop button is pressed while doing v3 Discovery in the Script Runner dialog.

Fixed issue with handling an Inform Request in the Script Runner.

Restored the Security Level when a Report PDU is received during Diffie-Hellman Key Ignition.

SimpleTester Ver 5.0 Release Notes (November 7th, 2000)

These notes describe the new enhancements made to the SimpleTester since Version 4.0.

Diffie-Hellman

The SimpleTester now adds support for Diffie-Hellman USM Key as described in RFC2786. This includes support for both Diffie-Hellman Key Change and Key Ignition.

Ability to create random numbers and corresponding public keys is also added.
SNMPv3

The SimpleTester now adds support for CBC-DES Symmetric Encryption Protocol (U.S. Version only) to support Auth/Priv, in addtion to NoAuth/NoPriv and Auth/noPriv.

The international version adds support for MD5 and SHA based Authentication (Auth/NoPriv).

All the SNMPv3 security parameters can be collectively saved in one named profile.
Semantic Tests

The SimpleTester now adds over 350 additional tests for USM, VACM, Co-existence MIBs as well as the newer systems and interfaces MIBs, to bring the total of semantic tests available to over 800.

As before, the source code of all the tcl scripts is available for perusal.
Ease of use

A software key based licensing is now optionally available.

Test results can now be written in a record format, with each result printed on one line, to ease the parsing of results files via scripts or programs.

The MIB Browser maps OID values to their corresponding object descriptors.

Various changes to ST_puts, ST_abort and ST_setsession Tcl commands to ease the update of summary files, telnet transactions, and setting of session parameters.

Support for additional arguments like IPAddress during command line invocation.

Bug fixes for Counter64 encodings, changing default editors, out of time window result pdus and closing of script files in Script Runner.

SimpleTester Ver 4.0 Release Notes (June 12, 2000)

These notes describe the new enhancements made to the SimpleTester since Version 3.5.

Semantic Tests

The SimpleTester now adds over 450 test scripts that validate the semantics of popular MIBs like MIB-II, mini-RMON, and SNMPv3. These tests further ensure that the agent has implemented the behavior of each MIB object properly. For example, in MIB-II if 2000 octets are sent to the agent, the value of the corresponding ifInOctets should atleast increment by 2000. Another example is that the value of ifNumber.0 should be the same as the number of rows in the ifTable. The tests can be easily customized by making changes to a configuration file. In addition the source code of all the 450+ tcl scripts is also available for perusal.

A new window has been added to manage the hundreds of scripts that have been included in this release. The framework is documented to allow users to add their own test suites. The semantic tests can also be carried out via the command file feature allowing for automated testing.
Tcl Commands

A new library of Tcl commands has been added to ease the burden of writing scripts. For example there is a new command which will retrieve all the rows in a table, or there is a set of commands to validate the status during row creation.

Telnet commands have been added to the Tcl interpreter, which allow a script to open a Telnet connection, send and receive data and finally close the connection. This will enable the testing of the device's Command Line Interface (CLI) in conjunction with its SNMP interface.

Serial interface support has also been added to the Tcl interpreter, which allow a script to open a local serial port, send and receive data and finally close the port. This also can be used to test a directly connected device's CLI in conjunction with its SNMP interface.
Ease of use

Two new configuration options control what files are displayed in the MIB Compile window. MIBDIRLIST configures additional directories the SimpleTester will look into and MIBEXTLIST configures additional file extensions that it will filter on.

The Browser and the Semantic Test window, the main window is automatically minimized to save screen real estate.

Double clicking a node in the MIB browser window starts a query.

SimpleTester Ver 3.5 Release Notes (Feb 2, 2000)

These notes describe the new enhancements made to the SimpleTester since Version 3.0.

SNMPv3

The SimpleTester now adds support for SNMPv3 Auth/noPriv. Both HMAC-MD5-96 and HMAC-SHA-96 are supported. (These capabilities are currently only available for US customers only.)

SNMPv3 Discovery is now only carried out when required for the selected tests.
MIB Browser

A search capability has been added to the MIB Browser to easily locate a MIB object in the object registration tree displayed.

Results of SNMP queries are displayed in a tabular style to provide an easy to read format.
Inform Request

Support for Inform Request was added to both the Trap Receiver and the Script Runner. A new command was added to the scripting syntax that waits for an inform request and sends a response once it is received.
Ease of use

A detailed status line was added to the main window. Trap names are now printed when the trap contents are displayed. Enumeration labels are also displayed in addition to the numbers for enumerated integers.

SimpleTester Ver 3.0 Release Notes (June 14, 1999)

These notes describe the new enhancements made to the SimpleTester since Version 2.5.

SNMPv3

The SimpleTester now adds support for SNMPv3. This version only supports NoAuth/NoPriv. Selecting this version can be done through the Configure-->Settings menu.
MIB Browser

Also included is a GUI based MIB browser. This can be invoked from the Tools menu, and gives the tester a graphical view of the MIB structure.
Compiler options

A new compile option was added to specify the strictness of the mib compiler.
Ease of use

A "Settings..." button has been added to the Command Line window and Get Variables To Test window to make it more convient to change settings without leaving the original window.

SimpleTester Ver 2.5 Release Notes (Jan 27, 1999)

These notes describe the new enhancements made to the SimpleTester since Version 2.0.

Tcl scripting support

The SimpleTester now includes the Tool Command Language (Tcl) interpreter version 8.0. This version of Tcl has been enhanced to support generating SNMP requests and handling SNMP responses. You can send GET, GETNEXT, GETBULK, and SET requests. Waiting for Traps is also supported. This capability allows for writing more complex and specialized tests that can be tailored for a specific agent and its behavior.
Errors during "Getting Variables To Test"

In the past the "Get Variables To Test" phase would be halted upon encountering lexicographical erorrs, or errors other than NO_SUCH_NAME from the agent.

New options have been added to the SimpleTester that allow the user to specify that these errors be ignored during the "Get Variables To Test" phase, but reported as errors in the tests. This allows the entire MIB to be tested in one shot, inspite of agent errors.
Stop Script on Error

A new option was added which causes the SimpleTester to stop the execution of a script when an error is first detected. This is particularly useful, when running load tests by recursively running a script, in a forever loop, in an unattended mode.
Bug fixes

The SimpleTester will accept both SNMPv1 and SNMPv2C traps and match them with their corresponding MIB defintions in either SNMPv1 or SNMPv2C SMI format.

Handles GetResponse PDU's with no variable bindings, from agents responding to GetRequests with one or more variable bindings.

Enhanced support for Bits and Counter64.