Repro Sanity Tests

From reSIProcate
Jump to navigation Jump to search

Repro Sanity Tests[edit]

In the tfm directory, you'll find a program named sanityTests.

This program uses the test framework to put the proxy through a series of tests.

Currently there are around 99 tests. We're seeing (typically) 2 failures and 12 to 13 errors:

 Test Results:
 Run:  99   Failures: 2   Errors: 12

There is a bug we're chasing in TFM that is leading to these errors (state from previous tests not being completely cleaned perhaps). When the erroring tests are run in isolation, they pass.


To run a single test, copy sanityTests to something like "smallTests", then go into the section of smallTests that starts:

class MyTestCase
{
   public:
      static CppUnit::Test* suite()
      {
         CppUnit::TestSuite *suiteOfTests = new CppUnit::TestSuite( "Suite1" );
//Registrar tests
        TEST(testRegisterBasic);
        TEST(testMultiple1);
        TEST(testMixedExpires);
        TEST(testThirdPartyRegistration);
        TEST(testDetailsQValue);


and comment out the tests you don't want to run. Using the evil #if 0 technique works well for this.

        TEST(testRegisterBasic);
 #if 0
        TEST(testMultiple1);
        TEST(testMixedExpires);
        TEST(testThirdPartyRegistration);
        TEST(testDetailsQValue);
        .
        .
        .
        TEST(testInviteAllBusyContacts);
 #endif
        return suiteOfTests;
     }


You'll see some of the tests being loaded into the suite using BADTEST and BUGTEST instead of TEST. From the code:

 // Tests that fail because there are problems in the test case implementation
 // should be labeled as BADTEST.  Tests that fail because they identify a
 // known bug in rePro should be labeled as BUGTEST.


By default, the BADTEST and BUGTEST tests will not be executed. To run them, recompile sanityTests with RUN_ALL_TESTS defined.