Commit Checklist

First and easiest is unit tests:

$ ant clean test

All tests should pass, with the exception of these two:

[junit] Test edu.cmu.ml.praprolog.MultithreadedTrainerTest FAILED
[junit] Test edu.cmu.ml.praprolog.learn.SRWRestartTest FAILED

...which are mildly controversial but don't indicate any critical bugs.

Second is a sanity-check of typical running conditions using some prepackaged datasets:

$ cd examples/
$ make clean
...
$ make
...
$ ./review.sh 
./textcattoy
pre.train pairs 11.0 errors 11.0 errorRate 1.0 map 0.8181818181818182
pre.test pairs 7.0 errors 7.0 errorRate 1.0 map 0.7142857142857143
post.train pairs 11.0 errors 0.0 errorRate 0.0 map 1.0
post.test pairs 7.0 errors 0.0 errorRate 0.0 map 1.0
...

You may need to ./configure --proppr=absolute/path/to/proppr if you've never run anything in the examples set before.

In general, the pre.* map scores should be less than the post.* scores. For top-1000-near-google, the untrained system already gets perfect performance on the test set, to the pre- and post- training scores are identical.