1998 International Workshop on Description Logics
(DL'98)

Trento, Italy
Saturday June 6 - Monday June 8, 1998

(coordinated event with KR'98 -- Sixth International Conference on
Principles of Knowledge Representation and Reasoning)

[DL]

DL'98 Systems Comparison: official test data

Availability

The complete test suite is contained in a single gziped tar archive at:

http://dl.kr.org/dl98/comparison/dl98-test.tar.gz

The Test Suite

Assuming that you have downloaded, unziped and untared the distribution file dl98-sys.tar.gz into the current directory, you will have the following files:

Data - directory containing test data files
LaTeX - directory containing example LaTeX submission
README - this file
Results - directory for results files
crack-tests.lsp - lisp interface to run tests with Crack system
dl98-tests.lsp - lisp core program to run tests
fact-tests.lsp - lisp interface to run tests with FaCT system
krss.ps - postscript file of KRSS specification document

All of the test data files use the KRSS syntax as described in (1). The ideal for the first two sets of tests was borrowed form the Comparison of Theorem Provers for Modal Logics at Tableaux'98 (2) and use test data developed by Alain Heuerding and Stefan Schwendimann (3).

The Test Data

There are 4 sets of tests:

Performing the Tests

For systems written in Lisp, it should be easy to modify one of fact-tests.lsp or crack-tests.lsp to interface to the new system and run all the tests automatically. All you need to do is modify the code to load your system and make the appropriate function/macro calls for various Tbox and Abox definitions and queries. The tests can then be performed simply by loading the modified file and calling the function "run-tests"; keyword parameters can be supplied to this function in order to run a subset of the tests.
If your system does not include all the constructs in some test, you may remove or ignore the constructs that your system does not include, as long as you indicate which constructs you are ignoring for each test.

Presenting the Results

The results should be submitted as a self-contained LaTeX document, and occupy no more than 4 12-point article style pages (excluding the bibliography). The submission should contain brief details of the system and the test hardware followed by tabulated results for those tests which it was possible to perform. A small number of citations can be included to indicate where further details about the system can be found. The directory LaTeX contains example LaTeX and postscript files.

Submission

Electronic submission of postscript files is preferred. These can be sent by email to either of the organisers (horrocks@cs.man.ac.uk or pfps@research.bell-labs.com). Please try to ensure that the postscript file includes any special fonts that are required and that only standard compression methods (if any) have been used (preferably gzip).

References


franconi@irst.itc.it
Last modified: Tue Apr 14 16:55:49 METDST 1998