1998 International Workshop on
Description Logics
(DL'98) Trento, Italy
Saturday June 6 - Monday June
8, 1998 (coordinated event with KR'98 -- Sixth International
Conference on
Principles of Knowledge Representation and
Reasoning) |  | ![[DL]](../dl-small.gif) |
Call for DL Systems Comparison
DL'98 will host a DL systems
comparison session. Systems will be compared on the basis of
benchmarks which will be prepared in order to emphasize the
best out of each system. This first round of the comparison at
DL'98 will serve mostly as an
evaluation of the benchmark testing set, and it is not intended to be
the last word on a possible ranking.
Aims
We have four aims:
- To reach a consensus on a corpus of benchmarks for DL system,
which may become an agreed standard.
- Bringing people together with a common interest in implementation
work on various DL.
- Giving the implementors of reasoners for the considered logics
the opportunity to demonstrate their systems and to make them better
known. (There also exists the possibility to submit system
descriptions to DL'98).
- There are many reasoning methods for DL. We propose a comparison,
with no winners or prizes, of these approaches.
Criteria
A fair comparison is very difficult, since different computers are
used, systems are implemented in different programming languages,
and expressivity of the different logics varies considerably. We hope to
overcome these problem to a certain extent by using different families
of parametrized knowledge bases. The time to decide the provability of
these KBs is expected to be exponential in the parameter for most
reasoners. The participants will determine the largest parameter value that
their system can handle within a given time limit. Consequently, the
results can only be slightly improved by using a faster computer or a
better programming language.
Moreover, we believe that expressivity and efficiency are not the only
qualities of a reasoner. Even if reasoner B is `really' considerably
less expressive or slower than reasoner A, one may prefer B to A,
e.g. if:
- B is very comfortable to use.
- B is very small and simple, and thus more secure.
- B is written is a standard language and easily portable.
- B can deal with many logics.
- The correctness and completeness of B have been verified.
- B requires little space for the computation.
- B is tailored for specific applications.
- ...
Such advantages can be stated in entries. For example,
the number of lines of the source code of of a system could be mentioned
(if the small size is an advantage for that system), or so could the storage
required to process certain KBs (if the system requires little space).
Who can take part?
Everybody who has written a reasoner for DL which accepts a
reasonable subset of the language specified by the KRSS
document can submit an entry.
All the knowledge bases interchanges will be done using
the neutral syntax specified by the KRSS document. We prefer reasoners
that are easily available to everybody or can be used by some means
(e.g., over the WWW).
Everybody who takes part automatically accepts the procedure of the
comparison.
Procedure
The benchmark KBs will be available via the comparison home page
at http://dl.kr.org/dl/dl98/comparison/.
People willing to participate must confirm their participation to
Peter Patel-Schneider at pfps@research.bell-labs.com
or to Ian Horrocks at horrocks@cs.man.ac.uk
by Friday March 6, 1998.
We encourage participants to indicate their interest as soon as possible
and to participate in the selection of benchmarks.
Participants may additionally submit a set of benchmarks which
possibly show where their systems excel. These benchmarks should be
written in the neutral KRSS syntax, and should be representable in at
least one other available DL system.
If these benchmarks are submitted early enough, they may be used as the
basis of a parameterized set.
Important:
The participants time their systems at their site before the workshop.
The result should contain the following information:
- Name of the system, if possible a WWW address
or a reference to the literature.
- Some information about the algorithms.
- Advantages of your system besides efficiency
- The programming language used to implement the system.
- The hardware used for the timings.
- The results of the timings,
plus a note if you deviated from the limits in the specified procedure.
- Optional comments on the various results.
The result should be submitted as a self-contained postscript document
to Peter Patel-Schneider at pfps@research.bell-labs.com
or to Ian Horrocks at horrocks@cs.man.ac.uk
by Friday April 17, 1998.
Publication/presentation of results
The organisers will compile short summaries of these results. If the
organisers want to check the results of some participants, then the
participants are responsible for providing sufficient support. They must
give the organisers the possibility to check the results at the
participant's site.
The summaries will appear in the proceedings of the DL'98 workshop.
At the workshop, the participants will have the possibility to demonstrate
their systems either on site or over the internet to another site. Posters
describing the systems may be displayed.
Everybody is welcome to submit for evaluation by the PC of DL'98 either full papers
describing any new approaches or system descriptions describing new
implementations (or both).
Timetable
|
Preliminary disclosure of the basic benchmarks:
|
February 16, 1998
|
|
Identification of the DL systems willing to partecipate:
|
March 6, 1998
|
|
Optional submission of partecipants' benchmarks:
|
March 6, 1998
|
|
Disclosure of the official comparison benchmarks:
|
March 23, 1998
|
|
Deadline for the submission of the results:
|
April 17, 1998
|
|
Summaries of the results available:
|
May 18, 1998
|
Meeting at DL'98:
- Presentation of the results
- Demonstration of the reasoners
- Comments and evaluation of the benchmarks
- Agreement on a corpus of benchmarks
|
June 6 - 8, 1998
|
Committee:
RESOURCES
Enquiries about the DL'98 workshop can be made by mailing to dl98@dl.kr.org.
The WWW page of the DL'98 workshop is at http://dl.kr.org/dl/dl98/;
check it for updates.
The official Description Logics WWW home page is at http://dl.kr.org/dl/.
Advertising (click for more info):
|
This page maintained by Enrico Franconi <franconi@irst.itc.it>.
Comments and suggestions are welcome.
Last modified: Wed Apr 15 12:06:37 METDST 1998