Alternatively, you can use the MELT Evaluation Client which is discussed in this section. The client is a simple java command line tool that is executed in a terminal and does not require any coding skills or deep understanding of MELT, SEALS, or the Docker Web format. The client requires as input IDs of the available built-in tracks. Alternatively, local tracks/ontologies can alo be given as input.
You can download the latest version of the command line tool on GitHub.
The most straight forward way to use the tool is as follows:
java -jar matching-eval-client-latest.jar --systems <system-1-path> <system-2-path> <...> --track <location-URI> <collection-name> <version>
The results will be printed on the command line.
java -jar matching-eval-client-latest.jar --help
Types of Systems
The evaluation client will automatically detect the following types of matchers:
- Web package (docker file) - automatically detected by path ending with
- SEALS package - automatically detected by path ending with
- Web package (URL) - in all other cases an URL is assumed
Evaluating SEALS packages requires Java 8. If you do not use Java 8 as your default java runtime, you can specify the Java 8 location using parameter
--java8 <path to Java 8>.
If you want to access the system alignments and/or have a more detailed evaluation report, you can specify an output directory via
--results. The reports are generated by the
EvaluatorCSV and you can find an explanation of the generated file structure in the corresponding section of the user guide.
If you have a local test case (i.e. two ontologies and a reference alignment) on disk that you want to evaluate, you can use option
--local-testcase <onto1-path> <onto2-path> <reference-path> instead of the
If you have multiple local test cases that you want to evaluate simultaneously (micro/macro precision etc.), you can create a local track. Create a folder structure as described in the local track section of the user guide and use the
--local-track <folder> <name> <version> option where
version can be freely set.