Class EvaluatorCSV
java.lang.Object
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.Evaluator
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.EvaluatorCSV
This evaluator is capable of persisting the results of the matching process in a CSV file (which can be consumed
in Excel, for example).
It divides mappings into certain groups, namely: classes, properties, instances, and allConfusionMatrix.
If the alignment file are very large, it is better to use EvaluatorBasic.
- Author:
- Jan Portisch
-
Field Summary
Modifier and TypeFieldDescriptionThe alignment extensions that are to be printed.private AlignmentsCube
Analytical Store for all alignments.private eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge
Baseline matcher for residual results.private TypeRefiner
private ConfusionMatrixMetric
private boolean
If true: system alignments will be copied to the evaluation directories.The correspondence extensions that are to be printed.private static org.apache.commons.csv.CSVFormat
private TypeRefiner
private boolean
If true: Alignment extensions are printed in CSV.private boolean
Indicates whether the CSV shall be printed with shortened strings.private boolean
If true: Correspondence extensions are printed in CSV.private static final org.slf4j.Logger
Default Loggerprivate TypeRefiner
private ResidualRefiner
private List<IExplainerResource>
The explainers to be used in the CSV that will be written.private org.apache.commons.csv.CSVPrinter
Printer which can be used to print an individual matcher performance for a test case (e.g.private static final String
private org.apache.commons.csv.CSVPrinter
Printer which can be used to print an individual matcher performance for a track (e.g. -
Constructor Summary
ConstructorDescriptionEvaluatorCSV
(ExecutionResultSet results) ConstructorEvaluatorCSV
(ExecutionResultSet results, boolean isPrintAsShortenedString) ConstructorEvaluatorCSV
(ExecutionResultSet results, ConfusionMatrixMetric metric) ConstructorEvaluatorCSV
(ExecutionResultSet results, ConfusionMatrixMetric metric, boolean isPrintAsShortenedString) Constructor -
Method Summary
Modifier and TypeMethodDescriptionprivate void
This method flushes and closes global printers.private String[]
private String[]
determineAlignmentExtensionValuesToWriteForCSV
(Map<String, Object> existingExtensionValues) Given the existing extension values of an alignment, determine what to write in the CSV file.getAlignmentExtensions
(ExecutionResultSet results) This method determines the uniqueAlignment
extensions that are used in the alignments in the ExecutionResultSet.private String
Obtain an output stream that can be used to write the CSV file.Obtain an output stream that can be used to write the CSV file.eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge
This method determines the uniqueCorrespondence
extensions that are used in the alignments.static org.apache.commons.csv.CSVFormat
Returns the CSV format used to write all CSV files.static String
getFormattedRuntime
(long nanoSeconds) Get the header row for the aggregated results on a per matcher basis.Get the header row for the individual statistics.Get the header row for the individual statistics in the overall CSV file.Get the header row for the individual statistics in the overall CSV file.static long
getSummedRuntimeOfResults
(Set<ExecutionResult> results) static String
private void
initializePrinters
(File baseDirectory) This method initializes global writers for the performance KPI CSV files.boolean
boolean
boolean
boolean
void
setBaselineMatcher
(eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge baselineMatcher) void
setConfusionMatrixMetric
(ConfusionMatrixMetric confusionMatrixMetric) void
setCopyAlignmentFiles
(boolean copyAlignmentFiles) static void
setCsvFormat
(org.apache.commons.csv.CSVFormat csvFormat) void
setPrintAlignmentExtensions
(boolean printAlignmentExtensions) void
setPrintAsShortenedString
(boolean printAsShortenedString) void
setPrintCorrespondenceExtensions
(boolean printCorrespondenceExtensions) void
setResourceExplainers
(List<IExplainerResource> resourceExplainers) (package private) String[]
toStringArrayWithArrayAtTheEnd
(String[] putAtTheEnd, Object... individualValues) Creates one string array where theputAtTheEnd
values are arranged at the end of the string.private void
writeAggregatedFileMatcherTrack
(Track track, String matcher, File baseDirectory) Write the aggregated overview file, i.e.private void
writeOverviewFileMatcherTestCase
(TestCase testCase, String matcher, File baseDirectory, boolean onlyCalculateCube) Write the overview file, i.e.void
writeResultsToDirectory
(File baseDirectory) Methods inherited from class de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.Evaluator
checkAndCreateDirectory, getDefaultResultsDirectory, getDirectoryWithCurrentTime, getResults, getResultsDirectoryTrack, getResultsDirectoryTrackMatcher, getResultsDirectoryTrackTestcaseMatcher, setDefaultResultsDirectory, writeToDirectory, writeToDirectory, writeToDirectory
-
Field Details
-
copyAlignmentFiles
private boolean copyAlignmentFilesIf true: system alignments will be copied to the evaluation directories. Default: true -
isPrintAlignmentExtensions
private boolean isPrintAlignmentExtensionsIf true: Alignment extensions are printed in CSV. -
isPrintCorrespondenceExtensions
private boolean isPrintCorrespondenceExtensionsIf true: Correspondence extensions are printed in CSV. -
baselineMatcher
private eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge baselineMatcherBaseline matcher for residual results. -
LOGGER
private static final org.slf4j.Logger LOGGERDefault Logger -
confusionMatrixMetric
-
classRefiner
-
propertyRefiner
-
instanceRefiner
-
residualRefiner
-
isPrintAsShortenedString
private boolean isPrintAsShortenedStringIndicates whether the CSV shall be printed with shortened strings. Default: true. -
alignmentsCube
Analytical Store for all alignments. -
testCasePerformanceCubePrinter
private org.apache.commons.csv.CSVPrinter testCasePerformanceCubePrinterPrinter which can be used to print an individual matcher performance for a test case (e.g. precision, recall, f1). -
resourceExplainers
The explainers to be used in the CSV that will be written. -
trackPerformanceCubePrinter
private org.apache.commons.csv.CSVPrinter trackPerformanceCubePrinterPrinter which can be used to print an individual matcher performance for a track (e.g. macro-precision, micro-precision). -
alignmentExtensions
The alignment extensions that are to be printed. -
correspondenceExtensions
The correspondence extensions that are to be printed. -
TRACK_PERFORMANCE_CUBE_FILE_NAME
- See Also:
-
csvFormat
private static org.apache.commons.csv.CSVFormat csvFormat
-
-
Constructor Details
-
EvaluatorCSV
public EvaluatorCSV(ExecutionResultSet results, ConfusionMatrixMetric metric, boolean isPrintAsShortenedString) Constructor- Parameters:
results
- The execution results for which an evaluation shall be performed.metric
- The confusion matrix metric to be used.isPrintAsShortenedString
- The CSV output will be written with shortened URIs.
-
EvaluatorCSV
Constructor- Parameters:
results
- The execution results for which an evaluation shall be performed.metric
- The confusion matrix metric to be used.
-
EvaluatorCSV
Constructor- Parameters:
results
- The execution results for which an evaluation shall be performed.
-
EvaluatorCSV
Constructor- Parameters:
results
- The execution results for which an evaluation shall be performed.isPrintAsShortenedString
- The CSV output will be written with shortened URIs.
-
-
Method Details
-
getCorrespondenceExtensions
This method determines the uniqueCorrespondence
extensions that are used in the alignments.- Parameters:
results
- The result set.- Returns:
- A set of unique correspondence extensions.
-
getAlignmentExtensions
This method determines the uniqueAlignment
extensions that are used in the alignments in the ExecutionResultSet.- Parameters:
results
- The result set.- Returns:
- A list of unique alignment extensions that are used.
-
writeResultsToDirectory
- Specified by:
writeResultsToDirectory
in classEvaluator
-
initializePrinters
This method initializes global writers for the performance KPI CSV files.- Parameters:
baseDirectory
- The base directory to which the CSV files shall be written to.
-
closePrinters
private void closePrinters()This method flushes and closes global printers. -
writeAggregatedFileMatcherTrack
Write the aggregated overview file, i.e. KPIs such as recall or precision, for a matcher on a particular track.- Parameters:
track
- Trackmatcher
- Matcher namebaseDirectory
- Base directory where file shall be written.
-
getFormattedRuntime
-
getSummedRuntimeOfResults
-
toStringArrayWithArrayAtTheEnd
Creates one string array where theputAtTheEnd
values are arranged at the end of the string.- Parameters:
putAtTheEnd
- To be put at the end.individualValues
- Some String values.- Returns:
- One String array where first values from
individualValues
and then the values fromputAtTheEnd
are appearing.
-
writeOverviewFileMatcherTestCase
private void writeOverviewFileMatcherTestCase(TestCase testCase, String matcher, File baseDirectory, boolean onlyCalculateCube) Write the overview file, i.e. KPIs such as recall or precision, for a matcher on a particular test case.- Parameters:
testCase
- Test casematcher
- Matcher namebaseDirectory
- Base directory where file shall be written.onlyCalculateCube
- Indicator whether only thealignmentsCube
shall be calculated.
-
determineAlignmentExtensionValuesToWriteForCSV
private String[] determineAlignmentExtensionValuesToWriteForCSV(Map<String, Object> existingExtensionValues) Given the existing extension values of an alignment, determine what to write in the CSV file.- Parameters:
existingExtensionValues
- The existing extension values in the alignment.- Returns:
- Tokenized extension values in the correct order for the CSV file to print.
-
determineAggregatedAlignmentExtensionValuesToWriteForCSV
private String[] determineAggregatedAlignmentExtensionValuesToWriteForCSV(Set<ExecutionResult> results) -
getHeaderIndividual
Get the header row for the individual statistics.- Returns:
- Header row in the form of a String-List.
-
getHeaderTrackPerformanceCube
Get the header row for the individual statistics in the overall CSV file.- Returns:
- CSV header row in the form of a String-List.
-
getHeaderAggregated
Get the header row for the aggregated results on a per matcher basis.- Returns:
- Header row in the form of a String-List.
-
getHeaderTestCasePerformanceCube
Get the header row for the individual statistics in the overall CSV file.- Returns:
- CSV header row in the form of a String-List.
-
getAlignmentsCubeAsString
Obtain an output stream that can be used to write the CSV file.- Returns:
- CSV String representation of the alignment cube.
-
getAlignmentsCubeAsShortenedString
Obtain an output stream that can be used to write the CSV file.- Returns:
- CSV String representation of the alignment cube.
-
isCopyAlignmentFiles
public boolean isCopyAlignmentFiles() -
setCopyAlignmentFiles
public void setCopyAlignmentFiles(boolean copyAlignmentFiles) -
getBaselineMatcher
public eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge getBaselineMatcher() -
setBaselineMatcher
public void setBaselineMatcher(eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge baselineMatcher) -
getConfusionMatrixMetric
-
setConfusionMatrixMetric
-
getResourceExplainers
-
setResourceExplainers
-
isPrintAsShortenedString
public boolean isPrintAsShortenedString() -
setPrintAsShortenedString
public void setPrintAsShortenedString(boolean printAsShortenedString) -
isPrintCorrespondenceExtensions
public boolean isPrintCorrespondenceExtensions() -
setPrintCorrespondenceExtensions
public void setPrintCorrespondenceExtensions(boolean printCorrespondenceExtensions) -
isPrintAlignmentExtensions
public boolean isPrintAlignmentExtensions() -
setPrintAlignmentExtensions
public void setPrintAlignmentExtensions(boolean printAlignmentExtensions) -
getTrackPerformanceCubeFileName
-
getCsvFormat
public static org.apache.commons.csv.CSVFormat getCsvFormat()Returns the CSV format used to write all CSV files.- Returns:
- CSVFormat
-
setCsvFormat
public static void setCsvFormat(org.apache.commons.csv.CSVFormat csvFormat)
-