java.lang.Object
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.Evaluator
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.EvaluatorCSV

public class EvaluatorCSV extends Evaluator
This evaluator is capable of persisting the results of the matching process in a CSV file (which can be consumed in Excel, for example). It divides mappings into certain groups, namely: classes, properties, instances, and allConfusionMatrix. If the alignment file are very large, it is better to use EvaluatorBasic.
Author:
Jan Portisch
  • Field Details

    • copyAlignmentFiles

      private boolean copyAlignmentFiles
      If true: system alignments will be copied to the evaluation directories. Default: true
    • isPrintAlignmentExtensions

      private boolean isPrintAlignmentExtensions
      If true: Alignment extensions are printed in CSV.
    • isPrintCorrespondenceExtensions

      private boolean isPrintCorrespondenceExtensions
      If true: Correspondence extensions are printed in CSV.
    • baselineMatcher

      private eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge baselineMatcher
      Baseline matcher for residual results.
    • LOGGER

      private static final org.slf4j.Logger LOGGER
      Default Logger
    • confusionMatrixMetric

      private ConfusionMatrixMetric confusionMatrixMetric
    • classRefiner

      private TypeRefiner classRefiner
    • propertyRefiner

      private TypeRefiner propertyRefiner
    • instanceRefiner

      private TypeRefiner instanceRefiner
    • residualRefiner

      private ResidualRefiner residualRefiner
    • isPrintAsShortenedString

      private boolean isPrintAsShortenedString
      Indicates whether the CSV shall be printed with shortened strings. Default: true.
    • alignmentsCube

      private AlignmentsCube alignmentsCube
      Analytical Store for all alignments.
    • testCasePerformanceCubePrinter

      private org.apache.commons.csv.CSVPrinter testCasePerformanceCubePrinter
      Printer which can be used to print an individual matcher performance for a test case (e.g. precision, recall, f1).
    • resourceExplainers

      private List<IExplainerResource> resourceExplainers
      The explainers to be used in the CSV that will be written.
    • trackPerformanceCubePrinter

      private org.apache.commons.csv.CSVPrinter trackPerformanceCubePrinter
      Printer which can be used to print an individual matcher performance for a track (e.g. macro-precision, micro-precision).
    • alignmentExtensions

      private ArrayList<String> alignmentExtensions
      The alignment extensions that are to be printed.
    • correspondenceExtensions

      private ArrayList<String> correspondenceExtensions
      The correspondence extensions that are to be printed.
    • TRACK_PERFORMANCE_CUBE_FILE_NAME

      private static final String TRACK_PERFORMANCE_CUBE_FILE_NAME
      See Also:
    • csvFormat

      private static org.apache.commons.csv.CSVFormat csvFormat
  • Constructor Details

    • EvaluatorCSV

      public EvaluatorCSV(ExecutionResultSet results, ConfusionMatrixMetric metric, boolean isPrintAsShortenedString)
      Constructor
      Parameters:
      results - The execution results for which an evaluation shall be performed.
      metric - The confusion matrix metric to be used.
      isPrintAsShortenedString - The CSV output will be written with shortened URIs.
    • EvaluatorCSV

      public EvaluatorCSV(ExecutionResultSet results, ConfusionMatrixMetric metric)
      Constructor
      Parameters:
      results - The execution results for which an evaluation shall be performed.
      metric - The confusion matrix metric to be used.
    • EvaluatorCSV

      public EvaluatorCSV(ExecutionResultSet results)
      Constructor
      Parameters:
      results - The execution results for which an evaluation shall be performed.
    • EvaluatorCSV

      public EvaluatorCSV(ExecutionResultSet results, boolean isPrintAsShortenedString)
      Constructor
      Parameters:
      results - The execution results for which an evaluation shall be performed.
      isPrintAsShortenedString - The CSV output will be written with shortened URIs.
  • Method Details

    • getCorrespondenceExtensions

      private ArrayList<String> getCorrespondenceExtensions(ExecutionResultSet results)
      This method determines the unique Correspondence extensions that are used in the alignments.
      Parameters:
      results - The result set.
      Returns:
      A set of unique correspondence extensions.
    • getAlignmentExtensions

      private ArrayList<String> getAlignmentExtensions(ExecutionResultSet results)
      This method determines the unique Alignment extensions that are used in the alignments in the ExecutionResultSet.
      Parameters:
      results - The result set.
      Returns:
      A list of unique alignment extensions that are used.
    • writeResultsToDirectory

      public void writeResultsToDirectory(File baseDirectory)
      Specified by:
      writeResultsToDirectory in class Evaluator
    • initializePrinters

      private void initializePrinters(File baseDirectory)
      This method initializes global writers for the performance KPI CSV files.
      Parameters:
      baseDirectory - The base directory to which the CSV files shall be written to.
    • closePrinters

      private void closePrinters()
      This method flushes and closes global printers.
    • writeAggregatedFileMatcherTrack

      private void writeAggregatedFileMatcherTrack(Track track, String matcher, File baseDirectory)
      Write the aggregated overview file, i.e. KPIs such as recall or precision, for a matcher on a particular track.
      Parameters:
      track - Track
      matcher - Matcher name
      baseDirectory - Base directory where file shall be written.
    • getFormattedRuntime

      public static String getFormattedRuntime(long nanoSeconds)
    • getSummedRuntimeOfResults

      public static long getSummedRuntimeOfResults(Set<ExecutionResult> results)
    • toStringArrayWithArrayAtTheEnd

      String[] toStringArrayWithArrayAtTheEnd(String[] putAtTheEnd, Object... individualValues)
      Creates one string array where the putAtTheEnd values are arranged at the end of the string.
      Parameters:
      putAtTheEnd - To be put at the end.
      individualValues - Some String values.
      Returns:
      One String array where first values from individualValues and then the values from putAtTheEnd are appearing.
    • writeOverviewFileMatcherTestCase

      private void writeOverviewFileMatcherTestCase(TestCase testCase, String matcher, File baseDirectory, boolean onlyCalculateCube)
      Write the overview file, i.e. KPIs such as recall or precision, for a matcher on a particular test case.
      Parameters:
      testCase - Test case
      matcher - Matcher name
      baseDirectory - Base directory where file shall be written.
      onlyCalculateCube - Indicator whether only the alignmentsCube shall be calculated.
    • determineAlignmentExtensionValuesToWriteForCSV

      private String[] determineAlignmentExtensionValuesToWriteForCSV(Map<String,Object> existingExtensionValues)
      Given the existing extension values of an alignment, determine what to write in the CSV file.
      Parameters:
      existingExtensionValues - The existing extension values in the alignment.
      Returns:
      Tokenized extension values in the correct order for the CSV file to print.
    • determineAggregatedAlignmentExtensionValuesToWriteForCSV

      private String[] determineAggregatedAlignmentExtensionValuesToWriteForCSV(Set<ExecutionResult> results)
    • getHeaderIndividual

      private List<String> getHeaderIndividual()
      Get the header row for the individual statistics.
      Returns:
      Header row in the form of a String-List.
    • getHeaderTrackPerformanceCube

      private List<String> getHeaderTrackPerformanceCube()
      Get the header row for the individual statistics in the overall CSV file.
      Returns:
      CSV header row in the form of a String-List.
    • getHeaderAggregated

      private List<String> getHeaderAggregated()
      Get the header row for the aggregated results on a per matcher basis.
      Returns:
      Header row in the form of a String-List.
    • getHeaderTestCasePerformanceCube

      private List<String> getHeaderTestCasePerformanceCube()
      Get the header row for the individual statistics in the overall CSV file.
      Returns:
      CSV header row in the form of a String-List.
    • getAlignmentsCubeAsString

      public String getAlignmentsCubeAsString()
      Obtain an output stream that can be used to write the CSV file.
      Returns:
      CSV String representation of the alignment cube.
    • getAlignmentsCubeAsShortenedString

      private String getAlignmentsCubeAsShortenedString()
      Obtain an output stream that can be used to write the CSV file.
      Returns:
      CSV String representation of the alignment cube.
    • isCopyAlignmentFiles

      public boolean isCopyAlignmentFiles()
    • setCopyAlignmentFiles

      public void setCopyAlignmentFiles(boolean copyAlignmentFiles)
    • getBaselineMatcher

      public eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge getBaselineMatcher()
    • setBaselineMatcher

      public void setBaselineMatcher(eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge baselineMatcher)
    • getConfusionMatrixMetric

      public ConfusionMatrixMetric getConfusionMatrixMetric()
    • setConfusionMatrixMetric

      public void setConfusionMatrixMetric(ConfusionMatrixMetric confusionMatrixMetric)
    • getResourceExplainers

      public List<IExplainerResource> getResourceExplainers()
    • setResourceExplainers

      public void setResourceExplainers(List<IExplainerResource> resourceExplainers)
    • isPrintAsShortenedString

      public boolean isPrintAsShortenedString()
    • setPrintAsShortenedString

      public void setPrintAsShortenedString(boolean printAsShortenedString)
    • isPrintCorrespondenceExtensions

      public boolean isPrintCorrespondenceExtensions()
    • setPrintCorrespondenceExtensions

      public void setPrintCorrespondenceExtensions(boolean printCorrespondenceExtensions)
    • isPrintAlignmentExtensions

      public boolean isPrintAlignmentExtensions()
    • setPrintAlignmentExtensions

      public void setPrintAlignmentExtensions(boolean printAlignmentExtensions)
    • getTrackPerformanceCubeFileName

      public static String getTrackPerformanceCubeFileName()
    • getCsvFormat

      public static org.apache.commons.csv.CSVFormat getCsvFormat()
      Returns the CSV format used to write all CSV files.
      Returns:
      CSVFormat
    • setCsvFormat

      public static void setCsvFormat(org.apache.commons.csv.CSVFormat csvFormat)