Package de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator

Class EvaluatorCSV

java.lang.Object
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.Evaluator
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.EvaluatorCSV
public class EvaluatorCSV extends Evaluator
This evaluator is capable of persisting the results of the matching process in a CSV file (which can be consumed in Excel, for example). It divides mappings into certain groups, namely: classes, properties, instances, and allConfusionMatrix. If the alignment file are very large, it is better to use EvaluatorBasic.
Author:
Jan Portisch

  • Field Details

    • copyAlignmentFiles

      private boolean copyAlignmentFiles
      If true: system alignments will be copied to the evaluation directories. Default: true

    • isPrintAlignmentExtensions

      private boolean isPrintAlignmentExtensions
      If true: Alignment extensions are printed in CSV.

    • isPrintCorrespondenceExtensions

      private boolean isPrintCorrespondenceExtensions
      If true: Correspondence extensions are printed in CSV.

    • baselineMatcher

      private eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge baselineMatcher
      Baseline matcher for residual results.

    • LOGGER

      private static final org.slf4j.Logger LOGGER
      Default Logger

    • confusionMatrixMetric

      private ConfusionMatrixMetric confusionMatrixMetric

    • classRefiner

      private TypeRefiner classRefiner

    • propertyRefiner

      private TypeRefiner propertyRefiner

    • instanceRefiner

      private TypeRefiner instanceRefiner

    • residualRefiner

      private ResidualRefiner residualRefiner

    • isPrintAsShortenedString

      private boolean isPrintAsShortenedString
      Indicates whether the CSV shall be printed with shortened strings. Default: true.

    • alignmentsCube

      private AlignmentsCube alignmentsCube
      Analytical Store for all alignments.

    • testCasePerformanceCubePrinter

      private org.apache.commons.csv.CSVPrinter testCasePerformanceCubePrinter
      Printer which can be used to print an individual matcher performance for a test case (e.g. precision, recall, f1).

    • resourceExplainers

      private List<IExplainerResource> resourceExplainers
      The explainers to be used in the CSV that will be written.

    • trackPerformanceCubePrinter

      private org.apache.commons.csv.CSVPrinter trackPerformanceCubePrinter
      Printer which can be used to print an individual matcher performance for a track (e.g. macro-precision, micro-precision).

    • alignmentExtensions

      private ArrayList<String> alignmentExtensions
      The alignment extensions that are to be printed.

    • correspondenceExtensions

      private ArrayList<String> correspondenceExtensions
      The correspondence extensions that are to be printed.

    • TRACK_PERFORMANCE_CUBE_FILE_NAME

      private static final String TRACK_PERFORMANCE_CUBE_FILE_NAME
      See Also:

    • csvFormat

      private static org.apache.commons.csv.CSVFormat csvFormat

  • Constructor Details

    • EvaluatorCSV

      public EvaluatorCSV(ExecutionResultSet results, ConfusionMatrixMetric metric, boolean isPrintAsShortenedString)
      Constructor
      Parameters:
      results - The execution results for which an evaluation shall be performed.
      metric - The confusion matrix metric to be used.
      isPrintAsShortenedString - The CSV output will be written with shortened URIs.

    • EvaluatorCSV

      public EvaluatorCSV(ExecutionResultSet results, ConfusionMatrixMetric metric)
      Constructor
      Parameters:
      results - The execution results for which an evaluation shall be performed.
      metric - The confusion matrix metric to be used.

    • EvaluatorCSV

      public EvaluatorCSV(ExecutionResultSet results)
      Constructor
      Parameters:
      results - The execution results for which an evaluation shall be performed.

    • EvaluatorCSV

      public EvaluatorCSV(ExecutionResultSet results, boolean isPrintAsShortenedString)
      Constructor
      Parameters:
      results - The execution results for which an evaluation shall be performed.
      isPrintAsShortenedString - The CSV output will be written with shortened URIs.

  • Method Details

    • getCorrespondenceExtensions

      private ArrayList<String> getCorrespondenceExtensions(ExecutionResultSet results)
      This method determines the unique Correspondence extensions that are used in the alignments.
      Parameters:
      results - The result set.
      Returns:
      A set of unique correspondence extensions.

    • getAlignmentExtensions

      private ArrayList<String> getAlignmentExtensions(ExecutionResultSet results)
      This method determines the unique Alignment extensions that are used in the alignments in the ExecutionResultSet.
      Parameters:
      results - The result set.
      Returns:
      A list of unique alignment extensions that are used.

    • writeResultsToDirectory

      public void writeResultsToDirectory(File baseDirectory)
      Specified by:
      writeResultsToDirectory in class Evaluator

    • initializePrinters

      private void initializePrinters(File baseDirectory)
      This method initializes global writers for the performance KPI CSV files.
      Parameters:
      baseDirectory - The base directory to which the CSV files shall be written to.

    • closePrinters

      private void closePrinters()
      This method flushes and closes global printers.

    • writeAggregatedFileMatcherTrack

      private void writeAggregatedFileMatcherTrack(Track track, String matcher, File baseDirectory)
      Write the aggregated overview file, i.e. KPIs such as recall or precision, for a matcher on a particular track.
      Parameters:
      track - Track
      matcher - Matcher name
      baseDirectory - Base directory where file shall be written.

    • getFormattedRuntime

      public static String getFormattedRuntime(long nanoSeconds)

    • getSummedRuntimeOfResults

      public static long getSummedRuntimeOfResults(Set<ExecutionResult> results)

    • toStringArrayWithArrayAtTheEnd

      String[] toStringArrayWithArrayAtTheEnd(String[] putAtTheEnd, Object... individualValues)
      Creates one string array where the putAtTheEnd values are arranged at the end of the string.
      Parameters:
      putAtTheEnd - To be put at the end.
      individualValues - Some String values.
      Returns:
      One String array where first values from individualValues and then the values from putAtTheEnd are appearing.

    • writeOverviewFileMatcherTestCase

      private void writeOverviewFileMatcherTestCase(TestCase testCase, String matcher, File baseDirectory, boolean onlyCalculateCube)
      Write the overview file, i.e. KPIs such as recall or precision, for a matcher on a particular test case.
      Parameters:
      testCase - Test case
      matcher - Matcher name
      baseDirectory - Base directory where file shall be written.
      onlyCalculateCube - Indicator whether only the alignmentsCube shall be calculated.

    • determineAlignmentExtensionValuesToWriteForCSV

      private String[] determineAlignmentExtensionValuesToWriteForCSV(Map<String,Object> existingExtensionValues)
      Given the existing extension values of an alignment, determine what to write in the CSV file.
      Parameters:
      existingExtensionValues - The existing extension values in the alignment.
      Returns:
      Tokenized extension values in the correct order for the CSV file to print.

    • determineAggregatedAlignmentExtensionValuesToWriteForCSV

      private String[] determineAggregatedAlignmentExtensionValuesToWriteForCSV(Set<ExecutionResult> results)

    • getHeaderIndividual

      private List<String> getHeaderIndividual()
      Get the header row for the individual statistics.
      Returns:
      Header row in the form of a String-List.

    • getHeaderTrackPerformanceCube

      private List<String> getHeaderTrackPerformanceCube()
      Get the header row for the individual statistics in the overall CSV file.
      Returns:
      CSV header row in the form of a String-List.

    • getHeaderAggregated

      private List<String> getHeaderAggregated()
      Get the header row for the aggregated results on a per matcher basis.
      Returns:
      Header row in the form of a String-List.

    • getHeaderTestCasePerformanceCube

      private List<String> getHeaderTestCasePerformanceCube()
      Get the header row for the individual statistics in the overall CSV file.
      Returns:
      CSV header row in the form of a String-List.

    • getAlignmentsCubeAsString

      public String getAlignmentsCubeAsString()
      Obtain an output stream that can be used to write the CSV file.
      Returns:
      CSV String representation of the alignment cube.

    • getAlignmentsCubeAsShortenedString

      private String getAlignmentsCubeAsShortenedString()
      Obtain an output stream that can be used to write the CSV file.
      Returns:
      CSV String representation of the alignment cube.

    • isCopyAlignmentFiles

      public boolean isCopyAlignmentFiles()

    • setCopyAlignmentFiles

      public void setCopyAlignmentFiles(boolean copyAlignmentFiles)

    • getBaselineMatcher

      public eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge getBaselineMatcher()

    • setBaselineMatcher

      public void setBaselineMatcher(eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge baselineMatcher)

    • getConfusionMatrixMetric

      public ConfusionMatrixMetric getConfusionMatrixMetric()

    • setConfusionMatrixMetric

      public void setConfusionMatrixMetric(ConfusionMatrixMetric confusionMatrixMetric)

    • getResourceExplainers

      public List<IExplainerResource> getResourceExplainers()

    • setResourceExplainers

      public void setResourceExplainers(List<IExplainerResource> resourceExplainers)

    • isPrintAsShortenedString

      public boolean isPrintAsShortenedString()

    • setPrintAsShortenedString

      public void setPrintAsShortenedString(boolean printAsShortenedString)

    • isPrintCorrespondenceExtensions

      public boolean isPrintCorrespondenceExtensions()

    • setPrintCorrespondenceExtensions

      public void setPrintCorrespondenceExtensions(boolean printCorrespondenceExtensions)

    • isPrintAlignmentExtensions

      public boolean isPrintAlignmentExtensions()

    • setPrintAlignmentExtensions

      public void setPrintAlignmentExtensions(boolean printAlignmentExtensions)

    • getTrackPerformanceCubeFileName

      public static String getTrackPerformanceCubeFileName()

    • getCsvFormat

      public static org.apache.commons.csv.CSVFormat getCsvFormat()
      Returns the CSV format used to write all CSV files.
      Returns:
      CSVFormat

    • setCsvFormat

      public static void setCsvFormat(org.apache.commons.csv.CSVFormat csvFormat)