Package de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.metric.cm

Class ConfusionMatrixMetric

java.lang.Object
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.metric.Metric<ConfusionMatrix>
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.metric.cm.ConfusionMatrixMetric
public class ConfusionMatrixMetric extends Metric<ConfusionMatrix>
Confusion Matrix Metric. Can handle full and partial gold standards as well as explict null mappings (there is no match for one entity).
Author:
Sven Hertling, Jan Portisch
See Also:

  • Field Details

    • LOGGER

      private org.slf4j.Logger LOGGER
      Default Logger

  • Constructor Details

    • ConfusionMatrixMetric

      public ConfusionMatrixMetric()

  • Method Details

    • compute

      public ConfusionMatrix compute(ExecutionResult executionResult)
      Specified by:
      compute in class Metric<ConfusionMatrix>

    • compute

      public ConfusionMatrix compute(Alignment referenceAlignment, Alignment systemAlignment, GoldStandardCompleteness gsCompleteness, Alignment evaluationExclusionAlignment)

    • compute

      public ConfusionMatrix compute(Alignment referenceAlignment, Alignment systemAlignment, GoldStandardCompleteness gsCompleteness)

    • computeForPartialGoldStandard

      private ConfusionMatrix computeForPartialGoldStandard(Alignment referenceAlignment, Alignment systemAlignment, GoldStandardCompleteness gsCompleteness)
      Calculate the confusion matrix under the premises that the gold standard is incomplete, i.e., partial.
      Parameters:
      referenceAlignment - reference alignment
      systemAlignment - system alignment
      gsCompleteness - gold standard completeness
      Returns:
      The confusion matrix.

    • computeForCompleteGoldStandard

      private ConfusionMatrix computeForCompleteGoldStandard(Alignment referenceAlignment, Alignment systemAlignment)
      Calculate the confusion matrix under the premises that the gold standard is complete.
      Parameters:
      referenceAlignment - reference alignment
      systemAlignment - system alignment
      Returns:
      The confusion matrix.

    • calculateConfusionMatrixFromMappings

      public static ConfusionMatrix calculateConfusionMatrixFromMappings(Alignment truePositives, Alignment falsePositives, Alignment falseNegatives, int numberOfCorrespondences)
      Calculation method which calculates the confusion matrix given three mappings: tp, fp, and fn.
      Parameters:
      truePositives - True Positive (tp) mapping.
      falsePositives - False Positive (fp) mapping.
      falseNegatives - False Negative (fn) mapping.
      numberOfCorrespondences - The number of correspondences.
      Returns:
      The confusion matrix.

    • getMicroAveragesForResults

      public ConfusionMatrix getMicroAveragesForResults(Iterable<ExecutionResult> resultsForCalculation)
      Returns aggregated confusion matrices according to the micro average. The micro average sums up individual true positives, false positives and false negatives and calculates one confusion matrix.
      Parameters:
      resultsForCalculation - The results for which an aggregated confusion matrix shall be computed.
      Returns:
      Aggregated Confusion Matrices

    • getMacroAveragesForResults

      public ConfusionMatrix getMacroAveragesForResults(Iterable<ExecutionResult> resultsForCalculation)
      Returns aggregated confusion matrices according to the macro average.
      Parameters:
      resultsForCalculation - The results for which an aggregated confusion matrix shall be computed.
      Returns:
      Aggregated Confusion Matrices Aggregated Confusion Matrices

    • getMacroAveragesForResults

      public ConfusionMatrix getMacroAveragesForResults(Iterable<ExecutionResult> resultsForCalculation, int numberOfTestCases)
      Returns aggregated confusion matrices according to the macro average.
      Parameters:
      resultsForCalculation - The results for which an aggregated confusion matrix shall be computed.
      numberOfTestCases - The number of testcases which should be used for the calculation
      Returns:
      Aggregated Confusion Matrices Aggregated Confusion Matrices

    • getMicroAverages

      public ConfusionMatrix getMicroAverages(Iterable<ConfusionMatrix> confusionMatrices)
      Returns aggregated confusion matrices according to the micro average. The micro average sums up individual true positives, false positives and false negatives and calculates one confusion matrix.
      Parameters:
      confusionMatrices - The confusion matrices for which an aggregated confusion matrix shall be computed.
      Returns:
      Aggregated Confusion Matrices

    • getMacroAverages

      public ConfusionMatrix getMacroAverages(Iterable<ConfusionMatrix> confusionMatrices)
      Returns aggregated confusion matrices according to the macro average.
      Parameters:
      confusionMatrices - The confusion matrices for which an aggregated confusion matrix shall be computed.
      Returns:
      Aggregated Confusion Matrices Aggregated Confusion Matrices

    • executeAggregation

      private ConfusionMatrix executeAggregation(Iterable<ConfusionMatrix> confusionMatrices, ConfusionMatrixAggregationMode aggregationMode)
      Internal aggregation logic.
      Parameters:
      confusionMatrices - ConfusionMatrices that shall be used for the aggregation.
      aggregationMode - The kind of aggregation that is to be performed.
      Returns:
      Resulting confusion matrix.

    • divideWithTwoDenominators

      private static double divideWithTwoDenominators(double numerator, double denominatorOne, double denominatorTwo)
      Simple division that is to be performed. The two denominators will be added.
      Parameters:
      numerator - Numerator of fraction
      denominatorOne - Denominator 1
      denominatorTwo - Denominator 2
      Returns:
      Result as double.