Package de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator

Class EvaluatorRankGroup

java.lang.Object
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.Evaluator
de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.EvaluatorRankGroup
public class EvaluatorRankGroup extends Evaluator
An evaluator that calculates rank metrics on an per-element basis for each element of a specified source ontology. An example is provided below. If you look for a plain general implementation of NDCG and DCG (not grouped per source node), you have to use EvaluatorRank. Example: Be O1 and O2 different ontologies that are to be matched. 
 O1:a----------O2:a (FP)
        |
        ------O2:b (TP)

 O1:b----------O2:b (FP)
       |
       ------O2:c (TP)
       |
       ------O2:d (TP)
 
 Reference Alignment:
 (O1:a, O2:b, =)
 (O1:b, O2:c, =)
 (O1:b, O2:d, =)
 
 Given a HITS@2 configuration with source=O1 (examples of many KPIs calculated):
 P@2 = (1/2 + 1/2)/2 = 1/2
 HITS@2 = 1
 F1@2 = (2 * P@2 * R@2)/(P@2 + R@2)

  • Field Details

    • LOGGER

      private static final org.slf4j.Logger LOGGER
      Default logger.

    • sameConfidenceRankingList

      private SameConfidenceRanking[] sameConfidenceRankingList
      The strategies to be evaluated for cases in which two correspondences carry the same confidence.

    • RESULT_FILE_NAME

      public static final String RESULT_FILE_NAME
      The CSV file name which will be written to the results directory.
      See Also:

    • kOfHitsAtK

      private int kOfHitsAtK
      K of HITS@K-based metrics.

    • refinerList

      private List<Refiner[]> refinerList
      Refinement operations to be applied.

  • Constructor Details

    • EvaluatorRankGroup

      public EvaluatorRankGroup(ExecutionResultSet results, int kOfHitsAtK, SameConfidenceRanking... sameConfidenceRankingList)
      Constructor
      Parameters:
      results - The results to be evaluated.
      kOfHitsAtK - K of HTIS@K (must be a positive integer > 0)
      sameConfidenceRankingList - The confidence ranking strategies to be applied in case of multiple correspondences with the same confidence. If multiple strategies are given, all will be evaluated and appear in the resulting CSV file (for comparison).

    • EvaluatorRankGroup

      public EvaluatorRankGroup(ExecutionResultSet results, int kOfHitsAtK)
      Convenience Constructor. Clashes with multiple correspondences carrying the same score will be resolved by alphabetical ordering.
      Parameters:
      results - The results of the matching process.
      kOfHitsAtK - K of HTIS@K (must be a positive integer > 0)

  • Method Details

    • writeResultsToDirectory

      protected void writeResultsToDirectory(File baseDirectory)
      Specified by:
      writeResultsToDirectory in class Evaluator

    • runMetricAndPrintToFile

      private void runMetricAndPrintToFile(RankingMetricGroup[] metrics, ExecutionResult executionResult, String refinerString, org.apache.commons.csv.CSVPrinter printer) throws IOException
      Runs the metric on the given result and prints a line to the CSV file.
      Parameters:
      metrics - Metrics for evaluation.
      executionResult - Refined result for evaluation.
      refinerString - String identifying the refinement operations that have been performed.
      printer - Printer to be used for printing.
      Throws:
      IOException - IOException may be thrown while printing.

    • getRefinerList

      public List<Refiner[]> getRefinerList()

    • setRefinerList

      public void setRefinerList(List<Refiner[]> refinerList)