Package de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator

Class EvaluatorRank

java.lang.Object

de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.Evaluator

de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.EvaluatorRank

public class EvaluatorRank
extends Evaluator

A rank evaluator which writes a file resultsRanking.csv. It contains the rankings for each executed testcase. Currently it contains DCG, nDCG, MAP in three variants: random, alphabetically, top (this only comes into play when there are correspondences with the same confidence).

Field Summary

Fields

Modifier and Type	Field	Description
private static final org.slf4j.Logger	LOGGER
private List<Refiner[]>	refinerList	Refinement operations to be applied.
static final String	RESULT_FILE_NAME	The CSV file name which will be written to the results directory.
private SameConfidenceRanking[]	sameConfidenceRankingList	The strategies to be evaluated for cases in which two correspondences carry the same confidence.

Fields inherited from class de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.Evaluator

results

Constructor Summary

Constructors

Constructor	Description
EvaluatorRank(ExecutionResultSet results)	Convenience Constructor.
EvaluatorRank(ExecutionResultSet results, SameConfidenceRanking... sameConfidenceRankingList)	Constructor

Method Summary

Modifier and Type	Method	Description
List<Refiner[]>	getRefinerList()
private void	runMetricAndPrintToFile(RankingMetric[] metrics, ExecutionResult executionResult, String refinerString, org.apache.commons.csv.CSVPrinter printer)	Runs the metric on the given result and prints a line to the CSV file.
void	setRefinerList(List<Refiner[]> refinerList)
void	writeResultsToDirectory(File baseDirectory)

Methods inherited from class de.uni_mannheim.informatik.dws.melt.matching_eval.evaluator.Evaluator

checkAndCreateDirectory, getDefaultResultsDirectory, getDirectoryWithCurrentTime, getResults, getResultsDirectoryTrack, getResultsDirectoryTrackMatcher, getResultsDirectoryTrackTestcaseMatcher, setDefaultResultsDirectory, writeToDirectory, writeToDirectory, writeToDirectory

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

sameConfidenceRankingList

private SameConfidenceRanking[] sameConfidenceRankingList

The strategies to be evaluated for cases in which two correspondences carry the same confidence.

RESULT_FILE_NAME

public static final String RESULT_FILE_NAME

The CSV file name which will be written to the results directory.

See Also:

• Constant Field Values

refinerList

private List<Refiner[]> refinerList

Refinement operations to be applied.

Constructor Details

EvaluatorRank

public EvaluatorRank(ExecutionResultSet results,
 SameConfidenceRanking... sameConfidenceRankingList)

Constructor

Parameters:

results - The results to be evaluated.

sameConfidenceRankingList - The confidence ranking strategies to be applied in case of multiple correspondences with the same confidence. If multiple strategies are given, all will be evaluated and appear in the resulting CSV file (for comparison).

EvaluatorRank

public EvaluatorRank(ExecutionResultSet results)

Convenience Constructor. Clashes with multiple correspondences carrying the same score will be resolved by alphabetical ordering.

Parameters:

results - The execution result set.

Method Details

writeResultsToDirectory

public void writeResultsToDirectory(File baseDirectory)

Specified by:

writeResultsToDirectory in class Evaluator

runMetricAndPrintToFile

private void runMetricAndPrintToFile(RankingMetric[] metrics,
 ExecutionResult executionResult,
 String refinerString,
 org.apache.commons.csv.CSVPrinter printer)
                              throws IOException

Runs the metric on the given result and prints a line to the CSV file.

Parameters:

metrics - Metrics for evaluation.

executionResult - Refined result for evaluation.

refinerString - String identifying the refinement operations that have been performed.

printer - Printer to be used for printing.

Throws:

IOException - IOException may be thrown while printing.

getRefinerList

public List<Refiner[]> getRefinerList()

setRefinerList

public void setRefinerList(List<Refiner[]> refinerList)