edu.mit.jmwe.harness.result
Class MWEResult<T extends IToken,S extends IMarkedSentence<T>>

java.lang.Object
  extended by edu.mit.jmwe.harness.result.MWEResult<T,S>
Type Parameters:
T - the type of tokens contained in the unit and its associated multi-word expressions.
S - the type of sentence whose results are stored. Is parameterized by tokens of type T.
All Implemented Interfaces:
IBaseResult, IOverallResult<T,S>

public class MWEResult<T extends IToken,S extends IMarkedSentence<T>>
extends Object
implements IOverallResult<T,S>

Default implementation of IOverallResult interface. Contains the overall precision and recall scores with and without partial credit. All of the data stored in this result is organized by part of speech.

Since:
jMWE 1.0.0
Version:
$Id: MWEResult.java 623 2011-05-09 01:52:30Z markaf $
Author:
M.A. Finlayson, N. Kulkarni

Constructor Summary
MWEResult(Map<MWEPOS,Integer> answer, Map<MWEPOS,Integer> found, Map<MWEPOS,Integer> correct, Map<MWEPOS,Double> partial, Map<String,ISentenceResult<T,S>> detailed)
          Constructs the result from the answer, found and correct data, and the precision, recall and partial credit scores.
 
Method Summary
static double calcF1Score(double precision, double recall)
          Calculates the f1 score.
protected  double calcPartialPrecision(MWEPOS pos)
          Calculates the precision over the specified part of speech, taking into account partial credit for the specified MWE part of speech.
protected  double calcPartialRecall(MWEPOS pos)
          Calculates the recall over the specified part of speech, taking into account partial credit for the specified MWE part of speech.
static double calcPrecision(double correctlyRetrieved, double totalRetrieved)
          Given the total number of MWEs retrieved and the number of MWEs that are correct, calculates precision.
protected  double calcPrecision(MWEPOS pos)
          Calculates the precision over the specified parts of speech, or all parts of speech if the specified part of speech is null
static double calcRecall(double correctlyRetrieved, double totalAnswers)
          Given the total number of answer MWEs and the number of MWEs that are correct, calculates recall.
protected  double calcRecall(MWEPOS pos)
          Calculates the recall over the specified parts of speech, or all parts of speech if the specified part of speech is null
 Map<MWEPOS,Integer> getAnswerData()
          Returns a map that stores the number of answer multi-word expressions for each part of speech.
 Map<MWEPOS,Integer> getCorrectData()
          Returns a map that stores the number of multi-word expressions correctly found by the detector for each part of speech.
 Map<String,ISentenceResult<T,S>> getDetails()
          Returns a map that stores the results for a unit under its ID.
 Map<MWEPOS,Double> getF1Scores()
          Returns a map that stores the f1 measures of the detector for each part of speech.
 Map<MWEPOS,Integer> getFoundData()
          Returns a map that stores the number of multi-word expressions found by the detector for each part of speech.
 double getFScore()
          Returns the F1 score, a double between zero and 1, inclusive.
 double getPartialF1Score()
          Returns the total F score earned by the detector, taking into account partial credit.
 Map<MWEPOS,Double> getPartialF1Scores()
          Returns a map that stores the f1 measure of the detector for each part of speech after adding the partial credit to its correct score.
 double getPartialPrecision()
          Returns the total precision of the detector after counting partial credit.
 Map<MWEPOS,Double> getPartialPrecisionScores()
          Returns a map that stores the precision of the detector for each part of speech after adding the partial credit to its correct score.
 double getPartialRecall()
          Returns the total recall of the detector after counting partial credit.
 Map<MWEPOS,Double> getPartialRecallScores()
          Returns a map that stores the recall of the detector for each part of speech after adding the partial credit to its correct score.
 double getPartialScore()
          Returns the total partial credit earned by the detector.
 Map<MWEPOS,Double> getPartialScores()
          Returns a map that stores the partial credit for the partially correct multi-word expressions found by the detector for each part of speech.
 double getPrecision()
          Returns the precision, a double between zero and 1, inclusive.
 Map<MWEPOS,Double> getPrecisionScores()
          Returns a map that stores the precision of the detector for each part of speech.
 double getRecall()
          Returns the recall, a double between zero and 1, inclusive.
 Map<MWEPOS,Double> getRecallScores()
          Returns a map that stores the recall of the detector for each part of speech.
 int getTotalAnswers()
          Returns the total number of answers.
 int getTotalCorrect()
          Returns the total number of items correctly identified.
 int getTotalFound()
          Returns the total number items found.
static double sumDbl(Iterable<? extends Double> m)
          Return the sum of all the values stored in the map.
static int sumInt(Iterable<? extends Integer> m)
          Return the sum of all the values stored in the map.
 String toString()
           
static String toString(IOverallResult<?,?> result)
          Creates a table displaying the number of answer, found and correct multi-word expressions and the precision, recall and partial credit scores of the detector for each part of speech.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

MWEResult

public MWEResult(Map<MWEPOS,Integer> answer,
                 Map<MWEPOS,Integer> found,
                 Map<MWEPOS,Integer> correct,
                 Map<MWEPOS,Double> partial,
                 Map<String,ISentenceResult<T,S>> detailed)
Constructs the result from the answer, found and correct data, and the precision, recall and partial credit scores. Also takes a map that stores the ISentenceResult objects for a unit under its ID.

Parameters:
answer - a map that stores the number of answer multi-word expressions for each part of speech.
found - a map that stores the number of multi-word expressions found by the detector for each part of speech.
correct - a map that stores the number of multi-word expressions correctly found by the detector for each part of speech.
partial - a map that stores the partial credit for the partially correct multi-word expressions found by the detector for each part of speech.
detailed - a map that stores the results for a unit under its ID.
Since:
jMWE 1.0.0
Method Detail

getPrecision

public double getPrecision()
Description copied from interface: IBaseResult
Returns the precision, a double between zero and 1, inclusive.

Specified by:
getPrecision in interface IBaseResult
Returns:
the precision, a double between zero and 1, inclusive.

getRecall

public double getRecall()
Description copied from interface: IBaseResult
Returns the recall, a double between zero and 1, inclusive.

Specified by:
getRecall in interface IBaseResult
Returns:
the recall, a double between zero and 1, inclusive.

getPrecisionScores

public Map<MWEPOS,Double> getPrecisionScores()
Description copied from interface: IOverallResult
Returns a map that stores the precision of the detector for each part of speech. Should never be null.

Specified by:
getPrecisionScores in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the precision the detector for each part of speech.

getRecallScores

public Map<MWEPOS,Double> getRecallScores()
Description copied from interface: IOverallResult
Returns a map that stores the recall of the detector for each part of speech. Should never be null.

Specified by:
getRecallScores in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the recall the detector for each part of speech.

getF1Scores

public Map<MWEPOS,Double> getF1Scores()
Description copied from interface: IOverallResult
Returns a map that stores the f1 measures of the detector for each part of speech. Should never be null.

Specified by:
getF1Scores in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the f1 measures the detector for each part of speech.

getTotalAnswers

public int getTotalAnswers()
Description copied from interface: IBaseResult
Returns the total number of answers.

Specified by:
getTotalAnswers in interface IBaseResult

getTotalCorrect

public int getTotalCorrect()
Description copied from interface: IBaseResult
Returns the total number of items correctly identified.

Specified by:
getTotalCorrect in interface IBaseResult

getTotalFound

public int getTotalFound()
Description copied from interface: IBaseResult
Returns the total number items found.

Specified by:
getTotalFound in interface IBaseResult

getAnswerData

public Map<MWEPOS,Integer> getAnswerData()
Description copied from interface: IOverallResult
Returns a map that stores the number of answer multi-word expressions for each part of speech. Should never be null.

Specified by:
getAnswerData in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the number of answer multi-word expressions for each part of speech.

getFoundData

public Map<MWEPOS,Integer> getFoundData()
Description copied from interface: IOverallResult
Returns a map that stores the number of multi-word expressions found by the detector for each part of speech. Should never be null.

Specified by:
getFoundData in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the number of number of multi-word expressions found by the detector for each part of speech.

getCorrectData

public Map<MWEPOS,Integer> getCorrectData()
Description copied from interface: IOverallResult
Returns a map that stores the number of multi-word expressions correctly found by the detector for each part of speech. Should never be null.

Specified by:
getCorrectData in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the number of number of multi-word expressions correctly found by the detector for each part of speech.

getPartialPrecision

public double getPartialPrecision()
Description copied from interface: IOverallResult
Returns the total precision of the detector after counting partial credit. Must be a number between 0 and 1.

Specified by:
getPartialPrecision in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
the total precision with partial credit

getPartialRecall

public double getPartialRecall()
Description copied from interface: IOverallResult
Returns the total recall of the detector after counting partial credit. Must be a number between 0 and 1.

Specified by:
getPartialRecall in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
the total precision with partial credit

getPartialScores

public Map<MWEPOS,Double> getPartialScores()
Description copied from interface: IOverallResult
Returns a map that stores the partial credit for the partially correct multi-word expressions found by the detector for each part of speech. Partial credit should be given when the detector correctly identifies some but not all of the tokens in an expression or if it includes some extra incorrect tokens in the expression. Should never be null.

Specified by:
getPartialScores in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the partial credit for the partially correct multi-word expressions found by the detector for each part of speech.

getPartialPrecisionScores

public Map<MWEPOS,Double> getPartialPrecisionScores()
Description copied from interface: IOverallResult
Returns a map that stores the precision of the detector for each part of speech after adding the partial credit to its correct score. Should never be null.

Specified by:
getPartialPrecisionScores in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the precision the detector for each part of speech after counting partial credit.

getPartialRecallScores

public Map<MWEPOS,Double> getPartialRecallScores()
Description copied from interface: IOverallResult
Returns a map that stores the recall of the detector for each part of speech after adding the partial credit to its correct score. Should never be null.

Specified by:
getPartialRecallScores in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the recall the detector for each part of speech after counting partial credit.

getPartialF1Scores

public Map<MWEPOS,Double> getPartialF1Scores()
Description copied from interface: IOverallResult
Returns a map that stores the f1 measure of the detector for each part of speech after adding the partial credit to its correct score. Should never be null.

Specified by:
getPartialF1Scores in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores the f1 measure the detector for each part of speech after counting partial credit.

getPartialScore

public double getPartialScore()
Description copied from interface: IOverallResult
Returns the total partial credit earned by the detector. Partial credit should be given when the detector correctly identifies some but not all of the tokens in a multi-word expression or if it includes some extra incorrect tokens in the expression.

Specified by:
getPartialScore in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
the total partial credit earned by the detector

getFScore

public double getFScore()
Description copied from interface: IBaseResult
Returns the F1 score, a double between zero and 1, inclusive. The F1 score, or f-measure, is the harmonic mean of the precision and the recall.

Specified by:
getFScore in interface IBaseResult
Returns:
the F1 score, a double between zero and 1, inclusive.

getPartialF1Score

public double getPartialF1Score()
Description copied from interface: IOverallResult
Returns the total F score earned by the detector, taking into account partial credit.

Specified by:
getPartialF1Score in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
the total F score earned by the detector, taking into account partial credit.

getDetails

public Map<String,ISentenceResult<T,S>> getDetails()
Description copied from interface: IOverallResult
Returns a map that stores the results for a unit under its ID. Should never be null.

Specified by:
getDetails in interface IOverallResult<T extends IToken,S extends IMarkedSentence<T>>
Returns:
a map that stores a unit result under its ID.

toString

public String toString()
Overrides:
toString in class Object

calcPrecision

protected double calcPrecision(MWEPOS pos)
Calculates the precision over the specified parts of speech, or all parts of speech if the specified part of speech is null

Parameters:
pos - the part of speech over which the precision is to be calculated; null means all parts of speech
Returns:
the precision over the specified part of speech
Since:
jMWE 1.0.0

calcRecall

protected double calcRecall(MWEPOS pos)
Calculates the recall over the specified parts of speech, or all parts of speech if the specified part of speech is null

Parameters:
pos - the part of speech over which the recall is to be calculated; null means all parts of speech
Returns:
the recall over the specified part of speech
Since:
jMWE 1.0.0

calcPartialPrecision

protected double calcPartialPrecision(MWEPOS pos)
Calculates the precision over the specified part of speech, taking into account partial credit for the specified MWE part of speech. The partial credit is the alignment score between a partially correct MWE and an answer MWE.

Parameters:
pos - the MWE pos for which the precision should be calculated; or null if for all
Returns:
the total precision with partial credit
Since:
jMWE 1.0.0

calcPartialRecall

protected double calcPartialRecall(MWEPOS pos)
Calculates the recall over the specified part of speech, taking into account partial credit for the specified MWE part of speech. The partial credit is the alignment score between a partially correct MWE and an answer MWE.

Parameters:
pos - the MWE pos for which the recall should be calculated; or null if for all
Returns:
the total precision with partial credit
Since:
jMWE 1.0.0

toString

public static String toString(IOverallResult<?,?> result)
Creates a table displaying the number of answer, found and correct multi-word expressions and the precision, recall and partial credit scores of the detector for each part of speech.

Parameters:
result - a non-null IResult
Returns:
A String that displays the information stored in the result in a table.
Since:
jMWE 1.0.0

calcPrecision

public static double calcPrecision(double correctlyRetrieved,
                                   double totalRetrieved)
Given the total number of MWEs retrieved and the number of MWEs that are correct, calculates precision. Will be Double.NaN if the number of correct MWEs is greater than 0 and there are no found MWEs, since this is undefined. Will return 1.0, a perfect score, if both the number of found MWEs and correct MWEs are 0 or if both numbers are equal.

Parameters:
correctlyRetrieved - the number of MWEs that are correctly identified
totalRetrieved - the number of MWEs retrieved
Returns:
the precision. Will be Double.NaN if the number of correct MWEs is greater than 0 and there are no found MWEs. Will return 1.0, a perfect score, if both the number of found MWEs and correct MWEs are 0 or if both numbers are equal.
Since:
jMWE 1.0.0

calcRecall

public static double calcRecall(double correctlyRetrieved,
                                double totalAnswers)
Given the total number of answer MWEs and the number of MWEs that are correct, calculates recall. Will be Double.NaN if the number of correct MWEs is greater than 0 and there are no answer MWEs, since this is undefined. Will return 1.0, a perfect score, if both the number of answer MWEs and correct MWEs are 0 or if both numbers are equal.

Parameters:
correctlyRetrieved - the number of MWEs that are correctly identified
totalAnswers - the total number of answer MWEs
Returns:
the recall. Will be Double.NaN if the number of correct MWEs is greater than 0 and there are no answer MWEs. Will return 1.0, a perfect score, if both the number of answer MWEs and correct MWEs are 0 or if both numbers are equal.
Since:
jMWE 1.0.0

calcF1Score

public static double calcF1Score(double precision,
                                 double recall)
Calculates the f1 score. It is the harmonic mean of the precision and recall, reaching its best value at 1 and worst value at 0. If either the precision or recall is undefined, will return Double.NaN.

Parameters:
precision - the precision. Must be a non-negative double less than 1.0.
recall - the recall. Must be a non-negative double less than 1.0.
Returns:
the f1 score. Will be Double.NaN if either the precision or recall is undefined; otherwise, will be a nonnegative double between 0.0 and 1.0.
Since:
jMWE 1.0.0

sumInt

public static int sumInt(Iterable<? extends Integer> m)
Return the sum of all the values stored in the map.

Parameters:
m - the map whose values are to be summed
Returns:
the sum of all the values stored in the map.
Since:
jMWE 1.0.0

sumDbl

public static double sumDbl(Iterable<? extends Double> m)
Return the sum of all the values stored in the map.

Parameters:
m - the map whose values are to be summed
Returns:
the sum of all the values stored in the map.
Since:
jMWE 1.0.0


Copyright © 2011 Massachusetts Institute of Technology. All Rights Reserved.