|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use IToken | |
---|---|
edu.mit.jmwe.data | Provides the basic data structures used by the library and their default implementations. |
edu.mit.jmwe.data.concordance | Provides interfaces and classes for accessing data taken Semcor-formatted concordances, useful for benchmarking detectors. |
edu.mit.jmwe.detect | Provides MWE detector API, a baseline detector, plus numerous other detector implementations. |
edu.mit.jmwe.detect.score | Provides various scoring mechanisms that can be used by subclasses of the FilterByScore and ResolveByScore detectors. |
edu.mit.jmwe.harness | Provides testing harness infrastructure |
edu.mit.jmwe.harness.result | Provides objects that encapsulate the results of a test harness run |
edu.mit.jmwe.harness.result.error | Provides error detectors to evaluate the results of a test harness run |
edu.mit.jmwe.index | Provides the MWE index interfaces and default implementations, which allow one to look up an MWE given one of its parts. |
Uses of IToken in edu.mit.jmwe.data |
---|
Classes in edu.mit.jmwe.data with type parameters of type IToken | |
---|---|
interface |
IMarkedSentence<T extends IToken>
A marked sentence is a sentence (i.e., a list of tokens) that has been tagged with a unique id. |
interface |
IMWE<T extends IToken>
A multi-word expression found in a list of tokens. |
class |
MWE<T extends IToken>
Default implementation of the IMWE interface. |
class |
MWEComparator<T extends IToken>
A comparator that compares IMWE s by checking which MWE starts earlier
in the list of tokens used to construct this comparator. |
Classes in edu.mit.jmwe.data that implement IToken | |
---|---|
class |
Token
Default implementation of the IToken interface. |
Methods in edu.mit.jmwe.data with parameters of type IToken | |
---|---|
static boolean |
AbstractMWEDesc.isFillerForSlot(IToken token,
IMWEDesc.IPart part)
Returns true if the part's lemma matches either the surface form of the given token or any of the token's stems, regardless of case. |
Uses of IToken in edu.mit.jmwe.data.concordance |
---|
Subinterfaces of IToken in edu.mit.jmwe.data.concordance | |
---|---|
interface |
IConcordanceToken
A token from a Semcor sentence. |
Classes in edu.mit.jmwe.data.concordance that implement IToken | |
---|---|
protected static class |
ConcordanceTagger.TaggerToken
Represents a semcor token that is not yet tagged. |
class |
ConcordanceToken
Default implementation of IConcordanceToken . |
Uses of IToken in edu.mit.jmwe.detect |
---|
Classes in edu.mit.jmwe.detect with type parameters of type IToken | |
---|---|
class |
MWEBuilder<T extends IToken>
A record that is used to hold tokens as the detector passes over a sentence. |
Methods in edu.mit.jmwe.detect with type parameters of type IToken | ||
---|---|---|
protected
|
Exhaustive.containsDuplicate(Collection<? extends IMWE<T>> results,
IMWE<T> mwe)
Returns true if the given collection of MWEs already contains a particular MWE. |
|
|
StopWords.detect(List<T> sentence)
|
|
|
ResolveByScore.detect(List<T> sentence)
|
|
|
ProperNouns.detect(List<T> sentence)
|
|
|
Perfect.detect(List<T> sentence)
|
|
|
NoProperNouns.detect(List<T> sentence)
|
|
|
NoInflection.detect(List<T> sentence)
|
|
|
LMLR.detect(List<T> s)
|
|
|
InOrder.detect(List<T> sentence)
|
|
|
InflectionPattern.detect(List<T> sentence)
|
|
|
InflectionLookup.detect(List<T> sentence)
|
|
|
IMWEDetector.detect(List<T> sentence)
Given a list of tokens, the detector searches for the MWEs in the list. |
|
|
HasMWEDetector.detect(List<T> sentence)
|
|
|
FilterByScore.detect(List<T> sentence)
|
|
|
Exhaustive.detect(List<T> sentence)
|
|
|
Continuous.detect(List<T> sentence)
|
|
|
Consecutive.detect(List<T> sent)
|
|
|
CompositeDetector.detect(List<T> sentence)
|
|
static
|
MWEBuilder.fillNextSlot(MWEBuilder<T> builder,
T t)
Fills the first non-null (empty) slot in the given builder. |
|
protected
|
Consecutive.fillNextSlot(MWEBuilder<T> builder,
T t)
Fills the first non-null (empty) slot in the given builder. |
|
static
|
MWEBuilder.fillSlots(Set<MWEBuilder<T>> records,
T token)
Given a set of MWE builders, fills all the slots in the records that can be filled by the given token. |
|
static
|
LMLR.getFirstToken(Iterable<? extends T> tokens,
Comparator<T> c)
Returns the token that is the first in a given iterable collection of tokens. |
|
protected
|
SmallestVariance.getScorer(List<T> sentence)
|
|
protected abstract
|
ResolveByScore.getScorer(List<T> sentence)
Returns the scoring function for this filter. |
|
protected
|
MoreFrequentAsMWE.getScorer(List<T> sentence)
|
|
protected
|
Longest.getScorer(List<T> scorer)
|
|
protected
|
LeskAtLeast.getScorer(List<T> sentence)
|
|
protected
|
Leftmost.getScorer(List<T> sentence)
|
|
protected abstract
|
FilterByScore.getScorer(List<T> sentence)
Returns a scoring function for the specified sentence. |
|
protected
|
ConstrainLength.getScorer(List<T> sentence)
|
|
static
|
InflectionLookup.getSurfaceFormDescription(IRootMWEDesc root,
IMWE<T> mwe)
Returns a multi-word expression description with a lemma that is constructed by concatenating the tokens of the MWE exactly as they appear in the sentence with underscores. |
|
|
InflectionRule.getTagPattern(IMWE<T> mwe)
Concatenates the tags of each token in the MWE, separating each by underscores. |
|
static
|
InflectionRule.inflects(T token,
IMWE<T> mwe)
Returns true if a the text of a token from an MWE does not equal the corresponding part lemma. |
|
static
|
Continuous.isDiscontinuous(IMWE<T> mwe,
List<T> sentence)
Determines if the specified MWE is continuous, i.e., there are no interstitial tokens inside its boundaries that are not a part of the MWE. |
|
static
|
Continuous.isDiscontinuous(IMWE<T> mwe,
Map<T,Integer> indexMap)
Determines if the specified MWE is continuous, i.e., there are no interstitial tokens inside its boundaries that are not a part of the MWE. |
|
static
|
InOrder.isOutOfOrder(IMWE<T> mwe)
Determines if the constituents of the specified MWE are out of order. |
|
static
|
ProperNouns.isProperNoun(T token)
Checks if the token represents a proper noun by checking its part of speech tag. |
|
|
InflectionRule.isValid(IMWE<T> mwe)
|
|
|
IInflectionRule.isValid(IMWE<T> mwe)
Returns true if this MWE follows the rule;
false otherwise. |
|
protected
|
ProperNouns.isValidInterstitial(T token,
LinkedList<T> tokens)
Checks if a token that is not a proper noun may still be a part of a proper noun MWE. |
|
static
|
LMLR.longest(IMWE<T> one,
IMWE<T> two,
Comparator<T> c)
Compares two MWEs and returns the longest MWE. |
|
|
InflectionRule.matches(IMWE<T> mwe)
|
|
|
IInflectionRule.matches(IMWE<T> mwe)
Returns true if the given MWE has the same syntax as this
rule. |
|
protected
|
ProperNouns.removeIncorrectInterstitials(LinkedList<T> cs)
Removes all the tokens from the end of the given list that are not proper nouns. |
Methods in edu.mit.jmwe.detect with parameters of type IToken | |
---|---|
protected Set<? extends IMWEDesc> |
Consecutive.getMWEDescs(IToken token)
Returns all the MWE entries in the index that contain the given token or one of its stems as a part. |
Uses of IToken in edu.mit.jmwe.detect.score |
---|
Classes in edu.mit.jmwe.detect.score with type parameters of type IToken | |
---|---|
class |
FractionAsMWEScore<T extends IToken>
A scorer that scores with the fraction of times it appears marked as an MWE, as opposed to a run of unmarked tokens. |
class |
LengthScore<T extends IToken>
Scores a MWE with its length |
class |
LeskScore<T extends IToken>
Scores an object with its lesk-score overlap with dictionary glosses. |
class |
StartingIndexScore<T extends IToken>
Scores an MWE with its starting index. |
class |
VarianceScore<T extends IToken>
Scores each MWE with its index variance. |
Methods in edu.mit.jmwe.detect.score with type parameters of type IToken | ||
---|---|---|
static
|
LengthScore.getInstance()
Returns the singleton instance of this class, instantiating if necessary. |
|
static
|
FractionAsMWEScore.getInstance()
Returns the singleton instance of this class, instantiating if necessary. |
Uses of IToken in edu.mit.jmwe.harness |
---|
Methods in edu.mit.jmwe.harness with type parameters of type IToken | ||
---|---|---|
|
IAnswerKey.getAnswers(IMarkedSentence<T> sentence)
Gets the answer multi-word expressions from the given sentence. |
|
|
ConcordanceAnswerKey.getAnswers(IMarkedSentence<T> sent)
|
|
|
ConcordanceAnswerKey.getAnswers(IMarkedSentence<T> sent,
edu.mit.jsemcor.element.ISentence answers)
Extracts a set of MWE answers from a sentence and its corresponding answer sentence. |
|
protected
|
ConcordanceAnswerKey.getContinuousMWEs(IMarkedSentence<T> sent,
edu.mit.jsemcor.element.ISentence answer,
Set<edu.mit.jsemcor.element.IWordform> used)
Gets the multi-word expressions from the given sentence that are marked as single tokens. |
|
protected
|
ConcordanceAnswerKey.getNonContinuousMWEs(IMarkedSentence<T> sent,
edu.mit.jsemcor.element.ISentence answer,
Set<edu.mit.jsemcor.element.IWordform> used)
Gets the multi-word expressions from the given sentence that are non-contiguous (e.g., have a distance value not equal to zero). |
|
|
TestHarness.run(IMWEDetector detector,
IResultBuilder<T,S> result,
Iterator<S> itr,
IAnswerKey answers,
IProgressBar pb)
|
|
|
ITestHarness.run(IMWEDetector detector,
IResultBuilder<T,S> results,
Iterator<S> itr,
IAnswerKey answers,
IProgressBar pb)
Runs the detector in the test harness and stores the results in the provided result builder. |
|
|
TestHarness.run(Map<IMWEDetector,IResultBuilder<T,S>> detectors,
Iterator<S> itr,
IAnswerKey answers,
IProgressBar pb)
|
|
|
ITestHarness.run(Map<IMWEDetector,IResultBuilder<T,S>> detectors,
Iterator<S> itr,
IAnswerKey answers,
IProgressBar pb)
Runs the detectors in the test harness and stores the results in the associated result builder. |
|
protected
|
TestHarness.runDetector(IMWEDetector detector,
IResultBuilder<T,S> builder,
S sent,
List<IMWE<T>> answers)
Runs the detector over a single sentence, storing the result as an ISentenceResult in the given result builder. |
|
protected
|
TestHarness.runDetectors(Map<IMWEDetector,IResultBuilder<T,S>> detectors,
S sent,
List<IMWE<T>> answers)
Runs a set of detectors on the specified sentence, comparing the results to the specified answers. |
Uses of IToken in edu.mit.jmwe.harness.result |
---|
Methods in edu.mit.jmwe.harness.result with type parameters of type IToken | ||
---|---|---|
static
|
SentenceResult.printTable(StringBuilder sb,
ISentenceResult<T,S> result,
Formatter f)
Prints a table of the correct, false negative and false positive expressions found by the detector in columns. |
|
static
|
ErrorResult.toString(IErrorResult<T> result)
Creates a table displaying the number of instances of each error class. |
|
static
|
SentenceResult.toString(ISentenceResult<T,S> result,
S sentence,
boolean table)
Creates a graphical representation of the multi-word expressions found by the detector for a given sentence. |
|
static
|
SentenceResult.toString(ISentenceResult<T,U> result,
U sentence)
Creates a graphical representation of the multi-word expressions found by the detector for a given sentence. |
Uses of IToken in edu.mit.jmwe.harness.result.error |
---|
Methods in edu.mit.jmwe.harness.result.error with type parameters of type IToken | ||
---|---|---|
|
VBDVBN.detect(ISentenceResult<T,S> result)
|
|
|
UntaggedPNoun.detect(ISentenceResult<T,S> result)
|
|
|
MissingFromIndex.detect(ISentenceResult<T,S> result)
|
|
|
InflectionPatternError.detect(ISentenceResult<T,S> result)
|
|
|
InflectionError.detect(ISentenceResult<T,S> result)
|
|
|
IErrorDetector.detect(ISentenceResult<T,S> result)
Identifies the multi-word expressions in a unit result that fall under the specific error class this detector identifies. |
|
|
ExtraPrep.detect(ISentenceResult<T,S> result)
|
|
|
ExtraPOS.detect(ISentenceResult<T,S> result)
|
|
|
DetectorDisagreement.detect(ISentenceResult<T,S> result)
|
|
|
CompositeErrorDetector.detect(ISentenceResult<T,S> result)
|
|
|
AllStopWords.detect(ISentenceResult<T,S> result)
|
|
|
WrongPOS.detect(ISentenceResult<T,U> result)
|
|
|
PNounShort.detect(ISentenceResult<T,U> result)
|
|
|
PNounLong.detect(ISentenceResult<T,U> result)
|
|
|
OutOfOrder.detect(ISentenceResult<T,U> result)
|
|
|
InterstitialTokens.detect(ISentenceResult<T,U> result)
|
|
protected static
|
ExtraPrep.findTag(IMWE<T> test,
String tag)
Returns the index of the first token in the MWE with the specified tag. |
|
static
|
InterstitialTokens.hasParticle(IMWE<T> mwe,
List<T> sentence)
Returns true if the given MWE contains a token that is a particle and is separated from the previous token in the MWE by one or more non-MWE tokens in the sentence. |
|
static
|
InterstitialTokens.isParticle(T token)
Returns true if the specified token is tagged as a particle;
false otherwise |
|
static
|
VBDVBN.isProblem(IMWE<T> mwe)
Determines if the specified MWE is a problem according to this error class. |
|
static
|
MissingFromIndex.isProblem(IMWE<T> mwe,
IMWEIndex index)
Determines if the specified MWE is a problem, relative to the specified index, according to this error class. |
|
static
|
DetectorDisagreement.isProblem(IMWE<T> mwe,
ISentenceResult<T,S> result,
IMWEDetector detector)
Determines if the specified MWE is a problem relative to the specified sentence according to this error class. |
Uses of IToken in edu.mit.jmwe.index |
---|
Methods in edu.mit.jmwe.index with type parameters of type IToken | ||
---|---|---|
|
IndexBuilder.findMissingMWEs(List<IMWE<T>> mwes,
Map<IMWEDescID,IndexBuilder.MutableRootMWEDesc> index,
Set<IndexBuilder.MutableRootMWEDesc> missing)
Finds MWEs that are marked in the the specified list, but not in the index. |
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |