Uses of Package
edu.mit.jsemcor.detokenize

Packages that use edu.mit.jsemcor.detokenize

edu.mit.jsemcor.detokenize

Provides interfaces and default implementations for classes that provide the ability to generate free text from Semcor files.

Classes in edu.mit.jsemcor.detokenize used by edu.mit.jsemcor.detokenize

ApostropheSRule
A separator rule encoding the fact that the string separating a word and it's following apostrophe-s ('s) should not be separated by whitespace.

CompositeDetokenizer
An implementation of a detokenizer that is a composite of other detokenizers.

ContractionRule
Implements a rule encoding the fact that the contractions 're, n't and 've come directly after the previous token, with no space.

DelimiterRule
Implements a rule encoding the fact that there are spaces before but not after opening parentheses, brackets, and quotes, and spaces after and not before closing instances of the same.

DollarSignRule
Implements a rule encoding the fact that if a dollar sign is followed by an adjective (JJ*) or a cardinal number (CD), there is no space between them, e.g., Vandiver likely will mention the $100 million highway bond issue approved earlier... (JJ, from br-a01), and The bill would increase from $5000000 to $15000000 the maximum loan the state could make... (CD, from br-a02).

IDetokenizer
A detokenizer is a class that can transform a list of tokens into a properly-whitespace delimited string.

ISeparatorRule
Classes that implement this interface provide the information necessary to re-concatenating tokens into properly formatted text.

PhraseDelimiterRule
Implements a rule indicating that a comma has no space between it and previous token.

SentenceFinalPuncRule
Implements the rule that there is no space before sentence-final punctuation (periods, question marks, and exclamation points), and a space after, unless the next token is a closing delimiter.