public class Token extends java.lang.Object implements IToken
IToken
interface.Constructor and Description |
---|
Token(java.lang.String text,
java.lang.String tag)
Constructs a new token object with the specified text and tag,
with no stems yet assigned.
|
Token(java.lang.String text,
java.lang.String tag,
java.lang.String... stems)
Constructs a new token object with the specified text, tag, and stems.
|
Modifier and Type | Method and Description |
---|---|
static java.util.List<java.lang.String> |
checkStems(java.lang.String[] stems)
Checks the specified array of strings to ensure each one is non-
null , and, once trimmed, is not empty and does not contain
whitespace or an underscore. |
static java.lang.String |
checkString(java.lang.String text)
Checks the specified string to see that, once trimmed, it is not empty
and does not contain whitespace.
|
java.lang.String |
getForm()
Returns the object's surface form text, exactly as it appears in its
original context, with capitalization intact.
|
java.util.List<java.lang.String> |
getStems()
Returns an unmodifiable list of stems, all in lowercase.
|
java.lang.String |
getTag()
Returns the part of speech tag for this token, or
null if
the token is not tagged. |
java.lang.String |
toString() |
public Token(java.lang.String text, java.lang.String tag)
text
- the surface form of the token as it appears in the sentence,
capitalization intacttag
- the tag of the token, if assigned, otherwise null
java.lang.NullPointerException
- if the text is null
java.lang.NullPointerException
- if the text is null
java.lang.IllegalArgumentException
- if the trimmed text is empty or contains whitespacepublic Token(java.lang.String text, java.lang.String tag, java.lang.String... stems)
text
- the surface form of the token as it appears in the sentence,
capitalization intacttag
- the tag of the token, if assigned, otherwise null
stems
- the array of stems, possibly empty or null
, but
not containing null
. If null
, this
means that no stemming has yet been attempted. If empty, this
means the token is not stemmable.java.lang.NullPointerException
- if the text is null
, or any of the stems are
null
java.lang.IllegalArgumentException
- if the trimmed text is empty or contains whitespacepublic java.lang.String getForm()
IHasForm
null
.public java.lang.String getTag()
IToken
null
if
the token is not tagged. If the part of speech is null
, no
part of speech has yet been assigned.public java.util.List<java.lang.String> getStems()
IToken
null
, this means no
stemming has yet been attempted.public java.lang.String toString()
toString
in class java.lang.Object
public static java.lang.String checkString(java.lang.String text)
text
- the text to be checkedjava.lang.NullPointerException
- if the specified String is null
java.lang.IllegalArgumentException
- if, after being trimmed, the specified String is empty, or
contains whitespace or an underscorepublic static java.util.List<java.lang.String> checkStems(java.lang.String[] stems)
null
, and, once trimmed, is not empty and does not contain
whitespace or an underscore. If all strings check out, an unmodifiable
list of the trimmed, lowercase strings is returned. Otherwise, the method
throws an exception.stems
- the list of stems to check; may be null
or empty,
but may not contain null
java.lang.NullPointerException
- if the any string in the array is null
java.lang.IllegalArgumentException
- if, after being trimmed, any string in the array is empty or
contains whitespaceCopyright © 2011 Massachusetts Institute of Technology. All Rights Reserved.