All Packages Class Hierarchy This Package Previous Next Index
Class sdsu.util.TokenCharacters
java.lang.Object
|
+----sdsu.util.TokenCharacters
- public class TokenCharacters
- extends Object
This class maintains special characters used in parsing strings into tokens. It keeps
track of whitespace, characters to indicate the start of a comment, quote characters
used to quote tokens that contain special characters, an escape character, and characters
that separate tokens.
- Author:
- Roger Whitney (whitney@cs.sdsu.edu)
- See Also:
- SimpleTokenizer, Stringizer
-
COMMENT_CHAR
- Default character (#) used to indicate start of comment.
-
ESCAPE_CHAR
- Character (/) used to preceed a quote character or escape character in a quoted token.
-
QUOTE_CHAR
- Default character (') used to delineate the start and end of a quoted token.
-
WHITESPACE
- Default characters treated as whitespace.
-
TokenCharacters()
- Create TokenCharacters with default values.
-
TokenCharacters(String)
- Create TokenCharacters with given characters for token separators and default values
for the rest of parameters.
-
TokenCharacters(String, String, char, char, String)
- Create a TokenCharacters object with given values
-
addQuoteChars(char, char)
- Add the quote pair beginQuote-endQuote to the pairs recognized as char pairs
to quote a token.
-
containsEscapeableChar(String)
- Returns true if c needs to be escaped in a quoted token.
-
containsTokenTerminator(String)
- Returns true if contains a character that indicates the end of a token.
-
escapeToken(String)
- Places escape character before any quote character or the escape character
Returns the modified token
-
getCommentChar()
- Returns a character that indicates start of a comment.
-
isBeginQuote(char)
- Returns true if c indicates the start of a quoted token
-
isComment(char)
- Returns true if c indicates the start of a comment
-
isEndQuote(char)
- Returns true if c indicates the end of a quoted token
-
isEOL(char)
- Returns true if c is Mac, Unix, or PC EOL character
-
isEscape(char)
- Returns true if c is an escape character
-
isQuotePair(char, char)
- Returns true if beginQuote and endQuote are matching begin ending
quote characters
-
isSeparator(char)
- Returns true if c is a separator character
-
isTokenTerminator(char)
- Returns true if c indicates the end of an unquoted token
IE c is a whitespace, separator or comment character
-
isWhitespace(char)
- Returns true if c is a whitespace character
-
quoteToken(String)
- Surrounds token with begin-end quote pair
Returns the quoted token
-
requiresEscaping(char)
- Returns true if c needs to be escaped in a quoted token.
-
setSeparatorChars(String)
- Set the current set of separators to newSeparators
WHITESPACE
public static final String WHITESPACE
- Default characters treated as whitespace. Default is space, tab, line feed
(ascii 10, \n in unix)
and carriage return ( ascii 13, usd as newline char on PC and Macs )
ESCAPE_CHAR
public static final char ESCAPE_CHAR
- Character (/) used to preceed a quote character or escape character in a quoted token.
COMMENT_CHAR
public static final String COMMENT_CHAR
- Default character (#) used to indicate start of comment.
QUOTE_CHAR
public static final char QUOTE_CHAR
- Default character (') used to delineate the start and end of a quoted token.
Tokens are quoted when they contain special characters.
TokenCharacters
public TokenCharacters()
- Create TokenCharacters with default values. You must set separators before using
the new object.
TokenCharacters
public TokenCharacters(String separators)
- Create TokenCharacters with given characters for token separators and default values
for the rest of parameters.
TokenCharacters
public TokenCharacters(String separators,
String commentChars,
char beginQuoteChar,
char endQuoteChar,
String whitespace)
- Create a TokenCharacters object with given values
- Parameters:
- commentChar - character used to indicate start of a comment
- beginQuoteChar - character used to start a quote of a string containing special characters
- endQuoteChar - character used to end a quote of a string containing special characters
- whitespace - characters used for whitespace. Use null or empty string
for no whitespace characters
addQuoteChars
public void addQuoteChars(char beginQuote,
char endQuote)
- Add the quote pair beginQuote-endQuote to the pairs recognized as char pairs
to quote a token.
getCommentChar
public char getCommentChar()
- Returns a character that indicates start of a comment.
setSeparatorChars
public void setSeparatorChars(String newSeparators)
- Set the current set of separators to newSeparators
isEOL
public boolean isEOL(char c)
- Returns true if c is Mac, Unix, or PC EOL character
isEscape
public boolean isEscape(char c)
- Returns true if c is an escape character
isWhitespace
public boolean isWhitespace(char c)
- Returns true if c is a whitespace character
isSeparator
public boolean isSeparator(char c)
- Returns true if c is a separator character
isBeginQuote
public boolean isBeginQuote(char c)
- Returns true if c indicates the start of a quoted token
isEndQuote
public boolean isEndQuote(char c)
- Returns true if c indicates the end of a quoted token
isComment
public boolean isComment(char c)
- Returns true if c indicates the start of a comment
isTokenTerminator
public boolean isTokenTerminator(char c)
- Returns true if c indicates the end of an unquoted token
IE c is a whitespace, separator or comment character
isQuotePair
public boolean isQuotePair(char beginQuote,
char endQuote)
- Returns true if beginQuote and endQuote are matching begin ending
quote characters
requiresEscaping
public boolean requiresEscaping(char c)
- Returns true if c needs to be escaped in a quoted token.
That is if c is a quote character or the escape character
containsEscapeableChar
public boolean containsEscapeableChar(String token)
- Returns true if c needs to be escaped in a quoted token.
That is if c is a quote character or the escape character
escapeToken
public String escapeToken(String token)
- Places escape character before any quote character or the escape character
Returns the modified token
containsTokenTerminator
public boolean containsTokenTerminator(String token)
- Returns true if contains a character that indicates the end of a token.
That is whitespace, comment char or a separator
quoteToken
public String quoteToken(String token)
- Surrounds token with begin-end quote pair
Returns the quoted token
All Packages Class Hierarchy This Package Previous Next Index