public interface Token extends TokenTypes
A Token
is a piece of text representing some logical token in
source code for a programming language. For example, the line of C code:
int i = 0;would be broken into 8
Token
s: the first representing
int
, the second whitespace, the third i
, the fourth
whitespace, the fifth =
, etc.
Note: The tokens returned by RSyntaxDocument
s are pooled and
should always be treated as immutable. Modifying tokens you did not create
yourself can and will result in rendering issues and/or runtime exceptions.
You have been warned!
ANNOTATION, COMMENT_DOCUMENTATION, COMMENT_EOL, COMMENT_KEYWORD, COMMENT_MARKUP, COMMENT_MULTILINE, DATA_TYPE, DEFAULT_NUM_TOKEN_TYPES, ERROR_CHAR, ERROR_IDENTIFIER, ERROR_NUMBER_FORMAT, ERROR_STRING_DOUBLE, FUNCTION, IDENTIFIER, LITERAL_BACKQUOTE, LITERAL_BOOLEAN, LITERAL_CHAR, LITERAL_NUMBER_DECIMAL_INT, LITERAL_NUMBER_FLOAT, LITERAL_NUMBER_HEXADECIMAL, LITERAL_STRING_DOUBLE_QUOTE, MARKUP_CDATA, MARKUP_CDATA_DELIMITER, MARKUP_COMMENT, MARKUP_DTD, MARKUP_ENTITY_REFERENCE, MARKUP_PROCESSING_INSTRUCTION, MARKUP_TAG_ATTRIBUTE, MARKUP_TAG_ATTRIBUTE_VALUE, MARKUP_TAG_DELIMITER, MARKUP_TAG_NAME, NULL, OPERATOR, PREPROCESSOR, REGEX, RESERVED_WORD, RESERVED_WORD_2, SEPARATOR, VARIABLE, WHITESPACE
限定符和类型 | 方法和说明 |
---|---|
java.lang.StringBuilder |
appendHTMLRepresentation(java.lang.StringBuilder sb,
RSyntaxTextArea textArea,
boolean fontFamily)
Appends HTML code for painting this token, using the given text area's
color scheme.
|
java.lang.StringBuilder |
appendHTMLRepresentation(java.lang.StringBuilder sb,
RSyntaxTextArea textArea,
boolean fontFamily,
boolean tabsToSpaces)
Appends HTML code for painting this token, using the given text area's
color scheme.
|
char |
charAt(int index)
Returns the character at the specified offset in the token.
|
boolean |
containsPosition(int pos)
Returns whether the token straddles the specified position in the
document.
|
int |
documentToToken(int pos)
Returns the position in the token's internal char array corresponding
to the specified document position.
|
boolean |
endsWith(char[] ch)
Returns whether this token's lexeme ends with the specified characters.
|
int |
getEndOffset()
Returns the end offset of this token in the document (exclusive).
|
java.lang.String |
getHTMLRepresentation(RSyntaxTextArea textArea)
Returns a
String containing HTML code for painting this
token, using the given text area's color scheme. |
int |
getLanguageIndex()
Returns the language index of this token.
|
Token |
getLastNonCommentNonWhitespaceToken()
Returns the last token in this list that is not whitespace or a
comment.
|
Token |
getLastPaintableToken()
Returns the last paintable token in this token list, or
null
if there is no paintable token. |
java.lang.String |
getLexeme()
Returns the text of this token, as a string.
|
int |
getListOffset(RSyntaxTextArea textArea,
javax.swing.text.TabExpander e,
float x0,
float x)
Determines the offset into this token list (i.e., into the
document) that covers pixel location
x if the token list
starts at pixel location x0 . |
Token |
getNextToken()
Returns the token after this one in the linked list.
|
int |
getOffset()
Returns the offset into the document at which this token resides.
|
int |
getOffsetBeforeX(RSyntaxTextArea textArea,
javax.swing.text.TabExpander e,
float startX,
float endBeforeX)
Returns the position in the document that represents the last character
in the token that will fit into
endBeforeX-startX pixels. |
char[] |
getTextArray()
Returns the character array backing the lexeme of this token.
|
int |
getTextOffset()
Returns the offset into the character array of the lexeme in
getTextArray() . |
int |
getType()
Returns the type of this token.
|
float |
getWidth(RSyntaxTextArea textArea,
javax.swing.text.TabExpander e,
float x0)
Returns the width of this token given the specified parameters.
|
float |
getWidthUpTo(int numChars,
RSyntaxTextArea textArea,
javax.swing.text.TabExpander e,
float x0)
Returns the width of a specified number of characters in this token.
|
boolean |
is(char[] lexeme)
Returns whether this token's lexeme matches a specific character array.
|
boolean |
is(int type,
char[] lexeme)
Returns whether this token is of the specified type, with the specified
lexeme.
|
boolean |
is(int type,
java.lang.String lexeme)
Returns whether this token is of the specified type, with the specified
lexeme.
|
boolean |
isComment()
Returns whether this token is a comment.
|
boolean |
isCommentOrWhitespace()
Returns whether this token is a comment or whitespace.
|
boolean |
isHyperlink()
Returns whether this token is a hyperlink.
|
boolean |
isIdentifier()
Returns whether this token is an identifier.
|
boolean |
isLeftCurly()
Returns whether this token is a
TokenTypes.SEPARATOR representing a single
left curly brace. |
boolean |
isPaintable()
Returns whether or not this token is "paintable;" i.e., whether or not
the type of this token is one such that it has an associated syntax
style.
|
boolean |
isRightCurly()
Returns whether this token is a
TokenTypes.SEPARATOR representing a single
right curly brace. |
boolean |
isSingleChar(char ch)
Returns whether this token is the specified single character.
|
boolean |
isSingleChar(int type,
char ch)
Returns whether this token is the specified single character, and of a
specific type.
|
boolean |
isWhitespace()
Returns whether or not this token is whitespace.
|
int |
length()
Returns the length of this token.
|
java.awt.Rectangle |
listOffsetToView(RSyntaxTextArea textArea,
javax.swing.text.TabExpander e,
int pos,
int x0,
java.awt.Rectangle rect)
Returns the bounding box for the specified document location.
|
void |
setHyperlink(boolean hyperlink)
Sets whether this token is a hyperlink.
|
void |
setLanguageIndex(int languageIndex)
Sets the language index for this token.
|
void |
setType(int type)
Sets the type of this token.
|
boolean |
startsWith(char[] chars)
Returns whether this token starts with the specified characters.
|
int |
tokenToDocument(int pos)
Returns the position in the document corresponding to the specified
position in this token's internal char array (
textOffset -
textOffset+textCount-1 ). |
java.lang.StringBuilder appendHTMLRepresentation(java.lang.StringBuilder sb, RSyntaxTextArea textArea, boolean fontFamily)
sb
- The buffer to append to.textArea
- The text area whose color scheme to use.fontFamily
- Whether to include the font family in the HTML for
this token. You can pass false
for this parameter
if, for example, you are making all your HTML be monospaced,
and don't want any crazy fonts being used in the editor to be
reflected in your HTML.getHTMLRepresentation(RSyntaxTextArea)
java.lang.StringBuilder appendHTMLRepresentation(java.lang.StringBuilder sb, RSyntaxTextArea textArea, boolean fontFamily, boolean tabsToSpaces)
sb
- The buffer to append to.textArea
- The text area whose color scheme to use.fontFamily
- Whether to include the font family in the HTML for
this token. You can pass false
for this parameter
if, for example, you are making all your HTML be monospaced,
and don't want any crazy fonts being used in the editor to be
reflected in your HTML.tabsToSpaces
- Whether to convert tabs into spaces.getHTMLRepresentation(RSyntaxTextArea)
char charAt(int index)
boolean containsPosition(int pos)
pos
- The position in the document to check.int documentToToken(int pos)
Note that this method does NOT do any bounds checking; you can pass in a document position that does not correspond to a position in this token, and you will not receive an Exception or any other notification; it is up to the caller to ensure valid input.
pos
- A position in the document that is represented by this token.textOffset
and
< textOffset+textCount
.tokenToDocument(int)
boolean endsWith(char[] ch)
ch
- The characters.startsWith(char[])
int getEndOffset()
[getOffset(), getEndOffset())
.getOffset()
java.lang.String getHTMLRepresentation(RSyntaxTextArea textArea)
String
containing HTML code for painting this
token, using the given text area's color scheme.textArea
- The text area whose color scheme to use.appendHTMLRepresentation(StringBuilder, RSyntaxTextArea, boolean)
int getLanguageIndex()
0
denotes the
"main" language, any positive value denotes a specific secondary
language.setLanguageIndex(int)
Token getLastNonCommentNonWhitespaceToken()
null
if there isn't one.Token getLastPaintableToken()
null
if there is no paintable token.java.lang.String getLexeme()
Note that this method isn't used much by the
ryntaxtextarea
package internally, as it tries to limit
memory allocation.
int getListOffset(RSyntaxTextArea textArea, javax.swing.text.TabExpander e, float x0, float x)
x
if the token list
starts at pixel location x0
.
This method will return the document position "closest" to the
x-coordinate (i.e., if they click on the "right-half" of the
w
in awe
, the caret will be placed in
between the w
and e
; similarly, clicking on
the left-half places the caret between the a
and
w
). This makes it useful for methods such as
viewToModel
found in javax.swing.text.View
subclasses.
textArea
- The text area from which the token list was derived.e
- How to expand tabs.x0
- The pixel x-location that is the beginning of
tokenList
.x
- The pixel-position for which you want to get the corresponding
offset.tokenList
is
null
or has type Token.NULL
, then
-1
Token getNextToken()
int getOffset()
getEndOffset()
int getOffsetBeforeX(RSyntaxTextArea textArea, javax.swing.text.TabExpander e, float startX, float endBeforeX)
endBeforeX-startX
pixels.
For example, if you're using a monospaced 8-pixel-per-character font,
have the token "while" and startX
is 0
and
endBeforeX
is 30
, this method will return the
document position of the "i" in "while", because the "i" ends at pixel
24
, while the "l" ends at 32
. If not even the
first character fits in endBeforeX-startX
, the first
character's position is still returned so calling methods don't go into
infinite loops.textArea
- The text area in which this token is being painted.e
- How to expand tabs.startX
- The x-coordinate at which the token will be painted. This
is needed because of tabs.endBeforeX
- The x-coordinate for which you want to find the last
character of t
which comes before it.char[] getTextArray()
getTextOffset()
,
length()
int getTextOffset()
getTextArray()
.getTextArray()
int getType()
TokenTypes
,
setType(int)
float getWidth(RSyntaxTextArea textArea, javax.swing.text.TabExpander e, float x0)
textArea
- The text area in which the token is being painted.e
- Describes how to expand tabs. This parameter cannot be
null
.x0
- The pixel-location at which the token begins. This is needed
because of tabs.getWidthUpTo(int, com.fr.design.gui.syntax.ui.rsyntaxtextarea.RSyntaxTextArea, javax.swing.text.TabExpander, float)
float getWidthUpTo(int numChars, RSyntaxTextArea textArea, javax.swing.text.TabExpander e, float x0)
3
here returns the width of the "whi" portion of the token.numChars
- The number of characters for which to get the width.textArea
- The text area in which the token is being painted.e
- How to expand tabs. This value cannot be null
.x0
- The pixel-location at which this token begins. This is needed
because of tabs.getWidth(com.fr.design.gui.syntax.ui.rsyntaxtextarea.RSyntaxTextArea, javax.swing.text.TabExpander, float)
boolean is(char[] lexeme)
lexeme
- The lexeme to check for.is(int, char[])
,
is(int, String)
,
isSingleChar(int, char)
,
startsWith(char[])
boolean is(int type, char[] lexeme)
This method is preferred over the other overload in performance-critical code where this operation may be called frequently, since it does not involve any String allocations.
type
- The type to check for.lexeme
- The lexeme to check for.is(int, String)
,
is(char[])
,
isSingleChar(int, char)
,
startsWith(char[])
boolean is(int type, java.lang.String lexeme)
The other overload of this method is preferred over this one in performance-critical code, as this one involves a String allocation while the other does not.
type
- The type to check for.lexeme
- The lexeme to check for.is(int, char[])
,
isSingleChar(int, char)
,
startsWith(char[])
boolean isComment()
isWhitespace()
,
isCommentOrWhitespace()
boolean isCommentOrWhitespace()
isComment()
,
isWhitespace()
boolean isHyperlink()
setHyperlink(boolean)
boolean isIdentifier()
boolean isLeftCurly()
TokenTypes.SEPARATOR
representing a single
left curly brace.isRightCurly()
boolean isRightCurly()
TokenTypes.SEPARATOR
representing a single
right curly brace.isLeftCurly()
boolean isPaintable()
Token.NULL
.boolean isSingleChar(char ch)
ch
- The character to check for.isSingleChar(int, char)
boolean isSingleChar(int type, char ch)
type
- The token type.ch
- The character to check for.isSingleChar(char)
boolean isWhitespace()
true
iff this token is whitespace.isComment()
,
isCommentOrWhitespace()
int length()
getOffset()
java.awt.Rectangle listOffsetToView(RSyntaxTextArea textArea, javax.swing.text.TabExpander e, int pos, int x0, java.awt.Rectangle rect)
null
is returned.textArea
- The text area from which the token list was derived.e
- How to expand tabs.pos
- The position in the document for which to get the bounding
box in the view.x0
- The pixel x-location that is the beginning of
tokenList
.rect
- The rectangle in which we'll be returning the results. This
object is reused to keep from frequent memory allocations.void setHyperlink(boolean hyperlink)
hyperlink
- Whether this token is a hyperlink.isHyperlink()
void setLanguageIndex(int languageIndex)
0
, this token is in the "main" language being edited.
Negative values are invalid and treated as 0
.languageIndex
- The new language index. A value of
0
denotes the "main" language, any positive value
denotes a specific secondary language. Negative values will
be treated as 0
.getLanguageIndex()
void setType(int type)
type
- The new token type.TokenTypes
,
getType()
boolean startsWith(char[] chars)
chars
- The characters.endsWith(char[])
,
is(int, char[])
int tokenToDocument(int pos)
textOffset
-
textOffset+textCount-1
).Note that this method does NOT do any bounds checking; you can pass in an invalid token position, and you will not receive an Exception or any other indication that the returned document position is invalid. It is up to the user to ensure valid input.
pos
- A position in the token's internal char array
(textOffset
- textOffset+textCount
).documentToToken(int)