lexer
美 
英
例句
The lexer actually does some of the work of figuring out where in a message it is, but the parser still ties everything together.
lexer實際上完成了指出它位于消息中的哪個地方的一些任務，但是解析器仍將所有內容放在一起。
Additionally there is implemented a IDL lexer, which allows to use the preprocessor library as the lexing component of a IDL oriented tool.
此外還實現了一個IDLlexer，允許將預處理器庫用作某個面向IDL的工具的lexing組件。
Whenever possible, try to develop separate tests for your lexer and your parser.
只要有可能，請嘗試為您的lexer和解析器開發單獨的測試。
The lexer for this example is more complicated than the one for the calculator; quite a bit so.
該示例中的lexer比計算器的更為復雜；復雜得多。
So this lexer does not recognize any keywords (except true and false, which are needed by the preprocessor itself).
因此，這個lexer不識別任何關鍵字（除了true和false，因為預處理器本身需要它們）。
A lexer is a software component that divides text strings into individual words, or tokens, so that the individual words can be indexed.
lexer是一個能將文本字符串分成單個命令，或者標記的軟件組分，這樣使得單個命令可以被檢索到。
On the basis of the test tool of executing characteristics of programs, we modify the lexer and parser to implement it.
該工具是在程序執行特征測試工具的基礎上，將其詞法分析器和語法分析器加以修改而實現的。
These directives are used to define the tokens the lexer can return.
這些指令用來定義lexer可以返回的記號。
The lexer is the part of our language knowledge that says "this is a sentence; this is punctuation; twenty-three is a single word. "
詞法分析器是我們的語言知識中識別“這是一個句子；這是標點；twenty-three是一個單一的詞”的那一部分。
The symbols produced by the lexer are called terminals or tokens.
lexer生成的符號稱為終結符（terminals）或者記號（tokens）。
JAPANESE_LEXER: A lexer for extracting tokens from Japanese text.
JAPANESE_LEXER：一個從Japanese文本在中提取標記的lexer。
The C++ lexers packaged with the Wave library may be used standalone, too, and are not tied to the C++ preprocessor iterator at all.
與Wave庫一起打包的C++lexer也可以單獨使用，并沒有與C++預處理迭代器捆綁在一起。
So, in this example, NUMBER is a terminal; the lexer is producing this result.
所以，在這個例子中，NUMBER是一個終結符；lexer正是生成這種結果。
KOREAN_MORPH_LEXER: A lexer for extracting tokens from Korean text.
KOREAN_MORPH_LEXER：一個從Korean文本在中提取標記的lexer。
The tokens ALPHA, INT, and REAL are typically provided from the lexer.
INT和REAL通常由lexer提供。
KOREAN_LEXER: A lexer for extracting tokens from Korean text.
KOREAN_LEXER：一個從Korean文本在中提取標記的lexer。
This does mean that the lexer and the parser both have to know that an empty line separates a header from a body.
這確實意味著lexer和解析器都必須明白，是一個空行將標題從主體中分離了出來。
A listing of supported lexer types follows.
下面是一個支持lexer類型的清單。
JAPANESE_VGRAM: A lexer for extracting tokens from Japanese text.
JAPANESE_VGRAM：一個從Japanese文本在中提取標記的lexer。
Most simple programming projects, of course, can get by with very trivial lexers.
當然，大部分簡單編程項目使用非常普通的lexer就可以完成。
CHINESE_VGRAM: A lexer for extracting tokens from Chinese text.
CHINESE_VGRAM：一個從Chinese文本在中提取標記的lexer。
Thus, P: : RD on top of Perl 5 is a powerful parser and lexer combination.
因此，Perl5之上的P：：RD是一個解析器與詞法分析器的強大組合。
A good lexer example can help a lot with learning how to write a tokenizer.
一個好的lexer例子會非常有助于學習如何編寫斷詞器（tokenizer）。
