tokenized

美 　英

• 網(wǎng)絡(luò)標(biāo)記化的；詞法分析；符號(hào)化

例句

The contents of this file are tokenized into a list of strings and bound to a list called Rules.

文件的內(nèi)容被分割放入一個(gè)字符串列表，然后綁定到名為Rules的變量。

Should be specified as the string of hexadecimal digits representing the tokenized form of the public key.

應(yīng)指定為十六進(jìn)制數(shù)字的字符串以表示標(biāo)記形式的公鑰。

In some specific cases , changes made to the word breakers have the potential to impact how some data is tokenized .

在某些特定情況下，對(duì)斷字符所做的更改可能會(huì)影響對(duì)某些數(shù)據(jù)進(jìn)行標(biāo)記化的方式。

Erlang also gives us programmatic access to the tokenized, parsed and abstract forms of an expression.

Erlang還允許我們編程訪問經(jīng)過詞法分析（tokenized）、語法分析之后、抽象形式的表達(dá)式。

The identified messages are tokenized , classified, and aggregated and changes in the frequency of tokenized messages are identified.

識(shí)別過的消息被做標(biāo)記、分類和聚類處理，消息中標(biāo)記的頻率變化也被識(shí)別出來。

These considerations relate to how your text is tokenized and then indexed by Full-Text Engine.

這些注意事項(xiàng)均與全文引擎如何對(duì)文本進(jìn)行詞匯切分再編制其索引有關(guān)。

A display of the tokenized messages are identified.

標(biāo)記消息通過用戶界面顯示。