Splits given text to elements (tokens).
<tokenize delimiters="delimiters"
trimtokens="trimtokens"
allowemptytokens="allowemptytokens">
content to tokenize
</tokenize>
| Name | Required | Default | Description |
|---|---|---|---|
| delimiters | no | new line character | Tells which characters are used as token delimiters. |
| trimtokens | no | yes | Tells whether to trim resulting tokens. |
| allowemptytokens | no | no | Tells whether to include empty tokens in the resulting list (consisting only of whitespaces). |