Search Java Classes and Packages
#Opennlp.tools.tokenize Classes and Interfaces - 22 results found.
| Name | Description | Type | Package | Framework |
| DefaultTokenContextGenerator | Generate events for maxent decisions for tokenization. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| DetokenizationDictionary | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) | |
| DetokenizationDictionary .Operation | Attaches the token to the token on the left and right sides. | Class | opennlp.tools.tokenize.DetokenizationDictionary | Natural Language Processing (OpenNLP) |
| Detokenizer | A Detokenizer merges tokens back to their untokenized representation. | Interface | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| Detokenizer .DetokenizationOperation | This enum contains an operation for every token to merge the tokens together to their detokenized form. | Class | opennlp.tools.tokenize.Detokenizer | Natural Language Processing (OpenNLP) |
| DictionaryDetokenizer | A rule based detokenizer. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| Factory | Class | opennlp.tools.tokenize.lang | Natural Language Processing (OpenNLP) | |
| SimpleTokenizer | Performs tokenization using character classes. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokenContextGenerator | Interface for TokenizerME context generators. | Interface | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| Tokenizer | The interface for tokenizers, which segment a string into its tokens. | Interface | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokenizerCrossValidator | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) | |
| TokenizerEvaluator | The TokenizerEvaluator measures the performance of the given Tokenizer with the provided reference | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokenizerFactory | The factory that provides Tokenizer default implementations and resources. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokenizerME | A Tokenizer for converting raw text into separated tokens. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokenizerModel | The TokenizerModel is the model used by a learnable Tokenizer. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokenizerStream | The TokenizerStream uses a tokenizer to tokenize the input string and output TokenSamples. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokenSample | A TokenSample is text with token spans. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokenSampleStream | Class which produces an Iterator | Class | opennlp.tools.tokenize.lang.en | Natural Language Processing (OpenNLP) |
| TokenSampleStream | This class is a stream filter which reads in string encoded samples and creates TokenSamples out of them. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| TokSpanEventStream | This class reads the TokenSamples from the given Iterator and converts the TokenSamples into Events which | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| WhitespaceTokenizer | This tokenizer uses white spaces to tokenize the input text. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |
| WhitespaceTokenStream | This stream formats a TokenSamples into whitespace separated token strings. | Class | opennlp.tools.tokenize | Natural Language Processing (OpenNLP) |