Package sklearn.feature_extraction.text

Class Tokenizer

java.lang.Object

java.util.AbstractMap<K,V>

java.util.HashMap<String,Object>

net.razorvine.pickle.objects.ClassDict

org.jpmml.python.PythonObject

sklearn.feature_extraction.text.Tokenizer

All Implemented Interfaces:

collections.Callable, Serializable, Cloneable, Map<String,Object>

Direct Known Subclasses:

Matcher, Splitter

public abstract class Tokenizer
extends org.jpmml.python.PythonObject
implements collections.Callable

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class java.util.AbstractMap

AbstractMap.SimpleEntry<K extends Object,V extends Object>, AbstractMap.SimpleImmutableEntry<K extends Object,V extends Object>

Nested classes/interfaces inherited from interface java.util.Map

Map.Entry<K extends Object,V extends Object>

Constructor Summary

Constructors

Constructor	Description
Tokenizer(String module, String name)

Method Summary

Modifier and Type	Method	Description
abstract org.dmg.pmml.TextIndex	configure(org.dmg.pmml.TextIndex textIndex)
abstract String	formatStopWordsRE(List<String> stopWords)

Methods inherited from class org.jpmml.python.PythonObject

get, getArray, getArray, getArray, getArrayShape, getArrayShape, getBoolean, getBooleanArray, getClassName, getDict, getInteger, getIntegerArray, getList, getList, getListLike, getListLike, getNumber, getNumberArray, getObject, getOptional, getOptionalBoolean, getOptionalObject, getOptionalScalar, getOptionalString, getPythonModule, getPythonName, getPythonObject, getScalar, getString, getTuple, getTupleList, setClassName

Methods inherited from class net.razorvine.pickle.objects.ClassDict

__setstate__

Methods inherited from class java.util.HashMap

clear, clone, compute, computeIfAbsent, computeIfPresent, containsKey, containsValue, entrySet, forEach, get, getOrDefault, isEmpty, keySet, merge, put, putAll, putIfAbsent, remove, remove, replace, replace, replaceAll, size, values

Methods inherited from class java.util.AbstractMap

equals, hashCode, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map

equals, hashCode

Constructor Details

Tokenizer

public Tokenizer(String module,
 String name)

Method Details

configure

public abstract org.dmg.pmml.TextIndex configure(org.dmg.pmml.TextIndex textIndex)

formatStopWordsRE

public abstract String formatStopWordsRE(List<String> stopWords)