sklearn.feature_extraction.text

Class Tokenizer

java.lang.Object

java.util.AbstractMap<K,V>

java.util.HashMap<String,Object>

net.razorvine.pickle.objects.ClassDict

org.jpmml.python.PythonObject

sklearn.feature_extraction.text.Tokenizer

All Implemented Interfaces:

collections.Callable, Serializable, Cloneable, Map<String,Object>

Direct Known Subclasses:

Matcher, Splitter

public abstract class Tokenizer
extends org.jpmml.python.PythonObject
implements collections.Callable

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class java.util.AbstractMap

AbstractMap.SimpleEntry<K,V>, AbstractMap.SimpleImmutableEntry<K,V>

Nested classes/interfaces inherited from interface java.util.Map

Map.Entry<K,V>

Constructor Summary

Constructors

Constructor and Description
Tokenizer(String module, String name)

Method Summary

Modifier and Type	Method and Description
abstract org.dmg.pmml.TextIndex	configure(org.dmg.pmml.TextIndex textIndex)
abstract String	formatStopWordsRE(List<String> stopWords)

Methods inherited from class org.jpmml.python.PythonObject

get, getArray, getArray, getArray, getArrayShape, getArrayShape, getBoolean, getBooleanArray, getClassName, getDict, getInteger, getIntegerArray, getList, getList, getListLike, getListLike, getNumber, getNumberArray, getObject, getOptional, getOptionalBoolean, getOptionalBoolean, getOptionalInteger, getOptionalNumber, getOptionalObject, getOptionalScalar, getOptionalString, getPythonModule, getPythonName, getPythonObject, getScalar, getString, getStringArray, getTuple, getTupleList, setClassName

Methods inherited from class net.razorvine.pickle.objects.ClassDict

__setstate__

Methods inherited from class java.util.HashMap

clear, clone, compute, computeIfAbsent, computeIfPresent, containsKey, containsValue, entrySet, forEach, get, getOrDefault, isEmpty, keySet, merge, put, putAll, putIfAbsent, remove, remove, replace, replace, replaceAll, size, values

Methods inherited from class java.util.AbstractMap

equals, hashCode, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map

equals, hashCode

Constructor Detail

Tokenizer

public Tokenizer(String module,
                 String name)

Method Detail

configure

public abstract org.dmg.pmml.TextIndex configure(org.dmg.pmml.TextIndex textIndex)

formatStopWordsRE

public abstract String formatStopWordsRE(List<String> stopWords)