Package sklearn.feature_extraction.text

Class Tokenizer

  • All Implemented Interfaces:
    collections.Callable, java.io.Serializable, java.lang.Cloneable, java.util.Map<java.lang.String,​java.lang.Object>
    Direct Known Subclasses:
    Matcher, Splitter
    public abstract class Tokenizer
extends org.jpmml.python.PythonObject
implements collections.Callable
    See Also:
    Serialized Form

    • Nested Class Summary

      • Nested classes/interfaces inherited from class java.util.AbstractMap

        java.util.AbstractMap.SimpleEntry<K extends java.lang.Object,​V extends java.lang.Object>, java.util.AbstractMap.SimpleImmutableEntry<K extends java.lang.Object,​V extends java.lang.Object>

      • Nested classes/interfaces inherited from interface java.util.Map

        java.util.Map.Entry<K extends java.lang.Object,​V extends java.lang.Object>

    • Constructor Summary

      Constructors 
      Constructor Description
      Tokenizer​(java.lang.String module, java.lang.String name)  

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      abstract org.dmg.pmml.TextIndex configure​(org.dmg.pmml.TextIndex textIndex)  
      abstract java.lang.String formatStopWordsRE​(java.util.List<java.lang.String> stopWords)  

      • Methods inherited from class org.jpmml.python.PythonObject

        containsKey, delattr, get, get, getArray, getArray, getArray, getArrayList, getArrayList, getArrayShape, getArrayShape, getattr, getattr, getBoolean, getBooleanArray, getClassName, getDict, getEnum, getEnumList, getIdentifiable, getInteger, getIntegerArray, getList, getList, getListLike, getListLike, getNumber, getNumberArray, getObject, getObjectArray, getObjectList, getOptional, getOptionalBoolean, getOptionalBoolean, getOptionalDict, getOptionalEnum, getOptionalIdentifiable, getOptionalInteger, getOptionalNumber, getOptionalObject, getOptionalScalar, getOptionalString, getOptionalTuple, getPythonModule, getPythonName, getScalar, getString, getStringArray, getStringList, getTuple, getTupleList, hasattr, put, putAll, remove, setattr, setClassName, update

      • Methods inherited from class net.razorvine.pickle.objects.ClassDict

        __setstate__

      • Methods inherited from class java.util.HashMap

        clear, clone, compute, computeIfAbsent, computeIfPresent, containsValue, entrySet, forEach, getOrDefault, isEmpty, keySet, merge, putIfAbsent, remove, replace, replace, replaceAll, size, values

      • Methods inherited from class java.util.AbstractMap

        equals, hashCode, toString

      • Methods inherited from class java.lang.Object

        getClass, notify, notifyAll, wait, wait, wait

      • Methods inherited from interface java.util.Map

        equals, hashCode

    • Constructor Detail

      • Tokenizer

        public Tokenizer​(java.lang.String module,
                 java.lang.String name)

    • Method Detail

      • configure

        public abstract org.dmg.pmml.TextIndex configure​(org.dmg.pmml.TextIndex textIndex)

      • formatStopWordsRE

        public abstract java.lang.String formatStopWordsRE​(java.util.List<java.lang.String> stopWords)