Package org.gorpipe.spark.redis

Class RedisBatchConsumer

  • All Implemented Interfaces:
    java.io.Serializable, java.lang.AutoCloseable, org.apache.spark.api.java.function.VoidFunction2<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>,​java.lang.Long>
    public class RedisBatchConsumer
extends java.lang.Object
implements org.apache.spark.api.java.function.VoidFunction2<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>,​java.lang.Long>, java.lang.AutoCloseable
    See Also:
    Serialized Form

    • Constructor Summary

      Constructors 
      Constructor Description
      RedisBatchConsumer​(org.apache.spark.sql.SparkSession sparkSession, java.lang.String redisUri)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void call​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> v1, java.lang.Long v2)  
      void close()  
      void runGorJobs​(java.lang.String projectDirStr, java.util.Set<java.lang.Integer> gorJobs, java.lang.String[] queries, java.lang.String[] fingerprints, java.lang.String[] jobIds, java.lang.String[] cachefiles)  
      void runSparkJob​(java.lang.String projectDirStr, java.lang.String cmd, java.lang.String jobId, java.lang.String cacheFile)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • RedisBatchConsumer

        public RedisBatchConsumer​(org.apache.spark.sql.SparkSession sparkSession,
                          java.lang.String redisUri)

    • Method Detail

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable

      • runGorJobs

        public void runGorJobs​(java.lang.String projectDirStr,
                       java.util.Set<java.lang.Integer> gorJobs,
                       java.lang.String[] queries,
                       java.lang.String[] fingerprints,
                       java.lang.String[] jobIds,
                       java.lang.String[] cachefiles)

      • runSparkJob

        public void runSparkJob​(java.lang.String projectDirStr,
                        java.lang.String cmd,
                        java.lang.String jobId,
                        java.lang.String cacheFile)

      • call

        public void call​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> v1,
                 java.lang.Long v2)
        Specified by:
        call in interface org.apache.spark.api.java.function.VoidFunction2<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>,​java.lang.Long>