Package org.gorpipe.spark

Class GeneralSparkQueryHandler

java.lang.Object
org.gorpipe.spark.GeneralSparkQueryHandler
All Implemented Interfaces:
org.gorpipe.gor.model.GorParallelQueryHandler
public class GeneralSparkQueryHandler
extends java.lang.Object
implements org.gorpipe.gor.model.GorParallelQueryHandler

  • Field Summary

    Fields 
    Modifier and Type Field Description
    static java.lang.String queue  

  • Constructor Summary

    Constructors 
    Constructor Description
    GeneralSparkQueryHandler​(org.gorpipe.spark.GorSparkSession gorPipeSession, java.lang.String sparkRedisUri)  

  • Method Summary

    Modifier and Type Method Description
    java.lang.String[] executeBatch​(java.lang.String[] fingerprints, java.lang.String[] commandsToExecute, java.lang.String[] batchGroupNames, org.gorpipe.gor.monitor.GorMonitor mon)  
    static java.lang.String[] executeSparkBatch​(org.gorpipe.spark.GorSparkSession session, java.lang.String projectDir, java.lang.String cacheDir, java.lang.String[] fingerprints, java.lang.String[] commandsToExecute, java.lang.String[] jobIds, java.lang.String[] cacheFiles)  
    long getWaitTime()  
    void init​(org.gorpipe.spark.GorSparkSession gorPipeSession)  
    void setCluster​(GorClusterBase cluster)  
    void setForce​(boolean force)  
    void setQueryTime​(java.lang.Long time)  

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

    • GeneralSparkQueryHandler

      public GeneralSparkQueryHandler​(org.gorpipe.spark.GorSparkSession gorPipeSession, java.lang.String sparkRedisUri)

  • Method Details

    • setCluster

      public void setCluster​(GorClusterBase cluster)

    • init

      public void init​(org.gorpipe.spark.GorSparkSession gorPipeSession)

    • executeSparkBatch

      public static java.lang.String[] executeSparkBatch​(org.gorpipe.spark.GorSparkSession session, java.lang.String projectDir, java.lang.String cacheDir, java.lang.String[] fingerprints, java.lang.String[] commandsToExecute, java.lang.String[] jobIds, java.lang.String[] cacheFiles)

    • executeBatch

      public java.lang.String[] executeBatch​(java.lang.String[] fingerprints, java.lang.String[] commandsToExecute, java.lang.String[] batchGroupNames, org.gorpipe.gor.monitor.GorMonitor mon)
      Specified by:
      executeBatch in interface org.gorpipe.gor.model.GorParallelQueryHandler

    • setForce

      public void setForce​(boolean force)
      Specified by:
      setForce in interface org.gorpipe.gor.model.GorParallelQueryHandler

    • setQueryTime

      public void setQueryTime​(java.lang.Long time)
      Specified by:
      setQueryTime in interface org.gorpipe.gor.model.GorParallelQueryHandler

    • getWaitTime

      public long getWaitTime()
      Specified by:
      getWaitTime in interface org.gorpipe.gor.model.GorParallelQueryHandler