Package org.gorpipe.spark

Class GorSparkUtilities

  • public class GorSparkUtilities
extends java.lang.Object

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static void closePySpark()  
      static java.util.Optional<java.lang.String> getJupyterPath()  
      static org.apache.spark.api.python.Py4JServer getPyServer()  
      static int getPyServerPort()  
      static java.lang.String getPyServerSecret()  
      static java.lang.String getSparkGorRedisUri()  
      static org.apache.spark.sql.SparkSession getSparkSession()  
      static void initPySpark​(java.util.Optional<java.lang.String> standaloneRoot)  
      static java.util.List<org.apache.spark.sql.Row> stream2SparkRowList​(java.util.stream.Stream<org.gorpipe.gor.model.Row> str, org.apache.spark.sql.types.StructType schema)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • getPyServer

        public static org.apache.spark.api.python.Py4JServer getPyServer()

      • getPyServerPort

        public static int getPyServerPort()

      • getPyServerSecret

        public static java.lang.String getPyServerSecret()

      • getJupyterPath

        public static java.util.Optional<java.lang.String> getJupyterPath()

      • closePySpark

        public static void closePySpark()

      • initPySpark

        public static void initPySpark​(java.util.Optional<java.lang.String> standaloneRoot)

      • getSparkGorRedisUri

        public static java.lang.String getSparkGorRedisUri()

      • getSparkSession

        public static org.apache.spark.sql.SparkSession getSparkSession()

      • stream2SparkRowList

        public static java.util.List<org.apache.spark.sql.Row> stream2SparkRowList​(java.util.stream.Stream<org.gorpipe.gor.model.Row> str,
                                                                           org.apache.spark.sql.types.StructType schema)