Install Intellij & run first Spark program in windows | Configure spark in Intellij

Поділитися
Вставка
  • Опубліковано 2 жов 2024
  • If you are looking for spark training call: +91-8500002025 or join t.me/SparkTrai... telegram
    In this video explains how to install IntelliJ and configure spark in IntelliJ and how to execute the sample spark program explained in this video.
    if you want to install spark and Hadoop follow this link • Latest Hadoop 3.2.2 Sp...
    Download IntelliJ from this link.
    www.jetbrains....
    Paste this code in intellin . File.. Setting..Editor ... File & Templates .. Scala object ... replace with given code
    #if ((${PACKAGE_NAME} && ${PACKAGE_NAME} != ""))package ${PACKAGE_NAME} #end
    #parse("File Header.java")
    import org.apache.spark.sql._
    import org.apache.spark.sql.functions._
    object ${NAME} {
    def main(args: Array[String]) {
    val spark = SparkSession.builder.master("local[*]").appName("${NAME}").getOrCreate()
    // val ssc = new StreamingContext(spark.sparkContext, Seconds(10))
    val sc = spark.sparkContext
    sc.setLogLevel("ERROR")
    import spark.implicits._
    import spark.sql
    spark.stop()
    }
    }

КОМЕНТАРІ • 10