Відео

Please remove background sound

Hello sir meru reel peduthunnaru koddhiga explain cheyocchuga kadha sir

Hello Abhiram Sir ... Today's GCP interview question?

I believe you can read documentation very well rather than understanding it

what is the age criteria and career criteria for Jobs after getting IICS Training. Please Give Genuine Answer, is it possible to at the age in 38.

Music 😅

may i have your contact to have training on pyspark

may i have your contact to have training on pyspark

may i have your contact to have training on pyspark

Method: 1 Reading from CSV file from pyspark.sql import SparkSession spark = SparkSession.builder.appName("DataFrame creation example").getOrCreate() df = spark.read.csv("file_path.csv", header=True, inferSchema=True Method: 2 Reading from JSON file from pyspark.sql import SparkSession spark = SparkSession.builder.appName("DataFrame creation example").getOrCreate() df = spark.read.json("file_path.json" Method 3: Reading from Parquet file: from pyspark.sql import SparkSession spark = SparkSession.builder.appName("DataFrame creation example").getOrCreate() df = spark.read.parquet("file_path.parquet" Method 4: Reading from a database from pyspark.sql import SparkSession spark = SparkSession.builder.appName("DataFrame creation example").getOrCreate() jdbc_url = "jdbc:mysql://localhost:3306/mydb" table_name = "my_table" properties = {"user": "my_user", "password": "my_password"} df = spark.read.jdbc(url=jdbc_url, table=table_name, properties=properties)

Syntax: from pyspark.sql import SparkSession # Initialize SparkSession spark = SparkSession.builder.appName("DFSize").getOrCreate() # create DataFrame df = spark.range(1000000) # Force cache DataFrame df.cache().count() # Get size of DataFrame in bytes size_in_bytes = sc._jvm.org.apache.spark.util.SizeEstimator.estimate(df._jdf) # size in megabytes size_estimate_mb = size_estimate_bytes / (1024**2) # size in gigabytes size_estimate_gb = size_estimate_bytes / (1024**3) df.unpersist()

Syntax for Pipelines in PySpark MLlib from pyspark.ml import Pipeline from pyspark.ml.feature import StringIndexer from pyspark.ml.classification import RandomForestClassifier indexer = StringIndexer(inputCol=’category’, outputCol=’categoryIndex’) rf = RandomForestClassifier(featuresCol=’features’, labelCol=’categoryIndex’) pipeline = Pipeline(stages=[indexer, rf]) model = pipeline.fit(trainingData)

Syntax: PySpark’s MLlib for machine learning tasks from pyspark.ml.classification import LogisticRegression from pyspark.ml.feature import VectorAssembler assembler = VectorAssembler(inputCols=[‘feature1’, ‘feature2’], outputCol=’features’) df_transformed = assembler.transform(df) lr = LogisticRegression(featuresCol=’features’, labelCol=’label’) model = lr.fit(df_transformed)

Syntax for Find the top N most frequent words in a large text file from pyspark import SparkContext # create your spark context sc = SparkContext("local", "WordCount") # import a text file from a local path lines = sc.textFile("path/to/your/text/file.txt") # split and map the words # then reduce by using the words as keys and add to the count word_counts = lines.flatMap(lambda line: line.split(" ")) \ .map(lambda word: (word, 1)) \ .reduceByKey(lambda a, b: a + b) # order the words and take only the top N frequent words top_n_words = word_counts.takeOrdered(N, key=lambda x: -x[1]) print(top_n_words)

import pyspark from pyspark.sql import SparkSession from pyspark.sql.functions import expr #Create spark session data = [("Devara",1000,"India"), ("Kalki",1500,"India"), ("Pushpa",1600,"India"), \ ("Devara",4000,"USA"), \ ("Pushpa",1200,"USA"),("Kalki",1500,"USA"), \ ("Pushpa",2000,"Canada"),("Kalki",2000,"Canada"),("Devara",2000,"Mexico")] columns= ["Product","Amount","Country"] df = spark.createDataFrame(data = data, schema = columns) df.printSchema() df.show(truncate=False) Output: root | -- Product: string (nullable = true) | -- Amount: long (nullable = true) | -- Country: string (nullable = true) +-------------+-------------+-------------+ | Product | Amount | Country | +-------------+-------------+--------------+ | Devara | 1000 | India | | Pushpa | 1600 | India | | Kalki | 1500 | India | | Devara | 4000 | USA | | Pushpa | 1200 | USA | | Kalki | 1500 | USA | | Devara | 2000 | Mexico | | Pushpa | 2000 | Canada | | Kalki | 2000 | Canada | +-------------+--------------+--------------+ To determine the entire amount of each product's exports to each nation, we'll group by Product, pivot by Country, and sum by Amount. pivotDF = df.groupBy("Product").pivot("Country").sum("Amount") pivotDF.printSchema() pivotDF.show(truncate=False) This will convert the nations from DataFrame rows to columns, resulting in the output seen below. Output: root | -- Product: string (nullable = true) | -- Canada: long (nullable = true) | -- Mexico: long (nullable = true) | -- USA: long (nullable = true) | -- India: long (nullable = true) +-------------+-------------+------------+-------+---------+ | Product | Canada | Mexico | USA | India | +-------------+-------------+------------+--------+---------+ | Devara | null | 2000 | 4000 |1000 | | Pushpa | 2000 | null | 1200 | 1600 | | Kalki | 2000 | null | 1500 | 1500 | +-------------+-------------+------------+---------+---------+

Syntax kidhr he 😂

Good.. keep it up

Please do not add background music it is too much distracting. Or you can add silent music

Hi Abhiram Could you please let me know how to migrate job from datastage to informatica power center or iics

from pyspark.sql import SparkSession, Row from pyspark.sql import functions as F # Initialize Spark session spark = SparkSession.builder.appName("concatenate_columns").getOrCreate() # Sample DataFrame data = [ Row(struct_col=Row(a=1, b="foo"), array_col=[Row(c=3, d="bar"), Row(c=4, d="baz")]) ] df = spark.createDataFrame(data) # Flatten the struct column flattened_struct_col = F.concat_ws( ",", *[F.col("struct_col." + field.name) for field in df.schema["struct_col"].dataType.fields] ) # Flatten the array of structs column flattened_array_col = F.expr(""" concat_ws(",", transform(array_col, x -> concat_ws(",", x.*))) """) # Concatenate the two columns df = df.withColumn( "concatenated_col", F.concat_ws(",", flattened_struct_col, flattened_array_col) ) # Show result df.show(truncate=False)

Please don't put any background music and give little more time to read the slides

Kindly stop that background music

Pls send data file

Voice rale sir

No voice

Thank you so much. Starting lo nay input data tho dataframe create chesay time lo arguments error vachindhi kadha. Adhi okasari cheppandi anna fix

Nice xplaining sir

Nice explanation sir

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 👉Features of Online Training: 👉Real-Time Oriented Training 👉Live Training Sessions 👉Interview Preparation Tips 👉FAQ’s 👉100% Job Guarantee Program 👉Mock Interviews

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 👉Features of Online Training: 👉 Real-Time Oriented Training 👉Live Training Sessions 👉Interview Preparation Tips 👉FAQ’s 👉100% Job Guarantee Program 👉Mock Interviews

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 👉Features of Online Training: 👉 Real-Time Oriented Training 👉Live Training Sessions 👉Interview Preparation Tips 👉FAQ’s 👉100% Job Guarantee Program 👉Mock Interviews

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 👉Features of Online Training: 👉 Real-Time Oriented Training 👉Live Training Sessions 👉Interview Preparation Tips 👉FAQ’s 👉100% Job Guarantee Program 👉Mock Interviews

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 👉Features of Online Training: 👉 Real-Time Oriented Training 👉Live Training Sessions 👉Interview Preparation Tips 👉FAQ’s 👉100% Job Guarantee Program 👉Mock Interviews

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 👉Features of Online Training: 👉 Real-Time Oriented Training 👉Live Training Sessions 👉Interview Preparation Tips 👉FAQ’s 👉100% Job Guarantee Program 👉Mock Interviews

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 Features of Online Training: 👉Real-Time Oriented Training 👉Live Training Sessions 👉 Interview Preparation Tips 👉FAQ’s 👉100% Job Guarantee Program 👉Mock Interviews

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 Features of Online Training Real-Time Oriented Training Live Training Sessions Interview Preparation Tips FAQ’s 100% Job Guarantee Program Mock Interviews

How to join

Spark=sparksession.builder.master THIS LINE NEED!?

What about standard cluster....?

👍

Thanks Abhiram for this informative content.

👍

Enroll Now: "Azure Data Engineer Training & Placement Program" Start Date: Every Month 1st week || 7:00 pm IST For More Details: Call: +91 9281106429 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 Features of Online Training Real-Time Oriented Training Live Training Sessions Interview Preparation Tips FAQ’s 100% Job Guarantee Program Mock Interviews

✍Enroll Now: "Azure Data Engineer Training & Placement Program" 📅 Start Date: Every Month 1st week || 7:00 pm IST For More Details: 📱 Call: +91 9281106429 👉 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 💥 Features of Online Training ✅ Real-Time Oriented Training ✅ Live Training Sessions ✅ Interview Preparation Tips ✅ FAQ’s ✅ 100% Job Guarantee Program ✅ Mock Interviews

✍Enroll Now: "Azure Data Engineer Training & Placement Program" 📅 Start Date: Every Month 1st week || 7:00 pm IST For More Details: 📱 Call: +91 9281106429 👉 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 💥 Features of Online Training ✅ Real-Time Oriented Training ✅ Live Training Sessions ✅ Interview Preparation Tips ✅ FAQ’s ✅ 100% Job Guarantee Program ✅ Mock Interviews

✍Enroll Now: "Azure Data Engineer Training & Placement Program" 📅 Start Date: Every Month 1st week || 7:00 pm IST For More Details: 📱 Call: +91 9281106429 👉 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 💥 Features of Online Training ✅ Real-Time Oriented Training ✅ Live Training Sessions ✅ Interview Preparation Tips ✅ FAQ’s ✅ 100% Job Guarantee Program ✅ Mock Interviews

✍Enroll Now: "Azure Data Engineer Training & Placement Program" 📅 Start Date: Every Month 1st week || 7:00 pm IST For More Details: 📱 Call: +91 9281106429 👉 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 💥 Features of Online Training ✅ Real-Time Oriented Training ✅ Live Training Sessions ✅ Interview Preparation Tips ✅ FAQ’s ✅ 100% Job Guarantee Program ✅ Mock Interviews

Good explanation

✍Enroll Now: "Azure Data Engineer Training & Placement Program" 📅 Start Date: Every Month 1st week || 7:00 pm IST For More Details: 📱 Call: +91 9281106429 👉 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 💥 Features of Online Training ✅ Real-Time Oriented Training ✅ Live Training Sessions ✅ Interview Preparation Tips ✅ FAQ’s ✅ 100% Job Guarantee Program ✅ Mock Interviews

✍Enroll Now: "Azure Data Engineer Training & Placement Program" 📅 Start Date: Every Month 1st week || 7:00 pm IST For More Details: 📱 Call: +91 9281106429 👉 Chat with us on WhatsApp: wa.me/qr/PSW2ILTYJHTZI1 💥 Features of Online Training ✅ Real-Time Oriented Training ✅ Live Training Sessions ✅ Interview Preparation Tips ✅ FAQ’s ✅ 100% Job Guarantee Program ✅ Mock Interviews