Відео

Yes

Please check the commands in the video description to start the redis container and how to create the cluster.

Check here github.com/arunsrajan/tutorial/tree/main/kafkaconfig-kraft

Check in the web console or use "artemis browse" or "artemis consumer" command options. For more information check in chatgpt.

@configlearninghub chatgpt command only showing the messages and characters not showing tinestamp

Very nice

Thanks

Master and slave is meant for 1 live and 1 backup. You can configure for multiple backups also. If you want more backups and primary you can go for the colocated cluster

@@configlearninghub colocated is possible

Colocated cluster is for multiple primary and multiple backup

@configlearninghub But actually iam not use colocated cluster, I set up 1 master and 2 backups only 1 backup is live 2 backups is not in live...

Only one primary is active at a time and if primary fails any of the backup servers will become primary and other will become backup

Please use kubectl port-forward command for accessing the dashboard for each pod or try this artemiscloud.io/docs/tutorials/send_receive_port_forwarding/

Please reach me arunsrajan@gmail.com

Thanks and welcome

@@AnkurRanjan-rm5jd yes it a standalone cluster in Kubernetes.

@@configlearninghub In production it will be better if we use K8s as cluster manager. Am I right?

@@AnkurRanjan-rm5jd apache spark community supports only k8s cluster manager in production.

@@configlearninghub Do you have any articles or videos for this? I am getting confused about this. I am trying to run my PySpark job using k8s cluster

You can try spark-submit to submit a python job in the k8s cluster with python support spark docker image. For submitting job on Kubernetes cluster please watch this video. ua-cam.com/video/6ZYvH7dINig/v-deo.htmlsi=hc_ouqlyXIPGyvKb

Thankyou

Thanks

Thank you! Cheers!

Thank you! Cheers!

Welcome!

You should access Kubernetes dashboard using master node ip address with node port. Don't use cluster ip of Kubernetes dashboard service it is internal to the vm.

I will try

Please check this ua-cam.com/video/_4F_pV04csk/v-deo.html

In the video description section I have mentioned the repository which contains jenkinsagents.txt where you can see build command at the end. You can use that. You can also use the docker image that I have built which is configured in the deployment yaml file

Glad it was helpful!

Yes

I have sent steps for running apache airflow for spark in Kubernetes. Was it helpful?

@@configlearninghub Hi Sir ji yes it is very helpful, 😊 I'm sending my raspberry pi login credentials you. It is like remote machine. Sir ji both attached file work but after creating spark driver and worker node i want to submit my job where I want to write a simple program to calculate the average value. Thank you so much for help dear sir ji. I seen in airflow there connection id where lots of type of connection available like Amazon web services, MySQL, postgres, but there is not available spark. Please i need little more help. I'm really sorry for distributing you but please if you got time i need help. 🙏🙏🙏

@@configlearninghub Hi Sir ji it is very helpful thank you so much 🙏 Sir ji both file work I'm able to see master driver and worker . I want to submit my job to calculate average value in master spark driver. Please where I can write my spark code and how to submit. You send me yaml file where i can able to create spark master driver and worker but how to submit my job in it. One more help sir ji in airflow in connection id I'm able to see MySQL, Amazon web series, postgres but I'm not able to spark connection.

I will share you today my raspberry pi login credenti@ls it is just like remote machine where you can see sir ji my installed applications. 😊 And you can use it as well i will be so happy ❤️

Is the spark job in python language or in java or scala DSL. Where is your dataset stored?