Data pipeline using Kafka - Elasticsearch - Logstash - Kibana | ELK Stack | Kafka

[@praveenchakravarthula8806]

This is the one of the best video tutorial I have seen for education.
very intuitive and required level of details

[@AshuToshSharma99]

Facing issues with cmak unable to create topic can you help

[@pholohohlaha10]

Wonderful work my brother, this has helped me a lot, what a presentation.

[@codewithrajranjan]

Thanks

[@shaiovadia5766]

Great tutorial - extensive enough and doen't skip any step.
excellent work !

[@BhavinShahBlog]

Excellent post. The only problem is the pace is very fast.. I guess using AI it can be re-recorded with slower pace professionally

[@leamon9024]

Awesome. Looking forward to more in-depth contents about ELK stack

[@gouravkhatter2520]

Informative, easily and cleanly explained.... subscribed (you owned) 😀
Can I have reference guide for processing data before sending to elasticsearch?

[@pavankumard5276]

Really good video, I have subscribed to your channel looking forward to more such videos👍

[@codewithrajranjan]

Awesome, thank you!

[@dannotes9396]

Thanks for the video bro, very informative :)
One non-related query, what font do you configured in the jet brains IDE? :)

[@ksavinash]

@SelfTuts
please share the working docker-compose setup lagstash as well
Note: The video shows logstash local setup its working fine for me in a local setup. I'm trying add logstash to docker compose file as well but I'm unable connect to Kafka

[@nicwhites]

15:30 You should simply do a lambda instead of created a one line function. Other than that, good video!

[@rabithbo7089]

A very useful tutorial 😘🙏

[@shridarshanramanand4766]

This is best video i have ever been seen . Cool nice thank you . point to point any videos for ansible script patching or provision linux server through vmware clone ?

[@ElHiro03]

subscribed. Great tutorial and amazing channel.
Saludos desde Argentina.

[@AbhimanyuKumar-ko1ih]

Very informative thanks

[@codewithrajranjan]

Thanks

[@srikanthchitturi7283]

Excellent explanation. Good job

[@shafiquegkhan6400]

Great video. Thnx 4 sharing

[@salihuumar3138]

A wonderful video

[@lalitshukla6585]

Thanks Sir, do we have to pay or take youtube subscription for accessing your website ?

[@uploadvoice]

Excellent !!!
Great video

[@balinasuryachandra9337]

Hey SelfTuts Awesome explanation
i have small doubt, i installed kafka, elk environment on different instances IN AWS
to establish a connection between kafka and logstash which ip address should i provide either public IP or private IP in conf directory of logstash

[@AnthonyPinero]

Hi nice tutorial, do you think you can share the sources in any repository?

[@sany2k8]

Great tutorials, subscribed

[@sakshivats7574]

How do I start logstash using cmd or shell . Bcs systemctl doesn’t work

[@satishchhatpar]

Very good demo. Thank you

[@dmcp3261]

subscribed - awesome content.

[@anupamchowdhury0]

How logstash connect to secure kafka using keycloak?

[@lucaspendola7696]

version: "3.3"
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.4.0
container_name: elasticsearch
restart: always
environment:
- xpack.security.enabled=false
- discovery.type=single-node
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
cap_add:
- IPC_LOCK
volumes:
- elasticsearch-data:/usr/share/elasticsearch/data
ports:
- 9200:9200
kibana:
container_name: kibana
image: docker.elastic.co/kibana/kibana:7.4.0
restart: always
environment:
- ELASTICSEARCH_HOSTS=elasticsearch:9200 # address of elasticsearch docker container which kibana will connect
ports:
- 5601:5601
depends_on:
- elasticsearch # kibana will start when elasticsearch has started
volumes:
elasticsearch-data:

[@lucaspendola7696]

version: "3"
services:
zookeeper:
image: zookeeper
restart: always
container_name: zookeeper
hostname: zookeeper
ports:
- 2181:2181
environment:
ZOO_MY_ID: 1
kafka:
image: wurstmeister/kafka
container_name: kafka
ports:
- 9092:9092
environment:
KAFKA_ADVERTISED_HOST_NAME: [machine-ip-address]
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
kafka_manager:
image: hlebalbau/kafka-manager:stable
container_name: kakfa-manager
restart: always
ports:
- "9000:9000"
environment:
ZK_HOSTS: "zookeeper:2181"
APPLICATION_SECRET: "random-secret"
command: -Dpidfile.path=/dev/null

[@vineethkumar7803]

@@lucaspendola7696 Thanks for sharing. Saves lot of time when the website provided in description is down.

[@edgarnaula2835]

All services get up except kafka. Any idea why that happens?

[@HindiEditzCountdown]

so nice tutoorial bro

[@dheerajasnani7918]

Great tutorial, just what i was looking for. Thanks

[@skidrow406]

great, thanks

[@aj_abdel7113]

Best videos!

[@Vivek.1011]

the links are broken, can you update them

[@tatavarthyvenkatesh4390]

Can we monitor.. consumer lag , fetch lag and all.. using logstach and kibana?

[@satishchhatpar]

your website is not working. Gives error "Error establishing a database connection"

[@ajay-lu5gh]

Thanks a lot!

[@billmcguire6128]

Great video - thanks

[@86Saurabh1]

Hello, great tutorial. I am having some issues when both containers are running. I can see that all the 5 services are running when using command "docker ps" but when I try to open the UI for either kibana or kafka_manager , both won't open, only one would open. When I run the containers one at a time then kibana or kafka_manager would open, depending upon which service is running, but not together. Any help would be appreciated.

[@86Saurabh1]

Issue seems to be with Kafka Manager service, as without this service I can run Kibana and other services. Need to find an alternative to this.

[@hemp_gaming]

Hello, can you do a video about api connections in kafka,
About real-time, data processing.

[@himanshuagarwal7628]

why your site is currently not reachable?

[@rvpandey99]

your website is down.

[@saisharath5515]

Your website is not reachable

[@srikanthjnr]

very good video! keep it up!

[@mosseery8118]

Awesome

[@krystian-magdziarz]

Amazing.

[@freefallinu3446]

Cool!

[@localboyvijay9628]

website is not working, Can you please check

[@rufianarchamdani3848]

good very cool

[@harrydam_]

Does Your website still down?

[@hemantjain9850]

website is down

[@stockmarketBullsVsBears]

Can you add ssl info for secure data transfer

[@joepoptiya]

@selftuts, did you know your website is down?

[@codewithrajranjan]

Thanks it's up now

[@mohammadsalehsaeidi378]

@@codewithrajranjan it's down again.

[@lalitshukla6585]

@@codewithrajranjan still down

[@preethikannan98]

Given links are not reachable

[@davidmadronalsanchez4377]

links are down?