How to Centralize SpringBoot logs to ELK Elasticsearch using Filebeat and Logstash

How to Centralize SpringBoot logs to ELK Elasticsearch using Filebeat and Logstash
In this session we are going to implement Centralized Logging In Spring Boot Microservices using Elastic Search, Filebeat, LogStash and Kibana, in other words ElasticSearch Logstash, Filebeat and Kibana for application logging in microservice architecture
Logging SpringBoot apps using Filebeat + ELK(Elasticsearch,Logstash,Kibana)
Download ElasticSearch: www.elastic.co...
Download Kibana: www.elastic.co...
Download Logstash: www.elastic.co...
Download Filebeat: www.elastic.co...
Springboot Project : start.spring.io/
How to Install and configure Elasticsearch & Kibana : • How to Install Elastic...
How to use Logstash Pipeline : • How To Load Data from ...
Reference GitHub:
github.com/pat...
______________________
⏲️Time Stamps⏲️
00:20 - Introduction
️▶️ Watch our latest videos: www.youtube.co....
▶️ Apache Kafka series : www.youtube.co....
▶️ Elasticsearch java - Playlist : www.youtube.co....
▶️ MongoDB java -Playlist : www.youtube.co....
▶️ Spring Boot Java -Playlist : www.youtube.co....
________________________________________
✉️ Contact: [e-mail]
🐦 Twitter: / simplifyingtech
📷 Insta: / simplifyingtech
---------------------------------------------------------------
Production Credits:
Edited by: Simplifying Tech
Edited with Tool : Filmora 10
Produced by: Simplifying Tech
WebSite: simplifyingtec......
---------------------------------------------------------------
About Simplifying Tech :
👉 Lets Make it Simplified and illustrative....
The main Objective of Simplifying Tech is to have people learn new technologies and programming in more simplified and illustrative way.

[@buksvdl]

Why take the easy path? No xml/yaml/json - application.properties only. Why? How many cfg files do you maintain? How many places did you define your elastic config? management.elastic.metrics.export.host etc

[@simplifyingtech]

It's springboot app no other configs required like xml yml etc,
All answers are already in video if you watch video without skipping... elastic config only need to define in logstash file that's it..

[@akashKVlogs-l9s]

I wanted to parse data to json format?

[@simplifyingtech]

Yes elastic output data in json format only.

[@gokulakrishnanr2675]

[ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
how to resolve this ?

[@simplifyingtech]

Check where you placed config file .. place where its mentioned in video

[@fahadsiddiqui4359]

What to do when an error is coming as "Failed to execute action{:action=>LogStash}"

[@simplifyingtech]

There is your error, logstash could not find java . You will need to install java or set JAVA_HOME for the logstash processs
run below equivalent in windows and check
sudo -u logstash which java

[@RahulSaini-pp5kz]

you also have tomcat in the folder but you never told about it? Will everything work well without that? Pls tell

[@simplifyingtech]

Hey this is microservice , tomcat is in built in springboot . There is no use of external tomcat here, if its web application other than springboot then its required.

[@RahulSaini-pp5kz]

@@simplifyingtech thank you it is working now, just want to ask how can we also see live logs in observability->logs->stream, as if now it is not showing

[@anon_geek]

what if we don't want to use logstash? Can we directly ship logs from the java application console without writing to a file with slf4j to elasticsearch using filebeat?
Example: Upon starting up my application I start getting logs which are printed in the IDE console. Is it possible to ship those logs to Elasticsearch directly?

[@simplifyingtech]

you need shipper to ship your logs to elasticsearch, and logstash is shipper. alternative splunk/fluentd

[@anon_geek]

@@simplifyingtech So I used your example and a different example using which I was able to directly connect filebeat with elasticsearch as per the documentation and visualize them using Kibana.
So, was concerned on how logstash comes in handy? like is it an optional use case?

[@simplifyingtech]

Good qus
To answer this we can communicate to ES from filebeat locally but if we talk on multi cluster environment and client server model we need logstash to aggregate logs from many servers and apply common transformations and filtering to your events...

[@rahulsoni-lx5rb]

🤩🤩🤩