Step by step guide how to build a real-time anomaly detection system using Apache Spark Streaming

Поділитися
Вставка
  • Опубліковано 14 жов 2024

КОМЕНТАРІ • 15

  • @laurynasstasys
    @laurynasstasys 6 років тому +3

    Fine tutorial, however, there's underlying issue with k-means, that it will always split data into k clusters even though you'd actually have less than k clusters in your data. Please consider DBSCAN as an alternative :)

  • @iidtxbc
    @iidtxbc 3 роки тому +1

    It might be ok to use k-means clustering for anomaly detection but, as mentioned in your video, there are things we need to think over about the threshold to tell it is an anomaly and the optimal number of clusters. Finding the number of clusters using Elbow method is not really applicable to the real time situation because we don't really know the optimal number of clusters unless we really do the data mining to find it.

    • @SolarisAgentSystem
      @SolarisAgentSystem  3 роки тому

      Hi you are correct - unless you have very well explored domain problem and know how many clusters to expect, k-means may be difficult to apply in more complex areas. It's mostly used for data exploration.

  • @sarthak7413
    @sarthak7413 6 років тому +2

    How to find threshold for detecting outliers?

  • @akhileshsingh569
    @akhileshsingh569 4 роки тому

    Hello bro can you please make a video on real time forecasting on live stock market

  • @inderkar8349
    @inderkar8349 6 років тому

    why have you not used SVMModel in Apache spark to create the model? is it because of no of data is too large?

  • @huili4364
    @huili4364 6 років тому

    Great!

  • @AndrewCLatham
    @AndrewCLatham 2 роки тому

    Be great to access the code.

  • @arunbm123
    @arunbm123 7 років тому

    nice technique, is it possible to share the code Please...

  • @lawalsherif5242
    @lawalsherif5242 6 років тому

    Thanks for this video and tutorial, please can someone try this on Matlab or Devc++ Applications? Then this is related to my work can you please share the codes with me privately? I will so much appreciate it. Thanks once again.

  • @datasciencewithr1039
    @datasciencewithr1039 7 років тому

    Great tutorial. please share code.

  • @Madhu9738
    @Madhu9738 6 років тому

    Great tutorial. please share code :)

  • @bekofdevs
    @bekofdevs 5 років тому

    can you send the code to me, please?

  • @ug0ts3rvd
    @ug0ts3rvd 7 років тому

    my nigga mariusz

  • @dinasamir2778
    @dinasamir2778 6 років тому

    can you send the code to me, please?