Django Celery Advanced Concepts | How does Celery Work? | Pool, Concurrency, Autoscale |Scaling Apps

Поділитися
Вставка
  • Опубліковано 7 лис 2024

КОМЕНТАРІ • 162

  • @gammezzy
    @gammezzy 2 роки тому +15

    Where were you, this channel is so underrated.

  •  2 роки тому +3

    Always maintain quality work and solve production and top-level problems. thanks dude

  • @deez_dev
    @deez_dev 6 місяців тому +3

    great videos on celery, I was afraid first to implement it in my project since it's a big change but now am comfortable.
    keep going.

  • @supritkumar2624
    @supritkumar2624 Рік тому +2

    No one in UA-cam has explained Celery in depth till now.

  • @josehermosillaavila7889
    @josehermosillaavila7889 2 роки тому +2

    oh men thanks for the tutorial, i am not english spekaer but i undertand all of this tutorial! thank you for sharing

  • @user-hr8iz9lb3g
    @user-hr8iz9lb3g 3 роки тому +16

    Great tutorial. You really are a good teacher: you understand that people need to really understand how tools work to use it confidently and genuinely. I am starting a job (new role) that requires to master celery quickly and your videos make the difference. The documentation is confusing and the majority of tutorials are only practical "getting started" stuff. On the other-side conferences lecturers talk to people who already understand them.
    I would be interested interested in more details that are covered in the user guide.

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  3 роки тому +5

      Yes, you got me right. Thanks by the way.

    • @bharatisagar9336
      @bharatisagar9336 3 роки тому +1

      All the stuff is very good and the way you explained is brilliant 👍 thank you . When I searched for the celery, got your videos and I strated working on it . Its going very smoothly 😊 thank you for sharing your knowledge 👏

    • @kkj8601
      @kkj8601 11 місяців тому

      @@PriyanshuGuptaOfficial Hi, how could i use greenlet.settrace inside celery task so that i could know whenever control switches between greenlets?

  • @realdevdiv
    @realdevdiv 11 місяців тому +2

    This tutorial has broadened my perspective. Thank you for taking the time to explain.

  • @GiaBao-nc8ox
    @GiaBao-nc8ox 6 місяців тому +1

    Thanks for this series, I learned a lot from it. And this is the most detailed explain about Celery that I have ever seen.

  • @RahulGupta-pw5ei
    @RahulGupta-pw5ei 2 роки тому +1

    Great sir i got it ur concept priyanshu sir

  • @bharatisagar9336
    @bharatisagar9336 3 роки тому +3

    All the stuff is very good And the way you Explained is also brillant 👏When I searched for the celery and I got your series And started working on it.Its going very smoothly 😊thank you for sharing your knowledge 👍

  • @PriyanshuGuptaOfficial
    @PriyanshuGuptaOfficial  3 роки тому +7

    🔴 Don't miss the ending part of this video, I have an important message for you - 59:15
    ✅ Do watch this video after you have watched this if you want to know when to use celery and when to use multithreading, multiprocessing etc. - ua-cam.com/video/y_FkspGs6ro/v-deo.html
    👉 In case anyone wants to connect with me 1:1, feel free to schedule it on my calendar - topmate.io/priyanshugupta
    🗒Article: www.priyanshuofcl.com/blogs/celery
    Don't forget to like 👍 the video and subscribe 🔴 to my channel if you really want to level up your skills to have an edge over others.

  • @kanyantamusonda711
    @kanyantamusonda711 11 місяців тому +2

    this a great tutorial, first time landing on your channel and i can say your explanation on this concept is the best so far to what i have come across. i keep learning from your channel

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  11 місяців тому

      Hope you learn more from the channel and see something new

    • @kkj8601
      @kkj8601 11 місяців тому

      @@PriyanshuGuptaOfficial how do i use greenlet.settrace from within the celery task to monitoe whenever greenlets yield control to other greenlets . I need to log this event for a given celery worker of gevent pool with concurreny 3

  • @AnkitMishra-ys7fb
    @AnkitMishra-ys7fb Рік тому +1

    Very well explained. Everthing at one place for django background tasking.
    Thank You Sir

  • @alycheikhouldsmail7576
    @alycheikhouldsmail7576 2 роки тому +2

    thank you.
    explanation focus on concept not on writing code.
    good explanation.
    if you split this video in several small videos as playlist about celery it will be good.
    I suggess to comment PAAS like Heroku and serveless like AWS support for celery in deployed django project

  • @priyankagorkhe2870
    @priyankagorkhe2870 10 місяців тому +1

    Thanks a million to brother, I really appreciate the value of knowledge you share and the nonstop hardwork you did to create these videos for us. Thank u so much.

  • @archit_singh15
    @archit_singh15 Рік тому +1

    thanks man, nice refresher for deeper concepts and not just a basic hello world setup

  • @siddharthsrivastava4491
    @siddharthsrivastava4491 Рік тому +1

    Crystal clear explanation
    Thanks buddy!!

  • @jessicaryan9820
    @jessicaryan9820 3 роки тому +4

    Excellent presentation!!! Thank you!!!

  • @bogi0509
    @bogi0509 3 роки тому +2

    Thank you because you are shearing this knowledge. Greetings from Serbia.

  • @sillywalker2992
    @sillywalker2992 2 роки тому +2

    Greetings from Brazil! Thanks for sharing such knowledge!

  • @rynkbell
    @rynkbell 2 роки тому

    Thanks Priyanka ji..you are too nice and smart

  • @nidhi26100
    @nidhi26100 3 роки тому +2

    Superb video. Thankyou so much

  • @dabasishkundushento5992
    @dabasishkundushento5992 2 роки тому +2

    Great tutorial for celery. Detail explanation, seamless throughout the tutorial. Keep up the good work.

  • @trosgate
    @trosgate 3 роки тому +3

    This is amazing. I just landed on your channel and I am glad I did.

  • @rifatahsan5636
    @rifatahsan5636 2 роки тому +2

    Thanks a tonnn brother. Im stunned by the clarity of your knowledge.

  • @boogeyman9824
    @boogeyman9824 3 роки тому +4

    Thanks Broda you actually did a right thing making this playlist i was fed up searching for the right resources about celery this tutorial covered nicely and one more suggestion this being totally theorotical u can push this video to part 1 and keep the rest of the videos after this coz most of the other tutorials have covered the other videos but this particular video is the highlight of this playlist anyways Kudos to you :)

  • @abdullahamin4504
    @abdullahamin4504 2 роки тому

    Hellow -----
    My self Abdullah from bangladesh .
    I Watch your video regularly and now i complete django playlist.
    i learn everything from your youtube ---
    thank you ---
    I have a request to you that please make a video how to use django custom admin panel instead django default admin templare....
    please
    please
    please --------- i wait for your notification

  • @mrcharm767
    @mrcharm767 Рік тому +1

    brother , your doing super duper job please continue the good work

  • @mahirmuhammedali8293
    @mahirmuhammedali8293 2 роки тому +2

    Thanks a lot brother ❤

  • @nabinchaudhary73
    @nabinchaudhary73 2 роки тому +3

    what a mind blowing video bro? keep uploading this types of content.

  • @alx913
    @alx913 2 роки тому +1

    Priyanshu! Great video. I'd like to translate these series to spanish but I don't find the option of adding transcript. Let me know if you can add it!

  • @abdullahamin4504
    @abdullahamin4504 2 роки тому +1

    Love You Brother -------
    Hellow -----
    Request to you that please make a video how to use django custom admin panel instead django default admin templare....
    Wait For Your video Notificaiton

  • @ganeshdabral8963
    @ganeshdabral8963 3 роки тому +2

    Why this video have less like... This is very amazing and helpful video... 🙏 Thanks buddy

  • @fransubaru
    @fransubaru 2 роки тому +1

    Thanks for the explaining !! Very good video, now I understand a bit more Celery!

  • @abhisheks1036
    @abhisheks1036 2 роки тому +1

    I really liked your explanation bro excellent, and you really nailed it man.

  • @markgreiner3237
    @markgreiner3237 2 роки тому +2

    You are awesome for making this series of videos! Thank you!

  • @nampallyanurag3620
    @nampallyanurag3620 Рік тому +1

    Keep making such content, very much useful for making highly scalable apps. Not only celery, this concept itself is much needed irrespective of framework

  • @ajayprajapati3097
    @ajayprajapati3097 3 роки тому +3

    Great video, Thanks for sharing.
    You can also add managing queue, adding ofair and also one ques is it best to add asyncio inside celery task?

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  3 роки тому

      Thanks. Use asyncio inside your celery task if that single task has to peform multiple I/O operations.

  • @thanisalbert8316
    @thanisalbert8316 Рік тому +1

    Excelent🙂

  • @emadheidari-p6b
    @emadheidari-p6b Рік тому +1

    Thanks for the video
    It helped me a lot

  • @mifrajvlogs778
    @mifrajvlogs778 3 роки тому +1

    Please make a video regarding Eventlet & gevent please

  • @abhishek171278
    @abhishek171278 Рік тому +1

    awesome lecture...

  • @Namastedbt
    @Namastedbt 2 роки тому +1

    Very useful

  • @jasmeetpabla378
    @jasmeetpabla378 2 роки тому +1

    Thanks a lot for your hard work.

  • @kumarshivam2076
    @kumarshivam2076 3 роки тому +2

    This is a great playlist !!

  • @corey_pratiksarangi857
    @corey_pratiksarangi857 2 роки тому +2

    Wow Amazed !! thanks for the help ! really helps a lot

  • @RaushanKumar-uj2ew
    @RaushanKumar-uj2ew 7 місяців тому +1

    Can you please make me understand how can I use compute of ec2 for celery ? how scaling will happen over there ?

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  7 місяців тому +1

      So what you can do is first decide the pool of celery according to your tasks(cpu intensive/io operations). Try to increase the workers and use the full capacity of your ec2. You can also make dedicated queues for different task types and respectively make the separate celery workers(listening to dedicated queue). This will reduce the load on one queue. Then you can deploy rabbitmq on a separate server. Then you can deploy the celery workers on a new ec2 machine in order to scale horizontally. In order to manage services over different machines easily, you can use ec2 auto scaling feature, or you can use docker swarm/kubernetes.

    • @RaushanKumar-uj2ew
      @RaushanKumar-uj2ew 7 місяців тому

      @@PriyanshuGuptaOfficial Thank you so much for quick reply. Much appreciated !!!

  • @danishmaqsood9969
    @danishmaqsood9969 2 роки тому +2

    best video

  • @sunilmaurya6594
    @sunilmaurya6594 3 роки тому +2

    i find your channel is like i find diamond 💎 😀

  • @prashantthumar75
    @prashantthumar75 Рік тому +1

    This was very helpful. Can you please also upload it on green threads?

  • @gamerwolf5242
    @gamerwolf5242 2 роки тому +2

    Thanks for the motivation brother :)

  • @Komedy1Reels
    @Komedy1Reels Місяць тому +1

    Now i can keep celery in my resume

  • @abhishekbudruk6619
    @abhishekbudruk6619 3 роки тому +2

    Best Video

  • @krishdb38
    @krishdb38 Рік тому

    thankyou for your video
    i have some problem with celery
    my work station has 64 core (128 threads)
    251GB RAM
    i want to parallel run my process
    running my single one task require around 30 minutes and internally one task use internally 10 processor (threads) and around 10 GB RAM required
    i hae used concurrency , autoscale also
    when i run manually 20 ~ 25 tasks are optimal at a time
    if i set concurrency 20 then the celery is waiting all 20 tasks to complete
    not jumping to 21 if first 1 , 2 task completed
    do you have any idea ?
    can you write me a celery options so i can try ?
    thank you
    from S.Korea

  • @sergey5551
    @sergey5551 2 роки тому +2

    Great Tutorial!

  • @iamanonymous2
    @iamanonymous2 5 місяців тому +1

    great tutorial as always😀

  • @moneeshkumar1838
    @moneeshkumar1838 Рік тому

    Detailed explanation
    Great work
    Keep posting

  • @AtulPatel-vu9gf
    @AtulPatel-vu9gf Рік тому +1

    Can we pause and resume celery task?

  • @ayushjaiswal167
    @ayushjaiswal167 Рік тому

    Great Content Bhai! Thank you, keep making such content.

  • @sandipsadhukhan9628
    @sandipsadhukhan9628 2 роки тому

    Good course

  • @pythondjango1798
    @pythondjango1798 2 роки тому +1

    I did complete your 700 Likes😇😇

  • @girishgirijan9900
    @girishgirijan9900 2 роки тому +1

    Great, thanks for the video.

  • @dev-ql8fj
    @dev-ql8fj Рік тому +1

    Awesome content

  • @aashayamballi
    @aashayamballi 3 роки тому +2

    very informative! thank you...

  • @seerozhaa2656
    @seerozhaa2656 2 роки тому +1

    Thank u so much!

  • @PrAsHaNtHdUke1093
    @PrAsHaNtHdUke1093 9 місяців тому

    thanks a lot bro...upload more videos like this

  • @uljhe_hue_wires_ka_khambha
    @uljhe_hue_wires_ka_khambha Рік тому

    great explanation

  • @cleanpathtravels
    @cleanpathtravels 3 роки тому +2

    brilliant ❤️

  • @awsdayoub2963
    @awsdayoub2963 Рік тому +1

    Great tutorial❤❤

  • @abhishek171278
    @abhishek171278 Рік тому

    Can you make video on flask redis and celery to cache any task query from database

  • @abhishekk1231
    @abhishekk1231 3 роки тому +3

    It's Amazing!

  • @gauravkandoria6622
    @gauravkandoria6622 2 роки тому +1

    Great work

  • @HarshitGupta-vo5bm
    @HarshitGupta-vo5bm 7 місяців тому

    brother windows par concurrency and autoscale karne par celery task receive karta hai but execute nahi karta. i want to autosclae my celery workers according to processes load on windows. should i create multiple pool=solo but it will run fixed amount workers even when there is not need. so what should i do tasks are processing type

  • @ajetunmobitolulope7745
    @ajetunmobitolulope7745 3 роки тому +2

    you are the best

  • @hafeezpp
    @hafeezpp 2 роки тому +2

    Nice video 👍

  • @numberjuan297
    @numberjuan297 Рік тому +1

    Wow amazing content man instantly subscribed

  • @nooruzbai
    @nooruzbai Рік тому

    Thanks for the video, if possible could you show how to use celery with genericApi views and class based views?🙏

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  Рік тому

      If you want to call celery task from view, its very easy. Just call task_name.delay()

  • @twins_coder
    @twins_coder 3 роки тому

    Sir ek video django testing using faker in depth
    One video on django ORM deep knowledge
    Then start a series in API with project or project kisi ka clone ya phir esa kuch jisme doubt clear ho jaye ki api mai validation ko kese lagde jese python mai lgate the

  • @mahfuzurrahman4517
    @mahfuzurrahman4517 2 роки тому +1

    Great tutorial 👍

  • @preetiduhan9937
    @preetiduhan9937 3 роки тому +1

    This was very helpful.Can you please also upload on green threads?

  • @PrAsHaNtHdUke1093
    @PrAsHaNtHdUke1093 20 днів тому

    bro how much time it will take for sending 10,000 to clients using celery concept.. i mean how many minutes it will take?

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  20 днів тому

      It should take around 0.5 seconds to send each mail. So you can estimate accordingly. Plus you can optimize it further by sending in batches and using pool of workers(here thread or gevent makes more sense as its an io operation)

    • @PrAsHaNtHdUke1093
      @PrAsHaNtHdUke1093 20 днів тому

      @@PriyanshuGuptaOfficial thank ui

    • @PrAsHaNtHdUke1093
      @PrAsHaNtHdUke1093 20 днів тому

      @@PriyanshuGuptaOfficial now celery threading will be support in windows machine or it will support only solo?

    • @PrAsHaNtHdUke1093
      @PrAsHaNtHdUke1093 17 днів тому

      @@PriyanshuGuptaOfficial bro pool of workers (thread) this thread will support on windows 11?

  • @engineer8340
    @engineer8340 2 роки тому +1

    Awesome Explanation Bro :)

  • @melihamanav
    @melihamanav Рік тому +1

    Awesome!

  • @MukulSinghAdhikari-n6s
    @MukulSinghAdhikari-n6s 7 місяців тому

    can you share your ppt of celery ?

  • @avishisainii
    @avishisainii 3 роки тому +2

    Superb work🙌🙌

  • @rikenshah1861
    @rikenshah1861 2 роки тому

    Thanks a lot bro!

  • @LucasFontini
    @LucasFontini 9 місяців тому +1

    good video brother

  • @reactjsmedia6406
    @reactjsmedia6406 3 роки тому +2

    Amazing video!

  • @ricardorodriguez4180
    @ricardorodriguez4180 2 роки тому +1

    Subbed, excellent content.

  • @NiM44
    @NiM44 3 роки тому

    Hi
    Priyanshu Gupta
    46:27 - regard the gevent and eventet can you plese create video ? also about stateful tasks
    thanks

  • @altarcalsaneller2281
    @altarcalsaneller2281 10 місяців тому

    perfect !

  • @hafeezpp
    @hafeezpp 2 роки тому

    Please provide ppt, it will be very helpful

  • @sivakumarn2052
    @sivakumarn2052 Рік тому

    how can i deploy my celery web application for free ? currently im using pythonanywhere

  • @mentore9696
    @mentore9696 7 місяців тому +1

    it feels like, you have to work more on delivering.

  • @amitkhandelwal6866
    @amitkhandelwal6866 Рік тому

    Hi Bro can you provide some hands on coding for celery

  • @gabriellopezmontesinos2951
    @gabriellopezmontesinos2951 8 місяців тому

    really good content, but your voice tone makes it difficult to follow. You could try toning it down maybe.. just a suggestion :P

  • @a036nikhilsannat4
    @a036nikhilsannat4 2 роки тому

    WHat is the need of solo? Like why do we even need this?

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  2 роки тому

      You can use it if you want to perform a single task dedicatedly without any context switching

  • @vaibhavgirase3021
    @vaibhavgirase3021 5 місяців тому

    In my project celery not receive task and not performed task in background , please reply on my comment

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  4 місяці тому +1

      This can happen due to various reasons. Please message on discord server or telegram with more details, so that I can understand your problem properly.

    • @vaibhavgirase3021
      @vaibhavgirase3021 4 місяці тому +1

      @@PriyanshuGuptaOfficial thanks for reply, celery is working now in my project

  • @AlphaSlyer87
    @AlphaSlyer87 3 роки тому

    Bhai Hindi Mai Video banana kyu band kardi...please video Hindi Mai Bana dijiye if not Hindi mai ek channel Banayo Thanks

    • @PriyanshuGuptaOfficial
      @PriyanshuGuptaOfficial  3 роки тому +2

      See, many people dont know hindi, thats why I am making videos in English. I will think about making a separate channel in hindi.

    • @AlphaSlyer87
      @AlphaSlyer87 3 роки тому

      @@PriyanshuGuptaOfficial then it’s good idea to make separate Channel in Hindi Will be subscribe there as well Thanks 🙏

    • @user-hr8iz9lb3g
      @user-hr8iz9lb3g 3 роки тому

      @@PriyanshuGuptaOfficial I only speak three European languages. Fortunately, for me, it was not in Hindi.

  • @qubitsmith7090
    @qubitsmith7090 2 роки тому

    Biggest waste of time. The author just discusses terms and no code at all. Sensational title though. Lower your expectations.