The Only .NET Scheduler You Should Be Using!
Вставка
- Опубліковано 1 тра 2024
- Get 20% off our new Deep Dive: Domain-Driven Design course on Dometrain: bit.ly/44pf7sE
Get the source code: mailchi.mp/dometrain/4wurs-67mb0
Become a Patreon and get special perks: / nickchapsas
Hello, everybody, I'm Nick, and in this video I will introduce you to Hangfire. The best .NET library for task or job scheduling in .NET.
Workshops: bit.ly/nickworkshops
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: github.com/Elfocrash
Follow me on Twitter: / nickchapsas
Connect on LinkedIn: / nick-chapsas
Keep coding merch: keepcoding.shop
#csharp #dotnet
Thanks for the video. Can you do a comparison of Hangfire, Coravel and Quartz, please?
Quartz is extremely reliable and works very well. Hangfire is a very heavy dependency, I'd only adopt it if I really needed it.
The title of the video sounds like a clickbait. I think you need a comparison video why is it better than other similar libraries, e.g. Quartz, Coravel, FluentScheduler.
Agree.
Quartz FTW!
I love Nick and pretty much watch all his videos but bless him, he does love the clickbait titles and images.
For a small site where you don't need fancy features, Coravel is the easiest one to use by far. I love it.
Nick almost always uses clickbait titles. Often has the opposite effect on me making me not watch his videos lmao.
Hangfire is solid but you should try Quartz. Over time I have replaced hangfire with Quartz in most of my projects. I find the extensibility and performance better
Perfect comment. Even to do a simple benchmark Hangfire can be a pain due to its heavy static based usage. But after looking into the source code and managing to find its proper instance structures to benchmark I have found that Quartz performs much better. Hangfire also seems to have some kind of internal process that runs from time to time that causes some hanging and delays the scheduled tasks by some seconds (up to 20s)
Hang fire doesn't do well in a containerized environment. The workers stops working at some point and that affects recurring tasks.
quartz is better especially in containerized environment.
Quartz is in the middle of a rewrite for nosql and other improvements, though, so support is currently slow or lacking
Totally agree, Hangfire almost the same no updates or major enhancements for many years. Quartz does the job and more.
Also agree, I would never recommend Hangfire over Quartz unless a commercial support contract was mandatory.
Perfect timing on this video! I am just starting to build an application that needs to execute tasks on a background service, with a follow-up n hours later AND have scheduled jobs. Hangfire looks to do all of that and includes a nice dashboard to boot. I will also look at some of the other libraries mentioned in the comments--Quartz, Coravel, FluentScheduler. Very excited I won't need to build from scratch :D
Wow! Awesome library! I will definitely be checking it out. Thank you Nick!
Thank you Nick for listening to us and making valuable videos again ❤
Please make a video to show all possible things we can do with Hangfire. I love your content so much.
It'll be nice to have a deep dive into hangfire
Could you make a comparison to Quartz. We are using Quartz but had some problems of scheduled Jobs that were never executed (due to a Bug I then fixed in v3.4). In addition we needed to extend a bunch of extra stuff. So, would be nice to know if there is another scheduling framework which is worth it switching.
Have used Quartz, HF and Coravel and for me personally, choosing one over another depends on the environment and app requirements.
Big fan of hangifre, I use it heavily at work and would love to see a deeper dive into how you use it!
Used hangfire in production for 4 years now, never had any issues and a great builtin dashboard
Thank you a lot, you are not going to believe the timing of this video is inch-perfect.
Another solution to consider is employing an external job scheduler that triggers tasks via API, effectively decoupling processes in an elegant manner.
We also use this a lot in our projects. With some nice extension packages you can have progress bars in the dashboard wich is cool. The dashboard itself is useful, but if you have a lot of jobs that get triggered very often it can become cumbersome to find an invokation of certain job types. Some sort of filter on date or job would be nice.
Hi there. Apologies if you already know this, but thought I would offer a suggestion. You can give your jobs a Display Name and incorporate instance arguments into the name. The display name is then displayed as the job name in the dashboard. This doesn't solve the problem of being able to search for a specific job, but you could for example set your paging to 50 or 100 per page, and then use the in browser search.
Your code might look something like this:
public class MyJobClass
{
[DisplayName("Job for customer {3} with argument {0} and number {1}")]
public async Task MyJobMethod(string argument1, int argument2, string customerName)
{
.... // Your method code
}
}
This can be helpful for tracking down specific instances of regularly run jobs.
I have used hangfire I generally like it. Dashboard is good but not properly documented how to secure it. Having multiple servers is a good benefit to scale.
dashboard security and scale with multiple servers are the next video post ideas from me. Perhaps it's good to mention about async await problem of it too
To secure it, you need to pass an authorization filter in the options for UseHangfireDashboard.
Curious, what's the async/await problem?
We've got a private/custom implementation for recurring and on demand tasks actually. Likely because we've had this need for many, many years now. But having a fully custom thing does have it's own advantages (and some disadvantages too of course, mostly in cost to build + maintain it).
However, that's mostly legacy at this point though, since most projects these days are cloud first and as such Azure's replaced all of that. Basically, Azure Functions are generally the first stop for whatever needs to happen. Easy enough to get those scheduled if need be.
Seems very cool, I would love to see an advance use cases video!
An in-depth video on Hangfire would be awesome!
Really interesting, I'd love to see more in-depth preview, especially since I've been looking at Microsoft's Durable Functions, but also been wanting to move away from requiring MSSQL (esp since they don't support an ARM64 Docker container atm).
Hangfire is amazing. And their dashboard too. The worst part of the engine, that their client API is still synchronous meanwhile the most of storages they use have asynchronous API. Anyway, it works fine executing asynchronous methods enqueued.
Interesting content. Thanks! I still prefer message based scheduling, like Azure service bus or even rabbit mq, no pooling necessary.
Surely Nick you have played around with the scheduling options that you can build in .Net Core? What are the drawbacks that make you prefer Hangfire?
Hi Nick, quick question. If you want to schedule jobs but also queue tasks. Would you recommend using this solely. Or would you only use this for scheduling and then something like Mass Transit for queuing?
I think a video about scaling out would be cool. So many asp core api's are setup using scaleout in some way or another. So how do i make sure a job is only executed once across the instances of my api. Can i have another aspcore api be the worker and others be "enquers". Stuff like that. Looking forward to more of your videos !
My company has built in-house a background worker ui. It shows what background tasks are running, and how many instances of each per host, and we can scale up/down per host through the dashboard if needed
It would be cool if we could move it to something off the shelf. I'd love a deeper dive if something like this is possible.
Hi Nick. Please make more videos about Hangfire
While I love Hangfire and we also used it in various production environments, my by far biggest problem is that under the hood everything is static. While the surface looks nicely integrated into the .net service architecture, it actually really isn't. This makes it absolutely impossible to write reproducable integration tests because there is no way to reset the state of the objects.
i agree, while i understand that hangfire is from a time looong before ServiceCollection or other "modern" patterns (like a non-static builder pattern)
i really wish that hangfire upgrades to a non-static builder. I really see no reason not to.
If i want a static IJobScheduler, let me do that, but don't force it on me (have everything available static even if i don't want).
I do personally created some 'system tests' which is pretty much reproducable. (except the part when it is affected by other service - but you might prepare similar thing with more static data). However initial phase of it is to drop and recreate database and start from 'blank environment' -> next step is to create a HangfireServer which is assigned to 'test customer' (each server which we have is responsible of different customers with different databases) - after that I've created a 'WaitForJob' method which works on while loop searching for not finished (failed) jobs and let rest of the process to continue when it is '0'.
but indeed - there is a lot of static and is like part of documentantion also in places when you have other options. Dashboard however is a bit annoying, becouse if I want to cover new 'customer database' it have to be re-released becouse route have to be registred
Would very much like more info on this! Looks really nice, especially since it's not a windows service!
Hello everybody I'm Nick
no, i am
@@jacobmars1902 No, IM Nick, and so's my wife
I’m Nick and so is my wife!
It says in your Username you are Philip tho
@@BetrayedEnemy I'm Nick
Yes i do use hangfire in production, its reliable but need some validation and etc if job run long time it can be run again even if the job still run
Yes, you need to make sure that your job processing is deterministic
Buy the full version, this can guarantee that jobs do not run more than once even across different deployments
Unfortunately my post was deleted.
So can use it this way: private readonly ReaderWriterLockSlim locker = new();
public void MyReccuringTask() {
try
{
locker.EnterWriteLock();
// Do your stuff
}
finally
{
locker.ExitWriteLock();
}
}
Have you tried increasing invisibility timeout/sliding invis timeout? It's 30 min by default iirc so any job that takes longer than that will start over
Best video thanks!
It's the first time I'm hearing of Insomnia. Is this what you prefer using over Postman or is there certain features Insomnia has over Postman that makes it better?
It’s matter of taste in general or what’s in use in company you working at. In fact it’s all json files and ui - forms and buttons, cli or your preferred prog language script.
Insomnia is way better at saving params and collections in a Git friendly form (although not perfect), making usable for sharing across the team.
And I think it even has a free feature for sharing, but not sure on that one.
can you make it cluster aware using the settings ? if yes dose it take a strategy like round robin ... etc ?
Yes we want more Hagfire!
I am using it for years and it's very very reliable
Nik finally switched to a new JetBrains IDE design
How do you prevent the IIS hosted app from going into idle?
How do you authenticate the dashboard using Azure AD?
Please make that video! Very interesting stuff.
I’d use Quartz instead, at least for simple scheduled jobs that does not need any admin gui.
What do you mean "trust"? These are open source, you can look at the code. That being said if you want how everything works you should be using timers (and eventually roll your own scheduling system).
HF just works and fairly simple and easy to setup
How about securing dashboard (exposing it only to authenticated users) and splitting it from the project to a different domain?
Hello, I can't find the monthly subscription on the website, I think I have seen it at some point. I don't know if it's a bug on the website or it's just something I imagined. It would be great to have her back. Greetings!
There is no monthly subscription and there won't be until at least mid to late 2025
@@nickchapsas sad
This is very cool, but I can't really think of many use cases for this. What are people using it for?
I also liked Coravel
Is it next possible ?
- have web API with Hangfire as described in the video but only for job registrations
- have a separate service (several may be) on different host\hosts connected to the same Hangfire database but actually processed registered by API jobs.
Yes. Hangfire separates the client (job scheduling) from the server(s) (job processing). They don't have to run on the same host. Common approach is to deploy a Windows Service as a server instance. You can have multiple servers that are processing the jobs. They just need access to the Hangfire data storage. And for each server you can specify if it should only process jobs from specific queues (default is to process from all queues).
I use Hangfire, but sometimes it happens my job is run twice (I have 3 replicas in k8s). Maybe its because I'm using redis with two replicas as job storage?
One thing that wasn't clear in the video, when the scheduled job runs, is that code executing on the hang fire server, or is it doing a call back into the dispatching service and running the code in the service? If it's calling back into the service for execution, how does that work? When you have multi nodes of your service, does it always call back to the exact instance of your service that dispatched the job? Could that end up hotspotting execution? Thanks!
It's running in the server instance. Everything needed to run the command has to have been serialised, and if you are running a custom class you need to make sure that DLL is present and registered on each destination server.
Job dispatching is done from outside of the service, so there is no 'call back' occurring. In theory you could insert a record straight into the database from a command prompt, and one of your Hangfire server instances would see it, consume it, run it, then save the result in the job result.
Remember that this is not some form of RPC, it is a scheduler, so as far as client apps are concerned it's fire and forget. You have API calls to request the status of jobs by your own criteria, so you might add a GUID to your job data, then poll Hangfire afterwards and ask it if the job with that GUID passed or failed, but it is a scheduler.
I use this for large scale ETL processes and client emailers, with custom jobs. I run thousands of jobs per day, and send out hundreds emails, each being its own job. I can check the failed jobs list in the Dashboard for problems, and rerun them manually.
It is an extremely powerful system, and the Dashboard makes it more acceptable to clients than Quartz.
As said, it''s run on the server.
If you need scalability (and don't want to send a bunch of dlls to another server after each build...), use a webApi endpoint on your main application.
It may be slower, but you never will need to touch your Hangfire server.
Or messaging. Messaging works too.
@@RaMz00z I'm not sure what you're suggesting here.
Creating a webAPI prevents scalability, as it's a single endpoint. Whatever you do for scaling you will have to distribute custom DLLs (unless it's a simple enough function to serialise as in Nick's example which is simply auto-self-deployment). This is not a disadvantage or negative, it's just a necessity when you have custom code.
You could deploy your solution to an image then run it scalable on Azure or AWS, as Hangfire will simply add them as new servers available to process tasks, but the whole point of a scalable service is that it can run on separate resources to allow parallelism. Funneling it back into a webAPI seems to defeat the purpose of a scalable solution.
I didn’t find it super reliable, maybe it was the version I was on but things like attributes for controlling concurrency and retries just wouldn’t behave. I got on better with queues and serverless functions myself, which ended up being cheaper than hangfire as well as you don’t need the always on compute sat there polling and costing money.
Concurrencies and retries are not Hangfire's responsibility but your code's.
It is not meant for executing complex code. Call a webApi if you need to do that.
@@RaMz00zWhy does it have concurrency and retry attributes then?
I need a scheduling library that uses "circuit breaker - retry policy" pattern. Not on job level, bu across jobs. What I mean by that is the ability to group jobs by some sort of typing. Firt and foremost when they fail - how they fail. Let's assume I have many instances of the same job with different parameters, but relying on the same third party service. Or different jobs using that same service. And let's suppose that service is unavailable for some period. I need a mechanism that can put on hold all subsequent job instances relying on that service if N consequent attemtps to use it fail (spread in the job queue), and have some retry policy, that can even put them fully suspended after M failed retry sessions, but with the ability to switch "manually" that circuit breaker back on.
I am aware of both Hangfire and Quartz, bud haven't used them yet. Is any of these or any other similar tool providing something like this?
That's not the job of *any* external library, that's the job of your application code
Polly is a nice library to do that
After working a lot with Hangfire i kind of have to disagree on it beeing the scheduler you should use. Yes its a solid solution but for staging with a single db and multiple devs its at least for me a bit frustrating whereas coravel shines with its simplisity. Dont get me wrong i do thing i didnt dig enough to get it working as i want it too but mostly you dont want to dig that deep
Been using this for years.
I made my own scheduler with only five scripts. A tiny library that can run every and execute any action that you throw in there, using cron expressions .You have also full control of the action because it returns a unique Id every time you schedule a task so you can cancel it at any point by api call.
Hangfire sends an ID that you can cancel as well.
how about compiring hangfire to Coravel ??
Why would I use this instead of windows task scheduler
What if the prod user is iis don't have permission of create table?
Hangfire still executing everything (jobs, interacting with the storage) synchronously which is a big bummer. Everything will be a blocking call under the hood.
Hopefully integration with Aspire will become available soon.
Hangfire looks good, but personally I prefer Quartz 😅
Since dotnet core, when hosted inside IIS, Hangfire goes down when app/api goes idle. Hence scheduled jobs are not executed on time. None of the solution worked for me ever and I moved to dotnet background workers (WS). Did anyone else faced similar issue, or was it only me doing something silly?
Hangfire causes some high cpu usage on my dedicated server sometimes
I don’t get the point of hard coding of adding jobs in the code, does the dashboard allow me to add jobs dynamically?
Not to my knoledge no, it isn't the usage intended.
Hangfire uses code static anylisis to serialize the code needed to be executed. How would you do that in a web form ?
isn't serverless already replaced it?
Once my database reached its storage quota causing website downtime.. Hangfire did not delete its jobs
Doesn't do any deleting by default, but there's a param for that
Why just do not use a message broker with default dotnet background worker (hosted service)?
Cannot do scheluding, i.e. executing code in the future or every monday at 6
I had to dump hangfire for quartz net because hangfire retry policyfor schedule jobs doesn't work as expected, and it does not allow me to run jobs in parallel.
Care to elaborate? I though parallel jobs with Hangfire were kinda the thing it was supposed to do (other than scheduling) based on the worker count? Maybe I have misunderstood something.
I have been using HangFire for a while but I don't like that I need to have the connection to the data storage every place I use the scheduler, there is not a concept of HangFire Server and HangFire client libraries.
You can use the SideCar pattern for that.
Basically you have one specific service that has those infos, and you call that service when needed. By means of webApi or messaging.
yeah, this makes sense in certain scenarios, however I pretended to use Hangfire as a separated pod or as a separated service, to manage scheduled services across organization, maybe is not intended for this scenario.
How much are they paying you
Hangfire is great to do background job but the dashboard becomes really slow after some time because of past data.
You can setup a cleaning policy on the server (I'm pretty sure).
Or just clean the database yourself.
@@RaMz00z yes, that is one of the way but it does not help for case like you want to process thousands of data on a day and you just want to monitor of what went well and what went wrong. You open the dashboard and it feels really slow.
Neat Ad
Does HangFire support Timezone scheduling yet?
Yup, TimeZoneInfo is an optional parameter when scheduling recurring jobs
How do you debug it?
The usual way?
Yes
Hello Nick I’m everybody
Ciao Vittorio, bene colpani, buon gregario.
Per Luis Alberto ripeto quello che dico da vent'anni, non capisco perché aspettare le offerte,
Le società serie sono proattive sul mercato perché preferiscono monetizzare il giocatore piuttosto che lasciare che il morbo covi nello spogliatoio fino a corrompere tutto il gruppo come più volte denunciato da Sarri
there is so much ways to create background threads, I would not use this lib at all. Only if I had to
It's not intended for background threads but for scheduling. Even tho it can effectively do that as well
What IDE is he using?
Rider
Rider
7:28 What's Insomnia?
Postman alternative.
Hangfire is terrible. I’ve had cases where I’ve disabled schedules and they still run. The database has a fully qualified namespace to a method that will run and it invokes them when it shouldn’t.
This Hangfire gives me so much headache everyday...
What kind of issues have you had?
Hey Nick. Where were you born?) just curious
I would rather see a video that explains how to make your own scheduler. Hangfire is badly implemented, Quartz requires a big chunk of manual labor, and Coravel is behind a paywall. Making your own is very easy anyway.
Not invented here, why use other people http servers and messaging frameworks, when you can do it yourself.
@@AndersReinhardtHansen My point is that you can easily write a scheduler that does the job instead of using a library that either contains a massive amount of overhead or doesn't to the job you expect it to do.
Agreed. The only reason I tested Hangfire but was using Quartz in the end, was the convenience of CRON expression.
However, even Quartz generated some headaches, which motivates me to make an own scheduler. The mess with timers in .NET and how to pick the right one, was the "hardest" part of it.
If you need a scheduler only, build one and sleep well.
@@fusedqyou yeah and that is my point, you cannot make a better scheduler in the time it takes to setup a hangfire job. There is also the added advantage in the fact that if you have used hangfire before it is even faster yo setup and read the code.
@@AndersReinhardtHansen I think this is up for debate. Most people can just write a simple timer to satisfy their needs of a scheduler. I didn't spend longer than a day to write a basic scheduler that handles multiple tasks in a specific interval, so if there was an actual tutorial on it then anybody could make one.
This course advertising is so anoying
Problem is, it was developed from Russia. I introduced it with a customer 1 month before the war and I was unable to get the licensed version which I requested, because of company policy.
Edit: is is no longer based in Russia
Started in Russia, they are now in Estonia. Same with Jetbrains, do you stop using Jetbrains as well?
I thnk hangfire is free are
@@guy-heylens they moved during the war, jetbrains before. It was not my choice not to use it. The decision was done in the purchase department. And at that time the company was based in Russia, they were writing a blog about moving to Estonia.
@@xelesarc1680the basic version yes. I needed the pro version.
What stupid reasons, just because it is "Russian" software. What a pathetic world we lived in? We wouldn't even have to use American software at this point if we were consistent
very buggy lib, this is a promotional video
First comment! Send money. 😂