Stop using the Process class for CLI interactions in .NET
Вставка
- Опубліковано 28 вер 2024
- Check out the ABP Framework here: bit.ly/3NGacLw
Check out my courses: dometrain.com
Become a Patreon and get source code access: / nickchapsas
Hello everybody I'm Nick and in this video I will show you how you can control your machine's CLI in an elegant way in C#. .NET's built in Process class is a total mess and really hard to work with so we will be using a Nuget package called CliWrap, to fix all of those problems and add some very nice features on top of it.
Give CliWrap a star on GitHub: github.com/Tyr...
This video is sponsored by abp.io.
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: bit.ly/ChapsasG...
Follow me on Twitter: bit.ly/ChapsasT...
Connect on LinkedIn: bit.ly/ChapsasL...
Keep coding merch: keepcoding.shop
#csharp #dotnet
Being a .NET developer with over 20 years of experience, I am still getting awed over learning new things. Nick does such a great job stretching my knowledge on topics that really matter to my day-to-day work. Nick's examples are spot on and his ramping up from a basic "hello world" to the more advanced is perfect. Excellent topic and something I'll definitely be using.
Exactly this. This is literally the only coding channel i actually subscribed to, you just always learn something useful, regardless of your experience with the subject.
It helps that .Net has evolved significantly in 20 years. Like with most things, it's important to stay on top of the changes.
Your comment as a whole just shows that you're a decent developer.
You have a lot of experience, but you're open to keep on learning more.
So many times I've worked with contractors who are arrogant and are stuck in their ways.
I'm a tech lead that oversees a lot of senior developers and I have a similar amount of experience as you do, but I always think you can keep on learning in this profession.
I've learned tricks from Junior devs countless times and I love it.
That would hurt some developers egos and I find that attitude bizarre.
C# devs: This is cursed, I don't like overloaded operators for gimmicks like this
C++ devs with printing using bit-shift operator: Haha, classic
haskell devs: oh, you guys finally started overload operators
@@maybe4900 haskell overload? what you need to define
Thanks for another great video. I discovered CliWrap about 3 years ago and its been my go-to library any time I need to shell out from a process to call another executable. Especially when I need to process the stdout in a specific way. It has many many options for handling stdout.
OH MY GOD I'm so happy that you make a video for this package.....this is for me THE package of 2022!!!!11!!!!
Just used this library on a project after discovering it on your channel a while back. It's a joy to use indeed!
This library looks incredibly powerful. Didn't know this existed and have needed something like this several times in my projects. Thanks for sharing!
This is exactly what I need. Had to use ffmpeg before and it was a huge pain :)
Wow. That video came into my feed just at the right time. As I was about to start a new self learning project around using git from C# through command line.
Great stuff keep it up Nick.
Given .NET's relationship with PowerShell, and PowerShell's use of the pipe operator in the same context, it seems natural to use this syntax. Elixir and F# also use |> for piping functions together, and there is a proposal for the same use case in JavaScript. Other than that, great vid and cool looking lib!
I remember when I implemented opening a file in a WPF application and every time I would press the button, the console screen would briefly pop up for some reason. I couldn't figure out why it would happen and just went with it. I really hate how process class works. And I had some weird experience that it wouldn't work in .NET core unless I used ProcessStartInfo overload. I have no clue to this day why is it so weird, but thank you very much for showing a nice alternative.
That is pretty neat, wish I'd known about it last time I used the Process class. I didn't even think to look for a package for it.
I've been waiting for this nugget for so many years ...
11:20 👀 what kind of sorcery is this. Very nice library!
A valuable component !! Thanks
Now sometimes in the near future you have to make a "Top 10 cursed C#" video! 🤗
I’m actually gonna make a talk out of that
Have you tried Podman & Podman desktop? at work we've switched over to this due to it being open source; you can even alias docker=podman and it works pretty seamlessly
Wow you had me scared until you mentioned CliWrap which is what I have been using.
Did not know you could overload the OR operator! Could make things very messy if you started abusing that I think
Hi Nick! Currently Amazon is helding giveaway of 120 courses for AWS. Could you please advise what technologies are crucial or will benefit me as a .net backend developer and I must know it. I'm new to aws
Sounds just like powershell but inside a program
🤤 🤤 🤤
In elegant loops video you abuse operator overload in all sort of ways and suddenly here you're being skeptical about pipe operator 🙃
Actaully it isn't operator overload in that video. It is duck typing. Equally cursed
*Summary*
Problem of the video: _Process_ is hard to work with for executing commands in CLI
Proposed solution: use a library called CliWrap (link in desc)
Most important: 1 -> 2 -> 6 -> 5 -> 9 -> 12
Cool stuff: 5, 9, 12
1) Sample code - 3:08
2) _Execute_ can't do stdout/stdin, _ExecuteBufferedAsync_ can - 4:11
3) CliWarp supports parameters, so no need to escape strings - 5:37
4) You can use a builder instead of an array - 6:02
5) CLI events - 7:08
6) Simpler stdout - 10:18
7) Even simpler with an operator - 11:08
8) The *|* operator - 11:24
9) _CancellationToken_ - 12:10
FFmpeg example, Nick doesn't run the code:
10) With the *|* operator -13:12
11) Same but looks more C# - 13:48
12) Process-to-process piping - 14:09, notice the "|" operator from 7) and 8)
Finally I am ahead of the legendary Nick Chapsas on this one as I have been using it for years and even sucked in the source code so I could make a change I needed to it. Take that Nick: booyeah!! ;-)
Matthew I will disappoint you but I’ve been using it for years too. I don’t make videos on thing as I discover them 😂
Seems very useful. It would've been nice if you had shown how the equivalent is done using Process class, even just going over pre-written code.
What a timing! I've started using Process in one of my projects just this week, and this library looks quite interesting.
I agree the Process class is clunky, but I don't think it's _HOORRRRIBBLE_, as you say. I also agree this is an improvement, but it's not a striking improvement.
Been using this dream-like library for maybe 4 years now? It has made my wrapped CLIs look clean and understand easily for new developers. So happy you covered this gem.
I was skeptical about "is null" syntax, because it didn't felt C# like.
But now I prefer it above "== null" 😆
The cliwrap | looks great, just let it grow on you.
x is null is equivalent to ReferenceEquals(x, null)... It doesnt use the potentially overwritten == operator
@@DennisHaney True, but to be honest: I have never ever seen anyone overriding the == operator in production code ;-)
@@thomai5287
It doesn't mean it can't happen, also is null and is not null is way more fluently readable
@@thomai5287 Unity does it
That package got some political terms of use though :/
i saw him like it in github so i knew the video was coming hehehe
CliGyros 😂😂😂
this in F# would be cool af (specially with the pipe lol)
I had and hated to have to deal with the Process class, flags that you have to set correctly to get process output via events... This looks so much better.
For me, even with the right flags, it always misses the first few output lines, I don't understand how and I don't wanna spend so much energy on that garbage
i can’t wait for the System.CommandLine namespace to go public
The overloading of | might look weird from a "C# dev" perspective but it looks intuitive from a "unix command line tool user" perspective. Note how Oleksii's layout of the actions corresponds to unix piping of > redirecting to stdout and 2> redirecting to stderr.
Hi! Amazing! Thanks for this! So much technology used in 1 API. This is really unmatched and deserves a price! Basically it's turning your application into an extended shell script, which is indeed very fascinating. The whole pipeline part also immediately reminds also me of PowerShell (since it's also a .NET environment combined with shell functionality). I love to see a more of this functionality in C#. And of course the C++ style operator overloading thing makes it even cooler, even though that's also a bit less typical for the C# language indeed.
I am a total shell and Linux geek as well and I also see use cases for it in relation to this. It's likely going to replace some of my shell script work on both windows and linux.
But do I always need such a whole extra package to my project? For very simple commands in code I might still use the old default Process.Start(string) when I don't need any extra features or read any output from it. I will find that out.
You don't need most shell scripts if you have PowerShell. If you like C# and are a shell geek, you should definitely learn PS Core.
The | override is sick! Tyrrrz is awesome!
Hey Nick! Could you do a video of what is Nuke (the build tool), how to use it, and when to use it? Thanks!
Reinventing UNIX shell scripts? I used them in the 80ties of last century. UNIX is a great OS, and nice to see few features now in this library!
Anyone know how to use CliWrap with libnotify / notify-send? I'm having issues running those on a C# Linux daemon.
Normal .Net Process library works fine, but only when the dotnet CLI is running a dev copy. My Linux daemon says it's running `notify-send .... ` from Bash, but nothing pops up.
Great video. This looks way easier to use than the Process class. I can't believe I didn't know about it sooner. And that pipe syntax is so sweet; looks like writing PowerShell in C#, and is very intuitive if you've ever written PowerShell. Very cool.
I'm getting System.ComponentModel.Win32Exception errors on multiple machines in different environments. Windows 10 and 11 .net5 and .net6 console apps.
We have app/site hosted using Nancy and when we run it form cmd it print "Press ENTER to close the host". I tried to run this API using CliWrap - but it starts and closes instandly - looks like CliWrap clicks/passes ENTER. How I could prevent that and keep app running?
No. Any API that allows you to invoke another process by specifying arguments in a single string separated by spaces will INEVATIBLY lead to bugs or security issues. Programmers will stitch together the argument list by string concat, will not escape the parameters, one of them will have a space either by accident or maliciously, and yiu have a guaranteed time bomb ticking. All, literally ALL languages and frameworks that allow the programmer to do this had introduced major CVE security flaws that can be traced to this.
Never. Ever. Ever allow to pass arguments to a process like cliwrap does. Never. It will ALWAYS lead to issues. It does not matter that you offer a safe way, programmers will always abuse the EASY, UNSAFE WAY. This is a chance missed by cliwrap to only allow safe passing.
Besides of that, of course cliwrap is great especially because of the piping. Ffmpeg is such a classical example, with the typical use case being the.net app being a stream source for custom networking, or streaming. Great how cliwrap allows you to stitch together pipelines.
Man i build 5 years ago, an entire app on top of CMD, if this library was available, I would build a more stable code and probable this app will still run. Men I am so happy
Also, thanks for the video. As always, super practical and helpful.
Haha, Nick you are being too nice to say "Process" is a mess.... I would call it crap! Or something worse, that I wouldn't argue with you. Can't agree with you more that the CliWrapper package is great. It saved my life to getting an install i needed to automate. The nice with with the CliWrapper, there is no upper limit on the console output as their is with Process(), think it is 1MB IIRC.
The overloading of the bit or operator isn't good, if it can be done with the fluent api then i'd assert overloading the bit operator is so so terrible...
Hi Nick, i just tried to buy one of your courses and i keep getting error “The card was declined”. Im using just a regular credit card and i buy stuff on internet without any problems but unfortunately i can’t buy you course for some reason. Also - not clear how register on you website, ther is no register button/form
Seems the owner of that project should add a named pipes client to spawn an app and communicate back and forth. Essentially a scaleable app via multiprocess vs multithreading
Looks very good. I use process and really only found a problem when I was want to do multiple processes and get the output from each process. I haven't yet seen whether cli wrap can do this as well. I have used process to do such things as call ffmpeg and convert say a range of durations of mp4 to a streamed raw format. The processes of durations all run at same time and use hardware acceleration and gpu. Yes cli wrap looks easy but I think have to find whether it will do what I want. At present I am trying to create a program that creates a processes for tracking counts in a video or series of videos and reports outputs of counts for various range of frames and finds the maximum count per minute for the 10hrs of video. I can do it in 4 minutes but there is a problem that each process even though I set them off together the subsequent jobs wait for the error and STD output to finish before outputting next.
Would love to here from someone that has used cli wrap to perform such an operation.
No way, I was just searching about what to use instead of Process because it was giving me horrible experience.
Then I see this on my feed a day later.
No idea why i would use this library, it is not like i have ever deadlocked an application due to some error in my code handling the streams... Oh wait... Never mind.
This pipe operator makes it read closer to PowerShell. I guess having a method like .Pipe() or .PipeTo() would make it more “C#-like”. In any case, this is very cool stuff.
Hi sir can you make a video on .net core application architectures like you have a Asp. Net core Architecture and in that upload videos one by one two tier, three tier , N tier and clean Architecture etc with folder make separate video for each please sir
Always learning something new with Nick, great vid!
It reads just fine if you're coming from PowerShell lol
Hey nick can you please overview the Microsoft orleans framework
tfw you realize the traditional pipe operator uses the same symbol as the bitwise OR operator
Passing the arguments via an array seems like a good idea. However, on windows, the command line parsed to an application is always a single string (contrary to linux, where it is a string array). This means that argv is dependend on the compiler/language used to compile the target application. This results in different escape mechanisms being used by different compilers and thus, applications. So the only way to be sure that argv escaping is done right for an application is to use a single string. On linux, it's the other way around.
Amazing library, thanks for this video
Looks real nice. I'm gonna give it a shot the next time I'll need to use Process
I actually wrapped Process myself in a professional context so simplify the API for integrated testing, so I think that designing a fluent builder around it is a good common sense idea.
That being said, overloading the | operator is rather counter-intuitive. For one, it changes the semantics of the operator, which is currently reserved for bitwise OR operations. While you might be used to seeing it in a CLI context for piping, the fact that this is essentially the only CLI trapping they're reproducing in C# results in code that ends up looking like neither C# nor CLI. I don't feel it makes the code cleaner; on the contrary, it introduces more confusion as to the role of the operator, given how it changes behavior based on whether it precedes or follows the Command object. Heck, even from the perspective of reviewing code changes, I might be tempted to isolate the redirection arguments on their own lines, so the added verbosity of WithStandardInput/WithStandardOutput/WithStandardError doesn't end up taking significant space, while making the intent explicit.
Operator overloading should be used when you can remain consistent with the semantics for that operator in the language you're using. Don't introduce confusion to developers by playing around with language conventions.
Mongo driver already is doing that, it overrides the & operator for adding multiple query items and it may seem weird at first.. But it becomes very quickly easy and intuitive
@@p4xx07 anything becomes easier with practice. That doesn't make it intuitive. This is why tools like VI can be incredibly efficient in the hands of proficient users. They're not designed to be intuitive, but they're designed to leverage expertise faster. Same goes with efficiency suites like ReSharper.
And while I think that this is a great thing to have when it comes to tools, I think that this has the adverse effect when it comes to language syntax. If you're playing with the syntax of the language in a way that is not consistent with the conventions of the language, what you are doing is by definition counter-intuitive because you change expectations about what the code is telling you. You might think it's convenient because you have practice with it, but unless you're familiar with the intricacies of the Mongo driver, the code itself leads to confusion.
That doesn't mean that operator overloading is bad per se, even though this fundamentally changes the syntax of the language. For example, in C++, smart pointer types overload the dereferencing and member access operators because they allow variables of those types to be interacted with as you would a standard pointer. And this is by design because smart pointers should be interchangeable as far as using the variable is concerned. No expectations are changed for consuming code.
So unless operator overloading plays to the expectations of developers proficient with the language, you're likelier to make maintenance more complex than what you can hope to make up for in cleanliness.
I was thinking the same but it's undeniably the best option for readability if you're chaining commands like the example at the end.
@@dominicc1804 I disagree that this is the best option for readability because you have to ask yourself which of the output pipes is being used as an input to the second command. So a C# developer cannot easily infer behavior by looking at the code.
Not to mention the fact that the readers an writers used for input and output pipes are IDisposable objects, and they beg the question as to when and where you might want to dispose of their resources.
Less verbose code doesn't always make things more readable. There are reasons why variable names are no longer limited in length. Sure, less verbose code might seem more elegant, but if it introduces ambiguity, it's self-defeating.
@@marcusmajarra in the example that was given, calling a command and piping the out of that command to the in of the next, I think it's more readable than the alternative. I would envisage using this in a helper program for some mundane task, rather than production code, and it's certainly prettier than powershell. It reads pretty good to me because I use piping in terminals anyway.
What if I want to keep the terminal open for CMD? It always exits.
CliWrap is so easy to use, even I'm able to use it effectively.
"But as someone who's a fan of cursed C#"
I would love a video on this
The process class has an absolute garbage developer experience. I have had to build multiple applications with ffmpeg and this would have made it so much simpler for sure. I'll definately try it out next time!
I hope it supports ensuring the application is killed when your app exits
So I'm not the only one that gets deadlocks due to unread std buffers?
Even from inside google you won't get 4ms ping lol
It kind of makes me wish I had some nails to use this hammer on XD
I've been meaning to try CliWrap for months now. Thanks for this
A glance of power only PowerShell has
That overloaded pipe operator made me do a double take
234profit
I like Nick and his videos, but I don’t like how he cargo cults some rando library every single video. Maybe interrogating the internals of a library and adopting the design patterns rather than the whole glob of code?
He says Process is a pain to use, but also extremely flexible? 🤔
Honestly, making Process easier to use is a single gist/helper method away.
I find this take very odd. CliWrap has been around for 5 years, it has 2.7 million downloads, 93% code coverage and it is being developed by one of what I consider to be the 10 most important .NET OSS maintainers. The Process class is a mess and Microsoft knows it. It is as flexible as being given, water, flour, tomatoes, yeast and cheese and was asked to make a pizza without a recipe. It’s a terrible developer experience in which CliWrap improves in every way. You will find very quickly in the comments that a single gist/helper method aren’t enough.
@@nickchapsas Have to agree, was tasked to write and installer that would go into a docker image. I got to the point if i never captured logging output, it all ran fine. But registering an output stream on the Process, guaranteed hand and had to abort my processing after 20 or 30 minutes... Process if you hook up standard out had a hard coded limit at 1MB(???), and not easy to get around. Found CliWrap after searching for the known limitation in Process, and not an easy work around, but pluged in CliWrap, no more issue and got logging and the installer to complete. Not sure it would have even been possible with Process... You want the red pill or blue...?
Escaping parameters ... why not handle an interpolated string as such? OK, not a well known goodness, but it's useful.
At the time, string interpolated handlers weren't a thing and now it would be a big breaking change to replace
@@nickchapsas EF did it with a separate method. ExecuteSomethingInterplatedOrSomething (too late for google). 🤣
Very cursed C#, I love it
The name could be better thou
Library is awesome....
Excellent !! Would love to see more CLI / Console videos.... Who would have thunk that Console / CLI are back in style thanks to DotNet and all of the automation happening today.
We had Brandon Herrera with the heading "Cursed images of weapons". Now, apparently, we will also have the heading "Cursed C# libraries" =)
14:21 100% cursed syntax =)
In the next video with a similar library, I suggest starting with the scene of a bottle of strong alcohol like this
ua-cam.com/video/eiGUJV4d_Vc/v-deo.html 🤣🤣
This is a life changer! At least for me :D
👌
Hi Nick. Way do you make this video only now ?
I already have lot of code that use the new Process() with process.OutputDataReceived() to catch the output.
CliWrap is better
I have some code using Process code to "pipe" a cloud storage stream into sox and then back into a cloud storage stream. It took so long to get working properly, and it's super ugly. This... this is fantastic. Gives me a reason to upgrade that service to net7.0 and use CliWrap and PipeSource!
I tried MedallionShell before, but it didn't work as expected for some reason. System.Diagnostics.Process has always been a pain. Thanks a lot for this !!
This is awesome. All of a sudden I can do C# commands. Before it was just too painful. Is there a `Tee` available?
Absolutely great library design to leave the decision on how to use the piping up to the developer.
I mean, this library truly enables you to rewrite all those hacky bash, PS or python scripts for automating everyday stuff in a very script like maner, using our much loved language of choice ;-)
And if there's the need for production code quality: no problem. 2mins of Refactoring and it's done.
Also great presentation - as always. 15mins of Nick every now and then saves lots of hours of research :-)
Sophisticated scripting of CLI using c# 😁 Could also be used with c# scrips?
Man, thank you! I just started a project where i build an internal tool to basically just fire of scrips to remote machines, and i did run into everything that you mentioned in your video. I will change to CliWrap asap.
Well, I don't really like those pipe operator shenanigans. This looks cursed as hell, it's so confusing.
Good one nick, liked it!
Ok, this is pretty cool. I was a bit skeptical in the beginning but it grew on me.
I love CliWrap. I thought you showed it to me a long time ago, but maybe it was Tim Corey
This very week I have been struggling with Process because I have a requirement where I have to create and pack Conda environments from C#, using very elaborate commands to prepare the environments. I leveraged F#’s asyncs and callback hell and have a hard time getting the arguments properly escaped.
This library looks like a godsend, I will try it for what I want to achieve. Thanks for showing it!
I discovered this library by the exact same reason, to avoid Process. Great video!
10:55 So by overloading the | he just PowerShell'ed C# :D - This is very common in PowerShell and Unix/Linux where you can pipe one result to another command by piping (|). Having done lots of PowerShell this actually looks quite "normal" to me.
Yeah the idea is borrowed by that concept. It’s just that this operator is used for something completely different in this language so unless you know what it is supposed to be doing, it can be very confusing
Nice! Was looking for this cause I need to automate a ton of things and in particular the piping mechanism from one to the next or context relevant method is what I need.
did you know you can color the console with RGB on Windows :)