How to use sed to automate big code changes (GUIs hate this trick)
Вставка
- Опубліковано 26 чер 2023
- sed is a great command line tool for making code changes to codebases of any size. In this video, we'll look at several tips for quickly getting up and running with safely making these types of changes.
The General Problem: xkcd.com/974/
My Links
shaky.sh
shaky.sh/tools
/ andrew8088
mastodon.social/@andrew8088
#terminal #commandline - Наука та технологія
3:50 Correction: It's called Forward Slash (/). Backslash is \
Great video Andrew 👏
Great video!!🎉
great vid Andrew!
Amazing description of sed substitutions. The extra arg in sed -i always drives me nuts in scripts that run on OSX and CI Linux
This is a great video, coreutils are awesome and every dev should get familiar with them
You don’t need to find the exact files that you want sed to process. sed -i does nothing to input files that have no matching lines.
I typically run sed with -n, which removes all non-printed lines from its output, and the “p” modifier, which prints the changed lines, on all of the “s” commands (yes, I put several “s” commands in one sed command), and then pass all of the files, or all the C/C++ source and header files, in the directory to my sed command. The output is a dry run that shows me what sed will change. When I’m satisfied with the changes, I remove the “p” modifiers and change the -n option to -i.
Bonus tip: If you run sed -i, but didn’t remove all of the “p” modifiers, sed will edit the matching lines in place and print them, resulting in duplicate lines in your files. The uniq command can be used to remove the duplicates.
Great video! I never would have thought of using sed in such a situation 😅
One note is that you have to make sure that you're running the command from a src folder, as to exclude node_modules, documentation and whatnot.
Actually, most grep-like commands follow your gitignore so node_modules would probably not be a problem
You should be careful that you don't replace more than you wanted, in particular you should take care that you only match "I" as a complete word. In your example that can be done by putting "\
Cool way of doing it! But this seems to be breaking imports, no? I also have not seen an editor that can’t do this built in and even update imports etc. at the same time. Is this ever better than a standard editor?
I'm new to bash. I know this wasn't the point of the video but wanted to share what I'll be using moving forward. I've been doing this, "for file in $(grep -l ) ; do ; done; " . Very cool to see the cleaner "grep -l | while read file; do ; done; "
Also, the tip of leaving the echo in place to see the command that will run is awesome. I'm going to use that. One last thing when I'm doing some for (now while) loop like this in bash command, I'll add a " | head" in some place so my output doesn't fill my terminal, and remove it when I'm ready to run the full thing.
awesome!
You might want to consider a perl one-liner in place of sed. You can get the same in-place behavior and you can easily work with multiline regexes.
Yeah! I've done just a little bit with perl, and it's scary powerful!
OK what I would love to have in a future video, is our beloved content creator... I would love to have kind of a biographical time line but of the learning process, like at first someone barley know the existence of a computer and then discovers its passion for something in particular, I would like to understand the order in which he learned his skills and art... especially every think that is accessory to the main languages and technologies and stacks... Type touch, vim or Neovim, git and git hub (or maybe he started using GitLab or what not) and bash scripting (this is something that I have learned and I am always curious about how normal human being are learning that kind of cursed language), I am using linux so I would have a lot to say about my learning curve of the OS, personally I was unable to use Windows anymore after owning a macBook Pro for almost 10 years and not having enough money to get a new one while having an expensive desktop computer made me chose Fedora but having a macOS based system is amazing and maybe te prodigious and modest superhero would be interested in telling us if he started with Windows or since when he is using macOS and what not... for my part then listing all the main stack(s) libraries and programming language he learned would be enough but maybe for other it would be worth explaining in as much detail the time line for that... I am very interested in all that's not the main chunk of the stack because all those other accessory elements that are super important in fact are always forgotten and never shine in any stories about the thing one might know, love and use... _(so future me told me not to use ChatGPT to fix any typos because it would end up taking 90 minutes and I would not be able to get the message I really wanted)_
Or just use a good editor that can find and replace all over the project, even vim can do that quite easily. coreutils are great, but there are much better use cases to demonstrate their power.
My thought exactly. If you are simply doing a search and replace any IDE/editor can do that with ease. I was hoping for something that could do more complex code manipulation but this clearly isn't it. Just an overly complex way to to basic search/replace.
The point of sed is that it a _non-interactive_ editor. If you want to write a script that updates the version number of your project you will use something like sed.
Obviously, you could also do that by hand, using an _interactive_ editor, but why would you if you could just use a script?
@@bierrollerful There is nothing 'interactive' about normal search and replace in a code editor. It is no more or less interactive than a script or command line tool. It IS easier. Search and replace by command line is stupid.
@@TimothyWhiteheadzm Non-interactive means that it does not require further user input.
Imagine you want to make the work of finding and replacing some text repeatable. For example replacing the version number of your project, which might be in multiple places all over your project.
With an IDE, you would note down the steps (or the _interactions_ you have with the editor) you need to do. Then, whenever you need to do the same work again, you retrace those steps, interacting with the IDE in the same way as you did before.
Or you can do the same work from the command line. You'd use grep to find all the files with the version number, and you'd then use sed to replace that version number with the new one. This is all done just with function arguments and entirely without user prompts.
Why bother? *Because you can reuse those commands in a script* (or makefile). Doing the same work again then becomes a matter of milliseconds. You can also have that work be automated (f.ex. a github hook). But most importantly: *anyone can reproduce the exact same work without fail by simply running the script.* If you work on a project with multiple people, scripts (and thereby using command line utils like sed) become mandatory, because people are terrible at following manual instructions.
I actually think this is a good tutorial because a lot of the time you don’t have fancy editors and especially custom, configurations and remote machines so being able to do something this complicated with something as simple as sed is actually pretty handy
OR you can use vim's quickfix list, with extra superiority over gui boys too:
`:grep -r /path` (or otherwise populate quickfix list, e.g. in Telescope)
(optional) `:copen`
`:cdo ` do any command on matches, e.g. `:cdo s/pattern/replacement/`, `:cdo norm! }P` (go to next paragraph, paste before the cursor)
`:cdo update` (write modified files)
I've been putting off learning how to use the quickfix list, but I really need to get into it.
Crazy just this week i was working on a bash script, wanted a basic PascalCase util with sed but macOs sed is just different and output some ugly characters like "U" for no reason
how do you do the same using MACOS sed
I really like your content Andrew, (I do!) however this seems to be an awfully hard and error prone way to replace built-in editor functions such as 'rename' or 'search and replace'. In my editors can also be supplemented with regular expressions to be even more powerful. Learning completely new set of tools just to mimic built-in editor function seems... like overdoing it a bit ;)
That's what I thought initially, but probably there are people around the world that doesn't use "fancy IDEs" or just love doing everything in the terminal, specially those who use vim, it ends up being easier for them.
Edit: Sometimes you won't have an IDE to help you out, it happened a lot for me and knowing those type of command would be very handy at the time
@@fagnersales532 Can you give an example of when you won't have an IDE to help?
@@mr.vladislav5746 When I connect to a computer through a terminal.
new set of tools? regex is regex (usually)
I use vim and sed extensively. vim’s “s” command is fine for changing patterns within single lines, and the “g” command (executing a “d”, “t” or “m” command) for adding or removing lines around patterns. But nothing beats sed for finding and changing patterns spread across two or more consecutive lines. I also use a code formatter, which breaks lines at unpredictable points, so I have to find and replace multi-line patterns often.
That's when I started to learn Perl. So I didn't have to learn bash, sed and awk. ;o)
/ is actually a forward slash ;)
That slash is a forward slash. That one is the back slash: \
Main reason I learned 'sed'?? So I could use 'awk' a lot better. :) Really, once you've got 'sed' down pat, 'awk' is super. Over the years, I've done so many code imports and refactors using awk.
3:50 that's a forward slash /, not a back slash \, doesn't matter much tho
You could use sd from rust
Thanks for telling about this tool. It looks pretty good.
But there is only one Monday per week and only one 8 am EST/EDT per Monday every other time slots is a *never*
Lol, I'm switching it up a bit, thanks for noticing :) need to update that banner
@@andrew-burgess I was complaining that it was only once per week and would have not being using such a sarcastic tone if it was not because I was implying a compliment ― that is that I would love tho have more content implies that I love it... but to be hones I do love the presentation and the presenter also... Edit: I am not « in love with » but I meant to say I do appreciate a lot the beloved content creator _(I did't wanted to sound like anyone was exposed to Pink Kryptonite here) ..._
What editors are you using that don't have a findAll/replaceAll functions?
Nice title lol
I used sed to modify playlist files (.xapf) so that I could use them on other devices. It’s really handy.
My solution would be similar to yours:
perl -i -pe 's/I/AsyncIterable/g' $(find -type f)
-p would be the implicit while loop with automatic "printing" the changed lines into the file.
Love this! I recently learned about "Perl pies", so cool!