We recently had an email service token leak, which resulted in 1.2 mil scam emails being sent from the company noreply mail, so this is very important advice!
@@IsaacShoebottom I think if that works for you then go for it! In my case I much prefer to do a quick git reset which also prevents any false positives in the future.
@@dreamsofcode I think I would do the following: 1. Identify existing leaks and change secrets 2. Integrate the tool as part of my workflow thereby preventing any new leaks IF a new leak somehow occurs and is a false positive you can update the baseline of the tool so it wont look any further back
8:12 force pushing to a remote is fine if you own the branch. Don't force push shared branches like main. Do force push branches that only you have committed to, it's a very good way to clean up and reorganize a branch pre-merge. If you feel uncomfortable knowing when it's ok to force push and when it's not, study and practice git, it's worth it.
Agreed! Your own branch is fine (I believe I mentioned that), but never main (unless it's your own and you know what you're doing). I was definitely erring on the side of caution with this message as I didn't want to encourage people to force push without knowing when it is useful.
@dreamsofcode I think you did too - just hoping to add a little extra clarification for people new to git who are a bit too scared of using it effectively (in my experience)
If they're on a remote repo, as I mentioned in the video I would instead rather do an investigation otherwise you might be encouraging unnecessary downtime.
I'd love to see videos in the future which go deeper into especially those things said in the last minute (Pre commits generally, other secret solutions and generally secret managers) ❤
I'd go with `git commit --amend` if you really just need to fixup the last commit. No need to temporarily undo it completely with `git reset --soft HEAD~1`. Also, one should use the "dotenvy" crate for Rust; "dotenv" is no longer maintained.
Your spinning warning symbol graphic at 8:07 seems to rotate around an axis aligned with its face rather than its midpoint, and has a weird smear on the frame its parallel to camera.
Thanks for thoroughly explaining how to resolve these issues as I bet this guidance will help give people the solutions they need to get these issues approved to fix.
I liked your video. Thanks a lot for doing them. What about the git filter-repo and other tools that might edit only the needed parts of the commits history?
git-filter repo is pretty dope from what I've seen! I've never used it personally however as I tend to stick with just the basic git commands. Perhaps that's a good excuse to do a video and check it out!
3:00 This is MUUUCH easier with lazygit, you just go to the leaking commit, press enter, find the file, press enter again, select the line of the secret, press ctrl+p to open custom patch options menu and then press d or select Remove patch from original commit and you're done
A note on `gitleaks`! Please think about how the tool does detection and react accordingly. Do not blindly run the tool on code you do not personally have distribute rights to. Definitely perform a full audit of the tool when running against mission critical code.
Although I share the belief that good hygiene is better than tooling the git commit around with pre-commit, in a not-so-big team that is already difficult to enforce. And code-reviews could be used in a better way.
Hey! Love your content! I'm at work right now, so can't watch the video, but the title really bugs me for a really important reason. Anyone who's searching for this video when they really need it will never be able to find it. If someone pushes an API key to origin/main and they need to find out how to fix it, they'll never be able to find this particular video. I don't mind clickbait most of the time, but I strongly urge you to make the title more searchable since this topic, in particular, can be a make-or-break seconds-on-the-clock job-on-the-line emergency for someone or several someones.
Hey! Thanks for the feedback and the great suggestion! I will definitely change this title once the algorithm has done it's thing as I 100% agree with you. I usually do this for more searchable content once the recommendation process has finished as this also helps to get higher views, which will help with searchability as well. Thank you again for the great suggestion
@@dreamsofcode I would generally recommend to roll over secrets when in doubt, it should be done regularly anyway and not at all an operational hassle. If people don’t ask, I won’t recommend it, so if they feel confident then let them be the decider. On the other hand I also feel very bad about .env files or environment variables on my own machine
I see even well-versed devs make mistakes. Sometimes, as with crypto, this causes the loss of thousands or millions of dollars. Pre-commit hooks in sensitive situations (many such cases) are needed, as a last resort.
Correct, I believe I mention this. The commit will be garbage collected eventually, but the idea is that you don't push these commits up which becomes a much larger problem.
It's a completely personal opinion (I can see the benefits) but I don't like the idea of becoming dependent on them and having too much trust in the process. That's totally a me issue though
Someone mentioned this but I've never seen this happen, (mainly because I don't commit them). But the cache password did still exist in the private repo.
@@zhytedark2859 I should do a video on my dotfiles, but I actually use password store, a Gpg public key, and a yubikey to help with secrets in mine. That way all I need to do is plug in the yubikey and it'll work
I love your videos, but this video seriously missed the mark in terms of accuracy, if someone actually does this in a real environment they will not be safe. Step 1 should be invalidating the current secret, Step 2 should be then figuring out how to replace it without leaking the new key. The millisecond you push a key to a public repo, it gets logged by many git logger services, so no matter what you do, it’s comprised. That’s why you must first delete, figure out later.
sorry i ruthlessly disliked due to the terrible joke in the beginning "dont commit crime", but liked bcz it got better in the later part of the video :)
Using such a tool in this way is useless for a dev and cleaning up the history is utterly unnecessary. Just invalidate all relevant secrets if it happened and move on. Where tools like this really shine for us devs is running it as a pre-commit hook, because then it isn't too late yet.
I think it's worth cleaning up your history before pushing. I also mention the pre-commit hook at the end, (although I do think having better hygiene is more important).
@@dreamsofcode Oh yea, for sure, if you haven't pushed anything yet, nothing did happen "yet", as you do here. But having to force push the trunk will make a bunch of your team members very unhappy. But since this is one of these outliers where developers can do permanent "damage" in a software system, putting the guardrails upfront in the video (the pre-commit part) and then showing what the tool does and how to work with it would've been better. I didn't even make it to the pre-commit part before leaving my comment - and for that I am sorry.
Thanks for the video. Small suggestion: reduce the amount of bass in your voice recordings, it's a bit distracting and unpleasant (maybe it's just me though) ;)
We recently had an email service token leak, which resulted in 1.2 mil scam emails being sent from the company noreply mail, so this is very important advice!
Just always use commit in combination with push -f to annoy your comrades
bonus points if you automate it to happen in save.
git add -A && git commit --amend --author "Senior Developer" --no-edit --no-verify && git push -f
@@FlorianWendelborn now put it in an funny alias and it's perfect!
wait you guys dont do that?
Tbh if i find a leak with this tool im just gonna rotate the secret rather than mess around with git
Even if it's only on your local machine?
@@dreamsofcode probably less time than messing with git. if you have to move to a secret file anyways, when you move the key, just rotate it then
@@IsaacShoebottom I think if that works for you then go for it! In my case I much prefer to do a quick git reset which also prevents any false positives in the future.
@@dreamsofcode I think I would do the following:
1. Identify existing leaks and change secrets
2. Integrate the tool as part of my workflow thereby preventing any new leaks
IF a new leak somehow occurs and is a false positive you can update the baseline of the tool so it wont look any further back
@@IsaacShoebottom you probably need to learn how to use git, because this is faster, cleaner and safer
8:12 force pushing to a remote is fine if you own the branch. Don't force push shared branches like main. Do force push branches that only you have committed to, it's a very good way to clean up and reorganize a branch pre-merge.
If you feel uncomfortable knowing when it's ok to force push and when it's not, study and practice git, it's worth it.
Agreed! Your own branch is fine (I believe I mentioned that), but never main (unless it's your own and you know what you're doing). I was definitely erring on the side of caution with this message as I didn't want to encourage people to force push without knowing when it is useful.
@dreamsofcode I think you did too - just hoping to add a little extra clarification for people new to git who are a bit too scared of using it effectively (in my experience)
@@orterves gotcha! And you're correct as well. I actually have another video planned looking more at force push!
Well Done Tris. I am still not used to seeing your mug in these videos. Clearly articulated, demonstrated, and now to get lazy devs to follow suit....
Step 1 after encountering a leak should be to invalidate the credentials.
If they're on a remote repo, as I mentioned in the video I would instead rather do an investigation otherwise you might be encouraging unnecessary downtime.
I'd love to see videos in the future which go deeper into especially those things said in the last minute (Pre commits generally, other secret solutions and generally secret managers) ❤
Absolutely! I'll make sure to add them to my backlog
I'd go with `git commit --amend` if you really just need to fixup the last commit. No need to temporarily undo it completely with `git reset --soft HEAD~1`.
Also, one should use the "dotenvy" crate for Rust; "dotenv" is no longer maintained.
Ahh shoot! I did not realize it was no longer maintained! Thanks for letting me know!
amend is also a good option as well!
Thanks you magic anime girl of wisdom
Thanks for featuring my tool! Great video!
Your spinning warning symbol graphic at 8:07 seems to rotate around an axis aligned with its face rather than its midpoint, and has a weird smear on the frame its parallel to camera.
Please make a video going deeper into this , gitleaks, thanks in advance
this is more of a git tutorial on interactive rebasing for me, extremely educational stuff
This was really informative Thank you. I just recently gave an interview and I was presented with this EXACT question.
Superb content as always. I am waiting for a full microservices course from you. 😁
Thanks for thoroughly explaining how to resolve these issues as I bet this guidance will help give people the solutions they need to get these issues approved to fix.
I liked your video. Thanks a lot for doing them. What about the git filter-repo and other tools that might edit only the needed parts of the commits history?
git-filter repo is pretty dope from what I've seen! I've never used it personally however as I tend to stick with just the basic git commands. Perhaps that's a good excuse to do a video and check it out!
3:00 This is MUUUCH easier with lazygit, you just go to the leaking commit, press enter, find the file, press enter again, select the line of the secret, press ctrl+p to open custom patch options menu and then press d or select Remove patch from original commit and you're done
A note on `gitleaks`! Please think about how the tool does detection and react accordingly. Do not blindly run the tool on code you do not personally have distribute rights to. Definitely perform a full audit of the tool when running against mission critical code.
Good advice on all code, if you're extra paranoid, run it with your network disabled!
One of my biggest fear is to commit my home address by accident, and not even noticing 😅
I accidentally doxxed myself on stream one time so I know this fear!
Thank you for the excellent video. Also, using git filter-repo is a good way to remove a file from history.
Although I share the belief that good hygiene is better than tooling the git commit around with pre-commit, in a not-so-big team that is already difficult to enforce. And code-reviews could be used in a better way.
would have been nice if I had this information about 2 months ago :) thanks for pushing this out though!
How do you feel about the “git commit --fixup”/“git rebase --autosquash --interactive” combo?
the terminal status bar is amazing also the neovim colorscheme. May I ask what are you using to get this done.
A video about Gitleaks would be very useful
I second this motion
I think you must make one video about setting up neovim for go developers. I am diehard Goland fan but I would like to try neovim.
Damn you, Dingus!
At 2:55 "Dingus Dev ", hilarious 🤣
That was an interesting video!
I'd love to watch more useful git-topics if you have more.
Absolutely!
video on GitLeaks, pls!
Hey! Love your content! I'm at work right now, so can't watch the video, but the title really bugs me for a really important reason. Anyone who's searching for this video when they really need it will never be able to find it. If someone pushes an API key to origin/main and they need to find out how to fix it, they'll never be able to find this particular video. I don't mind clickbait most of the time, but I strongly urge you to make the title more searchable since this topic, in particular, can be a make-or-break seconds-on-the-clock job-on-the-line emergency for someone or several someones.
Hey! Thanks for the feedback and the great suggestion! I will definitely change this title once the algorithm has done it's thing as I 100% agree with you. I usually do this for more searchable content once the recommendation process has finished as this also helps to get higher views, which will help with searchability as well. Thank you again for the great suggestion
@@dreamsofcode That's great!
rust dotenv crate is no longer maintained, prefer dotenvy instead.
Awesome tool!
the must watch video
Hey man,
Mind sharing what you used to make the animations?
I make them myself using After Effects!
@@dreamsofcodeVery cool. Thanks
more on gitleaks pls
does `git gc --aggressive´ do anything in this case (to gc the previous, compromised commits)?
The leaked secrets should always be considered compromised
That's the most stupid thing I have heard this week
Even if they haven't left your own machine?
@@dreamsofcode I would generally recommend to roll over secrets when in doubt, it should be done regularly anyway and not at all an operational hassle. If people don’t ask, I won’t recommend it, so if they feel confident then let them be the decider. On the other hand I also feel very bad about .env files or environment variables on my own machine
@@dreamsofcode if they haven't left your machine they are not leaked 🙃
@@ImperiumLibertas Yep.
I see even well-versed devs make mistakes. Sometimes, as with crypto, this causes the loss of thousands or millions of dollars. Pre-commit hooks in sensitive situations (many such cases) are needed, as a last resort.
Gitleaks opportunity missed. GitLeeks... it reeks of leeks. Writes itself
4:10 Hey, actuelly you didn't remove from the git history, you just created à commit with the same parent as the HEAD but we can still find it
Correct, I believe I mention this. The commit will be garbage collected eventually, but the idea is that you don't push these commits up which becomes a much larger problem.
I still haven't seen the end of the video so idk
Cool mug.
Are... are you using... Arch with Hyprland?
👉👈
My Solution used to be: Delete the Repository :D
Please make a video that goes more in depth about leaks.
Why are you not a fan of pre-commit hooks? I find them rather useful as a guard against user mistakes (secrets, but also linting etc)
It's a completely personal opinion (I can see the benefits) but I don't like the idea of becoming dependent on them and having too much trust in the process. That's totally a me issue though
@@dreamsofcodetrust but verify. The verify part being a pre commit hook
Loved the crime joke!
Thank you! I went a lot harder in the earlier edits but decide to tone it down a little haha
Just change passwords or revoke tokens asap.
doesn't github automatically detect this for many common types of secrets (and revokes them)?
Someone mentioned this but I've never seen this happen, (mainly because I don't commit them). But the cache password did still exist in the private repo.
In my experience it does not work well, I even opened a bug for a sample file with a false positive and a false negative within 3 lines of script ,)
If i accidentally push password or secret key i'll change password/secret key and its enough.
I thought you had moved to nixOS?
Is it also a big problem if it's in private repo ?
I think it depends on how many people have access, but really I'd try to avoid hardcoding as much as possible.
@@dreamsofcode Yeah I get it. My problem is to keep my exported key in a my dots file for full reset
@@zhytedark2859 I should do a video on my dotfiles, but I actually use password store, a Gpg public key, and a yubikey to help with secrets in mine. That way all I need to do is plug in the yubikey and it'll work
@@dreamsofcode Would love to see a video on this!
@@thimvana8030I thought he already had a video on that topic? I am using the same password manager due to that video, I believe
FAST WATCHER SQUAD
I love your videos, but this video seriously missed the mark in terms of accuracy, if someone actually does this in a real environment they will not be safe. Step 1 should be invalidating the current secret, Step 2 should be then figuring out how to replace it without leaking the new key.
The millisecond you push a key to a public repo, it gets logged by many git logger services, so no matter what you do, it’s comprised. That’s why you must first delete, figure out later.
Did you watch the full video? The first half is for locally committed secrets.
sorry i ruthlessly disliked due to the terrible joke in the beginning "dont commit crime", but liked bcz it got better in the later part of the video :)
Using such a tool in this way is useless for a dev and cleaning up the history is utterly unnecessary. Just invalidate all relevant secrets if it happened and move on.
Where tools like this really shine for us devs is running it as a pre-commit hook, because then it isn't too late yet.
I think it's worth cleaning up your history before pushing.
I also mention the pre-commit hook at the end, (although I do think having better hygiene is more important).
@@dreamsofcode Oh yea, for sure, if you haven't pushed anything yet, nothing did happen "yet", as you do here. But having to force push the trunk will make a bunch of your team members very unhappy. But since this is one of these outliers where developers can do permanent "damage" in a software system, putting the guardrails upfront in the video (the pre-commit part) and then showing what the tool does and how to work with it would've been better. I didn't even make it to the pre-commit part before leaving my comment - and for that I am sorry.
Can't relate. I have commitment issues.
Great video as usual!
Thanks for the video. Small suggestion: reduce the amount of bass in your voice recordings, it's a bit distracting and unpleasant (maybe it's just me though) ;)
Sorry to hear that!