CI/CD In 5 Minutes | Is It Worth The Hassle: Crash Course System Design #2
Вставка
- Опубліковано 7 тра 2024
- To get better at system design, subscribe to our weekly newsletter: bit.ly/3tfAlYD
Checkout our bestselling System Design Interview books:
Volume 1: amzn.to/3Ou7gkd
Volume 2: amzn.to/3HqGozy
ABOUT US:
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.
The thing with CI/CD is that it can lead to a hugh bureaucracy overhead. Each of the steps on the pipeline is more often than not, out of reach for a developer and locked away behind a plethora of access request tickets. If you, as a developer, have a issue in one of those steps, you're in for 1-2 weeks of waiting for devops/infra/whoever to sort things out. You are still responsible, but you will not have the tools to fix it. As opposed to other approaches where devops, who has the tools, is also responsible for making things work. Companies seem to think that CD/CI is fire-and-forget miracle solution when in reality, it requires a lot of supervision and revision all the time to insure that it not riddled with roadblocks.
CI, or more specifically the pipeline on your Git platform, is useful though. As soon someone commits to their Merge Request, tests/builds/others checks are ran. This somewhat forces devs to use code style, linting and tools, and reviewer doesn't have to get their own copy and run these just to be sure.
As for "no branches, merge everything into develop branch" or something... yeah, I am not buying it either.
What you explain is not an issue with CI/CD but a concrete (and incorrect) implementation that you probably encountered. A correct implementation would have all necessary build, test and deployment scripts in the same repository as the application and those scripts are constantly exercised (on each commit). If one commit breaks the pipeline (be it a change in code, a test or in the pipeline itself), whoever broked it should either fixit or reverse the commit within 10 minutes. On top of that, companies doing CI/CD would normally be following real agile practices and be structured with small autonomous teams, so in no way they'd have bureaucracy overhead.
@@Buffalo5 Yes, that is true. BUT, and here's my beef with CD/CI, under previous "less refined" models, when something was problematic, the people that got screwed were in a position to complain (and be heard) or simply do something about it. That is devops when it was called devops. Nowadays it seems that eventually CD/CI ends up being a mess where if something problematic is going on that delays things, the people that gets screwed (developers) are NOT in a position to do something about it because the pipeline (and products and services acquired to be used in said pipeline) are not their decision or domain either.
There's nothing wrong with CD/CI in principle, it's just that it spirals out of control and becomes bottleneck after bottleneck and "it looks alright" from the higherups perspective because "we are following a process".
Also, in the past, when things were more manual and less automated, if you had 2 devops for 100 developers (exaggerating of course) you knew you'd have a bottle neck. Now that everything is automated, it appears that you CAN have 2 devops for 100 developers because "everything is automated with our CD/CI pipeline"... until something doesn't work and then you have two poor bastards trying to get things done as fast as possible.
@@sanguchito7381 yeah, of course. At the end, no matter what processes you follow, there are humans involved and that means that what will matter the most is the company culture. It seems that the struggles that you mention come from some misalignments that only people can resolve, not processes.
At my company, we've been (for better or worse) encouraged to adopt nearly the full CI/CD process as part of our scope. Part of building our tool is creating our pipelines, or if we're on the cloud, using terraform to define our architecture.
4:54 "This effect requires GPU acceleration" 👀
Subliminal advertising for NVidia
Plot twist: They deployed this video using CD and didn't catch the errors
@@DavidsKanal Regression: Add test for blinking in red frames with "This effect requires GPU acceleration"
@@SirRFI Regression: Title in video intro says "CI/C in 5D Minutes"
@@DavidsKanal 5-dimensional hypertime discovered
Thank you! This is a shockingly well put together and succinct explanation with awesome graphics to reinforce the talk track without bloat. Fantastic work!
brilliant. clear, concise and a great overview
As usual, a great overview! Keep up the good work!
Thank you for this short but very insightful video. The graphics helped explain CI/CD in a way other talking head with static slide videos do not. You got a new sub!
Really appreciate these abbreviated overviews!! 🎉
Thanks so much for your explainer videos! They are very helpful for me as a non technical person who have to work with the devs and engineers
Thank you! Thank you! Thank you! Please, don't stop making these videos!
Great work with the animations, I love them!
Beautifully explained overview, I would love to see more content like this for DevSecOps. I’m taking the SANS GIAC course and this ties in perfectly and covers high level gaps.
Thanks for the video! Just short info for people who are not know much about CI/CD. CI/CD is not about tools at all, it's about approach/practice. Tools just help to reach the goal.
CI is about continuous integration our changes into a single unit. It's the practice that allow us to integrate many changes made by many people into the single unit. For example in dev area, the unit can be some branch in the git repository.
CD is about keeping the unit deployable to the prod environment. It means that our unit should have ready to prod state. It means that where we integrate should have a fully working state.
Great stuff!
Great video for learning CI/CD in 5 minutes. Would you recommend a course for learning CI/CD in, say, 5 hours(in depth system design)?
Great video, really well done. Thank you 🙏🏼
4:53 "This effect requires GPU acceleration." 😄
well made video with beautiful animations which made it even more easier to watch
super useful channel! thanks for sharing your knowledge
This is one of the best channels i knlw. Thanks for sharing.
I have an apinion on this CI CD topic.
Why would a functional or automation tester ever use CI CD, or any repository other than Jira, or Testcase management tools?
This is required for code integration and automated deployment... for developers. A software tester has notving to do with it.. 😅
Testers dont use Guthub/Lab, or any pipeline. Testers use Jira X-Ray, HP Alm, Selenium, Postman, etc.
Your video is small & crisp with exact high level information.
Thank you so much, you provide such a great quality videos, thank you infinitely
You explain things so well🎉
Awesome material. Good to know some of the top tools for CI/CD. How to get this picture with the tools?
Thanks for the great explanation
Awesome 🙌. By the way which tool are you using to create animated diagrams 😬
Very simple and good explanations
Wow! I am a first viewer of this video😅. I must say, the animation is amazing and content is easy to understand! Thanks 😊
Great video! A small comment. The original definition of CD should refer to Continuous Delivery, not Deployment.
loved your content
Thanks for free knowledge
The foundational principle of SRE is that developers should be unable to execute their Yaml oriented program without submitting it to another team's rental hardware. The industry standard Yaml first design pattern ensures software engineers will never notice debugging is a forbidden worst practice.
As a beginner youtuber in the programming space, I have to say, I LOVE YOUR EDITING! It seems to take so much effort to animate all these parts. What are you using for that?
Thank you for the great information and me personally I would love a deep dive into your youtube creation process
They use aftereffects.
Great presentation
could I please a link or reference to source of ci/cd double loop graphic I would love to read source of this awesome video
Hi, Please do a video on how webRTC & group video call applications work on large scale.. Thank you.
great animation for easy explanation
Great video!Question: how do you design and create these awesome animations? Could you provide some example source element file ? Thanks, pls
Thank you for explaining in a nutshell
Thank you!
perfect
What do you use to make your videos / animations?
Amazing video!
Question: how do you design and create these awesome animations?
I believe he has a team behind it 😋
How do you create these presentations? Which tool do you use ??
So, complex system could need same parts of manual deployment?
I didn't notice any quality improvements in sw (it's slower, more bloated, less intuitive, ad packed), but damn, those frequent deployments are definitely noticeable. This is all software does these days - constantly updates itself.
Nice take 😂
Great Video . Please talk about redis and its internals in depth if possible
Did you hear something? Must've been the wind
Great video
Thanks
which tools you are using for animation editing ?
Thanks, this kind of helped. It seems like CI CD is something worth looking at when you are a mature dev, right? I am just starting with development, have static website online, and was wondering if I could start now with CI CD. The idea: Whenever I decide to upload the newest changes to my website (which I currently am doing with scp via terminal), I'd like that to happen from...VS Code? Github? At least a lot more automatic than opening terminal and typing that long command.
Would love a video covering WebRTC!
Guru, what program do you use for the infographics?
This video effect requires GPU acceleration? @4:52
What are the frames that say "This effect requires GPU acceleration"?
@@konrness Actually Premiere Pro is for editing. For animation you should use After Effects.
@@ARmy2510 Adobe Animate is used for 2D animations like these.
@@mallipooalwa9085 Not everything. Intro itself is a pure AE. Well, all I see here is After Effects :D
Oooh, I see a meme reference here :D nice job.
All dev should now about CI/CD
My thought is: Automate as much as possible before production. That includes, building, testing and deploying to test/acceptance etc. That's where the fast stuff happens, the quicker you can get something tested and verified the quicker you can move on with your life.
Production deployments i take case by case, I always assume it's needs to be manual but with automation until you can "press the button" and go from there.
Ok, Does it mean not lot of companies are doing automated deployment when it comes to production deployments?
@@rajr5936 all depends on the size. If it's a large scale distributed system, it's probably fully automated with clusters and everything. But i've worked on a few highly sensitive systems and it was usually 80% manual and 20% automated. We never got to fully automate it...
@@fronix5060 ok if it's not automated who performs these tasks manually? Still devops eng?
Great content! Am I the only one who read the intro as "CI/C in 5D minutes" 🤔
4:53 the text that scrolls on screen is "This effect requires GPU acceleration"
very nice video
I'm new to this, but why does build goes before testing, shouldn't it the other way around?
What's that about the effect requiring GPU acceleration at 4:51?
some Adobe Premiere issue that was not caught with the automatic deployment of this video
Ok so if this is not necessary who and how its deployed?
4:56 there is an issue with the video?
Is it possible to implement a CI/CD without all these tools in AWS?
I dont like CI online, send things to server, to test, is always slow, I run everything on my machine and after success send to git, the code must be ready to be deployed when uploaded. So deploy to dev/test environment and after, if everything is ok, to prod. Im searching a good set of tools to do the monitoring process.
I hardly understand these videos since they're high level stuff while I'm still a noob. What part does system design play in web development? I hope you would cover some parts related to web dev.
A lot of web app shops use CI/CD, or at least CI as he says.
@@peterstedman6140 Thank you for the explanation.
CD means Continuous Delivery #beeyem
CircleCI is missing from video
In my opinion, no it's not worth on small teams and during the inital development. You can start adding all this stuff for version 3 (remember version 1 is to get out software fast, 2 is to fix the bugs, 3 is to add new surprising ideas). Before that it's a waste of time and money
Okay. Will these jobs still required manually?
0:07 CI/C in 5D minutes :P
Tracking CICD compliance is another headache. Tracking crazy metrics and trying to achieve unreasonable targets don't make sense.
5:40 ish - "this effect requires GPU acceleration"
I think you are missing Harness in a lot of these spaces
Exactly. It's a revolution tool.
"This effect requires GPU acceleration"
I freaked out on ""This effect requires GPU acceleration""
This effect requires GPU acceleration
Nice catch man!
HELP TEAMS SHIP FASTER AND BETTER QUALITY SOFTWARE DEPENDS ON COMPLEXITY OF THE SOFTWARE THAT IS BUILD AND HARDER AND SLOWER TO TROUBLESHOOT / FIX ALL THESE PLATFORM in the same time
Once prog. languages were straight and efficient and everything worked. Today programming environment is so cluttered with so many mostly useless add-ons that it is a miracle if everything works while efficiency has gone thru the window.
Popularity of the IT industry and money have lured in so many players that do more damage than good. Bad time to be programmer who has to cope with so many inflated egos
In 5D minutes xDDD
CI/C ;)))
Thank you!
This effect requires GPU acceleration