I didn’t expect this video to get that many views. I just wanted to make a video to give people who work solo or or on a smaller project more insight into a larger scale project. Your mileage may very. Every company does things their own way from what I’ve seen, and most companies have various teams that all work on their own sub systems and integrate with others.
Nah, man it's useful to have stuf like this ( BASICS ) in understandable English. That last part is important. Also, interesting to see what the "Zeitgeist" is in Enterprise Software Development, because what you outline here is what we are doing too. Getting major DEJA VU
As an enterprise developer myself, this is by far one of the best breakdowns of how enterprise software development works and I wish there was a video like this when I got my first developer job.
@@lit22006 It sounds like you’ve never worked in enterprise if you can’t recognize an actual team structure in an organization, but please do enlighten me about what’s actually going on in such an environment if you’re so experienced in the matter.
This is actually a terrible / antiquated way to handle development. Breaking out branches into dev/test/prod just creates this nightmare scenario where you're constantly managing the state of the branches and is very prone to error. A much better way is TBD with environment configurations and artifact promotion. This way you can "version" your software and control releases based off the build artifacts, instead of trying to maintain the state at the source code level through branches. Git is already versioned. You don't need to implement a branching strategy to add a needless abstraction around versioning.
Bro... it's as much real as it can be. It's THE BEST description of how different teams in 80% of the industry do their job. And you also covered rest of the 20% by including redundant systems, and multi layered prod systems (Production Microservices like architecture). Great stuff.👏
Yes, you're correct. Startups may have lower complexity, but the underlying concepts and patterns remain consistent. This video is truly exceptional, highlighting this fact.
Yeah, the concepts definitely apply to smaller startups as well, just at a smaller scale since you don't need to be as rigorous since it doesn't hurt as much when stuff breaks (compared to like some big company with 100M+ DAU where stuff breaking costs a ton).
Since the start of coding 2010 I was always on my own with solo projects as fullstack dev. I had always big questionmarks regarding those enterprise teams working together. Those 24 min of your video were so incredible eye opening to me and it made me a good feeling to finally understand how this works. Awesome work here Cody
This is epic. I've worked in enterprise environments for the last 10 years and this is by far the best intro explanation I have ever seen. Well done :)
Beginning of year I singned a simple VPS, the best choice I made so far, started learning linux, ssh, git hub actions, automatic deploy, git (for real), managing db, staging (my personal PC), production environment, nginx, and those things made me so more aware of how development works and how can I really deliver software as a product.
which VPS did you buy and how much does it cost per month? For now I am using aws ec2(my 1 year free tier is done) and when I am done, I just stop the instance(I still have to pay for storage, but I dont have to pay for instance since its not running). I started last week only, so not sure how much its going to cost after month. I am thinking that it should be less than $1. If its more than $5 then I might just buy digital ocean droplet or something.
could you elaborate on that VPS? I do want to (more like "have to") learn those components. It might sound silly, but I came from old monolithic way and my responsibility are only for Dev side, which is outdated and now we Dev are required to also hands on the operation side.
Thank you very much for this brillian discuss on how enterprise systems are developed and how enterprise teams work on large scale projects. This video is way more than a random video as you share a lot of insights into how large team are structured and how their projects are planned, developed and deployed. Quite a rare video on youtube. Thanks once again.
I am on a 5 man scrum team implementing robotics in software and this video holds pretty true. Depending on the scale of the company you find yourself in, that whole user interaction part can sometimes get absorbed by the roll of a pm and perhaps other individuals or teams, but this was a great summary of how things work. Great video.
I'm a backend developer for an enterprise company, and this is exactly the way we do work as well. This is pretty much exactly the same way we work, but we have around 400-500 developers I'd say, and probably like 80 different development teams, and most teams are integrated with other team's services and products in some way to produce one entire seamless product that end users use (simplified).
As a web dev newbie who just came out of a coding bootcamp, this kind of content is super valuable since the teachers never really talk about this too much. I am so glad that I found your channel! This type of real world content is also rather rare on UA-cam, and I am looking forward to seeing more of this from you 😀
As a SM this is a great for devs to understand why there's so much syncs and managment stuff or meetings. Enterprise will scalate like crazy were you can have like 100 teams trying to push features. Great video!
With 10+ years of enterprise web dev experience under my belt, i approve this. One thing you missed though are the many endless product team meetings. The flows you talk about here are happening in an ideal scenario. In reality there's a lot of big talks and "we're all a big family", feature rushes, egocentric people, pizza parties, "synergistic" in person meetings and not so much time actually writing good software. Btw, perhaps we should also open the topic of "cloud" costs. I've worked with one that started with gcloud and datastore and it turned into a hot mess that needed almost 1 year of refactoring and paying 10k+/month for infra alone
I've had only one professional experience with working with a bigger team and I have to say this video is spot on. Literally everything we did, we did exactly as explained in the video. The only thing I would say differed, is that we had no contact with users other than error logs on AWS, our client was pretty much 100% responsible for gathering user experience and send that our way, digested, so we could turn those into issues. We also didn't have automated smoke tests, we just had QA do those manually before releasing
This was amazing man. I always felt like some dots were missing in understanding the full picture. Thanks for helping me connect more dots. Happy Coding!!
solid video; to make it more adequate and complex - Business Analyst is a must (partly overtaken by UI/UX designer here) and QA allocation in entire process; regardless the role inside organization - there is always BA being a system functional expert being able to present and negotiate options - customers in general have ideas and do not go into details since they are not technical, do not know system limitations; do not undestand full workflow scenarios; are not capable to have a holistic view, identify gaps and risks. Same goes with Devs - they don't have to understand entire functional scope of the system, to memorize or the details, or at least it is often not achievable due to e.g. rotation inside company. BA role is key to be a proxy; while UI/UX designer is a part of that process, by role definition, it is not only about interface design; but I believe the simplification here is that in most cases - BA and UX experts coexists where UX designer is more technical role (UI) focusing on workflow and visual aspect
We've been burnt multiple times using dev -> feature branch flow and I don't recommend my teams to do it. Now we always fork features from main, and work it up through the stages so that if we need to make a cut without some buggy feature we can and feel comfortable not leaking changes into prod. Our team also use a pre-prod environment to verify changes we release before they are done to make sure we have an exact replica of what prod will be/ or what it is, because staging may contain bugs that need to resolved. As you said multiple times it gets complex if your team is bigger or have multiple team working on solution spread across the organization. BTW. very good video, you got most of it clearly explained.
It is pretty much abandoned as a process and the original git-flow author has disowned it as a defective way to collaborate! People keep finding git-flow though so he put a big disclaimer at the top of the page. They still keep recommending it!
@@ltbarcly I think once you search for git flow on Google one of the first thing that pop up is Atlassian's git flow recommendation/description of the process and there they branching from dev in there flow and i guess people just accept it because it Atlassian recommending it.
@@ltbarcly Could you please explain what we should avoid ? I haven't experienced any large scaled project but i'm planning to work on some large scaled projects so I would like to know what I will encounter. Personally, I also use dev and feature branches because I heard it's one of most common way.
Do you still have 3 enviornments? so you fork from the master branch, and merge into dev, if the feature is approved you merge that branch to staging which should be basically a copy of master and test it there, and then back to master?
I'm relieved that I (accidentally) made us follow some existing standard at my work! We ended up doing roughly the same things and it's been great for our productivity. I would like to emphasize that testing is what makes this work so well.
I was the Director of engineering at Byte for many years. Now the head of engineering at Sirge. Because we end up building products for enterprise businesses, the org it setup a bit differently. We have different business units that represent keys parts of the business. Accountings, Operations, Engineering, Marketing, Growth, Design, Sales, Customer Support/Success, just to name of few. When I run team I like to ensure that product is always representing the "What" we are going to build, and engineering is representing the "How". Product will normally work directly with design, along with customers, gaining feedback from surveying and customer interviews. Through the ideation cycles product will define that features/products we need to build next based on customer collected data. From their we have a series of meetings within an Agile/Scrum structure that allows us to carve our the user stories, functional requirements, point, then allocate to sprints. Thats just a high level. But my team sizes grow pretty large.
Hey Cody, Love the content. I've worked professionally as an engineer for 13+ years amongst varies enterprises and this top tier content for helping dev's understand enterprise software life cycles! I will say, I'm surprised that a Business Analysts isn't sandwiched somewhere between your PM, Designer, and Client.
So many companies have replaced business analyst with this perverted concept of Product Manager entirely (to their peril imho) to the extent that those following "agile knockoff" as I call it don't know what BA's do. Where I've seen this, arbitrary rectangles in Figma replace proper documentation, diagrams, capturing of scenarios + use-cases, etc. You probably have worked deep enough Cody just at places where the bootcamp rectangle kids took over.
when us devs can't keep up with requirements from clients, BAs be outhere meeting with them 8 hours a day and drawing all kinds of diagrams for us. coding comes easy after that. in smaller teams, BAs sometimes work as frontend dev/designer as well.
In my experience there has generally been a layer of separation between Client/Support and Developers. It is important that issues are properly ticketed and discussed in daily standups and assigned to either the backlog or to the appropriate developer. Also, at least where I work, there is an "Architect" team that are often the most senior developers of the various teams working on different projects, and the architect team will have meetings where they look at what code is duplicated and evaluate where it is worth developing an internal shared library that multiple projects would benefit from. We have one team that develops those libraries and they're often for cases where performance is critical or reliability is critical, so for example we have a shared library for direct communication to Mellanox network cards, and we have a shared library for 3-mode redundancy.
Great as always! Would love to hear your thoughts about git flow. How do you maintain commits while merging, apply hotfixes for several branches, rebase or not, etc... This would be a great addition for this video!
Great questions! In my experience, I think having multiple active branches causes more problems than it solves. I'm one for using trunk-based development but it requires having tests that run when PR from a feature branch is open. Master/main is always a deployable state. We always did squash merge.
Thanks for taking the time and making this video. I didn't know I knew how to navigate all of this mess, and having it laid down so beautifully is amazing.
Have worked as a senior architect for many years now and have to say this is one of the better videos I have seen on Enterprise Development. Having to deal with 5 or more of those projects simultaneously while understanding all of the legal, technical and other dependencies is why the complexity exists even though it can be extremely frustrating for developers and others on the front lines. Hopefully it helps people understand why, sometimes, they are told no when they just want to add a "simple" maven plugin or NuGet package to their project that will make things so much easier... there is often a lot more going on that might not be immediately visible. Great video 😎 (edit: Architects can't spell)
Dude! This was amazing. As developers we never actually get taught this (not even in college), but we kind of pick it up in the different organizations we get to work for under our career. Seeing it laid out like this is beautiful, thanks.
Literally what we do summed up in a neat video. I will be sharing this aggressively to everyone who wants to become a cs major to set some expectations of their daily, as not many know this before getting their 1st actual job.
As someone who hasn’t worked as a developer at a company yet, this info is invaluable. Thanks for taking the time to talk through the process and share your knowledge 🙏🏼
Your explanation has given me a much better grasp of the complex ecosystem involved in larger scale software development, both from a technical and organizational perspective. The visuals and real-world examples made the concepts more concrete and easier to understand. Thank you for taking the time to share your knowledge and expertise in such a clear and compelling manner.
different branches for different environments (aka git flow) works well if you are building software with specific versions and you are cutting different releases with a specific list of features included in each release. but for development of public-facing web products trunk based development (aka github flow) is more common - a dev can pick up a ticket, make the change, have it tested by QAs/automation & have it in prod by the end of the day. then you don't need to worry so much about what's getting shipped when, you are just constantly building stuff and it's in prod as soon as it's ready.
Very good point. We need to explicitly say that for new devs, I would even discourage introducing new devs to gitflow. It's bad, it only works for situations as you mentioned. Trunk based is much simpler and in most cases preferable. I've seen so many devs using branch names with prefixes such as "feature/..." and that's just because they've picked it up somewhere else without even knowing what they're doing - now they think it's a git standard. That's completely unnecessary and a bad habit imo. Simplicity is the key.
This could seem to imply that switching to trunk based automatically means your tickets are small (small enough to do in a day) but I don't see what one has to do with the other. I'm sure you'll agree that ticket size is basically unrelated to the branch strategy? Someone on my team is trying to get us to switch to trunk development and I'm having trouble seeing the benefits. FWIW in my team we do feature branches but we do not have a dev branch like is sketched in the video between feature branches and master.
@@xellestar trunk based does work best with smaller tickets. they don’t need to be doable within a single day but it shouldn’t take more than a sprint. the main idea is that you build features with small iterations which each go to prod individually over time rather than dropping a whole finished fully working feature in a specific release. it just lets you work in a more agile way where you can pivot more easily as new priorities come in. if you’re not able to break down your tickets into smaller chunks of work that can be deployed independently without breaking stuff, then trunk based might not be right for the software you’re working on. of course you also need to have the appropriate infrastructure eg. CI/CD pipelines, automated testing in pre-prod environments etc.
@@a__guy I'm all for building smaller things in iterations. But let me repeat a comment/question I left on a reply to someone else: There seems to be either nuance or confusion around what is and is not trunk development, so please tell me where I am going wrong. When people say "merge directly to master", merge what exactly? Your feature branch, right? People seem to employ the terminology as if they do not branch at all. If a team uses feature branches, but they are branching from and merging to master directly (forget feature flags) is that trunk development or is it using feature branches? Is it trunk development only if they are using feature flags?
Ticket size and how long it takes to finish a feature has nothing to do with branching model. You can develop on your branch and merge to master once it's ready. That's still trunk development. It's your responsibility to keep your branch up to date with master (which you should be doing as often as possible). And if this doesn't work, then you have different kinds of problems that aren't related to branching model. Probably bad code quality and poor planning.
I started my journey in software development in a very small startup - no automated tests, code hosted on VPS with manual pull and build, no docker or anything. so I started to discover that there is more to software dev than writing bunch of code and push it to the main directly. now am discovering testing, cloud, docker and deployment and trying to understand how to implement all of this in my personal / freelance projects. Your video helps a lot and I want to thank you
This video is fantastic at explaining high friction software development but it's surprising the amount of companies still to this day that have no automation with CI/CD or even proper use of version control (as the company I work for daily drives Team Foundation Server Version Control). Obviously everyone's use case it different as some companies still have on premise servers but overall companies from big to small should take note of the usability and simplicity of making a good software development workflows that's right for them (and that's leverages more modern tooling). Good stuff!
For large and important systems, in practice you have to use incremental deployment of new features into production for a limited set of users. Of course, this also brings complications - different versions of the application must be compatible with each other, and the production environment must allow gradual deployment.
I'm a co founder at a startup, during the video i was starting to lose faith on our way of doing things, but the last couple of minutes your restored all my confidence and giced me courage to continue the amazing journey I'm in.
how are y'all doing things at your start up? honest, keep it simple is what I've learned and add process when and only when you keep having real issues
Great video. I see quite a few similarities to what my team does. We don’t deploy branches other than master to environments. Feature work and bug fixes are done in branches then merged into master upon PR review and CI checks passing. My team composition is just about the same but we have QA attending all our standups. I wasn’t sure if QA is part of your standups because they were not listed alongside devs, scrum masters, and PMs. Interesting to see how other teams work. Would love to see more videos like this about SDLC.
yeah it sounds like you do trunk based development. So do you merge code daily to master, or do you have long living feature branches and merge those when they are all done?
Thanks for this. I'm a self taught developer, got a job 3 years ago in a small company with 4 devs, all of which use different languages. Since then I've always been left 100% by myself -- this is the exact type of stuff I need to learn to move to the next step but don't know how. Most videos just focus on the superficial stuff anyone with some experience already knows.
Hey, Cody! Awesome video, man. It's so great to see experienced developers breaking down the process of how things work behind the scenes to bring others up. Keep pushing, bro.
I like that you introduce the detailed process of team development at a macro level, which can bring a clear reason for the task to the newbie to better understand the specific role of their project, which can think of a variety of excellent solutions for the purpose when encountering difficulties and reduce a lot of errors caused by the main line development
One problem we hit is deploy to ephemeral (for the feature branch/PR) environments for testing. Building a dev branch to a dev environment is quite easy. Also, we are developing an application that is cloud neutral. Each customer wants to run the cloud based application on different clouds. This is actually more difficult than you would imagine.
This is a great birdseye view, thanks so much Cody! For those curious about more, there is more context in the software architecture, software / solution designing phases in the SDLC that are just as complex. Proof of concepting, complex testing infrastructure and levels. AppSec / DevSecOps such as docker container scanning, container / code linting, static code analysis, dependency scanning and so much more. It is a very exciting, and sometimes daunting world! So much to love and hate, haha!
I worked with a team of nine developers, one UI/UX , one tester, can confirm this is exactly the process that we go through when we want to release a feature. Great video for sure
This is so cool. Even as an experienced developer, I sometimes find it difficult to explain complex tasks in a simplified way (like talking to a 5-year-old). So I enjoyed watching this video.
I'd disagree. I think there should always be at least a staging environment. You should avoid PRing into main/ production directly. If you have created the automation or have the automation available for CI/CD, it isn't that complex to create a second environment. Definitely do it. In my experience in the past of doing side projects all the time for small businesses, there is always the "can you add this feature" requests and putting it straight into their production system is almost always very difficult without confusing the end user or rolling back if the feature isn't wanted in the end, doing it in production is just something to avoid.
I've been developing for 15 years and always wondered where my personal position is in terms of large scale enterprise projects. Glad to get an insight and overview of everything and reassuring my confidence! Thanks (:
This actually touches on a LOT of important stuff and is a great break down of all things that go into enterprise design, implementaiton and maitenance.
Great video, I just wanted to add something, in my personal experience (9 years), I only once had a team of 11 people, and it was a complete disaster. Most of the time, teams are 4 to 7 devs, from my experience the smaller the better
We never cut a release off of "dev", we re-branch a fresh release branch off of "main" and merge in individual completed feature branches ready for "stage". It keeps the mess thats in "dev" isolated.
This is really impressive. I was afraid that the approaches we follow in our project were not upto the standards or different from what industry uses. But I am glad we follow almost a similar kind of approach. This reassured me.
This is a must watch for beginners developer! As an enterprise software engineer myself, this is the most accurate and detailed workflow of how developing on enterprise works. Nice video man!
I practice almost every single one of these techniques in my software development as well. The only difference is there’s only 1 dev on my team, me. Oh and there’s only 1 PM on my team, me. Oh and there’s only 1 client, me. Oh and there’s only 1 user, me. Oh, and I’m not even a software developer. I’m a photographer. But my software works as prescribed and designed. So I guess that’s a good thing.
I've been a web developer for 3 years now but only worked for startups/small companies, 2 so far to be more specific, and even if the development flow is similar in some ways, I have never been involved in such a complex scenario, although I'm not sure if this is better or worst, since being in a startup requires you to know at least a bit about how everything works in the system. If all our codebase were deleted overnight for some strange reason, every single dev alone in our team (with enough time of course, probably years) could build the whole system from scratch, but I don't think this can be possible to achieve in a big company.
The way you explain things about the dev process, it feels like a great challenge. Would be great to have an opportunity like that. Working on my certs. Great presentation!
20:00 onward - a lot of this feels like implementation details. The approach that our team settled on is making master the source of truth and having new features under feature flags turned on/off remotely.
I do a bit different work (HPC), but good intro to the topic. If your team isn't entirely crazy it's going to look something like this. Everything in source control, nothing pushed to prod without testing (the more testing the better), and automate as much as you can so nothing falls through the cracks
Great caution at the end to avoid complexity. It is very tempting to create branches for environments but it goes against the thinking of continuous integration - changes are kept isolated from each other (dev vs main). Instead branch by abstraction and use toggles, use the same code history but deploy to different environments.
A lot of others might do trunk based development (where you merge directly to main, have a ton of automated testing, and use feature flags to prevent unfinished code from being used by users), but in my situation we need to use long living feature branches. Sometimes you have to use a process that works best for your client and team.
That's true 💯 There are advocates for both Trunk based & Feature branches. Personally I like the FB approach you explained here. Mostly the final call must be based on the size & complexity of the product & the team
Hey Cody, this is one awesome video here bro 🔥💥🥳 You clearly brokedown the entire structure of a SW dev team piece by piece. I really liked how you explained the dev workflow FB > Dev > Staging > Main > Prod> User This is exactly how we work. 👍🏽
Also it sounded like like you were unsure about this kinda content. I kid you not, this has literally been one of THE BEST tech video I've seen on YT. Pls keep making more content like this. If you can pls talk a little bit more how a the PM interfaces with the Dev team and what's his role in day-to-day affairs. TIA bye
@@RamKumar-kv1fx this was magic in a bottle. I literally just turned on my camera and started talking while drawing diagrams. Maybe I can figure out the secret sauce
I watched this video like a year ago and didn't understand a thing. But now after getting my first job in the industry and our team is 400+ I can follow this video. We have our dev environment and multiple system testing environments and user acceptance testing environments and prod environment ofc. It's so complex and your video does a great job of depicting this. Thank you!
I am more shocked at how accurate this information is and the fact that this is available for free to see for anyone(not that it shouldn’t be, but that as far as I’ve seen, no one ever put out such a comprehensive video on enterprise level workflow). This is precisely the same workflow we have at our startup. Thanks for educating peeps!
It's a very simplistic way of explaining it, and somewhat accurate, but it depends on the maturity of the company. At Amazon we deploy 10 or 20 times per day, so if any bug is detected, we just treat it as a normal feature and the pull request goes to dev... testing is fully automated, so it usually takes 30 mins to reach production (depending on the blast radius of your change, because if you need to destroy and recreate infrastructure it might take a lot more). This explanation you gave also doesn't take into account supporting multiple versions of your software at the same time.
If anyone has a CS degree we learned most of this in school. It's really the foundation and building blocks of the SDLC but in enterprise form. It can be a single operation all the way to a large operation. It's nice to view the perspective on whiteboard to display to people what the process looks like. Good job making the video.
I'm about to complete my IT degree with RMIT and yeah we have focused heavily on SDLC too, Java C# C++ python and a few other languages in there. My capstone was essentially an Uber-style application but with a lot more complexity SEPM etc and with all the documentation, I have industry experience but I feel that university has a higher standard than my experience has offered me before my degree.
A lot of devs are self taught, so a lot of this infrastructure education is missing, as the primary focus is just making code projects that no-where near approach this level of complexity. I think it's great that he's taken the time to plot it out in such an intuitive fashion.
Unfortunately this is not true for every CS program. Many CS programs focus primarily on the theory, data structures & algos, and spend very little time on building applications or the SDLC. Even if they do touch the SDLC, it is never as hands on or concise. Not to mention, many, many schools are underfunded and DATED, such that they are barely even touching things like Agile, Version Control, or anything close to devops and how to deploy software in general.
@@codingwithcodi that's true even places like Udemy don't actually mark student projects and they focus on very narrow fields of study . Learning all the aspects of software development is more involved then what people might think . I know for certain. The programming takes centre stage but unfortunately programmers spend less time programming and more time problem solving it's about 20/80 in my opinion
I’ve been leading teams and contributing to large enterprise SaaS products. The Teams I work with have a large number of developers, in excess of a hundred. I think this video does a great job of breaking up the overall big picture into essential things to know. Yes there will be jira and other agile jargon and engineering jargon overlaid on top of this in addition to development jargon
Awesome insights Cody, I had asked yesterday during the livestream about cloud monitoring for GCP and this video is also super helpful since I am part of a team that doesn't have very good response time to bugs that make it into production... It's surprisingly difficult to figure out how to get better at knowing how a systems to deploy code and maintain it should actually work... Seems like maybe I need to find a new team that looks at these issues more seriously. Anyways thank you as always!
Some things we do is log every exception, the request, what endpoint was invoked, what user hit that endpoint, and what time. You need a way to inspect what the user did and sent in if you ever plan to fix issues
The hotfix scenario is a good example of why trunk-based development is preferred. Having different branches for different environments is a recipe for disaster. Each branch is a separate code state that has to be managed, and they get out of sync really easily.
agreed, but trunk based development usually means your team has the ability to do feature flags. Our project has a requirement that no unfinished stories can be merged to main. constraints often dictate processes
@@WebDevCody no, trunk based development means everything is merged directly into master. Feature flags should be a requirement in any decently sized project. its not that hard to implement. However even without feature flagging, as long as your tests are robust (prevent errors in the first place) and your CI/CD pipeline is good your time to resolution should be quick enough to mitigate any errors (whether through a revert/code fix).
If you can get away from the idea that "HEAD" in main is what is in production, you can start to realize why you don't need to use branches to manage your software versioning. I know you, and many companies, think that this is the best way to manage their SDLC, but I can assure you that the same thing you're trying to accomplish can be accomplished without maintaining long running dev/test branches.
There seems to be either nuance or confusion around what is and is not trunk development, so please tell me where I am going wrong. When people say "merge directly to master", merge what exactly? Your feature branch, right? People seem to employ the terminology as if they do not branch at all. If a team uses feature branches, but they are branching from and merging to master directly (forget feature flags) is that trunk development or is it using feature branches? Is it trunk development only if they are using feature flags?
Such a great explanation, thank you! As a junior software engineer, after watching this video, I can confidently say that I have gained a good understanding of how things are related to each other.
It can be very difficult to rollback code in a microservice architecture. Your deployed code represents many different components e.g. Av1, Bv1, Cv3, Dv4. Lets say you have an error in D version 4 so you rollback to V3. How do you know that Dv3 works alongside Av1, Bv1 and Cv3 if they never existed together in the first place? You have to be very careful when making changes to apply some of sort of semantic versioning to your system and even then I wouldn't be too confident without having things like canary deploys and a solid automated test suite in place.
That's why you use Infra as Code. And this code live in a repo of its own. When ever any new service is added or updated, this code updates too. And is versioned with Git 🤓
I have been in the industry for two+ years and I can say that this is a very realistic introduction to how things work in the industry for anyone who is trying to get into this field of cs.
Man, super nice video. I work in an pipeline like that, more or less. Not completely as a dev, but setting requirements, as client. I think it was really well summarized and well explained. Thank you! have an awesome one.
Awesome video! Your explanation of the AWS infrastructure is amazing and also explains why devops came about. Back in the olden days as an infrastructure dev, we were somewhat disconnected from what was happening on the developer side. Things like a build taking too long and our infra side being like well, that's a you issue since we only handle the infrastructure aspect and we don't know what you're trying to do and we don't have time to understand that because we have infra to manage. Developers also don't have time to understand the infra side because they're busy writing code. Devops came about to more closely integrate the infrastructure side with the developers, making it easier for us infra people to do things like work on speeding up build times, work on integrating tests, establish good infra security policies, etc. Devops of course then became a more generic "you work on infra? cool you're devops" title. Great video! What a wacky industry 🤣
Thanks a lot for this video ! No one talks about how the real every-day work at a company/enterprise is, and it really helps in understanding the skills we should have or how we should think about software development.
This is so helpful for me as a self-taught doing my best type coder. It also serves as a rough roadmap of what to learn beyond the direct coding skills, like more familiarity with Git and working with AWS. Thank you so much for this
Solid video. I just got done deploying all my personal projects so I got a big appreciation for the difference between working locally and working deployed.
Interesting. I deal with this type of stuff a lot at work and your channel is the first I've came across to actually discuss it. You even did it in an easy to understand way! Great stuff!
As a 22 year old just starting my full-time job as a SRE for a large company, this video was incredibly helpful and helped solidify my understanding about how all this stuff works. Really accurate to how things are set up at my company. Thanks
I didn’t expect this video to get that many views. I just wanted to make a video to give people who work solo or or on a smaller project more insight into a larger scale project. Your mileage may very. Every company does things their own way from what I’ve seen, and most companies have various teams that all work on their own sub systems and integrate with others.
please make more videos like this.
As a solo dev for a small business this video has been a huge boon :)
Nah, man it's useful to have stuf like this ( BASICS ) in understandable English. That last part is important. Also, interesting to see what the "Zeitgeist" is in Enterprise Software Development, because what you outline here is what we are doing too. Getting major DEJA VU
Waiting for more videos like this :) Thank you
@@emmettkeyser1110 better than cloudformation imo
As an enterprise developer myself, this is by far one of the best breakdowns of how enterprise software development works and I wish there was a video like this when I got my first developer job.
+1
@@lit22006 It sounds like you’ve never worked in enterprise if you can’t recognize an actual team structure in an organization, but please do enlighten me about what’s actually going on in such an environment if you’re so experienced in the matter.
This is actually a terrible / antiquated way to handle development. Breaking out branches into dev/test/prod just creates this nightmare scenario where you're constantly managing the state of the branches and is very prone to error.
A much better way is TBD with environment configurations and artifact promotion. This way you can "version" your software and control releases based off the build artifacts, instead of trying to maintain the state at the source code level through branches. Git is already versioned. You don't need to implement a branching strategy to add a needless abstraction around versioning.
@@lit22006This is literally enterprise level, whatever over-engineered heap of crap you have been involved with doesn’t make the cut, sorry
@@elliott8596 isn't a lot of enterprise antiquated?
Bro... it's as much real as it can be. It's THE BEST description of how different teams in 80% of the industry do their job. And you also covered rest of the 20% by including redundant systems, and multi layered prod systems (Production Microservices like architecture). Great stuff.👏
@WebDevCody I wish you could've also put this diagram in the description.
Yes, you're correct. Startups may have lower complexity, but the underlying concepts and patterns remain consistent. This video is truly exceptional, highlighting this fact.
Yeah, the concepts definitely apply to smaller startups as well, just at a smaller scale since you don't need to be as rigorous since it doesn't hurt as much when stuff breaks (compared to like some big company with 100M+ DAU where stuff breaking costs a ton).
Startups make features not projects.
merdat everywhere
Since the start of coding 2010 I was always on my own with solo projects as fullstack dev. I had always big questionmarks regarding those enterprise teams working together. Those 24 min of your video were so incredible eye opening to me and it made me a good feeling to finally understand how this works. Awesome work here Cody
This is epic. I've worked in enterprise environments for the last 10 years and this is by far the best intro explanation I have ever seen. Well done :)
thanks man
@@WebDevCody 20 years in the industry here - completely agree, 10/10 summary video.
As a final year CS student, I am so grateful to have watched this before my first actual job. Thanks a lot
Beginning of year I singned a simple VPS, the best choice I made so far, started learning linux, ssh, git hub actions, automatic deploy, git (for real), managing db, staging (my personal PC), production environment, nginx, and those things made me so more aware of how development works and how can I really deliver software as a product.
Explanation of DevOps :)
Where are you learning?
which VPS did you buy and how much does it cost per month? For now I am using aws ec2(my 1 year free tier is done) and when I am done, I just stop the instance(I still have to pay for storage, but I dont have to pay for instance since its not running). I started last week only, so not sure how much its going to cost after month. I am thinking that it should be less than $1. If its more than $5 then I might just buy digital ocean droplet or something.
could you elaborate on that VPS? I do want to (more like "have to") learn those components. It might sound silly, but I came from old monolithic way and my responsibility are only for Dev side, which is outdated and now we Dev are required to also hands on the operation side.
Thank you very much for this brillian discuss on how enterprise systems are developed and how enterprise teams work on large scale projects. This video is way more than a random video as you share a lot of insights into how large team are structured and how their projects are planned, developed and deployed. Quite a rare video on youtube. Thanks once again.
I am on a 5 man scrum team implementing robotics in software and this video holds pretty true. Depending on the scale of the company you find yourself in, that whole user interaction part can sometimes get absorbed by the roll of a pm and perhaps other individuals or teams, but this was a great summary of how things work.
Great video.
I'm a backend developer for an enterprise company, and this is exactly the way we do work as well. This is pretty much exactly the same way we work, but we have around 400-500 developers I'd say, and probably like 80 different development teams, and most teams are integrated with other team's services and products in some way to produce one entire seamless product that end users use (simplified).
As a web dev newbie who just came out of a coding bootcamp, this kind of content is super valuable since the teachers never really talk about this too much. I am so glad that I found your channel! This type of real world content is also rather rare on UA-cam, and I am looking forward to seeing more of this from you 😀
that was the worst thing about bootcamp
Bootcamp grad you say? Lemme underpay you real quick
As a SM this is a great for devs to understand why there's so much syncs and managment stuff or meetings. Enterprise will scalate like crazy were you can have like 100 teams trying to push features. Great video!
With 10+ years of enterprise web dev experience under my belt, i approve this. One thing you missed though are the many endless product team meetings. The flows you talk about here are happening in an ideal scenario. In reality there's a lot of big talks and "we're all a big family", feature rushes, egocentric people, pizza parties, "synergistic" in person meetings and not so much time actually writing good software. Btw, perhaps we should also open the topic of "cloud" costs. I've worked with one that started with gcloud and datastore and it turned into a hot mess that needed almost 1 year of refactoring and paying 10k+/month for infra alone
I've had only one professional experience with working with a bigger team and I have to say this video is spot on. Literally everything we did, we did exactly as explained in the video. The only thing I would say differed, is that we had no contact with users other than error logs on AWS, our client was pretty much 100% responsible for gathering user experience and send that our way, digested, so we could turn those into issues. We also didn't have automated smoke tests, we just had QA do those manually before releasing
Finally! Somebody outlined how this process works in teams at different scales! Excellent video. Thanks for making it.
This was amazing man. I always felt like some dots were missing in understanding the full picture.
Thanks for helping me connect more dots.
Happy Coding!!
solid video; to make it more adequate and complex - Business Analyst is a must (partly overtaken by UI/UX designer here) and QA allocation in entire process; regardless the role inside organization - there is always BA being a system functional expert being able to present and negotiate options - customers in general have ideas and do not go into details since they are not technical, do not know system limitations; do not undestand full workflow scenarios; are not capable to have a holistic view, identify gaps and risks. Same goes with Devs - they don't have to understand entire functional scope of the system, to memorize or the details, or at least it is often not achievable due to e.g. rotation inside company. BA role is key to be a proxy; while UI/UX designer is a part of that process, by role definition, it is not only about interface design; but I believe the simplification here is that in most cases - BA and UX experts coexists where UX designer is more technical role (UI) focusing on workflow and visual aspect
Thank you for this, I have mostly worked in small projects and this was an eye-opener.
We've been burnt multiple times using dev -> feature branch flow and I don't recommend my teams to do it. Now we always fork features from main, and work it up through the stages so that if we need to make a cut without some buggy feature we can and feel comfortable not leaking changes into prod. Our team also use a pre-prod environment to verify changes we release before they are done to make sure we have an exact replica of what prod will be/ or what it is, because staging may contain bugs that need to resolved. As you said multiple times it gets complex if your team is bigger or have multiple team working on solution spread across the organization. BTW. very good video, you got most of it clearly explained.
It is pretty much abandoned as a process and the original git-flow author has disowned it as a defective way to collaborate! People keep finding git-flow though so he put a big disclaimer at the top of the page. They still keep recommending it!
Trunk based is the way to go. Make small commits, deliver fast. Use feature toggle not to break prod.
@@ltbarcly I think once you search for git flow on Google one of the first thing that pop up is Atlassian's git flow recommendation/description of the process and there they branching from dev in there flow and i guess people just accept it because it Atlassian recommending it.
@@ltbarcly Could you please explain what we should avoid ? I haven't experienced any large scaled project but i'm planning to work on some large scaled projects so I would like to know what I will encounter. Personally, I also use dev and feature branches because I heard it's one of most common way.
Do you still have 3 enviornments? so you fork from the master branch, and merge into dev, if the feature is approved you merge that branch to staging which should be basically a copy of master and test it there, and then back to master?
I’m a junior web dev looking for my first job and this video is so valuable. Now I have an overview of the workflow and different roles. Thank you!🙏
I'm relieved that I (accidentally) made us follow some existing standard at my work! We ended up doing roughly the same things and it's been great for our productivity. I would like to emphasize that testing is what makes this work so well.
I was the Director of engineering at Byte for many years. Now the head of engineering at Sirge. Because we end up building products for enterprise businesses, the org it setup a bit differently. We have different business units that represent keys parts of the business. Accountings, Operations, Engineering, Marketing, Growth, Design, Sales, Customer Support/Success, just to name of few. When I run team I like to ensure that product is always representing the "What" we are going to build, and engineering is representing the "How". Product will normally work directly with design, along with customers, gaining feedback from surveying and customer interviews. Through the ideation cycles product will define that features/products we need to build next based on customer collected data. From their we have a series of meetings within an Agile/Scrum structure that allows us to carve our the user stories, functional requirements, point, then allocate to sprints. Thats just a high level. But my team sizes grow pretty large.
Ah story points the arbitrary unicorn poop that even their creator has recanted on
@@furycorpcan confirm. Story points never align with what Devs predict.
Hey Cody, Love the content. I've worked professionally as an engineer for 13+ years amongst varies enterprises and this top tier content for helping dev's understand enterprise software life cycles! I will say, I'm surprised that a Business Analysts isn't sandwiched somewhere between your PM, Designer, and Client.
Idk what a business analyst is, so I may not even be working deep enough in enterprise 😂
@@WebDevCody Hahahahahaha
Yeah in my company we have Business Analysts, Payment Analysts and Payment operations
So many companies have replaced business analyst with this perverted concept of Product Manager entirely (to their peril imho) to the extent that those following "agile knockoff" as I call it don't know what BA's do. Where I've seen this, arbitrary rectangles in Figma replace proper documentation, diagrams, capturing of scenarios + use-cases, etc.
You probably have worked deep enough Cody just at places where the bootcamp rectangle kids took over.
when us devs can't keep up with requirements from clients, BAs be outhere meeting with them 8 hours a day and drawing all kinds of diagrams for us. coding comes easy after that. in smaller teams, BAs sometimes work as frontend dev/designer as well.
In my experience there has generally been a layer of separation between Client/Support and Developers. It is important that issues are properly ticketed and discussed in daily standups and assigned to either the backlog or to the appropriate developer.
Also, at least where I work, there is an "Architect" team that are often the most senior developers of the various teams working on different projects, and the architect team will have meetings where they look at what code is duplicated and evaluate where it is worth developing an internal shared library that multiple projects would benefit from. We have one team that develops those libraries and they're often for cases where performance is critical or reliability is critical, so for example we have a shared library for direct communication to Mellanox network cards, and we have a shared library for 3-mode redundancy.
Great as always!
Would love to hear your thoughts about git flow.
How do you maintain commits while merging, apply hotfixes for several branches, rebase or not, etc...
This would be a great addition for this video!
Great questions! In my experience, I think having multiple active branches causes more problems than it solves. I'm one for using trunk-based development but it requires having tests that run when PR from a feature branch is open. Master/main is always a deployable state. We always did squash merge.
A video soon web dev cody ?
@@roach_iam I agree, short lived feature branches that are merge back into main makes for easier testing and faster releases.
Thanks for taking the time and making this video. I didn't know I knew how to navigate all of this mess, and having it laid down so beautifully is amazing.
Have worked as a senior architect for many years now and have to say this is one of the better videos I have seen on Enterprise Development. Having to deal with 5 or more of those projects simultaneously while understanding all of the legal, technical and other dependencies is why the complexity exists even though it can be extremely frustrating for developers and others on the front lines.
Hopefully it helps people understand why, sometimes, they are told no when they just want to add a "simple" maven plugin or NuGet package to their project that will make things so much easier... there is often a lot more going on that might not be immediately visible.
Great video 😎
(edit: Architects can't spell)
Legend
Lllkk0ļ...... MmÒp) kojkhtttttrr😅😮😮😮😮 10:03
Dude! This was amazing. As developers we never actually get taught this (not even in college), but we kind of pick it up in the different organizations we get to work for under our career. Seeing it laid out like this is beautiful, thanks.
As a senior software engineer for about 10 years, I take all this for granted. Really good video!
Cody's advise near the end takes "testing in production" to a new level.
Literally what we do summed up in a neat video. I will be sharing this aggressively to everyone who wants to become a cs major to set some expectations of their daily, as not many know this before getting their 1st actual job.
Thank you Cody. As a startup founder with no industry experience, it's helpful to get an overview of how it works!
As someone who hasn’t worked as a developer at a company yet, this info is invaluable. Thanks for taking the time to talk through the process and share your knowledge 🙏🏼
Your explanation has given me a much better grasp of the complex ecosystem involved in larger scale software development, both from a technical and organizational perspective. The visuals and real-world examples made the concepts more concrete and easier to understand. Thank you for taking the time to share your knowledge and expertise in such a clear and compelling manner.
different branches for different environments (aka git flow) works well if you are building software with specific versions and you are cutting different releases with a specific list of features included in each release. but for development of public-facing web products trunk based development (aka github flow) is more common - a dev can pick up a ticket, make the change, have it tested by QAs/automation & have it in prod by the end of the day. then you don't need to worry so much about what's getting shipped when, you are just constantly building stuff and it's in prod as soon as it's ready.
Very good point. We need to explicitly say that for new devs, I would even discourage introducing new devs to gitflow. It's bad, it only works for situations as you mentioned. Trunk based is much simpler and in most cases preferable. I've seen so many devs using branch names with prefixes such as "feature/..." and that's just because they've picked it up somewhere else without even knowing what they're doing - now they think it's a git standard. That's completely unnecessary and a bad habit imo. Simplicity is the key.
This could seem to imply that switching to trunk based automatically means your tickets are small (small enough to do in a day) but I don't see what one has to do with the other. I'm sure you'll agree that ticket size is basically unrelated to the branch strategy? Someone on my team is trying to get us to switch to trunk development and I'm having trouble seeing the benefits.
FWIW in my team we do feature branches but we do not have a dev branch like is sketched in the video between feature branches and master.
@@xellestar trunk based does work best with smaller tickets. they don’t need to be doable within a single day but it shouldn’t take more than a sprint.
the main idea is that you build features with small iterations which each go to prod individually over time rather than dropping a whole finished fully working feature in a specific release. it just lets you work in a more agile way where you can pivot more easily as new priorities come in. if you’re not able to break down your tickets into smaller chunks of work that can be deployed independently without breaking stuff, then trunk based might not be right for the software you’re working on.
of course you also need to have the appropriate infrastructure eg. CI/CD pipelines, automated testing in pre-prod environments etc.
@@a__guy I'm all for building smaller things in iterations. But let me repeat a comment/question I left on a reply to someone else:
There seems to be either nuance or confusion around what is and is not trunk development, so please tell me where I am going wrong. When people say "merge directly to master", merge what exactly? Your feature branch, right? People seem to employ the terminology as if they do not branch at all. If a team uses feature branches, but they are branching from and merging to master directly (forget feature flags) is that trunk development or is it using feature branches? Is it trunk development only if they are using feature flags?
Ticket size and how long it takes to finish a feature has nothing to do with branching model. You can develop on your branch and merge to master once it's ready. That's still trunk development. It's your responsibility to keep your branch up to date with master (which you should be doing as often as possible). And if this doesn't work, then you have different kinds of problems that aren't related to branching model. Probably bad code quality and poor planning.
Huge thanks man!
You have no idea how valuable this is for someone who is self-taught and been working solo!
Watching this put me in a state of anxiety.
I started my journey in software development in a very small startup - no automated tests, code hosted on VPS with manual pull and build, no docker or anything. so I started to discover that there is more to software dev than writing bunch of code and push it to the main directly. now am discovering testing, cloud, docker and deployment and trying to understand how to implement all of this in my personal / freelance projects. Your video helps a lot and I want to thank you
This video is fantastic at explaining high friction software development but it's surprising the amount of companies still to this day that have no automation with CI/CD or even proper use of version control (as the company I work for daily drives Team Foundation Server Version Control). Obviously everyone's use case it different as some companies still have on premise servers but overall companies from big to small should take note of the usability and simplicity of making a good software development workflows that's right for them (and that's leverages more modern tooling). Good stuff!
For large and important systems, in practice you have to use incremental deployment of new features into production for a limited set of users. Of course, this also brings complications - different versions of the application must be compatible with each other, and the production environment must allow gradual deployment.
I'm a co founder at a startup, during the video i was starting to lose faith on our way of doing things, but the last couple of minutes your restored all my confidence and giced me courage to continue the amazing journey I'm in.
how are y'all doing things at your start up? honest, keep it simple is what I've learned and add process when and only when you keep having real issues
Great video. I see quite a few similarities to what my team does. We don’t deploy branches other than master to environments. Feature work and bug fixes are done in branches then merged into master upon PR review and CI checks passing. My team composition is just about the same but we have QA attending all our standups. I wasn’t sure if QA is part of your standups because they were not listed alongside devs, scrum masters, and PMs. Interesting to see how other teams work. Would love to see more videos like this about SDLC.
yeah it sounds like you do trunk based development. So do you merge code daily to master, or do you have long living feature branches and merge those when they are all done?
Thanks for this. I'm a self taught developer, got a job 3 years ago in a small company with 4 devs, all of which use different languages. Since then I've always been left 100% by myself -- this is the exact type of stuff I need to learn to move to the next step but don't know how. Most videos just focus on the superficial stuff anyone with some experience already knows.
Hey, Cody! Awesome video, man. It's so great to see experienced developers breaking down the process of how things work behind the scenes to bring others up. Keep pushing, bro.
I like that you introduce the detailed process of team development at a macro level, which can bring a clear reason for the task to the newbie to better understand the specific role of their project, which can think of a variety of excellent solutions for the purpose when encountering difficulties and reduce a lot of errors caused by the main line development
That's why your one of my favorite programming channel. No bullshit and going straight to what real business is. Props to you for that !
One problem we hit is deploy to ephemeral (for the feature branch/PR) environments for testing. Building a dev branch to a dev environment is quite easy. Also, we are developing an application that is cloud neutral. Each customer wants to run the cloud based application on different clouds. This is actually more difficult than you would imagine.
This is a great birdseye view, thanks so much Cody! For those curious about more, there is more context in the software architecture, software / solution designing phases in the SDLC that are just as complex. Proof of concepting, complex testing infrastructure and levels. AppSec / DevSecOps such as docker container scanning, container / code linting, static code analysis, dependency scanning and so much more. It is a very exciting, and sometimes daunting world! So much to love and hate, haha!
really appreciate hearing about the organizational aspect of software development
I worked with a team of nine developers, one UI/UX , one tester, can confirm this is exactly the process that we go through when we want to release a feature. Great video for sure
This is so cool. Even as an experienced developer, I sometimes find it difficult to explain complex tasks in a simplified way (like talking to a 5-year-old). So I enjoyed watching this video.
I'd disagree. I think there should always be at least a staging environment. You should avoid PRing into main/ production directly. If you have created the automation or have the automation available for CI/CD, it isn't that complex to create a second environment. Definitely do it. In my experience in the past of doing side projects all the time for small businesses, there is always the "can you add this feature" requests and putting it straight into their production system is almost always very difficult without confusing the end user or rolling back if the feature isn't wanted in the end, doing it in production is just something to avoid.
I've been developing for 15 years and always wondered where my personal position is in terms of large scale enterprise projects. Glad to get an insight and overview of everything and reassuring my confidence! Thanks (:
I love these more advanced videos that are not just "should you use Prisma?". Thanks a lot for this!
This actually touches on a LOT of important stuff and is a great break down of all things that go into enterprise design, implementaiton and maitenance.
Thanks
Great video, I just wanted to add something, in my personal experience (9 years), I only once had a team of 11 people, and it was a complete disaster. Most of the time, teams are 4 to 7 devs, from my experience the smaller the better
One of the best explanations of how the sausage is made. And I also worked in the industry for over a decade. great stuff, keep it coming!
We never cut a release off of "dev", we re-branch a fresh release branch off of "main" and merge in individual completed feature branches ready for "stage". It keeps the mess thats in "dev" isolated.
We need more videos like this, for professionals and who already works as a developer, good job, thank you.
This is really impressive. I was afraid that the approaches we follow in our project were not upto the standards or different from what industry uses.
But I am glad we follow almost a similar kind of approach. This reassured me.
This is a must watch for beginners developer!
As an enterprise software engineer myself, this is the most accurate and detailed workflow of how developing on enterprise works. Nice video man!
I practice almost every single one of these techniques in my software development as well.
The only difference is there’s only 1 dev on my team, me. Oh and there’s only 1 PM on my team, me. Oh and there’s only 1 client, me. Oh and there’s only 1 user, me.
Oh, and I’m not even a software developer. I’m a photographer.
But my software works as prescribed and designed. So I guess that’s a good thing.
😆
This 100% jibes with my experience as a senior dev in saas orgs. Super clear explanation and diagramming!
I've been a web developer for 3 years now but only worked for startups/small companies, 2 so far to be more specific, and even if the development flow is similar in some ways, I have never been involved in such a complex scenario, although I'm not sure if this is better or worst, since being in a startup requires you to know at least a bit about how everything works in the system. If all our codebase were deleted overnight for some strange reason, every single dev alone in our team (with enough time of course, probably years) could build the whole system from scratch, but I don't think this can be possible to achieve in a big company.
The way you explain things about the dev process, it feels like a great challenge. Would be great to have an opportunity like that. Working on my certs. Great presentation!
That bird-eye view looks like an electric circuit.
haha for real, it's why larger projects become to complex
I had a similar thought. To me it started looking more and more like a CPU schematic (which is also a circuit I guess)
20:00 onward - a lot of this feels like implementation details. The approach that our team settled on is making master the source of truth and having new features under feature flags turned on/off remotely.
I do a bit different work (HPC), but good intro to the topic. If your team isn't entirely crazy it's going to look something like this. Everything in source control, nothing pushed to prod without testing (the more testing the better), and automate as much as you can so nothing falls through the cracks
Great caution at the end to avoid complexity. It is very tempting to create branches for environments but it goes against the thinking of continuous integration - changes are kept isolated from each other (dev vs main). Instead branch by abstraction and use toggles, use the same code history but deploy to different environments.
A lot of others might do trunk based development (where you merge directly to main, have a ton of automated testing, and use feature flags to prevent unfinished code from being used by users), but in my situation we need to use long living feature branches. Sometimes you have to use a process that works best for your client and team.
That's true 💯 There are advocates for both Trunk based & Feature branches. Personally I like the FB approach you explained here.
Mostly the final call must be based on the size & complexity of the product & the team
Hey Cody, this is one awesome video here bro 🔥💥🥳
You clearly brokedown the entire structure of a SW dev team piece by piece. I really liked how you explained the dev workflow
FB > Dev > Staging > Main > Prod> User
This is exactly how we work. 👍🏽
Also it sounded like like you were unsure about this kinda content. I kid you not, this has literally been one of THE BEST tech video I've seen on YT. Pls keep making more content like this.
If you can pls talk a little bit more how a the PM interfaces with the Dev team and what's his role in day-to-day affairs. TIA bye
@@RamKumar-kv1fx this was magic in a bottle. I literally just turned on my camera and started talking while drawing diagrams. Maybe I can figure out the secret sauce
Thanks for aligning that UX/UI icon, much appreciated.
As a backend developer looking for a job, this video really made me understand the real word development cycle. Thanks for this awesome video man.
I watched this video like a year ago and didn't understand a thing. But now after getting my first job in the industry and our team is 400+ I can follow this video. We have our dev environment and multiple system testing environments and user acceptance testing environments and prod environment ofc. It's so complex and your video does a great job of depicting this. Thank you!
I am more shocked at how accurate this information is and the fact that this is available for free to see for anyone(not that it shouldn’t be, but that as far as I’ve seen, no one ever put out such a comprehensive video on enterprise level workflow). This is precisely the same workflow we have at our startup. Thanks for educating peeps!
It's a very simplistic way of explaining it, and somewhat accurate, but it depends on the maturity of the company. At Amazon we deploy 10 or 20 times per day, so if any bug is detected, we just treat it as a normal feature and the pull request goes to dev... testing is fully automated, so it usually takes 30 mins to reach production (depending on the blast radius of your change, because if you need to destroy and recreate infrastructure it might take a lot more). This explanation you gave also doesn't take into account supporting multiple versions of your software at the same time.
Well, I'm in the banking sector and on Mainframe, we deploy 4 times a year to production ;-)
Fail forward?
If anyone has a CS degree we learned most of this in school. It's really the foundation and building blocks of the SDLC but in enterprise form. It can be a single operation all the way to a large operation. It's nice to view the perspective on whiteboard to display to people what the process looks like. Good job making the video.
I'm about to complete my IT degree with RMIT and yeah we have focused heavily on SDLC too, Java C# C++ python and a few other languages in there. My capstone was essentially an Uber-style application but with a lot more complexity SEPM etc and with all the documentation, I have industry experience but I feel that university has a higher standard than my experience has offered me before my degree.
A lot of devs are self taught, so a lot of this infrastructure education is missing, as the primary focus is just making code projects that no-where near approach this level of complexity. I think it's great that he's taken the time to plot it out in such an intuitive fashion.
Unfortunately this is not true for every CS program. Many CS programs focus primarily on the theory, data structures & algos, and spend very little time on building applications or the SDLC. Even if they do touch the SDLC, it is never as hands on or concise. Not to mention, many, many schools are underfunded and DATED, such that they are barely even touching things like Agile, Version Control, or anything close to devops and how to deploy software in general.
@@codingwithcodi that's true even places like Udemy don't actually mark student projects and they focus on very narrow fields of study . Learning all the aspects of software development is more involved then what people might think . I know for certain. The programming takes centre stage but unfortunately programmers spend less time programming and more time problem solving it's about 20/80 in my opinion
I learned a lot about algos and math when getting my CS degree, but nothing about the stuff in this video. I wish university would teach this stuff.
Man I’ve been tuned in for a while, looks like this video will get you a trip to Disney world for Christmas 😂. Congrats bro
Good job babe!!!! Also 👸🏿
You are always here first😂😂
True support ❤
I’ve been leading teams and contributing to large enterprise SaaS products. The Teams I work with have a large number of developers, in excess of a hundred. I think this video does a great job of breaking up the overall big picture into essential things to know. Yes there will be jira and other agile jargon and engineering jargon overlaid on top of this in addition to development jargon
I've been working in Product for over 7 years and you beautifully sum up the whole idea of entrerprise software development. Best wishes from Brazil
Awesome insights Cody, I had asked yesterday during the livestream about cloud monitoring for GCP and this video is also super helpful since I am part of a team that doesn't have very good response time to bugs that make it into production... It's surprisingly difficult to figure out how to get better at knowing how a systems to deploy code and maintain it should actually work... Seems like maybe I need to find a new team that looks at these issues more seriously. Anyways thank you as always!
Some things we do is log every exception, the request, what endpoint was invoked, what user hit that endpoint, and what time. You need a way to inspect what the user did and sent in if you ever plan to fix issues
As a person who are not SWE but interested in software engineer adaptation, this has fully satisfied my curiousity. Thank you Cody.
The hotfix scenario is a good example of why trunk-based development is preferred. Having different branches for different environments is a recipe for disaster. Each branch is a separate code state that has to be managed, and they get out of sync really easily.
agreed, but trunk based development usually means your team has the ability to do feature flags. Our project has a requirement that no unfinished stories can be merged to main. constraints often dictate processes
@@WebDevCody no, trunk based development means everything is merged directly into master.
Feature flags should be a requirement in any decently sized project. its not that hard to implement. However even without feature flagging, as long as your tests are robust (prevent errors in the first place) and your CI/CD pipeline is good your time to resolution should be quick enough to mitigate any errors (whether through a revert/code fix).
@@WebDevCody On a project with 480+ developers have a similar approach, no unfinished stories merged to the main.
If you can get away from the idea that "HEAD" in main is what is in production, you can start to realize why you don't need to use branches to manage your software versioning.
I know you, and many companies, think that this is the best way to manage their SDLC, but I can assure you that the same thing you're trying to accomplish can be accomplished without maintaining long running dev/test branches.
There seems to be either nuance or confusion around what is and is not trunk development, so please tell me where I am going wrong. When people say "merge directly to master", merge what exactly? Your feature branch, right? People seem to employ the terminology as if they do not branch at all. If a team uses feature branches, but they are branching from and merging to master directly (forget feature flags) is that trunk development or is it using feature branches? Is it trunk development only if they are using feature flags?
Such a great explanation, thank you! As a junior software engineer, after watching this video, I can confidently say that I have gained a good understanding of how things are related to each other.
It can be very difficult to rollback code in a microservice architecture. Your deployed code represents many different components e.g. Av1, Bv1, Cv3, Dv4. Lets say you have an error in D version 4 so you rollback to V3. How do you know that Dv3 works alongside Av1, Bv1 and Cv3 if they never existed together in the first place? You have to be very careful when making changes to apply some of sort of semantic versioning to your system and even then I wouldn't be too confident without having things like canary deploys and a solid automated test suite in place.
That's why you use Infra as Code.
And this code live in a repo of its own. When ever any new service is added or updated, this code updates too. And is versioned with Git 🤓
@@RamKumar-kv1fx infa as code has nothing to do with this problem
Helm is your best friend. To rollback to a previous build or a suite of builds, while you and your devs actually resolve the issue.
Had to learn all of this the hard way in my first job. Great video, deserves to be more popular!
A freelancer watching this will go : I AM WHOLE ENTERPRISE :)
I have been in the industry for two+ years and I can say that this is a very realistic introduction to how things work in the industry for anyone who is trying to get into this field of cs.
glad you found it useful
Man, super nice video.
I work in an pipeline like that, more or less.
Not completely as a dev, but setting requirements, as client.
I think it was really well summarized and well explained. Thank you! have an awesome one.
Awesome video! Your explanation of the AWS infrastructure is amazing and also explains why devops came about. Back in the olden days as an infrastructure dev, we were somewhat disconnected from what was happening on the developer side. Things like a build taking too long and our infra side being like well, that's a you issue since we only handle the infrastructure aspect and we don't know what you're trying to do and we don't have time to understand that because we have infra to manage. Developers also don't have time to understand the infra side because they're busy writing code. Devops came about to more closely integrate the infrastructure side with the developers, making it easier for us infra people to do things like work on speeding up build times, work on integrating tests, establish good infra security policies, etc. Devops of course then became a more generic "you work on infra? cool you're devops" title. Great video! What a wacky industry 🤣
Thank you,just started my first internship a month ago and alot of what you have shown is pretty much spot on.
Thanks a lot for this video ! No one talks about how the real every-day work at a company/enterprise is, and it really helps in understanding the skills we should have or how we should think about software development.
Missing from this diagram: each pull request should be gated on some automated testing to prevent your dev branch from getting constantly broken.
This is so helpful for me as a self-taught doing my best type coder. It also serves as a rough roadmap of what to learn beyond the direct coding skills, like more familiarity with Git and working with AWS. Thank you so much for this
Solid video. I just got done deploying all my personal projects so I got a big appreciation for the difference between working locally and working deployed.
Interesting. I deal with this type of stuff a lot at work and your channel is the first I've came across to actually discuss it. You even did it in an easy to understand way! Great stuff!
exceptional! you just draw the enterprise software development picture perfectly. Thanks, Cody.
As a 22 year old just starting my full-time job as a SRE for a large company, this video was incredibly helpful and helped solidify my understanding about how all this stuff works. Really accurate to how things are set up at my company. Thanks
how did you get that job
@@ss-to7iiI got a bachelors degree in computer science, minor in IT. Interned there during school, got signed on full time after I graduated.
fantastic video, been in enterprise for a little over 2 years and this cleared up a lot of questions that've been dancing in my head