8 Terraform Best Practices that will improve your TF workflow immediately
Вставка
- Опубліковано 8 тра 2024
- 8 Terraform Best Practices that will improve your Terraform workflow immediately
▬▬▬▬▬▬ Learn more about Terraform? 🚀 ▬▬▬▬▬▬
Terraform explained in 15mins ► • Terraform explained in...
Complete Terraform Course for Beginners ► bit.ly/3OCoCPu
Terraform in complete DevOps process ► bit.ly/3WtBARg
Terraform is one of the most popular Infrastructure as Code tools out there. And if you’ve just started working with Terraform, you may be asking yourself, whether you are doing things in the right way. So in this video, you will learn 8 Terraform best practices that will improve your Terraform workflows immediately and make you feel more confident when using Terraform in your projects.
► This video is sponsored by env0 (Terraform Cloud alternative) 🙌🏼
► Check out www.env0.com/ for more information!
#terraform #devops #techworldwithnana
▬▬▬▬▬▬ Useful Links 🔗 ▬▬▬▬▬▬
► Remote State: www.terraform.io/docs/languag...
► State Locking: www.terraform.io/docs/languag...
▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬
0:00 - Intro
0:25 - Terraform State & State File - Best Practices around State
1:18 - BP 1: Manipulate state only through TF commands
1:46 - BP 2: Remote State
2:44 - BP 3: State Locking
3:43 - BP 4: Back up State File
4:23 - BP 5: Use 1 State per Environment
5:36 - BP 6: Host TF code in Git repository
6:56 - BP 7: CI for TF Code
7:39 - BP 8: Execute TF only in an automated build
8:28 - Wrap Up & More TF Resources
▬▬▬▬▬▬ Want to learn more? 🚀 ▬▬▬▬▬▬
Full Docker course ► • Docker Tutorial for Be...
Full Python course ► • Python Tutorial for Be...
Full K8s course course ► • Kubernetes Tutorial fo...
DevOps Tools, like Terraform, Prometheus ► bit.ly/2W9UEq6
▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
INSTAGRAM ► bit.ly/2F3LXYJ
TWITTER ► bit.ly/3i54PUB
LINKEDIN ► bit.ly/3hWOLVT
Facebook group ► bit.ly/32UVSZP
DEV ► bit.ly/3h2fqiO
▬▬▬▬▬▬ Courses & Bootcamp & Ebooks 🚀 ▬▬▬▬▬▬
► Become a DevOps Engineer - full educational program 👉🏼 bit.ly/45mXaer
► High-Quality and Hands-On Courses 👉🏼 bit.ly/3BNS8Kv
► Kubernetes 101 - compact and easy-to-read ebook bundle 👉🏼 bit.ly/3Ozl28x
Follow me on Instagram for behind-the-scene content 😊 bit.ly/2F3LXYJ
If you want to support this channel, please leave a like :)
▬▬▬▬▬▬ Learn more about Terraform? 🚀 ▬▬▬▬▬▬
Terraform explained in 15mins ► ua-cam.com/video/l5k1ai_GBDE/v-deo.html
Complete Terraform Course for Beginners ► bit.ly/3OCoCPu
Terraform in complete DevOps process ► bit.ly/3WtBARg
▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬
0:00 - Intro
0:25 - Terraform State & State File - Best Practices around State
1:18 - BP 1: Manipulate state only through TF commands
1:46 - BP 2: Remote State
2:44 - BP 3: State Locking
3:43 - BP 4: Back up State File
4:23 - BP 5: Use 1 State per Environment
5:36 - BP 6: Host TF code in Git repository
6:56 - BP 7: CI for TF Code
7:39 - BP 8: Execute TF only in an automated build
8:28 - Wrap Up & More TF Resources
▬▬▬▬▬▬ Useful Links 🔗 ▬▬▬▬▬▬
► Remote State: www.terraform.io/docs/language/state/remote.html
► State Locking: www.terraform.io/docs/language/state/locking.html
Hey Nana, I searched for your terraform course on Udemy but find it but costly considering in Indian currency. Is there anyway I could get some discount on it?
Hi Nana,
A very nice video on Terraform. Can you also create one a basic Terraform Tutorial video and share your views on it.
Thanks
We are honored to sponsor this great video!
- Use remote state with versioning and locking;
- Use workspace for multiple environments;
- Use for_each instead of count if it's possible;
- Never save TF state files in git, they can contain sensitive information in plain text format;
- Use modules for code reuse (DIY);
Thanks for a video :)
For bullet 4 - why? If we're speaking in best practice terms, you shouldn't have sensitive information in plain text anyway. This should mitigate any issues with maintaining state files in git, no?
@@lmind2 It could store auto-generated secrets like service account keys. Also if you already have remote state why do you want it on git? It would be a hassle to maintain the current state across your branches.
Great job! Love your presentation style and have enjoyed many of your videos over the past year.
Your explanations are concise and to the point. Keep creating these videos it helps a lot.
Love your videos as always, concise, precise and crisp, thank you
Amazing best practices, one to add is to create names or identifiers dynamically for the resources that cannot be deployed multiple times, this way you can deploy IaC for features branches to test your changes before merging to the main branch
I just love the way you present the information in your videos! Best videos out there
Love the content, and I’m happy to report that I’ve been following these best practices for several years now.
Only thing I’d recommend is to use modules as a best practice. It’s just easier to manage components vs one file with all the resources.
Nice video, you can also include -:
Using of TF modules to follow DRY code practices.
Use of terraform workflows to deploy similar type is multiple environments.
Securing sensitive variable to output on console.
Securing state files wherever kept.
one of the best videos that actually shows how big orgs do it. Thanks
This env0 for both Terraform and DevOps as a whole looks and sounds so cool! Great information there Ms.Nana. Thank you so much!
- Use workspaces to better organize state files
- Use pre-commit hooks to do basic Terraform fmt, linting before commiting changes
Hi Nirav,
I never have used workspaces in real life, I think that if you can use different state files for different environment it wouldn't be necessary. Could you give any reason to use it? I could be wrong.
Regards
Thanks for sharing Nirav! 👍
actually using workspaces is a pretty bad practice I feel as differents workspaces are juste subset files of a main state so all your states are hosted within a single remote backend, so it becomes a single point of failure, you wouldnt want you prod state deleted by a mistake made on a temp workspace.
There is a book called terraform up and running published by Oreilly and if I recall well, it is stated toavoid using workspace if you arent doing it on terraform enterprise.
@@lmart7402 Thanks for your reply.
@@lmart7402 what about terragrunt to manage envs?
One thing worth mentioning regarding tfstate and putting your terraform into version control is that you do not want to version control your .tfstate files. These can potentially contain secrets in plain text. The best thing to do in this case is switch to a remote storage method like s3 even if you're not part of a team, just to be on the safe side.
Thank you Nana. This is a handful for Terraform beginner developers.
What I would add from my experience - creating only one state file for one environment can be an issue if the environment has a lot of resources. A bigger state means a much longer plan and apply (and very often more $$$ if someone uses CI/CD as SaaS), so my suggestion is to divide code into workspaces and use state files for each of them. Keep up to 50 resources per state.
Mariusz, I agree with you, for me the most important thing in Terraform is to avoid the Terralith. Terraform code should be split in multiple dimensions: environments, frequency of updates (of infra components), type of resources, permissions (avoid to run with TF with SuperAdmin SAs), providers. The downside when splitting terraform in multiple dimensions is to keep track of the dependencies. I think IaC is still very early stages and needs more automation.
She have already mentioned this point
Hi Nana, you are awesome.the way you explain things is super easy.
You know what, I became Devops expert only watching your videos.
Hi Nana, I just wanted to give kudos to you for your Terraform course on Udemy. I finished it some weeks ago and I can use my new knowledge in my daily work.
Thank you Tilo for your feedback! It's really great to hear that my course is helpful for your job 😊👍
Using terrform modules instead of repetative resources and pass the required input arguments to this module, this module can also be versioned and kept in git and use tags (with incremental versions eg: v0.1) to refer in the main code. This enhances our code to become better in terms of readability.
Also, we have to hide the sensitive content in tf output vars ( if any) and donot hardcode sensitive protected info in tf code, instead handle them through CI ( store them as jenkins credentials ), write jpac to read it and pass it as CLI arg to tf commands as needed.
Stellar content! Can you discuss environment management strategies for Kubernetes workloads leveraging the GitOps model? 🙏🏽
Thanks for this video Nana. One thing I was expecting in this was to organise terraform code in reusable modules.
Great Best Practices Explanation! Really awesome explanation on how to utilize 1 state per environment and how to organize states!
Thank you :)
It is a very good practice to set default_tags on aws infrastructure. And also to create a resource group to group easily all cloud resources by filtering on some tags.
Perfect timing!
Your videos are few in thousands that I enjoy watching till the end.
Hello Nana.
Another best practice it's trying to avoid throwing resources to the main files in all the environments that's has the same deployment. This should live in a separate versioned/tagged module allowing to have a standard way to deploy new environments.
Thank you 🙏. Gonna give env0 a try!
i love this video, thanks for sharing.
Thx for sharing! How about a video on managing terraform modules? Or maybe how to separate out IaC for staging, production, and dev environments :)
+1 about terraform modules, and perhaps outputs
+1 that would be a great one, looking forward to it
great video Nana, Love all of these
Just a quick tip regarding remote state. In my practice I'm using state separation per service as well, not just per env.
For example, gke cluster or cloud SQL tf modules should store their state files within different folders. It could be easily configured in Terragrunt - backend.tf could be generated automatically with required configuration.
Finally it will look like: /terraform/state/environment/service.
Usage of terraform workspaces is not a good idea especially with distributed teams. IMHO.
Hi Nana, thank you for the great content you produce, what's your preferred choice for testing IaC (terraform in particular)? Thanks
Thanks for this great video.
I would also add the use of Terraform Workspace to separate different environments as best practice. Thoughts?
Great video as always. Can you tell me what application did you use for working on the animations in your video?
Hi Nana, Thanks for the video. Terra grunt can be use for DRY your terraform configuration across multiple environments.
Terraform modules also one of the best practice to share other terraform users for quick start/update without reinventing or duplicating the terraform code
Great content, I really appreciate your time. I am just curious, what tool do you use to animate your slides?
Perfect Nana!
Hello Nana
Great job as usual 👏
I think there is a another important point to discuss regarding Secrets in plain text
Thank you 😊
THANK YOU!
Very good explanation now i have good idea about TF State. Thanks
Hi Nana, may I ask which software are you using to edit such beautiful and interactive videos? Thanks a lot
your youtube tutorials are very high-quality content, I wanted to purchase a course and i did for the terraform course, but for the DevOps course it's very expensive, could you consider please some discounts :)
Hi Nana, great tutorial. Is there a way to get notify when someone in the team execute a terraform plan?
Great video!
Yes I Got Best UA-cam Channel For My DevOps Journey
Thank you
Nice video, if you remake please include more about secrets management and what files you don't want to commit to any public repositories.
Does using a repository to host your terraform code eliminates the need to have a dedicated storage for the state file ?
Or , we use repository to host only the code and after completing the pipelines the new changes will get the latest state and update it ?
Thanks in advance ^^
Always like, and Already Subscribed with opening bell icon 😅
I’m actually interested on how to test IaC code.
Very useful information 👌
Hi Nana, could you share your source for the popularity statistic of IaC tools at 0:03? I've been looking for something similar for my Bachelors thesis. Thanks in advance
Can You please cover What all topics to study for Hashicorp Terraform : Associate Exam
hey, one question regarding the state file for each environment, since I'm creating a state file for each env, is it best practice for all to be in the same bucket with different directories or different buckets? and regarding the dynamodb, should be one db for the state files?
Hello Nana, yet another great tutorial 👍 But I was just wondering why we couldn’t use git or another scm tool to store stats ?!? 🤔
We should not store state file in SCM as it contains credentials. You can still store but it is not recommended
@@sagarmamidala5242 thanks for the feedback
Great video thanks. Do you have some recommendation on how do we test terraform code in CI/CD pipeline?
terratest ? Which runs your tf code against a empty cloud environment, then terratest interacts with the cloud, to validate whether the freshly provisioned environment meets expectation.
There are different alternatives to terratest, my understanding is terratest works best with Azure, as it provides better integration with AAD to access the provisioned resources for validation.
use terraform validate to test terraform code
Hi Nana, how do I upgrade argoCD version from 1.5.5 to 1.6? Looking forward to hearing from you
Would request a tutorial series or bootcamp on MLOPS or AIOPS?
It is perhaps obvious, but I think it's worth mentioning that auth tokens etc. should not be stored in variables file, but in terraform.tfvars. And this file should not be included in the source project.
DBAs at my work decided to remove 2 servers and keep number 3,4,5 only
This was a problem to do it in TF as we use count and count was 5.
If you changed count to 3 it would remove servers 4 and 5
So I had to amend the module we had and also change indexing inside the tfstate to make it work
Hello, Your explanation was simply superb and easy to understand as always. As mentioned in this video 4th best practice -> I am using GCS bucket and storing my terraform state file there after every terraform apply but unfortunately I have deleted GCS Bucket and now i have lost state file...Could you please let me know how can I create or get back the terraform state file. Any solution.
According to best practice 6, it is in Git, so according to best practice 8, let the CD put it back, once you have a new bucket. (I know this is theoretical now...)
Nice video. I'm surprised you didn't mention Terragrunt though.
thank you Nanaaaaaaaaaaaa Glory to you
What do you use to make these nice presentations?
Good but if you could make complete TERRAFORM video in which case studies to implement in different platforms of technical perspective
Hi Fahad, I have a complete course, in case you want to check it out. Check the video description :)
Danke!
Danke für die Unterstützung Roberto! 💙
Gern geschehen, Nana, deine Kurse sind wundervoll. Schöne Festtage! @@TechWorldwithNana
I'm currently using a tool called atlantis, that should be an alternative to env0
env0 supports atlantis workflow and has more features
Awesome!!! Thank you Nana
Woahhhh .! My org follows all of these…
Can you please upload the video on Terraform vs Terragrunt
Hi Nana, I am struggling to download the latest builded artifact from artifactory which is a war file basically, I have many artifacts on my artifactory repo and I need to download the latest builded one..can you is there any way to do it?unfortunately I’m not using the artifactory pro version which makes it more complicated to do.please help
Can anyone tell me how to get a software developer job as I am a fresher. And also learning the Devops bootcamp for further knowledge
No, when using Amazon S3 as a backend for state file storage, the default state locking mechanism does not automatically utilize DynamoDB, so your statement at 3:27 is not correct.
i don't get the best practise having a terraform state file remotely.
why not use git for that too in order to have "locking" or versioning and a pipeline to push that against a server?
i don't see why use git for terraform code, but not for state files.
maybe i am missing something
Nice video but I think you missed the point with GitOps and Terraform, in order to implement GitOps, a tool like Crossplane instead of Terraform must be used (or at least create your own terraform tool)
Hi Nana! why are these courses not available in Udemy Business? :(
4:40 you didn’t say how to have multiple state files. I recommend you do it with work spaces.
@TechWorld with Nana you missed security best practises like storing secrets and also terragrunt.
Wish to have an email connection ☺️
Also would like to know more about integrating CIs or env0 into PRs.
Nana, are all these tips integrated with the boot camp?
Hi Shailesh, yes they're integrated. You can check out the curriculum!
Please make full course
Nana has one here: www.udemy.com/course/complete-terraform-course-beginner-to-advanced/
Hey, could somebody please explain to me one thing? Why is it necessary to store `tfstate` file at all? An alternative: just query the provider about what infrastructure it has and use this response result instead of tfstate on the moment of code execution. It would require to implement a small number of additional API for providers, but it reduces so much complexity for the end users...
I can see only one possible reason - it is difficult to perform locking, when N people simultaneously execute `terraform apply` without state file. But here comes the punchline - since Terraform does not provide this feature out of the box, you still have to enforce mutual exclusion yourself, this reason must not be the actual reason why was it not done!
As I see it, querying state explicitly would solve all the complexity of maintaining this state shared between team members, which is a SIGNIFICANT complexity if you did not use static file storage in your project before. Literally, WHY???
Probably it's because of Terraform's approach, or because of some restrictions of targeted infrastructure, but solving parallel editing using locking is ridiculous, especially considering reinvention of the wheel; there is "code" in infrastructure as code, and nowadaysr the best solution to maintain the code is using source control.
Thank you Nana 😎 Since you are using Git, the development process must follow Gitflow, you must have branches for your dev, test and prod environments, and the state files must match those branches. This can be considered a best practice. But everything is learned in practice in a specific project.
How about secrets management?
Best
I don't understand point 4 (backing up/versioning terraform state). Isn't the terraform state just supposed to be tracking the actual state of the deployed resources? If that's the case, and it gets nuked, am I not in the same position as I was in before I ran "terraform apply" or "terraform plan" for the first time? Can't I just run "terraform apply" again? Seems to me it's an easily-recomputable resource - so why back it up? (But maybe I'm missing something?)
And as for versioning state... again, why? I don't see any use case for rewinding to older state (as you seem to suggest) - again (as I understand it) it's just supposed to be a reflection of the current actual deployed state, so rewinding would just make it inconsistent with reality. If you want to rewind your system state, that's what versioning your actual terraform code is for (and yes of course do that!): check out an old version of _that_ and plan/apply. The only purpose I can think of for versioning state is as a historical record for audit purposes (i.e. attesting that "this changed at this time").
But maybe I'm missing something...?
Genuinely asking, as I'm relatively new to terraform. Everything else you're saying makes sense to me, so if there's something I'm missing here I'd love to know about it. Thanks!
1. You created VPC -> subnet -> VMs.
How on the earth terraform will know if you want recreate subnet with terraform taint without 'state'?
2. You 120+ servers have names server-random suffix (module random string)
how terraform will know if 99th server should be replaced?
Another best practise is to use terraform modules, which will allow you to reuse your terraform across all your environments.
hi..nana ..you can make one video for learners...write terraform script and deploy through ci/cd pipeline...please my request...madam .
I am a beginner and I learn HTML CSS and I want to be a Back End Developer
( Node js or Spring BooT )
Please reply me and Thanks 🥰
Why not just keep your state file on GitHub? You'd get remote access, versioning, conflict resolution system, locking, support for different environments as well as reviewing, CI/CD out of the box.
Terraform state can contain passwords, secrets, and basically all the stuff you don't want to store anywhere in plaintext. That's why when using remote state it's best practice to actually use encrypted storage.
როგორც ყოველთვის მოკლედ და გასაგებად
What is the difference between a software engineer and programmer, even though they both write code
Respect from Pakistan 🇵🇰
Why didn't you mention in 1st best practice nothing about terraform import, state list, state rm ?
DRY with terragrunt, modules for reuse.
Hi Nana,
Good work.. Could you please upload reskill Spring and Spring boot videos. It will help the developer to refresh ourselves.
i'm not sure i understand about restoring previous versions of corrupted state files.
if the state files represent the cloud resources, then using a previous version would mean it will not be in sync with what actually exists in the cloud?
in regards to using your own TF repository, i have to say i object to that ,(even though it what we do) because in my opinion, tf code should reside in the SAME repo as the application code that uses it, this is because the commit should contain both the resource creation and the code that uses that resource.
assume a developer needs to establish an event bridge, SQS, SNS and s3 bucket resources, his code would need to address and use them, which would require creating them in advance, which would require devops to do this process, making the developer wait till they are finished, (same goes for changing said resources ) thus making devops a bottle neck
(yes i'm aware env0 is meant to solve this, but to me it seems kind of an over kill)
additionally let's say i'm using a new syntax in terraform (like using dynamic blocks) which may not always work, the constant iteration of a build server deploy would THAT bottle neck as it would start queues on these changes.
and lastly , i'm wondering, if i can combine localStack into this, buy somehow managing the code to be selectively executed on localstack container, (where i want only certain amount of resources created and override all endpoints to my own.
Probably you don't use ring deployment.
Probably you don't have multiple DCs and rules.
Probably you never faced the infrastructure changes not related to the code (mostly it has another lifecycle)..
Probably you never faced the migration - for example your code is not enough to handle 2x rpm and need to create second EventHub or emergency to recreate a SQS.
Probably you never faced the shut with a buil of the code when you can't release when it needed for infrastructure changes.
@@ValeriySolovyov shut with a buil?
@@emaayan autocorrection:)
The problems with Artifactory or dependencies(servers or repo shutdown)- when you cannot release a new version but you have an urgent infra changes.
@@ValeriySolovyov yes, but those could happen any way, i"m talking about a system the has it's tf source code and state committed in each repo that uses the code for it, but the process that launches and coordinates it is in one place using Tf remote to access the actual templates, otherwise, what will happen is that your own code will always fall out of sync with cloud resources. I have no idea how it's being managed these days
I got a Durex ad while watching this video. You know what that means nana
How do I automate the terraform for 1000+ accounts?