I appreciate a lot the focus you put on the importance of personal life in this video :) Imagine thinking like this all the time - I make all these fancy advanced K8s things so I can spend more time with my family and friends.
Hey Victor , Can you explain the difference between crossplane crd vs upbound crd , Is upbound using terraform under the hood for creating resouce (what is terrajet , upjet )
There are two main components of Crossplane. One are providers that, when installed, extend Kubernetes with CRDs and controllers that manage resources somewhere. That could be AWS, Azure, GCP, GitHub, or anything else. Those providers can be built 1) from scratch, 2) by using Terrajet which is mostly code generation tool that converts Terraform modules into code as a Crossplane provider, or 3) can be built on top of Terraform, or on top of Pulumi, or Ansible, etc. As long as the APIs of those providers comply with Crossplane "standards", it does not matter. The second, and much more important part of Crossplane are Compositions. That's how you create your own CRDs and controllers that compose the resources you need and expose new APIs. That is, in my opinion, the main advantage of Crossplane, and that's what DevEx explained in that video focuses on. Those Compositions might or might not even use the providers I mentioned above.
Thanks for another great video ! Ok, it looks like now we're covered from the "platform engineer" or "service owner" perspective. I would say that even before the Status Transformer it was still possible to get some validation done via Kyverno for the "developer" perspective, when users input invalid values. Definitively a big step forward in the right direction! Now I still need to get the validation of user Claim's like SQLClaim before they even reach the cluster covered 😉
That's a tricky one... On one hand, Kyverno is the best option to prevent "wrong" things entering the cluster while the status transformer is there to propagate information (often errors) back to users. However, if Kyverno "prevents" something from entering the cluster, then users would never get the feedback without entering into low-level details. If a claim is synced with Argo CD (or Flux), there would be nothing but a Kyverno custom resource with the details why it rejected the claim. My assumption is that looking at that resource as-is is not really "developer-friently". Still, my example with the incorrect region was not the best one, especially since, in that video, I did not want to enter into "what is Kyverno and what is Status Transformer domain" type of a debate. I should have used a better example of an issue that cannot (shouldn't) be picked up by Kyverno.
@@DevOpsToolkit I didn't mean to enter the kyverno Vs status transformer domain either .I just wanted to say that from the user input perspective there was something we could do to prevent user errors in the first place. But as I said I really think this status transformer is a huge step in the right direction whenever you get errors and they should be reported at the right level of abstraction. Still, as you said, the tricky problem is with validation of the Claims before they even reach the cluster or Argo, ideally something to catch early in the pipelines. That's why I'm waiting for your video on the subject 😉💪
@Victor can you please show how to have crossplane in backstage :) and if possible in Port as well, we are still busy comparing both IDP. And if you had to chose between them which one would you do?
Port is capable of "discovering" Kubernetes schemas (CRDs) so it works very well with any resources applied to Kubernetes, including Crossplane. You can see an example of the "discovery" at ua-cam.com/video/PV1sBiC85Yc/v-deo.html. Backstage cannot figure out what's in Kubernetes (as far as I know). There is a Kubernetes plugin that shows only a few basic resources (e.g., Deployment, Service, Pod, and one or two more) and it does that in an awful way. As far as I know, you'd need to build your own plugin and a Kubernetes controller to accompany it. Between the two, my choice is, without doubt, Port. The only advantage Backstage has is that it is open source (free). However, that comes at a potentially very steep cost in development and maintenance.
I understand why you would want to have backstage or port in cloud or on-prem. I'm not sure why you would have one of those in both. In any case... If you need it on-prem, Backstage is the one. Port, as far as I know, cannot be self managed. You can have an agent feeding it data anywhere but port itself Is saas-only.
@@DevOpsToolkit Sorry, I did not mend the IDP be on-prem but rather the IDP manage and create resources on-prem and in the cloud. :) we still in the cloud migration journey. I am going to wait for your video where you combine Port and Backstage as that might be what I need
So we can call this "a try-catch mechanism for deployment"? So all of the best practices of try-catch from the world of developers can be applied here as well.
It'd be quite nice if Crossplane had some better CLI tools to assist in debugging XRs. Komoplane is super useful, but seems quite unfinished. Just some feedback from a mostly happy user.
@@DevOpsToolkit understood, but it can emit events too, and the docs tells correctly the limitations. Anyway, about the Status and Respective Conditions, is there any good resource for best Practice how to use conditions, If I look on the operator landscape we see that everyone is using them differently. Is there any opinionated crossplane way how to structure conditions most valuable? From what I see, there can be at least many Conditions per resource.
Are you aware of businesses that actually successfully implemented a day 2 operations w/ Kubernetes and Crossplane - at scale, i.e. with >50 software development teams using a large number of centrally provided (abstracted) resources? As you mentioned, Day 1 is much more easy, but also enables dev teams only so much.
Not many managed to do that and those who did rely mostly on their own custom code. Existing projects tend to focus on the easy part (day 1 operations). Crossplane is, in my opinion, the one that is closest to that goal.
A K8s Admin AI Agent can do all what you need to be done, integrating that agent with ticketing systems, resource reservation systems etc would offer seamless & smooth developer relations.
I appreciate a lot the focus you put on the importance of personal life in this video :) Imagine thinking like this all the time - I make all these fancy advanced K8s things so I can spend more time with my family and friends.
Hey Victor , Can you explain the difference between crossplane crd vs upbound crd , Is upbound using terraform under the hood for creating resouce (what is terrajet , upjet )
There are two main components of Crossplane. One are providers that, when installed, extend Kubernetes with CRDs and controllers that manage resources somewhere. That could be AWS, Azure, GCP, GitHub, or anything else. Those providers can be built 1) from scratch, 2) by using Terrajet which is mostly code generation tool that converts Terraform modules into code as a Crossplane provider, or 3) can be built on top of Terraform, or on top of Pulumi, or Ansible, etc. As long as the APIs of those providers comply with Crossplane "standards", it does not matter.
The second, and much more important part of Crossplane are Compositions. That's how you create your own CRDs and controllers that compose the resources you need and expose new APIs. That is, in my opinion, the main advantage of Crossplane, and that's what DevEx explained in that video focuses on. Those Compositions might or might not even use the providers I mentioned above.
Thanks for another great video ! Ok, it looks like now we're covered from the "platform engineer" or "service owner" perspective. I would say that even before the Status Transformer it was still possible to get some validation done via Kyverno for the "developer" perspective, when users input invalid values. Definitively a big step forward in the right direction! Now I still need to get the validation of user Claim's like SQLClaim before they even reach the cluster covered 😉
That's a tricky one...
On one hand, Kyverno is the best option to prevent "wrong" things entering the cluster while the status transformer is there to propagate information (often errors) back to users.
However, if Kyverno "prevents" something from entering the cluster, then users would never get the feedback without entering into low-level details. If a claim is synced with Argo CD (or Flux), there would be nothing but a Kyverno custom resource with the details why it rejected the claim. My assumption is that looking at that resource as-is is not really "developer-friently".
Still, my example with the incorrect region was not the best one, especially since, in that video, I did not want to enter into "what is Kyverno and what is Status Transformer domain" type of a debate. I should have used a better example of an issue that cannot (shouldn't) be picked up by Kyverno.
@@DevOpsToolkit I didn't mean to enter the kyverno Vs status transformer domain either .I just wanted to say that from the user input perspective there was something we could do to prevent user errors in the first place. But as I said I really think this status transformer is a huge step in the right direction whenever you get errors and they should be reported at the right level of abstraction. Still, as you said, the tricky problem is with validation of the Claims before they even reach the cluster or Argo, ideally something to catch early in the pipelines. That's why I'm waiting for your video on the subject 😉💪
@Victor can you please show how to have crossplane in backstage :) and if possible in Port as well, we are still busy comparing both IDP. And if you had to chose between them which one would you do?
Port is capable of "discovering" Kubernetes schemas (CRDs) so it works very well with any resources applied to Kubernetes, including Crossplane. You can see an example of the "discovery" at ua-cam.com/video/PV1sBiC85Yc/v-deo.html.
Backstage cannot figure out what's in Kubernetes (as far as I know). There is a Kubernetes plugin that shows only a few basic resources (e.g., Deployment, Service, Pod, and one or two more) and it does that in an awful way. As far as I know, you'd need to build your own plugin and a Kubernetes controller to accompany it.
Between the two, my choice is, without doubt, Port. The only advantage Backstage has is that it is open source (free). However, that comes at a potentially very steep cost in development and maintenance.
As a side note, I will be working on a material that combines (or tries to combine) backstage and port. I can't promise the release date just yet.
@@DevOpsToolkit Thanks for the info, if you have to consider to still manage both on premise and cloud would you still make the same decision ?
I understand why you would want to have backstage or port in cloud or on-prem. I'm not sure why you would have one of those in both.
In any case... If you need it on-prem, Backstage is the one. Port, as far as I know, cannot be self managed. You can have an agent feeding it data anywhere but port itself Is saas-only.
@@DevOpsToolkit Sorry, I did not mend the IDP be on-prem but rather the IDP manage and create resources on-prem and in the cloud. :) we still in the cloud migration journey.
I am going to wait for your video where you combine Port and Backstage as that might be what I need
So we can call this "a try-catch mechanism for deployment"? So all of the best practices of try-catch from the world of developers can be applied here as well.
It's a bit more than that. You could define it as try, catch, propagate up if relevant.
any plans to create more detailed crossplane course on udemy or others educational portals?
More detailed than Crossplane Tutorial
ua-cam.com/play/PLyicRj904Z99i8U5JaNW5X3AyBvfQz-16.html?
It'd be quite nice if Crossplane had some better CLI tools to assist in debugging XRs. Komoplane is super useful, but seems quite unfinished. Just some feedback from a mostly happy user.
That's coming very very soon...
Thanks for the heaadsup... now we might have the event throttling, but thats another story
Those are statuses rather than events. Still, you're right. The more we do the more strain we are putting on kubernetes.
@@DevOpsToolkit understood, but it can emit events too, and the docs tells correctly the limitations. Anyway, about the Status and Respective Conditions, is there any good resource for best Practice how to use conditions, If I look on the operator landscape we see that everyone is using them differently. Is there any opinionated crossplane way how to structure conditions most valuable? From what I see, there can be at least many Conditions per resource.
@holgerwinkelmann6219 I haven't seen an best practice document 😟
I want to see this in backstage!
Are you aware of businesses that actually successfully implemented a day 2 operations w/ Kubernetes and Crossplane - at scale, i.e. with >50 software development teams using a large number of centrally provided (abstracted) resources? As you mentioned, Day 1 is much more easy, but also enables dev teams only so much.
Not many managed to do that and those who did rely mostly on their own custom code. Existing projects tend to focus on the easy part (day 1 operations). Crossplane is, in my opinion, the one that is closest to that goal.
A K8s Admin AI Agent can do all what you need to be done, integrating that agent with ticketing systems, resource reservation systems etc would offer seamless & smooth developer relations.
🎉
Cry at 5:10