TensorRT is more complicated than what I have covered. I have restricted the explain in relevancy to Comfy UI and Stable Diffusion. LoRA works, ControlNet support is not added yet. Hope the video is helpful.
That won’t help. Training the tensor takes like only 3 to 5 minutes. Whatever figures I show is highly dependant on my system. I have shown some benchmarks in the video. As far as I know for turbo the gain is least. I suggest you just train the engine and throw it in your workflow to test. Tensor RT has zero effect on quality. If you optimize a turbo model, the quality is based on that model. TensorRT only impacts the performance.
@@adisatrio3871 I have not tried with LCM. Just tried with turbo. I doubt it would make any difference. Since only with turbo gets me 14 to 15 percent faster on average, on my 4090. Maybe lower cards have more performance boost.
I don't often leave comments but I had to for this video! I came in just wanting to learn about Tensor RT(which I did) but I also came out with a killer All-In-One workflow! Seriously, this is great, thank you!
context is the multiplier for 75 token size, setting the max to 128 will slow down tensorRT a lot and increase vram, for average use case it should be set to around 2-4 (about several dozen tags worth)
You can put static batch size to whatever you want. If you put a number you can use the engine only with that batch size. Typically 1, 2 or 4 as per your requirement
Error occurred when executing TensorRTLoader: 'NoneType' object has no attribute 'create_execution_context', only on SD3 workflow, when try to load the tensorrt... any suggestion? Thanks!!
Are you using the sd3 medium vae? Also have you trained the engine on your system or using the shared engine. The shared engine file will work only on a 4090 system.
Hi. I need a trick to use higher gpu Vram. For example i have 2 PC and want to render with both. Do you know a solution for that? There is a node called NetDist, but its very bad documented i am lost. Maybe you are smarter than me and could help somehow ?
can you generate a picture and upload it to a folder that we can download from? this would allow us to use the workflow easily without having to build it all manually.
Sorry, ready workflows are only available for paid UA-cam channel members. However I don’t hide anything in the UA-cam video tutorial. It just something extra I give to who support that channel monetarily (over and above the standard views, likes support).
No its not practical for animate diff at this moment. ControlNet doesn't work, it would be highly restrictive. You can train the main checkpoint with your resolution. But the workflow would be extremely limiting seeing how ControlNet is very much used with animate diff.
"ComfyUI TensorRT engines are not yet compatible with ControlNets or LoRAs. Compatibility will be enabled in a future update." Not interesting for the moment
TensorRT is more complicated than what I have covered. I have restricted the explain in relevancy to Comfy UI and Stable Diffusion. LoRA works, ControlNet support is not added yet. Hope the video is helpful.
could you make a video to compare between, Turbo, LCM, RT, FreeU, and which combination are compatible and the best in terms of speed and quality?
That won’t help. Training the tensor takes like only 3 to 5 minutes. Whatever figures I show is highly dependant on my system. I have shown some benchmarks in the video. As far as I know for turbo the gain is least. I suggest you just train the engine and throw it in your workflow to test. Tensor RT has zero effect on quality. If you optimize a turbo model, the quality is based on that model. TensorRT only impacts the performance.
@@controlaltai so is it possible to get way more better performance combining, turbo model + LCM + RT?
@@adisatrio3871 I have not tried with LCM. Just tried with turbo. I doubt it would make any difference. Since only with turbo gets me 14 to 15 percent faster on average, on my 4090. Maybe lower cards have more performance boost.
Happy to be a member... Thank you for all your work!!
I don't often leave comments but I had to for this video! I came in just wanting to learn about Tensor RT(which I did) but I also came out with a killer All-In-One workflow! Seriously, this is great, thank you!
context is the multiplier for 75 token size, setting the max to 128 will slow down tensorRT a lot and increase vram, for average use case it should be set to around 2-4 (about several dozen tags worth)
I am always grateful for the best lectures. I may have missed something, but the explanation about Static batch size is missing.
You can put static batch size to whatever you want. If you put a number you can use the engine only with that batch size. Typically 1, 2 or 4 as per your requirement
@@controlaltai thank you!!!!!
Error occurred when executing TensorRTLoader:
'NoneType' object has no attribute 'create_execution_context', only on SD3 workflow, when try to load the tensorrt... any suggestion? Thanks!!
Are you using the sd3 shared workflow or have you created one?
@@controlaltai shared
Are you using the sd3 medium vae? Also have you trained the engine on your system or using the shared engine. The shared engine file will work only on a 4090 system.
Hi. I need a trick to use higher gpu Vram. For example i have 2 PC and want to render with both. Do you know a solution for that? There is a node called NetDist, but its very bad documented i am lost. Maybe you are smarter than me and could help somehow ?
can you generate a picture and upload it to a folder that we can download from? this would allow us to use the workflow easily without having to build it all manually.
Sorry, ready workflows are only available for paid UA-cam channel members. However I don’t hide anything in the UA-cam video tutorial. It just something extra I give to who support that channel monetarily (over and above the standard views, likes support).
Hello sir. Were you able to make it work with cosxl models?
Hi. Doesn't work. Not compatible for the moment. Tried and tested.
@@controlaltai Sounds good. Thank you for your answer. Been following you for a little while now, and your videos are gold sir.
Thanks
Does tensor rt work with animatediff already by any chance?
Not tested. Will test and let you know.
@@controlaltaitested?
Sorry could not, give me a day or two will get back.
No its not practical for animate diff at this moment. ControlNet doesn't work, it would be highly restrictive. You can train the main checkpoint with your resolution. But the workflow would be extremely limiting seeing how ControlNet is very much used with animate diff.
"ComfyUI TensorRT engines are not yet compatible with ControlNets or LoRAs. Compatibility will be enabled in a future update."
Not interesting for the moment
Turbo and sd3 are closed licensed. Wgaf about them except hobbyists? And sdxl lightning is open license but you ignored it. Very disappointing.
Lightning is not supported. Already mentioned what models are supported.