TouchDesigner Tutorial - Generate AI Images with Stable Diffusion using Image-to-Image Generation
Вставка
- Опубліковано 23 чер 2024
- Hey! In this tutorial, we'll go over how to use Stable Diffusion with a custom component I created to generate images in TouchDesigner. The project supports 2 forms of input using prompt generation and image to image so you can use any TOP in TouchDesigner as a starting point.
Project File: drive.google.com/file/d/1nJiO...
⚠️NOTE If you're on an M1 or M2 Mac⚠️ there's a bug in TD that causes the component to be stuck on Loading. The next release of TD will include a fix. In the meantime, there's a workaround on the form here: forum.derivative.ca/t/fixed-s...
Link to the API: computerender.com/
Link to my Patreon: / tblankensmith
Tutorial Part 2 on Audio Reactive Particle Systems with Stable Diffusion: • Generate AI Images wit...
Prompt Engineering with Lexica: lexica.art/
Opt out of Ai Training: haveibeentrained.com/
Huge thank you to Peter Whidden for his support on this and his work on computerender which makes this project possible
0:00 Overview and Examples
2:00 Setup The Computerender Component
2:36 Generating Single Images
4:04 Generate Multiple Images
6:18 Working with Iterations
8:05 Working with Guidance
9:23 Customize Resolution
10:19: Handling Errors
11:08 Auto Saving Images
12:18 Prompt Engineering with Lexica
13:09 Ethics of Copying Prompts
13:50 Generating More Image Examples
15:22 Using Any TOP as Input
Thank you for setting this up, it's really awesome. Fantastic work!
Thanks I was watching old TDNeuron videos all yesterday and this is pretty much what I was actually looking for so Thank you so much 😊
Awesome work - and very well explained!
thank you so much for setting this up! It is super helpful and great work!
This is incredible! thank you so much.
fantastic, thanks for sharing your work!
amazing , tx for that, it's been 6 month I was waiting for AI to hit touchdesigner with good explanations! I'd also be interested by the python side of things.
Amazing.. cannot wait until realtime video generation is possible
Is this not possible real time at the moment? I haven't tried it out yet. But it's so cool!
I would be curious to know this as well!
thank you sososososooooo much. you are a literal angel and saved my life and i swear i'll join your patreon when i'm not a broke college student 💗💗💗💗
EXCELENT! Bravo.
Wow, incredible work!
Thanks Stuart!
Hey friend! Great work!
Thanks Scottie!
Hey Torin, thanks for amazing tutorial. Is it possible to use other models inside the computerender module? And which version of SD we are using with compute render ? Again thank you thank you .
At 11:30, to make it write to the correct folder after selecting a location using the File Browser, edit the script in generate_file_name to add the slash between folder and filename
op('moviefileout1').par.file = parent().par.Imagefolderpath+ '/' + parent().par.Prompt...
dude is great! thank you so much
amazing work!!! thank u, soo cool :)))
amazing!. Is there a way to run the stable diffusion model locally without the API?
thanks, greetings from Mexico
Hi man! Thanks for sharing this.
I've a question: is it possible to connect the chop 'keyboardin1' into the 'button1'?
How?
This was an amazing video. I am trying to get serious about doing this type of stuff and I was wondering how you learned to code python specifically for touchdesigner. I am thinking of learning python but want to learn it in a way that is directly relevant to touchdesigner. Also how did you find computerrender are there other websites like it?
Hey Sade, thanks so much! Matthew Regan also has some great tutorials on using Python in TD: ua-cam.com/video/gjaDA1k5KG0/v-deo.html . Depending on your experience with python I'd actually recommend just learning the programming language outside of TD first and then diving into using it in TouchDesigner. Udemy has some great courses, as well as MIT OpenCourseWare and many others.
The main way I've learned how to integrate Python into TD projects is through their wiki page. My friend Peter Whidden developed computerender so I've picked up a lot of info from him. For finding this type of stuff there're many resources but can find a lot of new info on stable diffusion on reddit.com/r/stablediffusion. For new AI research, I really love Two Minute Papers: ua-cam.com/channels/bfYPyITQ-7l4upoX8nvctg.html.
@@blankensmithingReally appreciate the advice (: Love the work you are doing hope to continue following your youtube journey!
Hey Torin, thanks for the tutorial! It seems that when I try to open the .toe file on my TD, it says that this type of file is not supported. Do you have any idea why?
What version of TD are you running?
Amazing work!! Is there any way to generate audio through images using stable diffusion?
amazing!
Hi Torin, any chance to add more parameters to the compute render component ? im missing negative prompts and weights :)
Hey It's definitly something that Peter who made the Computerender API is looking into. I think negative weights are the next on his list. I'm about to add a strength parameter into the component. New tutorial out soon :)
hey if you check out the file on my newest tutorial there are negative prompts 😄
hey question! noticing sometimes when changing the prompt, the prompt will change the name of an already existing image in my directory but the output wont save.. not sure if that makes sense but have you encountered that and what's the resolve?
I created new folders
hey! thanks for your sharing! and I wanna to ask a question: if it is possible to use Arduino(which could read the data from outside) to control the prompt? thanks for your answering:😃)
Hey totally. It just depends on how you want that sensor data to control the prompt. I'd recommend using Firmata with Arduino to easily read sensor data from the Arduino. There's a Firmata component in the pallet of TD and you'd need to upload the Firmata sketch to the Arduino
@@blankensmithing 😃thanks for answering,but when I download the project, the computerender in TD just keep loading and nothing happened, my computer is dell and is based on Windows, is it because of my computer or I just missed something? (also I did not know where to download the macosx and images🥲)
Thank you for share this great project with us ! And I have a problem at first, which is i clicked and nothing happened, I checked Textport and DATs, it shows Failed to establish a new connection. Then I figured out this problem, by using Astrill VPN stealth mode. After that everything works properly:) This suggestion specially for China users.
😂请问 我用梯子去登录computerender 却没有弹出购买API服务的原因 也是因为墙吗？
@@riverrem738 api链接不用买 要买的是余额 你等免费次数用完了 估计就提醒你买了
您好，请问第一步添加了API KEY然后一直loading是因为vpn的问题吗？我不知道如何调整到您所说的Astrill VPN stealth mode，希望您可以分享更多的细节，我将不胜感激
This is great, thank you! I was playing around with a higher resolution, but it seems to cap out at 640x640. I don't see how to increase the output resolution past that, is there a way to go higher?
This happens when I do image->image and the resolution is higher than 640x640 on the input image.
img2img automatically downscales to 640x640, as it’s faster and results tend to get distorted at resolutions much higher than this
Heya Torin, thanks for this wonderful tutorial! I had it working wonderfully for a while on my M1 Macbook pro with and now it just loops the loading text indefinitely... I tried downloading a fresh copy, running it on your 2022.29850 version, regenerating the key and refreshing the folder path with the proper ./images/ format... and still all i get is loading... Any idea what could have gone wrong? I've got $10 in credit which should go a long way so its not that...
Hey did you happen to update your TD version? There's a bug in TD for M1 Mac where the SSL python library is missing and it looks for it on your system. The TD team fixed it in the upcoming build, but someone also find a temporary fix: forum.derivative.ca/t/fixed-ssl-python-module-missing-from-apple-silicon-build/308299
@@blankensmithing that did it! Thanks again!
Awasome tutorial~ I'm a freshman in Touchdesigner~Would you like to share how to make the "computerender component" ?
any way to implment this on local gpu with local stable diffusion install ?
Yes, you definitely can. If you have experience in python you can bring in external libraries. There's a good tutorial here: ua-cam.com/video/_U5gcTEsupE/v-deo.html and the model is available on Hugging face's website: huggingface.co/spaces/stabilityai/stable-diffusion. You can use a Script TOP to run python code and output the content to a TOP. My goal for this is that no one would need to install any libraries and I'm a huge fan of computerender.com since they're running their service essentially at the same cost as renting the GPU machines so more people can experiment with this technology.
@@blankensmithing sadly I have no python experience ;) I did install SD in a folder to run on a web ui and colab. I should have all the requirements but the scripts would not work as intended here I suppose
Awesome work, making all these custom components. I'm a total newbe with AI so forgive my stupid question but, when I've entered my API key and hit Pulse it just keeps staying stuck on Loading...
Is there something stupidly obvious I'm missing?
Any chance you're on an M1 Mac?
@@blankensmithing Yes I am, is that the problem?
@@erwintuijl6342 There's a bug I submitted to TD that's going to be fixed in the next release. There's a workaround someone found though. forum.derivative.ca/t/fixed-ssl-python-module-missing-from-apple-silicon-build/308299
@@blankensmithing You rock, sir. Thanks for the help, I'm gonna check out that workaround.
Have problems with the m1 processor, the computer render loads all the time. Does anyone have a similar problem and can help?
Hey Torin,
First of all, thank you very much for the video! It is so nice of you to share your knowledge with us.
I have followed your first steps and generated an API Key + copy/paste on TD and clicked on pulse but nothing happened. It is just loading and on my Computerender account it states that no invocations took place. Can you help me fix the problem?
What version of TD are you on? Make sure you're on the newest
@@blankensmithing downloaded new version, is working now! :)
Hey Torin! Thank you so much for the great tutorial and for sharing this TD project file with us! I have a weird problem and maybe you know the answer. I was playing around with different keywords to generate some images and everything worked just fine. Suddenly it all stopped working. There was an error coming up that says "{"status":"error","message":"User has no credit","errorType":"INVALID_AUTH"}". I thought it might have something to do with my API key but it's still in place. I even opened your original file new to see if it has something to do with the changes I made but it still doesn't work. Can you help me please?
Hey yeah, you get $0.25 of credit on the api to start with. You can add more credit on the account page of computerender
@@blankensmithing oh okay, thanks for the fast reply. how long does like $1 last then?
@@kalugrafik 4x the amount of time it took you to use the starting funds. i’m a huge fan of using computerender because it’s been reliable and also the lowest cost api
@@blankensmithing I see, it's definitely worth the money. Thanks for answering!
Hello! I have a question. Where can I find a tutorial to make a light disperse like in your opening title?
Hey! It's using a feedback network. Noto has a great tutorial on it: ua-cam.com/video/w4DXtVWdlYg/v-deo.html
I think I added in some of the RGB delay component from the pallette as well
@@blankensmithing thanks a lot for answer!
Will there be no charge?
Wow
Hi Torin! How could it be done in real time by connecting the Webcam?
hey you can use img to img with the webcam as input. it wont be fully real-time. also i’d recommend working off the project file from my more recent video on making an audio reactive animation. There's an extra parameter on the component called strength that will give you better results
hi torin! thanks for sharing!
my TD computerender op stays in "Loading...." scrolling, and no result happen,. im on a mac m1 laptop... any idea on what the problem may be?
thnks!
i also created the api key...
What version of TD are you on? You should update to the latest
@@blankensmithing That was it, didnt knew there was a new version. thnx!
How do I import this computer render into TouchDesigner. I am trying to load the .toe file into my application and nothing is showing up
.toe files are stand alone projects. I have an example network inside it. You can copy the component over to your other project once you have it open
6:50 iterations 머신 러닝 모델 반복
8:25 guidance 프롬프트에 얼마나 밀접하냐
Hey man! Thanks for sharing! I have a question..so I added my own API Key and tried to generate an image but it loads and nothing happens. My Internet connection is very good. I use a macbook pro (M1) Thank you! Bests! Alex
Hey! Yes, there was one other person on an M1 that ran into this. What version of TD are you using? Also just want to double-check your TD timeline is in play mode. As a temporary solution, running the latest Intel version of TD seemed to fix it which is strange because I assumed that version wouldn't run on an M1. I tried this on a friend's M1 on the latest silicon version of TD and it worked totally fine. I should have access to an M1 mac soon and can try testing on my end
@@blankensmithing I'm encountering the same issue on an M1. Dropped some funds into computerender but that didn't solve the issue. No errors reported, no invocations in computerender, and just the loading screen appears. TD version 2022.28040, currently in play mode. Digging into the code to see if I can spot something. Really cool use of the tech!
Traceback (most recent call last):
File "/project1/computerender1/run_sd", line 12, in post
File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "/project1/computerender1/run_sd", line 12, in
File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.computerender.com', port=443): Max retries exceeded with url: /generate/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."))
@@brentmarshall Ok, I tracked it down!! It looks like the ssl python library isn't included in the apple silicon build of TouchDesigner. I'm not sure if it's deliberate. I submitted a bug request to TD, we'll see what they say: forum.derivative.ca/t/ssl-python-module-missing-from-apple-silicon-build/308299
@@blankensmithing Hey man! Thanks for your help! I have no experience in python so I am just staying thankful!😅 I hope you enjoy a wonderful Christmas! 🎄
hey Torin, thanks for sharing with us :)
i have a problem, when i open the file in TD 2022.24200 (windows version) and put my key on the computerender Comp and generate image, nothing happened. i just see the loading on Comp. can you help me to fix that?
Hey! Could you open up the Text Port & Dats Preview(drive.google.com/file/d/16Ha12t8MDptR55FkYRjwCt5ugbPQc3HG/view?usp=sharing) after you run generate and let me know what shows up in the logs there?
@@blankensmithing Hey! it's the result
drive.google.com/file/d/10A9PWwgzGqTPDf-in_pAJQFU1GTTX57V/view?usp=share_link
@@detailsstudio Could you scroll down on the text port and also share the log?
@@detailsstudio I've tested on the latest version of TD using windows. Could you also try updating your TD version?
@@blankensmithing Hi, I'm sharing this because I'm probably encountering the same error.
The environment is Windows 11,Touchdesigner 2022.26590.
(This is a newer version than the project file here)
File "/ui/dialogs/dialog_error/list/list/script1_callbacks", line 24, in cook
TypeError: unsupported operand type(s) for /: 'str' and 'str'
error
So, this is not a Project1 error maybe.
And I set "int(availSize)/int(textSize)" to make "availSize/textSize" valid, but the image is still not generated.
There are no other errors, and it seems the API call has never been made yet.(my StableDiffusion account is newly created today.)
I will continue to investigate the cause.
Thanks for sharing this great project!
Attached image (drive.google.com/file/d/18ooX9MPNkIoEvzqXP6HAWrEvbQP8Fnyg/view?usp=share_link)
It can work according to the steps before, but after 1 month, there are words that the user has no credit on the computerrender, do you have a solution?
Hey, the computerender API costs money to run. You need to log back in and add credit. You started off with $0.25. at the moment it's the least expensive stable diffusion API I've found
Is it possible to connect TouchDesigner with local StableDiffusion build?
yeah, you need to have the proper GPU to run it though. Lyell Hintz (dotsimulate) made a component and Bileam has tutorial here: ua-cam.com/video/4wpn_3JNaIc/v-deo.html
When I use the project, the computerender node has a big fat red X and says error on line 5 of script2_callbacks. How do I fix this?
what version of TouchDesigner are you using?
@@blankensmithing The most recent. I switched to your version and it works so ye
I think I love you
I have this error: (Parameter : Experimental) AtrributeError: ´td.App´object has no atrribute ´experimental´. Is it because of my version of TD? 2021.16419
Yeah you should update to the latest version of TD
@@blankensmithing Thanks!
hi there just came across this, really nice!!:) I am trying to configure an API key in computer render, any chance anyone knows how to do that now? ( I can only use the request builder but not actually get an api key)...
Go to the Account page. There's a form where you can enter an App Name. Once you add the name it'll generate an API key for you
hei hei , what about of an integration of invoke ai!! ?? anyway thanks same code can be used for other API with little changes!! BRAVO BRAVO !!!
woooooow
anyone getting this error? happened after i tried to modify the iterations and guidance. gets stuck on loading then returns the following error. using latest build on windows computer
{"status":"error","message":"Could not find \"complete\" state within timeout. You may try again.","errorType":"INTERNAL_ERROR","jobId":"a3ec249b954ceb9c114d1df71c6c21240f16b4d4"}
the server running compute render may have been down. Try running it again
@@blankensmithing i think that was it! running smoothly today :) thanks btw for the work you've put into this
🐯
New discovery: Chinese friends can try to close your vpn after clicking pulse without response, I was able to use it normally after trying this.（That's a little bit wired cause you always should using vpn can get to the websites like chatgpt...
or use phone, it works for me
👋
Hey,Torin.This video is very very good, but after I modified the keywords in TD, it kept showing loading and nothing happened. My version number is 2022.32660. What should I do?