If you're interested in something like DALL-E 2 and what it could mean for the future of creativity, last summer, I released a YT series entirely on this topic. It's called GPT-X , DALL-E, and our Multimodal Future it's available right now for free on youtube: ua-cam.com/play/PLza3gaByGSXjUCtIuv2x9fwkx3K_3CDmw.html
The issue of Dall-E generating consistent styles for concept art could easily be resolved by (professional) artists using inpainting of their existing works. They could probably use it to generate new alterations or variations of their works. At a faster & probably more time efficient rate. Very exciting stuff for small businesses!
I'd really like to use dalle 2 to generate images on the fly in my d&d campaign or just see it used in storytelling. Like imagine taking a really descriptive book and creating the scenario for every description and having it integrated in audiobooks
That's an interesting thought. It just occurred to me that if we go far enough in the future with this technology and other AI related technologies like human-like voices etc.. there could eventually be a technology similar to captions but in reverse. You could open up your e-book, hit a button, and see the text on the page turn into images, and the book could be narrated by AI, with unique voices for every character. In theory it's already possible at this current stage, except that it would have to be created manually section of text by section - but I could see it being a fully automated rapid process in future in the same way that auto-CC is now.
D&D would be SUCH a great use-case for this technology! I didn't even think about that. Only problem I can foresee is that there aren't as many general fantasy reference images to train the AI on as there are just simple photos of things like people/places/animals/etc. Hopefully the training data set for Dall-E 3 is even more massive, and includes more things from more different genres!
The cars generated from Blade Runner image maybe even better. You just gave me an idea, you could try different car designs based on the Cybertruck/Delorean.
Lovely video, fun to see how you utilized DALL-E 2 AI in actual application to create a story through a series of concept arts. This is a huge game changer and can help artists to further push their creativity during the ideation process. I can already see how it can be of great application in my creative process :)
Isn’t it a bit redundant doing the images is say a concept art style? As in reality the styles exist because they are fast, whereas dalle takes the same amount of time to do any style
If the AI only did what it did with when you give it an image I would be fine with it....it's the fact you can go "Give me ___ and ____ in the style of Studio Ghibli" and it just rips all the Ghibli art pallets and such and applies them over other mashed photos and such and makes it happen. I dunno how I feel about that. Like it's not copyright infringement by any means, but it's not...honest...either.
Would it also be possible to get a drawing by lesser known draftsmen? Like a mural by Kim Jung Gi, a caricature by Jack Davis, an anatomy study by Frank Frazetta etc. It would e so cool to study art and anatomy from drawings by Ai, for example you want to draw a character like Kim Jung Gi but you're not sure what it looks like under a certain angle etc
The variants of an already finished piece is probably the most useful thing here for concept artists, however even then Clients are very specific with changes. I'm not sure how useful this would be with the pseudo random variants it spits out. Will be interesting to see how it's put to use if allowed to the public.
Great video! Just as a heads up, DALL-E is pronounced with an 'aah' sound (Open back unrounded vowel) as in "ball" and "tall". The name is a blend of WALL-E and Salvador Dalí, which both use that vowel sound.
The idea of generating from reference images is extraordinary. It adds serendipity in a consistent style otherwise missing. Did you use GPT-3 to co-write this story?
How much RAM is needed for this. And how long did this take? Thank you. I'm on the waitlist for Dalle2. I have a laptop with 4 gigs of ram. Presently using Google Colab with interesting but unpredictable and irrelevant outcomes. Too many results have a Shutterstock logo watermarked which I guess will change in time. And it takes quite a while - hours if you tweak stuff. This is exciting but frustrating as I'm just starting out and I'm old. I can see this helping my online home decor business if and when I get it figured out how to print my AI designs. I already sell some concept art which is fun to create but hellish time-consuming. I'll watch your YT series.
Based on what you mentioned on a podcast a while back (preparing for DALL-E) would like to know how your advice changes (if at all) now that you’re hands-on. I imagine the speed (and getting 10x outputs each time) makes it a challenge to keep track of what you’ve attempted. Did you manage to remain systematic, or just jump into sandbox-mode right away? Love the way you’re going through it based on use-cases. Such unlimited possibilities can be exciting, but also potentially paralyzing.
You're right on the money with this one Josh. I'm going to provide updated tips on DALL-E 2 usage hopefully in the next podcast episode. The iteration rate is far greater. Also, I agree with you, so many directions and ways you can go with things. This is also why I like to take notes, we can isolate things out that way at least mentally a bit.
Can you import an image while adding a modified text description, or a text description of itself? If so, would they turn out more homogenous given the right prompt? Example: Same Bladerunner shot, but the man's POV" or "the exact same image but a wider shot" Also, does it work with more detailed positional prompts? "Girl standing on the left third, pyramid taking up the top right third, water-bottle in the bottom middle foreground"
Great questions. At the moment (again I'm in the research beta) you can't import simultaneously with a prompt. You upload and then either edit or generate variations. To answer your other question, I haven't had any time to test the positional prompt stuff but it's an interesting area I've wanted to check since the beginning.
Could you do a video about Aleph Alpha's luminous? I mean DALL-E and GPT-3 are certainly very fascinating but I believe they get so much exposure compared to other great models out there. From what I've seen their model is performing really well but I'm not too experienced so I'd like to hear your opinion! Also: I love your videos!
AI dangeoun for story + art by DALL-E 2 + GPT 3 as NPCs + ultra realistic VR headset I know it's not that easy to put everyone together but let me dream ok lol
Using Dall-e's Inpainting tool, is it possible to replace parts of a character? for example. load in a character concept art, then replace the head, arms, body, etc untill you have a new character design?
@@bakztfuture Awesome. there are so many things I want to try with Dall-E 2. I'm a concept artist, and i'm always trying to figure out ways of speeding up my workflow. I'm on the waitlist, but in the meantime, I'd love to see the results of your experimentation. even failed attempts will be useful for seeing the limitations. These are exciting times.
@@hamidmohamadzade1920 It can generate variations on a single image, which is a sort of brainstorming; but for the kind of brainstorming where you come up with many distinct ideas, I think combining it with GPT-3 would achieve that result better than trying to get DALL-E to do that on its own. GPT-3 to generate the different ideas, DALL-E 2 to generate them as images and produce variations on each idea.
Rendering in extremely high fidelity and doing it quickly, even with initial concepts and iterations, is extremely common in concept design. Not saying this tool can't be useful in v creative ways along the process, but I get the very palpable sense that a lot of these Dall-E videos are done by people that are shockingly ignorant about common artistic practices in the commercial art community, including concept artists. Like, the potential isn't really being explored. It's just a bunch of assumptions about how real commercial artists work. Again, this tool is revolutionary. But devs seem to miss the whole freaking point of their own tools.
@@bakztfuture I know u were 😌. Hence my response. That is what 'extremely high fidelity' means. That's not even the real issue tho. Although the capacity for photorealism already exists at the initial stages of concept design, the reason it isn't pursued has to do with foundational principles about design. For example, in architecture (my field) first-year students love to render their initial concepts with high fidelity, and we always have to move them away from this strategy because it isn't conducive to studying formal properties that are infinitely more important than photorealistic representation. The use of non-photorealistic representation allows us to study volume, massing strategies, circulation patterns, or all the relevant spatial strategies that matter way more than end-result representation. Photorealistic representation (at this stage) gets in the way of all this for many reasons, not the least of which is that it generates so much visual noise. Reducing or abstracting initial concepts produces better designs down the line. Again, this is a foundational practice in the theory of design and similar analogues can be drawn from concept design. It seems to me that devs are so arrogant that they cannot even fathom their ignorance about fields they do not work in. Sorry to be harsh. But this pattern persists. Again, this tech is amazing. And I can envision so many interesting use-cases. But hardly anyone in the dev community has spoken about them lol. It speaks to the greater need of cross-disciplinary strategies going forward in Ai. And btw, I think u can make the same criticism about most designers and our general ignorance about CS.
I really feel sorry for digital artists out there 😔 Lets be real guys making these mind blowing arts so easy with ai u don't even need any knowledge about art or ai.
Calm down buddy. I doubt he wants your money lol. No, Dall-E 2 cannot do any of that. Future programs will probably have this functionality, but I'd imagine we're at least 5-10 years out from that. Dall-E 2 generates an image based on a text prompt. It does not understand anything like real-world measurements (although it probably has some grasp of things like relative scale), let alone something as specific as all the components in a proper CAD file. That would require an AI that could convert an image into a 3D representation, along with all the data that makes it usable as a CAD file, and while we're entering into that field with things like Nvidia NeRF, we're still quite a away from the fidelity needed for something so precise. TL;DR, no, but check back every couple years.
If you're interested in something like DALL-E 2 and what it could mean for the future of creativity, last summer, I released a YT series entirely on this topic. It's called GPT-X , DALL-E, and our Multimodal Future it's available right now for free on youtube:
ua-cam.com/play/PLza3gaByGSXjUCtIuv2x9fwkx3K_3CDmw.html
The issue of Dall-E generating consistent styles for concept art could easily be resolved by (professional) artists using inpainting of their existing works. They could probably use it to generate new alterations or variations of their works. At a faster & probably more time efficient rate. Very exciting stuff for small businesses!
Yes!!! But a systematic level design system tool would be nice too
I'd really like to use dalle 2 to generate images on the fly in my d&d campaign or just see it used in storytelling.
Like imagine taking a really descriptive book and creating the scenario for every description and having it integrated in audiobooks
Do you have aphantasia? No problem! We supply the imagination!
That's an interesting thought. It just occurred to me that if we go far enough in the future with this technology and other AI related technologies like human-like voices etc.. there could eventually be a technology similar to captions but in reverse. You could open up your e-book, hit a button, and see the text on the page turn into images, and the book could be narrated by AI, with unique voices for every character.
In theory it's already possible at this current stage, except that it would have to be created manually section of text by section - but I could see it being a fully automated rapid process in future in the same way that auto-CC is now.
D&D would be SUCH a great use-case for this technology! I didn't even think about that. Only problem I can foresee is that there aren't as many general fantasy reference images to train the AI on as there are just simple photos of things like people/places/animals/etc.
Hopefully the training data set for Dall-E 3 is even more massive, and includes more things from more different genres!
love the fact that ur a tech guy who also loves fine art! what an amazing product from openai!
Thanks so much, really appreciate you saying this!
The cars generated from Blade Runner image maybe even better. You just gave me an idea, you could try different car designs based on the Cybertruck/Delorean.
Cool idea, I was going to do something similar
@@bakztfuture You would be the first.
Lovely video, fun to see how you utilized DALL-E 2 AI in actual application to create a story through a series of concept arts. This is a huge game changer and can help artists to further push their creativity during the ideation process. I can already see how it can be of great application in my creative process :)
Maybe a couple of fairly specific elements joined by one or more generalities when making a prompt would be a method worth looking into?
Ooh I hadn't considered that this could be used to illustrate short stories real quick! Another use case for me :D
Isn’t it a bit redundant doing the images is say a concept art style? As in reality the styles exist because they are fast, whereas dalle takes the same amount of time to do any style
Haha super fair point
When will Dall-e 2 released for all?
If the AI only did what it did with when you give it an image I would be fine with it....it's the fact you can go "Give me ___ and ____ in the style of Studio Ghibli" and it just rips all the Ghibli art pallets and such and applies them over other mashed photos and such and makes it happen. I dunno how I feel about that.
Like it's not copyright infringement by any means, but it's not...honest...either.
Would it also be possible to get a drawing by lesser known draftsmen? Like a mural by Kim Jung Gi, a caricature by Jack Davis, an anatomy study by Frank Frazetta etc. It would e so cool to study art and anatomy from drawings by Ai, for example you want to draw a character like Kim Jung Gi but you're not sure what it looks like under a certain angle etc
Congrats on getting access!! Really jealous
Thanks!
Exciting stuff.
At some point this will be generating animations, imagine inputing your favorite story and having it brought to life.
Your girl story sounds similar to Moses’. Have you seen the movie Prince of Egypt?
I use Dall-e mini as starting point too. But I’m only beginning.
In that blade runner shot, so we just going to act like dalle didn’t design some really cool practical futuristic car.
Underrated.
The variants of an already finished piece is probably the most useful thing here for concept artists, however even then Clients are very specific with changes. I'm not sure how useful this would be with the pseudo random variants it spits out. Will be interesting to see how it's put to use if allowed to the public.
Great video! Just as a heads up, DALL-E is pronounced with an 'aah' sound (Open back unrounded vowel) as in "ball" and "tall". The name is a blend of WALL-E and Salvador Dalí, which both use that vowel sound.
Thanks for the space ship;)
This would be amazing for me as I have so many creative ideas in my head but I can never get them onto paper.
The idea of generating from reference images is extraordinary. It adds serendipity in a consistent style otherwise missing.
Did you use GPT-3 to co-write this story?
Lol I made up the story on the fly 🙃
How much RAM is needed for this. And how long did this take? Thank you. I'm on the waitlist for Dalle2. I have a laptop with 4 gigs of ram. Presently using Google Colab with interesting but unpredictable and irrelevant outcomes. Too many results have a Shutterstock logo watermarked which I guess will change in time. And it takes quite a while - hours if you tweak stuff. This is exciting but frustrating as I'm just starting out and I'm old. I can see this helping my online home decor business if and when I get it figured out how to print my AI designs. I already sell some concept art which is fun to create but hellish time-consuming. I'll watch your YT series.
"And when everyone is super... No one will be"
Based on what you mentioned on a podcast a while back (preparing for DALL-E) would like to know how your advice changes (if at all) now that you’re hands-on.
I imagine the speed (and getting 10x outputs each time) makes it a challenge to keep track of what you’ve attempted. Did you manage to remain systematic, or just jump into sandbox-mode right away?
Love the way you’re going through it based on use-cases. Such unlimited possibilities can be exciting, but also potentially paralyzing.
You're right on the money with this one Josh. I'm going to provide updated tips on DALL-E 2 usage hopefully in the next podcast episode. The iteration rate is far greater. Also, I agree with you, so many directions and ways you can go with things. This is also why I like to take notes, we can isolate things out that way at least mentally a bit.
Great video! Love your content...
Thank you!!
Can you import an image while adding a modified text description, or a text description of itself? If so, would they turn out more homogenous given the right prompt? Example: Same Bladerunner shot, but the man's POV" or "the exact same image but a wider shot"
Also, does it work with more detailed positional prompts? "Girl standing on the left third, pyramid taking up the top right third, water-bottle in the bottom middle foreground"
Great questions. At the moment (again I'm in the research beta) you can't import simultaneously with a prompt. You upload and then either edit or generate variations. To answer your other question, I haven't had any time to test the positional prompt stuff but it's an interesting area I've wanted to check since the beginning.
Could you do a video about Aleph Alpha's luminous? I mean DALL-E and GPT-3 are certainly very fascinating but I believe they get so much exposure compared to other great models out there. From what I've seen their model is performing really well but I'm not too experienced so I'd like to hear your opinion! Also: I love your videos!
AI dangeoun for story + art by DALL-E 2 + GPT 3 as NPCs + ultra realistic VR headset
I know it's not that easy to put everyone together but let me dream ok lol
Using Dall-e's Inpainting tool, is it possible to replace parts of a character?
for example. load in a character concept art, then replace the head, arms, body, etc untill you have a new character design?
Theoretically yes, very interesting idea!
@@bakztfuture Awesome. there are so many things I want to try with Dall-E 2.
I'm a concept artist, and i'm always trying to figure out ways of speeding up my workflow.
I'm on the waitlist, but in the meantime,
I'd love to see the results of your experimentation. even failed attempts will be useful for seeing the limitations.
These are exciting times.
What about animation key frames?
of God...
Is openAI open source?
Use the other ai to create a story then use words from that to generate the pictures
can it turn your sketches into a final image? like those sketches that you had in your last story?
Those sketches had the DALL-E colored box signature in the bottom-right; they *were* the DALL-E output, not hand made :)
Yes @icemetalpunk nailed it as always. DALL-E made those sketches too and yes it can convert between sketches and paintings
@@bakztfuture can AI do brain storming? like a colleague?
@@hamidmohamadzade1920 It can generate variations on a single image, which is a sort of brainstorming; but for the kind of brainstorming where you come up with many distinct ideas, I think combining it with GPT-3 would achieve that result better than trying to get DALL-E to do that on its own. GPT-3 to generate the different ideas, DALL-E 2 to generate them as images and produce variations on each idea.
OH MY GODDD YOU ARE SO UNCREATIVE
Rendering in extremely high fidelity and doing it quickly, even with initial concepts and iterations, is extremely common in concept design. Not saying this tool can't be useful in v creative ways along the process, but I get the very palpable sense that a lot of these Dall-E videos are done by people that are shockingly ignorant about common artistic practices in the commercial art community, including concept artists. Like, the potential isn't really being explored. It's just a bunch of assumptions about how real commercial artists work. Again, this tool is revolutionary. But devs seem to miss the whole freaking point of their own tools.
I was talking about very very photo realistic renders
@@bakztfuture I know u were 😌. Hence my response. That is what 'extremely high fidelity' means.
That's not even the real issue tho. Although the capacity for photorealism already exists at the initial stages of concept design, the reason it isn't pursued has to do with foundational principles about design. For example, in architecture (my field) first-year students love to render their initial concepts with high fidelity, and we always have to move them away from this strategy because it isn't conducive to studying formal properties that are infinitely more important than photorealistic representation. The use of non-photorealistic representation allows us to study volume, massing strategies, circulation patterns, or all the relevant spatial strategies that matter way more than end-result representation. Photorealistic representation (at this stage) gets in the way of all this for many reasons, not the least of which is that it generates so much visual noise. Reducing or abstracting initial concepts produces better designs down the line. Again, this is a foundational practice in the theory of design and similar analogues can be drawn from concept design. It seems to me that devs are so arrogant that they cannot even fathom their ignorance about fields they do not work in. Sorry to be harsh. But this pattern persists. Again, this tech is amazing. And I can envision so many interesting use-cases. But hardly anyone in the dev community has spoken about them lol. It speaks to the greater need of cross-disciplinary strategies going forward in Ai. And btw, I think u can make the same criticism about most designers and our general ignorance about CS.
3:23 imagine inpainting the person into a capybara lol
I really feel sorry for digital artists out there 😔
Lets be real guys making these mind blowing arts so easy with ai u don't even need any knowledge about art or ai.
I think it will affect moreso beginner artists doing small commissions who's clients don't care who's doing the work.
@@definitelynotnick2454 although I can see my self using this as web designer. This will save my life.
now you can even generate your own unique design for your UA-cam channel for free
0:40 Is this just Naruto characters 😆
try to use more generic words, like "a monster attacks a character" and see how the variations are different
Man I'm so jealous
I want to make t-shirts designs using dalle, hopefully they accept me 🙏
Great idea. Tag me on Twitter when you launch and I'll check it out!
I asked a question on your Yeezy video, please answer. Update: He deleted it anyway.
Calm down buddy. I doubt he wants your money lol. No, Dall-E 2 cannot do any of that. Future programs will probably have this functionality, but I'd imagine we're at least 5-10 years out from that. Dall-E 2 generates an image based on a text prompt. It does not understand anything like real-world measurements (although it probably has some grasp of things like relative scale), let alone something as specific as all the components in a proper CAD file. That would require an AI that could convert an image into a 3D representation, along with all the data that makes it usable as a CAD file, and while we're entering into that field with things like Nvidia NeRF, we're still quite a away from the fidelity needed for something so precise. TL;DR, no, but check back every couple years.
@@acemagex13 Don't start off beta, it's called a proposal. If he is a man he can speak for himself or not. Thanks for the info.
@@cortezforever "Don't start off beta" lmaooo, go watch your fresh & fit podcast💀and don't @ me
@@SpeedrunsMakkelijk Bye Mr Hitler.
@@cortezforever Godwin's law any% WR ez
Ъ