Watching the beginnings of a new art form emerge these last few weeks. We're at the cutting edge, thanks so much for putting the effort in to share your techniques!
The img2img alternative script runs the model in reverse to find the noise pattern that would generate an approximation of that image and then uses it as seed for the actual generation. So it is a way to do fine adjustments to an image. Also it is only precise in the euler sampler.
A few other tricks that could help; 1- reducing the number of frames without deleting every other frame; you can do that in adobe premiere, after effects or whatever software you use to export into single frames. You just reduce the frame rate before exporting. 2- Use DAIN ai frame interpolator or similar, so you can use with even less frames per second. Then input into DAIN and it will create the in-between frames even smoother. 3- 2D animations are usually made with 12 or 15 frame per second. So you can reduce the jittery-ness even more by just reducing the frame per second to 12 or 15 fps, and it won't look weird at all. Because animation audience are used to reduced frame per second videos. For example Rick and Morty characteres move at about 12 fps. I am an animator and thanks for your videos. I am really excited to try this technique out and make some content.
Thank you for the tricks! I’m definitely interested in reducing frame rate to something 12 fps and testing that with frame interpolation! I will definitely try that next time!
You can control the background by reducing the "denoising strength", but you will lose the changes in the character. I watched some other videos about video generation via img2img but yeah, the "frame blending" is the catch for smoother non-stuttering AI transition videos. This video totally dope! Thanks for sharing it 👍
well done video honestly I love your "how to delete every second file" technique. it's such smooth little hack. also never new about mass renaming. the stable diffusion stuff is cool too tough :P
I've been waiting for this one. I don't have nearly as dramatic subject matter, so my first attempt will be turning my hopping bunny into a hopping potato.
Great tips! I hadn't thought about using the frame blending method to smooth it out, but that makes sense. It also makes me wonder whether using optical flow might help sometimes, too. But in your example, when you hit that frame blending tab, it really made a difference. Thanks again!
If you open your original clip in premiere,create sequence from clip, then go to sequence settings, you can drop the frame rate from what you have it to less, say 24fps to 12fps, then export as png img sequence without having to delete the pictures...
Here is a tip: create 12FPS composition in AE when you are exporting your video as PNG sequence and it will make your life easier to import it back again and you don't have to rename files.
Maybe also worth checking out Pixelmotion blur in AE. Double the length to get original time-frame back (from deleting every 2nd frame) keep frame blending and also add pixelmotion blur. It interpolates the expected blur on moving images to bring back natural motion.
Dude! your 0:37 animation just got shown live on stage at the Stable Diffusion presentation today. Look up "Stability Diffusion announcements" from Robert Scoble. It appears at 50:27. Congrats!
Wow!!! Thats so cool! I’m so happy that people are showcasing what I’ve work on as an example. Thank you for letting me know about this, definitely made my day!
cool stuff, thanks for sharing! quick tip with the first part of the video: you can set different frame rates in your AE comp on the original footage (8, 12 etc) in Composition Settings, then export picture sequences before you plug them into SD. this way it's easier to play around with lower framerates instead of having to delete them manually. Instead of frame blending, you can try the Time Warp effect in AE, which is pretty much the same just with more settings. But what I would try next, is dropping the video frame rate to something like 10, then using an AI frame interpolation algorythim like TVP instead of the built in AE one. Those algorythms are trained for cartoons and will give a lot better results than the pixel morphing AE effects. this is just theory though, I'm yet to try this next week.
thank youuuuuuuuuuu. i found your channel. been looking for this style, cant find. i saw this style from WebUigirls, but they dont have a tutorials. though they used LoRA model. but hey thnk this is very helpfulllll! more power!
bro. thankyou very much. This tutorial really helpful to me for getting better output (90% expectations are reached) - I used the High Resolution image for sample (500 image), everyone can downscalling and adjusting the resolutions for get output quickly - used simple prompt (the output can to reuse for changing the main sample picture for next train Image) - img2img alternative enabled with same setting in this video. thats really helpful to getting output image - i used OpenPose [dw openpose full with sd15 openpose controlnetmodel] controlnet (getting same output the object and expression from data image train source) and lineart [lineart_realistic and sd15 lineart model controlnet] controlnet (I Think this controlnet really useful for painting the full area of each data image train) Again bro. Thankyou very much 😃😃👍👍👍👍👍
You can also do another thing, increase the fps to like 120 and then slow down video, smoother video with overall less jittery video + spamming de-flicker node on davinci resolve
I have been using SD v1.5 and it is SIGNIFICANTLY better than 1.4. In fact I now prefer the overall quality over MJ. But since MJ uses SD the next release of MJ v4 will incorporate SD v1.5. Things are moving so fast.
in after effects you can export your original frames at a lower frame rate - which should save a bit of messing around. Also, try using timewarp instead of frame blending. This uses a softer morphing technique between frames and you'll be able to get away with doing maybe 1 frame in 5 rather than 1 in 2 for a much smoother animation. also, how about separating the background and foreground so you can give them different settings. Maybe you could even motion track the background, export and run SD on just one frame of it (with some outpainting to extend the edges) then use the tracking data to animate that background. That way you'll have a rock-steady background painting. Finally, experiment with a "difference matte". This is a great way to key out only those parts of an image that stay the same between frames. Use a difference matte (slightly blurred) to create a b/w mask. then have after effects only "repaint" the sections of the image that change from one frame to the next. That way, if you've got (say) a closeup of a face, speaking, only the mouth will be redrawn for different positions, leaving the rest of the face untouched.
Hey, this is fantastic! Thanks for putting this together. You should drop a link in for aitrepeneur's video about training styles in your description, he'd probably appreciate it since you reference his midjourney training. Would anybody by chance happen to know if that frame smoothing option or something similar is available in Apple Motion? I'd like to give it a go, but I refuse to give Adobe a lead cent, screw those guys.
so im curious about your framerates on this. Is the initial video at 30 fps for your png sequence, then you delete half, then take the you ai rendered sequence back into AE, but is it now faster than the original video? or does the frame blending change that? Going to try some of this out now, I like seeing this difference between the more jumpy part, frame blending is a cool trick, somehow never noticed that box was there
this is funny because I was wondering if I should remove every other frame and then use Topaz Video Enhance AI to do a slow mo version of my video, which would then use AI to fill in the frame gaps and then use Premier Pro to speed the video back up ... but watching your video, you've found an easier method
@@enigmatic_eYou'll love it. NMKD made GUIs like Flowframes for easy Frame Interpolation, Cupscale for Image and Video Enhancement and Stable Diffusion GUI.
@@enigmatic_e Flowframes is the best. I've thrashed it pretty hard with 4k MB3D renders. Interpolating crisp fractals is hard, and this will easily do 4 frames out of 5 and you won't notice a thing. It also only takes 20 minutes to do a vid clip that Topaz wants to spend all night on.
Could “frame blending” be another name for “image interpolation “? Cause that would make perfect sense why it becomes so smooth and retains the consistency longer. image interpolation is an amazing tool that I rarely see used. Basically predicts the frames in between two separate stills. It can easily turn a 30 fps vid into a 60fps. It works wonders on old (and new) claymation videos!!
Thanks for the trick on deleting every other frame on windows. I wasted so much time the other day doing it manually. 😪 Have you tried flowframes for the frame interpolation?
It's interesting, I've tried this technique shown here, on nerdy rodents page, and on max novak's channel and the alternative test scripts dont seem to function the same way at all. Turning on the script immediately breaks any style tokens you add in the prompt, override denoising strength doesn't seem to have the same function as you guys display, and the sigma checkbox locks it down even further. Are you guys testing this on the most recent git pull? All the videos showing this off are from a couple days ago and i dont think the updated version runs the same. I've tested this on brand new clean installs on 2 machines and it just doesn't function the way you guys are showcasing.
Hey, i can tell you that its just tweaking things. Ive had my image look good and sudden i change the a word in the prompt or change a value by 1 unit and it becomes a total messed.
@@enigmatic_e appreciate the reply and im sure that plays a part, but using the same source images settings and prompts yield completely different results. Particularly with the override check boxes that were introduced that do not exist in any of the videos on this topic including yours since they were made before the update. Something has changed recently guess anyone trying this now will have to wait on clarification as people experiment.
Great video! Very useful tips! Could you tell us what is the framerate of your original video, and once you import your custom png's what's the "assumed framerate" in after effects? Thanks!
Very helpful! Im rendering a long video now and Im going to try this. I do have a question. Ive been shooting my videos in 60fps in the hopes of having more smooth end results, but if this video is the case, would you suggest shooting in 24 or 30fps and still deleting every other frame?
Me thinking about deleting every other frame: should be straighforward in Bash / GitBash, just make a seq of odd numbers and pipe it into xargs rm with a glob mask. enigmatic_e: literally just resizes the explorer window Me: (•́Ô•̀)
So the whole video has to already be in a stylized frame? My video has a lot of different people and scene changes. Because what if I only styled the first 10 secs will eybsnth now to stylize the rest of the video into like a drawing. Or will it only stylize the frame I gave it. I have a 5 minute clip I want to stylize but I only have a stylized frame for the first 10 secs? Will eybsnth know to stylize the rest of my clip because it has a lot of moving parts it?
I was thinking this too. I doubt the results would be perfect, but I think there's a strong chance they'd look better than AE's frame blending. I'm also curious how Adobe's other frame interpolation methods would perform. I'm guessing worse (in this situation). I don't think it'll be all that long before this is an actual feature in SD, though. I don't know the ideal path to achieving that, but given that many AI algorithms have focused on compensating for change over time in the last couple of years, I wouldn't be surprised if it happens. Image generation is already progressing at an insanely fast rate. I wish I had more time to play around with these techniques right now.
Yes you can technically use this version of SD using Colab. I show how to use it in this video ua-cam.com/video/qmnXBx3PcuM/v-deo.html check the second half of the video.
Thank you for this video! I have an error using img2img alternative test "OutOfMemoryError: CUDA out of memory". What can i do with it? I can't buy a new videocard as for now
Artstyles can be made by yourself but it take a lot of steps. AItreprenuer has some tutorials on how to do that. ua-cam.com/video/tgRiZzwSdXg/v-deo.html
When I delete the odd frames and then bring the even into After Effects or even Media encoder it is showing color bars or media off line for the deleted frames. Can anyone remedy?
Frame blending works (was using it) but still don't have stability in SD (even with seed & proper setup). And it's difficult to track video properly. (At this stage)
I know its a bit tricky to get it right. Theres a lot of variables that go into it and now that i updated to new ui things seems to be different. I will def be making another video on the topic
what frame rate are your videos set to ? say if you have it set to 60fps and do every other frame like you suggest, would changing the frame rate to 30fps and doing every frame result in the same smoothness?
I would say using them together creates great results when there are subtle movements. SD img2img is getting better though now with ControlNet. Have you heard of it?
If you are animating a video with audio like singing, deleting every other frame will shorten the final animation and cause the audio to not be in sync correct?
Blending a Rife frame interpolated and a every frame version could make the video more consistent and more accurate. The other thing is using the original video and converting it to a black and white image. This will help with keeping the brightness consistent, and then you can adjust the color saturation and vibrancy to bring back the color. If the color is shifting around a lot, you might need to just blend in the original video. In my opinion, if you aren't willing to put in the work to keep the frames consistent by looking at each converted image by hand, then you should just go with the jittery image. Trust me, making it the style is a lot better than doing a poor job trying to make it consistent. The AI will get better over time, so hopefully it won't be a problem in the future.
I followed what you were doing but when I turn on Img2Img Alternative my image completely changes and doesn't resemble my original image, any idea why this might be happening?
Yes, when you import sequence into editing program, just interpret frame rate to original clip. Most programs will automatically make it 30 fps, so change it if you have to. Then just change speed to double the length and it should sync with audio
![АЛАУДИНОВ у Скабеевой: эти их ВСУ нас НЕ ДОГОНЯТ 😁 [Пародия]](http://i.ytimg.com/vi/an0anqU4WGQ/mqdefault.jpg)
Watching the beginnings of a new art form emerge these last few weeks. We're at the cutting edge, thanks so much for putting the effort in to share your techniques!
🙏🏽
Basically it's a rotoscoping setup for much cheaper and quicker than the traditional methods.
Seriously, this is an artistic revolution, the likes of which we haven't seen... ever!
The img2img alternative script runs the model in reverse to find the noise pattern that would generate an approximation of that image and then uses it as seed for the actual generation. So it is a way to do fine adjustments to an image. Also it is only precise in the euler sampler.
Thank you for sharing this info. It’s good to know.
A few other tricks that could help;
1- reducing the number of frames without deleting every other frame; you can do that in adobe premiere, after effects or whatever software you use to export into single frames. You just reduce the frame rate before exporting.
2- Use DAIN ai frame interpolator or similar, so you can use with even less frames per second. Then input into DAIN and it will create the in-between frames even smoother.
3- 2D animations are usually made with 12 or 15 frame per second. So you can reduce the jittery-ness even more by just reducing the frame per second to 12 or 15 fps, and it won't look weird at all. Because animation audience are used to reduced frame per second videos. For example Rick and Morty characteres move at about 12 fps.
I am an animator and thanks for your videos. I am really excited to try this technique out and make some content.
Thank you for the tricks! I’m definitely interested in reducing frame rate to something 12 fps and testing that with frame interpolation! I will definitely try that next time!
You can control the background by reducing the "denoising strength", but you will lose the changes in the character. I watched some other videos about video generation via img2img but yeah, the "frame blending" is the catch for smoother non-stuttering AI transition videos. This video totally dope!
Thanks for sharing it 👍
well done video
honestly I love your "how to delete every second file" technique. it's such smooth little hack.
also never new about mass renaming.
the stable diffusion stuff is cool too tough :P
Dude really appreciate your videos. Awesome work and thanks for sharing your knowledge on this. This tech is so ground breaking ...
Thank you! I agree is ground breaking and…… it kinda scares me 😂
I've been waiting for this one. I don't have nearly as dramatic subject matter, so my first attempt will be turning my hopping bunny into a hopping potato.
😂 nice
Great tips! I hadn't thought about using the frame blending method to smooth it out, but that makes sense. It also makes me wonder whether using optical flow might help sometimes, too. But in your example, when you hit that frame blending tab, it really made a difference. Thanks again!
If you open your original clip in premiere,create sequence from clip, then go to sequence settings, you can drop the frame rate from what you have it to less, say 24fps to 12fps, then export as png img sequence without having to delete the pictures...
I love you for making this video, thank you so much
Here is a tip: create 12FPS composition in AE when you are exporting your video as PNG sequence and it will make your life easier to import it back again and you don't have to rename files.
Note for people who don't work with video, "12FPS" is only true for 23.976 FPS footage. Essentially half your normal frame rate in the composition.
Bruh that was the most entertaining n original intro 👍👍
Maybe also worth checking out Pixelmotion blur in AE. Double the length to get original time-frame back (from deleting every 2nd frame) keep frame blending and also add pixelmotion blur. It interpolates the expected blur on moving images to bring back natural motion.
Nice, is that a plug in already in AE?
@@enigmatic_e Yes it's default in AE.
Deleting half of frames really game changer . Nice and wonderful trick you `ve been share in this video . Super thanks !!!
Great job my broër°°°
Thanks bro!!
I love your channel !
🙏🙏
Thank you so much for taking the time to help us. You are a blessing to humanity brother. I mean it. ❤
pretty cool!!!
lmao look whoz here
maybe next video is ai based triology
Dude! your 0:37 animation just got shown live on stage at the Stable Diffusion presentation today. Look up "Stability Diffusion announcements" from Robert Scoble. It appears at 50:27. Congrats!
Wow!!! Thats so cool! I’m so happy that people are showcasing what I’ve work on as an example. Thank you for letting me know about this, definitely made my day!
@@enigmatic_e When I saw them show it on stage, I was like, OMG I just saw a video about how this was made. I had to tell you.
This is the first video I stumbled on for this channel. I can already tell it's always top quality 👌
👍🏽
Thanks! I've been searching how to get it and this is brilliant :D
Nice Tutorial. Subscribed. Good luck with the channel!
Thank you!
Keep making the content bro
Thanks! 🙏🏽🙏🏽🙏🏽
cool stuff, thanks for sharing! quick tip with the first part of the video: you can set different frame rates in your AE comp on the original footage (8, 12 etc) in Composition Settings, then export picture sequences before you plug them into SD. this way it's easier to play around with lower framerates instead of having to delete them manually. Instead of frame blending, you can try the Time Warp effect in AE, which is pretty much the same just with more settings. But what I would try next, is dropping the video frame rate to something like 10, then using an AI frame interpolation algorythim like TVP instead of the built in AE one. Those algorythms are trained for cartoons and will give a lot better results than the pixel morphing AE effects. this is just theory though, I'm yet to try this next week.
Thank you for sharing all of these informations.... God bless you
You're awesome and friendly. Thanks for the video
awesome video Mate 🙂
works, chock-full thanks!
Great video!
Great video ❤
great video bro
Looks pretty good
the original prompt is for what the original image is supposed to be. And the prompt at the top should be what you want it to be.
Thanks for the tips!
Dope 🙏🏽✨
Good job man, thank you for putting that out !
Thank you! I appreciate you checking out the video.
thank youuuuuuuuuuu. i found your channel. been looking for this style, cant find. i saw this style from WebUigirls, but they dont have a tutorials. though they used LoRA model. but hey thnk this is very helpfulllll! more power!
AI Researchers: Let's have a model with temporal consistency
Creators: Blend the frames 😂
Very useful ! Thanks !
THANKS DUDE
Thanks!
Thank you!!!!
This is so cool
bro. thankyou very much. This tutorial really helpful to me for getting better output (90% expectations are reached)
- I used the High Resolution image for sample (500 image), everyone can downscalling and adjusting the resolutions for get output quickly
- used simple prompt (the output can to reuse for changing the main sample picture for next train Image)
- img2img alternative enabled with same setting in this video. thats really helpful to getting output image
- i used OpenPose [dw openpose full with sd15 openpose controlnetmodel] controlnet (getting same output the object and expression from data image train source) and lineart [lineart_realistic and sd15 lineart model controlnet] controlnet (I Think this controlnet really useful for painting the full area of each data image train)
Again bro. Thankyou very much 😃😃👍👍👍👍👍
a young woman looking towards camera with green hair holding wearing gloves with a blue shirt, anime, midjoumeyart style
blotches, blown-out, maximalist, teeth, mouth open
Thank you!
you rock!!!
You can also do another thing, increase the fps to like 120 and then slow down video, smoother video with overall less jittery video + spamming de-flicker node on davinci resolve
You can also use SD to style animations you render in Blender. SD is more creative and diverse than EbSynth.
I have been using SD v1.5 and it is SIGNIFICANTLY better than 1.4. In fact I now prefer the overall quality over MJ. But since MJ uses SD the next release of MJ v4 will incorporate SD v1.5. Things are moving so fast.
So interesting dude, well explained too. At least for an AI beginner like me it was easy to follow.🤙
in after effects you can export your original frames at a lower frame rate - which should save a bit of messing around. Also, try using timewarp instead of frame blending. This uses a softer morphing technique between frames and you'll be able to get away with doing maybe 1 frame in 5 rather than 1 in 2 for a much smoother animation.
also, how about separating the background and foreground so you can give them different settings. Maybe you could even motion track the background, export and run SD on just one frame of it (with some outpainting to extend the edges) then use the tracking data to animate that background. That way you'll have a rock-steady background painting.
Finally, experiment with a "difference matte". This is a great way to key out only those parts of an image that stay the same between frames. Use a difference matte (slightly blurred) to create a b/w mask. then have after effects only "repaint" the sections of the image that change from one frame to the next.
That way, if you've got (say) a closeup of a face, speaking, only the mouth will be redrawn for different positions, leaving the rest of the face untouched.
Thanks for advice. Will try this!
Hey, this is fantastic! Thanks for putting this together. You should drop a link in for aitrepeneur's video about training styles in your description, he'd probably appreciate it since you reference his midjourney training. Would anybody by chance happen to know if that frame smoothing option or something similar is available in Apple Motion? I'd like to give it a go, but I refuse to give Adobe a lead cent, screw those guys.
you can use optical flow in fcp
Thank you
thanks ma a brotha
thanks for the tutorial~ by the way, what device do you use for lighting the background and yourself it looks cool~
Thanks! The lights are called CAME-TV which aren’t sold anymore. But there are plenty of RBG saber type lights out there.
@@enigmatic_e thanks a lot~
so im curious about your framerates on this. Is the initial video at 30 fps for your png sequence, then you delete half, then take the you ai rendered sequence back into AE, but is it now faster than the original video? or does the frame blending change that? Going to try some of this out now, I like seeing this difference between the more jumpy part, frame blending is a cool trick, somehow never noticed that box was there
this is funny because I was wondering if I should remove every other frame and then use Topaz Video Enhance AI to do a slow mo version of my video, which would then use AI to fill in the frame gaps and then use Premier Pro to speed the video back up ... but watching your video, you've found an easier method
Try Flowframes + EbSynth. I bet that could be something cool
I heard of flowframes and even downloaded it but never tried it! I will try it soon!
@@enigmatic_eYou'll love it. NMKD made GUIs like Flowframes for easy Frame Interpolation, Cupscale for Image and Video Enhancement and Stable Diffusion GUI.
@@enigmatic_e anf IF it helps you, PLEEEASE give me a credit in a Video it would mean so much to me 💜🥹
@@enigmatic_e Flowframes is the best. I've thrashed it pretty hard with 4k MB3D renders. Interpolating crisp fractals is hard, and this will easily do 4 frames out of 5 and you won't notice a thing.
It also only takes 20 minutes to do a vid clip that Topaz wants to spend all night on.
thanks!
thanks ^^
Could “frame blending” be another name for “image interpolation “?
Cause that would make perfect sense why it becomes so smooth and retains the consistency longer. image interpolation is an amazing tool that I rarely see used. Basically predicts the frames in between two separate stills. It can easily turn a 30 fps vid into a 60fps. It works wonders on old (and new) claymation videos!!
gracias!
Nice tuts sir, pelase do animatediff tuts for webui instead of comfy, im ur #1 fan!
Thanks for that. Also try flowframes for smoothing the animation
Thanks for the trick on deleting every other frame on windows. I wasted so much time the other day doing it manually. 😪 Have you tried flowframes for the frame interpolation?
Glad i could help!
No i havent but i heard a lot about. I will try it out.
It's interesting, I've tried this technique shown here, on nerdy rodents page, and on max novak's channel and the alternative test scripts dont seem to function the same way at all. Turning on the script immediately breaks any style tokens you add in the prompt, override denoising strength doesn't seem to have the same function as you guys display, and the sigma checkbox locks it down even further. Are you guys testing this on the most recent git pull? All the videos showing this off are from a couple days ago and i dont think the updated version runs the same. I've tested this on brand new clean installs on 2 machines and it just doesn't function the way you guys are showcasing.
Hey, i can tell you that its just tweaking things. Ive had my image look good and sudden i change the a word in the prompt or change a value by 1 unit and it becomes a total messed.
@@enigmatic_e appreciate the reply and im sure that plays a part, but using the same source images settings and prompts yield completely different results. Particularly with the override check boxes that were introduced that do not exist in any of the videos on this topic including yours since they were made before the update. Something has changed recently guess anyone trying this now will have to wait on clarification as people experiment.
Ahh ok. I wasn’t aware of an update. I will def look into it.
The generated image is similar to the original image, I want to generate a woman on the moon, in this case the result is no longer stable
Great video! Very useful tips! Could you tell us what is the framerate of your original video, and once you import your custom png's what's the "assumed framerate" in after effects? Thanks!
Maybe image morphing could help? There is a stable-diffusion-morphing repo on Github...
que hermoso video se me enciende la imaginación gracias por las clases... pero el programa donde esta?
Very helpful! Im rendering a long video now and Im going to try this. I do have a question. Ive been shooting my videos in 60fps in the hopes of having more smooth end results, but if this video is the case, would you suggest shooting in 24 or 30fps and still deleting every other frame?
You can shoot in 60 fps and test results. You can always remove frames after you want to.
Me thinking about deleting every other frame: should be straighforward in Bash / GitBash, just make a seq of odd numbers and pipe it into xargs rm with a glob mask.
enigmatic_e: literally just resizes the explorer window
Me: (•́Ô•̀)
😂 hey as long as it gets the job done
More people don't use this often? I'm using it all time since two weeks. I should start posting more lol
Good thing I got 10gbs of VRAM
So the whole video has to already be in a stylized frame? My video has a lot of different people and scene changes. Because what if I only styled the first 10 secs will eybsnth now to stylize the rest of the video into like a drawing. Or will it only stylize the frame I gave it. I have a 5 minute clip I want to stylize but I only have a stylized frame for the first 10 secs? Will eybsnth know to stylize the rest of my clip because it has a lot of moving parts it?
Batch Img2Img
AttributeError: 'NoneType' object has no attribute 'ema_scope'
Error Fixed :
www.reddit.com/r/StableDiffusion/comments/yjbc2d/batch_img2img_help/?context=3
I wonder how well the blending would be if you ran the images through a RIFE or DAIN
I was thinking this too. I doubt the results would be perfect, but I think there's a strong chance they'd look better than AE's frame blending. I'm also curious how Adobe's other frame interpolation methods would perform. I'm guessing worse (in this situation). I don't think it'll be all that long before this is an actual feature in SD, though. I don't know the ideal path to achieving that, but given that many AI algorithms have focused on compensating for change over time in the last couple of years, I wouldn't be surprised if it happens. Image generation is already progressing at an insanely fast rate. I wish I had more time to play around with these techniques right now.
thank you , also I wonder if you can do the same on google colab.. maybe slow but maybe not .. most of people use colab
Yes you can technically use this version of SD using Colab. I show how to use it in this video ua-cam.com/video/qmnXBx3PcuM/v-deo.html check the second half of the video.
I feel like frame interpolation could be cool instead of frame blending
Thank you for this video! I have an error using img2img alternative test "OutOfMemoryError: CUDA out of memory". What can i do with it? I can't buy a new videocard as for now
nice one! where/how did you get that art style exactly?
Artstyles can be made by yourself but it take a lot of steps. AItreprenuer has some tutorials on how to do that. ua-cam.com/video/tgRiZzwSdXg/v-deo.html
@@enigmatic_e thanks alot mate!
When I delete the odd frames and then bring the even into After Effects or even Media encoder it is showing color bars or media off line for the deleted frames. Can anyone remedy?
that moment when you want stable diffusion into your iphone, it's already someone have make an app to replace sd filters in camera on the go?
That would be nice!
Please do one using google colab, I currently do not have enough gpu
Frame blending works (was using it) but still don't have stability in SD (even with seed & proper setup). And it's difficult to track video properly. (At this stage)
I know its a bit tricky to get it right. Theres a lot of variables that go into it and now that i updated to new ui things seems to be different. I will def be making another video on the topic
@@enigmatic_e Thank you, great work! It's fun learning & growing together with this technology.
was there a reason you didn't post your negative prompts into the Script settings?
what frame rate are your videos set to ? say if you have it set to 60fps and do every other frame like you suggest, would changing the frame rate to 30fps and doing every frame result in the same smoothness?
Mmm I think 30 fps works slightly better. 60 might be too many frames in a short amount of time. Test it and see how you like the results.
Hi please tell me there must be some way to install im2img into the MAC iOS system 😢
Otherwise I need to buy a new windows computer….no…
since you have used both, what do you think is better, img2img SD like this video, or using EbSynth? pros and cons to both?
I would say using them together creates great results when there are subtle movements. SD img2img is getting better though now with ControlNet. Have you heard of it?
@@enigmatic_e no i haven't. imma check that out
If you are animating a video with audio like singing, deleting every other frame will shorten the final animation and cause the audio to not be in sync correct?
Just make the sequence same fps and should look fine.
Blending a Rife frame interpolated and a every frame version could make the video more consistent and more accurate. The other thing is using the original video and converting it to a black and white image. This will help with keeping the brightness consistent, and then you can adjust the color saturation and vibrancy to bring back the color. If the color is shifting around a lot, you might need to just blend in the original video. In my opinion, if you aren't willing to put in the work to keep the frames consistent by looking at each converted image by hand, then you should just go with the jittery image. Trust me, making it the style is a lot better than doing a poor job trying to make it consistent. The AI will get better over time, so hopefully it won't be a problem in the future.
ouch, any way to blend frames in blender?
Could you edit video like Second gear UA-cam channel?
do you have the disco diffusion style for this? I see that in the description there is only the one from midjourney.
ok
I followed what you were doing but when I turn on Img2Img Alternative my image completely changes and doesn't resemble my original image, any idea why this might be happening?
Try clicking on sigma adjustment right the bottom of script. Sometimes that fixes it.
Aaah you should have redone your wife's shot with the decimated frames technique to show the difference
I know! I thought of that very thing after watching it over. 😅
@@enigmatic_e maybe an idea for another video ☺
How about using some ai interpolation instead of frame blending ?
I will try that. Maybe even do a short video showing results. Thank you!
how can i get to this site? I use mac os and I can't find it at all.
if you delete half of the the frames, will you still be able to sync audio you may have ?
Yes, when you import sequence into editing program, just interpret frame rate to original clip. Most programs will automatically make it 30 fps, so change it if you have to. Then just change speed to double the length and it should sync with audio