Better Than DAIN? NEW BEST Tool for Boosting Video's FPS with AI [RIFE/Flowframes]
Вставка
- Опубліковано 21 лис 2020
- Boost FPS for anime, stop motions, animations easily and fast with RIFE.
Increase video's FPS with this new AI! After quite some thoughts I think this is definitely slightly more advantageous than DAIN, which is really amazing!
RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation
RIFE APP FREE: nmkd.itch.io/flowframes
GitHub: github.com/hzwer/Arxiv2020-RIFE
Paper: arxiv.org/abs/2011.06294
Project Page: rife-vfi.github.io/
Colab: colab.research.google.com/git...
RIFE APP: nmkd.itch.io/flowframes
RIFE Installation Tutorial
• RIFE Video Frame Inter...
Today's Sponsor is Infinite Red
Infinite Red consulting handles your mobile, web, and AI needs
Check it out here: bit.ly/2UwddmM
This video is supported by the kind Patrons:
🙏Marc Schwyn, Mazen Alotaibi, Jason Nickel, Wampipti, Sascha Henrichs, Jake Disco
Support me on Patreon if you hope to see more:
/ bycloud
[Discord] / discord
[Twitter] / bycloudai
[Patreon] / bycloud
[Music] Steaminwaffles - Wait
[Profile Art] / bynicalcynical
Video credits:
Dog fight
• A Dog Fight Animation ...
Pumpkin
www.reddit.com/r/ArtExplorer/...
Among us fight
• AMONG US FIGHT ANIMATI...
Green vs red
• Green vs Red vs Blue
K/DA - Villain
• K/DA - VILLAIN ft. Mad... - Наука та технологія
![Smoother animation ≠ Better animation [read description]](http://i.ytimg.com/vi/_KRb_qV9P4g/mqdefault.jpg)
![Smoother animation ≠ Better animation [read description]](/img/tr.png)
Check out the full K/DA - Villian 60FPS here: ua-cam.com/video/YZy6l2dBbR4/v-deo.html
Longer RIFE vs DAIN comparison compilation: ua-cam.com/video/2BwC_utLhH8/v-deo.html
RIFE APP with Windows exe is available for free: nmkd.itch.io/flowframes
Hi 🌍💯🔭
Should note the creator is planning on making a model to interpolate anime, fingers crossed it comes out soon!
Gay
gay
Awesome.
@@123495734 how is that gay? It depends on the anime. Anime stands for animation, that's what Japanese people call cartoons. Japanese people watch anime... Japanese cartoons are basically anime and anime is Japanese cartoons. So are you making fun of another nation's word for cartoons oranother nation's cartoons? There are many many different types of anime.
@@Stealthful_ gay
Anime
0:19 Cowboy Bepop
0:43 One Punch Man
1:28 Akira
2:23 Konosuba
3:19 Attack on Titan
4:08 One Punch Man
4:27 Jojo's Bizarre Adventure
4:34 Jujutsu Kaisen
1:00 to 1: 18 show name pls?
@@athul_c1375 It's not a show. It's a music video called K/DA - VILLAIN by Riot Games.
1:38 amogus
The movement looks sluggish sometimes
Ok, but Why
That's what I've seen with a lot of people just plugging interpolation into anime, they don't recognize that the low framerate is just going to make it so that it will hold the drawn poses for longer than the interpolation frames, because since the original footage holds onto a single pose for 2 or more frames, then the animation will still look jittery if the ai only creates an in between every other frame. to get a better result someone would have to first export an animation on 1s, then change the timing of the frames after the AI does it's job.
Finally some one who understands interpolation AI. It must be used with very intentionally and not as a crutch to save cost. It can hard carry slow mo action scenes and cloth physics but 24 FPS impact frames still should be done by a key animator.
Just look at CGI, barely any studios understand why just cutting frames down to 24 FPS doesnt make CGI models look better. Because CGI is a perfect model that warps and transforms perfectly to a perspective grid. It cannot exaggerate or hold a pose unless it it intentionally done.
This is also why scenes which intentionally have 60 FPS look better when interpolated, because the amount of detail that the audience can perceive is far greater, giving them more time to process the whole scene in question.
You can choose to remove duplicate frames, you know.
True but that is already a solution, you can "de-duplicate" the frames which I think mainly just fixes up the frames on twos, it is but of the software though not the AI Model itself.
flowframes has a feature to fix that lol
Since it isn't exactly perfect as of yet, I think this has a lot of use for creative efforts rather than remastering efforts.
I've done a bit of animation work here and there, frame-by-frame rotoscoping being the most nightmare-ish type I've done, and the idea of possibly being able to skip out on drawing a lot of the frames in the less active scenes and yet keep the framerate consistent across the whole animation is pretty dang appealing.
I once did rotoscoping with morphing shapes rather than raster drawings, and that allowed me to skip frames, but the workload ended up being about the same as rather than drawing each shape, every keyframe that I animated had to have the pre-existing shapes morphed manually into the shapes of the next keyframe, sometimes the object moved too fast or at an inconsistent velocity thus requiring a lot of correction, and new objects that popped up had to have a whole new shape drawn in and manually morphed, every shape doubled the workload required for the next keyframes. Sure I could animate at like 12 per second and interpolate it as high as I wanted, but the in-between frames got really ugly going past 24 fps.
I have used this for a couple of days.The main two advs are fast speed and low gpu memory usage.
Are the results as good as DAIN ?
@@ifanmorgan8070 It faster than Dain but Dain has better result
@@faizyunus6618 Thanks. I think for my purposes (home movies) the RIFE is good enough. I honestly could not see a difference between them
@@ifanmorgan8070 For movies Rife is good enough but for anime DAIN is better 😁
Hey I need help, I’m trying to use rife but when I interpolate videos it only gives me the frames as Images not as a mp4. Is this how it suppose to work?
For the given time,it is more than okay with higher resolution.I will check this one for sure.Thank you for the video.
well thank you for this quick overview. it worked for a smooth camera motion from a Blender animation. finally i got this slow mo working
FOR ALL GPUUUU
DAMNNNN YEEESSSSSSSSSS
I had been using Hybrid, but this tool seems better and faster.
@@2BsWraith i wish i have good pc
@@pops7249 If you cant run RIFE then try hybrid, i believe it's easier on the system and compatible with intel and AMD cards as well.
@@2BsWraith can you send the link for it?
@@2BsWraith That's all i need to hear :D
you deserve more subscribers, brother
The only problem with Flowframes is you can't specify your output frames yourself, you need to choose the ones proposed on the list
Thanks a lot
Already test it, works good!
Underrated video, thanks mate!
Quick question, where do you learn/find the papers you talk about?
I've noticed that with all of the tools I've used so far (dain, cain, and rife), they still sometimes have trouble with things like text on top of a moving object. Sometimes it treats the text as being a part of the object and warps it as the object moves.
Noted, thank you for your info sir
So if using this for 3d animation rendering (as a time saver), would it be better to leave motion blur off?
Hi, 2 years later, is RIFE the best available option for frame interpolation?
watching high-fps anime on the high-hz displays...
LETS GO
THIS! I NEED BUTTERY SMOOTH ANIMATION AND MOVIES/SHOWS
@@MultiGotch how to ruin animation 101
@@Midrule depends on preference tbh. I prefer anime interpolated with rife, because I get motion sick with the juttery movements in 24fps after awhile
Learn so much~ thanks😃
Any update on higher achievements on Dain or Rife (now known as Flowframes)? - Paul
1:40 When the imposter is interpolated to 60fps 😳
Sus
among us is gay. overrated. never played it. never will.
@@intelgen7860 bruh you can't say a game is gay when you haven't played it. Just abouit discounts your opinion on it instantly
@@wilbofreeshan6340 Not really. I've seen enough to know it's transgender.
@@intelgen7860 Why do you keep calling it things like gay and transgender? Have your opinions on the game sure, but I'd be careful on the way you phrase it.
Trying to use the (flowframes) experimental interpolation, but 8gb vram isn't enough for 1080p, and it only supports Nvidia. What's the quality difference between normal interpolation and above?
Is this a live application? Like instant use of motion interpolation like tv? Will it work with browser streams?
was about to buy any nvidia card w/ cuda v5+ cores, man, you saved me from unnecessary spendings
Wish I'd found this before starting a DAIN job. 7 hours for a 3248×2736, 67-frame, 15 to 60 FPS interpolation of an object rotating in stop motion. Granted I only have a 1050Ti, but whatever. Still preferable to doing the extra 300% of picture-taking work with my particular setup.
Let's say that I render 15 fps and I render a 30fps depth map image sequence, can use it to get a better result out of rife?
DAIN eat much vram and slow, but i'm stil impressed.
if you value time use RIFE
The issue with for example AOT is that the framerate of the background does not match with the framerate of the foreground.
This is usually a no go in frame interpolation. To get an actual clean result you need to split those two and do it seperatly. Also you need to do some artifact corrections frame by frame.
To interpolate the composition and expect things to be 99% clean without needs for correction is just... cringe.
I dont know if there will be a work around in future for such things, but Dain/Rife is imo not ment to be used on a endconsumer level, where you put 30 mins of film in it, click a button and fin, but within animation/film studios, where they are prototyping the scenes themselves.
So, how do I use it on Colab again?
Im not the smartest lol
what a time to be alive!
Is there any frame interpolation app for m1 mac gpu?
I wonder could this be used in video games to bost fps similar to how the RTX AI works
does i get it right if i have an error of ssl/tls chanel in Flowframes - that mean i can not try to dain or rife?
So does this allow me to produce a new file that's smoother, or is it just for viewing in realtime?
What a time to be alive
I still think DAIN is the absolute King, only huge drawback is the processing time. My brain is kind of seeing an elite "soap opera effect" when looking at RIFE processed videos
I dont have nvidea device and i have a low cpu low end pc ..i tried this flowframes and it doesnt even render and causes too much error warnings
thanks for this special video
In 3D motion graphic CGI, should you render the frames with or without motion blur if you want to boost the FPS? Which would yield better results? I haven't found any answers regarding motion blur on/off with DAIN/RIFE/CAIN.
You never really want motion blur, because it's really just an artifact masking technique. More clear frames will be a better artifact masking technique. So if you have a 30fps video, putting it at 4x and watching it in 1x or 2x speed will be more visually appealing, because you get at least 2-4 frames between every frame when viewing, which smooths out the video. If you just add blur to the frames, all motion will be blurry, especially when slowed down. The only time you would want blurring is if you have video with constant erratic movement that makes path prediction difficult to the point it has tons of artifacts.
can i ask if rife can convert 60fps of 4k reso video ?? i have rtx 3070 laptop ??
Can you combine all AI graphics into one system. Continuous graphics enhance into very very detail.
do you know if the FR can go higher than 60fps?
Using the flowframes I do get Error "Your GPU run out of VRAM!!.." after trying to use like 1.8GB of GPU ram ...no idea why but I do get this tried different settings limiting the max video input size but still the same pff
Love. all. your. videos.
I cant figure out how to use this thing. it just kept downloading model file unet.pkl over and over again for hours.
what about GPU requirements? Dain is super demanding when it comes to vRAM
i must say that i prefer DAIN with thier own Blur settings like the area and threshold config for better smooth movements of the background, even when RIFE has a much faster processing of all the rendering it still doesnt look as expected since there arent very much options in RIFE compared to DAIN
gonna boost the brain of this person cause alot of the artistic vision is lost when you boost legit everyvideo like some videos are rendered in 23 fps for a reason
Wait, so if I got it right, this is not interpolating in real time? Do I have to "upscale" to 60 fps before I watch a video file? Or does it work in real time?
Real time means the processing time is similar to the original length of the video,with a power gpu of course.
Inspired me to start my own channel
i dont know, im render video 30fps to ke 60fps with subtitle but subtitle Little Flashing !!!
i hv question how to render with subtitle no flashing ?
I kept getting this error.
RuntimeError: unexpected EOF, expected 1826598 more bytes. The file might be corrupted.
Can i use this progam in a phone?
does DAIN or RIFE work on a mac m1 or m2?
When used dain it took my cpu to 100% usage and i got a "warning cpu usage" while in the app it shows the device to use is the gpu! whats wrong with dain?
please help me . Its keep saying Download failed during thr interpolation
Is there anything similar that works on Apple Mac?
Do you know how to install the program?
Tutorial on how to use rife?
Question about frame rate capability. What is the lowest you can go? For example, if I wanted to convert 15 to 24, can it do it? I don't need anything above 24. What I'm trying to do, is smooth out camera moves.
You can double it to 30 fps and then use openshot editor to export it at 24 fps.
Is it good for fast Anime edits?
Is there a way to use flowframes on iPhone ?
a vulkan error occurred during interpolation
How to fix this ?
its cool but nothing can beat that 3D depth awareness in DAIN
Does rife work good with motion blur? Dain app didn't
Where will be the output file of colab
It sounds like the creators of RIFE needs to use more frames to deal with quick motion. They already have the core so theoretically it shouldn't be difficult to do.
The rife Google colab has an error please help
RIFE is better than Hydrid ? 🤔 I have been using Hybrid to upscale movies to 60fps
I don’t think the technology will ever be perfect but it has potential as a tool to speed up animation work, the ai creates the base image and then an animator goes in to patch up the artifacts that the ai produces rather than doing it frame by frame. I don’t see it replacing parts of post production, rather speeding it up.
How to use this can you tell?
Hey, Nice Video. thx for the Information. I got 2 Questions.
1. shell i wait for an update for this programm, so it will work for anime aswell?
2. Do you think an Intel 630 grafic card is enough for this master piece, or do I have to get a new PC/ Graphic Card? :D
Hace a nice day
any ideas for mac users?
Can we use face frontalization
I wonder whats the newest algorithm right now.
Why not revise the interpolation algorithm into a plugin compatible with a variety of animation workflow platforms to be able to interpolate various animation sequences more efficiently?
The real value of this algorithm lies in what can be done with it right this very moment.
yeah but there's literally an option on dain to turn off the depth for 3D media if you want
which mobile version is there or not?
Well yeah this exact what I saw with DAIN. For scenes with high action and fast frames you are getting artifacts and for scenes with low action it looks smoother but is unnecessary because you dont really need more frames here.... . So basically it is a lost-lost situation... .
fast frames dont give me artifacts tho?
Amv editors bout to go crazy
tried this in windows 10, had either crash, every second frame black, out of memory error, nothing worked , just amazing
cuda wont work it keeps sayin ran outta vram even tho no other program is using my 3060
Do you think anything like this could apply for gaming in the future? I get that it's just for video interpolation and it needs a file to convert to 60fps but Maybe for things like game capture software that could smooth out the 30 fps games of last gen since sony and xbox only made like 15 next gen consoles.
Also great video. It was recommended on a digital foundry video about why destiny 2 can't run at 60 fps on ps4 pro using examples of frame interpolation. I was also looking for anyways to perhaps mod spiderman ps4 or miles morales to 60fps. Obviously no luck there but I was hoping frame interpolation could probably be an option.
Frame interpolation can already be done in real time, but it's a lot less refined looking. Something like this would not be good for smoothing out console frames, because it takes as much or more power than the game itself.....so you'd be needing more than twice the graphics power to turn 30fps into 60fps....which means you might as well use that power to get more gaming fps instead. On top of that, since games are interactive and rely on input, if you ran a 30fps game at 60fps visually, you'd still have the input lag of 30fps, not to mention since it's interactive, frame trajectory prediction wouldn't work without a second frame as a reference. How these work is they look at the differences between a first and second frame, then interpolate a third frame in the middle. With no second frame to interpolate, there is no prediction, so not only would you have a 30fps input lag, you'd also have a 3fps delay on top of that, or about 100ms visual delay between actions, which would make the game even less playable than 30fps without interpolation.
@@peoplez129 "but Maybe for things like game capture software that could smooth out the 30 fps games"
He never talked about interpolating frames in real-time gameplay, he was talking about game recordings.
Something similar to DLSS but for frame interpolation could be cool. Imo the problem is that games are not just seen like videos, but also felt, so if the motion is wrong it would be a lot more jarring
Let's teach IA to convert 3D anime models into 2D to fill the gaps in the interpolation frame job.
Don't know anything this guy said but I love it.
give it 5 year and an ambitious studio whos willing to incorporate this tech in their design process, and anime will change forever.
anime should never go 60fps
@@entsu88 true.
@@entsu88 But it wouldn't harm for anime to be 24fps (since they animate at two's)
@@mrlightwriter Do you know anything about anime? Anime is not constantly animated in 2, sometimes it is in 1, 3 or even 4 and that's because the animator purposely decided to do so. More frames is not better animation
@@guancho9142 Yes, I know the difference between the different types of animations in anime. I was generalizing. In a longer answer, it wouldn't harm having the possibility of increasing the fps of animes (for the ones who want to see more fluid animation). Of course, animation in 1, 2, 3 or 4s is related to artistic choices and/or time and money contraints, and it should continue to be produced that way.
Wonder if you could use this to enhance fps of games in realtime given a velocity map and exact coordinates of objects in each native frame.
No you can't that's not how it works...
you can use this on games (but its better to use this in live action or camera caught videos rather than minecraft for example
Good video
Me transforming and watching maybe all of jujutsu kaisen episode with 60fps: *"well hello there"*
What about SVP?
'Just a few more papers down the line' lol yeah 2 minute papers. GANG GANG.
there is nothing on the link
i use shotcut and i just changed the GOP from 300 to 600 and its actually smoother :/
Topaz is good ?
How do I use this?
Is this available for mobile
Seems like RIFE supports at least 3840x2160 input now... I tried this with RIFE 3.1
3 years later we have Flowframes... 🤯
Looking closer, RIFE could not handle motion blur quite well sometimes.
I wonder if in the future, hand drawn animation (which drawn in 24FPS or less) that is AI interpolated to 60FPS would be standard in anime industry.
It won't, because it looks bad
@@Mqstodon i said "in the future", the ai technology would be much better by then... And perhaps there will be specialized ai for hand drawn animation
Oh yes, fuck timing, fuck intent, fuck the talent of an artist. Pls i WanT mORE fps 🤡
@Ekklipse What noodle video? Are you unironically using snowflake?
Also, it's fine for to be an experiment but animation being produced this way is bad for so many reasons.
@@guancho9142 Stop lying