Thanks for adding this tutorial, really glad to see an example of the usefulness of this tool. After following the directions I wasn't getting any useful data when my Copycat was done processing. So I went and listened once more and noticed something that you said that you actually didn't do in your script. In the first Remove Node (connected upstream from Copycat 'Input') you say to Keep the RGBA, but in your Node Graph you actually only kept the RGB. I think that's why I wasn't getting any results the first time through.
While the idea of AI assisted roto is impressive, you could have picked a trickier shot for demonstration. The guy nearly moves, mocha would take seconds to track.
@Behram Patel I agree with you. He does say though that this approach pretty much saved the entire project and at some point there is a tricky shot with Gozer that he says he used copycat to extract a matte for. Cheers!
It definitely does work! I used it for the whole entire project. Some shots the actors were flying around on harnesses, landing and doing flips then taking off again... It was crazy. And copycat managed to more or less nail the entire 5 minute sequence. Seriously impressive!
@@AlfieVaughan That's great to hear! We are still stuck with Nuke 12.7 where I work but an upgrade is on the way. Looking forward giving it a spin. Cheers!
Wow this is awesome. I had a very complicated project almost 3 years ago that required roto for 2 actors and each of theyr body parts including many objects for 2000+ frames nd a very short deadline. Was a cartoon conversion from real footage to 2D. I had to write a python AI program to run on 3 high end nvidia gpu's of that time and even after letting it run 20+ hours over a weekend on 4k frames it had a lot of trash in it and was never so clean like this, but was usable and better then nothing, was 70% there, so i had to literally paint each frame to get the trash out. Was crazy work and i wish i had this tool. Is incredible how the technology evolves so fast. Resolve 18 just came out and the Ai roto in it is also extremly impressive.
Somethings in Nuke are just easy while other simple things are nearly impossible but with Mocha in After Effects this can be done in seconds. Mind boggling.
Sure, this specific one is easy. But I just picked this as the example. A lot of the shots were hundreds if not thousands of frames and much more complicated. The reason I used copycat for this is because once you train it on one shot it can do all of them as long as they're similar.
Can we address the fact that this is a production level with actors, a studio BUT just one artist in post production !? Can we replace producers with AI already ?
I really don't understand why this isn't keyable - the greenscreen looks absolutely fine, just need to remove markers near the actors and use a degrained plate to stop the buzzing edges. It honestly blows my mind when compositors don't know the most basic (and advanced) art of keying - NOBODY gets a good key with 1 keying node, it's naive to think you could. And having worked on many many large productions over the years, we would NEVER roto a shot like this, that's not normal if its on a decent green/bluescreen (such as this), people just need to learn how to key.
If it took 40 min for copycat to produce an alpha for a 66 frame shot, how long did it take for that 2300 frame shot?? I can definitely imagine this being very useful, thanks for the tips!
For the really long shots I left it training overnight for about 9 hours. I don't think it needed all that time it was just a good opportunity to leave it running for ages. Most the time it was useable within a few hours :)
Hi! Of course you can! Feel free to share your thoughts in our community once you give it a try too community.foundry.com/discuss/forum/197/nuke-non-commercial-users
@@FoundryTeam Thank you for supplying the non-commercial version. I have large time gaps in working on Nuke and although I have been using it for 12+ years I find that downloading and practicing for a few days before I go back into a studio brings back all that muscle memory. At 68 the little grey cells need a bit of a kick occasionally.
is there any difference if we do key for a single frame and use that to generate matte and do roto for the single frame? I appreciate if you would have used copycat to generate matte for the Screenleft character as that's the tougher task in the shot
I don't think so - as he mentions the copycat node is interpreting the input as Images rather than roto shapes, so I believe you could even have different numbers of roto shapes in your reference frames and it will be fine.
@AlfieVaughan, Hey, I'm prepping for some post work and want to use a similar approach. In my case it's a rigid bodied object in a roughly similar position in varying lighting conditions. Do you need to do per shot roto + training? Or can I do 2-3 frames on let's say 10 shots and use those as my ground truths and use one inference for 60+ shots for example?
You can yes! As long as they're the same sort of shot then you can use an inference node from the training on other shots and it will do the same thing.
It does a pretty good job. You can either roto the motion blur in the reference frames by blurring and feathering the edges of the mattes. Or what I sometimes did is run the hard roto through a 2D motion blur node to fake it a bit. It depended on the shot which one I chose to do. Hope that helps!
Hi! yes, copycat can be used for any image to image process. So if doing paint cleanup of markers, or things like dust, scratches etc, you could give copycat a few frames of the cleaned-up version and train it on that to have this applied to the rest of the frames in the shot. Like with any machine learning the quality of the data you put in will help determine the quality of the result you get out.
If anything, it will create even more roto/comp work because you can now crank out these shots in a fraction of the time. So you'll be able to get even more shots done with less work.
For more Nuke tutorials, check out: learn.foundry.com/nuke
For Nuke product information, go to: www.foundry.com/products/nuke-family
When is Nuke going to run natively on Apple Silicon, when so many other software developers kicked in so quickly. Seem to be missing the game changer
Thanks for adding this tutorial, really glad to see an example of the usefulness of this tool. After following the directions I wasn't getting any useful data when my Copycat was done processing. So I went and listened once more and noticed something that you said that you actually didn't do in your script. In the first Remove Node (connected upstream from Copycat 'Input') you say to Keep the RGBA, but in your Node Graph you actually only kept the RGB. I think that's why I wasn't getting any results the first time through.
While the idea of AI assisted roto is impressive, you could have picked a trickier shot for demonstration. The guy nearly moves, mocha would take seconds to track.
@Behram Patel I agree with you.
He does say though that this approach pretty much saved the entire project and at some point there is a tricky shot with Gozer that he says he used copycat to extract a matte for.
Cheers!
It definitely does work! I used it for the whole entire project. Some shots the actors were flying around on harnesses, landing and doing flips then taking off again... It was crazy. And copycat managed to more or less nail the entire 5 minute sequence. Seriously impressive!
@@AlfieVaughan That's great to hear! We are still stuck with Nuke 12.7 where I work but an upgrade is on the way. Looking forward giving it a spin. Cheers!
@@doginconfusion me too, there is many plugins still not up-to-date that the company usually used..I have to stick with nuke 12
Yeah I was disappointed also with the example used. Not challenging at all.
Wow this is awesome. I had a very complicated project almost 3 years ago that required roto for 2 actors and each of theyr body parts including many objects for 2000+ frames nd a very short deadline. Was a cartoon conversion from real footage to 2D. I had to write a python AI program to run on 3 high end nvidia gpu's of that time and even after letting it run 20+ hours over a weekend on 4k frames it had a lot of trash in it and was never so clean like this, but was usable and better then nothing, was 70% there, so i had to literally paint each frame to get the trash out. Was crazy work and i wish i had this tool. Is incredible how the technology evolves so fast. Resolve 18 just came out and the Ai roto in it is also extremly impressive.
Somethings in Nuke are just easy while other simple things are nearly impossible but with Mocha in After Effects this can be done in seconds. Mind boggling.
Sure, this specific one is easy. But I just picked this as the example. A lot of the shots were hundreds if not thousands of frames and much more complicated. The reason I used copycat for this is because once you train it on one shot it can do all of them as long as they're similar.
Mocha works for Nuke as well
Thanks for sharing your knowledge...
SO DOPE! ⭐
thank's for making this video !!
have you tried white balance the footage before keying,might have helped
Perfect, quick and precise explanation. Thanx!
It will be great to have the footage to practice copy cat
To me this looks like perfect task for Mocha.
Can we address the fact that this is a production level with actors, a studio BUT just one artist in post production !?
Can we replace producers with AI already ?
I really don't understand why this isn't keyable - the greenscreen looks absolutely fine, just need to remove markers near the actors and use a degrained plate to stop the buzzing edges.
It honestly blows my mind when compositors don't know the most basic (and advanced) art of keying - NOBODY gets a good key with 1 keying node, it's naive to think you could.
And having worked on many many large productions over the years, we would NEVER roto a shot like this, that's not normal if its on a decent green/bluescreen (such as this), people just need to learn how to key.
Clean-Plate + IBK Gizmo would have done the job. :D
Thank you the good explanation.
Brilliant! I'll try this tomorrow on a green screen plate that I've been having issues with and was dreading doing a heap of roto on
If it took 40 min for copycat to produce an alpha for a 66 frame shot, how long did it take for that 2300 frame shot?? I can definitely imagine this being very useful, thanks for the tips!
For the really long shots I left it training overnight for about 9 hours. I don't think it needed all that time it was just a good opportunity to leave it running for ages. Most the time it was useable within a few hours :)
@@AlfieVaughan how do you feel copycat compares to runway's tools? I'm not sure if you've talked about it online somewhere.
Where can I find the ColourEdge gizmo? Please, it will come very handy.
Did you find colouredge gizmo ?
@@ivaylo.paskalev he didn't find the colouredge gizmo
@@TheNjordy is there anybody who found the colouredge gizmo?
What Ghostbusters project is this? It looks great!
Can I try this in the non-commercial version? This is insane!
Hi! Of course you can! Feel free to share your thoughts in our community once you give it a try too community.foundry.com/discuss/forum/197/nuke-non-commercial-users
I have been practising CopyCat on the non-commercial version ... so yes.
@@FoundryTeam Thank you for supplying the non-commercial version. I have large time gaps in working on Nuke and although I have been using it for 12+ years I find that downloading and practicing for a few days before I go back into a studio brings back all that muscle memory. At 68 the little grey cells need a bit of a kick occasionally.
yo. tnx!
whats the seventh doctor doing here?
"Unimaginable power, unlimited rice pudding!"
Fantastic explanation, thankyou
Thanks!
Is the production video somewhere we can see ? This looks great !
Unfortunately not. It was for a live experience project so I don't think there are any videos online
is there any difference if we do key for a single frame and use that to generate matte and do roto for the single frame?
I appreciate if you would have used copycat to generate matte for the Screenleft character as that's the tougher task in the shot
I would think it wouldn't make a difference, as long as the key is clamped to min 0 and max 1 values, the end result should be the same.
You can do that too. When I show the bigger shot at 2:50 you can see I'm using a mix of keying and roto to generate the mattes
@@AlfieVaughan Did you tried copycat with the other character in the FG?
Mocha can do that in seconds.
dont you think it can be done with a planar tracker . Cheers!
At 5:26 you say to keep RGBA though I‘m sure you only mean RGB
Does the roto you create for each reference require the same amount of anchor points?
I don't think so - as he mentions the copycat node is interpreting the input as Images rather than roto shapes, so I believe you could even have different numbers of roto shapes in your reference frames and it will be fine.
Nope it just uses the alpha! :)
Is there a big improvment from copycat in 13.1? I have tried it on a pretty simple shot and it produced totally unusable results.
We are still on the stoneage, speaking as as Vfx artist here
Someone need to fire the VFX supervisors on that set lol...
Is that shot you were working on at 2:57 from Ghostbusters Afterlife!?
@AlfieVaughan, Hey, I'm prepping for some post work and want to use a similar approach. In my case it's a rigid bodied object in a roughly similar position in varying lighting conditions. Do you need to do per shot roto + training? Or can I do 2-3 frames on let's say 10 shots and use those as my ground truths and use one inference for 60+ shots for example?
where do I find colouredge?
Can I use the reference frames from one shot and use copycat with those reference frames in another shot? How would be the process?
You can yes! As long as they're the same sort of shot then you can use an inference node from the training on other shots and it will do the same thing.
how good does this work on shots that have motion blur? All the examples I have seen of it always has fairly static subjects.
It does a pretty good job. You can either roto the motion blur in the reference frames by blurring and feathering the edges of the mattes. Or what I sometimes did is run the hard roto through a 2D motion blur node to fake it a bit. It depended on the shot which one I chose to do. Hope that helps!
can copy cat node be used for paint-prep shots ?
Hi!
yes, copycat can be used for any image to image process. So if doing paint cleanup of markers, or things like dust, scratches etc, you could give copycat a few frames of the cleaned-up version and train it on that to have this applied to the rest of the frames in the shot.
Like with any machine learning the quality of the data you put in will help determine the quality of the result you get out.
Has anyone had issues with it saying "Empty frame range" when trying to restart it? Can't seem to crack it
I`ve got the same problem
Hello Alfie how do I contact you to ask question on application using copycat for project, best joe
Hey Joe. I guess here! What would you like to know?
This would be like 5 minutes of work with AE roto brush or Davinci Resolve's Magic Mask. This is too much work for automation..
Is that Sylvester McCoy?
It is!
I'm a year late, but finally someone pointed that out. Like was I the only one who noticed the 7th Doctor??
Roto is another useless job AI will obliterate...easiest thing it can do
Will it end roto jobs? Plz answer
no
@@igormajdandzic7587thnku
Not now, but may be in future 🤔
If anything, it will create even more roto/comp work because you can now crank out these shots in a fraction of the time. So you'll be able to get even more shots done with less work.
Adobe roto brush is killing them already.
boring 1 min done in after effects.
I'd be interested in how long it would take you to do one of the 2500 frame shots of a woman flailing around in a harness in After Effects
@@AlfieVaughan not more than 20 minutes if it is similarly shot.