I'm honestly really grateful for this video, I remember a while back I modified your original FFmpeg script to use x265 instead of x264 with limited success. Although I got the script working it wasn't practical with the absurd amount of time it would take to compress footage, this on the other hand just burns through my footage and is 6x faster on my system with close to similar visual fidelity. Thank you so much!
whats the typical bitrate for the compressed files? i usually encode in handbrake with cq 0 and it turned a 200+mbps 20gb video to a 20 mbps 2 gb video.
Dude, you're the best. I always love your uploads but then again I'm in the center of your target demographic so I guess that makes sense. Keep up the great work!
When you're selecting those files to delete you can probably do it a lot more easily by sorting by Type, shift-select the block of old files, then sort back by filename to look for empty/bad files and correct from there.
I bereave a full tutorial is needed for this video since not everyone is script savvy. i can tare down and rebuild computers all day and fix almost all software and hardware related issues but scripting is a whole different knowledge base and skill set. please make this a tutorial video :)
@@meolessi5227 The description links to ffmpeg.exe - you can't run this script without ffmpeg, which he is not going to host as it is likely copyrighted. All he's automating is an ffmpeg command, you could simply strip that out of his script and as long as you change the input file ("%%A") and output file ("%%A~dnpA_CRF%ffmpeg_qv%_HEVC.mp4") parts it should work manually.
After a bit tinkering, I think I fixed the script to work with AMD cards. Only did a few changes to line 33 on the script and it seems to be working and doing the same stuff without any problems now. Changed hevc_nvenc to hevc_amf and added -gops_per_idr 1 after to solve the seek problem that happened in videos and also changed "constqp -qp %ffmpeg_qv%" to "cqp -qp_p %ffmpeg_qv% -qp_i %ffmpeg_qv%". Also had to change -pix_fmt to nv12 as was recommended during the encoding, but that's just optional and can be changed to something else.
Hello, thx for script. It us working for amd, but from what I see Nvidia get much smaller files with better quality than amd.? I get a bit washed out colors and size for example 1.2GB from 14.7GB file (from resolve ) where I expected around 600MB max. It's sad. I have rx 570 8GB. From what I understand, amd could fix this with just some software patch but they have no plan to do it in near future.
I strongly advise you to NOT USE AMD hardware decoding, it is the worst technology (literally) you can use if you care about quality OR /AND if you care about file size. This is not a secret this is a well-known fact. Very easy to google.
Hey Eposvox great video just got a quick question, at 10:00 you said that the encodes come in 8-bit 4:2:0 but according to NVIDIA your cards should support 10bit 444 and lossless, so why not use that and save more space? developer.nvidia.com/video-encode-decode-gpu-support-matrix
It does when compared to raw doesn't it, I don't have alot of experience with video compression but from me messing around lossless files definitely where smaller than raw with no data loss ( Couse you know lossless 😂) when I encoded. Thought of course they took way more space than "visually lossless" which I ended up using because they where nothing important, if you need all video data I don't get why lossless wouldn't save space on drives compared to the high bitrate raw.
Oh and I'm talking specifically about the black magic files you said you might need to color correct later and such not the normal ones. And thanks for replying btw 😁
@@amrnassef5344 Ohh I see. Yeah I need to work on that, but that requires changing which encoding I do based on the source which gets more messy. Something I'd like to work on.
@@EposVox Oh I thought you would have the raw files in a separate folder per project or something so you would just be able to specifie the codec in the script (another version for lossless) and run it on the folder containing the raw files. But if you have a different workflow then that changes things of course, though I do hope you look into this in the future and do a video about it as it would be really useful and interesting. :D
Have you ever tried using this on AviSynth scripts? Wanna make sure I'm doing this right, it seems to be using more of my CPU than my GPU, if it's even using my GPU.
You compared x264 -crf 14 with nvenc -qp 24 as if they were interchangeable. This is not true, x264's crf mode is substantially different from nvenc's qp mode. Your results will have a substantially different quality profile. Which in your use case may not be critical, but in other scenarios this difference in how these modes assign quality and file sizes in different ways are quite substantial. Also please remember that the licensing situations of h264 and h265 are very different. The additional fees and more complex licensing requirements for x265 almost disqualifies the format for most scenarios that not either strictly private use or very big scale commercial.
You're right about the CRF parameters, but I want address the licensing. So, while the HEVC codec does limit playability and editability in mainstream NLEs and players, I still think saving in that format is safe for the long run because we haven't yet hit the best compression quite yet, which means that one should use and make use, of what's best now. Even if the licensing is exclusive in the future, I'm on with paying once to be able to open a work of video footage that I can still reconvert for viewing pleasure. Yes, for streaming, not the best, BUT I want to archive, and so I've actually changed to HEVC encoding a while ago even before this video was posted on the same premise and results. So, I applaud this move and approve his message. The script was actually very helpful.
Why am i only seeing this now thank you UA-cam algorithm and epos vox!! I have 40tb+ worth of client files that i need to compress so i can start re using hard drives to save money, and this is a gamechanger. Thank you
I am "only" getting about 1.5x speed on 1080p footage, anything I can do to make it quicker? or is this just how it works with certain files etc?(1080ti)
Did you update it perhaps after the rtx ampere GPU's came out? I have a 3080 and I'm wondering if it's worth to update the ffmpeg script. I don't have the skills to update it.
Ok, the topic looks interesting , and now, I'm a novice in this. Where do I start? I have some .mkv's than I'd like to compress, so how do I use this script? Thanks a lot for help!!
For anyone who isn't an expert and are trying to figure out how to do this: 1) download the "crf24_hevc.bat" file from EposVox's video description link 2) go to ffmpeg.org, click download, click on Windows, and click on "Windows Build by BtbN" 3) download this file: "ffmpeg-N-100554-g89c9c42c5b-win64-gpl.zip " 4) extract the "ffmpeg.zip" file, and look for "ffpmeg.exe" 5) move the "ffmpeg.exe" and "crf24_hevc.bat" into a new folder, and move the video file you want to convert in that folder also 6) double click on "crf24_hevc.bat" and it will start encoding Some things to note: 1) you may need to disable your virus scanner temporarily since mine prevented the .bat from running 2) you need an Nvidia video card for this. If you don't have one, it won't work Good luck. Mine worked fine with the instructions above. Thank you EposVox for this stellar script as I'm seeing a noticeable difference in file size and the quality is identical!
man you seem to be the handbrake/obs/premiere wizard. how tf do i fix my shyt? i'm recording with a focusrite scarlett 2i2 2nd gen. in obs I record with mkv format. when video is brought into handbrake i encode or whatever to mp4. So I have 3 files when it's encoding in handbrake. game (video) game(audio) and mic (audio) if i import a video in i've converted then cut it in 2 different areas and try draggin another one on top and cropping it my playback doesn't work and it just skips all around like crazy or locks up. i have constant bitrate selected in handbrake or whatever.. not sure what im doing wrong. can you help me out?
Please do note that QP24 and CRF24 are different. CRF will use different QP values for each frame depending on what is deemed necessary to reach the specified quality level.
Quick question - any particular reason to use '-rc constqp -qp' over other possibilities? Many say, including slhck site, to avoid -qp and opt for -cq (similar to -crf in x264). Quality with -qp is guaranteed, however, it can use more space, so savings with -cq could be higher. I am still testing hevc in order to see what is best for archival purposes. BTW, you can omit both -map 0, the default map behavior is to select best video+audio, or in case of single video/audio track it is not even needed. Then for -b:v you can simply use 0, 'K' is not needed, ie. '-b:v 0'.
1 - no particular reason, just what I was most familiar with. 2 - no, those cannot be omitted for people with my kind of workflow, they're required to safely keep multitrack recordings in tact
I see. I did some SSIM and PSNR tests, '-qp' definitely preserves quality, and yes, it uses slightly more bits, but there is no drastic increase compared to '-cq' with the same value, so both are great. Observations around '-cq': - it seems to default to '-rc vbr' and it had slightly better score than '-rc vbr_hq', which some people recommend, almost no difference in metrics - '-preset slow' surprisingly lowered metrics a little, but not perceivable as well. In short, it seems nvenc is smart enough to use best defaults in given scenario, and '-qp' or '-cq' is enough. Same values provide similar results, both require '-b:v 0', '-qp' seems to default to '-rc constqp'. I used this to test: # ffmpeg -i new.mp4 -i original.mp4 -lavfi "ssim;[0:v][1:v]psnr" -f null - ... and this to see all options: # ffmpeg -hide_banner -h encoder=hevc_nvenc FFmpeg was latest build from Zeranoe site.
These are great settings for ffmpeg. Instead of using a script you can easily automate everything with tdarr (I run in a docker container) which will monitor folders and automatically fan in and fan out GPU encoding and give you all the stats you want on storage saved.
EposVox, is there a version of this script that uses AMD HVEC? You have sung its praises lately and I'd love to try a version of this script with my AMD hardware.
AMF's ffmpeg implementation is a tad confusing atm and I haven't gotten CQP working in it yet. Theoretically just swapping "hevc_nvenc" to "hevc_amf" would work, but it throws errors for me. Will update if I get it sorted!
i seen a thing but i don't know if it's normal or not , in your recent bat , you use the mp4 format , everything work , then when i change to MKV , it does loop , something is wrong or it's perfectly normal ?
Hey Epos, have you considered using LTO tapes for archiving your cineform/massive video files so this way, if you absolutely need a clip and in it's original quality, you've got it in ~24 hours? That's personally what I'd do instead of deleting files (I'm running ~100TB locally and over 1PB in the cloud/colocated servers)
EposVox I have a question I use OBS studio to record with and I have recently started recording in lossless quality do you know any file converters that are out there that I can use to convert the file to MP4 because the editor that I use does not support Avi and that's the only way you can record lossless quality is Avi right?
I don't understand the map-part: 0:a and 0:v. The a stands for audio and v for video but what does that 0 mean? I quickly looked in the man-page of ffmpeg (I searched for map), it doesn't make it much clearer. It is not relevant for using it but I am curious what that number stands for.
It's for multiple tracks. I mentioned that in the video. I utilize multiple audio tracks in OBS and this makes sure that they're all mapped from source to the new file.
I'm using Handbrake to do the same thing. I tried the script, but ffmpeg complained about a missing a library, which is odd. But I got Handbrake to do the same thing. I am currently chewing through 64 Blu Ray 1080p movies, encoding them to H.265 HEVC using my 8700K and nVidia 2080 ti. It takes about 25 to 30 minutes per movie to transcode them. These files took up about 1.7TB of space on my NAS, but based on the compression so far, I expect that I will save about ~60% of that. So the end result is that I should save about 700GB across all 64 files. Not bad!! Plus, I can now direct play these to my Sony TV, which means the CPU usage for my NAS is about 0-1% while Plex streams them!! I'm happy!!
UPDATE: I just had to change it to output an .mkv file (since that is a regression for ffmpeg when the input is .mkv) and it worked. Comparing the output of that to Handbrake however, Handbrake is better for only one reason. Everything is the same, video quality, audio quality, except the ffmpeg produces a video image that won't scale to the size of the player. So on my TV or gaming PC, it is a small rectangle with black borders on the top, bottom, left and right. Whereas Handbrake scales the video to any size view port.
does plex work with nvenc/amf? I am planning to retire my current rig which has an old i3 to be a tv tuner/plex server/htpc thing but cpu encoding/upgrade isn't going to work for me or my budget. So I'm considering getting it a gpu to use for compressing & 4k decode. Should I use plex to capture broadcasts or have other software simply write the raw MPEG2/H264 stream to disk and process later?
Plex works fine* with h265. The asterisk is because lots of client devices can't handle h265. Plex will gladly attempt to transcode it to h264, but you need either a very modern CPU or something capable of doing it in hardware to transcode in realtime for playback.
I have a question, I use handbrake to convert footage from shadowplay so it's constant frame rate which is needed for Resolve. I use H.264 Intel QSV and the file sizes are massive. I want it to be lossless because it's before the video is edited, but my God, a 5GB file can become 30GB. Is there something I'm missing?
I wonder if this could be useful in cutting down Shadowplay footage in a reliable way. I don't generate much of a footprint with my recordings still, but those files can get out of hand sometimes, since they're recorded at 50Mbit on the old NVENC.
How are the quality profiles equivalent between h264 and h265? Would you get better compression with CPU encoding over GPU? I have so many unanswered questions because I'm currently re-encoding my GoPro footage
CPU-based encoding always results in better quality and compression rates (at least for the foreseeable future). Hardware encoders don't offer all the instructions a software encoder can offer which results in the losses. I only use GPU-based hardware encoding when rendering a rough-cut of a project and want it to happen relatively fast. When streaming, the GPU does the live-stream and my CPU does a local recording for better quality.
If you have an old Android phone you can save space by rooting it and installing a Magisk module (or just changing the "build.prop" file) to make Google Photos think you are using a Google Pixel and it will give you unlimited uncompressed storage. Then you just sync your footage with a cross platform app/program like Syncthing, and you can drag footage into a folder on your PC for example, and it will automatically send it to your phone first and upload it to Google Photos. I use it all the time on my phone and PC, works great
hey EposVox! I use this script religiously :) I was just wondering If you've made any improvements to this script after all these years ? Thank you very much
So I gave this a shot with my stream recordings. I changed the audio bitrate on your script from 384k to 192k (because that's the quality I stream/record). I tried a short 20 minute mp4 recording, before it was 858,370KB and after it was 1,657,192KB. It's just about twice the size as before I done this. What could be going on here?
Not sure if you still check the comments on this video but I tried this on some videos i recorded with shadowplay and while everything worked properly the colors on the encoded video weren't quite right. On the encoded version of the video I tried the colors seem a bit dull for example the reds are a bit more orange than red and the blues are a bit darker. I don't really understand all the commands in the batch file but I tried to up the cqp to 22 like it said in the script and while the file size went up the colors weren't fixed. I tested out encoding the videos through handbrake using nvenc x265 and it fixed the color issue but obviously a script would be way easier to encode videos than handbrake so I would like to use the script if possible. Do you know the reason why this happens with your script and if it's possible to fix?
Nice! AV1 Could be quick enough next year. OPUS is slightly more compressed with the same quality. AVIF has a potential for image compression as of now WEBP is quick enough. For lossless audio FLAC is an open option and there is also APE which can be set to slightly higher compression. For archival purposes of raw audio and images 7-Zip plugin MFilter by T4Shell could be used. Also Brotli and Zfast could be used for xml, html, etc. speedy text compression.
Great video, I'm back-archiving a lot of videos with HEVC now with a variation of your preset, it's mostly the same as yours and I'll likely use your script in the future. Question @eposvox, if you want to edit the HEVC converted video in a video editor, you would have to losslessly convert it back to x264, right? I can't seem to find any video editor that can edit it natively.
I found x264 at CRF 18 and veryslow to be imperceptibly close to lossless with really good file sizes. I use this for my Plex server to stream media faster and with less bandwidth usage. I wanted to halve the file sizes but I can't do it at acceptable quality. x265 at quality 22 looks alright in bright scenes but falls apart in dark scenes and has severe banding issues. 10 bit h265 is way better but not quite half the file size of h264. H265 is really not half the file size at the same quality. I tried over and over but h265 just shows more artifacts than h264. I wonder how good your stuff looks at CRF 24 because software x265 is crap at 24.
For dark scenes I've found you really have to tinker with x264 settings, 2 pass is best. For general video I've actually got NVENC settings that beat x264 in quality and size, but you may not find my quality level acceptable if you're having issues with 18. Using NVENC VBRHQ is essentially the NVENC version of 2 pass so even using 23 and bitrate of 1100 on 720P is nearly indistinguishable from source video except for really dark scenes and/or fog.
@@glenwaldrop8166 q18 h264 looks nearly perfect. The difference is tiny and only visible on a large screen that has bad black levels. It is most visible in dark areas where you can see blocking but h265 is way worse in dark areas. H265 produces different artifacts like banding and weird noise blocks. The banding is extremely noticable if you encode at 8 bit. Some of my movies are under 2 GB and still look good in h265 but the quality drop is a bit larger.
No idea what the capacity/age of those harddrives but i'd assume they're 8TB each if these were built post 2015 so why not just combine the 2 and get a bit of extra storage? All in all though it's pretty rare to see a channel with only 240k subscribers spend $2500 on storage. I assume most of them are sitting on maybe $300 of storage for their videos at most.
Are you using HEVC? X264 and HEVC CQP/CRF values work a bit differently. But no, I haven't. If it looks good enough upon spot checking and running through UA-cam compression again, it's more than good enough for this purpose. I'm not aiming to squeeze out perfect duplicates, just "good enough". That being said, most of what I have is screen capture, which compresses well anyway, and uncompressed DNxHR/Cineform/ProRes files which are going to squish down a ton.
@@EposVox Yes, hvec and same hardware. As long as you understand that it is not a visually lossless conversion, you should be okay. It sounds like you're doing this just for archiving footage you're not sure if you need any more, aka "rainy day" backups, so it's fine.
Awesome! Will check this out at home later, any idea if this work on AMD's codec? How will I change the script to AMD or if I wanna try run it on 265 codec? Thanks ☺️😅
Any script to convert h265 back to h264 for old TV? Also what about a scrip too for downmixing properly the audios 5.1 to stereo 2.0 again for some old TV , thanks.
I want to use this but keep getting errors before the video starts to encode,. Not sure if my files are in the correct folders etc??? Great Idea, but I'm totally lost.
could this be automated on linux for x265? i have an old HP DL160 G6 server and i've manually been needing to move files to and from it every time its done so just dumping a bunch of crap over and letting it run for the day would help
Hey ... I Love this, but i have (programming) issues: I Put your scripts/files, ffmpeg and my video (.mov File from Iphone) in one folder Got an Error after executing cfr24_hevc.bat ... Can you Help Me ?
Also found this error: [mov,mp4,m4a,3gp,3g2,mj2 @ 000001f13b20df40] Format mov,mp4,m4a,3gp,3g2,mj2 detected only with low score of 1, misdetection possible! [mov,mp4,m4a,3gp,3g2,mj2 @ 000001f13b20df40] moov atom not found C:\Users\Home\Desktop\Test\IMG_0005 2.MOV~dnpA_CRF24_HEVC.mp4: Invalid data found when processing input
@EposVox I have been using HandBrake to compress my movie collection. Everyone says that you should use your CPU and X.246 for the compression. I see that you are using ffmep which is the same as HandBrake but you are using the Video card to do the compression. I'm guessing you would not being do this if the compressed version was a bad quality. I can't see why there would be any difference between what you are doing and me with compressing a movie. Do you know of any settings that I can set in HandBrake to produce the same quality that you are doing? If I could same a bunch of time that would be wonderful. Maybe this is something you could make a video about. It would be nice if you could address the CPU verses GPU compression. Great video by the way. One comment. You can install Linux in Windows 10 (WSL). So you could write a bash script to do your encoding.
Well its actually pretty simple. CPU (aka software) encoding will always be more efficient compared to GPU, meaning same quality at smaller file size or better quality with same file size. The reason why that is, is because hardware encoding (GPU) isnt as flexible as a written software and does not 100% support all the features of the codec (for example nvidia h.265 encoder before 20 series did not support B-Frames, which is very important for good efficiency with hevc).
Tip: This doesn't work on 900 series cards, because they don't support 265 encoding. I just found this out since I wanted to try it on my laptop, as I'm not home to test it on my PC.
Does this work on .mp4 files as well? Tried doing this, and the result was getting a "compressed" video that was a larger file size. Went from 1.05GB to 1.08GB
@@EposVox Nice! I just tried this on a screen recording and unfortunately, I noticed a little artifacting on CRF 24 with handbrake. I actually did a test with x264 and x265 with the same recording and it seemed like x264 looked better strangely. There's some recordings that I don't have a reason to keep but will do anyway, so I will use x265 for that just for archival purposes. I'm actually doing CRF 20 on that just to be safe. The size is still a lot smaller than x264, so I'm still getting a lot of benefit! I'm not storing an insane amount of things like you are, but I can still definitely receive the storage benefits despite this.
Hello! And what about video metadata (for example created at)? For old videos it's important to have these data to be easiest to find and sort files. I saw that ffmpeg has an option to copy the original video metadata to the converted one. Thanks!
Have you tried doing the same on Linux? Actually to check if it would be faster with the same parameters despite the GPU drivers being considerate bad. I guess CPU wise it would have better performance (?)
Epos, just subbed......solid vids bro u nerdout so hard lol............do u hav any plans to do a vid on a 2070super gpu card combined with a 3700/3900 xt amd chip and how well it performs for streaming? If not would u?
Dam, quick reply......Badass bro, regards to my 1st reply keep seeing alot of problems for streamers in regards to 5700 gpus so for the sake of time im gonna purchase a 2070 super and a ryzen 3700
okey, so U cant record with the codec u used to compress the files? cuz I was thinking that this video explains how to record "lossles" high compressed video hm
@@EposVox I going to add this to my workflow, pop a new raid in and run this through my server this weekend. Thank you so much your channel has always been a go to place for process improvement for me.
This is awesome. I think somthing like this would be useful for me shooting footage in a t3i. How much do you think would have to be done in order to use this script on linux
The "%%A" / "~%%dnpA" variables aren't the same in Linux, but otherwise the normal ffmpeg command is the same. So just translating the rest from Batch to Bash.
I've been playing around with this script and different 'original' recording settings over the past 3 months and I find that if I'm re-encoding something at H265 CQP 22 or worse, you can get *some* artifacting but it's in very niche situations that usually happen with dark environments. I've seen blocking on steam vent effects in a dark area, or a face that passes by in a split second in the King's Row (night time) map in Overwatch. This is made *very slightly* more significant if you're running a garbage-in-garbage-out workflow where you already compressed the recording at a very high setting (like H264 CQP14, well past diminishing returns) but even CQP22 from a lossless AVI can produce some flaws here and there With that said, 'watching the video, fullscreen, pausing on busy frames and actively hunting for flaws' is quite different from how the content is actually watched, and for the amount of space saved, oh, it's absolutely worth it, especially considering that at this point youtube processing is the tightest bottleneck even at 1440p.
Just opened my 45 drives server today. I installed 120 terabytes and I never thought nine 14 terabyte drives wood look so small but next to 36 empty drive bays it's sad. First world problems
@@EposVox lol I've got lots of empty bays you're welcome to come by my office in LA throw in some drives, make a storage pool, and bang ! Free cloud storage. 2gig symmetrical fiber in it also helps. Speed things up
Yeh - hvec is a huge step up in power over 264. I use it for 12bit HDR content... Being a mad programmer I edit and master all my videos with ffmpeg on a Mac using zsh. Sadly, the Mac hardware hvec encoder does not seem to handle smpte2084 HDR, so it is a slow and laborious process... Anyhow - loved the video and thst you are getting the message out. ffmpeg rocks. Thanks for the vid!
@EposVox. Hi. Just curios if you've switched to RX 5700/5700 XT for this script, considering you've mentioned (ua-cam.com/video/Yi-_T3vsv-Q/v-deo.html) you could run 5-6 encoding streams simultaneously? Thanks!
For deleting those .mov files just search the folder for mov and you'll just see those .mov files and then simply ctrl+a and delete. So much quicker :)
Coulda, if I had the experience. Could've made it a full gui and something that ran itself for me once footage copies over while I'm afk - but I work with what I know how to do. And I'm a control freak and don't want something deleting the files for me.
If only I could use this for my 500gb+ steam library...But otherwise, very useful for the old bits of video here and there that take up way too much space!
I considered doing something similar last year. I set up a script that worked well, the quality was barely perceptibly different, with significantly smaller files (better than 1/2 size in most cases). However I came to the conclusion that it actually wasn't worth it - because the Electricity cost to re-encode was higher than the cost of just buying additional HDs. Obviously the maths is different if you have an ongoing cost for cloud storage, vs the one-off cost of local storage.
@@EposVox Not really - hard disks, even resonable quality ones, are cheap. I worked out it was going to take over a month to re-encode. vs the maybe £100 for a suitably sized HD. Now, at the time, I had solar panels and so free electricity during the day. I did have a plan to make a script that would run in the daytime only, and while I was out of the house. But in the end it just didn't seem worthwhile. Not everyone's needs are the same though
@@EposVox correct, although assuming a linear relationship between encoding time and space required - the maths still works the same. That's a big assumption though, and unlikely to be true at the higher end. As I say, different requirements for different people 🙂
@@zinsy23 Depends on viewing distance, display size and resolution... *I* notice it on my projects and setup. It's just a technical fact h265 got less details aka smoothens/blurs the image. Same reason your JPEG file got smaller when you add some blur in Photoshop "save for web" export dialog.
@@surject Is there a CR value that won't reduce sharpness compared to original & does that value still make this HEVC archival process worth it if attempting to reduce file size (Not that I expect you to know all the answers but I thought I'd ask :) )
@@IDKOKIDK More details mean bigger file size. Test "--tune grain" and play especially around with "--psy-rd" / "--psy-rdoq" / "--rdoq-level" depending on your source.
I'm one of the biggest IT guys in my area and I only have about 15TB between 7 machines and several external drives. No telling what I've got archived to DVD though... that's probably a pretty ridiculous number there. While I typed this up I had to keep bumping up the number though, keep forgetting about this drive and that drive and that one over there, backup in my bag... It is time to pick up a couple of 4 to 6TB rust drives for storage though before the prices go up. It's only a matter of time before SSD and HD won't be so far apart in price and I'm not sure SSD are going to get a lot cheaper.
@@EposVox very true. My original DVR system recorded at roughly 15GB/hr, but I only recorded about an hour a night, compressed it the same night. Nvenc made my life a lot easier.
![I lost all my data. You don't have to. [World Backup Day]](http://i.ytimg.com/vi/7eHOCr-HMKo/mqdefault.jpg)
![I lost all my data. You don't have to. [World Backup Day]](/img/tr.png)
Video: how to save 20tbs on hard drive
*looks at 1tb hard drive
I'm listening.
@@CodingUnited I feel you bruh
Same here.
Hahaha, true! xD
Wkwkkw
200* for me
Can't wait to store my "homework folder" while saving alot of space
"Research material" ;)
How's the homework coming?
Worth it. Experiment with switches that increase parallelism without reducing quality. Double worth it.
My "internet explorer" folder
I'm honestly really grateful for this video, I remember a while back I modified your original FFmpeg script to use x265 instead of x264 with limited success. Although I got the script working it wasn't practical with the absurd amount of time it would take to compress footage, this on the other hand just burns through my footage and is 6x faster on my system with close to similar visual fidelity. Thank you so much!
Software HEVC is way too slow
Next video:
How I saved my kidney yet again!
How I saved my kidney with one basic Script
whats the typical bitrate for the compressed files? i usually encode in handbrake with cq 0 and it turned a 200+mbps 20gb video to a 20 mbps 2 gb video.
The point of CQP is that it bases the bitrate on the content. So I have files that are 1-2mbps, 12mbps, 100mbps, etc.
Dude, you're the best. I always love your uploads but then again I'm in the center of your target demographic so I guess that makes sense. Keep up the great work!
When you're selecting those files to delete you can probably do it a lot more easily by sorting by Type, shift-select the block of old files, then sort back by filename to look for empty/bad files and correct from there.
I bereave a full tutorial is needed for this video since not everyone is script savvy. i can tare down and rebuild computers all day and fix almost all software and hardware related issues but scripting is a whole different knowledge base and skill set. please make this a tutorial video :)
Definitely I can't get the script to work and I don't know when it works. It's very confusing.
I'm very interested in how this works but I have zero scripting skills. Couldn't find a single good UA-cam tutorial on batch encoding files.
@@KnowIin Bit late to you guys but in the description there's an .exe file which you can put in a folder and just double click it
@@brandon_downes where is the .exe file ? I downloaded the zip from github but I didn't find the .exe
@@meolessi5227 The description links to ffmpeg.exe - you can't run this script without ffmpeg, which he is not going to host as it is likely copyrighted. All he's automating is an ffmpeg command, you could simply strip that out of his script and as long as you change the input file ("%%A") and output file ("%%A~dnpA_CRF%ffmpeg_qv%_HEVC.mp4") parts it should work manually.
After a bit tinkering, I think I fixed the script to work with AMD cards. Only did a few changes to line 33 on the script and it seems to be working and doing the same stuff without any problems now. Changed hevc_nvenc to hevc_amf and added -gops_per_idr 1 after to solve the seek problem that happened in videos and also changed "constqp -qp %ffmpeg_qv%" to "cqp -qp_p %ffmpeg_qv% -qp_i %ffmpeg_qv%". Also had to change -pix_fmt to nv12 as was recommended during the encoding, but that's just optional and can be changed to something else.
This was super helpful, I kept getting errors regarding constqp and could not figure it out!
Hello, thx for script. It us working for amd, but from what I see Nvidia get much smaller files with better quality than amd.? I get a bit washed out colors and size for example 1.2GB from 14.7GB file (from resolve ) where I expected around 600MB max. It's sad. I have rx 570 8GB. From what I understand, amd could fix this with just some software patch but they have no plan to do it in near future.
I strongly advise you to NOT USE AMD hardware decoding, it is the worst technology (literally) you can use if you care about quality OR /AND if you care about file size.
This is not a secret this is a well-known fact. Very easy to google.
Remembered this when I needed to compress some training videos, thanks for the content!
Really needed this. Hard drives were starting to really fill up. Thanks for sharing!
Me too, but for videos in Google Photos! They take up so much space on my SSD! D:
Hey Eposvox great video just got a quick question, at 10:00 you said that the encodes come in 8-bit 4:2:0 but according to NVIDIA your cards should support 10bit 444 and lossless, so why not use that and save more space?
developer.nvidia.com/video-encode-decode-gpu-support-matrix
Er, encoding to 444/lossless doesn't save more space...
It does when compared to raw doesn't it, I don't have alot of experience with video compression but from me messing around lossless files definitely where smaller than raw with no data loss ( Couse you know lossless 😂) when I encoded. Thought of course they took way more space than "visually lossless" which I ended up using because they where nothing important, if you need all video data I don't get why lossless wouldn't save space on drives compared to the high bitrate raw.
Oh and I'm talking specifically about the black magic files you said you might need to color correct later and such not the normal ones. And thanks for replying btw 😁
@@amrnassef5344 Ohh I see. Yeah I need to work on that, but that requires changing which encoding I do based on the source which gets more messy. Something I'd like to work on.
@@EposVox Oh I thought you would have the raw files in a separate folder per project or something so you would just be able to specifie the codec in the script (another version for lossless) and run it on the folder containing the raw files.
But if you have a different workflow then that changes things of course, though I do hope you look into this in the future and do a video about it as it would be really useful and interesting. :D
6GB to 6MB haha that is nice!
Have you ever tried using this on AviSynth scripts? Wanna make sure I'm doing this right, it seems to be using more of my CPU than my GPU, if it's even using my GPU.
Man I don’t even have one terabyte. That’s really insane bro.
KiiDFLiiGHT I have 240 gb lol
@@treeleaf6471 i have 128gbs
of ram
LOL
You compared x264 -crf 14 with nvenc -qp 24 as if they were interchangeable. This is not true, x264's crf mode is substantially different from nvenc's qp mode. Your results will have a substantially different quality profile. Which in your use case may not be critical, but in other scenarios this difference in how these modes assign quality and file sizes in different ways are quite substantial.
Also please remember that the licensing situations of h264 and h265 are very different. The additional fees and more complex licensing requirements for x265 almost disqualifies the format for most scenarios that not either strictly private use or very big scale commercial.
I'm aware, I mention that every time I discuss streaming scenarios and why HEVC isn't used for live. For random people via my UA-cam video, it's fine.
You're right about the CRF parameters, but I want address the licensing. So, while the HEVC codec does limit playability and editability in mainstream NLEs and players, I still think saving in that format is safe for the long run because we haven't yet hit the best compression quite yet, which means that one should use and make use, of what's best now. Even if the licensing is exclusive in the future, I'm on with paying once to be able to open a work of video footage that I can still reconvert for viewing pleasure. Yes, for streaming, not the best, BUT I want to archive, and so I've actually changed to HEVC encoding a while ago even before this video was posted on the same premise and results. So, I applaud this move and approve his message. The script was actually very helpful.
Brilliant, will look into this!
Why am i only seeing this now thank you UA-cam algorithm and epos vox!! I have 40tb+ worth of client files that i need to compress so i can start re using hard drives to save money, and this is a gamechanger. Thank you
I am "only" getting about 1.5x speed on 1080p footage, anything I can do to make it quicker? or is this just how it works with certain files etc?(1080ti)
Depends on your chosen encoder using hvec_nvenc speeds it up but bit bigger file size
Just a correction, the max bitrate allowed for the AAC codec isn't 384 Kbps, but 512 Kbps.
Did you update it perhaps after the rtx ampere GPU's came out? I have a 3080 and I'm wondering if it's worth to update the ffmpeg script. I don't have the skills to update it.
Ok, the topic looks interesting , and now, I'm a novice in this. Where do I start? I have some .mkv's than I'd like to compress, so how do I use this script? Thanks a lot for help!!
For anyone who isn't an expert and are trying to figure out how to do this:
1) download the "crf24_hevc.bat" file from EposVox's video description link
2) go to ffmpeg.org, click download, click on Windows, and click on "Windows Build by BtbN"
3) download this file: "ffmpeg-N-100554-g89c9c42c5b-win64-gpl.zip
"
4) extract the "ffmpeg.zip" file, and look for "ffpmeg.exe"
5) move the "ffmpeg.exe" and "crf24_hevc.bat" into a new folder, and move the video file you want to convert in that folder also
6) double click on "crf24_hevc.bat" and it will start encoding
Some things to note:
1) you may need to disable your virus scanner temporarily since mine prevented the .bat from running
2) you need an Nvidia video card for this. If you don't have one, it won't work
Good luck. Mine worked fine with the instructions above. Thank you EposVox for this stellar script as I'm seeing a noticeable difference in file size and the quality is identical!
This is exactly what I've been looking for. Thanks!
man you seem to be the handbrake/obs/premiere wizard. how tf do i fix my shyt? i'm recording with a focusrite scarlett 2i2 2nd gen. in obs I record with mkv format. when video is brought into handbrake i encode or whatever to mp4.
So I have 3 files when it's encoding in handbrake. game (video) game(audio) and mic (audio)
if i import a video in i've converted then cut it in 2 different areas and try draggin another one on top and cropping it my playback doesn't work and it just skips all around like crazy or locks up. i have constant bitrate selected in handbrake or whatever.. not sure what im doing wrong. can you help me out?
Please do note that QP24 and CRF24 are different. CRF will use different QP values for each frame depending on what is deemed necessary to reach the specified quality level.
Quick question - any particular reason to use '-rc constqp -qp' over other possibilities? Many say, including slhck site, to avoid -qp and opt for -cq (similar to -crf in x264). Quality with -qp is guaranteed, however, it can use more space, so savings with -cq could be higher. I am still testing hevc in order to see what is best for archival purposes.
BTW, you can omit both -map 0, the default map behavior is to select best video+audio, or in case of single video/audio track it is not even needed. Then for -b:v you can simply use 0, 'K' is not needed, ie. '-b:v 0'.
1 - no particular reason, just what I was most familiar with.
2 - no, those cannot be omitted for people with my kind of workflow, they're required to safely keep multitrack recordings in tact
I see. I did some SSIM and PSNR tests, '-qp' definitely preserves quality, and yes, it uses slightly more bits, but there is no drastic increase compared to '-cq' with the same value, so both are great.
Observations around '-cq':
- it seems to default to '-rc vbr' and it had slightly better score than '-rc vbr_hq', which some people recommend, almost no difference in metrics
- '-preset slow' surprisingly lowered metrics a little, but not perceivable as well.
In short, it seems nvenc is smart enough to use best defaults in given scenario, and '-qp' or '-cq' is enough. Same values provide similar results, both require '-b:v 0', '-qp' seems to default to '-rc constqp'.
I used this to test:
# ffmpeg -i new.mp4 -i original.mp4 -lavfi "ssim;[0:v][1:v]psnr" -f null -
... and this to see all options:
# ffmpeg -hide_banner -h encoder=hevc_nvenc
FFmpeg was latest build from Zeranoe site.
I'm coming from the future, oOoOo!
Great informative video my dude! This was really interesting.
who the hell are you?
A spy
@@EposVox a spy indeed ;)
These are great settings for ffmpeg. Instead of using a script you can easily automate everything with tdarr (I run in a docker container) which will monitor folders and automatically fan in and fan out GPU encoding and give you all the stats you want on storage saved.
as a atomation lover, this video was delightfull. great job as aways.
:D
an*
In 2020 this video has been extremely helpful. :)
EposVox, is there a version of this script that uses AMD HVEC? You have sung its praises lately and I'd love to try a version of this script with my AMD hardware.
AMF's ffmpeg implementation is a tad confusing atm and I haven't gotten CQP working in it yet. Theoretically just swapping "hevc_nvenc" to "hevc_amf" would work, but it throws errors for me. Will update if I get it sorted!
i seen a thing but i don't know if it's normal or not , in your recent bat , you use the mp4 format , everything work , then when i change to MKV , it does loop , something is wrong or it's perfectly normal ?
Hey Epos, have you considered using LTO tapes for archiving your cineform/massive video files so this way, if you absolutely need a clip and in it's original quality, you've got it in ~24 hours? That's personally what I'd do instead of deleting files (I'm running ~100TB locally and over 1PB in the cloud/colocated servers)
It's something I've wanted to look into, but don't have the budget for anytime soon
EposVox I have a question I use OBS studio to record with and I have recently started recording in lossless quality do you know any file converters that are out there that I can use to convert the file to MP4 because the editor that I use does not support Avi and that's the only way you can record lossless quality is Avi right?
You can use "Indistinguishable quality" for MKV/MP4
@@EposVox Thank you for your help
I don't understand the map-part: 0:a and 0:v. The a stands for audio and v for video but what does that 0 mean? I quickly looked in the man-page of ffmpeg (I searched for map), it doesn't make it much clearer. It is not relevant for using it but I am curious what that number stands for.
It refers to source 0. You can have multiple input files with ffmpeg.
It's for multiple tracks. I mentioned that in the video. I utilize multiple audio tracks in OBS and this makes sure that they're all mapped from source to the new file.
@@EposVox Matthew Bilker
Thanks.
I'm using Handbrake to do the same thing. I tried the script, but ffmpeg complained about a missing a library, which is odd. But I got Handbrake to do the same thing. I am currently chewing through 64 Blu Ray 1080p movies, encoding them to H.265 HEVC using my 8700K and nVidia 2080 ti. It takes about 25 to 30 minutes per movie to transcode them. These files took up about 1.7TB of space on my NAS, but based on the compression so far, I expect that I will save about ~60% of that. So the end result is that I should save about 700GB across all 64 files. Not bad!! Plus, I can now direct play these to my Sony TV, which means the CPU usage for my NAS is about 0-1% while Plex streams them!! I'm happy!!
UPDATE: I just had to change it to output an .mkv file (since that is a regression for ffmpeg when the input is .mkv) and it worked. Comparing the output of that to Handbrake however, Handbrake is better for only one reason. Everything is the same, video quality, audio quality, except the ffmpeg produces a video image that won't scale to the size of the player. So on my TV or gaming PC, it is a small rectangle with black borders on the top, bottom, left and right. Whereas Handbrake scales the video to any size view port.
does plex work with nvenc/amf? I am planning to retire my current rig which has an old i3 to be a tv tuner/plex server/htpc thing but cpu encoding/upgrade isn't going to work for me or my budget. So I'm considering getting it a gpu to use for compressing & 4k decode. Should I use plex to capture broadcasts or have other software simply write the raw MPEG2/H264 stream to disk and process later?
Plex works fine* with h265. The asterisk is because lots of client devices can't handle h265. Plex will gladly attempt to transcode it to h264, but you need either a very modern CPU or something capable of doing it in hardware to transcode in realtime for playback.
@@ClockworkD0rk can it use gpu to do real time?
@@skatcat743 I was just researching this earlier today, and according to documentation / forum posts it will, but I haven't verified it myself yet.
@@ClockworkD0rk bet
Hardware encoding only works if you have a Plex pass. 1080p and 720p transcoding from 1080p h264 isn't very demanding so at least that should work.
I have a question, I use handbrake to convert footage from shadowplay so it's constant frame rate which is needed for Resolve. I use H.264 Intel QSV and the file sizes are massive. I want it to be lossless because it's before the video is edited, but my God, a 5GB file can become 30GB. Is there something I'm missing?
I wonder if this could be useful in cutting down Shadowplay footage in a reliable way. I don't generate much of a footprint with my recordings still, but those files can get out of hand sometimes, since they're recorded at 50Mbit on the old NVENC.
Every once in a while the file will be bigger due to extra noise or motion, but yep! Here's an example i.imgur.com/HR7STlL.png
Can’t wait for the H266 VVC update 🤓!
How are the quality profiles equivalent between h264 and h265? Would you get better compression with CPU encoding over GPU? I have so many unanswered questions because I'm currently re-encoding my GoPro footage
CPU-based encoding always results in better quality and compression rates (at least for the foreseeable future). Hardware encoders don't offer all the instructions a software encoder can offer which results in the losses.
I only use GPU-based hardware encoding when rendering a rough-cut of a project and want it to happen relatively fast. When streaming, the GPU does the live-stream and my CPU does a local recording for better quality.
If you have an old Android phone you can save space by rooting it and installing a Magisk module (or just changing the "build.prop" file) to make Google Photos think you are using a Google Pixel and it will give you unlimited uncompressed storage. Then you just sync your footage with a cross platform app/program like Syncthing, and you can drag footage into a folder on your PC for example, and it will automatically send it to your phone first and upload it to Google Photos. I use it all the time on my phone and PC, works great
reencoding video sure, who cares, but i really don't get why you would reencode audio
hey EposVox! I use this script religiously :) I was just wondering If you've made any improvements to this script after all these years ? Thank you very much
So I gave this a shot with my stream recordings. I changed the audio bitrate on your script from 384k to 192k (because that's the quality I stream/record). I tried a short 20 minute mp4 recording, before it was 858,370KB and after it was 1,657,192KB. It's just about twice the size as before I done this. What could be going on here?
If you did not press 'Like' for this video, then you shall never press 'like' on another video ever! Great efforts!
Thank you! I am using this now to save a bunch of space, much easier than messing with Handbrake. Hero!
How To Do This..? Is He Doing Without Software ?
Very nice!! Do you have a video were you go in to your mapped drives?
Cool script, thanks for sharing man!
Not sure if you still check the comments on this video but I tried this on some videos i recorded with shadowplay and while everything worked properly the colors on the encoded video weren't quite right. On the encoded version of the video I tried the colors seem a bit dull for example the reds are a bit more orange than red and the blues are a bit darker. I don't really understand all the commands in the batch file but I tried to up the cqp to 22 like it said in the script and while the file size went up the colors weren't fixed. I tested out encoding the videos through handbrake using nvenc x265 and it fixed the color issue but obviously a script would be way easier to encode videos than handbrake so I would like to use the script if possible. Do you know the reason why this happens with your script and if it's possible to fix?
Nice! AV1 Could be quick enough next year. OPUS is slightly more compressed with the same quality. AVIF has a potential for image compression as of now WEBP is quick enough. For lossless audio FLAC is an open option and there is also APE which can be set to slightly higher compression.
For archival purposes of raw audio and images 7-Zip plugin MFilter by T4Shell could be used. Also Brotli and Zfast could be used for xml, html, etc. speedy text compression.
Can you do a video basically just setting this stuff up, just a short do this, download this and put this here video.
@@darkcggaming I am a hardware engineer, I just have little to no knowledge of automation
Great video, I'm back-archiving a lot of videos with HEVC now with a variation of your preset, it's mostly the same as yours and I'll likely use your script in the future. Question @eposvox, if you want to edit the HEVC converted video in a video editor, you would have to losslessly convert it back to x264, right? I can't seem to find any video editor that can edit it natively.
Premiere and Resolve both edit HEVC fine
I found x264 at CRF 18 and veryslow to be imperceptibly close to lossless with really good file sizes. I use this for my Plex server to stream media faster and with less bandwidth usage. I wanted to halve the file sizes but I can't do it at acceptable quality. x265 at quality 22 looks alright in bright scenes but falls apart in dark scenes and has severe banding issues. 10 bit h265 is way better but not quite half the file size of h264. H265 is really not half the file size at the same quality. I tried over and over but h265 just shows more artifacts than h264. I wonder how good your stuff looks at CRF 24 because software x265 is crap at 24.
For dark scenes I've found you really have to tinker with x264 settings, 2 pass is best.
For general video I've actually got NVENC settings that beat x264 in quality and size, but you may not find my quality level acceptable if you're having issues with 18. Using NVENC VBRHQ is essentially the NVENC version of 2 pass so even using 23 and bitrate of 1100 on 720P is nearly indistinguishable from source video except for really dark scenes and/or fog.
@@glenwaldrop8166 q18 h264 looks nearly perfect. The difference is tiny and only visible on a large screen that has bad black levels. It is most visible in dark areas where you can see blocking but h265 is way worse in dark areas. H265 produces different artifacts like banding and weird noise blocks. The banding is extremely noticable if you encode at 8 bit. Some of my movies are under 2 GB and still look good in h265 but the quality drop is a bit larger.
No idea what the capacity/age of those harddrives but i'd assume they're 8TB each if these were built post 2015 so why not just combine the 2 and get a bit of extra storage?
All in all though it's pretty rare to see a channel with only 240k subscribers spend $2500 on storage. I assume most of them are sitting on maybe $300 of storage for their videos at most.
hi could you please tell me is that script made just for Nvidia cards cos it tells me it Cannot load nvcuda.dll ??? i have RX580 VGA card
Yes it is because of the hevc_nvenc encoder, if you change that to the amd version of it then it will use that hardware encoder
Another nice video!
Why are my notifs always 30 minutes late
You got here right on time for this one!
Can you make a tutorial how to actually encode a video with your script? Put a video file into the same folder and press "crf24_hevc"?
Have you done SSIM comparisons with qp 24? I see loss (with the naked eye) with 1080p footage at 24. 4k is less important, but it's still there.
Are you using HEVC? X264 and HEVC CQP/CRF values work a bit differently.
But no, I haven't. If it looks good enough upon spot checking and running through UA-cam compression again, it's more than good enough for this purpose. I'm not aiming to squeeze out perfect duplicates, just "good enough". That being said, most of what I have is screen capture, which compresses well anyway, and uncompressed DNxHR/Cineform/ProRes files which are going to squish down a ton.
@@EposVox Yes, hvec and same hardware. As long as you understand that it is not a visually lossless conversion, you should be okay. It sounds like you're doing this just for archiving footage you're not sure if you need any more, aka "rainy day" backups, so it's fine.
Awesome! Will check this out at home later, any idea if this work on AMD's codec? How will I change the script to AMD or if I wanna try run it on 265 codec?
Thanks ☺️😅
did you have to custom build the ffmpeg to get hardware acceleration working?
Nope, just the latest Windows build from their website/zeranoes
Any script to convert h265 back to h264 for old TV?
Also what about a scrip too for downmixing properly the audios 5.1 to stereo 2.0 again for some old TV , thanks.
I want to use this but keep getting errors before the video starts to encode,. Not sure if my files are in the correct folders etc??? Great Idea, but I'm totally lost.
Are you getting "ffmpeg is not recognized as an internal or external command" as error?
could this be automated on linux for x265? i have an old HP DL160 G6 server and i've manually been needing to move files to and from it every time its done so just dumping a bunch of crap over and letting it run for the day would help
Can I compress Alpha channel videos like this?
powershell is the answer, supports all path types and easier than batch
Hey ... I Love this, but i have (programming) issues:
I Put your scripts/files, ffmpeg and my video (.mov File from Iphone) in one folder
Got an Error after executing cfr24_hevc.bat ...
Can you Help Me ?
Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> hevc (hevc_nvenc))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[hevc_nvenc @ 0000024d33ba81c0] 10 bit encode not supported2.77 bitrate= -0.0kbits/s speed=N/A
[hevc_nvenc @ 0000024d33ba81c0] No capable devices found
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0000024d33ba7440] Qavg: 54808.117
[aac @ 0000024d33ba7440] 2 frames left in the queue on closing
Conversion failed!
Ryzen 9 3900x / gtx 980ti
Also found this error:
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001f13b20df40] Format mov,mp4,m4a,3gp,3g2,mj2 detected only with low score of 1, misdetection possible!
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001f13b20df40] moov atom not found
C:\Users\Home\Desktop\Test\IMG_0005 2.MOV~dnpA_CRF24_HEVC.mp4: Invalid data found when processing input
@EposVox I have been using HandBrake to compress my movie collection. Everyone says that you should use your CPU and X.246 for the compression. I see that you are using ffmep which is the same as HandBrake but you are using the Video card to do the compression. I'm guessing you would not being do this if the compressed version was a bad quality. I can't see why there would be any difference between what you are doing and me with compressing a movie. Do you know of any settings that I can set in HandBrake to produce the same quality that you are doing? If I could same a bunch of time that would be wonderful. Maybe this is something you could make a video about. It would be nice if you could address the CPU verses GPU compression.
Great video by the way.
One comment. You can install Linux in Windows 10 (WSL). So you could write a bash script to do your encoding.
Well its actually pretty simple. CPU (aka software) encoding will always be more efficient compared to GPU, meaning same quality at smaller file size or better quality with same file size.
The reason why that is, is because hardware encoding (GPU) isnt as flexible as a written software and does not 100% support all the features of the codec (for example nvidia h.265 encoder before 20 series did not support B-Frames, which is very important for good efficiency with hevc).
@EposVox might be a good idea to create a video about this subject. I'm sure a lot of people might have the same questions.
@@_--_--_
Tip: This doesn't work on 900 series cards, because they don't support 265 encoding.
I just found this out since I wanted to try it on my laptop, as I'm not home to test it on my PC.
Yeah only 10 and up :/
Wait, what...? This is a 980: i.imgur.com/2aQVT8r.png
@@EposVox So if i use Handbrake H.265 (NVEnc) it works, why i cant use it with this? Nvidia GTX 980
so i need to change the pixel format to -pix_fmt yuv420p
@@FilipeFernandes7 I don't know why it would work in Handbrake, 900 series doesn't support it.
I see Synology, i upvote
Does this work on .mp4 files as well? Tried doing this, and the result was getting a "compressed" video that was a larger file size. Went from 1.05GB to 1.08GB
Yep
In your experience so far, is CRF 24 H.265 still basically equivalent to CRF 14 H.264?
So far, yep!
@@EposVox Nice! I just tried this on a screen recording and unfortunately, I noticed a little artifacting on CRF 24 with handbrake. I actually did a test with x264 and x265 with the same recording and it seemed like x264 looked better strangely. There's some recordings that I don't have a reason to keep but will do anyway, so I will use x265 for that just for archival purposes. I'm actually doing CRF 20 on that just to be safe. The size is still a lot smaller than x264, so I'm still getting a lot of benefit! I'm not storing an insane amount of things like you are, but I can still definitely receive the storage benefits despite this.
Hello! And what about video metadata (for example created at)? For old videos it's important to have these data to be easiest to find and sort files. I saw that ffmpeg has an option to copy the original video metadata to the converted one. Thanks!
Probably something I should include, haven't looked into it.
@@EposVox i saw something like: "-movflags use_metadata_tags" video.stackexchange.com/questions/23741/how-to-prevent-ffmpeg-from-dropping-metadata
I've also got "-map_metadata 0" added in on github
This script is getting rid of the original subtitles, any help?
how did you set up the stream deck button to encode the folder you are currently active in?
Have you tried doing the same on Linux? Actually to check if it would be faster with the same parameters despite the GPU drivers being considerate bad. I guess CPU wise it would have better performance (?)
Epos, just subbed......solid vids bro u nerdout so hard lol............do u hav any plans to do a vid on a 2070super gpu card combined with a 3700/3900 xt amd chip and how well it performs for streaming? If not would u?
It's possible! I don't have access to the cards, but I think Wendell got one in, so it's something we can look at.
Dam, quick reply......Badass bro, regards to my 1st reply keep seeing alot of problems for streamers in regards to 5700 gpus so for the sake of time im gonna purchase a 2070 super and a ryzen 3700
okey, so U cant record with the codec u used to compress the files? cuz I was thinking that this video explains how to record "lossles" high compressed video hm
Great Video, Subscribed.
Thanks!
Amazingly I hadn't considered this before. Will davinci resolve allow hvec editing
Yep!
@@EposVox I going to add this to my workflow, pop a new raid in and run this through my server this weekend. Thank you so much your channel has always been a go to place for process improvement for me.
@@salaciouscreations4323 :D
Bro, this was insanely awesome. You gained a Sub for sharing, and I'm a really hard sell. Looking forward to videos like this.
Thanks!!
This is awesome. I think somthing like this would be useful for me shooting footage in a t3i. How much do you think would have to be done in order to use this script on linux
The "%%A" / "~%%dnpA" variables aren't the same in Linux, but otherwise the normal ffmpeg command is the same. So just translating the rest from Batch to Bash.
I love EPOSVOX
Hi, do you have an updated version of this script?
I've been playing around with this script and different 'original' recording settings over the past 3 months and I find that if I'm re-encoding something at H265 CQP 22 or worse, you can get *some* artifacting but it's in very niche situations that usually happen with dark environments. I've seen blocking on steam vent effects in a dark area, or a face that passes by in a split second in the King's Row (night time) map in Overwatch. This is made *very slightly* more significant if you're running a garbage-in-garbage-out workflow where you already compressed the recording at a very high setting (like H264 CQP14, well past diminishing returns) but even CQP22 from a lossless AVI can produce some flaws here and there
With that said, 'watching the video, fullscreen, pausing on busy frames and actively hunting for flaws' is quite different from how the content is actually watched, and for the amount of space saved, oh, it's absolutely worth it, especially considering that at this point youtube processing is the tightest bottleneck even at 1440p.
Did the script but the compressed file is larger than the original?
I thought about this too, legit script there 👍🏻
Would this also be recommended with movies?
Just opened my 45 drives server today. I installed 120 terabytes and I never thought nine 14 terabyte drives wood look so small but next to 36 empty drive bays it's sad. First world problems
HNNNNG jealous
@@EposVox lol I've got lots of empty bays you're welcome to come by my office in LA throw in some drives, make a storage pool, and bang ! Free cloud storage. 2gig symmetrical fiber in it also helps. Speed things up
Yeh - hvec is a huge step up in power over 264. I use it for 12bit HDR content... Being a mad programmer I edit and master all my videos with ffmpeg on a Mac using zsh. Sadly, the Mac hardware hvec encoder does not seem to handle smpte2084 HDR, so it is a slow and laborious process... Anyhow - loved the video and thst you are getting the message out. ffmpeg rocks. Thanks for the vid!
@EposVox. Hi. Just curios if you've switched to RX 5700/5700 XT for this script, considering you've mentioned (ua-cam.com/video/Yi-_T3vsv-Q/v-deo.html) you could run 5-6 encoding streams simultaneously? Thanks!
how do u run 5-6 encoding streams simultaneously..what the set up?
For deleting those .mov files just search the folder for mov and you'll just see those .mov files and then simply ctrl+a and delete. So much quicker :)
or you coulda wrote a small python program/script to do it all at once thus being able to deal with the issues inside the same program?
Coulda, if I had the experience. Could've made it a full gui and something that ran itself for me once footage copies over while I'm afk - but I work with what I know how to do. And I'm a control freak and don't want something deleting the files for me.
If only I could use this for my 500gb+ steam library...But otherwise, very useful for the old bits of video here and there that take up way too much space!
... hmmmm maybe something similar but steam gets pretty upset usually over file changes
waiting for an update to this video using AV1
I considered doing something similar last year. I set up a script that worked well, the quality was barely perceptibly different, with significantly smaller files (better than 1/2 size in most cases).
However I came to the conclusion that it actually wasn't worth it - because the Electricity cost to re-encode was higher than the cost of just buying additional HDs.
Obviously the maths is different if you have an ongoing cost for cloud storage, vs the one-off cost of local storage.
Expensive electricity o_o
@@EposVox Not really - hard disks, even resonable quality ones, are cheap. I worked out it was going to take over a month to re-encode. vs the maybe £100 for a suitably sized HD. Now, at the time, I had solar panels and so free electricity during the day. I did have a plan to make a script that would run in the daytime only, and while I was out of the house. But in the end it just didn't seem worthwhile.
Not everyone's needs are the same though
@@LeeSmith-cf1vo Clearly you don't need the same scale of storage lol
@@EposVox correct, although assuming a linear relationship between encoding time and space required - the maths still works the same. That's a big assumption though, and unlikely to be true at the higher end. As I say, different requirements for different people 🙂
Holy crap! CRF 24 and no noticeable quality loss? I'll have to try that!
You lose sharpness. Else it's awesome / way better than h264.
surject It's not noticable unless you zoom way in, is it?
@@zinsy23 Depends on viewing distance, display size and resolution... *I* notice it on my projects and setup. It's just a technical fact h265 got less details aka smoothens/blurs the image. Same reason your JPEG file got smaller when you add some blur in Photoshop "save for web" export dialog.
@@surject Is there a CR value that won't reduce sharpness compared to original & does that value still make this HEVC archival process worth it if attempting to reduce file size (Not that I expect you to know all the answers but I thought I'd ask :) )
@@IDKOKIDK More details mean bigger file size. Test "--tune grain" and play especially around with "--psy-rd" / "--psy-rdoq" / "--rdoq-level" depending on your source.
I'm one of the biggest IT guys in my area and I only have about 15TB between 7 machines and several external drives.
No telling what I've got archived to DVD though... that's probably a pretty ridiculous number there.
While I typed this up I had to keep bumping up the number though, keep forgetting about this drive and that drive and that one over there, backup in my bag...
It is time to pick up a couple of 4 to 6TB rust drives for storage though before the prices go up. It's only a matter of time before SSD and HD won't be so far apart in price and I'm not sure SSD are going to get a lot cheaper.
General datahoarding and video production business are very different scales of data, hehe
@@EposVox very true.
My original DVR system recorded at roughly 15GB/hr, but I only recorded about an hour a night, compressed it the same night.
Nvenc made my life a lot easier.
Could you possibly make a newer video, but this time for AV1?
Unfortunately, theres a noticeable green tint applied across al my GH5 original videos, also on the exported color graded ones. Anyone knwos a fix ?