Wendel... You are the only person on UA-cam, that I can truly trust your information. I have learned so much from you over the years. Getting my feet wet with TruNAS core and built my first NAS/Server about 3 months ago now on an Optiplex 990 (i7-2600). Creating a ZFS Pool was a 1st for me, and setting up an SMD Share... I'm sure I haven't made it the most efficient config, But I'm learning! Thank you for everything you share!!!
Truenas Scale and Truecharts being "sort of clunky" is a ridiculous understatement. The documentation is horrendous as well. I am 2 seconds away from going back to proxmox and never looking back
Wow I ran into the network share issue last year and thought I was crazy. Lazy solution, I setup a 2nd truenas system in the lab to act as a network share for VMs on the first. Glad to finally find a workaround. Leave it to Wendell to solve a problem I never asked about 😂
I had this issue trying to set up twingate on my Truenas set up. I could not for the life of me figure out why I could connect to other virtual machines hosted in truenas, but could not connect to the truenas dashboard. I've been wracking my brain for a few weeks trying to figure out why it wasn't working. Can confirm the network bridge fixed it. Such an easy fix but easily overlooked. Thanks Wendell. :)
The best thing about this video was listening to Wendell's aggrieved tone of voice when discussing the broken network stack for TrueNAS Scale. I imagine that's what I sound like to other people when I'm complaining about it stuff myself.
I learned a lot from you in past 4 - 5 months! At this moment my home server setup: 12th Gen i7 Intel NUC with 64GB Ram, 240SSD hosting Proxmox and 2TB NVME for VMs. One VM is TrueNAS Scale which handles 16TB Mirror rust drives (Attached Drive BAY). Stores everything from Proxmox (Backups, ISOs, CT Templates, VM Additional drives, iSCSI 5TB drive for my work PC) Addition to TrueNAS, Proxmox spinning another 5 VMs and 10 CT Containers. Drive bay + NUC with all the stuff using on average 45 - 50w
Nice video Wendell. The wonkiness of the networking is a big turn off for new DIYers, whereas us old farts just sigh and fight thru it. Sadly this means when non techs ask what I recommend for home NAS storage and basic messing around, I just sigh and say Synology.
One important thing to mention about the bridging. If you have a VM running or set to 'Autostart', make sure you have it off, or none of the bridge changes will work.
Eek. Glad i watched this. Been wanting to consolidate a few systems into one more modern, and this has me rethinking my direction, which was basically this without knowing the landmines.
My thoughts exactly. Call me crazy but I think iX is massively over complicated things by not natively supporting plain vanilla Docker. I’m happy to use portainer as the webUI, but I’d like to run the Docker on the host itself, not in a VM. Maybe that is considered bad practice and that is why they are not doing it that way? But they are offering K3s on the host so I can’t imagine how Docker on host and K3s on host are very different in terms of ‘best practices’ Oh well. Stuck with 2 home servers
I've been using truecharts for some of my containers and, as you said early on, they're not very transparent and tend to force you to use their specific workflows. On top of that, I always run into issues with their containers whenever I try to do updates. It's definitely going to be a rough process but this video will definitely help me to transition away from truecharts apps. Also, Tailscale also opens up a lot of possibilities when it comes to remotely accessing my services securely and more simply. So thanks for that too!
i was wondering why you went through a VM just to install Portainer instead of through docker on TrueNAS... but then i read the writeup! ok. was very well explained and makes a lot (more) sense. thanks for the writeup!
That's weird! I thought that network share issues (not allowing guests to ping the host) was by design. All of the virtual network setups in KVM for example (I used it on Fedora, Ubuntu) required removing the default virtual network and setting up a bridge manually. I think generally it may kinda sorta be a good idea in the majority of virtualization scenarios (by default).
If you move the system dataset to your boot drive, HDDs shouldn't have activity anymore. But with 100 VMs/containers asking for stuff, you can't really send HDDs to sleep. You can squeeze a minute or two out of the writes with zfs_txg_timeout and over engineered cache for reads, but ZFS is meant for performance, not idle state. Let the drives spin in idle (
Love your videos Wendell, working on my own home NAS atm. But an idea for a future video because its a huge point of stress for me... disaster recovery for this kind of setup. FreeNAS scale, Portainer... what needs to be done when you loose a drive, or even your boot drive, or other hardware failures.
I'm using a Fractal R6 for my home server, and yeah the bridge network took me awhile to figure out when I tried the truenas scale beta. Also personally I moved my docker stuff to a rpi4 due to the memory leak issue awhile back, not moving back since I find it easier to manage for docker.
A good video overall but its the first video with Wendell where my own professional experience differs from what he is advising users to do. Certain types of data should never be ran over NFS. Specifically SQLite databases will encounter all kinds of bugs and are commonly used in docker. There are lots of other software that is incompatible with NFS also which will cause bugs and because users typically don't configure or install any software within a docker image there is little way most users are going to know about a potential problem until it just starts happening. It is possible to still use NFS with docker but really it should be limited only to data files which are unlikely to cause any issues. I know this undermines Wendell's goal of being able to snapshot his docker applications. I've been trying to solve the same problem myself for a long time and have yet to actually solve the problem.
This makes sense. Look at what type of data you have and then follow the best practices for that. I can't remember what it was but I remember seeing some open source software, that would plug into SQLite db's and replicate them. So personally I'd rather do that.
I somehow suspect Wendell read my threads on TrueNAS community forums. I have a SuperMicro build. I have two nics. I did try to set up bridge. And passthroughs. And it’s all buggy and broken but it works just enough so that if you learn of all the issues you can tip toe your way around those. I finished the build like 5 days ago, and I have my signature on their forum full of issues I encountered with links to relevant posts. To much of the annoyance of their mods, might I add.
@@Jimmy_Jones It feels like they focused most of their QA to make sure the part that's responsible for your data works. That part actually works flawlessly.
Thanks for doing this video. And thanks for sharing the frustration with TRUENAS SCALE. It drove me nuts to set the networking between VM and truenas host. They have some serious issues in their feedback processes. Do you think you could use your influence and push on them a little bit? Something seriously needs to change there.
Even on Truenas core things like VM's are totally broken. Windows VM's are almost impossible to keep stable. Had to move to scale to get stable VM's and there is still stuff like this that i had to do even after that.
@@The123tactics use XCP-ng for virtualization. Run truenas core as a VM and pass the hba through to truenas. Learn to tell which Intel NICs are counterfeit and get a real one.
@@blackrockcity I switched over to truenas scale. Works a lot better. Currently not using a HBA anyway but could passthrough the ssds I have in my pool the same. No issues and working as expected.
@@The123tactics I’m testing out scale but haven’t tried docker or VMs yet. It sounds like it’s not there yet, especially regarding networking. Having fought core, I don’t trust truenas to get virtualization right. XCP-ng is solid. I’ve never tried proxmox.
Unfortunately, I've found the mods on TrueNAS forums being rude is their natural state. This isn't a bug, being rude and condescending is aTrueNAS mod feature. Thats why I always look elsewhere and only go there if I absolutely have too. Really like TrueNAS, really dislike how they treat others.. Might be different for enterprise customers; thats the path they've really taken as a company.
This is probably the best guide on this but there's still so much that's confusing tbh. It would have been more clear for a Level0 user like myself if you actually walked through more of the steps instead of just have scroll overlays from the forum post with half-cut images for some steps
Where do you buy your spinning rust? I tried Amazon for a 16TB drive that was an utter fail. Drive came in a baggy with no protection. Immediate return. I bought same from NewEgg drive came in a sealed box with those plastic inflatable pouches. Unfortunately the drive wasn't packaged properly in the cushioning and sure enough it was a dead clicking drive. Thankfully NewEgg refunded my money -- this was before the Tech Jesus motherboard controversy. Any good seller recommendations appreciated.
I would go straight to a company specialising in reselling old server hardware, skip the middle man like Amazon or Newegg. A lot of these companies list on ebay but don't think of it as buying from embay, read the reviews and check out their products. I've got one in the UK who sells old SAS drives 10 at a time.
It is actually very common to transport HDD's in just a baggy, without any other form of protection. Much more drives can be transported in one go this way and it is cheaper to replace broken drives than it is to package them properly. What is strange is that not too many drives break during transport this way. Not nearly enough to offset the price of packaging drives and sending much less of them in a container from the factory to the country where they are being sent. More drives get damaged when they are transported by truck/delivery company than by container transport, because packages get seriously mistreated by overworked/underpaid employees unfortunately. No amount of properly hard drive packaging helps with that.
Amazing guide! Finally set up nextcloud and tailscale and everything works over the tailscale-provided IPs; but I still can't get IPs from my local subnet to work... I guess I'll keep debugging, but even at this point the functionality is fantastic!
❤ thank you Wendell Enterprise king 👑 of nerds Seriously though, these walkthroughs are amazing and having an old HP z420 server and running TrueNAS + apps makes my brain go bbbrrrrr with all this optimization 🙏🏼
I always find it amusing when tech tubers say things like "2x 20TB hard drives, not that much". Here in Canada, those IronWolfs cost $700 each. At $1400 for storage, that's worth more than most people's computers :) There are so many other computer parts I would spend that money on before I would spend that much on disk drives.
20TB is bad $/TB right now because "new" and "biggest". I can get a Toshiba MG08 16TB for 230$ here in Europe which is really the sweet spot for $/TB atm.
Here in the UK a company is auctioning 10 3TB Hitachi SAS drives. I picked up a set for £80 ($100). That's 8 in the system and 2 spares. That's about 17TB after ZFS has done it's thing. A bit cheaper than C$1400!
Can someone help with step 2 "Tell the NFS share system to treat the “root” user on the client as root on this system. Map user and Map root should both be set to rot."? I have set Taproot User and group both to room and have also tried nfsdckr but I'm getting access denied when I try to mount /nfsDocker/nfsdckr
Awesome, thanks for that! Can we get that guide extended with VLANs, to have some network separation? Bunch of containers should be visible only to user devices, other to servers, yet another to IOT...
Honestly, I would use Truenas for the thing that it's extremely good at, and have another machine running Docker. Right now, my homelab is a hodgepodge. Unraid running on a machine that is VERY overkill (Dual socket 6core/12thread Xeons with 128GB of ram), my Proxmox server (Which is overkill, but is severely lacking in local hard drive space) and then a couple of laptops just sitting in my server rack. I'm getting a disk shelf soon and want to make the unraid server Truenas, but JUST for storage.
@@wayland7150 I mean, if you were going to just play with it, sure? But I want to use truenas for HA and storage. If I'm storing my VMs on truenas, why would I want to run it as a VM?
@@hotstovejer Well that's the point, I don't trust TrueNAS with my VMs but I like how it's a NAS. Proxmox for the VMs. It was OK, but slow NAS as a VM. Probably required some fine tuning, which I'm not prepared to do.
Great video as always. I ran into exactly this issue after i gave up on the (horrible) kubernetes implentation. Went with Ubuntu server with docker instead. Wishlist for videos: 1. Traefik for Nextcloud 2. Obsidian
@@hotstovejer Thanks mate 🙏 No need though I also have it running. But it would be great content :) Also obsidian is one of the best things I am using, would be fun to see how Wendell would use it.
Hard to expect a response to a question on an 8 month old video, but here goes. Can you comment on the pros/cons of having the docker vm mount the nfs share vs setting up an nfs based volume and having the container mount it?
If TrueNAS Scale has terrible hypervisor support, what’s the benefit of Scale over Core? I was thinking of moving to TrueNAS from Unraid, but now I’m thinking I keep one NAS on Unraid with my containers, TrueNAS with a copy of the data and some fast file and block storage, and a Proxmox cluster for my homelabbinf nonsense and VMs.
Have you tried Storj as a cloud backup solution for TrueNAS Scale? My initial tests are that it’s slow and difficult to upload files to it, and even harder/bordering on impossible to download/recover a full backup from it. How about a full review of the various cloud backup solutions for TrueNAS like Storj? Thanks!
Should probably mention that UEFI is basically kinda sorta broken on TrueNAS Scale for Linux where GRUB doesn't work so you have to basically create your own boot stuff....
Just spent the entire day stressing over not being able to mount any NFS share. Turns out that TrueNAS Scale (Cobia?) will not export ANY NFS shares if there is an user missing or something like that. I added a missing (I think it was missing) user on the host side and deleted (then recreated ) most of my NFS shares and lo and behold, I can now mount the shares!
Somewhat new to Truenas, what I don't understand is what is the benefit of setting up these applications through Portainer? When you can just set them up using Truecharts?
i think where must be a mounting issue with my vm.. in the nextcloud setting it says it only has 132Gib of space... can anyone help. i followed everything
Any chance of getting an explainer on the NFS setup portion for Linux dummies like myself? I don't understand what it is we are trying to do or what mount/umount does in this context.
Hi, Thanks a lot .. but you said in the beginning of the video you show us how to stop the spinning rust from spinning and by that save engerie .. this i still don't know!?! i really want to lower my truenas server power consumption!
Any thoughts on ZeroTier? It seems to work in a similar way, and it can be self hosted, etc. I was able to use a Caddy container in front of my VaultWarden instance to only allow access from my ZeroTier network. Works well, but I always see stuff about TailScale, so I am interested in how they compare. Also, anyone having issues with TrusNas Scale running on a QNap? I can't restart it without it basically locking up from an endless flood of "rc rc0: receive overflow" messages, which force me to have to hard power it down because they just never stop and block all input.
is it safe to have your hdds start and stop spining so often? or will it degrade the hhd and make it fail earlier then if its spining all day. and after watching the whole video im kind of confused about the power saving thing mentioned at the start. you didnt speak about it again through out the video
If Wendell manages to break the built in Docker then you know it's bad lol. The problem is IX wanted it done in a particular way and from my understanding even people experienced with Kubernetes can't work out why the hell they did it in this way, it's bassakwards. The VM network thing seems to be something IX just "does". Core does exactly the same thing, the VM cant talk to the host without stuffing around with a bridge. I had so many issues with the Docker / Kubernetes implementation in Scale I went to Core with a Plex jail and anything that didn't need to access the NAS went on a VM. Might tinker with this method instead of setting a Debian VM on one of my other systems.
@@wayland7150 My thoughts as well. Hell even a bog standard Debian install with Docker and ZFS is much better, more work obviously but it'll do what you want. The issue with Scale is Kubernetes, the implementation is stupid and clearly doesn't work but IX refuse to change it or give an option during install to just use vanilla Docker. A lot of people on Reddit and IX forums want vanilla Docker as well so it's not like the demand isn't there. Scale could've easily dominated the homelab, small and medium business sector but they screwed it up going after big business and enterprise. Might've worked, if they didn't usually use custom solutions and not these fairly restrictive prebuilt solutions... Dunno what the hell IX was thinking.
@@rebsdioramas There is probably a nice webGUI I don't know about that could be put on a Debian ZFS server that would get you a decent NAS. Webmin perhaps hahaha.
@@wayland7150 Cockpit works well but it can be a massive pain in the ass to set up correctly on Debian, lots of dependencies needed. It's easier on Ubuntu but... Ubuntu lol. Any RHEL distro it'll work mostly flawlessly on but getting ZFS on RHEL is a hassle lol.
Having memory not shared by the host, thus reducing zfs cache capability makes this seem like a workable solution that limits my machine. What about using portainer as a chart in truenas and then running all dockers with NFS. Does that work and solve the network, data/codebase, and reliability issues.
The whole solution sounds good. would love to try this soon. I goign to setup new lab on some old server I found from work. I am looking forward to tailscale as a solution to connect to home devices when not at home. :)
What i'd like to do is set up TrueNAS 'apps' to run plex, as well as TDARR to transcode recorded TV/movies, i just recently discovered that TrueNAS Scale can forward hardware decoder/encoder so i can now use my TrueNAS Plex instance as my primary playback server without worrying about weird CPU performance degredation for critical file share services instead of a seperate windows machine(i've recently had issues in linux with the PCIE TV tuners i need for my area for reliable signal). Though i wouldnt be transcoding to H265/AV1 with hardware, as all hardware encoders are very poor at space efficency, especially when going from one codec to another. In my case MPEG2 to H265 or AV1, both are qually bad, often either worse quality, or larger than the original MPEG2 file ~2GB/hour for 1080p, where as CPU is often 1/3rd-1/5th the size of the original for the same quality. I just need to be able to add in a PCIE TV tuner and i'll be set with a single device solution (seperate machines for on site redundancy and off site backup)
Soooo the short version is that TrueNAS Core is good for storage, but still sucks at virtualization? This the reason why people run Proxmox on metal, and TrueNAS Core in a VM?
Great video! Do you have any recommendation for a quiet PSU to build a NAS. The eight year old psu of my homeserver just went up in smokes and I am looking for a new one. Thanks for your help in advance.
Hey Wendell, for that exactly use case, couldn't you make a case to use proxmox instead of truenas? If your intent is virtualization that would seem like a good start point.
You end up with virtualized storage that adds complexity and trouble or basically re-building Proxmox from scratch. TrueNAS with KVM as both storage server and hypervisor is a more elegant and better performing solution while keeping complexity manageable.
It's the unsolvable problem, TrueNAS or Proxmox. I put TrueNAS inside Proxmox and passed the drive controller to it. It worked and I thought I was a genius. However passing out an iSCSI drive from VM TrueNAS to a Proxmox VM really started to cause a rift in the space time continuum. By the time that VM was serving files it was only running at 30MB/s and it's unreliable. I am going back to two physical servers, Proxmox and TrueNAS.
@@wayland7150 as I said in other threads, I’ve never use proxmox but I like XCP-ng. Virtualize truenas core in it. Pass the hba through. Use a real Intel NIC. I never got iSCSI to work in core. Too many bugs. Some said core needs a ton of ram for iSCSI to work. Not sure how much…Way more than 20GB.
got stuck on giving my client permissions so I just made it open to everyone on my local network to use. Probably not best practice but this nas is just for fun I'm fine with it. Thanks for the great guide!
I have 4 nics on my scale server. Can I change this for a vm to be different that the storage instead of a bridge to be able to mount the vm to the storage?
Have you not experienced the constant snapshot creation? Check your snapshots. I installed Scale and before long I had hundreds of snapshots after using the apps. Admittedly, I didn't spend a lot of time trying to stop them but from some of the research I did, it appears to be a very annoying bug. Curious if anyone else was able to get around this.
@@niallthebomb hundreds or thousands of snapshots don't really matter. I have 16000 snapshots on my TrueNAS Core and all is running fine and well. All those tiny subdirectories for docker barely have any data in it anyway, so it's not like actual space that's wasted.
@@niallthebomb IT seems like there are a large number of snapshots on my time machine dataset, which doesnt make sense, because the apps/plugins/vms are on a completely different array/pool, and only access my share dataset, though, so far, they are not writing to these datasets.
Is there a way to host the containerization system on bare metal instead of visualizing it so you can use native filesystem calls instead of having to invoke the network subsystem and nfs and all the overhead involved with that? If that's possible then you can also worry less about segmenting memory to the virtual machine or applications and allow for dynamic memory allocation and/or better management of memory on an container by container basis. Also, and this is probably a containerization n00b question, but do containers group themselves into application clusters or when i look at all the containers running do i just see the complete list of containers? - I'm just curious about managing the various containers. I'm used to TrueNAS Core with jails and putting a complete application stack in each jail. What is the purpose of splitting off portions of an application into various containers anyway (like for the example of nextcloud a database container and a web server container)?
Yes! I use a docker VM with only Rancher on it to manage containers on the host. I would say it has a steep learning curve, but the UI is beautiful and unrestricted unlike the TrueNAS UI. The containers/info, however, are not integrated with the TrueNAS UI at all, it shows "no applications" on mine, which is currently a good thing IMO because ix-systems... So, the docker mentality is "one process per container" that's why the DB is its own container. Jails are more similar to LXC. Additionally, TrueNAS Scale is using docker, yes, but it's also using k3s (kubernetes) which most people I've heard call it 'docker on space steroids' or some other such nonsense. This is also why the term charts is used rather than compose. There is such a mixup of information on kubernetes/k8s/k3s/so-on, so take care while researching it. P.S. Rancher is for k3s like portainer is for docker.
I think the lesson from this video is have a separate volume or partition for your own stuff. For instance my PCs have /home mounted from a separate partition in the case of the laptop and a separate drive in the case of the desktop. I can zap the OS and not lose my files. This approach works for VM etc too.
@@wayland7150 I understand, but how that's accessed is also important. If i can bypass the network stack, loopback (or the network card), and NFS, things operate much faster and smoother in terms of accessing files on the disk. I'd rather access files directly. Also resource management. if you lock up ram in a vm, it's no longer availible to the system and you have to get that configuration right instead of just letting containers use as much ram as it needs. Otherwise, I might as well create virtual machines for each application which is way heavier and a waste of resources... running containerization inside of a virtual machine seems needlessly compute and memory expensive
@@joshhardin666 On Proxmox you have LXC (Lexy) which look and work like full Linux VMs but are much lighter. What I've done with Windows VMs in Proxmox is run them as a full VM off a volume on the SSD but have their D: drive as a huge volume off the hard drive array. You get the speed for Windows OS plus the size and separation required for your files. I did a similar thing for NextCloud run as an LXC off SSD with the user files mounted from a hard drive based volume.
I was wondering if I could bounce ideas off of someone. Would you rather setup a config such as this (TrueNAS Scale + VM with docker inside of it) or Proxmox with a TrueNAS Core/Scale VM (with passthrough of an HBA of course). They seem to accomplish the same end result but I'm unsure as to which approach is "best" to manage long term. Thanks!
TrueNAS is reliable in some ways and broken in others. If you setup NFS it will probably just work. Use XCP-ng as your hypervisor. Run TrueNAS core in a VM. Pass the HBA through to this VM.
For a home server, I would've just installed tailscale on the host truenas installation. Would've made configuration a lot easier. Probably not the best way though
Ha, the inability to access the host was driving me nuts! I didn't even know what terms to search for. I worked around it by using dual NICs and a physical switch.
can someone explain how this tailscale works? I've set this up and get it to work, but the performance drop is HUGE. I suspect the packets are relayed through some other node and not a real P2P.. i'm a noob on networking but the numbers im getting suggested there's throttling somewhere... my NAS is on 200Mbps link and I connected using another broadband at 50 Mbps file transfer stucks at 500Kb to 1Mb
Given that nothing else in the title works in Core, I think it's pretty clear. The first 30 seconds of the video also shows Scale documentation and links also say "TrueNAS Scale". Blame IX systems for their shitty branding.
If you ike Ubuntu a lot then you have a point. However TrueNAS is like an appliance, it would be even better if they removed the menu options for VMs, they just spoil an otherwise well made system. What would make it perfect is if when you created a pool it created the SMB and NFS shares at the same time.
I was very excited for TrueNAS Scale. TrueNAS is my choice of NAS OS and I run primarily Docker containers for home/homelab services and have a few VMs. I am really disappointed that TrueNAS Scale has such crap support for native Docker and weak VM usage. I was hopefully to consolidate the homelab to 1 server. But given the current state of Docker/VMs on Scale, I think I will stick with Core as a NAS and a second host for VMs and Docker.
I just wanted to mention that I get practically full freedom of containers on the Scale host by managing it with Rancher, NOT THEIR UI PLEASE! k3s vs docker and all that jazz. I prefer it to Wendall's docker VM method, which I tried myself at the end of last year. But I'm still struggling with trying to get disk/usb passthrough and .img or "Appliance OSes" working on Scale's KVM. I've heard of people being successful, but I can't get them to boot. Also, you can get Virtual Machine Manager to work with it if you change the KVM socket (I cannot fathom why they would change/hide it).
Two seperate hosts is always preferred if the double power bill and space is ok. I'm running Proxmox with virtualized TrueNAS Core but I'm consolidating everything into Scale soon, because I want to decrease complexity and easier management. And Container VM with nested containers as shown by this video is a great alternative to k3s+docker on Scale. Not everyone likes having kubernetes by default nor has a need for it. But I can't deny Core being THE choice for storage.
@@TheExard3k I would agree, BUT space is a huge concern for me. I live in an NYC apartment so space is the big factor. Not too worried about power, the two hosts I have now are both white box builds from used desktop hardware and are pretty power efficient. Of course only 1 host would be more efficient but for me physical space is a bigger concern than power.
Spoiler alert: Red Shirt Wendel survives the NAS software configuration.
I was wondering if he dead by the end, lol
Wendel... You are the only person on UA-cam, that I can truly trust your information. I have learned so much from you over the years. Getting my feet wet with TruNAS core and built my first NAS/Server about 3 months ago now on an Optiplex 990 (i7-2600). Creating a ZFS Pool was a 1st for me, and setting up an SMD Share... I'm sure I haven't made it the most efficient config, But I'm learning! Thank you for everything you share!!!
Amen! Long live Saint Wendell! 🎉
Thank God for written guides. It's a rare occurrence nowadays.
Truenas Scale and Truecharts being "sort of clunky" is a ridiculous understatement. The documentation is horrendous as well. I am 2 seconds away from going back to proxmox and never looking back
Wow I ran into the network share issue last year and thought I was crazy. Lazy solution, I setup a 2nd truenas system in the lab to act as a network share for VMs on the first. Glad to finally find a workaround. Leave it to Wendell to solve a problem I never asked about 😂
It seems like a cludge work around because it's tied to physical hardware with a 1gb/s limitation where as virtually it could be 10gb/s.
over 2 years later the issue still exists, they will never fix it
I had this issue trying to set up twingate on my Truenas set up. I could not for the life of me figure out why I could connect to other virtual machines hosted in truenas, but could not connect to the truenas dashboard. I've been wracking my brain for a few weeks trying to figure out why it wasn't working. Can confirm the network bridge fixed it. Such an easy fix but easily overlooked. Thanks Wendell. :)
The best thing about this video was listening to Wendell's aggrieved tone of voice when discussing the broken network stack for TrueNAS Scale. I imagine that's what I sound like to other people when I'm complaining about it stuff myself.
I learned a lot from you in past 4 - 5 months!
At this moment my home server setup: 12th Gen i7 Intel NUC with 64GB Ram, 240SSD hosting Proxmox and 2TB NVME for VMs.
One VM is TrueNAS Scale which handles 16TB Mirror rust drives (Attached Drive BAY). Stores everything from Proxmox (Backups, ISOs, CT Templates, VM Additional drives, iSCSI 5TB drive for my work PC)
Addition to TrueNAS, Proxmox spinning another 5 VMs and 10 CT Containers.
Drive bay + NUC with all the stuff using on average 45 - 50w
How is it going with TrueNas Scale running on a VM? Did you pass-through all the drives? And does it properly use the allocated RAM?
Nice video Wendell. The wonkiness of the networking is a big turn off for new DIYers, whereas us old farts just sigh and fight thru it. Sadly this means when non techs ask what I recommend for home NAS storage and basic messing around, I just sigh and say Synology.
One important thing to mention about the bridging. If you have a VM running or set to 'Autostart', make sure you have it off, or none of the bridge changes will work.
^ This. Stopping any active VMs will allow it to work from the gui
Eek. Glad i watched this. Been wanting to consolidate a few systems into one more modern, and this has me rethinking my direction, which was basically this without knowing the landmines.
My thoughts exactly.
Call me crazy but I think iX is massively over complicated things by not natively supporting plain vanilla Docker. I’m happy to use portainer as the webUI, but I’d like to run the Docker on the host itself, not in a VM.
Maybe that is considered bad practice and that is why they are not doing it that way? But they are offering K3s on the host so I can’t imagine how Docker on host and K3s on host are very different in terms of ‘best practices’
Oh well. Stuck with 2 home servers
unRAID will do the trick
I've been using truecharts for some of my containers and, as you said early on, they're not very transparent and tend to force you to use their specific workflows. On top of that, I always run into issues with their containers whenever I try to do updates. It's definitely going to be a rough process but this video will definitely help me to transition away from truecharts apps. Also, Tailscale also opens up a lot of possibilities when it comes to remotely accessing my services securely and more simply. So thanks for that too!
I pulled the trigger on unraid after trying and horribly failing at setting up truenas. Needed this video a year ago :P Awesome content as always!
unRAID is much better...
Switching from unraid to truenas. Followed this tutorial and got portainer going. Now to get all my apps going!
Didn't know about Tailscale, was exactly what I always wanted to access my NAS over Internet. Thank you!
This channel is SO UNDERAPPRECIATED!! Great video! I'm waiting to have the budget to start tipping my toes.
Thanks Wendel, I was stuck on this for a few days until I ran across your video.
i was wondering why you went through a VM just to install Portainer instead of through docker on TrueNAS... but then i read the writeup! ok. was very well explained and makes a lot (more) sense. thanks for the writeup!
Duuuuude thank you Wendel!!! Fixed my issue with JellyFin so now it should be able to see my nas shares! FINALLY!
That's weird! I thought that network share issues (not allowing guests to ping the host) was by design. All of the virtual network setups in KVM for example (I used it on Fedora, Ubuntu) required removing the default virtual network and setting up a bridge manually.
I think generally it may kinda sorta be a good idea in the majority of virtualization scenarios (by default).
Wendell, you've mentioned a few times sleeping HDDs in TrueNAS; Is that (and hopefully other TrueNAS tuning) coming in a future video/forum post?
This and automated cold storage have been ideas I am trying to make happen for my setup. Very nice.
If you move the system dataset to your boot drive, HDDs shouldn't have activity anymore. But with 100 VMs/containers asking for stuff, you can't really send HDDs to sleep. You can squeeze a minute or two out of the writes with zfs_txg_timeout and over engineered cache for reads, but ZFS is meant for performance, not idle state. Let the drives spin in idle (
Did you really think the older videos would escape my gaze? Muhahahah
Love your videos Wendell, working on my own home NAS atm. But an idea for a future video because its a huge point of stress for me... disaster recovery for this kind of setup. FreeNAS scale, Portainer... what needs to be done when you loose a drive, or even your boot drive, or other hardware failures.
Do you even network br0?
I'm using a Fractal R6 for my home server, and yeah the bridge network took me awhile to figure out when I tried the truenas scale beta. Also personally I moved my docker stuff to a rpi4 due to the memory leak issue awhile back, not moving back since I find it easier to manage for docker.
A good video overall but its the first video with Wendell where my own professional experience differs from what he is advising users to do. Certain types of data should never be ran over NFS. Specifically SQLite databases will encounter all kinds of bugs and are commonly used in docker. There are lots of other software that is incompatible with NFS also which will cause bugs and because users typically don't configure or install any software within a docker image there is little way most users are going to know about a potential problem until it just starts happening.
It is possible to still use NFS with docker but really it should be limited only to data files which are unlikely to cause any issues. I know this undermines Wendell's goal of being able to snapshot his docker applications. I've been trying to solve the same problem myself for a long time and have yet to actually solve the problem.
This makes sense. Look at what type of data you have and then follow the best practices for that. I can't remember what it was but I remember seeing some open source software, that would plug into SQLite db's and replicate them. So personally I'd rather do that.
I somehow suspect Wendell read my threads on TrueNAS community forums. I have a SuperMicro build. I have two nics. I did try to set up bridge. And passthroughs. And it’s all buggy and broken but it works just enough so that if you learn of all the issues you can tip toe your way around those.
I finished the build like 5 days ago, and I have my signature on their forum full of issues I encountered with links to relevant posts. To much of the annoyance of their mods, might I add.
I'm surprised it came out of beta with the amount of bugs this version has.
@@Jimmy_Jones It feels like they focused most of their QA to make sure the part that's responsible for your data works. That part actually works flawlessly.
Thanks for doing this video. And thanks for sharing the frustration with TRUENAS SCALE. It drove me nuts to set the networking between VM and truenas host. They have some serious issues in their feedback processes. Do you think you could use your influence and push on them a little bit? Something seriously needs to change there.
Even on Truenas core things like VM's are totally broken. Windows VM's are almost impossible to keep stable. Had to move to scale to get stable VM's and there is still stuff like this that i had to do even after that.
@@The123tactics use XCP-ng for virtualization. Run truenas core as a VM and pass the hba through to truenas. Learn to tell which Intel NICs are counterfeit and get a real one.
@@blackrockcity I switched over to truenas scale. Works a lot better. Currently not using a HBA anyway but could passthrough the ssds I have in my pool the same. No issues and working as expected.
@@The123tactics I’m testing out scale but haven’t tried docker or VMs yet. It sounds like it’s not there yet, especially regarding networking. Having fought core, I don’t trust truenas to get virtualization right. XCP-ng is solid. I’ve never tried proxmox.
All of this is easy in unRAID...
Thank you for another wonderful upload! I'm enjoying the homelab content so much!
Unfortunately, I've found the mods on TrueNAS forums being rude is their natural state. This isn't a bug, being rude and condescending is aTrueNAS mod feature. Thats why I always look elsewhere and only go there if I absolutely have too. Really like TrueNAS, really dislike how they treat others.. Might be different for enterprise customers; thats the path they've really taken as a company.
This is probably the best guide on this but there's still so much that's confusing tbh. It would have been more clear for a Level0 user like myself if you actually walked through more of the steps instead of just have scroll overlays from the forum post with half-cut images for some steps
Thanks! I had the same issue with the tailscale instance not showing up on the tailscale web ui. The command you showed fixed it
Please do the Tailscale video. Thanks for the video!
Where do you buy your spinning rust? I tried Amazon for a 16TB drive that was an utter fail. Drive came in a baggy with no protection. Immediate return. I bought same from NewEgg drive came in a sealed box with those plastic inflatable pouches. Unfortunately the drive wasn't packaged properly in the cushioning and sure enough it was a dead clicking drive. Thankfully NewEgg refunded my money -- this was before the Tech Jesus motherboard controversy. Any good seller recommendations appreciated.
I would go straight to a company specialising in reselling old server hardware, skip the middle man like Amazon or Newegg. A lot of these companies list on ebay but don't think of it as buying from embay, read the reviews and check out their products. I've got one in the UK who sells old SAS drives 10 at a time.
It is actually very common to transport HDD's in just a baggy, without any other form of protection. Much more drives can be transported in one go this way and it is cheaper to replace broken drives than it is to package them properly. What is strange is that not too many drives break during transport this way. Not nearly enough to offset the price of packaging drives and sending much less of them in a container from the factory to the country where they are being sent.
More drives get damaged when they are transported by truck/delivery company than by container transport, because packages get seriously mistreated by overworked/underpaid employees unfortunately. No amount of properly hard drive packaging helps with that.
Thanks for the tutorial! Gonna buy Synology
Amazing guide! Finally set up nextcloud and tailscale and everything works over the tailscale-provided IPs; but I still can't get IPs from my local subnet to work... I guess I'll keep debugging, but even at this point the functionality is fantastic!
Configuring some engagement for the algorithm
❤ thank you Wendell Enterprise king 👑 of nerds
Seriously though, these walkthroughs are amazing and having an old HP z420 server and running TrueNAS + apps makes my brain go bbbrrrrr with all this optimization 🙏🏼
I always find it amusing when tech tubers say things like "2x 20TB hard drives, not that much". Here in Canada, those IronWolfs cost $700 each. At $1400 for storage, that's worth more than most people's computers :) There are so many other computer parts I would spend that money on before I would spend that much on disk drives.
Good lord! Yeah find the sweet spot for capacity vs cost.
20TB is bad $/TB right now because "new" and "biggest". I can get a Toshiba MG08 16TB for 230$ here in Europe which is really the sweet spot for $/TB atm.
Here in the UK a company is auctioning 10 3TB Hitachi SAS drives. I picked up a set for £80 ($100). That's 8 in the system and 2 spares. That's about 17TB after ZFS has done it's thing. A bit cheaper than C$1400!
Can someone help with step 2 "Tell the NFS share system to treat the “root” user on the client as root on this system. Map user and Map root should both be set to rot."?
I have set Taproot User and group both to room and have also tried nfsdckr but I'm getting access denied when I try to mount /nfsDocker/nfsdckr
I'm glad I saw this. Seriously rethinking if I would be able to get this up and running. Haha
Awesome, thanks for that!
Can we get that guide extended with VLANs, to have some network separation? Bunch of containers should be visible only to user devices, other to servers, yet another to IOT...
Your videos are always top quality. Thanks!
Honestly, I would use Truenas for the thing that it's extremely good at, and have another machine running Docker. Right now, my homelab is a hodgepodge. Unraid running on a machine that is VERY overkill (Dual socket 6core/12thread Xeons with 128GB of ram), my Proxmox server (Which is overkill, but is severely lacking in local hard drive space) and then a couple of laptops just sitting in my server rack. I'm getting a disk shelf soon and want to make the unraid server Truenas, but JUST for storage.
Get TrueNAS Core for that.
I combined Proxmox and TrueNAS in one machine but I don't recommend it.
@@wayland7150 I mean, if you were going to just play with it, sure? But I want to use truenas for HA and storage. If I'm storing my VMs on truenas, why would I want to run it as a VM?
@@hotstovejer Well that's the point, I don't trust TrueNAS with my VMs but I like how it's a NAS. Proxmox for the VMs. It was OK, but slow NAS as a VM. Probably required some fine tuning, which I'm not prepared to do.
Great video as always. I ran into exactly this issue after i gave up on the (horrible) kubernetes implentation.
Went with Ubuntu server with docker instead.
Wishlist for videos:
1. Traefik for Nextcloud
2. Obsidian
I have Nextcloud running through Traefik and Authelia (Gonna switch over to Authentik soon...) So if you would like the labels I used, let me know. :)
@@hotstovejer Thanks mate 🙏
No need though I also have it running. But it would be great content :)
Also obsidian is one of the best things I am using, would be fun to see how Wendell would use it.
@Level1Techs Is there anyway to write a script to automate the process of setting up the bridge networking?
Hard to expect a response to a question on an 8 month old video, but here goes. Can you comment on the pros/cons of having the docker vm mount the nfs share vs setting up an nfs based volume and having the container mount it?
Depends if you wanna manage containers individually of as a fleet. For homelab cases fleet is easier?
can you do a video on how to properly migrate from Core to Scale? Specifically how to move Jails (Nextcloud, UniFI, Plex, Tailscale, etc.)
If TrueNAS Scale has terrible hypervisor support, what’s the benefit of Scale over Core? I was thinking of moving to TrueNAS from Unraid, but now I’m thinking I keep one NAS on Unraid with my containers, TrueNAS with a copy of the data and some fast file and block storage, and a Proxmox cluster for my homelabbinf nonsense and VMs.
Have you tried Storj as a cloud backup solution for TrueNAS Scale? My initial tests are that it’s slow and difficult to upload files to it, and even harder/bordering on impossible to download/recover a full backup from it. How about a full review of the various cloud backup solutions for TrueNAS like Storj? Thanks!
Should probably mention that UEFI is basically kinda sorta broken on TrueNAS Scale for Linux where GRUB doesn't work so you have to basically create your own boot stuff....
Whats better between Zerotier and Tailscale?
That was a nice 1 minute case review. Looks good
Just spent the entire day stressing over not being able to mount any NFS share. Turns out that TrueNAS Scale (Cobia?) will not export ANY NFS shares if there is an user missing or something like that. I added a missing (I think it was missing) user on the host side and deleted (then recreated ) most of my NFS shares and lo and behold, I can now mount the shares!
Somewhat new to Truenas, what I don't understand is what is the benefit of setting up these applications through Portainer? When you can just set them up using Truecharts?
Why is TrueNas using macvlan by default?!
i think where must be a mounting issue with my vm.. in the nextcloud setting it says it only has 132Gib of space... can anyone help. i followed everything
Any chance of getting an explainer on the NFS setup portion for Linux dummies like myself? I don't understand what it is we are trying to do or what mount/umount does in this context.
Hi, Thanks a lot .. but you said in the beginning of the video you show us how to stop the spinning rust from spinning and by that save engerie .. this i still don't know!?! i really want to lower my truenas server power consumption!
The KILL A WATT(tm) points out watt consumed per second ?
Any thoughts on ZeroTier? It seems to work in a similar way, and it can be self hosted, etc. I was able to use a Caddy container in front of my VaultWarden instance to only allow access from my ZeroTier network. Works well, but I always see stuff about TailScale, so I am interested in how they compare. Also, anyone having issues with TrusNas Scale running on a QNap? I can't restart it without it basically locking up from an endless flood of "rc rc0: receive overflow" messages, which force me to have to hard power it down because they just never stop and block all input.
is it safe to have your hdds start and stop spining so often? or will it degrade the hhd and make it fail earlier then if its spining all day. and after watching the whole video im kind of confused about the power saving thing mentioned at the start. you didnt speak about it again through out the video
If Wendell manages to break the built in Docker then you know it's bad lol. The problem is IX wanted it done in a particular way and from my understanding even people experienced with Kubernetes can't work out why the hell they did it in this way, it's bassakwards. The VM network thing seems to be something IX just "does". Core does exactly the same thing, the VM cant talk to the host without stuffing around with a bridge.
I had so many issues with the Docker / Kubernetes implementation in Scale I went to Core with a Plex jail and anything that didn't need to access the NAS went on a VM. Might tinker with this method instead of setting a Debian VM on one of my other systems.
I fail to see the point of Scale if Docker does not work. May as well use either Proxmox or Core.
@@wayland7150 My thoughts as well. Hell even a bog standard Debian install with Docker and ZFS is much better, more work obviously but it'll do what you want. The issue with Scale is Kubernetes, the implementation is stupid and clearly doesn't work but IX refuse to change it or give an option during install to just use vanilla Docker. A lot of people on Reddit and IX forums want vanilla Docker as well so it's not like the demand isn't there.
Scale could've easily dominated the homelab, small and medium business sector but they screwed it up going after big business and enterprise. Might've worked, if they didn't usually use custom solutions and not these fairly restrictive prebuilt solutions... Dunno what the hell IX was thinking.
@@rebsdioramas There is probably a nice webGUI I don't know about that could be put on a Debian ZFS server that would get you a decent NAS. Webmin perhaps hahaha.
@@wayland7150 Cockpit works well but it can be a massive pain in the ass to set up correctly on Debian, lots of dependencies needed. It's easier on Ubuntu but... Ubuntu lol. Any RHEL distro it'll work mostly flawlessly on but getting ZFS on RHEL is a hassle lol.
@@rebsdioramas It sounds like an opportunity for someone to make a distro like that. I know I'd prefer to get that than roll my own.
Nom, nom, nom, let me consume your knowledge 👾
That was an a amazing tutorial... any idea about setting up ZFS on a linux distro without using TrueNAS almost seems like it would be easier
I think Ubuntu comes with that if you like Ubuntu.
I'm a Nub, first time running NAS, VM. @09:39 I ran into that issue when i realized I needed Win Drivers, and couldn't access my SMB's.
This is Level 4 tech fixing.
Why use Scale if you create a VM anyway for Docker containers? Isn’t running Core faster in that case?
Having memory not shared by the host, thus reducing zfs cache capability makes this seem like a workable solution that limits my machine. What about using portainer as a chart in truenas and then running all dockers with NFS. Does that work and solve the network, data/codebase, and reliability issues.
The whole solution sounds good. would love to try this soon. I goign to setup new lab on some old server I found from work.
I am looking forward to tailscale as a solution to connect to home devices when not at home. :)
What i'd like to do is set up TrueNAS 'apps' to run plex, as well as TDARR to transcode recorded TV/movies, i just recently discovered that TrueNAS Scale can forward hardware decoder/encoder so i can now use my TrueNAS Plex instance as my primary playback server without worrying about weird CPU performance degredation for critical file share services instead of a seperate windows machine(i've recently had issues in linux with the PCIE TV tuners i need for my area for reliable signal).
Though i wouldnt be transcoding to H265/AV1 with hardware, as all hardware encoders are very poor at space efficency, especially when going from one codec to another. In my case MPEG2 to H265 or AV1, both are qually bad, often either worse quality, or larger than the original MPEG2 file ~2GB/hour for 1080p, where as CPU is often 1/3rd-1/5th the size of the original for the same quality.
I just need to be able to add in a PCIE TV tuner and i'll be set with a single device solution (seperate machines for on site redundancy and off site backup)
It's 2024 and bridging is *still* broken. Great job, iXsystems!
When I install debian and it goes to reboot it wont load the grub and brings me to the UEFI Bios shell....any suggestions?
Soooo the short version is that TrueNAS Core is good for storage, but still sucks at virtualization?
This the reason why people run Proxmox on metal, and TrueNAS Core in a VM?
Great video! Do you have any recommendation for a quiet PSU to build a NAS. The eight year old psu of my homeserver just went up in smokes and I am looking for a new one. Thanks for your help in advance.
Which motherboard was this
Hey Wendell, for that exactly use case, couldn't you make a case to use proxmox instead of truenas?
If your intent is virtualization that would seem like a good start point.
You end up with virtualized storage that adds complexity and trouble or basically re-building Proxmox from scratch. TrueNAS with KVM as both storage server and hypervisor is a more elegant and better performing solution while keeping complexity manageable.
It's the unsolvable problem, TrueNAS or Proxmox. I put TrueNAS inside Proxmox and passed the drive controller to it. It worked and I thought I was a genius. However passing out an iSCSI drive from VM TrueNAS to a Proxmox VM really started to cause a rift in the space time continuum. By the time that VM was serving files it was only running at 30MB/s and it's unreliable. I am going back to two physical servers, Proxmox and TrueNAS.
@@wayland7150 as I said in other threads, I’ve never use proxmox but I like XCP-ng. Virtualize truenas core in it. Pass the hba through. Use a real Intel NIC. I never got iSCSI to work in core. Too many bugs. Some said core needs a ton of ram for iSCSI to work. Not sure how much…Way more than 20GB.
got stuck on giving my client permissions so I just made it open to everyone on my local network to use. Probably not best practice but this nas is just for fun I'm fine with it. Thanks for the great guide!
Thank you, Wendel :)
I'll install Proxmox
I initially wanted to host TrueNAS and Debian VMs inside Proxmox but now I'll probably host a Debian VM in TrueNAS.
I have 4 nics on my scale server. Can I change this for a vm to be different that the storage instead of a bridge to be able to mount the vm to the storage?
This doesn't work anymore, now we need cloudnative-pg and prometheus-operator. I have no idea how to install now...
Does TrueNAS support DOS/Win95 guest VMs for retro gaming?
This is so awesome. Thanks Wendell
is it better to install it on VM or truecharts via truenas scale apps?
Wtf... l just installed truenas 2 days ago... WHERE WERE YOU 2 DAYS AGO!? :(
I havent had issues with the 'apps' in Truenas Scale, and IIRC it uses docker, i'll have to check when i get home.
Have you not experienced the constant snapshot creation? Check your snapshots. I installed Scale and before long I had hundreds of snapshots after using the apps. Admittedly, I didn't spend a lot of time trying to stop them but from some of the research I did, it appears to be a very annoying bug. Curious if anyone else was able to get around this.
@@niallthebomb hundreds or thousands of snapshots don't really matter. I have 16000 snapshots on my TrueNAS Core and all is running fine and well. All those tiny subdirectories for docker barely have any data in it anyway, so it's not like actual space that's wasted.
@@niallthebomb IT seems like there are a large number of snapshots on my time machine dataset, which doesnt make sense, because the apps/plugins/vms are on a completely different array/pool, and only access my share dataset, though, so far, they are not writing to these datasets.
Is there a way to host the containerization system on bare metal instead of visualizing it so you can use native filesystem calls instead of having to invoke the network subsystem and nfs and all the overhead involved with that? If that's possible then you can also worry less about segmenting memory to the virtual machine or applications and allow for dynamic memory allocation and/or better management of memory on an container by container basis.
Also, and this is probably a containerization n00b question, but do containers group themselves into application clusters or when i look at all the containers running do i just see the complete list of containers? - I'm just curious about managing the various containers. I'm used to TrueNAS Core with jails and putting a complete application stack in each jail. What is the purpose of splitting off portions of an application into various containers anyway (like for the example of nextcloud a database container and a web server container)?
Yes! I use a docker VM with only Rancher on it to manage containers on the host. I would say it has a steep learning curve, but the UI is beautiful and unrestricted unlike the TrueNAS UI. The containers/info, however, are not integrated with the TrueNAS UI at all, it shows "no applications" on mine, which is currently a good thing IMO because ix-systems...
So, the docker mentality is "one process per container" that's why the DB is its own container. Jails are more similar to LXC.
Additionally, TrueNAS Scale is using docker, yes, but it's also using k3s (kubernetes) which most people I've heard call it 'docker on space steroids' or some other such nonsense. This is also why the term charts is used rather than compose. There is such a mixup of information on kubernetes/k8s/k3s/so-on, so take care while researching it. P.S. Rancher is for k3s like portainer is for docker.
I think the lesson from this video is have a separate volume or partition for your own stuff. For instance my PCs have /home mounted from a separate partition in the case of the laptop and a separate drive in the case of the desktop. I can zap the OS and not lose my files. This approach works for VM etc too.
@@wayland7150 I understand, but how that's accessed is also important. If i can bypass the network stack, loopback (or the network card), and NFS, things operate much faster and smoother in terms of accessing files on the disk. I'd rather access files directly. Also resource management. if you lock up ram in a vm, it's no longer availible to the system and you have to get that configuration right instead of just letting containers use as much ram as it needs. Otherwise, I might as well create virtual machines for each application which is way heavier and a waste of resources... running containerization inside of a virtual machine seems needlessly compute and memory expensive
@@joshhardin666 On Proxmox you have LXC (Lexy) which look and work like full Linux VMs but are much lighter. What I've done with Windows VMs in Proxmox is run them as a full VM off a volume on the SSD but have their D: drive as a huge volume off the hard drive array. You get the speed for Windows OS plus the size and separation required for your files. I did a similar thing for NextCloud run as an LXC off SSD with the user files mounted from a hard drive based volume.
I was wondering if I could bounce ideas off of someone.
Would you rather setup a config such as this (TrueNAS Scale + VM with docker inside of it) or Proxmox with a TrueNAS Core/Scale VM (with passthrough of an HBA of course).
They seem to accomplish the same end result but I'm unsure as to which approach is "best" to manage long term.
Thanks!
TrueNAS is reliable in some ways and broken in others. If you setup NFS it will probably just work.
Use XCP-ng as your hypervisor. Run TrueNAS core in a VM. Pass the HBA through to this VM.
@@blackrockcity TIL about XCP-ng thanks!
@@carlosmunoz_gp let me know if you have questions. Be careful to pass the correct pci address for the HBA. The format of that number matters a lot.
For a home server, I would've just installed tailscale on the host truenas installation. Would've made configuration a lot easier. Probably not the best way though
17:23 I thought it said *"I added this line to the end of my life"* 🤣
We need an update using jailmaker!
Ha, the inability to access the host was driving me nuts! I didn't even know what terms to search for. I worked around it by using dual NICs and a physical switch.
can someone explain how this tailscale works? I've set this up and get it to work, but the performance drop is HUGE. I suspect the packets are relayed through some other node and not a real P2P.. i'm a noob on networking but the numbers im getting suggested there's throttling somewhere...
my NAS is on 200Mbps link and I connected using another broadband at 50 Mbps file transfer stucks at 500Kb to 1Mb
How could something not work for two years when the version you're using wasn't released until this year?
Just a note that it looks like you're using scale rather than core here. The video title/description were ambiguous. Scale ftw!
Given that nothing else in the title works in Core, I think it's pretty clear. The first 30 seconds of the video also shows Scale documentation and links also say "TrueNAS Scale".
Blame IX systems for their shitty branding.
@@handspiker1994 core can run virtual machines just fine
@@Emerald13 They break after a few goes.
You're calling the bridge Bea Arthur?
I don't get all this true Nas love all over youtube. What's wrong with Ubuntu server and zfs?
Do you get with ubuntu a proper WEB UI, specialized for NAS and a company grade dedicated team behind for ensuring the software functions as a NAS?
If you ike Ubuntu a lot then you have a point. However TrueNAS is like an appliance, it would be even better if they removed the menu options for VMs, they just spoil an otherwise well made system. What would make it perfect is if when you created a pool it created the SMB and NFS shares at the same time.
There is a problem with truenas13 and debian, where it comes to efi and bootx64.efi(grubx64.efi) location file
Ohh how I would love the ability to persist the uefi settings in my VMs.
I was very excited for TrueNAS Scale. TrueNAS is my choice of NAS OS and I run primarily Docker containers for home/homelab services and have a few VMs.
I am really disappointed that TrueNAS Scale has such crap support for native Docker and weak VM usage. I was hopefully to consolidate the homelab to 1 server. But given the current state of Docker/VMs on Scale, I think I will stick with Core as a NAS and a second host for VMs and Docker.
I just wanted to mention that I get practically full freedom of containers on the Scale host by managing it with Rancher, NOT THEIR UI PLEASE! k3s vs docker and all that jazz. I prefer it to Wendall's docker VM method, which I tried myself at the end of last year.
But I'm still struggling with trying to get disk/usb passthrough and .img or "Appliance OSes" working on Scale's KVM. I've heard of people being successful, but I can't get them to boot. Also, you can get Virtual Machine Manager to work with it if you change the KVM socket (I cannot fathom why they would change/hide it).
Two seperate hosts is always preferred if the double power bill and space is ok. I'm running Proxmox with virtualized TrueNAS Core but I'm consolidating everything into Scale soon, because I want to decrease complexity and easier management. And Container VM with nested containers as shown by this video is a great alternative to k3s+docker on Scale. Not everyone likes having kubernetes by default nor has a need for it. But I can't deny Core being THE choice for storage.
@@TheExard3k I would agree, BUT space is a huge concern for me. I live in an NYC apartment so space is the big factor. Not too worried about power, the two hosts I have now are both white box builds from used desktop hardware and are pretty power efficient. Of course only 1 host would be more efficient but for me physical space is a bigger concern than power.
I think two separate machines is best. I also put TrueNAS inside of Proxmox but I'm not happy with the reliability.