Another cheap option for point-to-point 10 GbE is to get some fiber SFP+ modules - get the kind with 850nm multimode LC connectors and you can get super cheap fiber on eBay (under 20 bucks for 50+ feet). It works really well. I have my two workstations connected to a 2 port card on my FreeNAS server.
Some performance issues on ZFS with HDDs can be attributed to not running a really fast ZIL. (grab a intel 900p ssd and use 50~GB for ZIL and the rest for ARC cache)
Nice tutorial. What Google Chrome version or skin you are using that have movement animation? And what software for network topology you use? Cheers, amazing work, im a starter on FREENAS myself.
I found this video doing research on SFP+ because I wanted to deploy my NAS off-site at my neighbors house. What kind of cabling and modules would you need hang or bury between the homes?
@@LAWRENCESYSTEMS I was hoping for a fiber cable type. Below is what I think I need. Adapter: Chelsio T520-CR on the used market it seems like a good LR card for around $150. Cable: LC UPC to LC UPC Duplex OM3 Multimode outdoor rated (can't find any, only OM2 breakout) Transceiver Module: Chelsio SM10G-LR The only reason I think I need LR is every cable I see maxes out at 30m and my run is just inside of 100m, maybe I should just run CAT6A but I like to learn new stuff and I would like an off-site backup.
thanks for sharing, i have a question right now i am building a lab with some vms and freenas as storage server, i got a san switch cisco and i want to create the san network over there, i know that for connecting this to the switch i need to create a zone then add the WWPN on it, my question is can i set the WWPN or the WWNN to the freenas initiators or how do i map the luns to the specic servers on the san network using the swith.
I have the same setup, as inspired by this video! Once I add the 10g nic, my jails that I want to use my regular nic, looses connectivity and says it has no route. I’m trying to figure out where I can bind the jail activity to the 1g network.
Great video, I'm actually looking to try this myself for transferring large Blu-ray videos from my workstation to my Freenas with plex. I'm curious Tom if you saw a performance boost on your VM's when moving from 1Gb to 10Gb?
Great videos! New to pfsense and freenas. Still learning. I've been thinking of trying a 10GB nic in my pfsense box. Using pfsense and a dual port 10gb nic to bridge to the LAN. I'm looking to not get a 10gb switch. Expensive! The nas connect over 10gb to the pfsense and all traffic to the nas would go over the 10gb. Instead of dual 1gb bridged to pfsense and the rest of the LAN. Currently I only have 10gb from the nas to my windows 10 computer with a DAC cable. I would like to get 10gb to my LAN for the nas. Would something like that work? I hope I'm explaining this so it makes sense. Thanks
For those interested, the official documentation as of version 11.3 U2 and above now states you can use up to 80% for iSCSI ZVol instead of 50% - much better.
I'm getting 650(+) MB/s on my iSCSI link as well... The iSCSI link is from my FreeNAS to a Lenovo server running ESXi. The FreeNAS has two links to my GB switch and the ESXi only has one... There is something about that report that isn't working correctly.
Thanks. It is information I wish to see in your film. BTW yours videos are great. I always lost connectivity to the server when I add new interface by the gui.
I am having the worst luck with this exact card when installing to Windows 10. Drivers are installed and card is recognized. I see my network and can access, but as soon as I access a file on the freenas box, i get the BSOD saying that NDIS.SYS is the problem. Could you tell me which driver you used for this card?
I need a reality check. I want to run UniFi NVR in a vm and have it record to a freenas box (not vm) over a sfp+ 10gb direct connection. Is this possible? Could it be reliable?
you should try MTU 9710 as this is biggest currently supported industry-wide - it should work on chelsio too, but I have not tested it, as Chelsio is quite expensive in Europe and I rather sell Chelsio and keep cheap, crappy Intel PRO 10G XF, as it have lower value
Ive been wanting to change to free nas for a while now but not having ecc ram for me has been the issue. Now I see that you are using a non ecc machine. For how long have you been running this configuration and have you ever had issues with bitrot?
There is some much controversy over this but in reality, It is statistically improbable that non ecc memory would cause an array to go bad. This system has been running for a few years without issue, but I ALWAYS recommend backing up to a separate machine as having all of your data in one place is never a good idea.
Lawrence Systems / PC Pickup yea, but I've been more worried about corruption of the files themselves, does that make sense? Additionally, in regards of the backups, I'm aware but I haven't had the cash to make an off-site backup of the critical data.
Lawrence Systems / PC Pickup ok, just found an incredible article about. jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/ that will make me want to switch now xD
Lawrence Systems / PC Pickup - Do you have to buy an addon cage/card for the Mikrotik CSS326-24G-2S+RM 24 port to enable the 10GBE SFP+ ports ? I found the switch on Amazon US for USD$133.33 - www.amazon.com/gp/product/B0723DT6MN
I seem to be stuck on only 400 MB\s transfers and can't seem to get any higher speeds on my 10GbE cards between my FreeNas and Workstation. My FreeNas 11 has WD Reds 5400rpm and Workstation has Samsung 960 EVO NVMe. I have set Jumbo packets on both systems to 9000. Not sure why I am not seeing higher, unless it is because my NVMe might be sharing PCIe from mobo PCH with the 10GbE card. Any ideas why I am not seeing higher transfer speeds? Anything I can check?
I'm having an issue with my setup. Got a couple chelsio n320e cards and a dac cable to go between a freenas box and xenserver. The nics show up in freenas but media status just shows "No Carrier". Light on card blinks at a steady rate. (That's in the freenas box) On the xenserver box the nics show up but I get no lights on the card when the cable is connected... hmmmm
Hmm. Yeah, I was thinking that was a possibility but I connected the same cable to and the sfp port of an edge router and to a switch and got activity that way. Guess it could still be the cable. May have to try to get ahold of another cable.
how safe is that?As i want to build a nas for home backups etc.... I built one for my father with ECC it's just the price for ecc board mem etc makes the build that much more expansive.
you are wrong. this is not how CoW behaves if it would behave this way you will find ZFS to be crashing after at most few weeks and BTRFS would survive at best a week. There will be a copy made on write if you would be writing to file, which is either inside snapshot and have not been modified from last time is was snapshoted or if you would duplicate it (not copy), otherwise it will just write to file and no copy will be made. Do not talk about technology, which you do not understand
Another cheap option for point-to-point 10 GbE is to get some fiber SFP+ modules - get the kind with 850nm multimode LC connectors and you can get super cheap fiber on eBay (under 20 bucks for 50+ feet).
It works really well. I have my two workstations connected to a 2 port card on my FreeNAS server.
Some performance issues on ZFS with HDDs can be attributed to not running a really fast ZIL. (grab a intel 900p ssd and use 50~GB for ZIL and the rest for ARC cache)
PLEASE DONT USE 50GB for ZIL unless yur raidz handles ~50gb per 5 sec
Yes, I watched a zil-less ZFS box eat 3 2 tb drives; ruin them.
thanks for this video I am planning to do same in my home network soon
Nice tutorial. What Google Chrome version or skin you are using that have movement animation? And what software for network topology you use? Cheers, amazing work, im a starter on FREENAS myself.
I found this video doing research on SFP+ because I wanted to deploy my NAS off-site at my neighbors house. What kind of cabling and modules would you need hang or bury between the homes?
Fiber is good for longer distances.
@@LAWRENCESYSTEMS I was hoping for a fiber cable type. Below is what I think I need.
Adapter: Chelsio T520-CR on the used market it seems like a good LR card for around $150.
Cable: LC UPC to LC UPC Duplex OM3 Multimode outdoor rated (can't find any, only OM2 breakout)
Transceiver Module: Chelsio SM10G-LR
The only reason I think I need LR is every cable I see maxes out at 30m and my run is just inside of 100m, maybe I should just run CAT6A but I like to learn new stuff and I would like an off-site backup.
thanks for sharing, i have a question right now i am building a lab with some vms and freenas as storage server, i got a san switch cisco and i want to create the san network over there, i know that for connecting this to the switch i need to create a zone then add the WWPN on it, my question is can i set the WWPN or the WWNN to the freenas initiators or how do i map the luns to the specic servers on the san network using the swith.
I have the same setup, as inspired by this video! Once I add the 10g nic, my jails that I want to use my regular nic, looses connectivity and says it has no route. I’m trying to figure out where I can bind the jail activity to the 1g network.
Great video, I'm actually looking to try this myself for transferring large Blu-ray videos from my workstation to my Freenas with plex. I'm curious Tom if you saw a performance boost on your VM's when moving from 1Gb to 10Gb?
I am looking into 10Gbe thanks for the great video. Any advice on how to get a full height bracket for these?
I like the Mellanox cards better. Same price/performance but they run a lot cooler! Those Chelsio cards run really hot.
Any link on what good Mellanox cards to get for freenas and windows 10 pro OS?
Great videos! New to pfsense and freenas. Still learning. I've been thinking of trying a 10GB nic in my pfsense box. Using pfsense and a dual port 10gb nic to bridge to the LAN. I'm looking to not get a 10gb switch. Expensive! The nas connect over 10gb to the pfsense and all traffic to the nas would go over the 10gb. Instead of dual 1gb bridged to pfsense and the rest of the LAN. Currently I only have 10gb from the nas to my windows 10 computer with a DAC cable. I would like to get 10gb to my LAN for the nas. Would something like that work? I hope I'm explaining this so it makes sense. Thanks
Can we get the Amazon link to that coffee maker too? 😆
I bought 2 of HP NC552SFP and when I installed on the freenas, I get bunch of memory dump on the terminal and not connected. Why I am puzzle?
For those interested, the official documentation as of version 11.3 U2 and above now states you can use up to 80% for iSCSI ZVol instead of 50% - much better.
I'm getting 650(+) MB/s on my iSCSI link as well... The iSCSI link is from my FreeNAS to a Lenovo server running ESXi. The FreeNAS has two links to my GB switch and the ESXi only has one...
There is something about that report that isn't working correctly.
How do you add static IP to NIC in FreeNAS? By GUI or int the shell?
the web interface under "Network" and then "Interface"
Thanks. It is information I wish to see in your film. BTW yours videos are great. I always lost connectivity to the server when I add new interface by the gui.
+1 for "Xenifer"
Can you use those cards for hooking them up to say a SFP+ Switch with Fiber with normal IP networking? thanks
Yes
OK thanks, also can you use that specific card for windows 10? Thanks :-)
I am having the worst luck with this exact card when installing to Windows 10. Drivers are installed and card is recognized. I see my network and can access, but as soon as I access a file on the freenas box, i get the BSOD saying that NDIS.SYS is the problem. Could you tell me which driver you used for this card?
Took a look on the Canadian amazon site and this is the only card listed by chelsio for less then $CDN1000 is it the same card or compatible?
Hi. Is this setup valid for 2021? I want 10g network next year.
I need a reality check. I want to run UniFi NVR in a vm and have it record to a freenas box (not vm) over a sfp+ 10gb direct connection. Is this possible? Could it be reliable?
if you set it up right, yes.
thanks for all of your videos. can you tell me what's your email solution / server?
+Nuno Silva I used to run my own mail server, but it has become so much easier to have it all in gsuite, which is where my mail is now.
I am new to free nas. Can it be used for ESV VMWare shared storage for 2 or more host servers?
Yes, it supports ISCSI, NFS, & Samba
you should try MTU 9710 as this is biggest currently supported industry-wide - it should work on chelsio too, but I have not tested it, as Chelsio is quite expensive in Europe and I rather sell Chelsio and keep cheap, crappy Intel PRO 10G XF, as it have lower value
Ive been wanting to change to free nas for a while now but not having ecc ram for me has been the issue. Now I see that you are using a non ecc machine. For how long have you been running this configuration and have you ever had issues with bitrot?
There is some much controversy over this but in reality, It is statistically improbable that non ecc memory would cause an array to go bad. This system has been running for a few years without issue, but I ALWAYS recommend backing up to a separate machine as having all of your data in one place is never a good idea.
Lawrence Systems / PC Pickup yea, but I've been more worried about corruption of the files themselves, does that make sense? Additionally, in regards of the backups, I'm aware but I haven't had the cash to make an off-site backup of the critical data.
Lawrence Systems / PC Pickup ok, just found an incredible article about. jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/ that will make me want to switch now xD
Gday Do you know of a good pro-consumer 24 port gig with 2 or 4 10GbE SFP+ ports?
Ubiquiti US-48 Unifi Switch has SFP+ but I don't think any of their 24 port models do.
You are correct, the CSS326-24G-2S+RM has SFP+
Lawrence Systems / PC Pickup - Do you have to buy an addon cage/card for the Mikrotik CSS326-24G-2S+RM 24 port to enable the 10GBE SFP+ ports ? I found the switch on Amazon US for USD$133.33 - www.amazon.com/gp/product/B0723DT6MN
www.netgear.com/business/products/switches/web-managed/XS724EM.aspx#tab-techspecs
“It’s bigger on the inside”, every person that enters the tardis on Dr Who Storage allocates vs Storage used
Check out solarflare cards I got 2 for $40 on eBay they work with pretty much any DAC and transceiver. Support for FreeBSD Linux, Mac OS and Windows.
Got a model number for those cards?
@@jeffheaven9261 rover.ebay.com/rover/0/0/0?mpre=https%3A%2F%2Fwww.ebay.com%2Fulk%2Fitm%2F372571914273 do best offer
I seem to be stuck on only 400 MB\s transfers and can't seem to get any higher speeds on my 10GbE cards between my FreeNas and Workstation. My FreeNas 11 has WD Reds 5400rpm and Workstation has Samsung 960 EVO NVMe. I have set Jumbo packets on both systems to 9000. Not sure why I am not seeing higher, unless it is because my NVMe might be sharing PCIe from mobo PCH with the 10GbE card. Any ideas why I am not seeing higher transfer speeds? Anything I can check?
How you tried testing with iperf to see what the raw network speed is?
I haven't, but will check that out. Basing it only what Win 10 is showing.
I'm having an issue with my setup. Got a couple chelsio n320e cards and a dac cable to go between a freenas box and xenserver. The nics show up in freenas but media status just shows "No Carrier". Light on card blinks at a steady rate. (That's in the freenas box)
On the xenserver box the nics show up but I get no lights on the card when the cable is connected... hmmmm
Check the cables, the light should be solid
Hmm. Yeah, I was thinking that was a possibility but I connected the same cable to and the sfp port of an edge router and to a switch and got activity that way. Guess it could still be the cable. May have to try to get ahold of another cable.
anyone still here that understands the chelsio cards and getting them to work in my windows 10 and windows sever 2019?thanks guys
Ecc?
In the Dell yes, in the FreeNAS no.
how safe is that?As i want to build a nas for home backups etc....
I built one for my father with ECC it's just the price for ecc board mem etc makes the build that much more expansive.
Forgot to say thank you for a great channel and content.
WHAT!?
NO spare cable or cards to show!?
I would never present a solution without spares on hand to show!
Which means, you DO HAVE spares, don't you??
+P V I didn't while making this video
you are wrong.
this is not how CoW behaves
if it would behave this way you will find ZFS to be crashing after at most few weeks and BTRFS would survive at best a week.
There will be a copy made on write if you would be writing to file, which is either inside snapshot and have not been modified from last time is was snapshoted or if you would duplicate it (not copy), otherwise it will just write to file and no copy will be made.
Do not talk about technology, which you do not understand