ZFS

Deebs

Chief Arsewipe
Staff member
Moderator
FH Subscriber
Joined
Dec 11, 1997
Messages
9,076,920
A cache makes no difference to the speed of the scrub as it is essentially reading every used block from your pools and verifying the checksum. It can only go as fast as your read speed of your pools.
 

TdC

Trem's hunky sex love muffin
Joined
Dec 20, 2003
Messages
30,801
hmmmkay that makes sense; my bad, I never actually checked!
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
Ok good, I won't bother with an SSD at all then. I've got 32gb of ECC RAM in there and it's not gone higher than about 20gb used with everything I've thrown at it so far.

Have some pictures. I just liked the lighting in the first one (light shining through open top of case) and you can see the LSI card. Number 2 looks awful but there's very little airflow in the middle bit of the case so I'm not fussed. Number 3 is because @Deebs likes wheels. :)

20140517_183200.jpg

20140517_183229.jpg

20140521_133258.jpg

You could honestly fit a child inside the case. I haven't tried. Honest.
 

TdC

Trem's hunky sex love muffin
Joined
Dec 20, 2003
Messages
30,801
has anyone ever confronted you about your storage fetish, callous?
 

Deebs

Chief Arsewipe
Staff member
Moderator
FH Subscriber
Joined
Dec 11, 1997
Messages
9,076,920
Ok good, I won't bother with an SSD at all then. I've got 32gb of ECC RAM in there and it's not gone higher than about 20gb used with everything I've thrown at it so far.

Have some pictures. I just liked the lighting in the first one (light shining through open top of case) and you can see the LSI card. Number 2 looks awful but there's very little airflow in the middle bit of the case so I'm not fussed. Number 3 is because @Deebs likes wheels. :)

View attachment 18739

View attachment 18740

View attachment 18741

You could honestly fit a child inside the case. I haven't tried. Honest.
Using an SSD for the ZIL (making it an SLOG) will massively help with synchronous writes but I suspect you are not running NFS or ESXi and iSCSI so it will not matter one bit, all writes are grouped into a transaction group which by default is flushed out every 5 seconds to the pool. If you do not have a SLOG then your ZIL lives in reserved areas on your pools and it is this which kills synch performance as HDD are slow. Another thing to note, the ZIL exists in RAM too, the storage is only used to back synchronous writes (POSIX).
 

TdC

Trem's hunky sex love muffin
Joined
Dec 20, 2003
Messages
30,801
the leccy company prolly sends a senior exec with the bill every now and then. they drink champagne and chat like old mates..
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
@TdC I rejiggled things a bit and look how fast it scrubs now!
Code:
  scan: scrub in progress since Thu Sep 18 12:17:38 2014
  547G scanned out of 28.5T at 702M/s, 11h37m to go
  0 repaired, 1.87% done
 

TdC

Trem's hunky sex love muffin
Joined
Dec 20, 2003
Messages
30,801
haha not bad chum, not bad at all! I'd be jealous but it's unrealistic to compare my ickle 4 disk setup to your infrastructure ;-)
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
RIP the first 4TB Red in my box, annoyingly it was one of the newer ones. Welcome ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1460297!
Code:
scan: resilvered 2.75T in 22h27m with 0 errors on Sun Feb  8 11:52:29 2015
 

MYstIC G

Official Licensed Lump of Coal™ Distributor
Staff member
Moderator
FH Subscriber
Joined
Dec 22, 2003
Messages
12,362
One of mine did the DoA when I switched to 4TB Reds. Thankfully they've been solid ever since.

Did it tell you what it died from?
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
I've not checked it yet tbh, it came up as REMOVED in zpool status and then OFFLINE. I rebooted and it resilved a bit then said FAULTED so I switched it out pronto and set the resilver going on the new one. I'll plug it into this box and see what the SMART stuff tells me. It wasn't even listing in /dev/disks/by-id/ though so I think it might well be terminal. Still well within its warranty though.
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
Weird. It appears to be literally dead. Doesn't spin up at all. Maybe a problem on the PCB, I don't know. This has never happened to a disk of mine before.
 

TdC

Trem's hunky sex love muffin
Joined
Dec 20, 2003
Messages
30,801
the NSA-A-A took your data away :(
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
RAIDZ2 saved me though! Take that NSA-A-A!

Seriously, it's the first resilver I've had to do and it was absolutely seamless.
 

Draylor

Part of the furniture
Joined
Dec 23, 2003
Messages
2,591
So the first sign of trouble from the drive was when it effectively bricked itself? Very odd indeed.
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
Basically yes. It was towards the end of a scrub and the first I saw was that it had come up as REMOVED. As if it thought that the drive just plain wasn't there any more.
 

Deebs

Chief Arsewipe
Staff member
Moderator
FH Subscriber
Joined
Dec 11, 1997
Messages
9,076,920
I had a similar incident a few months ago in one of my NAS. The drive dropped offline so I bought it back online and resilvered, was fine for a week, no S.M.A.R.T. errors, then it dropped again. Replaced with a new RED and all my pr0n is now safe again.

This year is the year I totally replace my 3 NAS units with a big single one, just need to start researching for a case to hold 12-18 disks in a hot swap manner.
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
X-Case do some great 4u rackmount cases. Everyone in America raves about the Norco 4u cases but X-Case are the (better) UK equivalent. I'm looking at the RM 424Pro - Short for my first foray into rackmounting. It's a 4u case with 24 hotswap bays and some very fancy stuff inside for fan control and whatnot.

If you want a standalone case you can't go wrong with the Lian-Li PC-D8000 (I'm sure we've been here before :) ).
 

Deebs

Chief Arsewipe
Staff member
Moderator
FH Subscriber
Joined
Dec 11, 1997
Messages
9,076,920
X-Case do some great 4u rackmount cases. Everyone in America raves about the Norco 4u cases but X-Case are the (better) UK equivalent. I'm looking at the RM 424Pro - Short for my first foray into rackmounting. It's a 4u case with 24 hotswap bays and some very fancy stuff inside for fan control and whatnot.

If you want a standalone case you can't go wrong with the Lian-Li PC-D8000 (I'm sure we've been here before :) ).
Think we have to be honest :) That case is now on my shopping list, just need to find a MB which can take 64mb ECC RAM. Unfortunately my garage is not geared out to take a rack and my tiny office is too much of a shit pit to take a half rack..... or is it.....
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
I'm thinking a nice little freestanding 16 or 18u cabinet for my needs, something like this. Honestly I don't think the footprint would be THAT much bigger than the PC-D8000...
 

Deebs

Chief Arsewipe
Staff member
Moderator
FH Subscriber
Joined
Dec 11, 1997
Messages
9,076,920
Bought a D8000 case, have 14 3tb WD Red NAS drives installed. Just moving all the files (using ZFS Send/Recv of course) and then I am good to go and decommission the old NAS.
 

Zarjazz

Identifies as a horologist.
Joined
Dec 11, 2003
Messages
2,383
Bought a D8000 case, have 14 3tb WD Red NAS drives installed. Just moving all the files (using ZFS Send/Recv of course) and then I am good to go and decommission the old NAS.

Blimey, that's certainly some bulk storage. Not jealous, nope, not me ... :)

What OS you gone with ?
 

Deebs

Chief Arsewipe
Staff member
Moderator
FH Subscriber
Joined
Dec 11, 1997
Messages
9,076,920
Blimey, that's certainly some bulk storage. Not jealous, nope, not me ... :)

What OS you gone with ?
Freenas 9.10 U4
Code:
  pool: datapool
state: ONLINE
  scan: none requested
config:

        NAME                                            STATE     READ WRITE CKSUM
        datapool                                        ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            gptid/3baef772-a6a2-11e6-af00-0cc47ae11ad2  ONLINE       0     0     0
            gptid/3cb98234-a6a2-11e6-af00-0cc47ae11ad2  ONLINE       0     0     0
            gptid/3dc1a42e-a6a2-11e6-af00-0cc47ae11ad2  ONLINE       0     0     0
            gptid/3ec8ddda-a6a2-11e6-af00-0cc47ae11ad2  ONLINE       0     0     0
            gptid/3fcbb00f-a6a2-11e6-af00-0cc47ae11ad2  ONLINE       0     0     0
            gptid/40db2b86-a6a2-11e6-af00-0cc47ae11ad2  ONLINE       0     0     0
            gptid/41e0d274-a6a2-11e6-af00-0cc47ae11ad2  ONLINE       0     0     0
          raidz2-1                                      ONLINE       0     0     0
            gptid/4c02eee9-a75f-11e6-9609-0cc47ae11ad2  ONLINE       0     0     0
            gptid/4cca29c3-a75f-11e6-9609-0cc47ae11ad2  ONLINE       0     0     0
            gptid/4dcbcfd0-a75f-11e6-9609-0cc47ae11ad2  ONLINE       0     0     0
            gptid/4e803c7f-a75f-11e6-9609-0cc47ae11ad2  ONLINE       0     0     0
            gptid/4f3fc79e-a75f-11e6-9609-0cc47ae11ad2  ONLINE       0     0     0
            gptid/4feab5b0-a75f-11e6-9609-0cc47ae11ad2  ONLINE       0     0     0
            gptid/50995aaa-a75f-11e6-9609-0cc47ae11ad2  ONLINE       0     0     0

errors: No known data errors
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
Bought a D8000 case
The D8000 is something else, isn't it? I've filled mine now (including hdd cages on those 3 fan mounts on the back panel) and have overflowed into another case. I remember putting my original disks from the previous case in it at the start and thinking I'd never fill it. :\ It's properly spaghetti inside but I have tidied it up a bit recently.
 

Deebs

Chief Arsewipe
Staff member
Moderator
FH Subscriber
Joined
Dec 11, 1997
Messages
9,076,920
The D8000 is something else, isn't it? I've filled mine now (including hdd cages on those 3 fan mounts on the back panel) and have overflowed into another case. I remember putting my original disks from the previous case in it at the start and thinking I'd never fill it. :\ It's properly spaghetti inside but I have tidied it up a bit recently.
Yeh very impressed with the case. I have a supermicro x11 board with an onboard Avago 3008 which I cross flashed to IT mode. This can manage 8 drives, I also have a 2008 card which is can also manage 8 drives.
 

caLLous

I am a FH squatter
FH Subscriber
Joined
Dec 23, 2003
Messages
18,426
Yup there's no downside to enabling compression with ZFS. A lot of file types won't benefit from it but it's very easy going on the CPU.
 

Users who are viewing this thread

Top Bottom