Home Server / NAS

[Syri]

I'm thinking 2x3tb WD red drives, as they're optimised for NAS use, from what I can gather, unless there's a better option.
I was planning to set them up in RAID 0, as I believe that will work better if more than one person is accessing files at once, but I'm new to this, so not really sure on anything, and open to suggestions.

 

[Bodhi]

Big thanks to everyone in the thread who helped (and those who didn't, as I'm feeling generous) - I now have an Ubuntu Server sitting under the desk merrily sharing files to the rest of the network. Had to reinstall Ubuntu Server as I managed to cock up the LVM and left myself with a 30Gb disk and 450Gb I could do nothing with, but that only took 30 mins, and the issues I had with installing Webmin went away after that.

Only got a 500Gb drive in there at the mo as I need to wait until payday to get any more, but so far so good. I assume it's easy enough to add more storage later on?

Oh and Linux folder permissions. They're fun aren't they!

 

[Poag]

TBH, before you get too far in. I'd look at freenas. Its supporting dockers now and, from my unraid experience, makes life alot easier when it comes to installing applications which require certain versions.

Got to admit i am taking a look at moving off of unraid to freenas, but i have a tendency to toy with things that work and get annoyed so it might be my 'next' server


I've got 16GB of ram in my Gen8 now, 2x3TB disks (adding more as they fill up). Got a 240GB SSD where the ODD was using for my docker img and the supporting docker configs, also my scripts n stuff.
Next big thing will be replacing the Celeron with a 1245v2 which is the top end the mobo can take, the v3 wont fit.

http://ark.intel.com/products/65729/Intel-Xeon-Processor-E3-1245-v2-8M-Cache-3_40-GHz

 

[Trem]

Oh and Linux folder permissions. They're fun aren't they!

Seeing as we are meant to be familiar with Windows the permissions in Windows Server can be AIDS riddled and utter pish (Jock word there for you skirt lad) as well but at least this way you have spent some time learning something new

 

[Bodhi]

Seeing as we are meant to be familiar with Windows the permissions in Windows Server can be AIDS riddled and utter pish (Jock word there for you skirt lad) as well but at least this way you have spent some time learning something new

Aye, I've learned that when in doubt - 777 everything

 

[caLLous]

Ugh. Don't 777 everything. At the very most, 775 everything and add your user to whatever group owns the files/folders. Setting 777 is horrible.

 

[Trem]

I don't know what 777 is

 

[soze]

Linux speak for Full Access for Everyone (I think) I know it is permissions and bad practice.

 

[caLLous]

Yup. Each number represents a level of access: 4 allows read access, 2 allows write access and 1 allows execution. You then add those up to give a total for each scenario. So 7 is allow everything, basically. So, for 777, the first 7 is the owner of the file, which is fine, the second 7 is the group of the owner of the file (which means you can give full access to other members of that group) and the third and most silliest one to set as 7 is "everybody else".

It's octal notation for what you see when you do a ls -l in terminal in a Unix OS - you will see on the left hand side something like -rwxrw-r--, which is the symbolic notation. This splits up to "- rwx rw- r--". The very first character is either "-" or "d", which denotes a directory (or not in this case), the first group of 3 characters after that (rwx) means the file owner has full access (7), the next 3 (rw-) means members of the same group as the file owner have read and write access and the last 3 (r--) mean everybody else only has read access (they can't change the files at all). So the octal notation for that lot would be 764.

See here for more infos. At the end of the day setting the last of the 3 digits to 7 is unnecessary and horrible. You can set them (with chmod) to any number between 0 and 7 and it will be valid:

0 = nothing, not even read,
1 = execute only,
2 = write only,
3 = write and execute,
4 = read only,
5 = read and execute,
6 = read and write,
7 = read, write and execute

 

[Bodhi]

Is it so much of an issue if I'm the only person accessing it? The OH won't be using fileshares, just Plex etc.

2 other questions:

1) has any tried using subsonic? Plex seems great for video but a bit flaky with music, got a couple of DLNA devices that can't pick it up.

2) what's the best way to back the server up? Back up the fileshares in Windows or is there some way of doing it directly off the box?

 

[caLLous]

To be fair it doesn't matter if you're the only user. Having said that, I don't even give myself write access from the account I use to access the archive shares from my Windows box. I've seen too many stories about people accidentally wiping huge swathes of files and I'm quite paranoid about that sort of thing. Only for the archive pools though, I imagine that would be annoying if you're regularly moving files over or deleting stuff or what have you.

 

[TdC]

No, not really, it doesn't matter...but it is certainly a good practice to set things up properly.

Tbh, if it were me and I wasn't doing anything fancy I would just use 755 or rwx r-x r-x on everything. Why everything? Well, in Unixland you need to have execute rights on directories in order to be able to "see" and manipulate what's in them. Think of a directory as a list of locations of things rather than a folder containing things and you're totes there brah. Fun fact: you can remove read-only files in a directory that you have execute on. The OS will ask you if you're sure, which even in modern Linux is fairly exceptional.

andrew@work:~$ mkdir test
andrew@work:~$ touch test/foo
andrew@work:~$ ls -ltr test
total 0
-rw-r--r-- 1 andrew andrew 0 Dec 12 18:47 foo
andrew@work:~$ ls -ltrd test
drwxr-xr-x 2 andrew andrew 4096 Dec 12 18:47 test
andrew@work:~$ chmod u-w test/foo
andrew@work:~$ ls -ltr test
total 0
-r--r--r-- 1 andrew andrew 0 Dec 12 18:47 foo
andrew@work:~$ rm test/foo
rm: remove write-protected regular empty file ‘test/foo’? y
andrew@work:~$ ls -ltr test
total 0
andrew@work:~$

andrew@work:~$ touch test/foo
andrew@work:~$ chmod 444 test
andrew@work:~$ ls -ltrd test
dr--r--r-- 2 andrew andrew 4096 Dec 12 18:52 test
andrew@work:~$ ls -ltr test
ls: cannot access test/foo: Permission denied
total 0
-????????? ? ? ? ?            ? foo

tbh I was surprised that the ls command had any output at all. it's likely that it's coming from the inode cache or some other source.

 

[TdC]

aha! it wasn't a cache, it was rights: I *was* allowed to read the list of files in the directory named test, but I wasn't allowed to do anything with them or access file rights or time data.

if I take the r away, no dice

andrew@work:~$ chmod 200 test
andrew@work:~$ ls -ltrd test
d-w------- 2 andrew andrew 4096 Dec 12 18:52 test
andrew@work:~$ ls -ltr test
ls: cannot open directory test: Permission denied

I am allowed to write in the directory, but also nada I am allowed to write, but the lack of execute does not allow me to manipulate objects in the directory!

andrew@work:~$ touch test/bar
touch: cannot touch ‘test/bar’: Permission denied

Put execute back aaaaand....

andrew@work:~$ chmod 300 test
andrew@work:~$ ls -ltrd test
d-wx------ 2 andrew andrew 4096 Dec 12 18:52 test
andrew@work:~$ touch test/bar
andrew@work:~$

everything works just fine. I've now made a one-way window into the directory. I can put things in, but I can not see them. If I know their exact names I can remove them. A bit pointless tbh, but a good demonstration of Unix filesystem permissions oddness

 

[Bodhi]

And also a good demonstration of why most people use Windows

 

[MYstIC G]

If you've gone Linux then it might be worth looking at off site backups using something like crashplan

 

[Trem]

Some amazingly detailed answers, thank you. Unfortunately I may remain flacid for life now

 

[Bodhi]

Having a bit of a weird issue with the Ubuntu box - file transfers over Samba have gone pathetically slow - like 1Mb/s over 100Mb LAN.

Thought it was just Windows 10, but my work laptop is doing the same.

Anyone seen anything similar before?

 

[Embattle]

I'm heading towards 70% storage capacity so it'll soon be time to consider my options, I suspect something like an upgrade to 1815 or the newer version might be a good choice.

 

[Hawkwind]

Having a bit of a weird issue with the Ubuntu box - file transfers over Samba have gone pathetically slow - like 1Mb/s over 100Mb LAN.

Thought it was just Windows 10, but my work laptop is doing the same.

Anyone seen anything similar before?

Do you really need to use SMB protocols now? Thought that went out the window with any post 3.11.

 

[Bodhi]

Do you really need to use SMB protocols now? Thought that went out the window with any post 3.11.

They help with sharing files between a Linux server and Windows machine.

Anyway I updated to the latest version of Samba on the Ubuntu box and it seems to work much better now.

 

[Embattle]

Well my 1815+ arrived with an additional 4gb ram taking the total up to 6gb. Personally right now I'm moving data around to enable me to shift from Ext4 to btrfs although I suspect you Linux nerds know more about these filesystems than me

 

[caLLous]

Depending on which RAID mode you're going with, you might want to put the brakes on re: btrfs. I still don't think RAID5 and 6 are "production ready".

 

[Embattle]

Yeah I saw something on that, I believe most issues were resolved with 3.19 or something but soon find out I suspect

 

[MYstIC G]

Yeah, what @caLLous said. I wouldn't trust btrfs with anything you're not prepared to risk.

 

[Embattle]

Yeah I've got 2 backups of the important stuff, one external hard drive and one in the cloud.

 

[GReaper]

Been using btrfs for over a year on my NAS - and so far it's working fine for me, although I've had it fairly easy as I haven't had any drive failures yet. I've gone from 2x3TB originally in the pool, to 4TB + 2x3TB + 2x1TB.

Performance can occasionally be a bit weak, but it's not something that's bothered me given my usage of it. I wouldn't use it on a public server I rely on - the sysadmin in me will be sticking to ext4 for years for that!

Various bits of advice would be... Don't use RAID5/RAID6 - only use RAID1. Don't use quotas. Don't use the inplace conversion from ext3 to btrfs. Scrub regularly - at least weekly. Balance regularly - weekly again. Use a decent stable yet modern(ish) distro - Debian jessie does the job for me here.

As always - have backups of important data elsewhere.

 

[caLLous]

It's been said before but now it seems to be more definite: do NOT use RAID 5 or RAID 6 on btrfs if you value the data you're storing with it.

 

[MYstIC G]

It's been said before but now it seems to be more definite: do NOT use RAID 5 or RAID 6 on btrfs if you value the data you're storing with it.

Ouch, that's horrible.

 

[caLLous]

In other news I just upgraded to Ubuntu Server 16.04.1 LTS a few days ago and zfs is now fully included, which is nice. It took me a few heart attacks to realise that zfs is now fully included but once I'd got rid of the old zol ppa and installed zfs "properly" (zfsutils-linux) and re-imported my pools, everything was fine.

 

[MYstIC G]

I'm waiting for FreeBSD 11 and FreeNAS 10 at the moment. Really want to start playing with bhyve and containers using jails.