Safe and Easy Data Storage Outside the /home
Don't crowd your /home! Use Symbolic Links to access organized data storage.
Having one large /home is like putting all your eggs in one basket, so to speak. If you drop that basket, you've lost it all.
I have both /[root] and /home on a single 10GB partition, which is more than I even need. The rest of my data, [not operating system data!], I keep on separate partitions. If you screw up /home by user tweaking, experimentation —whatever— it can easily be replaced.
If you keep all your personal data on /home as well —can it be replaced as easily?
— a MepisLover known as “traveler”
The Argument: Your Command Centre is not a data dump!
The general organizational scheme of GNU/Linux is derived from Unix, the classic mainframe operating system where many users shared one large computer with a unified filesystem. Under the /home directory, every user who had login access had a personal directory with a certain amount of space to save her/his private files.
Small-office and home users of todays GNU/Linux OS's usually operate in a far different environment. Personal computers may have more than one user — but often not. Even when there is more than one user, computer access is almost always by one user at a time, except in server set-ups. In the typical program-rich environment of the Linux Desktop Computer, the most vital function of the /home directory is to preserve the program configuration settings of each user.
This is a distinctly different function from the storage of accumulated personal data files like emails, pictures, music, videos, downloaded html pages, text documents, spreadsheets, and so on.
Traveler is right in one way: the vital information in /home is really part of the Operating System —as far as the user is concerned. The /home/user directory exists primarily to configure the OS for optimal personal efficiency. That task cannot be done anywhere else [except by the root user]. The storage of collected data is a secondary function of the /home/user directory, not an essential one. Collected data can be stored in any directory on any partition of any disk. All the user needs is full rwx permission for that directory.
Unlike traveler, I do not recommend putting /home in the same partition as the OS filesystem root. I have encountered situations in which an OS reinstall was desireable —and pretty easy, because my configuration data was safely in another partition. Reinstalling the OS took between 10 and 15 minutes, and I just told the installer to use the existing /home partition. The point is: my application configuration files did not get wiped out along with the old OS, because they were separated, not mixed together.
For the same reason, it is prudent to separate the /home configuration data from other classes of personal data. Because, really, if you separate /home from OS to save a half-hour's work reconfiguring your favorite apps, why ignore the hundreds of hours of work you'll spend in trying to reassemble all the other accumulated data. . . if it's suddenly gone?
And it's not just safe — it's convenient! When I boot into PCLinuxOS or Vector, I have full access to all the same web-authoring projects, all my music files, graphic files, downloaded email, and filesharing uploads/downloads that I have in my workhorse Mepis OS.
For historical reasons that do not really apply to modern desktop computers, the prevailing practise is for the /home/user directory to be the default location where every file you generate gets dumped. Good in 1987 on a Unix mainframe with 50 users. Bad here, now, on your personal computer.
Linux distro developers have many more immediate problems to work on; they are not going to restrategize personal data storage schemes. So for the forseeable future, until GNU/Linux becomes the preferred OS among the majority of the technically literate population of computer users [SilverBear's crystal ball says: 2012 or so], redesigning your data storage and retreival system is up to you!
I've been successfully using several separate data partitions with no inconvenience. The power of Symbolic Links in a Linux system is available to all Linux users. You can use it to make your data safer and just as handy as it is now —if not more so.
Recap of “The Argument”:
- Make the OS root partition and the /home partition separate.
- Make your /home partition and your saved data partition separate.
- Use symbolic links to access your data partitions [wherever thaey may be] from ithin your /home user directory.
Overview:
1. Ask yourself, “ What is a directory?”
A directory is a list of where things can be found.
Contrary to the Microsoft “let's dumb it down for them” characterization of a filesystem directory as “a folder” in computer storage filesystems, a directory in a digital storage unit is not a contaier. It is a list. You cannot put anything physically “inside” a directory.
To learn more about how digital information differs from paper when it comes to directories, right-click to open this link in a new tab or window for this SilverBear Linux page on the technical details of Unix/Linux Directories
2. Deprogramming your Microsoftened mind:
Forget “directory = folder”. You're not a file clerk. That's a screen in front of you. Think Visual. You're watching television. Directory = list of channels. Get it?
KMenu = remote control selection for applications.
/home/user = remote control selection for stored data files.
The apps are not physically “in” the K Menu. The stored data files do not have to be physically “in” /home/user. They are both directories, each with its own purpose.
3: Visualize an efficient data storage schema
Relax. Visualize, then design your Big Picture. Then use these tips to implement it. A directory is a list of how to find what you want. Remember that the purpose of storing data is to keep it in a safe place. You want to be able to find it, and you want to be able to copy it to backup locations. Everyone's data organization scheme will be different. But everyone's data organization scheme can be easily connected to the /home/user remote control. Look at this graphic for “the big picture:”
Illustration: Most data [other than your hidden .config files] can be outside your /home/user directory, while functionally appearing to be inside it.
Each of the categories on the left can be in different partitions or different disk drives, all accessible from within your /home/user directory.
4: Use a piece of paper to make notes on how to repartition your disk drives for compartmentalized storage according to function.
So you don't forget what's what while you're repartitioning your HDDs, make notes of each disk and it's proposed new partitions. Here are some good rules of thumb, but your circumstances may call for variation.
- Smaller partitions are easier to move and back up. But with the newer Linux kernel 2.6.20 and higher, all HDDs are treated as if they were Serial devices. So the maximum number of functional partitions is 15, not 63 like IDE drives under earlier kernels.
- If you're dual-booting Windows, it is going to be happier in sda1 than antplace else.
- If possible, keep all your OSs on partitions of sda. The /home partitions that go with them can be on another HDD if necessary. If you aren't booting too many OSs, the root partitions and /home partitions can all fit on sda.
- Allocate 6-8 GB for a Linux OS root partition and 8-10GB for its corresponding /home partition.
The ext3 filesystem format works most efficiently when less than 80% full. These sizes leave room for additional programs and config files to be installed, plus 20% free space. Since your data storage is going to be on other partitions, /home does not have to be more than 10GB. If you are nervous, make it 20GB, just so you don't crack under the strain ;-) - Although data storage partitions can be any size, in practise sizes over 40GB are harder to back up or move if you need to do that someday. And chances are you will.
- If you have 1 GB or more of RAM you do not need to make a linux-swap partition larger than 1GB. I use the same linux-swap partition for all my Linux OSs, and I have never had any problems. Put your swap partition either on a sparate HDD or else in a partition that is physically close to the OS that uses it. Otherwise your R/W head will be skittering back and forth along the entire surface of the HDD.
Phase One: Setting up Data Partitions
Before you can symlink them, you need to set up your data partitions so that they are acessible. These instructions are intended for a personal computer where there is no need to grant or to limit access to files from other users who may log in. If other users logged in under different usernames are a consideration, post in the SBLinux.org forum and I will address changes that need to be made in your case.
Automatically mounting a partition when the computer boots
The configuration file that governs the mounting of partitions to the root filesystem is /etc/fstab —and since it is a plain text file, it can easily be edited to suit your needs. Like all system files, editing it requires root privileges. First, back it up!
If you don't already have one, create a subdirectory for storing backups of various system files inside your /home/user directory with this CLI command:
mkdir ~/backups
Then copy /etc/fstab by issuing this command:
cp /etc/fstab ~/backups/fstab.original
Brief tips for changing your fstab
- Every partition should only be listed once. If you goof and list a partition twice, lines lower down in the fstab list will be the ones that stick if the two lines have instructions that are contradictory, like automounting versus mounting only by a root mount command.
- Lines that deal with partitions formatted with different filesystems may have different mounting options than a standard Linux ext3 filesystem. This is especially true with a FAT filesystem from Microsoft [in a Linux fstab it is called vfat].
- There are three ways to designate a partition: by bus device, by UUID and by label. Each line must be consistant internally, but different lines need not be all of the same designating schema. This is especially good to know because an OS that likes to use UUIDs by default will cause trouble if you reformat a partition: it will get a different UUID. It might be easier to go back to the good old “/dev/sda6 /mnt/sda6” schema just for that changed line that mounts the newly formatted partition.
- All partitions that are symlinked to your /home/user directory shgould be automounted at boot time with full read-write-execute permissions, unless there are specific reasons why not.
Sample fstab lines
To automount an ext3 formatted partition with user read/write/execute permissions:
To automount an NTFS formatted partition with user read/write/execute permissions:
To automount a FAT formatted partition with user read/write/execute permissions:
Two steps to basic implementation
1a] Creating a Symbolic link: using the CLI
As a regular user here's what you can do:
ln -s /mnt/hdb3 /home/silverbear/graphics
Of course, since you are not the SilverBear, I'd suggest putting your own username in. But this creates a symbolic link in /home/silverbear that is named “graphics”.
I click on it, and all the directories and files contained in /mnt/hdb3 show up. I can open files, and save files, and copy files, and write new files in this "directory" in my user home just as if that's where they were, physically. Except they're not. They're in hdb3!
If something were to happen during an upgrade, or beta-test, and my entire /home partition got put thru the digital wood-chipper, none of my graphics files would be affected. And with my sounds stored and accessed via the symbolic link I created by typing:
ln -s /mnt/hdb1 /home/silverbear/mp3-flac
. . .I don't lose my 10 year's worth of collected digital music, either.
1b] Creating a Symbolic link: using the KDE GUI
If you don't like working with the Command Line Interface, the KDE file manager Konqueror mades it easy to create symbolic links.
- Open Konqueror in the split-pane mode: View > View Mode > Detailed List View and then left-click the red root folder icon in the left margin.
- Left-click the + sign to the left of the mnt directory folder icon to see all the partitions on your computer.
- Drag the directory folder you want linked into your /home/username directory and when the mini-menu pops up, choose link here, not move here or copy here.
- Rename the link in your home directory to anything you want to call it. Or leave it sdb8 [or whatever] if you need a reminder of where that partition exists.
2] Automounting data partitions
For this system to work, you need:
2a - Separate data partitions where your files are stored.
2b- Those data partitions must automatically mount at boot time so they always will be accessible.
2a: Plan ahead.
If you have a 10GB partition, the most you want to put on there is 8GB. The ext3 filesystem likes to have 20% free space for "housekeeping." That's an oversimplification, but the 20% is hard fact. So if you plan on adding more to a partition you're creating, say, as you convert your vinyl collection to digital flac files like I'm very slowly doing, you need to leave extra room.
Existing music = 6GB
music to come = maybe 10GB
16GB = 80% —so, algebraically: 16/x = 80/100
160 = 8x
20 = x
q.e.d.: You should make the partition 20GB, rounded to the nearest cylinder.
If you have a large HDD, make all your data partitions a standard 40GB. Leave one blank alll the time, so it can be used to hold the contents of another if you need to rescue the data off one and reformat it. See the link in the middle of the page, above, for sample partitioning schemata.
2b: Automount the necessary partition!
In my fstab I can automatically mount my hdb1 partition with the line:
/dev/sdb1 /mnt/sdb1 ext3 users,async,atime,auto,rw,exec,dev,suid 0 2
Change the first two columns [the /dev and the /mnt specifics] to match your own partitions, and everytime you boot , this partition will be mounted with read, write and execute permissions.
So as to not make this too long, I won't go into detailed explanations of all the options in the 6 columns in the fstab. For that, go here:
RECAP:
- make your data partitions first.
- change your fstab to automount those partitions to which you want your symbolic links to point.
- make sure you have each hard disk partition [ /dev/sdx ] listed only once!
- run the ln -s command in your user shell, not as root, so that you will set the correct rw permissions, like those in the rest of your home directory.
Internet Explorer?