User Tools

Site Tools


howto:using_lvm2_to_extend_linux_onto_your_sdhc_card

This page is still a work in progress please do not make changes until this warning is removed

Using LVM2 to span the SDD and an SDHC card

Due to the limited space of the SDD inside the EeePC (8GB at most these days), and because most of these SDDs are not upgradable without voiding your warranty, there exists a challenge amongst those of us who yearn for more space, but do not wish to void our warranty. This solution uses an SD Card to basically expand the internal SDD storage, yet still present a coherent filesystem to the Linux OS.

This guide is intended for those who want to setup their system to be the most flexible and are going to be installing a different flavour of Linux anyways.

While it is likely possible to perform this operation on an already installed Linux distro, the process can be difficult (compounded by the fact that the older EEE's have little extra space). The basic steps would be to use gparted (or similar) to shrink your root partition to small enough (you will need to transfer a complete copy of your files from the EEE to a USB key or network share or something), follow the relevant parts of this guide to setup the logical volumes. Next restore your backup over the mounted root and lvm volumes, edit fstab, ensure that LVM2 is installed on your system and that your kernel supports it. Finally, reboot and hope for the best.

Expectations of the Reader

In this document, I expect that the reader has basic understanding of the following. If not, then maybe someone will update this tutorial for you, but for now, you need to understand:

* using disks by their device nodes * partitioning:

  • what partitions are
  • how to create partitions (use the tool you like)

* creating filesystems on a partition

Caveats and Issues

The following are things that you need to be aware of before you start:

  • This procedure will wipe the entire drive of your EEE PC, and is best performed at the time of install of a non-standard OS. This tutorial is based on Debian, but will work equally well for Gentoo or Ubuntu (both of which I have use LVM with), or any Linux whose installer supports and detects LVM.
  • Because LVM presents the combination of the SDD and SD card as logical devices, both physical devices need to be available at boot if you want to use hibernate or suspend functionality. At the moment, since the Card Reader is USB based, this is not possible because the USB subsystem is relatively slow to come back up. I personally do not use hibernate and suspend as I find I can tune the boot process of my chosen distro to bring the machine up in fast enough time (not as fast as the default Xandros, but good enough). Hibernate and Suspend are not recommended with this setup (if someone has more info on how to get this working, by all means please change the appropriate sections of this tutorial).
  • SDHC cards are slower than the SDD to write to. You may notice the occasional lag with this setup because of that. However, using a Class 6 (the fastest available on the market today) SDHC card will alleviate this as much as possible. I actually don't actually add my SD card to LVM until after the default OS and my chosen set of applications are installed. This helps to ensure that as much as possible is running off the SSD.
  • You can theoretically substitute a USB stick for the SDHC card, but I haven't tried that. If you do, make certain that the USB stick is fully USB 2.0 compliant, or you will have a significant bottleneck.

Introduction to LVM

If you don't know what LVM (logical volume management) is then it is highly recommended that you read Redhat's excellent introduction at http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin-guide/ch-lvm-intro.html before continuing.

What you need

You will need the following in order to complete this setup:

  • An boot disk for a Linux that supports LVM2 and has the tools for your chosen filesystem available. The Debian USB install image from http://wiki.debian.org/DebianEeePC/HowTo/Install is what I used. You only need this to initially set up LVM.
  • An SDHC card or USB stick to use as a second physical drive. (Remember this has to be inserted before every time you boot after this procedure is done).
  • Once LVM is set up, you will need to install a linux distro that supports LVM2. Most will, but some of their installers don't which is an issue. See below for a list of distributions that both support LVM and have installation disks that recognize LVM automatically.

The process

WARNING this process will destroy all data currently residing on the SDD and SDHC card or USB stick you are using as an extra drive. Make sure you back up the contents of these first.

Preparation

  1. Insert the CD or USB Drive that contains the boot disk with LVM support
  2. Power on the EEE and hit ESC right way to select this disk to boot from (you may need to disable Boot Booster in the BIOS if you have it turned on)
  3. If you used the Debian USB linked to from above, just hit enter to boot the default.
  4. Open up a console (Xterm or otherwise). On the Debian Image referenced above, once you get to the first installation screen press Alt-F2 to get to a console.
  5. If you know your boot disk has LVM support built-in to the kernel or init scripts move past these steps, otherwise we need to ensure that the LVM modules are loaded:
    1. On Ubuntu: you will need to install the lvm tools, so ensure you have a net connection and execute:
      ~ # sudo apt-get install lvm2

      . Gentoo and Debian have these already installed.

    2. Execute the following to see if the correct module is loaded:
      ~ # lsmod | grep dm_mod
    3. If nothing is returned then you need to load the module:
      ~ # modprobe dm_mod

      and execute the previous step to ensure the module was loaded.

  6. Next we need to figure out which drive is the SDD and Which one is the SDHC Card. This can be done with the fdisk tool:
    ~ # fdisk -l

    You will get output similar to the following (NOTE: this is my output after setting up LVM. If someone has an example with the original partitions, feel free to place it here, instead of mine):

    Disk /dev/sda: 4001 MB, 4001292288 bytes
    255 heads, 63 sectors/track, 486 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0x00058e84
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1          32      257008+  83  Linux
    /dev/sda2              33          95      506047+  82  Linux swap / Solaris
    /dev/sda3              96         486     3140707+  8e  Linux LVM
    
    Disk /dev/sdb: 8195 MB, 8195670016 bytes
    253 heads, 62 sectors/track, 1020 cylinders
    Units = cylinders of 15686 * 512 = 8031232 bytes
    Disk identifier: 0x000659d3
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sdb1               1        1020     7999829   8e  Linux LVM

    I can tell that /dev/sda is the internal SSD because of the size (4GB). /dev/sdb is obviously the SDHC card because of it's size (8GB). These device nodes may or may not be the same, depending upon how you booted and you may see other devices as well (for example if you booted from a USB stick, you may see another /dev/sdX device).

  7. Now that you have your device nodes, it is time to figure out our partitioning scheme. The goal here is to maximize the storage capacity across both disks. That being said, we still need one or two non-LVM partitions for the machine to boot without a lot of extra work (apparently you can get a linux install going with / running on LVM, but I have never tried it, and you may need to patch the bootloader). Basically, I recommend that we create mount points for the following:
Mount point Partition Type Notes Recommended size for Debian
/ physical Although possible booting with a root or boot FS on LVM is out of scope for this tutorial. 128MB (without seperate /boot), 100MB (with separate /boot)
/boot physical Optional, but a good idea for extra security, because you won't mount it unless you explicitly need to change something. I do this on Gentoo, but not on Debian/Ubuntu, because on the latter an initramfs is used and will be updated when certain packages are installed. On Gentoo I use a custom kernel without an initramfs, so I only need to mount this partition when I recompile the kernel. 30MB
/var LVM Debian uses a lot of space initially in /var for temporary storage of install files, this can be resized if you distribution is not Debian based, or uses less /var space on installation. 1.5 GB
/usr LVM This is where most programs are installed to on most Linuxes. This figure will vary slightly depending upon Desktop choice and application choice 2.0 GB (I am using 60% of this with a very basic KDE installation)
/opt LVM This is where external third-party components generally go. With Debian based stuff, I haven't had a lot of stuff go here. With Gentoo, if source hasn't been available for a package, pre-compiled packages willgenerally go here. 100 MB
/home LVM unneeded for installation, but is best to create it at installation so that you don't have to move files around later. At least 5MB/user for personalized KDE settings, 2MB for personalized Gnome settings. If you are running a Firefox based browser, then add 40-50M/user (why? read this). Other applications will require more.
swap physical or LVM If you want to take a crack at getting suspend/hibernate working, then make this physical and at least the size of your RAM. Otherwise it is optional and depends only on what you run. I have 512MB of RAM and have yet to have my machine ever swap out. I have successfully run See notes.
/tmp LVM or tmpfs Pick LVM if you have a distro that likes temp space or want your /tmp to persist through reboots. I use tmpfs and am currently utilizing only 36K
/var/log LVM or tmpfs tmpfs is a pain if you need to do kernel debugging, etc as the directory is reset on each reboot. I use tmpfs and am currently utilizing only 200K
/var/tmp LVM or tmpfs Pick LVM if you have a distro that likes temp space or wants/needs your /var/tmp to persist through reboots. I use tmpfs and am currently utilizing only 1.1MB

At this point we are ready to setup the LVM and partitioning scheme. You will be creating physical partitions for any drive that is marked physical above. Remember that each drive can only have 4 primary partitions, if you need more then you'll need to create extended partitions under one of the primaries. I highly recommend that you place / and /boot on your SSD, as booting will be much faster from this media. If you are going with physical swap, then either the SSD or SDHC will work (SSD is probably better as it is a faster device). In my setup, I am using a physical / and an lvm swap. The rest of the space is setup for LVM.

Partitioning

So now you have figured out what your plan is (which mountpoints are physical and which are logical volumes). The next thing to do is to create the partitions for your physical volumes and one for each disk that needs to be part of LVM.

In my situation, the only mount point I had chosen to be physical was /. The rest of my disk space is going to be used by logical volumes. So I have created a 256M primary partion on my SSD (/dev/sda for me) as type 83 (Linux) and the rest was allocated to a second primary partition with type 8E (Linux LVM). I partitioned the SDHC card as a single primary partition with type 8E.

Configuring LVM

The last step before finishing your installation is to set up LVM. Before we start, here are some terms you need to be aware of: * Physical Volume - AKA pv - These are the physical partitions that will be used for hosting the lvm volumes. * Volume Group - AKA vg - this a group of physical volumes that are grouped together to look like a single disk * Logical Volume - AKA lv - this is a “partition” on the volume group. To the OS, it looks and behaves like a normal disk, although there may be a number of actual disks (PVs) supporting an LV.

The first step is to ensure that the correct modules are loaded into the kernel and that the tools are available in your environment: - Ensure that mod_dm is loaded:

modprobe | grep mod_dm

If something is returned, then it is loaded. If not continue. - Load it is it isn't:

insmod mod_dm

Some installation environments may or may not have the lvm2 tools installed. Debian and Gentoo install disks include them. Ubuntu does not. On Ubuntu simply run

sudo apt-get install lvm2

Next we need to tell the device mapper (mod_dm) and the lvm tools about the PVs we will be using. To do this we will call the pvcreate command.

# pvcreate <partition device>

Where <partition device> is the device node, such as /dev/sda2, that will be used for LVM. We need to call this command once for each partition that we want in the logical volume array.

Now we need to create a VG that maps the LVs to the PVs on which they will actually be stored. The volume group is conceptually just a big area of unpartitioned disk space that logical volumes are carved out of (LVs are like partitions). You can have multiple VGs if you wish, but for most desktop installs, there is not much of a reason to go with more than one. To create this volume group, we use the vgcreate command.

# vgcreate <vg name> <device1> <device2> ... <deviceN>

Where <vg name> is the name you wish to refer to this volume group by. It can be anything that is supported by a regular file name, but I suggest you keep it short and sweet, as it creates a device node called /dev/<vg name>. I called mine “vol1”. The device parameters are the space delimited list of PVs that will support the volume group. I used /dev/sda2 and /dev/sdc1.

As a quick aside, you can set up LVM just on your SSD right now and not worry about the SDHC card. You can do this if you do not have an SDHC card at the moment, or if you want to ensure the base install is definately installed on the SSD. When you wish you can add the SDHC card later by inserting it and issuing the following command (make sure the SDHC is NOT mounted though):

# pvcreate <device>
# vgmodify --add <device>

Where device is of course the device node of the LVM partition on the SDHC card.

Okay, so now we have consolidated all of our physical partitions (PVs) to appear as one (a VG). Now we need to carve up that VG into its own partitions, called logical volumes (LVs). It is these LVs that will have the filesystems written to them, and which will be mounted to the mount points. To create a logical volume we use…you guessed it! lvcreate:

# lvcreate <some options>

You need to run this command once for each LV you want (refer to your planning table). Here are the commands I issued for my device.

At this point we need to continue with the installation of the OS.

Configuring the (soon to be) installed OS

As each installer is different, the actual steps will be different for each linux flavour, however they all will end up doing the same things: - Format the each LV with the appropriate filesystem - Mount each LV to the place it is supposed to reside with on the end system - Install the components of the OS - Compile the kernel with device mapper and LVM support, or update the initramfs to support this - Update the target system's /etc/fstab to mount these logical volumes on boot

During this phase, you will need to determine which File system you are are going to use for each LV. Here are a list of the currently popular filesystems and how they relate to LVM and the EEE: * ext2 - supports growing and shrinking, and as such is a a decent choice for LVM, however, it does not journal, and has a number of other limitations, so I personally do not recommend it for an EEE. * ext3 - supports growing and shrinking. Because it journals it is a better choice for an EEE, however I find that EXT3's implementation of journalling eats up too much disk space for such a small disk, so I avoid it. * reiserfs 3 - supports growing and shrinking. It also journals, and has less journalling overhead than ext3. Unfortunately, it was written by a convicted murderer. It is currently my choice for now, due to limitations of other journalling file systems. * JFS and XFS - supports growing the FS. Has less journalling overhead than reiserfs. Because I like to reclaim disk space occasionally (I am a tinkerer), I don't use this filesystem because I can't shrink it. If there ever is shrinking support, I'll probably move away from reiser.

I have experience with the top three filesystems in this list and have never lost data due to a bug in any of them (and in fact have not yet lost data in EXT3 and Reiser thanks to journalling) and they are all stable and well supported byt the major distros. JFS and XFS look extremely interesting and promising to me, but because of the lack of shrink capabilities, I have not used them in any significant fashion. Others feel free to post your experiences on JFS and XFS here.

There are other up and coming filesystems (ext4, brtfs and arguably reiser4), but I have no experience with these so cannot comment.

insert OS specific instructions here

howto/using_lvm2_to_extend_linux_onto_your_sdhc_card.txt · Last modified: 2009/03/10 13:50 by albkwan