Zumastor Installation Notes

This document describes how to get a basic zumastor master node with snapshots running on Debian or Ubuntu, and then how to replicate that to a second slave server. Actual deployments may be more complex - the goal here is simplicity to get started.

Common node setup

The following sections are required for both master and slave nodes.

1. Partition and install OS

Zumastor requires some storage space for storing snapshot data (and for the volume itself if you are creating a new filesystem). If you are starting from scratch, install operating systems using custom partitioning, preferably leaving a large LVM volume group (named sysvg in this example) for use when creating origin and snapshot devices. Eg.

pvcreate /dev/hda7
vgcreate sysvg /dev/hda7

2. Build Debian packages

There are three required packages to build: the patched kernel package, ddsnap (snapshot device userspace), and zumastor (management cli) package.
  1. Make sure you have the necessary build tools install, including the following:

    aptitude install debhelper devscripts fakeroot kernel-package \ libc6-dev libevent-dev libkrb5-dev libkrb53 libldap2-dev libpopt-dev \ ncurses-dev slang1-dev subversion zlib1g-dev

  2. Get the latest code:

    svn checkout http://zumastor.googlecode.com/svn/trunk/ zumastor

  3. Run the build script. This will build all three required packages.

    $ cd zumastor
    $ ./buildcurrent kernel/config/full

    Note: This will download the required kernel image from kernel.org (currently 2.6.21.1). The script takes a parameter which is a .config file for the kernel. There is distribution-like config in the repository in kernel/config/full. If you need to use a different config, make sure you enable CONFIG_DM_DDSNAP (y or m)

If you do not wish to use the build script, you can follow the "tarball" installation instructions in the ddsnap INSTALL file. You can also use these instruction only for the kernel install, then build the ddsnap and zumastor packages by running dpkg-buildpkg -rfakeroot in both the ddsnap and zumastor subdirectories.

3. Install packages

  1. Install kernel packages. If you have built the kernel packages about, install using dpkg:

    # dpkg -i kernel-image-2.6.21.1-zumastor-r880_1.0_i386.deb

  2. Reboot on to the new kernel

    # shutdown -r now

  3. Install ddsnap and zumastor packages. First, ensure the dependcies are installed.

    # aptitude install dmsetup libpopt0 zlib1g util-linux cron
    # dpkg -i ddsnap_0.4-r596_i386.deb zumastor_0.4-r596_i386.deb

  4. Ensure zumastor is installed
    # zumastor status --usage
          RUN STATUS:
    /var/run/zumastor
    |-- agents/
    |-- cron/
    |-- mount/
    |-- running
    `-- servers/
        
Now you're ready to start using zumastor.

4. Create block devices for origin and snapshot stores

lvcreate --size 20g -n test sysvg
lvcreate --size 40g -n test_snap sysvg

These snapshot stores are large compared to the origin store since the tests below will create and destroy large amounts of random data with nothing shared between snapshots in most cases. Choose sizes appropriate to your usage.

Snapshot Management Steps

define testvol

On the master node: zumastor define volume testvol /dev/sysvg/test /dev/sysvg/test_snap --initialize

The output should resemble:

js_bytes was 512000, bs_bits was 12 and cs_bits was 12
cs_bits 14
chunk_size is 16384 & js_chunks is 32
Initializing 5 bitmap blocks...
pid = 5879
Thu May 10 13:45:54 2007: [5880] snap_server_setup: ddsnapd server bound to socket /var/run/zumastor/servers/testvol
pid = 5881
Successfully created and initialized volume 'testvol'.
You can now create a filesystem on /dev/mapper/testvol
    

Create a filesystem on testvol

mkfs.ext3 /dev/mapper/testvol

Automate hourly and daily snapshots

zumastor define master testvol -h 24 -d 7

zumastor status --usage

VOLUME testvol:
Data chunk size: 16384
Used data: 0
Free data: 0
Metadata chunk size: 16384
Used metadata: 56
Free metadata: 2621384
Origin size: 21474836480
Write density: 0
Creation time: Tue May 15 18:33:23 2007
  Snap            Creation time   Chunks Unshared   Shared
totals                                 0        0        0
Status: running
Configuration:
/var/lib/zumastor/volumes/testvol
|-- device/
|   |-- origin -> /dev/sysvg/test
|   `-- snapstore -> /dev/sysvg/test_snap
|-- master/
|   |-- next
|   |-- schedule/
|   |   |-- daily
|   |   `-- hourly
|   |-- snapshots/
|   `-- trigger|
`-- targets/

RUN STATUS:
/var/run/zumastor
|-- agents/
|   `-- testvol=
|-- cron/
|   `-- testvol
|-- mount/
|   `-- testvol/
|-- running
`-- servers/
    `-- testvol=
    
As hours roll by, snapshots should appear as well under master/snapshot.

df

/dev/mapper/testvol   20642428    131228  19462624   1% /var/run/zumastor/mount/testvol
    
The origin filesystem on /var/run/zumastor/mount/testvol is where all testing and fielsystem activity should take place.

To verify snapshots are working correctly:
date >> /var/run/zumastor/mount/testvol/testfile
sync

zumastor snapshot testvol hourly

date >> /var/run/zumastor/mount/testvol/testfile

zumastor snapshot testvol hourly

cat /var/run/zumastor/mount/testvol\(2\)/testfile
cat /var/run/zumastor/mount/testvol\(4\)/testfile
cat /var/run/zumastor/mount/testvol/testfile

Remote replication

Install both an origin and a slave node. These will be lab1 and lab2 below.

ssh authorization

On each machine, run ssh-keygen as root. Copy the .ssh/id_dsa.pub file from each account to .ssh/authorized_keys on the other. This authorizes root on each machine to connect without a password to the other. More restricted access may be used in actual deployment.

Define target on master

root@lab1:~# zumastor define target testvol lab2.example.com:11235 -p 30

This tells lab1 to replicate to port 11235 on lab2 every 30 seconds. If the period is omitted, replication will not occur.

zumastor status --usage

...
`-- targets/
    `-- lab2.example.com/
        |-- period
        |-- port
        `-- trigger|
...
    

Define volume and configure source on target

root@lab2:~# zumastor define volume testvol /dev/sysvg/test /dev/sysvg/test_snap --initialize

root@lab2:~# zumastor define source testvol lab1.example.com --period 600

zumastor status --usage

VOLUME testvol:
Data chunk size: 16384
Used data: 0
Free data: 0
Metadata chunk size: 16384
Used metadata: 56
Free metadata: 2621384
Origin size: 21474836480
Write density: 0
Creation time: Wed May 16 10:28:27 2007
  Snap            Creation time   Chunks Unshared   Shared
totals                                 0        0        0
Status: running
Configuration:
/var/lib/zumastor/volumes/testvol
|-- device/
|   |-- origin -> /dev/sysvg/test
|   `-- snapstore -> /dev/sysvg/test_snap
|-- source/
|   |-- hostname
|   `-- period
`-- targets/

RUN STATUS:
/var/run/zumastor
|-- agents/
|   `-- testvol=
|-- cron/
|-- mount/
|-- running
`-- servers/
    `-- testvol=
    

Start replication

root@lab2:~# zumastor start source testvol

Alternatively, you may kick off replication manually each time from the master.

root@lab1:~# zumastor replicate testvol lab2.example.com

Once initial replication is started, ddsnap processes should begin consuming CPU cycles and moving tens of megabits per second between the nodes. Initially, the entire origin device will need to be moved, so wait 15 minutes before looking on the target for snapshots. When replication is complete, df on the slave should show the same /var/run/zumstor/testvol volume locally mounted.

Verify replication

root@lab1:~# date >> /var/run/zumastor/mount/testvol/testfile

Wait 30-60 seconds for next replication cycle to complete.

root@lab2:~# cat /var/run/zumastor/mount/testvol/testfile

You may need to leave the current directory to see incoming changes. If the same file is there on the slave (lab2), replication is working.

Stopping zumastor

If a zumastor volume is exported via nfs, the kernel server must be stopped before zumastor is stopped in order to allow unmounting of the volume. Due to what appears to be a bug, nfs-common also needs to be stopped in some cases.