Septimius Network The jurnal of a system administrator

Howto – Mdadm cheat cheat

mdadm is a program that can be used to create, manage, and monitor MD devices.

The name is derived from the md (multiple device) device nodes it administers or manages, and it replaced a previous utility mdctl. The original name was "Mirror Disk", but was changed as the functionality increased.

As such it provides a similar set of functionality to the raidtools packages. The key differences between mdadm and raidtools are:

1. mdadm is a single program and not a collection of programs.
2. mdadm can perform (almost) all of its functions without having a configuration file and does not use one by default. Also mdadm helps with management of the configuration file.
3. mdadm can provide information about your arrays (through Query, Detail, and Examine) that raidtools cannot. mdadm does not use /etc/raidtab, the raidtools configuration file, at all. It has a different configuration file with a different format and an different purpose.

RAID devices are virtual devices created from two or more real block devices. This allows multiple devices (typically disk drives or partitions there-of) to be combined into a single device to hold (for example) a single filesystem. Some RAID levels include redundancy and so can survive some degree of device failure.

Linux Software RAID devices are implemented through the md (Multiple Devices) device driver.

Currently, Linux supports LINEAR md devices, RAID0 (striping), RAID1 (mirroring), RAID4, RAID5, RAID6, and MULTIPATH.

MULTIPATH is not a Software RAID mechanism, but does involve multiple devices. For MULTIPATH each device is a path to one common physical storage device.

1. Howto mount existing raid

An raid configuration is usually saved into mdadm.conf file (/etc/mdadm.conf on redhat or /etc/mdadm/mdadm.conf on debian).

In order to scan all the raid devices and generate a new configuration file we can use something like this

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

 2. Howto create a new raid array

In this example we created a mirror array between /dev/sda1 and /dev/sdb1

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sd[ab]1

or

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1

You can also create an degraded array with a 'missing'   drive that can be added latter

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 missing

and you can add the missing drive with (--manage can be ommitted)

mdadm --manage /dev/md0 --add /dev/sdb1

Note that ideally the drives should:

- not contain any raid information in the superblock (like if they were used in another array), you can clean the old data using :

mdadm --zero-superblock /dev/sdb

- be identical (in size),  but if you have drives with different sizes or you want to grow your array size by adding bigger drives you still can create an array by using the --force option.

- be identical in the way they are partitioned, you can "dump" the partion from one drive (sda) to a new drive (sdb) with the following command.

sfdisk -d /dev/sda | sfdisk /dev/sdb

in this case I like to also use partprobe to inform the OS of partition table change

3. Howto remove a drive from an array

In order to remove a drive from the array, the drive needs to me marked as "failed".

mdadm --fail /dev/md0 /dev/sda1

then we can remove it

mdadm --remove /dev/md0 /dev/sda1

We can do this also in a single step:

mdadm /dev/md0 --fail /dev/sda1 --remove /dev/sda1

4. Howto stop/start and array

An array can be stopped only if it is not mounted.

To stop an array :

mdadm --stop /dev/md0

To start an array (even if fewer drives were given than are needed for a full array):

mdadm --run /dev/md0

5. To remove (delete) an array

mdadm --remove /dev/md0

 6. To see to status of an array durring build/rebuild

The array information can be displayed by :

cat /proc/mdstat

or to have a constant update every 10 seconds

watch -n 10 'cat /proc/mdstat'

for more information about a particular md device you can see all the information in detail:

mdadm --detail /dev/md0

this may be usefull to see details about drives in degraded arrays.

 7. To limit/increase the resync speed

- setup a minimum speed (10000k)

echo 10000 > /proc/sys/dev/raid/speed_limit_min

-to limit the maximum speed to 15000k (usually used to reduce the system stress during rebuilds)

echo 15000 > /proc/sys/dev/raid/speed_limit_max

the can be setup using sysctl also

sysctl -w dev.raid.speed_limit_max=15000

to have a permanent limit you can edit /etc/sysctl.conf and add "dev.raid.speed_limit_min=10000" or "dev.raid.speed_limit_max=15000", folowed by a

sysctl -a -f /etc/sysctl.conf