Storage Management :: Global File Systems and ZFS
One enterprise architecture feature that every admin dreams of is simple storage management. Having one storage network and a global filesystem that all nodes can access, and being able to easily have space “appear” on the file system by adding LUNs to the global file system is a huge time saver.
Redhat Linux
GFS(2)
Redhat has done well with GFS(2) from a systems management perspective (I have read that it doesn’t scale as well as lustre). The cluster file system technology is pretty simple comparatively speaking even though there are several layers involved. You don’t need metadata controllers or independent systems acting as lock managers. You do need to either have fencing hardware, or a manual fencing process in place to account for failed nodes.
Adding storage involves using LVM to add the LUN to the volume, expand the volume, then using gfs2_grow to expand the filesystem.
At its simplest a configuration could look like the following:
Sun Solaris/OpenSolaris
SAM-QFS
Sun has had SAM-QFS for quite some time. It is more complex than GFS(2). To be fair it also has more features. It requires dedicated metadata controllers. This solution doesn’t interest me much due to the fact that I use Solaris/OpenSolaris for ZFS to simplify storage management.
Adding storage to the shared QFS file system appears to involve a similar process to GFS(2).
At its simplest a configuration could look like the following:
Lustre, BTRFS/CRFS, ZFS and the Future
Lustre
Lustre seems to be more complex than both GFS(2) and SAM-QFS. It also seems to be more scalable. It requires 2 types of controllers: MDS (metadata) and OSS (storage). So from a management perspective it requires quite a bit more overhead.
It looks like ZFS will be added as a backend storage format in the future versions 2.x+ this might simplify LUN management a little for the solution.
BTRFS/CRFS
BTRFS is a Linux ZFS workalike. It is still in development but the final version is shaping up to have ZFS feature parity (minus easy device/raid management, zvols, and other sweet things like zfs send recv). What interests me however is CRFS.
CRFS also in development, appears to be a global network filesystem that basically exports BTRFS file systems.
ZFS
What I would like to see is ZFS to become cluster aware. Being able to use OpenSolaris as a storage host and export either a zfs or a zpool to nodes, and perhaps run a lockmanager process similar to GFS(2) would be pretty slick.
For now with ZFS
Creating ZFS file systems on disk boxes connected to a SAN fabric really isn’t that difficult. It is however a bit time consuming, and you really should configure LUN masking for each LUN and node. Other wise you run the risk of accidentally corrupting data by using the same LUN at the same time on different nodes.
Exporting zvols with Comstar/iSCSI also doesn’t seem like the best idea, because again you should configure target portal groups and host access, and I don’t think that running ZFS on top of a ZFS based virtual block device (zvol) would be great for performance. Jeff Bonwick had stated something about improving the zvol performance (block pass through?) in his last ZFS talk. I am not sure wether it has been done yet or not. There also are no best practice notes about ZFS on iSCSI zvols.
I would think using UFS on a zvol would be better performance wise, but the whole purpose of the exercise for me is to provide snapshot functionality to the host consuming the filesystem, and filesystem delegation to the zones on the host.
I would really like to be able to use zvols as opposed to SAN fabric LUNs because then the zones could use snapshots, I could mirror the zvols from a Comstar storage host to a remote system for disaster recovery, and only have to deal with one zpool.
So how about it? Is anyone else using Comstar, ZFS, and OpenSolaris in the fashion I have described What is your strategy for consolidating storage management with OpenSolaris?
