ZFS vs ILNP and ‘good enough’ availability

December 5th, 2011 No comments

I have been busy setting up ZFS. I am late to the party, as ever. This has been painful because my time has been at a real premium these days.

Somewhat ironically, the great advance offered by ZFS, in it sphere of storage and filing, is analogous to the great advance offered by ILNP, in its sphere of network naming and location. Mathematicians have a great word for this; isomorphism.

What I can say is this: a few of the third party packages available for FreeBSD make it easier to manage ZFS out-of-the-box. However, even with this kind of help, some self assembly may still be required for systems administrators to make it useful at their own sites.

I had hoped that tools would have matured enough for someone, without being overly familiar with ZFS, would be able to pick it up and start using it, and its new features, without fuss. I know some open source folk wax lyrical about forcing people to learn the tools, by not providing GUIs or other required integration for daily use; or indeed frowning upon people who use them.

This is a bit like forcing your child to drink cod liver oil. We know that it is ultimately good for us, but there are easier and more appropriate ways of getting the same results.  I consider this proselytizing. Your mileage may vary.

What the tools in ports don’t do, is offer out-of-the-box configuration which does your thinking for you. However, I believe that what we need here is quite common. I was surprised that this was not something which I could just order like fried chicken in Hackney on a Friday night.

We have two servers: one master, one warm backup.  The master is where all the action happens. If it fails, the warm backup will be switched in to replace it, albeit with some manual fail-over, which we can live with. We aren’t aiming for ‘High Availability’, just good enough availability.

To start, I configured and installed the zfs-periodic port on both master and backup, and fired up zxfer to incrementally pull snapshots from the master.

Bzzzt – wrong answer. Over the weekend, after a barrage of email from cron, I came to a stunning epiphany: the name space for ZFS snapshots, must be common to all nodes involved, for incremental replication to work amongst them.

After disabling zfs-periodic on the backup server, I was able to use zxfer to pull the hourly snapshot, without it being clobbered by zfs-periodic running on the backup itself.

All zfs-periodic does is impose a naming scheme on this; snapshots are taken locally, and pruned according to simple rules. It has no notion of network backups. zxfer is a wrapper for ‘zfs send’ and rsync which automates certain repetitive command stanzas, but does so in a way which makes it very useful to have around.

The name space used by ZFS itself is flat. The way to deal with the split is probably to add a property to any snapshots copied by zxfer, marking them as ‘common’.

I haven’t implemented the necessary changes yet, as I have a zillion other things to do, but at least I am now more familiar with how ZFS snapshots and replication interact.

Again, this should all be taken with a pinch of salt. We know that ZFS is stable, because many people over the world have taken the time, and effort, to become familiar with it, and deploy it in their work. Perhaps the hard work involved is part of that, as it. However, we could perhaps all do better in promoting it.

A change in thinking is certainly required to get the best from it. To be certain, using ZFS in the infrastructure needs changes in practice also, and I know I am going to have fun explaining it to people who aren’t familiar with it.

I am happy to trade-off the blind flexibility of rsync, against getting the same job done at least forty times quicker, with none of the race conditions.

It is perhaps a case where some of us didn’t get the memo…

Categories: Uncategorized Tags:

Configuring Mac OS X to Install Perl Modules

November 29th, 2011 No comments

Here is a step-by-step guide to installing Perl modules using the version of the interpreter which ships with MacOS X. Why? In the copious free time which I don’t have, I will try to find time to informally evaluate the Convergence peer-to-peer Certificate Authority replacement. Some interesting discussion of Convergence can be found here.

Categories: Uncategorized Tags:

Network Bibliography

November 29th, 2011 No comments

The most useful link I’ve found all day today, is to Henning Schulzrinne’s Network Bibliography. Whilst no longer updated, it is a vast repository of BibTeX format citation entries for papers, drafts, and standards documents, commonly used by researchers in telecommunications as normative references.

Tags: ,

Categories: Uncategorized Tags:

Schneier on Android Malware

November 29th, 2011 No comments

I do have an Android mobile phone. I am pretty careful about what I run on it. The trust relationships involved in "cloud" are something of which I am quite wary. So it comes as no surprise that Bruce Schneier has started making noise about it.

On the surface, the notion of remote credential storage reminds me of a feature called Factotum in the research operating system, Plan 9 from Bell Labs.

It seems the default is to trust your vendor absolutely, out of the box. Can anyone see inherent problems with this? This article from UT Austin has some straightforward, common sense suggestions about how to lock down your Android device.

Tags: , ,

Categories: Uncategorized Tags:

Software contracts “better than licenses”

November 28th, 2011 No comments

Food for thought: Problems in Open Source Licensing by Jeremy Malcolm, of iLaw in Perth, Western Australia.

Categories: Uncategorized Tags:

How to disable IPv6 in OS X Lion

November 28th, 2011 No comments

It seems Apple messed up the Networking preference pane again. There are legitimate reasons why you’d want to disable IPv6 completely. This thread documents how to turn IPv6 off; this involves using the Terminal with a text editor.

Categories: Uncategorized Tags:

My first LaTeX paper

November 28th, 2011 No comments

One of the first LaTeX papers I wrote was an overview of the FreeBSD boot process for the Xen folk, who were at that time bootstrapping the project. Unfortunately I’ve lost the original .tex source, and I’m kicking myself.

Categories: Uncategorized Tags:

Guts of FreeBSD in Fusion

November 25th, 2011 No comments

This represents an ongoing dump/structuring of rough notes over the past few weeks, from Stickies on my MacOS X desktop, into something more palatable.

Some of this will be merged with the earlier post on FreeBSD guests below.

  • The low fat window manager is FluxBox.
  • open-vm-tools is more immediately useful than VMware Tools for FreeBSD, because it actually supports Shared Folders.
  • The patch to fix detection of VMware’s mouse by hald, and thus load the correct driver in Xorg for mouse desktop integration with the host, can be found here. It has not been integrated yet, as the bug report was prematurely closed.
  • More as it happens.

Believe it or not, I’ve never used WordPress in my life.

Categories: Uncategorized Tags:

A 6LOWPAN capable wrist watch?

November 25th, 2011 No comments

The TI EZ430 Chronos is a wrist watch which may be capable of running 6LOWPAN.

More to follow…

Categories: Uncategorized Tags:

Homebrew branch on Github

November 25th, 2011 No comments

I have forked the Homebrew package manager tree on Github. There isn’t much in there right now, apart from one little tool I find very useful when looking at normative references: rfc.

Categories: Uncategorized Tags: