Heroku
A ruby app hosting company with a slick API and a fully managed stack. I love their web design too!

Why is this significant? Because it opens up the package system for commercial software distributors. If you couple this with the IPS repository functionality and require keys for the repo you have subscription updates, and license management covered!

Your software purchase and install path looks like:

Go to web site -> do they support opensolaris -> yes -> buy -> add repo -> pkg install software

Update your software from now until forever:

pkg install software or

pkg image-update

I really don’t like having to run custom installers from vendors that run scripts that do all kinds of stuff to the system and also install package sets. If the software packaging system is opened up to vendors we can manage our open source and commercial applications from one place.

I would also like to see open source projects build their own package repositories for all the major distributions. This would allow us to stay current with some software packages who’s development out paces the release cycles of the enterprise distros (RHEL, Ubuntu LTS).

Redhat Linux

GFS(2)
Redhat has done well with GFS(2) from a systems management perspective (I have read that it doesn’t scale as well as lustre). The cluster file system technology is pretty simple comparatively speaking even though there are several layers involved. You don’t need metadata controllers or independent systems acting as lock managers. You do need to either have fencing hardware, or a manual fencing process in place to account for failed nodes.

Adding storage involves using LVM to add the LUN to the volume, expand the volume, then using gfs2_grow to expand the filesystem.

At its simplest a configuration could look like the following:

Sun Solaris/OpenSolaris

SAM-QFS
Sun has had SAM-QFS for quite some time. It is more complex than GFS(2). To be fair it also has more features. It requires dedicated metadata controllers. This solution doesn’t interest me much due to the fact that I use Solaris/OpenSolaris for ZFS to simplify storage management.

Adding storage to the shared QFS file system appears to involve a similar process to GFS(2).

At its simplest a configuration could look like the following:

Lustre, BTRFS/CRFS, ZFS and the Future

Lustre
Lustre seems to be more complex than both GFS(2) and SAM-QFS. It also seems to be more scalable. It requires 2 types of controllers: MDS (metadata) and OSS (storage). So from a management perspective it requires quite a bit more overhead.

It looks like ZFS will be added as a backend storage format in the future versions 2.x+ this might simplify LUN management a little for the solution.

BTRFS/CRFS
BTRFS is a Linux ZFS workalike. It is still in development but the final version is shaping up to have ZFS feature parity (minus easy device/raid management, zvols, and other sweet things like zfs send recv). What interests me however is CRFS.

CRFS also in development, appears to be a global network filesystem that basically exports BTRFS file systems.

ZFS
What I would like to see is ZFS to become cluster aware. Being able to use OpenSolaris as a storage host and export either a zfs or a zpool to nodes, and perhaps run a lockmanager process similar to GFS(2) would be pretty slick.

For now with ZFS

Creating ZFS file systems on disk boxes connected to a SAN fabric really isn’t that difficult. It is however a bit time consuming, and you really should configure LUN masking for each LUN and node. Other wise you run the risk of accidentally corrupting data by using the same LUN at the same time on different nodes.

Exporting zvols with Comstar/iSCSI also doesn’t seem like the best idea, because again you should configure target portal groups and host access, and I don’t think that running ZFS on top of a ZFS based virtual block device (zvol) would be great for performance. Jeff Bonwick had stated something about improving the zvol performance (block pass through?) in his last ZFS talk. I am not sure wether it has been done yet or not. There also are no best practice notes about ZFS on iSCSI zvols.

I would think using UFS on a zvol would be better performance wise, but the whole purpose of the exercise for me is to provide snapshot functionality to the host consuming the filesystem, and filesystem delegation to the zones on the host.

I would really like to be able to use zvols as opposed to SAN fabric LUNs because then the zones could use snapshots, I could mirror the zvols from a Comstar storage host to a remote system for disaster recovery, and only have to deal with one zpool.

So how about it? Is anyone else using Comstar, ZFS, and OpenSolaris in the fashion I have described What is your strategy for consolidating storage management with OpenSolaris?

1
2
3
4
5
6
7
8
9
10
11
12

$view = new Zend_View();
$view->addScriptPath(BASE_PATH.'/tpl');
$view->somevar = $somevar;
$html = $view->render('someTemplate.phtml');
 
$mail = new Zend_Mail();
$mail->setBodyHtml($html);
$mail->setFrom($config->site->serviceAddress, $config->site->appName);
$mail->addTo($email);
 
$mail->setSubject($config->site->appName.' - Message Subject');
$mail->send();

Zend_View handles the output buffering and rendering of the template. I have never really had a problem with it.

In Grails this is supposed to work:

1
2
3
4
5
6

sendMail {
	to user.email
        from 'email@emails.com'
        subject 'Subject'
        body (view:'/mail/myTemplate', model[user:user])
}

sendMail is a mail service plugin that accepts a closure as an argument, it also has a shortcut to the gsp template engine to render the template. Seems more simplistic, however there is a lot of code under the hood driving it. It seems that the output buffering within the Grails framework is very tied to the servlet response.

Anyway within a Grails Service (haven’t tried it within a controller yet) it munges the output buffer. Probably due to the output buffer juggling that is going on within various layers of the response and template rendering wrappers. The result is a failed remoting call.

The fix (figured this out after profiling and digging around the web) is to wrap sendMail in a thread:

1
2
3
4
5
6
7
8
9

def t = new Thread() {
	sendMail {
		to user.email
                from 'email@emails.com'
                subject 'Subject'
                body (view:'/mail/myTemplate', model[user:user])
	}
}
t.start()

This leads me to believe that somewhere within GrailsPageResponseWrapper the original output buffer is being overwritten or lost which is killing the response writer. Spawning a new thread for the operation preserves the original output buffer.

I am surprised that no one else is using this functionality outside of their controllers, and only 2 people have encountered the issue. I don’t know enough about the grails stack to debug it fully myself yet.

Here is a link to the bug report:
http://jira.codehaus.org/browse/GRAILSPLUGINS-1548

Please fix?

Things that still bother me vs Ubuntu LTS:

• Comparatively small distro package set.
• File system layout in respect to etc and var: stuff is all over the place. Seriously each configurable software package having its own directory in etc and var sub directories (log etc) is nice. It is uniform and easy to guess where things might be.
• $300/yr

In an interview I was posed a problem: given 2 arrays a and b write a program that will print the elements of a that are not in b.

The easiest (incorrect solution) in PHP:

1
2
3
4
5
6
7
8
9
10
11
12
13

#!/usr/bin/php
<?
$a = array(1, 2, 3, 4, 9, 20, 30, 82, 57);
$b = array(1, 2, 3, 82, 57, 100, 32);
 
$c = array();
foreach($a as $aval) {
	if(!in_array($aval, $b)) $c[] = $aval;
}
 
print_r($c);
 
?>

Now as I had stated in the interview this is NOT the best way to accomplish this. Of course that would all depend on how in_array is implemented in PHP. If in_array is simply a short cut to a true false foreach loop then we have a problem. If n and m represent the number of elements of a and b respectively iterations = n * m. Or O(mn). Not efficient. Since this is PHP I am guessing this is the case (a simple sequential search).

So what is a better solution (best?) to allow for better performance as n and m increase? This is the best I could come up with after a quick look at a couple of algorithms.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

#!/usr/bin/php
<?
$a = array(3, 1, 2, 4, 9, 20, 30, 82, 57);
$b = array(1, 2, 3, 82, 57, 100, 32);
 
//binary search function
function binSearch($ar, $n, $value) {
	$l = 0;
	$r = $n;
	while($r >= $l) {
		$mid = ($l + $r)/2;
		if($ar[$mid] == $value) {
			return true;
		} else if ($ar[$mid]< $value) {
			$l = $mid + 1;
		} else {
			$r = $mid - 1;
		}
	}
 
	return false;
}
 
//sort the first array used in the bin search
sort($b);
$n = count($b);
$m = count($a);
$c = array();
for($i=0;$i<$m;$i++) {
	if(!binSearch($b, $n, $a[$i])) {
		$c[] = $a[$i];
	} 
}
 
print_r($c);
 
?>

Removing the negation operator prior to the function call will change the program to array intersection vs array diff. The math for this binary search solution is O(m log2 n).

So consider the following:
m = 8
n = 8

Solution 1:
(mn) = 64

Solution 2:
16*log2(8) = 48

Basically due to the first array being sorted and chunking we are able to do fewer comparisons instead of a comparison for each element. As the size of the chunk decreases the worst case is that the value doesn’t exist.

It also occurs to me that you will want to count each array and sort the shortest and use that as the first argument and the longer array[val] as the second to further reduce the number of iterations. Of course the order of the arguments matters for this problem (in a not in b). I suppose you could flip flop logic/operator in/on the binary search call/function though to account for that as well.

Anyway this is basic CS, everyone probably knows this but I couldn’t come up with it off the top of my head, and the problem was driving me nuts cycling in the back of my mind so I had to write it out.

UPDATE
I still kept thinking about this so here is another based on array merge and comparing next array elements. Since this is a diff and not an intersection we need to merge the first array twice. Any value that shows up twice in the merged array is in a and not in b.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

#!/usr/bin/php
<?
$a = array(3, 1, 2, 4, 9, 20, 30, 82, 57);
$b = array(1, 2, 3, 82, 57, 100, 32);
 
$c = array_merge($a, $b);
$c = array_merge($a, $c);
$d = array();
sort($c);
 
 
$i = 0;
 
while($i <= count($c)) {
 
	if($c[$i+1] == $c[$i] && $c[$i+2] == $c[$i]) { 
		$i+=3;
	} else if ($c[$i+1] == $c[$i]) {
		$d[] = $c[$i];
		$i+=2;
	} else {
		$i++;
	}
}
 
print_r($d);
?>

The math ends up being something like ~O(m + m + n).