Opposable Thumbs

Here's a little tidbit that took me somewhat by surprise. I was poking around on my laptop trying to debug an issue with a Ruby library I'm trying to use. I ended up trying to find the man pages for the TCP socket API. Not the Ruby interface, but the native stack. The thing is, the Ruby TCPSocket class just wraps around the native implementation. But when I tried to find the man pages, they weren't there. I thought I had all the basic development packages installed, but apparently, there is a separate package for the development man pages. It turns out you need to do an 'apt-get install manpages-dev' to get them.

So, in the effort to get myself setup to work on my desktop machine, I've been thinking about a different approach to my workspace configuration. I'm not talking about the physical workspace, though I've had many thoughts about that, too. This is about my virtual workspace. Really, truly virtual.

Some people know that I've been a long-time fan of VMware. I think I first started using around 1999 or 2000, when it was still a very early and somewhat rough bit of software. But it was already quite exciting to me then and quite useful.

I'm also a hardcore Linux user -- it's my preferred desktop. Though I have, at time, given in and used various Windows flavors when employers have required, it's always been with extreme reluctance. And, of course, I always end up tricking it out with Cygwin and a million little utilities to make it as unix-like as possible.

Anyway, given my fondness for Linux and VMware, I often think about ways to use virtual Linux machines to improve my setup. I've already been using a development system contained in a virtual machine for quite a while -- at least, for the day job development needs. I run Jboss and Eclipse directly from the development VM, running the Eclipse UI over an SSH tunnel, which automatically redirects the X-Windows connection to my local system.

But it finally occurred to me that I could take this a couple steps further. In particular, I realized that I could log directly into a desktop running in a VM via XDMCP. So, while I was backing up the latest stuff from my soon to be missing laptop, I began setting up a new Linux VM on my desktop with the latest release candidate of Ubuntu.

I got everything installed and updated and finally today had time to log out of my current session on the desktop machine and into the VM. It turns out that it works great! Everything runs over a local, virtual network, so it's almost as fast as if it were all running completely locally (really, it is completely local -- it just has a little more indirection in the mix).

The great thing about this setup is that I can configure a cron job to put the VM to sleep using VMware's scripting tools, creating a quick backup of the system to my NAS, and then wake it back up without even noticing it. Of course, I'll set it up to run at some obscene hour of the night when I'm not likely to be using it. I can always add a few checks to make sure I'm not currently logged in.

Once I get a new laptop, I'll just transfer this whole rig over to the new system. I'm not quite sure how I'll handle things like putting the whole laptop to sleep, but I'm sure a little trickery (read: lots of Bash scripting) can handle that. I'll report back here when the experiment is complete.

So, the other night at DorkbotPDX, there was a discussion about techniques for making images searchable. The idea was to come up with something beyond what Google Images gives you and have something more in depth -- among other things, the idea of categorizing things like Pandora does, using collective organization based on a large body of contributors, was discussed. There were a lot of interesting ideas thrown around, but most of them seemed to rely on this approach of using large social structures to generate the relevant metadata.

In a bit of the sort of serendipity that the internet seems to bring with near frightening regularity, the next day I read this post on O'Reilly Radar about the new O'Reilly book Programming Collective Intelligence. This book appears to discuss exactly the kind of development tasks that are necessary to make possible the systems we were discussing.

Intrigued by this, I downloaded the sample chapter and started reading. It's a very engaging book and I'm looking forward to getting my own copy. However, the example code is all in Python. I'm not a Python programmer, though I do have a good bit of familiarity with the language and, in the past, have written a handful of utilities using it. But even still, I don't think this would be an issue: I'm well familiar and to a degree quite versed in a handful of languages that are very similar to Python. Which makes it easy enough to understand what the examples are trying to do.

This brought up a stream of thoughts, though, that have been rattling around in my head for some time now. The fact is that when it comes to languages like Python, Ruby, Java, C, etc. there is enough similarity that I can make my way through a given example (for those with which I am less familiar) without any great difficulty. The problem is that outside of these common paradigms (for lack of better descriptors, imperative and object-oriented) of programming languages, I am not so comfortable.

Unfortunately, there is a growing undercurrent of languages that work outside of that world. Haskell Erlang, OCaml, Scala, Prolog... the list goes on. All very interesting and worth understanding, but I feel largely out of my depth when looking at examples in these environments. It's just not right.

For a long time it's been clear to me that in order to continue to push my abilities as a developer and technologist in general, I need to continue to expand the scope of my understanding and knowledge. Functional, logical and other language styles are clearly a necessary addition to my repertoire.

So, it occured to me that after soliciting everyone in DorkbotPDX to post the announcement for this to their blogs, I failed to do so myself. Bad self! So, here it is, the official announcement for the first ever DorkbotPDX event:

Come join DorkbotPDX, people doing strange things with electricity, for our inaugural event at Vendetta on June 24th at 5pm. If you're a hacker, painter, engineer or sculpture, musician or maker you'll fit right in. We bring together the tech and art worlds and enjoy it all over a pint of beer. We'll have presentations and performances by these fine folks:

Jason Plumb is a software engineer by day...hardware hacker, reverse engineer, and experimental sound geek by night. He will provide an overview of the Essential Reality P5 glove controller and explain how it can be used with free and open-source software to create and manipulate sound.

Jesse Fox studied music composition and physics at Bates College before getting a Master's Degree from the Center for Computer Research in Music and Acoustics at Stanford University. He will discuss his involvement with the League of Electronic Musical Urban Robots (LEMUR) and describe the detailed technical recreation of George Antheil's "Ballet Mecanique", which includes xylophones, bass drums, tam-tam, pianists, electric bells, a siren, airplane propellers, and a volley of player pianos.

Donald Delmar Davis, principal research anarchist at D3 Laboratories, will overview the deconstruction of Arduino and Wiring platforms to create artistic robot platforms with AVR microcontrollers. "AI Begins With Self Destruction"

paint & copter create multi-media experiences of regurgitated and improvised media. By synthesizing live and pre-manipulated video feeds, field recordings and live instrumentation, Paint and Copter filter cultural noise and reprocess it into a new, mesmerizing thread.

We will also have a brief open-mic of sorts referred to as Open Dork. This is a show and tell where you can have the mic for a few minutes to discuss your latest project, vent about frustrations trying to get your art grant or tell us about the intricacies of the color blue. It's your time to tell us what you think we need to hear.

Festivities will begin at 5pm and you can expect them to last until they kick us out. Please bring yourself, your friends and any thing you'd like to share.

http://dorkbotpdx.org/wiki/dorkbotpdx_0x00

A while back I agreed to do tech reviews for two Ruby books from O’Reilly. The first of these, Ajax on Rails by Scott Raymond, was just published a few weeks ago. It looks to be quite an excellent piece of work (of course, I know the tech reviewing was top notch ;~). I just received word that the second, Rails Cookbook by Rob Orsini, has been published, too.

Both look to be worth picking up. I’ll be posting more about both soon, I hope.

I've been working on setting up some servers for a project with which I'm involved (in all my infinite spare time). These servers are going to have a variety of web applications running (some of which will be third-party), but we (that is, me and the other guy ;~) would like to keep a somewhat consistent look & feel across the board.

Of course, the obvious solution is to just edit the templates provided with each application (assuming the have templates at all). As the number of applications grows, the enjoyment level of doing this decreases rapidly.

Another solution that was thrown around was to make sure all the applications output well-formed XHTML and use XSLT to churn the output into something we like. However, this involves a number of assumptions about the quality of the markup being generated by each application (not something we necessarily have control over) and requires installing a front end filtering system to process all the content before it's sent out over the interweb.

One approach which I've looked at in the past is to use something like SiteMesh which also does filtering, but is a bit more direct and doesn't require well-formed XHTML to do it's dirty work. On the other hand, it's a java-based system, and I'm hoping to avoid having to run a java application server for the foreseeable future (I know it's coming at some point, actually, but I'm going to limit that to just the servers that require it, if possible).

While chatting about this problem on the PDX.rb IRC channel a few weeks back, rcoder pointed me to this wonderful gem from why. I tried it out later that day and never could get it to work.

But tonight, I started fresh with a nice clean build of Apache and why's modified mod_ruby and I'm happy to report that I got his example working perfectly.

So now, the next step is to lay out a design for a SiteMesh like system, but using some nice Ruby idioms and conventions and see where I can get with this. If it gets to a point where it's halfway usable, I might just have something that I can put out for all to see. Of course, it will be a little weird releasing some code that depends on a hacked version of a little used (at least so far as I can tell) module for Apache, but that's the way things go sometimes.

Late last night while trying to get rails running on a server I was setting up, I ran into an odd error. Sadly, I failed to save the error message itself and my browser history has been woefully unhelpful in finding the google searches I did to track down the issue. However, I can recreate the sequence of steps I took to fix the problem.

First, though, let me give you the scenario. This box is running Ubuntu Dapper (6.06) but the scenario could apply to just about any Debian-based distro. I had been planning on running rails on top of the ruby .deb packages since I'll be eventually maintaining a potentially large number of boxes and installing from source is not so fun across a bunch of systems.

I did stumble on this patch for the issue, but it wasn't immediately obvious how straightforward it would be to patch the source and repackage it. I'm here to tell you, it turned out to be really damn easy. For more background info on the approach I took, you might want to read this article, but I'll outline the steps here for you if you don't want to skip over there first.

The first thing you will want to do is make sure you have the prerequisites for building packages. You can load them like so:

sudo apt-get install devscripts build-essential fakeroot

Next, you'll need to get the source for the package you want to patch:

sudo apt-get source ruby1.8

You will probably want to do this in a directory of it's own, since it can dump a number of files in your current directory depending upon the package you're updating.

After you've grabbed the source, you'll need to get any dependencies for this package:

sudo apt-get build-deps ruby1.8

At this point, you should have a directory called ruby1.8-1.8.4 in your current directory. Change into this directory.

You should see a directory under there called debian, in which will be another directory called patches. This is where we will add our patch file.

You'll notice that each patch file has a numeric prefix. You'll want to add a higher numbered patch than any of the existing files so that your patch gets applied last. To do so, I created a file called 904_extend32_fix.patch. The contents of this file were (based on the contents in the email linked above):

Index: pack.c
===================================================================
RCS file: /src/ruby/pack.c,v
retrieving revision 1.62.2.12
diff -u -w -b -p -r1.62.2.12 pack.c
--- ruby-1.8.4/pack.c   13 Oct 2005 14:30:49 -0000 1.62.2.12
+++ ruby-1.8.4/pack.c   16 Feb 2006 06:01:07 -0000
@@ -347,11 +347,11 @@ num2i32(x)
     return 0;                  /* not reached */
 }

-#if SIZEOF_LONG == SIZE32 || SIZEOF_INT == SIZE32
+#if SIZEOF_LONG == SIZE32
 # define EXTEND32(x) 
 #else
 /* invariant in modulo 1<<31 */
-# define EXTEND32(x) do {if (!natint) {(x) = (I32)(((1<<31)-1-(x))^~(~0<<31));}} while(0)
+# define EXTEND32(x) do { if (!natint) {(x) = (((1L<<31)-1-(x))^~(~0L<<31));}} while(0)
 #endif
 #if SIZEOF_SHORT == SIZE16
 # define EXTEND16(x)

At this point, you're just about done, all you need to do is run debuild -us -uc from that ruby1.8-1.8.4 directory. This will create a bunch of .deb files in the parent of this directory. At that stage, you're ready to install.

There are certainly some improvements to be made to this process. For one, I have yet to attempt to lock or hold the package, which means that next time there is an updated version of the package available, it will get updated without my patch. However, I'm not entirely sure whether that's going to be a problem. I'll do more research into it, but for now my problem is solved.

Another change to the approach that would improved things would be to edit the changelog that lives in that debian directory. Adding a new entry to the change log is both a good practice and tells the package build tools to update the version number of the package. However, that file is rather strictly formatted and there are special tools for editing it. I haven't investigate them, yet, so I won't discuss them any further here.

So, just a quick update to let everyone know why I've disappeared and have been unable to find even a few spare seconds to update the blog. I've been just a wee bit busy.

Last month I attended OSCON and helped put on FOSCON II. Both of which were awesome and exhausting. I met a lot of great people and learned about some interesting things. Hopefully I'll find time to dig in a bit deeper on some of it.

Since then I've been working on two new projects. The first has been tech reviewing a new book for O'Reilly that's coming out later this year. More on that later.

But the most important activity in my mind has been putting together dorkbotpdx, the Portland chapter of the truely amazing dorkbot group. This one is certainly going to lead to more blog fodder, but that, too, will have to wait.

A few days ago, Chad Fowler 's new book Rails Recipes was released in beta by the Pragmatic Programmers. As soon as I saw this, I printed out and read the excerpts they were making available. As I had hoped, they were great. Aside from a few typos (to be expected for a beta), the quality was excellent and I've already learned a few tricks. It was fun setting up mini-rails projects and quickly running through each recipe. I find examples are often the best way to learn things and this book is exactly the ticket.

Last night I went ahead and bought the beta PDF. I'm looking forward to reading the rest of it and watching it evolve through the beta process.

Sorry for those of you coming here expecting non-technical posts, but I did start this thing for technical purposes, after all, and I figured it was about time to return to form.

Tonight I was working on a little side project I'm doing. I needed to pull a bunch of legacy data into a new table for event data. To make this work, I created a set of models that referenced the old tables, using the syntax ActiveRecord provides to override it's default settings.

After figuring out how to make it work (I had to play with the database connection settings a bit), I decided to take a break and catch up on some blogs. Low and behold, I find that Dave Thomas went and described almost the exact same situation on his blog! I mean, even down to the "Legacy" prefix on his table names. And the reassuring bit is that he arrived at the same solution I did. Gotta love it.

Tim O'Reilly proposes the term "make tank" for the companies like Applied Minds that:

figure out how to design and build hardware that solves interesting new problems.

Sounds interesting. Now, how to get a job working for them... ;~)

This is just a collection of random thoughts that were floating through my head last night before I fell asleep. And, yes, I do think about these sorts of things often. That's why I'm a geek... It seems to me that searching for information on the web really makes more sense when done in context. That is, Google and the like are great for finding certain kinds of information when you're interests are very general, but they can often be a real pain when searching for something very specific. That is, unless you know some relevant details about the item you're searching for. But that's not often the way it works when you're trying to track down a long lost resource. For instance, you might remember that you saw this web page you're looking for sometime last month. But Google doesn't know anything about that. Now, bookmarks and browser history provide this kind of information. That is, they can provide it. But there's currently no way to bridge this gap. Not to mention that services like del.icio.us and Furl add another layer on there. Besides that fact that they store the information about your bookmarks remotely (making things in theory much easier to interface with a service like a search engine), they also add on features like tagging. Which, obviously can be exploited.

All the recent frenzy over AJAX has gotten me looking at a lot of really great javascript. Anytime I see good code, I get inspired to learn something from it and push my abilities a bit. So I've been playing around with something that has long been an issue for me. You see, in a lot of my applications I end up dynamically generating at least certain parts of the page via javascript. I find it's often easier to return data from the server in a javascript datastructure and then rely on the javascript to build up whatever form elements I need so that the user can modify it. But one thing that's been hard to deal with is how to get this data from the form back into the original datastructure. Sure, it's possible to manually add onclick/onchange/etc. handlers by concatenating a lot of strings, but it's just ugly and not very reusable. For instance, in a recent application, I found myself doing this:

td.innerHTML = 
  '';

Yuck! But reading through all this AJAX code and the related libraries that it's brought to attention has led me to investigate javascript support for closures. Closures, for those of you who don't already know are a way to preserve a state across function calls. This is, they let you carry around a set of variables that seemly would have gone out of scope by the time your function is called. This is a very powerful idea (read more here).

Rails Day 2005 - June 4th - seems to me this could be the right opportunity to get at least the basic functionality of Esperanticon off the ground. It would be at least some free publicity. On the other hand, I don't know how we'd get one piece done since it won't be able to use Rails. I need to talk to Justin about this. Sorry for the vagueness of this post, but I'm not sure we're ready to unleash this idea on the world yet. It might be too big for some of you to handle!

All this new hype about AJAX has helped to bring out in the open a lot of interesting developments in the JavaScript world. http://www.clarkware.com/cgi/blosxom/2005/03/18#RLT1 http://www.edwardh.com/jsunit/ http://dev.rubyonrails.com/file/trunk/actionpack/lib/action_view/helpers/javascripts/prototype.js http://www.dojotoolkit.org/