Tag: sysadmin

Loving Git

I suppose this must be the zillionth post about how Git is so cool, but I’ll tell you what I love about it:

  1. Ultra fast commits. Coming from subversion, typing commit and getting a prompt back instantly is so surprising that you double check the first few times.
  2. Branching! God, you don’t realize how much you miss it until you have fast branching and merging. Without the pain of creating cps, just a simple git checkout -b branch_name and git merge branch_name and everything just works.
  3. The above was the reason I tried git in the first place. mobME’s an SVN shop and we do the usual trunk, tags, branches dance. When the trunk shapes up to be stable though I can’t seem to do anything on it. I can fork off a new branch in SVN but that’s too painful to even think about. What do I do now? git clone it, create a branch, and do regular git svn rebases and dcommits.
  4. Oh did I mention bidirectional SVN support? Without which I wouldn’t/couldn’t have switched no matter how much I wanted to try this cool new thing.

But it’s great and it really changes the way you think about VCS. Notice something? I didn’t even talk about distributed source control, and that’s coz even when I use git like I use SVN – committing to a central repo at the end and pulling changes from it, it’s brilliant. I’d definitely want to explore cool stuff like github soon for personal use (some gracious soul gave me an invite some time back).

dtach, a brilliant daemonizer

Most people use GNU screen (or equivalent) to daemonize processes. There is a better alternative called dtach which should become the definitive way to run a process in the background as a daemon.

Installing dtach is easy. If you’re on gentoo, it is a simple:


$ emerge dtach

dtach --help provides the following output:


Usage: dtach -a <socket> <options>
       dtach -A <socket> <options> <command...>
       dtach -c <socket> <options> <command...>
       dtach -n <socket> <options> <command...>
Modes:
  -a		Attach to the specified socket.
  -A		Attach to the specified socket, or create it if it
		  does not exist, running the specified command.
  -c		Create a new socket and run the specified command.
  -n		Create a new socket and run the specified command detached.

This basically means that you start dtach like so:


$ dtach -c ~/rtorrent/.socket/dtach rtorrent

The command-shortcut Ctrl+\ detaches the running command and runs it in the background. To re-attach, just enter:


$ dtach -a ~/rtorrent/.socket/dtach

dtach hasn’t got much more functionality, but that’s what makes it superb. Tiny and functional. As you can see, I’m using dtach to run a Seedbox 🙂

EC2-like instances with Slicehost

Slicehost’s backup and add new slice

Virtual hosting and infrastructure is coming up in a big way, and Amazon’s EC2 is leading the pack. However, there doesn’t exist a comprehensive & provably scalable system for deploying applications to such a stack yet: the components are in place or being developed (SimpleDB or equivalent, EC2 and S3) but the crucial connecting layer is missing. Several attempts are on to fill the gap.

Until it becomes a reality (and perhaps even then), a great intermediate tool is Slicehost‘s xen-based virtual machines: “slices”, that allow for a quick snapshot from any particular slice and the creation of a new slice from that snapshot.

The way it works is a bit cumbersome now (and they really should expose this feature more), but the idea is simple:

  1. Take a snapshot of a slice, currently you have to go to the backups tab to take a snapshot backup.
  2. When you choose to create a new slice, you have the option to create from that backup snapshot (see picture).
  3. Change a few variables specific to the host (maybe add a puppet instance to reconfigure automatically), and wham! EC2-like virtual infrastructure at half the pain.

I plan to use this for factory-like instances across MobME‘s infrastructure.

Deploying JAR/JAD content via WAP

An easy way to deploy JAR/JAD content on all phones with GPRS is to create a WML site and provide a link to the JAR file that you have. [As an aside, the whole J2ME scene is so fragmented that it sucks completely. Let’s hope Android sorts it out.]

Here’s how you go about setting up that in Lighttpd:

Add these mimetypes (to lighttpd.conf or mime-types.conf):

mimetype.assign += (
  ".wml"     =>      "text/vnd.wap.wml",
  ".wmlc"   =>      "application/vnd.wap.wmlc",
  ".wmls"   =>      "text/vnd.wap.vmlscript",
  ".vmlsc"  =>      "application/vnd.wap.wmlscriptc",
  ".wbmp"   =>     "image/vnd.wap.wbmp",
  ".jar"      =>      "application/java-archive",
  ".jad"     =>      "text/vnd.sun.j2me.app-descriptor"
)

Add the index.wml file as a valid DirectoryIndex:

server.indexfiles    += ("index.wml")

Make this .wml file and upload it somewhere (replace JARCLIENT with your .jar file, and JADCLIENT with your .jad file):

<?xml version="1.0"?>
<!DOCTYPE wml PUBLIC "-//WAPFORUM//DTD WML 1.1//EN"
"http://www.wapforum.org/DTD/wml_1.1.xml">

<wml>
<card title="Download">

<p>Download the Jar file:</p>
<p>
<a href="JARCLIENT.jar">Download</a>
</p>
<p>
<a href="JADCLIENT.jad">Download (JAD)</a>
</p>

</card>
</wml>

Restart your server, and that should be it! If you’re using Apache, the steps are similar.

Kai Voigt's MySQL talk

Condensed: One gora, a madrassi and two longdas (of whom only one deigns to be online) attended Kai Voigt’s MySQL Cluster talk at ValueOne today. Here are my thoughts:

First off, the talk was pretty interesting, and any talk that has a live demo in it has my vote on being useful and down to the metal. An interesting overview of the whole process, including what clustering means and an honest appraisal about its pretty crippling deficiencies in 5.0 [no on-disk store, no persistence, anyone?]. 5.1 is supposed to Make Things Better, but Fulltext Search is still hazy.

Which made me think about memcached and how similar clustering as it stands right now is to danga’s (semi)beast. Except of course, the pretty important difference of having an SQL interface to the data on top. Well guess what, I’m not the first person (Google Cache link) to think about this, and
even a thread on memcached lists asks this question. Multiple levels of caching seems to be the way to go though, right from Squid (or better yet, Varnish) down to an in-memory cache.
Right now, stuff that looks really cool for server deployment (from my 2 month experience at Slideshare):

  1. Xen and virtualization. Dom0 Gentoo boxen running Gentoo kernels optimized for a web/app/db etc (see EngineYard [movie]).
  2. Solaris and ZFS pools for reliable local storage (clashes with 1, (open)Solaris is not even yet domU). S3 or homegrown MogileFS-ish platforms for distributed storage.
  3. Webcache – Squid & Varnish
  4. MySQL tuned, PostgreSQL
  5. Puppet, for dependency-based VM configuraton (?).
  6. Monitoring: monit (watch process), munin (report), or Cacti
  7. S3+EC2 (reduce latency to S3 via an EC2 stack and make S3 behave like memcached or better.)

Well, I want to play with all of these seven. Good news is that Leopard will have dtrace (and possibly even ZFS), the cool things from Sun, and I’ll be able to play with the rest on my Mac anyways.

Kai’s code is online. Word is, his slides will be too, pretty soon.

Pat on the back: the first line in the blog post reveals my growing understanding of hindiland. 🙂