Translating Listserv archive to mbox format

We wanted to import our old mailing list entries from the OpenMRS mailing list Listserv archives into Nabble.  No problem.  Finding the GET listname FILELIST and GET listname file1, GET listname file2, … commands was easy enough.  A quick search of Nabble support made it clear that I needed to send them mbox files.  So, I set out in search of a Listserv to mbox converter.  I found a couple scripts: one in perl and another in PHP.  But trying them out, made it clear that I was going to have to do some tweaking.  After a few near misses, I thought: “I could do this easier in Groovy.”  So, I ended up with this script.  Basically, it came down to leaving the messages and their headers alone and just adding a From_ line in front.  Otherwise, the only tricky part was getting the dates right (GMT time without timezone specified in the From_ line and a some reshuffling of the date format in the message header).

Both for future me and anyone else who might benefit, here’s the script I ended up with:

import java.text.SimpleDateFormat

delim = '=' * 73	// LISTSERV separates messages with a bar of equal signs
foundDelim = false	// we skip all content until first delimiter
inHeader = true		// true when processing header data
def header = ""		// holds current header data

dfListserv = new SimpleDateFormat("E, d MMM yyyy HH:mm:ss z")
dfHeader = new SimpleDateFormat("E MMM dd HH:mm:ss yyyy z")
dfMbox = new SimpleDateFormat("E MMM dd HH:mm:ss yyyy")
dfMbox.timeZone = TimeZone.getTimeZone("GMT")	// for mbox, convert to GMT and drop timezone reference
cal = Calendar.instance

// Process input line by line from stdin
System.in.eachLine() { line ->
  if (!foundDelim)
    foundDelim = (line == delim)	// skip until we find first delim
  else if (inHeader) {
    // within header
    if (line =~ /^s*$/) {
      // empty line signals end of header
      
      // fetch Date from header and reformat it for output
      m1 = header =~ /(?ms)^Date:s+(.*?)s*$/
      date = dfListserv.parse(m1[0][1])
      cal.time = date
      mboxDate = dfMbox.format(cal.time)
      headerDate = dfHeader.format(cal.time)

      // fetch From from header
      m2 = header =~ /(?ms)^From:s+(.*?)s*$/
      fromHeader = m2[0][1]
      leftBracket = fromHeader.indexOf('<')
      rightBracket = fromHeader.indexOf('>')
      if (leftBracket > 0 && rightBracket > leftBracket)
        from = fromHeader.substring(leftBracket+1, rightBracket)
      else
        from = fromHeader

      // output header with mbox-required From_ line up front and reformatted date
      header = "From $from $mboxDaten" + header.replaceAll(/(?m)^Date:s+.*$/, "Date: $headerDate")
      println "$headern"

      inHeader = false	// no longer in header
      header = ""	// clear for next message
    } else {
      header += "$linen"	// accumulate full header data
    }
  } else if (line == delim) {
    // if we find a delim, begin processing next line as header
    print "nn"
    inHeader = true
  } else {
    // within a message, just send it through untouched
    println line
  }
}

Creating a Desktop Shortcut on a Mac

Making a desktop shortcut icon on the Mac is pretty easy.  Open up the AppleScript script editor and make a script like this:

do shell script “/Applications/Firefox.app/Contents/MacOS/firefox-bin -P Developer -no-remote &”

Then save as an application on the desktop.  Done.

This example shows how to pass parameters from AppleScript.  In particular, it shows how to open a 2^nd Firefox window with the “Developer” profile by passing some command line parameters from AppleScript.

Three-and-out Rodriguez

Rich Rodriguez, besides achieving the most number of losses in 129 years of Michigan football and ending the longest bowl streak, has managed to stand out in the NCAA this year for going three-and-out.  Well you got three wins this year, Mr. Rodriguez.  You’re out.

One small problem at AMIA…

Some of our Regenstrief presentations are too crowded to attend. 🙂 Here, Marc Overhage shows a packed room for Jeff Friedlin’s presentation.

AMIA 2008 … A one two punch!

I’m sitting in the opening ceremony for AMIA 2008, where they announced the three student paper competition winners. I’m thrilled to say that Regenstrief fellows captured both first (Martin Were) and second (Jeff Friedlin) place! Way to go Martin & Jeff! And way to go Regenstrief!!

Logic-a-thon Wrap-up

This was a great hack-a-thon.  We tackled a lot of different issues for the logic service.  Unfortunately, a number of these will require additional effort to be completed.

We knocked out some of the simpler tickets:

• Rules tokens need to be case insensitive (#1104)
• Encounter datasource expansion (#1105)
• Creating an Operand interface (#1106)

We made some decisions (like not moving Result methods to a utility class) and we make some serious inroads (including code) on how to solve some of the more difficult issues (persisting tokens #924, persisting token tags #969, caching issues #1107, and changing “today” within criteria #929).

All-in-all.  A great job by all involved!  A special thanks to Tammy for making it a success!

OpenMRS Logic-a-thon

So far, so good at the OpenMRS! To our surprise, we’re actually getting code written. We’re not sure if it’s because of Tammy’s amazing prep work or because Paul & Hamish couldn’t make it. 🙂

Looking forward to tomorrow…