Recipe to see changes in repo in real time
Today, I was playing with Markdown in a README.md file for a repository and thought it’d be helpful if I could see my changes in real time (this was a junk repo I was going to delete, so I wasn’t worried about a bunch of commits). Just thought I’d blog this for my own future reference.
Create a script to commit changes and reload Safari…
update.sh
#!/bin/bash
# Commit changes
git add README.md
git commit -m "formatting"
git push
# Reload Safari
osascript -e 'tell first window of application "Safari" to do JavaScript "window.location.reload(true)" in current tab'
Install fswatch (Mac’s version of inotifywait), a tool to monitor for changes to a file or folder.
brew install fswatch
With the following command, any change to our file will get committed and the page in Safari will be reloaded.
fswatch -o README.md | xargs -n1 ./update.sh
Tab Stack for Safari
I recently decided to try switching from Chrome to Safari for a while… to see if it might buy me a little more battery life on my aging MacBook Pro. One of the immediate drawbacks I discovered was realizing how accustomed I had become to the Tab Stack extension for Chrome. It’s a simple workaround for MRU (most recently used) tab ordering that moves the active tab to the left after a brief delay (the delays lets you scan through tabs without re-ordering them). It’s a bit hacky and I’m sure a lot of people wouldn’t care for the approach, but I’m used to it and immediately noticed its absence in Safari.
I googled around and didn’t find an equivalent extension for Safari… so I made one and submitted it to Apple’s gallery. I’ve added an extra hack to mitigate Safari’s insistence on clearing the address bar when tabs are moved (move new tabs immediately) as a version 1.1.
Informal Feedback in a Single Click
There are many ways to capture feedback from users & testers, from feedback buttons built into the app to issue trackers and tools like JIRA Capture. Another method we have been using, especially for upcoming releases or new features or widgets, within the OpenMRS community is a side-by-side feedback page.
The application is on the left and an etherpad on the right. While I am not suggesting this as an approach for issue tracking, but we have found it to be a quick & easy way of collecting community feedback. The combination of a link taking them directly to the product to be tested along with the near-zero activation energy required by etherpad makes it a handy combination. It’s also nice to be able to throw a brief intro into the etherpad to direct people on what to test and how to report feedback. And lastly, there’s a nice side effect of people seeing each other’s activity in real time. When combined with a developer responding to feedback and re-deploying fixes in real time, it can be incredibly powerful.
Anyway, the main reason I decided to blog on this is because I tweaked our side-by-side tool a bit and wanted to throw my one-page feedback HTML in here for the next time I need it. Here it is:
Feedback
OpenMRS Data Model Browser
Ever since the beginnings of OpenMRS, we’ve used the data model as a reference and as a teaching tool. As the number of tables has grown, it has become harder to keep the data model diagram updated. I also wanted an easy way to search for tables, columns, or foreign keys. So, I created dbtohtml to generate an easily browsable, standalone, singe-page HTML view of the data model.
Freezing and Thawing Droplets in a DigitalOcean
DigitalOcean has been a game-changer for me. Why create another space-hungry VM locally when you can spin up a new machine in 60 seconds on DO? And it gets better: Tugboat. Now, I can manage my droplets from the command line. Since, I typically use (and reuse) droplets like I did local VMs, I often want to set a droplet aside for a while (maybe weeks or months) and return to it later. Fortunately, DO provides a way to put snapshots into cold storage and then retrieve them later. But freezing and thawing droplets wasn’t easy enough. I made a suggestion to DO, but I doubt it’ll be implemented anytime soon (if ever), so I used Tugboat and some Groovy scripts to roll my own.
Here is what I was looking for:
Freezing a droplet
- Take an existing droplet, create a snapshot using the same name, then destroy the droplet.
- If a snapshot already exists with the same name, delete it (so freezing a previously thawed droplet replaces the previous snapshot).
Thawing a droplet
- As long as a droplet doesn’t already exist with the same name, create new droplet from a snapshot using the same name and start it up.
The goal:
$ # Assume we have a droplet foo
$ tugboat create foo
$ # Imagine you're done working with foo for now
$ freeze foo
$ # Foo is a snapshot & the droplet is destroyed.
$ # ... weeks pass and you have a hankering for foo ...
$ thaw foo
$ # Foo is back, Baby!
While it would be great to have freeze & thaw button on the DO website freeze & thaw parameters for Tugboat, I didn’t have the time to make a pull request for Tugboat… so here are the scripts:
~/bin/freeze
This script will snapshot a droplet, replacing any snapshot of the same name, and destroy the droplet.
#!/usr/bin/env groovy
class TugboatException {
// Our very own little exception is born. It's a buoy!
}
def getImageInfo = {
"tugboat images".execute().text.split("\n").find{
it.startsWith(imageName+" ")
}
}
def waitFor = { imageName, to='appear' -> /* to='appear' or 'disappear' */
attempts = 0
while (true) {
attempts++
imageInfo = "tugboat images".execute().text.split("\n").find{
it.startsWith(imageName+" ")
}
if ((imageInfo && to=='appear') || (!imageInfo && to=='disappear')) break
if (imageInfo || attempts > 20) {
throw new TugboatException("$imageName did not $to within 3 min. Gave up.")
}
sleep(10000) // wait 10 seconds between checks
}
}
def cmd = { description, command ->
print description
response = command.execute().text
println "done."
}
def cli = new CliBuilder(usage:'freeze DROPLET_NAME')
cli.q(longOpt:'quiet', '')
def options = cli.parse(args)
if (!options.arguments() || options.arguments().size != 1) {
cli.usage()
System.exit(0)
}
imageName = options.arguments()[0]
if (! "tugboat droplets".execute().text ==~ /(?ims).*^$imageName\s.*/) {
println "$imageName droplet does not exist"
System.exit(1)
}
imageInfo = getImageInfo()
if (imageInfo) {
imageId = (imageInfo =~ /id:\s*(\d+)/)[0][1]
if (imageId) {
cmd("Destroying old $imageName image...", "tugboat destroy-image -c -i $imageId")
waitFor(imageName, 'disappear')
}
}
cmd('Telling droplet to halt...', "tugboat halt $imageName")
cmd('Waiting for droplet to shut down...', "tugboat wait $imageName -s off")
sleep(3000)
cmd('Taking snapshot of droplet...', "tugboat snapshot $imageName $imageName")
print "Waiting for image to complete..."
waitFor(imageName)
println "done."
cmd("Destroying $imageName droplet...", "tugboat destroy -c $imageName")
~/bin/thaw
This script will restore a frozen droplet and start it up for you.
#!/usr/bin/env groovy
def getImageInfo = {
"tugboat images".execute().text.split("\n").find{
it.startsWith(imageName+" ")
}
}
def cmd = { description, command ->
print description
response = command.execute().text
println "done."
}
def cli = new CliBuilder(usage:'thaw IMAGE_NAME')
cli.q(longOpt:'quiet', '')
def options = cli.parse(args)
if (!options.arguments() || options.arguments().size != 1) {
cli.usage()
System.exit(0)
}
imageName = options.arguments()[0]
if ("tugboat droplets".execute().text ==~ /(?ims).*^$imageName\s.*/) {
println "$imageName droplet already exists"
System.exit(1)
}
imageInfo = getImageInfo()
if (!imageInfo) {
println "Image $imageName not found"
System.exit(2)
}
imageId = (imageInfo =~ /id:\s*(\d+)/)[0][1]
if (!imageId) {
println "Unable to parse $imageName image id"
System.exit(3)
}
print "Thawing image..."
response = "tugboat create $imageName -i $imageId".execute().text
println "done."
print "Waiting for droplet to start..."
response = "tugboat wait $imageName".execute().text
println "done."
Faster resets for the OpenMRS Demo
I’ve long been bothered by how long it takes for our demo site (http://demo.openmrs.org) to be reset. The site goes down for a few minutes each hour as the database is restored and the application restarted. I was working on the demo site today in the middle of one of these resets and so I decided to look around for a better solution.
Fortunately, Akshay Suryawanshi from Percona (percona.com) recently shared a nifty rename_db script that can rename a MySQL database in a couple of seconds. So, we put our credentials and host information into a file (don’t forget chmod 600 .my.cnf to protect those credentials):
.my.cnf
[client]
user=openmrs
password=secret
host=127.0.0.1
port=3306
Assuming the credentials and other settings are correct, then this script should show existing databases:
show-databases
#!/bin/bash
mysql --defaults-extra-file=.my.cnf -e "show databases"
Then we take Akshay’s rename_db script, define DEFAULTS=.my.cnf at the beginning, replace the “-h $1” references with --defaults-extra-file=$DEFAULTS, and remove the initial host parameter:
rename_db
Adapted from Akshay Suryawanshi’s blog post, thanks to the generous folks at Percona for sharing.
#!/bin/bash
# Copyright 2013 Percona LLC and/or its affiliates
# http://www.mysqlperformanceblog.com/2013/12/24/renaming-database-schema-mysql/
#
# Changes
# - Uses --defaults-extra-file for credentials, host, and port settings
set -e
DEFAULTS=.my.cnf
if [ -z "$2" ]; then
echo "rename_db "
echo "(assumes $DEFAULTS defines user, password, host, and port under [client] section)"
exit 1
fi
db_exists=`mysql --defaults-extra-file=$DEFAULTS -e "show databases like '$2'" -sss`
if [ -n "$db_exists" ]; then
echo "ERROR: New database already exists $2"
exit 1
fi
TIMESTAMP=`date +%s`
character_set=`mysql --defaults-extra-file=$DEFAULTS -e "show create database $1\G" -sss | grep ^Create | awk -F'CHARACTER SET ' '{print $2}' | awk '{print $1}'`
TABLES=`mysql --defaults-extra-file=$DEFAULTS -e "select TABLE_NAME from information_schema.tables where table_schema='$1' and TABLE_TYPE='BASE TABLE'" -sss`
STATUS=$?
if [ "$STATUS" != 0 ] || [ -z "$TABLES" ]; then
echo "Error retrieving tables from $1"
exit 1
fi
echo "create database $2 DEFAULT CHARACTER SET $character_set"
mysql --defaults-extra-file=$DEFAULTS -e "create database $2 DEFAULT CHARACTER SET $character_set"
TRIGGERS=`mysql --defaults-extra-file=$DEFAULTS $1 -e "show triggers\G" | grep Trigger: | awk '{print $2}'`
VIEWS=`mysql --defaults-extra-file=$DEFAULTS -e "select TABLE_NAME from information_schema.tables where table_schema='$1' and TABLE_TYPE='VIEW'" -sss`
if [ -n "$VIEWS" ]; then
mysqldump --defaults-extra-file=$DEFAULTS $1 $VIEWS > /tmp/${1}_views${TIMESTAMP}.dump
fi
mysqldump --defaults-extra-file=$DEFAULTS $1 -d -t -R -E > /tmp/${1}_triggers${TIMESTAMP}.dump
for TRIGGER in $TRIGGERS; do
echo "drop trigger $TRIGGER"
mysql --defaults-extra-file=$DEFAULTS $1 -e "drop trigger $TRIGGER"
done
for TABLE in $TABLES; do
echo "rename table $1.$TABLE to $2.$TABLE"
mysql --defaults-extra-file=$DEFAULTS $1 -e "SET FOREIGN_KEY_CHECKS=0; rename table $1.$TABLE to $2.$TABLE"
done
if [ -n "$VIEWS" ]; then
echo "loading views"
mysql --defaults-extra-file=$DEFAULTS $2 < /tmp/${1}_views${TIMESTAMP}.dump
fi
echo "loading triggers, routines and events"
mysql --defaults-extra-file=$DEFAULTS $2 < /tmp/${1}_triggers${TIMESTAMP}.dump
TABLES=`mysql --defaults-extra-file=$DEFAULTS -e "select TABLE_NAME from information_schema.tables where table_schema='$1' and TABLE_TYPE='BASE TABLE'" -sss`
if [ -z "$TABLES" ]; then
echo "Dropping database $1"
mysql --defaults-extra-file=$DEFAULTS $1 -e "drop database $1"
fi
if [ `mysql --defaults-extra-file=$DEFAULTS -e "select count(*) from mysql.columns_priv where db='$1'" -sss` -gt 0 ]; then
COLUMNS_PRIV=" UPDATE mysql.columns_priv set db='$2' WHERE db='$1';"
fi
if [ `mysql --defaults-extra-file=$DEFAULTS -e "select count(*) from mysql.procs_priv where db='$1'" -sss` -gt 0 ]; then
PROCS_PRIV=" UPDATE mysql.procs_priv set db='$2' WHERE db='$1';"
fi
if [ `mysql --defaults-extra-file=$DEFAULTS -e "select count(*) from mysql.tables_priv where db='$1'" -sss` -gt 0 ]; then
TABLES_PRIV=" UPDATE mysql.tables_priv set db='$2' WHERE db='$1';"
fi
if [ `mysql --defaults-extra-file=$DEFAULTS -e "select count(*) from mysql.db where db='$1'" -sss` -gt 0 ]; then
DB_PRIV=" UPDATE mysql.db set db='$2' WHERE db='$1';"
fi
if [ -n "$COLUMNS_PRIV" ] || [ -n "$PROCS_PRIV" ] || [ -n "$TABLES_PRIV" ] || [ -n "$DB_PRIV" ]; then
echo "IF YOU WANT TO RENAME the GRANTS YOU NEED TO RUN ALL OUTPUT BELOW:"
if [ -n "$COLUMNS_PRIV" ]; then echo "$COLUMNS_PRIV"; fi
if [ -n "$PROCS_PRIV" ]; then echo "$PROCS_PRIV"; fi
if [ -n "$TABLES_PRIV" ]; then echo "$TABLES_PRIV"; fi
if [ -n "$DB_PRIV" ]; then echo "$DB_PRIV"; fi
echo " flush privileges;"
fi
We use the following script to backup the OpenMRS database:
backup-openmrs
#!/bin/bash
mysqldump --defaults-extra-file=.my.cnf --add-drop-database --extended-insert \
--single-transaction openmrs > openmrs.sql
Then we create the following scripts:
drop-openmrs
#!/bin/bash
mysql --defaults-extra-file=.my.cnf -e "drop database openmrs"
restore-openmrs1
#!/bin/bash
mysql --defaults-extra-file=.my.cnf \
-e "create database openmrs1 DEFAULT CHARACTER SET utf8"
mysql --defaults-extra-file=.my.cnf openmrs1 < openmrs.sql
replace-openmrs
#!/bin/bash
./drop-openmrs
./rename_db openmrs1 openmrs
Now, when we are preparing to reset the openmrs database, we can execute the restore-openmrs1 script, which will place a fresh copy of the default OpenMRS data into the openmrs1 database (restoring the database can take several seconds, but since we are doing this into openmrs1 and the demo is using openmrs, we can perform this step before involving the demo site). Then, when we are ready to reset the database, simply executing replace-openmrs will reset the openmrs database in 2-3 seconds or less. Given that the demo data is a non-production system, we could even perform this data replacement without restarting the demo application.
Now, instead of taking 30-90 seconds or more, the OpenMRS Demo can be reset in 2-3 seconds or less.
Thank you Akshay Suryawanshi and the folks at Percona for sharing!
Fun with HTTP
Playing around with cURL and learning about some handy online tools…
Visit requestb.in and you’ve instantly got a link against which you can post data and see the results.
CLBIN
clbin.com is a nifty tool for directing command line output straight to a short url.
IRCCloud Last Read Bookmarklet
If you are like me and participate in several IRC channels but don’t get to be in them all the time and sometime are away from them for several days, then IRCCloud is the perfect solution (a web-based IRC client that rivals any other and awesome mobile clients). While IRCCloud makes it very easy to load a chunk of history (simply scrolling up or clicking on a bar), loading the history of a busy IRC channel for several days or weeks that can span multiple chunks of history is cumbersome. I’ve bugged James a few times to ask for a click-once-to-load-all-history-back-to-last-read-message link. When I recently suggested it again in the #feedback channel, I got some javascript tips from James that allowed me to do the next best thing: make a bookmarklet. Simply drag this link to your bookmark bar:
When you are using IRCCloud and enter a channel that you haven’t visited in a while (your last read message is several hours or days ago), click the bookmarklet and it will load history until it reaches your last read message. In the rare case that you’ve been away for weeks or months and it takes more than a minute to load all the history you’ve missed, the bookmarklet will stop after a minute; just click it again to keep loading.
Alfred Workflows
If you own a Mac and haven’t discovered Alfred 2 yet, well I’m sorry. Combined with its PowerPack, it’s a formidable tool for doing things quickly on the Mac. I’ve used QuickSilver and LaunchBar. I held out for a long time as a staunch LaunchBar user, but recently made the switch to Alfred 2 and I’ve got it doing anything everything I need (and boy do I need a lot… I’m lazy and I’ll love keyboard shortcuts). The only shortcoming with Alfred is its clipboard history, which is limited to text (no images… bummer). LaunchBar does the clipboard history just right, so I map Alt+Cmd+K to LaunchBar’s clipboard history and I’m good to go. Sure, LaunchBar is an expensive tool just for clipboard history, but I already owned it and why use Alfred’s crappy clipboard history when I have access to LaunchBar? 🙂
Anyway, I thought I’d share a few of my (now ~24) Alfred 2 workflows for anyone interested:
 |
Arrows Easily type arrow characters (↑, ↓, ←, →) using HTML entity names: uarr, darr, larr, and rarr. |
 |
IRC Cloud Open IRC Cloud in Chrome with irc. If you’ve already got IRC Cloud in a tab, it doesn’t open it in a second tab. |
 |
OpenMRS A workflow for OpenMRS developers with a few handy shortcuts.
|
 |
Volume Controls Provides some shortcuts for controlling your volume. You thought you were lazy? I know there are dedicated keys for this, but they’re way over there. Alfred shortcuts can be habituated (once learned, you just think and it happens) and don’t require your hands to leave the “home row” of the keyboard.
|
jsonpretty: handy when playing with web services
I just ran into a handy little utility: jsonpretty.
$sudo gem install json jsonpretty
While
$curl -i http://localhost:8081/openmrs-standalone/ws/rest/v1/catalog
gives you something like this:
{"catalog":[{"name":"Cohort","operations":[{"name":"GET http://localhost:8081/openmrs/ws/rest/v1/cohort?q","description":"Fetch all non-retired that match this parameter"},{"name":"GET http://localhost:8081/openmrs/ws/rest/v1/cohort/{uuid}","description":"Fetch by unique uuid"},{"name":"GET http://localhost:8081/openmrs/ws/rest/v1/cohort","description":"Fetch all non-retired"},{"name":"POST http://localhost:8081/openmrs/ws/rest/v1/cohort","description":"Create with properties in request"},{"name":"POST http://localhost:8081/openmrs/ws/rest/v1/cohort/{uuid}","description":"Edit with given uuid, only modifying properties in request"},{"name":"DELETE http://localhost:8081/openmrs/ws/rest/v1/cohort/{uuid}?!purge","description":"Delete this object from the database"},{"name":"DELETE http://localhost:8081/openmrs/ws/rest/v1/cohort/{uuid}?purge","description":"Delete this object from the database"}],"url":"http://localhost:8081/openmrs/ws/rest/v1/cohort","representations":[{"name":"ref","properties":["uuid","display","links"]},{"name":"default","properties":["uuid","name","description","voided","memberIds","links"]},{"name":"full","properties":["uuid","name","description","memberIds","voided","auditInfo","links"]}]},{"name":"CohortMember","operations":[{"name":"GET http://localhost:8081/openmrs/ws/rest/v1/cohort/{parentUuid}/members/{uuid}","description":"Fetch by unique uuid"},{"name":"GET http://localhost:8081/openmrs/ws/rest/v1/cohort/{parentUuid}/members","description":"Fetch all non-retired"},{"name":"POST http://localhost:8081/openmrs/ws/rest/v1/cohort/{parentUuid}/members","description":"Create with properties in request"},{"name":"POST http://localhost:8081/openmrs/ws/rest/v1/cohort/{parentUuid}/members/{uuid}","description":"Edit with given uuid, only modifying properties in request"},{"name":"DELETE http://localhost:8081/openmrs/ws/rest/v1/cohort/{parentUuid}/members/{uuid}?!purge","description":"Delete this object from the database"},{"name":"DELETE http://localhost:8081/openmrs/ws/rest/v1/cohort/{parentUuid}/members/{uuid}?purge","description":"Delete this object from the database"}],...
adding jsonpretty:
$curl -i http://localhost:8081/openmrs-standalone/ws/rest/v1/catalog | jsonpretty
gets you something much prettier:
{
"catalog": [
{
"name": "Cohort",
"operations": [
{
"name": "GET http://localhost:8081/openmrs/ws/rest/v1/cohort?q",
"description": "Fetch all non-retired that match this parameter"
},
{
"name": "GET http://localhost:8081/openmrs/ws/rest/v1/cohort/{uuid}",
"description": "Fetch by unique uuid"
},
{
"name": "GET http://localhost:8081/openmrs/ws/rest/v1/cohort",
"description": "Fetch all non-retired"
},
{
"name": "POST http://localhost:8081/openmrs/ws/rest/v1/cohort",
"description": "Create with properties in request"
},
{
"name": "POST http://localhost:8081/openmrs/ws/rest/v1/cohort/{uuid}",
"description": "Edit with given uuid, only modifying properties in request"
},
{
"name": "DELETE http://localhost:8081/openmrs/ws/rest/v1/cohort/{uuid}?!purge",
"description": "Delete this object from the database"
},
{
"name": "DELETE http://localhost:8081/openmrs/ws/rest/v1/cohort/{uuid}?purge",
"description": "Delete this object from the database"
}
],
"url": "http://localhost:8081/openmrs/ws/rest/v1/cohort",
"representations": [
{
"name": "ref",
"properties": [
"uuid",
"display",
"links"
]
},
{
"name": "default",
"properties": [
"uuid",
"name",
"description",
"voided",
"memberIds",
"links"
]
},
{
"name": "full",
"properties": [
"uuid",
"name",
"description",
"memberIds",
"voided",
"auditInfo",
"links"
]
}
]
},
...
Nice! Thank you jsonpretty! 🙂