Pete's Points

Flickr Date Fixer

2013-04-28T04:36:41.827Z

So the startup/solopreneur community really likes little mantras and axioms. I wrote a little web application that follows mantras such as:

solve a problem
focus on a small problem in a niche market
scratch your own itch
build the minimum viable product and ship quickly

I have built and released a web app that focuses on an itch I have and may serve a small niche market consisting of only myself, but with that understood, my app is able to:

Locate your flickr photos with an incorrect "taken" date
Show you a list of photos with this problem and let you choose to fix them individually or en masse

In order to find this app useful, you probably have to:

Take photos on an Android smartphone
Be running Android 2.2 "Froyo" (I suspect other version have resolved this bug)
Upload them to flickr

Not sure if anyone other than me fits all these criteria, but this is fundamentally a bug in the camera app on these phones and for reasons I cannot fathom, Google seems unwilling or uninterested in releasing an update with a fix. The problem is incorrect EXIF metadata within the photo image files themselves. The bug causes every photo to be marked as December 8, 2002 at noon.

The app is pretty small and gave me a nice opportunity to try some new technologies (as well as use many that have been in my preferred stack for months or years).

node.js
coffee-script
express.js
passport.js for OAuth integration with flickr
backbone.js and jQuery in the browser
require.js
grunt.js
nodejitsu

The nice thing about this app is given that it is all OAuth based and is basically a custom UI using the flickr API for all the business logic, it doesn't need any database at all. This means it is easy to deploy. So for the novelty of it, I have chosen to deploy it onto nodejitsu, and in theory they will host it for free since it is open source.

Resources

Flickr Date Fixer running live on nodejitsu's http://flickrdatefixer.jit.su
flickr-date-fixer source code on github
Nodejitsu

AirPair on TechCrunch

2013-03-04T16:30:24.614Z

TechCrunch has this article on AirPair today with a nice quote about me. This is my first ever mention on TechCrunch (but hopefully not my first and only!).

My Airpair Session was written up

2013-02-26T04:28:29.842Z

Here's a post about my recent experience with the new AirPair service connecting entrepreneurs with technology experts.

Airpair

2013-02-23T22:26:52.624Z

So I've just been one of the earliest progragrammers to get involved with AirPair. It's a service that pairs entrepreneurs with technology experts for code review and pair programming sessions to help them with their coding projects.

I did some google hangouts with an entrepreneur who built his own Ruby on Rails app that analyzes geology data files for oil & gas companies and is just starting to port it over to node.js. There's a youtube recording video of the 2-hour google hangout session this morning up. You can skip around in that if you want to get a feel for the service and experience.

I'm perpetually interested in the startup community in general and connecting people across geographies via the Internet, so I was excited to try it out. I've done 3 hour-long sessions so far and so far so good.

npm modules: development and release versions

2013-02-01T20:17:35.407Z

You are working on two npm modules that are both actively being developed:

my_app: some application, which depends on my_lib
my_lib: a shared library

There are 2 modes you'll need inside my_app: 1) using a published release of my_lib and 2) using a local development version of my_lib.

Using a published release of my_lib is standard faire:

list my_lib as a dependency in my_app/package.json
1. dependencies: {"my_lib": "1.2.x"}
cd my_app && npm install
npm will populate my_app/node_modules/my_lib for you as you would expect

To use your development version so you can easily make changes to both codebases and have them available with minimal fuss, set up this structure

~/projects/my_app (working directory for the app)
~/projects/my_lib   (working directory for the lib)
~/projects/node_modules/my_lib (symlink to ../../my_lib)

You can set this up with

mkdir -p ~/projects/my_app ~/projects/my_lib ~/projects/node_modules
cd ~/projects/node_modules
ln -nsf ../../my_lib

So now when developing my_app and you want to use the local development version of my_lib, just do :

cd ~/projects/my_app
npm uninstall my_lib

Now node's require function will load my_lib from ~/projects/node_modules/my_lib and you are good to go. To switch back to a public of my_lib just do:

cd ~/projects/my_app
npm install

Of course, there is the npm link command which does something nearly identical to this, so far I prefer this approach as I find it a little simpler.

Hilarious Recruiter Email

2012-12-12T21:49:52.792Z

Here is the full text of a recruiter email I received today.

Dear Pete,

Wouldn't it feel great to wake up each day knowing that you're part of a dynamic organization that's on the cutting edge of innovation web products? That’s what the team experiences and you could too!

What you will find challenging about this job is to develop revolutionary and challenging web applications using object oriented javascript frameworks like backbone.js, angular.js, HTML, and CSS.

As a result, there are many exciting decisions to be made to keep it in line with demands. You won't find formulaic thinking here. As a company that specializes in innovation, they want the best and brightest creative visionaries who think so far out of the box that the box isn't even in the picture anymore. Where others say "can't", you say "how".

Keeping in mind that most successful people find opportunity not when they're looking but when it knocks, how's my timing?

I'd be happy to talk to you further about it. I'm reaching out to key individuals before I go on a full scale search for this. You have an ideal background and I think it's a good match for you geographically.

Fortune Favors the bravo!

Find closer OKCupid matches with this bookmarklet

2012-10-23T03:15:05.735Z

I have written a little bookmarklet that will allow you to filter your OKCupid matches by a smaller geographic region. By default, their search filters allow no smaller than a 25-mile radius. Drag the bookmarklet link below onto your bookmarks toolbar, then click it when you are on the OKC search page, and voilà, you will have the options for 5 and 10-mile searches. 10-mile radius will be selected, so just click "search" and you will have your results.

OKCupid Closer Matches

Some notes:

If you have no idea WTF a bookmarklet is or how to use one, read this
I live close to both Boulder and Denver Colorado. These are two quite different worlds, but they are both within 25 miles of my town. Since Denver's population is 10x the size of Boulder's, most of my search results are Denver residents, when I'm really only insterested in folks within biking distance.
After you click search, the default search form will re-appear and it will look like your search has a 25 mile radius. Never fear, the query string has filter3=10, which is what is required, and based on my results, yes, the OKC servers do seem to actually respect this query string parameter even if the value is not offered in the search form.
You need to re-click the bookmarklet for each new search. Sorry. Send feature requests to OKCupid until they add this permanently. Technically I could make this into a Chrome extension and have it always work, so I may do that at some point.
Thanks to Chris Zarate's handy Bookmarkleter utility
And for you web dev nerds out there, I even wrote this with (gasp!) Vanilla JS

Great Support from Herman Miller

2012-09-27T16:48:31.202Z

So I use one of the famous Herman Miller Aeron chairs in my home office. I love it primarily because the mesh seat prevents swampass, but of course overall it is a highly ergonomic chair, nicely adjustable, rugged, and holds its value well. My specific chair was made in 2000 and I bought it used on craigslist for $250 in 2009. Today exactly the 12-year warranty finally expires. About two weeks ago it suddenly broke. A thin rod broke free of its clamp and started protruding from the bottom such that the casters wouldn't hit the ground anymore. I was worried that I would need to replace the chair, but given my trend toward more DIY and frugality, I called up Herman Miller support to see what they would say. Staff was out at training that day so I left a voice mail and called again Monday. I spoke with a woman who had heard my voice mail and had me on her list to call back. I told her what my chair was doing and she shipped me out a repair kit for the main problem as well as a set of repair kits to fix the arms, which wouldn't hold their adjusted height. These arrived a bit later with hand-drawn instructions on how to do the repair. The diagrams weren't entirely clear (earth to furniture industry: we have these things called photographs. Stop sending out line drawing pencil art instructions.), but after some futzing I was able to properly repair the main hydraulic rod as well as the arms. I managed to bust the tilt adjuster in the process of putting my chair on its side many times, though. The back was now stuck dead vertical. I called support again. The rep's business card and direct phone number were including in the package of repair kits. She instructed me to stand on the back legs of the chair and lift the chair back up to unlock it. Now my Aeron is back to working condition, and it didn't cost me a time. Herman Miller shipped the repair kits free and the support consultation was free. That's great service!

Managing Per-Project Interpreters and the PATH

2012-09-02T21:35:19.427Z

So let's talk about managing runtimes and interpreters for projects and applications. This post comes about after I have seen a vast jungle of non-solutions and dead ends out there for managing installations of interpreters such as Python, Ruby, node.js, etc. First, let's clear the air of a bunch of nonesense you may find out there that makes this problem confusing.

Never use the interpreter provided by your operating system

If you are building a web application or any other project that should by all rights be cross-platform and doesn't ship with OS, then you should have absolutely nothing to do with the interpreter that may be included with the OS. The OS interpreters are there for components of the OS itself written in those languages. They have no business being used by third party projects and applications that run on top of the OS as opposed to being part of the OS. Forget about the Ruby that comes with OS X. Forget about the Python that comes with Ubuntu. Forget about the Debian packages for node.js (slightly different/better, but still, ignore them).

The reasoning behind this guideline is as follows.

Exact version: Applications need exact and strict control of the version of their interpreter. You should be using the exact same version of your interpreter across all of your development, test, staging, and production environments. This will avoid problems which are easily-avoidable, so do it.
Modern version: OSes tend to ship versions of these interpreters that are significantly behind the latest stable version. New applications should be written to work with the latest stable version and should keep up with ongoing releases, never getting more than 3 months behind.
Independence Applications need independence from one another. If you have 3 Django projects on the same machine, each one needs to have the ability to use whatever interpreter version it needs on its own independent schedule. Due to this fact, that means the correct location for these interpretters is within your application's directory alongside your application code, which is why I advise you to ignore the node.js debian packages you may find out there because it installs into a shared location, which does not meet our goals here.

Keep the app-specific interpreter within the application install directory

Again, don't let the OS's notion of shared interpreters in a shared location distract you from the right layout here. The app-specific interpreter installation belongs inside you project's installation directory.

project_root/python
project_root/node
project_root/ruby

Basically, the old school unix principles have gone stale on us. Years ago, sysadmins had rules for filesystem layout with different goals. For example, sysadmins wanted to be able to NFS mount a shared directory where binaries could live, be maintained in a single place, and be mounted and run by many additional servers. They wanted to do this to be efficient with disk space and to be able to make manual changes in one place and have them to affect immediately on an arbitrary number of servers that use the same NFS volume.

Now we use automated tools to manage deployments to clusters of independent servers, and disk space is cheap, so we want each server to have its own copy of what it needs to run with as few external dependencies as possible. We want to be able to do rolling deploys across a cluster or run 1/2 the cluster on the new code and half on the old code. Disk space is cheap and plentiful, so if we have 5 or 10 apps running on the same staging server, we could not care less about a few megabytes of duplication to handle a bunch of python installations.

Installing local versions of node.js

Now that Joyent is shipping pre-compiled binaries, installing node interpreters in the manner described here is a snap. Here's an install_node.sh script. Pass it the version you want and where you want it installed (works with node >= 0.8).

#!/bin/sh
VERSION=${1-0.8.8}
PREFIX=${2-node}
PLATFORM=$(uname | tr A-Z a-z)
ARCH=x64
case $(uname -p) in
    i686)
        ARCH=x86
    ;;
esac
if [ -e "${PREFIX}" ]; then
    TS=$(date +%Y%m%d-%H%M%S)
    echo "WARNING: Moving existing file at ${PREFIX} to ${PREFIX}-previous-${TS}" 1>&2
    mv "${PREFIX}" "${PREFIX}-previous-${TS}"
fi
mkdir -p "${PREFIX}"
curl --silent \
  "http://nodejs.org/dist/v${VERSION}/node-v${VERSION}-${PLATFORM}-${ARCH}.tar.gz" \
  | tar xzf - --strip-components=1 -C "${PREFIX}"

Never use npm -g

This is a follow up to my earlier blog post about avoiding npm -g, now improved and revised. For the most part, I believe npm to be the state-of-the-art package management system and to be superior to the messes available for python and ruby. However, the -g switch, which installs commands globally, should be avoided in favor of the system described here. You don't want to have to upgrade all your express.js apps at once, so give them each their own copy of the express script.

Provide a single script to launch your application commands

Encapsulate each version with a wrapper shell script that understands the project directory layout and manages your PATH appropriately. I tend to call this file project_root/do but project_root/bin/tasks.sh or similar are good locations for this. This script should handle your service operations like start, stop, reload, etc, as well as any one-off commands you make have like clearing a cache, regenering static files, and so forth.

Here's a snippet of my project_root/do script which locates the correct installation of python and fabric and passes control to them.

#!/bin/sh -e
cd $(dirname "${0}")
exec ./python/bin/fab "${@}"

Thus I can run this script from any directory, or from an init/upstart script, with any PATH, and the application correctly handles its own required settings. The above is the bare bones and the crux of the separation of concerns in the design. I normally have some other code in there to bootstrap the project's dependencies, but I'll save that topic for another blog post.

For local development, manage your PATH intelligently and automatically

As you work on many projects which contain their own interpreter installations, you don't want to always have to A) work from the project root directory and B) run commands like ./python/bin/python myapp.py. So here are some utilities that can intelligently manage your PATH similar to what is done by rbenv, but not tied to ruby and based on you changing project directories.

First, here's how I set up my PATH in my ~/.zshrc file (works equally well for bash or bourne shell). I've added extra explanatory comments inline.

#This helper function will add a directory to the PATH if it exists
#This is a simple way to handle different machines, OSes, and configurations
addPath() {
    if [ -d "${1}" ]; then
        if [ -z "${PATH}" ]; then
            export PATH="${1}"
        else
          export PATH=$PATH:"${1}"
        fi
    fi
}

setupPath() {
    #Start with an empty PATH
    PATH=
    #Local pwd stuff
    addPath "${PWD}/script"
    addPath "${PWD}/bin"
    #For node
    addPath "${PWD}/node_modules/.bin"
    addPath "${PWD}/node/bin"
    #For python virtualenvs
    addPath "${PWD}/python/bin"

    #Personal home dir stuff
    addPath ~/bin
    #For rbenv
    addPath ~/.rbenv/bin
    addPath ~/.cabal/bin
    #Homebrew
    addPath ~/Library/Python/2.7/bin
    addPath /usr/local/share/python
    addPath /usr/local/bin
    #XCode/Developer
    addPath /Developer/usr/bin
    #Normal system stuff
    addPath /bin
    addPath /usr/bin
    addPath /sbin
    addPath /usr/sbin
    addPath /usr/X11/bin
}
#Run this during shell startup. Can be re-run as needed manually as well
setupPath

OK, so that's how the PATH gets built up, but we want to change the PATH as we move our current working directory between projects. For that we use a shell hook function. What this does is try to detect if we've changed into a project directory, and if so, rebuild the PATH, which will put our project-specific directories early in the PATH list, so when we type node or python or coffee, etc, we get the project specific one under the project root. Because this adds absolute paths and only changes the PATH when we cd to a project root, we can cd to subdirectories within the project and still be running the correct project-specific interpreter. This does breakdown, however, if you cd directly into a project subdirectory without stopping in the project root. I don't hit that problem because I'm not in the habit of doing that, but YMMV. Here's the zsh version, which uses the chpwd hook function.

if [ -n "${ZSH_VERSION}" ]; then
    chpwd() {
        [ -d .git -o \
          -d  node_modules/.bin -o \
          -d python/bin -o \
          -d node/bin ] && setupPath
    }
fi

Bash users, you're on your own.

Here's an example of this at work.

~-> cd projects/peterlyons.com
~/projects/peterlyons.com-> which node
/Users/plyons/projects/peterlyons.com/node/bin/node
~/projects/peterlyons.com-> cd ../craft
~/projects/craft-> which node
/Users/plyons/projects/craft/node/bin/node
~/projects/craft-> cd ../othenticate.com
~/projects/othenticate.com-> which node
/Users/plyons/projects/othenticate.com/node/bin/node
~/projects/othenticate.com-> cd ../m-cm/indivo_provision
~/projects/m-cm/indivo_provision-> which python
/Users/plyons/projects/m-cm/indivo_provision/python/bin/python
~/projects/m-cm/indivo_provision-> cd ./conf
~/projects/m-cm/indivo_provision/conf-> which python
/Users/plyons/projects/m-cm/indivo_provision/python/bin/python

(Bonus item) Keep variable files for data and logging under your project directory

project_root/var/log
project_root/var/data

This is a mindset shift from traditional unix administration best practices. It's in my opinion a less complex and more application-centric design that makes better sense given our focus on applications that tend to be providing network services and generally are less tightly coupled to the underlying OS these days. Traditional unix administration (as documented in the Filesystem Heirarchy Standard) has a strong and system-wide distinction that runtime variable data like data files and log files go under /var and everything else except for /home and /tmp is static data. Again, this no longer applies to modern applications. These rules had to do with preventing key filesystems from filling up, primarily. They wanted application data to be static and allocate a certain amount of space that had separate filesystem limits from the variable data, which they wanted organized centrally under /var so they could manage log file growth and space disk space centrally. There were reasons for these designs at the time that made sense given the constraints and goals, but times have changed.

CoffeeScript and Progress

2012-03-05T04:36:35.083Z

If you like CoffeeScript, you should write applications in CoffeeScript. If you prefer JavaScript, you should write applications in JavaScript. In this post I want to address the common attitude I have found among JavaScript programmers that CoffeeScript is a priori somehow wrong or inferior or a bad choice. I'll briefly recount some of the technical reasons why I like and use CoffeeScript at the end of this post, but the focus is a rejection of the attitude above on philosophical grounds. To be clear, the position I'm taking issue with is the position that JavaScript is the one and only language we should be writing in for the browser and node.js.

Philosophical Rejection of Monoglotism

Availability of a multitude of highly varied programming languages is good
- Check out Dart, Haxe, CoffeeScript, GWT, or Caterwaul for examples
The goal of a single programming language for the browser is a detrimental non-goal
Openness and freedom are important, as are standards and interoperability, but standards that don't evolve on an appropriate schedule are detrimental.

I brought up the topic of Google's Dart language at a local tech meetup when Dart was first announced. I asked people what their reaction was. The few who responded seemed to dislike it, not for any particular technical objection to the language or its features, but simply to its existence at all, which surprised me. This struck me as completely unexpected and strange. Why, I asked. "JavaScript is a standard" came back from one person. "Why do you like it?", he asked. I said I didn't yet know much of anything about it, but I liked the creation of new programming languages for the browser as a basic idea. I said that choice of programming tools is a good thing, and JavaScript is clearly flawed in extremely well-understood ways. Why is there this relentless stranglehold on universal and eternal backward compatibility? What is the cost/benefit analysis that goes into that?

Software in general, including programming languages and tools, must continually advance, and the faster, the better. Addressing backward compatibility by slowing forward progress is a losing solution. Ye olde for loop, while venerable, has been surpassed. Let it go. I agree with Bruce Eckel here in that once you have a stable version of a language, authors who "don't want to deal with those changes" should be able to just continue using that version while the current version of the language advances. If you wrote a JDK 1.2 application in 2000 or so and you still want to run it, just keep running it on JDK 1.2. Don't hold the entire java language back for the lifetime of this application which you are unwilling to maintain and advance along with the platform. Browsers should do the same thing with their javascript engines every 2 years or so. If you want to write a codebase once and not update it, ship it on custom hardware that does not include networking capability and call it a day. If you want to write software that works on the Internet, commit to a certain amount of ongoing maintenance. Even if we had perfect technical backward compatible support, the state of everything else: UX, UI, performance, data formats, protocols, continues to advance. Craigslist.org can get away with keeping the same UI for a decade because they make their own rules. Every other site needs to modernize.

I overheard a well-regarded server-side JavaScript programmer complain about CoffeeScript saying something along the lines of "Why would you write a module in some language I don't know?". WTF? How would it sound if I said that about any other server side programming language? It would sound ridiculous, because it is.

This notion that the web is ever going to be some all-purpose panacea built upon a single markup language plus a single stylesheet language plus a single programming language is a false goal. Let it go. Embrace a variety of tools. The vast majority of web sites are not built from direct HTML files. HTML is generated by dozens of highly divergent programming languages across many paradigms with all combinations of trade-offs. So it should be for the application in the browser. It's not a problem on the server, and it's not going to be a problem in the browser.

Rejection of FUD

CoffeeScript is clearly a well-thought-through and beautiful language. It brilliantly removes the terrible parts of javascript and replaces verbose javascript boilerplate with elegant expressiveness. For example, the function keyword being replaced with the gorgeous arrow symbol -> and the introduction of list comprehensions. It is so small and concise that there are no glaring warts to be ironed out. If you like python or ruby, you will probably like CoffeeScript better than JavaScript. If you try it for 6 months and change your mind, compile it to javascript and use that as your no codebase. No harm, no foul.

But but but...

Debugging. I hear arguments that because the line numbers in the browser stack traces don't align with the line numbers in your .coffee files, debugging is harder in CoffeeScript than JavaScript. I don't find this a problem. I just don't. My variable and function names are preserved. I anchor to a function name and my functions don't get exceedingly long. I see that in the browser the error is a few lines down in the "loadUserList" function and I go look at the "loadUserList" function in CoffeeScript and find the problem without a lot of fuss. In my experience there's just no impact here. I don't really know why people bring this up so often. It's just a non-problem for me. We use multiple preprocessing techniques including concatenating JS files together. Line numbers just ain't never gonna map back to actual files in your editor. Get over it.

In case you didn't know

Multiline strings. OMFG yes, duh.
String interpolation. So nice. So nice.
Destructuring assignment. Coming to JavaScript circa 2019...
No 8-letter function keyword. These days we're building anonymous functions every 4 lines or so. Skinny arrow FTW!
Array literals with newlines and no commas. I'm not sure what the obsession with commas is, but people, we don't need them. I don't even think we need them in one-line lists. We have the mighty space character, which is how you eye parses code already! If we can do [1, 2, 3, 4], then why can't we just do [1 2 3 4]?
Automatic IIFE wrapper. No var keyword. Default values for function parameters. Die boilerplate, die!
It's like skinny dipping, but for punctuation! Look at some Jasmine or Mocha BDD tests in JavaScript and then CoffeeScript. One looks like a garbled mess of crap with line after dedented line of }); }); });, and one looks like poetry.

Here's a sample BDD spec in CoffeeScript from a gist on github.

describe 'Math:', ->
  describe 'fib()', ->
    it 'should calculate the numbers correctly up to fib(16)', ->
      fib = [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987]
      expect(Math.fib(i)).toEqual fib[i] for i in [0..16]

  describe 'uuid()', ->
    it 'should have the proper UUID format', ->
      expect(Math.uuid()).toMatch /[A-Z0-9]{8}-[A-Z0-9]{4}-[A-Z0-9]{4}-[A-Z0-9]{4}-[A-Z0-9]{13}/

    it 'should have always the numer 4 at position 14', ->
      expect(Math.uuid()).toMatch /[A-Z0-9]{8}-[A-Z0-9]{4}-4[A-Z0-9]{3}-[A-Z0-9]{4}-[A-Z0-9]{13}/
      expect(Math.uuid()).toMatch /[A-Z0-9]{8}-[A-Z0-9]{4}-4[A-Z0-9]{3}-[A-Z0-9]{4}-[A-Z0-9]{13}/
      expect(Math.uuid()).toMatch /[A-Z0-9]{8}-[A-Z0-9]{4}-4[A-Z0-9]{3}-[A-Z0-9]{4}-[A-Z0-9]{13}/

    it 'should generate a unique uuid for 1000 generated uuids at least', ->
      uuids = []
      counter = 0

      while counter < 1000
        uuids.push Math.uuid()
        counter++

      expect(uuids.length).toEqual _.uniq(uuids).length

describe("NotesView", function() {
  beforeEach(function() {
    var notes = new NotesCollection();
    spyOn(notes, "fetch");

    var view = new NotesView({collection: notes});
  });

  describe("#initialize", function() {
    it("should fetch the notes", function() {
      expect(notes.fetch).toHaveBeenCalled();
    });

    describe("on notes fetch success", function() {
      beforeEach(function() {
        var request = mostRecentAjaxRequest();
        request.response({
          status: 200,
          responseText: JSON.stringify([
            {body: "Blog post #1", id: "1"}
          ])
        });
      });

      it("should render the view", function() {
        expect($(view.el).find(".post")).toHaveText("Blog post #1");
      });
    });
  });
});

Ouch, my eyes! All those closing punctuation groups!

Reduction of boilerplate and increasing expressiveness are what I want from a language. CoffeeScript delivers strongly in these areas. It compiles to JavaScript and works right now in all browsers and in node.js.

Conclusion

It's a polyglot world. Every app is going to use more than one programming language. Work toward making the programming languages we use better instead of trying to shame us into continuing to use languages as better alternatives are created. Shame on you. Move forward.