How to install Python pandas Development Version on Mac OS X

The pandas data analysis module is quickly becoming the go-to tool for data analysis in Python. Certain features, such as in memory joins and sorts, become extremely powerful when dealing with in-memory datasets. Often times, operations that take hours in Excel to execute take only seconds using pandas.

As the recent re-covert to Mac OS X, I wanted to get setup with the development version of pandas on my new machine running Mac OS X 10.8.

To begin, we need to have a few things installed, particularly pip and homebrew.

If you have not yet installed pip, and have a valid Python installation on your machine, simply run sudo easy_install pip in your terminal.

Once that’s done, we need to install a few libraries before trying to install our Python libraries.

This will bring in all the compilers and libraries that we’re going to need to build our stuff later on.

Assuming that you want the following libraries installed at the global Python install level, rather than a virtual environment, you can install the requirements to build pandas in a single line.

With that, you should be able to clone the latest pandas repository and install the latest development version.

That’s pretty much it, if you have any problems, feel free to leave a comment.

Twitter as a Cure for Perfectionism

Perfectionism can cripple productivity in that it will stop you from even getting started. Why do something now if it will not be perfect? Why should I not shave this yak? This problem is particularly insidious in programming and design,  in that you can argue that there may very well be a “perfect” way of doing it. The global optimum of some function, or the minimum amount of ink to convey an idea. You can essentially work forever and never achieve perfection. There is a cure, in an unlikely place: Twitter.

I was (and to some degree still am) guilty of perfectionism, but I found the most therapeutic device in combating this was having a space to spout off half-cocked ideas into the ether and watch them linger. I noticed that no matter how much or how little time I spent crafting these short messages, it was always kind of short of perfect and it did not matter.

If you find yourself bored one day, log into a streaming Twitter client (e.g. TweetDeck) and add a trending hash tag column. What you’ll see immediately is that almost no tweet is significant. You could craft the most beautiful, intelligent short poem and post it to an unknown quantity of people. Could no one see it? Perhaps, though unlikely. Could someone see it and move on to the thousands of other messages they’re trying to consume? Absolutely.

This exercise reinforces that more important than holding something to perfection, you should let it go. Someone may call you mediocre, but that is just as insignificant. If they someone sees it and doesn’t think it’s great, it’s unlikely they’ll do anything but move on (unbeknownst to you). A kind of social nihilism.

This echos with almost all social networks where there really is no way to be negative using the software (e.g. favorite, like, heart), Hater App excluded.

While this technique may not work in quite the same way if you have over 100k followers, it addresses the real problem at hand: worrying that you’re not as great as you want to be and people will find out. Tweet more, ship more, write more, design more, who cares?

If you have failed to grok vim, ditch the arrow keys

As much as I am embarrassed to admit, I have been using vi/vim for the last 10 years or so (admittedly only on remote servers for editing configs for 8 of those years) and I am just now getting around to getting better at it.

In this, one recent post on Hacker News reiterated something that was the single most important step to take to improve your vim fluidity: use hjkl instead of the arrows keys.

Folks may argue that the transaction cost of getting to the arrow keys is insignificant, but it ignores that when you use the arrow keys, you essentially are moving your hand away from every important vim command. That minor increase in transaction cost by moving over to the arrow keys is multiplicative with the transaction cost of using the standard vim commands. In other words, you are far less likely to learn other vim commands when you have the friction of moving your hand back in addition to the friction of learning the command itself.

Once you have relinquished your hands to being ready to learn, you can start adding on top of that with new commands as you move around.

Keep in mind whilst using vim: it is more-or-less feature complete. If there is something that you feel you are not doing as fast as possible–you are doing it wrong. So whenever you feel the friction you felt when you used the arrow keys, look around and find that command that lubricates that action.

Ultimately, the reason that vim zealots are so frothy about their editor is that once you have the muscle memory to be completely fluid in the editing of text, you can focus on the actual task at hand (and not the tool being used). Every time you have to do that minor transaction between your hand and move it to the mouse, or feel encumbered by your editor, you are an artist concerned with the sanding of your brush handle or the easel holding your canvas. These are things incidental to the end result: the product. The focus. The very point of using a tool: to produce.