Automatically Compress/Sync Your S3 Hosted Static Website

For those of you that use Amazon Web Services (AWS) Simple Storage Service (S3), you might find yourself wondering how to serve gzip compressed files for better performance. While the instructions available online work well, manually compressing, renaming and editing metadata of all your hosted files can be super tedious; let’s see what we can do about that.

In order to setup, we’re going to need to have installed and configured s3cmd. For OS X users, this is available via Homebrew with simply  brew update && brew install s3cmd && s3cmd --configure .

Once that’s setup, we’re pretty much ready to write the script.

I saved this as a script called  gzsync and put it in my scripts directory (part of my $PATH , chmod a+x ~/.bin/gzsync ).

Now, say we’re in our static website directory that we want to upload to S3.

This would be a clever git post-receive hook to automatically deploy our site, but we’ll save that for next time. For CloudFront users, this would be a good time to invalidate your distribution. We can actually script this, and if you would like to contribute this, that would be great!

If you would like to make any modifications to the script and recontribute them, I have put this up as a GitHub gist.

Configure a USB foot pedal (or remap any key) on Linux

I wanted a USB foot pedal solely for the purposes of push-to-talk (PTT) on Mumble, so I bought one and eventually got it working. Configuring any off-the-beaten-path device on Linux can be kind of a pain, but it does not always have to be. Hopefully this helps someone else with the same problem.

If you are really into it, you can build your own foot pedal or you can simply buy one for about $12 USD online. The pedal I bought was a just generic USB gaming foot pedal on Amazon, but there are fancier ones out there with multiple pedals as well, which we can address later on.

Essentially, the foot pedal will act as a USB keyboard with some kind of default key press. This should work the same for nearly every USB foot pedal, but the default key is not always ideal. In my case, the press was mapped to a lowercase b, which does not really help anyone. So the real fun to configuring this pedal is rebinding that key press to something else, exclusively for the foot pedal. If you are happy with your default foot pedal press, then you can stop here, otherwise let’s get to rebinding.

Note: For the entirety of this tutorial, I will be using my own unique values returned from various commands we will be using. I have done my best to note those values that you should replace with your own unique results. One other thing, we’re basically going over how to remap a key in Linux under the specific case for a USB foot pedal. You can use this tutorial to remap any key on any keyboard if you wish!

Once you have plugged in your foot pedal, we need to find out how the foot pedal is addressed by the operating system.

Looking down the list, I see a device that is probably my foot pedal,  Bus 006 Device 004: ID 0c45:7403 Microdia. We will need more information on it than that, so let’s go a little deeper.

Note the argument  -d 0c45:7403 here is my device ID that shows up in lsusb to limit the verbose output, which we widdle down to the just the parts we need.

Moving on, we need to find how the USB foot pedal is addressed as an input device so that we can remap it.

The results of  /lib/udev/findkeyboards tells us what’s plugged in and how we can address it specifically. I suspect that input/event11 is my device, so I will try that out with our next command. If you accidentily choose your primary keyboard, press ESC to get back to the command prompt.

There we can see the scan code that we detect when the foot pedal is pressed, in my case that was 0x70005. Quick note for those with a multiple pedal system, click each pedal individually and note which pedal corresponds to which scan code for later.

Alright, we’re almost there, but we’re about to get a little weird. Using your favorite editor as superuser, open up  /lib/udev/rules.d/95-keymap.rules.

At the bottom of the this file, we’re going to append a new line that is similar to the others, except with our new device’s configuration.

Very important note here, the ID_VENDOR  is set to our result from  lsusb before while the idProduct  matches the 0x7403  we got again from lsusb and it comes just before the line LABEL="keyboard_end" . This will very likely be different from your configuration, so be sure to substitute your unique values in here, as with the entirety of this tutorial.

Save and close the file. We now can remap the key press to something more palatable for PTT stuff on Mumble by creating a keymap file.

Using your favorite editor, create a new keymap file at /lib/udev/keymaps/microdia (substituting your LABEL  from before as the filename). In that new file, it is as simple as using the scan code we got before and the new key we want to map it to. In my case, I wanted to map it to the phantom F13  key, so that it never gets in the way.

Save and close the file, and we are basically done. Run the last command to get it up and running,

And we’re done! Note that you will need to reboot your machine for the change to be permanent, but otherwise you should be good.

If you have any problems, questions or suggestions, leave a comment!

Deploying Your Flask Website with Python and Fabric

By the end of this post, you will be able to deploy your local code to any number of remote servers with a single command. This assumes, of course, that you have configured your web server with uWSGI to host your Flask website and you have shell access to this server. It is encouraged that prior to proceeding you at least understand hosting a Flask site, otherwise a lot of this might not make sense, as most details relating to hosting are omitted here (particularly the idiosyncrasies of of using a virtualenv in your hosting environment).

If you do not have shell access to your web server, maybe someone else can help. Maybe you need to check out the myriad of cloud hosting services out there (e.g. Amazon Web Services, which I use to host this blog, among other sites) and make the jump. You will learn a lot more this way and be able to do much more interesting things!

So, we have a little Flask website that we wish to be able to develop locally and then later deploy to our live (perhaps production) site.

First, locally, we should install Fabric, this can be in a virtual environment or…

Well, let’s assume that we have a Flask site, example_site, we’re going to first require a setup.py.

What we’re doing above is saying that our Flask site, example_site, is a package in a directory with the same name. We define our requirements, package name, etc. here.

After we have a nice setup file, we can start to script what’s called our fabfile. We must define two actions for the fabfile, a pack and a deploy function.

pack() defines how to ball up our sources.

deploy() defines how to push it to the server and what to do once our sources are there.

I chose to configure my fabfile like below, however the Fabric documentation offers the whole suite of features that are available to you.

Once we are satisfied that our deploy scripts look good, we are ready to rock!

We can execute our pack and deploy in a single command:

That’s it! Fabric will then ball up your sources, upload them to your remote server, and execute a script that handles those sources on the remote end. Once this is configured properly, developing is bliss–no matter what solution you choose, Fabric or otherwise, easy deployment of local code is incredibly important for your web project.

It allows you to focus on code and not operations (that are repeatable and trivial).

Configuring multiple Flask sites with uWSGI and nginx on an Amazon EC2 Instance

For the lazy, we have a shell script that will do all this for you on GitHub!

While trying to configure our new Amazon EC2 instance, it was a little too cumbersome and somewhat poorly documented how to setup Flask with uWSGI on nginx. While there were a few great writeups on how to configure this, they fell short to get a working configuration on Amazon’s Linux AMI, which we will try to describe here.

After creating a new EC2 instance and logging in, we will need to install nginx, uWSGI and required libraries.

Now we need a Flask project! Let’s just grab the latest simple repo from GitHub:

Alternatively, you can fork the project on GitHub and pull that one–this is probably the desirable option.

We will quickly setup a virtualenv in our new simple repository.

Now let’s give it a test, shall we?

Looking good, let’s move it real quick to where we want it and fix the permissions to our liking.

Next, let’s configure uWSGI. Following Tyler’s writeup, let’s make some directories.

Now let’s create the uWSGI configuration file with:sudo vim /etc/init/uwsgi.confWith the following content:

A few things to note here, we are setting up the general uWSGI emperor log and telling it to run as the nginx user for permission purposes.

With uWSGI configured, we can start it up withsudo start uwsgi

Now we can begin to configure our simple fork which runs this blog by creating a new file with:

And we will configure it with the following content:

Now we need to link our configuration file to the enabled sites folder:

Finally, we can configure nginx to serve up our new blog.

And we will configure nginx to serve up content from our uWSGI emperor like so:

Save the file and let’s fire up nginx, we are ready for launch!

Hope that you have found this post helpful. Later we hope to describe how to automate this on Amazon EC2 for automatic scaling of your server fleet.