Ectyper is a Python project designed to make it trivial to create an image transformation web service. It's still in the early stages, but is being used in production here at Hulu. We're releasing it under an open source license, so you're welcome to use it (see the end of this post for details).

All you have to do to get started is extend the base Ectyper classes to define where your source images reside. The codebase is a set of Tornado request handlers that allows on-the-fly conversion of images according to a request's query string options. By default it provides a way to resize, reflect and reformat images.

Motivation

Every video, series, network, and genre on Hulu has an image to represent it. Originally these images were generated as part of our ingestion and publish workflows. This is basically when content providers provide raw assets and we perform the automated and manual steps to prepare it for user-visible products. A predefined set of image formats and dimensions were generated and stored alongside the output of our video transcoding tasks. These cut images were then pushed to our origin servers and/or our content delivery network (CDN) servers for storage and highly cached delivery.

This worked fine for hulu.com, but as we started to build new user experiences for Hulu Plus on mobile phones, tablets, gaming consoles, set-top boxes, and connected TV/Blu-Ray players, it became clear that we would need a variety of new image dimensions. While we could just have the device applications resize the existing images, we had a few motivations for creating custom sizes to fit each UI layout. First, visual quality can suffer from scaling artifacts on certain devices. Also, for performance reasons, it's preferable to fetch the smallest possible file size, especially on mobile devices on cell networks. And we observed that some low-powered connected TVs could not scale images at all without incurring drastic performance hits.

Additional Transformations

While image resizing was the initial motivation for Ectyper, we continued to find opportunities to leverage server-side image transformations. For example, the new UIs our design team dreamed up made use of effects such as image reflections. Certain devices had the power and APIs to do these client-side, but most low-powered devices didn’t, so we added additional parameters to Ectyper to perform these types of operations. Anything you can perform using the underlying image libraries could be exposed in this way.

Implementation

Ectyper provides a primary ImageHandler class (extending Tornado’s RequestHandler class). This class parses each request’s query string and converts it into a command-line call into ImageMagick’s convert. This class is wrapped by ectyper.magick.ImageMagick which can be overridden to provide other options.

A request into an ImageHandler will automatically generate an ImageMagick object based on standard options (see help(ectyper.handlers.ImageHandler) for more). Once your handler calls convert_image() with a local file path or a remote URL, ImageHandler kicks off an ImageMagick convert process with the calculated options. The result is streamed to the browser. You can optionally extend CachingImageHandler or FileCachingImageHandler to stream the output elsewhere for caching purposes.

A full list of options supported by default:

&size=NxM
   Resize the source image to N pixels wide and M pixels high.

&maintain_ratio=1
   Maintain aspect ratio when resizing (ignored if size is not
   provided).  If requested size is a different aspect ratio than
   the source image, the resize will scale to fit the width or height,
   whichever is reached first.  The other dimension will be centered
   vertically or horizontally as necessary.  Defaults to 0.

&reflection_height=N
   Flip the image upside down and apply a gradient to mimic a
   reflected image.  reflection_alpha_top and reflection_alpha_bottom
   can be used to set the gradient parameters.

&reflection_alpha_top=N
   The top value to use when generating the gradient for
   reflection_height, ignored if that parameter is not set.  Should be
   between 0 and 1. Defaults to 1.

&reflection_alpha_bottom=N
   The bottom value to use when generating the gradient for
   reflection_height, ignored if that parameter is not set.  Should be
   between 0 and 1.  Defaults to 0.

&format=(jpeg|png|png16)
   Format to convert the image into.  Defaults to jpeg.
   png16 is 24-bit png pre-dithered for 16-bit (RGB555) screens.

Examples

Here’s a simple example. Assume you have a directory of images at /path/to/images and this hulu.jpg is one of them:

Here's a trivial ImageHandler that directs the request path to the image directory:

from ectyper.handlers import ImageHandler
import os
from tornado import web, ioloop

class StreamLocal(ImageHandler):
    def handler(self, *args):
        self.convert_image(os.path.join("/path/to/images", args[0]))

app = web.Application([
    ('/images/(.*)', StreamLocal),
])

if __name__ == "__main__":
    app.listen(8888)
    ioloop.IOLoop.instance().start()

Now if you run the above to start up this ImageHandler, a resize can be performed like this: http://localhost:8888/images/hulu.jpg?size=200x96

A reformat like this: http://localhost:8888/images/hulu.jpg?format=png

And a reflection like so: http://localhost:8888/images/hulu.jpg?size=200x96&format=png&reflection_height=60

Check out examples.py in the source code for samples of how to introduce logic in the handler to achieve more interesting use cases.

Tornado

Tornado serves as the underlying web server. It’s well suited for the use case due to its asynchronous nature. As a result, both the image transformations and necessary interactions with other Hulu services are non-blocking.

ImageMagick

The initial implementation leveraged Python Image Library, which was intuitive to develop the necessary transformation operations. After a variety of tests, we found the output quality of ImageMagick to be superior, although the programmatic interface is less intuitive.

Ectyper in Production

Image processing is an inherently costly operation relative to many of the tasks our backend services perform. We’ve gone through a variety of designs to bring the Ectyper functionality we require up to production scale, and we’re still tweaking it. Caching is critically important. We leverage an on-disk cache for each of the Ectyper machines (refer to FileCachingImageHandler), as well as edge caching on our CDNs’ servers. While it’s possible to perform transformations on-the-fly, we instead favor a fail-first approach where new transformations are entered into a queue for background processing. When we know new transformations are going to be utilized in production, we have a simple warmer to backfill the new requests without impacting live user requests.

Future

Remember, Ectyper is still in the early stages. Here are a few possible next steps:

  • Switch to MagickWand/AsyncHTTPClient instead of forking convert/curl.
  • Change the ImageMagick loading method (right now you set a static variable on the ImageHandler to the class which is a little wonky).
  • Push more of the backfill and warming functionality we're using in production into the base Ectyper codebase.
  • Add more image transformations (http://www.imagemagick.org/image/examples.jpg).
  • Extract Tornado and ImageMagick functionality into pluggable modules.

Source

The Ectyper repository lives at https://github.com/hulu/ectyper, and we would encourage you to join the discussion group at http://groups.google.com/group/ectyper-dev and let us know how you’re using Ectyper. Your feedback and patches are welcome!

Daniel Bear leads software development for connected device applications.