Thursday, January 3, 2013

Receipts and the Mechanical Turk

Last time I discussed how using a QR code to categorize receipts would make data entry easier. Unfortunately this solution requires the cooperation of the stores. While easily doable it is hard to get the momentum going to add such a feature. Another thought I had at the same time was using Amazon's Mechanical Turk to do a similar process. I have actually implemented a simple version of this and run a few examples through the production Turk. I think people will find this very interesting both for the specific problem and for a good, yet still fairly simple example of using MTurk (as Amazon likes to call it). See for the official Amazon site.

First, for those not familiar, MTurk is a platform that allows the creation of small tasks that are meant to be done by real people in exchange for a small fee, typically a few cents. Examples tend to include "identify the color of the automobile in a picture" or "determine the hours of operation at a particular business". These are trivially easy for a person to accomplish but not for a computer.

My idea is to take a picture of a receipt and extract information from it such as the date of purchase, the name of the business, and the category of spending. Now some of you out there are probably image processing experts and might try to do a full computerized solution to this problem, but I think most people will admit that although you can convert the image to text trying to parse out information like the date of purchase is really hard. Go ahead and take a look at a few receipts and see what you think, I'll wait...

So the idea is to submit the image to MTurk and let a human do the job in a few seconds. I found this a very interesting experiment both from the software and human perspective. It is true that MTurk costs real money but I spent less than $5.00 on my experiments and you can spend nothing to use and develop for it, it only costs money when real people start actually working for you.

To keep this from being an enormous blog post I am going to split this up over several posts and include implementation details later. I will end here with a screen shot of what the person saw when they processed my receipt using MTurk. Keep in mind we are still at an experimental stage and more information would be added in the future.


  1. The Australian powers has put aside a spending plan adding up to $4 billion to execute its Power Efficient Houses Package and among which is the Residence Insulation Plan.weblink