The Descriptive Camera by Matthew Richardson at NYU’s Interactive Technology Program is another camera to add to the growing list of networked enabled cameras I’ve been posting about (see the Lens-less Camera and Buttons). Using it is the same as other cameras, simply point and click, however the output produced is very different. Instead of producing a photographic representation of the space in front of the lens, the camera produces (via a mechanical turk) a description of the scene printed to paper. The rationale for the work is as follows:
Modern digital cameras capture gobs of parsable metadata about photos such as the camera’s settings, the location of the photo, the date, and time, but they don’t output any information about the content of the photo. The Descriptive Camera only outputs the metadata about the content.
As we amass an incredible amount of photos, it becomes increasingly difficult to manage our collections. Imagine if descriptive metadata about each photo could be appended to the image on the fly—information about who is in each photo, what they’re doing, and their environment could become incredibly useful in being able to search, filter, and cross-reference our photo collections. Of course, we don’t yet have the technology that makes this a practical proposition, but the Descriptive Camera explores these possibilities
The camera utilises some similar technologies to the cameras posted about previously however particular to this is the human element, an amusing use of Amazon’s Mechanical Turk service:
The technology at the core of the Descriptive Camera is Amazon’s Mechanical Turk API. It allows a developer to submit Human Intelligence Tasks (HITs) for workers on the internet to complete. The developer sets the guidelines for each task and designs the interface for the worker to submit their results. The developer also sets the price they’re willing to pay for the successful completion of each task. An approval and reputation system ensures that workers are incented to deliver acceptable results.
An example image converted to text description output is shown below.
Originally seen on Today and Tomorrow.