Possible Project: Automated Adaptation of 3D Scenes to Hand-Drawn Images

Contact: Moritz Gerl and/or Tobias Isenberg

This project deals with generating a 3D replication of a scene depicted in a given hand-drawn illustration. In a current research project, a 3D model, lighting and projection setup, generating an image that closely matches a sample illustration, is employed. It enables taking 3D measurements related to the 2D illustration. These measurements are used in an approach for capturing and reproducing the drawing style present in a hand-drawn sample image. Currently, this replication is done 'manually' in a 3D software. The aim of this project is to automate this procedure (also a semi-automatic, interactive technique would be a good solution). Given is a hand-drawn sample illustration and a polygonal model of the depicted object, the shape of which corresponds, but not entirely matches the shape of the object in the drawing (see images). The goal is to automatically or semi-automatically remodel this deviation of shape as a deformation of the model, as well as to estimate camera parameters and a lighting setup to create a 3D scene which renders an image closely matching the sample illustration.

A literature study to find suitable existing techniques will be part of the project, a foundation can be found in the work of David Stork. He used a 3D replication of a painted scene to estimate the location of the light source in the painting (http://www.diatrope.com/stork/delaTourChrist.html)

Reference: Kimo Johnson and David G. Stork, 2006. Estimating the location of illuminants in realist master paintings: Computer image analysis addresses a debate in art history of the Baroque. In International Conference on Pattern Recognition (ICPR06), Volume 1: 255–258. (DOI: 10.1109/ICPR.2006.501) (URL)