Jelenlegi hely

Content Based Image Retrieval Using Stochastic Paintbrush Image Transformation

Lifetime from: 
Lifetime to: 
Short description: 
Stochastic paintbrush rendering is an algorithm to simulate the painting process to get a picture similar to a real painting. The image can be described by the parameters of the consecutive paintbrush strokes, resulting in a parameter-series. In this project, we will explore this stroke-representation of images for Content Based Image Retrieval (CBIR).

Images can be interpreted in several ways by decomposition into basic functions: strokes, fractals, wavelets, etc...Each of these is natural in some sense: strokes are good representations of letters or shapes, fractals originate from the self-similarity, wavelets are useful for multi-scale representation. When observing an image, our visual system is searching for small image-fragments to describe the scene: contours described by lines and curves, patches, simple textures, strokes. How can we interpret an image? In the biological sense, it is not an easy question. Brain and eye-researchers are looking for the right answer. However, there is an offering answer for this question from an artistic point of view: ask a talented painter and he/she will give a painted interpretation of the world: the scene as the artist sees it. Such an image is made of brush-strokes of different sizes and colors put on the screen one after the other in a sequence by the painter. Small articles are elaborated with fine brushes, while plain surfaces are painted with greater strokes.

Sample paintings produced by increasing brush sizes

Stochastic Paintbrush Transformation (SPT) is a new method to simulate such a painting process. It is based on a random search to insert brush-strokes into a generated image at decreasing scale of brush-sizes, without predefined models or interaction. The main goal of the method is to transform the image into a representation that is very similar to the human sensation of classical artistic images. When defining what we hope from a new process that follows the main features of representational painting, we can describe the main concepts of the algorithm:

  • It should have sharp edges at any level of image-construction;
  • There are no fine details below a limit;
  • There are sharp details at the finest level as well;
  • From a given distance the image should give the same visual scenery as the original.

The overall goal of this project is to investigate how SPT provides a description of an image via parameter-series of brush-strokes. This information can then be used to capture the (low-level) content of the original image. We extract various statistics of paintbrush strokes and define a similarity measure based on them. Such measures can be used in an image indexing and retrieval system. Current brush parameters include color, size, and orientation. Therefore not only the classical color or texture content can be measured but it also provides some information about the structure of the image: fine/coarse details, typical orientations, etc...Spatial distribution of brush-parameters provide further information about higher level content: presence of objects, composition.


The method has been tested on a large collection of images with human selected ground truth, which is available for download. Below, we show a comparative retrieval result obtained by the proposed algorithm, histogram intersection, and Oracle 9i’s CBIR function along with ground truth. Further results are available here.

Ji X, Kato Z, Huang Z. Non-Photorealistic Rendering and Content-Based Image Retrieval. In: Rokne J, Klein R, Wang W, editors. Proceedings of 11th Pacific Conference on Computer Graphics and Applications (PG). New York: IEEE Computer Soc. Pr.; 2003. 1. p. 153-162p.
Kato Z, Ji X, Sziranyi T, Tóth Z, Czúni L. Content-based image retrieval using stochastic paintbrush transformation. In: , editor. IEEE - International Conference on Image Processing: ICIP. Aix-en-Provence: IEEE Computer Society Press; 2002. 9. p. 944-947p.
Image Manipulation