# Connected Morphological Operators for Tensor Images (COMOTI) (2010-2014)

## Funding

The Netherlands Organization for Scientific Research (NWO).

## Project in the News

Research Media published an item about the COMOTI project in its August 2013 International Innovation report. Click here.

## Collaboration

This research project is a collaborative effort of the research groups Scientific Visualization & Computer Graphics http://www.cs.rug.nl/svcg and Intelligent Systems http://www.cs.rug.nl/is of the Johann Bernoulli Institute for Mathematics and Computer Science (JBI).

## People

**PhD student**: Jasper van de Gronde**Supervisors**: The PhD student will be supervised by prof. dr. Jos Roerdink and dr. Michael Wilkinson of the JBI.

## Project goal

In many scientific areas one increasingly makes use of images where each pixel holds a matrix or tensor value. This project addresses the development of a solid mathematical and algorithmic framework for connectivity-based morphological filtering and visualization of tensor images. The goal is to extend (hyper)connected, adaptive and multiscale morphological filters to tensor images. Tensor images contain a wealth of information which allows adaptive directional steering of morphological image operators. As the image data sets can grow to very large sizes, the development of efficient algorithms for the new filters is an important part of the project. A prime application area is medical imaging of the brain, where diffusion tensor magnetic resonance imaging enables the in-vivo exploration of nerve fiber bundles. We will collaborate with neuroscience researchers to explore the potential of the new morphological filters for brain connectivity analysis using diffusion tensor imaging (DTI).

## Project description

**Problem Definition and Scientific Goals**
In this project we address the development of a solid mathematical and
algorithmic framework for connectivity-based morphological filtering and
visualization of *tensor* fields. A prime application area is medical
imaging, where diffusion tensor magnetic resonance imaging (DT-MRI or DTI)
enables *in vivo* exploration of nerve fiber bundles in the brain,
see the following figure for an example.

(a) | (b) | (c) | (d) |

**Illustrative visualization of DTI fiber tracts. (a): Initial set. (b)-(d): Three example stages of filtering, using the fractional anisotropy (FA) value of a DTI fiber tract data set. With the growing filtering threshold for FA, more of the internal structure of the data set is revealed. Taken from Maarten H. Everts, Henk Bekker, Jos B.T.M. Roerdink, and Tobias Isenberg: Depth-Dependent Halos: Illustrative Rendering of Dense Line Data. IEEE Transactions on Visualization and Computer Graphics, 15(6):1299–1306, November/December 2009.**

Initiated in the 1960s, mathematical morphology was developed to describe image operators for enhancement, segmentation and extraction of shape information from digital images (Matheron 1975, Serra 1982). In contrast to traditional linear image processing, the morphological image operators focus on the

*geometrical*content of images and are

*nonlinear*. Their mathematical description has been extensively developed within the framework of complete lattice theory (Serra 1988, Heijmans 1994), and many efficient algorithms are available for binary and grey scale images. Also, the case of vector-valued data, such as color or hyper-spectral images, has been addressed.Our scientific focus in this project is to extend (hyper)connected, adaptive and multiscale morphological filters to tensor images, with special attention to invariance properties, such as translation, rotation or scale invariance.

*(Hyper)connected filters*. Connected filters are used to perform filtering based on various shape and size attributes. A key property of connected filters is their edge preserving nature. Connected filters rely on an axiomatic definition of connectivity within a complete lattice framework. To address some shortcomings, hyperconnectivity and hyperconnected filters were introduced by Serra (1998). In particular, hyperconnected filters can deal with overlapping objects as separate entities (Serra 1998, Braga-Neto 2003, Wilkinson 2007) and can prevent the so-called*leakage*problem of connected filters.*Adaptive morphological operators*. Adaptive operators have been introduced as a generalization of the standard morphological operators, by letting the size or shape of structuring elements (small subsets of various forms and sizes used to probe the image) adapt to image position or to image features. Another form of adaptivity is to extend translation invariance of morphological operators to more general invariance groups, where the shape of the structuring element spatially adapts in such a way that global group invariance is maintained (Maragos 2009, Roerdink 2009). Tensor images in particular contain a wealth of information which allows adaptive directional steering of morphological operators. Here one may also think of tensors which define intrinsic image properties, such as the structure tensor or the Hessian matrix.*Multiscale morphological operators*. Multiscale connected operators (Braga-Neto 2003, 2005), morphological scale spaces (Welk 2003, Jalba 2006) and morphological pyramids (Goutsias and Heijmans 2000) have been proposed. Such multiscale representations have been applied to granulometric image analysis (Breen and Jones 1996, Urbach 2007), image filtering or compression, or interactive visualization of large volume data sets (Roerdink 2005). Connected filters in particular allow efficient computation and representation of scale spaces.

L=0 | L=10000 | L=60000 | L=90000 |

** Connectivity-based morphological filtering of a volume data set. As the threshold L of the volume attribute increases, more and more objects disappear. Taken from Michel A. Westenberg, Jos B. T. M. Roerdink, and Michael H. F. Wilkinson: Volumetric attribute filtering and interactive visualization using the Max-Tree representation. IEEE Transactions on Image Processing, 16(2):2943–2952, 2007. **

(a) | (b) | (c) | (d) |

** Processing a historical document. (a) Original image showing much detail in the background. Background removed by (b) anisotropic diffusion; (c) connected attribute filter; (d) hyperconnected attribute filter. Taken from Ouzounis, G. K., and Wilkinson, M. H. F. Hyperconnected attribute filters based on k-flat zones. IEEE Trans. Pattern Anal. Mach. Intell. (2010). In press.**

## Work packages

**WP-1) Theoretical foundation**. In this package, we will extend the existing axiomatic work on hyperconnectivity to tensor-valued data, following the work of Burgeth et al. (2007, 2009) on mathematical morphology for matrix fields.**WP-2) Algorithms**. This package concerns the development of efficient algorithms for tensor-valued data.**WP-3) Multiscale analysis**. Multiscale analysis based on connectivity pyramids will be extended to hyperconnectivity and then generalized to tensor-data.**WP-4) Validation**. To validate the new methods we will need a suite of applications to test them. As main application area we will explore brain connectivity analysis and visualization of DTI or HARDI data, in collaboration with the BCN-Neuroimaging Center.**WP-5) Code and demonstrator development**. During the entire project, a great deal of code will be developed, both as stand-alone demonstration programs, and as libraries of image analysis and visualization methods.

## References

- Braga-Neto, U., and Goutsias, J. A multiscale approach to connectivity. Comp. Vis. Image Understand. 89 (2003), 70–107.
- Braga-Neto, U., and Goutsias, J. A theoretical tour of connectivity in image processing and analysis. J. Math. Imag. Vis. 19 (2003), 5–31.
- Braga-Neto, U., and Goutsias, J. Object-based image analysis using multiscale connectivity. IEEE Trans. Pattern Anal. Mach. Intell. 27, 6 (2005), 892–907.
- Breen, E. J., and Jones, R. Attribute openings, thinnings and granulometries. Computer Vision and Image Understanding 64, 3 (1996), 377–389.
- Burgeth, B., Bruhn, A., Didas, S., Weickert, J., and Welk, M. Morphology for matrix data: Ordering versus PDE-based approach. Image Vision Comput. 25, 4 (2007), 496–511.
- Burgeth, B., Bruhn, A., Papenberg, N., Welk, M., and Weickert, J. Mathematical morphology for matrix fields induced by the Loewner ordering in higher dimensions. Signal Processing 87, 2 (2007), 277–290.
- Burgeth, B., Breuss, M., Didas, S., andWeickert, J. PDE-based morphology for matrix fields: Numerical solution schemes. In Tensors in Image Processing and Computer Vision, S. Aja-Fernandez, R. de Luis-Garcia, D. Tao, and X. Li, Eds. Springer, London, 2009, pp. 125–150.
- Goutsias, J., and Heijmans, H. J. A. M. Multiresolution signal decomposition schemes. Part 1: Linear and morphological pyramids. IEEE Trans. Image Processing 9, 11 (2000), 1862–1876.
- Heijmans, H. J. A. M. Morphological Image Operators, vol. 25 of Advances in Electronics and Electron Physics, Supplement. Academic Press, New York, 1994.
- Jalba, A. C., Wilkinson, M. H., and Roerdink, J. B. T. M. Shape representation and recognition through morphological curvature scale spaces. IEEE Trans. Image Processing 15, 2 (Feb. 2006), 331–341.
- Maragos, P., and Vachier, C. Overview of adaptive morphology: Trends and perspectives. In Proc. ICIP’2009 (Int. Conf. Image Processing), Cairo, Egypt, Nov. 7-10 (2009).
- Matheron, G. Random Sets and Integral Geometry. John Wiley & Sons, New York, NY, 1975.
- Roerdink, J. B. T. M. Morphological pyramids in multiresolution MIP rendering of large volume data: Survey and new results. Journal of Mathematical Imaging and Vision 22, 2/3 (2005), 143–157.
- Roerdink, J. B. T. M. Adaptivity and group invariance in mathematical morphology. In Proc. ICIP’2009 (Int. Conf. Image Processing), Cairo, Egypt, Nov. 7-10 (2009), pp. 2253–2256.
- Serra, J. Image Analysis and Mathematical Morphology. Academic Press, New York, 1982.
- Serra, J., Ed. Image Analysis and Mathematical Morphology. II: Theoretical Advances. Academic Press, New York, 1988.
- Serra, J. Connectivity on complete lattices. J. Math. Imag. Vis. 9, 3 (1998), 231–251.
- Urbach, E. R., Roerdink, J. B. T. M., and Wilkinson, M. H. F. Connected shape-size pattern spectra for rotation and scale-invariant classification of gray-scale images. IEEE Trans. Pattern Anal. Machine Intell. 29, 2 (2007), 272–285.
- Welk, M. Families of generalised morphological scale spaces. In Scale Space Methods in Computer Vision (2003), L. D. Griffin and M. Lillholm, Eds., vol. 2695 of Lecture Notes in Computer Science, Springer, Berlin, pp. 770–784.
- Wilkinson, M. H. F. Attribute-space connectivity and connected filters. Image Vis. Comput. 25 (2007), 426–435.
- Wilkinson, M. H. F. An axiomatic approach to hyperconnectivity. In Proc. ISMM 2009 (2009), M. H. F. Wilkinson and J. B. T. M. Roerdink, Eds., vol. 5720 of LNCS, pp. 35–46.
- Wilkinson, M. H. F. Hyperconnectivity, attribute-space connectivity and path-openings: Theoretical relationships. In Proc. ISMM 2009 (2009), M. H. F. Wilkinson and J. B. T. M. Roerdink, Eds., vol. 5720 of LNCS, pp. 47–58.