21. Improved interactive medical image analysis through haptic display
methods
Erik Vidholm, Filip Malmberg, Ingela Nyström, Ewert Bengtsson, Stefan Seipel
Partners: Lennart Thurfjell, GE Healthcare, Uppsala/London, UK; Gunnar Jansson, Dept. of Psychology, UU; Hans Frimmel, Dept. of Oncology, Radiology, and Clinical Immunology, UU Hospital
Funding: Swedish Research Council; TN-faculty, UU
Period: 0301-
Abstract: Modern medical imaging techniques provide 3D images of
increasing complexity. Better ways of exploring these images for diagnostic
and treatment planning purposes are needed. Combined stereoscopic and haptic
display of the images form a powerful platform for such image analysis.
In order to work with specific patient cases, it is necessary to be able
to work directly with the medical image volume and to generate the relevant
3D structures directly as they are needed for the visualization. Most work
so far on haptic display has used predefined object surface models.
In this project, we are creating the tools necessary
for effective interactive exploration of complex medical image volumes
for diagnostic or treatment planning purposes through combined use of
haptic and 3D stereoscopic display techniques. The developed
methods are tested on real medical application data.
Our current applications are interactive liver segmentation from CT images,
see Project 22,
hardware assisted visualization of breast MR images, see
Project 23, and interactive segmentation of back muscles in MR images, see Project 24. In additon to this, we are
working on haptic interaction with 3D deformable surface meshes and
3D live-wire from a more theoretical point of view.
A software package for interactive visualization and segmentation developed within this project has been released under an open-source license. The package is available for download at http://www.cb.uu.se/research/haptics.
Erik Vidholm defended his doctoral thesis in February 2008, see Section 4.3. PhD student Filip Malmberg continues the project.
22. Interactive organ segmentation from abdominal CT images
Erik Vidholm, Filip Malmberg, Ingela Nyström, Ewert Bengtsson
Partners: Sven Nilsson, Milan Golubovic, Dept. of Oncology, Radiology, and Clinical Immunology, UU Hospital
Funding: Swedish Research Council; TN-faculty, UU
Period: 0501-
Abstract: The manual step in semi-automatic segmentation of medical volume images
typically involves initialization procedures, such as placement of
seed-points or positioning of surface models inside the object to be
segmented. The initialization is then used as input to an automatic
segmentation algorithm. We investigate how such initialization tasks can
be facilitated by using haptic feedback.
In this project, we develop interactive methods for segmenting the organs
from abdominal CT scans. For example, liver segmentation is of importance in hepatic surgery planning, where it is a first step in the process of finding vessels and tumours, and the classification of liver segments. Liver segmentation may also be useful for monitoring patients with liver metastases, where disease progress is correlated to enlargement of the liver.
We have developed a fully 3D liver segmentation method where high accuracy and precision is efficiently obtained via haptic interaction in a 3D user interface. Our method makes it possible to avoid time-consuming manual delineation, which otherwise is a common option prior to, e.g., hepatic surgery planning. This work was presented at SPIE Medical Imaging 2008: Visualization, Image-Guided Procedures, and Modeling.
Currently, we are incorporating and adapting region-based segmentation methods, such as the image foresting transform (IFT), into our software. Fig. 11 illustrates interactive organ segmentation with IFT.
|
23. Analysis of dynamic breast MRI
Ewert Bengtsson, Erik Vidholm, Ingela Nyström
Partners: Stuart Crozier, Andrew Mehnert, ITEE Dept., University of Queensland, Brisbane, Australia
Funding: TN-faculty, UU; The Australian Research Council
Period: 0503-
Abstract: The pattern of change of signal intensity over time in contrast enhanced magnetic resonance (MR) images of the breast is a useful indicator of malignancy. The methods used for assesing and visualizing this in current clinical practice are rather tedious; it is difficult to visualise and evaluate 4D (3D volumes over time) data effectively. During his sabbatical at the University of Queensland in 2004-2005, Bengtsson joined a new project in this area which developed into a collaborative project which is continuing. A basic concept for the visualisation is to convert the time course for each voxel into a colour coded representation where intensity and saturation represents the uptake of contrast and hue represents the washout which is different for normal and malignant tissue. We thus obtain a colour coded volume image. During 2006, a program where this image is visualized with hardware accelerated maximum intensity projection (MIP) in the hue-saturation-value (HSV) colour space was developed. This project is part of Vidholm's doctoral thesis, defended in February 2008, see Section 4.3. Currently the resulting visualizations are beeing evaluated on a larger material collected in Brisbane.
24. Interactive segmentation of back muscles in MR images
Filip Malmberg, Ewert Bengtsson, Ingela Nyström
Partners: Andrew Mehnert and Craig Engstrom, ITEE Dept., University of Queensland, Brisbane, Australia
Funding: TN-faculty, UU; The Australian Research Council
Period: 0803-
Abstract: In cricket fast bowlers, substantial volume asymmetries of the quadratus lumborum (QL) muscle have been associated with a significantly increased risk of developing lumbar spine stress fractures. In this project, we aim to measure such asymmetries by segmenting the QL muscle in MR images. At the university of Queensland, an automatic segmentation method has been developed for this purpose. The method uses a shape atlas, built from a large number of reference segmentations, to segment a specific muscle.
Currently, we are investigating if interactive segmentation methods can be used to accelerate the creation of reference segmentations. By accelerating this process, we make it easier to adapt the atlas based automatic segmentation method to other interesting muscles.
25. Efficient algorithms for computer graphics
Ewert Bengtsson, Anders Hast
Partner: Tony Barrera, Uppsala
Funding: TN-faculty, UU
Period: 9911-
Abstract: Computer graphics is increasingly being used to create realistic images of 3D objects for applications in entertainment, (animated films, games), commerce (showing 3D images of products on the web), industrial design and medicine. For the images to look realistic high quality shading and surface texture and topology rendering is necessary. A proper understanding of the mathematics behind the algorithms can make a big difference in rendering quality as well as speed. We are in this project re-examining some of the established algorithms and are finding new mathematical ways of simplifying the expressions and increasing the implementation speeds without sacrificing image quality. We have also invented a number of completely new algorithms. The project is carried out in close collaboration with Tony Barrera, an autodidact mathematician. It has been running since 1999 and resulted in more than 25 international publications and a PhD thesis. During 2008 the work resulted in two book chapters one on trigonometric splines the other on an alternative model for shading of diffuse light for rough materials. Another paper was accepted for a conference in early 2009.
26. Whole hand haptics with true 3D displays
Ingrid Carlbom; Ewert Bengtsson, Ingela Nyström, Stefan Seipel
Partners: Jonny Gustafsson, Industrial Metrology and Optics Group, KTH; Gunnar Jansson, Dept. of Psychology, UU; Roland Johansson, Dept. of Neurophysiology, Umeå University; Stefan Johansson, PiezoMotor AB; Håkan Lanshammar, Dept. of Information Technology, UU; Lars Mattsson, Industrial Metrology and Optics Group, KTH; Demetri Terzopoulos, Dept. of Computer Science, UCLA; Northern Light Studios; SenseGraphics AB
Period: 0810-
Abstract: We propose to build a system using true volumetric haptics for interfacing with a true 3D display of virtual models. The result will provide an unprecedented experience allowing the user to touch and manipulate high contrast, high resolution, three-dimensional (3D) virtual objects suspended in space using a glove that gives such realistic whole hand haptic feedback that the interaction closely resembles interaction with a real objects using a bare hand. The system will allow multiple users simultaneous access to the same virtual object in a fully lit room, providing a natural environment for collaboration on complex tasks. This project is highly multidisciplinary, and we have assembled a team of researchers from computer science, engineering, material science, neurophysiology, and psychology.
Such a system has numerous applications. Most interaction would benefit from using your own, gloved hand reaching around and inside a 3D virtual model or even using a real tool to interact with the virtual object, rather than using a traditional interaction tool such as a mouse or a stylus to manipulate a virtual hand or a virtual tool that in turn interacts with a flat display of an object. One application area may be computer-aided design where the designer not only wants to see a true rendition of the proposed design but also wants to touch it thereby using two of his or her senses to verify its correctness. Another application may be a training system for assembly of complex objects, where the trainee can both touch and see the object under construction. Yet another application may be a surgery or medical diagnosis training tool. The user would be able to touch a virtual patient and perform surgery using his or her own hands holding real surgery tools while simultaneously getting haptic feedback in all fingertips and the whole hand. Finally, this project would give game manufacturers a novel interaction tool for 3D games. See Fig. 12.
During the fall of 2008, we have planned this project, assembled the research team, and sought funding from major research funding agencies.
27. ProViz -- Interactive visualization of in situ 3D protein images
Lennart Svensson, Stina Svensson, Ingela Nyström, Ewert Bengtsson
Partners: Dept. of Cell and Molecular Biology, Karolinska Institute; Sidec AB; SenseGraphics AB
Funding: The Visualization Program by Knowledge Foundation; Vaardal Foundation; Foundation for Strategic Research; VINNOVA; Invest in Sweden Agency
Period: 0807-
Abstract: The traditional methods for solving the structure of proteins are X-Ray crystallography and NMR spectroscopy. An alternative approach, cryo-electron tomography (cryo-ET), has more recently gained interest within the field of structural biology as it enables studies of individual structures in tissue at nanometer scale, whereas the old methods only allowed studies in solution. Another important fact is that Cryo-ET enables studies of the dynamics of proteins. As Cryo-ET results in images of low resolution (lower than e.g., X-ray crystallography) and low signal-to-noise ratio a new kind of software for postprocessing is required, which allows for proper visualization and analysis, as well as data fusion with measurements from X-Ray crystallography and NMR. The target of the ProViz project is to create these software tools. Moreover to make use of stereo visualization and haptic rendering of the image in order to facilitate a good understanding of the data as well as interactivity in semi-automatic methods.
As the ProViz project was initiated during the summer of 2008, no new methods have been developed yet. In Fig. 13, we show an example on how we depict an interactive data fusion method. First, an initial position for the high resolution data is found automatically. Thereafter the position is refined to better fit the Cryo-ET data. The refinement is guided by the user who is interacting with the simplex mesh using a haptic probe.
28. PAP-cell detection, analysis and classification
Ewert Bengtsson, Patrik Malm, Hyun-Ju Choi, Carolina Wählby, Bo Nordin
Funding: Vinnova
Period: 0801-
Abstract: Cervical cancer is killing a quarter million women every year. Screening based on so called PAP-smears have proven very effective to reduce cancer mortality but require much work of well trained cytotechnologist. For 50 years research to automate the screening has been in progress, Bengtsson was very active in this field 1973-1993. Since about 10 years, commercial automated systems have been in operation but unfortunately those systems have many limitations.
In India there is no effective screening program in operation and around 70,000 women die from the disease each year. Now an effort to develop a screening system adapted to Indian needs is planned and CBA has been invited to cooperate. In this preparative project, funded by Vinnova, Bengtsson has visited India twice to work out the detailed project plans. The planning also resulted in an application to the Swedish Research Council for funding of a more detailed study of the 3D chromatin texture of the cervical cells. It is well known that the chromatin texture of cell nuclei is a strong indicator of malignancy. It has even been shown that cells in the vicinity of malignancies have a different chromatin texture than normal cells from healthy specimens. Unfortunately the extraction of texture features is strongly focus dependent making automation based on texture features unreliable.We propose to solve that problem by developing novel 3D texture analysis methods and apply those to image stacks acquired through a high numerical aperture lens.
The VR application has been granted and that part of the project has already started although funding will only be available from the beginning of 2009. Dr Hyun-Ju Choi from Korea has studied 3D nuclear texture for other applications and she has received a sholarship to be a visiting researcher at CBA working on this project, starting November 2008 and continuing for one year. The Indian funding for the project is still pending but we hope to get a decision and project start before mid 2009.
|
