Icon Legend

This session is not in your schedule.

This session is in your schedule. Click again to remove it.

Biomedical Imaging and Instrumentation

Biomedical Imaging and Instrumentation - Poster Session A

Poster F4 - A Dynamic Time Warping-Based Approach for 2D Affine Image Alignment

Thursday, October 24, 2024

10:00 AM - 11:00 AM EST

Location: Exhibit Hall E, F & G

Presenting Author(s)

Lindsey Wang

Electrical Engineering & Applied Mathematics Undergraduate Student
University of Delaware
Fairfax, Virginia, United States

Co-Author(s)

Logan P. Hallee (he/him/his)

Graduate Research Assistant
Center for Bioinformatics and Computational Biology, University of Delaware
Newark, Delaware, United States

Primary Investigator(s)

JG

Jason P. Gleghorn

Associate Professor
University of Delaware, United States

Introduction: Dynamic Time Warping (DTW) is a well-known algorithm for measuring similarity and finding an optimal match between sequences, making it useful for image alignment – a crucial step in 3D reconstruction of microscopy data for applications including anatomical computational modeling and quantitative morphodynamics studies. While much progress has been made towards high-throughput anatomical 3D reconstruction, 2D image alignment of small multicomponent features such as complex vascular networks often pose significant technical challenges for high-resolution 3D reconstruction. Here, we present a framework to correct for serial image misalignment – due to minor displacements and rotations of tissue under a microscope when manually imaging different sections of an entire organ – using DTW and affine transformations. Two popular approaches to solve image alignment problems are 1) the Fourier-Mellin Transform[1], and 2) pre-assigning points with fiducial markers for transformation[2]; however, these approaches are not applicable to correct for rotation and displacement transformations of medical data due to 1) the Transform only accounting for rotation and scale transformations, and 2) a lack of well-annotated data. In the past twenty years, however, multiple papers have been published about DTW, affine transformations, and image alignment to independently solve this problem[3],[4]; this research combines these three techniques to develop a new and robust image alignment framework, including a DTW-based pixel-level mapping that eliminates the need for laborious fiducial markers. Based on the theoretical framework described in the next section, real-world lymph node images were used to demonstrate that latent rotation and displacement transformations can be uncovered.

Materials and

Methods: The proof of concept for our theoretical framework centres on solving the following inverse problem: we affine transform one input image by horizontal and vertical translations u and v and rotation θ (Fig. 1A), and we aim to estimate this latent (u,v,θ) transformation. First, a vector field is calculated using DTW to summarize the pixel-level mappings between images, where each vector starts at the original image pixel location and points to the pixel it maps to in the synthetically misaligned image (Fig. 1B). Notably, the latent (u,v,θ) transformation that we aim to predict can be visually confirmed in the vector field. Next, we derive a system of three nonlinear equations to mathematically describe the latent transformation (Fig. 1C), involving u, v, θ, image centroid coordinates α and β, and three randomly-selected input-output coordinate pairs (xk,yk)-(xk’,yk’). Finally, we estimate the transformation for a particular iteration through a multidimensional root-finder to solve the nonlinear system, and images are iteratively aligned through a multi-shot analysis in the case of imperfect, noisy real data. Our framework is implemented in C++, with domain assumptions that the latent angle θ and displacement values u and v are within the range of -20°~20° and -10~10 pixels, respectively. The experimental implementation involves: OpenCV for affine transformation, FastDTW (DTW) for vector field, and GNU / GSL for multi-root finding.

Results, Conclusions, and Discussions: From the parameters described in the experimental implementation, 200 different configurations of (u,v,θ) were used to produce 200 synthetic cross-sectional misaligned image pairs. Two DTW paths – produced from image pixels read in row-wise and column-wise directions – were generated for each image pair for each of the 200 configurations. For each latent configuration, 1000 solutions were aggregated and their average was regarded as the values of the three latent variables for the current iteration. Since the vector field was obtained from DTW paths – which dynamically warp pixels of two images to form the most likely alignment – that can be highly noisy, as dictated by real-world scenarios, the calculated values of angle and displacements cannot immediately match the ground truth in one shot. Therefore, these calculated latent values were used to affinely transform the original image as the basis for the next iteration to improve accuracy. Specifically, the newly transformed image, based on the most recently calculated latent variables, was then compared with the previously affine transformed ground-truth image (the synthetically misaligned ground truth image). Through multiple iterations (multi-shot analysis) to find the values of the latent variables, our preliminary results (Fig. 1D) show that the formulation of the nonlinear system in conjunction with the DTW-based vector field can produce the estimation of angle and displacement values to correct misalignment to such a level we can preserve anatomical accuracy and detail. We aim to improve the accuracy of our implementation by eliminating the image artifact (blurring) that arises from successive affine transformations.

Image alignment remains a long-standing unsolved problem in the biomedical field. However, we present a novel framework for 2D affine image alignment, using DTW to eliminate the need for fiducial markers – a common yet laborious approach for simplifying alignment. We successfully apply our theoretical framework to noisy real-world data, and preliminary results of a DTW-based pixel-level mapping between images combined with an iterative, multi-shot process show that our mathematical formulation can estimate the latent displacements and rotation (u,v,θ) of two misaligned images in a z-stack.

Acknowledgements (Optional):