Poster R4 - 3D rendering and quantification of lymphatics activity using RGB-D and near-infrared fluorescence lymphatic imaging

Introduction: The lymphatic system is a unidirectional circulatory system that plays a vital role in maintaining cardiovascular health and facilitating immune surveillance. It is often medically overlooked despite its importance. The lymphatics provide a primary pathway for metastasis and are frequently targeted for the surveillance and treatment of cancer. Cancer treatment can result in cancer-related lymphedema (LE) for survivors, marked by increased tissue volumes, presenting notable management difficulties given its incurable nature. Cancer-related LE in the limbs is clinically diagnosed based on a 5-10% increase in the relative limb volume compared to the contralateral limb1. Diagnosis of head and neck cancer-related LE remains challenging due to the lack of contralateral comparison. Recent advancements indicate that the presence of dermal backflow (DBF) in lymphatics is predictive of LE and can be captured using near-infrared fluorescence lymphatic imaging (NIRF-LI) of indocyanine green (ICG) injections in the lymph nodes2. DBF refers to the retrograde flow of lymphatic fluid from collecting lymphatic vessels into the lymphatic capillaries, and not only affects fluid dynamics but also induces irreversible tissue changes, including tissue swelling, subcutaneous adipose tissue deposition, and ultimately fibrosis. Studies indicate that early intervention, before tissue swelling occurs, could improve LE outcomes. However, the limited diagnostic tools available for monitoring lymph flow and identifying early lymphatic dysfunction pose a challenge in preventing LE onset.

Materials and

Methods: We propose a framework to visualize and quantify the lymphatics’ activity in the head or arm using data from an imaging device integrating NIRF-LI and the Intel RealSense RGB-D cameras3. Figure 1 provides an overview of the proposed framework. Initially, we save a set of fluorescent images from the NIRF camera stream capturing lymph flow in the arm following ICG injections, and color and depth images from the RGB-D streams for 3D rendering. Stereo-camera calibration aims at finding the relative pose of the RGB to the depth camera and that of the NIRF to the RGB camera. RGB-D cameras are pre-calibrated with metadata including their intrinsic/extrinsic matrices. We use a custom checkerboard to estimate the transformation of the NIRF camera to the RGB camera and evaluate calibration quality using mean reprojection error. We perform preliminary processing to enhance depth quality through an edge-preserving temporal filter. We compute the 3D point cloud (PC) using the depth image and the depth camera intrinsic matrix. We provide color texture to this PC from the RGB and NIRF images using their intrinsic/extrinsic matrices. During multiview registration, we capture distinct poses from various angles and create a PC for each pose. These PCs are fused using fiducial markings and Iterative Closest Point (ICP) algorithm4 to create a complete PC. We apply the ball pivoting algorithm5 on the resulting PC to generate a mesh representation. Then, we use adaptive thresholding to segment fluorescence in the mesh and calculate the surface area of relevant segmented components.

Results, Conclusions, and Discussions: We generate a 3D point cloud with color texture using the depth image, color image, intrinsic, and extrinsic calibration acquired from the intel RealSense RGB-D camera and the metadata provided by the manufacturer as shown in Figures 2E and 2H. Using the inferred transformation between the color and NIRF cameras in the stereo calibration step, we estimate the correspondences between the pixels of images of the same scene captured by both cameras. We use these correspondences to provide the 3D PC with a second texture from the NIRF camera (Figures 2F and 2I). Prior to generating the PC, we apply edge-preserving filtering to the depth maps using temporal smoothing filters. This allows for the creation of PCs with smoother surfaces without losing important details.
Figure 3 highlights the registration of multiple 3D views to produce a more complete surface rendering of the arm. A full mesh is created by applying the ball pivoting algorithm to the resulting PC. A color correction algorithm is in development to address the texture disparities in the full mesh. After applying thresholding routines on the rendered complete mesh with NIRF texture, lymph flow can be segmented as shown in Figure 4 and its corresponding surface area is computed.
Our preliminary results demonstrate the reconstruction of head and arm 3D models with NIRF texture overlay using data from our NIRF and RGB-D stereo-camera device. We also demonstrate the feasibility of quantification of lymph fluid on clinically relevant 3D surfaces. This framework can provide a clinically useful measurement for monitoring lymphatics to identify dysfunction and assess early physiotherapy impact on the progression of LE in cancer survivors.

Acknowledgements (Optional): This work was funded in part by the Cancer Prevention and Research Institute of Texas [RP220204 (JCR)].

References:

1. Armer, J. M. The problem of post-breast cancer lymphedema: Impact and measurement issues. Cancer Invest 23, 76–83 (2005).
2. Aldrich, M. B. et al. Prediction of breast cancer-related lymphedema by dermal backflow detected with near-infrared fluorescence lymphatic imaging. Breast Cancer Res Treat 195, 33–41 (2022).
3. Bouhali, S., Merchant, F., Karni, R. J., Gutierrez, C. & Rasmussen, J. C. 3D rendering and analysis of dermal backflow as an early indicator of cancer-acquired lymphedema using RGB-D and near-infrared fluorescence lymphatic imaging. in Medical Imaging 2024: Clinical and Biomedical Imaging (eds. Gimi, B. S. & Krol, A.) vol. 12930 1293008 (SPIE, 2024).
4. Segal, A., Haehnel, D. & Thrun, S. Generalized-icp. in Robotics: science and systems vol. 2 435 (Seattle, WA, 2009).
5. Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C. & Taubin, G. The ball-pivoting algorithm for surface reconstruction. IEEE Trans Vis Comput Graph 5, 349–359 (1999).