Poster M11 - Motion tracking and machine learning approach for segmentation of a 3D printed Ice structures in real time

Introduction: Bio-fabrication of vascular tissue presents significant challenges in biomanufacturing, stemming from the micrometer-scale resolution of the process, the mechanical properties of tissue constructs, cell viability issues, and the necessity to replicate the scale and geometrical complexity of in-vivo vascular systems for accurate modelling.

Ice printing is a promising method for vasculature fabrication. Ice is biocompatible and can be used as a sacrificial material that will melt at room temperature and leave cross-linked biomaterials in place, creating a hollow vasculature geometry. Printing ice at the micrometer-millimeter scale is a challenging, high-frequency additive manufacturing process subject to many variables. An important direction forward is to integrate real-time sensing and control which would help enable automated vascular geometry biomanufacturing with Ice 3D Printing, which currently is mostly manually driven for new structures. To be able to advance our goal Machine learning is utilized with fast imaging to be able to control the printing process. While this imaging is needed, unfortunately ice is transparent and there is no dataset of labeled images to train Machine Learning systems.

Materials and

Methods: To work on this automated approach with Machine Learning, our group developed a feature and motion tracking hybrid segmentation algorithm (FAST/Farneback) for ice, which can be translatable to translucent/transparent objects in other advanced material manufacturing applications (glass, gels, etc).

A combination of a motion tracking algorithm Gunnar-Farneback, and a feature detection algorithm, FAST were implemented to segment transparent ice videos. The FAST feature detectors were used to selectively erode the Farneback motion profile, creating a more accurate segmentation of printed Ice.

Results, Conclusions, and Discussions: The detection and partial segmentation of the 3D ice constructs has been achieved using the FAST-Farneback segmentation and its accuracy is within the range of manual measurements, but with a millisecond time frame ability. Frames are segmented at a rate of 5-10Hz which would enable real time closed loop control under low droplet ejection frequencies to enable 3D Ice printing without the need for manual trial and error approaches. Furthermore, the ability to segment Ice 3D printing videos has enabled the creation of datasets that will be used to train Machine Learning segmentation systems based on linear matrix operations which that run-in sub 100Hz latencies using Graphic Processing Units (GPUs). Image segmentation running faster than droplet ejection will enable robust sensor feedback for manufacturing control schemes, using ice geometry widths and centroid as sensor feedback for digital control strategies.

For automated 3D Ice printing, the FAST/Farneback segmentator will allow the creation of labelled datasets that can help future segmentation applications for additive manufacturing of transparent materials. Datasets of transparent 3D printed constructs will allow supervised learning pipelines for this application. This work will have impact in a variety of areas including tissue engineering, artificial intelligence, and advanced biomanufacturing.

Acknowledgements (Optional):