Pandemic Accelerated Human-Machine Collaboration (PACMAN) : a framework for pulse oximeter digit detection and reading in a low-resource setting

In light of the COVID-19 pandemic, patients were required to manually input their daily oxygen saturation (SpO2) and pulse rate (PR) values into a health monitoring system—unfortunately, such a process trend to be an error in typing. Several studies attempted to detect the physiological value from the captured image using optical character recognition (OCR). However, the technology has limited availability with high cost. In low-resource settings, where access to advanced medical equipment is limited, the accurate detection and reading of pulse oximeters can be challenging. To address this issue, a groundbreaking framework called PACMAN has been developed.

PACMAN is an innovative framework designed to revolutionize pulse oximetry in low-resource settings with a low-resource deep learning-based computer vision. This groundbreaking technology enables the precise detection and reading of pulse oximeters, empowering healthcare providers, and improving patient care in resource-constrained environments.

Figure 1 : PACMAN framework

We compared state-of-the-art object detection algorithms (scaled YOLOv4, YOLOv5, and YOLOR), including the commercial OCR tools for digit recognition on the captured images from the pulse oximeter display. All images were derived from crowdsourced data collection with varying quality and alignment. YOLOv5 was the best-performing model against the given model comparison across all datasets, notably the correctly orientated image dataset. We further improved the model performance with the digits auto-orientation algorithm and applied a clustering algorithm to extract SpO2 and PR values. The accuracy performance of YOLOv5 with the implementations was approximately 81.0-89.5%, which was enhanced compared to without any additional implementation. 

PACMAN function also be further used as one of the functions in the CHIVID system. According to patient data derived from the CHIVID database, among 14,817 patients who use the system, the PACMAN function was utilized by patients on the system more frequently in the CHIVID system during the peak of the outbreak of the omicron strain in Thailand, which 28.37 of all patients who used the system used the PACMAN function, with 82.53% of users who used using it more than once during treatment and 13.63 of users who used using this function every time they provided their daily symptom progress. Moreover, the result shows that the accuracy of the data was assessed to be 81.0%.

Accordingly, this study highlighted the completion of the PACMAN framework to detect and read digits in real-world datasets. The proposed framework has been currently integrated into the patient monitoring system utilized by hospitals nationwide.

Figure 2 : Comparison with existing studies on medical device display dataset

Affiliations / Departments / Agencies:

• C. Boonnag is affiliated with Biomedical Informatics Center, Faculty of Medicine, Chiang Mai University, Thailand.
• W. Saengmolee, N. Seesawad, A. Chinkamol, K. Thanontip, W. Limpornchitwilai, and T. Wilaiprasitporn are with Bio-inspired Robotics and Neural Engineering (BRAIN) Lab, School of Information Science and Technology (IST), Vidyasirimedhi Institute of Science & Technology (VISTEC), Rayong, Thailand.
• S. Rattanasomrerk is with Medensy Co., Ltd., Bangkok, Thailand.
• K. Veerakanjana and P. Ittichaiwong are with Siriraj Informatics and Data Innovation Center, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand.

Contact Us

Asst. Prof. Dr. Theerawit Wilaiprasitporn (Team Leader)

Bio-inspired Robotics and Neural Engineering (BRAIN) Lab, 

School of Information Science and Technology (IST), 

Vidyasirimedhi Institute of Science & Technology (VISTEC), 

Rayong 21210, Thailand

Email: theerawit.w@vistec.ac.th