AuroXRay Bringing AI To Medical Imaging.
AuroXRay is an AI-assisted diagnostic platform that helps radiologists interpret chest X-Rays with greater confidence: combining deep learning, explainable heatmaps, and LLM-driven analysis into one reliable “second opinion”.
- Java
- Spring
Angular
Weka- PostgreSQL
Overview
Image processing has been proven effective across many disciplines; medicine is certainly no exception. Chest X-Rays remains one of the most important tools for diagnosing thoracic diseases, but radiology is a complex discipline where even experienced clinicians can still disagree, and a missed or misread finding carries real consequences.
AuroXRay is designed to support radiologist in making more reliable diagnoses, offering a valuable “second opinion”. Rather than producing a single opaque prediction, it lets radiologists classify based on patient’s specific symptoms, narrowing the selection to the radiological findings and pathologies relevant to that condition; instead of running every possible label against every image.
To make its reasoning legible, AuroXRay pairs its classifications with two layers of explanation:
- LLM-generated analysis: Precise prompting is used to have an LLM interpret each finding, discussing visual patterns and symptoms typically associated with the selected radiological observations.
- ScoreCAM heatmaps: Visual attention maps that highlight exact regions of the X-Ray which the model weighted most heavily in reaching its conclusion.
The heatmap methodology is inspired by WekaDeeplearning4J, an excellent Weka implementation for Deeplearning4J.
Honestly, I never envisioned this turning to a full-stack application. The original idea was much smaller: experiment with the CheXpert dataset and build a “magical” SDK which combines Weka, Mulan and Deeplearning4J. The core plan was to process the dataset in iterative subsets, extracting features with a pre-trained transfer learning model at each step, appending them to disk as ARFF, and finally training with Weka, Deeplearning4J, and Mulan.
I never expected it to grow into a full implementation. But somewhere along the way, Spring Boot 3.0 and Angular 17 talked me into making it something better.
Glimpse For X-Ray Classification & Heatmap
AuroXRay: Glimpse For X-Ray Classification & Heatmap
Development Process of AuroXRay
Building AuroXRay: The Development Process (Part 1)
Dataset Overview
AuroXRay is built upon the CheXpert dataset, a large, well-known collection of 224,316 chest X-ray images, each annotated across multiple pathologies and radiological findings.
Much of the prior work treats CheXpert as a multi-label classification problem, often evaluating performance on substantially smaller subsets (as few as 234 images across only five labels). A common approach here is to treat missing labels as U_ZEROs, which further skews the label distribution and leads models to overfit to that assumption rather than learn genuine clinical patterns, highlighting the Label Sparsity problem.
AuroXRay takes a different approach. Instead of treating it as a multi-label classification task, each label is treated as its own binary classification problem. Instances with uncertain or missing label are dynamically excluded on the fly for that specific label, allowing each model to train only on the most reliable available supervision. The sole exception is Support Device, where missing values are treated as 0s to mitigate class imbalance.
The is conceptually similar to Binary Relevance, but with the sparsity-handling baked in, referred to in this project as Sparse Binary Relevance. It's the mechanism that ultimately enables radiologists to classify based on a patient's specific symptoms, rather than forcing every image through every label.
How it works
224,316 images are preprocessed, normalized for uniformity, in iterative subsets. A pretrained transfer model extracts features which are written to disk in ARFF, and Weka-based classifier is trained using the Sparse Binary Relevance approach.
Uses ScoreCAM methodology to generate a heatmap, giving the radiologist a visual anchor for the model's reasoning, not just a number.
In parallel, the selected finding is passed through an LLM with carefully engineered prompt, producing a written interpretation alongside its associated visual patterns and symptoms.
Explainable Heatmap. Bright yellow represents the most salient regions; those with the highest saliency values. Purple/Pink indicates moderate influence: somewhat important, but not highly significant. Dark blue marks areas of very low importance. This allows radiologists to see where the model is focusing and understand the reasoning behind it.
This whole visualization approach and honestly the architectural backbone of the project draws direct inspiration from WekaDeeplearning4J, Weka and Mulan. Credit goes to the developers and maintainers of these projects.
Evaluation
The binary classification results varied across different labels, but several key conditions, i.e., Pleural Effusion, Consolidation, Cardiomegaly and Edema showed strong AUROC and F1-scores, AUROC ranging from 0.7991 (Pneumonia) to 0.9556 (Pleural Other). This indicates the model's strong ability to distinguish between positive and negative cases.
| Pathology | Train | Valid | Test | AUROC | Recall | Precision | F1-Score |
|---|---|---|---|---|---|---|---|
| Enlarged Cardiomediastinum | 24862 | 3081 | 3095 | 0.8406 | 0.5688 | 0.6988 | 0.6272 |
| Cardiomegaly | 38190 | 4674 | 4756 | 0.8753 | 0.89 | 0.8332 | 0.8606 |
| Lung Opacity | 87215 | 10970 | 10903 | 0.826 | 0.9957 | 0.9541 | 0.9745 |
| Lung Lesion | 9271 | 1170 | 1155 | 0.8476 | 0.8837 | 0.9032 | 0.8934 |
| Edema | 60160 | 7526 | 7512 | 0.8828 | 0.9308 | 0.8457 | 0.8862 |
| Consolidation | 36899 | 4531 | 4676 | 0.9165 | 0.7926 | 0.7607 | 0.7763 |
| Pneumonia | 7223 | 960 | 889 | 0.7991 | 0.8472 | 0.6621 | 0.7433 |
| Atelectasis | 28397 | 3581 | 3513 | 0.9233 | 0.9923 | 0.9775 | 0.9849 |
| Pneumothorax | 61929 | 7750 | 7749 | 0.8061 | 0.4842 | 0.5509 | 0.5154 |
| Pleural Effusion | 101065 | 12703 | 12720 | 0.9288 | 0.9462 | 0.8847 | 0.9144 |
| Pleural Other | 3953 | 541 | 505 | 0.9556 | 0.953 | 0.9673 | 0.9601 |
| Fracture | 10807 | 1417 | 1316 | 0.833 | 0.9379 | 0.7603 | 0.8398 |
| Support Devices | 178000 | 22251 | 22246 | 0.8221 | 0.7458 | 0.7475 | 0.7467 |
| Average | – | – | – | 0.8659 | 0.8437 | 0.8112 | 0.8248 |
F1-score reflects the balance between precision and recall, but in medical imaging context, the two errors are not equally costly. A missed diagnosis (false negative) is typically more costly than a false positive. For this reason, Recall (True Positive Rate) is prioritized in the training process, ensuring the model captures as many true cases as possible, even at some cost to precision.
AUROC Performance Per Finding
The false positive rate (FPR) for Lung Opacity is comparatively high (0.90), partly due to data imbalance and partly because it’s a finding associated with several underlying conditions i.e., consolidation, pneumonia and pleural effusion. Given this limitation and the potential redundancy it introduces, Lung Opacity was excluded from the final model.
In summary: the model demonstrates strong performance across this range of thoracic diseases and findings, with AUROC and F1-scores indicated strong classification and diagnostics performance overall.
Bottom Line
Gives radiologists a symptom-driven second opinion instead of a flat list of every possible finding.
Recasting CheXpert as per-pathology binary classification, avoided the label-sparsity overfitting that limited earlier multi-label attempts.
Pairs each prediction with an LLM explanation and a ScoreCAM heatmap, so the output is something a clinician can actually evaluate, not just trust blindly.
A look at the interface
