2.1. Sample Data Collection and Preprocessing

In order to develop the database for the ML tool to process and analyze, we conducted a retrospective study between 2019 and 2023, collecting data from the Pathology Department of “Saint John” Hospital in Galati, Romania, after obtaining the approval of the Ethics Committee. For each case, informed consent was obtained from the patient, indicating agreement with the collection, storage, and use of biological samples for both diagnostic and scientific purposes. Any data related to patient identification, except the histopathological diagnosis and the IHC results, were blinded before data processing. The diagnoses following immunohistochemical evaluation were collected, and for each case, the specific IHC marker panels used, and their positive or negative results, were documented.

Cases with insufficient IHC profiles—those without any markers indicating the origin of the tumor, IHC profiles with fewer than three markers, or unclear findings—were eliminated.

After collecting the data, a total of 686 cases were included in the primary dataset. IHC antibodies, averaging 5.4 per case (range 3–11), were utilized in the diagnostic process, with a total of 58 IHC markers identified and used in various combinations in panels to support the histopathological diagnosis of each case. These markers were utilized based on their relevance to various tumor types and included common antibodies such as Cytokeratin AE1/AE3, Cytokeratin 7, Cytokeratin 20, p63, Cytokeratin 5/6, E-cadherin, Estrogen Receptor, Progesterone Receptor, p53, S100, Melan-A, Human Melanoma Black 45 (HMB45), Epithelial Membrane Antigen (EMA), CD45, CD20, CD3, CD10, CD34, Thyroid Transcription Factor 1 (TTF1), Vimentin, Actin, Calretinin, Alpha Methyl Acyl CoA Racemase (AMACR), Prostate-Specific Antigen (PSA), and Glial Fibrillary Acidic Protein (GFAP). A total of 32 histopathological diagnoses were identified, but due to the limited sample size for some diagnoses, only 20 were included in the computations performed by the ML model, with the following distribution: invasive breast carcinoma, ductal type (15.31%), squamous cell carcinoma (12.54%), prostatic acinar adenocarcinoma (9.33%), pulmonary adenocarcinoma, not otherwise specified (NOS) (8.75%), colorectal adenocarcinoma, NOS (7.43%), serous carcinoma (4.37%), endometrioid carcinoma (3.21%), melanoma (3.21%), invasive breast carcinoma, lobular type (3.06%), basal cell carcinoma (2.62%), urothelial carcinoma (2.62%), gastric adenocarcinoma, NOS (2.04%), small cell neuroendocrine carcinoma (1.89%), invasive breast cancer, other types (1.75%), colorectal mucinous adenocarcinoma (1.60%), glioblastoma (1.60%), hepatocellular carcinoma (1.46%), cholangiocarcinoma (1.46%), Kaposi sarcoma (1.46%), and undifferentiated pleomorphic sarcoma (1.46%). Diagnoses like malignant mesothelioma, mixed germ cell tumor, or non-Hodgkin lymphoma, with less than 10 cases, were categorized as “Other” by the program due to their limited numbers, representing 12.83% of the total cases.

To test the ML model, a secondary dataset comprising 21 cases from an alternative center was created to mitigate the bias associated with diagnoses conducted by the same pathologists. This new dataset adhered to the same inclusion and exclusion criteria as the primary dataset, encompassing the most frequent diagnoses, such as invasive ductal carcinoma of the breast or pulmonary adenocarcinoma, as well as diagnoses categorized as “Other”, such as pancreatic adenocarcinoma.

All the collected data were compiled into two Excel tables with 59 columns, with the first column containing the histopathological diagnosis and the remaining columns listing the IHC markers with their respective positive or negative results, or not applicable (N/A) if the marker was not used for that specific case.

2.2. Machine-Learning Algorithm Model and Model Development

The ML algorithm employed in our study leverages Bayes’ theorem, a fundamental concept in probability theory and statistics that connects conditional probabilities with marginal probabilities. It states that if the pre-event probability is known, the post-event probability can be determined. Conditional probabilities indicate the likelihood of one event occurring, given that another event has occurred. Additionally, the probabilities of detecting events separately are also considered [15].

IHC results are binary, indicating either positive or negative outcomes, and pathologists possess practical knowledge about the probability of specific IHC markers being positive or negative in each tumor, also based on well-documented information in textbooks and other publications. The probability of positivity represents the number of positive cases out of all the diagnoses. After obtaining the test results, the likelihood of each diagnosis can be computed by multiplying the prior probability by the chance of each test being positive or negative. This allows for the comparison of post-probability values to determine which diagnosis has the highest likelihood.

To generate a prediction model, the Exploratory Desktop software, version 6.12.5.2, developed by Exploratory, Inc. was used. Exploratory Desktop offers a straightforward and user-friendly interface for accessing several data sources, transforming and cleaning data, visualizing and analyzing data, and using a wide range of machine-learning algorithms to explore data and gain deeper insights quickly. XGBoost, or eXtreme Gradient Boosting, is a well-liked machine-learning algorithm that creates prediction models in the form of an ensemble of weak prediction models—usually Decision Trees—for regression and classification tasks. Similar to previous boosting techniques, it constructs the model step-by-step and extends it by permitting optimization of any differentiable loss function.

2.3. Model Training, Validation and Testing

The first step in training the model was to import our Excel dataset into the data frame from the Exploratory main interface. Once the data were loaded, the wrangling tools available were used to make some minor adjustments to eliminate any program errors, misinterpretation of data, or false results. In the second step, the XGBoost learning model was applied, where the input variable (target variable) was the histopathological diagnosis and the predictors (independent variables influencing the target variable) were the immunohistochemical markers. In the final step, the model training parameters were set; in the first instance, the default parameters provided by XGBoost were used and the model performance was checked followed by a grid search for hyperparameter tuning to find the best parameter combination.

To validate the model, the XGBoost advanced settings have an option that allows splitting the dataset into various ratios of training data and validation data, performing the training and validation at the same time. We chose a ratio of 7:3, and to avoid overfitting, we increased the number of iterations and the number of early stopping iterations. Finally, the negative log-likelihood was chosen for the multiclass classification metric. After the model was properly trained and validated, we saved it to apply the testing data set.

Testing the XGBoost model in Exploratory involved the following steps: importing the testing dataset in Exploratory, applying the trained model to the test data, and the program outputting the predicted values alongside the actual value in a new data frame. By following these detailed steps, we could effectively assess our model’s generalization ability and make informed decisions on any further tuning or adjustments.

3.2. Model Testing Results

After training and validating the model, a new set of 21 patients was used for testing. The following results were obtained after applying the model (Table 2). The model exhibits strong performance in predicting the following histopathologic diagnoses: squamous cell carcinoma, basal cell carcinoma, prostatic acinar adenocarcinoma, glioblastoma, invasive breast carcinoma ductal type, small cell neuroendocrine carcinoma, Kaposi sarcoma, and serous carcinoma. The precise alignment between the true labels and predicted labels for these diagnoses indicates the model’s high reliability and robustness, ensuring both high precision and recall. This consistency is crucial for clinical accuracy and minimizing the risk of misdiagnosis.

However, the model frequently misclassified pancreatic adenocarcinoma, gastric poorly cohesive carcinoma, pulmonary adenocarcinoma NOS, undifferentiated pleomorphic sarcoma, and endometrioid carcinoma into the “Other” category. This misclassification suggests that the model struggled to distinguish these specific cancer types, likely due to insufficient training data or overlapping features with other cancer types. For instance, pancreatic adenocarcinoma and undifferentiated pleomorphic sarcoma had few cases in the training set, resulting in inadequate learning by the model. Additionally, the misclassification of gastric poorly cohesive carcinoma as invasive breast carcinoma, lobular type indicates potential feature similarities or insufficient distinction in the IHC markers used.

The model’s high performance for several common cancer types indicates its potential value in clinical settings, aiding in early and accurate diagnosis, which is critical for effective treatment planning. However, the misclassifications and lower performance for certain rare or complex cancer types highlight the need for more comprehensive training data and the incorporation of additional diagnostic markers or advanced feature-engineering techniques to improve the model’s robustness.

Overall, the model shows promising results for many cancer types. Addressing its limitations through targeted improvements can enhance its diagnostic accuracy and reliability, thus providing better support for clinical decision-making.

4.2. Performance Metrics and Generalization

In the present study, we validated the estimated diagnostic probability of specific histopathological diagnoses based on particular immunohistochemical markers. This was achieved using a machine-learning system employing a probabilistic Decision Tree model, such as XGBoost. These models have been applied to identify lung cancer and colon cancer subtypes [32], predict lung metastases from thyroid cancer [33], and develop risk models for detecting lung cancer [34], all of which have demonstrated high performance.

Our study demonstrated a precision rate of 85.97% within the dataset, indicating a high level of performance and suggesting that the model is generally reliable in producing accurate predictions. The precision rate of the predicted diagnoses was generally higher compared to prior research on tumors of unknown origin, which reported a precision rate of 79.9% [35]. However, it was lower than a study using lymphoma cases, which achieved a precision rate of 95% [36].

A high precision rate was observed in diagnoses such as adenocarcinoma with various localizations and melanoma. The authors attributed these findings to the fact that the machine-learning system provides more accurate predictions when trained on a larger number of cases. Other studies have shown that the accuracy rate for each tumor type is closely related to the corresponding sample size. Moreover, an imbalance in the dataset complicates the achievement of high performance [37,38,39].

Trying to understand the issues that led to cases being classified under a different diagnosis than the correct one, we analyzed each case in the testing set individually. In the case of pancreatic adenocarcinoma classified as “Other”, the issue is that the training set included only six cases with this diagnosis, which was not sufficient for learning and creating a distinct class, as this diagnosis was not among the 20 subtypes mentioned in Table 1. A similar explanation can be applied to undifferentiated pleomorphic sarcoma, which had five cases in the training set. For poorly cohesive gastric carcinoma, the training set had only three cases, which did not allow for the creation of a class. However, due to the use of common markers for both diagnoses (CK7, HER2, E-cadherin), the model classified it as lobular breast carcinoma instead of labeling it as “Other”. The case of lung adenocarcinoma classified as “Other” is atypical; as there were sufficient data in the training set, the F1 score was quite high and the precision was 1.0000. The explanation for this could be that it was a poorly differentiated brain metastasis, requiring the use of a larger number of markers (specifically 11) to establish the diagnosis, which likely led to an inaccurate prediction. The situation where an endometrioid carcinoma had the predicted label “Other” is attributed to the fact that this class has an F1 score of 0.7778, indicating moderate performance, most likely due to the lack of specificity of the IHC markers for this tumor subtype. The challenge of imbalanced data is critical in histopathology, where rare cancer types are often underrepresented. This study’s results align with the broader literature emphasizing the need for comprehensive and balanced datasets to enhance model accuracy [25,31]. While the model showed high performance for common cancer types, its generalization to rare types was limited. This is consistent with the challenges faced by other state-of-the-art models, suggesting improvements in training data diversity and advanced feature-engineering are necessary to enhance generalization capabilities [23,31].

As aforementioned, the present study employed a binary tumor marker expression system (positive or negative) to forecast the histopathological diagnosis. Nevertheless, alternative investigations have focused on developing mathematical models capable of predicting cancer types using molecular characteristics, such as gene expression, DNA methylation profiles, or somatic alteration analysis [40,41,42,43]. Through the inclusion of driver gene mutations and complex structural variant-related features, Nguyen et al. [43] achieved the prediction of approximately 35 different cancer (sub)types with an accuracy of approximately 90%. Marquard et al. [44] successfully distinguished between 10 cancer types with 69% accuracy and 6 cancer types with 85% accuracy when considering solely somatic mutations or both somatic mutations and copy number alterations, respectively. Utilizing the presence of somatic point mutations and copy number alterations in 50 genes as predictive features, Soh et al. [38] attained an accuracy of approximately 77% across 28 distinct cancer types.

Another method for predicting histopathological diagnoses, documented in the scientific literature, involves employing machine-learning systems that analyze cellular properties like nuclear size and the homogeneity of cell populations. This approach holds clinical utility for distinguishing various cell groups. Moran et al. [45] utilized the cell detection module within QuPath, an open-source image analysis toolbox for digital pathology, to extract morphological features of Merkel cell carcinoma (MCC). Their findings indicated that the nuclear area and circularity were significant prognostic factors for MCC. Faridi et al. [46] employed region-growing methods to segment cells, extracting morphological features such as the nuclear size, solidity, and eccentricity to grade the pleomorphism scores of whole-slide images of breast cancer cases, and attained an accuracy of 86.6%.