Classification of MRI scans using radiomics and MLP

Tumors, which are abnormal growths that can develop in brain tissues, pose significant challenges to the central nervous system. To detect unusual activities in the brain, we rely on advanced medical imaging techniques, such as MRIs and CT scans. However, accurately identifying tumors can be complex due to their various shapes and textures, requiring careful analysis by medical professionals. This is where the power of MRI scans using radiomics comes into play. By implementing handcrafted feature extraction followed by classification techniques, we can improve the speed and efficiency with which clinicians analyze imaging data, ultimately leading to more accurate diagnoses and better outcomes for patients.

Learning objectives

• Going deeper into the domain of artisanal characteristics.
• Understand the importance of Radiomics in the extraction of artisanal features.
• Learn how radiomics MRI scans improve tumor detection and classification, allowing for more accurate medical diagnoses.
• Use the extracted features to classify into different classes.
• Harnessing the power of radiomics and multilayer perceptron for classification.

This article was published as part of the Data Science Blogathon.

Understanding Radiomics for Feature Extraction

Radiomics is the technique used in the medical field to detect handicraft characteristics. By handmade characteristics, we mean texture, density, intensities, etc. These features are useful as they help understand complex disease patterns. Basically, it uses mathematical and statistical operations to calculate the values ​​of the characteristics. The final values ​​provide us with deep insights that can be used later for additional clinical observations. Here we must point out one thing. Feature extraction is basically done in the Region of Interest.

Common radiomic features for tumor detection

Here we will discuss the features that are extracted using Radiomics. Some of them are the following:

• Shape Features: In this Radiomics the geometric features of the Region of interest are extracted. It includes volume, area, length, amplitude, compactness, etc.
• Statistical characteristics: As the name suggests, it uses statistical techniques like mean, standard deviation, skewness, kurtosis, and randomness. With these we can evaluate the intensity of the ROI.
• Texture characteristics: These characteristics focus on the homogeneity and heterogeneity of the surface of the Region of Interest. Some examples are the following:
  • GLCM or Gray Level Co-occurrence Matrix: Measures the contrast, correlation of the pixels or voxels in the ROI
  • Gray Level Zone Size Matrix or GLZSM: Used to calculate the zonal percentage of homogeneous areas in the ROI.
  • Gray Level Run Length Matrix or GLRLM: It is used to measure the uniformity of intensities throughout the region of interest.
• Advanced math functions: Advanced mathematical techniques such as Laplacian, Gaussian and gradient formulas capture patterns in depth by applying filters.

Dataset Overview

Here we will use the brain tumor dataset which is present on Kaggle. The link to download the data set is here. The data set has two categories or classes: yes or no. Each class has 1500 images.

• it does denote the presence of the tumor.
• does not denote that the tumor is not present.

Below are some sample images:

Setting up the environment and libraries

We used the PyRadiomics library to extract features and chose Google Colab for this process as it provides the latest version of Python, ensuring PyRadiomics runs smoothly. Otherwise, using older versions may cause errors. Apart from PyRadiomics, we have used other libraries like SITK, Numpy, Torch to create multilayer perceptrons. We have also used Pandas to store the functions in the data frame.

As mentioned above, we will use the brain tumor dataset. But there are no masks here that can be used to highlight the brain tissue that is our Region of Interest. Then we will create binary masks and extract features from the masked region. So, we will first load the image dataset using the OS library and create a data frame comprising image paths and labels.

# 1. Import necessary libraries
import os
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, accuracy_score
from radiomics import featureextractor
import SimpleITK as sitk

# 2. Mount Google Drive
from google.colab import drive
drive.mount('/content/drive')

# 3. Define the dataset path
base_path="/content/drive/MyDrive/brain"

# 4. Prepare a DataFrame with image paths and labels
data = ()
for label in ('yes', 'no'):
    folder_path = os.path.join(base_path, label)
    for filename in os.listdir(folder_path):
        if filename.endswith(('.png', '.jpg', '.jpeg')):  # Ensure you're reading image files
            image_path = os.path.join(folder_path, filename)
            data.append({'image_path': image_path, 'label': label})

df = pd.DataFrame(data)

We will use the Simple Image Tool Kit (SITK) library to read images, since SITK preserves voxel intensities and orientation, features that OpenCV or Pillow do not maintain. Additionally, SITK is supported by Radiomics, ensuring consistency. After reading the image, we convert it to grayscale and create a binary mask using Otsu thresholding, which provides optimal values ​​for grayscale images. Finally, we extract the radiomics features, label each feature as “yes” or “no”, store them in a list, and convert the list to a DataFrame.

# 5. Initialize the Radiomics feature extractor
extractor = featureextractor.RadiomicsFeatureExtractor()
k=0
# 6. Extract features from images
features_list = ()
for index, row in df.iterrows():
    image_path = row('image_path')
    label = row('label')

    # Load image
    image_sitk = sitk.ReadImage(image_path)

    # Convert image to grayscale if it is an RGB image
    if image_sitk.GetNumberOfComponentsPerPixel() > 1:  # Check if the image is color (RGB)
        image_sitk = sitk.VectorIndexSelectionCast(image_sitk, 0)  # Use the first channel (grayscale)

    # Apply Otsu threshold to segment brain from background
    otsu_filter = sitk.OtsuThresholdImageFilter()
    mask_sitk = otsu_filter.Execute(image_sitk)  # Create binary mask using Otsu's method

    # Ensure the mask has the same metadata as the image
    mask_sitk.CopyInformation(image_sitk)

    # Extract features using the generated mask
    features = extractor.execute(image_sitk, mask_sitk)
    features('label') = label  # Add label to features
    features_list.append(features)
    print(k)
    k+=1

# 7. Convert extracted features into a DataFrame
features_df = pd.DataFrame(features_list)

# 8. Split the dataset into training and testing sets
x = features_df.drop(columns=('label'))  # Features
y = features_df('label')  # Labels

Preprocessing feature data

When Radiomics extracts features from images, it also adds the version of the features to the feature matrices. Therefore, we must include the feature values ​​that have the feature name with 'original_'. For non-numeric feature values, we coerce and then fill that data with 0. For the labels part, we convert the strings to 0 or 1. After that, we split the data into train and test in a ratio of 80:20. Lastly, functions are standardized using StandardScaler. We also check if the classes are unbalanced or not.

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
import pandas as pd
import matplotlib.pyplot as plt

# Assuming features_df is already defined and processed

feature_cols = (col for col in features_df.columns if col.startswith('original_'))

# Convert the selected columns to numeric, errors="coerce" will replace non-numeric values with NaN
features_df(feature_cols) = features_df(feature_cols).applymap(lambda x: x.item() if hasattr(x, 'item') else x).apply(pd.to_numeric, errors="coerce")

# Replace NaN values with 0 (you can use other strategies if appropriate)
features_df = features_df.fillna(0)

# Split the dataset into training and testing sets
x = features_df(feature_cols).values  # Features as NumPy array
y = features_df('label').map({'yes': 1, 'no': 0}).values  # Labels as NumPy array (0 or 1)
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)


class_counts = pd.Series(y_train).value_counts()

# Get the majority and minority classes
majority_class = class_counts.idxmax()
minority_class = class_counts.idxmin()
majority_count = class_counts.max()
minority_count = class_counts.min()

print(f'Majority Class: {majority_class} with count: {majority_count}')
print(f'Minority Class: {minority_class} with count: {minority_count}')

Using multilayer perceptron for classification

In this step, we will create a multilayer perceptron. But before that we convert the train and test data to tensors. DataLoaders are also created with a batch size of 32.

X_train_tensor = torch.tensor(X_train, dtype=torch.float32)
y_train_tensor = torch.tensor(y_train, dtype=torch.long)
X_test_tensor = torch.tensor(X_test, dtype=torch.float32)
y_test_tensor = torch.tensor(y_test, dtype=torch.long)

# Create PyTorch datasets and dataloaders
train_dataset = TensorDataset(X_train_tensor, y_train_tensor)
test_dataset = TensorDataset(X_test_tensor, y_test_tensor)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)  # Adjust batch size as needed
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=False)

The MLP defined below has two hidden layers, ReLU as the activation function and the dropout rate is 50%. The loss function used is Cross Entropy Loss and the optimizer used is Adam with a learning rate of 0.001.

class MLP(nn.Module):
    def __init__(self, input_size, hidden_size1, hidden_size2, output_size):
        super(MLP, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size1)
        self.relu1 = nn.ReLU()
        self.dropout1 = nn.Dropout(0.5)  # Dropout layer with 50% dropout rate
        self.fc2 = nn.Linear(hidden_size1, hidden_size2)
        self.relu2 = nn.ReLU()
        self.dropout2 = nn.Dropout(0.5)
        self.fc3 = nn.Linear(hidden_size2, output_size)

    def forward(self, x):
        x = self.fc1(x)
        x = self.relu1(x)
        x = self.dropout1(x)
        x = self.fc2(x)
        x = self.relu2(x)
        x = self.dropout2(x)
        x = self.fc3(x)
        return x

# Create an instance of the model
input_size = X_train.shape(1)  # Number of features
hidden_size1 = 128  # Adjust hidden layer sizes as needed
hidden_size2 = 64
output_size = 2  # Binary classification (yes/no)
model = MLP(input_size, hidden_size1, hidden_size2, output_size)

# Define loss function and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)  # Adjust learning rate as needed

# Initialize a list to store loss values
loss_values = ()

# Train the model
epochs = 200  # Adjust number of epochs as needed
for epoch in range(epochs):
    model.train()  # Set model to training mode
    running_loss = 0.0

    for i, (inputs, labels) in enumerate(train_loader):
        # Zero the gradients
        optimizer.zero_grad()

        # Forward pass
        outputs = model(inputs)

        # Compute the loss
        loss = criterion(outputs, labels)

        # Backward pass and optimization
        loss.backward()
        optimizer.step()

        # Accumulate the running loss
        running_loss += loss.item()

    # Store average loss for this epoch
    avg_loss = running_loss / len(train_loader)
    loss_values.append(avg_loss)  # Append to loss values
    print(f"Epoch ({epoch+1}/{epochs}), Loss: {avg_loss:.4f}")

# Test the model after training
model.eval()  # Set model to evaluation mode
correct = 0
total = 0

with torch.no_grad():  # Disable gradient computation for testing
    for inputs, labels in test_loader:
        outputs = model(inputs)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

# Calculate and print accuracy
accuracy = 100 * correct / total
print(f'Test Accuracy: {accuracy:.2f}%')

# Plot the Loss Graph
plt.figure(figsize=(10, 5))
plt.plot(loss_values, label="Training Loss", color="blue")
plt.title('Training Loss Curve')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.grid()
plt.show()

As we can see, the model is trained for 200 epochs and the loss is recorded in each epoch, which will then be used for plotting. The optimizer is used to optimize the weights. We will now test the model by turning off the gradient calculations.

As we can see from the result below, the accuracy is 94.50% on the test data set. From this we can conclude that the model generalizes well based on radiomic features.

Conclusion

Leveraging radiomics and multilayer perceptrons (MLP) in brain tumor classification can streamline and improve the diagnostic process for medical professionals. By extracting handcrafted features from brain images, we can capture subtle patterns and features that help accurately identify the presence of tumors. This approach minimizes the need for manual analysis, allowing clinicians to make informed, data-driven decisions more quickly. Integrating feature extraction with MLP classification demonstrates the potential of ai in medical imaging, presenting an efficient and scalable solution that could greatly assist radiologists and healthcare providers in diagnosing complex cases.

Click here for the Google collaboration link.

Key takeaways

• Radiomics captures detailed image features, allowing for more precise brain tumor analysis.
• Multilayer perceptrons (MLPs) improve classification accuracy by processing complex data patterns.
• Feature extraction and MLP integration streamline brain tumor detection, helping in faster diagnosis.
• Combining ai with radiology offers a scalable approach to assist healthcare professionals.
• This technique exemplifies how ai can improve the efficiency and accuracy of diagnosis in medical imaging.

Frequently asked questions

The media shown in this article is not the property of Analytics Vidhya and is used at the author's discretion.