Collaborative Research Project

PaTEx (Partition-Aware Tree Explanations) is a framework for generating faithful and interpretable tree-structured explanations of neural malware classifiers.

It decomposes the input space into locally coherent regions and learns compact surrogate trees, enabling structured reasoning, symbolic rule extraction, and counterfactual analysis.

---

Repository Structure

patex-malware-explainability/
│
├── notebooks/
│   └── PaTEx_main_experiments.ipynb
│
├── scripts/
│   └── fidelity_lime_shap.py
│
├── datasets/
│   ├── FakeInstaller_selected_features_SKBest.csv
│   ├── dynamic_malware_dataset.csv
│   └── sember_static_malware_dataset.csv
│
├── figures/
├── results/
├── requirements.txt
└── README.md

---

Installation

Install all required dependencies using:

pip install -r requirements.txt

Alternatively, the main dependencies can be installed manually:

pip install numpy pandas scikit-learn tensorflow matplotlib networkx lime shap jupyter

A minimal requirements.txt file may include:

numpy
pandas
scikit-learn
tensorflow
matplotlib
networkx
lime
shap
jupyter

---

Datasets

This repository contains three preprocessed malware classification datasets:

1. EMBER Static Windows Malware Dataset
   File: datasets/sember_static_malware_dataset.csv

2. Dynamic Windows Malware Dataset
   File: datasets/dynamic_malware_dataset.csv

3. FakeInstaller Android Malware Dataset from Drebin
   File: datasets/FakeInstaller_selected_features_SKBest.csv

All datasets used in the experiments are preprocessed and binarized.

---

Switching Between Datasets

By checking the main notebook, the dataset can be changed by modifying the DATASET_NAME variable.

DATASET_NAME = "EMBER"   # Options: "EMBER", "Dynamic", "Drebin"

DATASET_PATHS = {
    "EMBER": "datasets/sember_static_malware_dataset.csv",
    "Dynamic": "datasets/dynamic_malware_dataset.csv",
    "Drebin": "datasets/FakeInstaller_selected_features_SKBest.csv"
}

CSV_PATH = DATASET_PATHS[DATASET_NAME]

Available dataset options are:

"EMBER"
"Dynamic"
"Drebin"

---

Running the Main PaTEx Experiment

The main PaTEx experiment is implemented in the Jupyter notebook:

notebooks/PaTEx_main_experiments.ipynb

To run the main experiment:

1. Install the required dependencies.
2. Open the notebook in Jupyter Notebook, JupyterLab, Google Colab, or VS Code.
3. Select the dataset by setting DATASET_NAME.
4. Run all cells.

The notebook performs the following steps:

• Loads and preprocesses the selected dataset.
• Trains the sparse MLP malware classifier.
• Performs malware-focused clustering.
• Trains cluster-specific surrogate decision trees.
• Applies logical pruning to simplify the trees.
• Extracts symbolic malware rules from the pruned trees.
• Evaluates the fidelity of PaTEx to the trained MLP model.
• Generates interpretable tree-structured explanations.

---

PaTEx Experimental Settings

The main experimental settings are:

• Neural model: Multi-Layer Perceptron (MLP)
• Hidden layers: 128 and 64 neurons
• Activation: ReLU
• Output activation: Sigmoid
• Regularization: L1 regularization
• Optimizer: Adam
• Loss function: Binary cross-entropy
• Train/validation/test split: 60% / 20% / 20%
• Random seed: 42
• Clustering method: KMeans
• Clustering representation: binarized input feature space
• Number of clusters:
  • EMBER: K = 4
  • Dynamic: K = 4
  • Drebin/FakeInstaller: K = 2
• Local tree maximum depth: 5
• Minimum samples per leaf: 10

---

Fidelity Evaluation for LIME and SHAP

The fidelity evaluation for LIME and SHAP is implemented in:

scripts/fidelity_lime_shap.py

This script evaluates how faithfully LIME and SHAP approximate the trained MLP model.

The fidelity evaluation follows these steps:

1. For each test instance, generate a local neighborhood of perturbed samples.
2. Since the features are binary, perturbations are generated using bit flipping.
3. The trained MLP predicts probabilities for the perturbed samples.
4. LIME and SHAP generate local linear approximations.
5. The difference between the MLP predictions and the local approximation is measured using RMSE.
6. Fidelity is reported as:

Fidelity = 1 - RMSE

To run the fidelity evaluation:

python scripts/fidelity_lime_shap.py

Make sure that the dataset path and trained model path inside the script match the dataset/model you want to evaluate.

---

Key Features

PaTEx provides:

• Tree-structured explanations for neural malware classifiers.
• Local surrogate trees for different malware regions.
• Logical pruning to remove redundant decision branches.
• Symbolic rule extraction from pruned trees.
• Counterfactual reasoning through alternative tree paths.
• Fidelity comparison with common post-hoc explainers such as LIME and SHAP.

---

Citation

If you use this code, dataset preparation, or experimental framework, please cite:

@misc{anthony2026patex,
  title        = {PaTEx: Decision-aware partitioning for tree-structured explanations in neural malware classification},
  author       = {Anthony, Peter and Galadima, Kefas Rimanuske and Homola, Martin and Balogh, {\v{S}}tefan},
  year         = {2026},
  note         = {Manuscript submitted to the 30th International Conference on Knowledge-Based and Intelligent Information \& Engineering Systems (KES 2026)}
}

Reference:

Anthony, P., Galadima, K. R., Homola, M., & Balogh, Š. (2026). PaTEx: Decision-aware partitioning for tree-structured explanations in neural malware classification. Manuscript submitted to the 30th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems (KES 2026).

---

Associated Paper

This repository contains the implementation for:

PaTEx: Decision-Aware Partitioning for Tree-Structured Explanations in Neural Malware Classification

Manuscript submitted to KES 2026.