From 5f40a406f0152b340e7999481f07a6989612dbb4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mario=20Andr=C3=A9s=20Mu=C3=B1oz?= Date: Sat, 17 Oct 2026 23:49:34 +1100 Subject: [PATCH 1/3] Prepare v0.9.2 metadata and close stale documentation status --- CITATION.cff | 4 +- CLAUDE.md | 32 +++++------ Contents.m | 4 +- README.md | 2 + RELEASE_NOTES.md | 30 +++++++++-- doc/html/404.html | 3 +- doc/html/CLOISTER.html | 2 +- doc/html/Deprecated.html | 2 +- doc/html/FILTER.html | 2 +- doc/html/FunctionList.html | 2 +- doc/html/GettingStarted.html | 4 +- doc/html/INIT.html | 2 +- doc/html/ISAdefaults.html | 2 +- doc/html/ISAgetClassifierFcn.html | 2 +- doc/html/ISAmigrateModel.html | 2 +- doc/html/ISArecallView.html | 2 +- doc/html/ISAsubsetData.html | 2 +- doc/html/ISAvalidateOpts.html | 2 +- doc/html/InstanceSpace.html | 2 +- doc/html/InteractiveWalkthrough.html | 2 +- doc/html/MetadataFormat.html | 2 +- doc/html/MigratingLegacyModel.html | 2 +- doc/html/OptionsReference.html | 2 +- doc/html/PILOT.html | 2 +- doc/html/PILOTviewpoint.html | 2 +- doc/html/PRELIM.html | 2 +- doc/html/PYTHIA.html | 2 +- doc/html/SIFTED.html | 2 +- doc/html/TRACE.html | 2 +- doc/html/WhatsNew.html | 29 +++++----- doc/html/assets/search-index.js | 2 +- doc/html/buildIS.html | 2 +- doc/html/exploreIS.html | 2 +- doc/html/index.html | 3 +- doc/html/scriptcsv.html | 2 +- doc/html/scriptpng.html | 2 +- doc/html/scriptweb.html | 2 +- doc/src/GettingStarted.md | 2 +- doc/src/WhatsNew.md | 26 ++++++--- doc/src/index.md | 2 + docs/release-0.9.2.md | 81 ++++++++++++++++++++++++++++ 41 files changed, 200 insertions(+), 78 deletions(-) create mode 100644 docs/release-0.9.2.md diff --git a/CITATION.cff b/CITATION.cff index c85aeb8..b6dc46a 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -1,8 +1,8 @@ cff-version: 1.2.0 message: "If you use this software, please cite it as below." title: "Instance Space Analysis: A toolkit for the assessment of algorithmic power" -version: 0.9.1 -date-released: 2026-08-20 +version: 0.9.2 +date-released: 2026-10-17 doi: 10.5281/zenodo.4484107 authors: - family-names: "Muñoz" diff --git a/CLAUDE.md b/CLAUDE.md index e1ece32..884aed1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,26 +5,21 @@ - **Owner:** sole developer (Mario Andrés Muñoz) — no human reviewers; GitHub Copilot is the automated PR review gate. `master` has branch protection requiring changes go through a PR. - **Primary API:** the `InstanceSpace` class (`InstanceSpace.m`) — `build()`/`explore()`/`save()`/`InstanceSpace.load()`/`getResults()`/`plot()`. `buildIS.m`/`exploreIS.m` are thin backward-compatibility wrappers around it (kept for callers like the MATILDA web platform); don't add new logic to them directly. - **Completed refactor plan:** `docs/tech/isa_refactor_plan_v1.7.pdf` — historical source of truth for the v0.9.0 refactor (all ten phases). Now fully shipped; treat it as an archival reference, not a live backlog. -- **Current backlog:** GitHub Issues are the live source of truth. Every `v0.9.1`-milestone issue is now resolved and merged to `master` (see "Current state" below) — what's left is scoped for a **v0.9.2** milestone that doesn't exist as a GitHub milestone *object* yet (no milestone-management tool has been available in this environment across several sessions now; #30/#50/#52/#53 are the filed items, each needs its milestone field set by hand once v0.9.2 exists). +- **Current backlog:** GitHub Issues are the live source of truth. As of 17 October 2026, #63 (PILOT numerical restart sensitivity to floating-point jitter) is the only open MATLAB issue. Do not treat old release plans as pending work. - **Reference papers:** `docs/papers/Smith-Miles2023.pdf` (general ISA methodology), `docs/papers/Simpson2025.pdf` (ISA3, the 3D extension — cited only in `PILOT.m`, `PILOTviewpoint.m`, `TRACE.m`, and the 3D-output files in `output/`, not repo-wide). PILOT's origin (Muñoz et al. 2018), legacy TRACE's origin (Muñoz & Smith-Miles 2017), and FILTER's origin (Alipour et al. 2023) are cited in each file's own docstring `Reference:` block but aren't archived as PDFs here. ## Test harness Run from the repo root; `buildIS.m`/`exploreIS.m` print `EOF:SUCCESS` on success (errors abort before printing), and `test_integration.m` prints `EOF:SUCCESS`/`EOF:ERROR` at the end: example test_integration -`example.m` already runs the full `buildIS`+`exploreIS` pipeline internally on the bundled reference dataset — it's the one to start with. `test_integration.m` is a thin `matlab.unittest`-based runner (#39) over `tests/*.m` (`PipelineOptionsTest` — every classifier, tuning strategy, 2D/3D, PLS, viewpoint groups, parameterised; `ClassApiTest` — staged `build()`/`explore()`/save-load round-trips; `MigrationTest` — the full `ISAmigrateModel` legacy-migration table; `RegressionTest` — targeted fixed-bug regressions), with a `CodeCoveragePlugin` producing `coverage.xml`. New features and bug fixes get a test *method* under `tests/`, not a new block in `test_integration.m` itself. Good as a reference for how an option is meant to be used, not the place to start reading. Runs automatically on every push/PR via CI (#34, `.github/workflows/tests.yml`); pinned to `release: R2025a`, not `latest` (deliberate — see the `matlab-ci-testing` skill below for why). +`example.m` already runs the full `buildIS`+`exploreIS` pipeline internally on the bundled reference dataset — it's the one to start with. `test_integration.m` is a thin `matlab.unittest`-based runner (#39) over `tests/*.m` (`PipelineOptionsTest` — every classifier, tuning strategy, 2D/3D, PLS, viewpoint groups, parameterised; `ClassApiTest` — staged `build()`/`explore()`/save-load round-trips; `MigrationTest` — the full `ISAmigrateModel` legacy-migration table; `RegressionTest` — targeted fixed-bug regressions), with a `CodeCoveragePlugin` producing `coverage.xml`. New features and bug fixes get a test *method* under `tests/`, not a new block in `test_integration.m` itself. Good as a reference for how an option is meant to be used, not the place to start reading. Runs automatically on every push/PR via CI (#34, `.github/workflows/tests.yml`); pinned to `release: R2025a`, not `latest` (deliberate — this is the documented minimum supported MATLAB release). -## Current state (v0.9.0 and v0.9.1 both shipped to `master`; v0.9.2 not started) -- **v0.9.0** (July 2026): all ten refactor-plan phases — rewritten PRELIM/SIFTED/PILOT/PYTHIA/TRACE pipeline, the `InstanceSpace` class API, `ISAmigrateModel`/`ISAvalidateOpts`, 3D projection/viewpoint support, and a licence change from GPL v3 to PolyForm Noncommercial 1.0.0. -- **v0.9.1** (August 2026): engineering-quality, architecture, and infrastructure follow-ups — no algorithmic changes, no new user-facing options, per that milestone's own scope rule. Landed on `master` via three merged PRs; **`RELEASE_NOTES.md`'s v0.9.1 section is the full changelog** (organised into New functionality/Better engineering/Bug fixes) — this file only tracks what a future session specifically needs to know, not a duplicate list: - - **#51**: the bulk of the engineering work (INIT.m/PRELIM train-eval dual mode, per-stage contract validation and `onStage` inspection callbacks, CI, `matlab.unittest` migration, several confirmed correctness fixes). Went through **five rounds** of Copilot review; each round found genuine bugs — including, more than once, in a fix landed earlier in the *same* PR (e.g. an added safety warning that didn't actually fire; a guard that proved two values matched each other but not that either was still correct). If touching any of that code again, don't assume it's settled just because it's merged — check the PR's review-comment history first. - - **#54**: added the `matlab-ci-testing` Claude Code skill (`.claude/skills/matlab-ci-testing/`) — CI setup, `matlab.unittest` architecture, MATLAB-specific test-writing patterns, distilled directly from the #51 review cycle. Also added Codecov and GitHub-downloads badges to `README.md`. - - **#55**: `README.md` had accumulated internal development-process narration (issue numbers, "confirmed via CI", references to a prior implementation, an internal spec-section reference) — removed; see "Documentation conventions" below for the standing rule this established. Same PR also fixed a repo-wide docstring/doc-formatting audit: README's ~208 inline-code spans were triple-backtick instead of single; `core/PYTHIA.m` was the one `core/` pipeline file missing the `Inputs`/`Outputs` docstring section its siblings all have; a couple of files had inconsistent `%---` divider-comment widths. - - **Not yet done, not code**: the Codecov badge shows no data until the repo owner links the repo at codecov.io and adds a `CODECOV_TOKEN` GitHub Actions secret — the CI-side upload step is already in place (`fail_ci_if_error: false`, so its absence can't break CI). -- **v0.9.2 backlog** (filed, none started, no milestone object yet — see "Current backlog" above): #30 (PILOT projection-orientation canonicalisation), #50 (CLOISTER's convex hull is 2D-only even for a 3D projection), #52 (`traceOneRegion` doesn't trace a hole's boundary cycle), #53 (a MATLAB-toolbox-style reference documentation site — `doc InstanceSpace` Help-browser integration via `info.xml`/`helptoc.xml` plus GitHub Pages, ~29 pages; full scope is in the issue body). Unlike v0.9.1, several of these (#30/#50/#52) are explicitly algorithmic and #53 is a new deliverable — don't assume v0.9.1's "no algorithmic changes" rule carries over to v0.9.2 by default; check with the user. - - **Closed as not planned**: a cross-repo reference-fixture export tool for the Python port (`pyInstanceSpace`, #35) — per a standing preference, anything dealing with the Python port is handled from that repo, not this one. - - **Closed as resolved, no code change**: #45 (verify `PRELIM.m`'s `boxcox()` call) — confirmed it resolves to the Financial Toolbox's `boxcox.m`, the same dependency CI already installs for this call site; combined with the already-verified positive-shift guard, no gap was found. -- **Repository layout**: `core/` (INIT, PRELIM, SIFTED, PILOT, PILOTviewpoint, CLOISTER, PYTHIA, TRACE, TRACE_legacy, FILTER), `output/` (scriptcsv, scriptpng, scriptweb, scriptfcn, ISArecallView), `utils/` (ISAdefaults, ISAvalidateOpts, ISAgetClassifierFcn, ISAmigrateModel, ISAsubsetData), `deprecated/` (PYTHIA2, PYTHIAtest, SIFTED2 — warn-and-forward shims kept for backward compatibility only; don't build new code on these), `.claude/skills/` (`instance-space-analysis` — ISA methodology plus this repo's operational reference; `matlab-ci-testing` — MATLAB CI/`matlab.unittest` setup and debugging, not ISA-specific, reusable in other MATLAB repos). +## Current state (v0.9.2 release preparation) +- v0.9.0 and v0.9.1 are published. The v0.9.2 changes are merged to `master` through #60, #62, #64, #65 and #66. Release metadata and documentation are being prepared separately from publishing a tag/release; use `docs/release-0.9.2.md` for the release checklist. +- Closed/completed: #30 (projection orientation), #50 (3D CLOISTER), #52 (hole boundary cycles), #53 (reference site), #57 (JUnit/Codecov reporting), #58 (unobserved test outcomes), #59 (Oracle probability), #61 (SVM calibration seeding). These are not pending merges. `RELEASE_NOTES.md` is the complete user-facing change record. +- Current contracts include independent INIT snapshots, preprocessing recomputation after portfolio pruning, SIFTED cache isolation and replay inputs, fitted PLS centering, training-only selection policy, honest CV/test summaries, stage provenance, persisted TRACE distance tolerance and serial Octave support. Review `doc/src/`, `docs/octave-compatibility.md` and `review/2026-09-30/FINAL_REPORT.md` before changing them. +- Repository layout: `core/` stages, `output/` exporters, `utils/` utilities and `+isacompat` runtime adapters, `deprecated/` warn-and-forward shims, `tests/` MATLAB tests and `tests/portable/` shared contracts. `.claude/skills/instance-space-analysis/` contains methodology and repository usage guidance. The older referenced `matlab-ci-testing` skill is not present in this checkout. +- CI targets MATLAB R2025a; local release validation uses R2026a Update 5. The full serial Octave workflow is validated with 11.3.0, Statistics 2.0.0 and Datatypes 1.5.0. Core-only checks on older Octave do not imply full workflow support. ## Workflow - Work on feature branches; merge to `master` after Copilot review. `master`'s branch protection requires a PR — don't push directly to it. @@ -45,21 +40,22 @@ Run from the repo root; `buildIS.m`/`exploreIS.m` print `EOF:SUCCESS` on success - **Inline code in Markdown docs uses single backticks** (`` `like this` ``), not triple backticks. `README.md` had drifted to ~208 triple-backtick inline spans before #55's cleanup; `CONTRIBUTING.md`/`SECURITY.md`/`CLAUDE.md`/`RELEASE_NOTES.md` were already consistent. Triple backticks are still correct for actual fenced code blocks. - **Every `.m` file's docstring** starts with the same copyright/SPDX-license header block (copy an existing file's, e.g. `core/PILOT.m`) followed by a `Reference:` section citing only the paper(s) that specific file implements. `core/` pipeline-stage files (`PRELIM`/`SIFTED`/`PILOT`/`PILOTviewpoint`/`CLOISTER`/`PYTHIA`/`TRACE`/`TRACE_legacy`/`FILTER`/`INIT`) and `InstanceSpace.m` additionally use a structured `Inputs`/`Outputs` section (see any of them for the exact format); `utils/`/`output/`/`deprecated/`/`buildIS.m`/`exploreIS.m` intentionally use lighter narrative-prose docstrings instead — that split (algorithm implementations vs. supporting utilities) is deliberate, not an inconsistency to "fix." -## Known deferred items (tracked, not stale — check the linked issue before touching) -- **#30** (PILOT projection-orientation canonicalisation) is the remaining open algorithmic item. -- **Resolved on `claude/docs-testing-improvements-amt87s` (v0.9.2 work, pending merge):** #50 (CLOISTER now builds a 3D hull with `ZedgeFaces`/`ZecorrFaces`; `plot('boundary')`/`scriptpng` draw it; a 3D model built before the fix still raises `ISA:InstanceSpace:boundaryNot3D`), #52 (`traceOneRegion` traces every boundary cycle, NaN-separated; the `boundaryHoleOmitted` warning is gone), #58 (PYTHIA eval scores only rows with observed `Y`; unobserved algorithms get `NaN` metrics), #59 (Oracle `Probability_of_good` = `mean(any(Ybin,2))`), #57 (JUnit report + Codecov Test Analytics). +## Known limitations +- **#63 remains open:** numerical PILOT restart selection can amplify BLAS/thread-level floating-point jitter. Fixed-seed fixture repeats in a pinned environment do not establish cross-environment reproducibility or resolve this issue. +- `opts.pilot.topoWeight` remains disabled. Do not silently activate it as documentation cleanup. +- Octave supports only the bounded serial workflow documented in `docs/octave-compatibility.md`; its model archives are runtime-specific. The TRACE boundary policy agrees with Python, but arbitrary numerical/geometry identity is not promised. ## Documentation site (#53) - Sources: `doc/src/*.md`; generator: `doc/generate.py` (python-markdown, pinned in `doc/requirements.txt`); output: `doc/html/` (committed — the MATLAB Help browser reads it via `info.xml` from a plain clone). - The `TOC` list in `doc/generate.py` is the single source for the sidebar, breadcrumbs, `helptoc.xml`, the generated *All Functions* page and search. A page not in `TOC` fails the build. - Page conventions (Syntax/Description/Examples/Input Arguments/Output Arguments/Version History/See Also, ``### `name` — Short description`` argument headings, `` for bare `opts.x` links) are documented in the generator's docstring. -- `.github/workflows/docs-pages.yml`: `generate.py --check` + `.github/scripts/check_doc_links.py` on PRs; deploys `doc/html` to GitHub Pages from `master`. Pages must be enabled once in the repo settings (Source: GitHub Actions) — not something this environment can do. +- `.github/workflows/docs-pages.yml`: `generate.py --check` + `.github/scripts/check_doc_links.py` on PRs; deploys `doc/html` to GitHub Pages from `master`. Deployment uses the repository's GitHub Pages settings (Source: GitHub Actions). - When a function's behaviour or signature changes, update its `doc/src` page (and a Version History entry), then regenerate. **LIBSVM MEX-files removed (#29).** `svmpredict.mexw64`/`svmtrain.mexw64` were precompiled binaries with no corresponding source in the tree — removed entirely rather than kept-with-provenance, since LIBSVM is already fully deprecated for new runs. `PYTHIA`'s eval mode now raises a clear `ISA:PYTHIA:noLibsvm` error (naming the algorithm and pointing to the official LIBSVM project) if it ever needs to dispatch to a legacy LIBSVM-format classifier struct and `svmpredict` isn't on the path, instead of MATLAB's generic undefined-function error. Don't re-add these binaries; if LIBSVM support is ever needed again, get it from upstream rather than re-committing a precompiled MEX-file with no visible build source. ## Cross-repo note -The user has asked (in this session) about a repository called `claude-skills` (owner `andremun`) twice — once too ambiguously to act on, once explicitly as `andremun/claude-skills`, at which point cloning it failed: this session's GitHub access (both the MCP tools and git-level credentials) is hard-scoped to `andremun/InstanceSpace` only, confirmed by a 403 on an unauthenticated API probe and `git clone` refusing to prompt for credentials. If a future session has broader repo access and this comes up again, that's the blocker that was hit — not a transient error worth silently retrying. +`andremun/pyInstanceSpace` is the Python implementation. Keep Python-specific changes and fixture tooling there. Python issue #367 tracks improving its documentation website toward this reference site's navigation and presentation. Shared boundary policy does not imply full numerical parity. Check current connector permissions rather than relying on historical access failures. ## `.gitignore` notes - `docs/**/*.pdf` — tracked (negation rule), recursive under `docs/`. diff --git a/Contents.m b/Contents.m index dbbecd8..3931aa0 100644 --- a/Contents.m +++ b/Contents.m @@ -1,7 +1,8 @@ % Instance Space Analysis Toolkit -% Version 0.9.1 (R2025a) 20-Aug-2026 +% Version 0.9.2 (R2025a) 17-Oct-2026 % % Standalone pipeline functions +% INIT - Read metadata and validate training/evaluation inputs % PRELIM - Data preprocessing (outlier bounding, normalisation) % SIFTED - Feature selection (correlation + GA clustering) % PILOT - 2D/3D projection (PBLDR algorithm) @@ -21,6 +22,7 @@ % exploreIS - Thin wrapper around InstanceSpace.explore() % % Utilities +% ISAfootprintContains - Closed-boundary footprint membership with distance tolerance % ISAdefaults - Fill in missing opts fields with default values % ISAvalidateOpts - Validate user-supplied opts fields % ISAsubsetData - Subset rows (and optionally feature columns) of a data struct diff --git a/README.md b/README.md index 66f9590..1986c6b 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,8 @@ Or if you specifically use [MATILDA](http://matilda.unimelb.edu.au/matilda/), pl The reference documentation is at ****: a page for every function and for the `InstanceSpace` class, with examples, a getting-started guide, a stage-by-stage walkthrough, the metadata file format, and the full options reference. The same pages open inside MATLAB: with the repository root on the path, open the Help browser and choose **Supplemental Software → Instance Space Analysis Toolbox**. +For v0.9.2 changes, upgrade guidance and remaining limitations, see [What's New](https://andremun.github.io/InstanceSpace/WhatsNew.html) and the [release notes](RELEASE_NOTES.md). Rebuild existing experiments from their original metadata to obtain the corrected fitted results. + ## Installation Instructions The main requirement for the software to run is to have MATLAB R2025a or later, with the [Global Optimization](https://au.mathworks.com/help/gads/index.html), [Parallel Computing](https://www.mathworks.com/products/parallel-computing.html), [Optimization](https://au.mathworks.com/products/optimization.html), [Statistics and Machine Learning](https://au.mathworks.com/help/stats/index.html), and [Financial](https://au.mathworks.com/products/finance.html) toolboxes installed. The LIBSVM MEX-files (`svmpredict`/`svmtrain`), used for legacy models, are **not bundled with this repository** (no build source is available for them). They're only relevant for evaluating a pre-v0.9.0 model whose classifiers `ISAmigrateModel` couldn't retrain (missing original training data); `ISAmigrateModel` prefers retraining from scratch with the current registry whenever the training data is available, which needs no LIBSVM dependency. If you do hit that path, obtain LIBSVM yourself from [the official project](https://www.csie.ntu.edu.tw/~cjlin/libsvm/) and add its MEX-files to the MATLAB path — `PYTHIA`'s eval mode raises a clear `ISA:PYTHIA:noLibsvm` error naming the missing dependency if you don't. diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 3674cc4..07bfc42 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,6 +1,6 @@ -# Instance Space Analysis Toolkit — v0.9.2 (in development) +# Instance Space Analysis Toolkit — v0.9.2 -Bug fixes from the v0.9.2 backlog, serial GNU Octave support, a reference documentation site, and test infrastructure. It supports **MATLAB R2025a or later** and the bounded Octave workflow described below. +Corrected training, exploration and reporting contracts, serial GNU Octave support, an independent INIT stage, a reference documentation site, and test infrastructure. It supports **MATLAB R2025a or later** and the bounded Octave workflow described below. --- @@ -8,6 +8,8 @@ Bug fixes from the v0.9.2 backlog, serial GNU Octave support, a reference docume **Independent INIT stage.** `InstanceSpace` now exposes `init` before `prelim`, including stage callbacks, completion tracking, option validation and saved input data. PRELIM reruns reuse `model.init.data`; explicitly rerunning INIT reloads metadata and invalidates downstream results. Existing staged calls beginning with PRELIM initialize automatically when the snapshot is absent, and legacy models remain usable for exploration. Full builds retain their numerical behaviour. +**Explicit footprint boundary policy.** TRACE uses closed boundaries in 2D and 3D. `opts.trace.boundaryTolerance` is a finite nonnegative Euclidean distance in projected coordinates; its default is **0**, with no implicit numerical margin. The fitted value is saved in the model and reused during exploration and legacy contradiction counting. Older models without a saved tolerance use zero. MATLAB and Python share this policy; matching policies do not guarantee identical floating-point geometry. + **Serial GNU Octave workflow.** Octave supports 2D/3D build → save/load → explore with normalization, feature selection, analytic/numerical/SIMPLS projection, KNN classifiers with none/Sobol tuning, TRACE3 footprints, CSV and PNG. The full workflow requires Octave **11.1 or later**, Datatypes **1.5.0** and Statistics **2.0.0**; validation uses Octave **11.3.0**. Bayesian tuning, parallel execution, other classifiers, legacy TRACE, FIG files and web palette export remain unsupported. 3D PNG requires Qt and a display or virtual display; `outputs.fig` defaults to false in Octave. Build and explore reject unsupported output options before processing or replacing outputs. Octave saves versioned MAT v7 archives that preserve trained KNN models without retraining; these classifier archives are runtime-specific, while MATLAB retains native MAT v7.3 persistence. Schema 2 protects ordinary structs from collisions with internal type markers and reads existing schema-1 archives; ambiguous marker-shaped user structs in old archives cannot be distinguished retrospectively. CSV is the supported data interchange format. Equal seeds do not promise identical optimizer, Sobol or geometry results across runtimes. See the [Octave compatibility guide](docs/octave-compatibility.md) for setup and the complete support boundary. @@ -29,11 +31,31 @@ Octave saves versioned MAT v7 archives that preserve trained KNN models without ## Bug fixes - **PYTHIA evaluation scored algorithms against data that was never observed.** In `PYTHIA`'s evaluation mode, a trained algorithm absent from `metadata_test.csv` has an all-`NaN` performance column, which `PRELIM` turns into all-false labels; accuracy, precision and recall were then computed against those labels. Only instances with observed performance are now scored, and an algorithm with none reports `NaN`. Accuracy is divided by the number of scored instances, and the summary's per-algorithm `Probability_of_good` is also computed over observed instances only (blank when there are none) ([#58](https://github.com/andremun/InstanceSpace/issues/58)). -- **Oracle probability of good performance was always 1.** The Oracle row of `PYTHIA`'s summary now reports `mean(any(Ybin,2))`, the fraction of instances on which any algorithm is good. This is below 1 when `opts.perf.AbsPerf = true` and no algorithm meets the threshold on some instances ([#59](https://github.com/andremun/InstanceSpace/issues/59)). +- **Oracle probability of good performance was always 1.** The Oracle row of `PYTHIA`'s summary now reports the fraction of instances with any observed performance on which at least one observed algorithm is good (equivalent to `mean(any(Ybin,2))` when all outcomes are observed). Rows with no observations are excluded; no observations gives `NaN`. This is below 1 when `opts.perf.AbsPerf = true` and no algorithm meets the threshold on some instances ([#59](https://github.com/andremun/InstanceSpace/issues/59)). - **`TRACE` failed when called without PYTHIA predictions.** `TRACE`'s documentation allows `Yhat = []`, falling back to the true labels, but the footprint loop indexed `Yhat(:,i)` inside a `parfor`, which slices it for every iteration and so failed with an out-of-bounds index. The pipeline itself never passed `[]`, so only direct callers were affected. Predictions are now split per algorithm before the loop. - **`ISArecallView` on a figure with no axes** failed inside `ismember` instead of raising its own `ISA:ISArecallView:noAxes` error. - **Footprint holes were dropped from the traced boundary.** `traceOneRegion` followed only the boundary cycle that contained its start vertex, so the boundary of a hole inside a footprint region was omitted (with the `ISA:scriptfcn:boundaryHoleOmitted` warning). Every cycle is now traced, and cycles are separated by a `NaN` row as regions already were; the warning is removed ([#52](https://github.com/andremun/InstanceSpace/issues/52)). -- **[Behaviour-changing] `PYTHIA`'s SVM posterior probabilities were not reproducible.** In `fitOneClassifier`'s `'svm'` case, `fitSVMPosterior`'s own sigmoid-calibration draws depended on however many random numbers `fitcsvm`'s SMO solver had just consumed, so `Pr0hat`/`Pr0sub` for `opts.classifier = 'svm'` could differ between two runs with the same `opts.seed`. `fitOneClassifier` now reseeds to the same per-algorithm/per-fold seed immediately before `fitSVMPosterior`, mirroring the #41 fix for PILOT/SIFTED/PILOTviewpoint. Existing saved SVM models are unaffected; rerunning `PYTHIA` training with `opts.classifier = 'svm'` will change `Pr0hat`/`Pr0sub` (not `Yhat`/`Ysub`, which threshold at 0.5 and were already reproducible) ([#61](https://github.com/andremun/InstanceSpace/issues/61)). +- **[Behaviour-changing] `PYTHIA`'s SVM posterior probabilities were not reproducible.** In `fitOneClassifier`'s `'svm'` case, `fitSVMPosterior`'s own sigmoid-calibration draws depended on however many random numbers `fitcsvm`'s SMO solver had just consumed, so `Pr0hat`/`Pr0sub` for `opts.classifier = 'svm'` could differ between two runs with the same `opts.seed`. `fitOneClassifier` now reseeds to the same per-algorithm/per-fold seed immediately before `fitSVMPosterior`, mirroring the #41 fix for PILOT/SIFTED/PILOTviewpoint. Existing saved SVM models are unaffected; rerunning `PYTHIA` training with `opts.classifier = 'svm'` will change `Pr0hat`/`Pr0sub` and can also affect predicted labels, tuning decisions and downstream footprints through the calibrated classifier ([#61](https://github.com/andremun/InstanceSpace/issues/61)). + +## Further correctness fixes + +- **Training data and portfolio consistency.** After algorithms that are never good are pruned, PRELIM recomputes labels, winners, transforms and portfolio indices from the retained raw outcomes. Constant columns use a unit scale. Training inputs remaining incomplete after ingestion filtering raise errors identifying the instances; test outcomes can remain unobserved. Raw performance must be nonnegative, with zero allowed; the original relative ratio formulas and documented zero substitution are retained. +- **Repeatable feature selection and projection.** SIFTED resets its fitness caches between runs, including workers, and retains pre-selection inputs for reruns. PLS persists its fitted feature mean and applies it during exploration and CLOISTER projection. Analytic PILOT solves least squares directly, and standalone numerical fallbacks receive complete defaults. Seed bounds and per-stage seeding are validated. +- **Training/test separation and honest summaries.** Recommendations use saved training fallback choices and precision weights, regardless of test outcomes. Selector summaries use out-of-fold selections for training and explicitly labelled test metrics for exploration. Selector recall counts disjoint successes and missed opportunities; cost-sensitive weights use per-instance regret. Invalid classifier CV searches fail explicitly. Score metadata distinguishes posterior probabilities from raw scores and mixed fold outputs. These CV summaries remain conditional on fitted preprocessing, feature selection, projection and tuning; they are not outer cross-validation of the complete pipeline. +- **Staged builds and persistence.** Options consumed by each stage are saved with provenance. Partial models can be saved, loaded and resumed; incompatible option edits cannot silently retain stale fitted artifacts. Training feature exclusions are replayed during exploration. Builds respect serial settings and close only parallel pools they created. +- **Footprint evaluation and exports.** Exploration reuses trained space geometry and accepts single, repeated or lower-dimensional test batches. TRACE records whether the purity target was achieved and why its search stopped; a retained candidate below the target is not labelled accepted. CSV exports include 3D vertices and triangle faces, explicit empty geometry and `geometry_manifest.json`. Export replaces obsolete toolkit-owned geometry and plot files; user files must use other names. Viewpoint generation and recall use the same camera convention and equal data-axis scales. + +## Upgrading from v0.9.1 + +Keep a backup of existing models and result directories. Rebuild experiments from the original metadata to obtain the corrected preprocessing, feature selection, PLS coordinates, classifiers, selector summaries and footprints; loading an old model does not retroactively apply training fixes. Rebuild CLOISTER to obtain a valid stored 3D boundary. Models predating the saved pre-SIFTED snapshot must rebuild preprocessing before rerunning feature selection. `ISAmigrateModel` remains available for pre-v0.9.0 layouts, but migration is not a substitute for rebuilding corrected experiments. + +Options now reject ambiguous subsetting modes, invalid index files, unsupported seed values and negative raw performance earlier. Consumers of exported geometry should read the manifest, preserve NaN-separated 2D rings and use one-based 3D face indices. Saved Octave classifiers are runtime-specific; use CSV for data interchange. + +## Known limitations + +Numerical PILOT restart selection can amplify small floating-point differences across BLAS implementations or thread settings ([#63](https://github.com/andremun/InstanceSpace/issues/63)). A fixed seed is not a guarantee of identical numerical projections across environments. Keep runtime, libraries and thread settings fixed for reproducibility-sensitive work. This issue remains open; the closed projection-orientation issue #30 does not establish that this separate problem is solved. + +`opts.pilot.topoWeight` remains reserved and disabled. Octave support is limited to the workflow above. Geometric degeneracies and optimizer differences can produce different MATLAB/Python/Octave results even when the same boundary policy is used. --- diff --git a/doc/html/404.html b/doc/html/404.html index 15b3af1..5babfd0 100644 --- a/doc/html/404.html +++ b/doc/html/404.html @@ -13,7 +13,7 @@
Instance Space Analysis Toolbox - v0.9.1 + v0.9.2

Required products: MATLAB R2025a or later with the Statistics and Machine Learning, Optimization, Global Optimization, Parallel Computing, and Financial toolboxes.

+

A bounded serial GNU Octave workflow is also supported; see What's New for supported classifiers, outputs and persistence limits.

With the repository root on the path, this documentation also appears in the MATLAB Help browser under Supplemental Software.

References

  • Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys, 55(12), Article 255. https://doi.org/10.1145/3572895
    • diff --git a/doc/html/CLOISTER.html b/doc/html/CLOISTER.html index f3a9764..f54c00e 100644 --- a/doc/html/CLOISTER.html +++ b/doc/html/CLOISTER.html @@ -13,7 +13,7 @@
    Instance Space Analysis Toolbox - v0.9.1 + v0.9.2

Build the Instance Space

obj = InstanceSpace(rootdir, opts);
 obj = obj.build();
-

build runs the whole pipeline: PRELIM labels and normalises the data, SIFTED selects features, PILOT projects the instances to 2D, CLOISTER estimates the boundary of the space, PYTHIA trains a classifier per algorithm, and TRACE finds the footprints. It then saves model.mat and writes CSV and PNG files to rootdir. It takes a few minutes, most of it in SIFTED and PYTHIA.

+

build runs the whole pipeline: INIT reads and validates the metadata, PRELIM labels and normalises the data, SIFTED selects features, PILOT projects the instances to 2D, CLOISTER estimates the boundary of the space, PYTHIA trains a classifier per algorithm, and TRACE finds the footprints. It then saves model.mat and writes CSV and PNG files to rootdir. It takes a few minutes, most of it in SIFTED and PYTHIA.

Look at the Results

figure, obj.plot('portfolio')        % best algorithm on each instance
 figure, obj.plot('footprint', 6)     % footprint of the 6th algorithm
 disp(obj.model.trace.summary)        % footprint area, density and purity
diff --git a/doc/html/INIT.html b/doc/html/INIT.html
index 01c97c6..bf07bd4 100644
--- a/doc/html/INIT.html
+++ b/doc/html/INIT.html
@@ -13,7 +13,7 @@
 

   
   Instance Space Analysis Toolbox
-  v0.9.1
+  v0.9.2

Required products: MATLAB R2025a or later with the Statistics and Machine Learning, Optimization, Global Optimization, Parallel Computing, and Financial toolboxes.

+

A bounded serial GNU Octave workflow is also supported; see What's New for supported classifiers, outputs and persistence limits.

With the repository root on the path, this documentation also appears in the MATLAB Help browser under Supplemental Software.

References

  • Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys, 55(12), Article 255. https://doi.org/10.1145/3572895
    • diff --git a/doc/html/scriptcsv.html b/doc/html/scriptcsv.html index 21d38fd..c4dcb86 100644 --- a/doc/html/scriptcsv.html +++ b/doc/html/scriptcsv.html @@ -13,7 +13,7 @@
    Instance Space Analysis Toolbox - v0.9.1 + v0.9.2

Algorithms

For the standard method, PILOT solves

min  ||X - Z*B'||^2 + alpha*||Y - Z*C||^2    subject to  Z = X*A'
-

where X and Y are the preprocessed feature and performance matrices. The analytic solution takes the leading dims eigenvectors of [X Y]'*[X Y], with the Y block weighted by sqrt(alpha) (Muñoz et al., 2018). The numerical solution runs BFGS (fminunc) from opts.ntries random starts and keeps the trial with the highest correlation between the pairwise distances of the instances before and after projection, so the chosen space also preserves the topology of the data.

+

where X and Y are the preprocessed feature and performance matrices. The analytic solution takes the leading dims eigenvectors of [X Y]'*[X Y], with the Y block weighted by sqrt(alpha) (Muñoz et al., 2018). The numerical solution runs BFGS (fminunc) from opts.ntries random starts and keeps the trial with the highest correlation between the pairwise distances of the instances before and after projection, so the chosen space also preserves the topology of the data.

+

The projection has no automatic canonical orientation: equivalent spaces can appear rotated or reflected. A fixed seed also does not guarantee the same numerical restart winner across BLAS implementations or thread settings. Keep the numerical environment fixed when reproducibility is required.

GNU Octave

Octave supports serial analytic/numerical standard projection and centered SIMPLS in 2D/3D, including optimized 3D viewpoints. Native MATLAB backends are retained. Optimizer trajectories may differ; parallel execution is rejected under Octave.

Version History

v0.9.2 — Projection corrections and Octave support

Added serial Octave analytic, numerical and SIMPLS projections in 2D/3D.

PLS stores out.Xmean. Apply its projection as (X-out.Xmean)*out.A' for new instances. Standard PILOT projections do not subtract a mean.

diff --git a/doc/html/WhatsNew.html b/doc/html/WhatsNew.html index 6fd75e1..66b18cd 100644 --- a/doc/html/WhatsNew.html +++ b/doc/html/WhatsNew.html @@ -93,7 +93,7 @@

Correctness and Reporting

Upgrading and Limitations

Rebuild experiments from the original metadata to obtain corrected results. Loading an existing model does not apply fixes to its fitted artifacts. Back up models and outputs first; rebuild CLOISTER for older 3D models and preprocessing before rerunning SIFTED if the saved feature-selection input is absent. Raw performance must be nonnegative (zero is allowed), and incomplete training data remaining after ingestion filtering is rejected. See Migrating a Legacy Model for pre-v0.9.0 layouts.

-

Classifier CV summaries are conditional on the fitted preprocessing, feature selection, projection and tuning. They are not an outer-CV estimate of the full pipeline. Numerical PILOT restart selection can amplify floating-point differences across BLAS implementations and thread settings, so a seed alone does not promise identical projections across environments. opts.pilot.topoWeight remains reserved and disabled.

+

Classifier CV summaries are conditional on the fitted preprocessing, feature selection, projection and tuning. They are not an outer-CV estimate of the full pipeline. Numerical PILOT restart selection can amplify floating-point differences across BLAS implementations and thread settings, so a seed alone does not promise identical projections across environments. Projection orientation is not canonicalized automatically. opts.pilot.topoWeight remains reserved and disabled.

Documentation

Reference documentation is available in the MATLAB Help browser and on the web, with examples for every function. The version header, release notes and citation metadata now identify v0.9.2.

v0.9.1

Engineering release; results of the pipeline are unchanged.

diff --git a/doc/html/assets/search-index.js b/doc/html/assets/search-index.js index 1989380..318ab72 100644 --- a/doc/html/assets/search-index.js +++ b/doc/html/assets/search-index.js @@ -1,2 +1,2 @@ // Generated by doc/generate.py. Do not edit. -window.ISA_SEARCH = [{"t":"CLOISTER","u":"CLOISTER.html","p":"Estimate the boundary of the instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","GNU Octave","Version History","v0.9.2 — 3D boundaries and Octave support","v0.9.0 — Feature limit","References","See Also"],"x":"Syntax out = CLOISTER(X,A,opts) out = CLOISTER(X,A,opts,Xmean) Description out = CLOISTER(X,A,opts) estimates the region of the instance space where instances could exist, using correlations between features (Correlated Limits of the Instance Space's Theoretical or Experimental Regions). It takes every corner of the box bounded by the minimum and maximum of each feature, discards the corners that contradict a strong, significant correlation between two features, and projects the remaining corners with the PILOT matrix A . The convex hull of the projected corners is the estimated boundary. out = CLOISTER(X,A,opts,Xmean) subtracts the fitted feature mean before projection. For PLS, pass model.pilot.Xmean to keep the boundary in the same coordinates as model.pilot.Z . The three-argument form uses a zero mean. The same centring applies to the projected-instance hull used when the feature-count limit is exceeded. Parts of the boundary far from any instance show where new test instances would extend the benchmark. For a 2D projection the boundary is a closed polygon. For a 3D projection it is a triangulated convex surface; when the projected corners all lie on one plane (for example, a 3D projection of only two features), it is the flat polygon, triangulated. Examples Expand all Plot the boundary of a 2D instance space Copy Plot the boundary of a 3D instance space Copy obj.plot('boundary') draws the same figure over the instances. Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed, selected features, one row per instance. May contain NaN . A — Projection matrix dims -by- nfeats numeric matrix model.pilot.A . opts — Boundary options structure Normally obj.opts.cloister . opts.pval — Significance level 0.05 (default) | scalar in (0, 1) Correlations with a larger p-value are ignored. opts.corrThreshold — Correlation threshold 0.70 (default) | scalar in [0, 1] A corner is discarded when it contradicts the sign of a correlation stronger than this. opts.maxFeatures — Feature limit 20 (default) | positive integer The number of corners grows as 2^nfeats . With more features than this, CLOISTER warns ( ISA:CLOISTER:tooManyFeatures ) and returns the convex hull of the projected instances instead. Xmean — Fitted feature mean 1-by- nfeats numeric row vector | default: zeros(1,nfeats) The mean fitted during PILOT training. Pass model.pilot.Xmean for PLS. Corners and fallback instances are projected as (X-Xmean)*A' ; CLOISTER does not estimate a new mean. Omit the argument for an uncentred projection. Output Arguments Expand all out — Boundary structure out.Zedge — Boundary of all corners 2D: closed polygon, first vertex repeated last. 3D: vertices of the convex hull. out.Zecorr — Boundary of the correlation-consistent corners Same format as Zedge . Equals Zedge when no corner is discarded, when too many corners are discarded to form a hull, or when opts.maxFeatures is exceeded. out.ZedgeFaces , out.ZecorrFaces — Hull triangulations 3D only: nfaces -by-3 indices into the rows of Zedge and Zecorr . Empty in 2D. Tips CLOISTER expects mean-centred features. InstanceSpace warns ( ISA:InstanceSpace:cloisterNotMeanCentred ) when opts.auto.preproc or opts.norm.flag is false . scriptcsv writes Zedge to bounds.csv and Zecorr to bounds_prunned.csv . If every projected corner lies on one line (a degenerate projection matrix or features), there is no region to bound and CLOISTER raises ISA:CLOISTER:degenerateBoundary . GNU Octave Octave supports 2D/3D hulls using namespaced Pearson-significance and NaN-aware extrema adapters. The Pearson adapter requires at least three observations. Degenerate/coplanar cases and PLS centering are covered by portable tests. Version History v0.9.2 — 3D boundaries and Octave support Added serial Octave support for Pearson significance and 2D/3D hulls. An optional fourth argument, Xmean , applies the fitted PLS feature centring during projection. The three-argument call uses a zero mean. Corner enumeration"},{"t":"Deprecated Functions","u":"Deprecated.html","p":"Functions kept only for compatibility with older code","h":["Updating Your Code","See Also"],"x":"These functions still work but print a warning and forward to their replacement. Do not use them in new code. Function Replacement Deprecated in PYTHIA2(Z,Y,Ybin,Ybest,algolabels,opts) PYTHIA with opts.classifier = 'knn' v0.9.0 PYTHIAtest(model,Z,Y,Ybin,Ybest,algolabels) PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,model) (evaluation mode) v0.9.0 SIFTED2(X,Y,Ybin,featlabels,opts) SIFTED (renamed) v0.9.0 Updating Your Code Copy See Also PYTHIA | SIFTED | What's New"},{"t":"FILTER","u":"FILTER.html","p":"Remove near-duplicate instances to even out the instance density","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.2 — Bounded neighbour queries","v0.9.0 — Uniformity output","References","See Also"],"x":"Syntax [subsetIndex,isDissimilar,isVISA,unif] = FILTER(X,Y,Ybin,opts) Description [subsetIndex,isDissimilar,isVISA,unif] = FILTER(X,Y,Ybin,opts) marks instances as redundant when they are closer than opts.mindistance in feature space to an instance already kept, and they also meet the condition in opts.type . Removing them gives a smaller, more evenly spread set of instances, which reduces the bias of densely sampled regions on the projection and the footprints (Alipour et al., 2023). InstanceSpace calls FILTER when opts.selvars.densityflag is true : once on all features during PRELIM, and again after SIFTED on the selected features only. Examples Expand all Filter the reference data Copy Filter inside the pipeline Copy Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed features, one row per instance. Y — Performance matrix numeric matrix Ybin — Good-performance labels logical matrix opts — Filter options structure Normally obj.opts.selvars . opts.mindistance — Distance threshold 0.10 (default) | positive scalar Instances closer than this in feature space are candidates for removal. opts.type — Removal condition 'Ftr&Good' (default) | 'Ftr' | 'Ftr&AP' | 'Ftr&AP&Good' Condition, in addition to feature closeness, for an instance to be removed: Value The instance is removed when 'Ftr' it is close in feature space 'Ftr&AP' the two performance vectors are also within sqrt(nalgos/nfeats)*mindistance 'Ftr&Good' every algorithm is also good on both instances 'Ftr&AP&Good' both of the above Output Arguments Expand all subsetIndex — Redundant instances logical vector true for each instance marked redundant. Keep ~subsetIndex . isDissimilar — Isolated instances logical vector false for each instance that was close to another instance. isVISA — Close but kept logical vector true for an instance that was close to a kept instance in feature space but did not meet the opts.type condition, so it was kept. unif — Uniformity of the kept set scalar One minus the coefficient of variation of the nearest-neighbour distances of the kept instances. Values near 1 mean an even spread. Version History v0.9.2 — Bounded neighbour queries The spatial index is queried only for each retained instance, which bounds stored neighbour lists without changing the greedy row order. v0.9.0 — Uniformity output unif is returned; before, it was computed and discarded. References Alipour, H., Muñoz, M.A. & Smith-Miles, K. (2023). Enhanced instance space analysis for the maximum flow problem. European Journal of Operational Research , 304(2), 411–428. https://doi.org/10.1016/j.ejor.2022.04.012 Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also ISAsubsetData | PRELIM | SIFTED | Options Reference"},{"t":"All Functions","u":"FunctionList.html","p":"Every function and class in the toolbox, by category.","h":["Pipeline Stages","Class Interface","Backward-Compatible Wrappers","Utilities","Output"],"x":"Pipeline Stages Name Purpose INIT Read and filter instance metadata PRELIM Label good performance and normalise features and performance FILTER Remove near-duplicate instances to even out the instance density SIFTED Select the features that best explain algorithm performance PILOT Project instance features onto a 2D or 3D instance space PILOTviewpoint Find the best viewing angles of a 3D instance space CLOISTER Estimate the boundary of the instance space PYTHIA Train classifiers that predict which algorithms perform well TRACE Compute algorithm footprints in the instance space Class Interface Name Purpose InstanceSpace Build, explore and save an instance space Backward-Compatible Wrappers Name Purpose buildIS Build an instance space from a folder (backward-compatible wrapper) exploreIS Evaluate a saved instance space on new instances (backward-compatible wrapper) Utilities Name Purpose ISAdefaults Fill in missing options with their default values ISAvalidateOpts Check the type and range of user-supplied options ISAgetClassifierFcn Look up a classifier in the PYTHIA registry ISAmigrateModel Update a model from an earlier toolbox version to the current layout ISAsubsetData Keep a subset of the instances in a data structure Output Name Purpose scriptcsv Write the results of an instance space to CSV files scriptpng Save figures of an instance space as PNG files scriptweb Write colour-scaled data for the MATILDA web platform ISArecallView Return a 3D figure to its optimised viewpoint"},{"t":"Getting Started with Instance Space Analysis","u":"GettingStarted.html","p":"Build and explore your first instance space","h":["Prepare a Folder","Describe the Performance Measure","Build the Instance Space","Look at the Results","Evaluate New Instances","Next Steps","References"],"x":"This example builds an instance space for the reference dataset shipped with the toolbox: 212 classification problems (the instances ), described by 10 features, on which 10 classification algorithms were scored by misclassification error (Muñoz et al., 2018). A further 23 instances are held out for testing. The same steps apply to your own data once it is in the metadata format . example.m in the repository root runs this example from start to finish. Prepare a Folder The toolbox reads its inputs from, and writes its outputs to, one folder. Copy the reference metadata into a new folder: Copy Describe the Performance Measure Raw algorithm performance must be finite and nonnegative. Zero is allowed; negative values are rejected. Missing test outcomes can be represented by NaN . This restriction applies to raw performance, not features or transformed coordinates. The one decision you must always make is how to judge performance. Here performance is an error rate, so lower is better, and an algorithm counts as good on an instance when its error is below 20%: Copy With AbsPerf = false , an algorithm is good when it is within epsilon (a fraction) of the best algorithm on that instance. Every other option has a default; see the Options Reference . Build the Instance Space Copy build runs the whole pipeline: INIT reads and validates the metadata, PRELIM labels and normalises the data, SIFTED selects features, PILOT projects the instances to 2D, CLOISTER estimates the boundary of the space, PYTHIA trains a classifier per algorithm, and TRACE finds the footprints. It then saves model.mat and writes CSV and PNG files to rootdir . It takes a few minutes, most of it in SIFTED and PYTHIA. Look at the Results Copy The PNG files in rootdir show the same views for every feature and algorithm. Evaluate New Instances Copy explore applies the trained preprocessing, projection, classifiers and footprints to the new instances; nothing is refitted. Next Steps Change an option and re-run only the affected stages — see InstanceSpace . Run and inspect each stage in turn — see the Stage-by-Stage Walkthrough . Build a 3D instance space with opts.pilot.dims = 3 — see PILOT and PILOTviewpoint . Use your own data — see Metadata File Format . References Muñoz, M.A., Villanova, L., Baatar, D. & Smith-Miles, K. (2018). Instance spaces for machine learning classification. Machine Learning , 107(1), 109–147. https://doi.org/10.1007/s10994-017-5629-5 Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895"},{"t":"INIT","u":"INIT.html","p":"Read and filter instance metadata","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","Version History","v0.9.2 — Independent pipeline stage","v0.9.1 — Introduced","References","See Also"],"x":"Syntax [data,extra] = INIT(rootdir,opts) [data,extra] = INIT(rootdir,opts,trainedModel) Description [data,extra] = INIT(rootdir,opts) reads rootdir/metadata.csv , keeps the features and algorithms listed in opts.selvars.feats and opts.selvars.algos (all of them when these fields are absent), and returns the matrices the other stages need. [data,extra] = INIT(rootdir,opts,trainedModel) reads rootdir/metadata_test.csv for evaluation. It checks that the test file has the trained model's features in the same order, and aligns the algorithm columns with the trained model: known algorithms go to their trained position, new algorithms are appended, and trained algorithms missing from the test file get a column of NaN . InstanceSpace.build exposes this as the first stage, init . Its output is cached in model.init.data for subsequent prelim runs. Both build and explore callbacks report init before prelim . Examples Expand all Read the reference metadata Copy Read only some algorithms Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains metadata.csv (training) or metadata_test.csv (evaluation). Must end with a file separator. opts — Toolbox options structure The complete options structure. INIT reads opts.selvars.feats and opts.selvars.algos , cell arrays of column names including their feature_ / algo_ prefix. trainedModel — Trained model structure The model property of a built InstanceSpace object. Its presence selects evaluation mode. Output Arguments Expand all data — Metadata structure data.instlabels — Instance names From the Instances column. data.X , data.Xraw — Features Numeric matrix; both are the unprocessed values at this point. data.Y , data.Yraw — Performance Numeric matrix; both are the unprocessed values at this point. data.algolabels — Algorithm names Without the algo_ prefix. data.featlabels — Feature names Training mode only, without the feature_ prefix. data.S — Instance sources Categorical vector, present only when the file has a source column. extra — Evaluation bookkeeping structure Empty in training mode. In evaluation mode: featlabelsAll , modelalgos (number of trained algorithms), newalgos (algorithms new in the test file), nalgos and ninst . Tips See Metadata File Format for the file layout. Evaluation fails with ISA:InstanceSpace:featureOrderMismatch when metadata_test.csv lists its features in a different order from the training file. Version History v0.9.2 — Independent pipeline stage InstanceSpace exposes init as a selectable build stage, retains its output for preprocessing reruns, and reports it through build and explore callbacks. v0.9.1 — Introduced Replaces two separate readers inside InstanceSpace (build and explore) with one function. References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also PRELIM | InstanceSpace | Metadata File Format"},{"t":"ISAdefaults","u":"ISAdefaults.html","p":"Fill in missing options with their default values","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax opts = ISAdefaults(opts) Description opts = ISAdefaults(opts) returns opts with every missing field set to its default, so that every stage receives a complete options structure. Fields already present are not changed. The defaults are listed in the Options Reference . ISAdefaults also maps a few legacy field names to their current names when the current name is absent: opts.parallel.flag / .ncores to opts.general.parallel / .ncores , opts.pilot.ISA3D to opts.pilot.dims , opts.cloister.cthres to opts.cloister.corrThreshold , opts.pythia.cvfolds to opts.pythia.kFold , and opts.pythia.useknn = false to opts.pythia.classifier = 'svm' . InstanceSpace calls ISAdefaults after ISAvalidateOpts when an object is created or loaded. Examples Expand all See every default Copy Complete a partial options file Copy Input Arguments Expand all opts — Options structure Any subset of the options, for example as decoded from options.json . Output Arguments Expand all opts — Complete options structure Version History v0.9.0 — Introduced Defaults moved from each stage into one function. See Also ISAvalidateOpts | InstanceSpace | Options Reference"},{"t":"ISAgetClassifierFcn","u":"ISAgetClassifierFcn.html","p":"Look up a classifier in the PYTHIA registry","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax [fitFcn,p1label,p2label] = ISAgetClassifierFcn(name) Description [fitFcn,p1label,p2label] = ISAgetClassifierFcn(name) returns the Statistics and Machine Learning Toolbox fitting function for the classifier name , and the names of the hyperparameters that PYTHIA tunes for it. name is the value of opts.pythia.classifier . name Fitting function First hyperparameter (search range) Second hyperparameter 'knn' fitcknn NumNeighbors [1, 25] Distance (categorical) 'svm' fitcsvm BoxConstraint (log2 scale) KernelScale (log2 scale) 'tree' fitctree MinLeafSize [1, 100] — 'nb' fitcnb Bandwidth (log10 scale) — 'linear' fitclinear Lambda (log10 scale) — 'ensemble' fitcensemble NumLearningCycles [10, 200] MinLeafSize [1, 20] Each classifier is binary: PYTHIA trains one per algorithm, so multiclass learners such as fitcecoc are not needed. Examples Expand all Look up the SVM entry Copy Copy Supply fixed hyperparameters for a single-parameter classifier With opts.pythia.tuning = 'none' , opts.pythia.params needs one column per hyperparameter. Copy Input Arguments Expand all name — Classifier name 'knn' | 'svm' | 'tree' | 'nb' | 'linear' | 'ensemble' Case insensitive. Any other value raises ISA:ISAgetClassifierFcn:unknownClassifier . Output Arguments Expand all fitFcn — Fitting function function handle p1label — First hyperparameter name character vector p2label — Second hyperparameter name character vector 'N/A' for classifiers with one hyperparameter. Version History v0.9.0 — Introduced See Also PYTHIA | Options Reference"},{"t":"ISAmigrateModel","u":"ISAmigrateModel.html","p":"Update a model from an earlier toolbox version to the current layout","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — PLS boundary alignment and Octave archives","v0.9.0 — Introduced","See Also"],"x":"Syntax model = ISAmigrateModel(rootdir) model = ISAmigrateModel(rootdir,'backupSuffix',suffix) model = ISAmigrateModel(model) Description model = ISAmigrateModel(rootdir) migrates rootdir/model.mat in place. The original file is first copied to model_legacy.mat in the same folder. Use this form once, to convert a saved model. model = ISAmigrateModel(rootdir,'backupSuffix',suffix) names the backup model.mat . An existing backup is never overwritten; ISAmigrateModel raises ISA:ISAmigrateModel:backupExists instead. model = ISAmigrateModel(model) migrates a model structure already in memory and writes no files. InstanceSpace.load and exploreIS use this form on every load, so a legacy model.mat also works without converting it first. Examples Expand all Convert a saved model Copy Migrate in memory Copy Input Arguments Expand all rootdir — Folder containing model.mat character vector | string scalar model — Model structure structure A model loaded with load('model.mat') . suffix — Backup file suffix '_legacy' (default) | character vector File form only. Output Arguments Expand all model — Migrated model structure Algorithms Legacy content Becomes opts.oracle , opts.pbldr , opts.sbound , opts.footprint opts.pythia , opts.pilot , opts.cloister , opts.trace opts.corr.flag , opts.corr.threshold , opts.clust.flag fields of opts.sifted opts.perf.MaxMin opts.perf.MaxPerf model.data.bestPerformace model.data.Ybest PLS model without pilot.Xmean recover the mean from training features and translate saved cloist.Zedge / Zecorr by -Xmean*A' ; face indices are unchanged model.pythia.svm{i} , model.pythia.knn{i} model.pythia.classifiers{i} model.pythia.boxcosnt , model.pythia.kscale model.pythia.param1 , model.pythia.param2 LIBSVM classifier structures retrained with opts.pythia.classifier (default 'knn' ), because LIBSVM structures have no predict method model.trace in the DBSCAN/polyshape format recomputed with TRACE3, using model.pythia.Yhat when available, otherwise model.data.Ybin no completedStages inferred from the stage fields present A model.pilot.A without B and C cannot be repaired and raises a warning. GNU Octave File-based migration recognizes versioned Octave archives and preserves their explicit reconstruction format. MATLAB legacy migration retains its existing file format. Version History v0.9.2 — PLS boundary alignment and Octave archives File-based migration now reads versioned Octave model archives. Schema-1 archives remain readable; saving writes schema 2, which keeps ordinary struct fields separate from internal object markers. Recovering a missing PLS feature mean also translates any saved CLOISTER vertices into the fitted PLS coordinate system. Empty or absent boundaries are retained. Repeated migration leaves already centred geometry unchanged. Models without the training features needed to recover the mean must be rebuilt. v0.9.0 — Introduced See Also Migrating a Legacy Model | InstanceSpace | exploreIS"},{"t":"ISArecallView","u":"ISArecallView.html","p":"Return a 3D figure to its optimised viewpoint","h":["Syntax","Description","Examples","Input Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax ISArecallView(fig) ISArecallView(fig,algoIdx) Description ISArecallView(fig) sets the camera of the 3D instance-space figure fig to the global viewpoint found by PILOTviewpoint . Use it after rotating a figure saved by scriptpng . ISArecallView(fig,algoIdx) uses the viewpoint of the group in opts.pilot.viewGroups that contains algorithm algoIdx , or the global viewpoint if no group contains it. The viewpoints are stored in the figure's UserData by scriptpng . Examples Expand all Restore the view of a saved footprint Copy Input Arguments Expand all fig — Figure figure handle A 3D figure saved by scriptpng . algoIdx — Algorithm index [] (default) | positive integer Column of the algorithm in model.data.algolabels . Version History v0.9.0 — Introduced See Also PILOTviewpoint | scriptpng"},{"t":"ISAsubsetData","u":"ISAsubsetData.html","p":"Keep a subset of the instances in a data structure","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax data = ISAsubsetData(data,subsetIndex) data = ISAsubsetData(data,subsetIndex,featIdx) Description data = ISAsubsetData(data,subsetIndex) keeps the rows subsetIndex of every per-instance field of data : X , Y , Xraw , Yraw , Ybin , beta , numGoodAlgos , Ybest , P , instlabels and, if present, S . data = ISAsubsetData(data,subsetIndex,featIdx) also keeps only the feature columns featIdx of data.X and data.featlabels . InstanceSpace uses this form after SIFTED when density filtering is on. Examples Expand all Keep the beta-easy instances Copy Input Arguments Expand all data — Instance data structure model.data , with the fields listed above. subsetIndex — Instances to keep logical vector | vector of indices featIdx — Features to keep vector of indices Output Arguments Expand all data — Subset structure Version History v0.9.0 — Introduced See Also FILTER | InstanceSpace"},{"t":"ISAvalidateOpts","u":"ISAvalidateOpts.html","p":"Check the type and range of user-supplied options","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax opts = ISAvalidateOpts(opts) Description opts = ISAvalidateOpts(opts) checks each recognised field that is present in opts and raises an error on the first invalid one. Absent fields are not checked; ISAdefaults fills them afterwards. opts is returned unchanged. An error here names the field and the expected value, instead of failing later inside a stage. A field set to [] counts as present and is rejected. Field names the function does not recognise, including misspellings, are not reported. InstanceSpace calls ISAvalidateOpts when an object is created. Examples Expand all Catch an invalid value early Copy Input Arguments Expand all opts — Options structure Output Arguments Expand all opts — Options structure The input, unchanged. Tips Error identifiers, by the check that failed: Identifier Expected ISA:ISAvalidateOpts:notStruct a structure, for opts and each group such as opts.pilot ISA:ISAvalidateOpts:notLogical a logical scalar, true or false (not 0 or 1) ISA:ISAvalidateOpts:notFiniteNumericScalar a finite real number ISA:ISAvalidateOpts:notInteger a whole number ISA:ISAvalidateOpts:notPositive a positive number (0 is allowed for seeds) ISA:ISAvalidateOpts:notInUnitRange a number in [0, 1] ISA:ISAvalidateOpts:notMember one of a fixed set of values ISA:ISAvalidateOpts:notText a character vector or string ISA:ISAvalidateOpts:notCellOfText a cell array of character vectors ISA:ISAvalidateOpts:badViewGroups a cell array of algorithm index vectors Version History v0.9.0 — Introduced See Also ISAdefaults | InstanceSpace | Options Reference"},{"t":"InstanceSpace","u":"InstanceSpace.html","p":"Build, explore and save an instance space","h":["Syntax","Description","Examples","Input Arguments","Properties","Object Functions","build","explore","getResults","plot","save","load","GNU Octave","Version History","v0.9.2 — Stage validation and Octave workflows","v0.9.1 — Stage callbacks and boundary plot","v0.9.0 — Introduced","References","See Also"],"x":"Syntax obj = InstanceSpace(rootdir) obj = InstanceSpace(rootdir,opts) Description An InstanceSpace object holds the options, the trained model, and the evaluation results of one instance space analysis. It runs the pipeline stages INIT , PRELIM , SIFTED , PILOT , CLOISTER , PYTHIA and TRACE in order, lets you change options and re-run individual stages, evaluates the trained model on new instances, and saves and loads the model. obj = InstanceSpace(rootdir) creates an object for the metadata in rootdir/metadata.csv . Options are read from rootdir/options.json if the file exists; missing options take their defaults. No computation runs until you call build . obj = InstanceSpace(rootdir,opts) uses the options structure opts instead of options.json . During build , algorithms with no good training instances are removed. Preprocessing is then fitted to the retained portfolio. Best-algorithm indices, best performance, beta-easy flags, and saved transforms all refer to that portfolio. With an absolute threshold, removing an algorithm can change the best performance on an instance where no algorithm is good. InstanceSpace is a value class: a method that changes the object returns the changed copy, so assign the result, as in obj = obj.build() . Examples Expand all Build and explore an instance space Copy Change an option and re-run one stage Re-running a stage discards the results of every later stage, which you then re-run. Copy Inspect each stage as it finishes Copy Save and load a model Copy Input Arguments Expand all rootdir — Data folder character vector | string scalar Folder containing metadata.csv . Output files and model.mat are written here. See Metadata File Format . opts — Options structure Any subset of the fields in the Options Reference . Values are checked by ISAvalidateOpts , and missing fields filled by ISAdefaults . Properties Expand all rootdir — Data folder character vector Always ends with a file separator. opts — Options structure Complete options. Change fields between build calls to re-run stages with new settings. explore uses the options stored with the model at training time instead. model — Trained model structure One field per completed stage: data , prelim , featsel , sifted , pilot , cloist (CLOISTER), pythia , trace , and opts once every stage has run. completedStages — Completed stages cell array of character vectors testDirs , testResults — Evaluation results cell arrays One entry per explore call: the folder and the result structure, which has the same fields as model . Object Functions Function Purpose build Run pipeline stages explore Evaluate the trained model on new instances getResults Return the training or an evaluation result plot Plot a view of the instance space save Write model.mat InstanceSpace.load Create an object from model.mat build Copy Runs every stage, or only the named stages from {'init','prelim','sifted','pilot','cloister','pythia','trace'} , always in pipeline order. A stage whose prerequisites have not completed raises ISA:InstanceSpace:missingPrereq . callback(stageName, model) is called after each stage, including init . For compatibility, requesting prelim automatically runs init first if its input snapshot is missing. init loads and filters metadata, publishes it in model.data , and retains an independent input snapshot in model.init.data . Rerunning prelim reuses that snapshot. To reread a changed metadata file, rerun init ; this invalidates every later stage. Changes to selvars.feats , selvars.algos , or prelim.nanThreshold require rerunning init . Full builds retain the same computations and numerical outputs. Saved models now also contain the input snapshot. Legacy models without this snapshot remain usable for exploration; rebuilding their prelim stage automatically runs init and therefore requires the original metadata file. When all stages have completed, build saves model.mat and writes the outputs selected in opts.outputs with scriptcsv , scriptweb and scriptpng"},{"t":"Stage-by-Stage Walkthrough","u":"InteractiveWalkthrough.html","p":"Run each stage of the pipeline and inspect its output","h":["PRELIM: Label and Normalise","SIFTED: Select Features","PILOT: Project to 2D","CLOISTER: Estimate the Boundary","PYTHIA: Predict Good Algorithms","TRACE: Find the Footprints","Save and Explore","See Also"],"x":"This walkthrough builds the instance space one stage at a time so you can see what each stage produces. It follows liveDemoIS.m in the repository root, which you can open in the Live Editor and run section by section. It uses the same reference data and performance settings as Getting Started . Copy PRELIM: Label and Normalise PRELIM decides which algorithms are good on each instance, bounds outlying feature values, and normalises features and performance. Copy An instance is beta-easy when more than opts.perf.betaThreshold of the algorithms are good on it. SIFTED: Select Features SIFTED keeps the features that best explain performance: a correlation filter, then a genetic algorithm that picks one feature from each cluster of correlated features. Copy Set opts.sifted.flag = false to keep every feature. PILOT: Project to 2D PILOT finds the linear projection Z = X*A' from which features and performance can best be reconstructed. Z is the instance space. Copy opts.pilot.dims = 3 gives a 3D space; opts.pilot.method = 'pls' uses Partial Least Squares. CLOISTER: Estimate the Boundary CLOISTER estimates where instances could exist, given the ranges and correlations of the features. Empty areas inside the boundary are candidates for new test instances. Copy PYTHIA: Predict Good Algorithms PYTHIA trains one classifier per algorithm that predicts, from an instance's position, whether the algorithm is good there, and combines them into an algorithm selector. Copy TRACE: Find the Footprints TRACE finds the region where each algorithm is good (its footprint) and where it is best, and measures their size, density and purity. Copy Save and Explore Once every stage has run, build has already saved model.mat and written the output files. explore evaluates new instances with the trained model: Copy To change a setting later, edit obj.opts and re-run from the stage it affects, for example obj = obj.build('stages', {'pythia','trace'}) after changing opts.pythia.classifier . Re-running a stage discards the results of later stages. See Also InstanceSpace | Getting Started | Options Reference"},{"t":"Metadata File Format","u":"MetadataFormat.html","p":"Prepare metadata.csv for your own problem domain","h":["Columns","Example","Missing Values","Test Instances","Choosing Features","Version History","v0.9.2 — Training feature schema","References","See Also"],"x":"An instance space is built from one CSV file, metadata.csv , with one row per problem instance. Instances to evaluate later with explore go in metadata_test.csv , in the same format. Columns Columns are recognised by their header. The order of the column groups does not matter, and headers are case insensitive. Header Required Contents Instances yes Instance name. Text or numbers. Source no Where the instance comes from, for example the benchmark suite. Drawn by obj.plot('sources') . feature_ at least two Numeric feature value. algo_ at least one Numeric performance of the algorithm on the instance. Other columns are ignored. Names after the prefix must be valid MATLAB identifiers: use underscores instead of spaces, for example feature_edge_density or algo_simulated_annealing . Example The first rows of the reference data ( test/data/metadata.csv ), shortened to three features and three algorithms: Copy Missing Values Leave a cell empty or write NaN for a missing value. A feature with at least opts.prelim.nanThreshold missing values is removed. After filtering, training requires complete features and performance values. Remaining missing values cause an error that names the affected instances. Evaluation requires complete retained features but permits missing performance values, which never count as observed failures in classifier scoring. Avoid NA , spreadsheet error codes such as #DIV/0! , and empty rows: they make MATLAB read the whole column as text, which then fails later in the pipeline. Test Instances metadata_test.csv must contain the same feature_ columns as metadata.csv , in the same order. It may contain: a subset of the trained algorithms — the missing ones are not scored; algorithms that were not trained — they are appended to the results, without a classifier. Choosing Features The features determine what the instance space can reveal. Good features: are cheap to compute compared with running the algorithms; capture properties that plausibly affect how hard an instance is for each algorithm; vary across the instances, with few repeated values. Start with more candidate features than you need; SIFTED removes the ones that do not explain performance. See Smith-Miles & Muñoz (2023) for guidance on feature design. Version History v0.9.2 — Training feature schema Training stores the feature columns removed by nanThreshold . Exploration removes those same columns before checking the required feature schema, even when their test values are complete. References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also INIT | InstanceSpace | Options Reference"},{"t":"Migrating a Legacy Model","u":"MigratingLegacyModel.html","p":"Use a model.mat created by a toolbox version before v0.9.0","h":["Automatic Migration on Load","Converting the File Once","What Changes","When Migration Cannot Complete","See Also"],"x":"Version 0.9.0 renamed several option groups and model fields and replaced the LIBSVM classifiers and the footprint algorithm. A model.mat saved by an earlier version therefore needs migrating before the current code can use it. In most cases this is automatic. Automatic Migration on Load InstanceSpace.load and exploreIS migrate a legacy model in memory each time they load it. Nothing is written to disk: Copy Converting the File Once To avoid repeating the migration, including any classifier retraining, on every load, convert the file: Copy The original is kept as model_legacy.mat . Choose another backup name with ISAmigrateModel(rootdir,'backupSuffix','_v08') . What Changes Options are renamed to the current groups: opts.oracle → opts.pythia , opts.pbldr → opts.pilot , opts.sbound → opts.cloister , opts.footprint → opts.trace , and opts.perf.MaxMin → opts.perf.MaxPerf . The feature-selection flags of opts.corr and opts.clust move to opts.sifted . Classifiers. LIBSVM models cannot be evaluated by the current code. They are retrained with the classifier in opts.pythia.classifier ( 'knn' if unset) on the training data stored in the model. Predictions of the retrained model can differ slightly from those of the original. Footprints in the old DBSCAN/polyshape format are recomputed with TRACE3. Their areas, densities and purities can differ from the values reported by the old version. Field names such as data.bestPerformace are corrected. The complete table is on the ISAmigrateModel reference page. When Migration Cannot Complete A projection without its reconstruction matrices ( model.pilot.B , model.pilot.C ) is kept as is, with a warning. Retraining LIBSVM classifiers and recomputing footprints need the training data stored in the model ( model.pilot.Z and model.data ). When it is missing, the fields are renamed only, with a warning. If a LIBSVM model has to be evaluated as-is, PYTHIA raises ISA:PYTHIA:noLibsvm unless the LIBSVM MEX files are on the path. The toolbox does not include them; get them from the LIBSVM project . Retraining is recommended instead. See Also ISAmigrateModel | InstanceSpace | What's New"},{"t":"Options Reference","u":"OptionsReference.html","p":"Every field of the options structure, with its default","h":["opts.general","opts.perf","opts.prelim","opts.auto","opts.bound","opts.norm","opts.selvars","opts.sifted","opts.pilot","opts.cloister","opts.pythia","opts.trace","opts.outputs","Version History","v0.9.2 — Option validation","See Also"],"x":"Options control every stage of the pipeline. Give them to InstanceSpace as a structure, or as an options.json file in the data folder with the same nesting: Copy Copy Set only the fields you want to change. ISAvalidateOpts checks the fields you set, and ISAdefaults fills in the rest with the defaults below. Between staged build calls you can change obj.opts directly. opts.general Settings for the whole pipeline. Field Default Description seed 42 Random seed. Every stochastic stage derives its seed from this value, so the same seed and data reproduce a run. verbose true Detailed progress output. Stage start and end messages are always printed. parallel false Use a parallel pool (Parallel Computing Toolbox) for SIFTED, PILOT, PYTHIA and TRACE. ncores 18 Workers when opening a pool. An existing pool is reused unchanged. opts.perf How performance is judged. Used by PRELIM . Observed raw performance must be finite and nonnegative. Zero is allowed; negative scores are rejected in both absolute and relative modes. Relative mode substitutes machine epsilon for exact zero numerators and denominators; see PRELIM for the resulting zero-reference convention. Field Default Description MaxPerf false true if larger performance values are better; false for a cost such as error or run time. AbsPerf false true : good means better than epsilon . false : good means within a fraction epsilon of the best algorithm on the instance. epsilon 0.05 Good-performance threshold. In [0, 1] when AbsPerf is false ; any real number otherwise. betaThreshold 0.55 An instance is beta-easy when more than this fraction of the algorithms are good on it. opts.prelim Data preparation. Used by INIT and PRELIM . Field Default Description iqrMultiplier 5 Features are bounded to median ± iqrMultiplier*IQR . nanThreshold 0.20 A feature with at least this fraction of missing values is removed. opts.auto Used by PRELIM . Field Default Description preproc true Run the automatic outlier bounding and normalisation. false leaves the data as given. opts.bound Used by PRELIM . Field Default Description flag true Bound feature outliers. Features with very little variation can have an IQR of zero; remove them or turn bounding off. opts.norm Used by PRELIM . Field Default Description flag true Apply Box-Cox and Z-score transforms, so features and performance are close to normally distributed. Recommended, because PILOT is a linear projection and CLOISTER expects centred data. opts.selvars Which features, algorithms and instances to use. Used by INIT , InstanceSpace and FILTER . Field Default Description feats all Cell array of feature column names to use, with their feature_ prefix. algos all Cell array of algorithm column names to use, with their algo_ prefix. smallscaleflag false Build from a random fraction of the instances. Useful to try settings on a large dataset. smallscale 0.30 Fraction of instances kept when smallscaleflag is true . fileidxflag false Build from the instances listed in the file fileidx . fileidx '' CSV file with one column of instance indices (row numbers of metadata.csv ). densityflag false Remove near-duplicate instances with FILTER . mindistance 0.10 FILTER distance threshold in feature space. type 'Ftr&Good' FILTER removal condition: 'Ftr' , 'Ftr&AP' , 'Ftr&Good' or 'Ftr&AP&Good' . opts.sifted Feature selection. Used by SIFTED . Field Default Description diagnostics true Run the advisory silhouette sweep. Disable it to reduce selection overhead. flag true Run SIFTED. false keeps every feature. rho 0.10 Minimum absolute correlation between a feature and an algorithm's performance. pval 0.05 Significance level of the correlations. K 10 Number of feature clusters, which is the number of features selected. We recommend 10 or fewer. MaxIter 1000 Maximum k-means iterations. Replicates 100 Number of k-means replicates. seed opts.general.seed Random seed of SIFTED. opts.pilot Projection. Used by PILOT and PILOTviewpoint . Field Default Description dims 2 Dimensio"},{"t":"PILOT","u":"PILOT.html","p":"Project instance features onto a 2D or 3D instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Projection corrections and Octave support","v0.9.1 — Seed control","v0.9.0 — 3D projections and PLS","References","See Also"],"x":"Syntax out = PILOT(X,Y,featlabels,opts) Description out = PILOT(X,Y,featlabels,opts) finds a linear projection Z = X*A' of the feature matrix X into opts.dims dimensions, chosen so that both the features and the algorithm performance Y can be reconstructed from Z as well as possible (Prediction-based Linear Dimensionality Reduction). Instances with similar features and similar performance land close together, so trends in performance become visible as directions in the space. PILOT has three solvers, selected by opts.method and opts.analytic : Numerical (default): BFGS minimisation of the reconstruction error of X and Y from Z (see Algorithms), restarted opts.ntries times. The trial whose projection best preserves the pairwise distances of the original data is kept. Analytic : the closed-form eigenvector solution. Fast and deterministic. If X is rank deficient, PILOT falls back to the numerical solver. PLS : Partial Least Squares regression of Y on X with plsregress . opts.alpha does not apply. Examples Expand all Project the reference data in 2D Prepare the features with the earlier pipeline stages, then call PILOT directly. Copy Compare the solvers The analytic solver is deterministic and much faster than BFGS. Compare how well each reconstructs the data. Copy Build a 3D instance space Set opts.pilot.dims to 3 before building. InstanceSpace then also calls PILOTviewpoint to find the best 2D viewing angles. Copy Input Arguments Expand all X — Feature matrix numeric matrix Instance features, one row per instance and one column per feature, normally the output of PRELIM and SIFTED . Y — Performance matrix numeric matrix Algorithm performance, one row per instance and one column per algorithm. Must have the same number of rows as X . featlabels — Feature names cell array of character vectors One name per column of X . Used for the row labels of out.summary . opts — Projection options structure Normally obj.opts.pilot . Missing fields take the defaults set by ISAdefaults . opts.dims — Number of dimensions 2 (default) | 3 Dimension of the instance space. opts.method — Projection method 'standard' (default) | 'pls' 'standard' uses the analytic or numerical PILOT solution. 'pls' uses Partial Least Squares. opts.analytic — Use the closed-form solution false (default) | true Only used when opts.method is 'standard' . Falls back to the numerical solver when X is rank deficient. opts.ntries — Number of BFGS restarts 10 (default) | positive integer More restarts reduce the chance of a poor local minimum and take proportionally longer. opts.alpha — Weight of the performance term 1.0 (default) | positive scalar Weight of ||Y - C*Z||^2 relative to the feature term. Larger values favour a projection that explains performance over one that explains features. Not used by 'pls' . opts.seed — Random seed opts.general.seed (default) | integer Seeds the BFGS starting points, so repeated runs give the same projection. opts.verbose — Report progress opts.general.verbose (default) | logical opts.X0 — Starting points numeric matrix Optional user-supplied BFGS starting points, one column per trial. opts.precalcAlpha — Precomputed solution numeric vector Optional full BFGS solution vector from an earlier run. Skips the optimisation. Output Arguments Expand all out — Projection structure out.A — Projection matrix dims -by- nfeats matrix. Project new, identically preprocessed instances with Z = Xnew*out.A' . out.Z — Instance coordinates ninst -by- dims coordinates of each instance in the instance space. out.B , out.C — Reconstruction matrices Reconstruct the features and the performance from Z . out.error — Reconstruction error Sum of squared errors of the reconstruction of [X Y] . out.R2 — Coefficient of determination R^2 of the reconstruction of each column of [X Y] . out.summary — Projection table Cell array showing A with the feature labels. Also written to projection_matrix.csv by scriptcsv . out.alpha , out.X0 , out.eoptim , out.perf — Trial results"},{"t":"PILOTviewpoint","u":"PILOTviewpoint.html","p":"Find the best viewing angles of a 3D instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Camera conventions and Octave support","v0.9.0 — Introduced","References","See Also"],"x":"Syntax out = PILOTviewpoint(Z,Y,opts) Description out = PILOTviewpoint(Z,Y,opts) finds, for each group of algorithms in opts.viewGroups , the 2D view of the 3D projection Z from which the performance of those algorithms is best explained. Each view is a pair of orthogonal directions v1 , v2 ; the returned azimuth and elevation point the MATLAB camera along cross(v1,v2) (Simpson et al., 2025, Equation 2). InstanceSpace calls PILOTviewpoint after PILOT when opts.pilot.dims is 3 and stores the result in model.pilot.viewpoint . scriptpng then renders 3D figures from these angles. Examples Expand all View a 3D instance space from its best angle Copy One viewpoint per group of algorithms Copy Input Arguments Expand all Z — 3D instance coordinates ninst -by-3 numeric matrix model.pilot.Z from a 3D build. Y — Performance matrix numeric matrix Same number of rows as Z . opts — Viewpoint options structure Normally obj.opts.pilot . opts.viewGroups — Algorithm groups {} (default) | cell array of index vectors One viewpoint is computed per group, from the columns of Y in that group. Empty means one viewpoint for all algorithms. opts.ntries — Number of BFGS restarts 10 (default) | positive integer opts.seed — Random seed opts.general.seed (default) | integer opts.X0 — Starting points numeric matrix Optional, 2*3 + 2*numel(group) rows by one column per trial. Used for each group of matching size. Output Arguments Expand all out — Viewpoints structure out.groups — Algorithm groups The groups used, one cell per viewpoint. out.A — View matrices Cell array; each cell is a 2-by-3 matrix [v1; v2] that flattens Z onto the viewing plane. out.azimuth , out.elevation — Camera angles In radians, one per group. Use view(rad2deg(az), rad2deg(el)) . Algorithms For each group, PILOTviewpoint jointly fits the view A (2-by-3) and a performance reconstruction C by BFGS, minimising Copy The second term keeps the two view directions orthogonal. Like PILOT, it keeps the trial whose view best preserves the pairwise distances of Z . GNU Octave Octave runs the viewpoint optimization serially through a namespaced fminunc option adapter. MATLAB keeps native optimization options. Equal starting seeds do not guarantee identical optimized camera angles. Version History v0.9.2 — Camera conventions and Octave support Added serial Octave viewpoint optimization with runtime-specific optimizer options. Azimuth and elevation use the cart2sph convention in radians. Plot helpers convert azimuth to the MATLAB camera convention and use equal data-axis scales to preserve the fitted plane. v0.9.0 — Introduced Part of the 3D instance space (ISA3) support. References Simpson, C., Muñoz, M.A., Kandanaarachchi, S. & Campello, R.J.G.B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning , 114, 240. https://doi.org/10.1007/s10994-025-06871-5 See Also PILOT | ISArecallView | scriptpng"},{"t":"PRELIM","u":"PRELIM.html","p":"Label good performance and normalise features and performance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Performance labels and Octave normalization","v0.9.1 — Evaluation mode","v0.9.0 — Outlier bound option","References","See Also"],"x":"Syntax [X,Y,out] = PRELIM(X,Y,opts) [X,Y,out] = PRELIM(X,Y,opts,trainedPrelim) Description [X,Y,out] = PRELIM(X,Y,opts) fits the preprocessing to the training data and applies it. PRELIM labels each algorithm as good or not good on each instance, finds the best algorithm per instance, bounds feature outliers, and applies Box-Cox and Z-score transforms to the features and the performance. The fitted bounds and transform parameters are returned in out . [X,Y,out] = PRELIM(X,Y,opts,trainedPrelim) applies the bounds and transforms of an earlier training call, trainedPrelim , to new data instead of fitting new ones. The good/best labelling is the same code in both modes, so training and evaluation label ties the same way. Examples Expand all Preprocess a performance matrix Misclassification error is a cost ( MaxPerf = false ). Call an algorithm good when its error is below 0.20. Copy Apply a trained preprocessing to new instances Copy Input Arguments Expand all X — Feature matrix numeric matrix One row per instance, one column per feature. May contain NaN . Y — Performance matrix numeric matrix One row per instance, one column per algorithm. Observed raw performance must be finite and nonnegative in both training and evaluation. Zero is allowed; negative values raise ISA:PRELIM:invalidPerformance . May contain NaN ; a missing value never counts as good. opts — Preprocessing options structure When PRELIM is called by InstanceSpace , this structure is assembled from opts.perf , opts.auto , opts.bound , opts.norm and opts.prelim . opts.MaxPerf — Performance direction false (default) | true true if larger performance values are better (for example accuracy). false for a cost, such as run time or error. Set from opts.perf.MaxPerf . opts.AbsPerf — Threshold type false (default) | true true : an algorithm is good when its performance is better than epsilon . false : an algorithm is good when its performance is within a fraction epsilon of the best algorithm on that instance. Set from opts.perf.AbsPerf . opts.epsilon — Good-performance threshold 0.05 (default) | scalar Absolute threshold, or relative tolerance, depending on AbsPerf . Set from opts.perf.epsilon . opts.betaThreshold — Easy-instance fraction 0.55 (default) | scalar in [0, 1] An instance is beta-easy when more than this fraction of the algorithms are good on it. Set from opts.perf.betaThreshold . opts.auto — Run the preprocessing true (default) | false false skips the outlier bounds and the transforms; only the labelling runs. Set from opts.auto.preproc . opts.bound — Bound outliers true (default) | false Clip each feature to median ± iqrMultiplier*IQR . Set from opts.bound.flag . opts.norm — Normalise true (default) | false Apply Box-Cox and then Z-score transforms to X and Y . Set from opts.norm.flag . opts.iqrMultiplier — Outlier bound width 5 (default) | positive scalar Set from opts.prelim.iqrMultiplier . trainedPrelim — Trained preprocessing structure The out of an earlier training-mode call. Its presence selects evaluation mode. Output Arguments Expand all X — Preprocessed features numeric matrix Y — Preprocessed performance numeric matrix With AbsPerf = false , Y is first converted to performance relative to the best algorithm, then normalised. out — Labels and fitted parameters structure out.Ybin — Good-performance labels ninst -by- nalgos logical matrix. out.Ybest — Best performance Best raw performance on each instance. out.P — Best algorithm Index of the best algorithm on each instance. Ties are broken at random. out.numGoodAlgos — Number of good algorithms Per instance. out.beta — Beta-easy instances Logical vector; true when more than betaThreshold of the algorithms are good. out.hibound , out.lobound , out.medval , out.iqrange — Outlier bounds Training mode only. out.minX , out.lambdaX , out.muX , out.sigmaX — Feature transform Training mode only. Shift, Box-Cox parameter, mean and standard deviation per feature. out.minY , out.lambdaY , out.muY , out.sigmaY — P"},{"t":"PYTHIA","u":"PYTHIA.html","p":"Train classifiers that predict which algorithms perform well","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","GNU Octave","Version History","v0.9.2 — Evaluation corrections and Octave KNN","v0.9.0 — Classifier registry","References","See Also"],"x":"Syntax out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts) out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,trainedModel) Description out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts) trains one binary classifier per algorithm that predicts, from an instance's coordinates Z , whether the algorithm is good on it (Performance Yielding Heuristic Identification of Algorithms). Hyperparameters are tuned by k -fold cross-validation. PYTHIA then combines the predictions into an algorithm selector: for each instance it picks the algorithm predicted to be good that has the highest cross-validated precision. out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,trainedModel) applies the classifiers of an earlier training call to new instances and reports how accurate they are there. Classifiers are not retrained. The classifier type is set by opts.classifier ; see ISAgetClassifierFcn for the registry and the tuned hyperparameters. Examples Expand all Train k-NN classifiers on a 2D instance space Copy Use an SVM with Bayesian optimisation Copy Evaluate trained classifiers on new instances InstanceSpace.explore does this for you. The explicit call is: Copy Input Arguments Expand all Z — Instance coordinates numeric matrix model.pilot.Z . PYTHIA standardises it internally and stores the mean and standard deviation. Y — Raw performance matrix numeric matrix model.data.Yraw . Used for the performance columns of the summary and, with opts.useweights , for the class weights. NaN marks a missing value. Ybin — Good-performance labels logical matrix The classification targets. Ybest — Best performance per instance numeric vector algolabels — Algorithm names cell array of character vectors opts — Classifier options structure Normally obj.opts.pythia . opts.classifier — Classifier type 'knn' (default) | 'svm' | 'tree' | 'nb' | 'linear' | 'ensemble' opts.tuning — Hyperparameter search 'sobol' (default) | 'bayes' | 'none' 'sobol' : a scrambled Sobol sequence of nTuningIter candidates. 'bayes' : bayesopt with the same budget. 'none' : use opts.params . opts.nTuningIter — Search budget 20 (default) | positive integer opts.kFold — Cross-validation folds 5 (default) | positive integer opts.params — Fixed hyperparameters [] (default) | nalgos -by- nparams matrix Required when opts.tuning is 'none' ; one row per algorithm. opts.useweights — Cost-sensitive training false (default) | true Weight each instance by how far its performance is from the mean performance. opts.ispolykrnl — Polynomial kernel false (default) | true SVM only. false uses a Gaussian kernel. opts.ensembleMethod — Ensemble method 'Bag' (default) | any fitcensemble method Only for 'ensemble' . opts.skip — Skip training false (default) | true Return empty predictions without training. TRACE then uses Ybin directly. opts.seed — Random seed opts.general.seed (default) | integer opts.verbose — Report progress opts.general.verbose (default) | logical trainedModel — Trained classifiers structure The out of an earlier training call ( model.pythia ). Its presence selects evaluation mode. Output Arguments Expand all out — Classifiers, predictions and summary structure out.classifiers — Trained classifiers Cell array with one ClassificationModel per algorithm. out.Yhat — Predicted labels ninst -by- nalgos logical matrix. In training mode these are the predictions of the final model on the training data. out.Pr0hat — Predicted bad-class scores Bad-class score for each algorithm on each instance. Consult the corresponding out.scoreType entry before interpreting a column: only probability denotes the probability that the algorithm is not good. decision-score and class-score are classifier scores and need not lie in [0,1]; unknown has no declared score semantics, and unavailable marks a placeholder rather than a usable score. out.Ysub , out.Pr0sub — Cross-validated predictions Training mode only. Ysub contains predicted labels; Pr0sub contains bad-class scores. out.scoreTypeCV describes each score column. A mixed column combines "},{"t":"SIFTED","u":"SIFTED.html","p":"Select the features that best explain algorithm performance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","GNU Octave","Version History","v0.9.2 — Diagnostics, cache handling and Octave support","v0.9.1 — Seed control","v0.9.0 — Renamed from SIFTED2","References","See Also"],"x":"Syntax [X,out] = SIFTED(X,Y,Ybin,featlabels,opts) Description [X,out] = SIFTED(X,Y,Ybin,featlabels,opts) returns the columns of X that best explain algorithm performance, and the indices of those columns in out.selvars . Selection has two steps: Correlation filter. For every algorithm, keep the feature most correlated with its performance. Also keep every feature whose absolute Pearson correlation with the performance of at least one algorithm is at least opts.rho and significant at level opts.pval . Clustering and genetic search. Group the remaining features into opts.K clusters by correlation distance (k-means). A genetic algorithm then picks one feature per cluster, scoring each candidate set by how well a k -nearest-neighbour classifier predicts Ybin in a PILOT projection of those features. Step 2 is skipped when three or fewer features remain, or when no more than opts.K remain. Examples Expand all Select features from the reference data Copy Check the choice of K out.eva holds the silhouette value for each number of clusters. When the value for opts.K is poor, SIFTED also suggests a better K . Copy Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed features, one row per instance. Y — Performance matrix numeric matrix Preprocessed performance, one row per instance. Ybin — Good-performance labels logical matrix From PRELIM . Used by the genetic search's fitness function. featlabels — Feature names cell array of character vectors opts — Selection options structure Normally obj.opts.sifted . opts.rho — Minimum correlation 0.10 (default) | scalar in [0, 1] opts.pval — Significance level 0.05 (default) | scalar in (0, 1) opts.K — Number of clusters 10 (default) | positive integer Number of features SIFTED selects. When no more than K features pass the correlation filter, all of them are kept. opts.MaxIter — k-means iterations 1000 (default) | positive integer opts.Replicates — k-means replicates 100 (default) | positive integer opts.seed — Random seed opts.general.seed (default) | integer Seeds k-means, the cross-validation partition, and the genetic algorithm. opts.dims — Projection dimension for the fitness function 2 (default) | 3 InstanceSpace sets this from opts.pilot.dims , so features are chosen for the same dimension as the final projection. Output Arguments Expand all X — Selected features numeric matrix The columns out.selvars of the input X . out — Selection details structure out.selvars — Selected feature indices Indices into the columns of the input X . out.rho , out.p — Correlations Correlation of each feature with each algorithm's performance, and the p-values. out.eva — Cluster evaluation evalclusters result with the silhouette value for each number of clusters. out.clust — Cluster membership nfeats -by- K logical matrix. out.Ksuggested — Suggested number of clusters Present only when the silhouette value for opts.K is below 0.5. Tips opts.sifted.flag = false skips SIFTED in InstanceSpace and keeps every feature. SIFTED is the slowest stage. Set opts.general.parallel = true to evaluate the genetic algorithm's population in parallel. GNU Octave Octave runs serial correlation clustering and CV/KNN fitness with bounded integer search. Small search spaces are enumerated; larger ones use a serial GA. The backend and fitness are recorded in search, with cvpartition retaining the folds. Octave uses batch k-means updates; MATLAB retains its online phase and native GA. Version History v0.9.2 — Diagnostics, cache handling and Octave support Added serial Octave clustering and integer search, with the search backend and CV partition recorded in the result. Set opts.sifted.diagnostics=false to skip the advisory silhouette sweep. Feature clustering and selection still run with the configured K. A call with opts.parallel=false resets only its client fitness cache and does not dispatch work to an existing pool. A later parallel call resets its worker caches before using them. v0.9.1 — Seed control k-mean"},{"t":"TRACE","u":"TRACE.html","p":"Compute algorithm footprints in the instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Footprint evaluation and Octave support","v0.9.0 — TRACE3","References","See Also"],"x":"Syntax out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts) out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts,trainedTrace) Description out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts) finds each algorithm's footprint : the regions of the instance space where the algorithm is good ( out.good ) and where it is the best algorithm ( out.best ), plus the region of beta-hard instances ( out.hard ). For each footprint TRACE reports its area (2D) or volume (3D), density and purity, relative to the whole space. out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts,trainedTrace) keeps the footprints of an earlier training call, trainedTrace , and recomputes their density and purity with new instances. Footprints are not rebuilt. The default method, TRACE3, works in 2D and 3D. opts.method = 'legacy' selects the earlier DBSCAN-based method, which is 2D only. Examples Expand all Footprints of a trained instance space Copy Rebuild footprints with a stricter purity Copy Input Arguments Expand all Z — Instance coordinates ninst -by-2 or ninst -by-3 numeric matrix Ybin — Good-performance labels logical matrix Yhat — Predicted labels logical matrix | [] model.pythia.Yhat . TRACE3 uses only instances that are both good and predicted good, which removes instances the classifier cannot separate. With [] it uses Ybin alone and warns ( ISA:TRACE3:noPYTHIA ). P — Best algorithm per instance integer vector beta — Beta-easy instances logical vector ~beta defines the hard-instance footprint. algolabels — Algorithm names cell array of character vectors opts — Footprint options structure Normally obj.opts.trace . opts.method — Footprint method 'trace3' (default) | 'legacy' 'legacy' falls back to 'trace3' with a warning for a 3D space. opts.PI — Purity threshold 0.6 (default) | scalar in [0, 1] Target fraction of good instances inside a footprint. Check accepted before treating a training footprint as meeting this target. opts.minInstances — Minimum instances 4 (default) | positive integer Fewer instances than this give an empty footprint. opts.minAreaFrac — Minimum size 0.01 (default) | scalar in [0, 1] A footprint smaller than this fraction of the whole space is discarded. opts.contra — Remove contradictions false (default; true for 'legacy' ) | logical Legacy method only: remove the overlap between best-algorithm footprints. opts.boundaryTolerance — Explicit boundary distance 0 (default) | finite nonnegative scalar Euclidean distance in projection-coordinate units. A positive value includes points within this distance of a fitted edge (2D) or triangular face (3D), including boundaries of holes. Hole interiors farther away remain excluded. Zero preserves exact closed-boundary membership. There is no automatic epsilon multiplier, coordinate scaling or dependence on the query batch. The same rule is used for construction metrics, legacy overlap counts and trimming, and test-set evaluation. Geometry and area/volume are not expanded. The fitted value is saved as out.boundaryTolerance ; evaluation reuses it even if a different option is supplied. Older models without this field use zero. Choose a positive distance from an explicit uncertainty budget in the projection units; this option does not guarantee stable legacy trimming under arbitrary coordinate perturbations. trainedTrace — Trained footprints structure model.trace . Its presence selects evaluation mode. Output Arguments Expand all out — Footprints and summary structure out.space — Whole space The convex hull of all instances: measure (area or volume), measureLabel , elements , density , purity . out.good — Good-performance footprints Cell array, one footprint per algorithm. Each footprint has polygon (an alphaShape ), measure , measureLabel , elements , goodElements , density and purity . out.best — Best-algorithm footprints Same format as out.good . out.hard — Beta-hard footprint out.summary — Summary table Cell array: for each algorithm, the area or volume of its good and best footprints (absolute and as a fracti"},{"t":"What's New","u":"WhatsNew.html","p":"Changes in recent versions of the toolbox","h":["v0.9.2","New Functionality","Correctness and Reporting","Upgrading and Limitations","Documentation","v0.9.1","v0.9.0","See Also"],"x":"The complete list of changes, including every bug fix, is in RELEASE_NOTES.md in the repository root. v0.9.2 New Functionality Independent INIT stage for metadata ingestion, callbacks and saved input snapshots. PRELIM reruns reuse the snapshot; rerunning INIT reloads metadata and invalidates downstream results. Explicit opts.trace.boundaryTolerance , a nonnegative distance in projected coordinates, defaults to zero. Footprints include their exact boundaries; the saved tolerance is reused during exploration. Serial GNU Octave 2D/3D build, save/load and exploration with KNN and none/Sobol tuning, TRACE3, CSV and PNG. Validation uses Octave 11.3, Statistics 2.0.0 and Datatypes 1.5.0. Qt and a display are required for 3D PNG. Bayesian tuning, parallel execution, other classifiers, legacy TRACE, FIG and web palette output remain MATLAB-only. Octave classifier archives are runtime-specific. 3D CLOISTER boundaries and CSV triangle-face exports, including a geometry manifest and explicit empty footprints. Correctness and Reporting Recompute preprocessing after pruning algorithms that are never good, reset SIFTED caches between runs, retain inputs for feature-selection reruns, and apply fitted PLS centering during exploration. Recommendations depend on training policy and new features. Missing test outcomes are excluded from metrics. Oracle probability uses only rows with observed outcomes; selector recall avoids double counting. Classifier summaries distinguish CV from test metrics and probabilities from raw scores. Persist stage options and provenance for safe partial build/save/load/resume; reject incompatible partial rebuilds and replay training feature exclusions during exploration. Evaluate footprints against trained geometry for small or degenerate test batches. Preserve holes in 2D exports and record footprint acceptance and search termination. Replace obsolete toolkit-owned output files on export. Reproducible seeding immediately before SVM posterior calibration, complete numerical projection defaults, constant-column scaling and validation of seed bounds and input data. Upgrading and Limitations Rebuild experiments from the original metadata to obtain corrected results. Loading an existing model does not apply fixes to its fitted artifacts. Back up models and outputs first; rebuild CLOISTER for older 3D models and preprocessing before rerunning SIFTED if the saved feature-selection input is absent. Raw performance must be nonnegative (zero is allowed), and incomplete training data remaining after ingestion filtering is rejected. See Migrating a Legacy Model for pre-v0.9.0 layouts. Classifier CV summaries are conditional on the fitted preprocessing, feature selection, projection and tuning. They are not an outer-CV estimate of the full pipeline. Numerical PILOT restart selection can amplify floating-point differences across BLAS implementations and thread settings, so a seed alone does not promise identical projections across environments. opts.pilot.topoWeight remains reserved and disabled. Documentation Reference documentation is available in the MATLAB Help browser and on the web, with examples for every function. The version header, release notes and citation metadata now identify v0.9.2. v0.9.1 Engineering release; results of the pipeline are unchanged. build and explore accept an 'onStage' callback, called after each stage. obj.plot('boundary') and distribution_boundary.png show the CLOISTER boundary of a 2D space. INIT reads the metadata for both training and evaluation, and PRELIM gained an evaluation mode, so evaluation now breaks ties exactly as training does. opts.general.seed now also controls PILOT and SIFTED . InstanceSpace checks that each stage's inputs exist before running it. Continuous integration and a matlab.unittest test suite. v0.9.0 Complete refactor of the toolbox. The InstanceSpace class is the main interface; buildIS and exploreIS remain as wrappers. 3D instance spaces ( opts.pilot.dims = 3 ) with op"},{"t":"buildIS","u":"buildIS.html","p":"Build an instance space from a folder (backward-compatible wrapper)","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Wrapper around InstanceSpace","See Also"],"x":"Syntax model = buildIS(rootdir) Description model = buildIS(rootdir) builds an instance space from rootdir/metadata.csv and rootdir/options.json , writes model.mat and the output files to rootdir , prints EOF:SUCCESS , and returns the trained model. It is equivalent to Copy buildIS keeps the calling convention of versions before v0.9.0, which the MATILDA web platform and other scripts use. For new code, use InstanceSpace . Examples Expand all Build from a folder Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains metadata.csv and, optionally, options.json . Output Arguments Expand all model — Trained model structure The model property of the built InstanceSpace object. Version History v0.9.0 — Wrapper around InstanceSpace buildIS became a wrapper around InstanceSpace . Its inputs, outputs and files are unchanged. See Also exploreIS | InstanceSpace"},{"t":"exploreIS","u":"exploreIS.html","p":"Evaluate a saved instance space on new instances (backward-compatible wrapper)","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Wrapper around InstanceSpace","See Also"],"x":"Syntax out = exploreIS(rootdir) Description out = exploreIS(rootdir) loads rootdir/model.mat , evaluates rootdir/metadata_test.csv with it, writes the output files to rootdir , prints EOF:SUCCESS , and returns the result. It is equivalent to Copy A model.mat from a version before v0.9.0 is migrated in memory by ISAmigrateModel . For new code, use InstanceSpace . Examples Expand all Evaluate test instances after buildIS Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains model.mat and metadata_test.csv . Output Arguments Expand all out — Evaluation result structure Same fields as a trained model, computed for the test instances. Version History v0.9.0 — Wrapper around InstanceSpace exploreIS became a wrapper around InstanceSpace . Its inputs, outputs and files are unchanged. See Also buildIS | InstanceSpace | ISAmigrateModel"},{"t":"Instance Space Analysis Toolbox","u":"index.html","p":"Objective algorithm testing through the instance space of a problem","h":["Get Started","Getting Started","Stage-by-Stage Walkthrough","Metadata File Format","Options Reference","Functions and Classes","InstanceSpace","Pipeline Stages","Utilities","Output","Topics","Installation","References"],"x":"The Instance Space Analysis (ISA) Toolbox relates the structural features of problem instances to the performance of a portfolio of algorithms. It projects every instance into a 2D or 3D instance space , shows where each algorithm performs well (its footprint ), predicts which algorithm to use for a new instance, and estimates where in the space no instances have been observed yet. The toolbox implements the methodology in Smith-Miles & Muñoz (2023), including the 3D extension (ISA3) of Simpson et al. (2025). It runs in MATLAB R2025a or later and powers the MATILDA web platform. The ISA pipeline: INIT and PRELIM prepare the metadata, SIFTED selects features, PILOT projects them, then CLOISTER, PYTHIA and TRACE work in the projected space. INIT read metadata PRELIM label & normalise SIFTED select features PILOT project to 2D/3D CLOISTER empirical bounds PYTHIA predict algorithms TRACE footprints Outputs CSV · PNG · .mat The ISA pipeline. InstanceSpace runs the stages in this order; each stage is also a standalone function. Get Started Getting Started Build and explore an instance space from the bundled reference data in a few lines. Stage-by-Stage Walkthrough Run each stage separately and inspect what it produces. Metadata File Format Prepare metadata.csv for your own problem domain. Options Reference Every field of opts and options.json , with defaults. Functions and Classes InstanceSpace Build, explore, save, load and plot an instance space. The main entry point. Pipeline Stages INIT, PRELIM, FILTER, SIFTED, PILOT, PILOTviewpoint, CLOISTER, PYTHIA, TRACE. Utilities Option defaults and validation, classifier registry, legacy-model migration. Output Write CSV files and figures, and restore 3D viewpoints. Topics Migrating a Legacy Model — use a model.mat from a version before v0.9.0. Backward-Compatible Wrappers — buildIS and exploreIS , used by MATILDA. Deprecated Functions — PYTHIA2 , PYTHIAtest , SIFTED2 . What's New — release notes. Installation Clone or download the repository, then add it to the MATLAB path from its root folder: Copy Required products: MATLAB R2025a or later with the Statistics and Machine Learning, Optimization, Global Optimization, Parallel Computing, and Financial toolboxes. A bounded serial GNU Octave workflow is also supported; see What's New for supported classifiers, outputs and persistence limits. With the repository root on the path, this documentation also appears in the MATLAB Help browser under Supplemental Software . References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 Simpson, C., Muñoz, M.A., Kandanaarachchi, S. & Campello, R.J.G.B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning , 114, 240. https://doi.org/10.1007/s10994-025-06871-5"},{"t":"scriptcsv","u":"scriptcsv.html","p":"Write the results of an instance space to CSV files","h":["Syntax","Description","Examples","Input Arguments","Version History","v0.9.2 — Footprint boundaries and geometry exports","v0.9.1 — Multiple footprint regions","See Also"],"x":"Syntax scriptcsv(container,rootdir) Description scriptcsv(container,rootdir) writes the coordinates, data, predictions, footprints and summary tables of a trained model or an evaluation result to CSV files in rootdir . Coordinate columns are z_1 , z_2 and, for a 3D space, z_3 . InstanceSpace.build and InstanceSpace.explore call scriptcsv when opts.outputs.csv is true . File Contents coordinates.csv instance coordinates Z bounds.csv , bounds_prunned.csv CLOISTER boundary Zedge and Zecorr (training only) feature_raw.csv , feature_process.csv selected features, before and after preprocessing algorithm_raw.csv , algorithm_process.csv performance, before and after preprocessing algorithm_bin.csv good-performance labels Ybin good_algos.csv , beta_easy.csv , portfolio.csv number of good algorithms, beta-easy flag, best algorithm algorithm_svm.csv , portfolio_svm.csv PYTHIA predictions and selected algorithm footprint__good.csv , footprint__best.csv footprint boundary vertices; separate regions and holes are separated by a row of NaN footprint_performance.csv TRACE summary table classifier_table.csv PYTHIA summary table projection_matrix.csv PILOT projection matrix (training only) Examples Expand all Write the CSV files of an evaluation to another folder Copy Input Arguments Expand all container — Model or evaluation result structure obj.model or an element of obj.testResults . rootdir — Output folder character vector Must exist and end with a file separator. Version History v0.9.2 — Footprint boundaries and geometry exports Geometry exports include empty vertex tables when a footprint is empty. For 3D, each vertex table has a corresponding _faces.csv with one-based triangle indices. CLOISTER uses bounds_faces.csv and bounds_prunned_faces.csv . geometry_manifest.json records dimensionality, footprint filenames, and empty status. Two-dimensional rings are NaN-separated, including holes. The exporter replaces the toolkit-owned footprint_*_best.csv , footprint_*_good.csv , their face tables, and boundary CSVs on each run. Keep user-authored files outside these reserved names. A footprint region with a hole writes both its outer boundary and the hole's boundary, separated by a NaN row. v0.9.1 — Multiple footprint regions Every region of a footprint is written, separated by NaN rows, instead of only the first. See Also scriptpng | scriptweb | InstanceSpace"},{"t":"scriptpng","u":"scriptpng.html","p":"Save figures of an instance space as PNG files","h":["Syntax","Description","Examples","Input Arguments","GNU Octave","Version History","v0.9.2 — Plot export updates and Octave support","v0.9.1 — Boundary figure","v0.9.0 — 3D figures","See Also"],"x":"Syntax scriptpng(container,rootdir) Description scriptpng(container,rootdir) draws the standard views of a trained model or an evaluation result and saves them in rootdir : each feature, each algorithm's performance, good/bad labels, predictions, footprints, the algorithm portfolio, the number of good algorithms, the beta score, instance sources, and the CLOISTER boundary. For a 3D space the figures use the viewpoints found by PILOTviewpoint . Each footprint is also saved as a MATLAB figure ( footprint_.fig , footprint_portfolio.fig ) for interactive rotation, unless opts.outputs.fig is false . ISArecallView returns a rotated figure to its stored viewpoint. InstanceSpace.build and InstanceSpace.explore call scriptpng when opts.outputs.png is true . File Figure distribution_feature_.png feature value over the space distribution_performance_global_normalized_.png , ..._individual_normalized_.png performance, scaled over all algorithms or per algorithm binary_performance_.png , binary_classifier_.png good/bad labels, and PYTHIA's predictions footprint_.png , footprint_portfolio.png footprints distribution_portfolio.png , distribution_svm_portfolio.png best and selected algorithm distribution_number_good_algos.png , distribution_beta_score.png number of good algorithms, beta-easy instances distribution_sources.png instance sources, if the metadata has a source column distribution_boundary.png CLOISTER boundary (training only) Examples Expand all Save figures at a larger font size Copy Input Arguments Expand all container — Model or evaluation result structure obj.model or an element of obj.testResults . rootdir — Output folder character vector Must exist and end with a file separator. GNU Octave Octave exports PNG through print and uses viridis. Qt is selected when available and is required for 3D export; use a virtual display for headless execution. MATLAB retains exportgraphics and parula. FIG output is MATLAB-only. Version History v0.9.2 — Plot export updates and Octave support Added Octave PNG export through Qt, with viridis colours and MATLAB-only FIG output. Direct Octave calls requesting 3D FIG output fail before removing existing output files. Each plot export removes prior toolkit-owned plot filenames before drawing the current run. This clears plots for removed algorithms and features, absent sources or bounds, and old 3D FIG files. Keep user-authored files outside the toolkit's distribution_* , binary_* , and footprint_* output names. distribution_boundary.png is also drawn for a 3D space. v0.9.1 — Boundary figure Adds distribution_boundary.png . v0.9.0 — 3D figures 3D spaces are drawn from their optimised viewpoints and saved as .fig files too. See Also scriptcsv | ISArecallView | PILOTviewpoint | InstanceSpace"},{"t":"scriptweb","u":"scriptweb.html","p":"Write colour-scaled data for the MATILDA web platform","h":["Syntax","Description","Examples","Input Arguments","See Also"],"x":"Syntax scriptweb(container,rootdir) Description scriptweb(container,rootdir) writes the features, performance and number of good algorithms as colour indices, with the colour table, to CSV files in rootdir . The MATILDA web platform uses these files to draw its interactive plots. You do not need them for local work. InstanceSpace.build and InstanceSpace.explore call scriptweb when both opts.outputs.csv and opts.outputs.web are true . File Contents color_table.csv the colour map feature_raw_color.csv , feature_process_color.csv feature colour indices algorithm_raw_color.csv , algorithm_process_color.csv performance colour indices, scaled over all algorithms algorithm_raw_single_color.csv , algorithm_process_single_color.csv performance colour indices, scaled per algorithm good_algos_color.csv colour index of the number of good algorithms Examples Expand all Turn on the web output Copy Input Arguments Expand all container — Model or evaluation result structure rootdir — Output folder character vector Must exist and end with a file separator. See Also scriptcsv | InstanceSpace"}]; +window.ISA_SEARCH = [{"t":"CLOISTER","u":"CLOISTER.html","p":"Estimate the boundary of the instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","GNU Octave","Version History","v0.9.2 — 3D boundaries and Octave support","v0.9.0 — Feature limit","References","See Also"],"x":"Syntax out = CLOISTER(X,A,opts) out = CLOISTER(X,A,opts,Xmean) Description out = CLOISTER(X,A,opts) estimates the region of the instance space where instances could exist, using correlations between features (Correlated Limits of the Instance Space's Theoretical or Experimental Regions). It takes every corner of the box bounded by the minimum and maximum of each feature, discards the corners that contradict a strong, significant correlation between two features, and projects the remaining corners with the PILOT matrix A . The convex hull of the projected corners is the estimated boundary. out = CLOISTER(X,A,opts,Xmean) subtracts the fitted feature mean before projection. For PLS, pass model.pilot.Xmean to keep the boundary in the same coordinates as model.pilot.Z . The three-argument form uses a zero mean. The same centring applies to the projected-instance hull used when the feature-count limit is exceeded. Parts of the boundary far from any instance show where new test instances would extend the benchmark. For a 2D projection the boundary is a closed polygon. For a 3D projection it is a triangulated convex surface; when the projected corners all lie on one plane (for example, a 3D projection of only two features), it is the flat polygon, triangulated. Examples Expand all Plot the boundary of a 2D instance space Copy Plot the boundary of a 3D instance space Copy obj.plot('boundary') draws the same figure over the instances. Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed, selected features, one row per instance. May contain NaN . A — Projection matrix dims -by- nfeats numeric matrix model.pilot.A . opts — Boundary options structure Normally obj.opts.cloister . opts.pval — Significance level 0.05 (default) | scalar in (0, 1) Correlations with a larger p-value are ignored. opts.corrThreshold — Correlation threshold 0.70 (default) | scalar in [0, 1] A corner is discarded when it contradicts the sign of a correlation stronger than this. opts.maxFeatures — Feature limit 20 (default) | positive integer The number of corners grows as 2^nfeats . With more features than this, CLOISTER warns ( ISA:CLOISTER:tooManyFeatures ) and returns the convex hull of the projected instances instead. Xmean — Fitted feature mean 1-by- nfeats numeric row vector | default: zeros(1,nfeats) The mean fitted during PILOT training. Pass model.pilot.Xmean for PLS. Corners and fallback instances are projected as (X-Xmean)*A' ; CLOISTER does not estimate a new mean. Omit the argument for an uncentred projection. Output Arguments Expand all out — Boundary structure out.Zedge — Boundary of all corners 2D: closed polygon, first vertex repeated last. 3D: vertices of the convex hull. out.Zecorr — Boundary of the correlation-consistent corners Same format as Zedge . Equals Zedge when no corner is discarded, when too many corners are discarded to form a hull, or when opts.maxFeatures is exceeded. out.ZedgeFaces , out.ZecorrFaces — Hull triangulations 3D only: nfaces -by-3 indices into the rows of Zedge and Zecorr . Empty in 2D. Tips CLOISTER expects mean-centred features. InstanceSpace warns ( ISA:InstanceSpace:cloisterNotMeanCentred ) when opts.auto.preproc or opts.norm.flag is false . scriptcsv writes Zedge to bounds.csv and Zecorr to bounds_prunned.csv . If every projected corner lies on one line (a degenerate projection matrix or features), there is no region to bound and CLOISTER raises ISA:CLOISTER:degenerateBoundary . GNU Octave Octave supports 2D/3D hulls using namespaced Pearson-significance and NaN-aware extrema adapters. The Pearson adapter requires at least three observations. Degenerate/coplanar cases and PLS centering are covered by portable tests. Version History v0.9.2 — 3D boundaries and Octave support Added serial Octave support for Pearson significance and 2D/3D hulls. An optional fourth argument, Xmean , applies the fitted PLS feature centring during projection. The three-argument call uses a zero mean. Corner enumeration"},{"t":"Deprecated Functions","u":"Deprecated.html","p":"Functions kept only for compatibility with older code","h":["Updating Your Code","See Also"],"x":"These functions still work but print a warning and forward to their replacement. Do not use them in new code. Function Replacement Deprecated in PYTHIA2(Z,Y,Ybin,Ybest,algolabels,opts) PYTHIA with opts.classifier = 'knn' v0.9.0 PYTHIAtest(model,Z,Y,Ybin,Ybest,algolabels) PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,model) (evaluation mode) v0.9.0 SIFTED2(X,Y,Ybin,featlabels,opts) SIFTED (renamed) v0.9.0 Updating Your Code Copy See Also PYTHIA | SIFTED | What's New"},{"t":"FILTER","u":"FILTER.html","p":"Remove near-duplicate instances to even out the instance density","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.2 — Bounded neighbour queries","v0.9.0 — Uniformity output","References","See Also"],"x":"Syntax [subsetIndex,isDissimilar,isVISA,unif] = FILTER(X,Y,Ybin,opts) Description [subsetIndex,isDissimilar,isVISA,unif] = FILTER(X,Y,Ybin,opts) marks instances as redundant when they are closer than opts.mindistance in feature space to an instance already kept, and they also meet the condition in opts.type . Removing them gives a smaller, more evenly spread set of instances, which reduces the bias of densely sampled regions on the projection and the footprints (Alipour et al., 2023). InstanceSpace calls FILTER when opts.selvars.densityflag is true : once on all features during PRELIM, and again after SIFTED on the selected features only. Examples Expand all Filter the reference data Copy Filter inside the pipeline Copy Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed features, one row per instance. Y — Performance matrix numeric matrix Ybin — Good-performance labels logical matrix opts — Filter options structure Normally obj.opts.selvars . opts.mindistance — Distance threshold 0.10 (default) | positive scalar Instances closer than this in feature space are candidates for removal. opts.type — Removal condition 'Ftr&Good' (default) | 'Ftr' | 'Ftr&AP' | 'Ftr&AP&Good' Condition, in addition to feature closeness, for an instance to be removed: Value The instance is removed when 'Ftr' it is close in feature space 'Ftr&AP' the two performance vectors are also within sqrt(nalgos/nfeats)*mindistance 'Ftr&Good' every algorithm is also good on both instances 'Ftr&AP&Good' both of the above Output Arguments Expand all subsetIndex — Redundant instances logical vector true for each instance marked redundant. Keep ~subsetIndex . isDissimilar — Isolated instances logical vector false for each instance that was close to another instance. isVISA — Close but kept logical vector true for an instance that was close to a kept instance in feature space but did not meet the opts.type condition, so it was kept. unif — Uniformity of the kept set scalar One minus the coefficient of variation of the nearest-neighbour distances of the kept instances. Values near 1 mean an even spread. Version History v0.9.2 — Bounded neighbour queries The spatial index is queried only for each retained instance, which bounds stored neighbour lists without changing the greedy row order. v0.9.0 — Uniformity output unif is returned; before, it was computed and discarded. References Alipour, H., Muñoz, M.A. & Smith-Miles, K. (2023). Enhanced instance space analysis for the maximum flow problem. European Journal of Operational Research , 304(2), 411–428. https://doi.org/10.1016/j.ejor.2022.04.012 Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also ISAsubsetData | PRELIM | SIFTED | Options Reference"},{"t":"All Functions","u":"FunctionList.html","p":"Every function and class in the toolbox, by category.","h":["Pipeline Stages","Class Interface","Backward-Compatible Wrappers","Utilities","Output"],"x":"Pipeline Stages Name Purpose INIT Read and filter instance metadata PRELIM Label good performance and normalise features and performance FILTER Remove near-duplicate instances to even out the instance density SIFTED Select the features that best explain algorithm performance PILOT Project instance features onto a 2D or 3D instance space PILOTviewpoint Find the best viewing angles of a 3D instance space CLOISTER Estimate the boundary of the instance space PYTHIA Train classifiers that predict which algorithms perform well TRACE Compute algorithm footprints in the instance space Class Interface Name Purpose InstanceSpace Build, explore and save an instance space Backward-Compatible Wrappers Name Purpose buildIS Build an instance space from a folder (backward-compatible wrapper) exploreIS Evaluate a saved instance space on new instances (backward-compatible wrapper) Utilities Name Purpose ISAdefaults Fill in missing options with their default values ISAvalidateOpts Check the type and range of user-supplied options ISAgetClassifierFcn Look up a classifier in the PYTHIA registry ISAmigrateModel Update a model from an earlier toolbox version to the current layout ISAsubsetData Keep a subset of the instances in a data structure Output Name Purpose scriptcsv Write the results of an instance space to CSV files scriptpng Save figures of an instance space as PNG files scriptweb Write colour-scaled data for the MATILDA web platform ISArecallView Return a 3D figure to its optimised viewpoint"},{"t":"Getting Started with Instance Space Analysis","u":"GettingStarted.html","p":"Build and explore your first instance space","h":["Prepare a Folder","Describe the Performance Measure","Build the Instance Space","Look at the Results","Evaluate New Instances","Next Steps","References"],"x":"This example builds an instance space for the reference dataset shipped with the toolbox: 212 classification problems (the instances ), described by 10 features, on which 10 classification algorithms were scored by misclassification error (Muñoz et al., 2018). A further 23 instances are held out for testing. The same steps apply to your own data once it is in the metadata format . example.m in the repository root runs this example from start to finish. Prepare a Folder The toolbox reads its inputs from, and writes its outputs to, one folder. Copy the reference metadata into a new folder: Copy Describe the Performance Measure Raw algorithm performance must be finite and nonnegative. Zero is allowed; negative values are rejected. Missing test outcomes can be represented by NaN . This restriction applies to raw performance, not features or transformed coordinates. The one decision you must always make is how to judge performance. Here performance is an error rate, so lower is better, and an algorithm counts as good on an instance when its error is below 20%: Copy With AbsPerf = false , an algorithm is good when it is within epsilon (a fraction) of the best algorithm on that instance. Every other option has a default; see the Options Reference . Build the Instance Space Copy build runs the whole pipeline: INIT reads and validates the metadata, PRELIM labels and normalises the data, SIFTED selects features, PILOT projects the instances to 2D, CLOISTER estimates the boundary of the space, PYTHIA trains a classifier per algorithm, and TRACE finds the footprints. It then saves model.mat and writes CSV and PNG files to rootdir . It takes a few minutes, most of it in SIFTED and PYTHIA. Look at the Results Copy The PNG files in rootdir show the same views for every feature and algorithm. Evaluate New Instances Copy explore applies the trained preprocessing, projection, classifiers and footprints to the new instances; nothing is refitted. Next Steps Change an option and re-run only the affected stages — see InstanceSpace . Run and inspect each stage in turn — see the Stage-by-Stage Walkthrough . Build a 3D instance space with opts.pilot.dims = 3 — see PILOT and PILOTviewpoint . Use your own data — see Metadata File Format . References Muñoz, M.A., Villanova, L., Baatar, D. & Smith-Miles, K. (2018). Instance spaces for machine learning classification. Machine Learning , 107(1), 109–147. https://doi.org/10.1007/s10994-017-5629-5 Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895"},{"t":"INIT","u":"INIT.html","p":"Read and filter instance metadata","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","Version History","v0.9.2 — Independent pipeline stage","v0.9.1 — Introduced","References","See Also"],"x":"Syntax [data,extra] = INIT(rootdir,opts) [data,extra] = INIT(rootdir,opts,trainedModel) Description [data,extra] = INIT(rootdir,opts) reads rootdir/metadata.csv , keeps the features and algorithms listed in opts.selvars.feats and opts.selvars.algos (all of them when these fields are absent), and returns the matrices the other stages need. [data,extra] = INIT(rootdir,opts,trainedModel) reads rootdir/metadata_test.csv for evaluation. It checks that the test file has the trained model's features in the same order, and aligns the algorithm columns with the trained model: known algorithms go to their trained position, new algorithms are appended, and trained algorithms missing from the test file get a column of NaN . InstanceSpace.build exposes this as the first stage, init . Its output is cached in model.init.data for subsequent prelim runs. Both build and explore callbacks report init before prelim . Examples Expand all Read the reference metadata Copy Read only some algorithms Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains metadata.csv (training) or metadata_test.csv (evaluation). Must end with a file separator. opts — Toolbox options structure The complete options structure. INIT reads opts.selvars.feats and opts.selvars.algos , cell arrays of column names including their feature_ / algo_ prefix. trainedModel — Trained model structure The model property of a built InstanceSpace object. Its presence selects evaluation mode. Output Arguments Expand all data — Metadata structure data.instlabels — Instance names From the Instances column. data.X , data.Xraw — Features Numeric matrix; both are the unprocessed values at this point. data.Y , data.Yraw — Performance Numeric matrix; both are the unprocessed values at this point. data.algolabels — Algorithm names Without the algo_ prefix. data.featlabels — Feature names Training mode only, without the feature_ prefix. data.S — Instance sources Categorical vector, present only when the file has a source column. extra — Evaluation bookkeeping structure Empty in training mode. In evaluation mode: featlabelsAll , modelalgos (number of trained algorithms), newalgos (algorithms new in the test file), nalgos and ninst . Tips See Metadata File Format for the file layout. Evaluation fails with ISA:InstanceSpace:featureOrderMismatch when metadata_test.csv lists its features in a different order from the training file. Version History v0.9.2 — Independent pipeline stage InstanceSpace exposes init as a selectable build stage, retains its output for preprocessing reruns, and reports it through build and explore callbacks. v0.9.1 — Introduced Replaces two separate readers inside InstanceSpace (build and explore) with one function. References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also PRELIM | InstanceSpace | Metadata File Format"},{"t":"ISAdefaults","u":"ISAdefaults.html","p":"Fill in missing options with their default values","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax opts = ISAdefaults(opts) Description opts = ISAdefaults(opts) returns opts with every missing field set to its default, so that every stage receives a complete options structure. Fields already present are not changed. The defaults are listed in the Options Reference . ISAdefaults also maps a few legacy field names to their current names when the current name is absent: opts.parallel.flag / .ncores to opts.general.parallel / .ncores , opts.pilot.ISA3D to opts.pilot.dims , opts.cloister.cthres to opts.cloister.corrThreshold , opts.pythia.cvfolds to opts.pythia.kFold , and opts.pythia.useknn = false to opts.pythia.classifier = 'svm' . InstanceSpace calls ISAdefaults after ISAvalidateOpts when an object is created or loaded. Examples Expand all See every default Copy Complete a partial options file Copy Input Arguments Expand all opts — Options structure Any subset of the options, for example as decoded from options.json . Output Arguments Expand all opts — Complete options structure Version History v0.9.0 — Introduced Defaults moved from each stage into one function. See Also ISAvalidateOpts | InstanceSpace | Options Reference"},{"t":"ISAgetClassifierFcn","u":"ISAgetClassifierFcn.html","p":"Look up a classifier in the PYTHIA registry","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax [fitFcn,p1label,p2label] = ISAgetClassifierFcn(name) Description [fitFcn,p1label,p2label] = ISAgetClassifierFcn(name) returns the Statistics and Machine Learning Toolbox fitting function for the classifier name , and the names of the hyperparameters that PYTHIA tunes for it. name is the value of opts.pythia.classifier . name Fitting function First hyperparameter (search range) Second hyperparameter 'knn' fitcknn NumNeighbors [1, 25] Distance (categorical) 'svm' fitcsvm BoxConstraint (log2 scale) KernelScale (log2 scale) 'tree' fitctree MinLeafSize [1, 100] — 'nb' fitcnb Bandwidth (log10 scale) — 'linear' fitclinear Lambda (log10 scale) — 'ensemble' fitcensemble NumLearningCycles [10, 200] MinLeafSize [1, 20] Each classifier is binary: PYTHIA trains one per algorithm, so multiclass learners such as fitcecoc are not needed. Examples Expand all Look up the SVM entry Copy Copy Supply fixed hyperparameters for a single-parameter classifier With opts.pythia.tuning = 'none' , opts.pythia.params needs one column per hyperparameter. Copy Input Arguments Expand all name — Classifier name 'knn' | 'svm' | 'tree' | 'nb' | 'linear' | 'ensemble' Case insensitive. Any other value raises ISA:ISAgetClassifierFcn:unknownClassifier . Output Arguments Expand all fitFcn — Fitting function function handle p1label — First hyperparameter name character vector p2label — Second hyperparameter name character vector 'N/A' for classifiers with one hyperparameter. Version History v0.9.0 — Introduced See Also PYTHIA | Options Reference"},{"t":"ISAmigrateModel","u":"ISAmigrateModel.html","p":"Update a model from an earlier toolbox version to the current layout","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — PLS boundary alignment and Octave archives","v0.9.0 — Introduced","See Also"],"x":"Syntax model = ISAmigrateModel(rootdir) model = ISAmigrateModel(rootdir,'backupSuffix',suffix) model = ISAmigrateModel(model) Description model = ISAmigrateModel(rootdir) migrates rootdir/model.mat in place. The original file is first copied to model_legacy.mat in the same folder. Use this form once, to convert a saved model. model = ISAmigrateModel(rootdir,'backupSuffix',suffix) names the backup model.mat . An existing backup is never overwritten; ISAmigrateModel raises ISA:ISAmigrateModel:backupExists instead. model = ISAmigrateModel(model) migrates a model structure already in memory and writes no files. InstanceSpace.load and exploreIS use this form on every load, so a legacy model.mat also works without converting it first. Examples Expand all Convert a saved model Copy Migrate in memory Copy Input Arguments Expand all rootdir — Folder containing model.mat character vector | string scalar model — Model structure structure A model loaded with load('model.mat') . suffix — Backup file suffix '_legacy' (default) | character vector File form only. Output Arguments Expand all model — Migrated model structure Algorithms Legacy content Becomes opts.oracle , opts.pbldr , opts.sbound , opts.footprint opts.pythia , opts.pilot , opts.cloister , opts.trace opts.corr.flag , opts.corr.threshold , opts.clust.flag fields of opts.sifted opts.perf.MaxMin opts.perf.MaxPerf model.data.bestPerformace model.data.Ybest PLS model without pilot.Xmean recover the mean from training features and translate saved cloist.Zedge / Zecorr by -Xmean*A' ; face indices are unchanged model.pythia.svm{i} , model.pythia.knn{i} model.pythia.classifiers{i} model.pythia.boxcosnt , model.pythia.kscale model.pythia.param1 , model.pythia.param2 LIBSVM classifier structures retrained with opts.pythia.classifier (default 'knn' ), because LIBSVM structures have no predict method model.trace in the DBSCAN/polyshape format recomputed with TRACE3, using model.pythia.Yhat when available, otherwise model.data.Ybin no completedStages inferred from the stage fields present A model.pilot.A without B and C cannot be repaired and raises a warning. GNU Octave File-based migration recognizes versioned Octave archives and preserves their explicit reconstruction format. MATLAB legacy migration retains its existing file format. Version History v0.9.2 — PLS boundary alignment and Octave archives File-based migration now reads versioned Octave model archives. Schema-1 archives remain readable; saving writes schema 2, which keeps ordinary struct fields separate from internal object markers. Recovering a missing PLS feature mean also translates any saved CLOISTER vertices into the fitted PLS coordinate system. Empty or absent boundaries are retained. Repeated migration leaves already centred geometry unchanged. Models without the training features needed to recover the mean must be rebuilt. v0.9.0 — Introduced See Also Migrating a Legacy Model | InstanceSpace | exploreIS"},{"t":"ISArecallView","u":"ISArecallView.html","p":"Return a 3D figure to its optimised viewpoint","h":["Syntax","Description","Examples","Input Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax ISArecallView(fig) ISArecallView(fig,algoIdx) Description ISArecallView(fig) sets the camera of the 3D instance-space figure fig to the global viewpoint found by PILOTviewpoint . Use it after rotating a figure saved by scriptpng . ISArecallView(fig,algoIdx) uses the viewpoint of the group in opts.pilot.viewGroups that contains algorithm algoIdx , or the global viewpoint if no group contains it. The viewpoints are stored in the figure's UserData by scriptpng . Examples Expand all Restore the view of a saved footprint Copy Input Arguments Expand all fig — Figure figure handle A 3D figure saved by scriptpng . algoIdx — Algorithm index [] (default) | positive integer Column of the algorithm in model.data.algolabels . Version History v0.9.0 — Introduced See Also PILOTviewpoint | scriptpng"},{"t":"ISAsubsetData","u":"ISAsubsetData.html","p":"Keep a subset of the instances in a data structure","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax data = ISAsubsetData(data,subsetIndex) data = ISAsubsetData(data,subsetIndex,featIdx) Description data = ISAsubsetData(data,subsetIndex) keeps the rows subsetIndex of every per-instance field of data : X , Y , Xraw , Yraw , Ybin , beta , numGoodAlgos , Ybest , P , instlabels and, if present, S . data = ISAsubsetData(data,subsetIndex,featIdx) also keeps only the feature columns featIdx of data.X and data.featlabels . InstanceSpace uses this form after SIFTED when density filtering is on. Examples Expand all Keep the beta-easy instances Copy Input Arguments Expand all data — Instance data structure model.data , with the fields listed above. subsetIndex — Instances to keep logical vector | vector of indices featIdx — Features to keep vector of indices Output Arguments Expand all data — Subset structure Version History v0.9.0 — Introduced See Also FILTER | InstanceSpace"},{"t":"ISAvalidateOpts","u":"ISAvalidateOpts.html","p":"Check the type and range of user-supplied options","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax opts = ISAvalidateOpts(opts) Description opts = ISAvalidateOpts(opts) checks each recognised field that is present in opts and raises an error on the first invalid one. Absent fields are not checked; ISAdefaults fills them afterwards. opts is returned unchanged. An error here names the field and the expected value, instead of failing later inside a stage. A field set to [] counts as present and is rejected. Field names the function does not recognise, including misspellings, are not reported. InstanceSpace calls ISAvalidateOpts when an object is created. Examples Expand all Catch an invalid value early Copy Input Arguments Expand all opts — Options structure Output Arguments Expand all opts — Options structure The input, unchanged. Tips Error identifiers, by the check that failed: Identifier Expected ISA:ISAvalidateOpts:notStruct a structure, for opts and each group such as opts.pilot ISA:ISAvalidateOpts:notLogical a logical scalar, true or false (not 0 or 1) ISA:ISAvalidateOpts:notFiniteNumericScalar a finite real number ISA:ISAvalidateOpts:notInteger a whole number ISA:ISAvalidateOpts:notPositive a positive number (0 is allowed for seeds) ISA:ISAvalidateOpts:notInUnitRange a number in [0, 1] ISA:ISAvalidateOpts:notMember one of a fixed set of values ISA:ISAvalidateOpts:notText a character vector or string ISA:ISAvalidateOpts:notCellOfText a cell array of character vectors ISA:ISAvalidateOpts:badViewGroups a cell array of algorithm index vectors Version History v0.9.0 — Introduced See Also ISAdefaults | InstanceSpace | Options Reference"},{"t":"InstanceSpace","u":"InstanceSpace.html","p":"Build, explore and save an instance space","h":["Syntax","Description","Examples","Input Arguments","Properties","Object Functions","build","explore","getResults","plot","save","load","GNU Octave","Version History","v0.9.2 — Stage validation and Octave workflows","v0.9.1 — Stage callbacks and boundary plot","v0.9.0 — Introduced","References","See Also"],"x":"Syntax obj = InstanceSpace(rootdir) obj = InstanceSpace(rootdir,opts) Description An InstanceSpace object holds the options, the trained model, and the evaluation results of one instance space analysis. It runs the pipeline stages INIT , PRELIM , SIFTED , PILOT , CLOISTER , PYTHIA and TRACE in order, lets you change options and re-run individual stages, evaluates the trained model on new instances, and saves and loads the model. obj = InstanceSpace(rootdir) creates an object for the metadata in rootdir/metadata.csv . Options are read from rootdir/options.json if the file exists; missing options take their defaults. No computation runs until you call build . obj = InstanceSpace(rootdir,opts) uses the options structure opts instead of options.json . During build , algorithms with no good training instances are removed. Preprocessing is then fitted to the retained portfolio. Best-algorithm indices, best performance, beta-easy flags, and saved transforms all refer to that portfolio. With an absolute threshold, removing an algorithm can change the best performance on an instance where no algorithm is good. InstanceSpace is a value class: a method that changes the object returns the changed copy, so assign the result, as in obj = obj.build() . Examples Expand all Build and explore an instance space Copy Change an option and re-run one stage Re-running a stage discards the results of every later stage, which you then re-run. Copy Inspect each stage as it finishes Copy Save and load a model Copy Input Arguments Expand all rootdir — Data folder character vector | string scalar Folder containing metadata.csv . Output files and model.mat are written here. See Metadata File Format . opts — Options structure Any subset of the fields in the Options Reference . Values are checked by ISAvalidateOpts , and missing fields filled by ISAdefaults . Properties Expand all rootdir — Data folder character vector Always ends with a file separator. opts — Options structure Complete options. Change fields between build calls to re-run stages with new settings. explore uses the options stored with the model at training time instead. model — Trained model structure One field per completed stage: data , prelim , featsel , sifted , pilot , cloist (CLOISTER), pythia , trace , and opts once every stage has run. completedStages — Completed stages cell array of character vectors testDirs , testResults — Evaluation results cell arrays One entry per explore call: the folder and the result structure, which has the same fields as model . Object Functions Function Purpose build Run pipeline stages explore Evaluate the trained model on new instances getResults Return the training or an evaluation result plot Plot a view of the instance space save Write model.mat InstanceSpace.load Create an object from model.mat build Copy Runs every stage, or only the named stages from {'init','prelim','sifted','pilot','cloister','pythia','trace'} , always in pipeline order. A stage whose prerequisites have not completed raises ISA:InstanceSpace:missingPrereq . callback(stageName, model) is called after each stage, including init . For compatibility, requesting prelim automatically runs init first if its input snapshot is missing. init loads and filters metadata, publishes it in model.data , and retains an independent input snapshot in model.init.data . Rerunning prelim reuses that snapshot. To reread a changed metadata file, rerun init ; this invalidates every later stage. Changes to selvars.feats , selvars.algos , or prelim.nanThreshold require rerunning init . Full builds retain the same computations and numerical outputs. Saved models now also contain the input snapshot. Legacy models without this snapshot remain usable for exploration; rebuilding their prelim stage automatically runs init and therefore requires the original metadata file. When all stages have completed, build saves model.mat and writes the outputs selected in opts.outputs with scriptcsv , scriptweb and scriptpng"},{"t":"Stage-by-Stage Walkthrough","u":"InteractiveWalkthrough.html","p":"Run each stage of the pipeline and inspect its output","h":["PRELIM: Label and Normalise","SIFTED: Select Features","PILOT: Project to 2D","CLOISTER: Estimate the Boundary","PYTHIA: Predict Good Algorithms","TRACE: Find the Footprints","Save and Explore","See Also"],"x":"This walkthrough builds the instance space one stage at a time so you can see what each stage produces. It follows liveDemoIS.m in the repository root, which you can open in the Live Editor and run section by section. It uses the same reference data and performance settings as Getting Started . Copy PRELIM: Label and Normalise PRELIM decides which algorithms are good on each instance, bounds outlying feature values, and normalises features and performance. Copy An instance is beta-easy when more than opts.perf.betaThreshold of the algorithms are good on it. SIFTED: Select Features SIFTED keeps the features that best explain performance: a correlation filter, then a genetic algorithm that picks one feature from each cluster of correlated features. Copy Set opts.sifted.flag = false to keep every feature. PILOT: Project to 2D PILOT finds the linear projection Z = X*A' from which features and performance can best be reconstructed. Z is the instance space. Copy opts.pilot.dims = 3 gives a 3D space; opts.pilot.method = 'pls' uses Partial Least Squares. CLOISTER: Estimate the Boundary CLOISTER estimates where instances could exist, given the ranges and correlations of the features. Empty areas inside the boundary are candidates for new test instances. Copy PYTHIA: Predict Good Algorithms PYTHIA trains one classifier per algorithm that predicts, from an instance's position, whether the algorithm is good there, and combines them into an algorithm selector. Copy TRACE: Find the Footprints TRACE finds the region where each algorithm is good (its footprint) and where it is best, and measures their size, density and purity. Copy Save and Explore Once every stage has run, build has already saved model.mat and written the output files. explore evaluates new instances with the trained model: Copy To change a setting later, edit obj.opts and re-run from the stage it affects, for example obj = obj.build('stages', {'pythia','trace'}) after changing opts.pythia.classifier . Re-running a stage discards the results of later stages. See Also InstanceSpace | Getting Started | Options Reference"},{"t":"Metadata File Format","u":"MetadataFormat.html","p":"Prepare metadata.csv for your own problem domain","h":["Columns","Example","Missing Values","Test Instances","Choosing Features","Version History","v0.9.2 — Training feature schema","References","See Also"],"x":"An instance space is built from one CSV file, metadata.csv , with one row per problem instance. Instances to evaluate later with explore go in metadata_test.csv , in the same format. Columns Columns are recognised by their header. The order of the column groups does not matter, and headers are case insensitive. Header Required Contents Instances yes Instance name. Text or numbers. Source no Where the instance comes from, for example the benchmark suite. Drawn by obj.plot('sources') . feature_ at least two Numeric feature value. algo_ at least one Numeric performance of the algorithm on the instance. Other columns are ignored. Names after the prefix must be valid MATLAB identifiers: use underscores instead of spaces, for example feature_edge_density or algo_simulated_annealing . Example The first rows of the reference data ( test/data/metadata.csv ), shortened to three features and three algorithms: Copy Missing Values Leave a cell empty or write NaN for a missing value. A feature with at least opts.prelim.nanThreshold missing values is removed. After filtering, training requires complete features and performance values. Remaining missing values cause an error that names the affected instances. Evaluation requires complete retained features but permits missing performance values, which never count as observed failures in classifier scoring. Avoid NA , spreadsheet error codes such as #DIV/0! , and empty rows: they make MATLAB read the whole column as text, which then fails later in the pipeline. Test Instances metadata_test.csv must contain the same feature_ columns as metadata.csv , in the same order. It may contain: a subset of the trained algorithms — the missing ones are not scored; algorithms that were not trained — they are appended to the results, without a classifier. Choosing Features The features determine what the instance space can reveal. Good features: are cheap to compute compared with running the algorithms; capture properties that plausibly affect how hard an instance is for each algorithm; vary across the instances, with few repeated values. Start with more candidate features than you need; SIFTED removes the ones that do not explain performance. See Smith-Miles & Muñoz (2023) for guidance on feature design. Version History v0.9.2 — Training feature schema Training stores the feature columns removed by nanThreshold . Exploration removes those same columns before checking the required feature schema, even when their test values are complete. References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also INIT | InstanceSpace | Options Reference"},{"t":"Migrating a Legacy Model","u":"MigratingLegacyModel.html","p":"Use a model.mat created by a toolbox version before v0.9.0","h":["Automatic Migration on Load","Converting the File Once","What Changes","When Migration Cannot Complete","See Also"],"x":"Version 0.9.0 renamed several option groups and model fields and replaced the LIBSVM classifiers and the footprint algorithm. A model.mat saved by an earlier version therefore needs migrating before the current code can use it. In most cases this is automatic. Automatic Migration on Load InstanceSpace.load and exploreIS migrate a legacy model in memory each time they load it. Nothing is written to disk: Copy Converting the File Once To avoid repeating the migration, including any classifier retraining, on every load, convert the file: Copy The original is kept as model_legacy.mat . Choose another backup name with ISAmigrateModel(rootdir,'backupSuffix','_v08') . What Changes Options are renamed to the current groups: opts.oracle → opts.pythia , opts.pbldr → opts.pilot , opts.sbound → opts.cloister , opts.footprint → opts.trace , and opts.perf.MaxMin → opts.perf.MaxPerf . The feature-selection flags of opts.corr and opts.clust move to opts.sifted . Classifiers. LIBSVM models cannot be evaluated by the current code. They are retrained with the classifier in opts.pythia.classifier ( 'knn' if unset) on the training data stored in the model. Predictions of the retrained model can differ slightly from those of the original. Footprints in the old DBSCAN/polyshape format are recomputed with TRACE3. Their areas, densities and purities can differ from the values reported by the old version. Field names such as data.bestPerformace are corrected. The complete table is on the ISAmigrateModel reference page. When Migration Cannot Complete A projection without its reconstruction matrices ( model.pilot.B , model.pilot.C ) is kept as is, with a warning. Retraining LIBSVM classifiers and recomputing footprints need the training data stored in the model ( model.pilot.Z and model.data ). When it is missing, the fields are renamed only, with a warning. If a LIBSVM model has to be evaluated as-is, PYTHIA raises ISA:PYTHIA:noLibsvm unless the LIBSVM MEX files are on the path. The toolbox does not include them; get them from the LIBSVM project . Retraining is recommended instead. See Also ISAmigrateModel | InstanceSpace | What's New"},{"t":"Options Reference","u":"OptionsReference.html","p":"Every field of the options structure, with its default","h":["opts.general","opts.perf","opts.prelim","opts.auto","opts.bound","opts.norm","opts.selvars","opts.sifted","opts.pilot","opts.cloister","opts.pythia","opts.trace","opts.outputs","Version History","v0.9.2 — Option validation","See Also"],"x":"Options control every stage of the pipeline. Give them to InstanceSpace as a structure, or as an options.json file in the data folder with the same nesting: Copy Copy Set only the fields you want to change. ISAvalidateOpts checks the fields you set, and ISAdefaults fills in the rest with the defaults below. Between staged build calls you can change obj.opts directly. opts.general Settings for the whole pipeline. Field Default Description seed 42 Random seed. Every stochastic stage derives its seed from this value, so the same seed and data reproduce a run. verbose true Detailed progress output. Stage start and end messages are always printed. parallel false Use a parallel pool (Parallel Computing Toolbox) for SIFTED, PILOT, PYTHIA and TRACE. ncores 18 Workers when opening a pool. An existing pool is reused unchanged. opts.perf How performance is judged. Used by PRELIM . Observed raw performance must be finite and nonnegative. Zero is allowed; negative scores are rejected in both absolute and relative modes. Relative mode substitutes machine epsilon for exact zero numerators and denominators; see PRELIM for the resulting zero-reference convention. Field Default Description MaxPerf false true if larger performance values are better; false for a cost such as error or run time. AbsPerf false true : good means better than epsilon . false : good means within a fraction epsilon of the best algorithm on the instance. epsilon 0.05 Good-performance threshold. In [0, 1] when AbsPerf is false ; any real number otherwise. betaThreshold 0.55 An instance is beta-easy when more than this fraction of the algorithms are good on it. opts.prelim Data preparation. Used by INIT and PRELIM . Field Default Description iqrMultiplier 5 Features are bounded to median ± iqrMultiplier*IQR . nanThreshold 0.20 A feature with at least this fraction of missing values is removed. opts.auto Used by PRELIM . Field Default Description preproc true Run the automatic outlier bounding and normalisation. false leaves the data as given. opts.bound Used by PRELIM . Field Default Description flag true Bound feature outliers. Features with very little variation can have an IQR of zero; remove them or turn bounding off. opts.norm Used by PRELIM . Field Default Description flag true Apply Box-Cox and Z-score transforms, so features and performance are close to normally distributed. Recommended, because PILOT is a linear projection and CLOISTER expects centred data. opts.selvars Which features, algorithms and instances to use. Used by INIT , InstanceSpace and FILTER . Field Default Description feats all Cell array of feature column names to use, with their feature_ prefix. algos all Cell array of algorithm column names to use, with their algo_ prefix. smallscaleflag false Build from a random fraction of the instances. Useful to try settings on a large dataset. smallscale 0.30 Fraction of instances kept when smallscaleflag is true . fileidxflag false Build from the instances listed in the file fileidx . fileidx '' CSV file with one column of instance indices (row numbers of metadata.csv ). densityflag false Remove near-duplicate instances with FILTER . mindistance 0.10 FILTER distance threshold in feature space. type 'Ftr&Good' FILTER removal condition: 'Ftr' , 'Ftr&AP' , 'Ftr&Good' or 'Ftr&AP&Good' . opts.sifted Feature selection. Used by SIFTED . Field Default Description diagnostics true Run the advisory silhouette sweep. Disable it to reduce selection overhead. flag true Run SIFTED. false keeps every feature. rho 0.10 Minimum absolute correlation between a feature and an algorithm's performance. pval 0.05 Significance level of the correlations. K 10 Number of feature clusters, which is the number of features selected. We recommend 10 or fewer. MaxIter 1000 Maximum k-means iterations. Replicates 100 Number of k-means replicates. seed opts.general.seed Random seed of SIFTED. opts.pilot Projection. Used by PILOT and PILOTviewpoint . Field Default Description dims 2 Dimensio"},{"t":"PILOT","u":"PILOT.html","p":"Project instance features onto a 2D or 3D instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Projection corrections and Octave support","v0.9.1 — Seed control","v0.9.0 — 3D projections and PLS","References","See Also"],"x":"Syntax out = PILOT(X,Y,featlabels,opts) Description out = PILOT(X,Y,featlabels,opts) finds a linear projection Z = X*A' of the feature matrix X into opts.dims dimensions, chosen so that both the features and the algorithm performance Y can be reconstructed from Z as well as possible (Prediction-based Linear Dimensionality Reduction). Instances with similar features and similar performance land close together, so trends in performance become visible as directions in the space. PILOT has three solvers, selected by opts.method and opts.analytic : Numerical (default): BFGS minimisation of the reconstruction error of X and Y from Z (see Algorithms), restarted opts.ntries times. The trial whose projection best preserves the pairwise distances of the original data is kept. Analytic : the closed-form eigenvector solution. Fast and deterministic. If X is rank deficient, PILOT falls back to the numerical solver. PLS : Partial Least Squares regression of Y on X with plsregress . opts.alpha does not apply. Examples Expand all Project the reference data in 2D Prepare the features with the earlier pipeline stages, then call PILOT directly. Copy Compare the solvers The analytic solver is deterministic and much faster than BFGS. Compare how well each reconstructs the data. Copy Build a 3D instance space Set opts.pilot.dims to 3 before building. InstanceSpace then also calls PILOTviewpoint to find the best 2D viewing angles. Copy Input Arguments Expand all X — Feature matrix numeric matrix Instance features, one row per instance and one column per feature, normally the output of PRELIM and SIFTED . Y — Performance matrix numeric matrix Algorithm performance, one row per instance and one column per algorithm. Must have the same number of rows as X . featlabels — Feature names cell array of character vectors One name per column of X . Used for the row labels of out.summary . opts — Projection options structure Normally obj.opts.pilot . Missing fields take the defaults set by ISAdefaults . opts.dims — Number of dimensions 2 (default) | 3 Dimension of the instance space. opts.method — Projection method 'standard' (default) | 'pls' 'standard' uses the analytic or numerical PILOT solution. 'pls' uses Partial Least Squares. opts.analytic — Use the closed-form solution false (default) | true Only used when opts.method is 'standard' . Falls back to the numerical solver when X is rank deficient. opts.ntries — Number of BFGS restarts 10 (default) | positive integer More restarts reduce the chance of a poor local minimum and take proportionally longer. opts.alpha — Weight of the performance term 1.0 (default) | positive scalar Weight of ||Y - C*Z||^2 relative to the feature term. Larger values favour a projection that explains performance over one that explains features. Not used by 'pls' . opts.seed — Random seed opts.general.seed (default) | integer Seeds the BFGS starting points, so repeated runs give the same projection. opts.verbose — Report progress opts.general.verbose (default) | logical opts.X0 — Starting points numeric matrix Optional user-supplied BFGS starting points, one column per trial. opts.precalcAlpha — Precomputed solution numeric vector Optional full BFGS solution vector from an earlier run. Skips the optimisation. Output Arguments Expand all out — Projection structure out.A — Projection matrix dims -by- nfeats matrix. Project new, identically preprocessed instances with Z = Xnew*out.A' . out.Z — Instance coordinates ninst -by- dims coordinates of each instance in the instance space. out.B , out.C — Reconstruction matrices Reconstruct the features and the performance from Z . out.error — Reconstruction error Sum of squared errors of the reconstruction of [X Y] . out.R2 — Coefficient of determination R^2 of the reconstruction of each column of [X Y] . out.summary — Projection table Cell array showing A with the feature labels. Also written to projection_matrix.csv by scriptcsv . out.alpha , out.X0 , out.eoptim , out.perf — Trial results"},{"t":"PILOTviewpoint","u":"PILOTviewpoint.html","p":"Find the best viewing angles of a 3D instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Camera conventions and Octave support","v0.9.0 — Introduced","References","See Also"],"x":"Syntax out = PILOTviewpoint(Z,Y,opts) Description out = PILOTviewpoint(Z,Y,opts) finds, for each group of algorithms in opts.viewGroups , the 2D view of the 3D projection Z from which the performance of those algorithms is best explained. Each view is a pair of orthogonal directions v1 , v2 ; the returned azimuth and elevation point the MATLAB camera along cross(v1,v2) (Simpson et al., 2025, Equation 2). InstanceSpace calls PILOTviewpoint after PILOT when opts.pilot.dims is 3 and stores the result in model.pilot.viewpoint . scriptpng then renders 3D figures from these angles. Examples Expand all View a 3D instance space from its best angle Copy One viewpoint per group of algorithms Copy Input Arguments Expand all Z — 3D instance coordinates ninst -by-3 numeric matrix model.pilot.Z from a 3D build. Y — Performance matrix numeric matrix Same number of rows as Z . opts — Viewpoint options structure Normally obj.opts.pilot . opts.viewGroups — Algorithm groups {} (default) | cell array of index vectors One viewpoint is computed per group, from the columns of Y in that group. Empty means one viewpoint for all algorithms. opts.ntries — Number of BFGS restarts 10 (default) | positive integer opts.seed — Random seed opts.general.seed (default) | integer opts.X0 — Starting points numeric matrix Optional, 2*3 + 2*numel(group) rows by one column per trial. Used for each group of matching size. Output Arguments Expand all out — Viewpoints structure out.groups — Algorithm groups The groups used, one cell per viewpoint. out.A — View matrices Cell array; each cell is a 2-by-3 matrix [v1; v2] that flattens Z onto the viewing plane. out.azimuth , out.elevation — Camera angles In radians, one per group. Use view(rad2deg(az), rad2deg(el)) . Algorithms For each group, PILOTviewpoint jointly fits the view A (2-by-3) and a performance reconstruction C by BFGS, minimising Copy The second term keeps the two view directions orthogonal. Like PILOT, it keeps the trial whose view best preserves the pairwise distances of Z . GNU Octave Octave runs the viewpoint optimization serially through a namespaced fminunc option adapter. MATLAB keeps native optimization options. Equal starting seeds do not guarantee identical optimized camera angles. Version History v0.9.2 — Camera conventions and Octave support Added serial Octave viewpoint optimization with runtime-specific optimizer options. Azimuth and elevation use the cart2sph convention in radians. Plot helpers convert azimuth to the MATLAB camera convention and use equal data-axis scales to preserve the fitted plane. v0.9.0 — Introduced Part of the 3D instance space (ISA3) support. References Simpson, C., Muñoz, M.A., Kandanaarachchi, S. & Campello, R.J.G.B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning , 114, 240. https://doi.org/10.1007/s10994-025-06871-5 See Also PILOT | ISArecallView | scriptpng"},{"t":"PRELIM","u":"PRELIM.html","p":"Label good performance and normalise features and performance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Performance labels and Octave normalization","v0.9.1 — Evaluation mode","v0.9.0 — Outlier bound option","References","See Also"],"x":"Syntax [X,Y,out] = PRELIM(X,Y,opts) [X,Y,out] = PRELIM(X,Y,opts,trainedPrelim) Description [X,Y,out] = PRELIM(X,Y,opts) fits the preprocessing to the training data and applies it. PRELIM labels each algorithm as good or not good on each instance, finds the best algorithm per instance, bounds feature outliers, and applies Box-Cox and Z-score transforms to the features and the performance. The fitted bounds and transform parameters are returned in out . [X,Y,out] = PRELIM(X,Y,opts,trainedPrelim) applies the bounds and transforms of an earlier training call, trainedPrelim , to new data instead of fitting new ones. The good/best labelling is the same code in both modes, so training and evaluation label ties the same way. Examples Expand all Preprocess a performance matrix Misclassification error is a cost ( MaxPerf = false ). Call an algorithm good when its error is below 0.20. Copy Apply a trained preprocessing to new instances Copy Input Arguments Expand all X — Feature matrix numeric matrix One row per instance, one column per feature. May contain NaN . Y — Performance matrix numeric matrix One row per instance, one column per algorithm. Observed raw performance must be finite and nonnegative in both training and evaluation. Zero is allowed; negative values raise ISA:PRELIM:invalidPerformance . May contain NaN ; a missing value never counts as good. opts — Preprocessing options structure When PRELIM is called by InstanceSpace , this structure is assembled from opts.perf , opts.auto , opts.bound , opts.norm and opts.prelim . opts.MaxPerf — Performance direction false (default) | true true if larger performance values are better (for example accuracy). false for a cost, such as run time or error. Set from opts.perf.MaxPerf . opts.AbsPerf — Threshold type false (default) | true true : an algorithm is good when its performance is better than epsilon . false : an algorithm is good when its performance is within a fraction epsilon of the best algorithm on that instance. Set from opts.perf.AbsPerf . opts.epsilon — Good-performance threshold 0.05 (default) | scalar Absolute threshold, or relative tolerance, depending on AbsPerf . Set from opts.perf.epsilon . opts.betaThreshold — Easy-instance fraction 0.55 (default) | scalar in [0, 1] An instance is beta-easy when more than this fraction of the algorithms are good on it. Set from opts.perf.betaThreshold . opts.auto — Run the preprocessing true (default) | false false skips the outlier bounds and the transforms; only the labelling runs. Set from opts.auto.preproc . opts.bound — Bound outliers true (default) | false Clip each feature to median ± iqrMultiplier*IQR . Set from opts.bound.flag . opts.norm — Normalise true (default) | false Apply Box-Cox and then Z-score transforms to X and Y . Set from opts.norm.flag . opts.iqrMultiplier — Outlier bound width 5 (default) | positive scalar Set from opts.prelim.iqrMultiplier . trainedPrelim — Trained preprocessing structure The out of an earlier training-mode call. Its presence selects evaluation mode. Output Arguments Expand all X — Preprocessed features numeric matrix Y — Preprocessed performance numeric matrix With AbsPerf = false , Y is first converted to performance relative to the best algorithm, then normalised. out — Labels and fitted parameters structure out.Ybin — Good-performance labels ninst -by- nalgos logical matrix. out.Ybest — Best performance Best raw performance on each instance. out.P — Best algorithm Index of the best algorithm on each instance. Ties are broken at random. out.numGoodAlgos — Number of good algorithms Per instance. out.beta — Beta-easy instances Logical vector; true when more than betaThreshold of the algorithms are good. out.hibound , out.lobound , out.medval , out.iqrange — Outlier bounds Training mode only. out.minX , out.lambdaX , out.muX , out.sigmaX — Feature transform Training mode only. Shift, Box-Cox parameter, mean and standard deviation per feature. out.minY , out.lambdaY , out.muY , out.sigmaY — P"},{"t":"PYTHIA","u":"PYTHIA.html","p":"Train classifiers that predict which algorithms perform well","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","GNU Octave","Version History","v0.9.2 — Evaluation corrections and Octave KNN","v0.9.0 — Classifier registry","References","See Also"],"x":"Syntax out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts) out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,trainedModel) Description out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts) trains one binary classifier per algorithm that predicts, from an instance's coordinates Z , whether the algorithm is good on it (Performance Yielding Heuristic Identification of Algorithms). Hyperparameters are tuned by k -fold cross-validation. PYTHIA then combines the predictions into an algorithm selector: for each instance it picks the algorithm predicted to be good that has the highest cross-validated precision. out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,trainedModel) applies the classifiers of an earlier training call to new instances and reports how accurate they are there. Classifiers are not retrained. The classifier type is set by opts.classifier ; see ISAgetClassifierFcn for the registry and the tuned hyperparameters. Examples Expand all Train k-NN classifiers on a 2D instance space Copy Use an SVM with Bayesian optimisation Copy Evaluate trained classifiers on new instances InstanceSpace.explore does this for you. The explicit call is: Copy Input Arguments Expand all Z — Instance coordinates numeric matrix model.pilot.Z . PYTHIA standardises it internally and stores the mean and standard deviation. Y — Raw performance matrix numeric matrix model.data.Yraw . Used for the performance columns of the summary and, with opts.useweights , for the class weights. NaN marks a missing value. Ybin — Good-performance labels logical matrix The classification targets. Ybest — Best performance per instance numeric vector algolabels — Algorithm names cell array of character vectors opts — Classifier options structure Normally obj.opts.pythia . opts.classifier — Classifier type 'knn' (default) | 'svm' | 'tree' | 'nb' | 'linear' | 'ensemble' opts.tuning — Hyperparameter search 'sobol' (default) | 'bayes' | 'none' 'sobol' : a scrambled Sobol sequence of nTuningIter candidates. 'bayes' : bayesopt with the same budget. 'none' : use opts.params . opts.nTuningIter — Search budget 20 (default) | positive integer opts.kFold — Cross-validation folds 5 (default) | positive integer opts.params — Fixed hyperparameters [] (default) | nalgos -by- nparams matrix Required when opts.tuning is 'none' ; one row per algorithm. opts.useweights — Cost-sensitive training false (default) | true Weight each instance by how far its performance is from the mean performance. opts.ispolykrnl — Polynomial kernel false (default) | true SVM only. false uses a Gaussian kernel. opts.ensembleMethod — Ensemble method 'Bag' (default) | any fitcensemble method Only for 'ensemble' . opts.skip — Skip training false (default) | true Return empty predictions without training. TRACE then uses Ybin directly. opts.seed — Random seed opts.general.seed (default) | integer opts.verbose — Report progress opts.general.verbose (default) | logical trainedModel — Trained classifiers structure The out of an earlier training call ( model.pythia ). Its presence selects evaluation mode. Output Arguments Expand all out — Classifiers, predictions and summary structure out.classifiers — Trained classifiers Cell array with one ClassificationModel per algorithm. out.Yhat — Predicted labels ninst -by- nalgos logical matrix. In training mode these are the predictions of the final model on the training data. out.Pr0hat — Predicted bad-class scores Bad-class score for each algorithm on each instance. Consult the corresponding out.scoreType entry before interpreting a column: only probability denotes the probability that the algorithm is not good. decision-score and class-score are classifier scores and need not lie in [0,1]; unknown has no declared score semantics, and unavailable marks a placeholder rather than a usable score. out.Ysub , out.Pr0sub — Cross-validated predictions Training mode only. Ysub contains predicted labels; Pr0sub contains bad-class scores. out.scoreTypeCV describes each score column. A mixed column combines "},{"t":"SIFTED","u":"SIFTED.html","p":"Select the features that best explain algorithm performance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","GNU Octave","Version History","v0.9.2 — Diagnostics, cache handling and Octave support","v0.9.1 — Seed control","v0.9.0 — Renamed from SIFTED2","References","See Also"],"x":"Syntax [X,out] = SIFTED(X,Y,Ybin,featlabels,opts) Description [X,out] = SIFTED(X,Y,Ybin,featlabels,opts) returns the columns of X that best explain algorithm performance, and the indices of those columns in out.selvars . Selection has two steps: Correlation filter. For every algorithm, keep the feature most correlated with its performance. Also keep every feature whose absolute Pearson correlation with the performance of at least one algorithm is at least opts.rho and significant at level opts.pval . Clustering and genetic search. Group the remaining features into opts.K clusters by correlation distance (k-means). A genetic algorithm then picks one feature per cluster, scoring each candidate set by how well a k -nearest-neighbour classifier predicts Ybin in a PILOT projection of those features. Step 2 is skipped when three or fewer features remain, or when no more than opts.K remain. Examples Expand all Select features from the reference data Copy Check the choice of K out.eva holds the silhouette value for each number of clusters. When the value for opts.K is poor, SIFTED also suggests a better K . Copy Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed features, one row per instance. Y — Performance matrix numeric matrix Preprocessed performance, one row per instance. Ybin — Good-performance labels logical matrix From PRELIM . Used by the genetic search's fitness function. featlabels — Feature names cell array of character vectors opts — Selection options structure Normally obj.opts.sifted . opts.rho — Minimum correlation 0.10 (default) | scalar in [0, 1] opts.pval — Significance level 0.05 (default) | scalar in (0, 1) opts.K — Number of clusters 10 (default) | positive integer Number of features SIFTED selects. When no more than K features pass the correlation filter, all of them are kept. opts.MaxIter — k-means iterations 1000 (default) | positive integer opts.Replicates — k-means replicates 100 (default) | positive integer opts.seed — Random seed opts.general.seed (default) | integer Seeds k-means, the cross-validation partition, and the genetic algorithm. opts.dims — Projection dimension for the fitness function 2 (default) | 3 InstanceSpace sets this from opts.pilot.dims , so features are chosen for the same dimension as the final projection. Output Arguments Expand all X — Selected features numeric matrix The columns out.selvars of the input X . out — Selection details structure out.selvars — Selected feature indices Indices into the columns of the input X . out.rho , out.p — Correlations Correlation of each feature with each algorithm's performance, and the p-values. out.eva — Cluster evaluation evalclusters result with the silhouette value for each number of clusters. out.clust — Cluster membership nfeats -by- K logical matrix. out.Ksuggested — Suggested number of clusters Present only when the silhouette value for opts.K is below 0.5. Tips opts.sifted.flag = false skips SIFTED in InstanceSpace and keeps every feature. SIFTED is the slowest stage. Set opts.general.parallel = true to evaluate the genetic algorithm's population in parallel. GNU Octave Octave runs serial correlation clustering and CV/KNN fitness with bounded integer search. Small search spaces are enumerated; larger ones use a serial GA. The backend and fitness are recorded in search, with cvpartition retaining the folds. Octave uses batch k-means updates; MATLAB retains its online phase and native GA. Version History v0.9.2 — Diagnostics, cache handling and Octave support Added serial Octave clustering and integer search, with the search backend and CV partition recorded in the result. Set opts.sifted.diagnostics=false to skip the advisory silhouette sweep. Feature clustering and selection still run with the configured K. A call with opts.parallel=false resets only its client fitness cache and does not dispatch work to an existing pool. A later parallel call resets its worker caches before using them. v0.9.1 — Seed control k-mean"},{"t":"TRACE","u":"TRACE.html","p":"Compute algorithm footprints in the instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Footprint evaluation and Octave support","v0.9.0 — TRACE3","References","See Also"],"x":"Syntax out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts) out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts,trainedTrace) Description out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts) finds each algorithm's footprint : the regions of the instance space where the algorithm is good ( out.good ) and where it is the best algorithm ( out.best ), plus the region of beta-hard instances ( out.hard ). For each footprint TRACE reports its area (2D) or volume (3D), density and purity, relative to the whole space. out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts,trainedTrace) keeps the footprints of an earlier training call, trainedTrace , and recomputes their density and purity with new instances. Footprints are not rebuilt. The default method, TRACE3, works in 2D and 3D. opts.method = 'legacy' selects the earlier DBSCAN-based method, which is 2D only. Examples Expand all Footprints of a trained instance space Copy Rebuild footprints with a stricter purity Copy Input Arguments Expand all Z — Instance coordinates ninst -by-2 or ninst -by-3 numeric matrix Ybin — Good-performance labels logical matrix Yhat — Predicted labels logical matrix | [] model.pythia.Yhat . TRACE3 uses only instances that are both good and predicted good, which removes instances the classifier cannot separate. With [] it uses Ybin alone and warns ( ISA:TRACE3:noPYTHIA ). P — Best algorithm per instance integer vector beta — Beta-easy instances logical vector ~beta defines the hard-instance footprint. algolabels — Algorithm names cell array of character vectors opts — Footprint options structure Normally obj.opts.trace . opts.method — Footprint method 'trace3' (default) | 'legacy' 'legacy' falls back to 'trace3' with a warning for a 3D space. opts.PI — Purity threshold 0.6 (default) | scalar in [0, 1] Target fraction of good instances inside a footprint. Check accepted before treating a training footprint as meeting this target. opts.minInstances — Minimum instances 4 (default) | positive integer Fewer instances than this give an empty footprint. opts.minAreaFrac — Minimum size 0.01 (default) | scalar in [0, 1] A footprint smaller than this fraction of the whole space is discarded. opts.contra — Remove contradictions false (default; true for 'legacy' ) | logical Legacy method only: remove the overlap between best-algorithm footprints. opts.boundaryTolerance — Explicit boundary distance 0 (default) | finite nonnegative scalar Euclidean distance in projection-coordinate units. A positive value includes points within this distance of a fitted edge (2D) or triangular face (3D), including boundaries of holes. Hole interiors farther away remain excluded. Zero preserves exact closed-boundary membership. There is no automatic epsilon multiplier, coordinate scaling or dependence on the query batch. The same rule is used for construction metrics, legacy overlap counts and trimming, and test-set evaluation. Geometry and area/volume are not expanded. The fitted value is saved as out.boundaryTolerance ; evaluation reuses it even if a different option is supplied. Older models without this field use zero. Choose a positive distance from an explicit uncertainty budget in the projection units; this option does not guarantee stable legacy trimming under arbitrary coordinate perturbations. trainedTrace — Trained footprints structure model.trace . Its presence selects evaluation mode. Output Arguments Expand all out — Footprints and summary structure out.space — Whole space The convex hull of all instances: measure (area or volume), measureLabel , elements , density , purity . out.good — Good-performance footprints Cell array, one footprint per algorithm. Each footprint has polygon (an alphaShape ), measure , measureLabel , elements , goodElements , density and purity . out.best — Best-algorithm footprints Same format as out.good . out.hard — Beta-hard footprint out.summary — Summary table Cell array: for each algorithm, the area or volume of its good and best footprints (absolute and as a fracti"},{"t":"What's New","u":"WhatsNew.html","p":"Changes in recent versions of the toolbox","h":["v0.9.2","New Functionality","Correctness and Reporting","Upgrading and Limitations","Documentation","v0.9.1","v0.9.0","See Also"],"x":"The complete list of changes, including every bug fix, is in RELEASE_NOTES.md in the repository root. v0.9.2 New Functionality Independent INIT stage for metadata ingestion, callbacks and saved input snapshots. PRELIM reruns reuse the snapshot; rerunning INIT reloads metadata and invalidates downstream results. Explicit opts.trace.boundaryTolerance , a nonnegative distance in projected coordinates, defaults to zero. Footprints include their exact boundaries; the saved tolerance is reused during exploration. Serial GNU Octave 2D/3D build, save/load and exploration with KNN and none/Sobol tuning, TRACE3, CSV and PNG. Validation uses Octave 11.3, Statistics 2.0.0 and Datatypes 1.5.0. Qt and a display are required for 3D PNG. Bayesian tuning, parallel execution, other classifiers, legacy TRACE, FIG and web palette output remain MATLAB-only. Octave classifier archives are runtime-specific. 3D CLOISTER boundaries and CSV triangle-face exports, including a geometry manifest and explicit empty footprints. Correctness and Reporting Recompute preprocessing after pruning algorithms that are never good, reset SIFTED caches between runs, retain inputs for feature-selection reruns, and apply fitted PLS centering during exploration. Recommendations depend on training policy and new features. Missing test outcomes are excluded from metrics. Oracle probability uses only rows with observed outcomes; selector recall avoids double counting. Classifier summaries distinguish CV from test metrics and probabilities from raw scores. Persist stage options and provenance for safe partial build/save/load/resume; reject incompatible partial rebuilds and replay training feature exclusions during exploration. Evaluate footprints against trained geometry for small or degenerate test batches. Preserve holes in 2D exports and record footprint acceptance and search termination. Replace obsolete toolkit-owned output files on export. Reproducible seeding immediately before SVM posterior calibration, complete numerical projection defaults, constant-column scaling and validation of seed bounds and input data. Upgrading and Limitations Rebuild experiments from the original metadata to obtain corrected results. Loading an existing model does not apply fixes to its fitted artifacts. Back up models and outputs first; rebuild CLOISTER for older 3D models and preprocessing before rerunning SIFTED if the saved feature-selection input is absent. Raw performance must be nonnegative (zero is allowed), and incomplete training data remaining after ingestion filtering is rejected. See Migrating a Legacy Model for pre-v0.9.0 layouts. Classifier CV summaries are conditional on the fitted preprocessing, feature selection, projection and tuning. They are not an outer-CV estimate of the full pipeline. Numerical PILOT restart selection can amplify floating-point differences across BLAS implementations and thread settings, so a seed alone does not promise identical projections across environments. Projection orientation is not canonicalized automatically. opts.pilot.topoWeight remains reserved and disabled. Documentation Reference documentation is available in the MATLAB Help browser and on the web, with examples for every function. The version header, release notes and citation metadata now identify v0.9.2. v0.9.1 Engineering release; results of the pipeline are unchanged. build and explore accept an 'onStage' callback, called after each stage. obj.plot('boundary') and distribution_boundary.png show the CLOISTER boundary of a 2D space. INIT reads the metadata for both training and evaluation, and PRELIM gained an evaluation mode, so evaluation now breaks ties exactly as training does. opts.general.seed now also controls PILOT and SIFTED . InstanceSpace checks that each stage's inputs exist before running it. Continuous integration and a matlab.unittest test suite. v0.9.0 Complete refactor of the toolbox. The InstanceSpace class is the main interface; buildIS and exploreIS remain as w"},{"t":"buildIS","u":"buildIS.html","p":"Build an instance space from a folder (backward-compatible wrapper)","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Wrapper around InstanceSpace","See Also"],"x":"Syntax model = buildIS(rootdir) Description model = buildIS(rootdir) builds an instance space from rootdir/metadata.csv and rootdir/options.json , writes model.mat and the output files to rootdir , prints EOF:SUCCESS , and returns the trained model. It is equivalent to Copy buildIS keeps the calling convention of versions before v0.9.0, which the MATILDA web platform and other scripts use. For new code, use InstanceSpace . Examples Expand all Build from a folder Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains metadata.csv and, optionally, options.json . Output Arguments Expand all model — Trained model structure The model property of the built InstanceSpace object. Version History v0.9.0 — Wrapper around InstanceSpace buildIS became a wrapper around InstanceSpace . Its inputs, outputs and files are unchanged. See Also exploreIS | InstanceSpace"},{"t":"exploreIS","u":"exploreIS.html","p":"Evaluate a saved instance space on new instances (backward-compatible wrapper)","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Wrapper around InstanceSpace","See Also"],"x":"Syntax out = exploreIS(rootdir) Description out = exploreIS(rootdir) loads rootdir/model.mat , evaluates rootdir/metadata_test.csv with it, writes the output files to rootdir , prints EOF:SUCCESS , and returns the result. It is equivalent to Copy A model.mat from a version before v0.9.0 is migrated in memory by ISAmigrateModel . For new code, use InstanceSpace . Examples Expand all Evaluate test instances after buildIS Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains model.mat and metadata_test.csv . Output Arguments Expand all out — Evaluation result structure Same fields as a trained model, computed for the test instances. Version History v0.9.0 — Wrapper around InstanceSpace exploreIS became a wrapper around InstanceSpace . Its inputs, outputs and files are unchanged. See Also buildIS | InstanceSpace | ISAmigrateModel"},{"t":"Instance Space Analysis Toolbox","u":"index.html","p":"Objective algorithm testing through the instance space of a problem","h":["Get Started","Getting Started","Stage-by-Stage Walkthrough","Metadata File Format","Options Reference","Functions and Classes","InstanceSpace","Pipeline Stages","Utilities","Output","Topics","Installation","References"],"x":"The Instance Space Analysis (ISA) Toolbox relates the structural features of problem instances to the performance of a portfolio of algorithms. It projects every instance into a 2D or 3D instance space , shows where each algorithm performs well (its footprint ), predicts which algorithm to use for a new instance, and estimates where in the space no instances have been observed yet. The toolbox implements the methodology in Smith-Miles & Muñoz (2023), including the 3D extension (ISA3) of Simpson et al. (2025). It runs in MATLAB R2025a or later and powers the MATILDA web platform. The ISA pipeline: INIT and PRELIM prepare the metadata, SIFTED selects features, PILOT projects them, then CLOISTER, PYTHIA and TRACE work in the projected space. INIT read metadata PRELIM label & normalise SIFTED select features PILOT project to 2D/3D CLOISTER empirical bounds PYTHIA predict algorithms TRACE footprints Outputs CSV · PNG · .mat The ISA pipeline. InstanceSpace runs the stages in this order; each stage is also a standalone function. Get Started Getting Started Build and explore an instance space from the bundled reference data in a few lines. Stage-by-Stage Walkthrough Run each stage separately and inspect what it produces. Metadata File Format Prepare metadata.csv for your own problem domain. Options Reference Every field of opts and options.json , with defaults. Functions and Classes InstanceSpace Build, explore, save, load and plot an instance space. The main entry point. Pipeline Stages INIT, PRELIM, FILTER, SIFTED, PILOT, PILOTviewpoint, CLOISTER, PYTHIA, TRACE. Utilities Option defaults and validation, classifier registry, legacy-model migration. Output Write CSV files and figures, and restore 3D viewpoints. Topics Migrating a Legacy Model — use a model.mat from a version before v0.9.0. Backward-Compatible Wrappers — buildIS and exploreIS , used by MATILDA. Deprecated Functions — PYTHIA2 , PYTHIAtest , SIFTED2 . What's New — release notes. Installation Clone or download the repository, then add it to the MATLAB path from its root folder: Copy Required products: MATLAB R2025a or later with the Statistics and Machine Learning, Optimization, Global Optimization, Parallel Computing, and Financial toolboxes. A bounded serial GNU Octave workflow is also supported; see What's New for supported classifiers, outputs and persistence limits. With the repository root on the path, this documentation also appears in the MATLAB Help browser under Supplemental Software . References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 Simpson, C., Muñoz, M.A., Kandanaarachchi, S. & Campello, R.J.G.B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning , 114, 240. https://doi.org/10.1007/s10994-025-06871-5"},{"t":"scriptcsv","u":"scriptcsv.html","p":"Write the results of an instance space to CSV files","h":["Syntax","Description","Examples","Input Arguments","Version History","v0.9.2 — Footprint boundaries and geometry exports","v0.9.1 — Multiple footprint regions","See Also"],"x":"Syntax scriptcsv(container,rootdir) Description scriptcsv(container,rootdir) writes the coordinates, data, predictions, footprints and summary tables of a trained model or an evaluation result to CSV files in rootdir . Coordinate columns are z_1 , z_2 and, for a 3D space, z_3 . InstanceSpace.build and InstanceSpace.explore call scriptcsv when opts.outputs.csv is true . File Contents coordinates.csv instance coordinates Z bounds.csv , bounds_prunned.csv CLOISTER boundary Zedge and Zecorr (training only) feature_raw.csv , feature_process.csv selected features, before and after preprocessing algorithm_raw.csv , algorithm_process.csv performance, before and after preprocessing algorithm_bin.csv good-performance labels Ybin good_algos.csv , beta_easy.csv , portfolio.csv number of good algorithms, beta-easy flag, best algorithm algorithm_svm.csv , portfolio_svm.csv PYTHIA predictions and selected algorithm footprint__good.csv , footprint__best.csv footprint boundary vertices; separate regions and holes are separated by a row of NaN footprint_performance.csv TRACE summary table classifier_table.csv PYTHIA summary table projection_matrix.csv PILOT projection matrix (training only) Examples Expand all Write the CSV files of an evaluation to another folder Copy Input Arguments Expand all container — Model or evaluation result structure obj.model or an element of obj.testResults . rootdir — Output folder character vector Must exist and end with a file separator. Version History v0.9.2 — Footprint boundaries and geometry exports Geometry exports include empty vertex tables when a footprint is empty. For 3D, each vertex table has a corresponding _faces.csv with one-based triangle indices. CLOISTER uses bounds_faces.csv and bounds_prunned_faces.csv . geometry_manifest.json records dimensionality, footprint filenames, and empty status. Two-dimensional rings are NaN-separated, including holes. The exporter replaces the toolkit-owned footprint_*_best.csv , footprint_*_good.csv , their face tables, and boundary CSVs on each run. Keep user-authored files outside these reserved names. A footprint region with a hole writes both its outer boundary and the hole's boundary, separated by a NaN row. v0.9.1 — Multiple footprint regions Every region of a footprint is written, separated by NaN rows, instead of only the first. See Also scriptpng | scriptweb | InstanceSpace"},{"t":"scriptpng","u":"scriptpng.html","p":"Save figures of an instance space as PNG files","h":["Syntax","Description","Examples","Input Arguments","GNU Octave","Version History","v0.9.2 — Plot export updates and Octave support","v0.9.1 — Boundary figure","v0.9.0 — 3D figures","See Also"],"x":"Syntax scriptpng(container,rootdir) Description scriptpng(container,rootdir) draws the standard views of a trained model or an evaluation result and saves them in rootdir : each feature, each algorithm's performance, good/bad labels, predictions, footprints, the algorithm portfolio, the number of good algorithms, the beta score, instance sources, and the CLOISTER boundary. For a 3D space the figures use the viewpoints found by PILOTviewpoint . Each footprint is also saved as a MATLAB figure ( footprint_.fig , footprint_portfolio.fig ) for interactive rotation, unless opts.outputs.fig is false . ISArecallView returns a rotated figure to its stored viewpoint. InstanceSpace.build and InstanceSpace.explore call scriptpng when opts.outputs.png is true . File Figure distribution_feature_.png feature value over the space distribution_performance_global_normalized_.png , ..._individual_normalized_.png performance, scaled over all algorithms or per algorithm binary_performance_.png , binary_classifier_.png good/bad labels, and PYTHIA's predictions footprint_.png , footprint_portfolio.png footprints distribution_portfolio.png , distribution_svm_portfolio.png best and selected algorithm distribution_number_good_algos.png , distribution_beta_score.png number of good algorithms, beta-easy instances distribution_sources.png instance sources, if the metadata has a source column distribution_boundary.png CLOISTER boundary (training only) Examples Expand all Save figures at a larger font size Copy Input Arguments Expand all container — Model or evaluation result structure obj.model or an element of obj.testResults . rootdir — Output folder character vector Must exist and end with a file separator. GNU Octave Octave exports PNG through print and uses viridis. Qt is selected when available and is required for 3D export; use a virtual display for headless execution. MATLAB retains exportgraphics and parula. FIG output is MATLAB-only. Version History v0.9.2 — Plot export updates and Octave support Added Octave PNG export through Qt, with viridis colours and MATLAB-only FIG output. Direct Octave calls requesting 3D FIG output fail before removing existing output files. Each plot export removes prior toolkit-owned plot filenames before drawing the current run. This clears plots for removed algorithms and features, absent sources or bounds, and old 3D FIG files. Keep user-authored files outside the toolkit's distribution_* , binary_* , and footprint_* output names. distribution_boundary.png is also drawn for a 3D space. v0.9.1 — Boundary figure Adds distribution_boundary.png . v0.9.0 — 3D figures 3D spaces are drawn from their optimised viewpoints and saved as .fig files too. See Also scriptcsv | ISArecallView | PILOTviewpoint | InstanceSpace"},{"t":"scriptweb","u":"scriptweb.html","p":"Write colour-scaled data for the MATILDA web platform","h":["Syntax","Description","Examples","Input Arguments","See Also"],"x":"Syntax scriptweb(container,rootdir) Description scriptweb(container,rootdir) writes the features, performance and number of good algorithms as colour indices, with the colour table, to CSV files in rootdir . The MATILDA web platform uses these files to draw its interactive plots. You do not need them for local work. InstanceSpace.build and InstanceSpace.explore call scriptweb when both opts.outputs.csv and opts.outputs.web are true . File Contents color_table.csv the colour map feature_raw_color.csv , feature_process_color.csv feature colour indices algorithm_raw_color.csv , algorithm_process_color.csv performance colour indices, scaled over all algorithms algorithm_raw_single_color.csv , algorithm_process_single_color.csv performance colour indices, scaled per algorithm good_algos_color.csv colour index of the number of good algorithms Examples Expand all Turn on the web output Copy Input Arguments Expand all container — Model or evaluation result structure rootdir — Output folder character vector Must exist and end with a file separator. See Also scriptcsv | InstanceSpace"}]; diff --git a/doc/src/PILOT.md b/doc/src/PILOT.md index ad048b0..286a0d6 100644 --- a/doc/src/PILOT.md +++ b/doc/src/PILOT.md @@ -189,6 +189,8 @@ min ||X - Z*B'||^2 + alpha*||Y - Z*C||^2 subject to Z = X*A' where `X` and `Y` are the preprocessed feature and performance matrices. The analytic solution takes the leading `dims` eigenvectors of `[X Y]'*[X Y]`, with the `Y` block weighted by `sqrt(alpha)` (Muñoz et al., 2018). The numerical solution runs BFGS (`fminunc`) from `opts.ntries` random starts and keeps the trial with the highest correlation between the pairwise distances of the instances before and after projection, so the chosen space also preserves the topology of the data. +The projection has no automatic canonical orientation: equivalent spaces can appear rotated or reflected. A fixed seed also does not guarantee the same numerical restart winner across BLAS implementations or thread settings. Keep the numerical environment fixed when reproducibility is required. + ## GNU Octave Octave supports serial analytic/numerical standard projection and centered SIMPLS in 2D/3D, including optimized 3D viewpoints. Native MATLAB backends are retained. Optimizer trajectories may differ; parallel execution is rejected under Octave. diff --git a/doc/src/WhatsNew.md b/doc/src/WhatsNew.md index a54dcfb..58a2d06 100644 --- a/doc/src/WhatsNew.md +++ b/doc/src/WhatsNew.md @@ -25,7 +25,7 @@ The complete list of changes, including every bug fix, is in `RELEASE_NOTES.md` Rebuild experiments from the original metadata to obtain corrected results. Loading an existing model does not apply fixes to its fitted artifacts. Back up models and outputs first; rebuild CLOISTER for older 3D models and preprocessing before rerunning SIFTED if the saved feature-selection input is absent. Raw performance must be nonnegative (zero is allowed), and incomplete training data remaining after ingestion filtering is rejected. See [Migrating a Legacy Model](MigratingLegacyModel.html) for pre-v0.9.0 layouts. -Classifier CV summaries are conditional on the fitted preprocessing, feature selection, projection and tuning. They are not an outer-CV estimate of the full pipeline. Numerical PILOT restart selection can amplify floating-point differences across BLAS implementations and thread settings, so a seed alone does not promise identical projections across environments. `opts.pilot.topoWeight` remains reserved and disabled. +Classifier CV summaries are conditional on the fitted preprocessing, feature selection, projection and tuning. They are not an outer-CV estimate of the full pipeline. Numerical PILOT restart selection can amplify floating-point differences across BLAS implementations and thread settings, so a seed alone does not promise identical projections across environments. Projection orientation is not canonicalized automatically. `opts.pilot.topoWeight` remains reserved and disabled. ### Documentation diff --git a/docs/release-0.9.2.md b/docs/release-0.9.2.md index 1385e53..ea2b79b 100644 --- a/docs/release-0.9.2.md +++ b/docs/release-0.9.2.md @@ -18,6 +18,10 @@ uses R2026a Update 5. The full Octave workflow is validated with 11.3.0, Statistics 2.0.0 and Datatypes 1.5.0. See `octave-compatibility.md` for the exact support boundary; portable archives do not imply cross-runtime model loading. +Closed issue #30 proposed optional canonical projection rotation, but that feature +is not implemented in current PILOT and is excluded from this release. Do not +use the closed issue state as evidence of implementation. + Issue #63 remains open: numerical PILOT restart ranking can amplify small BLAS/thread-level differences. The release does not promise cross-environment bitwise reproducibility. `opts.pilot.topoWeight` remains disabled. Classifier CV @@ -53,8 +57,28 @@ display. On this machine the usable local display is `:101`; the inherited remote display name rejected authorization. Do not interpret that initial configuration failure as a supported-runtime test pass. -Local verification results will be recorded here before the release PR is -ready. R2025a and pinned Octave CI checks must also be green on the reviewed PR. +Local checks completed: + +- Generated HTML matches all sources (36 generated files); link validation + checks 31 pages with zero problems. +- Octave 11.3.0 with the pinned packages passes the portable preprocessing, + analytic/numerical/PLS projection, KNN/CV/tuning, geometry, workflow edge, + archive and property contracts. Both 2D and 3D full build/save/load/explore + checks pass with CSV and PNG on the local Qt display. The initial remote + display configuration failed its 3D graphics prerequisite; the Qt rerun + passes without errors. +- Octave 6.4.0 passes the separate core-only analytic and preprocessing contracts; + this is not full-pipeline support on that runtime. +- The source candidate archive passes ZIP integrity, version and required-file + checks and excludes caches, environments and generated test/package outputs. + Archive checksums and command logs are retained in ignored + `.cache/release-0.9.2/` (runtime logs originate under `/tmp/`). Rebuild the + candidate after changes and the final archive from the reviewed merge commit. + +The full MATLAB result and CI status are recorded in +[release-preparation PR #67](https://github.com/andremun/InstanceSpace/pull/67). +Require a passing full MATLAB suite and green R2025a/pinned Octave CI checks on +the reviewed PR before publication. ## Publication checklist From 7208525f077ee9f989a95c3bb9d9eb45564eaadb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mario=20Andr=C3=A9s=20Mu=C3=B1oz?= Date: Sun, 18 Oct 2026 00:03:20 +1100 Subject: [PATCH 3/3] Document footprint membership and add it to Help navigation --- doc/generate.py | 1 + doc/html/404.html | 1 + doc/html/CLOISTER.html | 1 + doc/html/Deprecated.html | 1 + doc/html/FILTER.html | 1 + doc/html/FunctionList.html | 5 ++ doc/html/GettingStarted.html | 1 + doc/html/INIT.html | 1 + doc/html/ISAdefaults.html | 1 + doc/html/ISAfootprintContains.html | 113 +++++++++++++++++++++++++++ doc/html/ISAgetClassifierFcn.html | 1 + doc/html/ISAmigrateModel.html | 1 + doc/html/ISArecallView.html | 1 + doc/html/ISAsubsetData.html | 1 + doc/html/ISAvalidateOpts.html | 1 + doc/html/InstanceSpace.html | 1 + doc/html/InteractiveWalkthrough.html | 1 + doc/html/MetadataFormat.html | 1 + doc/html/MigratingLegacyModel.html | 1 + doc/html/OptionsReference.html | 1 + doc/html/PILOT.html | 1 + doc/html/PILOTviewpoint.html | 1 + doc/html/PRELIM.html | 1 + doc/html/PYTHIA.html | 1 + doc/html/SIFTED.html | 1 + doc/html/TRACE.html | 3 +- doc/html/WhatsNew.html | 1 + doc/html/assets/search-index.js | 2 +- doc/html/buildIS.html | 1 + doc/html/exploreIS.html | 1 + doc/html/helptoc.xml | 1 + doc/html/index.html | 1 + doc/html/scriptcsv.html | 1 + doc/html/scriptpng.html | 1 + doc/html/scriptweb.html | 1 + doc/src/ISAfootprintContains.md | 79 +++++++++++++++++++ doc/src/TRACE.md | 2 +- docs/release-0.9.2.md | 4 +- 38 files changed, 234 insertions(+), 5 deletions(-) create mode 100644 doc/html/ISAfootprintContains.html create mode 100644 doc/src/ISAfootprintContains.md diff --git a/doc/generate.py b/doc/generate.py index 6097ff7..6121406 100644 --- a/doc/generate.py +++ b/doc/generate.py @@ -83,6 +83,7 @@ ("Utilities", "FunctionList", [ ("ISAdefaults", "ISAdefaults"), ("ISAvalidateOpts", "ISAvalidateOpts"), + ("ISAfootprintContains", "ISAfootprintContains"), ("ISAgetClassifierFcn", "ISAgetClassifierFcn"), ("ISAmigrateModel", "ISAmigrateModel"), ("ISAsubsetData", "ISAsubsetData"), diff --git a/doc/html/404.html b/doc/html/404.html index 5babfd0..6c73f81 100644 --- a/doc/html/404.html +++ b/doc/html/404.html @@ -49,6 +49,7 @@
  • Utilities
    • ISAdefaults
    • ISAvalidateOpts
      • +
    • ISAfootprintContains
    • ISAgetClassifierFcn
    • ISAmigrateModel
    • ISAsubsetData
      • diff --git a/doc/html/CLOISTER.html b/doc/html/CLOISTER.html index f54c00e..24c1195 100644 --- a/doc/html/CLOISTER.html +++ b/doc/html/CLOISTER.html @@ -49,6 +49,7 @@
    • Utilities
      • ISAdefaults
      • ISAvalidateOpts
        • +
      • ISAfootprintContains
      • ISAgetClassifierFcn
      • ISAmigrateModel
      • ISAsubsetData
        • diff --git a/doc/html/Deprecated.html b/doc/html/Deprecated.html index 6c09aa9..bf57729 100644 --- a/doc/html/Deprecated.html +++ b/doc/html/Deprecated.html @@ -49,6 +49,7 @@
      • Utilities
        • ISAdefaults
        • ISAvalidateOpts
          • +
        • ISAfootprintContains
        • ISAgetClassifierFcn
        • ISAmigrateModel
        • ISAsubsetData
          • diff --git a/doc/html/FILTER.html b/doc/html/FILTER.html index 8c92f6a..9ae039b 100644 --- a/doc/html/FILTER.html +++ b/doc/html/FILTER.html @@ -49,6 +49,7 @@
        • Utilities
          • ISAdefaults
          • ISAvalidateOpts
            • +
          • ISAfootprintContains
          • ISAgetClassifierFcn
          • ISAmigrateModel
          • ISAsubsetData
            • diff --git a/doc/html/FunctionList.html b/doc/html/FunctionList.html index 8e9664e..f99eabc 100644 --- a/doc/html/FunctionList.html +++ b/doc/html/FunctionList.html @@ -49,6 +49,7 @@
          • Utilities
            • ISAdefaults
            • ISAvalidateOpts
              • +
            • ISAfootprintContains
            • ISAgetClassifierFcn
            • ISAmigrateModel
            • ISAsubsetData
              • @@ -170,6 +171,10 @@

              All Functions

              Check the type and range of user-supplied options +ISAfootprintContains +Test footprint membership with an explicit boundary distance + + ISAgetClassifierFcn Look up a classifier in the PYTHIA registry diff --git a/doc/html/GettingStarted.html b/doc/html/GettingStarted.html index 0bc6675..b33adf7 100644 --- a/doc/html/GettingStarted.html +++ b/doc/html/GettingStarted.html @@ -49,6 +49,7 @@
            • Utilities
              • ISAdefaults
              • ISAvalidateOpts
                • +
              • ISAfootprintContains
              • ISAgetClassifierFcn
              • ISAmigrateModel
              • ISAsubsetData
                • diff --git a/doc/html/INIT.html b/doc/html/INIT.html index bf07bd4..22f8bd0 100644 --- a/doc/html/INIT.html +++ b/doc/html/INIT.html @@ -49,6 +49,7 @@
              • Utilities
                • ISAdefaults
                • ISAvalidateOpts
                  • +
                • ISAfootprintContains
                • ISAgetClassifierFcn
                • ISAmigrateModel
                • ISAsubsetData
                  • diff --git a/doc/html/ISAdefaults.html b/doc/html/ISAdefaults.html index c8cafb8..38a425a 100644 --- a/doc/html/ISAdefaults.html +++ b/doc/html/ISAdefaults.html @@ -49,6 +49,7 @@
                • Utilities
                  • ISAdefaults
                  • ISAvalidateOpts
                    • +
                  • ISAfootprintContains
                  • ISAgetClassifierFcn
                  • ISAmigrateModel
                  • ISAsubsetData
                    • diff --git a/doc/html/ISAfootprintContains.html b/doc/html/ISAfootprintContains.html new file mode 100644 index 0000000..2fdbd00 --- /dev/null +++ b/doc/html/ISAfootprintContains.html @@ -0,0 +1,113 @@ + + + + + +ISAfootprintContains - Instance Space Analysis Toolbox + + + + + + +
                    + + Instance Space Analysis Toolbox + v0.9.2 + +
                    +
                    + +
                    + +
                    +

                    ISAfootprintContains

                    +

                    Test footprint membership with an explicit boundary distance

                    +
                    +

                    Syntax

                    +
                    inside = ISAfootprintContains(geometry,queries)
                    +
                    inside = ISAfootprintContains(geometry,queries,tolerance)
                    +
                    +

                    Description

                    inside = ISAfootprintContains(geometry,queries) tests whether each query point lies in a footprint, including its boundary. The default distance tolerance is zero.

                    +

                    inside = ISAfootprintContains(geometry,queries,tolerance) also includes points within the specified Euclidean distance of a boundary edge in 2D or triangular face in 3D. Hole interiors farther from the boundary remain excluded. The geometry and its area or volume are unchanged.

                    +

                    TRACE uses this helper for footprint metrics and evaluation. Set opts.trace.boundaryTolerance to configure the pipeline; TRACE saves the fitted value and reuses it during exploration. The distance is in projected-coordinate units, with no implicit epsilon multiplier, automatic scaling or dependence on other points in the query batch. The membership policy is shared with pyInstanceSpace; floating-point geometry can still differ between implementations.

                    +

                    Examples

                    Include the exact boundary of a square

                    Run startup first when calling utilities directly from a fresh session.

                    +
                    startup
+region = polyshape([0 0; 1 0; 1 1; 0 1]);
+queries = [0.5 0.5; 0.5 0; 0.5 -0.0005; 0.5 -0.002];
+inside = ISAfootprintContains(region, queries)
+% inside = [true; true; false; false]
                    +
                    Choose an explicit distance tolerance
                    inside = ISAfootprintContains(region, queries, 0.001)
+% inside = [true; true; true; false]
                    +

                    Choose a positive distance from the uncertainty budget of the projected coordinates. A tolerance is not a guarantee that legacy footprint trimming is invariant under arbitrary coordinate perturbations.

                    +

                    Input Arguments

                    geometry — Footprint geometry

                    2D polyshape | 2D or 3D alphaShape

                    +

                    The fitted region, normally a footprint's polygon field. Octave TRACE3 uses the toolkit's compatible alpha-shape class; MATLAB's polyshape and legacy TRACE remain MATLAB-only. An empty geometry produces false for every query.

                    +
                    queries — Points to test

                    numeric matrix

                    +

                    One row per point, with two or three columns matching the geometry's dimensionality. Nonfinite query points are outside.

                    +
                    tolerance — Boundary distance

                    0 (default) | finite nonnegative real scalar

                    +

                    Euclidean distance in projection units. Positive values include points at distance less than or equal to this value from a boundary, including hole boundaries.

                    +

                    Output Arguments

                    inside — Membership mask

                    logical column vector

                    +

                    One value per query row. Testing a point alone or alongside other queries uses the same rule.

                    +

                    Version History

                    v0.9.2 — Introduced

                    Closed-boundary membership with explicit distance tolerance for TRACE construction, legacy contradiction handling and evaluation.

                    +

                    See Also

                    TRACE | ISAdefaults | Options Reference

                    +
                    +

                    Report a documentation issue · Source on GitHub

                    +

                    Copyright © 2026 Mario Andrés Muñoz Acosta and contributors, School of Computing and Information Systems, The University of Melbourne. Released under the PolyForm Noncommercial License 1.0.0.

                    +
                    +
                    + +
                    + + diff --git a/doc/html/ISAgetClassifierFcn.html b/doc/html/ISAgetClassifierFcn.html index 1eb4a7d..34c10fc 100644 --- a/doc/html/ISAgetClassifierFcn.html +++ b/doc/html/ISAgetClassifierFcn.html @@ -49,6 +49,7 @@
                  • Utilities
                    • ISAdefaults
                    • ISAvalidateOpts
                      • +
                    • ISAfootprintContains
                    • ISAgetClassifierFcn
                    • ISAmigrateModel
                    • ISAsubsetData
                      • diff --git a/doc/html/ISAmigrateModel.html b/doc/html/ISAmigrateModel.html index 159edcf..86b1757 100644 --- a/doc/html/ISAmigrateModel.html +++ b/doc/html/ISAmigrateModel.html @@ -49,6 +49,7 @@
                    • Utilities
                      • ISAdefaults
                      • ISAvalidateOpts
                        • +
                      • ISAfootprintContains
                      • ISAgetClassifierFcn
                      • ISAmigrateModel
                      • ISAsubsetData
                        • diff --git a/doc/html/ISArecallView.html b/doc/html/ISArecallView.html index 5ff5646..0a265c6 100644 --- a/doc/html/ISArecallView.html +++ b/doc/html/ISArecallView.html @@ -49,6 +49,7 @@
                      • Utilities
                        • ISAdefaults
                        • ISAvalidateOpts
                          • +
                        • ISAfootprintContains
                        • ISAgetClassifierFcn
                        • ISAmigrateModel
                        • ISAsubsetData
                          • diff --git a/doc/html/ISAsubsetData.html b/doc/html/ISAsubsetData.html index 4919627..0fef89a 100644 --- a/doc/html/ISAsubsetData.html +++ b/doc/html/ISAsubsetData.html @@ -49,6 +49,7 @@
                        • Utilities
                          • ISAdefaults
                          • ISAvalidateOpts
                            • +
                          • ISAfootprintContains
                          • ISAgetClassifierFcn
                          • ISAmigrateModel
                          • ISAsubsetData
                            • diff --git a/doc/html/ISAvalidateOpts.html b/doc/html/ISAvalidateOpts.html index c67d66b..d82d55f 100644 --- a/doc/html/ISAvalidateOpts.html +++ b/doc/html/ISAvalidateOpts.html @@ -49,6 +49,7 @@
                          • Utilities
                            • ISAdefaults
                            • ISAvalidateOpts
                              • +
                            • ISAfootprintContains
                            • ISAgetClassifierFcn
                            • ISAmigrateModel
                            • ISAsubsetData
                              • diff --git a/doc/html/InstanceSpace.html b/doc/html/InstanceSpace.html index 4dc8bd0..c608e55 100644 --- a/doc/html/InstanceSpace.html +++ b/doc/html/InstanceSpace.html @@ -49,6 +49,7 @@
                            • Utilities
                              • ISAdefaults
                              • ISAvalidateOpts
                                • +
                              • ISAfootprintContains
                              • ISAgetClassifierFcn
                              • ISAmigrateModel
                              • ISAsubsetData
                                • diff --git a/doc/html/InteractiveWalkthrough.html b/doc/html/InteractiveWalkthrough.html index e610ea6..7db1b7f 100644 --- a/doc/html/InteractiveWalkthrough.html +++ b/doc/html/InteractiveWalkthrough.html @@ -49,6 +49,7 @@
                              • Utilities
                                • ISAdefaults
                                • ISAvalidateOpts
                                  • +
                                • ISAfootprintContains
                                • ISAgetClassifierFcn
                                • ISAmigrateModel
                                • ISAsubsetData
                                  • diff --git a/doc/html/MetadataFormat.html b/doc/html/MetadataFormat.html index 302673e..53acdf5 100644 --- a/doc/html/MetadataFormat.html +++ b/doc/html/MetadataFormat.html @@ -49,6 +49,7 @@
                                • Utilities
                                  • ISAdefaults
                                  • ISAvalidateOpts
                                    • +
                                  • ISAfootprintContains
                                  • ISAgetClassifierFcn
                                  • ISAmigrateModel
                                  • ISAsubsetData
                                    • diff --git a/doc/html/MigratingLegacyModel.html b/doc/html/MigratingLegacyModel.html index 6eda817..f1992b0 100644 --- a/doc/html/MigratingLegacyModel.html +++ b/doc/html/MigratingLegacyModel.html @@ -49,6 +49,7 @@
                                  • Utilities
                                    • ISAdefaults
                                    • ISAvalidateOpts
                                      • +
                                    • ISAfootprintContains
                                    • ISAgetClassifierFcn
                                    • ISAmigrateModel
                                    • ISAsubsetData
                                      • diff --git a/doc/html/OptionsReference.html b/doc/html/OptionsReference.html index 97b19cd..5cdd09f 100644 --- a/doc/html/OptionsReference.html +++ b/doc/html/OptionsReference.html @@ -49,6 +49,7 @@
                                    • Utilities
                                      • ISAdefaults
                                      • ISAvalidateOpts
                                        • +
                                      • ISAfootprintContains
                                      • ISAgetClassifierFcn
                                      • ISAmigrateModel
                                      • ISAsubsetData
                                        • diff --git a/doc/html/PILOT.html b/doc/html/PILOT.html index 79594c2..9fb591d 100644 --- a/doc/html/PILOT.html +++ b/doc/html/PILOT.html @@ -49,6 +49,7 @@
                                      • Utilities
                                        • ISAdefaults
                                        • ISAvalidateOpts
                                          • +
                                        • ISAfootprintContains
                                        • ISAgetClassifierFcn
                                        • ISAmigrateModel
                                        • ISAsubsetData
                                          • diff --git a/doc/html/PILOTviewpoint.html b/doc/html/PILOTviewpoint.html index 3f312e2..da7a38a 100644 --- a/doc/html/PILOTviewpoint.html +++ b/doc/html/PILOTviewpoint.html @@ -49,6 +49,7 @@
                                        • Utilities
                                          • ISAdefaults
                                          • ISAvalidateOpts
                                            • +
                                          • ISAfootprintContains
                                          • ISAgetClassifierFcn
                                          • ISAmigrateModel
                                          • ISAsubsetData
                                            • diff --git a/doc/html/PRELIM.html b/doc/html/PRELIM.html index 172de30..772faf4 100644 --- a/doc/html/PRELIM.html +++ b/doc/html/PRELIM.html @@ -49,6 +49,7 @@
                                          • Utilities
                                            • ISAdefaults
                                            • ISAvalidateOpts
                                              • +
                                            • ISAfootprintContains
                                            • ISAgetClassifierFcn
                                            • ISAmigrateModel
                                            • ISAsubsetData
                                              • diff --git a/doc/html/PYTHIA.html b/doc/html/PYTHIA.html index 347432d..f421c27 100644 --- a/doc/html/PYTHIA.html +++ b/doc/html/PYTHIA.html @@ -49,6 +49,7 @@
                                            • Utilities
                                              • ISAdefaults
                                              • ISAvalidateOpts
                                                • +
                                              • ISAfootprintContains
                                              • ISAgetClassifierFcn
                                              • ISAmigrateModel
                                              • ISAsubsetData
                                                • diff --git a/doc/html/SIFTED.html b/doc/html/SIFTED.html index 3ad5534..28a9519 100644 --- a/doc/html/SIFTED.html +++ b/doc/html/SIFTED.html @@ -49,6 +49,7 @@
                                              • Utilities
                                                • ISAdefaults
                                                • ISAvalidateOpts
                                                  • +
                                                • ISAfootprintContains
                                                • ISAgetClassifierFcn
                                                • ISAmigrateModel
                                                • ISAsubsetData
                                                  • diff --git a/doc/html/TRACE.html b/doc/html/TRACE.html index 1423407..d93dbef 100644 --- a/doc/html/TRACE.html +++ b/doc/html/TRACE.html @@ -49,6 +49,7 @@
                                                • Utilities
                                                  • ISAdefaults
                                                  • ISAvalidateOpts
                                                    • +
                                                  • ISAfootprintContains
                                                  • ISAgetClassifierFcn
                                                  • ISAmigrateModel
                                                  • ISAsubsetData
                                                    • @@ -121,7 +122,7 @@

                                                    TRACE

                                                    including boundaries of holes. Hole interiors farther away remain excluded. Zero preserves exact closed-boundary membership. There is no automatic epsilon multiplier, coordinate scaling or dependence on the query batch.

                                                    -

                                                    The same rule is used for construction metrics, legacy overlap counts and +

                                                    ISAfootprintContains applies the same rule for construction metrics, legacy overlap counts and trimming, and test-set evaluation. Geometry and area/volume are not expanded. The fitted value is saved as out.boundaryTolerance; evaluation reuses it even if a different option is supplied. Older models without this field use zero. diff --git a/doc/html/WhatsNew.html b/doc/html/WhatsNew.html index 66b18cd..6d049d1 100644 --- a/doc/html/WhatsNew.html +++ b/doc/html/WhatsNew.html @@ -49,6 +49,7 @@

                                                  • Utilities
                                                    • ISAdefaults
                                                    • ISAvalidateOpts
                                                      • +
                                                    • ISAfootprintContains
                                                    • ISAgetClassifierFcn
                                                    • ISAmigrateModel
                                                    • ISAsubsetData
                                                      • diff --git a/doc/html/assets/search-index.js b/doc/html/assets/search-index.js index 318ab72..6cdd8a7 100644 --- a/doc/html/assets/search-index.js +++ b/doc/html/assets/search-index.js @@ -1,2 +1,2 @@ // Generated by doc/generate.py. Do not edit. -window.ISA_SEARCH = [{"t":"CLOISTER","u":"CLOISTER.html","p":"Estimate the boundary of the instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","GNU Octave","Version History","v0.9.2 — 3D boundaries and Octave support","v0.9.0 — Feature limit","References","See Also"],"x":"Syntax out = CLOISTER(X,A,opts) out = CLOISTER(X,A,opts,Xmean) Description out = CLOISTER(X,A,opts) estimates the region of the instance space where instances could exist, using correlations between features (Correlated Limits of the Instance Space's Theoretical or Experimental Regions). It takes every corner of the box bounded by the minimum and maximum of each feature, discards the corners that contradict a strong, significant correlation between two features, and projects the remaining corners with the PILOT matrix A . The convex hull of the projected corners is the estimated boundary. out = CLOISTER(X,A,opts,Xmean) subtracts the fitted feature mean before projection. For PLS, pass model.pilot.Xmean to keep the boundary in the same coordinates as model.pilot.Z . The three-argument form uses a zero mean. The same centring applies to the projected-instance hull used when the feature-count limit is exceeded. Parts of the boundary far from any instance show where new test instances would extend the benchmark. For a 2D projection the boundary is a closed polygon. For a 3D projection it is a triangulated convex surface; when the projected corners all lie on one plane (for example, a 3D projection of only two features), it is the flat polygon, triangulated. Examples Expand all Plot the boundary of a 2D instance space Copy Plot the boundary of a 3D instance space Copy obj.plot('boundary') draws the same figure over the instances. Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed, selected features, one row per instance. May contain NaN . A — Projection matrix dims -by- nfeats numeric matrix model.pilot.A . opts — Boundary options structure Normally obj.opts.cloister . opts.pval — Significance level 0.05 (default) | scalar in (0, 1) Correlations with a larger p-value are ignored. opts.corrThreshold — Correlation threshold 0.70 (default) | scalar in [0, 1] A corner is discarded when it contradicts the sign of a correlation stronger than this. opts.maxFeatures — Feature limit 20 (default) | positive integer The number of corners grows as 2^nfeats . With more features than this, CLOISTER warns ( ISA:CLOISTER:tooManyFeatures ) and returns the convex hull of the projected instances instead. Xmean — Fitted feature mean 1-by- nfeats numeric row vector | default: zeros(1,nfeats) The mean fitted during PILOT training. Pass model.pilot.Xmean for PLS. Corners and fallback instances are projected as (X-Xmean)*A' ; CLOISTER does not estimate a new mean. Omit the argument for an uncentred projection. Output Arguments Expand all out — Boundary structure out.Zedge — Boundary of all corners 2D: closed polygon, first vertex repeated last. 3D: vertices of the convex hull. out.Zecorr — Boundary of the correlation-consistent corners Same format as Zedge . Equals Zedge when no corner is discarded, when too many corners are discarded to form a hull, or when opts.maxFeatures is exceeded. out.ZedgeFaces , out.ZecorrFaces — Hull triangulations 3D only: nfaces -by-3 indices into the rows of Zedge and Zecorr . Empty in 2D. Tips CLOISTER expects mean-centred features. InstanceSpace warns ( ISA:InstanceSpace:cloisterNotMeanCentred ) when opts.auto.preproc or opts.norm.flag is false . scriptcsv writes Zedge to bounds.csv and Zecorr to bounds_prunned.csv . If every projected corner lies on one line (a degenerate projection matrix or features), there is no region to bound and CLOISTER raises ISA:CLOISTER:degenerateBoundary . GNU Octave Octave supports 2D/3D hulls using namespaced Pearson-significance and NaN-aware extrema adapters. The Pearson adapter requires at least three observations. Degenerate/coplanar cases and PLS centering are covered by portable tests. Version History v0.9.2 — 3D boundaries and Octave support Added serial Octave support for Pearson significance and 2D/3D hulls. An optional fourth argument, Xmean , applies the fitted PLS feature centring during projection. The three-argument call uses a zero mean. Corner enumeration"},{"t":"Deprecated Functions","u":"Deprecated.html","p":"Functions kept only for compatibility with older code","h":["Updating Your Code","See Also"],"x":"These functions still work but print a warning and forward to their replacement. Do not use them in new code. Function Replacement Deprecated in PYTHIA2(Z,Y,Ybin,Ybest,algolabels,opts) PYTHIA with opts.classifier = 'knn' v0.9.0 PYTHIAtest(model,Z,Y,Ybin,Ybest,algolabels) PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,model) (evaluation mode) v0.9.0 SIFTED2(X,Y,Ybin,featlabels,opts) SIFTED (renamed) v0.9.0 Updating Your Code Copy See Also PYTHIA | SIFTED | What's New"},{"t":"FILTER","u":"FILTER.html","p":"Remove near-duplicate instances to even out the instance density","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.2 — Bounded neighbour queries","v0.9.0 — Uniformity output","References","See Also"],"x":"Syntax [subsetIndex,isDissimilar,isVISA,unif] = FILTER(X,Y,Ybin,opts) Description [subsetIndex,isDissimilar,isVISA,unif] = FILTER(X,Y,Ybin,opts) marks instances as redundant when they are closer than opts.mindistance in feature space to an instance already kept, and they also meet the condition in opts.type . Removing them gives a smaller, more evenly spread set of instances, which reduces the bias of densely sampled regions on the projection and the footprints (Alipour et al., 2023). InstanceSpace calls FILTER when opts.selvars.densityflag is true : once on all features during PRELIM, and again after SIFTED on the selected features only. Examples Expand all Filter the reference data Copy Filter inside the pipeline Copy Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed features, one row per instance. Y — Performance matrix numeric matrix Ybin — Good-performance labels logical matrix opts — Filter options structure Normally obj.opts.selvars . opts.mindistance — Distance threshold 0.10 (default) | positive scalar Instances closer than this in feature space are candidates for removal. opts.type — Removal condition 'Ftr&Good' (default) | 'Ftr' | 'Ftr&AP' | 'Ftr&AP&Good' Condition, in addition to feature closeness, for an instance to be removed: Value The instance is removed when 'Ftr' it is close in feature space 'Ftr&AP' the two performance vectors are also within sqrt(nalgos/nfeats)*mindistance 'Ftr&Good' every algorithm is also good on both instances 'Ftr&AP&Good' both of the above Output Arguments Expand all subsetIndex — Redundant instances logical vector true for each instance marked redundant. Keep ~subsetIndex . isDissimilar — Isolated instances logical vector false for each instance that was close to another instance. isVISA — Close but kept logical vector true for an instance that was close to a kept instance in feature space but did not meet the opts.type condition, so it was kept. unif — Uniformity of the kept set scalar One minus the coefficient of variation of the nearest-neighbour distances of the kept instances. Values near 1 mean an even spread. Version History v0.9.2 — Bounded neighbour queries The spatial index is queried only for each retained instance, which bounds stored neighbour lists without changing the greedy row order. v0.9.0 — Uniformity output unif is returned; before, it was computed and discarded. References Alipour, H., Muñoz, M.A. & Smith-Miles, K. (2023). Enhanced instance space analysis for the maximum flow problem. European Journal of Operational Research , 304(2), 411–428. https://doi.org/10.1016/j.ejor.2022.04.012 Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also ISAsubsetData | PRELIM | SIFTED | Options Reference"},{"t":"All Functions","u":"FunctionList.html","p":"Every function and class in the toolbox, by category.","h":["Pipeline Stages","Class Interface","Backward-Compatible Wrappers","Utilities","Output"],"x":"Pipeline Stages Name Purpose INIT Read and filter instance metadata PRELIM Label good performance and normalise features and performance FILTER Remove near-duplicate instances to even out the instance density SIFTED Select the features that best explain algorithm performance PILOT Project instance features onto a 2D or 3D instance space PILOTviewpoint Find the best viewing angles of a 3D instance space CLOISTER Estimate the boundary of the instance space PYTHIA Train classifiers that predict which algorithms perform well TRACE Compute algorithm footprints in the instance space Class Interface Name Purpose InstanceSpace Build, explore and save an instance space Backward-Compatible Wrappers Name Purpose buildIS Build an instance space from a folder (backward-compatible wrapper) exploreIS Evaluate a saved instance space on new instances (backward-compatible wrapper) Utilities Name Purpose ISAdefaults Fill in missing options with their default values ISAvalidateOpts Check the type and range of user-supplied options ISAgetClassifierFcn Look up a classifier in the PYTHIA registry ISAmigrateModel Update a model from an earlier toolbox version to the current layout ISAsubsetData Keep a subset of the instances in a data structure Output Name Purpose scriptcsv Write the results of an instance space to CSV files scriptpng Save figures of an instance space as PNG files scriptweb Write colour-scaled data for the MATILDA web platform ISArecallView Return a 3D figure to its optimised viewpoint"},{"t":"Getting Started with Instance Space Analysis","u":"GettingStarted.html","p":"Build and explore your first instance space","h":["Prepare a Folder","Describe the Performance Measure","Build the Instance Space","Look at the Results","Evaluate New Instances","Next Steps","References"],"x":"This example builds an instance space for the reference dataset shipped with the toolbox: 212 classification problems (the instances ), described by 10 features, on which 10 classification algorithms were scored by misclassification error (Muñoz et al., 2018). A further 23 instances are held out for testing. The same steps apply to your own data once it is in the metadata format . example.m in the repository root runs this example from start to finish. Prepare a Folder The toolbox reads its inputs from, and writes its outputs to, one folder. Copy the reference metadata into a new folder: Copy Describe the Performance Measure Raw algorithm performance must be finite and nonnegative. Zero is allowed; negative values are rejected. Missing test outcomes can be represented by NaN . This restriction applies to raw performance, not features or transformed coordinates. The one decision you must always make is how to judge performance. Here performance is an error rate, so lower is better, and an algorithm counts as good on an instance when its error is below 20%: Copy With AbsPerf = false , an algorithm is good when it is within epsilon (a fraction) of the best algorithm on that instance. Every other option has a default; see the Options Reference . Build the Instance Space Copy build runs the whole pipeline: INIT reads and validates the metadata, PRELIM labels and normalises the data, SIFTED selects features, PILOT projects the instances to 2D, CLOISTER estimates the boundary of the space, PYTHIA trains a classifier per algorithm, and TRACE finds the footprints. It then saves model.mat and writes CSV and PNG files to rootdir . It takes a few minutes, most of it in SIFTED and PYTHIA. Look at the Results Copy The PNG files in rootdir show the same views for every feature and algorithm. Evaluate New Instances Copy explore applies the trained preprocessing, projection, classifiers and footprints to the new instances; nothing is refitted. Next Steps Change an option and re-run only the affected stages — see InstanceSpace . Run and inspect each stage in turn — see the Stage-by-Stage Walkthrough . Build a 3D instance space with opts.pilot.dims = 3 — see PILOT and PILOTviewpoint . Use your own data — see Metadata File Format . References Muñoz, M.A., Villanova, L., Baatar, D. & Smith-Miles, K. (2018). Instance spaces for machine learning classification. Machine Learning , 107(1), 109–147. https://doi.org/10.1007/s10994-017-5629-5 Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895"},{"t":"INIT","u":"INIT.html","p":"Read and filter instance metadata","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","Version History","v0.9.2 — Independent pipeline stage","v0.9.1 — Introduced","References","See Also"],"x":"Syntax [data,extra] = INIT(rootdir,opts) [data,extra] = INIT(rootdir,opts,trainedModel) Description [data,extra] = INIT(rootdir,opts) reads rootdir/metadata.csv , keeps the features and algorithms listed in opts.selvars.feats and opts.selvars.algos (all of them when these fields are absent), and returns the matrices the other stages need. [data,extra] = INIT(rootdir,opts,trainedModel) reads rootdir/metadata_test.csv for evaluation. It checks that the test file has the trained model's features in the same order, and aligns the algorithm columns with the trained model: known algorithms go to their trained position, new algorithms are appended, and trained algorithms missing from the test file get a column of NaN . InstanceSpace.build exposes this as the first stage, init . Its output is cached in model.init.data for subsequent prelim runs. Both build and explore callbacks report init before prelim . Examples Expand all Read the reference metadata Copy Read only some algorithms Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains metadata.csv (training) or metadata_test.csv (evaluation). Must end with a file separator. opts — Toolbox options structure The complete options structure. INIT reads opts.selvars.feats and opts.selvars.algos , cell arrays of column names including their feature_ / algo_ prefix. trainedModel — Trained model structure The model property of a built InstanceSpace object. Its presence selects evaluation mode. Output Arguments Expand all data — Metadata structure data.instlabels — Instance names From the Instances column. data.X , data.Xraw — Features Numeric matrix; both are the unprocessed values at this point. data.Y , data.Yraw — Performance Numeric matrix; both are the unprocessed values at this point. data.algolabels — Algorithm names Without the algo_ prefix. data.featlabels — Feature names Training mode only, without the feature_ prefix. data.S — Instance sources Categorical vector, present only when the file has a source column. extra — Evaluation bookkeeping structure Empty in training mode. In evaluation mode: featlabelsAll , modelalgos (number of trained algorithms), newalgos (algorithms new in the test file), nalgos and ninst . Tips See Metadata File Format for the file layout. Evaluation fails with ISA:InstanceSpace:featureOrderMismatch when metadata_test.csv lists its features in a different order from the training file. Version History v0.9.2 — Independent pipeline stage InstanceSpace exposes init as a selectable build stage, retains its output for preprocessing reruns, and reports it through build and explore callbacks. v0.9.1 — Introduced Replaces two separate readers inside InstanceSpace (build and explore) with one function. References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also PRELIM | InstanceSpace | Metadata File Format"},{"t":"ISAdefaults","u":"ISAdefaults.html","p":"Fill in missing options with their default values","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax opts = ISAdefaults(opts) Description opts = ISAdefaults(opts) returns opts with every missing field set to its default, so that every stage receives a complete options structure. Fields already present are not changed. The defaults are listed in the Options Reference . ISAdefaults also maps a few legacy field names to their current names when the current name is absent: opts.parallel.flag / .ncores to opts.general.parallel / .ncores , opts.pilot.ISA3D to opts.pilot.dims , opts.cloister.cthres to opts.cloister.corrThreshold , opts.pythia.cvfolds to opts.pythia.kFold , and opts.pythia.useknn = false to opts.pythia.classifier = 'svm' . InstanceSpace calls ISAdefaults after ISAvalidateOpts when an object is created or loaded. Examples Expand all See every default Copy Complete a partial options file Copy Input Arguments Expand all opts — Options structure Any subset of the options, for example as decoded from options.json . Output Arguments Expand all opts — Complete options structure Version History v0.9.0 — Introduced Defaults moved from each stage into one function. See Also ISAvalidateOpts | InstanceSpace | Options Reference"},{"t":"ISAgetClassifierFcn","u":"ISAgetClassifierFcn.html","p":"Look up a classifier in the PYTHIA registry","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax [fitFcn,p1label,p2label] = ISAgetClassifierFcn(name) Description [fitFcn,p1label,p2label] = ISAgetClassifierFcn(name) returns the Statistics and Machine Learning Toolbox fitting function for the classifier name , and the names of the hyperparameters that PYTHIA tunes for it. name is the value of opts.pythia.classifier . name Fitting function First hyperparameter (search range) Second hyperparameter 'knn' fitcknn NumNeighbors [1, 25] Distance (categorical) 'svm' fitcsvm BoxConstraint (log2 scale) KernelScale (log2 scale) 'tree' fitctree MinLeafSize [1, 100] — 'nb' fitcnb Bandwidth (log10 scale) — 'linear' fitclinear Lambda (log10 scale) — 'ensemble' fitcensemble NumLearningCycles [10, 200] MinLeafSize [1, 20] Each classifier is binary: PYTHIA trains one per algorithm, so multiclass learners such as fitcecoc are not needed. Examples Expand all Look up the SVM entry Copy Copy Supply fixed hyperparameters for a single-parameter classifier With opts.pythia.tuning = 'none' , opts.pythia.params needs one column per hyperparameter. Copy Input Arguments Expand all name — Classifier name 'knn' | 'svm' | 'tree' | 'nb' | 'linear' | 'ensemble' Case insensitive. Any other value raises ISA:ISAgetClassifierFcn:unknownClassifier . Output Arguments Expand all fitFcn — Fitting function function handle p1label — First hyperparameter name character vector p2label — Second hyperparameter name character vector 'N/A' for classifiers with one hyperparameter. Version History v0.9.0 — Introduced See Also PYTHIA | Options Reference"},{"t":"ISAmigrateModel","u":"ISAmigrateModel.html","p":"Update a model from an earlier toolbox version to the current layout","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — PLS boundary alignment and Octave archives","v0.9.0 — Introduced","See Also"],"x":"Syntax model = ISAmigrateModel(rootdir) model = ISAmigrateModel(rootdir,'backupSuffix',suffix) model = ISAmigrateModel(model) Description model = ISAmigrateModel(rootdir) migrates rootdir/model.mat in place. The original file is first copied to model_legacy.mat in the same folder. Use this form once, to convert a saved model. model = ISAmigrateModel(rootdir,'backupSuffix',suffix) names the backup model.mat . An existing backup is never overwritten; ISAmigrateModel raises ISA:ISAmigrateModel:backupExists instead. model = ISAmigrateModel(model) migrates a model structure already in memory and writes no files. InstanceSpace.load and exploreIS use this form on every load, so a legacy model.mat also works without converting it first. Examples Expand all Convert a saved model Copy Migrate in memory Copy Input Arguments Expand all rootdir — Folder containing model.mat character vector | string scalar model — Model structure structure A model loaded with load('model.mat') . suffix — Backup file suffix '_legacy' (default) | character vector File form only. Output Arguments Expand all model — Migrated model structure Algorithms Legacy content Becomes opts.oracle , opts.pbldr , opts.sbound , opts.footprint opts.pythia , opts.pilot , opts.cloister , opts.trace opts.corr.flag , opts.corr.threshold , opts.clust.flag fields of opts.sifted opts.perf.MaxMin opts.perf.MaxPerf model.data.bestPerformace model.data.Ybest PLS model without pilot.Xmean recover the mean from training features and translate saved cloist.Zedge / Zecorr by -Xmean*A' ; face indices are unchanged model.pythia.svm{i} , model.pythia.knn{i} model.pythia.classifiers{i} model.pythia.boxcosnt , model.pythia.kscale model.pythia.param1 , model.pythia.param2 LIBSVM classifier structures retrained with opts.pythia.classifier (default 'knn' ), because LIBSVM structures have no predict method model.trace in the DBSCAN/polyshape format recomputed with TRACE3, using model.pythia.Yhat when available, otherwise model.data.Ybin no completedStages inferred from the stage fields present A model.pilot.A without B and C cannot be repaired and raises a warning. GNU Octave File-based migration recognizes versioned Octave archives and preserves their explicit reconstruction format. MATLAB legacy migration retains its existing file format. Version History v0.9.2 — PLS boundary alignment and Octave archives File-based migration now reads versioned Octave model archives. Schema-1 archives remain readable; saving writes schema 2, which keeps ordinary struct fields separate from internal object markers. Recovering a missing PLS feature mean also translates any saved CLOISTER vertices into the fitted PLS coordinate system. Empty or absent boundaries are retained. Repeated migration leaves already centred geometry unchanged. Models without the training features needed to recover the mean must be rebuilt. v0.9.0 — Introduced See Also Migrating a Legacy Model | InstanceSpace | exploreIS"},{"t":"ISArecallView","u":"ISArecallView.html","p":"Return a 3D figure to its optimised viewpoint","h":["Syntax","Description","Examples","Input Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax ISArecallView(fig) ISArecallView(fig,algoIdx) Description ISArecallView(fig) sets the camera of the 3D instance-space figure fig to the global viewpoint found by PILOTviewpoint . Use it after rotating a figure saved by scriptpng . ISArecallView(fig,algoIdx) uses the viewpoint of the group in opts.pilot.viewGroups that contains algorithm algoIdx , or the global viewpoint if no group contains it. The viewpoints are stored in the figure's UserData by scriptpng . Examples Expand all Restore the view of a saved footprint Copy Input Arguments Expand all fig — Figure figure handle A 3D figure saved by scriptpng . algoIdx — Algorithm index [] (default) | positive integer Column of the algorithm in model.data.algolabels . Version History v0.9.0 — Introduced See Also PILOTviewpoint | scriptpng"},{"t":"ISAsubsetData","u":"ISAsubsetData.html","p":"Keep a subset of the instances in a data structure","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax data = ISAsubsetData(data,subsetIndex) data = ISAsubsetData(data,subsetIndex,featIdx) Description data = ISAsubsetData(data,subsetIndex) keeps the rows subsetIndex of every per-instance field of data : X , Y , Xraw , Yraw , Ybin , beta , numGoodAlgos , Ybest , P , instlabels and, if present, S . data = ISAsubsetData(data,subsetIndex,featIdx) also keeps only the feature columns featIdx of data.X and data.featlabels . InstanceSpace uses this form after SIFTED when density filtering is on. Examples Expand all Keep the beta-easy instances Copy Input Arguments Expand all data — Instance data structure model.data , with the fields listed above. subsetIndex — Instances to keep logical vector | vector of indices featIdx — Features to keep vector of indices Output Arguments Expand all data — Subset structure Version History v0.9.0 — Introduced See Also FILTER | InstanceSpace"},{"t":"ISAvalidateOpts","u":"ISAvalidateOpts.html","p":"Check the type and range of user-supplied options","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax opts = ISAvalidateOpts(opts) Description opts = ISAvalidateOpts(opts) checks each recognised field that is present in opts and raises an error on the first invalid one. Absent fields are not checked; ISAdefaults fills them afterwards. opts is returned unchanged. An error here names the field and the expected value, instead of failing later inside a stage. A field set to [] counts as present and is rejected. Field names the function does not recognise, including misspellings, are not reported. InstanceSpace calls ISAvalidateOpts when an object is created. Examples Expand all Catch an invalid value early Copy Input Arguments Expand all opts — Options structure Output Arguments Expand all opts — Options structure The input, unchanged. Tips Error identifiers, by the check that failed: Identifier Expected ISA:ISAvalidateOpts:notStruct a structure, for opts and each group such as opts.pilot ISA:ISAvalidateOpts:notLogical a logical scalar, true or false (not 0 or 1) ISA:ISAvalidateOpts:notFiniteNumericScalar a finite real number ISA:ISAvalidateOpts:notInteger a whole number ISA:ISAvalidateOpts:notPositive a positive number (0 is allowed for seeds) ISA:ISAvalidateOpts:notInUnitRange a number in [0, 1] ISA:ISAvalidateOpts:notMember one of a fixed set of values ISA:ISAvalidateOpts:notText a character vector or string ISA:ISAvalidateOpts:notCellOfText a cell array of character vectors ISA:ISAvalidateOpts:badViewGroups a cell array of algorithm index vectors Version History v0.9.0 — Introduced See Also ISAdefaults | InstanceSpace | Options Reference"},{"t":"InstanceSpace","u":"InstanceSpace.html","p":"Build, explore and save an instance space","h":["Syntax","Description","Examples","Input Arguments","Properties","Object Functions","build","explore","getResults","plot","save","load","GNU Octave","Version History","v0.9.2 — Stage validation and Octave workflows","v0.9.1 — Stage callbacks and boundary plot","v0.9.0 — Introduced","References","See Also"],"x":"Syntax obj = InstanceSpace(rootdir) obj = InstanceSpace(rootdir,opts) Description An InstanceSpace object holds the options, the trained model, and the evaluation results of one instance space analysis. It runs the pipeline stages INIT , PRELIM , SIFTED , PILOT , CLOISTER , PYTHIA and TRACE in order, lets you change options and re-run individual stages, evaluates the trained model on new instances, and saves and loads the model. obj = InstanceSpace(rootdir) creates an object for the metadata in rootdir/metadata.csv . Options are read from rootdir/options.json if the file exists; missing options take their defaults. No computation runs until you call build . obj = InstanceSpace(rootdir,opts) uses the options structure opts instead of options.json . During build , algorithms with no good training instances are removed. Preprocessing is then fitted to the retained portfolio. Best-algorithm indices, best performance, beta-easy flags, and saved transforms all refer to that portfolio. With an absolute threshold, removing an algorithm can change the best performance on an instance where no algorithm is good. InstanceSpace is a value class: a method that changes the object returns the changed copy, so assign the result, as in obj = obj.build() . Examples Expand all Build and explore an instance space Copy Change an option and re-run one stage Re-running a stage discards the results of every later stage, which you then re-run. Copy Inspect each stage as it finishes Copy Save and load a model Copy Input Arguments Expand all rootdir — Data folder character vector | string scalar Folder containing metadata.csv . Output files and model.mat are written here. See Metadata File Format . opts — Options structure Any subset of the fields in the Options Reference . Values are checked by ISAvalidateOpts , and missing fields filled by ISAdefaults . Properties Expand all rootdir — Data folder character vector Always ends with a file separator. opts — Options structure Complete options. Change fields between build calls to re-run stages with new settings. explore uses the options stored with the model at training time instead. model — Trained model structure One field per completed stage: data , prelim , featsel , sifted , pilot , cloist (CLOISTER), pythia , trace , and opts once every stage has run. completedStages — Completed stages cell array of character vectors testDirs , testResults — Evaluation results cell arrays One entry per explore call: the folder and the result structure, which has the same fields as model . Object Functions Function Purpose build Run pipeline stages explore Evaluate the trained model on new instances getResults Return the training or an evaluation result plot Plot a view of the instance space save Write model.mat InstanceSpace.load Create an object from model.mat build Copy Runs every stage, or only the named stages from {'init','prelim','sifted','pilot','cloister','pythia','trace'} , always in pipeline order. A stage whose prerequisites have not completed raises ISA:InstanceSpace:missingPrereq . callback(stageName, model) is called after each stage, including init . For compatibility, requesting prelim automatically runs init first if its input snapshot is missing. init loads and filters metadata, publishes it in model.data , and retains an independent input snapshot in model.init.data . Rerunning prelim reuses that snapshot. To reread a changed metadata file, rerun init ; this invalidates every later stage. Changes to selvars.feats , selvars.algos , or prelim.nanThreshold require rerunning init . Full builds retain the same computations and numerical outputs. Saved models now also contain the input snapshot. Legacy models without this snapshot remain usable for exploration; rebuilding their prelim stage automatically runs init and therefore requires the original metadata file. When all stages have completed, build saves model.mat and writes the outputs selected in opts.outputs with scriptcsv , scriptweb and scriptpng"},{"t":"Stage-by-Stage Walkthrough","u":"InteractiveWalkthrough.html","p":"Run each stage of the pipeline and inspect its output","h":["PRELIM: Label and Normalise","SIFTED: Select Features","PILOT: Project to 2D","CLOISTER: Estimate the Boundary","PYTHIA: Predict Good Algorithms","TRACE: Find the Footprints","Save and Explore","See Also"],"x":"This walkthrough builds the instance space one stage at a time so you can see what each stage produces. It follows liveDemoIS.m in the repository root, which you can open in the Live Editor and run section by section. It uses the same reference data and performance settings as Getting Started . Copy PRELIM: Label and Normalise PRELIM decides which algorithms are good on each instance, bounds outlying feature values, and normalises features and performance. Copy An instance is beta-easy when more than opts.perf.betaThreshold of the algorithms are good on it. SIFTED: Select Features SIFTED keeps the features that best explain performance: a correlation filter, then a genetic algorithm that picks one feature from each cluster of correlated features. Copy Set opts.sifted.flag = false to keep every feature. PILOT: Project to 2D PILOT finds the linear projection Z = X*A' from which features and performance can best be reconstructed. Z is the instance space. Copy opts.pilot.dims = 3 gives a 3D space; opts.pilot.method = 'pls' uses Partial Least Squares. CLOISTER: Estimate the Boundary CLOISTER estimates where instances could exist, given the ranges and correlations of the features. Empty areas inside the boundary are candidates for new test instances. Copy PYTHIA: Predict Good Algorithms PYTHIA trains one classifier per algorithm that predicts, from an instance's position, whether the algorithm is good there, and combines them into an algorithm selector. Copy TRACE: Find the Footprints TRACE finds the region where each algorithm is good (its footprint) and where it is best, and measures their size, density and purity. Copy Save and Explore Once every stage has run, build has already saved model.mat and written the output files. explore evaluates new instances with the trained model: Copy To change a setting later, edit obj.opts and re-run from the stage it affects, for example obj = obj.build('stages', {'pythia','trace'}) after changing opts.pythia.classifier . Re-running a stage discards the results of later stages. See Also InstanceSpace | Getting Started | Options Reference"},{"t":"Metadata File Format","u":"MetadataFormat.html","p":"Prepare metadata.csv for your own problem domain","h":["Columns","Example","Missing Values","Test Instances","Choosing Features","Version History","v0.9.2 — Training feature schema","References","See Also"],"x":"An instance space is built from one CSV file, metadata.csv , with one row per problem instance. Instances to evaluate later with explore go in metadata_test.csv , in the same format. Columns Columns are recognised by their header. The order of the column groups does not matter, and headers are case insensitive. Header Required Contents Instances yes Instance name. Text or numbers. Source no Where the instance comes from, for example the benchmark suite. Drawn by obj.plot('sources') . feature_ at least two Numeric feature value. algo_ at least one Numeric performance of the algorithm on the instance. Other columns are ignored. Names after the prefix must be valid MATLAB identifiers: use underscores instead of spaces, for example feature_edge_density or algo_simulated_annealing . Example The first rows of the reference data ( test/data/metadata.csv ), shortened to three features and three algorithms: Copy Missing Values Leave a cell empty or write NaN for a missing value. A feature with at least opts.prelim.nanThreshold missing values is removed. After filtering, training requires complete features and performance values. Remaining missing values cause an error that names the affected instances. Evaluation requires complete retained features but permits missing performance values, which never count as observed failures in classifier scoring. Avoid NA , spreadsheet error codes such as #DIV/0! , and empty rows: they make MATLAB read the whole column as text, which then fails later in the pipeline. Test Instances metadata_test.csv must contain the same feature_ columns as metadata.csv , in the same order. It may contain: a subset of the trained algorithms — the missing ones are not scored; algorithms that were not trained — they are appended to the results, without a classifier. Choosing Features The features determine what the instance space can reveal. Good features: are cheap to compute compared with running the algorithms; capture properties that plausibly affect how hard an instance is for each algorithm; vary across the instances, with few repeated values. Start with more candidate features than you need; SIFTED removes the ones that do not explain performance. See Smith-Miles & Muñoz (2023) for guidance on feature design. Version History v0.9.2 — Training feature schema Training stores the feature columns removed by nanThreshold . Exploration removes those same columns before checking the required feature schema, even when their test values are complete. References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also INIT | InstanceSpace | Options Reference"},{"t":"Migrating a Legacy Model","u":"MigratingLegacyModel.html","p":"Use a model.mat created by a toolbox version before v0.9.0","h":["Automatic Migration on Load","Converting the File Once","What Changes","When Migration Cannot Complete","See Also"],"x":"Version 0.9.0 renamed several option groups and model fields and replaced the LIBSVM classifiers and the footprint algorithm. A model.mat saved by an earlier version therefore needs migrating before the current code can use it. In most cases this is automatic. Automatic Migration on Load InstanceSpace.load and exploreIS migrate a legacy model in memory each time they load it. Nothing is written to disk: Copy Converting the File Once To avoid repeating the migration, including any classifier retraining, on every load, convert the file: Copy The original is kept as model_legacy.mat . Choose another backup name with ISAmigrateModel(rootdir,'backupSuffix','_v08') . What Changes Options are renamed to the current groups: opts.oracle → opts.pythia , opts.pbldr → opts.pilot , opts.sbound → opts.cloister , opts.footprint → opts.trace , and opts.perf.MaxMin → opts.perf.MaxPerf . The feature-selection flags of opts.corr and opts.clust move to opts.sifted . Classifiers. LIBSVM models cannot be evaluated by the current code. They are retrained with the classifier in opts.pythia.classifier ( 'knn' if unset) on the training data stored in the model. Predictions of the retrained model can differ slightly from those of the original. Footprints in the old DBSCAN/polyshape format are recomputed with TRACE3. Their areas, densities and purities can differ from the values reported by the old version. Field names such as data.bestPerformace are corrected. The complete table is on the ISAmigrateModel reference page. When Migration Cannot Complete A projection without its reconstruction matrices ( model.pilot.B , model.pilot.C ) is kept as is, with a warning. Retraining LIBSVM classifiers and recomputing footprints need the training data stored in the model ( model.pilot.Z and model.data ). When it is missing, the fields are renamed only, with a warning. If a LIBSVM model has to be evaluated as-is, PYTHIA raises ISA:PYTHIA:noLibsvm unless the LIBSVM MEX files are on the path. The toolbox does not include them; get them from the LIBSVM project . Retraining is recommended instead. See Also ISAmigrateModel | InstanceSpace | What's New"},{"t":"Options Reference","u":"OptionsReference.html","p":"Every field of the options structure, with its default","h":["opts.general","opts.perf","opts.prelim","opts.auto","opts.bound","opts.norm","opts.selvars","opts.sifted","opts.pilot","opts.cloister","opts.pythia","opts.trace","opts.outputs","Version History","v0.9.2 — Option validation","See Also"],"x":"Options control every stage of the pipeline. Give them to InstanceSpace as a structure, or as an options.json file in the data folder with the same nesting: Copy Copy Set only the fields you want to change. ISAvalidateOpts checks the fields you set, and ISAdefaults fills in the rest with the defaults below. Between staged build calls you can change obj.opts directly. opts.general Settings for the whole pipeline. Field Default Description seed 42 Random seed. Every stochastic stage derives its seed from this value, so the same seed and data reproduce a run. verbose true Detailed progress output. Stage start and end messages are always printed. parallel false Use a parallel pool (Parallel Computing Toolbox) for SIFTED, PILOT, PYTHIA and TRACE. ncores 18 Workers when opening a pool. An existing pool is reused unchanged. opts.perf How performance is judged. Used by PRELIM . Observed raw performance must be finite and nonnegative. Zero is allowed; negative scores are rejected in both absolute and relative modes. Relative mode substitutes machine epsilon for exact zero numerators and denominators; see PRELIM for the resulting zero-reference convention. Field Default Description MaxPerf false true if larger performance values are better; false for a cost such as error or run time. AbsPerf false true : good means better than epsilon . false : good means within a fraction epsilon of the best algorithm on the instance. epsilon 0.05 Good-performance threshold. In [0, 1] when AbsPerf is false ; any real number otherwise. betaThreshold 0.55 An instance is beta-easy when more than this fraction of the algorithms are good on it. opts.prelim Data preparation. Used by INIT and PRELIM . Field Default Description iqrMultiplier 5 Features are bounded to median ± iqrMultiplier*IQR . nanThreshold 0.20 A feature with at least this fraction of missing values is removed. opts.auto Used by PRELIM . Field Default Description preproc true Run the automatic outlier bounding and normalisation. false leaves the data as given. opts.bound Used by PRELIM . Field Default Description flag true Bound feature outliers. Features with very little variation can have an IQR of zero; remove them or turn bounding off. opts.norm Used by PRELIM . Field Default Description flag true Apply Box-Cox and Z-score transforms, so features and performance are close to normally distributed. Recommended, because PILOT is a linear projection and CLOISTER expects centred data. opts.selvars Which features, algorithms and instances to use. Used by INIT , InstanceSpace and FILTER . Field Default Description feats all Cell array of feature column names to use, with their feature_ prefix. algos all Cell array of algorithm column names to use, with their algo_ prefix. smallscaleflag false Build from a random fraction of the instances. Useful to try settings on a large dataset. smallscale 0.30 Fraction of instances kept when smallscaleflag is true . fileidxflag false Build from the instances listed in the file fileidx . fileidx '' CSV file with one column of instance indices (row numbers of metadata.csv ). densityflag false Remove near-duplicate instances with FILTER . mindistance 0.10 FILTER distance threshold in feature space. type 'Ftr&Good' FILTER removal condition: 'Ftr' , 'Ftr&AP' , 'Ftr&Good' or 'Ftr&AP&Good' . opts.sifted Feature selection. Used by SIFTED . Field Default Description diagnostics true Run the advisory silhouette sweep. Disable it to reduce selection overhead. flag true Run SIFTED. false keeps every feature. rho 0.10 Minimum absolute correlation between a feature and an algorithm's performance. pval 0.05 Significance level of the correlations. K 10 Number of feature clusters, which is the number of features selected. We recommend 10 or fewer. MaxIter 1000 Maximum k-means iterations. Replicates 100 Number of k-means replicates. seed opts.general.seed Random seed of SIFTED. opts.pilot Projection. Used by PILOT and PILOTviewpoint . Field Default Description dims 2 Dimensio"},{"t":"PILOT","u":"PILOT.html","p":"Project instance features onto a 2D or 3D instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Projection corrections and Octave support","v0.9.1 — Seed control","v0.9.0 — 3D projections and PLS","References","See Also"],"x":"Syntax out = PILOT(X,Y,featlabels,opts) Description out = PILOT(X,Y,featlabels,opts) finds a linear projection Z = X*A' of the feature matrix X into opts.dims dimensions, chosen so that both the features and the algorithm performance Y can be reconstructed from Z as well as possible (Prediction-based Linear Dimensionality Reduction). Instances with similar features and similar performance land close together, so trends in performance become visible as directions in the space. PILOT has three solvers, selected by opts.method and opts.analytic : Numerical (default): BFGS minimisation of the reconstruction error of X and Y from Z (see Algorithms), restarted opts.ntries times. The trial whose projection best preserves the pairwise distances of the original data is kept. Analytic : the closed-form eigenvector solution. Fast and deterministic. If X is rank deficient, PILOT falls back to the numerical solver. PLS : Partial Least Squares regression of Y on X with plsregress . opts.alpha does not apply. Examples Expand all Project the reference data in 2D Prepare the features with the earlier pipeline stages, then call PILOT directly. Copy Compare the solvers The analytic solver is deterministic and much faster than BFGS. Compare how well each reconstructs the data. Copy Build a 3D instance space Set opts.pilot.dims to 3 before building. InstanceSpace then also calls PILOTviewpoint to find the best 2D viewing angles. Copy Input Arguments Expand all X — Feature matrix numeric matrix Instance features, one row per instance and one column per feature, normally the output of PRELIM and SIFTED . Y — Performance matrix numeric matrix Algorithm performance, one row per instance and one column per algorithm. Must have the same number of rows as X . featlabels — Feature names cell array of character vectors One name per column of X . Used for the row labels of out.summary . opts — Projection options structure Normally obj.opts.pilot . Missing fields take the defaults set by ISAdefaults . opts.dims — Number of dimensions 2 (default) | 3 Dimension of the instance space. opts.method — Projection method 'standard' (default) | 'pls' 'standard' uses the analytic or numerical PILOT solution. 'pls' uses Partial Least Squares. opts.analytic — Use the closed-form solution false (default) | true Only used when opts.method is 'standard' . Falls back to the numerical solver when X is rank deficient. opts.ntries — Number of BFGS restarts 10 (default) | positive integer More restarts reduce the chance of a poor local minimum and take proportionally longer. opts.alpha — Weight of the performance term 1.0 (default) | positive scalar Weight of ||Y - C*Z||^2 relative to the feature term. Larger values favour a projection that explains performance over one that explains features. Not used by 'pls' . opts.seed — Random seed opts.general.seed (default) | integer Seeds the BFGS starting points, so repeated runs give the same projection. opts.verbose — Report progress opts.general.verbose (default) | logical opts.X0 — Starting points numeric matrix Optional user-supplied BFGS starting points, one column per trial. opts.precalcAlpha — Precomputed solution numeric vector Optional full BFGS solution vector from an earlier run. Skips the optimisation. Output Arguments Expand all out — Projection structure out.A — Projection matrix dims -by- nfeats matrix. Project new, identically preprocessed instances with Z = Xnew*out.A' . out.Z — Instance coordinates ninst -by- dims coordinates of each instance in the instance space. out.B , out.C — Reconstruction matrices Reconstruct the features and the performance from Z . out.error — Reconstruction error Sum of squared errors of the reconstruction of [X Y] . out.R2 — Coefficient of determination R^2 of the reconstruction of each column of [X Y] . out.summary — Projection table Cell array showing A with the feature labels. Also written to projection_matrix.csv by scriptcsv . out.alpha , out.X0 , out.eoptim , out.perf — Trial results"},{"t":"PILOTviewpoint","u":"PILOTviewpoint.html","p":"Find the best viewing angles of a 3D instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Camera conventions and Octave support","v0.9.0 — Introduced","References","See Also"],"x":"Syntax out = PILOTviewpoint(Z,Y,opts) Description out = PILOTviewpoint(Z,Y,opts) finds, for each group of algorithms in opts.viewGroups , the 2D view of the 3D projection Z from which the performance of those algorithms is best explained. Each view is a pair of orthogonal directions v1 , v2 ; the returned azimuth and elevation point the MATLAB camera along cross(v1,v2) (Simpson et al., 2025, Equation 2). InstanceSpace calls PILOTviewpoint after PILOT when opts.pilot.dims is 3 and stores the result in model.pilot.viewpoint . scriptpng then renders 3D figures from these angles. Examples Expand all View a 3D instance space from its best angle Copy One viewpoint per group of algorithms Copy Input Arguments Expand all Z — 3D instance coordinates ninst -by-3 numeric matrix model.pilot.Z from a 3D build. Y — Performance matrix numeric matrix Same number of rows as Z . opts — Viewpoint options structure Normally obj.opts.pilot . opts.viewGroups — Algorithm groups {} (default) | cell array of index vectors One viewpoint is computed per group, from the columns of Y in that group. Empty means one viewpoint for all algorithms. opts.ntries — Number of BFGS restarts 10 (default) | positive integer opts.seed — Random seed opts.general.seed (default) | integer opts.X0 — Starting points numeric matrix Optional, 2*3 + 2*numel(group) rows by one column per trial. Used for each group of matching size. Output Arguments Expand all out — Viewpoints structure out.groups — Algorithm groups The groups used, one cell per viewpoint. out.A — View matrices Cell array; each cell is a 2-by-3 matrix [v1; v2] that flattens Z onto the viewing plane. out.azimuth , out.elevation — Camera angles In radians, one per group. Use view(rad2deg(az), rad2deg(el)) . Algorithms For each group, PILOTviewpoint jointly fits the view A (2-by-3) and a performance reconstruction C by BFGS, minimising Copy The second term keeps the two view directions orthogonal. Like PILOT, it keeps the trial whose view best preserves the pairwise distances of Z . GNU Octave Octave runs the viewpoint optimization serially through a namespaced fminunc option adapter. MATLAB keeps native optimization options. Equal starting seeds do not guarantee identical optimized camera angles. Version History v0.9.2 — Camera conventions and Octave support Added serial Octave viewpoint optimization with runtime-specific optimizer options. Azimuth and elevation use the cart2sph convention in radians. Plot helpers convert azimuth to the MATLAB camera convention and use equal data-axis scales to preserve the fitted plane. v0.9.0 — Introduced Part of the 3D instance space (ISA3) support. References Simpson, C., Muñoz, M.A., Kandanaarachchi, S. & Campello, R.J.G.B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning , 114, 240. https://doi.org/10.1007/s10994-025-06871-5 See Also PILOT | ISArecallView | scriptpng"},{"t":"PRELIM","u":"PRELIM.html","p":"Label good performance and normalise features and performance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Performance labels and Octave normalization","v0.9.1 — Evaluation mode","v0.9.0 — Outlier bound option","References","See Also"],"x":"Syntax [X,Y,out] = PRELIM(X,Y,opts) [X,Y,out] = PRELIM(X,Y,opts,trainedPrelim) Description [X,Y,out] = PRELIM(X,Y,opts) fits the preprocessing to the training data and applies it. PRELIM labels each algorithm as good or not good on each instance, finds the best algorithm per instance, bounds feature outliers, and applies Box-Cox and Z-score transforms to the features and the performance. The fitted bounds and transform parameters are returned in out . [X,Y,out] = PRELIM(X,Y,opts,trainedPrelim) applies the bounds and transforms of an earlier training call, trainedPrelim , to new data instead of fitting new ones. The good/best labelling is the same code in both modes, so training and evaluation label ties the same way. Examples Expand all Preprocess a performance matrix Misclassification error is a cost ( MaxPerf = false ). Call an algorithm good when its error is below 0.20. Copy Apply a trained preprocessing to new instances Copy Input Arguments Expand all X — Feature matrix numeric matrix One row per instance, one column per feature. May contain NaN . Y — Performance matrix numeric matrix One row per instance, one column per algorithm. Observed raw performance must be finite and nonnegative in both training and evaluation. Zero is allowed; negative values raise ISA:PRELIM:invalidPerformance . May contain NaN ; a missing value never counts as good. opts — Preprocessing options structure When PRELIM is called by InstanceSpace , this structure is assembled from opts.perf , opts.auto , opts.bound , opts.norm and opts.prelim . opts.MaxPerf — Performance direction false (default) | true true if larger performance values are better (for example accuracy). false for a cost, such as run time or error. Set from opts.perf.MaxPerf . opts.AbsPerf — Threshold type false (default) | true true : an algorithm is good when its performance is better than epsilon . false : an algorithm is good when its performance is within a fraction epsilon of the best algorithm on that instance. Set from opts.perf.AbsPerf . opts.epsilon — Good-performance threshold 0.05 (default) | scalar Absolute threshold, or relative tolerance, depending on AbsPerf . Set from opts.perf.epsilon . opts.betaThreshold — Easy-instance fraction 0.55 (default) | scalar in [0, 1] An instance is beta-easy when more than this fraction of the algorithms are good on it. Set from opts.perf.betaThreshold . opts.auto — Run the preprocessing true (default) | false false skips the outlier bounds and the transforms; only the labelling runs. Set from opts.auto.preproc . opts.bound — Bound outliers true (default) | false Clip each feature to median ± iqrMultiplier*IQR . Set from opts.bound.flag . opts.norm — Normalise true (default) | false Apply Box-Cox and then Z-score transforms to X and Y . Set from opts.norm.flag . opts.iqrMultiplier — Outlier bound width 5 (default) | positive scalar Set from opts.prelim.iqrMultiplier . trainedPrelim — Trained preprocessing structure The out of an earlier training-mode call. Its presence selects evaluation mode. Output Arguments Expand all X — Preprocessed features numeric matrix Y — Preprocessed performance numeric matrix With AbsPerf = false , Y is first converted to performance relative to the best algorithm, then normalised. out — Labels and fitted parameters structure out.Ybin — Good-performance labels ninst -by- nalgos logical matrix. out.Ybest — Best performance Best raw performance on each instance. out.P — Best algorithm Index of the best algorithm on each instance. Ties are broken at random. out.numGoodAlgos — Number of good algorithms Per instance. out.beta — Beta-easy instances Logical vector; true when more than betaThreshold of the algorithms are good. out.hibound , out.lobound , out.medval , out.iqrange — Outlier bounds Training mode only. out.minX , out.lambdaX , out.muX , out.sigmaX — Feature transform Training mode only. Shift, Box-Cox parameter, mean and standard deviation per feature. out.minY , out.lambdaY , out.muY , out.sigmaY — P"},{"t":"PYTHIA","u":"PYTHIA.html","p":"Train classifiers that predict which algorithms perform well","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","GNU Octave","Version History","v0.9.2 — Evaluation corrections and Octave KNN","v0.9.0 — Classifier registry","References","See Also"],"x":"Syntax out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts) out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,trainedModel) Description out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts) trains one binary classifier per algorithm that predicts, from an instance's coordinates Z , whether the algorithm is good on it (Performance Yielding Heuristic Identification of Algorithms). Hyperparameters are tuned by k -fold cross-validation. PYTHIA then combines the predictions into an algorithm selector: for each instance it picks the algorithm predicted to be good that has the highest cross-validated precision. out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,trainedModel) applies the classifiers of an earlier training call to new instances and reports how accurate they are there. Classifiers are not retrained. The classifier type is set by opts.classifier ; see ISAgetClassifierFcn for the registry and the tuned hyperparameters. Examples Expand all Train k-NN classifiers on a 2D instance space Copy Use an SVM with Bayesian optimisation Copy Evaluate trained classifiers on new instances InstanceSpace.explore does this for you. The explicit call is: Copy Input Arguments Expand all Z — Instance coordinates numeric matrix model.pilot.Z . PYTHIA standardises it internally and stores the mean and standard deviation. Y — Raw performance matrix numeric matrix model.data.Yraw . Used for the performance columns of the summary and, with opts.useweights , for the class weights. NaN marks a missing value. Ybin — Good-performance labels logical matrix The classification targets. Ybest — Best performance per instance numeric vector algolabels — Algorithm names cell array of character vectors opts — Classifier options structure Normally obj.opts.pythia . opts.classifier — Classifier type 'knn' (default) | 'svm' | 'tree' | 'nb' | 'linear' | 'ensemble' opts.tuning — Hyperparameter search 'sobol' (default) | 'bayes' | 'none' 'sobol' : a scrambled Sobol sequence of nTuningIter candidates. 'bayes' : bayesopt with the same budget. 'none' : use opts.params . opts.nTuningIter — Search budget 20 (default) | positive integer opts.kFold — Cross-validation folds 5 (default) | positive integer opts.params — Fixed hyperparameters [] (default) | nalgos -by- nparams matrix Required when opts.tuning is 'none' ; one row per algorithm. opts.useweights — Cost-sensitive training false (default) | true Weight each instance by how far its performance is from the mean performance. opts.ispolykrnl — Polynomial kernel false (default) | true SVM only. false uses a Gaussian kernel. opts.ensembleMethod — Ensemble method 'Bag' (default) | any fitcensemble method Only for 'ensemble' . opts.skip — Skip training false (default) | true Return empty predictions without training. TRACE then uses Ybin directly. opts.seed — Random seed opts.general.seed (default) | integer opts.verbose — Report progress opts.general.verbose (default) | logical trainedModel — Trained classifiers structure The out of an earlier training call ( model.pythia ). Its presence selects evaluation mode. Output Arguments Expand all out — Classifiers, predictions and summary structure out.classifiers — Trained classifiers Cell array with one ClassificationModel per algorithm. out.Yhat — Predicted labels ninst -by- nalgos logical matrix. In training mode these are the predictions of the final model on the training data. out.Pr0hat — Predicted bad-class scores Bad-class score for each algorithm on each instance. Consult the corresponding out.scoreType entry before interpreting a column: only probability denotes the probability that the algorithm is not good. decision-score and class-score are classifier scores and need not lie in [0,1]; unknown has no declared score semantics, and unavailable marks a placeholder rather than a usable score. out.Ysub , out.Pr0sub — Cross-validated predictions Training mode only. Ysub contains predicted labels; Pr0sub contains bad-class scores. out.scoreTypeCV describes each score column. A mixed column combines "},{"t":"SIFTED","u":"SIFTED.html","p":"Select the features that best explain algorithm performance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","GNU Octave","Version History","v0.9.2 — Diagnostics, cache handling and Octave support","v0.9.1 — Seed control","v0.9.0 — Renamed from SIFTED2","References","See Also"],"x":"Syntax [X,out] = SIFTED(X,Y,Ybin,featlabels,opts) Description [X,out] = SIFTED(X,Y,Ybin,featlabels,opts) returns the columns of X that best explain algorithm performance, and the indices of those columns in out.selvars . Selection has two steps: Correlation filter. For every algorithm, keep the feature most correlated with its performance. Also keep every feature whose absolute Pearson correlation with the performance of at least one algorithm is at least opts.rho and significant at level opts.pval . Clustering and genetic search. Group the remaining features into opts.K clusters by correlation distance (k-means). A genetic algorithm then picks one feature per cluster, scoring each candidate set by how well a k -nearest-neighbour classifier predicts Ybin in a PILOT projection of those features. Step 2 is skipped when three or fewer features remain, or when no more than opts.K remain. Examples Expand all Select features from the reference data Copy Check the choice of K out.eva holds the silhouette value for each number of clusters. When the value for opts.K is poor, SIFTED also suggests a better K . Copy Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed features, one row per instance. Y — Performance matrix numeric matrix Preprocessed performance, one row per instance. Ybin — Good-performance labels logical matrix From PRELIM . Used by the genetic search's fitness function. featlabels — Feature names cell array of character vectors opts — Selection options structure Normally obj.opts.sifted . opts.rho — Minimum correlation 0.10 (default) | scalar in [0, 1] opts.pval — Significance level 0.05 (default) | scalar in (0, 1) opts.K — Number of clusters 10 (default) | positive integer Number of features SIFTED selects. When no more than K features pass the correlation filter, all of them are kept. opts.MaxIter — k-means iterations 1000 (default) | positive integer opts.Replicates — k-means replicates 100 (default) | positive integer opts.seed — Random seed opts.general.seed (default) | integer Seeds k-means, the cross-validation partition, and the genetic algorithm. opts.dims — Projection dimension for the fitness function 2 (default) | 3 InstanceSpace sets this from opts.pilot.dims , so features are chosen for the same dimension as the final projection. Output Arguments Expand all X — Selected features numeric matrix The columns out.selvars of the input X . out — Selection details structure out.selvars — Selected feature indices Indices into the columns of the input X . out.rho , out.p — Correlations Correlation of each feature with each algorithm's performance, and the p-values. out.eva — Cluster evaluation evalclusters result with the silhouette value for each number of clusters. out.clust — Cluster membership nfeats -by- K logical matrix. out.Ksuggested — Suggested number of clusters Present only when the silhouette value for opts.K is below 0.5. Tips opts.sifted.flag = false skips SIFTED in InstanceSpace and keeps every feature. SIFTED is the slowest stage. Set opts.general.parallel = true to evaluate the genetic algorithm's population in parallel. GNU Octave Octave runs serial correlation clustering and CV/KNN fitness with bounded integer search. Small search spaces are enumerated; larger ones use a serial GA. The backend and fitness are recorded in search, with cvpartition retaining the folds. Octave uses batch k-means updates; MATLAB retains its online phase and native GA. Version History v0.9.2 — Diagnostics, cache handling and Octave support Added serial Octave clustering and integer search, with the search backend and CV partition recorded in the result. Set opts.sifted.diagnostics=false to skip the advisory silhouette sweep. Feature clustering and selection still run with the configured K. A call with opts.parallel=false resets only its client fitness cache and does not dispatch work to an existing pool. A later parallel call resets its worker caches before using them. v0.9.1 — Seed control k-mean"},{"t":"TRACE","u":"TRACE.html","p":"Compute algorithm footprints in the instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Footprint evaluation and Octave support","v0.9.0 — TRACE3","References","See Also"],"x":"Syntax out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts) out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts,trainedTrace) Description out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts) finds each algorithm's footprint : the regions of the instance space where the algorithm is good ( out.good ) and where it is the best algorithm ( out.best ), plus the region of beta-hard instances ( out.hard ). For each footprint TRACE reports its area (2D) or volume (3D), density and purity, relative to the whole space. out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts,trainedTrace) keeps the footprints of an earlier training call, trainedTrace , and recomputes their density and purity with new instances. Footprints are not rebuilt. The default method, TRACE3, works in 2D and 3D. opts.method = 'legacy' selects the earlier DBSCAN-based method, which is 2D only. Examples Expand all Footprints of a trained instance space Copy Rebuild footprints with a stricter purity Copy Input Arguments Expand all Z — Instance coordinates ninst -by-2 or ninst -by-3 numeric matrix Ybin — Good-performance labels logical matrix Yhat — Predicted labels logical matrix | [] model.pythia.Yhat . TRACE3 uses only instances that are both good and predicted good, which removes instances the classifier cannot separate. With [] it uses Ybin alone and warns ( ISA:TRACE3:noPYTHIA ). P — Best algorithm per instance integer vector beta — Beta-easy instances logical vector ~beta defines the hard-instance footprint. algolabels — Algorithm names cell array of character vectors opts — Footprint options structure Normally obj.opts.trace . opts.method — Footprint method 'trace3' (default) | 'legacy' 'legacy' falls back to 'trace3' with a warning for a 3D space. opts.PI — Purity threshold 0.6 (default) | scalar in [0, 1] Target fraction of good instances inside a footprint. Check accepted before treating a training footprint as meeting this target. opts.minInstances — Minimum instances 4 (default) | positive integer Fewer instances than this give an empty footprint. opts.minAreaFrac — Minimum size 0.01 (default) | scalar in [0, 1] A footprint smaller than this fraction of the whole space is discarded. opts.contra — Remove contradictions false (default; true for 'legacy' ) | logical Legacy method only: remove the overlap between best-algorithm footprints. opts.boundaryTolerance — Explicit boundary distance 0 (default) | finite nonnegative scalar Euclidean distance in projection-coordinate units. A positive value includes points within this distance of a fitted edge (2D) or triangular face (3D), including boundaries of holes. Hole interiors farther away remain excluded. Zero preserves exact closed-boundary membership. There is no automatic epsilon multiplier, coordinate scaling or dependence on the query batch. The same rule is used for construction metrics, legacy overlap counts and trimming, and test-set evaluation. Geometry and area/volume are not expanded. The fitted value is saved as out.boundaryTolerance ; evaluation reuses it even if a different option is supplied. Older models without this field use zero. Choose a positive distance from an explicit uncertainty budget in the projection units; this option does not guarantee stable legacy trimming under arbitrary coordinate perturbations. trainedTrace — Trained footprints structure model.trace . Its presence selects evaluation mode. Output Arguments Expand all out — Footprints and summary structure out.space — Whole space The convex hull of all instances: measure (area or volume), measureLabel , elements , density , purity . out.good — Good-performance footprints Cell array, one footprint per algorithm. Each footprint has polygon (an alphaShape ), measure , measureLabel , elements , goodElements , density and purity . out.best — Best-algorithm footprints Same format as out.good . out.hard — Beta-hard footprint out.summary — Summary table Cell array: for each algorithm, the area or volume of its good and best footprints (absolute and as a fracti"},{"t":"What's New","u":"WhatsNew.html","p":"Changes in recent versions of the toolbox","h":["v0.9.2","New Functionality","Correctness and Reporting","Upgrading and Limitations","Documentation","v0.9.1","v0.9.0","See Also"],"x":"The complete list of changes, including every bug fix, is in RELEASE_NOTES.md in the repository root. v0.9.2 New Functionality Independent INIT stage for metadata ingestion, callbacks and saved input snapshots. PRELIM reruns reuse the snapshot; rerunning INIT reloads metadata and invalidates downstream results. Explicit opts.trace.boundaryTolerance , a nonnegative distance in projected coordinates, defaults to zero. Footprints include their exact boundaries; the saved tolerance is reused during exploration. Serial GNU Octave 2D/3D build, save/load and exploration with KNN and none/Sobol tuning, TRACE3, CSV and PNG. Validation uses Octave 11.3, Statistics 2.0.0 and Datatypes 1.5.0. Qt and a display are required for 3D PNG. Bayesian tuning, parallel execution, other classifiers, legacy TRACE, FIG and web palette output remain MATLAB-only. Octave classifier archives are runtime-specific. 3D CLOISTER boundaries and CSV triangle-face exports, including a geometry manifest and explicit empty footprints. Correctness and Reporting Recompute preprocessing after pruning algorithms that are never good, reset SIFTED caches between runs, retain inputs for feature-selection reruns, and apply fitted PLS centering during exploration. Recommendations depend on training policy and new features. Missing test outcomes are excluded from metrics. Oracle probability uses only rows with observed outcomes; selector recall avoids double counting. Classifier summaries distinguish CV from test metrics and probabilities from raw scores. Persist stage options and provenance for safe partial build/save/load/resume; reject incompatible partial rebuilds and replay training feature exclusions during exploration. Evaluate footprints against trained geometry for small or degenerate test batches. Preserve holes in 2D exports and record footprint acceptance and search termination. Replace obsolete toolkit-owned output files on export. Reproducible seeding immediately before SVM posterior calibration, complete numerical projection defaults, constant-column scaling and validation of seed bounds and input data. Upgrading and Limitations Rebuild experiments from the original metadata to obtain corrected results. Loading an existing model does not apply fixes to its fitted artifacts. Back up models and outputs first; rebuild CLOISTER for older 3D models and preprocessing before rerunning SIFTED if the saved feature-selection input is absent. Raw performance must be nonnegative (zero is allowed), and incomplete training data remaining after ingestion filtering is rejected. See Migrating a Legacy Model for pre-v0.9.0 layouts. Classifier CV summaries are conditional on the fitted preprocessing, feature selection, projection and tuning. They are not an outer-CV estimate of the full pipeline. Numerical PILOT restart selection can amplify floating-point differences across BLAS implementations and thread settings, so a seed alone does not promise identical projections across environments. Projection orientation is not canonicalized automatically. opts.pilot.topoWeight remains reserved and disabled. Documentation Reference documentation is available in the MATLAB Help browser and on the web, with examples for every function. The version header, release notes and citation metadata now identify v0.9.2. v0.9.1 Engineering release; results of the pipeline are unchanged. build and explore accept an 'onStage' callback, called after each stage. obj.plot('boundary') and distribution_boundary.png show the CLOISTER boundary of a 2D space. INIT reads the metadata for both training and evaluation, and PRELIM gained an evaluation mode, so evaluation now breaks ties exactly as training does. opts.general.seed now also controls PILOT and SIFTED . InstanceSpace checks that each stage's inputs exist before running it. Continuous integration and a matlab.unittest test suite. v0.9.0 Complete refactor of the toolbox. The InstanceSpace class is the main interface; buildIS and exploreIS remain as w"},{"t":"buildIS","u":"buildIS.html","p":"Build an instance space from a folder (backward-compatible wrapper)","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Wrapper around InstanceSpace","See Also"],"x":"Syntax model = buildIS(rootdir) Description model = buildIS(rootdir) builds an instance space from rootdir/metadata.csv and rootdir/options.json , writes model.mat and the output files to rootdir , prints EOF:SUCCESS , and returns the trained model. It is equivalent to Copy buildIS keeps the calling convention of versions before v0.9.0, which the MATILDA web platform and other scripts use. For new code, use InstanceSpace . Examples Expand all Build from a folder Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains metadata.csv and, optionally, options.json . Output Arguments Expand all model — Trained model structure The model property of the built InstanceSpace object. Version History v0.9.0 — Wrapper around InstanceSpace buildIS became a wrapper around InstanceSpace . Its inputs, outputs and files are unchanged. See Also exploreIS | InstanceSpace"},{"t":"exploreIS","u":"exploreIS.html","p":"Evaluate a saved instance space on new instances (backward-compatible wrapper)","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Wrapper around InstanceSpace","See Also"],"x":"Syntax out = exploreIS(rootdir) Description out = exploreIS(rootdir) loads rootdir/model.mat , evaluates rootdir/metadata_test.csv with it, writes the output files to rootdir , prints EOF:SUCCESS , and returns the result. It is equivalent to Copy A model.mat from a version before v0.9.0 is migrated in memory by ISAmigrateModel . For new code, use InstanceSpace . Examples Expand all Evaluate test instances after buildIS Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains model.mat and metadata_test.csv . Output Arguments Expand all out — Evaluation result structure Same fields as a trained model, computed for the test instances. Version History v0.9.0 — Wrapper around InstanceSpace exploreIS became a wrapper around InstanceSpace . Its inputs, outputs and files are unchanged. See Also buildIS | InstanceSpace | ISAmigrateModel"},{"t":"Instance Space Analysis Toolbox","u":"index.html","p":"Objective algorithm testing through the instance space of a problem","h":["Get Started","Getting Started","Stage-by-Stage Walkthrough","Metadata File Format","Options Reference","Functions and Classes","InstanceSpace","Pipeline Stages","Utilities","Output","Topics","Installation","References"],"x":"The Instance Space Analysis (ISA) Toolbox relates the structural features of problem instances to the performance of a portfolio of algorithms. It projects every instance into a 2D or 3D instance space , shows where each algorithm performs well (its footprint ), predicts which algorithm to use for a new instance, and estimates where in the space no instances have been observed yet. The toolbox implements the methodology in Smith-Miles & Muñoz (2023), including the 3D extension (ISA3) of Simpson et al. (2025). It runs in MATLAB R2025a or later and powers the MATILDA web platform. The ISA pipeline: INIT and PRELIM prepare the metadata, SIFTED selects features, PILOT projects them, then CLOISTER, PYTHIA and TRACE work in the projected space. INIT read metadata PRELIM label & normalise SIFTED select features PILOT project to 2D/3D CLOISTER empirical bounds PYTHIA predict algorithms TRACE footprints Outputs CSV · PNG · .mat The ISA pipeline. InstanceSpace runs the stages in this order; each stage is also a standalone function. Get Started Getting Started Build and explore an instance space from the bundled reference data in a few lines. Stage-by-Stage Walkthrough Run each stage separately and inspect what it produces. Metadata File Format Prepare metadata.csv for your own problem domain. Options Reference Every field of opts and options.json , with defaults. Functions and Classes InstanceSpace Build, explore, save, load and plot an instance space. The main entry point. Pipeline Stages INIT, PRELIM, FILTER, SIFTED, PILOT, PILOTviewpoint, CLOISTER, PYTHIA, TRACE. Utilities Option defaults and validation, classifier registry, legacy-model migration. Output Write CSV files and figures, and restore 3D viewpoints. Topics Migrating a Legacy Model — use a model.mat from a version before v0.9.0. Backward-Compatible Wrappers — buildIS and exploreIS , used by MATILDA. Deprecated Functions — PYTHIA2 , PYTHIAtest , SIFTED2 . What's New — release notes. Installation Clone or download the repository, then add it to the MATLAB path from its root folder: Copy Required products: MATLAB R2025a or later with the Statistics and Machine Learning, Optimization, Global Optimization, Parallel Computing, and Financial toolboxes. A bounded serial GNU Octave workflow is also supported; see What's New for supported classifiers, outputs and persistence limits. With the repository root on the path, this documentation also appears in the MATLAB Help browser under Supplemental Software . References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 Simpson, C., Muñoz, M.A., Kandanaarachchi, S. & Campello, R.J.G.B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning , 114, 240. https://doi.org/10.1007/s10994-025-06871-5"},{"t":"scriptcsv","u":"scriptcsv.html","p":"Write the results of an instance space to CSV files","h":["Syntax","Description","Examples","Input Arguments","Version History","v0.9.2 — Footprint boundaries and geometry exports","v0.9.1 — Multiple footprint regions","See Also"],"x":"Syntax scriptcsv(container,rootdir) Description scriptcsv(container,rootdir) writes the coordinates, data, predictions, footprints and summary tables of a trained model or an evaluation result to CSV files in rootdir . Coordinate columns are z_1 , z_2 and, for a 3D space, z_3 . InstanceSpace.build and InstanceSpace.explore call scriptcsv when opts.outputs.csv is true . File Contents coordinates.csv instance coordinates Z bounds.csv , bounds_prunned.csv CLOISTER boundary Zedge and Zecorr (training only) feature_raw.csv , feature_process.csv selected features, before and after preprocessing algorithm_raw.csv , algorithm_process.csv performance, before and after preprocessing algorithm_bin.csv good-performance labels Ybin good_algos.csv , beta_easy.csv , portfolio.csv number of good algorithms, beta-easy flag, best algorithm algorithm_svm.csv , portfolio_svm.csv PYTHIA predictions and selected algorithm footprint__good.csv , footprint__best.csv footprint boundary vertices; separate regions and holes are separated by a row of NaN footprint_performance.csv TRACE summary table classifier_table.csv PYTHIA summary table projection_matrix.csv PILOT projection matrix (training only) Examples Expand all Write the CSV files of an evaluation to another folder Copy Input Arguments Expand all container — Model or evaluation result structure obj.model or an element of obj.testResults . rootdir — Output folder character vector Must exist and end with a file separator. Version History v0.9.2 — Footprint boundaries and geometry exports Geometry exports include empty vertex tables when a footprint is empty. For 3D, each vertex table has a corresponding _faces.csv with one-based triangle indices. CLOISTER uses bounds_faces.csv and bounds_prunned_faces.csv . geometry_manifest.json records dimensionality, footprint filenames, and empty status. Two-dimensional rings are NaN-separated, including holes. The exporter replaces the toolkit-owned footprint_*_best.csv , footprint_*_good.csv , their face tables, and boundary CSVs on each run. Keep user-authored files outside these reserved names. A footprint region with a hole writes both its outer boundary and the hole's boundary, separated by a NaN row. v0.9.1 — Multiple footprint regions Every region of a footprint is written, separated by NaN rows, instead of only the first. See Also scriptpng | scriptweb | InstanceSpace"},{"t":"scriptpng","u":"scriptpng.html","p":"Save figures of an instance space as PNG files","h":["Syntax","Description","Examples","Input Arguments","GNU Octave","Version History","v0.9.2 — Plot export updates and Octave support","v0.9.1 — Boundary figure","v0.9.0 — 3D figures","See Also"],"x":"Syntax scriptpng(container,rootdir) Description scriptpng(container,rootdir) draws the standard views of a trained model or an evaluation result and saves them in rootdir : each feature, each algorithm's performance, good/bad labels, predictions, footprints, the algorithm portfolio, the number of good algorithms, the beta score, instance sources, and the CLOISTER boundary. For a 3D space the figures use the viewpoints found by PILOTviewpoint . Each footprint is also saved as a MATLAB figure ( footprint_.fig , footprint_portfolio.fig ) for interactive rotation, unless opts.outputs.fig is false . ISArecallView returns a rotated figure to its stored viewpoint. InstanceSpace.build and InstanceSpace.explore call scriptpng when opts.outputs.png is true . File Figure distribution_feature_.png feature value over the space distribution_performance_global_normalized_.png , ..._individual_normalized_.png performance, scaled over all algorithms or per algorithm binary_performance_.png , binary_classifier_.png good/bad labels, and PYTHIA's predictions footprint_.png , footprint_portfolio.png footprints distribution_portfolio.png , distribution_svm_portfolio.png best and selected algorithm distribution_number_good_algos.png , distribution_beta_score.png number of good algorithms, beta-easy instances distribution_sources.png instance sources, if the metadata has a source column distribution_boundary.png CLOISTER boundary (training only) Examples Expand all Save figures at a larger font size Copy Input Arguments Expand all container — Model or evaluation result structure obj.model or an element of obj.testResults . rootdir — Output folder character vector Must exist and end with a file separator. GNU Octave Octave exports PNG through print and uses viridis. Qt is selected when available and is required for 3D export; use a virtual display for headless execution. MATLAB retains exportgraphics and parula. FIG output is MATLAB-only. Version History v0.9.2 — Plot export updates and Octave support Added Octave PNG export through Qt, with viridis colours and MATLAB-only FIG output. Direct Octave calls requesting 3D FIG output fail before removing existing output files. Each plot export removes prior toolkit-owned plot filenames before drawing the current run. This clears plots for removed algorithms and features, absent sources or bounds, and old 3D FIG files. Keep user-authored files outside the toolkit's distribution_* , binary_* , and footprint_* output names. distribution_boundary.png is also drawn for a 3D space. v0.9.1 — Boundary figure Adds distribution_boundary.png . v0.9.0 — 3D figures 3D spaces are drawn from their optimised viewpoints and saved as .fig files too. See Also scriptcsv | ISArecallView | PILOTviewpoint | InstanceSpace"},{"t":"scriptweb","u":"scriptweb.html","p":"Write colour-scaled data for the MATILDA web platform","h":["Syntax","Description","Examples","Input Arguments","See Also"],"x":"Syntax scriptweb(container,rootdir) Description scriptweb(container,rootdir) writes the features, performance and number of good algorithms as colour indices, with the colour table, to CSV files in rootdir . The MATILDA web platform uses these files to draw its interactive plots. You do not need them for local work. InstanceSpace.build and InstanceSpace.explore call scriptweb when both opts.outputs.csv and opts.outputs.web are true . File Contents color_table.csv the colour map feature_raw_color.csv , feature_process_color.csv feature colour indices algorithm_raw_color.csv , algorithm_process_color.csv performance colour indices, scaled over all algorithms algorithm_raw_single_color.csv , algorithm_process_single_color.csv performance colour indices, scaled per algorithm good_algos_color.csv colour index of the number of good algorithms Examples Expand all Turn on the web output Copy Input Arguments Expand all container — Model or evaluation result structure rootdir — Output folder character vector Must exist and end with a file separator. See Also scriptcsv | InstanceSpace"}]; +window.ISA_SEARCH = [{"t":"CLOISTER","u":"CLOISTER.html","p":"Estimate the boundary of the instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","GNU Octave","Version History","v0.9.2 — 3D boundaries and Octave support","v0.9.0 — Feature limit","References","See Also"],"x":"Syntax out = CLOISTER(X,A,opts) out = CLOISTER(X,A,opts,Xmean) Description out = CLOISTER(X,A,opts) estimates the region of the instance space where instances could exist, using correlations between features (Correlated Limits of the Instance Space's Theoretical or Experimental Regions). It takes every corner of the box bounded by the minimum and maximum of each feature, discards the corners that contradict a strong, significant correlation between two features, and projects the remaining corners with the PILOT matrix A . The convex hull of the projected corners is the estimated boundary. out = CLOISTER(X,A,opts,Xmean) subtracts the fitted feature mean before projection. For PLS, pass model.pilot.Xmean to keep the boundary in the same coordinates as model.pilot.Z . The three-argument form uses a zero mean. The same centring applies to the projected-instance hull used when the feature-count limit is exceeded. Parts of the boundary far from any instance show where new test instances would extend the benchmark. For a 2D projection the boundary is a closed polygon. For a 3D projection it is a triangulated convex surface; when the projected corners all lie on one plane (for example, a 3D projection of only two features), it is the flat polygon, triangulated. Examples Expand all Plot the boundary of a 2D instance space Copy Plot the boundary of a 3D instance space Copy obj.plot('boundary') draws the same figure over the instances. Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed, selected features, one row per instance. May contain NaN . A — Projection matrix dims -by- nfeats numeric matrix model.pilot.A . opts — Boundary options structure Normally obj.opts.cloister . opts.pval — Significance level 0.05 (default) | scalar in (0, 1) Correlations with a larger p-value are ignored. opts.corrThreshold — Correlation threshold 0.70 (default) | scalar in [0, 1] A corner is discarded when it contradicts the sign of a correlation stronger than this. opts.maxFeatures — Feature limit 20 (default) | positive integer The number of corners grows as 2^nfeats . With more features than this, CLOISTER warns ( ISA:CLOISTER:tooManyFeatures ) and returns the convex hull of the projected instances instead. Xmean — Fitted feature mean 1-by- nfeats numeric row vector | default: zeros(1,nfeats) The mean fitted during PILOT training. Pass model.pilot.Xmean for PLS. Corners and fallback instances are projected as (X-Xmean)*A' ; CLOISTER does not estimate a new mean. Omit the argument for an uncentred projection. Output Arguments Expand all out — Boundary structure out.Zedge — Boundary of all corners 2D: closed polygon, first vertex repeated last. 3D: vertices of the convex hull. out.Zecorr — Boundary of the correlation-consistent corners Same format as Zedge . Equals Zedge when no corner is discarded, when too many corners are discarded to form a hull, or when opts.maxFeatures is exceeded. out.ZedgeFaces , out.ZecorrFaces — Hull triangulations 3D only: nfaces -by-3 indices into the rows of Zedge and Zecorr . Empty in 2D. Tips CLOISTER expects mean-centred features. InstanceSpace warns ( ISA:InstanceSpace:cloisterNotMeanCentred ) when opts.auto.preproc or opts.norm.flag is false . scriptcsv writes Zedge to bounds.csv and Zecorr to bounds_prunned.csv . If every projected corner lies on one line (a degenerate projection matrix or features), there is no region to bound and CLOISTER raises ISA:CLOISTER:degenerateBoundary . GNU Octave Octave supports 2D/3D hulls using namespaced Pearson-significance and NaN-aware extrema adapters. The Pearson adapter requires at least three observations. Degenerate/coplanar cases and PLS centering are covered by portable tests. Version History v0.9.2 — 3D boundaries and Octave support Added serial Octave support for Pearson significance and 2D/3D hulls. An optional fourth argument, Xmean , applies the fitted PLS feature centring during projection. The three-argument call uses a zero mean. Corner enumeration"},{"t":"Deprecated Functions","u":"Deprecated.html","p":"Functions kept only for compatibility with older code","h":["Updating Your Code","See Also"],"x":"These functions still work but print a warning and forward to their replacement. Do not use them in new code. Function Replacement Deprecated in PYTHIA2(Z,Y,Ybin,Ybest,algolabels,opts) PYTHIA with opts.classifier = 'knn' v0.9.0 PYTHIAtest(model,Z,Y,Ybin,Ybest,algolabels) PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,model) (evaluation mode) v0.9.0 SIFTED2(X,Y,Ybin,featlabels,opts) SIFTED (renamed) v0.9.0 Updating Your Code Copy See Also PYTHIA | SIFTED | What's New"},{"t":"FILTER","u":"FILTER.html","p":"Remove near-duplicate instances to even out the instance density","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.2 — Bounded neighbour queries","v0.9.0 — Uniformity output","References","See Also"],"x":"Syntax [subsetIndex,isDissimilar,isVISA,unif] = FILTER(X,Y,Ybin,opts) Description [subsetIndex,isDissimilar,isVISA,unif] = FILTER(X,Y,Ybin,opts) marks instances as redundant when they are closer than opts.mindistance in feature space to an instance already kept, and they also meet the condition in opts.type . Removing them gives a smaller, more evenly spread set of instances, which reduces the bias of densely sampled regions on the projection and the footprints (Alipour et al., 2023). InstanceSpace calls FILTER when opts.selvars.densityflag is true : once on all features during PRELIM, and again after SIFTED on the selected features only. Examples Expand all Filter the reference data Copy Filter inside the pipeline Copy Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed features, one row per instance. Y — Performance matrix numeric matrix Ybin — Good-performance labels logical matrix opts — Filter options structure Normally obj.opts.selvars . opts.mindistance — Distance threshold 0.10 (default) | positive scalar Instances closer than this in feature space are candidates for removal. opts.type — Removal condition 'Ftr&Good' (default) | 'Ftr' | 'Ftr&AP' | 'Ftr&AP&Good' Condition, in addition to feature closeness, for an instance to be removed: Value The instance is removed when 'Ftr' it is close in feature space 'Ftr&AP' the two performance vectors are also within sqrt(nalgos/nfeats)*mindistance 'Ftr&Good' every algorithm is also good on both instances 'Ftr&AP&Good' both of the above Output Arguments Expand all subsetIndex — Redundant instances logical vector true for each instance marked redundant. Keep ~subsetIndex . isDissimilar — Isolated instances logical vector false for each instance that was close to another instance. isVISA — Close but kept logical vector true for an instance that was close to a kept instance in feature space but did not meet the opts.type condition, so it was kept. unif — Uniformity of the kept set scalar One minus the coefficient of variation of the nearest-neighbour distances of the kept instances. Values near 1 mean an even spread. Version History v0.9.2 — Bounded neighbour queries The spatial index is queried only for each retained instance, which bounds stored neighbour lists without changing the greedy row order. v0.9.0 — Uniformity output unif is returned; before, it was computed and discarded. References Alipour, H., Muñoz, M.A. & Smith-Miles, K. (2023). Enhanced instance space analysis for the maximum flow problem. European Journal of Operational Research , 304(2), 411–428. https://doi.org/10.1016/j.ejor.2022.04.012 Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also ISAsubsetData | PRELIM | SIFTED | Options Reference"},{"t":"All Functions","u":"FunctionList.html","p":"Every function and class in the toolbox, by category.","h":["Pipeline Stages","Class Interface","Backward-Compatible Wrappers","Utilities","Output"],"x":"Pipeline Stages Name Purpose INIT Read and filter instance metadata PRELIM Label good performance and normalise features and performance FILTER Remove near-duplicate instances to even out the instance density SIFTED Select the features that best explain algorithm performance PILOT Project instance features onto a 2D or 3D instance space PILOTviewpoint Find the best viewing angles of a 3D instance space CLOISTER Estimate the boundary of the instance space PYTHIA Train classifiers that predict which algorithms perform well TRACE Compute algorithm footprints in the instance space Class Interface Name Purpose InstanceSpace Build, explore and save an instance space Backward-Compatible Wrappers Name Purpose buildIS Build an instance space from a folder (backward-compatible wrapper) exploreIS Evaluate a saved instance space on new instances (backward-compatible wrapper) Utilities Name Purpose ISAdefaults Fill in missing options with their default values ISAvalidateOpts Check the type and range of user-supplied options ISAfootprintContains Test footprint membership with an explicit boundary distance ISAgetClassifierFcn Look up a classifier in the PYTHIA registry ISAmigrateModel Update a model from an earlier toolbox version to the current layout ISAsubsetData Keep a subset of the instances in a data structure Output Name Purpose scriptcsv Write the results of an instance space to CSV files scriptpng Save figures of an instance space as PNG files scriptweb Write colour-scaled data for the MATILDA web platform ISArecallView Return a 3D figure to its optimised viewpoint"},{"t":"Getting Started with Instance Space Analysis","u":"GettingStarted.html","p":"Build and explore your first instance space","h":["Prepare a Folder","Describe the Performance Measure","Build the Instance Space","Look at the Results","Evaluate New Instances","Next Steps","References"],"x":"This example builds an instance space for the reference dataset shipped with the toolbox: 212 classification problems (the instances ), described by 10 features, on which 10 classification algorithms were scored by misclassification error (Muñoz et al., 2018). A further 23 instances are held out for testing. The same steps apply to your own data once it is in the metadata format . example.m in the repository root runs this example from start to finish. Prepare a Folder The toolbox reads its inputs from, and writes its outputs to, one folder. Copy the reference metadata into a new folder: Copy Describe the Performance Measure Raw algorithm performance must be finite and nonnegative. Zero is allowed; negative values are rejected. Missing test outcomes can be represented by NaN . This restriction applies to raw performance, not features or transformed coordinates. The one decision you must always make is how to judge performance. Here performance is an error rate, so lower is better, and an algorithm counts as good on an instance when its error is below 20%: Copy With AbsPerf = false , an algorithm is good when it is within epsilon (a fraction) of the best algorithm on that instance. Every other option has a default; see the Options Reference . Build the Instance Space Copy build runs the whole pipeline: INIT reads and validates the metadata, PRELIM labels and normalises the data, SIFTED selects features, PILOT projects the instances to 2D, CLOISTER estimates the boundary of the space, PYTHIA trains a classifier per algorithm, and TRACE finds the footprints. It then saves model.mat and writes CSV and PNG files to rootdir . It takes a few minutes, most of it in SIFTED and PYTHIA. Look at the Results Copy The PNG files in rootdir show the same views for every feature and algorithm. Evaluate New Instances Copy explore applies the trained preprocessing, projection, classifiers and footprints to the new instances; nothing is refitted. Next Steps Change an option and re-run only the affected stages — see InstanceSpace . Run and inspect each stage in turn — see the Stage-by-Stage Walkthrough . Build a 3D instance space with opts.pilot.dims = 3 — see PILOT and PILOTviewpoint . Use your own data — see Metadata File Format . References Muñoz, M.A., Villanova, L., Baatar, D. & Smith-Miles, K. (2018). Instance spaces for machine learning classification. Machine Learning , 107(1), 109–147. https://doi.org/10.1007/s10994-017-5629-5 Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895"},{"t":"INIT","u":"INIT.html","p":"Read and filter instance metadata","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","Version History","v0.9.2 — Independent pipeline stage","v0.9.1 — Introduced","References","See Also"],"x":"Syntax [data,extra] = INIT(rootdir,opts) [data,extra] = INIT(rootdir,opts,trainedModel) Description [data,extra] = INIT(rootdir,opts) reads rootdir/metadata.csv , keeps the features and algorithms listed in opts.selvars.feats and opts.selvars.algos (all of them when these fields are absent), and returns the matrices the other stages need. [data,extra] = INIT(rootdir,opts,trainedModel) reads rootdir/metadata_test.csv for evaluation. It checks that the test file has the trained model's features in the same order, and aligns the algorithm columns with the trained model: known algorithms go to their trained position, new algorithms are appended, and trained algorithms missing from the test file get a column of NaN . InstanceSpace.build exposes this as the first stage, init . Its output is cached in model.init.data for subsequent prelim runs. Both build and explore callbacks report init before prelim . Examples Expand all Read the reference metadata Copy Read only some algorithms Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains metadata.csv (training) or metadata_test.csv (evaluation). Must end with a file separator. opts — Toolbox options structure The complete options structure. INIT reads opts.selvars.feats and opts.selvars.algos , cell arrays of column names including their feature_ / algo_ prefix. trainedModel — Trained model structure The model property of a built InstanceSpace object. Its presence selects evaluation mode. Output Arguments Expand all data — Metadata structure data.instlabels — Instance names From the Instances column. data.X , data.Xraw — Features Numeric matrix; both are the unprocessed values at this point. data.Y , data.Yraw — Performance Numeric matrix; both are the unprocessed values at this point. data.algolabels — Algorithm names Without the algo_ prefix. data.featlabels — Feature names Training mode only, without the feature_ prefix. data.S — Instance sources Categorical vector, present only when the file has a source column. extra — Evaluation bookkeeping structure Empty in training mode. In evaluation mode: featlabelsAll , modelalgos (number of trained algorithms), newalgos (algorithms new in the test file), nalgos and ninst . Tips See Metadata File Format for the file layout. Evaluation fails with ISA:InstanceSpace:featureOrderMismatch when metadata_test.csv lists its features in a different order from the training file. Version History v0.9.2 — Independent pipeline stage InstanceSpace exposes init as a selectable build stage, retains its output for preprocessing reruns, and reports it through build and explore callbacks. v0.9.1 — Introduced Replaces two separate readers inside InstanceSpace (build and explore) with one function. References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also PRELIM | InstanceSpace | Metadata File Format"},{"t":"ISAdefaults","u":"ISAdefaults.html","p":"Fill in missing options with their default values","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax opts = ISAdefaults(opts) Description opts = ISAdefaults(opts) returns opts with every missing field set to its default, so that every stage receives a complete options structure. Fields already present are not changed. The defaults are listed in the Options Reference . ISAdefaults also maps a few legacy field names to their current names when the current name is absent: opts.parallel.flag / .ncores to opts.general.parallel / .ncores , opts.pilot.ISA3D to opts.pilot.dims , opts.cloister.cthres to opts.cloister.corrThreshold , opts.pythia.cvfolds to opts.pythia.kFold , and opts.pythia.useknn = false to opts.pythia.classifier = 'svm' . InstanceSpace calls ISAdefaults after ISAvalidateOpts when an object is created or loaded. Examples Expand all See every default Copy Complete a partial options file Copy Input Arguments Expand all opts — Options structure Any subset of the options, for example as decoded from options.json . Output Arguments Expand all opts — Complete options structure Version History v0.9.0 — Introduced Defaults moved from each stage into one function. See Also ISAvalidateOpts | InstanceSpace | Options Reference"},{"t":"ISAfootprintContains","u":"ISAfootprintContains.html","p":"Test footprint membership with an explicit boundary distance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.2 — Introduced","See Also"],"x":"Syntax inside = ISAfootprintContains(geometry,queries) inside = ISAfootprintContains(geometry,queries,tolerance) Description inside = ISAfootprintContains(geometry,queries) tests whether each query point lies in a footprint, including its boundary. The default distance tolerance is zero. inside = ISAfootprintContains(geometry,queries,tolerance) also includes points within the specified Euclidean distance of a boundary edge in 2D or triangular face in 3D. Hole interiors farther from the boundary remain excluded. The geometry and its area or volume are unchanged. TRACE uses this helper for footprint metrics and evaluation. Set opts.trace.boundaryTolerance to configure the pipeline; TRACE saves the fitted value and reuses it during exploration. The distance is in projected-coordinate units, with no implicit epsilon multiplier, automatic scaling or dependence on other points in the query batch. The membership policy is shared with pyInstanceSpace; floating-point geometry can still differ between implementations. Examples Expand all Include the exact boundary of a square Run startup first when calling utilities directly from a fresh session. Copy Choose an explicit distance tolerance Copy Choose a positive distance from the uncertainty budget of the projected coordinates. A tolerance is not a guarantee that legacy footprint trimming is invariant under arbitrary coordinate perturbations. Input Arguments Expand all geometry — Footprint geometry 2D polyshape | 2D or 3D alphaShape The fitted region, normally a footprint's polygon field. Octave TRACE3 uses the toolkit's compatible alpha-shape class; MATLAB's polyshape and legacy TRACE remain MATLAB-only. An empty geometry produces false for every query. queries — Points to test numeric matrix One row per point, with two or three columns matching the geometry's dimensionality. Nonfinite query points are outside. tolerance — Boundary distance 0 (default) | finite nonnegative real scalar Euclidean distance in projection units. Positive values include points at distance less than or equal to this value from a boundary, including hole boundaries. Output Arguments Expand all inside — Membership mask logical column vector One value per query row. Testing a point alone or alongside other queries uses the same rule. Version History v0.9.2 — Introduced Closed-boundary membership with explicit distance tolerance for TRACE construction, legacy contradiction handling and evaluation. See Also TRACE | ISAdefaults | Options Reference"},{"t":"ISAgetClassifierFcn","u":"ISAgetClassifierFcn.html","p":"Look up a classifier in the PYTHIA registry","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax [fitFcn,p1label,p2label] = ISAgetClassifierFcn(name) Description [fitFcn,p1label,p2label] = ISAgetClassifierFcn(name) returns the Statistics and Machine Learning Toolbox fitting function for the classifier name , and the names of the hyperparameters that PYTHIA tunes for it. name is the value of opts.pythia.classifier . name Fitting function First hyperparameter (search range) Second hyperparameter 'knn' fitcknn NumNeighbors [1, 25] Distance (categorical) 'svm' fitcsvm BoxConstraint (log2 scale) KernelScale (log2 scale) 'tree' fitctree MinLeafSize [1, 100] — 'nb' fitcnb Bandwidth (log10 scale) — 'linear' fitclinear Lambda (log10 scale) — 'ensemble' fitcensemble NumLearningCycles [10, 200] MinLeafSize [1, 20] Each classifier is binary: PYTHIA trains one per algorithm, so multiclass learners such as fitcecoc are not needed. Examples Expand all Look up the SVM entry Copy Copy Supply fixed hyperparameters for a single-parameter classifier With opts.pythia.tuning = 'none' , opts.pythia.params needs one column per hyperparameter. Copy Input Arguments Expand all name — Classifier name 'knn' | 'svm' | 'tree' | 'nb' | 'linear' | 'ensemble' Case insensitive. Any other value raises ISA:ISAgetClassifierFcn:unknownClassifier . Output Arguments Expand all fitFcn — Fitting function function handle p1label — First hyperparameter name character vector p2label — Second hyperparameter name character vector 'N/A' for classifiers with one hyperparameter. Version History v0.9.0 — Introduced See Also PYTHIA | Options Reference"},{"t":"ISAmigrateModel","u":"ISAmigrateModel.html","p":"Update a model from an earlier toolbox version to the current layout","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — PLS boundary alignment and Octave archives","v0.9.0 — Introduced","See Also"],"x":"Syntax model = ISAmigrateModel(rootdir) model = ISAmigrateModel(rootdir,'backupSuffix',suffix) model = ISAmigrateModel(model) Description model = ISAmigrateModel(rootdir) migrates rootdir/model.mat in place. The original file is first copied to model_legacy.mat in the same folder. Use this form once, to convert a saved model. model = ISAmigrateModel(rootdir,'backupSuffix',suffix) names the backup model.mat . An existing backup is never overwritten; ISAmigrateModel raises ISA:ISAmigrateModel:backupExists instead. model = ISAmigrateModel(model) migrates a model structure already in memory and writes no files. InstanceSpace.load and exploreIS use this form on every load, so a legacy model.mat also works without converting it first. Examples Expand all Convert a saved model Copy Migrate in memory Copy Input Arguments Expand all rootdir — Folder containing model.mat character vector | string scalar model — Model structure structure A model loaded with load('model.mat') . suffix — Backup file suffix '_legacy' (default) | character vector File form only. Output Arguments Expand all model — Migrated model structure Algorithms Legacy content Becomes opts.oracle , opts.pbldr , opts.sbound , opts.footprint opts.pythia , opts.pilot , opts.cloister , opts.trace opts.corr.flag , opts.corr.threshold , opts.clust.flag fields of opts.sifted opts.perf.MaxMin opts.perf.MaxPerf model.data.bestPerformace model.data.Ybest PLS model without pilot.Xmean recover the mean from training features and translate saved cloist.Zedge / Zecorr by -Xmean*A' ; face indices are unchanged model.pythia.svm{i} , model.pythia.knn{i} model.pythia.classifiers{i} model.pythia.boxcosnt , model.pythia.kscale model.pythia.param1 , model.pythia.param2 LIBSVM classifier structures retrained with opts.pythia.classifier (default 'knn' ), because LIBSVM structures have no predict method model.trace in the DBSCAN/polyshape format recomputed with TRACE3, using model.pythia.Yhat when available, otherwise model.data.Ybin no completedStages inferred from the stage fields present A model.pilot.A without B and C cannot be repaired and raises a warning. GNU Octave File-based migration recognizes versioned Octave archives and preserves their explicit reconstruction format. MATLAB legacy migration retains its existing file format. Version History v0.9.2 — PLS boundary alignment and Octave archives File-based migration now reads versioned Octave model archives. Schema-1 archives remain readable; saving writes schema 2, which keeps ordinary struct fields separate from internal object markers. Recovering a missing PLS feature mean also translates any saved CLOISTER vertices into the fitted PLS coordinate system. Empty or absent boundaries are retained. Repeated migration leaves already centred geometry unchanged. Models without the training features needed to recover the mean must be rebuilt. v0.9.0 — Introduced See Also Migrating a Legacy Model | InstanceSpace | exploreIS"},{"t":"ISArecallView","u":"ISArecallView.html","p":"Return a 3D figure to its optimised viewpoint","h":["Syntax","Description","Examples","Input Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax ISArecallView(fig) ISArecallView(fig,algoIdx) Description ISArecallView(fig) sets the camera of the 3D instance-space figure fig to the global viewpoint found by PILOTviewpoint . Use it after rotating a figure saved by scriptpng . ISArecallView(fig,algoIdx) uses the viewpoint of the group in opts.pilot.viewGroups that contains algorithm algoIdx , or the global viewpoint if no group contains it. The viewpoints are stored in the figure's UserData by scriptpng . Examples Expand all Restore the view of a saved footprint Copy Input Arguments Expand all fig — Figure figure handle A 3D figure saved by scriptpng . algoIdx — Algorithm index [] (default) | positive integer Column of the algorithm in model.data.algolabels . Version History v0.9.0 — Introduced See Also PILOTviewpoint | scriptpng"},{"t":"ISAsubsetData","u":"ISAsubsetData.html","p":"Keep a subset of the instances in a data structure","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax data = ISAsubsetData(data,subsetIndex) data = ISAsubsetData(data,subsetIndex,featIdx) Description data = ISAsubsetData(data,subsetIndex) keeps the rows subsetIndex of every per-instance field of data : X , Y , Xraw , Yraw , Ybin , beta , numGoodAlgos , Ybest , P , instlabels and, if present, S . data = ISAsubsetData(data,subsetIndex,featIdx) also keeps only the feature columns featIdx of data.X and data.featlabels . InstanceSpace uses this form after SIFTED when density filtering is on. Examples Expand all Keep the beta-easy instances Copy Input Arguments Expand all data — Instance data structure model.data , with the fields listed above. subsetIndex — Instances to keep logical vector | vector of indices featIdx — Features to keep vector of indices Output Arguments Expand all data — Subset structure Version History v0.9.0 — Introduced See Also FILTER | InstanceSpace"},{"t":"ISAvalidateOpts","u":"ISAvalidateOpts.html","p":"Check the type and range of user-supplied options","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","Version History","v0.9.0 — Introduced","See Also"],"x":"Syntax opts = ISAvalidateOpts(opts) Description opts = ISAvalidateOpts(opts) checks each recognised field that is present in opts and raises an error on the first invalid one. Absent fields are not checked; ISAdefaults fills them afterwards. opts is returned unchanged. An error here names the field and the expected value, instead of failing later inside a stage. A field set to [] counts as present and is rejected. Field names the function does not recognise, including misspellings, are not reported. InstanceSpace calls ISAvalidateOpts when an object is created. Examples Expand all Catch an invalid value early Copy Input Arguments Expand all opts — Options structure Output Arguments Expand all opts — Options structure The input, unchanged. Tips Error identifiers, by the check that failed: Identifier Expected ISA:ISAvalidateOpts:notStruct a structure, for opts and each group such as opts.pilot ISA:ISAvalidateOpts:notLogical a logical scalar, true or false (not 0 or 1) ISA:ISAvalidateOpts:notFiniteNumericScalar a finite real number ISA:ISAvalidateOpts:notInteger a whole number ISA:ISAvalidateOpts:notPositive a positive number (0 is allowed for seeds) ISA:ISAvalidateOpts:notInUnitRange a number in [0, 1] ISA:ISAvalidateOpts:notMember one of a fixed set of values ISA:ISAvalidateOpts:notText a character vector or string ISA:ISAvalidateOpts:notCellOfText a cell array of character vectors ISA:ISAvalidateOpts:badViewGroups a cell array of algorithm index vectors Version History v0.9.0 — Introduced See Also ISAdefaults | InstanceSpace | Options Reference"},{"t":"InstanceSpace","u":"InstanceSpace.html","p":"Build, explore and save an instance space","h":["Syntax","Description","Examples","Input Arguments","Properties","Object Functions","build","explore","getResults","plot","save","load","GNU Octave","Version History","v0.9.2 — Stage validation and Octave workflows","v0.9.1 — Stage callbacks and boundary plot","v0.9.0 — Introduced","References","See Also"],"x":"Syntax obj = InstanceSpace(rootdir) obj = InstanceSpace(rootdir,opts) Description An InstanceSpace object holds the options, the trained model, and the evaluation results of one instance space analysis. It runs the pipeline stages INIT , PRELIM , SIFTED , PILOT , CLOISTER , PYTHIA and TRACE in order, lets you change options and re-run individual stages, evaluates the trained model on new instances, and saves and loads the model. obj = InstanceSpace(rootdir) creates an object for the metadata in rootdir/metadata.csv . Options are read from rootdir/options.json if the file exists; missing options take their defaults. No computation runs until you call build . obj = InstanceSpace(rootdir,opts) uses the options structure opts instead of options.json . During build , algorithms with no good training instances are removed. Preprocessing is then fitted to the retained portfolio. Best-algorithm indices, best performance, beta-easy flags, and saved transforms all refer to that portfolio. With an absolute threshold, removing an algorithm can change the best performance on an instance where no algorithm is good. InstanceSpace is a value class: a method that changes the object returns the changed copy, so assign the result, as in obj = obj.build() . Examples Expand all Build and explore an instance space Copy Change an option and re-run one stage Re-running a stage discards the results of every later stage, which you then re-run. Copy Inspect each stage as it finishes Copy Save and load a model Copy Input Arguments Expand all rootdir — Data folder character vector | string scalar Folder containing metadata.csv . Output files and model.mat are written here. See Metadata File Format . opts — Options structure Any subset of the fields in the Options Reference . Values are checked by ISAvalidateOpts , and missing fields filled by ISAdefaults . Properties Expand all rootdir — Data folder character vector Always ends with a file separator. opts — Options structure Complete options. Change fields between build calls to re-run stages with new settings. explore uses the options stored with the model at training time instead. model — Trained model structure One field per completed stage: data , prelim , featsel , sifted , pilot , cloist (CLOISTER), pythia , trace , and opts once every stage has run. completedStages — Completed stages cell array of character vectors testDirs , testResults — Evaluation results cell arrays One entry per explore call: the folder and the result structure, which has the same fields as model . Object Functions Function Purpose build Run pipeline stages explore Evaluate the trained model on new instances getResults Return the training or an evaluation result plot Plot a view of the instance space save Write model.mat InstanceSpace.load Create an object from model.mat build Copy Runs every stage, or only the named stages from {'init','prelim','sifted','pilot','cloister','pythia','trace'} , always in pipeline order. A stage whose prerequisites have not completed raises ISA:InstanceSpace:missingPrereq . callback(stageName, model) is called after each stage, including init . For compatibility, requesting prelim automatically runs init first if its input snapshot is missing. init loads and filters metadata, publishes it in model.data , and retains an independent input snapshot in model.init.data . Rerunning prelim reuses that snapshot. To reread a changed metadata file, rerun init ; this invalidates every later stage. Changes to selvars.feats , selvars.algos , or prelim.nanThreshold require rerunning init . Full builds retain the same computations and numerical outputs. Saved models now also contain the input snapshot. Legacy models without this snapshot remain usable for exploration; rebuilding their prelim stage automatically runs init and therefore requires the original metadata file. When all stages have completed, build saves model.mat and writes the outputs selected in opts.outputs with scriptcsv , scriptweb and scriptpng"},{"t":"Stage-by-Stage Walkthrough","u":"InteractiveWalkthrough.html","p":"Run each stage of the pipeline and inspect its output","h":["PRELIM: Label and Normalise","SIFTED: Select Features","PILOT: Project to 2D","CLOISTER: Estimate the Boundary","PYTHIA: Predict Good Algorithms","TRACE: Find the Footprints","Save and Explore","See Also"],"x":"This walkthrough builds the instance space one stage at a time so you can see what each stage produces. It follows liveDemoIS.m in the repository root, which you can open in the Live Editor and run section by section. It uses the same reference data and performance settings as Getting Started . Copy PRELIM: Label and Normalise PRELIM decides which algorithms are good on each instance, bounds outlying feature values, and normalises features and performance. Copy An instance is beta-easy when more than opts.perf.betaThreshold of the algorithms are good on it. SIFTED: Select Features SIFTED keeps the features that best explain performance: a correlation filter, then a genetic algorithm that picks one feature from each cluster of correlated features. Copy Set opts.sifted.flag = false to keep every feature. PILOT: Project to 2D PILOT finds the linear projection Z = X*A' from which features and performance can best be reconstructed. Z is the instance space. Copy opts.pilot.dims = 3 gives a 3D space; opts.pilot.method = 'pls' uses Partial Least Squares. CLOISTER: Estimate the Boundary CLOISTER estimates where instances could exist, given the ranges and correlations of the features. Empty areas inside the boundary are candidates for new test instances. Copy PYTHIA: Predict Good Algorithms PYTHIA trains one classifier per algorithm that predicts, from an instance's position, whether the algorithm is good there, and combines them into an algorithm selector. Copy TRACE: Find the Footprints TRACE finds the region where each algorithm is good (its footprint) and where it is best, and measures their size, density and purity. Copy Save and Explore Once every stage has run, build has already saved model.mat and written the output files. explore evaluates new instances with the trained model: Copy To change a setting later, edit obj.opts and re-run from the stage it affects, for example obj = obj.build('stages', {'pythia','trace'}) after changing opts.pythia.classifier . Re-running a stage discards the results of later stages. See Also InstanceSpace | Getting Started | Options Reference"},{"t":"Metadata File Format","u":"MetadataFormat.html","p":"Prepare metadata.csv for your own problem domain","h":["Columns","Example","Missing Values","Test Instances","Choosing Features","Version History","v0.9.2 — Training feature schema","References","See Also"],"x":"An instance space is built from one CSV file, metadata.csv , with one row per problem instance. Instances to evaluate later with explore go in metadata_test.csv , in the same format. Columns Columns are recognised by their header. The order of the column groups does not matter, and headers are case insensitive. Header Required Contents Instances yes Instance name. Text or numbers. Source no Where the instance comes from, for example the benchmark suite. Drawn by obj.plot('sources') . feature_ at least two Numeric feature value. algo_ at least one Numeric performance of the algorithm on the instance. Other columns are ignored. Names after the prefix must be valid MATLAB identifiers: use underscores instead of spaces, for example feature_edge_density or algo_simulated_annealing . Example The first rows of the reference data ( test/data/metadata.csv ), shortened to three features and three algorithms: Copy Missing Values Leave a cell empty or write NaN for a missing value. A feature with at least opts.prelim.nanThreshold missing values is removed. After filtering, training requires complete features and performance values. Remaining missing values cause an error that names the affected instances. Evaluation requires complete retained features but permits missing performance values, which never count as observed failures in classifier scoring. Avoid NA , spreadsheet error codes such as #DIV/0! , and empty rows: they make MATLAB read the whole column as text, which then fails later in the pipeline. Test Instances metadata_test.csv must contain the same feature_ columns as metadata.csv , in the same order. It may contain: a subset of the trained algorithms — the missing ones are not scored; algorithms that were not trained — they are appended to the results, without a classifier. Choosing Features The features determine what the instance space can reveal. Good features: are cheap to compute compared with running the algorithms; capture properties that plausibly affect how hard an instance is for each algorithm; vary across the instances, with few repeated values. Start with more candidate features than you need; SIFTED removes the ones that do not explain performance. See Smith-Miles & Muñoz (2023) for guidance on feature design. Version History v0.9.2 — Training feature schema Training stores the feature columns removed by nanThreshold . Exploration removes those same columns before checking the required feature schema, even when their test values are complete. References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 See Also INIT | InstanceSpace | Options Reference"},{"t":"Migrating a Legacy Model","u":"MigratingLegacyModel.html","p":"Use a model.mat created by a toolbox version before v0.9.0","h":["Automatic Migration on Load","Converting the File Once","What Changes","When Migration Cannot Complete","See Also"],"x":"Version 0.9.0 renamed several option groups and model fields and replaced the LIBSVM classifiers and the footprint algorithm. A model.mat saved by an earlier version therefore needs migrating before the current code can use it. In most cases this is automatic. Automatic Migration on Load InstanceSpace.load and exploreIS migrate a legacy model in memory each time they load it. Nothing is written to disk: Copy Converting the File Once To avoid repeating the migration, including any classifier retraining, on every load, convert the file: Copy The original is kept as model_legacy.mat . Choose another backup name with ISAmigrateModel(rootdir,'backupSuffix','_v08') . What Changes Options are renamed to the current groups: opts.oracle → opts.pythia , opts.pbldr → opts.pilot , opts.sbound → opts.cloister , opts.footprint → opts.trace , and opts.perf.MaxMin → opts.perf.MaxPerf . The feature-selection flags of opts.corr and opts.clust move to opts.sifted . Classifiers. LIBSVM models cannot be evaluated by the current code. They are retrained with the classifier in opts.pythia.classifier ( 'knn' if unset) on the training data stored in the model. Predictions of the retrained model can differ slightly from those of the original. Footprints in the old DBSCAN/polyshape format are recomputed with TRACE3. Their areas, densities and purities can differ from the values reported by the old version. Field names such as data.bestPerformace are corrected. The complete table is on the ISAmigrateModel reference page. When Migration Cannot Complete A projection without its reconstruction matrices ( model.pilot.B , model.pilot.C ) is kept as is, with a warning. Retraining LIBSVM classifiers and recomputing footprints need the training data stored in the model ( model.pilot.Z and model.data ). When it is missing, the fields are renamed only, with a warning. If a LIBSVM model has to be evaluated as-is, PYTHIA raises ISA:PYTHIA:noLibsvm unless the LIBSVM MEX files are on the path. The toolbox does not include them; get them from the LIBSVM project . Retraining is recommended instead. See Also ISAmigrateModel | InstanceSpace | What's New"},{"t":"Options Reference","u":"OptionsReference.html","p":"Every field of the options structure, with its default","h":["opts.general","opts.perf","opts.prelim","opts.auto","opts.bound","opts.norm","opts.selvars","opts.sifted","opts.pilot","opts.cloister","opts.pythia","opts.trace","opts.outputs","Version History","v0.9.2 — Option validation","See Also"],"x":"Options control every stage of the pipeline. Give them to InstanceSpace as a structure, or as an options.json file in the data folder with the same nesting: Copy Copy Set only the fields you want to change. ISAvalidateOpts checks the fields you set, and ISAdefaults fills in the rest with the defaults below. Between staged build calls you can change obj.opts directly. opts.general Settings for the whole pipeline. Field Default Description seed 42 Random seed. Every stochastic stage derives its seed from this value, so the same seed and data reproduce a run. verbose true Detailed progress output. Stage start and end messages are always printed. parallel false Use a parallel pool (Parallel Computing Toolbox) for SIFTED, PILOT, PYTHIA and TRACE. ncores 18 Workers when opening a pool. An existing pool is reused unchanged. opts.perf How performance is judged. Used by PRELIM . Observed raw performance must be finite and nonnegative. Zero is allowed; negative scores are rejected in both absolute and relative modes. Relative mode substitutes machine epsilon for exact zero numerators and denominators; see PRELIM for the resulting zero-reference convention. Field Default Description MaxPerf false true if larger performance values are better; false for a cost such as error or run time. AbsPerf false true : good means better than epsilon . false : good means within a fraction epsilon of the best algorithm on the instance. epsilon 0.05 Good-performance threshold. In [0, 1] when AbsPerf is false ; any real number otherwise. betaThreshold 0.55 An instance is beta-easy when more than this fraction of the algorithms are good on it. opts.prelim Data preparation. Used by INIT and PRELIM . Field Default Description iqrMultiplier 5 Features are bounded to median ± iqrMultiplier*IQR . nanThreshold 0.20 A feature with at least this fraction of missing values is removed. opts.auto Used by PRELIM . Field Default Description preproc true Run the automatic outlier bounding and normalisation. false leaves the data as given. opts.bound Used by PRELIM . Field Default Description flag true Bound feature outliers. Features with very little variation can have an IQR of zero; remove them or turn bounding off. opts.norm Used by PRELIM . Field Default Description flag true Apply Box-Cox and Z-score transforms, so features and performance are close to normally distributed. Recommended, because PILOT is a linear projection and CLOISTER expects centred data. opts.selvars Which features, algorithms and instances to use. Used by INIT , InstanceSpace and FILTER . Field Default Description feats all Cell array of feature column names to use, with their feature_ prefix. algos all Cell array of algorithm column names to use, with their algo_ prefix. smallscaleflag false Build from a random fraction of the instances. Useful to try settings on a large dataset. smallscale 0.30 Fraction of instances kept when smallscaleflag is true . fileidxflag false Build from the instances listed in the file fileidx . fileidx '' CSV file with one column of instance indices (row numbers of metadata.csv ). densityflag false Remove near-duplicate instances with FILTER . mindistance 0.10 FILTER distance threshold in feature space. type 'Ftr&Good' FILTER removal condition: 'Ftr' , 'Ftr&AP' , 'Ftr&Good' or 'Ftr&AP&Good' . opts.sifted Feature selection. Used by SIFTED . Field Default Description diagnostics true Run the advisory silhouette sweep. Disable it to reduce selection overhead. flag true Run SIFTED. false keeps every feature. rho 0.10 Minimum absolute correlation between a feature and an algorithm's performance. pval 0.05 Significance level of the correlations. K 10 Number of feature clusters, which is the number of features selected. We recommend 10 or fewer. MaxIter 1000 Maximum k-means iterations. Replicates 100 Number of k-means replicates. seed opts.general.seed Random seed of SIFTED. opts.pilot Projection. Used by PILOT and PILOTviewpoint . Field Default Description dims 2 Dimensio"},{"t":"PILOT","u":"PILOT.html","p":"Project instance features onto a 2D or 3D instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Projection corrections and Octave support","v0.9.1 — Seed control","v0.9.0 — 3D projections and PLS","References","See Also"],"x":"Syntax out = PILOT(X,Y,featlabels,opts) Description out = PILOT(X,Y,featlabels,opts) finds a linear projection Z = X*A' of the feature matrix X into opts.dims dimensions, chosen so that both the features and the algorithm performance Y can be reconstructed from Z as well as possible (Prediction-based Linear Dimensionality Reduction). Instances with similar features and similar performance land close together, so trends in performance become visible as directions in the space. PILOT has three solvers, selected by opts.method and opts.analytic : Numerical (default): BFGS minimisation of the reconstruction error of X and Y from Z (see Algorithms), restarted opts.ntries times. The trial whose projection best preserves the pairwise distances of the original data is kept. Analytic : the closed-form eigenvector solution. Fast and deterministic. If X is rank deficient, PILOT falls back to the numerical solver. PLS : Partial Least Squares regression of Y on X with plsregress . opts.alpha does not apply. Examples Expand all Project the reference data in 2D Prepare the features with the earlier pipeline stages, then call PILOT directly. Copy Compare the solvers The analytic solver is deterministic and much faster than BFGS. Compare how well each reconstructs the data. Copy Build a 3D instance space Set opts.pilot.dims to 3 before building. InstanceSpace then also calls PILOTviewpoint to find the best 2D viewing angles. Copy Input Arguments Expand all X — Feature matrix numeric matrix Instance features, one row per instance and one column per feature, normally the output of PRELIM and SIFTED . Y — Performance matrix numeric matrix Algorithm performance, one row per instance and one column per algorithm. Must have the same number of rows as X . featlabels — Feature names cell array of character vectors One name per column of X . Used for the row labels of out.summary . opts — Projection options structure Normally obj.opts.pilot . Missing fields take the defaults set by ISAdefaults . opts.dims — Number of dimensions 2 (default) | 3 Dimension of the instance space. opts.method — Projection method 'standard' (default) | 'pls' 'standard' uses the analytic or numerical PILOT solution. 'pls' uses Partial Least Squares. opts.analytic — Use the closed-form solution false (default) | true Only used when opts.method is 'standard' . Falls back to the numerical solver when X is rank deficient. opts.ntries — Number of BFGS restarts 10 (default) | positive integer More restarts reduce the chance of a poor local minimum and take proportionally longer. opts.alpha — Weight of the performance term 1.0 (default) | positive scalar Weight of ||Y - C*Z||^2 relative to the feature term. Larger values favour a projection that explains performance over one that explains features. Not used by 'pls' . opts.seed — Random seed opts.general.seed (default) | integer Seeds the BFGS starting points, so repeated runs give the same projection. opts.verbose — Report progress opts.general.verbose (default) | logical opts.X0 — Starting points numeric matrix Optional user-supplied BFGS starting points, one column per trial. opts.precalcAlpha — Precomputed solution numeric vector Optional full BFGS solution vector from an earlier run. Skips the optimisation. Output Arguments Expand all out — Projection structure out.A — Projection matrix dims -by- nfeats matrix. Project new, identically preprocessed instances with Z = Xnew*out.A' . out.Z — Instance coordinates ninst -by- dims coordinates of each instance in the instance space. out.B , out.C — Reconstruction matrices Reconstruct the features and the performance from Z . out.error — Reconstruction error Sum of squared errors of the reconstruction of [X Y] . out.R2 — Coefficient of determination R^2 of the reconstruction of each column of [X Y] . out.summary — Projection table Cell array showing A with the feature labels. Also written to projection_matrix.csv by scriptcsv . out.alpha , out.X0 , out.eoptim , out.perf — Trial results"},{"t":"PILOTviewpoint","u":"PILOTviewpoint.html","p":"Find the best viewing angles of a 3D instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Camera conventions and Octave support","v0.9.0 — Introduced","References","See Also"],"x":"Syntax out = PILOTviewpoint(Z,Y,opts) Description out = PILOTviewpoint(Z,Y,opts) finds, for each group of algorithms in opts.viewGroups , the 2D view of the 3D projection Z from which the performance of those algorithms is best explained. Each view is a pair of orthogonal directions v1 , v2 ; the returned azimuth and elevation point the MATLAB camera along cross(v1,v2) (Simpson et al., 2025, Equation 2). InstanceSpace calls PILOTviewpoint after PILOT when opts.pilot.dims is 3 and stores the result in model.pilot.viewpoint . scriptpng then renders 3D figures from these angles. Examples Expand all View a 3D instance space from its best angle Copy One viewpoint per group of algorithms Copy Input Arguments Expand all Z — 3D instance coordinates ninst -by-3 numeric matrix model.pilot.Z from a 3D build. Y — Performance matrix numeric matrix Same number of rows as Z . opts — Viewpoint options structure Normally obj.opts.pilot . opts.viewGroups — Algorithm groups {} (default) | cell array of index vectors One viewpoint is computed per group, from the columns of Y in that group. Empty means one viewpoint for all algorithms. opts.ntries — Number of BFGS restarts 10 (default) | positive integer opts.seed — Random seed opts.general.seed (default) | integer opts.X0 — Starting points numeric matrix Optional, 2*3 + 2*numel(group) rows by one column per trial. Used for each group of matching size. Output Arguments Expand all out — Viewpoints structure out.groups — Algorithm groups The groups used, one cell per viewpoint. out.A — View matrices Cell array; each cell is a 2-by-3 matrix [v1; v2] that flattens Z onto the viewing plane. out.azimuth , out.elevation — Camera angles In radians, one per group. Use view(rad2deg(az), rad2deg(el)) . Algorithms For each group, PILOTviewpoint jointly fits the view A (2-by-3) and a performance reconstruction C by BFGS, minimising Copy The second term keeps the two view directions orthogonal. Like PILOT, it keeps the trial whose view best preserves the pairwise distances of Z . GNU Octave Octave runs the viewpoint optimization serially through a namespaced fminunc option adapter. MATLAB keeps native optimization options. Equal starting seeds do not guarantee identical optimized camera angles. Version History v0.9.2 — Camera conventions and Octave support Added serial Octave viewpoint optimization with runtime-specific optimizer options. Azimuth and elevation use the cart2sph convention in radians. Plot helpers convert azimuth to the MATLAB camera convention and use equal data-axis scales to preserve the fitted plane. v0.9.0 — Introduced Part of the 3D instance space (ISA3) support. References Simpson, C., Muñoz, M.A., Kandanaarachchi, S. & Campello, R.J.G.B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning , 114, 240. https://doi.org/10.1007/s10994-025-06871-5 See Also PILOT | ISArecallView | scriptpng"},{"t":"PRELIM","u":"PRELIM.html","p":"Label good performance and normalise features and performance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Performance labels and Octave normalization","v0.9.1 — Evaluation mode","v0.9.0 — Outlier bound option","References","See Also"],"x":"Syntax [X,Y,out] = PRELIM(X,Y,opts) [X,Y,out] = PRELIM(X,Y,opts,trainedPrelim) Description [X,Y,out] = PRELIM(X,Y,opts) fits the preprocessing to the training data and applies it. PRELIM labels each algorithm as good or not good on each instance, finds the best algorithm per instance, bounds feature outliers, and applies Box-Cox and Z-score transforms to the features and the performance. The fitted bounds and transform parameters are returned in out . [X,Y,out] = PRELIM(X,Y,opts,trainedPrelim) applies the bounds and transforms of an earlier training call, trainedPrelim , to new data instead of fitting new ones. The good/best labelling is the same code in both modes, so training and evaluation label ties the same way. Examples Expand all Preprocess a performance matrix Misclassification error is a cost ( MaxPerf = false ). Call an algorithm good when its error is below 0.20. Copy Apply a trained preprocessing to new instances Copy Input Arguments Expand all X — Feature matrix numeric matrix One row per instance, one column per feature. May contain NaN . Y — Performance matrix numeric matrix One row per instance, one column per algorithm. Observed raw performance must be finite and nonnegative in both training and evaluation. Zero is allowed; negative values raise ISA:PRELIM:invalidPerformance . May contain NaN ; a missing value never counts as good. opts — Preprocessing options structure When PRELIM is called by InstanceSpace , this structure is assembled from opts.perf , opts.auto , opts.bound , opts.norm and opts.prelim . opts.MaxPerf — Performance direction false (default) | true true if larger performance values are better (for example accuracy). false for a cost, such as run time or error. Set from opts.perf.MaxPerf . opts.AbsPerf — Threshold type false (default) | true true : an algorithm is good when its performance is better than epsilon . false : an algorithm is good when its performance is within a fraction epsilon of the best algorithm on that instance. Set from opts.perf.AbsPerf . opts.epsilon — Good-performance threshold 0.05 (default) | scalar Absolute threshold, or relative tolerance, depending on AbsPerf . Set from opts.perf.epsilon . opts.betaThreshold — Easy-instance fraction 0.55 (default) | scalar in [0, 1] An instance is beta-easy when more than this fraction of the algorithms are good on it. Set from opts.perf.betaThreshold . opts.auto — Run the preprocessing true (default) | false false skips the outlier bounds and the transforms; only the labelling runs. Set from opts.auto.preproc . opts.bound — Bound outliers true (default) | false Clip each feature to median ± iqrMultiplier*IQR . Set from opts.bound.flag . opts.norm — Normalise true (default) | false Apply Box-Cox and then Z-score transforms to X and Y . Set from opts.norm.flag . opts.iqrMultiplier — Outlier bound width 5 (default) | positive scalar Set from opts.prelim.iqrMultiplier . trainedPrelim — Trained preprocessing structure The out of an earlier training-mode call. Its presence selects evaluation mode. Output Arguments Expand all X — Preprocessed features numeric matrix Y — Preprocessed performance numeric matrix With AbsPerf = false , Y is first converted to performance relative to the best algorithm, then normalised. out — Labels and fitted parameters structure out.Ybin — Good-performance labels ninst -by- nalgos logical matrix. out.Ybest — Best performance Best raw performance on each instance. out.P — Best algorithm Index of the best algorithm on each instance. Ties are broken at random. out.numGoodAlgos — Number of good algorithms Per instance. out.beta — Beta-easy instances Logical vector; true when more than betaThreshold of the algorithms are good. out.hibound , out.lobound , out.medval , out.iqrange — Outlier bounds Training mode only. out.minX , out.lambdaX , out.muX , out.sigmaX — Feature transform Training mode only. Shift, Box-Cox parameter, mean and standard deviation per feature. out.minY , out.lambdaY , out.muY , out.sigmaY — P"},{"t":"PYTHIA","u":"PYTHIA.html","p":"Train classifiers that predict which algorithms perform well","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","GNU Octave","Version History","v0.9.2 — Evaluation corrections and Octave KNN","v0.9.0 — Classifier registry","References","See Also"],"x":"Syntax out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts) out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,trainedModel) Description out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts) trains one binary classifier per algorithm that predicts, from an instance's coordinates Z , whether the algorithm is good on it (Performance Yielding Heuristic Identification of Algorithms). Hyperparameters are tuned by k -fold cross-validation. PYTHIA then combines the predictions into an algorithm selector: for each instance it picks the algorithm predicted to be good that has the highest cross-validated precision. out = PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts,trainedModel) applies the classifiers of an earlier training call to new instances and reports how accurate they are there. Classifiers are not retrained. The classifier type is set by opts.classifier ; see ISAgetClassifierFcn for the registry and the tuned hyperparameters. Examples Expand all Train k-NN classifiers on a 2D instance space Copy Use an SVM with Bayesian optimisation Copy Evaluate trained classifiers on new instances InstanceSpace.explore does this for you. The explicit call is: Copy Input Arguments Expand all Z — Instance coordinates numeric matrix model.pilot.Z . PYTHIA standardises it internally and stores the mean and standard deviation. Y — Raw performance matrix numeric matrix model.data.Yraw . Used for the performance columns of the summary and, with opts.useweights , for the class weights. NaN marks a missing value. Ybin — Good-performance labels logical matrix The classification targets. Ybest — Best performance per instance numeric vector algolabels — Algorithm names cell array of character vectors opts — Classifier options structure Normally obj.opts.pythia . opts.classifier — Classifier type 'knn' (default) | 'svm' | 'tree' | 'nb' | 'linear' | 'ensemble' opts.tuning — Hyperparameter search 'sobol' (default) | 'bayes' | 'none' 'sobol' : a scrambled Sobol sequence of nTuningIter candidates. 'bayes' : bayesopt with the same budget. 'none' : use opts.params . opts.nTuningIter — Search budget 20 (default) | positive integer opts.kFold — Cross-validation folds 5 (default) | positive integer opts.params — Fixed hyperparameters [] (default) | nalgos -by- nparams matrix Required when opts.tuning is 'none' ; one row per algorithm. opts.useweights — Cost-sensitive training false (default) | true Weight each instance by how far its performance is from the mean performance. opts.ispolykrnl — Polynomial kernel false (default) | true SVM only. false uses a Gaussian kernel. opts.ensembleMethod — Ensemble method 'Bag' (default) | any fitcensemble method Only for 'ensemble' . opts.skip — Skip training false (default) | true Return empty predictions without training. TRACE then uses Ybin directly. opts.seed — Random seed opts.general.seed (default) | integer opts.verbose — Report progress opts.general.verbose (default) | logical trainedModel — Trained classifiers structure The out of an earlier training call ( model.pythia ). Its presence selects evaluation mode. Output Arguments Expand all out — Classifiers, predictions and summary structure out.classifiers — Trained classifiers Cell array with one ClassificationModel per algorithm. out.Yhat — Predicted labels ninst -by- nalgos logical matrix. In training mode these are the predictions of the final model on the training data. out.Pr0hat — Predicted bad-class scores Bad-class score for each algorithm on each instance. Consult the corresponding out.scoreType entry before interpreting a column: only probability denotes the probability that the algorithm is not good. decision-score and class-score are classifier scores and need not lie in [0,1]; unknown has no declared score semantics, and unavailable marks a placeholder rather than a usable score. out.Ysub , out.Pr0sub — Cross-validated predictions Training mode only. Ysub contains predicted labels; Pr0sub contains bad-class scores. out.scoreTypeCV describes each score column. A mixed column combines "},{"t":"SIFTED","u":"SIFTED.html","p":"Select the features that best explain algorithm performance","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Tips","GNU Octave","Version History","v0.9.2 — Diagnostics, cache handling and Octave support","v0.9.1 — Seed control","v0.9.0 — Renamed from SIFTED2","References","See Also"],"x":"Syntax [X,out] = SIFTED(X,Y,Ybin,featlabels,opts) Description [X,out] = SIFTED(X,Y,Ybin,featlabels,opts) returns the columns of X that best explain algorithm performance, and the indices of those columns in out.selvars . Selection has two steps: Correlation filter. For every algorithm, keep the feature most correlated with its performance. Also keep every feature whose absolute Pearson correlation with the performance of at least one algorithm is at least opts.rho and significant at level opts.pval . Clustering and genetic search. Group the remaining features into opts.K clusters by correlation distance (k-means). A genetic algorithm then picks one feature per cluster, scoring each candidate set by how well a k -nearest-neighbour classifier predicts Ybin in a PILOT projection of those features. Step 2 is skipped when three or fewer features remain, or when no more than opts.K remain. Examples Expand all Select features from the reference data Copy Check the choice of K out.eva holds the silhouette value for each number of clusters. When the value for opts.K is poor, SIFTED also suggests a better K . Copy Input Arguments Expand all X — Feature matrix numeric matrix Preprocessed features, one row per instance. Y — Performance matrix numeric matrix Preprocessed performance, one row per instance. Ybin — Good-performance labels logical matrix From PRELIM . Used by the genetic search's fitness function. featlabels — Feature names cell array of character vectors opts — Selection options structure Normally obj.opts.sifted . opts.rho — Minimum correlation 0.10 (default) | scalar in [0, 1] opts.pval — Significance level 0.05 (default) | scalar in (0, 1) opts.K — Number of clusters 10 (default) | positive integer Number of features SIFTED selects. When no more than K features pass the correlation filter, all of them are kept. opts.MaxIter — k-means iterations 1000 (default) | positive integer opts.Replicates — k-means replicates 100 (default) | positive integer opts.seed — Random seed opts.general.seed (default) | integer Seeds k-means, the cross-validation partition, and the genetic algorithm. opts.dims — Projection dimension for the fitness function 2 (default) | 3 InstanceSpace sets this from opts.pilot.dims , so features are chosen for the same dimension as the final projection. Output Arguments Expand all X — Selected features numeric matrix The columns out.selvars of the input X . out — Selection details structure out.selvars — Selected feature indices Indices into the columns of the input X . out.rho , out.p — Correlations Correlation of each feature with each algorithm's performance, and the p-values. out.eva — Cluster evaluation evalclusters result with the silhouette value for each number of clusters. out.clust — Cluster membership nfeats -by- K logical matrix. out.Ksuggested — Suggested number of clusters Present only when the silhouette value for opts.K is below 0.5. Tips opts.sifted.flag = false skips SIFTED in InstanceSpace and keeps every feature. SIFTED is the slowest stage. Set opts.general.parallel = true to evaluate the genetic algorithm's population in parallel. GNU Octave Octave runs serial correlation clustering and CV/KNN fitness with bounded integer search. Small search spaces are enumerated; larger ones use a serial GA. The backend and fitness are recorded in search, with cvpartition retaining the folds. Octave uses batch k-means updates; MATLAB retains its online phase and native GA. Version History v0.9.2 — Diagnostics, cache handling and Octave support Added serial Octave clustering and integer search, with the search backend and CV partition recorded in the result. Set opts.sifted.diagnostics=false to skip the advisory silhouette sweep. Feature clustering and selection still run with the configured K. A call with opts.parallel=false resets only its client fitness cache and does not dispatch work to an existing pool. A later parallel call resets its worker caches before using them. v0.9.1 — Seed control k-mean"},{"t":"TRACE","u":"TRACE.html","p":"Compute algorithm footprints in the instance space","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Algorithms","GNU Octave","Version History","v0.9.2 — Footprint evaluation and Octave support","v0.9.0 — TRACE3","References","See Also"],"x":"Syntax out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts) out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts,trainedTrace) Description out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts) finds each algorithm's footprint : the regions of the instance space where the algorithm is good ( out.good ) and where it is the best algorithm ( out.best ), plus the region of beta-hard instances ( out.hard ). For each footprint TRACE reports its area (2D) or volume (3D), density and purity, relative to the whole space. out = TRACE(Z,Ybin,Yhat,P,beta,algolabels,opts,trainedTrace) keeps the footprints of an earlier training call, trainedTrace , and recomputes their density and purity with new instances. Footprints are not rebuilt. The default method, TRACE3, works in 2D and 3D. opts.method = 'legacy' selects the earlier DBSCAN-based method, which is 2D only. Examples Expand all Footprints of a trained instance space Copy Rebuild footprints with a stricter purity Copy Input Arguments Expand all Z — Instance coordinates ninst -by-2 or ninst -by-3 numeric matrix Ybin — Good-performance labels logical matrix Yhat — Predicted labels logical matrix | [] model.pythia.Yhat . TRACE3 uses only instances that are both good and predicted good, which removes instances the classifier cannot separate. With [] it uses Ybin alone and warns ( ISA:TRACE3:noPYTHIA ). P — Best algorithm per instance integer vector beta — Beta-easy instances logical vector ~beta defines the hard-instance footprint. algolabels — Algorithm names cell array of character vectors opts — Footprint options structure Normally obj.opts.trace . opts.method — Footprint method 'trace3' (default) | 'legacy' 'legacy' falls back to 'trace3' with a warning for a 3D space. opts.PI — Purity threshold 0.6 (default) | scalar in [0, 1] Target fraction of good instances inside a footprint. Check accepted before treating a training footprint as meeting this target. opts.minInstances — Minimum instances 4 (default) | positive integer Fewer instances than this give an empty footprint. opts.minAreaFrac — Minimum size 0.01 (default) | scalar in [0, 1] A footprint smaller than this fraction of the whole space is discarded. opts.contra — Remove contradictions false (default; true for 'legacy' ) | logical Legacy method only: remove the overlap between best-algorithm footprints. opts.boundaryTolerance — Explicit boundary distance 0 (default) | finite nonnegative scalar Euclidean distance in projection-coordinate units. A positive value includes points within this distance of a fitted edge (2D) or triangular face (3D), including boundaries of holes. Hole interiors farther away remain excluded. Zero preserves exact closed-boundary membership. There is no automatic epsilon multiplier, coordinate scaling or dependence on the query batch. ISAfootprintContains applies the same rule for construction metrics, legacy overlap counts and trimming, and test-set evaluation. Geometry and area/volume are not expanded. The fitted value is saved as out.boundaryTolerance ; evaluation reuses it even if a different option is supplied. Older models without this field use zero. Choose a positive distance from an explicit uncertainty budget in the projection units; this option does not guarantee stable legacy trimming under arbitrary coordinate perturbations. trainedTrace — Trained footprints structure model.trace . Its presence selects evaluation mode. Output Arguments Expand all out — Footprints and summary structure out.space — Whole space The convex hull of all instances: measure (area or volume), measureLabel , elements , density , purity . out.good — Good-performance footprints Cell array, one footprint per algorithm. Each footprint has polygon (an alphaShape ), measure , measureLabel , elements , goodElements , density and purity . out.best — Best-algorithm footprints Same format as out.good . out.hard — Beta-hard footprint out.summary — Summary table Cell array: for each algorithm, the area or volume of its good and best footprints (abs"},{"t":"What's New","u":"WhatsNew.html","p":"Changes in recent versions of the toolbox","h":["v0.9.2","New Functionality","Correctness and Reporting","Upgrading and Limitations","Documentation","v0.9.1","v0.9.0","See Also"],"x":"The complete list of changes, including every bug fix, is in RELEASE_NOTES.md in the repository root. v0.9.2 New Functionality Independent INIT stage for metadata ingestion, callbacks and saved input snapshots. PRELIM reruns reuse the snapshot; rerunning INIT reloads metadata and invalidates downstream results. Explicit opts.trace.boundaryTolerance , a nonnegative distance in projected coordinates, defaults to zero. Footprints include their exact boundaries; the saved tolerance is reused during exploration. Serial GNU Octave 2D/3D build, save/load and exploration with KNN and none/Sobol tuning, TRACE3, CSV and PNG. Validation uses Octave 11.3, Statistics 2.0.0 and Datatypes 1.5.0. Qt and a display are required for 3D PNG. Bayesian tuning, parallel execution, other classifiers, legacy TRACE, FIG and web palette output remain MATLAB-only. Octave classifier archives are runtime-specific. 3D CLOISTER boundaries and CSV triangle-face exports, including a geometry manifest and explicit empty footprints. Correctness and Reporting Recompute preprocessing after pruning algorithms that are never good, reset SIFTED caches between runs, retain inputs for feature-selection reruns, and apply fitted PLS centering during exploration. Recommendations depend on training policy and new features. Missing test outcomes are excluded from metrics. Oracle probability uses only rows with observed outcomes; selector recall avoids double counting. Classifier summaries distinguish CV from test metrics and probabilities from raw scores. Persist stage options and provenance for safe partial build/save/load/resume; reject incompatible partial rebuilds and replay training feature exclusions during exploration. Evaluate footprints against trained geometry for small or degenerate test batches. Preserve holes in 2D exports and record footprint acceptance and search termination. Replace obsolete toolkit-owned output files on export. Reproducible seeding immediately before SVM posterior calibration, complete numerical projection defaults, constant-column scaling and validation of seed bounds and input data. Upgrading and Limitations Rebuild experiments from the original metadata to obtain corrected results. Loading an existing model does not apply fixes to its fitted artifacts. Back up models and outputs first; rebuild CLOISTER for older 3D models and preprocessing before rerunning SIFTED if the saved feature-selection input is absent. Raw performance must be nonnegative (zero is allowed), and incomplete training data remaining after ingestion filtering is rejected. See Migrating a Legacy Model for pre-v0.9.0 layouts. Classifier CV summaries are conditional on the fitted preprocessing, feature selection, projection and tuning. They are not an outer-CV estimate of the full pipeline. Numerical PILOT restart selection can amplify floating-point differences across BLAS implementations and thread settings, so a seed alone does not promise identical projections across environments. Projection orientation is not canonicalized automatically. opts.pilot.topoWeight remains reserved and disabled. Documentation Reference documentation is available in the MATLAB Help browser and on the web, with examples for every function. The version header, release notes and citation metadata now identify v0.9.2. v0.9.1 Engineering release; results of the pipeline are unchanged. build and explore accept an 'onStage' callback, called after each stage. obj.plot('boundary') and distribution_boundary.png show the CLOISTER boundary of a 2D space. INIT reads the metadata for both training and evaluation, and PRELIM gained an evaluation mode, so evaluation now breaks ties exactly as training does. opts.general.seed now also controls PILOT and SIFTED . InstanceSpace checks that each stage's inputs exist before running it. Continuous integration and a matlab.unittest test suite. v0.9.0 Complete refactor of the toolbox. The InstanceSpace class is the main interface; buildIS and exploreIS remain as w"},{"t":"buildIS","u":"buildIS.html","p":"Build an instance space from a folder (backward-compatible wrapper)","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Wrapper around InstanceSpace","See Also"],"x":"Syntax model = buildIS(rootdir) Description model = buildIS(rootdir) builds an instance space from rootdir/metadata.csv and rootdir/options.json , writes model.mat and the output files to rootdir , prints EOF:SUCCESS , and returns the trained model. It is equivalent to Copy buildIS keeps the calling convention of versions before v0.9.0, which the MATILDA web platform and other scripts use. For new code, use InstanceSpace . Examples Expand all Build from a folder Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains metadata.csv and, optionally, options.json . Output Arguments Expand all model — Trained model structure The model property of the built InstanceSpace object. Version History v0.9.0 — Wrapper around InstanceSpace buildIS became a wrapper around InstanceSpace . Its inputs, outputs and files are unchanged. See Also exploreIS | InstanceSpace"},{"t":"exploreIS","u":"exploreIS.html","p":"Evaluate a saved instance space on new instances (backward-compatible wrapper)","h":["Syntax","Description","Examples","Input Arguments","Output Arguments","Version History","v0.9.0 — Wrapper around InstanceSpace","See Also"],"x":"Syntax out = exploreIS(rootdir) Description out = exploreIS(rootdir) loads rootdir/model.mat , evaluates rootdir/metadata_test.csv with it, writes the output files to rootdir , prints EOF:SUCCESS , and returns the result. It is equivalent to Copy A model.mat from a version before v0.9.0 is migrated in memory by ISAmigrateModel . For new code, use InstanceSpace . Examples Expand all Evaluate test instances after buildIS Copy Input Arguments Expand all rootdir — Data folder character vector Folder that contains model.mat and metadata_test.csv . Output Arguments Expand all out — Evaluation result structure Same fields as a trained model, computed for the test instances. Version History v0.9.0 — Wrapper around InstanceSpace exploreIS became a wrapper around InstanceSpace . Its inputs, outputs and files are unchanged. See Also buildIS | InstanceSpace | ISAmigrateModel"},{"t":"Instance Space Analysis Toolbox","u":"index.html","p":"Objective algorithm testing through the instance space of a problem","h":["Get Started","Getting Started","Stage-by-Stage Walkthrough","Metadata File Format","Options Reference","Functions and Classes","InstanceSpace","Pipeline Stages","Utilities","Output","Topics","Installation","References"],"x":"The Instance Space Analysis (ISA) Toolbox relates the structural features of problem instances to the performance of a portfolio of algorithms. It projects every instance into a 2D or 3D instance space , shows where each algorithm performs well (its footprint ), predicts which algorithm to use for a new instance, and estimates where in the space no instances have been observed yet. The toolbox implements the methodology in Smith-Miles & Muñoz (2023), including the 3D extension (ISA3) of Simpson et al. (2025). It runs in MATLAB R2025a or later and powers the MATILDA web platform. The ISA pipeline: INIT and PRELIM prepare the metadata, SIFTED selects features, PILOT projects them, then CLOISTER, PYTHIA and TRACE work in the projected space. INIT read metadata PRELIM label & normalise SIFTED select features PILOT project to 2D/3D CLOISTER empirical bounds PYTHIA predict algorithms TRACE footprints Outputs CSV · PNG · .mat The ISA pipeline. InstanceSpace runs the stages in this order; each stage is also a standalone function. Get Started Getting Started Build and explore an instance space from the bundled reference data in a few lines. Stage-by-Stage Walkthrough Run each stage separately and inspect what it produces. Metadata File Format Prepare metadata.csv for your own problem domain. Options Reference Every field of opts and options.json , with defaults. Functions and Classes InstanceSpace Build, explore, save, load and plot an instance space. The main entry point. Pipeline Stages INIT, PRELIM, FILTER, SIFTED, PILOT, PILOTviewpoint, CLOISTER, PYTHIA, TRACE. Utilities Option defaults and validation, classifier registry, legacy-model migration. Output Write CSV files and figures, and restore 3D viewpoints. Topics Migrating a Legacy Model — use a model.mat from a version before v0.9.0. Backward-Compatible Wrappers — buildIS and exploreIS , used by MATILDA. Deprecated Functions — PYTHIA2 , PYTHIAtest , SIFTED2 . What's New — release notes. Installation Clone or download the repository, then add it to the MATLAB path from its root folder: Copy Required products: MATLAB R2025a or later with the Statistics and Machine Learning, Optimization, Global Optimization, Parallel Computing, and Financial toolboxes. A bounded serial GNU Octave workflow is also supported; see What's New for supported classifiers, outputs and persistence limits. With the repository root on the path, this documentation also appears in the MATLAB Help browser under Supplemental Software . References Smith-Miles, K. & Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. ACM Computing Surveys , 55(12), Article 255. https://doi.org/10.1145/3572895 Simpson, C., Muñoz, M.A., Kandanaarachchi, S. & Campello, R.J.G.B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning , 114, 240. https://doi.org/10.1007/s10994-025-06871-5"},{"t":"scriptcsv","u":"scriptcsv.html","p":"Write the results of an instance space to CSV files","h":["Syntax","Description","Examples","Input Arguments","Version History","v0.9.2 — Footprint boundaries and geometry exports","v0.9.1 — Multiple footprint regions","See Also"],"x":"Syntax scriptcsv(container,rootdir) Description scriptcsv(container,rootdir) writes the coordinates, data, predictions, footprints and summary tables of a trained model or an evaluation result to CSV files in rootdir . Coordinate columns are z_1 , z_2 and, for a 3D space, z_3 . InstanceSpace.build and InstanceSpace.explore call scriptcsv when opts.outputs.csv is true . File Contents coordinates.csv instance coordinates Z bounds.csv , bounds_prunned.csv CLOISTER boundary Zedge and Zecorr (training only) feature_raw.csv , feature_process.csv selected features, before and after preprocessing algorithm_raw.csv , algorithm_process.csv performance, before and after preprocessing algorithm_bin.csv good-performance labels Ybin good_algos.csv , beta_easy.csv , portfolio.csv number of good algorithms, beta-easy flag, best algorithm algorithm_svm.csv , portfolio_svm.csv PYTHIA predictions and selected algorithm footprint__good.csv , footprint__best.csv footprint boundary vertices; separate regions and holes are separated by a row of NaN footprint_performance.csv TRACE summary table classifier_table.csv PYTHIA summary table projection_matrix.csv PILOT projection matrix (training only) Examples Expand all Write the CSV files of an evaluation to another folder Copy Input Arguments Expand all container — Model or evaluation result structure obj.model or an element of obj.testResults . rootdir — Output folder character vector Must exist and end with a file separator. Version History v0.9.2 — Footprint boundaries and geometry exports Geometry exports include empty vertex tables when a footprint is empty. For 3D, each vertex table has a corresponding _faces.csv with one-based triangle indices. CLOISTER uses bounds_faces.csv and bounds_prunned_faces.csv . geometry_manifest.json records dimensionality, footprint filenames, and empty status. Two-dimensional rings are NaN-separated, including holes. The exporter replaces the toolkit-owned footprint_*_best.csv , footprint_*_good.csv , their face tables, and boundary CSVs on each run. Keep user-authored files outside these reserved names. A footprint region with a hole writes both its outer boundary and the hole's boundary, separated by a NaN row. v0.9.1 — Multiple footprint regions Every region of a footprint is written, separated by NaN rows, instead of only the first. See Also scriptpng | scriptweb | InstanceSpace"},{"t":"scriptpng","u":"scriptpng.html","p":"Save figures of an instance space as PNG files","h":["Syntax","Description","Examples","Input Arguments","GNU Octave","Version History","v0.9.2 — Plot export updates and Octave support","v0.9.1 — Boundary figure","v0.9.0 — 3D figures","See Also"],"x":"Syntax scriptpng(container,rootdir) Description scriptpng(container,rootdir) draws the standard views of a trained model or an evaluation result and saves them in rootdir : each feature, each algorithm's performance, good/bad labels, predictions, footprints, the algorithm portfolio, the number of good algorithms, the beta score, instance sources, and the CLOISTER boundary. For a 3D space the figures use the viewpoints found by PILOTviewpoint . Each footprint is also saved as a MATLAB figure ( footprint_.fig , footprint_portfolio.fig ) for interactive rotation, unless opts.outputs.fig is false . ISArecallView returns a rotated figure to its stored viewpoint. InstanceSpace.build and InstanceSpace.explore call scriptpng when opts.outputs.png is true . File Figure distribution_feature_.png feature value over the space distribution_performance_global_normalized_.png , ..._individual_normalized_.png performance, scaled over all algorithms or per algorithm binary_performance_.png , binary_classifier_.png good/bad labels, and PYTHIA's predictions footprint_.png , footprint_portfolio.png footprints distribution_portfolio.png , distribution_svm_portfolio.png best and selected algorithm distribution_number_good_algos.png , distribution_beta_score.png number of good algorithms, beta-easy instances distribution_sources.png instance sources, if the metadata has a source column distribution_boundary.png CLOISTER boundary (training only) Examples Expand all Save figures at a larger font size Copy Input Arguments Expand all container — Model or evaluation result structure obj.model or an element of obj.testResults . rootdir — Output folder character vector Must exist and end with a file separator. GNU Octave Octave exports PNG through print and uses viridis. Qt is selected when available and is required for 3D export; use a virtual display for headless execution. MATLAB retains exportgraphics and parula. FIG output is MATLAB-only. Version History v0.9.2 — Plot export updates and Octave support Added Octave PNG export through Qt, with viridis colours and MATLAB-only FIG output. Direct Octave calls requesting 3D FIG output fail before removing existing output files. Each plot export removes prior toolkit-owned plot filenames before drawing the current run. This clears plots for removed algorithms and features, absent sources or bounds, and old 3D FIG files. Keep user-authored files outside the toolkit's distribution_* , binary_* , and footprint_* output names. distribution_boundary.png is also drawn for a 3D space. v0.9.1 — Boundary figure Adds distribution_boundary.png . v0.9.0 — 3D figures 3D spaces are drawn from their optimised viewpoints and saved as .fig files too. See Also scriptcsv | ISArecallView | PILOTviewpoint | InstanceSpace"},{"t":"scriptweb","u":"scriptweb.html","p":"Write colour-scaled data for the MATILDA web platform","h":["Syntax","Description","Examples","Input Arguments","See Also"],"x":"Syntax scriptweb(container,rootdir) Description scriptweb(container,rootdir) writes the features, performance and number of good algorithms as colour indices, with the colour table, to CSV files in rootdir . The MATILDA web platform uses these files to draw its interactive plots. You do not need them for local work. InstanceSpace.build and InstanceSpace.explore call scriptweb when both opts.outputs.csv and opts.outputs.web are true . File Contents color_table.csv the colour map feature_raw_color.csv , feature_process_color.csv feature colour indices algorithm_raw_color.csv , algorithm_process_color.csv performance colour indices, scaled over all algorithms algorithm_raw_single_color.csv , algorithm_process_single_color.csv performance colour indices, scaled per algorithm good_algos_color.csv colour index of the number of good algorithms Examples Expand all Turn on the web output Copy Input Arguments Expand all container — Model or evaluation result structure rootdir — Output folder character vector Must exist and end with a file separator. See Also scriptcsv | InstanceSpace"}]; diff --git a/doc/html/buildIS.html b/doc/html/buildIS.html index 0a0a2d4..1ef6015 100644 --- a/doc/html/buildIS.html +++ b/doc/html/buildIS.html @@ -49,6 +49,7 @@
                                                    • Utilities
                                                      • ISAdefaults
                                                      • ISAvalidateOpts
                                                        • +
                                                      • ISAfootprintContains
                                                      • ISAgetClassifierFcn
                                                      • ISAmigrateModel
                                                      • ISAsubsetData
                                                        • diff --git a/doc/html/exploreIS.html b/doc/html/exploreIS.html index 616a504..2dd7bde 100644 --- a/doc/html/exploreIS.html +++ b/doc/html/exploreIS.html @@ -49,6 +49,7 @@
                                                      • Utilities
                                                        • ISAdefaults
                                                        • ISAvalidateOpts
                                                          • +
                                                        • ISAfootprintContains
                                                        • ISAgetClassifierFcn
                                                        • ISAmigrateModel
                                                        • ISAsubsetData
                                                          • diff --git a/doc/html/helptoc.xml b/doc/html/helptoc.xml index 1af9f05..8e650a6 100644 --- a/doc/html/helptoc.xml +++ b/doc/html/helptoc.xml @@ -28,6 +28,7 @@ Utilities ISAdefaults ISAvalidateOpts + ISAfootprintContains ISAgetClassifierFcn ISAmigrateModel ISAsubsetData diff --git a/doc/html/index.html b/doc/html/index.html index 79ba227..d872431 100644 --- a/doc/html/index.html +++ b/doc/html/index.html @@ -49,6 +49,7 @@
                                                        • Utilities
                                                          • ISAdefaults
                                                          • ISAvalidateOpts
                                                            • +
                                                          • ISAfootprintContains
                                                          • ISAgetClassifierFcn
                                                          • ISAmigrateModel
                                                          • ISAsubsetData
                                                            • diff --git a/doc/html/scriptcsv.html b/doc/html/scriptcsv.html index c4dcb86..0eecabf 100644 --- a/doc/html/scriptcsv.html +++ b/doc/html/scriptcsv.html @@ -49,6 +49,7 @@
                                                          • Utilities
                                                            • ISAdefaults
                                                            • ISAvalidateOpts
                                                              • +
                                                            • ISAfootprintContains
                                                            • ISAgetClassifierFcn
                                                            • ISAmigrateModel
                                                            • ISAsubsetData
                                                              • diff --git a/doc/html/scriptpng.html b/doc/html/scriptpng.html index db801f0..1dfabb4 100644 --- a/doc/html/scriptpng.html +++ b/doc/html/scriptpng.html @@ -49,6 +49,7 @@
                                                            • Utilities
                                                              • ISAdefaults
                                                              • ISAvalidateOpts
                                                                • +
                                                              • ISAfootprintContains
                                                              • ISAgetClassifierFcn
                                                              • ISAmigrateModel
                                                              • ISAsubsetData
                                                                • diff --git a/doc/html/scriptweb.html b/doc/html/scriptweb.html index 89c747a..18395f7 100644 --- a/doc/html/scriptweb.html +++ b/doc/html/scriptweb.html @@ -49,6 +49,7 @@
                                                              • Utilities
                                                                • ISAdefaults
                                                                • ISAvalidateOpts
                                                                  • +
                                                                • ISAfootprintContains
                                                                • ISAgetClassifierFcn
                                                                • ISAmigrateModel
                                                                • ISAsubsetData
                                                                  • diff --git a/doc/src/ISAfootprintContains.md b/doc/src/ISAfootprintContains.md new file mode 100644 index 0000000..609760c --- /dev/null +++ b/doc/src/ISAfootprintContains.md @@ -0,0 +1,79 @@ +# ISAfootprintContains + +Test footprint membership with an explicit boundary distance + +## Syntax + +``` +inside = ISAfootprintContains(geometry,queries) +inside = ISAfootprintContains(geometry,queries,tolerance) +``` + +## Description + +`inside = ISAfootprintContains(geometry,queries)` tests whether each query point lies in a footprint, including its boundary. The default distance tolerance is zero. + +`inside = ISAfootprintContains(geometry,queries,tolerance)` also includes points within the specified Euclidean distance of a boundary edge in 2D or triangular face in 3D. Hole interiors farther from the boundary remain excluded. The geometry and its area or volume are unchanged. + +TRACE uses this helper for footprint metrics and evaluation. Set `opts.trace.boundaryTolerance` to configure the pipeline; TRACE saves the fitted value and reuses it during exploration. The distance is in projected-coordinate units, with no implicit epsilon multiplier, automatic scaling or dependence on other points in the query batch. The membership policy is shared with pyInstanceSpace; floating-point geometry can still differ between implementations. + +## Examples + +### Include the exact boundary of a square + +Run `startup` first when calling utilities directly from a fresh session. + +```matlab +startup +region = polyshape([0 0; 1 0; 1 1; 0 1]); +queries = [0.5 0.5; 0.5 0; 0.5 -0.0005; 0.5 -0.002]; +inside = ISAfootprintContains(region, queries) +% inside = [true; true; false; false] +``` + +### Choose an explicit distance tolerance + +```matlab +inside = ISAfootprintContains(region, queries, 0.001) +% inside = [true; true; true; false] +``` + +Choose a positive distance from the uncertainty budget of the projected coordinates. A tolerance is not a guarantee that legacy footprint trimming is invariant under arbitrary coordinate perturbations. + +## Input Arguments + +### `geometry` — Footprint geometry + +*2D `polyshape` | 2D or 3D `alphaShape`* + +The fitted region, normally a footprint's `polygon` field. Octave TRACE3 uses the toolkit's compatible alpha-shape class; MATLAB's `polyshape` and legacy TRACE remain MATLAB-only. An empty geometry produces false for every query. + +### `queries` — Points to test + +*numeric matrix* + +One row per point, with two or three columns matching the geometry's dimensionality. Nonfinite query points are outside. + +### `tolerance` — Boundary distance + +*`0` (default) | finite nonnegative real scalar* + +Euclidean distance in projection units. Positive values include points at distance less than or equal to this value from a boundary, including hole boundaries. + +## Output Arguments + +### `inside` — Membership mask + +*logical column vector* + +One value per query row. Testing a point alone or alongside other queries uses the same rule. + +## Version History + +### v0.9.2 — Introduced + +Closed-boundary membership with explicit distance tolerance for TRACE construction, legacy contradiction handling and evaluation. + +## See Also + +`TRACE` | `ISAdefaults` | [Options Reference](OptionsReference.html) diff --git a/doc/src/TRACE.md b/doc/src/TRACE.md index ac78c5e..6195274 100644 --- a/doc/src/TRACE.md +++ b/doc/src/TRACE.md @@ -121,7 +121,7 @@ including boundaries of holes. Hole interiors farther away remain excluded. Zero preserves exact closed-boundary membership. There is no automatic epsilon multiplier, coordinate scaling or dependence on the query batch. -The same rule is used for construction metrics, legacy overlap counts and +`ISAfootprintContains` applies the same rule for construction metrics, legacy overlap counts and trimming, and test-set evaluation. Geometry and area/volume are not expanded. The fitted value is saved as `out.boundaryTolerance`; evaluation reuses it even if a different option is supplied. Older models without this field use zero. diff --git a/docs/release-0.9.2.md b/docs/release-0.9.2.md index ea2b79b..ca91e1b 100644 --- a/docs/release-0.9.2.md +++ b/docs/release-0.9.2.md @@ -59,8 +59,8 @@ configuration failure as a supported-runtime test pass. Local checks completed: -- Generated HTML matches all sources (36 generated files); link validation - checks 31 pages with zero problems. +- Generated HTML matches all sources (37 generated files); link validation + checks 32 pages with zero problems. - Octave 11.3.0 with the pinned packages passes the portable preprocessing, analytic/numerical/PLS projection, KNN/CV/tuning, geometry, workflow edge, archive and property contracts. Both 2D and 3D full build/save/load/explore