diff --git a/.github/workflows/pypi-test.yml b/.github/workflows/pypi-test.yml
index b8edabe4..2c97a4ef 100644
--- a/.github/workflows/pypi-test.yml
+++ b/.github/workflows/pypi-test.yml
@@ -69,10 +69,6 @@ jobs:
working-directory: easyscience
run: pixi run python -m pytest ../tests/unit/ --color=yes -v
- - name: Run functional tests to verify the installation
- working-directory: easyscience
- run: pixi run python -m pytest ../tests/functional/ --color=yes -v
-
- name: Run integration tests to verify the installation
working-directory: easyscience
run: pixi run python -m pytest ../tests/integration/ --color=yes -n auto
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 776ed94f..b04d87bd 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -102,21 +102,6 @@ jobs:
pixi run --environment $env unit-tests
done
- - name: Run functional tests
- shell: bash
- run: |
- set -euo pipefail
-
- for py_ver in $PY_VERSIONS; do
- echo
- echo "πΉπΈπΉπΈπΉ Python: $py_ver πΉπΈπΉπΈπΉ"
-
- env="py-$(echo $py_ver | tr -d .)-env" # Converts 3.XX -> py-3XX-env
-
- echo "Running tests in environment: $env"
- pixi run --environment $env functional-tests
- done
-
- name: Run integration tests ${{ needs.env-prepare.outputs.pytest-marks }}
shell: bash
run: |
@@ -258,25 +243,6 @@ jobs:
cd ..
done
- - name: Run functional tests
- shell: bash
- run: |
- set -euo pipefail
-
- for py_ver in $PY_VERSIONS; do
- echo
- echo "πΉπΈπΉπΈπΉ Python: $py_ver πΉπΈπΉπΈπΉ"
-
- echo "Entering pixi project directory easyscience_py$py_ver"
- cd easyscience_py$py_ver
-
- echo "Running tests"
- pixi run python -m pytest ../tests/functional/ --color=yes -v
-
- echo "Exiting pixi project directory"
- cd ..
- done
-
- name: Run integration tests ${{ needs.env-prepare.outputs.pytest-marks }}
shell: bash
run: |
diff --git a/docs/docs/api-reference/index.md b/docs/docs/api-reference/index.md
index 4bd6f561..b6767ccf 100644
--- a/docs/docs/api-reference/index.md
+++ b/docs/docs/api-reference/index.md
@@ -8,19 +8,16 @@ This section contains the reference detailing the functions and modules
available in EasyScience.
- [base_classes](base_classes.md) β Core abstract and helper base
- classes used to build EasyScience objects (e.g. `ObjBase`,
- `ModelBase`).
-- [fitting](fitting.md) β Fitting utilities and interfaces, including
- `Fitter` and available minimizers.
+ classes used to build EasyScience objects (e.g. `EasyBase`,
+ `ModelBase`, `EasyList`, `SerializerBase`).
+- [fitting](fitting.md) β Fitting and sampling, including `Fitter`,
+ `Sampler` and the available minimizers.
- [global_object](global_object.md) β Global singleton providing shared
- services (logger, map, undo/redo stack, script manager).
-- [io](io.md) β Serialization and deserialization framework for
- persisting EasyScience objects (serializers and components).
+ services (logger and object registry).
- [job](job.md) β Job and experiment abstractions for running and
organizing analyses.
-- [models](models.md) β Predefined model implementations (e.g.
- polynomial models) used in fitting workflows.
- [utils](utils.md) β Miscellaneous utility functions and helpers (class
tools, decorators, type helpers).
- [variable](variable.md) β Descriptor types and variable abstractions
- (e.g. `DescriptorNumber`, `Parameter`, `DescriptorArray`).
+ (e.g. `DescriptorNumber`, `Parameter`, `DescriptorArray`). All of them
+ are `EasyBase` objects, serialized with `to_dict`/`from_dict`.
diff --git a/docs/docs/api-reference/io.md b/docs/docs/api-reference/io.md
deleted file mode 100644
index f06a9356..00000000
--- a/docs/docs/api-reference/io.md
+++ /dev/null
@@ -1 +0,0 @@
-::: easyscience.io
diff --git a/docs/docs/api-reference/models.md b/docs/docs/api-reference/models.md
deleted file mode 100644
index 2471f604..00000000
--- a/docs/docs/api-reference/models.md
+++ /dev/null
@@ -1 +0,0 @@
-::: easyscience.models
diff --git a/docs/docs/tutorials/fitting-bayesian.ipynb b/docs/docs/tutorials/fitting-bayesian.ipynb
index 302800c2..a2931634 100644
--- a/docs/docs/tutorials/fitting-bayesian.ipynb
+++ b/docs/docs/tutorials/fitting-bayesian.ipynb
@@ -28,7 +28,7 @@
"\n",
"where $\\theta$ are the model parameters, $d$ is the observed data, $p(d \\mid \\theta)$ is the likelihood, and $p(\\theta)$ is the prior. In `easyscience`, the `min`/`max` bounds of a `Parameter` are interpreted as a **uniform prior**, and a Gaussian likelihood is constructed from the data and supplied weights.\n",
"\n",
- "`easyscience` exposes a Bayesian Markov-chain Monte Carlo (MCMC) sampler through the `Sampler` class. Under the hood this uses BUMPS' DREAM sampler, so the underlying minimizer must be switched to BUMPS.\n",
+ "`easyscience` exposes a Bayesian Markov-chain Monte Carlo (MCMC) sampler through the `Sampler` class.\n",
"\n",
"```{note}\n",
"This tutorial focuses on Bayesian analysis with a simple QENS model for illustration. For dedicated QENS fitting with more sophisticated models, consider using [`EasyDynamics`](https://github.com/easyscience/easydynamics).\n",
@@ -138,12 +138,12 @@
},
{
"cell_type": "markdown",
- "id": "9",
+ "id": "7",
"metadata": {},
"source": [
"## Defining parameters with priors\n",
"\n",
- "Create four `Parameter` objects, for the area $A$, $\\gamma$, $\\omega_0$ and $\\sigma$. The `min` and `max` arguments define a **uniform prior** on each parameter — the sampler will only consider values inside this range and will treat every value inside the range as equally plausible *a priori*.\n",
+ "Create four `Parameter` objects, for the area $A$, $\\gamma$, $\\omega_0$ and $\\sigma$. The `min` and `max` arguments define a **uniform prior** on each parameter — the sampler will only consider values inside this range and will treat every value inside the range as equally plausible *a priori*. The parameters are then collected in an `EasyList` container: this is the model object that both `Fitter` and `Sampler` take, so it is defined here, before the optional fit.\n",
"\n",
"| Parameter | Initial Value | Min | Max |\n",
"| --- | --- | --- | --- |\n",
@@ -156,21 +156,24 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "10",
+ "id": "8",
"metadata": {},
"outputs": [],
"source": [
+ "from easyscience import EasyList\n",
"from easyscience import Parameter\n",
"\n",
- "area = Parameter(name='area', value=10, fixed=False, min=1, max=100)\n",
- "gamma = Parameter(name='gamma', value=8e-3, fixed=False, min=1e-4, max=1e-2)\n",
- "omega_0 = Parameter(name='omega_0', value=1e-3, fixed=False, min=0, max=2e-3)\n",
- "sigma = Parameter(name='sigma', value=1e-3, fixed=False, min=1e-5, max=1e-1)"
+ "area = Parameter(display_name='area', value=10, fixed=False, min=1, max=100)\n",
+ "gamma = Parameter(display_name='gamma', value=8e-3, fixed=False, min=1e-4, max=1e-2)\n",
+ "omega_0 = Parameter(display_name='omega_0', value=1e-3, fixed=False, min=0, max=2e-3)\n",
+ "sigma = Parameter(display_name='sigma', value=1e-3, fixed=False, min=1e-5, max=1e-1)\n",
+ "\n",
+ "parameter_container = EasyList(area, gamma, omega_0, sigma)"
]
},
{
"cell_type": "markdown",
- "id": "77ce3f63",
+ "id": "9",
"metadata": {},
"source": [
"## The model\n",
@@ -193,7 +196,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "36a0c4f4",
+ "id": "10",
"metadata": {},
"outputs": [],
"source": [
@@ -217,9 +220,7 @@
"source": [
"## Maximum-likelihood fit (optional, but recommended)\n",
"\n",
- "Perform a quick maximum-likelihood fit. This is **not** a prerequisite for sampling: `Sampler`\n",
- "needs a *configured* `Fitter`, not a *fitted* one, and you can sample straight from the initial\n",
- "parameter values.\n",
+ "Perform a quick maximum-likelihood fit.\n",
"\n",
"It is worth doing anyway, for two reasons:\n",
"\n",
@@ -239,9 +240,6 @@
"outputs": [],
"source": [
"from easyscience import Fitter\n",
- "from easyscience import ObjBase\n",
- "\n",
- "parameter_container = ObjBase(name='params', A=area, gamma=gamma, omega_0=omega_0, sigma=sigma)\n",
"\n",
"mle_fitter = Fitter(parameter_container, intensity_model)\n",
"mle_result = mle_fitter.fit(x=omega, y=intensity_obs, weights=1 / intensity_error)\n",
@@ -261,7 +259,7 @@
"\n",
"We now draw samples from the posterior distribution $p(\\theta \\mid d)$ using the BUMPS DREAM (DiffeRential Evolution Adaptive Metropolis) algorithm. DREAM is an ensemble MCMC method that runs multiple chains in parallel and automatically tunes the proposal distribution.\n",
"\n",
- "DREAM only works with the BUMPS minimizer. We reuse the ``mle_fitter`` created above β any configured `Fitter` would do, and it does not have to have been fitted β switch it to BUMPS, and create a `Sampler` instance bound to the fitter and data. Calling `sampler.sample()` returns a `SamplingResults` object with the following attributes:\n",
+ "Create a `Sampler` from the same `parameter_container` and `intensity_model` we gave the `Fitter`, bound to the data. Calling `sampler.sample()` returns a `SamplingResults` object with the following attributes:\n",
"\n",
"- `draws`: a `(n_samples, n_parameters)` array of posterior samples: each **row** is one complete draw from the joint posterior (one value for every parameter simultaneously), and each **column** holds all sampled values for a single parameter. Note this is a *trimmed* view of the chain rather than the raw buffer, so `n_samples` is smaller than `samples / thin` — see the note under [Extend the chain](#extend-the-chain-and-check-convergence);\n",
"- `param_names`: the unique names of the parameters, in the same column order as `draws`;\n",
@@ -274,7 +272,9 @@
"- `burn` (500): the number of initial *burn-in* generations to discard — the sampler needs time to find the typical set of the posterior, and early samples are not representative. Note this counts generations, not raw samples, so `burn=500` discards `500 Γ n_chains` raw samples;\n",
"- `thin` (2): the *thinning* interval — only every second generation is kept, which reduces autocorrelation between consecutive draws;\n",
"\n",
- "First, we switch to the BUMPS minimizer:"
+ "```{note}\n",
+ "If you already have a `Fitter`, `Sampler.from_fitter(mle_fitter, omega, intensity_obs, weights=1 / intensity_error)` builds the same sampler from the model bound to it. The fitter is neither needed afterwards nor modified.\n",
+ "```"
]
},
{
@@ -283,22 +283,12 @@
"id": "14",
"metadata": {},
"outputs": [],
- "source": [
- "from easyscience import AvailableMinimizers\n",
- "\n",
- "mle_fitter.switch_minimizer(AvailableMinimizers.Bumps)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "5a219fcd",
- "metadata": {},
- "outputs": [],
"source": [
"from easyscience.fitting import Sampler\n",
"\n",
- "sampler = Sampler(mle_fitter, omega, intensity_obs, weights=1 / intensity_error)\n",
+ "sampler = Sampler(\n",
+ " parameter_container, intensity_model, omega, intensity_obs, weights=1 / intensity_error\n",
+ ")\n",
"results = sampler.sample(samples=10000, burn=500, thin=2)\n",
"\n",
"print(f'Drew {results.draws.shape[0]} samples for {results.draws.shape[1]} parameters.')\n",
@@ -307,7 +297,7 @@
},
{
"cell_type": "markdown",
- "id": "8766b170",
+ "id": "15",
"metadata": {},
"source": [
"## Convergence diagnostics\n",
@@ -323,7 +313,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "3c49ab6f",
+ "id": "16",
"metadata": {},
"outputs": [],
"source": [
@@ -359,7 +349,7 @@
},
{
"cell_type": "markdown",
- "id": "15",
+ "id": "17",
"metadata": {},
"source": [
"## Posterior summaries\n",
@@ -377,7 +367,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "ce3e38a8",
+ "id": "18",
"metadata": {},
"outputs": [],
"source": []
@@ -385,7 +375,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "16",
+ "id": "19",
"metadata": {},
"outputs": [],
"source": [
@@ -407,7 +397,7 @@
},
{
"cell_type": "markdown",
- "id": "17",
+ "id": "20",
"metadata": {},
"source": [
"## Visualise the joint posterior\n",
@@ -418,7 +408,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "18",
+ "id": "21",
"metadata": {},
"outputs": [],
"source": [
@@ -452,7 +442,7 @@
},
{
"cell_type": "markdown",
- "id": "19",
+ "id": "22",
"metadata": {},
"source": [
"## Posterior-predictive band\n",
@@ -463,7 +453,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "20",
+ "id": "23",
"metadata": {},
"outputs": [],
"source": [
@@ -499,7 +489,7 @@
},
{
"cell_type": "markdown",
- "id": "03339658",
+ "id": "24",
"metadata": {},
"source": [
"## Extend the chain and check convergence\n",
@@ -543,7 +533,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "293b140b",
+ "id": "25",
"metadata": {},
"outputs": [],
"source": [
@@ -573,7 +563,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "9ec4302c",
+ "id": "26",
"metadata": {},
"outputs": [],
"source": [
@@ -614,7 +604,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "b0f30be6",
+ "id": "27",
"metadata": {},
"outputs": [],
"source": [
@@ -641,7 +631,7 @@
},
{
"cell_type": "markdown",
- "id": "50b7213a",
+ "id": "28",
"metadata": {},
"source": [
"### What is Gelman-Rubin R-hat?\n",
@@ -675,7 +665,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "3449e0a7",
+ "id": "29",
"metadata": {},
"outputs": [],
"source": [
diff --git a/docs/docs/tutorials/fitting-qens.ipynb b/docs/docs/tutorials/fitting-qens.ipynb
index a4929a5c..3809461e 100644
--- a/docs/docs/tutorials/fitting-qens.ipynb
+++ b/docs/docs/tutorials/fitting-qens.ipynb
@@ -246,10 +246,10 @@
"source": [
"from easyscience import Parameter\n",
"\n",
- "A = Parameter(name='A', value=10, fixed=False, min=0.01, max=100)\n",
- "gamma = Parameter(name='gamma', value=8e-3, fixed=False, min=1e-4, max=10e-3)\n",
- "omega_0 = Parameter(name='omega_0', value=0.001, fixed=False, min=0, max=0.002)\n",
- "sigma = Parameter(name='sigma', value=0.001, fixed=False, min=1e-5, max=0.1)"
+ "A = Parameter(display_name='A', value=10, fixed=False, min=0.01, max=100)\n",
+ "gamma = Parameter(display_name='gamma', value=8e-3, fixed=False, min=1e-4, max=10e-3)\n",
+ "omega_0 = Parameter(display_name='omega_0', value=0.001, fixed=False, min=0, max=0.002)\n",
+ "sigma = Parameter(display_name='sigma', value=0.001, fixed=False, min=1e-5, max=0.1)"
]
},
{
@@ -293,10 +293,10 @@
"metadata": {},
"outputs": [],
"source": [
+ "from easyscience import EasyList\n",
"from easyscience import Fitter\n",
- "from easyscience import ObjBase\n",
"\n",
- "params = ObjBase(name='params', A=A, gamma=gamma, omega_0=omega_0, sigma=sigma)\n",
+ "params = EasyList(A, gamma, omega_0, sigma)\n",
"f = Fitter(params, intensity)\n",
"\n",
"res = f.fit(x=q, y=i, weights=1 / di)"
diff --git a/docs/docs/tutorials/fitting-sans.ipynb b/docs/docs/tutorials/fitting-sans.ipynb
index 06b0cc36..84c3a1cd 100644
--- a/docs/docs/tutorials/fitting-sans.ipynb
+++ b/docs/docs/tutorials/fitting-sans.ipynb
@@ -236,10 +236,10 @@
"source": [
"from easyscience import Parameter\n",
"\n",
- "scale = Parameter(name='scale', value=1.4e-7, fixed=True)\n",
- "delta_rho = Parameter(name='delta_rho', value=3, fixed=False, min=0, max=10)\n",
- "r = Parameter(name='r', value=80, fixed=False, min=0, max=1000)\n",
- "bkg = Parameter(name='bkg', value=0.01, fixed=False, min=0.001, max=0.1)"
+ "scale = Parameter(display_name='scale', value=1.4e-7, fixed=True)\n",
+ "delta_rho = Parameter(display_name='delta_rho', value=3, fixed=False, min=0, max=10)\n",
+ "r = Parameter(display_name='r', value=80, fixed=False, min=0, max=1000)\n",
+ "bkg = Parameter(display_name='bkg', value=0.01, fixed=False, min=0.001, max=0.1)"
]
},
{
@@ -283,10 +283,10 @@
"metadata": {},
"outputs": [],
"source": [
+ "from easyscience import EasyList\n",
"from easyscience import Fitter\n",
- "from easyscience import ObjBase\n",
"\n",
- "params = ObjBase(name='params', delta_rho=delta_rho, r=r, bkg=bkg)\n",
+ "params = EasyList(delta_rho, r, bkg)\n",
"f = Fitter(params, sphere)\n",
"\n",
"res = f.fit(x=q, y=i, weights=1 / di)"
diff --git a/docs/docs/tutorials/progress-callback.ipynb b/docs/docs/tutorials/progress-callback.ipynb
index a5625a5a..f782f1b9 100644
--- a/docs/docs/tutorials/progress-callback.ipynb
+++ b/docs/docs/tutorials/progress-callback.ipynb
@@ -47,9 +47,10 @@
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"\n",
- "from easyscience import AvailableMinimizers\n",
+ "from easyscience import EasyList\n",
"from easyscience import Fitter\n",
- "from easyscience import ObjBase\n",
+ "from easyscience import Minimizers\n",
+ "from easyscience import ModelBase\n",
"from easyscience import Parameter"
]
},
@@ -82,32 +83,23 @@
},
"outputs": [],
"source": [
- "class MultiPeak(ObjBase):\n",
- " a0: Parameter\n",
- " a1: Parameter\n",
- " a2: Parameter\n",
- " A1: Parameter\n",
- " mu1: Parameter\n",
- " sigma1: Parameter\n",
- " A2: Parameter\n",
- " mu2: Parameter\n",
- " sigma2: Parameter\n",
- " A3: Parameter\n",
- " mu3: Parameter\n",
- " sigma3: Parameter\n",
- "\n",
+ "class MultiPeak(ModelBase):\n",
" def __init__(self, **values):\n",
- " params = {name: Parameter(name, value) for name, value in values.items()}\n",
- " super().__init__('multipeak', **params)\n",
+ " super().__init__(display_name='multipeak')\n",
+ " self._parameters = EasyList(\n",
+ " *(Parameter(value, display_name=name) for name, value in values.items())\n",
+ " )\n",
+ "\n",
+ " @property\n",
+ " def parameters(self) -> EasyList:\n",
+ " return self._parameters\n",
"\n",
" def __call__(self, x):\n",
- " bg = self.a0.value + self.a1.value * x + self.a2.value * x**2\n",
+ " p = {par.display_name: par.value for par in self.parameters}\n",
+ " bg = p['a0'] + p['a1'] * x + p['a2'] * x**2\n",
" peaks = 0.0\n",
- " for A, mu, sigma in (\n",
- " (self.A1.value, self.mu1.value, self.sigma1.value),\n",
- " (self.A2.value, self.mu2.value, self.sigma2.value),\n",
- " (self.A3.value, self.mu3.value, self.sigma3.value),\n",
- " ):\n",
+ " for i in (1, 2, 3):\n",
+ " A, mu, sigma = p[f'A{i}'], p[f'mu{i}'], p[f'sigma{i}']\n",
" peaks = peaks + A * np.exp(-0.5 * ((x - mu) / sigma) ** 2)\n",
" return bg + peaks"
]
@@ -205,7 +197,7 @@
" mu3=3.0,\n",
" sigma3=1.2,\n",
" )\n",
- " for p in model.get_parameters():\n",
+ " for p in model.get_all_parameters():\n",
" p.fixed = False\n",
" return model"
]
@@ -272,7 +264,7 @@
},
"outputs": [],
"source": [
- "def fit_with(backend: AvailableMinimizers):\n",
+ "def fit_with(backend: Minimizers):\n",
" model = make_initial_model()\n",
" fitter = Fitter(model, model)\n",
" fitter.switch_minimizer(backend)\n",
@@ -285,7 +277,7 @@
"\n",
"results = {}\n",
"histories = {}\n",
- "for backend in (AvailableMinimizers.LMFit, AvailableMinimizers.Bumps, AvailableMinimizers.DFO):\n",
+ "for backend in (Minimizers.LMFit, Minimizers.Bumps, Minimizers.DFO):\n",
" res, hist = fit_with(backend)\n",
" results[backend.name] = res\n",
" histories[backend.name] = hist\n",
@@ -378,7 +370,7 @@
"else:\n",
" model = make_initial_model()\n",
" fitter = Fitter(model, model)\n",
- " fitter.switch_minimizer(AvailableMinimizers.LMFit)\n",
+ " fitter.switch_minimizer(Minimizers.LMFit)\n",
" fitter.max_evaluations = 108\n",
"\n",
" pbar = tqdm(total=fitter.max_evaluations, desc='LMFit', unit='eval')\n",
diff --git a/docs/docs/user-guide/controlling-log-output.md b/docs/docs/user-guide/controlling-log-output.md
index 0403590a..12899e1e 100644
--- a/docs/docs/user-guide/controlling-log-output.md
+++ b/docs/docs/user-guide/controlling-log-output.md
@@ -35,14 +35,13 @@ individual subsystems:
```
easyscience # root β controls everything
βββ easyscience.base_classes # EasyList runtime warnings
-βββ easyscience.legacy # ObjBase / CollectionBase deprecations
βββ easyscience.deprecated # @deprecated decorator messages
βββ easyscience.fitting # import-availability warnings
β βββ easyscience.fitting.bumps # Bumps fitting runtime messages
β βββ easyscience.fitting.lmfit # LMFit fitting runtime messages
β βββ easyscience.fitting.dfo # DFO fitting runtime messages
βββ easyscience.variable # parameter warnings
-βββ easyscience.global_object # undo/redo, debugging diagnostics
+βββ easyscience.global_object # debugging diagnostics
```
Setting the level on a parent logger affects all its children, because
@@ -56,9 +55,6 @@ logging.getLogger('easyscience.fitting').setLevel(logging.ERROR)
# Suppress only Bumps runtime messages
logging.getLogger('easyscience.fitting.bumps').setLevel(logging.ERROR)
-
-# Suppress only legacy deprecation notices
-logging.getLogger('easyscience.legacy').setLevel(logging.ERROR)
```
## Environment Variable
diff --git a/docs/docs/user-guide/external-calculators.md b/docs/docs/user-guide/external-calculators.md
new file mode 100644
index 00000000..9b3c4f9f
--- /dev/null
+++ b/docs/docs/user-guide/external-calculators.md
@@ -0,0 +1,332 @@
+# Connecting an External Calculator
+
+Technique libraries built on EasyScience usually compute their model
+with an external package (for example refnx or refl1d for
+reflectometry). This page describes how such a calculator connects to
+the EasyScience model and to `Fitter`.
+
+EasyScience does not provide a calculator base class, a factory or a
+registry. A calculator is any object your library defines; the only
+thing EasyScience sees is the fit function passed to `Fitter`.
+
+## General overview
+
+```mermaid
+flowchart LR
+ subgraph lib["Technique library"]
+ calc["Calculator backend choice and configuration"]
+ func["Fit function calculator.fit_func_for(model)"]
+ backend["Backend package e.g. refnx, refl1d"]
+ end
+ subgraph core["EasyScience core"]
+ fitter["Fitter / MultiFitter"]
+ minimizer["Minimizer LMFit, Bumps, DFO"]
+ model["Model (ModelBase) Parameters: the only state"]
+ end
+
+ calc -- "creates" --> func
+ func -. "handed over once: Fitter(model, fit_function)" .-> fitter
+ fitter -- "get_free_parameters()" --> model
+ minimizer -- "trial values" --> fitter
+ fitter -- "1: sets Parameter.value" --> model
+ fitter -- "2: calls fit_function(x)" --> func
+ func -- "3: reads current values" --> model
+ func -- "4: builds and evaluates with plain numbers" --> backend
+ backend -- "5: y" --> func
+```
+
+The arrows between the library and the model only point one way: the fit
+function reads parameters, it never writes them. `Fitter` is the only
+common point between core and the library, and the only thing passed
+across is a callable.
+
+## The Contract
+
+- **EasyScience is the only store of parameter state.** Values,
+ uncertainties, bounds, fixed flags and dependencies live in the
+ `Parameter` objects of the model. The calculator does not keep its own
+ copy.
+- **The calculator reads the model when asked to calculate.** Each
+ evaluation uses the current values of the independent parameters and
+ of the dependent parameters derived from them.
+- **Reading a parameter has no side effects.** `Parameter.value` returns
+ the stored value; it never calls into a calculator.
+- **Evaluation does not change the model.** Calculating leaves values,
+ uncertainties, bounds, fixed flags and dependencies as they were. Only
+ the fitting engine writes parameter values during a fit, through the
+ ordinary `value` setter.
+- **Structural edits are seen by the next evaluation.** Adding or
+ removing a part of the model needs no extra step, because the
+ calculator reads the model fresh.
+
+## A Minimal Example
+
+The calculator below evaluates a straight line. It holds no parameter
+values; it reads them from the model on every call.
+
+```python
+import numpy as np
+
+from easyscience import Parameter
+from easyscience.base_classes import ModelBase
+from easyscience.fitting import Fitter
+
+
+class Line(ModelBase):
+ def __init__(self, slope: float, intercept: float):
+ super().__init__()
+ self._slope = Parameter(slope, display_name='slope')
+ self._intercept = Parameter(intercept, display_name='intercept')
+
+ @property
+ def slope(self) -> Parameter:
+ return self._slope
+
+ @property
+ def intercept(self) -> Parameter:
+ return self._intercept
+
+
+class LineCalculator:
+ """Owned by the technique library, unknown to EasyScience."""
+
+ def calculate(self, model: Line, x: np.ndarray) -> np.ndarray:
+ # Hand plain numbers to the backend
+ return model.slope.value * x + model.intercept.value
+
+ def fit_func_for(self, model: Line):
+ return lambda x: self.calculate(model, x)
+
+
+model = Line(slope=1.0, intercept=0.0)
+calculator = LineCalculator()
+
+x = np.linspace(0, 10, 50)
+y = 3.0 * x + 1.5 + np.random.default_rng(0).normal(0.0, 0.01, x.size)
+
+fitter = Fitter(model, calculator.fit_func_for(model))
+result = fitter.fit(x=x, y=y, weights=np.ones_like(x))
+
+print(model.slope.value, model.intercept.value) # close to 3.0 and 1.5
+```
+
+During the fit, the engine sets the parameter values and then calls the
+fit function, so the calculator always uses the values being tried.
+
+Dependent parameters need no special handling. If the intercept is made
+dependent on the slope, the engine fits only the slope and the
+calculator reads the recomputed intercept:
+
+```python
+model.intercept.make_dependent_on(
+ dependency_expression='0.5 * s',
+ dependency_map={'s': model.slope},
+)
+```
+
+## Several Datasets
+
+With `MultiFitter`, pass one fit function per dataset. A `Parameter`
+object used by several models is fitted once and every calculator call
+will read the same value:
+
+```python
+from easyscience.fitting.multi_fitter import MultiFitter
+
+fitter = MultiFitter(
+ [model_1, model_2],
+ [calculator.fit_func_for(model_1), calculator.fit_func_for(model_2)],
+)
+results = fitter.fit(x=[x_1, x_2], y=[y_1, y_2], weights=[w_1, w_2])
+```
+
+## Performance
+
+Rebuilding the backend object on every evaluation is simple and, for
+typical models, cheap. If profiling shows that construction dominates,
+build the backend object once when a fit starts, refresh only the
+numbers on each call, and discard it when the fit ends. Such a
+fit-scoped session lives inside the fit function in your library. The
+parameter set does not change during a fit, so the session does not have
+to track structural edits.
+
+## Writing Values Back
+
+EasyScience has no API for a backend to write values into the model, and
+ordinary evaluation should never need one. If your library has an
+operation that genuinely changes the model based on a backend result,
+perform it in your own code through the public `Parameter.value` setter,
+outside the fit, as an explicit step the user can see.
+
+## Following a Fit Iteration by Iteration
+
+Because the calculator only reads the model, it is not the place to
+watch a fit's progress. To see the parameter values at each iteration,
+pass a `progress_callback` to `fit()`. Starting again from a fresh,
+independent line (the dependent intercept from the previous section
+would not be in the payload, see below):
+
+```python
+model = Line(slope=1.0, intercept=0.0)
+history = []
+
+
+def on_iteration(payload: dict) -> None:
+ # Called by the minimizer as the fit progresses. Values are keyed
+ # by unique_name and describe the point the minimizer reports.
+ values = payload['parameter_values']
+ slope = values[model.slope.unique_name]
+ intercept = values[model.intercept.unique_name]
+ history.append((payload['iteration'], slope, intercept))
+ print(
+ f'iteration {payload["iteration"]:3d}: '
+ f'slope={slope:.6f} intercept={intercept:.6f} chi2={payload["chi2"]:.4g}'
+ )
+
+
+fitter = Fitter(model, calculator.fit_func_for(model))
+result = fitter.fit(x=x, y=y, weights=np.ones_like(x), progress_callback=on_iteration)
+```
+
+The callback receives a dictionary with these keys:
+
+| Key | Meaning |
+| --------------------------- | ------------------------------------------------------------------------------- |
+| `iteration` | Iteration or evaluation number, as counted by the minimizer |
+| `chi2` | Chi-squared at the reported point |
+| `reduced_chi2` | Chi-squared divided by the degrees of freedom |
+| `parameter_values` | Values of the fitted parameters at the reported point, indexed by `unique_name` |
+| `refresh_plots`, `finished` | Hints for user interfaces |
+
+`parameter_values` holds exactly the parameters returned by
+`model.get_free_parameters()`: the free, independent ones. Fixed and
+dependent parameters are not in it. To report a dependent value, read it
+from the model or recompute it from the free values in the payload.
+
+Its return value is ignored. To stop a fit from the callback, see
+[Stopping a Fit](#stopping-a-fit).
+
+When the callback is called, and which point it reports, depends on the
+minimizer:
+
+| Minimizer | Called | Reported point |
+| --------- | ---------------------------- | ------------------------------------------- |
+| LMFit | on every function evaluation | the point just evaluated |
+| Bumps | once per optimizer step | the optimizer's current point for that step |
+| DFO | on every function evaluation | the best point found so far |
+
+With LMFit `iteration` is an evaluation counter, not a step index: it
+starts at `-1`, the first few callbacks all describe the starting point,
+the small steps of about 1e-5 are LMFit estimating the gradient
+numerically, and the last number is reported twice. Compare the values
+rather than the counter if you only want real steps. With DFO the
+reported values change only when an evaluation improves on the best
+point.
+
+If you need a hook on every evaluation rather than every iteration, put
+it in the fit function your calculator returns: it is called each time
+and reads the current values from the model.
+
+## Stopping a Fit
+
+EasyScience has no single cancel switch that works the same way on every
+minimizer. The options and how each minimizer responds:
+
+| How | LMFit | Bumps | DFO |
+| --------------------------------- | ------------------------- | ------------------------------------------------ | ----------------------------------- |
+| Raise from `progress_callback` | stops, raises `FitError` | stops, raises `FitError` | stops, raises `FitError` |
+| Raise from the fit function | stops, raises `FitError` | stops only after logging every failed evaluation | stops, raises `FitError` |
+| `abort_test=` passed to `fit()` | ignored with a warning | stops, returns `success=False` | raises `FitError` (unknown keyword) |
+| `fitter.max_evaluations` property | stops after N evaluations | stops after N optimizer steps | stops after N evaluations |
+
+### Raising from the progress callback
+
+This is the one approach that works on all three minimizers. Each
+minimizer catches the exception, puts the fitted parameters back to the
+values they had before the fit, and raises `FitError` with your
+exception as its argument.
+
+A user interface can set a flag from its "Stop" button and let the
+callback check it:
+
+```python
+from easyscience.fitting.minimizers import FitError
+
+
+class FitCancelled(Exception):
+ pass
+
+
+stop_requested = False # set to True from the user interface
+
+
+def on_iteration(payload: dict) -> None:
+ if stop_requested:
+ raise FitCancelled()
+
+
+try:
+ result = fitter.fit(x=x, y=y, weights=np.ones_like(x), progress_callback=on_iteration)
+except FitError:
+ # The fit was cancelled; the model is back at its pre-fit values.
+ ...
+```
+
+With LMFit and DFO the callback runs on every function evaluation, so
+the fit stops at the next evaluation. With Bumps it runs once per
+optimizer step, so the fit stops at the end of the current step.
+
+### Raising from the fit function
+
+Avoid using the fit function as a stop signal. LMFit and DFO stop as
+they would for an exception from the callback, but Bumps catches the
+error on every evaluation, logs a traceback, and carries on for many
+failing evaluations before it gives up.
+
+### `abort_test` with Bumps
+
+Bumps, and the DREAM sampler, accept an `abort_test` argument: a
+function without arguments that returns `True` when the fit should stop.
+It is polled during the optimizer loop and once more after it finishes,
+so it must have no side effects:
+
+```python
+from easyscience import Minimizers
+
+fitter.switch_minimizer(Minimizers.Bumps)
+result = fitter.fit(
+ x=x, y=y, weights=np.ones_like(x), abort_test=lambda: stop_requested
+)
+if not result.success:
+ print(result.message) # 'Fit aborted before convergence'
+```
+
+Unlike raising an exception, an aborted Bumps fit returns normally and
+keeps the best values found so far in the model. Do not pass
+`abort_test` to the other minimizers: LMFit ignores it and runs the fit
+to the end, and DFO rejects it with a `FitError`.
+
+### Limiting the number of evaluations
+
+Set the budget on the `Fitter` before calling `fit()`. Do not pass
+`max_evaluations=` (or `tolerance=`) as a keyword to `fit()`: the
+`Fitter` already forwards its own value, and a second one raises
+`TypeError: got multiple values for keyword argument` on every
+minimizer.
+
+```python
+fitter.max_evaluations = 100
+result = fitter.fit(x=x, y=y, weights=np.ones_like(x))
+if not result.success:
+ print(result.message)
+```
+
+The budget means different things to the minimizers. For LMFit and DFO
+it is a limit on function evaluations. For Bumps it is forwarded as the
+optimizer's `steps` setting, and one step can contain several
+evaluations, so the calculator may be called many more times than the
+number you set. The `iterations` reported in the result and in the
+progress payload count the same quantity as the budget for each
+minimizer, so they are not comparable across minimizers. It is a budget,
+not a cancel button, and it does not bound elapsed time when an
+evaluation is slow.
diff --git a/docs/docs/user-guide/index.md b/docs/docs/user-guide/index.md
index 84fe5c09..4c31eb8c 100644
--- a/docs/docs/user-guide/index.md
+++ b/docs/docs/user-guide/index.md
@@ -8,3 +8,10 @@ icon: material/book-open-variant
β How to suppress, filter, or redirect the messages that EasyScience
produces. Covers the `EASYSCIENCE_LOG_LEVEL` environment variable, the
logger hierarchy, context managers, and common recipes.
+- [:material-link-variant: Serializing Dependent Parameters](parameter-dependency-serialization.md)
+ β How to save and load parameters that depend on other parameters, and
+ how dependencies are resolved after loading.
+- [:material-calculator-variant: Connecting an External Calculator](external-calculators.md)
+ β How a technique library connects its own calculator to an
+ EasyScience model and to `Fitter`, and what to change when migrating
+ from `InterfaceFactoryTemplate`.
diff --git a/docs/docs/user-guide/parameter-dependency-serialization.md b/docs/docs/user-guide/parameter-dependency-serialization.md
new file mode 100644
index 00000000..2a8254e8
--- /dev/null
+++ b/docs/docs/user-guide/parameter-dependency-serialization.md
@@ -0,0 +1,179 @@
+# Serializing Dependent Parameters
+
+A `Parameter` can depend on other parameters through an expression (see
+`Parameter.make_dependent_on` and `Parameter.from_dependency`). This
+page explains how such parameters are written to a dictionary or JSON
+and how their dependencies are rebuilt when they are loaded.
+
+## Quick Start
+
+Serialize every parameter involved with `to_dict`:
+
+```python
+import json
+
+from easyscience import Parameter
+
+a = Parameter(value=2.0, unit='m', min=0, max=10, display_name='a')
+b = Parameter.from_dependency(
+ dependency_expression='2 * a',
+ dependency_map={'a': a},
+ display_name='b',
+)
+
+with open('parameters.json', 'w') as f:
+ json.dump({'a': a.to_dict(), 'b': b.to_dict()}, f, indent=2)
+```
+
+To load them, deserialize all parameters first. Then resolve the
+dependencies in a single step:
+
+```python
+import json
+
+from easyscience import Parameter
+from easyscience.variable.parameter_dependency_resolver import (
+ resolve_all_parameter_dependencies,
+)
+
+with open('parameters.json') as f:
+ params_dict = json.load(f)
+
+new_a = Parameter.from_dict(params_dict['a'])
+new_b = Parameter.from_dict(params_dict['b'])
+
+resolve_all_parameter_dependencies({'a': new_a, 'b': new_b})
+
+new_a.value = 5.0
+print(new_b.value) # 10.0
+```
+
+!!! warning "Always resolve after loading"
+
+ `Parameter.from_dict` does **not** restore the dependency on its
+ own. Until `resolve_all_parameter_dependencies` (or
+ `Parameter.resolve_pending_dependencies`) is called, a loaded
+ dependent parameter behaves as an independent parameter holding the
+ value it had when it was saved.
+
+Alternatively, if all you have is a dictionary of serialized parameters,
+the convenience helper `deserialize_and_resolve_parameters` combines
+both steps:
+
+```python
+from easyscience.variable.parameter_dependency_resolver import (
+ deserialize_and_resolve_parameters,
+)
+
+params = deserialize_and_resolve_parameters(params_dict)
+params['a'].value = 3.0
+print(params['b'].value) # 6.0
+```
+
+## What Is Written
+
+An independent parameter is written with its constructor arguments
+(`value`, `unit`, `variance`, `min`, `max`, `fixed`, `description`,
+`url`, and `display_name`/`unique_name` when they were set explicitly).
+A dependent parameter gets the following additional fields:
+
+| Field | Meaning |
+| -------------------------------- | ----------------------------------------------------------------------------------- |
+| `_dependency_string` | The dependency expression, e.g. `"2 * a"`. |
+| `_dependency_map_serializer_ids` | Maps each symbol in the expression to the serializer id of the object it refers to. |
+| `_independent` | `false`, marking the parameter as dependent. |
+| `_desired_unit` | Only present if a `desired_unit` was given when making the dependency. |
+
+Any parameter that something else depends on also writes its own
+`__serializer_id`. This is a UUID assigned the first time another
+parameter depends on it. It is what `_dependency_map_serializer_ids`
+refers to:
+
+```json
+{
+ "@module": "easyscience.variable.parameter",
+ "@class": "Parameter",
+ "value": 4.0,
+ "unit": "m",
+ "display_name": "b",
+ "_dependency_string": "2 * a",
+ "_independent": false,
+ "_dependency_map_serializer_ids": {
+ "a": "a48db37b-519b-4bea-9873-aa3cc5b2ec76"
+ }
+}
+```
+
+Serializer ids are independent of `unique_name`. A loaded parameter can
+therefore receive a fresh unique name, avoiding a clash with objects
+that already exist in the session, and its dependencies still find each
+other.
+
+## How Loading Works
+
+1. `Parameter.from_dict` creates each parameter as an ordinary
+ independent parameter and restores its `__serializer_id`.
+2. For dependent parameters, the expression, the serializer-id map and
+ the desired unit are kept on the object as pending dependency
+ information.
+3. `resolve_all_parameter_dependencies(obj)` walks `obj` and finds every
+ parameter with pending information. `obj` can be a single
+ `Parameter`, a list, tuple or dict of them, or an object exposing
+ them as public attributes or properties. For each parameter it looks
+ up the referenced objects by serializer id among all objects alive in
+ the session and calls `make_dependent_on`. It then removes the
+ pending information.
+
+Because resolution happens only after everything is loaded, parameters
+can be deserialized in **any order**. A dependent parameter may come
+before the parameters it depends on.
+
+To check what still needs resolving, use
+`get_parameters_with_pending_dependencies`:
+
+```python
+from easyscience.variable.parameter_dependency_resolver import (
+ get_parameters_with_pending_dependencies,
+)
+
+pending = get_parameters_with_pending_dependencies({'a': new_a, 'b': new_b})
+print(f'{len(pending)} parameter(s) still need resolving')
+```
+
+## Errors
+
+`resolve_all_parameter_dependencies` tries every pending parameter and
+then raises a single `ValueError` listing all failures. The typical
+cause is a referenced parameter that was not loaded:
+
+```text
+ValueError: Failed to resolve dependencies for 1 parameter(s):
+Failed to resolve dependencies for parameter 'Parameter_0' (display_name: 'b', serializer_id: 'unknown'): Cannot find parameter with serializer_id 'f9040c2a-...'
+```
+
+An invalid expression, or one that would create a cyclic dependency, is
+reported the same way, with the underlying error message appended.
+
+## Best Practices
+
+- **Serialize the whole dependency graph.** Every parameter referenced
+ by a dependency expression must be saved and loaded together with the
+ parameters that depend on it.
+- **Load everything, then resolve once.** Deserialize all parameters
+ first and call `resolve_all_parameter_dependencies` on the collection
+ as a whole.
+- **Keep the referenced objects alive.** Lookup goes through the global
+ object registry, so a parameter that has been garbage-collected cannot
+ be found.
+- **Do not load the same data twice in one session.** Serializer ids are
+ restored verbatim. If the original parameters, or an earlier load of
+ the same file, are still alive, a dependency can be bound to that
+ older object instead of the one just loaded.
+
+!!! note "Parameters nested in models"
+
+ Dependency information is written by `Parameter.to_dict`. Calling
+ `to_dict` on a `ModelBase` that *contains* dependent parameters
+ does not currently go through `Parameter.to_dict`, so the
+ dependency fields are not included. Until this is addressed,
+ serialize dependent parameters directly as shown above.
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index 91165ebc..905fa409 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -192,6 +192,9 @@ nav:
- User Guide:
- User Guide: user-guide/index.md
- Controlling Log Output: user-guide/controlling-log-output.md
+ - Serializing Dependent Parameters:
+ user-guide/parameter-dependency-serialization.md
+ - Connecting an External Calculator: user-guide/external-calculators.md
- Tutorials:
- Tutorials: tutorials/index.md
- Bayesian analysis: tutorials/fitting-bayesian.ipynb
@@ -204,8 +207,6 @@ nav:
- base_classes: api-reference/base_classes.md
- fitting: api-reference/fitting.md
- global_object: api-reference/global_object.md
- - io: api-reference/io.md
- job: api-reference/job.md
- - models: api-reference/models.md
- utils: api-reference/utils.md
- variable: api-reference/variable.md
diff --git a/legacy/Examples/README.rst b/legacy/Examples/README.rst
deleted file mode 100644
index e2d0e47e..00000000
--- a/legacy/Examples/README.rst
+++ /dev/null
@@ -1,4 +0,0 @@
-Example Galleries
-=================
-
-These are a collection of examples covering aspects of the *EasyScience* project.
\ No newline at end of file
diff --git a/legacy/Examples/base/README.rst b/legacy/Examples/base/README.rst
deleted file mode 100644
index 46da573a..00000000
--- a/legacy/Examples/base/README.rst
+++ /dev/null
@@ -1,6 +0,0 @@
-.. _base_examples:
-
-Subclassing Examples
-------------------------
-
-This section gathers examples which correspond to subclassing the :class:`easyscience.base_classes.ObjBase` class.
diff --git a/legacy/Examples/base/plot_baseclass1.py b/legacy/Examples/base/plot_baseclass1.py
deleted file mode 100644
index b87c559e..00000000
--- a/legacy/Examples/base/plot_baseclass1.py
+++ /dev/null
@@ -1,97 +0,0 @@
-"""
-Subclassing ObjBase - Simple Pendulum
-=====================================
-This example shows how to subclass :class:`easyscience.base_classes.ObjBase` with parameters from
-:class:`EasyScience.variable.Parameter`. For this example a simple pendulum will be modeled.
-
-.. math::
- y = A \sin (2 \pi f t + \phi )
-
-Imports
-*******
-
-Firstly the necessary imports. Notice that we import numpy from easyscience. This is not done for any reason other than
-saving time from multiple imports.
-"""
-
-import matplotlib.pyplot as plt
-import numpy as np
-
-from easyscience.base_classes import ObjBase
-from easyscience.variable import Parameter
-
-# %%
-# Subclassing
-# ***********
-# To include embedded rST, use a line of >= 20 ``#``'s or ``#%%`` between your
-# rST and your code. This separates your example
-# into distinct text and code blocks. You can continue writing code below the
-# embedded rST text block:
-
-
-class Pendulum(ObjBase):
- def __init__(self, A: Parameter, f: Parameter, p: Parameter):
- super(Pendulum, self).__init__('SimplePendulum', A=A, f=f, p=p)
-
- @classmethod
- def from_pars(cls, A: float = 1, f: float = 1, p: float = 0):
- A = Parameter('Amplitude', A)
- f = Parameter('Frequency', f)
- p = Parameter('Phase', p)
- return cls(A, f, p)
-
- def __call__(self, t):
- return self.A.value * np.sin(2 * np.pi * self.f.value * t + self.p.value)
-
- def plot(self, time, axis=None, **kwargs):
- if axis is None:
- axis = plt
- else:
- axis.set_title(f'A={self.A.value}, F={self.f.value}, P={self.p.value}')
- p = axis.plot(time, self(time), **kwargs)
- return p
-
-
-# %%
-# Single Example
-# **************
-# To include embedded rST, use a line of >= 20 ``#``'s or ``#%%`` between your
-# rST and your code. This separates your example
-# into distinct text and code blocks. You can continue writing code below the
-# embedded rST text block:
-
-p1 = Pendulum.from_pars()
-# Another pendulum with Amplitude = 5
-p2 = Pendulum.from_pars(A=5)
-# Another pendulum with Frequency = 4
-p3 = Pendulum.from_pars(A=5, f=4)
-# Another pendulum with Phase = pi/2
-p4 = Pendulum.from_pars(A=5, f=4, p=np.pi / 2)
-
-# %%
-# Plotting
-
-t = np.linspace(0, 3, 601)
-fig = plt.figure()
-gs = fig.add_gridspec(2, 2)
-(ax1, ax2), (ax3, ax4) = gs.subplots(sharex='col', sharey='row')
-p1.plot(t, axis=ax1)
-p2.plot(t, axis=ax2)
-p3.plot(t, axis=ax3)
-p4.plot(t, axis=ax4)
-fig.show()
-
-# %%
-# Multiple Examples
-# *****************
-# To include embedded rST, use a line of >= 20 ``#``'s or ``#%%`` between your
-# rST and your code. This separates your example
-# into distinct text and code blocks. You can continue writing code below the
-# embedded rST text block:
-
-pendulum_array = [Pendulum.from_pars(p=phase) for phase in np.linspace(0, 1, 3)]
-fig = plt.figure()
-for pendulum in pendulum_array:
- pendulum.plot(t, label=f'Phase = {pendulum.p}')
-plt.legend(loc='lower right')
-fig.show()
diff --git a/legacy/Examples/fitting/README.rst b/legacy/Examples/fitting/README.rst
deleted file mode 100644
index 614960cb..00000000
--- a/legacy/Examples/fitting/README.rst
+++ /dev/null
@@ -1,6 +0,0 @@
-.. _fitting_examples:
-
-Fitting Examples
-----------------
-
-This section gathers examples which demonstrate fitting functionality using EasyScience's fitting capabilities.
diff --git a/legacy/PARAMETER_DEPENDENCY_SERIALIZATION.md b/legacy/PARAMETER_DEPENDENCY_SERIALIZATION.md
deleted file mode 100644
index 66a6659b..00000000
--- a/legacy/PARAMETER_DEPENDENCY_SERIALIZATION.md
+++ /dev/null
@@ -1,259 +0,0 @@
-# Parameter Dependency Serialization
-
-This document explains how to serialize and deserialize `Parameter`
-objects that have dependencies.
-
-## Overview
-
-Parameters with dependencies can now be serialized to dictionaries (and
-JSON) while preserving their dependency relationships. After
-deserialization, the dependencies are automatically reconstructed using
-the `serializer_id` attribute to match parameters, with `unique_name`
-attribute being used as a fallback.
-
-## Key Features
-
-- **Automatic dependency serialization**: Dependency expressions and
- maps are automatically saved during serialization
-- **Reliable dependency resolution**: Dependencies are resolved using
- stable `serializer_id` attributes with `unique_name` as fallback after
- deserialization
-- **Order-independent loading**: Parameters can be loaded in any order
- thanks to the reliable ID system
-- **Bulk dependency resolution**: Utility functions help resolve all
- dependencies at once
-- **JSON compatibility**: Full support for JSON
- serialization/deserialization
-- **Backward compatibility**: Existing code using `unique_name`
- continues to work as fallback
-
-## Usage
-
-### Basic Serialization/Deserialization
-
-```python
-import json
-from easyscience import Parameter, global_object
-from easyscience.variable.parameter_dependency_resolver import resolve_all_parameter_dependencies
-
-# Create parameters with dependencies
-a = Parameter(name="a", value=2.0, unit="m", min=0, max=10)
-b = Parameter.from_dependency(
- name="b",
- dependency_expression="2 * a",
- dependency_map={"a": a},
- unit="m"
-)
-
-# Serialize to dictionary and save to file
-params_dict = {"a": a.as_dict(), "b": b.as_dict()}
-with open("parameters.json", "w") as f:
- json.dump(params_dict, f, indent=2, default=str)
-
-print("Parameters saved to parameters.json")
-```
-
-In a new Python session:
-
-```python
-import json
-from easyscience import Parameter, global_object
-from easyscience.variable.parameter_dependency_resolver import resolve_all_parameter_dependencies
-
-# Load parameters from file
-with open("parameters.json", "r") as f:
- params_dict = json.load(f)
-
-# Clear global map (simulate new environment)
-global_object.map._clear()
-
-# Deserialize parameters
-new_a = Parameter.from_dict(params_dict["a"])
-new_b = Parameter.from_dict(params_dict["b"])
-
-# Resolve dependencies
-resolve_all_parameter_dependencies({"a": new_a, "b": new_b})
-
-# Dependencies are now working
-new_a.value = 5.0
-print(new_b.value) # Will be 10.0 (2 * 5.0)
-```
-
-### JSON Serialization
-
-```python
-import json
-
-# Serialize to JSON
-param_dict = parameter.as_dict()
-json_str = json.dumps(param_dict, default=str)
-
-# Deserialize from JSON
-loaded_dict = json.loads(json_str)
-new_param = Parameter.from_dict(loaded_dict)
-
-# Resolve dependencies
-resolve_all_parameter_dependencies(new_param)
-```
-
-### Bulk Operations
-
-```python
-from easyscience.variable.parameter_dependency_resolver import get_parameters_with_pending_dependencies
-
-# Create multiple parameters with dependencies
-params = create_parameter_hierarchy() # Your function
-
-# Serialize all
-serialized = {name: param.as_dict() for name, param in params.items()}
-
-# Clear and deserialize
-global_object.map._clear()
-new_params = {name: Parameter.from_dict(d) for name, d in serialized.items()}
-
-# Check which parameters have pending dependencies
-pending = get_parameters_with_pending_dependencies(new_params)
-print(f"Found {len(pending)} parameters with pending dependencies")
-
-# Resolve all at once
-resolve_all_parameter_dependencies(new_params)
-```
-
-## Implementation Details
-
-### Serialization
-
-During serialization, the following additional fields are added to
-dependent parameters:
-
-- `_dependency_string`: The original dependency expression
-- `_dependency_map_serializer_ids`: A mapping of dependency keys to
- stable dependency IDs (preferred)
-- `_dependency_map_unique_names`: A mapping of dependency keys to unique
- names (fallback)
-- `__serializer_id`: The parameter's own unique dependency ID
-- `_independent`: Boolean flag indicating if the parameter is dependent
-
-### Deserialization
-
-During deserialization:
-
-1. Parameters are created normally but marked as independent temporarily
-2. Dependency information is stored in `_pending_dependency_string`,
- `_pending_dependency_map_serializer_ids`, and
- `_pending_dependency_map_unique_names` attributes
-3. The parameter's own `__serializer_id` is restored from serialized
- data
-4. After all parameters are loaded,
- `resolve_all_parameter_dependencies()` establishes the dependency
- relationships using dependency IDs first, then unique names as
- fallback
-
-### Dependency Resolution
-
-The dependency resolution process:
-
-1. Scans for parameters with pending dependencies
-2. First attempts to look up dependency objects by their stable
- `serializer_id`
-3. Falls back to `unique_name` lookup in the global map if serializer_id
- is not available
-4. Calls `make_dependent_on()` to establish the dependency relationship
-5. Cleans up temporary attributes
-
-This dual-strategy approach ensures reliable dependency resolution
-regardless of parameter loading order while maintaining backward
-compatibility.
-
-## Error Handling
-
-The system provides detailed error messages for common issues:
-
-- Missing dependencies (parameter with required unique_name not found)
-- Invalid dependency expressions
-- Circular dependency detection
-
-## Utility Functions
-
-### `resolve_all_parameter_dependencies(obj)`
-
-Recursively finds all Parameter objects with pending dependencies and
-resolves them.
-
-**Parameters:**
-
-- `obj`: Object to search for Parameters (can be Parameter, list, dict,
- or complex object)
-
-**Returns:**
-
-- None (modifies parameters in place)
-
-**Raises:**
-
-- `ValueError`: If dependency resolution fails
-
-### `get_parameters_with_pending_dependencies(obj)`
-
-Finds all Parameter objects that have pending dependencies.
-
-**Parameters:**
-
-- `obj`: Object to search for Parameters
-
-**Returns:**
-
-- `List[Parameter]`: List of parameters with pending dependencies
-
-## Best Practices
-
-1. **Always resolve dependencies after deserialization**: Use
- `resolve_all_parameter_dependencies()` after loading serialized
- parameters
-
-2. **Handle the global map carefully**: The global map must contain all
- referenced parameters for dependency resolution to work
-
-3. **Use unique names for cross-references**: When creating dependency
- expressions that reference other parameters, consider using unique
- names with quotes: `'Parameter_0'`
-
-4. **Error handling**: Wrap dependency resolution in try-catch blocks
- for robust error handling
-
-5. **Bulk operations**: For complex object hierarchies, use the utility
- functions to handle all parameters at once
-
-6. **Reliable ordering**: With the new dependency ID system, parameters
- can be loaded in any order without affecting dependency resolution
-
-7. **Access dependency ID**: Use `parameter.serializer_id` to access the
- stable ID for debugging or manual cross-referencing
-
-## Example: Complex Hierarchy
-
-```python
-def save_model(model):
- \"\"\"Save a model with parameter dependencies to JSON.\"\"\"
- model_dict = model.as_dict()
- with open('model.json', 'w') as f:
- json.dump(model_dict, f, indent=2, default=str)
-
-def load_model(filename):
- \"\"\"Load a model from JSON and resolve dependencies.\"\"\"
- global_object.map._clear() # Start fresh
-
- with open(filename) as f:
- model_dict = json.load(f)
-
- model = Model.from_dict(model_dict)
-
- # Resolve all parameter dependencies
- resolve_all_parameter_dependencies(model)
-
- return model
-```
-
-This system ensures that complex parameter hierarchies with dependencies
-can be reliably serialized and reconstructed while maintaining their
-behavioral relationships.
diff --git a/legacy/examples_old/dataset_examples.ipynb b/legacy/examples_old/dataset_examples.ipynb
deleted file mode 100644
index 48f206c2..00000000
--- a/legacy/examples_old/dataset_examples.ipynb
+++ /dev/null
@@ -1,1254 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [],
- "source": [
- "import numpy as np\n",
- "from easyscience.Datasets.xarray import xr\n",
- "from easyscience.Objects.Base import Parameter, BaseObj\n",
- "from easyscience.fitting import Fitter"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "try:\n",
- " import hvplot.xarray\n",
- "except:\n",
- " import sys\n",
- " !{sys.executable} -m pip install hvplot\n",
- " import hvplot.xarray"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# 1D Data"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 13,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Create a dataset\n",
- "d = xr.Dataset()\n",
- "\n",
- "# Create a coreObject\n",
- "b = BaseObj('line',\n",
- " m=Parameter('m', 1),\n",
- " c=Parameter('c', 1))\n",
- "\n",
- "# Create a fit function to use with b\n",
- "def fit_fun(x, *args, **kwargs):\n",
- " return b.c.raw_value + b.m.raw_value * x\n",
- "\n",
- "\n",
- "# Create and initialize the fitter\n",
- "f = Fitter()\n",
- "f.initialize(b, fit_fun)\n",
- "\n",
- "# Generate some `fake` data to fit to\n",
- "nx = 1E3\n",
- "x_min = 0\n",
- "x_max = 100\n",
- "\n",
- "x = np.linspace(x_min, x_max, num=int(nx))\n",
- "y = 2*x - 1 + 5*(np.random.random(size=x.shape) - 0.5)\n",
- "\n",
- "# Add the data to the dataset\n",
- "d.easyscience.add_coordinate('x', x)\n",
- "d.easyscience.add_variable('y', ['x'], y, auto_sigma=False)\n",
- "\n",
- "# Perform the fit\n",
- "f_res = d['y'].easyscience.fit(f)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 14,
- "metadata": {},
- "outputs": [
- {
- "data": {},
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "data": {
- "application/vnd.holoviews_exec.v0+json": "",
- "text/html": [
- "