Version 5.2.0 is available as a signed, notarized disk image. Open it and drag mac4DSTEM to your Applications folder — it launches without a security warning, and needs nothing else installed.
+Version 5.2.0 is available as a signed, notarized disk image. Open it and drag mac4DSTEM to your Applications folder — it launches without a security warning, and needs nothing else installed.
The source is published on GitHub under GPL-3.0.
+The source is published on GitHub under GPL-3.0.
- Requires macOS 26 or later on Apple silicon (for macOS 14–25, version 3.0.0 remains available). No external runtime, toolchain, or library + Requires macOS 26 or later on Apple silicon (for macOS 14–25, version 3.0.0 remains available). No external runtime, toolchain, or library installation is needed — all dependencies are contained within the application. The Merlin MIB and EMPAD RAW readers are preview-grade pending further vendor data.
@@ -419,7 +419,7 @@Each kit against the Python package it ports, same inputs, on an Apple M-series Mac. Tables are copied from each kit's docs/benchmarks.md at its latest tag; where the kit is slower than Python, the table says so.
Small and typical sizes; the large rows are not measured yet (tools/benchmarks/bench.py run --large adds them). Same machine, one case at a time, no other benchmark running.
| Machine | Apple M5 Pro, arm64, macOS 27.0.1 |
| Kit | |
| Reference | RosettaSciIO 0.14.1.dev96+g049e7d707 (the kit's pin), Python 3.14.7, NumPy 2.5.4, h5py 3.16.0, tifffile 2026.9.20, numba 0.68.0, sparse 0.19.2, in one virtual environment |
| Operation | open the file and have all its data in memory: file_reader(lazy=False) against the kit's signal load() (Velox SI: VeloxSpectrumImageSignals; HDF5 4D cube: h5py dataset[...], the floor, py4DSTEM is not an rsciio plugin) |
| Protocol | one fresh process per side and case; one warm-up read, then five timed reads; median, with min to max in brackets; peak resident size of that process (includes the interpreter, NumPy and numba on the Python side) |
| Same work | both sides' data sums are compared (inputs are integer-valued, so the sums are exact) |
| Inputs | generated by tools/benchmarks/bench.py into a scratch folder (planted DM4, Velox, TIFF, MRC, MIB, EMPAD, HDF5 and MSA files); no measured data |
Re-run from a fresh clone of the kit, on a quiet Mac (run --large adds the large rows):
Listed first in docs/parity.md (known limits) with the cause. Found and fixed before this table (diagnosed with sample, then measured): MIB 934 ms to 35 ms (an ISO8601DateFormatter built per frame header, 1024 times, plus a per-pixel Data.append); MSA 111 ms to 17 ms (per-line allocation in the first reader).
Run 2026-10-21 (ImagoKit 0.6.0 sources). Header. Apple M5 Pro, macOS (Darwin 27.0.0), Swift 6.4 (swiftlang-6.4.0.34.1), swift test -c release; Python 3.12.14 with NumPy 2.5.4, SciPy 1.18.1, scikit-image 0.26.0 (single-threaded np.fft/ndimage). Method. The same function, dtype and input size on both sides (uniform random pixels 30 to 70); one warm-up call, then five timed calls; the table shows the median and (min–max) in milliseconds. FFTs reuse their plan (FFT2D, FFT2DDouble; NumPy caches its own); rfft2, butterworth, the shifts, registrations and filters are the public one-call API. Caveat. Other builds ran on the machine at the same time (load average 11 to 25 during these runs, 6 to 8 before): absolute times are noisy, and both sides were measured in the same minutes so the ratios are comparable but not exact. The sizes are the small (128²) and typical (512²) ones plus one non-power-of-two (500²); masked registration, which costs eleven transforms of (2n−1)², is at 64² and 256². Large sizes (2048², 4096², 8192²) are not measured yet: IMAGOKIT_BENCH_LARGE=1 and --large add them.
Re-run from a fresh clone of the kit, on a quiet Mac, one benchmark at a time:
The harness is Tests/ImagoKitParityTests/Benchmarks/Benchmarks.swift (its one test returns at once unless IMAGOKIT_BENCH=1); tools/benchmarks/bench.py is the Python side. "kit / Python" below 1 means the kit is faster.
Today every function above is CPU (Accelerate, Float64 reference plus Float32 where it exists). No function has a Metal path, so no Metal break-even has been measured; the column says what the CPU table implies and what a large-size run must measure. A guess is not recorded as a measurement.
| function | backend today | decision |
|---|---|---|
fft2 Float32/Float64 | CPU | CPU only at 128² to 512² (0.02 to 2 ms: a Metal dispatch cannot win). Float64 lengths of prime factors 2, 3, 5, 7 run on the mixed-radix plan. Work item: a Metal Float32 FFT prototype, break-even not measured yet (2048², 4096², 8192²) |
rfft2, irfft2, butterworth, masked registration | CPU | CPU; real-input transform since 2026-10-22 (0.7 ms Float64 at 512², 1.1 times NumPy); a Metal path would have to beat 0.4 to 0.7 ms at 512², which a dispatch cannot, so the break-even is again a question for 2048² and up, not measured yet |
gaussian_filter, uniform_filter, difference_of_gaussians | CPU | CPU only at 128² to 512² (1 to 4 ms, memory bound). Metal candidate above 2048² for large sigma; not measured yet |
shift order 1 | CPU | CPU only: 0.65 ms at 512² |
shift order 3 (prefilter plus taps) | CPU | CPU; the prefilter is sequential along lines, the tap sum is the Metal-friendly part: candidate above 2048², not measured |
phase_cross_correlation | CPU | CPU; FFT-bound, follows fft2 |
profile_line | CPU | CPU: small arrays stay on the CPU |
label | CPU | CPU: union-find is not a Metal workload |
peak_local_max | CPU | CPU; the window maximum is a Metal candidate only above 4096² |
Measured 2026-10-21. Small and typical sizes only; the large rows are not measured yet (they run with DIFFRACTO_BENCH_LARGE=1). Best of five after one warm-up: the median is the figure, the spread (min-max) is in brackets. Same inputs on both sides (tools/benchmarks/gen_inputs.py writes them), Float64 unless the function is Float32 by design (ACOM, ellipse pattern pixels).
| Machine | Apple M5 Pro, 64 GB, macOS 27.0.1 (Darwin 27.0.0) |
| Toolchain | Apple Swift 6.4, swift build -c release (separate package tools/benchmarks) |
| Python | 3.12.14, NumPy 2.5.4, SciPy 1.18.1, in two virtual environments: py4DSTEM 0.14.19 in one; pymatgen-core 2026.10.16 (the code; its pymatgen metapackage dist-info reads 2026.9.24), orix 0.15.0 and spglib 2.8.0 in the other |
| Load | 10.5 on a 12-core machine while other builds ran (uptime 17:39): every figure carries that noise, the single-core rows least. Re-run when the machine is quiet before quoting a ratio. |
| Pin note | The pymatgen environment runs pymatgen-core 2026.10.16 under a pymatgen 2026.9.24 metapackage, and every fixture that records pymatgen-core records 2026.10.16 (tools/pins/requirements/pymatgen.txt). Nothing here compares numbers, only time |
| 0.9.1 | The symmetry_detection and cif_import rows above are the 2026-10-21 measurement of 0.9.0. 0.9.1 replaced the closure check that took 95 % of them (see docs/parity.md, known limits); measured in-process on the same al_super2.cif (machine load 3.6): cif_import typical median 67 ms (min 67, max 77), against 297 ms (one run at load 9) before the change and the table's 351 ms. Not yet re-run through run.sh |
Re-run from a fresh clone of the kit, on a quiet Mac: gen_inputs.py writes the shared inputs, run.sh builds the benchmark package (tools/benchmarks, release) and prints the kit's JSON lines, bench_python.py prints the Python lines, and merge.py turns them into the table below. Add DIFFRACTO_BENCH_LARGE=1 to run.sh and bench_python.py for the large rows.
| Function | Backend | Why / break-even |
|---|---|---|
niggli_reduce, reflections, kinematic_reflections, symmetry_detection, cif_import | CPU only | small arrays, branchy integer logic, readers; the kit is already faster than Python except where docs/parity.md (known limits) lists it |
orientation_reduce | CPU only | 24 small matrix products per rotation; the batch (100 000 in 0.2 s) is bound by memory traffic, a GPU would pay the transfer for no gain at these sizes |
ellipse_fit_1d | CPU only | one 1D least-squares fit per pattern; level with Python |
acom_library | CPU only | built once per phase (60 ms for 600 axes), not per pattern |
acom_match_scan | CPU all cores; Metal not above any measured N | Metal 0.294 ms per pattern at 64 patterns, 0.236 at 1 024, against 0.245 and 0.216 on all CPU cores: the break-even is not reached up to 1 024 patterns (the Metal path is 7 to 22 % slower); the 128 x 128 row (16 384 patterns) is not measured yet, and the Metal path stays an option (ACOMBackend.metal), not the default. A larger library (2 400 axes) is the other axis still to measure |
Numbers in docs/parity.md are untouched by this table (time only).
Written 2026-10-21 for 0.8.0. Harness: Swift Tests/SpectroKitParityTests/BenchmarkQuantificationTests.swift and Tests/SpectroKitParityTests/BenchmarkThicknessTests.swift (built by swift test -c release, enabled by SPECTRO_BENCH_OUT), Python tools/benchmarks/bench.py, merged by tools/benchmarks/table.py. The Swift side peaks at 1.3 GB at the typical size. Inputs are closed-form on both sides (formulas in the harness headers), float64. One warm-up, then five runs, median and (min-max).
Machine: Apple M5 Pro, 64 GB, macOS 27.0.1 (26A434), Swift 6.4 (swiftlang-6.4.0.34.1), Python 3.12.14, NumPy 2.5.4, HyperSpy 2.4.0, exSpy 0.3.2 (no Numba). Other jobs were running on the machine, so the Python spreads are wide; read ratios within about 30 %.
Re-run from a fresh clone of the kit, on a quiet Mac, one benchmark at a time:
@@ -106,7 +106,7 @@| function | decision | why |
|---|---|---|
| quantification maps, all methods (the Python column converts to weight percent only; Swift returns both) | CPU only (parallel over pixels since 0.8.0) | 9 to 40 ms for 65 536 pixels; per pixel the work is 10 to 500 ns of arithmetic, so a Metal dispatch pays only above about 1e6 pixels; the break-even is not measured (the large size is not measured yet). Not a work item until a large-size run shows a slow large image |
| t/lambda log ratio, fixed threshold | CPU only; slower than Python on the typical size (0.49x), first in docs/parity.md (known limits) | single-threaded scalar passes against NumPy's vectorised axis sums |
| t/lambda log ratio, estimated threshold | CPU only | 5x faster than Python at the typical size (Python's per-pixel Savitzky-Golay search dominates) |
| EDX line fit, auto ID, EELS background fit, edge quantification | not benchmarked in 0.8.0 | need a Velox spectrum image or a planted EELS image on both sides; listed in docs/parity.md (known limits) |
Kit against py4DSTEM, same inputs, Float32 cubes, one run of tools/benchmarks on 2026-10-21. Harness: Tests/CuboVirtualTests/BenchmarkHarness.swift (Swift, built with swift test -c release, skipped unless CUBOKIT_BENCH_DIR is set) and tools/benchmarks/bench.py (the py4DSTEM side and the merge). Both read the same .npy cubes. Best of five after one warm-up; cells are best / median (spread = slowest - best) in milliseconds; "CPU vs py4DSTEM" is py4DSTEM's best divided by the kit CPU's best (above 1 the kit is faster). The cubes are held in memory (MemoryCube; DataCube(data=ndarray)). The Metal columns copy each tile to the GPU on every call; the _resident rows hold the cube in a ResidentCube (one Metal buffer) first.
Sizes (scan rows x columns x detector rows x columns): small 32x32x32x32 (1.0 M values), mid 64x64x32x32 (4.2 M), typical 100x100x64x64 (41.9 M, 168 MB). The large size (330x330x64x64, 1.8 GB) and the Bragg, strain, DPC, parallax and ptychography functions are not measured yet. The machine was shared during the run (load average 5 to 11 from other builds), so the spreads are wider than on a quiet Mac.
Re-run from a fresh clone of the kit, on a quiet Mac, one benchmark at a time (the typical cube is 168 MB on disk and in memory):
@@ -129,7 +129,7 @@The break-even is read off the three sizes above; between two measured sizes it is interpolated, and says so. The Mac was loaded (load average 10 to 11) during the final run, so a crossing near a tie moves with load.
| function | decision | why |
|---|---|---|
VirtualDiffraction.max | CPU always | block-parallel since 0.8.0: 5.9 ms at the typical size on the loaded 2026-10-21 run against NumPy's 2.8 and 29 ms with the host copy; 1.67 ms (was 2.24) on the resident MeanMaxBench since CuboKit#53; a vmax pass is memory-bound, so a copy to the GPU never pays |
VirtualDiffraction.mean, meanAndMax | CPU; resident Metal always | since 0.9.0 the compensated Double sum is a SIMD TwoSum across cores; since CuboKit#53 it is held in registers over four positions: 2.17 ms at the typical size (was 4.47, 22.0 before 0.9.0), ahead of NumPy's 2.7; host-copy Metal (27.5) does not pay; the resident cube (3.4 ms at 41.9 M values, measured before 0.9.0) has not been measured against the new CPU pass |
VirtualDiffraction.pattern with a mask | CPU (0.9.0); resident Metal for repeated reductions | CPU 3.9 ms at the typical size since 0.9.0 (was 47.6), 2.6x faster than py4DSTEM; host-copy Metal (29.0 before) now loses to it; resident Metal 4.6 ms is level |
VirtualDiffraction.pattern with weights | CPU (0.9.0); resident Metal for repeated reductions | CPU 5.1 ms at the typical size since 0.9.0 (was 61.7), 31x faster than py4DSTEM (157 ms); host-copy Metal (27.7) loses to it; resident Metal 3.6 ms |
VirtualDiffraction.median | CPU only | no Metal path; 3.4x faster than py4DSTEM at the typical size. A GPU sort per pixel needs the whole scan axis resident; revisit with the resident cube |
VirtualImaging.image | CPU only for a non-resident cube | the CPU dot product is 9.5 ms at the typical size against 21 ms with the host copy; resident Metal is not measured (open item) |
CenterOfMass.measure | CPU only | CPU 11.0 ms against Metal 26.7 at the typical size; the weighted-moment kernel is one thread per position |
CubeEditing.binScan, thinScan, padDetector, resampleDetector, medianFilterMaskedPixels, ViewedCube bin | CPU only | memory-bound copies and small per-pattern arithmetic; the copy to the GPU would cost more than the work. Faster than py4DSTEM since 0.9.0 (binScan 2.4 against 7.0 ms, the masked-pixel median 0.29 against 0.64 ms at 64 x 64; see the speed round) |
Calibration, CalibrationMeasurement, BraggMasking | CPU only | scalars and one pattern at a time |
Per-function timings of SimulatoKit against abTEM 1.0.10 on the same inputs (Al [001], 200 keV, a plane wave or a 20 mrad probe, 10 slices of 2.025 Å, about 0.1 Å per pixel, infinite projection). One warm-up, then five timed runs; each cell is the median in milliseconds with (min-max). Written by tools/benchmarks/run.sh and merge.py.
Machine: Apple M5 Pro, macOS 27.0.1, Apple Swift version 6.4 (swiftlang-6.4.0.34.1 clang-2100.3.34.1)
Python 3.12.14 (NumPy 2.5.4, SciPy 1.18.1), abTEM 1.0.10 (CPU, NumPy's FFT; no GPU)
diff --git a/suite/index.html b/suite/index.html index 506f7de..e9422db 100644 --- a/suite/index.html +++ b/suite/index.html @@ -36,14 +36,14 @@Six Swift packages that port the Python electron-microscopy stack to Apple silicon, and the two Mac apps built on them. Every number is checked against the Python package it comes from, within that package's own tolerance.
| Kit | What it does | Reference | Version | Parity: present of rows | Docs |
|---|---|---|---|---|---|
| FormatoKit | File formats for electron microscopy: Velox EMD, DM3/DM4, HDF5/EMD, MRC, TIFF, MIB, EMPAD. | RosettaSciIO | 0.12.1 | 2 of 38 | API |
| ImagoKit | Image numerics: FFT, profiles, filters, alignment, measurement, segmentation. | NumPy, SciPy, scikit-image | 0.9.1 | 36 of 93 | API |
| DiffractoKit | Crystallography, diffraction geometry, ring and ellipse fits, indexing, orientation and ACOM. | pymatgen, py4DSTEM, orix | 0.12.0 | 22 of 148 | API |
| SpectroKit | EDX quantification and EELS thickness, background and edges. | HyperSpy, exSpy | 0.13.2 | 58 of 100 | API |
| CuboKit | The 4D-STEM library: virtual imaging, centre of mass, DPC, strain, Bragg disks, parallax, ptychography. | py4DSTEM | 0.11.0 | 135 of 215 | API |
| SimulatoKit | Multislice TEM/STEM simulation: potentials, frozen phonons, scans, detectors, CTF. | abTEM | 0.5.0 | 38 of 129 | API |
| FormatoKit | File formats for electron microscopy: Velox EMD, DM3/DM4, HDF5/EMD, MRC, TIFF, MIB, EMPAD. | RosettaSciIO | 0.12.1 | 2 of 38 | API |
| ImagoKit | Image numerics: FFT, profiles, filters, alignment, measurement, segmentation. | NumPy, SciPy, scikit-image | 0.9.1 | 36 of 93 | API |
| DiffractoKit | Crystallography, diffraction geometry, ring and ellipse fits, indexing, orientation and ACOM. | pymatgen, py4DSTEM, orix | 0.12.0 | 22 of 148 | API |
| SpectroKit | EDX quantification and EELS thickness, background and edges. | HyperSpy, exSpy | 0.13.2 | 58 of 100 | API |
| CuboKit | The 4D-STEM library: virtual imaging, centre of mass, DPC, strain, Bragg disks, parallax, ptychography. | py4DSTEM | 0.11.0 | 135 of 215 | API |
| SimulatoKit | Multislice TEM/STEM simulation: potentials, frozen phonons, scans, detectors, CTF. | abTEM | 0.5.0 | 38 of 129 | API |
Beside the kits, MacEMUI 0.5.1 holds the apps' shared window, controls and Metal image view. It has no Python reference.
+Beside the kits, MacEMUI 0.5.1 holds the apps' shared window, controls and Metal image view. It has no Python reference.
"Parity: present of rows" counts the functions of the reference package that the kit has and has tested against it; each kit's docs/parity.md lists every function with its state and tolerance. See benchmarks and releases.
Apps: mac4DSTEM (4D-STEM) and macTEM (TEM images, spectroscopy and diffraction).
diff --git a/suite/releases.html b/suite/releases.html index f54e797..7dd823a 100644 --- a/suite/releases.html +++ b/suite/releases.html @@ -36,15 +36,15 @@Latest version of each kit and of MacEMUI. Tags are bare versions and equal the top entry of the package's CHANGELOG. The apps pin exact tags.
| Package | Version | Date | Tag | Changes |
|---|---|---|---|---|
| FormatoKit | 0.12.1 | 2026-11-01 | tag | changelog |
| ImagoKit | 0.9.1 | 2026-11-01 | tag | changelog |
| DiffractoKit | 0.12.0 | 2026-11-01 | tag | changelog |
| SpectroKit | 0.13.2 | 2026-11-01 | tag | changelog |
| CuboKit | 0.11.0 | 2026-11-05 | tag | changelog |
| SimulatoKit | 0.5.0 | 2026-10-31 | tag | changelog |
| MacEMUI | 0.5.1 | 2026-11-05 | tag | changelog |
| FormatoKit | 0.12.1 | 2026-11-01 | tag | changelog |
| ImagoKit | 0.9.1 | 2026-11-01 | tag | changelog |
| DiffractoKit | 0.12.0 | 2026-11-01 | tag | changelog |
| SpectroKit | 0.13.2 | 2026-11-01 | tag | changelog |
| CuboKit | 0.11.0 | 2026-11-05 | tag | changelog |
| SimulatoKit | 0.5.0 | 2026-10-31 | tag | changelog |
| MacEMUI | 0.5.1 | 2026-11-05 | tag | changelog |
The apps: mac4DSTEM releases and macTEM releases on GitHub.
+The apps: mac4DSTEM releases and macTEM releases on GitHub.
diff --git a/support.html b/support.html index f79a93c..ccc3709 100644 --- a/support.html +++ b/support.html @@ -40,8 +40,8 @@Questions, bug reports and feature requests for mac4DSTEM and macTEM: e-mail me, or open an issue on GitHub for - mac4DSTEM or - macTEM. + mac4DSTEM or + macTEM.