From dbdf5cb2664208db29c4fc93007fd6f3f6963f34 Mon Sep 17 00:00:00 2001 From: HarcoChen Date: Tue, 3 Nov 2026 10:36:50 +0800 Subject: [PATCH 1/3] docs: refresh repository presentation and benchmark visuals --- README.en.md | 156 +++++ README.md | 163 +++-- benchmarks/README.md | 83 ++- benchmarks/plot_throughput.py | 84 +++ docs/assets/benchmark-throughput.svg | 900 +++++++++++++++++++++++++++ docs/assets/hero.svg | 109 ++++ 6 files changed, 1428 insertions(+), 67 deletions(-) create mode 100644 README.en.md create mode 100644 benchmarks/plot_throughput.py create mode 100644 docs/assets/benchmark-throughput.svg create mode 100644 docs/assets/hero.svg diff --git a/README.en.md b/README.en.md new file mode 100644 index 0000000..bf5035e --- /dev/null +++ b/README.en.md @@ -0,0 +1,156 @@ +

+ ArkGuardRL-Core — a Gymnasium research environment for Stronghold Protocol +

+ +

+ 简体中文 · English +

+ +

+ Python 3.10+ + Node.js 22+ + Gymnasium 1.1+ + GPL-3.0-or-later +

+ +

+ Quick start · Performance · Environment API · Native Rust progress +

+ +**Bring Stronghold Protocol into your reinforcement learning experiments.** ArkGuardRL-Core +provides a Gymnasium interface to the unofficial +[Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) simulator, exposing +game decisions as structured observations, candidate actions, and action masks. + +| Connect experiments | Inspect decisions | Scale collection | +| --- | --- | --- | +| `gym.make("ArkGuardRL-v0")` with NumPy dictionary observations | Engine seeds, reward components, and action acceptance | `SyncVectorEnv` and shared-memory `AsyncVectorEnv` | +| Rust/PyO3 action and observation kernels | Separate termination and budget truncation | Environment-only benchmarks without Torch or a trainer | + +The current interface covers solo games with default loadouts, using **the app 0.3.1 engine (Node.js) for game execution and Rust for action and feature computation**. +A complete Rust environment is [in development](docs/NATIVE-033.md). Core owns the environment +and runtime; policy networks and training belong in downstream research projects. + +## Quick start + +### 1. Install the environment package + +You need **Python 3.10+, Node.js 22+, Git, Rust/Cargo, and system linker tools**. +A Python virtual environment is recommended. Installation builds the Rust/PyO3 extension in release mode. + +```sh +git clone https://github.com/ArkStrongholdRL/ArkGuardRL-Core.git +cd ArkGuardRL-Core +python -m pip install -e '.[gym]' +``` + +### 2. Prepare the external engine + +From this repository, clone the engine into a sibling directory: + +```sh +git clone https://github.com/sganggs/Stronghold-Protocol.git ../Stronghold-Protocol +git -C ../Stronghold-Protocol checkout c9d804f470b8d7422ae04eb66faf671cef21ad52 +``` + +`ArkGuardRL-v0` requires this exact revision (app 0.3.1), a **clean Git working tree**, and +the complete `data/*.json` files. This checkout includes the required JSON; Gym starts the engine +in the background without a web server or artwork downloads. Set `engine_root` to the engine repository root. See the +[upstream documentation](https://github.com/sganggs/Stronghold-Protocol/tree/c9d804f470b8d7422ae04eb66faf671cef21ad52) +for engine and resource information, and [NOTICE](NOTICE.md) for this project's resource boundaries. + +### 3. Run a minimal example + +Run this example from the Core repository. It samples masked actions at random; the default difficulty is `ABYSS`: + +```python +import gymnasium as gym +import arkguardrl_core.gym_env # Registers ArkGuardRL-v0. + +with gym.make("ArkGuardRL-v0", engine_root="../Stronghold-Protocol") as env: + env.action_space.seed(42) + obs, info = env.reset(seed=42) + + while True: + action = env.action_space.sample(mask=obs["action_mask"]) + obs, reward, terminated, truncated, info = env.step(action) + if terminated or truncated: + break +``` + +## Performance + +Recorded environment throughput includes observation construction, reward calculation, +Gym communication, and in-loop resets: + +![Environment throughput: six measurements and medians for one synchronous and four asynchronous environments](docs/assets/benchmark-throughput.svg) + +**Apple M5 · 16 GiB · November 2, 2026.** Uniform sampling over valid candidates, without model +inference or training. Each environment runs 64 warm-up steps and 1,024 measured decisions; +each variant has six runs. Short desktop measurements vary with load; these changes do not establish a training speedup. + +→ **[Results, measurement scope, and reproduction commands](benchmarks/README.md)** · +[Raw records](benchmarks/env-hotspots-20261102.json) · [Optimization notes](benchmarks/HOTSPOTS-20261102.md) + +## Environment API + +An action is a **row index into the current candidate set**; a row's meaning changes with the state. +Observations are dictionaries of NumPy arrays: + +| Field | Shape / type | Content | +| --- | --- | --- | +| `observation` | `(4784,) float32` | Dynamic state features | +| `candidates` | `(max_candidates, 41) float32` | Candidate action features; zero padding | +| `schedules` / `experts` | `(max_candidates,) int64` | Candidate groups | +| `action_mask` | `(max_candidates,) int8` | Current valid candidate rows | + +Candidates are constructed from public information; check `info["accepted"]` for the execution result. +The default candidate capacity is **16,384**. Overflow raises an error; selecting padding or an +out-of-range index raises `InvalidAction` without advancing the game. + +
+Seeds, rewards, episode boundaries, and vector execution + +- **Seeds:** the same Gym seed reproduces the environment. The derived engine seed is available + in `info["engine_seed"]`. `reset(seed=None)` continues the RNG stream; action sampling has its own RNG. +- **Rewards:** `NativeProgressCore` computes rewards; components are returned in `info["reward_components"]`. +- **Episode boundaries:** natural endings return `terminated`. Exhausting `max_decisions`, + `max_advances`, or `wall_seconds` returns `truncated`, a reason in `info["reason"]`, and the actual + successor observation. Engine and transport failures raise exceptions. +- **Actions:** the mask identifies candidate rows that can be submitted; the engine may still reject + execution. Unmasked random actions from a generic `check_env()` can select padding; sample with the mask. +- **Configuration:** set difficulty and budgets at construction. Single-environment `reset(options=...)` + accepts no extra configuration. Read static definitions after reset via `env.unwrapped.catalog.definitions()`. +- **Vector execution:** this repository's async benchmark sets `context="spawn"` and `shared_memory=True`. Custom entry points need + `if __name__ == "__main__"`. Each environment owns one Node process; default padded observations + use **2,984,640 bytes per environment**. + +
+ +## Documentation and development + +The detailed development and benchmark notes are currently in Chinese. + +| Guide | Contents | +| --- | --- | +| [Performance benchmarks](benchmarks/README.md) | Latest measurements, historical comparisons, reproduction, and timing scope | +| [Rust action and feature kernels](docs/RUST-ROUTES.md) | Kernel responsibilities and ABI in the installed package | +| [Complete Rust environment migration](docs/NATIVE-033.md) | Implementation scope, validation, and remaining work against app 0.3.3 | +| [Attribution and resource boundaries](NOTICE.md) | Code provenance, external engine, and resource ownership | + +Environment integration tests require the external engine and run actual games: + +```sh +ARKGUARD_ENGINE_ROOT=/path/to/Stronghold-Protocol \ + python -m unittest discover -s tests -p test_gym_env.py -v +``` + +Contributions are welcome: [report interface issues](https://github.com/ArkStrongholdRL/ArkGuardRL-Core/issues), +share reproducible environment measurements, or help with the native migration. + +## Acknowledgments and license + +Thanks to [Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) for the upstream simulator. +This is an unofficial research framework. Code is licensed under **GPL-3.0-or-later**; +game assets, data, model weights, and trajectories are not included in this repository. +See [NOTICE](NOTICE.md) and [LICENSE](LICENSE) for provenance and licensing details. diff --git a/README.md b/README.md index c825e4b..a203f52 100644 --- a/README.md +++ b/README.md @@ -1,85 +1,146 @@ -# ArkGuardRL-Core +

+ ArkGuardRL-Core — Stronghold Protocol 的 Gymnasium 研究环境 +

+ +

+ 简体中文 · English +

+ +

+ Python 3.10+ + Node.js 22+ + Gymnasium 1.1+ + GPL-3.0-or-later +

+ +

+ 快速开始 · 性能数据 · 环境接口 · Rust 迁移进度 +

+ +**把《卫戍协议:盟约》接入强化学习实验。** ArkGuardRL-Core 为非官方 +[Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) 提供 Gymnasium 接口, +将游戏决策表示为结构化观测、候选动作与动作掩码。 + +| 接入实验 | 检查决策 | 扩展采样 | +| --- | --- | --- | +| `gym.make("ArkGuardRL-v0")`,NumPy 字典观测 | 引擎种子、奖励分项、动作接受结果 | `SyncVectorEnv` / 共享内存 `AsyncVectorEnv` | +| Rust/PyO3 动作与观测内核 | 区分自然结束与预算截断 | 独立环境 benchmark,无需 Torch 或训练器 | + +当前接口覆盖单人、默认配置的游戏流程,由 **app 0.3.1 引擎(Node.js)执行游戏,Rust 加速动作与特征计算**。 +完整 Rust 环境正在[迁移](docs/NATIVE-033.md)。Core 负责环境与运行时;策略网络和训练由上层研究项目负责。 -[![Python](https://img.shields.io/badge/python-3.10%2B-3776AB?style=flat-square&logo=python&logoColor=white)](https://www.python.org/) -[![Rust](https://img.shields.io/badge/rust-PyO3%200.27-dea584?style=flat-square&logo=rust&logoColor=white)](https://www.rust-lang.org/) -[![Gymnasium](https://img.shields.io/badge/gymnasium-%E2%89%A51.1-4B8BBE?style=flat-square)](https://gymnasium.farama.org/) -[![Engine](https://img.shields.io/badge/engine-app%200.3.1-lightgrey?style=flat-square)](https://github.com/sganggs/Stronghold-Protocol) -[![License](https://img.shields.io/badge/license-GPL--3.0--or--later-blue?style=flat-square)](LICENSE) -[![Status](https://img.shields.io/badge/status-pre--release-orange?style=flat-square)](docs/NATIVE-033.md) +## 快速开始 -[Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) 的 Gymnasium 环境。 +### 1. 安装环境包 -## 安装 +准备 **Python 3.10+、Node.js 22+、Git、Rust/Cargo 与系统链接工具**。 +建议在 Python 虚拟环境中安装;安装过程会编译 Rust/PyO3 release 扩展。 -Python 3.10+、Node 22+ ```sh +git clone https://github.com/ArkStrongholdRL/ArkGuardRL-Core.git +cd ArkGuardRL-Core python -m pip install -e '.[gym]' ``` -引擎与数据单独准备:[Stronghold-Protocol](https://github.com/sganggs/Stronghold-Protocol) -`c9d804f470b8d7422ae04eb66faf671cef21ad52`(app 0.3.1) -本仓库不含游戏素材、数据、权重或轨迹。 +### 2. 准备外部引擎 + +在本仓库目录执行,将引擎放到同级目录: + +```sh +git clone https://github.com/sganggs/Stronghold-Protocol.git ../Stronghold-Protocol +git -C ../Stronghold-Protocol checkout c9d804f470b8d7422ae04eb66faf671cef21ad52 +``` -## 使用 +`ArkGuardRL-v0` 使用这个固定提交(app 0.3.1),需要**干净的 Git 工作区**及完整的 +`data/*.json`。该 checkout 已包含所需 JSON;Gym 在后台启动引擎,无需网页服务器或美术资源。 +`engine_root` 指向引擎仓库根目录;引擎与资源说明见 +[上游文档](https://github.com/sganggs/Stronghold-Protocol/tree/c9d804f470b8d7422ae04eb66faf671cef21ad52)。 +本仓库单独分发环境代码,资源边界见 [NOTICE](NOTICE.md)。 + +### 3. 运行最小示例 + +在本仓库目录运行以下示例。示例按掩码随机采样,默认难度为 `ABYSS`: ```python import gymnasium as gym import arkguardrl_core.gym_env # 注册 ArkGuardRL-v0 -with gym.make("ArkGuardRL-v0", engine_root="/path/to/engine") as env: +with gym.make("ArkGuardRL-v0", engine_root="../Stronghold-Protocol") as env: + env.action_space.seed(42) obs, info = env.reset(seed=42) - done = False - while not done: + + while True: action = env.action_space.sample(mask=obs["action_mask"]) obs, reward, terminated, truncated, info = env.step(action) - done = terminated or truncated + if terminated or truncated: + break ``` -观测是 NumPy `Dict`,动作是**当前候选行索引**: +## 性能速览 + +已有测量记录中的环境吞吐,包含观测构造、奖励计算、Gym 通信及循环内 reset: + +![环境吞吐:单环境与四环境并行的六次测量及中位数](docs/assets/benchmark-throughput.svg) -| 字段 | 形状 / 类型 | 含义 | +**Apple M5 · 16 GiB · 2026-11-02**。随机采样有效候选,无模型推理或训练; +每环境预热 64 步、测量 1,024 步,每个版本各 6 次。 +短时桌面测量存在波动,图中变化不等同于训练提速。 + +→ **[查看结果、测量边界与复现命令](benchmarks/README.md)** · +[原始记录](benchmarks/env-hotspots-20261102.json) · [优化说明](benchmarks/HOTSPOTS-20261102.md) + +## 环境接口 + +动作是**当前状态的候选行索引**,候选行的含义会随状态变化。观测为包含 NumPy 数组的字典: + +| 字段 | 形状 / 类型 | 内容 | | --- | --- | --- | -| `observation` | `(4784,) float32` | 动态观测 | -| `candidates` | `(max_candidates, 41) float32` | 候选特征,padding 为 0 | +| `observation` | `(4784,) float32` | 动态状态特征 | +| `candidates` | `(max_candidates, 41) float32` | 候选动作特征,padding 为 0 | | `schedules` / `experts` | `(max_candidates,) int64` | 候选分组 | -| `action_mask` | `(max_candidates,) int8` | 有效候选行 | +| `action_mask` | `(max_candidates,) int8` | 当前有效候选行 | -- 候选由公开信息构造;引擎仍可能拒绝合法候选,结果见 `info["accepted"]`。 -- padding 或越界索引抛 `InvalidAction`,不替换动作、不推进游戏。 -- 候选容量默认 16384,超限报错,不裁剪动作域。 -- 自然结束为 `terminated`;`max_decisions` / `max_advances` / `wall_seconds` 耗尽为 - `truncated`(`info["reason"]`)且返回真实后继观测;引擎/传输错误抛异常。 -- 奖励由 `NativeProgressCore` 计算,分项在 `info["reward_components"]`。 -- 相同 Gym 种子可复现,实际引擎种子见 `info["engine_seed"]`;`reset(seed=None)` 延续 RNG 流。 -- 静态定义通过 `env.unwrapped.catalog.definitions()` 读取。候选依赖游戏状态, - 通用 `check_env()` 的无掩码随机动作会选中 padding,请按掩码采样。 -- 难度与预算只能在构造时配置,`reset(options=...)` 不接受其他选项。 +候选由公开信息构造,执行结果通过 `info["accepted"]` 检查。 +默认候选容量为 **16,384**,超限报错;padding 或越界索引抛出 `InvalidAction`,且不推进游戏。 -## 并行与测量 +
+种子、奖励、结束条件与并行语义 -支持 `SyncVectorEnv` / `AsyncVectorEnv`。Async 使用 `spawn` 和共享内存,入口需要 -`if __name__ == "__main__"`。每个 env 独占一个 Node 进程,默认填充观测约 2.98 MB/env。 +- **种子**:相同 Gym 种子可复现环境;实际引擎种子见 `info["engine_seed"]`。 + `reset(seed=None)` 延续 RNG 流,动作采样使用单独的随机源。 +- **奖励**:由 `NativeProgressCore` 计算,分项见 `info["reward_components"]`。 +- **结束**:自然结束返回 `terminated`;`max_decisions`、`max_advances` 或 `wall_seconds` + 耗尽返回 `truncated`,原因见 `info["reason"]`,同时保留真实后继观测。引擎与传输错误抛异常。 +- **动作**:掩码标记可提交的候选行,引擎仍可能拒绝执行。通用 `check_env()` 的无掩码随机动作 + 可能选中 padding,请按掩码采样。 +- **配置**:难度与预算在构造时设置;单环境 `reset(options=...)` 不接受额外配置。 + 静态定义在 reset 后通过 `env.unwrapped.catalog.definitions()` 读取。 +- **并行**:本仓库的异步基准使用 `context="spawn"` 和 `shared_memory=True`;自定义入口需要 `if __name__ == "__main__"`。 + 每个 env 独占一个 Node 进程;默认填充观测为 **2,984,640 字节/env**。 -```sh -python benchmarks/gym_env.py --engine-root /path/to/engine \ - --vector async --envs 4 --steps 256 --repeats 3 --out artifacts/gym-async-4.json -``` +
+ +## 文档与开发 -## 测试 +| 入口 | 内容 | +| --- | --- | +| [性能基准](benchmarks/README.md) | 最新记录、历史对照、复现命令与计时边界 | +| [Rust 动作与特征内核](docs/RUST-ROUTES.md) | 当前安装包的内核职责与 ABI | +| [完整 Rust 环境迁移](docs/NATIVE-033.md) | 对照 app 0.3.3 的实现范围、验证与待完成工作 | +| [来源与资源边界](NOTICE.md) | 代码来源、外部引擎与资源归属 | -原生集成测试需显式提供引擎路径,会实际运行对局: +环境集成测试需要外部引擎,会实际运行对局: ```sh -ARKGUARD_ENGINE_ROOT=/path/to/engine python -m unittest discover -s tests -p test_gym_env.py -v +ARKGUARD_ENGINE_ROOT=/path/to/Stronghold-Protocol \ + python -m unittest discover -s tests -p test_gym_env.py -v ``` -## 文档 - -| 文档 | 内容 | -| --- | --- | -| [性能测量](benchmarks/README.md) | 负载定义、测量边界与基线数据 | -| [Rust 内核](docs/RUST-ROUTES.md) | 动作/观测/路线内核的职责与 ABI | -| [原生迁移状态](docs/NATIVE-033.md) | 0.3.3 Rust 引擎实现范围与开发命令 | +欢迎通过 [Issues](https://github.com/ArkStrongholdRL/ArkGuardRL-Core/issues) +报告接口问题、提交可复现的性能测量,或参与原生迁移。 +## 致谢与许可 -GPL-3.0-or-later;来源与外部资源边界见 [NOTICE](NOTICE.md) 与 [LICENSE](LICENSE)。 +感谢 [Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) 提供上游模拟实现。 +本项目是非官方研究框架,代码采用 **GPL-3.0-or-later**;游戏素材、数据、权重和轨迹不随仓库分发。 +代码来源与外部资源许可详见 [NOTICE](NOTICE.md) 和 [LICENSE](LICENSE)。 diff --git a/benchmarks/README.md b/benchmarks/README.md index 8a6e988..22e3b37 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -1,22 +1,71 @@ -# 性能测量 +# 性能基准 -独立 env 测量使用 `.[gym]` 安装与 `gym_env.py`,不加载模型、Torch、PPO 或轨迹采集器: +[项目主页](../README.md) · [最新结果](#最新结果) · [测量边界](#测量边界) · [运行基准](#运行基准) · [历史对照](#历史对照) + +测量 Gymnasium 环境的端到端决策吞吐,为运行时优化提供可复查的基线。 +脚本均匀抽样当前有效候选,不加载学习策略或训练器。 + +## 最新结果 + +**2026-11-02 · Apple M5 / 16 GiB · 0.3.1 引擎(Node.js)+ Rust 动作/观测内核。** +Python 3.12.13、Node.js 26.7.0、Gymnasium 1.4.0、NumPy 2.5.4。 +本次测量对应当前可运行的混合运行时,不代表纯 Rust 0.3.3 环境。 + +![环境吞吐:六次重复的散点及中位数;单环境旧版480.57、新版494.75决策每秒,四环境并行旧版899.14、新版937.19决策每秒。](../docs/assets/benchmark-throughput.svg) + +| 配置 | 优化前中位数(决策/秒) | 优化后中位数(决策/秒) | 本次变化 | +| --- | ---: | ---: | ---: | +| Sync,1 env | 480.57 | 494.75 | +2.95% | +| Async,共享内存,4 env | 899.14 | 937.19 | +4.23% | + +每个环境预热 64 步、测量 1024 步;每批重复 3 次,按旧/新/新/旧顺序运行,各版本合计 6 次。 +Gym seed 为 42、采样 seed 为 123,候选容量为 16384。重复从相同种子重新开始,图中散点反映计时波动。 +**桌面负载有波动,这些小幅变化不能证明稳定的训练提速。** + +[原始数值与校验记录](env-hotspots-20261102.json) · [优化内容与测量说明](HOTSPOTS-20261102.md) + +## 测量边界 + +- **单位是决策/秒。** 吞吐为全部环境的真实决策数除以墙钟时间;不是战斗帧率或完整对局数。 +- **计入循环内 reset。** 观测填充、奖励计算、动作抽样和 Gym 通信也在计时范围内;初始 reset、启动和预热单独记录,不进入吞吐分母。 +- **每个环境有独立采样 RNG。** 不截断候选域;超出容量时报告错误。自然结束与预算截断分别计数,reset 不算一步。 +- **分项时间是所有环境的累计值。** 并行环境的分项耗时不能当作墙钟时间占比,也不能把这些结果等同于模型训练或策略采集吞吐。 +- **短时桌面测量有适用范围。** 记录未列操作系统版本、CPU 核心配置及前后代码提交;本页保留原始测量口径,不将它扩展为跨硬件或跨版本结论。 + +## 运行基准 + +在仓库根目录执行;将 `ARKGUARD_ENGINE_ROOT` 改为固定版本、干净的 0.3.1 引擎 Git 仓库根目录。 +安装及引擎准备见[项目主页](../README.md)。 ```sh -python benchmarks/gym_env.py --engine-root /path/to/engine \ - --vector sync --envs 1 --steps 256 --repeats 3 --out artifacts/gym-sync-1.json -python benchmarks/gym_env.py --engine-root /path/to/engine \ - --vector async --envs 4 --steps 256 --repeats 3 --out artifacts/gym-async-4.json +python -m pip install -e '.[gym]' +export ARKGUARD_ENGINE_ROOT=/absolute/path/to/engine + +python benchmarks/gym_env.py --engine-root "$ARKGUARD_ENGINE_ROOT" \ + --vector sync --envs 1 --warmup 64 --steps 1024 --repeats 3 \ + --seed 42 --rng-seed 123 --max-candidates 16384 --max-decisions 1024 \ + --out artifacts/gym-sync-1.json + +python benchmarks/gym_env.py --engine-root "$ARKGUARD_ENGINE_ROOT" \ + --vector async --envs 4 --warmup 64 --steps 1024 --repeats 3 \ + --seed 42 --rng-seed 123 --max-candidates 16384 --max-decisions 1024 \ + --out artifacts/gym-async-4.json ``` -动作在有效候选行中均匀抽样,每个 env 使用独立 RNG。每次重复从相同游戏/RNG 种子重新开始; -不截断候选域。吞吐计入循环内 reset,单列初始 reset 与启动/预热时间。`steps` 是每个 env 的真实 -决策次数,reset 不计作 step;自然结束与预算截断分别计数。分项时间是所有 env 的累计值, -异步运行时不能当作墙钟时间占比。观测填充与 Gym 通信也计入测量,因此结果不是纯引擎速度。 -它与下面使用学习策略的完整采集 benchmark 负载不同,不直接比较倍率。 +每条命令测量当前检出版本,产生 3 次重复。前后对照需在相应版本分别执行,并保留运行顺序; +上表每个版本运行了两批。完整参数与输出字段见 [`gym_env.py`](gym_env.py)。 -2026-10-28,Apple M5 / 16 GB,Gymnasium 1.4.0,独立无 Torch 环境:每个 env -1024 步,预热 16 步,重复 3 次,Gym seed 42、采样 seed 123,默认候选容量 16384。 +图表仅从已提交的 JSON 生成,需要 Matplotlib,不会启动环境: + +```sh +python benchmarks/plot_throughput.py +``` + +## 历史对照 + +**2026-10-28:同一轮测量中的向量化比较。** Apple M5 / 16 GB、Gymnasium 1.4.0, +独立无 Torch 环境;每个环境 1024 步、预热 16 步、重复 3 次,Gym seed 42、采样 seed 123, +候选容量 16384。该轮预热与重复次数不同,且没有可直接对照的代码提交记录,**不能跨两轮计算提速或退化倍率**。 | 配置 | 中位吞吐(决策/秒) | 每次测量墙钟中位数 | | --- | ---: | ---: | @@ -24,7 +73,9 @@ python benchmarks/gym_env.py --engine-root /path/to/engine \ | Sync,4 env | 451 | 9.08 秒 / 4096 步 | | Async,共享内存,4 env | 982 | 4.17 秒 / 4096 步 | -4 env 的 Sync/Async 动作、奖励和结束标志摘要一致,每次均自然结束 18 局、截断 0 局; +同一负载的 4 env 对照中,Async 吞吐为 Sync 的 **2.18 倍**。 +动作、奖励与结束标志摘要一致,每次均自然结束 18 局、截断 0 局。 剩余轨迹不一定结束,因此不能用总耗时除以结束局数估算单局耗时。 -Async 相对同负载 Sync 吞吐为 2.18 倍。单 env 的约 2.24 毫秒/步包含观测、奖励及 Gym 开销。 -默认填充观测为每 env 2,984,640 字节;本次最多 401 个候选,不代表所有对局的候选上限。 + +单环境约 2.24 毫秒/步,包含观测、奖励及 Gym 开销。默认填充观测为每环境 2,984,640 字节, +这是数组大小,不是进程内存占用;该轮最多出现 401 个候选,不代表所有对局的候选上限。 diff --git a/benchmarks/plot_throughput.py b/benchmarks/plot_throughput.py new file mode 100644 index 0000000..198e492 --- /dev/null +++ b/benchmarks/plot_throughput.py @@ -0,0 +1,84 @@ +"""Render the recorded throughput results; does not import or run the environment.""" +import json +from pathlib import Path +import xml.etree.ElementTree as ET + +import matplotlib + +matplotlib.use("Agg") +import matplotlib.pyplot as plt +from matplotlib.lines import Line2D + + +root = Path(__file__).resolve().parents[1] +source = root / "benchmarks/env-hotspots-20261102.json" +output = root / "docs/assets/benchmark-throughput.svg" +data = json.loads(source.read_text()) +plt.rcParams.update({"font.family": "DejaVu Sans", "svg.fonttype": "path", + "svg.hashsalt": "arkguardrl-throughput", "font.size": 10}) +fig, ax = plt.subplots(figsize=(11, 5.4), facecolor="white") +fig.subplots_adjust(left=.16, right=.82, bottom=.24, top=.73) +fig.text(.055, .91, "Environment throughput", fontsize=21, weight="bold", color="#0f172a") +fig.text(.055, .85, "0.3.1 engine (Node.js) + Rust action / observation kernels | 2026-11-02", + fontsize=10.5, color="#475569") +colors = {"before": "#64748b", "after": "#0f766e"} +fills = {"before": "#e2e8f0", "after": "#ccfbf1"} +fig.legend(handles=[Line2D([], [], color=colors[key], lw=7, label=label) + for key, label in (("before", "Before"), ("after", "After"))], + loc="upper left", bbox_to_anchor=(.05, .81), frameon=False, ncol=2, fontsize=10) +fig.text(.84, .765, "MEDIAN", fontsize=9, color="#64748b") +jitter = [-.06, .06, -.10, .10, -.03, .03] +for config, title, top in (("sync-1", "1 environment · Sync", 3.25), + ("async-4", "4 environments · Async / shared memory", 1.35)): + item = data["measurements"][config] + ax.text(-.15, top + .48, title, transform=ax.get_yaxis_transform(), + color="#0f172a", weight="bold", fontsize=10.5) + for variant, offset in (("before", 0), ("after", -.65)): + y = top + offset + values, median = item[f"{variant}_rates"], item["medians"][variant] + ax.barh(y, median, height=.24, color=fills[variant], zorder=2) + ax.scatter(values, [y + j for j in jitter], color=colors[variant], + s=31, edgecolor="white", linewidth=.6, zorder=4) + ax.vlines(median, y - .20, y + .20, color=colors[variant], linewidth=2.4, zorder=5) + ax.text(-.025, y, variant.title(), transform=ax.get_yaxis_transform(), + ha="right", va="center", color="#475569", fontsize=9.5) + ax.text(1.035, y, f"{median:,.2f}", transform=ax.get_yaxis_transform(), + va="center", color=colors[variant], fontsize=12, weight="bold") +ax.set(xlim=(0, 1050), ylim=(.15, 3.95), yticks=[], xticks=[0, 250, 500, 750, 1000]) +ax.set_xlabel("Total environment decisions per second", color="#475569", labelpad=10) +ax.grid(axis="x", color="#e2e8f0", linewidth=.8, zorder=0) +ax.tick_params(axis="x", length=0, pad=7, colors="#64748b") +for spine in ax.spines.values(): + spine.set_visible(False) +fig.text(.055, .125, "Dots: all 6 runs per version. Vertical ticks: medians. | Apple M5 / 16 GiB", + fontsize=9.5, color="#475569") +fig.text(.055, .075, "1,024 decisions/env · 64 warm-up steps · loop resets included · no model or training", + fontsize=9, color="#64748b") +fig.text(.055, .03, "Shared-desktop timings vary; small gains do not establish a stable training speedup.", + fontsize=9, color="#64748b") +output.parent.mkdir(parents=True, exist_ok=True) +fig.savefig(output, format="svg", metadata={"Date": None, "Creator": "ArkGuardRL-Core"}) +plt.close(fig) + +ns = "http://www.w3.org/2000/svg" +ET.register_namespace("", ns) +ET.register_namespace("xlink", "http://www.w3.org/1999/xlink") +tree = ET.parse(output) +svg = tree.getroot() +svg.set("role", "img") +svg.set("aria-labelledby", "benchmark-title benchmark-desc") +title = ET.Element(f"{{{ns}}}title", id="benchmark-title") +title.text = "Environment throughput on 2026-11-02: six runs and medians" +desc = ET.Element(f"{{{ns}}}desc", id="benchmark-desc") +desc.text = ("Apple M5, 16 GiB. Engine 0.3.1 on Node.js with Rust action/observation kernels. " + "Uniform candidate sampling, no model or training. The zero-based horizontal axis " + "shows total decisions per second; dots show all six repeated timings per version. " + "Single environment median: 480.57 before, 494.75 after. Four asynchronous environments " + "with shared memory: 899.14 before, 937.19 after. Each run measures 1024 decisions " + "per environment after 64 warm-up steps. Loop resets count; initial reset and setup " + "do not. Shared-desktop variation prevents claiming a stable training speedup. " + "Source: benchmarks/env-hotspots-20261102.json.") +svg.insert(0, title) +svg.insert(1, desc) +tree.write(output, encoding="utf-8", xml_declaration=True) +print(output) diff --git a/docs/assets/benchmark-throughput.svg b/docs/assets/benchmark-throughput.svg new file mode 100644 index 0000000..b1ea540 --- /dev/null +++ b/docs/assets/benchmark-throughput.svg @@ -0,0 +1,900 @@ + + + Environment throughput on 2026-11-02: six runs and mediansApple M5, 16 GiB. Engine 0.3.1 on Node.js with Rust action/observation kernels. Uniform candidate sampling, no model or training. The zero-based horizontal axis shows total decisions per second; dots show all six repeated timings per version. Single environment median: 480.57 before, 494.75 after. Four asynchronous environments with shared memory: 899.14 before, 937.19 after. Each run measures 1024 decisions per environment after 64 warm-up steps. Loop resets count; initial reset and setup do not. Shared-desktop variation prevents claiming a stable training speedup. Source: benchmarks/env-hotspots-20261102.json. + + + + image/svg+xml + + + ArkGuardRL-Core + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/assets/hero.svg b/docs/assets/hero.svg new file mode 100644 index 0000000..1aeca99 --- /dev/null +++ b/docs/assets/hero.svg @@ -0,0 +1,109 @@ + + ArkGuardRL Core — a Gymnasium environment for Stronghold Protocol + A navy blue banner with the ArkGuardRL Core wordmark, structured observations, candidate actions, and parallel rollouts. An original abstract tactical grid on the right illustrates branching decisions and a highlighted route; it is not a game screenshot. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ARKSTRONGHOLDRL + + ArkGuardRL + CORE + Stronghold Protocol · Gymnasium environment + + + + Structured observations + + Candidate actions + + Parallel rollouts + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From ef3d8e7cebf2f1abe2d0779d29ea1cae9889de70 Mon Sep 17 00:00:00 2001 From: HarcoChen Date: Tue, 3 Nov 2026 10:41:20 +0800 Subject: [PATCH 2/3] docs: normalize generated SVG whitespace --- benchmarks/plot_throughput.py | 1 + docs/assets/benchmark-throughput.svg | 50 ++++++++++++++-------------- 2 files changed, 26 insertions(+), 25 deletions(-) diff --git a/benchmarks/plot_throughput.py b/benchmarks/plot_throughput.py index 198e492..0403630 100644 --- a/benchmarks/plot_throughput.py +++ b/benchmarks/plot_throughput.py @@ -81,4 +81,5 @@ svg.insert(0, title) svg.insert(1, desc) tree.write(output, encoding="utf-8", xml_declaration=True) +output.write_text("\n".join(line.rstrip() for line in output.read_text().splitlines()) + "\n") print(output) diff --git a/docs/assets/benchmark-throughput.svg b/docs/assets/benchmark-throughput.svg index b1ea540..0f9a60e 100644 --- a/docs/assets/benchmark-throughput.svg +++ b/docs/assets/benchmark-throughput.svg @@ -31,7 +31,7 @@ - + @@ -46,7 +46,7 @@ - + @@ -64,7 +64,7 @@ - + @@ -78,7 +78,7 @@ - + @@ -95,7 +95,7 @@ - + @@ -108,7 +108,7 @@ - + @@ -183,7 +183,7 @@ - + @@ -224,7 +224,7 @@ - + @@ -239,7 +239,7 @@ - + @@ -258,7 +258,7 @@ - + @@ -271,7 +271,7 @@ - + @@ -285,7 +285,7 @@ - + @@ -336,7 +336,7 @@ - + @@ -347,7 +347,7 @@ - + @@ -358,7 +358,7 @@ - + @@ -368,7 +368,7 @@ - + @@ -441,7 +441,7 @@ - + @@ -474,7 +474,7 @@ - + @@ -570,7 +570,7 @@ - + @@ -587,7 +587,7 @@ - + @@ -673,7 +673,7 @@ - + @@ -768,7 +768,7 @@ - + @@ -867,7 +867,7 @@ - + @@ -881,7 +881,7 @@ - + @@ -897,4 +897,4 @@ - \ No newline at end of file + From b2e7e7e6f11093edb9131f264c592067aad0bc6e Mon Sep 17 00:00:00 2001 From: HarcoChen Date: Wed, 11 Nov 2026 00:19:27 +0800 Subject: [PATCH 3/3] Reshape bilingual READMEs around a concise research landing page --- README.en.md | 170 ++++++++++++++++++++++----------------------------- README.md | 160 +++++++++++++++++++++--------------------------- 2 files changed, 142 insertions(+), 188 deletions(-) diff --git a/README.en.md b/README.en.md index bf5035e..195eb75 100644 --- a/README.en.md +++ b/README.en.md @@ -1,76 +1,64 @@ -

- ArkGuardRL-Core — a Gymnasium research environment for Stronghold Protocol -

+

- 简体中文 · English + ArkGuardRL-Core — Stronghold Protocol research environment

+

Stronghold Protocol · Reinforcement Learning Environment

+ +

简体中文 · English

+ +

Run complete solo games, connect a learning policy, and inspect each decision.

+

- Python 3.10+ - Node.js 22+ - Gymnasium 1.1+ GPL-3.0-or-later + Python 3.12 + Rust and PyO3 + Gymnasium 1.1+

-

- Quick start · Performance · Environment API · Native Rust progress -

+

Quick start · Environment API · Performance · Documentation · License

+ +## Overview -**Bring Stronghold Protocol into your reinforcement learning experiments.** ArkGuardRL-Core -provides a Gymnasium interface to the unofficial -[Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) simulator, exposing -game decisions as structured observations, candidate actions, and action masks. +ArkGuardRL-Core exposes the purchasing, team-building, and deployment decisions of Stronghold Protocol through Gymnasium. +Public observations, candidate actions, rewards, and episode boundaries let a policy drive complete games through `reset` and `step`. +Game rules follow the Stronghold Protocol simulator. [Upstream project](https://github.com/sganggs/Stronghold-Protocol) -| Connect experiments | Inspect decisions | Scale collection | -| --- | --- | --- | -| `gym.make("ArkGuardRL-v0")` with NumPy dictionary observations | Engine seeds, reward components, and action acceptance | `SyncVectorEnv` and shared-memory `AsyncVectorEnv` | -| Rust/PyO3 action and observation kernels | Separate termination and budget truncation | Environment-only benchmarks without Torch or a trainer | +| Core runtime | Policy research | +| --- | --- | +| Game execution, public observations, candidate enumeration, and numeric encoding | Model representations, policy networks, and learning algorithms | +| Seeded reproduction, decision budgets, and vector collection | Trajectory collection, training, and policy evaluation | +| Maintained in this repository | Maintained in [ArkGuardRL](https://github.com/HarcoChen/ArkGuardRL) | -The current interface covers solo games with default loadouts, using **the app 0.3.1 engine (Node.js) for game execution and Rust for action and feature computation**. -A complete Rust environment is [in development](docs/NATIVE-033.md). Core owns the environment -and runtime; policy networks and training belong in downstream research projects. +Maintenance targets solo Desperation mode (`mode_single_abyss`). This branch's quick start uses **the 0.3.1 Node engine and Rust feature kernels**. +The 0.3.3 native runtime, Rust slots, and public spatial APIs are developed in PR #4. [Native runtime PR](https://github.com/ArkStrongholdRL/ArkGuardRL-Core/pull/4) ## Quick start -### 1. Install the environment package - -You need **Python 3.10+, Node.js 22+, Git, Rust/Cargo, and system linker tools**. -A Python virtual environment is recommended. Installation builds the Rust/PyO3 extension in release mode. +Use Python 3.12, Node.js 22+, Git, and Rust/Cargo. Installation compiles a PyO3 extension and requires system linker tools. ```sh git clone https://github.com/ArkStrongholdRL/ArkGuardRL-Core.git cd ArkGuardRL-Core python -m pip install -e '.[gym]' -``` - -### 2. Prepare the external engine -From this repository, clone the engine into a sibling directory: - -```sh git clone https://github.com/sganggs/Stronghold-Protocol.git ../Stronghold-Protocol git -C ../Stronghold-Protocol checkout c9d804f470b8d7422ae04eb66faf671cef21ad52 ``` -`ArkGuardRL-v0` requires this exact revision (app 0.3.1), a **clean Git working tree**, and -the complete `data/*.json` files. This checkout includes the required JSON; Gym starts the engine -in the background without a web server or artwork downloads. Set `engine_root` to the engine repository root. See the -[upstream documentation](https://github.com/sganggs/Stronghold-Protocol/tree/c9d804f470b8d7422ae04eb66faf671cef21ad52) -for engine and resource information, and [NOTICE](NOTICE.md) for this project's resource boundaries. +Keep the external engine checkout clean with all `data/*.json` files present. Set `engine_root` to its repository root. +This revision includes the required JSON files. The environment starts the engine in the background. -### 3. Run a minimal example - -Run this example from the Core repository. It samples masked actions at random; the default difficulty is `ABYSS`: +### Run a game ```python import gymnasium as gym -import arkguardrl_core.gym_env # Registers ArkGuardRL-v0. +import arkguardrl_core.gym_env with gym.make("ArkGuardRL-v0", engine_root="../Stronghold-Protocol") as env: env.action_space.seed(42) obs, info = env.reset(seed=42) - while True: action = env.action_space.sample(mask=obs["action_mask"]) obs, reward, terminated, truncated, info = env.step(action) @@ -78,79 +66,65 @@ with gym.make("ArkGuardRL-v0", engine_root="../Stronghold-Protocol") as env: break ``` -## Performance +The example samples candidates using the mask. Replace `action` with the candidate row selected by your policy. -Recorded environment throughput includes observation construction, reward calculation, -Gym communication, and in-loop resets: +## Environment API -![Environment throughput: six measurements and medians for one synchronous and four asynchronous environments](docs/assets/benchmark-throughput.svg) +| Interface | Use | +| --- | --- | +| `ArkGuardRL-v0` | A single-environment Gymnasium decision loop | +| `SyncVectorEnv` / `AsyncVectorEnv` | Vector collection with an independent session per environment | +| `env.unwrapped.catalog.definitions()` | Read public static definitions | +| `info` | Inspect action acceptance, reward components, engine seeds, and episode reasons | -**Apple M5 · 16 GiB · November 2, 2026.** Uniform sampling over valid candidates, without model -inference or training. Each environment runs 64 warm-up steps and 1,024 measured decisions; -each variant has six runs. Short desktop measurements vary with load; these changes do not establish a training speedup. +Observations are dictionaries of NumPy arrays. Action indices refer to the current state's candidate rows. The default capacity is 16,384. -→ **[Results, measurement scope, and reproduction commands](benchmarks/README.md)** · -[Raw records](benchmarks/env-hotspots-20261102.json) · [Optimization notes](benchmarks/HOTSPOTS-20261102.md) +| Field | Shape and type | +| --- | --- | +| `observation` | `(4784,) float32` | +| `candidates` | `(max_candidates, 41) float32` | +| `schedules` / `experts` | `(max_candidates,) int64` | +| `action_mask` | `(max_candidates,) int8` | -## Environment API +
+Actions, seeds, and rollout semantics -An action is a **row index into the current candidate set**; a row's meaning changes with the state. -Observations are dictionaries of NumPy arrays: +- Candidates are constructed from public information. Check `info["accepted"]` for execution results. Padding or out-of-range indices raise `InvalidAction`. Candidate overflow raises an error. +- The same Gym seed reproduces the environment. `info["engine_seed"]` exposes the derived seed. `reset(seed=None)` continues the RNG stream. Action sampling uses a separate RNG. +- `NativeProgressCore` computes rewards. Components appear in `info["reward_components"]`. +- Natural endings return `terminated`. Exhausting `max_decisions`, `max_advances`, or `wall_seconds` returns `truncated`, the actual successor observation, and `info["reason"]`. Engine or transport failures raise exceptions. +- Set difficulty and budgets at construction. Single-environment `reset(options=...)` accepts an empty configuration. +- Async collection with `spawn` requires an `if __name__ == "__main__"` entry point. Each v0 environment owns an independent Node process. -| Field | Shape / type | Content | -| --- | --- | --- | -| `observation` | `(4784,) float32` | Dynamic state features | -| `candidates` | `(max_candidates, 41) float32` | Candidate action features; zero padding | -| `schedules` / `experts` | `(max_candidates,) int64` | Candidate groups | -| `action_mask` | `(max_candidates,) int8` | Current valid candidate rows | +
-Candidates are constructed from public information; check `info["accepted"]` for the execution result. -The default candidate capacity is **16,384**. Overflow raises an error; selecting padding or an -out-of-range index raises `InvalidAction` without advancing the game. +## Performance -
-Seeds, rewards, episode boundaries, and vector execution - -- **Seeds:** the same Gym seed reproduces the environment. The derived engine seed is available - in `info["engine_seed"]`. `reset(seed=None)` continues the RNG stream; action sampling has its own RNG. -- **Rewards:** `NativeProgressCore` computes rewards; components are returned in `info["reward_components"]`. -- **Episode boundaries:** natural endings return `terminated`. Exhausting `max_decisions`, - `max_advances`, or `wall_seconds` returns `truncated`, a reason in `info["reason"]`, and the actual - successor observation. Engine and transport failures raise exceptions. -- **Actions:** the mask identifies candidate rows that can be submitted; the engine may still reject - execution. Unmasked random actions from a generic `check_env()` can select padding; sample with the mask. -- **Configuration:** set difficulty and budgets at construction. Single-environment `reset(options=...)` - accepts no extra configuration. Read static definitions after reset via `env.unwrapped.catalog.definitions()`. -- **Vector execution:** this repository's async benchmark sets `context="spawn"` and `shared_memory=True`. Custom entry points need - `if __name__ == "__main__"`. Each environment owns one Node process; default padded observations - use **2,984,640 bytes per environment**. +| Configuration | Decisions per second | +| --- | ---: | +| Gym Sync, 1 environment | 494.75 | +| Gym Async, shared memory, 4 environments | 937.19 | -
+Recorded on November 2, 2026, with an Apple M5 / 16 GiB and the Node 0.3.1 engine plus Rust feature kernels. +Each environment runs 64 warm-up steps and 1,024 measured steps. Values are medians of six measurements. +Timing includes observations, rewards, Gym communication, and in-loop resets, with masked random actions. +Raw records, measurement plots, and reproduction commands are available in the benchmark guide. [Performance benchmarks](benchmarks/README.md) -## Documentation and development +## Documentation -The detailed development and benchmark notes are currently in Chinese. +Detailed development notes are currently in Chinese. | Guide | Contents | | --- | --- | -| [Performance benchmarks](benchmarks/README.md) | Latest measurements, historical comparisons, reproduction, and timing scope | -| [Rust action and feature kernels](docs/RUST-ROUTES.md) | Kernel responsibilities and ABI in the installed package | -| [Complete Rust environment migration](docs/NATIVE-033.md) | Implementation scope, validation, and remaining work against app 0.3.3 | -| [Attribution and resource boundaries](NOTICE.md) | Code provenance, external engine, and resource ownership | +| [Performance benchmarks](benchmarks/README.md) | Measurements, timing scope, and commands | +| [Rust feature kernels](docs/RUST-ROUTES.md) | Action and observation encoding responsibilities and ABI | +| [Native migration notes](docs/NATIVE-033.md) | Development history for the 0.3.3 engine | +| [Issue tracker](https://github.com/ArkStrongholdRL/ArkGuardRL-Core/issues) | Interface issues and reproducible reports | -Environment integration tests require the external engine and run actual games: - -```sh -ARKGUARD_ENGINE_ROOT=/path/to/Stronghold-Protocol \ - python -m unittest discover -s tests -p test_gym_env.py -v -``` +## License and acknowledgments -Contributions are welcome: [report interface issues](https://github.com/ArkStrongholdRL/ArkGuardRL-Core/issues), -share reproducible environment measurements, or help with the native migration. +Code is licensed under GPL-3.0-or-later. Stronghold Protocol provides the upstream simulator. See the license and attribution notes for game assets and external data. [LICENSE](LICENSE) · [NOTICE](NOTICE.md) -## Acknowledgments and license +The landing-page layout draws on AutoTavern. The banner is an original schematic illustration. [AutoTavern](https://github.com/MCG-NJU/AutoTavern) -Thanks to [Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) for the upstream simulator. -This is an unofficial research framework. Code is licensed under **GPL-3.0-or-later**; -game assets, data, model weights, and trajectories are not included in this repository. -See [NOTICE](NOTICE.md) and [LICENSE](LICENSE) for provenance and licensing details. +

Back to top

diff --git a/README.md b/README.md index a203f52..6916a51 100644 --- a/README.md +++ b/README.md @@ -1,74 +1,64 @@ -

- ArkGuardRL-Core — Stronghold Protocol 的 Gymnasium 研究环境 -

+

- 简体中文 · English + ArkGuardRL-Core — Stronghold Protocol research environment

+

卫戍协议 · 强化学习环境

+ +

简体中文 · English

+ +

复现完整单人对局,接入学习策略,检查每一步的决策结果。

+

- Python 3.10+ - Node.js 22+ - Gymnasium 1.1+ GPL-3.0-or-later + Python 3.12 + Rust and PyO3 + Gymnasium 1.1+

-

- 快速开始 · 性能数据 · 环境接口 · Rust 迁移进度 -

+

快速开始 · 环境接口 · 性能 · 文档 · 许可与致谢

+ +## 概览 -**把《卫戍协议:盟约》接入强化学习实验。** ArkGuardRL-Core 为非官方 -[Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) 提供 Gymnasium 接口, -将游戏决策表示为结构化观测、候选动作与动作掩码。 +ArkGuardRL-Core 将《卫戍协议:盟约》的购买、构筑和部署决策接入 Gymnasium。 +环境提供公开状态、候选动作、奖励与结束标志,策略代码可以通过标准 `reset`/`step` 循环驱动完整对局。 +游戏规则基于 Stronghold Protocol 的模拟实现。[上游项目](https://github.com/sganggs/Stronghold-Protocol) -| 接入实验 | 检查决策 | 扩展采样 | -| --- | --- | --- | -| `gym.make("ArkGuardRL-v0")`,NumPy 字典观测 | 引擎种子、奖励分项、动作接受结果 | `SyncVectorEnv` / 共享内存 `AsyncVectorEnv` | -| Rust/PyO3 动作与观测内核 | 区分自然结束与预算截断 | 独立环境 benchmark,无需 Torch 或训练器 | +| Core 环境运行时 | 策略研究 | +| --- | --- | +| 游戏推进、公开观测、候选枚举与数值编码 | 模型表示、策略网络与学习算法 | +| 种子复现、预算控制与多环境采样 | 轨迹采集、训练与策略评估 | +| 在本仓库维护 | 在 [ArkGuardRL](https://github.com/HarcoChen/ArkGuardRL) 维护 | -当前接口覆盖单人、默认配置的游戏流程,由 **app 0.3.1 引擎(Node.js)执行游戏,Rust 加速动作与特征计算**。 -完整 Rust 环境正在[迁移](docs/NATIVE-033.md)。Core 负责环境与运行时;策略网络和训练由上层研究项目负责。 +维护范围为单人绝境模式(`mode_single_abyss`)。本分支的快速开始对应 **0.3.1 Node 引擎 + Rust 特征内核**。 +0.3.3 原生运行时、Rust slots 和公开空间接口集中在 PR #4。[原生运行时 PR](https://github.com/ArkStrongholdRL/ArkGuardRL-Core/pull/4) ## 快速开始 -### 1. 安装环境包 - -准备 **Python 3.10+、Node.js 22+、Git、Rust/Cargo 与系统链接工具**。 -建议在 Python 虚拟环境中安装;安装过程会编译 Rust/PyO3 release 扩展。 +使用 Python 3.12、Node.js 22+、Git 和 Rust/Cargo。安装时编译 PyO3 扩展,需要系统链接工具。 ```sh git clone https://github.com/ArkStrongholdRL/ArkGuardRL-Core.git cd ArkGuardRL-Core python -m pip install -e '.[gym]' -``` - -### 2. 准备外部引擎 - -在本仓库目录执行,将引擎放到同级目录: -```sh git clone https://github.com/sganggs/Stronghold-Protocol.git ../Stronghold-Protocol git -C ../Stronghold-Protocol checkout c9d804f470b8d7422ae04eb66faf671cef21ad52 ``` -`ArkGuardRL-v0` 使用这个固定提交(app 0.3.1),需要**干净的 Git 工作区**及完整的 -`data/*.json`。该 checkout 已包含所需 JSON;Gym 在后台启动引擎,无需网页服务器或美术资源。 -`engine_root` 指向引擎仓库根目录;引擎与资源说明见 -[上游文档](https://github.com/sganggs/Stronghold-Protocol/tree/c9d804f470b8d7422ae04eb66faf671cef21ad52)。 -本仓库单独分发环境代码,资源边界见 [NOTICE](NOTICE.md)。 - -### 3. 运行最小示例 +保持外部引擎 checkout 干净,并保留完整的 `data/*.json`。`engine_root` 指向其仓库根目录。 +该版本包含所需 JSON,环境会在后台启动引擎。 -在本仓库目录运行以下示例。示例按掩码随机采样,默认难度为 `ABYSS`: +### 运行一局 ```python import gymnasium as gym -import arkguardrl_core.gym_env # 注册 ArkGuardRL-v0 +import arkguardrl_core.gym_env with gym.make("ArkGuardRL-v0", engine_root="../Stronghold-Protocol") as env: env.action_space.seed(42) obs, info = env.reset(seed=42) - while True: action = env.action_space.sample(mask=obs["action_mask"]) obs, reward, terminated, truncated, info = env.step(action) @@ -76,71 +66,61 @@ with gym.make("ArkGuardRL-v0", engine_root="../Stronghold-Protocol") as env: break ``` -## 性能速览 +示例按掩码随机选择候选。将 `action` 替换为策略选出的候选行索引,即可接入自己的决策器。 -已有测量记录中的环境吞吐,包含观测构造、奖励计算、Gym 通信及循环内 reset: +## 环境接口 -![环境吞吐:单环境与四环境并行的六次测量及中位数](docs/assets/benchmark-throughput.svg) +| 接口 | 用途 | +| --- | --- | +| `ArkGuardRL-v0` | 单环境 Gymnasium 决策循环 | +| `SyncVectorEnv`/`AsyncVectorEnv` | 多环境采样,每个环境持有独立会话 | +| `env.unwrapped.catalog.definitions()` | 读取公开静态定义 | +| `info` | 检查动作接受结果、奖励分项、引擎种子及结束原因 | -**Apple M5 · 16 GiB · 2026-11-02**。随机采样有效候选,无模型推理或训练; -每环境预热 64 步、测量 1,024 步,每个版本各 6 次。 -短时桌面测量存在波动,图中变化不等同于训练提速。 +观测为 NumPy 字典。动作索引引用当前状态的候选行,候选容量默认 16,384。 -→ **[查看结果、测量边界与复现命令](benchmarks/README.md)** · -[原始记录](benchmarks/env-hotspots-20261102.json) · [优化说明](benchmarks/HOTSPOTS-20261102.md) +| 字段 | 形状与类型 | +| --- | --- | +| `observation` | `(4784,) float32` | +| `candidates` | `(max_candidates, 41) float32` | +| `schedules`/`experts` | `(max_candidates,) int64` | +| `action_mask` | `(max_candidates,) int8` | -## 环境接口 +
+动作、种子与轨迹语义 -动作是**当前状态的候选行索引**,候选行的含义会随状态变化。观测为包含 NumPy 数组的字典: +- 候选根据公开信息构造,动作接受结果见 `info["accepted"]`。padding 或越界索引抛出 `InvalidAction`,候选超限报错。 +- 相同 Gym 种子可复现环境,`info["engine_seed"]` 给出派生种子。`reset(seed=None)` 延续 RNG 流,动作采样使用独立 RNG。 +- 奖励由 `NativeProgressCore` 计算,分项见 `info["reward_components"]`。 +- 自然结束返回 `terminated`。`max_decisions`、`max_advances` 或 `wall_seconds` 耗尽时返回 `truncated`、真实后继观测和 `info["reason"]`。引擎或传输错误抛异常。 +- 难度与预算在构造时设置。单环境 `reset(options=...)` 接受空配置。 +- Async 使用 `spawn` 时,入口需要 `if __name__ == "__main__"`。当前 v0 每环境持有独立 Node 进程。 -| 字段 | 形状 / 类型 | 内容 | -| --- | --- | --- | -| `observation` | `(4784,) float32` | 动态状态特征 | -| `candidates` | `(max_candidates, 41) float32` | 候选动作特征,padding 为 0 | -| `schedules` / `experts` | `(max_candidates,) int64` | 候选分组 | -| `action_mask` | `(max_candidates,) int8` | 当前有效候选行 | +
-候选由公开信息构造,执行结果通过 `info["accepted"]` 检查。 -默认候选容量为 **16,384**,超限报错;padding 或越界索引抛出 `InvalidAction`,且不推进游戏。 +## 性能 -
-种子、奖励、结束条件与并行语义 - -- **种子**:相同 Gym 种子可复现环境;实际引擎种子见 `info["engine_seed"]`。 - `reset(seed=None)` 延续 RNG 流,动作采样使用单独的随机源。 -- **奖励**:由 `NativeProgressCore` 计算,分项见 `info["reward_components"]`。 -- **结束**:自然结束返回 `terminated`;`max_decisions`、`max_advances` 或 `wall_seconds` - 耗尽返回 `truncated`,原因见 `info["reason"]`,同时保留真实后继观测。引擎与传输错误抛异常。 -- **动作**:掩码标记可提交的候选行,引擎仍可能拒绝执行。通用 `check_env()` 的无掩码随机动作 - 可能选中 padding,请按掩码采样。 -- **配置**:难度与预算在构造时设置;单环境 `reset(options=...)` 不接受额外配置。 - 静态定义在 reset 后通过 `env.unwrapped.catalog.definitions()` 读取。 -- **并行**:本仓库的异步基准使用 `context="spawn"` 和 `shared_memory=True`;自定义入口需要 `if __name__ == "__main__"`。 - 每个 env 独占一个 Node 进程;默认填充观测为 **2,984,640 字节/env**。 +| 配置 | 决策/秒 | +| --- | ---: | +| Gym Sync,1 环境 | 494.75 | +| Gym Async,共享内存,4 环境 | 937.19 | -
+2026-11-02,Apple M5/16 GiB,Node 0.3.1 + Rust 特征内核。每环境预热 64 步、测量 1,024 步,表中为六次测量中位数。 +计时包含观测、奖励、Gym 通信与循环内 reset,动作按掩码随机采样。完整原始记录、分布图和复现命令见性能文档。[性能基准](benchmarks/README.md) -## 文档与开发 +## 文档 -| 入口 | 内容 | +| 文档 | 内容 | | --- | --- | -| [性能基准](benchmarks/README.md) | 最新记录、历史对照、复现命令与计时边界 | -| [Rust 动作与特征内核](docs/RUST-ROUTES.md) | 当前安装包的内核职责与 ABI | -| [完整 Rust 环境迁移](docs/NATIVE-033.md) | 对照 app 0.3.3 的实现范围、验证与待完成工作 | -| [来源与资源边界](NOTICE.md) | 代码来源、外部引擎与资源归属 | +| [性能基准](benchmarks/README.md) | 测量结果、计时边界与运行命令 | +| [Rust 特征内核](docs/RUST-ROUTES.md) | 动作与观测编码的职责及 ABI | +| [原生迁移笔记](docs/NATIVE-033.md) | 0.3.3 引擎的开发记录 | +| [问题反馈](https://github.com/ArkStrongholdRL/ArkGuardRL-Core/issues) | 接口问题与可复现报告 | -环境集成测试需要外部引擎,会实际运行对局: - -```sh -ARKGUARD_ENGINE_ROOT=/path/to/Stronghold-Protocol \ - python -m unittest discover -s tests -p test_gym_env.py -v -``` +## 许可与致谢 -欢迎通过 [Issues](https://github.com/ArkStrongholdRL/ArkGuardRL-Core/issues) -报告接口问题、提交可复现的性能测量,或参与原生迁移。 +代码采用 GPL-3.0-or-later。Stronghold Protocol 提供上游模拟实现,游戏素材与外部数据的来源和使用边界见许可说明。[LICENSE](LICENSE) · [NOTICE](NOTICE.md) -## 致谢与许可 +首页编排参考 AutoTavern,横幅为本项目原创示意图。[AutoTavern](https://github.com/MCG-NJU/AutoTavern) -感谢 [Stronghold Protocol](https://github.com/sganggs/Stronghold-Protocol) 提供上游模拟实现。 -本项目是非官方研究框架,代码采用 **GPL-3.0-or-later**;游戏素材、数据、权重和轨迹不随仓库分发。 -代码来源与外部资源许可详见 [NOTICE](NOTICE.md) 和 [LICENSE](LICENSE)。 +

回到顶部