diff --git a/.env.example b/.env.example index cd8149c..2747a42 100644 --- a/.env.example +++ b/.env.example @@ -12,6 +12,10 @@ VIEW_PASS= ALLOW_SELF_APPROVAL=false HISTORY_KEEP=5000 +# The history also stops at this size (MB): each change keeps the edited document before and after it +# HISTORY_MAX_MB=200 +# Largest document saved at once (MB): an IS holds its VLANs and all its flows, about 300 bytes a flow +# MAX_DOC_MB=8 READ_ONLY=false BACKUP_KEEP=14 BACKUP_EVERY_HOURS=24 @@ -54,6 +58,7 @@ HOST=0.0.0.0 # SSO_HEADER_TRUST=127.0.0.1 # Local accounts once single sign-on is on: all (default), admins (break-glass only) or off # LOCAL_LOGIN=all +# How long a browser stays signed in, single sign-on and local accounts alike (hours) # SESSION_HOURS=8 # Connectors (Cyberwatch, GLPI, phpIPAM, Palo Alto, and modules). Set to false to turn them off. diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a6569e..045fea4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,42 @@ All notable changes to Strata. Versions follow [Semantic Versioning](https://semver.org/). +## [Unreleased] + +## [0.11.0] - 2026-11-08 +### Upgrading +- **HTTP Basic is gone.** Local accounts sign in with the form of the sign-in page and get a session cookie (`SESSION_HOURS`, 8 by default). Scripts that sent `curl -u name:password` get `401` with a message: give them an API token (*Account › API tokens*) and send `Authorization: Bearer stk_…`. `GET /?local` is gone too. +- Sessions are kept in memory: restarting Strata signs everyone out. + +### Added +- **Sign out** for every account, in the title block (it was in *Account*, for single sign-on only). +- **Import VLANs** (*Edit › Import VLANs*, in an IS): a CSV (header in English or French, `,` `;` or tab), a JSON list (`[{num, name, cidr, zone}]`, `{"vlans": […]}`, NetBox's `{"results": […]}`) or a Strata export. Tick the VLANs you want, filter, and choose the zone of each, or of all the ticked ones at once (an existing zone, a zone named in the file, or a new one). A VLAN whose number is already in the IS is updated when ticked. From a Strata export, the assets of the chosen VLANs and the flows between them come too. +- **Arrange VLANs** (*Layout*): each zone on rows and columns instead of one long row, the VLANs that talk to each other side by side (the most traffic to the plates left of and behind each one), VLANs without flows last. *VLANs per row*: automatic (about the square root of their number) or 3 to 12. Saved for everyone; *Line up* undoes it for a zone. +- **The Internet cloud can be moved**: dragged in *Arrange plates* (3D) and with *Move* in the 2D diagram. Its place is saved on the IS (`si.cloud`, `si.cloud2d`); the named Internet services and the flows follow; *Put Internet back* returns it to its default place. +- **Show all or no VLANs** of the IS, or of one zone (*All* / *None* next to each zone of the plate list). +- **Hide empty VLANs** (*Plates*): the plates without an asset leave the 3D model and the 2D diagram. Remembered by the browser. +- **A loading bar** at the top of the screen for any request still running after 300 ms, with the progress of batches of writes and imports. +- **Memory and CPU needs**, measured from an empty map to 30 000 assets (README, section 1), and resource limits, commented out, in `docker-compose.yml`. +- **The overview shows each IS in the compact form of its model**: each zone of the small model takes the rows and the order of *Arrange VLANs* (or of plates placed by hand); a zone never arranged keeps one row up to 8 VLANs, then wraps into a grid; the assets of a plate wrap into lines; past 5 zones the floors come closer. The small models share their geometries, lighter for a large IS. +- **The title block folds into one line** (▴ in its corner): the name of the IS or *Strata*, the four key figures, back to the overview, sign out and the incident band stay; the controls move up with it. Remembered by the browser. +- **Palo Alto: build the IS from a configuration** (*Build the IS*, a box of the file import): zones from the security zones, VLANs from the layer-3 sub-interfaces (tag, subnet, comment), assets from the host address objects (in the VLAN of their IP, the type from their name or description, else the ports opened to them, else a workstation when they only open connections) and the firewall itself, crossed by the flows that change VLAN; then the rules as flows, as before. What the map has is kept; the same file again changes nothing. On the 991 objects of the test configuration, 90% get the right type. +- Palo Alto: an import that draws nothing no longer opens an empty proposal. +- **Suggest zones from the flows** (*Edit*, and *Layout*): the zones of an IS from its flows, the Internet on top. Each VLAN goes to the floor of its distance from the Internet along the flows (*Exposed to the Internet*, *Reached from the exposed zone*, *Back end, 3 hops or more*); the VLANs the Internet never reaches go below by the role of their assets (internal servers, users, industrial, network and administration); empty VLANs stay where they are. Existing zones that hold most of the same VLANs keep their name. A window shows the result top first: rename a zone, move a VLAN, then *Apply* (approvers) or *Propose* (contributors: a draft proposal opens). +- **CSV VLANs**, a module in the store: the VLANs of an IS from a CSV (number, name, range, in any order, with or without a header). After the file, a window lists its VLANs: tick those to bring, filter, and choose the zone of each, or of all the ticked ones at once (a zone of the IS, a zone named in the file, or a new one). The result is a proposal, which any contributor can make; from a script, `picks` gives the same choice. +- **Modules can ask for a choice before an import** (`file.pick` in the manifest, a `preview(cfg, ctx)` function, `ctx.picks` in `run`, `POST /api/connectors/{id}/preview`): Strata shows what the file holds, people tick what to bring and, with `pick.zones`, the zone of each. Backward compatible: modules without `pick` work as before. See MODULES.md. +- **Graphics for small laptops** (*Display › Graphics*: Auto, Quality, Economy, and *Animated flows*). Economy: screen resolution, no shadows, no moving dots, labels hidden sooner from afar. Auto, the default, is Quality until moving the view turns out slow on this computer (then Economy, remembered by the browser), and Economy when the browser draws without a graphics card. +- **API**: `POST /auth/login` (the form, or JSON `{user, password}`); `analysis.js` exports `arrangeGrid` and `vlansFromFile`; `window.strataRenderStats()` in the page gives what the last frame cost. + +### Fixed +- **A large IS could no longer be saved** (“This item is too large to save”): one document was limited to 512 KB, and an IS holds all its flows, about 300 bytes each, so an IS of some 1 800 flows or more, imported through a proposal, could no longer be edited from the interface. The limit is now `MAX_DOC_MB`, 8 MB by default. +- **The history no longer grows without bound with a large IS**: each change keeps the edited document before and after it, about a megabyte a change for an IS of 2 000 flows, and the history is held in memory. It now also stops at `HISTORY_MAX_MB` (200 MB by default), the oldest changes first. + +### Changed +- **The 3D view is much lighter to draw**, and looks the same: 8 to 15 times fewer draw calls (1 569 → 208 for the Lyon demo) and half the triangles. It is drawn only when something changes, not 60 times a second; shadows are drawn again only when the model changes; the view is a little less sharp while it moves. Each type of equipment is one shared geometry; every flow of an IS is in one geometry with its own colour and opacity, and the moving dots are one instanced mesh; label textures are shared; labels too small to read are not drawn; flows no longer cast shadows; a rebuilt model frees what the GPU held for the previous one. +- Changing a password, or disabling or deleting an account, ends that account's other sessions. *End all sessions* (admin) ends the sessions of local accounts too, and keeps the admin's own. +- The Content-Security-Policy allows forms to post to Strata itself (`form-action 'self'`), for the sign-in form. +- `tests/e2e.mjs` takes `CHROMIUM_PATH`, a Chromium already installed. + ## [0.10.0] - 2026-10-28 ### Added - **Incident mode, with Muster** (*Incident* in the controls of an IS). Mark assets compromised, suspected or contained, from their sheet or from a playbook decision. The incident sheet then shows: @@ -218,6 +254,7 @@ All notable changes to Strata. Versions follow [Semantic Versioning](https://sem ## [0.1.0] - First release: 3D map of IS, zones, VLANs, assets, flows, exposure, vulnerabilities and incident contacts. +[0.11.0]: https://github.com/SolidGeeks/strata/releases/tag/v0.11.0 [0.10.0]: https://github.com/SolidGeeks/strata/releases/tag/v0.10.0 [0.9.0]: https://github.com/SolidGeeks/strata/releases/tag/v0.9.0 [0.8.0]: https://github.com/SolidGeeks/strata/releases/tag/v0.8.0 diff --git a/MODULES.md b/MODULES.md index a0ed480..c83c252 100644 --- a/MODULES.md +++ b/MODULES.md @@ -28,8 +28,9 @@ has; administrators also see a *Module store* tab. | Cyberwatch | Built-in | Servers and their CVEs (API v3) | Approvers | | GLPI | Built-in | Computers, network devices, printers, phones, software, people in charge | Approvers | | phpIPAM | Built-in | Sections, subnets, VLANs, addresses | Approvers | -| Palo Alto | Built-in | Allow rules, as flows: from the XML API, or a configuration file | Approvers; any contributor for a file | +| Palo Alto | Built-in | Allow rules, as flows: from the XML API, or a configuration file. *Build the IS*: also the zones, the VLANs (layer-3 sub-interfaces), the assets (host address objects) and the firewall | Approvers; any contributor for a file | | CSV inventory | Store | Assets of an IS from a spreadsheet saved as CSV | Any contributor (file import) | +| CSV VLANs | Store | VLANs of an IS (number, name, range): you choose the VLANs and the zone of each | Any contributor (file import) | | Nmap scan | Store | Hosts, systems and open ports from `nmap -oX` | Any contributor (file import) | | NetBox | Store | Devices, virtual machines, VLANs and prefixes | Approvers | @@ -378,6 +379,32 @@ export async function run(cfg, ctx) { } ``` +### Letting people choose before an import: `file.pick` and `preview()` + +Some files hold more than people want to bring: a list of every VLAN of the company, a whole CMDB +export. Add `pick` to `file` in the manifest, and export `preview(cfg, ctx)`: + +```json +"file": { "label": "Import VLANs from a CSV", "fields": ["site"], "role": "contributor", + "pick": { "title": "Choose the VLANs and their zone", "zones": true, "help": "Tick the VLANs to bring…" } } +``` + +1. People choose the file and the settings of `file.fields`, then *Next*: Strata calls + `preview(cfg, ctx)` (with `ctx.file`, and the map if the module reads it). Nothing is recorded. +2. `preview` returns `{items, summary, skipped}`. Each item: `key` (unique, given back in `ctx.picks`), + `label`, `detail`, `tags` and `warn` (short notes, shown as badges), `on` (ticked at first; default + `true`), `zone` (with `pick.zones`: the name of the zone suggested for it, an existing one or one to + create). At most 5000 items. +3. Strata shows the items: tick, filter, and with `pick.zones` choose a zone of the IS for each, or for + all the ticked ones at once (an existing zone, a suggested one, or a new one). `pick.zones` needs a + setting of type `site` in `file.fields`: the IS whose zones are offered. +4. *Import* runs `run(cfg, ctx)` with `ctx.picks`, and the result is a proposal, as usual. + +From a script: `POST /api/connectors/{id}/preview` with `{file, overrides}` answers `{items, summary, +skipped, pick}`; `POST /api/connectors/{id}/run` takes `picks` next to `file`. A Strata older than 0.11 +ignores `pick` and runs the import with `ctx.picks` undefined: handle that case as "no choice". +[`modules/csv-vlans`](modules/csv-vlans) is the example. + `test(cfg, ctx)` checks the access and returns a sentence for the person who clicked *Test the connection*. `run(cfg, ctx)` reads and returns `{proposals, summary, report}`: @@ -398,6 +425,7 @@ connection*. `run(cfg, ctx)` reads and returns `{proposals, summary, report}`: |---|---| | `ctx.state` | The map, `{si, assets, wan, services}`, when the manifest asks for it (`permissions.map`), `null` otherwise. A copy: changing it changes nothing. | | `ctx.file` | `{name, text}` for a file import, `null` for a sync. | +| `ctx.picks` | For a module with `file.pick`: what people chose after the preview, `{key: {zone}}` (the keys of the items chosen; `zone` with `pick.zones`: a zone id of the IS, or `"new:"` for a zone to create). `null` when no choice was made (a script that left `picks` out, or a sync): then do what the settings say. | | `ctx.sdk` | The helpers below. | | `ctx.log(...)` | A line in the server log, prefixed with the module id. | diff --git a/README.md b/README.md index 4a8064f..23da65e 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Strata — v0.10 +# Strata — v0.11 [![Release](https://img.shields.io/github/v/release/SolidGeeks/strata)](https://github.com/SolidGeeks/strata/releases) [![License AGPL-3.0](https://raw.githubusercontent.com/SolidGeeks/.github/main/assets/badges/license-agpl.svg)](LICENSE) [![Signed releases](https://raw.githubusercontent.com/SolidGeeks/.github/main/assets/badges/signed-releases.svg)](#2-quick-start-with-docker) [![Tests](https://github.com/SolidGeeks/strata/actions/workflows/ci.yml/badge.svg)](https://github.com/SolidGeeks/strata/actions/workflows/ci.yml) @@ -16,10 +16,21 @@ Strata is on its way to **1.0.0**: from then on, the API, the export format, the | [0.8.0](https://github.com/SolidGeeks/strata/milestone/2) ✅ | Connector modules and a module store, three.js 0.186. Released on 2026-10-26 | | [0.9.0](https://github.com/SolidGeeks/strata/milestone/3) ✅ | The last breaking changes: single sign-on, pre-releases, an API reference, the old product name removed. Released on 2026-10-26 | | 0.10.0 ✅ | Incident mode with Muster, the risk register with baselines, logs for the SIEM, the steering dashboard. Released on 2026-10-28 | +| 0.11.0 ✅ | Cookie sessions and sign-out, VLAN import with zones, Arrange VLANs and zones suggested from the flows, Palo Alto builds an IS, a lighter 3D view for small laptops. Released on 2026-11-08 | | [1.0.0](https://github.com/SolidGeeks/strata/milestone/4) | Release candidates (`1.0.0-rc.N`), then 1.0.0 | Follow it on the [milestones page](https://github.com/SolidGeeks/strata/milestones). Ideas for after 1.0 are discussed, and voted on, in [Discussions](https://github.com/orgs/SolidGeeks/discussions/categories/ideas). Want to help? Start with a [good first issue](https://github.com/SolidGeeks/strata/issues?q=is%3Aopen+label%3A%22good+first+issue%22). +## New in v0.11: sign-out, VLAN import, zones from the flows, a lighter 3D view + +- **Sessions and sign-out**: local accounts sign in with the form and get a session cookie; *Sign out* is in the title block for everyone. HTTP Basic is gone: scripts use API tokens (read *Upgrading* in the [CHANGELOG](CHANGELOG.md)). +- **Import VLANs** (*Edit › Import VLANs*): a CSV, a JSON list or a Strata export; tick the VLANs to bring and choose the zone of each. The **CSV VLANs** module of the store does the same as a proposal, and modules can now ask for such a choice before an import (`file.pick`, see [MODULES.md](MODULES.md)). +- **Arrange VLANs** lays each zone out on rows and columns by its flows; **Suggest zones from the flows** proposes the zones of an IS, the Internet on top, in a window to review before applying. +- **Palo Alto builds the IS** from a configuration alone: zones, VLANs from the sub-interfaces, assets from the address objects, the firewall, then the flows. +- **On the map**: show all or no VLANs per zone, hide empty VLANs, move the Internet cloud, fold the title block, a loading bar for long requests; the overview shows each IS in its compact form. +- **A lighter 3D view**: 8 to 15 times fewer draw calls, drawn only when something changes, and an Economy mode chosen automatically on slow laptops (*Display › Graphics*). +- **Large IS**: one document can now reach `MAX_DOC_MB` (8 MB), and the history stops at `HISTORY_MAX_MB` (200 MB). + ## New in v0.10: incident mode with Muster, the risk register, the steering dashboard - **Incident mode, with Muster** (*Incident* in the controls of an IS): mark assets compromised, suspected or contained; the incident sheet lists who to call, the flows to cut and the firewall where to cut each, the business services and assets that depend on them, what the attacker reaches next. A Muster incident opens on the map and its playbook decisions point at the assets. Each asset also has anonymised *AI prompts*: its major risks, and a guide for an AI to write the response playbooks of its plausible incidents in Muster's format. @@ -176,6 +187,41 @@ The machine needs access to the npm registry **once**, during the build, to down Users need a browser with WebGL 2 and JavaScript modules: Chrome or Edge 80 and later, Firefox 114 and later, Safari 15 and later (three.js has needed WebGL 2 since 2024). +### Memory and CPU + +Measured on Node.js 22 (the Docker image), one server process, maps generated with 2 flows per asset and a vulnerability on one asset in four. *Peak* is during an import of the whole map, ten browsers opening it at once, 50 edits and the dashboard. + +| Map | Server RAM, at rest → peak | Opening the map (10 browsers at once) | One edit | Dashboard (12 months) | +|---|---|---|---|---| +| Empty | 57 → 73 MB | — | — | — | +| Demo size: 5 IS, ~150 assets | 83 MB | 13 ms | 2 ms | 4 ms | +| 10 IS, 200 VLANs, 5 000 assets (2 MB of JSON) | 125 MB | 0.14 s | 17 ms | 70 ms | +| 20 IS, 1 000 VLANs, 30 000 assets (11.5 MB of JSON) | 290 → 340 MB | 0.9 s | 90 ms | 0.35 s | + +| Size of the map | Server: vCPU | Server: RAM | Disk (`/data`) | +|---|---|---|---| +| Up to ~5 000 assets | 0.5 | 256 MB | 1 GB | +| Up to ~30 000 assets | 1 to 2 | 512 MB (1 GB with connectors running) | 5 GB | + +The server is idle between requests: CPU is spent on writes (each one saves the map and appends to the history) and on the dashboard. The history is kept in memory too, up to `HISTORY_MAX_MB` (200 MB by default): count that much more RAM for an IS of thousands of flows edited often, or lower it. Each connector or module run starts a separate Node.js process, about 50 MB for as long as it runs. Disk: the map, the history (`HISTORY_KEEP` entries) and `BACKUP_KEEP` daily copies of the map, so roughly the size of the JSON export × (`BACKUP_KEEP` + 2), plus the history. + +**The browser does most of the work** (the 3D model, the 2D diagram, the analysis): 8 GB of RAM and WebGL 2 are enough, an integrated GPU included. An IS of 300 assets and 600 flows takes about 150 MB of browser memory. `docker-compose.yml` has the server limits, commented out. + +**Small laptops.** The 3D view is drawn only when something changes (the view, the data, a filter), sharper once the view stops moving, and *Display › Graphics* has three modes: *Quality*, *Economy* (screen resolution, no shadows, no moving dots on the flows, labels hidden sooner from afar) and *Auto*, the default, which is Quality until the view turns out slow on this computer, or when the browser draws without a graphics card. *Animated flows* turns the moving dots on or off in any mode. Measured on the same machine, without a GPU: + +| | Draw calls per frame | Triangles | Frames per second while turning the view | Frames drawn in 3 s, idle | +|---|---|---|---|---| +| Demo Lyon (34 assets), before 0.11 | 1 569 | 187 000 | 5 | 15 | +| … 0.11, Quality | 208 | 85 000 | 8 | only those of the moving dots (30 a second) | +| … 0.11, Economy | 173 | 30 000 | 35 | 2 | +| 120 assets, 260 flows, before 0.11 | 5 731 | 929 000 | 1.7 | 6 | +| … 0.11, Quality | 397 | 400 000 | 2.8 | only those of the moving dots | +| … 0.11, Economy | 365 | 134 000 | 21 | 2 | + +**Laying out a large IS.** *Layout › Suggest zones* proposes zones from the flows, the Internet on top (distance from the Internet along the flows, then the role of the assets for what the Internet never reaches); *Arrange VLANs* then puts each zone on rows and columns. Both are reviewed or can be undone (*History*). + +Without a GPU (a virtual desktop, a remote session), Auto chooses Economy; for very large IS, the 2D diagram and *Plates › None*, then the VLANs you need, stay the lightest. + --- ## 2. Quick start with Docker @@ -194,7 +240,7 @@ Or use the prebuilt image published with each release (tag `vX.Y.Z`, see the `Re Every published image is **signed**: an attestation proves it was built by this repository's `Release` workflow, from the code of the tag. To check it before deploying (you need the [GitHub CLI](https://cli.github.com/)): ```bash -gh attestation verify oci://ghcr.io/solidgeeks/strata:0.10.0 --owner SolidGeeks +gh attestation verify oci://ghcr.io/solidgeeks/strata:0.11.0 --owner SolidGeeks ``` The command must print `Verification succeeded!`. If it fails, do not use the image, and report it (see [SECURITY.md](SECURITY.md)). Images older than 0.5.4 are not signed. @@ -224,6 +270,8 @@ By default, the port only listens on the machine itself (`127.0.0.1:8080`). To o | `EDIT_PASS_FILE` / `VIEW_PASS_FILE` | — | Read the password from this file instead (Docker secrets, systemd credentials, sops-nix, agenix). The file wins over the variable; a trailing newline is ignored. If the file cannot be read, the server does not start. | | `ALLOW_SELF_APPROVAL` | `false` | `true` lets an approver approve their own proposals. | | `HISTORY_KEEP` | `5000` | Number of changes kept in the history. | +| `HISTORY_MAX_MB` | `200` | The history also stops at this size: each change keeps the whole edited document before and after it, so editing a large IS often (moving plates, importing flows) adds about twice its size each time. The oldest changes go first. | +| `MAX_DOC_MB` | `8` | Largest document saved at once. An IS holds its VLANs and all its flows (about 300 bytes a flow: 8 MB is some 25 000 flows). | | `READ_ONLY` | `false` | Freezes the map for everyone. | | `BACKUP_KEEP` | `14` | Number of automatic backups kept. | | `BACKUP_EVERY_HOURS` | `24` | How often automatic backups are made. | @@ -268,7 +316,7 @@ SSO_ROLE_CONTRIBUTOR=it-ops SSO_DEFAULT_ROLE=viewer # everyone else; empty: no access ``` -Opening Strata then shows a sign-in page: *Sign in with Microsoft Entra ID*, and *Sign in with a local account* if local accounts are allowed. The groups come from the `groups` claim (`SSO_GROUPS_CLAIM`; for Entra ID, add the groups claim to the token, or use app roles with `SSO_GROUPS_CLAIM=roles`), the user name from `preferred_username` (`SSO_USER_CLAIM`). Strata checks the ID token's signature against the provider's keys, its issuer, audience, expiry and nonce, and uses PKCE. *Account › Sign out* ends the session at the provider too. +Opening Strata then shows a sign-in page: *Sign in with Microsoft Entra ID*, and the form for a local account under it if local accounts are allowed. The groups come from the `groups` claim (`SSO_GROUPS_CLAIM`; for Entra ID, add the groups claim to the token, or use app roles with `SSO_GROUPS_CLAIM=roles`), the user name from `preferred_username` (`SSO_USER_CLAIM`). Strata checks the ID token's signature against the provider's keys, its issuer, audience, expiry and nonce, and uses PKCE. *Sign out* (in the title block, or *Account*) ends the session at the provider too. **A sign-in proxy you already run** (Authelia, oauth2-proxy, Pomerium, Authentik outpost): `SSO_HEADER_USER=Remote-User`, `SSO_HEADER_GROUPS=Remote-Groups` and `SSO_HEADER_TRUST`, the address the proxy connects to Strata from (`127.0.0.1` when it runs on the same machine; several addresses comma-separated, no range: anything in a range could claim to be anyone). The same `SSO_ROLE_*` apply. The proxy must set the headers itself and drop any coming from the browser. `TRUST_PROXY` stays what it was, for the client address. @@ -276,13 +324,13 @@ Opening Strata then shows a sign-in page: *Sign in with Microsoft Entra ID*, and **Names.** The history and the proposals name people by the name the provider gives. A name that is also a local account is refused (rename one of them), and an e-mail address is used only once the provider has verified it. Prefer a name people cannot change themselves at the provider (`SSO_USER_CLAIM=oid` on Entra ID, or `sub`), or make sure they cannot. -**Sessions** last `SESSION_HOURS` (8 by default) with the role given at sign-in. *Account › End all single sign-on sessions* (admin) signs everyone out, after a group change or when someone leaves. +**Sessions** last `SESSION_HOURS` (8 by default), for single sign-on and for local accounts alike: a local account signs in once with the form of the sign-in page and gets a session cookie. *Sign out*, in the title block, ends it. Changing a password, or disabling or deleting an account, ends that account's other sessions. *Account › End all sessions* (admin) signs everyone else out, after a group change or when someone leaves. Sessions are kept in memory: restarting Strata signs everyone out. -**API tokens** for scripts and integrations: *Account › API tokens* (admin), with a name, a role and a validity of 1 to 730 days. The token (`stk_…`) is shown once and stored hashed; send it as `Authorization: Bearer stk_…`. Basic Auth with a local account keeps working for scripts. +**API tokens** for scripts and integrations: *Account › API tokens* (admin), with a name, a role and a validity of 1 to 730 days. The token (`stk_…`) is shown once and stored hashed; send it as `Authorization: Bearer stk_…`. Since 0.11, scripts can no longer send a local account's password with HTTP Basic: the API answers 401 and says to use a token. ## 5. Network access and HTTPS -Passwords travel with Basic Auth: **never expose the app over plain HTTP outside the machine**. +Passwords travel in the sign-in form and sessions in a cookie: **never expose the app over plain HTTP outside the machine**. Set `PUBLIC_URL=https://…` behind HTTPS, so the session cookie is `Secure` and `__Host-` prefixed. ### Option A — the bundled HTTPS proxy (Caddy) @@ -458,7 +506,7 @@ The data lives in the Docker volume and is not touched by the rebuild. Before up The full reference, with every route's role, request and answer, the data format, the module API and the compatibility policy, is the [API reference](https://solidgeeks.github.io/strata/api.html) ([`docs/api.html`](docs/api.html)). -Every route needs the same credentials as the interface (a local account with Basic Auth, an API token as `Authorization: Bearer stk_…`, or the session of a signed-in browser), with the rights of the account's role (direct writes need `approver`). Writes require the headers `Content-Type: application/json` and `X-Requested-With: strata`. A script can group several writes into a single history entry with the headers `X-Batch: ` and `X-Batch-Label: `. To have a tool propose instead of write, give it a `contributor` account and use `POST /api/proposals` with `{"title", "siteId", "changes": {"si": {...}, "assets": {...}}}` (a document set to `null` = deletion). +Every route needs the same credentials as the interface (an API token as `Authorization: Bearer stk_…`, or the session of a signed-in browser; HTTP Basic is no longer accepted since 0.11), with the rights of the account's role (direct writes need `approver`). Writes require the headers `Content-Type: application/json` and `X-Requested-With: strata`. A script can group several writes into a single history entry with the headers `X-Batch: ` and `X-Batch-Label: `. To have a tool propose instead of write, give it a `contributor` account and use `POST /api/proposals` with `{"title", "siteId", "changes": {"si": {...}, "assets": {...}}}` (a document set to `null` = deletion). | Method and route | What it does | |---|---| @@ -484,11 +532,12 @@ Every route needs the same credentials as the interface (a local account with Ba | `GET/POST /api/proposals`, `GET/PUT/DELETE /api/proposals/{id}` | Proposals; `POST …/{id}/submit`, `/withdraw`, `/comment`, `/approve` (`{"force": true}` on a conflict), `/reject` | | `GET /api/users`, `PUT/DELETE /api/users/{name}` | Accounts (admin); `PUT /api/me/password`: `{"current","next"}` (local accounts) | | `GET/POST /api/tokens`, `DELETE /api/tokens/{id}` | API tokens (admin): `POST` `{"name", "role", "days"}` answers the token once | -| `GET /auth/login`, `/auth/oidc/start`, `POST /auth/logout` | Sign-in page, single sign-on, sign-out (`{"location"}` to go to) | +| `GET /auth/login`, `POST /auth/login`, `/auth/oidc/start`, `POST /auth/logout` | Sign-in page; a local account's sign-in (the form, or JSON `{"user", "password"}`: sets the session cookie; 401 wrong, 429 locked); single sign-on; sign-out (`{"location"}` to go to) | | `GET /api/connectors` | Connectors, settings (secrets masked) and latest syncs | | `PUT /api/connectors/{id}` | A connector's settings (admin); `id`: `cyberwatch`, `glpi`, `phpipam`, `paloalto`, or a module's id | | `POST /api/connectors/{id}/test` | Tests access (approver) | -| `POST /api/connectors/{id}/run` | Runs a sync (approver); `{"wait": true}` waits for the result. `{"file": {"name", "text"}, "overrides": {…}}` imports a file, for the modules that do (Palo Alto, CSV inventory: contributor); `overrides`: the settings the module lets one import change. The older Palo Alto form `{"xml", "fileName"}` still works | +| `POST /api/connectors/{id}/preview` | For a module that asks for a choice before its import (CSV VLANs): `{"file", "overrides"}` → the items of the file to choose from (`{items, summary, skipped, pick}`); nothing is recorded | +| `POST /api/connectors/{id}/run` | Runs a sync (approver); `{"wait": true}` waits for the result. With a choice: `"picks": {"": {"zone": ""}}` next to `file`. `{"file": {"name", "text"}, "overrides": {…}}` imports a file, for the modules that do (Palo Alto, CSV inventory: contributor); `overrides`: the settings the module lets one import change. The older Palo Alto form `{"xml", "fileName"}` still works | | `GET /api/modules` | The module store: what the registry offers, and what is installed (admin; `?fresh=1` reads the registry again) | | `POST /api/modules/{id}/install` | Installs or updates a module from the store (admin) | | `POST /api/modules/upload` | Installs a package: `{"data": "<.tgz in base64>"}` (admin, with `MODULE_UPLOAD=true`) | @@ -499,7 +548,7 @@ Every route needs the same credentials as the interface (a local account with Ba Example: adding an asset from a script (inventory, NetBox, scanner export…): ```bash -curl -u admin:PASSWORD -H 'Content-Type: application/json' -H 'X-Requested-With: strata' \ +curl -H "Authorization: Bearer $STRATA_TOKEN" -H 'Content-Type: application/json' -H 'X-Requested-With: strata' \ -X PUT https://strata.example.lan/api/assets/srv-erp-01 \ -d '{"site":"lyon","name":"srv-erp-01","type":"server","vlan":"v-lyo-20","ip":"10.20.0.20", "os":"RHEL 8","role":"ERP","crit":3, @@ -534,7 +583,7 @@ Fields added since v0.2 (all optional): Strata follows [Semantic Versioning](https://semver.org/). Before 1.0 a minor version may still break things, and the [CHANGELOG](CHANGELOG.md) says how to upgrade. From **1.0** on, every 1.x version keeps working: - the `/api/` routes, their methods, roles, request and answer fields; -- authentication (Basic, `Bearer stk_…` tokens), the write headers, `X-Batch` and `X-Batch-Label`; +- authentication (the session cookie of `POST /auth/login`, `Bearer stk_…` tokens), the write headers, `X-Batch` and `X-Batch-Label`; - status codes and the `error` key (not its wording); - the export format: every export written by an earlier version keeps importing; - the module API (`"strataModule": 1`, `ctx`, the SDK) and the store index (`"strataRegistry": 1`); @@ -569,15 +618,16 @@ This map describes your weak spots precisely: treat it as sensitive data. | "This browser only" badge at the top left | The page cannot reach the API: it is opened as a local file, or the proxy blocks `/api/`. | | Other users only see changes after a reload | The proxy buffers `/api/events` (see section 5, option B). | | "You have read-only access" | You are signed in with the viewer account, or `READ_ONLY=true`. | -| Switching accounts in the browser | The browser keeps Basic Auth credentials: open a private window or close the browser. | +| Switching accounts in the browser | *Sign out* in the title block, then sign in with the other account. | +| A script gets `401` with “HTTP Basic sign-in was removed” | Create an API token (*Account › API tokens*) and send `Authorization: Bearer stk_…`. | | Permission error on `/data` (Docker) | Keep the named volume from `docker-compose.yml`. If you mount a host folder, give it to UID 1000: `chown -R 1000:1000 ./data`. | --- -## 13. Limits of v0.10 +## 13. Limits of v0.11 - Storage is a single JSON file, suited to a few thousand assets. Beyond that, a real database will be needed. -- Sessions of single sign-on are kept in memory: restarting Strata signs SSO users out. Roles come from the groups at sign-in: a group change applies at the next sign-in (`SESSION_HOURS`, 8 by default), or at once with *End all single sign-on sessions*. LDAP is not read directly: use an OpenID Connect provider in front of it (Keycloak, Authentik, Entra ID). +- Sessions are kept in memory: restarting Strata signs everyone out. Roles come from the groups at sign-in: a group change applies at the next sign-in (`SESSION_HOURS`, 8 by default), or at once with *End all sessions*. LDAP is not read directly: use an OpenID Connect provider in front of it (Keycloak, Authentik, Entra ID). - Two approvers editing the same IS directly at the same time: the last save wins (the history lets you go back). Proposals, on the other hand, detect conflicts. - draw.io or Visio diagrams cannot be imported: only Strata JSON exports can become a proposal. - Opened without its server, the app keeps the map in the browser only: no accounts, history, proposals or connectors. @@ -610,7 +660,7 @@ A pull request must start from the latest release or later: the `base` job fails ### Releasing -1. Merge a pull request that sets the new version in `package.json` (`npm version 0.10.0 --no-git-tag-version`), `docker-compose.yml`, the interface and the docs, and dates its CHANGELOG section. +1. Merge a pull request that sets the new version in `package.json` (`npm version 0.11.0 --no-git-tag-version`), `docker-compose.yml`, the interface and the docs, and dates its CHANGELOG section. 2. *Actions › Release › Run workflow*, on `main`. The workflow reads the version from `package.json` and creates the `vX.Y.Z` tag itself, so there is nothing to type. It waits for a maintainer's approval (the `release` environment), then runs the tests, publishes the signed image and the GitHub release with its notes. Pushing a `vX.Y.Z` tag, or publishing a release from the web page with a new tag on `main`, still works: the tag must then be exactly `v` and the version (`v0.9.0`, not `V0.9.0` or `v.0.9.0`). diff --git a/docker-compose.yml b/docker-compose.yml index 965252f..e0a6574 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,7 +1,7 @@ services: strata: # Prebuilt image from GitHub (docker compose pull), or built here (docker compose up -d --build) - image: ghcr.io/solidgeeks/strata:0.10.0 + image: ghcr.io/solidgeeks/strata:0.11.0 build: . container_name: strata restart: unless-stopped @@ -14,6 +14,15 @@ services: # To open it to your LAN without the HTTPS proxy, use "8080:8080". ports: - "127.0.0.1:8080:8080" + # Resources (README, section 1, "Memory and CPU"): measured 125 MB for 5 000 assets, 340 MB at peak + # for 30 000. Uncomment to cap the container; raise the memory if connectors run often. + # deploy: + # resources: + # limits: + # cpus: "1.0" + # memory: 512M + # reservations: + # memory: 128M # Optional HTTPS reverse proxy: docker compose --profile https up -d caddy: diff --git a/docs/api.html b/docs/api.html index 62fe7fc..06796f5 100644 --- a/docs/api.html +++ b/docs/api.html @@ -71,7 +71,7 @@

Authentication

- + @@ -178,7 +178,8 @@

Connectors

- + +
MethodHowvia
Local accountHTTP Basic: Authorization: Basic base64(name:password), as curl -u name:password. Refused when LOCAL_LOGIN=off, and for non-admins when LOCAL_LOGIN=admins.local
Local accountThe session cookie set by POST /auth/login (the sign-in form, or JSON {"user", "password"}). Meant for browsers; scripts use an API token. HTTP Basic is refused since 0.11. No local sign-in when LOCAL_LOGIN=off, nor for non-admins when LOCAL_LOGIN=admins.local
API tokenAuthorization: Bearer stk_…. Created by an admin (below) with its own role and expiry. Not available on a server with no accounts.token
Single sign-on sessionThe strata_session cookie (__Host-strata_session over HTTPS) set by /auth/oidc/callback. Meant for browsers.sso
Trusted proxy headerSSO_HEADER_USER (and SSO_HEADER_GROUPS), believed only from the addresses in SSO_HEADER_TRUST.header
GET /api/connectorscontributor→ {connectors, canConfigure, canRun, store, problems?}: each connector with its fields, settings (secrets masked) and last run. A contributor sees only the file imports open to contributors.
PUT /api/connectors/{id}adminthe settings → {ok, connector}. A masked secret sent back unchanged keeps the stored one.
POST /api/connectors/{id}/testapprover→ {ok, message}. A failed test is 200 with ok: false and what to check.
POST /api/connectors/{id}/runapprover; contributor for a file import that allows it{wait?, file?: {name, text}, overrides?} → 202 {ok, started}, or with wait: true 200 {ok, run}. file imports a file instead of calling the tool; overrides: the settings the module lets one import change. 409 if already running. The older Palo Alto form {xml, fileName} still works.
POST /api/connectors/{id}/previewas its file importFor a module whose manifest has file.pick: {file: {name, text}, overrides?} → {ok, items, summary, skipped, pick}, the items of the file to choose from (key, label, detail, tags, warn, on, zone). Nothing is recorded. 400 for a module without file.pick.
POST /api/connectors/{id}/runapprover; contributor for a file import that allows it{wait?, file?: {name, text}, overrides?, picks?}. picks, after a preview: {key: {zone?}}, the items chosen and their zone (a zone id of the IS, or "new:Name"); the module reads it as ctx.picks. → 202 {ok, started}, or with wait: true 200 {ok, run}. file imports a file instead of calling the tool; overrides: the settings the module lets one import change. 409 if already running. The older Palo Alto form {xml, fileName} still works.

A run never writes the map: it opens proposals (one per IS), which an approver reviews. run is {id, at, by, file, ok, summary, error, proposals: [{id, title, siteId}], ms}.

@@ -226,11 +227,11 @@

Sign-in pages

- + + -
RouteDoes
GET /auth/loginThe sign-in page (with single sign-on)
GET /auth/loginThe sign-in page: the form for local accounts, and single sign-on
POST /auth/loginSigns a local account in: form fields or JSON {user, password} → session cookie (303 to the app, or {ok, user, role}); 401 wrong, 429 locked, 403 from another site
GET /auth/oidc/startStarts an OpenID Connect sign-in
GET /auth/oidc/callbackWhere the provider sends the browser back; sets the session cookie
POST /auth/logoutEnds the session → {ok, location}: where to go next (the provider’s sign-out)
GET /?localAsks for a local account’s password when single sign-on is on
@@ -317,7 +318,7 @@

Compatibility

What stays stable in 1.x

  • The routes on this page under /api/, their methods, the roles they need, the request fields they accept and the answer fields they return.
  • -
  • Authentication: Basic for local accounts, Bearer stk_… tokens, the write headers Content-Type: application/json and X-Requested-With: strata, X-Batch and X-Batch-Label.
  • +
  • Authentication: session cookies for browsers, Bearer stk_… tokens for scripts, the write headers Content-Type: application/json and X-Requested-With: strata, X-Batch and X-Batch-Label.
  • Status codes and the error key of refusals (not its wording).
  • The data format: the fields listed in Data format keep their meaning. Every export file written by an earlier version (version 3 and below) keeps importing; a 1.x version may write a higher version number, and still reads the lower ones.
  • The module API: a module written for "strataModule": 1 keeps running, with the same ctx and SDK helpers; a store index with "strataRegistry": 1 keeps being read.
  • @@ -343,7 +344,7 @@

    Deprecation

    Solid Geeks: open source the world.
    -

    Strata 0.10.0 · AGPL-3.0 · github.com/SolidGeeks/strata · Changelog

    +

    Strata 0.11.0 · AGPL-3.0 · github.com/SolidGeeks/strata · Changelog

    Cyberwatch, GLPI, phpIPAM and Palo Alto Networks are trademarks of their owners; Strata is not affiliated with them.

    diff --git a/docs/docs.html b/docs/docs.html index 0d7c81e..06a97ce 100644 --- a/docs/docs.html +++ b/docs/docs.html @@ -69,7 +69,7 @@

    Install with Docker

    nano .env # set EDIT_PASS docker compose up -d --build # builds the image here

    On Windows: copy .env.example .env then notepad .env.

    -

    Or skip the build: docker compose pull && docker compose up -d fetches the ready-made image ghcr.io/solidgeeks/strata of the release. Every image is signed; check it before deploying with gh attestation verify oci://ghcr.io/solidgeeks/strata:0.10.0 --owner SolidGeeks, which must print Verification succeeded!

    +

    Or skip the build: docker compose pull && docker compose up -d fetches the ready-made image ghcr.io/solidgeeks/strata of the release. Every image is signed; check it before deploying with gh attestation verify oci://ghcr.io/solidgeeks/strata:0.11.0 --owner SolidGeeks, which must print Verification succeeded!

    Open http://localhost:8080 and sign in with EDIT_USER / EDIT_PASS. The map starts empty: Load demo (5 IS) fills it with five sample sites, Delete everything clears it.

    docker compose logs -f strata     # logs
     docker compose restart strata     # restart
    @@ -86,6 +86,8 @@ 

    Configuration

    EDIT_PASS_FILE / VIEW_PASS_FILEnoneRead the password from this file instead (Docker secrets, systemd credentials, sops-nix, agenix). The file wins; a trailing newline is ignored; an unreadable file stops the server. ALLOW_SELF_APPROVALfalsetrue lets an approver approve their own proposals. HISTORY_KEEP5000How many document changes the history keeps. + HISTORY_MAX_MB200The history also stops at this size, the oldest changes first (each change keeps the edited document before and after it). + MAX_DOC_MB8Largest document saved at once: an IS with all its VLANs and flows (about 300 bytes a flow). READ_ONLYfalseFreezes the map for everyone (an audit, a migration). BACKUP_KEEP / BACKUP_EVERY_HOURS14 / 24Automatic backups: how many to keep, how often. MAX_IMPORT_MB20Largest JSON or firewall configuration file accepted. @@ -106,7 +108,7 @@

    Configuration

    HTTPS and network

    -

    Passwords travel as HTTP Basic credentials. Never expose Strata over plain HTTP beyond the machine it runs on.

    +

    Passwords travel in the sign-in form, and sessions in a cookie. Never expose Strata over plain HTTP beyond the machine it runs on; set PUBLIC_URL=https://… so the cookie is Secure.

    Bundled proxy (Caddy)

    1. In .env, set SITE_ADDRESS=strata.example.lan, the name your users will type.
    2. @@ -189,16 +191,20 @@

      Accounts and roles

      docker compose exec strata node scripts/users.mjs set alice contributor # asks for a password docker compose exec strata node scripts/users.mjs role alice approver docker compose exec strata node scripts/users.mjs disable alice
      +

      Local accounts sign in with the form of the sign-in page and get a session cookie for SESSION_HOURS (8); Sign out is in the title block. Changing a password, or disabling or deleting an account, ends its other sessions. Scripts use an API token: HTTP Basic is refused since 0.11.

      Repeated failed sign-ins are refused for a while (429), per account and per client address (see Configuration). Only different wrong passwords count, so a tab left open with an old password locks nobody out, and browsers already signed in keep working.

      Single sign-on

      OpenID Connect (Entra ID, Keycloak, Authentik, Google Workspace, Okta): register Strata at the provider with the redirect URI https://strata.example.org/auth/oidc/callback, then set PUBLIC_URL, SSO_ISSUER, SSO_CLIENT_ID, SSO_CLIENT_SECRET (or _FILE) and the groups that give each role (SSO_ROLE_ADMIN, SSO_ROLE_APPROVER, SSO_ROLE_CONTRIBUTOR, SSO_ROLE_VIEWER; SSO_DEFAULT_ROLE for everyone else). Opening Strata shows a sign-in page; the ID token’s signature, issuer, audience, expiry and nonce are checked, with PKCE.

      A sign-in proxy (Authelia, oauth2-proxy, Pomerium): SSO_HEADER_USER and SSO_HEADER_GROUPS name the headers it sets; they are believed only from the addresses in SSO_HEADER_TRUST (exact addresses, no range).

      -

      A name the provider gives that is also a local account is refused, and an e-mail address is used only once verified: prefer a name people cannot change (SSO_USER_CLAIM=oid on Entra ID). Sessions last SESSION_HOURS (8); Account › End all single sign-on sessions signs everyone out.

      +

      A name the provider gives that is also a local account is refused, and an e-mail address is used only once verified: prefer a name people cannot change (SSO_USER_CLAIM=oid on Entra ID). Sessions last SESSION_HOURS (8); Account › End all sessions signs everyone else out.

      Local accounts stay, for tests, small installations and as a way in when the provider is down: LOCAL_LOGIN=admins keeps them for administrators, off turns them off. API tokens for scripts: Account › API tokens (admin), shown once, sent as Authorization: Bearer stk_….

      Building the map

      Sites, zones, VLANs

      -

      An information system (IS) is a site. Zones, contacts sets its zones (one level each, for example Industrial, Office, Core IT, Exposed), its VLANs with their ranges, and the IS contacts shown on assets that have none of their own. Arrange plates lets you drag a plate anywhere on its level; Line up puts a level back in a row.

      +

      An information system (IS) is a site. Zones, contacts sets its zones (one level each, for example Industrial, Office, Core IT, Exposed), its VLANs with their ranges, and the IS contacts shown on assets that have none of their own. Arrange plates lets you drag a plate anywhere on its level, and the Internet cloud too (Put Internet back returns it); Line up puts a level back in a row.

      +

      Import VLANs (Edit › Import VLANs, in an IS) reads a CSV (header in English or French, , ; or tab), a JSON list ([{num, name, cidr, zone}], NetBox’s {"results": […]}) or a Strata export: tick the VLANs to bring, filter, and choose the zone of each, or of all the ticked ones at once. A VLAN whose number is already in the IS is updated. The CSV VLANs module of the store does the same as a proposal, open to contributors.

      +

      Arrange VLANs (Layout) lays each zone out on rows and columns, the VLANs that talk to each other side by side and those without flows last; VLANs per row is automatic or 3 to 12. Suggest zones from the flows (Edit or Layout) proposes zones from the Internet down: each VLAN goes to the floor of its distance from the Internet along the flows, and those the Internet never reaches go below by the role of their assets (internal servers, users, industrial, network and administration). Zones that already hold most of the same VLANs keep their name; a window lets you rename and move before Apply, or Propose for contributors.

      +

      In the plate list, All and None show or hide every VLAN of the IS or of one zone; Plates › Hide empty VLANs leaves out the plates without an asset.

      Assets

      Each asset has a type (server, firewall, router, switch, workstation, PLC…), a VLAN, an IP, a system, applications with their listening ports, vulnerabilities, a criticality, incident contacts and notes. A new asset starts on the first free IP of its VLAN range; an IP outside the range is pointed out. A vulnerability has a CVSS score or a severity, and optionally an EPSS (0 to 1, or a percentage) and a Known exploited (CISA KEV) mark. Search by name or IP to fly to it.

      Flows

      @@ -208,6 +214,9 @@

      Moving around the 3D view

      Drag with the left button to orbit, with the middle (or right) button to move the view left, right, up and down, and scroll to zoom. On a touch screen, one finger orbits, two fingers zoom and move. The view stops at the edge of the model; Reset view centres it again.

      2D diagram

      The 2D diagram view is the same model, flat: drag assets and plates, draw a flow from one asset to another (F), add an asset on a plate (A), move (V). Positions are shared with the 3D model.

      +

      Graphics on small laptops

      +

      Display › Graphics: Auto (the default) draws in Quality until moving the view turns out slow on this computer, then switches to Economy and remembers it; Economy uses the screen resolution, drops shadows and moving dots and hides labels sooner. Animated flows turns the moving dots off on its own. The 3D view is only drawn when something changes.

      +

      The title block folds into one line (▴ in its corner), keeping the name of the IS, the key figures, back to the overview and sign out. A long request shows a loading bar at the top of the screen.

      Theme

      Display › Theme: Auto follows the system, Day and Night force it. N switches.

      @@ -273,15 +282,17 @@

      Palo Alto

    3. any in an untrust, internet, outside or wan zone → the Internet;
    4. the chosen firewall as the device crossed; Internet-to-admin or database ports flagged as risky.
+

Build the IS (a box of the file import): on an empty or incomplete IS, the configuration also gives what the map lacks. A zone per security zone; a VLAN per layer-3 interface or sub-interface outside the Internet zones (its tag and subnet, named after its comment; an untagged interface gets a number from 4001 on, to change); an asset per host address object of a private range, in the VLAN whose range holds it, its type from its name or description (PC-, TEL-, SQL-, CAM-…), else from the ports opened to it, else a workstation if it only opens connections; and the firewall itself, from its host name and inside address, crossed by the flows that change VLAN. What the map has (a zone of the same name, a VLAN of the same number or range, an asset at the same IP) is kept; the same file again changes nothing. Systems, vulnerabilities and contacts still come from an inventory or a scanner.

What cannot be drawn (internal any→any, dynamic groups, negations, addresses with no asset) is listed in the proposal. Flows from a rule that disappeared are removed; flows drawn by hand are never touched. A proposal from a file counts as the importer’s own for the four-eyes rule.

Modules and the module store

-

Each connector is a module: a folder with a manifest (its settings, whether it imports files, what it may reach) and its code. The four above ship with Strata; three more come from the store:

+

Each connector is a module: a folder with a manifest (its settings, whether it imports files, what it may reach) and its code. The four above ship with Strata; four more come from the store:

  • CSV inventory imports the assets of an IS from a spreadsheet saved as CSV (columns name, ip, type, os, role, vlan, criticality, notes);
  • Nmap scan imports a scan saved with -oX: hosts that are up, the system Nmap guessed, open ports as listening ports of the applications;
  • +
  • CSV VLANs imports the VLANs of an IS from a CSV (number, name, range): after the file, a window lists them, to tick the ones to bring and choose their zone;
  • NetBox syncs devices, virtual machines, VLANs and prefixes with a read-only token, a NetBox site going to an IS through rules such as Lyon HQ = lyon.
-

The first two are file imports open to any contributor; NetBox is run by approvers.

+

The first three are file imports open to any contributor; NetBox is run by approvers.

  1. Install: Connectors › Module store (admin). Each card says what the module reaches (reads the map, connects to your tool over the network), its version, licence and source. Install downloads the package and refuses it if it differs from the checksum the store publishes.
  2. Set it up: Settings on its card under Installed, then Test the connection. Every module also has Enabled with Sync automatically every … hours (both needed for scheduled syncs), the TLS options and Limit to these IS under More settings.
  3. @@ -339,7 +350,7 @@

    API

    GET /healthzHealth check, no authentication -
    curl -u admin:PASSWORD -H 'Content-Type: application/json' -H 'X-Requested-With: strata' \
    +    
    curl -H "Authorization: Bearer $STRATA_TOKEN" -H 'Content-Type: application/json' -H 'X-Requested-With: strata' \
       -X PUT https://strata.example.lan/api/assets/srv-erp-01 \
       -d '{"site":"lyon","name":"srv-erp-01","type":"server","vlan":"v-lyo-20","ip":"10.20.0.20",
            "os":"RHEL 8","role":"ERP","crit":3,
    @@ -369,7 +380,7 @@ 

    Troubleshooting

    “This browser only” badgeThe page cannot reach the API: it was opened as a local file, or the proxy blocks /api/. Others see changes only after reloadingThe proxy buffers /api/events: see your own reverse proxy. “You have read-only access”You are signed in as a viewer, or READ_ONLY=true. - Switching accountsBrowsers keep Basic credentials: use a private window, or close the browser. + Switching accountsSign out in the title block, then sign in again. Permission error on /dataKeep the named volume. A host folder must belong to UID 1000: chown -R 1000:1000 ./data. A connector cannot connectTest the connection names the cause: DNS, refused port, untrusted certificate, refused credentials. @@ -378,7 +389,7 @@

    Troubleshooting

    Known limits

    • Storage is a single JSON file, fine for a few thousand assets.
    • -
    • Single sign-on sessions live in memory: a restart signs SSO users out, and a group change applies at the next sign-in.
    • +
    • Sessions live in memory: a restart signs everyone out, and a single sign-on group change applies at the next sign-in.
    • Two approvers editing the same IS directly at the same moment: the last save wins (history lets you go back). Proposals detect conflicts.
    • Connectors follow the public API documentation of each tool and were tested against simulators: test, sync, and read the first proposal carefully. GLPI 11’s new API (/api.php/v2) is not supported yet.
    • Palo Alto: NAT rules, security profiles, dynamic address groups and the Panorama device-group hierarchy are not interpreted.
    • @@ -395,7 +406,7 @@

      Known limits

      Solid Geeks: open source the world.
    -

    Strata 0.10.0 · AGPL-3.0 · github.com/SolidGeeks/strata · Changelog

    +

    Strata 0.11.0 · AGPL-3.0 · github.com/SolidGeeks/strata · Changelog

    Cyberwatch, GLPI, phpIPAM and Palo Alto Networks are trademarks of their owners; Strata is not affiliated with them.

    diff --git a/docs/index.html b/docs/index.html index 53b75e0..0537363 100644 --- a/docs/index.html +++ b/docs/index.html @@ -96,7 +96,7 @@

    What you get,

    A model you can walk around, and a diagram you can draw

    Each information system is a stack of plates: one per zone, one colour per VLAN, an object per asset. Orbit, move across with the middle button, zoom, filter by severity or exposure. Switch to the 2D diagram to add assets, draw flows and arrange plates; both views share the same positions, so what you draw flat is what you see in 3D.

    -
    • Flows routed through the firewalls, routers and switches they cross, each in its own lane
    • One colour per flow, on demand
    • Plates movable anywhere on their level
    • Five sites in the demo, linked by WAN
    +
    • Flows routed through the firewalls, routers and switches they cross, each in its own lane
    • One colour per flow, on demand
    • Plates movable anywhere on their level, or arranged by their flows
    • Zones suggested from the flows, the Internet on top
    • A lighter Economy mode for small laptops
    • Five sites in the demo, linked by WAN

    The 2D diagram of the Lyon site: four levels of VLAN plates with asset cards and flows
    2D diagram of the demo’s Lyon site.
    @@ -167,10 +167,11 @@

    Fed by the tools you already run

    Cyberwatchservers and their CVEs (API v3)vulnerabilities on matching assets, by host name then IP; hand-entered findings kept GLPIcomputers, network devices, printers, phones, software, technician and group, locationnew and updated assets, applications, incident contacts; the location picks the site phpIPAMsections, subnets, VLANs, addressesVLANs with their ranges, asset addresses, assets moved to their subnet’s VLAN - Palo Altoa configuration file, or the XML API of a firewall or Panoramaflows from every active allow rule, address objects matched to assets, public objects as Internet services + Palo Altoa configuration file, or the XML API of a firewall or Panoramaflows from every active allow rule, address objects matched to assets, public objects as Internet services; or the whole IS (zones, VLANs, assets) from the configuration alone NetBox storedevices, virtual machines, VLANs, prefixesVLANs with their ranges, new and completed assets; NetBox sites matched to IS by rules Nmap storea scan saved with -oXhosts with their system and open ports, new hosts typed from their ports CSV storea spreadsheet saved as CSVnew and updated assets of one site + CSV VLANs storea CSV of VLANs (number, name, range)the VLANs you tick, each in the zone you choose
    The Connectors panel: Cyberwatch and GLPI scheduled, their last runs and the proposals they prepared
    Connectors panel after a sync.
    @@ -212,7 +213,7 @@

    Your map stays Solid Geeks: open source the world.
    -

    Strata 0.10.0 · AGPL-3.0 · github.com/SolidGeeks/strata

    +

    Strata 0.11.0 · AGPL-3.0 · github.com/SolidGeeks/strata

    Cyberwatch, GLPI, phpIPAM and Palo Alto Networks are trademarks of their owners; Strata is not affiliated with them.

    diff --git a/docs/modules/csv-vlans-1.0.0.tgz b/docs/modules/csv-vlans-1.0.0.tgz new file mode 100644 index 0000000..880ce6f Binary files /dev/null and b/docs/modules/csv-vlans-1.0.0.tgz differ diff --git a/docs/modules/index.json b/docs/modules/index.json index db4e5d5..a36c76e 100644 --- a/docs/modules/index.json +++ b/docs/modules/index.json @@ -17,6 +17,22 @@ "url": "csv-inventory-1.0.0.tgz", "sha256": "a2c1bb73afd5102c7e91ce84046e4b9d96e1931bfdf2856af73f885708642fab" }, + { + "id": "csv-vlans", + "name": "CSV VLANs", + "version": "1.0.0", + "description": "VLANs of an IS from a CSV (number, name, address range): choose the VLANs to bring and the zone of each", + "author": "Solid Geeks", + "license": "AGPL-3.0-or-later", + "homepage": "https://github.com/SolidGeeks/strata/tree/main/modules/csv-vlans", + "badge": "VLAN", + "permissions": { + "map": true, + "network": false + }, + "url": "csv-vlans-1.0.0.tgz", + "sha256": "166097ebd979206b10d22831e0b70045b2861f101b1d0975e9d2ec736be783ef" + }, { "id": "netbox", "name": "NetBox", diff --git a/modules/csv-vlans/README.md b/modules/csv-vlans/README.md new file mode 100644 index 0000000..2d84f69 --- /dev/null +++ b/modules/csv-vlans/README.md @@ -0,0 +1,38 @@ +# CSV VLANs: a Strata connector module + +Brings VLANs into one IS from a spreadsheet saved as CSV, **choosing which VLANs and the zone of each**. +The result is a proposal: nothing changes on the map until an approver accepts it. + +1. *Collaboration › Connectors › CSV VLANs › Import VLANs from a CSV*: the IS and the file. +2. Strata lists the VLANs of the file: tick those to bring (the new ones are ticked), filter, and choose + the zone of each, or of all the ticked ones at once: a zone of the IS, a zone named in the file, or a + new one. +3. *Import*: the proposal holds the IS with those VLANs; an approver reviews it in *Proposals*. + +| Column (any order, any case, English or French) | Meaning | +|---|---| +| `number`, `vlan`, `vid`, `numéro` (required) | VLAN number, 1 to 4094 (`VLAN 10` is read as 10) | +| `name`, `nom`, `description` | VLAN name (`VLAN ` when empty) | +| `cidr`, `subnet`, `network`, `réseau`, `range` | Address range, such as `10.20.0.0/24` | +| `zone`, `group` (optional) | The zone suggested for the VLAN; created when the IS has none of that name | + +Without a header line, the columns are told apart by what they hold: ranges, numbers, and names, in any +order (`10.20.0.0/24;Servers;20` works). Commas, semicolons or tabs. + +A VLAN already in the IS (same number) starts unticked; ticked, its name, range and zone are updated. The +same file imported again gives the same proposal, and none when the IS already matches. + +From a script, without a choice (`picks` left out), every VLAN of the file is brought, in the zone the file +gives, else the *default zone* of the settings, else the top zone; those already in the IS are left alone +unless *update the VLANs already in the IS* is on. With a choice: + +```bash +jq -n --rawfile x vlans.csv '{wait: true, file: {name: "vlans.csv", text: $x}, overrides: {site: "lyon"}, + picks: {"300": {zone: "new:Lab"}, "301": {zone: "new:Lab"}, "20": {}}}' | + curl -sS -H "Authorization: Bearer $STRATA_TOKEN" -H 'Content-Type: application/json' -H 'X-Requested-With: strata' \ + -X POST https://strata.example.lan/api/connectors/csv-vlans/run --data-binary @- +``` + +`POST /api/connectors/csv-vlans/preview` with the same `file` and `overrides` answers the VLANs of the +file, as the import window shows them. Needs Strata 0.11 or later for the choice; an older Strata +imports every VLAN of the file. diff --git a/modules/csv-vlans/index.mjs b/modules/csv-vlans/index.mjs new file mode 100644 index 0000000..c281f86 --- /dev/null +++ b/modules/csv-vlans/index.mjs @@ -0,0 +1,151 @@ +// SPDX-License-Identifier: AGPL-3.0-or-later +// CSV VLANs: a Strata connector module (see MODULES.md). It reads the VLANs of a CSV file (number, name, +// address range, and optionally a zone), lets people choose which ones to bring into an IS and the zone of +// each (preview, then the import with ctx.picks), and proposes the IS with those VLANs. Nothing changes on +// the map until an approver accepts the proposal. It needs no network. + +const COLUMNS = { + num: ['vlan', 'vlan id', 'vlanid', 'vid', 'vlan number', 'number', 'num', 'numero', 'n', 'no', 'tag'], + name: ['name', 'nom', 'vlan name', 'description', 'desc', 'label', 'libelle'], + cidr: ['cidr', 'subnet', 'network', 'prefix', 'range', 'reseau', 'sous reseau', 'plage', 'ip range', 'address range', 'prefixe', 'adresse'], + zone: ['zone', 'group', 'groupe', 'vlan group'], +}; + +// Splits CSV text into rows, with quoted fields ("a; b" and "" for a quote) and the separator guessed +// from the first line +export function parseCsv(text) { + text = String(text).replace(/^/, ''); + const head = text.split(/\r?\n/, 1)[0]; + const sep = [';', '\t', ','].map(c => [c, head.split(c).length]).sort((a, b) => b[1] - a[1])[0][0]; + const rows = []; let row = [], cell = '', q = false; + for (let i = 0; i < text.length; i++) { + const c = text[i]; + if (q) { if (c === '"') { if (text[i + 1] === '"') { cell += '"'; i++; } else q = false; } else cell += c; continue; } + if (c === '"' && cell === '') q = true; + else if (c === sep) { row.push(cell); cell = ''; } + else if (c === '\n' || c === '\r') { if (c === '\r' && text[i + 1] === '\n') i++; row.push(cell); rows.push(row); row = []; cell = ''; } + else cell += c; + } + if (cell !== '' || row.length) { row.push(cell); rows.push(row); } + return rows.map(r => r.map(x => x.trim())).filter(r => r.some(Boolean)); +} + +const vlanNum = v => { const m = /^\s*(?:vlan\s*)?(\d{1,4})\s*$/i.exec(String(v ?? '')); const n = m ? +m[1] : NaN; return n >= 1 && n <= 4094 ? n : null; }; +const looksCidr = v => /^\d{1,3}(\.\d{1,3}){3}(\/\d{1,2})?$/.test(String(v || '').trim()); + +// The VLANs of the file: [{num, name, cidr, zone, line, warn}], and what was left out +export function readVlans(text, sdk) { + const rows = parseCsv(text); + if (!rows.length) throw new sdk.ConnectorError('The file is empty.'); + const head = rows[0].map(h => sdk.norm(h)), col = {}; + for (const [k, names] of Object.entries(COLUMNS)) { const i = head.findIndex(h => names.includes(h)); if (i >= 0) col[k] = i; } + let body = rows.slice(1), first = 2; + if (col.num === undefined) { + // no header (or one we do not know): each column is told by what it holds, in the order the file has + const sample = rows.slice(0, 50), width = Math.max(...sample.map(r => r.length)); + const share = (i, test) => sample.filter(r => r[i] && test(r[i])).length / Math.max(1, sample.filter(r => r[i]).length); + for (let i = 0; i < width; i++) { + if (col.cidr === undefined && share(i, looksCidr) > 0.6) col.cidr = i; + else if (col.num === undefined && share(i, x => vlanNum(x) !== null) > 0.6) col.num = i; + else if (col.name === undefined && share(i, x => !looksCidr(x) && vlanNum(x) === null) > 0.6) col.name = i; + } + if (col.num === undefined) throw new sdk.ConnectorError(`No column of VLAN numbers (1 to 4094) in this file. First line: ${rows[0].join(', ').slice(0, 120)}`); + if (vlanNum(rows[0][col.num]) !== null) { body = rows; first = 1; } + } + const vlans = [], skipped = [], seen = new Set(); + body.forEach((r, i) => { + const line = i + first, num = vlanNum(r[col.num]); + if (num === null) { skipped.push(`Line ${line}: no VLAN number from 1 to 4094 (${String(r[col.num] ?? '').slice(0, 30) || 'empty'}).`); return; } + if (seen.has(num)) { skipped.push(`Line ${line}: VLAN ${num} again, left out (the first one is kept).`); return; } + seen.add(num); + const cidr = col.cidr !== undefined ? String(r[col.cidr] || '').trim() : '', warn = []; + if (cidr && !sdk.parseRange(cidr)) warn.push('range not understood'); + vlans.push({ num, name: (col.name !== undefined ? String(r[col.name] || '').trim() : '').slice(0, 50) || `VLAN ${num}`, cidr: cidr.slice(0, 60), zone: (col.zone !== undefined ? String(r[col.zone] || '').trim() : '').slice(0, 40), line, warn }); + }); + if (!vlans.length) throw new sdk.ConnectorError('No VLAN with a number from 1 to 4094 in this file.'); + return { vlans, skipped }; +} + +const zonesOf = si => (Array.isArray(si.zones) && si.zones.length ? si.zones : [{ id: 'z-def', name: 'Main zone' }]); + +// Before the import: the VLANs of the file, ticked when new, with the zone the file or the IS suggests +export async function preview(cfg, ctx) { + const { state, sdk } = ctx, si = state.si[cfg.site]; + if (!si) throw new sdk.ConnectorError('Choose the IS the VLANs go to.'); + const { vlans, skipped } = readVlans(ctx.file.text, sdk), zones = zonesOf(si); + const have = new Map((si.vlans || []).map(v => [+v.num, v])); + const items = vlans.map(v => { + const ex = have.get(v.num), exZone = ex && zones.find(z => z.id === ex.zone); + const same = ex && ex.name === v.name && (!v.cidr || ex.cidr === v.cidr); + return { + key: String(v.num), label: `VLAN ${v.num} · ${v.name}`, detail: v.cidr, + tags: ex ? [same ? 'already in the IS, the same' : 'already in the IS: ticked, it is updated'] : [], + warn: v.warn, on: !ex, zone: v.zone || (exZone ? exZone.name : ''), + }; + }); + const n = items.filter(i => i.on).length; + return { items, skipped, summary: `${vlans.length} VLAN(s) in ${ctx.file.name}, ${n} not yet in IS ${si.name || cfg.site}.` }; +} + +export async function run(cfg, ctx) { + const { state, sdk } = ctx, sid = cfg.site, si = state.si[sid]; + if (!si) throw new sdk.ConnectorError('Choose the IS the VLANs go to.'); + if (!ctx.file) throw new sdk.ConnectorError('This module imports a CSV file: choose one.'); + const { vlans, skipped } = readVlans(ctx.file.text, sdk); + const work = JSON.parse(JSON.stringify(si)); + work.zones = JSON.parse(JSON.stringify(zonesOf(si))); + work.vlans = Array.isArray(work.vlans) ? work.vlans : []; + const picks = ctx.picks, report = { skipped, warnings: [] }, newZones = []; + // a zone by its id, or by its name ("new:", or a name from the file or the settings): created when missing + const zoneByName = name => { + let z = work.zones.find(x => sdk.norm(x.name) === sdk.norm(name)); + if (!z) { + let id = `z-csv-${sdk.slug(name)}`; + for (let i = 2; work.zones.some(x => x.id === id); i++) id = `z-csv-${sdk.slug(name)}-${i}`; + z = { id, name: String(name).slice(0, 40) }; + work.zones.push(z); newZones.push(z.name); + } + return z.id; + }; + const zoneOf = (choice, fileZone, current) => { + if (choice) { + if (choice.startsWith('new:') && choice.slice(4).trim()) return zoneByName(choice.slice(4).trim()); + if (work.zones.some(z => z.id === choice)) return choice; + report.warnings.push(`Zone ${choice} is not in the IS: the zone of the file or the default one is used.`); + } + if (fileZone) return zoneByName(fileZone); + if (current) return current; + if (cfg.defaultZone) return zoneByName(cfg.defaultZone); + return work.zones[work.zones.length - 1].id; + }; + let added = 0, updated = 0, same = 0; + for (const v of vlans) { + const pick = picks ? picks[String(v.num)] : null; + if (picks && !pick) continue; + const i = work.vlans.findIndex(x => +x.num === v.num), ex = i >= 0 ? work.vlans[i] : null; + // without a choice (a script), a VLAN already in the IS is only updated when the settings say so + if (ex && !picks && !cfg.updateExisting) { same++; continue; } + const zone = zoneOf(pick && pick.zone, v.zone, ex && ex.zone); + if (ex) { + const next = { ...ex, name: v.name, zone }; + if (v.cidr && !v.warn.length) next.cidr = v.cidr; + if (next.zone !== ex.zone) delete next.place; + if (JSON.stringify(next) === JSON.stringify(ex)) { same++; continue; } + work.vlans[i] = next; updated++; + } else { + let id = `v-csv-${sdk.slug(sid)}-${v.num}`; + for (let k = 2; work.vlans.some(x => x.id === id); k++) id = `v-csv-${sdk.slug(sid)}-${v.num}-${k}`; + work.vlans.push({ id, num: v.num, name: v.name, cidr: v.warn.length ? '' : v.cidr, zone }); + if (v.warn.length) report.warnings.push(`VLAN ${v.num}: range “${v.cidr}” not understood, left empty.`); + added++; + } + } + const summary = `${vlans.length} VLAN(s) read${picks ? `, ${Object.keys(picks).length} chosen` : ''}: ${added} to add, ${updated} to update${same ? `, ${same} unchanged` : ''}${newZones.length ? `; new zone(s): ${newZones.join(', ')}` : ''}.`; + ctx.log(summary); + const changed = added || updated; + return { + proposals: changed ? [{ siteId: sid, title: `VLANs from ${ctx.file.name}`, description: summary + (skipped.length ? ` ${skipped.length} line(s) left out.` : ''), changes: { si: { [sid]: work }, assets: {} } }] : [], + summary, + report, + }; +} diff --git a/modules/csv-vlans/strata-module.json b/modules/csv-vlans/strata-module.json new file mode 100644 index 0000000..f5c20d3 --- /dev/null +++ b/modules/csv-vlans/strata-module.json @@ -0,0 +1,31 @@ +{ + "strataModule": 1, + "id": "csv-vlans", + "name": "CSV VLANs", + "version": "1.0.0", + "description": "VLANs of an IS from a CSV (number, name, address range): choose the VLANs to bring and the zone of each", + "author": "Solid Geeks", + "license": "AGPL-3.0-or-later", + "homepage": "https://github.com/SolidGeeks/strata/tree/main/modules/csv-vlans", + "badge": "VLAN", + "main": "index.mjs", + "permissions": { "map": true, "network": false }, + "live": false, + "file": { + "label": "Import VLANs from a CSV", + "accept": ".csv,.txt,text/csv", + "role": "contributor", + "fields": ["site", "updateExisting", "defaultZone"], + "help": "One VLAN per line: its number, its name and its address range (CIDR), in any order, with or without a header (number, vlan or vid; name or nom; cidr, subnet, network or réseau; optionally zone). Separated by commas, semicolons or tabs.", + "pick": { + "title": "Choose the VLANs and their zone", + "zones": true, + "help": "Tick the VLANs to bring into the IS and choose the zone of each, or of all the ticked ones at once. A VLAN already in the IS (same number) is updated when ticked." + } + }, + "fields": [ + { "k": "site", "label": "IS the VLANs go to", "type": "site", "required": true }, + { "k": "updateExisting", "label": "Without a choice (from a script): also update the VLANs already in the IS", "type": "bool", "default": false }, + { "k": "defaultZone", "label": "Zone when none is chosen nor given by the file (its name; empty: the top zone of the IS)", "default": "" } + ] +} diff --git a/nix/module.nix b/nix/module.nix index 0986022..f73bc61 100644 --- a/nix/module.nix +++ b/nix/module.nix @@ -88,7 +88,7 @@ in ''; description = '' Environment variables of Strata, as listed in section 3 of the README and in .env.example: - EDIT_USER, VIEW_USER, ALLOW_SELF_APPROVAL, HISTORY_KEEP, READ_ONLY, BACKUP_KEEP, + EDIT_USER, VIEW_USER, ALLOW_SELF_APPROVAL, HISTORY_KEEP, HISTORY_MAX_MB, MAX_DOC_MB, READ_ONLY, BACKUP_KEEP, BACKUP_EVERY_HOURS, MAX_IMPORT_MB, LOGIN_MAX_FAILS, LOGIN_MAX_FAILS_PER_ADDRESS, LOGIN_LOCK_MINUTES, TRUST_PROXY (set it to "loopback" behind a reverse proxy on this host), PUBLIC_URL, SSO_ISSUER, SSO_CLIENT_ID, SSO_NAME, SSO_ROLE_ADMIN, SSO_ROLE_APPROVER, diff --git a/nix/package.nix b/nix/package.nix index a484d71..9129f54 100644 --- a/nix/package.nix +++ b/nix/package.nix @@ -35,7 +35,7 @@ buildNpmPackage { nodejs = nodejs_22; # Changes with package-lock.json: Nix prints the new value when it no longer matches # (README, "Nix and NixOS") - npmDepsHash = "sha256-fr4NO/VIuBQB9rc5TnwisLIvPWuTDnobwmiWin87+4A="; + npmDepsHash = "sha256-YGrqCpxo9cZZomxUqMsHz21uIBTZ91QwW3osm4NaNrs="; npmBuildScript = "vendor"; diff --git a/package-lock.json b/package-lock.json index cdc9ac7..30401ba 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "strata", - "version": "0.10.0", + "version": "0.11.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "strata", - "version": "0.10.0", + "version": "0.11.0", "license": "AGPL-3.0-or-later", "dependencies": { "@fontsource/archivo": "^5.0.0", diff --git a/package.json b/package.json index 1352fc7..64a9bde 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "strata", - "version": "0.10.0", + "version": "0.11.0", "description": "Strata — self-hosted 3D map of information systems: zones, VLANs, assets, flows, exposure, vulnerabilities and incident contacts.", "private": true, "license": "AGPL-3.0-or-later", @@ -12,7 +12,7 @@ "scripts": { "vendor": "node scripts/vendor.mjs", "start": "node server/server.js", - "test": "node --test tests/api.test.mjs tests/analysis.test.mjs tests/incident.test.mjs tests/risk.test.mjs tests/kpi.test.mjs tests/muster.test.mjs tests/secrets.test.mjs tests/signin.test.mjs tests/hardening.test.mjs tests/readonly.test.mjs tests/modules.test.mjs tests/sso.test.mjs", + "test": "node --test tests/api.test.mjs tests/analysis.test.mjs tests/incident.test.mjs tests/risk.test.mjs tests/kpi.test.mjs tests/muster.test.mjs tests/secrets.test.mjs tests/signin.test.mjs tests/hardening.test.mjs tests/readonly.test.mjs tests/modules.test.mjs tests/sso.test.mjs tests/paloalto.test.mjs", "test:e2e": "node --test tests/e2e.mjs", "setup": "npm ci && npm run vendor", "build:online": "node scripts/build-online.mjs" diff --git a/public/analysis.js b/public/analysis.js index bcdb07e..9d8f587 100644 --- a/public/analysis.js +++ b/public/analysis.js @@ -274,6 +274,163 @@ function chokePoints(site,byTarget){ return [...pts.values()].sort((x,y)=>y.every.length-x.every.length||y.paths-x.paths||byName(x.id,y.id)); } -root.StrataAnalysis={riskScore,riskPriorities,vulnRisk,likelihoodOf,inCidr,nextFreeIp,lanes,flowColours,dependencies,businessImpact,appGraph, +/* ---- VLANs read from a file, to import some of them ---- */ +// A CSV (header in English or French, separator , ; or tab), a JSON list ([{num, name, cidr, zone}] or +// {vlans: [...]}) or a Strata export (format "strata": every VLAN of every IS, with its zone and its IS). +// Returns {kind: 'csv'|'json'|'strata', items: [{num, name, cidr, zone, site, siteName, vlanId}], skipped} +// where skipped counts the lines without a valid VLAN number (1 to 4094). Throws when nothing is readable. +const VCOL={num:['vlan','vlan id','vlan_id','vlanid','vid','vlan number','num','number','numero','numéro','n°','tag'], + name:['name','nom','vlan name','description','desc','label','libellé','libelle'], + cidr:['cidr','subnet','network','prefix','range','réseau','reseau','sous-réseau','sous-reseau','plage','ip range','address range','prefixe','préfixe'], + zone:['zone','group','groupe','vlan group','site','location','emplacement','tenant']}; +function csvRows(text){ + const lines=String(text).replace(/^/,'').split(/\r?\n/).filter(l=>l.trim()); + if(!lines.length)return []; + const head=lines[0],sep=[';','\t',','].map(c=>[c,head.split(c).length]).sort((a,b)=>b[1]-a[1])[0][0]; + return lines.map(l=>{const out=[];let cur='',q=false; + for(let i=0;i{const m=/\d+/.exec(String(v==null?'':v));const n=m?parseInt(m[0],10):NaN;return n>=1&&n<=4094?n:null;}; +function vlansFromFile(text,fileName=''){ + const t=String(text||'').replace(/^/,'').trim(); + if(!t)throw new Error('the file is empty'); + const items=[];let skipped=0,kind; + const push=(o,extra={})=>{const num=vlanNum(o.num);if(num===null){skipped++;return;} + items.push(Object.assign({num,name:String(o.name||'').trim().slice(0,50)||`VLAN ${num}`,cidr:String(o.cidr||'').trim().slice(0,60),zone:String(o.zone||'').trim().slice(0,40)},extra));}; + if(/^[[{]/.test(t)&&!/\.csv$/i.test(fileName)){ + let o;try{o=JSON.parse(t);}catch(e){throw new Error('this is not valid JSON');} + if(o&&typeof o.si==='object'&&o.si&&!Array.isArray(o.si)){ + kind='strata'; + Object.entries(o.si).forEach(([sid,d])=>{if(!d||typeof d!=='object')return; + const zones=Array.isArray(d.zones)?d.zones:[]; + (Array.isArray(d.vlans)?d.vlans:[]).forEach(v=>{if(!v||typeof v!=='object')return;const z=zones.find(x=>x&&x.id===v.zone); + push({num:v.num,name:v.name,cidr:v.cidr,zone:z?z.name:''},{site:sid,siteName:String(d.name||sid),vlanId:String(v.id||'')});});}); + }else{ + kind='json'; + const list=Array.isArray(o)?o:Array.isArray(o&&o.vlans)?o.vlans:Array.isArray(o&&o.results)?o.results:null; + if(!list)throw new Error('expected a list of VLANs, {"vlans": [...]}, or a Strata export'); + const pick=(x,keys)=>{for(const k of keys){const hit=Object.keys(x).find(y=>y.toLowerCase()===k);if(hit!==undefined&&x[hit]!=null&&typeof x[hit]!=='object')return x[hit];} + // NetBox and others: {group: {name}}, {site: {name}} + for(const k of keys){const hit=Object.keys(x).find(y=>y.toLowerCase()===k);if(hit!==undefined&&x[hit]&&typeof x[hit]==='object'&&x[hit].name)return x[hit].name;}return '';}; + list.forEach(x=>{if(!x||typeof x!=='object'){skipped++;return;}push({num:pick(x,VCOL.num),name:pick(x,VCOL.name),cidr:pick(x,VCOL.cidr),zone:pick(x,VCOL.zone)});}); + } + }else{ + kind='csv'; + const rows=csvRows(t); + const norm=h=>h.toLowerCase().replace(/^["']|["']$/g,'').trim(); + const head=rows[0].map(norm),col={}; + Object.entries(VCOL).forEach(([k,names])=>{const i=head.findIndex(h=>names.includes(h));if(i>=0)col[k]=i;}); + let body=rows.slice(1); + // no header: number, name, range, zone in that order + if(col.num===undefined){if(vlanNum(rows[0][0])===null)throw new Error('no VLAN number column (vlan, vid, num, numéro…)');Object.assign(col,{num:0,name:1,cidr:2,zone:3});body=rows;} + body.forEach(r=>push({num:r[col.num],name:col.name!==undefined?r[col.name]:'',cidr:col.cidr!==undefined?r[col.cidr]:'',zone:col.zone!==undefined?r[col.zone]:''})); + } + if(!items.length)throw new Error('no VLAN with a number from 1 to 4094 in this file'); + return {kind,items,skipped}; +} + +/* ---- Zones suggested from the flows ---- */ +// The floors of an IS, the Internet on top: each VLAN goes to the floor of its depth, the number of flow +// hops from the Internet in the direction of the traffic (1: reached straight from the Internet), and the +// VLANs never reached from it go below, by the role of their assets (servers, users, industrial, +// network and administration). Empty VLANs stay in their zone. Existing zones that hold most of the same +// VLANs are kept, with their name and id. +// input: {vlans: [{id, num, name, zone}], assets: [{id, vlan, type}], flows: [{src, dst}], +// zones: [{id, name}] (bottom first, as stored), sources: ids that stand for the Internet (default +// 'internet' and every 'svc:…')} +// returns {zones: [{id, name, key, vlans: [ids], existing}] bottom first, byVlan: {vlanId: {zone, depth, role, +// moved}}, depth: {vlanId: n|null}} +const ROLE_OF={server:'srv',dc:'srv',db:'srv',hypervisor:'srv',workstation:'users',phone:'users',printer:'users',ap:'users', + plc:'ot',camera:'ot',iot:'ot',switch:'net',router:'net',firewall:'net',gateway:'net'}; +const ROLE_NAME={srv:'Internal servers',users:'Users',ot:'Industrial (OT)',net:'Network and administration',other:'Other'}; +// bottom to top below the floors reached from the Internet +const ROLE_ORDER=['net','ot','users','other','srv']; +function suggestZones({vlans,assets,flows,zones,sources}){ + const isNet=id=>sources?sources.includes(id):(id==='internet'||String(id).startsWith('svc:')); + const vOf={};assets.forEach(a=>{vOf[a.id]=a.vlan;}); + // hops from the Internet, following the flows from source to destination + const out={};flows.forEach(f=>{(out[f.src]=out[f.src]||[]).push(f.dst);}); + const dist={},q=[]; + Object.keys(out).filter(isNet).forEach(id=>{dist[id]=0;q.push(id);}); + while(q.length){const n=q.shift();(out[n]||[]).forEach(m=>{if(dist[m]===undefined&&!isNet(m)){dist[m]=dist[n]+1;q.push(m);}});} + const depth={},role={},count={},types={}; + vlans.forEach(v=>{depth[v.id]=null;count[v.id]={};types[v.id]={};}); + assets.forEach(a=>{if(!(a.vlan in depth))return;const d=dist[a.id];if(d!==undefined&&(depth[a.vlan]===null||d{const C=count[v.id],c=Object.entries(C).sort((x,y)=>y[1]-x[1]||ROLE_ORDER.indexOf(y[0])-ROLE_ORDER.indexOf(x[0])); + const n=c.reduce((t,x)=>t+x[1],0),ot=C.ot||0,name=v.name||'',plcs=types[v.id].plc||0; + if(!n)role[v.id]=null; + else if(plcs||(ot&&OT_NAME.test(name)))role[v.id]='ot'; + else if(USER_NAME.test(name)&&((C.users||0)+ot)*2>=n)role[v.id]='users'; + else if(ot*3>=n)role[v.id]='ot'; + else role[v.id]=c[0][0];}); + // the floors, bottom first + const maxD=Math.max(0,...vlans.map(v=>depth[v.id]||0)),deep=Math.min(maxD,3); + const floors=[...ROLE_ORDER.map(r=>({key:'role:'+r,name:ROLE_NAME[r]})), + ...[3,2,1].filter(d=>d<=deep).map(d=>({key:'depth:'+d,name:d===1?'Exposed to the Internet':d===2?'Reached from the exposed zone':'Back end, 3 hops or more'}))]; + const keyOf=v=>{if(role[v.id]===null)return null;const d=depth[v.id];if(d!==null)return 'depth:'+Math.min(d,3);return 'role:'+role[v.id];}; + const want={};vlans.forEach(v=>{const k=keyOf(v);if(k)(want[k]=want[k]||[]).push(v.id);}); + // keep an existing zone for the floor that holds most of its VLANs (and at least half of the floor) + const cur={};zones.forEach(z=>{cur[z.id]=new Set(vlans.filter(v=>v.zone===z.id).map(v=>v.id));}); + // (and that floor holds at least half of the zone's own VLANs with assets: a catch-all zone is not reused) + const full=z=>[...cur[z]].filter(id=>role[id]!==null).length; + const pairs=[];floors.forEach(f=>{const ids=want[f.key]||[];zones.forEach(z=>{const n=ids.filter(id=>cur[z.id].has(id)).length;if(n&&n*2>=ids.length&&n*2>=full(z.id))pairs.push({f:f.key,z:z.id,n});});}); + pairs.sort((a,b)=>b.n-a.n); + const zoneFor={},taken=new Set();pairs.forEach(p=>{if(!zoneFor[p.f]&&!taken.has(p.z)){zoneFor[p.f]=p.z;taken.add(p.z);}}); + const usedIds=new Set(zones.map(z=>z.id)); + const newId=key=>{let id='z-'+key.replace(/[^a-z0-9]+/g,'-');for(let i=2;usedIds.has(id);i++)id='z-'+key.replace(/[^a-z0-9]+/g,'-')+'-'+i;usedIds.add(id);return id;}; + const outZones=[],byVlan={}; + floors.forEach(f=>{const ids=want[f.key]||[];if(!ids.length)return; + const z=zoneFor[f.key],ex=z?zones.find(x=>x.id===z):null; + outZones.push({id:ex?ex.id:newId(f.key),name:ex?ex.name:f.name,key:f.key,vlans:ids.slice(),existing:!!ex,suggested:f.name});}); + // empty VLANs keep their zone: kept zones stay where they were, below the floors + const stay=vlans.filter(v=>role[v.id]===null); + const kept=new Set(outZones.map(z=>z.id)); + zones.forEach(z=>{const ids=stay.filter(v=>v.zone===z.id).map(v=>v.id);if(!ids.length)return; + const hit=outZones.find(o=>o.id===z.id);if(hit){hit.vlans.push(...ids);return;} + outZones.unshift({id:z.id,name:z.name,key:'keep:'+z.id,vlans:ids,existing:true,suggested:z.name});kept.add(z.id);}); + stay.filter(v=>!zones.some(z=>z.id===v.zone)).forEach(v=>{const z=outZones[0];if(z)z.vlans.push(v.id);}); + outZones.forEach(z=>z.vlans.forEach(id=>{const v=vlans.find(x=>x.id===id);byVlan[id]={zone:z.id,depth:depth[id],role:role[id],moved:v.zone!==z.id};})); + return {zones:outZones,byVlan,depth}; +} + +/* ---- Arranging the plates of a level on rows and columns ---- */ +// plates: [{id, w, D}] in their current order; weight(a, b): how much two VLANs talk (symmetric, >= 0); +// cols: the most plates per row (0 or less: about the square root of their number, a little wider than deep); +// gap: space between plates. Filled row by row from the back, each cell takes the plate with the most +// traffic to the plates already left of it and behind it, then the busiest plate, then the current order: +// VLANs that talk sit side by side, and VLANs with no flow end up last. Returns {id: [x, z]} (plate +// centres, the grid centred on 0) and the number of columns used. +function arrangeGrid(plates,weight,cols,gap=2){ + const n=plates.length;if(!n)return {places:{},cols:0,rows:0}; + const C=Math.max(1,Math.min(n,cols>0?Math.round(cols):Math.ceil(Math.sqrt(n*1.6)))); + const w=(a,b)=>{const v=+weight(a,b);return v>0&&isFinite(v)?v:0;}; + const total={};plates.forEach(p=>{total[p.id]=plates.reduce((t,q)=>t+(q.id!==p.id?w(p.id,q.id):0),0);}); + const left=plates.map((p,i)=>({p,i})),grid=[]; + for(let k=0;k0)nb.push(grid[k-1]);if(k>=C)nb.push(grid[k-C]); + let best=0,bs=null; + left.forEach((e,j)=>{const s=[nb.reduce((t,q)=>t+w(e.p.id,q.id),0),total[e.p.id],-e.i]; + if(!bs||s[0]>bs[0]||(s[0]===bs[0]&&(s[1]>bs[1]||(s[1]===bs[1]&&s[2]>bs[2])))){bs=s;best=j;}}); + grid.push(left.splice(best,1)[0].p); + } + const R=Math.ceil(n/C),colW=Array(C).fill(0),rowD=Array(R).fill(0); + grid.forEach((p,k)=>{colW[k%C]=Math.max(colW[k%C],p.w);rowD[Math.floor(k/C)]=Math.max(rowD[Math.floor(k/C)],p.D);}); + const sum=a=>a.reduce((t,x)=>t+x,0),W=sum(colW)+gap*(C-1),Dt=sum(rowD)+gap*(R-1); + const places={}; + grid.forEach((p,k)=>{const c=k%C,r=Math.floor(k/C); + const x=-W/2+sum(colW.slice(0,c))+gap*c+colW[c]/2,z=-Dt/2+sum(rowD.slice(0,r))+gap*r+rowD[r]/2; + places[p.id]=[Math.round(x*2)/2,Math.round(z*2)/2];}); + return {places,cols:C,rows:R,order:grid.map(p=>p.id)}; +} + +root.StrataAnalysis={suggestZones,arrangeGrid,vlansFromFile,riskScore,riskPriorities,vulnRisk,likelihoodOf,inCidr,nextFreeIp,lanes,flowColours,dependencies,businessImpact,appGraph, stepOdds,blastRadius,attackPaths,chokePoints,outside}; })(typeof globalThis!=='undefined'?globalThis:this); diff --git a/public/app.css b/public/app.css index ccf9818..7807946 100644 --- a/public/app.css +++ b/public/app.css @@ -52,6 +52,7 @@ button:disabled{opacity:.5;cursor:default} button:focus-visible,input:focus-visible,select:focus-visible,textarea:focus-visible,label:focus-within{outline:2px solid var(--accent);outline-offset:2px} .cap{font-family:var(--f-display);font-weight:600;font-size:11px;letter-spacing:.11em;text-transform:uppercase;color:var(--muted)} body.readonly .edit-only{display:none!important} +body.noprop .propose-only{display:none!important} /* Title block */ #cart{position:fixed;top:calc(14px + env(safe-area-inset-top,0px));left:max(16px,env(safe-area-inset-left,0px));z-index:5;display:grid;grid-template-columns:1fr auto;width:min(440px,calc(100% - 32px));box-sizing:border-box;font-size:12px} @@ -710,3 +711,47 @@ body.has-dash{overflow:hidden} #flatSvg .selhalo{animation:selhalo 1.6s ease-in-out infinite} @keyframes selhalo{0%,100%{opacity:.1}50%{opacity:.32}} @media (prefers-reduced-motion:reduce){#flatSvg .selhalo{animation:none}} +.ibtn.signout{border-color:var(--line);color:var(--fg);margin-left:6px;min-height:22px;padding:2px 8px} +.vtools{display:flex;align-items:center;gap:4px;margin:0 0 4px}.vtools .fl{margin-right:auto} +.zone-h{display:flex;align-items:center;gap:4px;min-height:24px}.zone-h>span,.zone-h>.ibtn{float:none}.zone-h>:first-child{margin-left:auto}.zone-h .zsel{display:inline-flex;gap:2px;opacity:1}.zone-h .zsel .ibtn{min-height:22px;padding:1px 6px} +.vrow.off{opacity:.45} +/* Loading bar for long operations */ +#busy{position:fixed;top:0;left:0;right:0;height:3px;z-index:1000;overflow:hidden;background:color-mix(in srgb,var(--fg) 10%,transparent);pointer-events:none} +#busy i{position:absolute;top:0;bottom:0;left:0;width:30%;background:var(--x-net,#2f6fdf);animation:busy 1.1s ease-in-out infinite} +#busy.det i{animation:none;transition:width .2s} +#busy span{position:fixed;top:6px;left:50%;transform:translateX(-50%);font-size:11px;padding:2px 8px;border-radius:3px;background:var(--panel,var(--bg));color:var(--fg);border:1px solid var(--line);white-space:nowrap} +#busy span:empty{display:none} +@keyframes busy{0%{left:-30%}100%{left:100%}} +@media (prefers-reduced-motion:reduce){#busy i{animation:none;width:100%;opacity:.5}} +.arrRow{display:flex;gap:6px;align-items:stretch}.arrRow select{width:auto;flex:0 0 76px;padding:6px 8px}.arrRow .btn{flex:1} +/* Import VLANs */ +.vi-tools{display:flex;gap:4px;align-items:center;margin:6px 0}.vi-tools .search{flex:1;padding:6px 8px;margin-left:4px} +.vi-bulk{margin:4px 0 8px}.vi-bulk>label{display:block;font-size:11px;color:var(--muted);margin-bottom:3px} +.vi-list{max-height:min(48vh,460px);overflow:auto;border:1px solid var(--line);border-radius:3px} +table.vi{width:100%;border-collapse:collapse;font-size:12px} +table.vi th{position:sticky;top:0;background:var(--panel-2);text-align:left;font-weight:600;padding:5px 6px;font-size:11px;color:var(--muted)} +table.vi td{padding:5px 6px;border-top:1px solid var(--line);vertical-align:top} +table.vi td small{display:block;color:var(--muted)}table.vi td.m,table.vi .m{font-family:var(--f-mono)} +table.vi tr:not(.on) td:not(:first-child){opacity:.55} +table.vi select{max-width:140px;font-size:11px;background:var(--panel-2);color:var(--fg);border:1px solid var(--line);border-radius:3px;padding:3px} +table.vi .tag{display:inline-block;font-size:10px;padding:0 5px;border:1px solid var(--line);border-radius:8px;margin:2px 3px 0 0;color:var(--muted)} +table.vi .tag.warn{border-color:var(--high);color:var(--high)} +/* Suggested zones */ +.sz-list{max-height:min(56vh,560px);padding:6px} +.sz{border:1px solid var(--line);border-radius:3px;margin:0 0 8px;background:var(--panel)} +.sz-h{display:flex;flex-direction:column;gap:2px;padding:6px;background:var(--panel-2)} +.sz-h input{padding:5px 7px;font-weight:600} +.sz-h small{color:var(--muted);font-size:11px} +.sz table.vi td{padding:4px 6px} +/* Title block folded into one line */ +#cart .cart-tog{position:absolute;top:3px;right:3px;min-height:24px;padding:0 7px;font-size:12px;z-index:1} +#cart:not(.min) .pl{padding-right:34px} +#cart.min{display:block;width:auto;max-width:calc(100% - 32px)} +#cart .mini{display:flex;align-items:center;gap:10px;padding:5px 38px 5px 10px;border-bottom:0;flex-wrap:wrap} +#cart .mini .back{padding:2px 6px} +#cart .mini .ttl{font-family:var(--f-display);font-weight:700;letter-spacing:.06em;text-transform:uppercase;font-size:14px;white-space:nowrap} +#cart .mini .mn{display:flex;gap:10px;font-family:var(--f-mono);font-size:12px;font-variant-numeric:tabular-nums;color:var(--muted);flex-wrap:wrap} +#cart .mini .mn .net{color:var(--x-net)}#cart .mini .mn .alert{color:var(--crit)} +#cart .mini .signout{margin-left:0} +#cart.min .incband{display:flex} +@media (max-width:760px){#cart.min{max-width:calc(100% - 140px)}#cart .mini{gap:4px 8px}#cart .mini .mn{gap:2px 8px;font-size:11px}} diff --git a/public/app.js b/public/app.js index d303df0..1dcaec3 100644 --- a/public/app.js +++ b/public/app.js @@ -382,9 +382,10 @@ const Store={ }, async pull(){ try{ - const r=await fetch('api/state',{cache:'no-store',credentials:'same-origin'}); - if(!r.ok)throw new Error('HTTP '+r.status); - const s=await r.json(); + Busy.start('Loading the map');let s; + try{const r=await fetch('api/state',{cache:'no-store',credentials:'same-origin'}); + if(!r.ok)throw new Error('HTTP '+r.status); + s=await r.json();}finally{Busy.end();} liveRaw={si:s.si||{},assets:s.assets||{},wan:Array.isArray(s.wan)?s.wan:[],services:Array.isArray(s.services)?s.services:[],risk:s.risk&&typeof s.risk==='object'?s.risk:{}};this.rev=s.rev||0; this.loaded=true;this.applyView(this.bulk?null:0); }catch(e){toast('Could not read the saved data. Check that the server is running, then reload.',true);} @@ -403,10 +404,11 @@ const Store={ async api(method,url,body){ const h={'content-type':'application/json','x-requested-with':'strata'}; if(this.batch){h['x-batch']=this.batch.id;h['x-batch-label']=encodeURIComponent(this.batch.label);} - const r=await fetch(url,{method,credentials:'same-origin',headers:h,body:body===undefined?undefined:JSON.stringify(body)}); + Busy.start();let r; + try{r=await fetch(url,{method,credentials:'same-origin',headers:h,body:body===undefined?undefined:JSON.stringify(body)});}finally{Busy.end();} let j=null;try{j=await r.json();}catch(e){} // a session that has ended (single sign-on): back to the sign-in page - if(r.status===401&&this.me&&this.me.via==='sso'){location.href='auth/login';} + if(r.status===401&&this.me&&this.me.signOut){location.href='auth/login';} if(!r.ok){const err=new Error((j&&j.error)||('HTTP '+r.status));err.status=r.status;err.body=j;throw err;} if(j&&j.rev)this.rev=j.rev; return j; @@ -429,7 +431,7 @@ const Store={ local();this.applyView(this.bulk?null:0); }catch(e){ if(e.status===403)toast(e.message||'You cannot change this.',true); - else if(e.status===413)toast('This item is too large to save.',true); + else if(e.status===413)toast(`This ${kind==='si'?'IS':'item'} is too large to save: the server accepts documents up to MAX_DOC_MB (8 MB unless set otherwise).`,true); else toast('Could not save: '+(e.message||'error')+'.',true); throw e; } @@ -459,12 +461,28 @@ const Store={ putRisk(doc){return this.write('risk',null,doc);}, async many(tasks,label){ this.bulk=true;this.batch={id:'b'+Date.now().toString(36)+Math.random().toString(36).slice(2,6),label};let done=0;const total=tasks.length; - toast(`${label}: 0 / ${total}`,false,true); - try{for(const t of tasks){await t();done++;if(done%5===0||done===total)toast(`${label}: ${done} / ${total}`,false,true);}toast(`${label}: done.`);} + toast(`${label}: 0 / ${total}`,false,true);Busy.start(label);Busy.progress(0,total,label); + try{for(const t of tasks){await t();done++;Busy.progress(done,total);if(done%5===0||done===total)toast(`${label}: ${done} / ${total}`,false,true);}toast(`${label}: done.`);} catch(e){toast(`${label} stopped after ${done} items.`,true);} - finally{this.bulk=false;this.batch=null;this.applyView(0);if(this.view==='draft'){clearTimeout(this._dt);this.saveDraft();}} + finally{Busy.end();this.bulk=false;this.batch=null;this.applyView(0);if(this.view==='draft'){clearTimeout(this._dt);this.saveDraft();}} } }; +// The loading bar at the top: any request still running after 300 ms shows it (moving stripe); a batch +// of writes or an import shows how far it got. Busy.track(promise, label) wraps any long task. +const Busy={n:0,timer:null,label:'',done:0,total:0, + el:()=>document.getElementById('busy'), + start(label){this.n++;if(label)this.label=label;if(this.n===1&&!this.timer)this.timer=setTimeout(()=>{this.timer=null;if(this.n)this.paint();},300);}, + end(){this.n=Math.max(0,this.n-1);if(this.n)return;clearTimeout(this.timer);this.timer=null;this.label='';this.done=this.total=0;const e=this.el();if(e)e.hidden=true;}, + progress(done,total,label){this.done=done;this.total=total;if(label)this.label=label;if(this.n&&!this.timer)this.paint();}, + paint(){const e=this.el();if(!e)return;e.hidden=false;const det=this.total>0,pc=det?Math.round(100*this.done/this.total):0; + e.classList.toggle('det',det);e.firstElementChild.style.width=det?pc+'%':''; + if(det)e.setAttribute('aria-valuenow',pc);else e.removeAttribute('aria-valuenow'); + e.lastElementChild.textContent=this.label?(det?`${this.label} · ${this.done} / ${this.total}`:`${this.label}…`):'';}, + async track(p,label){this.start(label);try{return await (typeof p==='function'?p():p);}finally{this.end();}} +}; +// Sign out (local account or single sign-on), from the header or the account panel +async function signOut(){try{const j=await Store.api('POST','auth/logout',{});location.href=j.location||'auth/login?out=1';}catch(err){toast(err.message,true);}} +document.addEventListener('click',e=>{if(e.target.closest('[data-signout]')){e.preventDefault();signOut();}}); // Replace the whole map (demo, import, delete everything) in one request async function replaceAll(data,label){ const next={si:clone(data.si||{}),assets:clone(data.assets||{}),wan:clone(data.wan||[]),services:clone(data.services||[]),risk:clone(data.risk||{})}; @@ -473,9 +491,10 @@ async function replaceAll(data,label){ if(Store.mode==='local'){liveRaw=next;Store.saveLocal();Store.applyView(0);toast(`${label}: done.`);return;} toast(`${label}…`,false,true); Store.batch={id:'imp'+Date.now().toString(36),label}; + Busy.start(label); try{await Store.api('POST','api/import',next);liveRaw=next;Store.applyView(0);toast(`${label}: done.`);} catch(e){toast(`${label} failed: ${e.message||'error'}.`,true);} - finally{Store.batch=null;} + finally{Store.batch=null;Busy.end();} } /* ================= Compiling the data ================= */ @@ -612,7 +631,12 @@ const nameOf=id=>id==='internet'?'Internet':SVC[id]?SVC[id].name:(byId[id]&&!byI const siteServices=()=>SITE?SITE.services.filter(x=>x.scope==='site'||SITE.usedSvc.has(x.id)):[]; const vnum=a=>{const v=vlanById[a.vlan];return v?v.num:'?';}; -const state={attack:{from:'internet'},attackHl:null,hoverFlow:null,lens:'physical',biz:null,view:'overview',siteId:null,ovSelId:null,selId:null,mode:'zones',hidden:new Set(),minSev:0,minExp:0,dir:'all',color:'risk',flows:true,riskOnly:false,labels:true,ports:false,rotate:false,gap:8,path:false,panel:null,posts:'off',compact:true,arrange:false,flat:false,tool:'move'}; +const state={attack:{from:'internet'},attackHl:null,hoverFlow:null,lens:'physical',biz:null,view:'overview',siteId:null,ovSelId:null,selId:null,mode:'zones',hidden:new Set(),minSev:0,minExp:0,dir:'all',color:'risk',flows:true,riskOnly:false,labels:true,ports:false,rotate:false,gap:8,path:false,panel:null,posts:'off',compact:true,arrange:false,flat:false,tool:'move',hideEmpty:false,anim:null}; +try{const v=localStorage.getItem('strata-anim');if(v==='1'||v==='0')state.anim=v==='1';}catch(e){} +try{state.hideEmpty=localStorage.getItem('strata-hide-empty')==='1';}catch(e){} +// VLANs left out of the model: those without an asset, when "Hide empty VLANs" is on +const vlanEmpty=id=>!ASSETS.some(a=>a.vlan===id); +const vlanShownInLayout=id=>!(state.hideEmpty&&vlanEmpty(id)); const reduceMotion=matchMedia('(prefers-reduced-motion: reduce)').matches; // saved theme applied before the first frame, so the page does not flash try{const t=localStorage.getItem('strata-theme');if(t==='light'||t==='dark')document.documentElement.dataset.theme=t;}catch(e){} @@ -632,17 +656,30 @@ function toast(msg,err,sticky){const t=$('#toast');t.textContent=msg;t.className /* ================= Title block and lists ================= */ const nums=arr=>`
    ${arr.map(([n,l,c])=>`
    ${n}${l}
    `).join('')}
    `; -function storeBadge(){return Store.mode==='api'?`Server${Store.user?' · '+esc(Store.user)+' ('+esc(Store.role)+')':''}`:`This browser only`;} +function storeBadge(){return Store.mode==='api'?`Server${Store.user?' · '+esc(Store.user)+' ('+esc(Store.role)+')':''}${Store.me&&Store.me.signOut?' ':''}`:`This browser only`;} +// The title block can be folded into one line (remembered in this browser): the name, the four key +// figures, back to the overview and sign out stay; the controls below move up with it +let cartMin=false;try{cartMin=localStorage.getItem('strata-cart-min')==='1';}catch(e){} +const cartTog=()=>``; function renderCart(){ const c=$('#cart'); + c.classList.toggle('min',cartMin); + if(cartMin){ + const ov=state.view==='overview',T=k=>SITES.reduce((n,s)=>n+s.stats[k],0); + const f=ov?{a:T('assets'),x:T('xNet'),c:T('crit'),r:T('risk')+WAN.filter(w=>w.risk).length}:{a:SITE.stats.assets,x:SITE.stats.xNet,c:SITE.stats.crit,r:SITE.stats.risk}; + c.innerHTML=`
    ${ov?'':''}${ov?'Strata':'IS '+esc(SITE.name)} + ${f.a} assets${f.x} exposed${f.c} crit.${f.r} risky + ${Store.me&&Store.me.signOut?'':''}${cartTog()}
    ${ov?'':incBand()}`; + document.body.classList.toggle('incident-on',incOn());placeCtl();return; + } if(state.view==='overview'){ const T=k=>SITES.reduce((n,s)=>n+s.stats[k],0); - c.innerHTML=`

    Strata

    Overview
    Sheet00
    + c.innerHTML=`${cartTog()}

    Strata

    Overview
    Sheet00
    Scope ${plural(SITES.length,'IS').replace('ISs','IS')} · ${plural(T('vlans'),'VLAN')}Storage ${storeBadge()}
    ${nums([[T('assets'),'assets'],[T('xNet'),'internet-facing','net'],[T('crit'),'critical vulns','alert'],[T('risk')+WAN.filter(w=>w.risk).length,'risky flows','alert']])}`; }else{ const s=SITE.stats; - c.innerHTML=`

    IS ${esc(SITE.name)}

    ${esc(SITE.kind||'Information system')}
    Sheet${SITE.planche}
    + c.innerHTML=`${cartTog()}

    IS ${esc(SITE.name)}

    ${esc(SITE.kind||'Information system')}
    Sheet${SITE.planche}
    Zones ${ZONES.length}Scale 1 plate = 1 VLANStorage ${storeBadge()}
    ${nums([[s.assets,'assets'],[s.xNet,'internet-facing','net'],[s.crit,'critical vulns','alert'],[s.risk,'risky flows','alert']])}${incBand()}`; } @@ -661,12 +698,14 @@ function renderSiteList(){ function renderVlanList(){ // By zone: listed left to right as on the level. One VLAN per floor: listed top to bottom. const zm=state.mode==='zones',L=zm?['←','→','Move left','Move right']:['↑','↓','Move up','Move down']; - $('#vlans').innerHTML=ZONES.slice().reverse().filter(z=>z.vlans.length).map(z=>{const vs=zm?z.vlans:z.vlans.slice().reverse();return `
    ${esc(z.name.toUpperCase())}${zm&&z.vlans.some(v=>v.place)?` `:zm&&vs.length>1?' left → right':''}
    `+vs.map((v,i)=>{ + $('#vlans').innerHTML=ZONES.slice().reverse().filter(z=>z.vlans.length).map(z=>{const vs=zm?z.vlans:z.vlans.slice().reverse();return `
    ${esc(z.name.toUpperCase())}${zm&&z.vlans.some(v=>v.place)?` `:zm&&vs.length>1?' left → right':''}
    `+vs.map((v,i)=>{ const first=i===0,last=i===vs.length-1,dA=zm?-1:1,dB=zm?1:-1; - return `
    +
    @@ -87,6 +88,8 @@

    Edit

    +
    +

    @@ -145,8 +148,19 @@

    Layout

    -

    Drag a plate anywhere on its level (left, right, front, back), or use the arrows that appear in the VLAN list. “Line up” puts a level back in a row.

    +

    Drag a plate anywhere on its level (left, right, front, back), or use the arrows that appear in the VLAN list. “Line up” puts a level back in a row. The Internet cloud can be dragged too.

    +
    + +

    The Internet on top: each VLAN on the floor of its distance from the Internet along the flows, the others by role. You review it before anything changes.

    +
    +
    + +
    +
    +

    Each zone on rows and columns: VLANs with flows between them side by side, VLANs without flows last. Saved for everyone; “Line up” undoes it for a zone.

    +
    +

    Internet services

    @@ -155,6 +169,8 @@

    Internet services

    Plates (VLANs)

    +
    Show
    +
    @@ -167,6 +183,12 @@

    Display

    +
    + Graphics +
    +

    +
    +
    @@ -175,7 +197,7 @@

    Legend

    -

    Strata 0.10.0, free software under the GNU AGPL v3. Source code

    +

    Strata 0.11.0, free software under the GNU AGPL v3. Source code

    @@ -214,6 +236,7 @@

    Legend

    + diff --git a/server/connectors/builtin/paloalto/index.js b/server/connectors/builtin/paloalto/index.js index 96c8516..4aa6edd 100644 --- a/server/connectors/builtin/paloalto/index.js +++ b/server/connectors/builtin/paloalto/index.js @@ -5,7 +5,7 @@ // Each "allow" security rule becomes flows: source address objects → assets of the IS (by IP) or the // Internet / an Internet service (public addresses, FQDN), destination likewise, services → ports, and the // firewall as the device the flows cross. -import { request, expectOk, ConnectorError, stripTags, parseXml, kids, kid, path, txt, members, parseRange, isPrivateRange, inRange, isIp, slug, hash, norm } from '../../sdk.js'; +import { request, expectOk, ConnectorError, stripTags, parseXml, kids, kid, path, txt, members, parseRange, isPrivateRange, inRange, isIp, slug, hash, norm, intToIp, vlanForIp, holdingVlan } from '../../sdk.js'; // Ports of the most common App-IDs, for rules set to "application-default" const APP_PORTS = { @@ -51,12 +51,13 @@ function findConfig(root) { } function objectsOf(el) { // keys are object names from the configuration file: no prototype, so "__proto__" or "constructor" are just names - const o = { address: Object.create(null), group: Object.create(null), service: Object.create(null), svcGroup: Object.create(null) }; + const o = { address: Object.create(null), group: Object.create(null), service: Object.create(null), svcGroup: Object.create(null), description: Object.create(null) }; if (!el) return o; for (const e of path(el, 'address', 'entry')) { const name = e.attrs.name; if (!name) continue; const v = kid(e, 'ip-netmask') || kid(e, 'ip-range') || kid(e, 'fqdn') || kid(e, 'ip-wildcard'); o.address[name] = v ? { kind: v.name === 'fqdn' ? 'fqdn' : v.name === 'ip-wildcard' ? 'wildcard' : 'ip', value: txt(v) } : { kind: 'unknown', value: '' }; + const d = txt(kid(e, 'description')); if (d) o.description[name] = d.slice(0, 120); } for (const e of path(el, 'address-group', 'entry')) { const name = e.attrs.name; if (!name) continue; @@ -105,7 +106,149 @@ export function parseConfig(xml) { // Panorama order: shared pre, device-group pre, device-group post, shared post (the evaluation order of PAN-OS) const rank = r => r.where === 'pre-rulebase' ? (r.scope === 'shared' ? 0 : 1) : r.where === 'post-rulebase' ? (r.scope === 'shared' ? 3 : 2) : 1; rules.sort((a, b) => rank(a) - rank(b)); - return { kind: dgs.length ? 'panorama' : 'firewall', objects, rules }; + return { kind: dgs.length ? 'panorama' : 'firewall', objects, rules, network: networkOf(config) }; +} +// The network side of the configuration, for building an IS: the layer-3 interfaces and sub-interfaces with +// their address and VLAN tag, the zone of each, and the firewall's host name. On Panorama, from its templates. +function networkOf(config) { + const roots = [config, ...path(config, 'devices', 'entry', 'template', 'entry', 'config')]; + const interfaces = [], zoneOf = Object.create(null); + let hostname = ''; + for (const root of roots) for (const dev of path(root, 'devices', 'entry')) { + hostname = hostname || txt(path(dev, 'deviceconfig', 'system', 'hostname')[0]); + const itf = kid(kid(dev, 'network'), 'interface'); + const add = (e, tag) => { + const ips = path(e, 'ip', 'entry').map(x => x.attrs.name).filter(Boolean); + if (e.attrs.name) interfaces.push({ name: e.attrs.name, tag, ips, comment: txt(kid(e, 'comment')) }); + }; + for (const kind of ['ethernet', 'aggregate-ethernet']) for (const e of path(itf, kind, 'entry')) { + const l3 = kid(e, 'layer3'); + if (l3) add(l3 && { ...l3, attrs: e.attrs, children: [...l3.children, ...kids(e, 'comment')] }, null); + for (const u of path(l3, 'units', 'entry')) add(u, parseInt(txt(kid(u, 'tag')), 10) || null); + } + const vl = kid(itf, 'vlan'); + if (vl) for (const u of path(vl, 'units', 'entry')) add(u, parseInt(txt(kid(u, 'tag')), 10) || parseInt(String(u.attrs.name).split('.')[1], 10) || null); + for (const v of [...path(dev, 'vsys', 'entry')]) for (const z of path(v, 'zone', 'entry')) { + const net = kid(z, 'network'); + for (const k of ['layer3', 'layer2', 'virtual-wire', 'tap']) for (const m of members(kid(net, k))) zoneOf[m] = z.attrs.name; + } + } + return { interfaces, zoneOf, hostname }; +} + +/* ---------------- building the IS ---------------- */ +// What the ports opened to an address say about it (the first that matches); a server otherwise +const TYPE_BY_PORT = [ + [['tcp/502', 'tcp/102', 'tcp/44818', 'udp/47808', 'tcp/20000', 'tcp/2404'], 'plc'], + [['tcp/1433', 'tcp/1521', 'tcp/3306', 'tcp/5432', 'tcp/27017', 'tcp/5022'], 'db'], + [['tcp/88', 'udp/88', 'tcp/636', 'tcp/3268'], 'dc'], + [['tcp/554'], 'camera'], [['tcp/9100', 'tcp/631', 'tcp/515'], 'printer'], + [['tcp/902', 'tcp/3260', 'tcp/8000'], 'hypervisor'], +]; +// What the name or description of an address object says (naming conventions: PC-…, TEL-…, CAM-…, SQL-…) +const TYPE_BY_WORD = [ + [['pc', 'ws', 'wks', 'poste', 'postes', 'laptop', 'desktop', 'workstation', 'portable', 'vdi'], 'workstation'], + [['tel', 'phone', 'voip', 'ipphone', 'telephone', 'sip'], 'phone'], + [['cam', 'camera', 'cctv', 'video', 'videosurveillance'], 'camera'], + [['imp', 'prn', 'print', 'printer', 'imprimante', 'mfp'], 'printer'], + [['ap', 'wifi', 'borne', 'wap'], 'ap'], + [['plc', 'automate', 'automates', 'rtu'], 'plc'], + [['cap', 'sensor', 'capteur', 'iot', 'badge', 'badgeuse', 'compteur', 'meter'], 'iot'], + [['sql', 'db', 'database', 'ora', 'oracle', 'pg', 'postgres', 'mysql', 'mongo'], 'db'], + [['dc', 'ad', 'adds', 'domain'], 'dc'], + [['esx', 'esxi', 'hv', 'hyperv', 'hypervisor', 'nutanix', 'san', 'nas', 'storage', 'stockage'], 'hypervisor'], + [['fw', 'firewall', 'pare'], 'firewall'], + [['rtr', 'router', 'routeur'], 'router'], + [['sw', 'switch', 'commutateur', 'net', 'netdev'], 'switch'], + [['vpn', 'gateway', 'gw'], 'gateway'], +]; +// Adds to the IS what the configuration describes and the map lacks: a zone per security zone, a VLAN per +// layer-3 sub-interface (tag and subnet; untagged interfaces get a number from 4001 on, to change), an asset +// per host address object of a private range (in the VLAN whose range holds it), and the firewall. What is +// already there (same zone name, same VLAN number or range, an asset with the same IP) is kept as it is. +function buildIs(parsed, cfg, state, si, assets, netZones) { + const siteId = cfg.site, N = parsed.network || { interfaces: [], zoneOf: {}, hostname: '' }, stats = { zonesAdded: 0, vlansAdded: 0, assetsAdded: 0, firewallAdded: 0 }; + si.zones = si.zones && si.zones.length ? si.zones : []; + si.vlans = si.vlans || []; + const zoneId = name => { + let z = si.zones.find(x => norm(x.name) === norm(name)); + if (!z) { let id = 'z-pa-' + slug(name); for (let i = 2; si.zones.some(x => x.id === id); i++) id = `z-pa-${slug(name)}-${i}`; z = { id, name: String(name).slice(0, 40) }; si.zones.push(z); stats.zonesAdded++; } + return z.id; + }; + const addrOf = name => { for (const o of Object.values(parsed.objects)) { const a = o.address[name]; if (a && a.kind === 'ip') return a.value; } return name; }; + // VLANs from the interfaces of the zones that are not the Internet + const fwIps = []; + let untagged = 4001; + for (const itf of N.interfaces) { + const zone = N.zoneOf[itf.name]; + for (const raw of itf.ips) { + const v4 = addrOf(raw), m = /^(\d+\.\d+\.\d+\.\d+)\/(\d{1,2})$/.exec(v4); + if (!m || !isIp(m[1])) continue; + if (!zone || netZones.includes(norm(zone))) { fwIps.push({ ip: m[1], net: true }); continue; } + fwIps.push({ ip: m[1], net: false }); + const r = parseRange(`${m[1]}/${m[2]}`), cidr = `${intToIp(r.lo)}/${m[2]}`; + if (+m[2] >= 31) continue; + const num = itf.tag && itf.tag >= 1 && itf.tag <= 4094 ? itf.tag : null; + const same = si.vlans.find(v => v.cidr === cidr) || (num && si.vlans.find(v => +v.num === num)); + if (same) { if (!same.cidr) same.cidr = cidr; continue; } + let n = num; + if (!n) { while (si.vlans.some(v => +v.num === untagged)) untagged++; n = untagged++; } + si.vlans.push({ id: `v-pa-${slug(siteId)}-${n}`, num: n, name: (itf.comment || `${zone} ${itf.name}${num ? '' : ' (untagged)'}`).slice(0, 50), cidr, zone: zoneId(zone) }); + stats.vlansAdded++; + } + } + if (!si.zones.length) si.zones.push({ id: 'z-def', name: 'Main zone' }); + // the ports opened to each address, from the allow rules, to guess its type + const portsTo = Object.create(null); + for (const r of parsed.rules) { + if (r.action !== 'allow' || r.disabled) continue; + const ports = r.service.flatMap(sv => { const o = Object.values(parsed.objects).map(x => x.service[sv]).find(Boolean); const mm = /^(tcp|udp)[-_/ ]?(\d+)$/i.exec(sv); return o || (mm ? [`${mm[1].toLowerCase()}/${mm[2]}`] : []); }); + for (const d of r.destination) (portsTo[d] = portsTo[d] || new Set()) && ports.forEach(p => portsTo[d].add(p)); + } + const sources = new Set(parsed.rules.filter(r => r.action === 'allow' && !r.disabled).flatMap(r => r.source)); + // the type: its name or description first, then the ports opened to it, then an address that only opens + // connections is a workstation, and anything else a server + const typeOf = (name, desc) => { + const words = new Set(norm(`${name} ${desc || ''}`).split(' ')); + for (const [list, t] of TYPE_BY_WORD) if (list.some(w => words.has(w))) return t; + const p = portsTo[name]; + if (p) for (const [list, t] of TYPE_BY_PORT) if (list.some(x => p.has(x))) return t; + if (!p && sources.has(name)) return 'workstation'; + return 'server'; + }; + // assets from host objects of private ranges that no asset of the IS has yet + const byIp = new Map(assets.map(([id, a]) => [a.ip, id])), out = {}; + for (const [scope, o] of Object.entries(parsed.objects)) { + if (cfg.scope && scope !== cfg.scope && scope !== 'shared') continue; + for (const [name, a] of Object.entries(o.address)) { + if (a.kind !== 'ip') continue; + const ip = /^(\d+\.\d+\.\d+\.\d+)(?:\/32)?$/.exec(a.value); + if (!ip || !isIp(ip[1]) || !isPrivateRange(parseRange(ip[1])) || byIp.has(ip[1]) || fwIps.some(f => f.ip === ip[1])) continue; + if (Object.keys(out).length >= 5000) break; + const vlan = vlanForIp(si, ip[1]) || holdingVlan(si, siteId, 'To sort (Palo Alto)'); + let id = `pa-${slug(siteId)}-${slug(name)}`; + for (let i = 2; state.assets[id] || out[id]; i++) id = `pa-${slug(siteId)}-${slug(name)}-${i}`; + out[id] = { site: siteId, name: name.slice(0, 80), type: typeOf(name, o.description && o.description[name]), vlan: vlan.id, ip: ip[1], os: '', role: (o.description && o.description[name]) || '', crit: 2, vulns: [], contacts: [], apps: [], notes: 'Created from a Palo Alto address object.', ext: { paloalto: name } }; + byIp.set(ip[1], id); stats.assetsAdded++; + } + } + // the firewall itself: an existing firewall asset at one of its addresses, or a new one + let firewall = null; + const inside = fwIps.find(f => !f.net); + const existing = assets.find(([, a]) => a.type === 'firewall' && fwIps.some(f => f.ip === a.ip)) || (!inside && assets.find(([, a]) => a.type === 'firewall')); + if (existing) firewall = existing[0]; + else if (inside) { + const name = N.hostname || 'Palo Alto firewall', id = `pa-${slug(siteId)}-fw-${slug(name)}`; + if (!state.assets[id]) { + const vlan = vlanForIp(si, inside.ip) || holdingVlan(si, siteId, 'To sort (Palo Alto)'); + out[id] = { site: siteId, name, type: 'firewall', vlan: vlan.id, ip: inside.ip, os: 'PAN-OS', role: 'Firewall', crit: 3, vulns: [], contacts: [], apps: [], notes: 'Created from the Palo Alto configuration.', ext: { paloalto: 'firewall' } }; + stats.firewallAdded = 1; + } + firewall = id; + } + // the new assets take part in the rules like the others + for (const [id, a] of Object.entries(out)) assets.push([id, a]); + return { assets: out, firewall, stats }; } /* ---------------- turning rules into flows ---------------- */ @@ -116,8 +259,11 @@ export function rulesToChanges(parsed, cfg, state) { const si = JSON.parse(JSON.stringify(si0)); si.flows = si.flows || []; si.services = si.services || []; const assets = Object.entries(state.assets).filter(([, a]) => a.site === siteId); - const fw = cfg.firewall && state.assets[cfg.firewall] && state.assets[cfg.firewall].site === siteId ? cfg.firewall : null; const netZones = String(cfg.internetZones || 'untrust, internet, outside, wan').split(/[\s,;]+/).map(norm).filter(Boolean); + // "Build the IS": zones, VLANs, assets and the firewall from the configuration, before the flows + const built = cfg.build ? buildIs(parsed, cfg, state, si, assets, netZones) : null; + const newAssets = built ? built.assets : {}; + const fw = cfg.firewall && state.assets[cfg.firewall] && state.assets[cfg.firewall].site === siteId ? cfg.firewall : built && built.firewall ? built.firewall : null; const maxExpand = Math.max(1, parseInt(cfg.maxExpand, 10) || 12); const scopeFilter = String(cfg.scope || '').trim(); const warn = [], skipped = []; @@ -194,6 +340,8 @@ export function rulesToChanges(parsed, cfg, state) { return [...out]; }; + const vlanOfId = id => (newAssets[id] || state.assets[id] || {}).vlan; + const sameVlan = (a, b) => !a.startsWith('svc:') && a !== 'internet' && !b.startsWith('svc:') && b !== 'internet' && vlanOfId(a) && vlanOfId(a) === vlanOfId(b); const imported = new Map(); // key src>dst>port → flow let used = 0, ignored = 0; for (const r of parsed.rules) { @@ -217,7 +365,7 @@ export function rulesToChanges(parsed, cfg, state) { imported.set(key, { id: 'f-pa-' + hash(key), src: s, dst: d, port: p, vol: 1, risk, note: `Palo Alto rule “${r.name}”${r.description ? ` — ${r.description.slice(0, 120)}` : ''}${r.application.length && !r.application.includes('any') ? ` · App-ID: ${r.application.join(', ')}` : ''}${risk ? ' · administration or database port open from the Internet' : ''}`, - ...(fw && s !== fw && d !== fw ? { via: [fw] } : {}), + ...(fw && s !== fw && d !== fw && !(built && sameVlan(s, d)) ? { via: [fw] } : {}), pan: { rules: [r.name], scope: r.scope }, }); } @@ -241,10 +389,11 @@ export function rulesToChanges(parsed, cfg, state) { si.flows = si.flows.filter(f => !f.pan || imported.has(keyOf(f))); removed = before - si.flows.length; } - const changed = JSON.stringify(si) !== JSON.stringify(si0); + // the empty lists added above are no change: compare the IS as it was, with them + const changed = JSON.stringify(si) !== JSON.stringify({ ...si0, flows: si0.flows || [], services: si0.services || [] }) || Object.keys(newAssets).length > 0; return { - changes: changed ? { si: { [siteId]: si }, assets: {} } : null, - stats: { rules: parsed.rules.length, used, ignored, flows: imported.size, added, tagged, removed, services: si.services.length - (si0.services || []).length }, + changes: changed ? { si: { [siteId]: si }, assets: newAssets } : null, + stats: { rules: parsed.rules.length, used, ignored, flows: imported.size, added, tagged, removed, services: si.services.length - (si0.services || []).length, ...(built ? built.stats : {}) }, warnings: [...new Set(warn)].slice(0, 40), skipped: [...new Set(skipped)].slice(0, 60), }; } @@ -255,7 +404,7 @@ export async function run(cfg, ctx) { const res = rulesToChanges(parsed, cfg, ctx.state); const siName = ctx.state.si[cfg.site]?.name || cfg.site; const st = res.stats; - const summary = `${st.rules} rule(s) read (${parsed.kind}), ${st.used} allow rule(s) drawn, ${st.ignored} deny or disabled ignored; ${st.added} new flow(s), ${st.removed} removed, ${st.services} new Internet service(s).`; + const summary = `${cfg.build ? `IS built: ${st.zonesAdded} new zone(s), ${st.vlansAdded} new VLAN(s), ${st.assetsAdded} new asset(s)${st.firewallAdded ? ' and the firewall' : ''}; ` : ''}${st.rules} rule(s) read (${parsed.kind}), ${st.used} allow rule(s) drawn, ${st.ignored} deny or disabled ignored; ${st.added} new flow(s), ${st.removed} removed, ${st.services} new Internet service(s).`; return { proposals: res.changes ? [{ siteId: cfg.site, diff --git a/server/connectors/builtin/paloalto/strata-module.json b/server/connectors/builtin/paloalto/strata-module.json index c745fa7..f662dd6 100644 --- a/server/connectors/builtin/paloalto/strata-module.json +++ b/server/connectors/builtin/paloalto/strata-module.json @@ -23,9 +23,10 @@ "fields": [ "site", "firewall", - "scope" + "scope", + "build" ], - "help": "On the firewall: Device › Setup › Operations › Export named configuration snapshot (running-config.xml). On Panorama: Panorama › Setup › Operations, same export; then give the device group below. Allow rules become flows: address objects are matched to the assets of the IS by IP, public addresses and FQDNs to Internet services." + "help": "On the firewall: Device › Setup › Operations › Export named configuration snapshot (running-config.xml). On Panorama: Panorama › Setup › Operations, same export; then give the device group below. Allow rules become flows: address objects are matched to the assets of the IS by IP, public addresses and FQDNs to Internet services. Tick “Build the IS” on an empty or incomplete IS: the configuration also gives the zones, the VLANs (layer-3 sub-interfaces), the assets (host address objects) and the firewall." }, "fields": [ { @@ -57,6 +58,12 @@ "type": "site", "required": true }, + { + "k": "build", + "label": "Build the IS: zones, VLANs (interfaces), assets (address objects) and the firewall, when the map lacks them", + "type": "bool", + "default": false + }, { "k": "firewall", "label": "Firewall asset the flows cross", diff --git a/server/connectors/host.js b/server/connectors/host.js index 56920d3..842e842 100644 --- a/server/connectors/host.js +++ b/server/connectors/host.js @@ -2,20 +2,20 @@ // Runs one connector module operation in its own process, started by runner.js with the Node.js // permission model: this process reads only its module's folder and the SDK, cannot write files or start // other processes, and its environment holds none of the server's settings or passwords. -// Messages: in {op: 'test' | 'run' | 'exports', main, cfg, state, file}; out {log} while it works, then +// Messages: in {op: 'test' | 'preview' | 'run' | 'exports', main, cfg, state, file, picks}; out {log} while it works, then // {ok: true, result} or {ok: false, error, expected}. import { pathToFileURL } from 'node:url'; import * as sdk from './sdk.js'; const send = m => new Promise(res => process.send(m, () => res())); -process.once('message', async ({ op, main, cfg, state, file }) => { +process.once('message', async ({ op, main, cfg, state, file, picks }) => { try { const mod = await import(pathToFileURL(main).href); // the functions the module offers, to check a package (scripts/check-registry.mjs) if (op === 'exports') { await send({ ok: true, result: Object.keys(mod).filter(k => typeof mod[k] === 'function').sort() }); process.exit(0); } const fn = mod[op]; if (typeof fn !== 'function') throw new sdk.ConnectorError(`This module has no ${op}() function.`); - const ctx = { state, file: file || null, sdk, log: (...a) => { process.send({ log: a.map(String).join(' ').slice(0, 2000) }); } }; + const ctx = { state, file: file || null, picks: picks || null, sdk, log: (...a) => { process.send({ log: a.map(String).join(' ').slice(0, 2000) }); } }; const result = await fn(cfg, ctx); await send({ ok: true, result: result === undefined ? null : JSON.parse(JSON.stringify(result)) }); } catch (e) { diff --git a/server/connectors/index.js b/server/connectors/index.js index db4aef3..28d03b0 100644 --- a/server/connectors/index.js +++ b/server/connectors/index.js @@ -41,6 +41,37 @@ function ownSite(changes, siteId, state) { return out; } +// The choices made after a preview, as the module receives them in ctx.picks: {key: {zone}}, the key of +// each item chosen and, when the module offers zones, the zone id of the IS or "new:" +export function cleanPicks(p) { + if (p === undefined || p === null) return null; + const list = Array.isArray(p) ? p.map(k => [k, {}]) : p && typeof p === 'object' ? Object.entries(p) : null; + if (!list) throw new ConnectorError('picks must be a list of keys or {key: {zone}}'); + if (list.length > 5000) throw new ConnectorError('At most 5000 items can be chosen in one import.'); + const out = Object.create(null); + for (const [k, v] of list) { + if (typeof k !== 'string' || !k || k.length > 200 || ['__proto__', 'constructor', 'prototype'].includes(k)) continue; + const o = {}; + if (v && typeof v === 'object' && typeof v.zone === 'string' && v.zone.trim()) o.zone = v.zone.trim().slice(0, 120); + out[k] = o; + } + return { ...out }; +} +// What a preview may answer: the items of the file people choose from +function cleanPreview(r) { + if (!r || typeof r !== 'object' || !Array.isArray(r.items)) throw new Error('the module answered its preview without a list of items'); + const seen = new Set(), items = [], s = (v, n) => (typeof v === 'string' || typeof v === 'number' ? String(v).slice(0, n) : ''); + for (const x of r.items.slice(0, 5000)) { + if (!x || typeof x !== 'object') continue; + const key = s(x.key, 200); + if (!key || seen.has(key)) continue; + seen.add(key); + items.push({ key, label: s(x.label, 200) || key, detail: s(x.detail, 300), tags: Array.isArray(x.tags) ? x.tags.map(t => s(t, 40)).filter(Boolean).slice(0, 6) : [], + warn: Array.isArray(x.warn) ? x.warn.map(t => s(t, 80)).filter(Boolean).slice(0, 4) : [], on: x.on !== false, zone: s(x.zone, 80), group: s(x.group, 80) }); + } + return { items, summary: s(r.summary, 500), skipped: Array.isArray(r.skipped) ? r.skipped.map(t => s(t, 200)).filter(Boolean).slice(0, 40) : [] }; +} + // opts: dataDir, providedDir (MODULES_DIR), installedDir (DATA_DIR/modules), strataVersion, timeoutMs, // and the server's getState, upsertProposal, log, writeJson, broadcast export function createConnectors({ dataDir, providedDir, installedDir, strataVersion, timeoutMs, getState, upsertProposal, log, writeJson, broadcast }) { @@ -122,7 +153,19 @@ export function createConnectors({ dataDir, providedDir, installedDir, strataVer finally { testing.delete(id); } return typeof r === 'string' ? r : r && typeof r.message === 'string' ? r.message : 'Connected.'; } - // Starts a run; resolves when it is finished. opts: {by, file: {name, text}, overrides} + // Reads a file for a module whose import starts with a choice (file.pick): returns the items to choose + // from. Nothing is recorded: the import itself is the run that follows, with the picks. + async function preview(id, { file, overrides } = {}) { + if (!known(id)) throw new ConnectorError('Unknown connector'); + const m = mod(id); + if (!m.file || !m.file.pick) throw new ConnectorError(`${m.name} has no choice to make before an import.`); + if (!file) throw new ConnectorError('Choose the file.'); + const cfg = effective(id, fileOverrides(id, overrides)); + const state = m.permissions.map ? JSON.parse(JSON.stringify(getState())) : null; + const r = await runModule(m, 'preview', { cfg, state, file: { name: String(file.name || 'file').slice(0, 120), text: String(file.text) } }, { timeoutMs: Math.min(timeoutMs, 5 * 60000), log }); + return cleanPreview(r); + } + // Starts a run; resolves when it is finished. opts: {by, file: {name, text}, overrides, picks} async function run(id, opts = {}) { if (!known(id)) throw new ConnectorError('Unknown connector'); const m = mod(id); @@ -142,6 +185,7 @@ export function createConnectors({ dataDir, providedDir, installedDir, strataVer const res = await runModule(m, 'run', { cfg, state: m.permissions.map ? state : null, file: opts.file ? { name: fileName, text: String(opts.file.text) } : null, + picks: opts.file && m.file.pick ? cleanPicks(opts.picks) : null, }, { timeoutMs, log }) || {}; if (res.proposals !== undefined && !Array.isArray(res.proposals)) throw new Error('the module answered without a list of proposals'); const sites = Array.isArray(cfg.sites) && cfg.sites.length ? new Set(cfg.sites) : null, rejected = []; @@ -195,7 +239,7 @@ export function createConnectors({ dataDir, providedDir, installedDir, strataVer list: () => ({ connectors: [...MODULES.keys()].map(publicView), runs: store.runs.slice(0, 30), problems }), get: id => (known(id) ? publicView(id) : null), module: id => (known(id) ? mod(id) : null), - update, test, run, reload, forget, + update, test, preview, run, reload, forget, isRunning: id => running.has(id) || testing.has(id), exists: known, }; diff --git a/server/connectors/modules.js b/server/connectors/modules.js index 201b4ba..b51a347 100644 --- a/server/connectors/modules.js +++ b/server/connectors/modules.js @@ -96,7 +96,19 @@ export function checkManifest(m, { strataVersion } = {}) { if (f.help !== undefined && !str(f.help, 1200)) throw new Error('file.help must be text'); if (f.role !== undefined && !['contributor', 'approver'].includes(f.role)) throw new Error('file.role must be contributor or approver'); if (f.fields !== undefined && !(Array.isArray(f.fields) && f.fields.every(k => keys.has(k)))) throw new Error('file.fields must list some of its settings'); - file = { label: f.label, accept: f.accept || '', help: f.help || '', role: f.role || 'approver', fields: f.fields || [] }; + // pick: before the import, the module reads the file (preview) and people choose what to bring, + // and, with zones: true, the zone of the IS each chosen item goes to + let pick = null; + if (f.pick !== undefined) { + const p = f.pick; + if (!p || typeof p !== 'object' || Array.isArray(p)) throw new Error('file.pick must be an object'); + if (p.title !== undefined && !str(p.title, 80)) throw new Error('file.pick.title must be text (80 characters at most)'); + if (p.help !== undefined && !str(p.help, 600)) throw new Error('file.pick.help must be text'); + if (p.zones !== undefined && typeof p.zones !== 'boolean') throw new Error('file.pick.zones must be true or false'); + if (p.zones && !fields.some(x => x.type === 'site' && (f.fields || []).includes(x.k))) throw new Error('file.pick.zones needs a setting of type site in file.fields (the IS whose zones are offered)'); + pick = { title: p.title || 'Choose what to import', help: p.help || '', zones: !!p.zones }; + } + file = { label: f.label, accept: f.accept || '', help: f.help || '', role: f.role || 'approver', fields: f.fields || [], pick }; } if (live === false && !file) throw new Error('a module with live: false must import files'); return { diff --git a/server/server.js b/server/server.js index 0f33822..1de7072 100644 --- a/server/server.js +++ b/server/server.js @@ -9,7 +9,7 @@ import fsp from 'node:fs/promises'; import path from 'node:path'; import crypto from 'node:crypto'; import { fileURLToPath } from 'node:url'; -import { createConnectors } from './connectors/index.js'; +import { createConnectors, cleanPicks } from './connectors/index.js'; import { createStore } from './connectors/store.js'; import { ID_RE as MODULE_ID_RE } from './connectors/modules.js'; import { ssoConfig, createSso, loginPage } from './sso.js'; @@ -44,8 +44,12 @@ const CONFIG = { backupKeep: Math.max(0, parseInt(env.BACKUP_KEEP || '14', 10)), backupEveryHours: Math.max(1, parseInt(env.BACKUP_EVERY_HOURS || '24', 10)), historyKeep: Math.max(50, parseInt(env.HISTORY_KEEP || '5000', 10)), + // the history also stops at this size: each change keeps the whole document before and after it, so a + // large IS edited often (plates moved, flows imported) would otherwise grow it by a megabyte a change + historyMaxBytes: Math.max(1, parseFloat(env.HISTORY_MAX_MB || '200') || 200) * 1024 * 1024, maxImportBytes: parseInt(env.MAX_IMPORT_MB || '20', 10) * 1024 * 1024, - maxDocBytes: 512 * 1024, + // one document: an IS holds its VLANs and all its flows (about 300 bytes a flow, 8 MB ≈ 25 000 flows) + maxDocBytes: Math.max(0.5, parseFloat(env.MAX_DOC_MB || '8') || 8) * 1024 * 1024, // Failed sign-ins: 0 turns a limit off loginMaxFails: Math.max(0, parseInt(env.LOGIN_MAX_FAILS ?? '10', 10) || 0), loginMaxFailsPerAddress: Math.max(0, parseInt(env.LOGIN_MAX_FAILS_PER_ADDRESS ?? '20', 10) || 0), @@ -134,7 +138,10 @@ async function backup(reason) { One line per changed document: {rev, at, user, batch, label, kind, id, before, after}. kind: si | assets | wan | services | risk | all (whole map, for imports). Rolling back replays the "before" of later lines, and is itself recorded, so a rollback can be undone too. */ -let history = []; +let history = [], historyBytes = 0; +// the size of each line in the history file, kept aside (the lines go to the file and the API as they are) +const LINE_BYTES = new WeakMap(); +const lineBytes = h => { let n = LINE_BYTES.get(h); if (n === undefined) { n = Buffer.byteLength(JSON.stringify(h)) + 1; LINE_BYTES.set(h, n); } return n; }; function loadHistory() { if (!fs.existsSync(HISTORY_FILE)) return; history = fs.readFileSync(HISTORY_FILE, 'utf8').split('\n').filter(Boolean).map(l => { try { return JSON.parse(l); } catch { return null; } }).filter(Boolean); @@ -145,12 +152,16 @@ function loadHistory() { log(`History: ${history.length} changes, back to rev ${history.length ? history[0].rev : state.rev}`); } function trimHistory(rewrite) { - if (history.length <= CONFIG.historyKeep && !rewrite) return; - if (history.length > CONFIG.historyKeep) { - // keep whole revisions: drop the oldest ones until under the limit - let cut = history.length - CONFIG.historyKeep; - while (cut < history.length && history[cut].rev === history[cut - 1]?.rev) cut++; - history = history.slice(cut); + if (rewrite) historyBytes = history.reduce((t, h) => t + lineBytes(h), 0); + if (history.length <= CONFIG.historyKeep && historyBytes <= CONFIG.historyMaxBytes && !rewrite) return; + if (history.length > CONFIG.historyKeep || historyBytes > CONFIG.historyMaxBytes) { + // keep whole revisions, and always the latest one: drop the oldest until under both limits + let cut = Math.max(0, history.length - CONFIG.historyKeep), bytes = historyBytes; + for (let i = 0; i < cut; i++) bytes -= lineBytes(history[i]); + while (cut < history.length - 1 && bytes > CONFIG.historyMaxBytes) bytes -= lineBytes(history[cut++]); + while (cut > 0 && cut < history.length && history[cut].rev === history[cut - 1].rev) bytes -= lineBytes(history[cut++]); + if (cut >= history.length) { cut = history.length - 1; while (cut > 0 && history[cut - 1].rev === history[cut].rev) cut--; bytes = history.slice(cut).reduce((t, h) => t + lineBytes(h), 0); } + if (cut > 0) { log(`History: ${cut} oldest change(s) dropped (limits: ${CONFIG.historyKeep} changes, ${Math.round(CONFIG.historyMaxBytes / 1048576)} MB)`); history = history.slice(cut); historyBytes = bytes; } } writeJson(HISTORY_FILE, history.map(h => JSON.stringify(h)).join('\n') + (history.length ? '\n' : '')); } @@ -163,6 +174,7 @@ async function commitChanges(changes, auth, meta = {}) { state.updatedAt = new Date().toISOString(); const lines = real.map(c => ({ rev: state.rev, at: state.updatedAt, user: auth.user || '', batch: meta.batch || `r${state.rev}`, label: meta.label || '', kind: c.kind, id: c.id ?? null, before: c.before ?? null, after: c.after ?? null })); history.push(...lines); + for (const l of lines) historyBytes += lineBytes(l); await writeJson(DB_FILE, state); const text = lines.map(l => JSON.stringify(l)).join('\n') + '\n'; writeChain = writeChain.then(() => fsp.appendFile(HISTORY_FILE, text, { mode: 0o600 })).catch(e => log('ERROR history', e.message)); @@ -298,9 +310,9 @@ const hashPass = (pass, salt) => crypto.scryptSync(String(pass), salt, 32).toStr function loadUsers() { try { const st = fs.statSync(USERS_FILE); usersMtime = st.mtimeMs; const p = JSON.parse(fs.readFileSync(USERS_FILE, 'utf8')); users = isObj(p.users) ? p.users : {}; } catch { users = {}; } - authCache.clear(); + } -function saveUsers() { authCache.clear(); return writeJson(USERS_FILE, JSON.stringify({ users }, null, 2)).then(() => { try { usersMtime = fs.statSync(USERS_FILE).mtimeMs; } catch { } }); } +function saveUsers() { return writeJson(USERS_FILE, JSON.stringify({ users }, null, 2)).then(() => { try { usersMtime = fs.statSync(USERS_FILE).mtimeMs; } catch { } }); } function setUser(name, { role, password, disabled }) { const u = own(users, name) || { createdAt: new Date().toISOString() }; if (role) u.role = role; @@ -368,50 +380,38 @@ function recordFailure(kind, key, credential) { if (m.size > 10000) m.delete(m.keys().next().value); } -// Successful logins are cached for 10 minutes, so scrypt runs once per session, not per request -const authCache = new Map(); -// Returns the account and role, null when the credentials are missing or wrong, or {retryAfter} (ms) -// while sign-ins are locked -// Single sign-on and API tokens come first (sso.js); {denied} is a refusal to explain (403) +// Returns the account and role, or null when there is no valid sign-in. Local accounts sign in once +// with the form (/auth/login) and then carry a session cookie; scripts use API tokens. HTTP Basic is +// no longer accepted (0.11). {denied} is a refusal to explain (403) function authenticate(req) { const other = sso.fromSession(req) || sso.fromToken(req) || sso.fromHeaders(req); + if (other && other.via === 'local') { + // the account may have been disabled, deleted, given another role or a new password since + const u = own(users, other.user); + if (!u || u.disabled || !u.hash || (u.passwordChangedAt && Date.parse(u.passwordChangedAt) > other.since) + || sso.localLogin === 'off' || (sso.localLogin === 'admins' && u.role !== 'admin')) { sso.endUserSessions(other.user); return null; } + return { ...other, role: CONFIG.readOnly ? 'viewer' : (RANK[u.role] !== undefined ? u.role : 'viewer') }; + } if (other) return other.denied ? other : { ...other, role: CONFIG.readOnly ? 'viewer' : other.role }; if (!authEnabled() && !sso.enabled()) return { user: '', role: CONFIG.readOnly ? 'viewer' : 'admin', open: true, via: 'open' }; - // LOCAL_LOGIN=off: no local sign-in at all; admins: only the accounts whose stored role is admin. The - // others are not even checked, so the answer (401) says nothing of their password - if (sso.localLogin === 'off') return null; - if (sso.localLogin === 'admins' && own(users, basicName(req))?.role !== 'admin') return null; - return authenticateLocal(req); -} -function basicName(req) { - const h = req.headers.authorization || ''; - if (!h.startsWith('Basic ')) return ''; - const raw = Buffer.from(h.slice(6), 'base64').toString('utf8'), i = raw.indexOf(':'); - return i < 1 ? '' : raw.slice(0, i); + return null; } -function authenticateLocal(req) { - const h = req.headers.authorization || ''; - if (!h.startsWith('Basic ')) return null; - const raw = Buffer.from(h.slice(6), 'base64').toString('utf8'), i = raw.indexOf(':'); - const name = i < 1 ? '' : raw.slice(0, i), pass = raw.slice(i + 1); - const key = crypto.createHash('sha256').update(raw).digest('hex'), hit = authCache.get(key), addr = clientAddress(req); - const ok = hit && hit.until > Date.now() && hit.addrs.has(addr); - if (!ok) { - const wait = Math.max(lockedFor('address', addr), name ? lockedFor('account', name) : 0); - if (wait) return { retryAfter: wait }; - const u = name ? own(users, name) : undefined; - let good = !!(u && !u.disabled && u.hash); - if (good) { const a = Buffer.from(hashPass(pass, u.salt), 'hex'), b = Buffer.from(u.hash, 'hex'); good = a.length === b.length && crypto.timingSafeEqual(a, b); } - if (!good) { recordFailure('address', addr, key); if (name) recordFailure('account', name, key); return null; } - failures.account.delete(name); - const addrs = hit && hit.until > Date.now() ? hit.addrs : new Set(); - addrs.add(addr); - authCache.set(key, { until: Date.now() + 600000, addrs }); - } - const u = own(users, name); - if (!u || u.disabled) return null; - const role = CONFIG.readOnly ? 'viewer' : (RANK[u.role] !== undefined ? u.role : 'viewer'); - return { user: name, role, via: 'local' }; +// Checks a local account's password, with the limits on failed sign-ins: true, false, or {retryAfter} (ms) +function checkLocalPassword(name, pass, addr) { + name = String(name || ''); pass = String(pass || ''); + // LOCAL_LOGIN=off: no local sign-in at all; admins: only the accounts whose stored role is admin. The + // others are not even checked, so the answer says nothing of their password + if (sso.localLogin === 'off') return false; + if (sso.localLogin === 'admins' && own(users, name)?.role !== 'admin') return false; + const wait = Math.max(lockedFor('address', addr), name ? lockedFor('account', name) : 0); + if (wait) return { retryAfter: wait }; + const key = crypto.createHash('sha256').update(`${name}:${pass}`).digest('hex'); + const u = name ? own(users, name) : undefined; + let good = !!(u && !u.disabled && u.hash); + if (good) { const a = Buffer.from(hashPass(pass, u.salt), 'hex'), b = Buffer.from(u.hash, 'hex'); good = a.length === b.length && crypto.timingSafeEqual(a, b); } + if (!good) { recordFailure('address', addr, key); if (name) recordFailure('account', name, key); return false; } + failures.account.delete(name); + return true; } const can = (auth, role) => RANK[auth.role] >= RANK[role]; const need = (auth, role) => { if (!can(auth, role)) throw httpError(403, `This needs the ${role} role (you are ${auth.role}).`); }; @@ -430,7 +430,7 @@ const SEC_HEADERS = { // no other site's window or page may hold a reference to the app, or load its responses 'Cross-Origin-Opener-Policy': 'same-origin', 'Cross-Origin-Resource-Policy': 'same-origin', - 'Content-Security-Policy': "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; font-src 'self'; connect-src 'self'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'; form-action 'none'", + 'Content-Security-Policy': "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; font-src 'self'; connect-src 'self'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'; form-action 'self'", }; function send(res, status, body, headers = {}) { const isJson = typeof body !== 'string' && !Buffer.isBuffer(body); @@ -442,6 +442,15 @@ const fail = (res, status, error, extra) => send(res, status, { error, ...(extra // (closing the connection at once left it with a network error). Past the limit by more than // DISCARD_MAX, the connection is cut: nobody waits for a body that large. const DISCARD_MAX = 64 * 1024 * 1024; +// The raw body as text (a sign-in form), up to limit bytes +function readRaw(req, limit) { + return new Promise((resolve, reject) => { + let size = 0; const chunks = []; + req.on('data', c => { size += c.length; if (size > limit) { reject(httpError(413, 'Payload too large')); req.destroy(); } else chunks.push(c); }); + req.on('end', () => resolve(Buffer.concat(chunks).toString('utf8'))); + req.on('error', reject); + }); +} function readBody(req, limit) { return new Promise((resolve, reject) => { let size = 0, over = false; const chunks = []; @@ -494,7 +503,7 @@ async function api(req, res, parts, url, auth) { const ok = rev => send(res, 200, { ok: true, rev: rev ?? state.rev }); if (r1 === 'me' && !id && method === 'GET') return send(res, 200, { - user: auth.user, role: auth.role, auth: !auth.open, via: auth.via, signOut: auth.via === 'sso', sso: sso.oidc || sso.header, readOnly: CONFIG.readOnly, allowSelfApproval: CONFIG.allowSelfApproval, + user: auth.user, role: auth.role, auth: !auth.open, via: auth.via, signOut: auth.via === 'sso' || auth.via === 'local', sso: sso.oidc || sso.header, readOnly: CONFIG.readOnly, allowSelfApproval: CONFIG.allowSelfApproval, canWrite: can(auth, 'approver'), canPropose: can(auth, 'contributor'), canReview: can(auth, 'approver'), canAdmin: can(auth, 'admin'), connectors: !!connectors, sourceUrl: CONFIG.sourceUrl, muster: muster ? { url: muster.url } : null, }); if (r1 === 'me' && id === 'password' && method === 'PUT') { @@ -504,7 +513,12 @@ async function api(req, res, parts, url, auth) { const given = u ? Buffer.from(hashPass(b?.current || '', u.salt), 'hex') : null, stored = u ? Buffer.from(u.hash, 'hex') : null; if (!u || given.length !== stored.length || !crypto.timingSafeEqual(given, stored)) throw httpError(403, 'Current password is wrong.'); if (String(b?.next || '').length < 10) throw httpError(400, 'Use at least 10 characters.'); - setUser(auth.user, { password: b.next }); await saveUsers(); return send(res, 200, { ok: true }); + setUser(auth.user, { password: b.next }); await saveUsers(); + // other browsers signed in with the old password are signed out; this one gets a fresh session + sso.endUserSessions(auth.user, auth.sessionId); + const fresh = auth.sessionId ? sso.createSession({ user: auth.user, role: u.role, via: 'local' }) : null; + if (fresh) sso.endUserSessions(auth.user, fresh.sessionId); + return send(res, 200, { ok: true }, fresh ? { 'Set-Cookie': fresh.setCookie } : {}); } if (r1 === 'state' && method === 'GET') { const at = url.searchParams.get('at'); @@ -759,6 +773,15 @@ async function api(req, res, parts, url, auth) { try { return send(res, 200, { ok: true, message: await connectors.test(id) }); } catch (e) { return send(res, 200, { ok: false, message: e.constructor.name === 'ConnectorError' ? e.message : `Unexpected error: ${e.message}` }); } } + // A module whose file import starts with a choice: reads the file and answers what there is to choose + if (r3 === 'preview' && method === 'POST') { + const b = await readBody(req, CONFIG.maxImportBytes) || {}, m = connectors.module(id); + if (!m.file || !m.file.pick) throw httpError(400, `${m.name} has no choice to make before an import.`); + need(auth, m.file.role === 'contributor' ? 'contributor' : 'approver'); + if (!isObj(b.file) || typeof b.file.text !== 'string' || !b.file.text.length) throw httpError(400, 'Choose the file.'); + try { return send(res, 200, { ok: true, pick: m.file.pick, ...(await connectors.preview(id, { file: b.file, overrides: isObj(b.overrides) ? b.overrides : {} })) }); } + catch (e) { if (e.constructor.name === 'ConnectorError') throw httpError(400, e.message); log('ERROR connector preview', id, e.stack || e.message); throw httpError(500, `Unexpected error: ${e.message}`); } + } if (r3 === 'run' && method === 'POST') { const b = await readBody(req, CONFIG.maxImportBytes) || {}; // {file: {name, text}}; the Palo Alto import also took {xml, fileName}, kept for scripts written for it @@ -770,7 +793,9 @@ async function api(req, res, parts, url, auth) { need(auth, f && m.file.role === 'contributor' ? 'contributor' : 'approver'); if (connectors.isRunning(id)) throw httpError(409, 'This connector is already running.'); if (!f && !connectors.get(id).live) throw httpError(400, m.live === false ? `${m.name} only imports files.` : 'Fill in its settings first.'); - const p = connectors.run(id, { by: auth.user || 'admin', file: f || undefined, overrides: isObj(b.overrides) ? b.overrides : {} }); + let picks; + try { picks = cleanPicks(b.picks); } catch (e) { throw httpError(400, e.message); } + const p = connectors.run(id, { by: auth.user || 'admin', file: f || undefined, overrides: isObj(b.overrides) ? b.overrides : {}, picks }); if (b.wait) return send(res, 200, { ok: true, run: await p }); p.catch(() => { }); return send(res, 202, { ok: true, started: true }); @@ -778,7 +803,7 @@ async function api(req, res, parts, url, auth) { } /* ---- API tokens (admin) ---- */ - if (r1 === 'sessions' && !id && method === 'DELETE') { need(auth, 'admin'); return send(res, 200, { ok: true, ended: sso.endSessions(auth.user || 'admin') }); } + if (r1 === 'sessions' && !id && method === 'DELETE') { need(auth, 'admin'); return send(res, 200, { ok: true, ended: sso.endSessions(auth.user || 'admin', auth.sessionId) }); } if (r1 === 'tokens') { need(auth, 'admin'); if (auth.open) throw httpError(400, 'Create an admin account first: with no account, a token would lock everyone else out.'); @@ -803,11 +828,12 @@ async function api(req, res, parts, url, auth) { if (b?.password && String(b.password).length < 10) throw httpError(400, 'Use at least 10 characters.'); if (id === auth.user && (b?.disabled || (b?.role && b.role !== 'admin'))) throw httpError(400, 'You cannot remove your own admin access.'); setUser(id, { role: b?.role || own(users, id)?.role || 'viewer', password: b?.password, disabled: b?.disabled }); + if (b?.password || b?.disabled) sso.endUserSessions(id); await saveUsers(); return send(res, 200, { ok: true }); } if (id && method === 'DELETE') { if (id === auth.user) throw httpError(400, 'You cannot delete your own account.'); - delete users[id]; await saveUsers(); return send(res, 200, { ok: true }); + delete users[id]; sso.endUserSessions(id); await saveUsers(); return send(res, 200, { ok: true }); } } return fail(res, 404, 'Unknown API route'); @@ -819,7 +845,29 @@ const localSignIn = () => sso.localLogin !== 'off' && authEnabled(); const html = (res, status, body, headers = {}) => send(res, status, body, { 'Content-Type': 'text/html; charset=utf-8', ...headers }); async function authRoute(req, res, url) { const p = url.pathname, page = (status, message, extra) => html(res, status, loginPage({ ssoName: sso.name, oidc: sso.oidc, local: localSignIn(), message, out: url.searchParams.has('out'), base: sso.publicUrl }), extra); - if (p === '/auth/login' && req.method === 'GET') return page(200); + if (p === '/auth/login' && req.method === 'GET') return authenticate(req) && !url.searchParams.has('out') ? send(res, 302, '', { Location: `${sso.publicUrl}/` }) : page(200); + // Local sign-in: the form of the sign-in page (urlencoded), or JSON from a script or a test + if (p === '/auth/login' && req.method === 'POST') { + const json = (req.headers['content-type'] || '').startsWith('application/json'); + // a page of another site must not sign a visitor in to an account of its choosing + const origin = req.headers.origin; + let fromHere = true; + if (origin && origin !== 'null') { try { fromHere = new URL(origin).host === req.headers.host || (!!sso.publicUrl && origin === new URL(sso.publicUrl).origin); } catch { fromHere = false; } } + if (!fromHere) return fail(res, 403, 'Sign-in from another site refused'); + if (!localSignIn()) return json ? fail(res, 404, 'Local sign-in is off') : page(404, 'Local sign-in is off on this server.'); + let b; + try { b = json ? await readBody(req, 4096) : Object.fromEntries(new URLSearchParams(await readRaw(req, 4096))); } + catch (e) { return fail(res, e.status || 400, e.message); } + const name = String(b?.user || '').slice(0, 64), r = checkLocalPassword(name, b?.password, clientAddress(req)); + if (r && r.retryAfter) { + const sec = Math.ceil(r.retryAfter / 1000), msg = `Too many failed sign-ins. Try again in ${sec >= 120 ? Math.ceil(sec / 60) + ' minutes' : sec + ' seconds'}.`; + return json ? send(res, 429, { error: msg }, { 'Retry-After': String(sec) }) : html(res, 429, loginPage({ ssoName: sso.name, oidc: sso.oidc, local: true, message: msg, base: sso.publicUrl, user: name }), { 'Retry-After': String(sec) }); + } + if (r !== true) return json ? fail(res, 401, 'Wrong user name or password') : html(res, 401, loginPage({ ssoName: sso.name, oidc: sso.oidc, local: true, message: 'Wrong user name or password.', base: sso.publicUrl, user: name })); + const u = own(users, name), s = sso.createSession({ user: name, role: u.role, via: 'local' }); + log(`Sign-in: ${JSON.stringify(name)} (local account)`); + return json ? send(res, 200, { ok: true, user: name, role: u.role }, { 'Set-Cookie': s.setCookie }) : send(res, 303, '', { Location: `${sso.publicUrl}/`, 'Set-Cookie': s.setCookie }); + } if (p === '/auth/oidc/start' && req.method === 'GET' && sso.oidc) { try { const s = await sso.start(); return send(res, 302, '', { Location: s.location, 'Set-Cookie': s.setCookie }); } catch (e) { log('SSO:', e.message); return page(502, `The identity provider cannot be reached: ${e.message}`); } @@ -847,20 +895,11 @@ const server = http.createServer(async (req, res) => { if (url.pathname.startsWith('/auth/')) return await authRoute(req, res, url); const auth = authenticate(req); if (auth && auth.denied) return send(res, 403, url.pathname.startsWith('/api/') ? { error: auth.denied } : loginPage({ ssoName: sso.name, oidc: sso.oidc, local: localSignIn(), message: auth.denied, base: sso.publicUrl }), url.pathname.startsWith('/api/') ? {} : { 'Content-Type': 'text/html; charset=utf-8' }); - if (auth && auth.retryAfter) { - const s = Math.ceil(auth.retryAfter / 1000); - res.writeHead(429, { ...SEC_HEADERS, 'Retry-After': String(s), 'Content-Type': 'text/plain; charset=utf-8', 'Cache-Control': 'no-store' }); - return res.end(`Too many failed sign-ins. Try again in ${s >= 120 ? Math.ceil(s / 60) + ' minutes' : s + ' seconds'}.`); - } if (!auth) { - // with OpenID Connect, a page asks for the sign-in page, and the API answers 401 without a password prompt - // …except "/?local": the browser's password prompt for a local account, asked at the root so that - // the browser then sends the password with every request of the app - if (sso.oidc && url.pathname === '/' && url.searchParams.has('local') && localSignIn()) - return send(res, 401, loginPage({ ssoName: sso.name, oidc: sso.oidc, local: true, message: 'Local sign-in cancelled or refused.', base: sso.publicUrl }), { 'Content-Type': 'text/html; charset=utf-8', 'WWW-Authenticate': 'Basic realm="Strata", charset="UTF-8"' }); - if (sso.oidc && !url.pathname.startsWith('/api/') && (req.method === 'GET' || req.method === 'HEAD')) return send(res, 302, '', { Location: `${sso.publicUrl}/auth/login` }); - res.writeHead(401, { ...SEC_HEADERS, ...(sso.oidc || !localSignIn() ? {} : { 'WWW-Authenticate': 'Basic realm="Strata", charset="UTF-8"' }), 'Content-Type': 'text/plain; charset=utf-8' }); - return res.end('Authentication required'); + // a page goes to the sign-in page; the API answers 401 (and says why when a script still sends HTTP Basic) + if (!url.pathname.startsWith('/api/') && (req.method === 'GET' || req.method === 'HEAD')) return send(res, 302, '', { Location: `${sso.publicUrl}/auth/login` }); + const basic = String(req.headers.authorization || '').startsWith('Basic '); + return fail(res, 401, basic ? 'HTTP Basic sign-in was removed in Strata 0.11: use an API token (Authorization: Bearer stk_…), created in Account › API tokens' : 'Authentication required'); } if (url.pathname.startsWith('/api/')) await api(req, res, url.pathname.slice(5).split('/').filter(Boolean).map(decodeURIComponent), url, auth); else if (req.method === 'GET' || req.method === 'HEAD') await serveStatic(req, res, url.pathname); diff --git a/server/sso.js b/server/sso.js index 3c43149..6a118ea 100644 --- a/server/sso.js +++ b/server/sso.js @@ -107,7 +107,19 @@ export function createSso({ config, dataDir, log, writeJson, peerAddress, localU const id = readCookie(req, SESSION); const s = id && sessions.get(id); if (!s || s.exp < Date.now()) { if (s) sessions.delete(id); return null; } - return { user: s.user, role: s.role, via: 'sso', sessionId: id }; + return { user: s.user, role: s.role, via: s.via || 'sso', sessionId: id, since: s.since }; + } + // A session for a local account signed in with the form (server.js checks the password) + function createSession({ user, role, via }) { + const sid = rand(32); + sessions.set(sid, { user, role, via, since: Date.now(), exp: Date.now() + C.sessionMs }); + return { sessionId: sid, setCookie: cookie(SESSION, sid, Math.round(C.sessionMs / 1000)) }; + } + // Ends the sessions of one person (password changed, account disabled or deleted), but the one given + function endUserSessions(user, exceptId) { + let n = 0; + for (const [k, s] of sessions) if (s.user === user && k !== exceptId) { sessions.delete(k); n++; } + return n; } /* ---- trusted proxy headers ---- */ @@ -208,7 +220,7 @@ export function createSso({ config, dataDir, log, writeJson, peerAddress, localU const role = roleOf(groups); if (!role) throw Object.assign(new Error(`${user} is in no group that gives access to Strata. Ask an administrator.`), { denied: true }); const sid = rand(32); - sessions.set(sid, { user, role, exp: Date.now() + C.sessionMs, idToken: tok.id_token }); + sessions.set(sid, { user, role, via: 'sso', since: Date.now(), exp: Date.now() + C.sessionMs, idToken: tok.id_token }); log(`SSO sign-in: ${JSON.stringify(user)} as ${role}`); return { user, role, setCookie: [cookie(SESSION, sid, Math.round(C.sessionMs / 1000)), cookie(STATE, '', 0)] }; } @@ -216,7 +228,7 @@ export function createSso({ config, dataDir, log, writeJson, peerAddress, localU const s = sessions.get(sessionId); sessions.delete(sessionId); let location = `${C.publicUrl}/auth/login?out=1`; - if (s && oidc) { + if (s && s.via === 'sso' && oidc) { try { const d = await discovery(); if (d.end_session_endpoint) { const u = new URL(d.end_session_endpoint); u.searchParams.set('id_token_hint', s.idToken); u.searchParams.set('post_logout_redirect_uri', location); location = u.href; } @@ -224,8 +236,8 @@ export function createSso({ config, dataDir, log, writeJson, peerAddress, localU } return { location, setCookie: cookie(SESSION, '', 0) }; } - // An administrator ends every single sign-on session (after a group change, or someone leaving) - function endSessions(by) { const n = sessions.size; sessions.clear(); log(`${n} single sign-on session(s) ended by ${JSON.stringify(by)}`); return n; } + // An administrator ends every session (after a group change, or someone leaving), but their own + function endSessions(by, exceptId) { let n = 0; for (const k of [...sessions.keys()]) if (k !== exceptId) { sessions.delete(k); n++; } log(`${n} session(s) ended by ${JSON.stringify(by)}`); return n; } /* ---- API tokens (DATA_DIR/tokens.json, mode 600) ---- */ const FILE = path.join(dataDir, 'tokens.json'); @@ -271,7 +283,7 @@ export function createSso({ config, dataDir, log, writeJson, peerAddress, localU oidc, header, name: C.name, localLogin: C.localLogin, publicUrl: C.publicUrl, // anyone could sign in some way other than a local account enabled: () => oidc || header || Object.keys(tokens).length > 0, - fromSession, fromHeaders, fromToken, start, callback, logout, endSessions, sessionCount: () => sessions.size, + fromSession, fromHeaders, fromToken, start, callback, logout, endSessions, endUserSessions, createSession, sessionCount: () => sessions.size, listTokens: () => Object.entries(tokens).map(([id, t]) => publicToken(id, t)).sort((a, b) => a.name.localeCompare(b.name)), createToken, deleteToken, }; @@ -280,19 +292,23 @@ export function createSso({ config, dataDir, log, writeJson, peerAddress, localU /* ---------------- Sign-in page ---------------- */ const escHtml = s => String(s).replace(/&/g, '&').replace(//g, '>').replace(/"/g, '"'); // base: PUBLIC_URL, or '' (links from the root) -export function loginPage({ ssoName, oidc, local, message, out, base = '' }) { +export function loginPage({ ssoName, oidc, local, message, out, base = '', user = '' }) { return `Sign in · Strata +a.btn,button.btn{display:block;width:100%;box-sizing:border-box;font:inherit;cursor:pointer;border:0;text-align:center;padding:11px 14px;border-radius:3px;text-decoration:none;font-weight:600;margin-top:10px} +a.primary,button.primary{background:var(--accent);color:var(--accent-fg)}a.second,button.second{background:transparent;border:1px solid var(--line);color:var(--fg)} +label{display:block;font-size:13px;font-weight:600;margin:12px 0 4px}input{display:block;width:100%;box-sizing:border-box;padding:9px 10px;font:inherit;color:var(--fg);background:transparent;border:1px solid var(--line);border-radius:3px} +.or{text-align:center;margin:16px 0 0;font-size:13px}.msg{color:var(--bad);border-left:3px solid var(--bad);padding:6px 10px;margin:0 0 16px}.ok{color:var(--muted)}

    Strata

    Map of the information systems

    ${message ? `` : out ? '

    You are signed out.

    ' : ''} ${oidc ? `Sign in with ${escHtml(ssoName)}` : ''} -${local ? `Sign in with a local account` : ''} +${local ? `${oidc ? '

    or with a local account

    ' : ''}
    + + +
    ` : ''}
    `; } diff --git a/tests/analysis.test.mjs b/tests/analysis.test.mjs index aa32dae..e54f2dd 100644 --- a/tests/analysis.test.mjs +++ b/tests/analysis.test.mjs @@ -233,3 +233,100 @@ test('attack paths: unlikely and long paths are left out, and a cut-short search // a target nothing leads to has no path assert.deepEqual(J(A.attackPaths(s, A.outside(s), ['bastion', 'fw']).byTarget.fw), []); }); + +test('arrange: plates of a level on rows and columns, the VLANs that talk side by side', () => { + const plates = 'abcdefghij'.split('').map((id, i) => ({ id, w: 8 + (i % 3), D: 6 + (i % 2) })); + const W = { 'a|h': 9, 'h|c': 7, 'c|j': 5 }; + const weight = (x, y) => W[`${x}|${y}`] || W[`${y}|${x}`] || 0; + const r = J(A.arrangeGrid(plates, weight, 4, 2)); + assert.equal(r.cols, 4); + assert.equal(r.rows, 3); + // the busiest VLAN first, then the one it talks to most beside it, and so on; VLANs without flows last + assert.deepEqual(r.order.slice(0, 4), ['h', 'a', 'c', 'j']); + assert.deepEqual(r.order.slice(4), ['b', 'd', 'e', 'f', 'g', 'i']); + // no two plates overlap, and the grid is centred on 0 + const box = id => { const p = plates.find(q => q.id === id), [x, z] = r.places[id]; return { x0: x - p.w / 2, x1: x + p.w / 2, z0: z - p.D / 2, z1: z + p.D / 2 }; }; + const ids = Object.keys(r.places); + for (const i of ids) for (const j of ids) if (i < j) { + const a = box(i), b = box(j); + assert.ok(a.x1 <= b.x0 || b.x1 <= a.x0 || a.z1 <= b.z0 || b.z1 <= a.z0, `${i} and ${j} overlap`); + } + const xs = ids.flatMap(id => [box(id).x0, box(id).x1]); + assert.ok(Math.abs(Math.min(...xs) + Math.max(...xs)) <= 1); +}); + +test('arrange: automatic columns, a single plate, and none', () => { + const plates = Array.from({ length: 40 }, (_, i) => ({ id: 'v' + i, w: 8, D: 6 })); + const r = J(A.arrangeGrid(plates, () => 0, 0, 2)); + assert.equal(r.cols, 8, 'about the square root, a little wider than deep: no more 40 plates on one row'); + assert.equal(r.rows, 5); + assert.deepEqual(r.order, plates.map(p => p.id), 'with no flow, the current order is kept'); + assert.deepEqual(J(A.arrangeGrid([{ id: 'x', w: 8, D: 6 }], () => 0, 6, 2)).places, { x: [0, 0] }); + assert.deepEqual(J(A.arrangeGrid([], () => 0, 6, 2)).places, {}); +}); + +test('VLAN files: a CSV with a French header, quotes and a semicolon separator', () => { + const r = J(A.vlansFromFile('Numéro;Nom;Réseau;Zone\n10;Serveurs;10.0.10.0/24;DC\n"20";"Postes; étage 1";10.0.20.0/24;Bureaux\nabc;x;;\n5000;trop grand;;\n')); + assert.equal(r.kind, 'csv'); + assert.deepEqual(r.items, [ + { num: 10, name: 'Serveurs', cidr: '10.0.10.0/24', zone: 'DC' }, + { num: 20, name: 'Postes; étage 1', cidr: '10.0.20.0/24', zone: 'Bureaux' }, + ]); + assert.equal(r.skipped, 2); + // without a header: number, name, range, zone + assert.deepEqual(J(A.vlansFromFile('30,Cameras,10.0.30.0/24\n40,Printers')).items.map(i => [i.num, i.name, i.cidr]), [[30, 'Cameras', '10.0.30.0/24'], [40, 'Printers', '']]); + assert.throws(() => A.vlansFromFile('name,range\nx,y'), /no VLAN number column/); + assert.throws(() => A.vlansFromFile(''), /empty/); +}); + +test('VLAN files: a JSON list (NetBox style) and a Strata export', () => { + const nb = J(A.vlansFromFile(JSON.stringify({ results: [{ id: 912, vid: 30, name: 'Cams', group: { name: 'OT' } }, { id: 913, name: 'no number' }] }))); + assert.equal(nb.kind, 'json'); + assert.deepEqual(nb.items, [{ num: 30, name: 'Cams', cidr: '', zone: 'OT' }]); + assert.equal(nb.skipped, 1); + const strata = J(A.vlansFromFile(JSON.stringify({ format: 'strata', si: { lyon: { name: 'Lyon', zones: [{ id: 'z1', name: 'DMZ' }], vlans: [{ id: 'v1', num: 100, name: 'Web', cidr: '192.0.2.0/24', zone: 'z1' }] } }, assets: {} }))); + assert.equal(strata.kind, 'strata'); + assert.deepEqual(strata.items, [{ num: 100, name: 'Web', cidr: '192.0.2.0/24', zone: 'DMZ', site: 'lyon', siteName: 'Lyon', vlanId: 'v1' }]); +}); + +test('zones from the flows: the Internet on top, by depth, then by role; matching zones kept, empty VLANs stay', () => { + const r = J(A.suggestZones({ + zones: [{ id: 'z1', name: 'Main' }, { id: 'zd', name: 'Our DMZ' }], + vlans: [{ id: 'dmz', zone: 'zd' }, { id: 'app', zone: 'z1' }, { id: 'db', zone: 'z1' }, { id: 'pc', zone: 'z1' }, { id: 'ot', zone: 'z1' }, { id: 'mg', zone: 'z1' }, { id: 'empty', zone: 'z1' }], + assets: [{ id: 'w', vlan: 'dmz', type: 'server' }, { id: 'a', vlan: 'app', type: 'server' }, { id: 'd', vlan: 'db', type: 'db' }, { id: 'd2', vlan: 'db', type: 'db' }, + { id: 'p', vlan: 'pc', type: 'workstation' }, { id: 'plc', vlan: 'ot', type: 'plc' }, { id: 'sw', vlan: 'mg', type: 'switch' }], + flows: [{ src: 'internet', dst: 'w' }, { src: 'w', dst: 'a' }, { src: 'a', dst: 'd' }, { src: 'p', dst: 'a' }, { src: 'p', dst: 'internet' }, { src: 'svc:x', dst: 'd2' }], + })); + // bottom first: empty VLANs keep their zone, then the roles of what the Internet never reaches, then depth, the exposed on top + // (db is reached by an Internet service: it is exposed too, on the floor of the DMZ) + assert.deepEqual(r.zones.map(z => [z.name, z.vlans]), [ + ['Main', ['empty']], ['Network and administration', ['mg']], ['Industrial (OT)', ['ot']], ['Users', ['pc']], + ['Reached from the exposed zone', ['app']], ['Our DMZ', ['dmz', 'db']], + ]); + assert.equal(r.zones.at(-1).id, 'zd', 'the DMZ zone is kept, with its name'); + assert.equal(r.byVlan.pc.depth, null, 'a VLAN that only calls the Internet is not reached from it'); + assert.equal(r.byVlan.db.depth, 1, 'an Internet service counts as the Internet'); + assert.equal(r.byVlan.empty.moved, false); + assert.equal(r.byVlan.app.moved, true); +}); + +test('zones from the flows: a VLAN with PLCs among its servers is industrial', () => { + const r = J(A.suggestZones({ zones: [{ id: 'z1', name: 'Main' }], + vlans: [{ id: 'v50', name: 'OT production', zone: 'z1' }, { id: 'v60', name: 'Lab', zone: 'z1' }], + assets: [{ id: 'plc', vlan: 'v50', type: 'plc' }, { id: 'hmi', vlan: 'v50', type: 'server' }, { id: 'hist', vlan: 'v50', type: 'server' }, + { id: 's1', vlan: 'v60', type: 'server' }, { id: 's2', vlan: 'v60', type: 'server' }, { id: 's3', vlan: 'v60', type: 'server' }, { id: 'cam', vlan: 'v60', type: 'camera' }], + flows: [] })); + assert.equal(r.byVlan.v50.role, 'ot'); + assert.equal(r.byVlan.v60.role, 'srv', 'one camera among four servers does not make it industrial'); +}); + +test('zones from the flows: meeting-room screens are the users\', PLCs make a VLAN industrial', () => { + const r = J(A.suggestZones({ zones: [{ id: 'z1', name: 'Main' }], + vlans: [{ id: 'room', name: 'Salles de réunion étage 1', zone: 'z1' }, { id: 'line', name: 'Line 1', zone: 'z1' }, { id: 'cams', name: 'Vidéosurveillance', zone: 'z1' }], + assets: [{ id: 's1', vlan: 'room', type: 'iot' }, { id: 's2', vlan: 'room', type: 'iot' }, { id: 'p', vlan: 'line', type: 'plc' }, { id: 'h', vlan: 'line', type: 'server' }, { id: 'h2', vlan: 'line', type: 'server' }, + { id: 'c1', vlan: 'cams', type: 'camera' }, { id: 'c2', vlan: 'cams', type: 'camera' }], + flows: [] })); + assert.equal(r.byVlan.room.role, 'users'); + assert.equal(r.byVlan.line.role, 'ot'); + assert.equal(r.byVlan.cams.role, 'ot'); +}); diff --git a/tests/api.test.mjs b/tests/api.test.mjs index fba0e86..7e579d0 100644 --- a/tests/api.test.mjs +++ b/tests/api.test.mjs @@ -8,6 +8,7 @@ import { request } from 'node:http'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth } from './session.mjs'; const root = join(dirname(fileURLToPath(import.meta.url)), '..'); const dataDir = mkdtempSync(join(tmpdir(), 'strata-api-')); @@ -34,10 +35,10 @@ after(async () => { }); class Client { - constructor(user, password = PASS[user]) { this.auth = user ? 'Basic ' + Buffer.from(`${user}:${password}`).toString('base64') : ''; } + constructor(user, password = PASS[user]) { this.user = user; this.password = password; } // headers: extra headers; a null value removes a default one (used by the CSRF test) async req(method, path, body, headers = {}) { - const h = { ...(body !== undefined ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}), ...(this.auth ? { Authorization: this.auth } : {}), ...headers }; + const h = { ...(body !== undefined ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}), ...(await auth(base, this.user, this.password)), ...headers }; for (const k of Object.keys(h)) if (h[k] === null) delete h[k]; const res = await fetch(base + path, { method, @@ -73,7 +74,14 @@ test('a malformed percent-encoding in the path gets 400, not a server error', as test('auth: no credentials, wrong password and unknown user are refused', async () => { const none = await new Client().get('/api/state'); assert.equal(none.status, 401); - assert.match(none.headers.get('www-authenticate'), /^Basic /); + assert.equal(none.headers.get('www-authenticate'), null); + // a page sends the browser to the sign-in form; HTTP Basic is refused, with what to use instead + const page = await fetch(`${base}/`, { redirect: 'manual' }); + assert.equal(page.status, 302); + assert.equal(page.headers.get('location'), '/auth/login'); + const basic = await fetch(`${base}/api/state`, { headers: { Authorization: 'Basic ' + Buffer.from(`admin:${PASS.admin}`).toString('base64') } }); + assert.equal(basic.status, 401); + assert.match((await basic.json()).error, /API token/); assert.equal((await new Client('admin', 'wrong-password').get('/api/state')).status, 401); assert.equal((await new Client('nobody', 'whatever-123').get('/api/state')).status, 401); const me = await as('alice').get('/api/me'); @@ -242,8 +250,9 @@ test('import: replaces the map, validates ids, and enforces MAX_IMPORT_MB', asyn }); test('static files: path traversal cannot reach the server code', async () => { + const vicAuth = await auth(base, 'vic', PASS.vic); const raw = p => new Promise((res, rej) => { - request({ host: '127.0.0.1', port, path: p, headers: { Authorization: as('vic').auth } }, r => { let b = ''; r.on('data', c => (b += c)); r.on('end', () => res({ status: r.statusCode, body: b })); }).on('error', rej).end(); + request({ host: '127.0.0.1', port, path: p, headers: vicAuth }, r => { let b = ''; r.on('data', c => (b += c)); r.on('end', () => res({ status: r.statusCode, body: b })); }).on('error', rej).end(); }); for (const p of ['/../server/server.js', '/%2e%2e/server/server.js', '/..%2fserver%2fserver.js', '/%2e%2e%2f%2e%2e%2fpackage.json']) { const r = await raw(p); @@ -316,7 +325,7 @@ test('reserved keys: the users.mjs command line refuses them too', async () => { // (": ping") are skipped async function openEvents(user) { const ctrl = new AbortController(); - const res = await fetch(base + '/api/events', { headers: user ? { Authorization: as(user).auth } : {}, signal: ctrl.signal }); + const res = await fetch(base + '/api/events', { headers: await auth(base, user, PASS[user]), signal: ctrl.signal }); if (res.status !== 200) { await res.body.cancel(); return { status: res.status }; } const reader = res.body.getReader(), dec = new TextDecoder(); let buf = ''; diff --git a/tests/e2e.mjs b/tests/e2e.mjs index 45d083d..872bcc4 100644 --- a/tests/e2e.mjs +++ b/tests/e2e.mjs @@ -6,10 +6,11 @@ import { test, before, after } from 'node:test'; import assert from 'node:assert/strict'; import { spawn } from 'node:child_process'; -import { mkdtempSync, rmSync } from 'node:fs'; +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth } from './session.mjs'; import { chromium } from 'playwright'; const root = join(dirname(fileURLToPath(import.meta.url)), '..'); @@ -31,7 +32,7 @@ before(async () => { assert.equal(r.status, 200); } // WebGL without a GPU: SwiftShader - browser = await chromium.launch({ args: ['--use-gl=angle', '--use-angle=swiftshader', '--enable-unsafe-swiftshader', '--ignore-gpu-blocklist'] }); + browser = await chromium.launch({ ...(process.env.CHROMIUM_PATH ? { executablePath: process.env.CHROMIUM_PATH } : {}), args: ['--use-gl=angle', '--use-angle=swiftshader', '--enable-unsafe-swiftshader', '--ignore-gpu-blocklist'] }); }); after(async () => { if (browser) await browser.close(); @@ -39,16 +40,23 @@ after(async () => { rmSync(dataDir, { recursive: true, force: true, maxRetries: 5 }); }); -function api(user, method, path, body) { +async function api(user, method, path, body) { return fetch(base + path, { method, - headers: { Authorization: 'Basic ' + Buffer.from(`${user}:${PASS[user]}`).toString('base64'), ...(body ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}) }, + headers: { ...(await auth(base, user, PASS[user])), ...(body ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}) }, body: body ? JSON.stringify(body) : undefined, }).then(async r => ({ status: r.status, data: await r.json().catch(() => null) })); } // A signed-in page; every page error and console error is recorded and fails the test async function signIn(user) { - const ctx = await browser.newContext({ viewport: { width: 1400, height: 900 }, httpCredentials: { username: user, password: PASS[user] } }); + const ctx = await browser.newContext({ viewport: { width: 1400, height: 900 } }); + // the sign-in form, as a person would fill it + const login = await ctx.newPage(); + await login.goto(base); + await login.fill('#u', user); + await login.fill('#p', PASS[user]); + await Promise.all([login.waitForURL(u => !String(u).includes('/auth/')), login.click('button[type=submit]')]); + await login.close(); const page = await ctx.newPage(); page.on('pageerror', e => errors.push(`${user}: ${e.message}`)); page.on('console', m => { if (m.type() === 'error') errors.push(`${user} console: ${m.text()}`); }); @@ -295,3 +303,126 @@ test('risk register: threats per asset on the matrix, requirements applied, sett await bob.click('#dash [data-dash="close"]'); assert.deepEqual(errors, [], 'JavaScript errors in the pages'); }); + +test('plates: all or none, empty VLANs hidden, VLANs imported into chosen zones, arranged on a grid, Internet moved, sign-out', async () => { + const admin = await signIn('admin'); + if (!(await admin.locator('#siteList [data-site="lyon"]').count())) { + await admin.click('#empty [data-act="demo"]'); + await admin.waitForFunction(() => document.querySelectorAll('#siteList [data-site]').length === 5); + } + await openIs(admin, 'lyon'); + await admin.waitForSelector('#vlans .vrow'); + const boxes = () => admin.locator('#vlans input[type=checkbox]:checked').count(); + const total = await admin.locator('#vlans input[type=checkbox]').count(); + + // the title block folds into one line, and the controls move up with it + const cartH = () => admin.evaluate(() => document.getElementById('cart').getBoundingClientRect().height); + const ctlTop = () => admin.evaluate(() => document.getElementById('ctl').getBoundingClientRect().top); + const [h0, t0] = [await cartH(), await ctlTop()]; + await admin.click('#cart [data-cart-tog]'); + assert.ok(await cartH() < h0 / 2, 'the title block did not fold'); + assert.ok(await ctlTop() < t0 - 50, 'the controls did not move up'); + assert.match(await admin.locator('#cart .mini').textContent(), /IS Lyon.*assets.*exposed.*crit.*risky/s); + await admin.click('#cart [data-cart-tog]'); + assert.ok(Math.abs(await cartH() - h0) < 2); + + // all / none, for the IS and for one zone + await admin.click('.vtools [data-vsel="none"]'); + assert.equal(await boxes(), 0); + await admin.click('.vtools [data-vsel="all"]'); + assert.equal(await boxes(), total); + const zone = admin.locator('#vlans .zone-h [data-vsel="none"]').first(); + const zoneId = await zone.getAttribute('data-zone'); + const inZone = (await api('admin', 'GET', 'api/state')).data.si.lyon.vlans.filter(v => v.zone === zoneId).length; + await zone.click(); + assert.equal(await boxes(), total - inZone); + await admin.click('.vtools [data-vsel="all"]'); + + // a CSV of VLANs: two ticked, put in a new zone at once, one in a zone named by the file + const csv = join(dataDir, 'vlans.csv'); + writeFileSync(csv, 'Numéro;Nom;Réseau;Zone\n300;Lab A;10.30.0.0/24;Labo\n301;Lab B;10.30.1.0/24;Labo\n302;Printers R&D;10.30.2.0/24;\n10;DMZ renamed;;\n'); + await admin.locator('#vlanFile').setInputFiles(csv); + await admin.waitForSelector('#viBody tr'); + assert.equal(await admin.locator('#viBody tr').count(), 4); + assert.equal(await admin.locator('#viBody tr').nth(3).locator('[data-vi-on]').isChecked(), false, 'a VLAN already in the IS starts unticked'); + await admin.click('[data-vi="none"]'); + await admin.locator('#viBody tr').nth(1).locator('[data-vi-on]').check(); + await admin.locator('#viBody tr').nth(2).locator('[data-vi-on]').check(); + await admin.selectOption('#viZone', '__new'); + await admin.fill('#viZoneNew', 'Peripherals'); + await admin.click('[data-vi="apply"]'); + await admin.locator('#viBody tr').nth(0).locator('[data-vi-on]').check(); + await admin.click('#frm button[type=submit]'); + await admin.waitForFunction(() => /Lab A/.test(document.querySelector('#vlans')?.textContent || '')); + const si = (await api('admin', 'GET', 'api/state')).data.si.lyon; + const zoneOf = num => si.zones.find(z => z.id === si.vlans.find(v => v.num === num).zone).name; + assert.equal(zoneOf(300), 'Labo'); + assert.equal(zoneOf(301), 'Peripherals'); + assert.equal(zoneOf(302), 'Peripherals'); + assert.equal(si.vlans.find(v => v.num === 10).name, 'DMZ', 'an unticked VLAN is left alone'); + + // the three new VLANs are empty: hidden from the model when asked + await admin.check('#optEmpty'); + await admin.waitForFunction(() => document.querySelectorAll('#vlans .vrow.off').length >= 3); + await admin.uncheck('#optEmpty'); + + // Arrange VLANs: 3 per row at most, saved as plate positions + await admin.selectOption('#arrCols', '3'); + await admin.click('#arrangeGridBtn'); + await admin.waitForFunction(async () => (await (await fetch('api/state')).json()).si.lyon.vlans.every(v => Array.isArray(v.place))); + const placed = (await api('admin', 'GET', 'api/state')).data.si.lyon; + for (const z of placed.zones) { + const rows = new Set(placed.vlans.filter(v => v.zone === z.id).map(v => v.place[1])); + const n = placed.vlans.filter(v => v.zone === z.id).length; + assert.equal(rows.size, Math.ceil(n / 3), `zone ${z.name}: ${n} VLANs on ${rows.size} rows`); + } + + // the Internet node, dragged in the 2D diagram, keeps its place + await admin.click('#viewSeg [data-v="2d"]'); + await admin.waitForSelector('#flatSvg [data-internet="internet"] rect'); + const b = await admin.locator('#flatSvg [data-internet="internet"] rect').boundingBox(); + await admin.mouse.move(b.x + b.width / 2, b.y + b.height / 2); + await admin.mouse.down(); + await admin.mouse.move(b.x + 300, b.y + 200, { steps: 8 }); + await admin.mouse.up(); + await admin.waitForFunction(async () => Array.isArray((await (await fetch('api/state')).json()).si.lyon.cloud2d)); + await admin.click('#cloudReset'); + await admin.waitForFunction(async () => !(await (await fetch('api/state')).json()).si.lyon.cloud2d); + + // Suggest zones: the Internet on top; applied by the admin + await admin.click('#viewSeg [data-v="3d"]'); + await admin.click('#suggestZonesBtn'); + await admin.waitForSelector('#szBody .sz'); + assert.equal(await admin.locator('#frm button[type=submit]').textContent(), 'Apply'); + await admin.click('#frm button[type=submit]'); + await admin.waitForFunction(async () => { const s = (await (await fetch('api/state')).json()).si.lyon; const top = s.zones[s.zones.length - 1]; return s.vlans.some(v => v.num === 10 && v.zone === top.id); }); + + // the 3D view is cheap to draw: a few hundred draw calls, nothing drawn while idle, nothing leaked on a rebuild + await admin.click('#viewSeg [data-v="3d"]'); + await admin.click('.vtools [data-vsel="all"]'); + await admin.waitForTimeout(1500); + const rs = () => admin.evaluate(() => window.strataRenderStats()); + const r0 = await rs(); + assert.ok(r0.calls > 20 && r0.calls < 400, `${r0.calls} draw calls for the Lyon model`); + await admin.click('#gfxSeg [data-gfx="eco"]'); + await admin.waitForTimeout(800); + assert.equal((await rs()).mode, 'eco'); + const idle0 = (await rs()).frames; + await admin.waitForTimeout(1500); + assert.ok((await rs()).frames - idle0 <= 1, 'the model is drawn again while nothing changes'); + // shared geometries (equipment models) go to the GPU the first time they are seen: one cycle warms them up, + // then rebuilding the model again and again must not add any + const cycle = async () => { for (const sp of ['wide', 'compact']) { await admin.click(`#spaceSeg [data-space="${sp}"]`); await admin.waitForTimeout(500); } await admin.waitForTimeout(600); return (await rs()).geometries; }; + const geo0 = await cycle(), geos = [await cycle(), await cycle(), await cycle()]; + assert.ok(Math.max(...geos) <= geo0 + 2, `GPU geometries grow on each rebuild (${geo0} → ${geos.join(', ')})`); + await admin.click('#gfxSeg [data-gfx="quality"]'); + await admin.waitForTimeout(800); + assert.equal((await rs()).mode, 'quality'); + await admin.click('#gfxSeg [data-gfx="auto"]'); + + // sign out: back to the sign-in page, and the session no longer opens the API + await admin.click('#cart [data-signout]'); + await admin.waitForURL(/\/auth\/login/); + assert.equal((await admin.request.get(base + 'api/me')).status(), 401); + assert.deepEqual(errors, [], 'JavaScript errors in the pages'); +}); diff --git a/tests/hardening.test.mjs b/tests/hardening.test.mjs index 31bf2fd..1f8d1c1 100644 --- a/tests/hardening.test.mjs +++ b/tests/hardening.test.mjs @@ -8,6 +8,7 @@ import { mkdtempSync, mkdirSync, rmSync, statSync, writeFileSync } from 'node:fs import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth } from './session.mjs'; import { parseXml } from '../server/connectors/sdk.js'; const root = join(dirname(fileURLToPath(import.meta.url)), '..'); @@ -35,7 +36,7 @@ test('data files are readable by the server user only, including those of older await new Promise((res, rej) => { proc.stdout.on('data', d => { if (String(d).includes('listening')) res(); }); proc.on('exit', c => rej(new Error('exit ' + c))); }); const r = await fetch(`http://127.0.0.1:${port}/api/si/s1`, { method: 'PUT', body: JSON.stringify({ name: 'IS one' }), - headers: { Authorization: 'Basic ' + Buffer.from(`admin:${pass}`).toString('base64'), 'Content-Type': 'application/json', 'X-Requested-With': 'strata' }, + headers: { ...(await auth(`http://127.0.0.1:${port}`, 'admin', pass)), 'Content-Type': 'application/json', 'X-Requested-With': 'strata' }, }); assert.equal(r.status, 200); await new Promise(res => setTimeout(res, 200)); @@ -48,3 +49,37 @@ test('data files are readable by the server user only, including those of older rmSync(dataDir, { recursive: true, force: true, maxRetries: 5 }); } }); + +test('a large IS (3500 flows) can be saved, and the history stops at HISTORY_MAX_MB', async () => { + const dataDir = mkdtempSync(join(tmpdir(), 'strata-big-')); + const port = 13100 + Math.floor(Math.random() * 400), pass = 'admin-pass-123', base = `http://127.0.0.1:${port}`; + const proc = spawn(process.execPath, [join(root, 'server/server.js')], { + env: { ...process.env, PORT: String(port), HOST: '127.0.0.1', DATA_DIR: dataDir, EDIT_USER: 'admin', EDIT_PASS: pass, VIEW_PASS: '', CONNECTORS: 'false', HISTORY_MAX_MB: '1' }, + stdio: ['ignore', 'pipe', 'inherit'], + }); + let out = ''; + try { + await new Promise((res, rej) => { proc.stdout.on('data', d => { out += d; if (String(d).includes('listening')) res(); }); proc.on('exit', c => rej(new Error('exit ' + c))); }); + const h = { ...(await auth(base, 'admin', pass)), 'Content-Type': 'application/json', 'X-Requested-With': 'strata' }; + // an IS of 150 VLANs and 3500 flows with notes, about 700 KB: above the old 512 KB limit + const doc = n => ({ name: 'Big', zones: [{ id: 'z1', name: 'Main' }], vlans: Array.from({ length: 150 }, (_, i) => ({ id: 'v' + i, num: i + 1, name: 'VLAN ' + i, zone: 'z1' })), + flows: Array.from({ length: 3500 }, (_, i) => ({ id: `f${i}`, src: `a${i}`, dst: `b${i}`, port: 'tcp/443', vol: n, risk: false, note: `Palo Alto rule “R${i}” — a note long enough to look like a real one, edit ${n}` })) }); + assert.ok(JSON.stringify(doc(0)).length > 512 * 1024); + for (let n = 0; n < 4; n++) { + const r = await fetch(`${base}/api/si/big`, { method: 'PUT', headers: h, body: JSON.stringify(doc(n)) }); + assert.equal(r.status, 200, `save ${n}: ${await r.text()}`); + } + // each change keeps the IS before and after it (more than 1 MB): only the latest is kept + const hist = await (await fetch(`${base}/api/history?limit=50`, { headers: h })).json(); + assert.equal(hist.entries.length, 1, JSON.stringify(hist.entries.map(e => e.rev))); + assert.equal(hist.rev, 4); + assert.ok(statSync(join(dataDir, 'history.jsonl')).size < 2 * 1024 * 1024); + assert.match(out, /History: \d+ oldest change\(s\) dropped \(limits: 5000 changes, 1 MB\)/); + // and still more than MAX_DOC_MB is refused + const huge = { ...doc(9), notes: 'x'.repeat(9 * 1024 * 1024) }; + assert.equal((await fetch(`${base}/api/si/big`, { method: 'PUT', headers: h, body: JSON.stringify(huge) })).status, 413); + } finally { + await new Promise(res => { if (proc.exitCode !== null) return res(); proc.once('exit', res); proc.kill(); }); + rmSync(dataDir, { recursive: true, force: true, maxRetries: 5 }); + } +}); diff --git a/tests/modules.test.mjs b/tests/modules.test.mjs index 8151989..474b309 100644 --- a/tests/modules.test.mjs +++ b/tests/modules.test.mjs @@ -11,6 +11,7 @@ import zlib from 'node:zlib'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth } from './session.mjs'; import { readTgz, writeTgz, sha256 } from '../server/connectors/store.js'; import { checkManifest, compareVersions } from '../server/connectors/modules.js'; @@ -109,7 +110,7 @@ after(async () => { async function api(user, method, path, body) { const r = await fetch(base + path, { method, - headers: { Authorization: 'Basic ' + Buffer.from(`${user}:${PASS[user]}`).toString('base64'), ...(body !== undefined ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}) }, + headers: { ...(await auth(base, user, PASS[user])), ...(body !== undefined ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}) }, body: body !== undefined ? JSON.stringify(body) : undefined, }); return { status: r.status, data: await r.json().catch(() => null) }; @@ -121,7 +122,7 @@ test('the built-in modules, the server-provided ones, and the folders that are n const by = Object.fromEntries(r.data.connectors.map(c => [c.id, c])); for (const id of ['cyberwatch', 'glpi', 'phpipam', 'paloalto']) assert.equal(by[id].origin, 'builtin', id); assert.equal(by.paloalto.badge, 'FW'); - assert.deepEqual(by.paloalto.file.fields, ['site', 'firewall', 'scope']); + assert.deepEqual(by.paloalto.file.fields, ['site', 'firewall', 'scope', 'build']); assert.equal(by.spy.origin, 'provided'); assert.equal(by.glpi.name, 'GLPI', 'a provided module cannot take a built-in id'); const problems = r.data.problems.map(p => p.folder + ': ' + p.error).join('\n'); @@ -130,7 +131,7 @@ test('the built-in modules, the server-provided ones, and the folders that are n // a contributor only sees the file imports open to them, with their own settings const a = await api('alice', 'GET', '/api/connectors'); assert.deepEqual(a.data.connectors.map(c => c.id), ['paloalto']); - assert.deepEqual(Object.keys(a.data.connectors[0].settings).sort(), ['firewall', 'scope', 'site']); + assert.deepEqual(Object.keys(a.data.connectors[0].settings).sort(), ['build', 'firewall', 'scope', 'site']); assert.equal(a.data.problems, undefined); }); @@ -283,6 +284,54 @@ test('store modules: an Nmap scan imported by a contributor, NetBox synced by an for (const id of ['nmap', 'netbox']) assert.equal((await api('admin', 'DELETE', `/api/modules/${id}`, {})).status, 200); }); +test('CSV VLANs: the VLANs of a file, a choice of VLANs and zones, then a proposal', async () => { + assert.equal((await api('admin', 'POST', '/api/modules/csv-vlans/install', {})).status, 200); + const c = (await api('alice', 'GET', '/api/connectors')).data.connectors.find(x => x.id === 'csv-vlans'); + assert.deepEqual(c.file.pick, { title: 'Choose the VLANs and their zone', help: c.file.pick.help, zones: true }); + // no header, columns in the order range, name, number; VLAN 20 is already in Lyon; a line without number + const file = { name: 'vlans.csv', text: '10.30.0.0/24;Lab A;300\n10.30.1.0/24;Lab B;301\n10.20.0.0/24;Servers (renamed);20\nbad;line;x\n10.40.0.0/24;Cameras;400\n' }; + const pv = await api('alice', 'POST', '/api/connectors/csv-vlans/preview', { file, overrides: { site: 'lyon' } }); + assert.equal(pv.status, 200, JSON.stringify(pv.data)); + assert.deepEqual(pv.data.items.map(i => [i.key, i.label, i.detail, i.on]), [ + ['300', 'VLAN 300 · Lab A', '10.30.0.0/24', true], ['301', 'VLAN 301 · Lab B', '10.30.1.0/24', true], + ['20', 'VLAN 20 · Servers (renamed)', '10.20.0.0/24', false], ['400', 'VLAN 400 · Cameras', '10.40.0.0/24', true]]); + assert.match(pv.data.items[2].tags[0], /already in the IS/); + assert.equal(pv.data.items[2].zone, 'Main', 'a VLAN already in the IS suggests its own zone'); + assert.match(pv.data.skipped[0], /Line 4: no VLAN number/); + assert.equal(pv.data.pick.zones, true); + // a preview records no run and opens no proposal + assert.equal((await api('admin', 'GET', '/api/proposals')).data.proposals.filter(p => p.connector === 'csv-vlans').length, 0); + + // the choice: 300 and 301 in a new zone Lab, 20 updated where it is, 400 left out + const picks = { 300: { zone: 'new:Lab' }, 301: { zone: 'new:Lab' }, 20: { zone: 'z1' } }; + const r = await api('alice', 'POST', '/api/connectors/csv-vlans/run', { wait: true, file, overrides: { site: 'lyon' }, picks }); + assert.equal(r.data.run.ok, true, r.data.run.error); + assert.match(r.data.run.summary, /4 VLAN\(s\) read, 3 chosen: 2 to add, 1 to update; new zone\(s\): Lab/); + const p = (await api('admin', 'GET', `/api/proposals/${r.data.run.proposals[0].id}`)).data.proposal; + const si = p.changes.si.lyon, lab = si.zones.find(z => z.name === 'Lab'); + assert.ok(lab, 'the new zone is in the proposal'); + assert.deepEqual(si.vlans.filter(v => v.zone === lab.id).map(v => v.num).sort(), [300, 301]); + assert.equal(si.vlans.find(v => v.num === 20).name, 'Servers (renamed)'); + assert.equal(si.vlans.find(v => v.num === 20).id, 'v20', 'an updated VLAN keeps its id, and so its assets'); + assert.ok(!si.vlans.some(v => v.num === 400), 'a VLAN left unticked is not brought'); + assert.equal(p.uploadedBy, 'alice'); + + // the same choice again refreshes the same proposal; from a script, without a choice, every new VLAN comes + const again = await api('alice', 'POST', '/api/connectors/csv-vlans/run', { wait: true, file, overrides: { site: 'lyon' }, picks }); + assert.equal(again.data.run.proposals[0].id, p.id); + await api('admin', 'PUT', '/api/connectors/csv-vlans', { defaultZone: 'Imported' }); + const all = await api('alice', 'POST', '/api/connectors/csv-vlans/run', { wait: true, file, overrides: { site: 'lyon' } }); + assert.match(all.data.run.summary, /4 VLAN\(s\) read: 3 to add, 0 to update, 1 unchanged; new zone\(s\): Imported/); + + // what is refused + assert.equal((await api('alice', 'POST', '/api/connectors/csv-vlans/preview', { file: { name: 'x.csv', text: 'name;range\nx;10.0.0.0/8' }, overrides: { site: 'lyon' } })).status, 400); + assert.equal((await api('alice', 'POST', '/api/connectors/csv-vlans/preview', { overrides: { site: 'lyon' } })).status, 400); + assert.equal((await api('alice', 'POST', '/api/connectors/csv-vlans/run', { wait: true, file, overrides: { site: 'lyon' }, picks: 'all' })).status, 400); + assert.equal((await api('bob', 'POST', '/api/connectors/csv-inventory/preview', { file, overrides: { site: 'lyon' } })).status, 404, 'csv-inventory is not installed'); + for (const pr of (await api('admin', 'GET', '/api/proposals')).data.proposals.filter(x => x.connector === 'csv-vlans')) await api('admin', 'DELETE', `/api/proposals/${pr.id}`, {}); + assert.equal((await api('admin', 'DELETE', '/api/modules/csv-vlans', {})).status, 200); +}); + test('the store refuses a package that does not match its checksum', async () => { tamper = true; try { @@ -323,6 +372,11 @@ test('package reader and manifest checks', () => { assert.throws(() => checkManifest({ ...man, permissions: { files: true } }), /permissions/); assert.throws(() => checkManifest({ ...man, live: false }), /must import files/); assert.throws(() => checkManifest({ ...man, minStrata: '99.0.0' }, { strataVersion: '0.8.0' }), /needs Strata 99/); + const fileOf = pick => ({ ...man, live: false, fields: [{ k: 'site', label: 'IS', type: 'site' }], file: { label: 'Import', fields: ['site'], pick } }); + assert.deepEqual(checkManifest(fileOf({ zones: true })).file.pick, { title: 'Choose what to import', help: '', zones: true }); + assert.equal(checkManifest({ ...man, live: false, file: { label: 'Import' } }).file.pick, null); + assert.throws(() => checkManifest(fileOf('yes')), /file.pick must be an object/); + assert.throws(() => checkManifest({ ...fileOf({ zones: true }), file: { label: 'Import', pick: { zones: true } } }), /needs a setting of type site/); assert.ok(compareVersions('0.10.0', '0.9.1') > 0 && compareVersions('1.0.0-rc.1', '1.0.0') < 0 && compareVersions('1.2.3', '1.2.3') === 0); // the top folder goes, whatever its name; a link inside a package is refused const tgz = writeTgz([{ path: 'strata-module.json', data: Buffer.from('{}') }, { path: 'lib/a.mjs', data: Buffer.from('') }], 'anything'); diff --git a/tests/muster.test.mjs b/tests/muster.test.mjs index 821ccef..36d9598 100644 --- a/tests/muster.test.mjs +++ b/tests/muster.test.mjs @@ -8,6 +8,7 @@ import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth } from './session.mjs'; const root = join(dirname(fileURLToPath(import.meta.url)), '..'); const dataDir = mkdtempSync(join(tmpdir(), 'strata-muster-')); @@ -38,8 +39,8 @@ function fakeMuster(req, res) { json(404, { error: 'Not found' }); }); } -const as = user => (method, path, body) => fetch(base + path, { - method, headers: { Authorization: 'Basic ' + Buffer.from(`${user}:${PASS[user]}`).toString('base64'), ...(body ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}) }, +const as = user => async (method, path, body) => fetch(base + path, { + method, headers: { ...(await auth(base, user, PASS[user])), ...(body ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}) }, body: body ? JSON.stringify(body) : undefined, }).then(async r => ({ status: r.status, data: await r.json().catch(() => null) })); diff --git a/tests/paloalto.test.mjs b/tests/paloalto.test.mjs new file mode 100644 index 0000000..46e752f --- /dev/null +++ b/tests/paloalto.test.mjs @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: AGPL-3.0-or-later +// Palo Alto module: building an IS from a configuration (zones, VLANs from the interfaces, assets from the +// address objects, the firewall), then the rules as flows. Run with: npm test +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { parseConfig, rulesToChanges } from '../server/connectors/builtin/paloalto/index.js'; + +const XML = ` + FW-PARIS + + + + + 20Servers + 30Users + + + + + ethernet1/1 + ethernet1/2ethernet1/2.20 + ethernet1/2.30 + +
    + 10.30.0.1/23 + 10.20.0.10ERP database + 10.20.0.11/32 + 10.0.0.50 + 10.30.0.77 + 192.168.99.9 + 10.30.0.0/23 + 203.0.113.10 +
    + 502 + + trusttrustapp-01SQL-01anytcp-1433allow + userstrusthost-77app-01anytcp-443allow + trusttrustapp-01plc-lineanytcp-502allow + untrusttrustanyapp-01anytcp-443allow + +
    +
    `; + +const build = state => rulesToChanges(parseConfig(XML), { site: 'p', build: true }, state); + +test('Palo Alto, build the IS: zones, VLANs from the interfaces, assets from the address objects, the firewall', () => { + const r = build({ si: { p: { name: 'Paris' } }, assets: {}, services: [] }); + const si = r.changes.si.p, A = r.changes.assets, by = n => Object.values(A).find(a => a.name === n); + // zones of the interfaces that are not the Internet (and the zone of "To sort", for the host outside them) + assert.deepEqual(si.zones.map(z => z.name), ['trust', 'users', 'Imported']); + // a VLAN per subnet: tagged sub-interfaces keep their tag, the untagged interface gets 4001; an interface + // address given as an object is read through it; the comment names the VLAN + assert.deepEqual(si.vlans.filter(v => !/unsorted$/.test(v.id)).map(v => [v.num, v.name, v.cidr, si.zones.find(z => z.id === v.zone).name]), + [[4001, 'trust ethernet1/2 (untagged)', '10.0.0.0/24', 'trust'], [20, 'Servers', '10.20.0.0/24', 'trust'], [30, 'Users', '10.30.0.0/23', 'users']]); + // the type: from the name or description, from the ports opened to it, a client only is a workstation + assert.equal(by('SQL-01').type, 'db'); + assert.equal(by('SQL-01').role, 'ERP database'); + assert.equal(by('plc-line').type, 'plc'); + assert.equal(by('host-77').type, 'workstation'); + assert.equal(by('app-01').type, 'server'); + assert.equal(by('SQL-01').vlan, si.vlans.find(v => v.num === 20).id); + // a host outside every VLAN waits in "To sort"; ranges, public addresses and the firewall's own are no assets + assert.match(by('lost').vlan, /unsorted$/); + for (const n of ['users-net', 'partner', 'gw-users']) assert.equal(by(n), undefined, n); + // the firewall: its host name, its inside address + const fw = Object.entries(A).find(([, a]) => a.type === 'firewall'); + assert.deepEqual([fw[1].name, fw[1].ip], ['FW-PARIS', '10.0.0.1']); + // flows: through the firewall when they change VLAN or come from the Internet, not inside a VLAN + const f = (s, d) => si.flows.find(x => x.src === s && x.dst === d); + const id = n => Object.entries(A).find(([, a]) => a.name === n)[0]; + assert.equal(f(id('app-01'), id('SQL-01')).via, undefined); + assert.deepEqual(f(id('host-77'), id('app-01')).via, [fw[0]]); + assert.deepEqual(f('internet', id('app-01')).via, [fw[0]]); + assert.equal(r.stats.vlansAdded, 3); + assert.equal(r.stats.assetsAdded, 5); +}); + +test('Palo Alto, build the IS: what the map has is kept, and the same file again changes nothing', () => { + const first = build({ si: { p: { name: 'Paris', zones: [{ id: 'z9', name: 'TRUST' }], vlans: [{ id: 'mine', num: 20, name: 'My servers', zone: 'z9' }] } }, + assets: { mydb: { site: 'p', name: 'erp-db', type: 'db', vlan: 'mine', ip: '10.20.0.10' } }, services: [] }); + const si = first.changes.si.p; + assert.equal(si.vlans.find(v => v.num === 20).id, 'mine', 'the VLAN of the map is kept, and gets its range'); + assert.equal(si.vlans.find(v => v.num === 20).cidr, '10.20.0.0/24'); + assert.equal(si.zones.filter(z => /trust/i.test(z.name)).length, 1, 'a zone of the same name is reused'); + assert.ok(!Object.values(first.changes.assets).some(a => a.ip === '10.20.0.10'), 'an asset already at that IP is not created again'); + const state = { si: { p: si }, assets: { mydb: { site: 'p', name: 'erp-db', type: 'db', vlan: 'mine', ip: '10.20.0.10' }, ...first.changes.assets }, services: [] }; + assert.equal(build(state).changes, null); +}); + +test('Palo Alto without "build": an empty IS gives nothing to draw, and no empty proposal', () => { + const r = rulesToChanges(parseConfig(XML), { site: 'p' }, { si: { p: { name: 'Paris' } }, assets: {}, services: [] }); + assert.equal(r.changes, null); + assert.ok(r.skipped.some(x => /no asset of the IS at 10\.20\.0\.10/.test(x))); +}); diff --git a/tests/readonly.test.mjs b/tests/readonly.test.mjs index 26d1d4b..6b4110c 100644 --- a/tests/readonly.test.mjs +++ b/tests/readonly.test.mjs @@ -8,6 +8,7 @@ import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth } from './session.mjs'; const root = join(dirname(fileURLToPath(import.meta.url)), '..'); @@ -21,7 +22,7 @@ test('READ_ONLY=true: an admin can read, and every write is refused with 403', a const call = async (method, path, body) => { const r = await fetch(`http://127.0.0.1:${port}${path}`, { method, - headers: { Authorization: 'Basic ' + Buffer.from(`admin:${pass}`).toString('base64'), ...(body ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}) }, + headers: { ...(await auth(`http://127.0.0.1:${port}`, 'admin', pass)), ...(body ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}) }, body: body ? JSON.stringify(body) : undefined, }); return { status: r.status, data: await r.json().catch(() => null) }; diff --git a/tests/secrets.test.mjs b/tests/secrets.test.mjs index 615b819..332c913 100644 --- a/tests/secrets.test.mjs +++ b/tests/secrets.test.mjs @@ -8,6 +8,7 @@ import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth } from './session.mjs'; const root = join(dirname(fileURLToPath(import.meta.url)), '..'); @@ -27,7 +28,7 @@ function start(extra) { proc.on('exit', code => { rmSync(dataDir, { recursive: true, force: true, maxRetries: 5 }); rej(Object.assign(new Error(`exit ${code}`), { code, err })); }); }); } -const status = (port, user, pass) => fetch(`http://127.0.0.1:${port}/api/state`, { headers: { Authorization: 'Basic ' + Buffer.from(`${user}:${pass}`).toString('base64') } }).then(r => r.status); +const status = async (port, user, pass) => (await fetch(`http://127.0.0.1:${port}/api/state`, { headers: await auth(`http://127.0.0.1:${port}`, user, pass) })).status; test('EDIT_PASS_FILE and VIEW_PASS_FILE: the passwords come from the files, which win over the variables', async () => { const dir = mkdtempSync(join(tmpdir(), 'strata-cred-')); diff --git a/tests/session.mjs b/tests/session.mjs new file mode 100644 index 0000000..1eab594 --- /dev/null +++ b/tests/session.mjs @@ -0,0 +1,27 @@ +// SPDX-License-Identifier: AGPL-3.0-or-later +// Signs a local account in with the sign-in form (POST /auth/login) and keeps its session cookie, per +// server, account and password: the tests send it as the browser does. HTTP Basic is gone since 0.11. +const cache = new Map(); +// base: http://127.0.0.1:port (with or without a trailing slash); extra: headers of the sign-in request +export async function login(base, user, pass, extra = {}) { + const r = await fetch(new URL('/auth/login', base), { + method: 'POST', redirect: 'manual', + headers: { 'Content-Type': 'application/json', ...extra }, + body: JSON.stringify({ user, password: pass }), + }); + await r.arrayBuffer(); + const set = r.headers.get('set-cookie') || ''; + return { status: r.status, retryAfter: r.headers.get('retry-after'), cookie: r.ok ? set.split(';')[0] : '' }; +} +// The headers that sign this account in ({} when the password is wrong: the request then gets a 401) +export async function auth(base, user, pass, extra) { + if (!user) return {}; + const key = `${base}|${user}|${pass}`; + if (!cache.has(key)) { + const r = await login(base, user, pass, extra); + if (!r.cookie) return {}; + cache.set(key, r.cookie); + } + return { Cookie: cache.get(key) }; +} +export const forget = () => cache.clear(); diff --git a/tests/signin.test.mjs b/tests/signin.test.mjs index 3ab084d..eb69ead 100644 --- a/tests/signin.test.mjs +++ b/tests/signin.test.mjs @@ -8,6 +8,7 @@ import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth, login } from './session.mjs'; const root = join(dirname(fileURLToPath(import.meta.url)), '..'); const ADMIN = 'admin-pass-123'; @@ -28,18 +29,19 @@ function start(extra) { proc.on('exit', code => rej(new Error(`exit ${code}`))); }); } -// GET /api/state as user:pass, from a client address given by X-Forwarded-For (only believed behind a trusted proxy) +// Signs in with the form as user:pass, from a client address given by X-Forwarded-For (only believed +// behind a trusted proxy), then reads /api/state with the session: 200, 401 (wrong) or 429 (locked) async function get(port, user, pass, from) { - const headers = { Authorization: 'Basic ' + Buffer.from(`${user}:${pass}`).toString('base64') }; - if (from) headers['X-Forwarded-For'] = from; - const r = await fetch(`http://127.0.0.1:${port}/api/state`, { headers }); - await r.arrayBuffer(); - return { status: r.status, retryAfter: r.headers.get('retry-after') }; + const r = await login(`http://127.0.0.1:${port}`, user, pass, from ? { 'X-Forwarded-For': from } : {}); + if (!r.cookie) return { status: r.status, retryAfter: r.retryAfter }; + const s = await fetch(`http://127.0.0.1:${port}/api/state`, { headers: { Cookie: r.cookie } }); + await s.arrayBuffer(); + return { status: s.status }; } async function addUser(port, name, password) { const r = await fetch(`http://127.0.0.1:${port}/api/users/${name}`, { method: 'PUT', - headers: { Authorization: 'Basic ' + Buffer.from(`admin:${ADMIN}`).toString('base64'), 'Content-Type': 'application/json', 'X-Requested-With': 'strata' }, + headers: { ...(await auth(`http://127.0.0.1:${port}`, 'admin', ADMIN)), 'Content-Type': 'application/json', 'X-Requested-With': 'strata' }, body: JSON.stringify({ role: 'viewer', password }), }); assert.equal(r.status, 200); @@ -91,14 +93,15 @@ test('a client address is locked after LOGIN_MAX_FAILS_PER_ADDRESS, read from X- } finally { await t.stop(); } }); -test('a browser already signed in keeps working from its address while the account is locked', async () => { +test('a browser already signed in keeps its session while the account is locked', async () => { const s = await start({ LOGIN_MAX_FAILS: '3', LOGIN_MAX_FAILS_PER_ADDRESS: '100', TRUST_PROXY: '127.0.0.1' }); try { - assert.equal((await get(s.port, 'admin', ADMIN, '192.0.2.10')).status, 200); + const signed = await login(`http://127.0.0.1:${s.port}`, 'admin', ADMIN, { 'X-Forwarded-For': '192.0.2.10' }); + assert.equal(signed.status, 200); // someone else guesses the admin password from elsewhere for (const p of ['guess-1', 'guess-2', 'guess-3']) assert.equal((await get(s.port, 'admin', p, '192.0.2.66')).status, 401); - // the signed-in browser goes on; the right password from any other address waits - assert.equal((await get(s.port, 'admin', ADMIN, '192.0.2.10')).status, 200); + // the signed-in browser goes on with its session; a new sign-in, from any address, waits + assert.equal((await fetch(`http://127.0.0.1:${s.port}/api/state`, { headers: { Cookie: signed.cookie, 'X-Forwarded-For': '192.0.2.10' } })).status, 200); assert.equal((await get(s.port, 'admin', ADMIN, '192.0.2.66')).status, 429); assert.equal((await get(s.port, 'admin', ADMIN, '192.0.2.11')).status, 429); } finally { await s.stop(); } diff --git a/tests/sso.test.mjs b/tests/sso.test.mjs index 0394caf..0883673 100644 --- a/tests/sso.test.mjs +++ b/tests/sso.test.mjs @@ -10,6 +10,7 @@ import crypto from 'node:crypto'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; +import { auth } from './session.mjs'; import { verifyJwt } from '../server/sso.js'; const root = join(dirname(fileURLToPath(import.meta.url)), '..'); @@ -98,7 +99,8 @@ async function signIn() { return { status: r.status, location: r.headers.get('location'), cookie: cookies(), text: r.status === 302 ? '' : await r.text() }; } const api = (path, cookie, init = {}) => fetch(base + path, { ...init, headers: { ...(cookie ? { Cookie: cookie } : {}), ...(init.body ? { 'Content-Type': 'application/json', 'X-Requested-With': 'strata' } : {}), ...init.headers } }); -const basic = (u, p) => ({ Authorization: 'Basic ' + Buffer.from(`${u}:${p}`).toString('base64') }); +// a local account, signed in with the sign-in form: its session cookie +const basic = (u, p) => auth(base, u, p); test('a page without a session goes to the sign-in page; the API answers 401 without a password prompt', async () => { const r = await fetch(`${base}/`, { redirect: 'manual' }); @@ -106,16 +108,27 @@ test('a page without a session goes to the sign-in page; the API answers 401 wit assert.equal(r.headers.get('location'), `${base}/auth/login`); const login = await (await fetch(`${base}/auth/login`)).text(); assert.match(login, /Sign in with Test IdP/); - assert.match(login, /Sign in with a local account/); + assert.match(login, /or with a local account/); const a = await fetch(`${base}/api/me`); assert.equal(a.status, 401); assert.equal(a.headers.get('www-authenticate'), null); - // "Sign in with a local account": the password prompt is asked at the root, so the browser then sends it everywhere - assert.ok(login.includes(`href="${base}/?local"`) && login.includes(`href="${base}/auth/oidc/start"`)); - const l = await fetch(`${base}/?local`, { redirect: 'manual' }); - assert.equal(l.status, 401); - assert.match(l.headers.get('www-authenticate'), /^Basic/); - assert.equal((await fetch(`${base}/?local`, { headers: basic('admin', 'admin-pass-123') })).status, 200); + // local accounts sign in with the form of the same page, which gives a session cookie + assert.ok(login.includes(`action="${base}/auth/login"`) && login.includes(`href="${base}/auth/oidc/start"`)); + const form = await fetch(`${base}/auth/login`, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, body: 'user=admin&password=admin-pass-123' }); + assert.equal(form.status, 303); + assert.equal(form.headers.get('location'), `${base}/`); + const cookie = form.headers.get('set-cookie').split(';')[0]; + assert.equal((await fetch(`${base}/`, { headers: { Cookie: cookie } })).status, 200); + const bad = await fetch(`${base}/auth/login`, { method: 'POST', headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, body: 'user=admin&password=nope' }); + assert.equal(bad.status, 401); + assert.match(await bad.text(), /Wrong user name or password/); + // another site cannot sign a visitor in + assert.equal((await fetch(`${base}/auth/login`, { method: 'POST', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Origin: 'https://evil.example' }, body: 'user=admin&password=admin-pass-123' })).status, 403); + assert.equal((await fetch(`${base}/auth/login`, { method: 'POST', headers: { 'Content-Type': 'application/json', Origin: 'not a url' }, body: '{}' })).status, 403); + // a local session signs out too + const out = await (await api('/auth/logout', cookie, { method: 'POST', body: '{}' })).json(); + assert.equal(out.location, `${base}/auth/login?out=1`); + assert.equal((await api('/api/me', cookie)).status, 401); }); test('OpenID Connect: groups give the role, the session cookie signs in, sign-out ends it', async () => { @@ -170,11 +183,11 @@ test('verifyJwt refuses "none" and unknown algorithms', () => { }); test('LOCAL_LOGIN=admins: the local admin keeps a way in, other local accounts do not', async () => { - assert.equal((await api('/api/users/bob', null, { method: 'PUT', headers: basic('admin', 'admin-pass-123'), body: JSON.stringify({ role: 'approver', password: 'bob-pass-12345' }) })).status, 200); - assert.equal((await api('/api/me', null, { headers: basic('admin', 'admin-pass-123') })).status, 200); + assert.equal((await api('/api/users/bob', null, { method: 'PUT', headers: await basic('admin', 'admin-pass-123'), body: JSON.stringify({ role: 'approver', password: 'bob-pass-12345' }) })).status, 200); + assert.equal((await api('/api/me', null, { headers: await basic('admin', 'admin-pass-123') })).status, 200); // refused like a wrong password: the answer says nothing of bob's password - assert.equal((await api('/api/me', null, { headers: basic('bob', 'bob-pass-12345') })).status, 401); - assert.equal((await api('/api/me', null, { headers: basic('bob', 'wrong-password') })).status, 401); + assert.equal((await api('/api/me', null, { headers: await basic('bob', 'bob-pass-12345') })).status, 401); + assert.equal((await api('/api/me', null, { headers: await basic('bob', 'wrong-password') })).status, 401); }); test('a provider name that is also a local account, or an unverified e-mail, is refused', async () => { @@ -200,7 +213,7 @@ test('a sign-in finishes once; an admin can end every session', async () => { assert.equal(again.status, 400); const s = await signIn(); assert.equal((await api('/api/me', s.cookie)).status, 200); - const end = await (await api('/api/sessions', null, { method: 'DELETE', headers: basic('admin', 'admin-pass-123'), body: '{}' })).json(); + const end = await (await api('/api/sessions', null, { method: 'DELETE', headers: await basic('admin', 'admin-pass-123'), body: '{}' })).json(); assert.ok(end.ended >= 1); assert.equal((await api('/api/me', s.cookie)).status, 401); }); @@ -214,7 +227,7 @@ test('headers need SSO_HEADER_TRUST with exact addresses', async () => { }); test('API tokens: created by an admin, shown once, used as Bearer, revoked', async () => { - const admin = basic('admin', 'admin-pass-123'); + const admin = await basic('admin', 'admin-pass-123'); const c = await (await api('/api/tokens', null, { method: 'POST', headers: admin, body: JSON.stringify({ name: 'ci', role: 'contributor', days: 7 }) })).json(); assert.match(c.token, /^stk_[0-9a-f]{16}_/); const bearer = { Authorization: `Bearer ${c.token}` };