diff --git a/docs/briefings/2026-09-26-morning.html b/docs/briefings/2026-09-26-morning.html new file mode 100644 index 000000000..9e8cb28da --- /dev/null +++ b/docs/briefings/2026-09-26-morning.html @@ -0,0 +1,929 @@ + + + + + + +House scanning briefing: what we are building, what works, what is blocked + + + + + + + +
+ +

House scanning: what we are building, what works, and what is blocked

+

Team briefing, 26 September 2026, facts as of 09:10. Readers: Sam, Hunter, Shrey, Aiden. About ten minutes.

+ +

A homeowner walks the outside wall around their electric meter with an iPhone. The phone measures the wall, a server applies the clearance rules in plain code, and the app shows where the battery goes, in AR, on the real wall.

+

Today it runs end to end in the iPhone Simulator on a recorded real walk and gets a real answer from the live server. It has never run on a physical iPhone at a real wall. That is the blocker, and the first job this morning.

+ + + + +

1. The problem: a battery has to fit between the rules

+ +
+ + An outside wall with the battery and its clearance rules, drawn to scale + Front view of a house wall. From left: an AC unit on the ground, a gas meter, the electric meter, the battery just to its right, a window, a door. Red hatched zones show 3 feet of clearance around the gas meter, AC unit, window and door. A bracket beside the battery shows 6.5 feet of headroom. A blue bracket along the ground shows 20 feet of cable reach either side of the electric meter. A tape along the bottom counts feet from the meter. Below, a view from above shows the battery footprint, 31 by 22 inches, and the 3 feet of clear space needed in front of it. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 20 ft of cable, either side of the meter + + + + AC unit + gas meter + electric meter + window + door + 6.5 ft headroom + battery + + + 3 ft + 3 ft + 3 ft + 3 ft + + + + + + + + + + + 151050 + 510152025 + + feet left and right of the electric meter + + + The same wall from above + + + + + + + + + + + + + + + + + + + + + + + + battery footprint, 31 by 22 in + 3 ft clear in front + AC unit + gas + + +

Wider than the screen: scroll sideways for the rest of the picture.

+
Everything is drawn at one scale: 5 ft between tape marks. The red hatched areas are places the battery may not go. The battery just fits between the electric meter and the window's zone. Rules shown: 3 ft from the gas meter, from AC units, fences and other batteries, and from doors and windows that enter the home; 3 ft of clear space in front; 6.5 ft of headroom; within 20 ft of cable run from the meter; a flat, seen stretch of wall behind it.
+
+ +
+

Base Power puts a home battery on the outside wall, close to the electric meter, because a cable runs from the meter to the battery. The battery is 31 in wide, 22 in deep and 39.5 in tall. Whether it fits, and where, depends on the clearance rules in the picture. Today a person has to go to the house and check.

+

Our product does that check with the homeowner's iPhone, in a couple of minutes. The app measures the wall. A server applies the rules in plain code. The app shows where the battery goes, in AR, on the real wall.

+

The demo's rules come from public sources and live in server/rules.yaml, each with its citation. Two numbers there are placeholders because no public value exists (pool 10 ft, driveway 5 ft). Base's own thresholds are private, load at runtime from a file that is not in git, and do not appear on this page.

+
+
+ The rules the demo uses, with sources +
+ + + + + + + + + + + +
RuleValueSource
Distance from the gas meter3 ftBase's public help page; Austin Energy; Texas Gas Service
Distance from AC units, fences, other batteries3 ftBase's help page
Distance from doors and windows that enter the home3 ftIRC R328.4
Clear space in front of the battery3 ftBase's help page
Cable run from the meterwithin 20 ftBase's help page
Headroom6.5 ftNEC 110.26, a demo choice
Wall behind the batteryflat, and seen by the camerademo requirement
Pool, driveway10 ft, 5 ftplaceholders; no public value exists
+
+ + +

2. How a scan becomes a decision

+

Six pictures. Each one is the last one with one thing added. The tape along the bottom is real: the app shows the wall unrolled like a tape measure, with the electric meter at zero.

+ +
    +
  1. + + Step 1: tap the electric meter, which becomes zero + The wall from the problem picture, smaller, with the electric meter in the middle. A blue ring marks a tap on the meter. A dashed line drops from the meter to the zero mark on a tape along the bottom. + + + + + tap the electric meter + it becomes 0 ft; all else is left or right of it + +
    Point at the electric meter and tap. The meter becomes zero. Every position from now on is measured left or right of it, in feet.
    +
  2. + +
  3. + + Step 2: the phone photographs the meter close up, then walks left, then right + Same wall. A dashed frame around the electric meter marks the close-up. A phone stands on the ground below the meter. Blue arrows run left along the wall and then right. + + + close-up, reads the number + + + + walk left + then right + +
    The phone takes a close-up of the meter by itself when the picture is sharp, to read the meter's number. Then you walk along the wall, left and then right. The phone keeps photos automatically.
    +
  4. + +
  5. + + Step 3: a haze covers wall and ground the phone has not seen yet + Same wall. The phone has walked left to about 8 feet and right to about 9 feet. Those parts of the wall and ground are clear. The rest is covered by a grey hatched haze. + + + + + + + + haze: + not seen yet + haze + seen: the haze lifts + +
    A haze covers wall and ground the phone has not seen and lifts as it sees them. The team calls this fog of war. The tape strip grows as you walk.
    +
  6. + +
  7. + + Step 4: mark things as you pass, and answer one request for another view + Same wall, almost all seen. Orange pins mark the gas meter, the window and the AC unit. A small hatched patch of ground about 6 feet right of the meter is still unseen. A message at the top reads: Show the ground about 6 ft right of your meter. + + + + + + gas + window + AC + + + Show the ground about 6 ft right of your meter + +
    Mark things as you pass: gas meter, AC unit, door, window, driveway edge, fence. If something a rule needs is still unseen, the app asks for one more view, in one sentence.
    +
  8. + +
  9. + + Step 5: a small measurement file goes to the server, which tries every spot + Same wall, fully seen. A small file icon labelled scene.json, 17 KB, rises from the phone toward a server box at the top. On the wall, dashed battery outlines slide along in small steps. + + + + + + tries every spot, 2 in apart + + + scene.json, 17 KB + photos stay on the phone + + + server: checks every rule + +
    The phone sends a small file of measurements, about 17 KB for a real walk. The server slides the battery's footprint along the wall in 2-inch steps and checks every rule at every position.
    +
  10. + +
  11. + + Step 6: the answer comes back and the battery is shown on the wall in AR + Same wall. The server box at the top sends an arrow back down to the phone. The battery is drawn on the wall just right of the electric meter with an orange outline, and a cable runs from the meter to it. A label reads: best spot, shortest cable. + + + server answers + + + + + + best spot, + shortest cable + shown in 3D and in AR, placed from the meter + +
    The server answers with the best spot (shortest cable run first), the cable route, and every check's outcome and reason. The phone draws the battery on the wall, placed relative to the meter, in 3D and in AR.
    +
  12. +
+ +
+ + The whole loop: phone to server and back + Two boxes. Left: the phone, which taps, walks, marks and shows. Right: the server, with the rules file and plain code. An arrow from phone to server carries scene.json. An arrow from server to phone carries the spot, the cable route and every check's outcome. + + Phone + tap, walk, mark + show the spot in AR + + Server + rules in a file + plain code, no AI + + scene.json + + spot, cable, checks + +
The recap. AI is used only to recognise things: it reads the meter's text and finds objects. The placement decision is ordinary code with the rules in a file, so every answer traces back to a rule and a measurement.
+
+ + +

3. Three ideas that make the answer honest

+ +

The phone counts its own steps, so error grows with distance

+
+ + Error bars widen the farther you walk from the meter + A tape along the top counts feet from the meter at 0, 5, 10, 15 and 20. Below it, four positions carry horizontal error bars that get wider with distance: plus or minus 0.3 feet at the meter, 1.1 feet at 5 feet, 1.9 feet at 10 feet and 3.5 feet at 20 feet. Footprints lead away from the meter. + + + + + + + 05101520 ft + + + meter + + + + + + + + + + + + + + + ± 0.3 ft at the meter + ± 1.1 ft at 5 ft + ± 1.9 ft at 10 ft + ± 3.5 ft at 20 ft + + error bar = 0.3 ft + 0.16 ft for every foot from the meter + +
AR tracking is dead reckoning: the phone estimates how far it moved from the camera and motion sensors, like counting your own steps from a doorway. Small errors add up, so the farther from the meter, the less sure a position is. The server widens every AR measurement's error bar by 0.3 ft plus 0.16 ft per foot along the wall. Where the analogy breaks: a miscounted step is only a stretch along the wall, while the phone's error also drifts sideways.
+
+ +

The error bar decides: PASS, FAIL or UNSURE

+
+ + Three measurements against the 3 foot gas meter rule + A number line from 0 to 6 feet with a vertical rule at 3 feet. Row one: 4.5 feet plus or minus 0.8, the whole bar sits right of 3, PASS. Row two: 2.0 feet plus or minus 0.8, the whole bar sits left of 3, FAIL. Row three: 3.4 feet plus or minus 0.8, the bar straddles 3, UNSURE. + + + + + 0123456 ft + + + 3 ft rule (gas meter) + too close + clear + + + + + + + 4.5 ft ± 0.8: clear by 1.5, more than 0.8 + + ✓ PASS + + + + + + + 2.0 ft ± 0.8: short by 1.0, more than 0.8 + + ✕ FAIL + + + + + + + 3.4 ft ± 0.8: clear by only 0.4 + + ? UNSURE + +
Every measurement carries an error bar. A check passes only when the margin beats the error, fails only when it is past the rule by more than the error, and is UNSURE otherwise. Equal is UNSURE. UNSURE means a person checks it, or the app asks for a better view. The three examples are the dossier's, for the 3 ft gas meter rule with ± 0.8 ft.
+
+ +

Unseen is not clear

+
+ + A check whose ground was never seen stays UNSURE and names the missing view + A candidate battery spot on the wall. The ground beside it is covered by grey haze. Beside it a check reads: ground near the spot, checks need 6 to 13 ft of seen ground, UNSURE, missing evidence: show the ground about 6 ft right of your meter. + + + + + + meter + + + candidate spot + + never seen + + Ground near the spot + checks need 6 to 13 ft of ground + + ? UNSURE + missing evidence: show the + ground 6 ft right of your meter + +
If the camera never saw part of the area a rule depends on, that check cannot pass. It stays UNSURE, and the server names the view that would settle it. Here the ground beside the spot was never seen, and the checks that use ground need 6 to 13 ft of it. A blank on the map is never treated as empty ground.
+
+ + +

4. Where it stands this morning

+
+ ✓ proven in the Simulator on a recorded real walk + ! fixed this morning + ? not yet proven on a real phone +
+
    +
  1. Tap the meterIt becomes zero.
    ✓Simulator?Real phone
  2. +
  3. Read the meter's numberClose-up; three candidates to tap.
    ✓Simulator?Real phone
  4. +
  5. Walk the wallHaze lifts; tape strip grows.
    ✓Simulator!Now reports ground depth, clear space in front, and a tilt-up view for headroom?Real phone
  6. +
  7. Mark things, one more viewGas, AC, door, window, fence.
    ✓Simulator!Server now publishes exactly what settles each check?Real phone
  8. +
  9. Upload scene.jsonMeasurements only.
    !17 KB, not 16.5 MB of photos; answer in 0.5 s?Real phone
  10. +
  11. Server decidesLive on Vercel.
    ✓252 tests; independent checker 50 of 50
  12. +
  13. Show the spot in ARPlaced from the meter.
    ✓Simulator, 3D and AR?Real phone
  14. +
+
✕
Never run on a physical iPhone at a real wall.Every stage marked "?" above shares this one gap. Everything that decides accuracy is waiting on that data.
+
+
Placeholder rulesWith the public placeholder rules, automatic approval is off, so every real result says "An installer will take a look."
+
Hidden wallThings standing in front of the wall can hide it, and nobody owns checking that yet (decision 7).
+
Parked: web review page (#15)It shows the server's answer in a browser. It helped check the server before the app could upload. It is not the product.
+
+ +
+
Does what we built work, and is it part of the app we'd ship?
In the Simulator on a recorded real walk, yes: meter to result, with a real answer from the live server. The app and the server are the product. The web review page is not, and it is parked.
+
Are the overnight tests doing anything?
Each of the five changed the app or the server (section 6 lists what). Three other pieces of overnight work changed nothing: the panel-label search, finer statistics on the modern-phone data, and the research notes.
+
What is done that achieves the goal?
The server, live with 252 tests. The whole app flow in the Simulator. Uploads fixed to 17 KB. The first-try fixes, merging now while the build is repaired.
+
What is the main blocker?
A real-phone session. See the next section.
+
+ + +

5. The main blocker, and what to do this morning

+
+ + The field test: the phone's reading against a tape on the same wall + A wall with the electric meter. A yellow tape runs along the wall with marks at 5, 10, 15 and 20 feet. A phone stands in front. Blue dots show where the phone thinks the same marks are; the gap between dot and mark grows with distance. A label reads: the gap is the phone's scale error. + + + + + + + + + + + 5101520 ft + + + + + + blue dots: where the phone says the tape marks are + the gap is the scale error + + a current iPhone, no LiDAR + +
Nothing decides accuracy until this runs. Hunter's research handoff (PR #18) reaches the same conclusion.
+
+
+

Run the field test first, about an hour. You need a current non-LiDAR iPhone, the Measure Lab build, a 30 ft tape and a real meter wall. Follow experiments/evals/field/FIELD_SHEET.md: type the tape readings into the prepared templates, and one command turns them into a score and the phone's real scale error. A dry run recovered a planted 1.5% error exactly, so the scoring end is ready.

+

The overnight tracking numbers (finding 1) come from a 2023 iPhone with LiDAR and a dataset whose reference may itself be scaled to the phone's tracking. Most homeowners' phones lack LiDAR. The field test measures our own phone on our own wall, which is the only direct measurement.

+
+ + +

6. What the overnight tests found, and what each changed

+

All five ran on public real-world datasets with laser-measured ground truth. Dataset images are never in git. Each finding below ends with the line that answers "did this test do anything".

+ +
+

Finding 1. How far off is the phone's own tracking?

+
+
+ 2018 iPhone 6s (ADVIO) + 2023 iPhone 14 Pro Max (MARViN) + server's allowance +
+ + p90 position error in inches against feet walked, two phones and the server's allowance + At 3, 10, 20 and 30 feet walked: 2018 iPhone 6s 18.6, 55.7, 94.6, 132.6 inches. 2023 iPhone 14 Pro Max 4.0, 8.6, 13.4, 18.5 inches. Server's allowance 5.8, 19.2, 38.4, 57.6 inches. The 2023 phone stays under the allowance at every distance. The 2018 phone is over it at every distance. + + + + 040 in80 in120 in + 3 ft10 ft20 ft30 ftdistance walked + + + + + + + + + 2018 phone, 132.6 in + allowance, 57.6 in + 2023 phone, 18.5 in + + 3 ft walked: 2018 phone 18.6 in, 2023 phone 4.0 in, allowance 5.8 in + 10 ft walked: 2018 phone 55.7 in, 2023 phone 8.6 in, allowance 19.2 in + 20 ft walked: 2018 phone 94.6 in, 2023 phone 13.4 in, allowance 38.4 in + 30 ft walked: 2018 phone 132.6 in, 2023 phone 18.5 in, allowance 57.6 in + + +
Position error after walking, p90 (9 in 10 cases are better). A current iPhone's own tracking stays inside the server's allowance at every distance; the 2018 phone is far over it. The allowance line is the 0.16 ft per foot part of the server's error bar. Caveats: the MARViN reference may itself be scaled to the phone's tracking, and that phone has LiDAR, which most homeowners' phones lack.
+
Data table +
+ + + + + + + +
Walked2018 iPhone 6s (ADVIO), in2023 iPhone 14 Pro Max (MARViN), inServer's allowance, in
3 ft18.64.05.8
10 ft55.78.619.2
20 ft94.613.438.4
30 ft132.618.557.6
+
+
+

What it changed: the server's error allowance comes from this. The field test measures our own phone.

+
+ +
+

Finding 2. Can photos alone give 3D measurements?

+
+ + Error of learned depth from photos: flat walls and edges + A horizontal scale from 0 to 10 inches. Flat walls: 5.0 inches at p90, with a 95 percent interval from 3.8 to 6.8. Window and door edges: 8 inches or worse in every setting, drawn as a bar starting at 8 and running off the right edge. + + + + 0246810 inerror, p90 + flat wall + + + + 5.0 in (3.8 to 6.8) + edges + + + 8 in or worse, every setting + +
Learned depth models (MoGe-2, Depth Anything 3), rescaled with the phone's poses. Flat walls come out at about 5 in at best at a realistic phone error. Window and door edges, which are what clearances are measured from, come out at 8 in or worse in every setting.
+
+

What it changed: no photo-to-3D measurement pipeline. Every point a check depends on is an AR tap; learned depth only draws the 3D picture. This saved building the wrong thing.

+
+ +
+

Finding 3. Does the app's coverage map claim wall it never saw?

+
+
+ wall a photo saw + claimed, never seen + good wall thrown away +
+ + A 19.3 foot wall under three versions of the coverage map + Three bars, each 19.3 feet long. First, as claimed: 1.1 feet marked as seen that no photo saw, hidden behind scaffold footings. Second, after requiring views from different angles: the false 1.1 feet is gone but 6 to 8.5 feet of good wall is also thrown away. Third, after a depth test on LiDAR phones: the false part is removed cleanly and the rest is kept. + + as claimed + + + 1.1 ft never seen + views from different angles + + + + 6 to 8.5 ft of good wall lost too + depth test (LiDAR phones) + + + clean + +
Run on real photos of a laser-scanned building. The map claimed 1.1 ft of a 19.3 ft wall that no photo saw, hidden behind scaffold footings. Requiring views from different angles removed it only by also discarding 6 to 8.5 ft of good wall. A depth test, available on LiDAR phones, removed it cleanly. The hatched span is drawn at 8.5 ft, the larger end of the range.
+
+

What it changed: decision 7 (who checks what is hidden). A wrong claim in the app's code, "the server rechecks from the images", was corrected: the server never sees images.

+
+ +
+

Finding 4. Can the phone read the meter?

+
+ + Meter reading success on real meter photos + Four horizontal bars on a 0 to 100 percent scale. Read the full number: 71 of 73, 97 percent. Picked the right line: 21 of 75. Barcode held the number: 20 of 24 photos that had one. A tap list of three candidates contained it: 85 percent of held-out photos. + + + + 0255075100% + + full number readright line pickedbarcode had itin the tap list of 3 + + + + + + 71 of 73, 97% + 21 of 75 + 20 of 24 photos that had one + 85% of held-out photos + + +
Apple's on-device text recognition read the full meter number on 71 of 73 real meter photos, but could not tell which line on the plate was the number (right on 21 of 75). A barcode held the number on 20 of 24 photos that had one. A tap-to-confirm list of three candidates contained it on 85% of held-out photos.
+
+

What it changed: the app shows three candidates to tap, barcode match first, plus two photo checks that trigger a retake (blurry; number too small, "move closer").

+
+ +
+

Finding 5. Can a first-time homeowner finish a scan?

+
+ + Ground depth the app reported against what the checks need + A scale from 0 to 14 feet. The app reported ground seen out to 4 feet. The checks need 6 to 13 feet. The two do not overlap, so no scan could finish. + + + + 04812 ft + app reported + + 4 ft of ground seen + checks need + + 6 to 13 ft + +
The verification agent's friction audit: no, not before this morning's fixes. However well it was walked, the app said it had seen ground only 4 ft out while checks need 6 to 13 ft, never reported the space in front of or above the wall, and turned corners into "walk past this" requests.
+
+

What it changed: the capture contract, and the app fixes in section 4: the server publishes exactly what settles each check, and the app reports real ground depth, the walked path as clear space in front, and a tilt-up view for headroom.

+
+ +

Three pieces of overnight work changed nothing in the product: the panel-label search (too few open photos: 7 US panels against 40 needed), finer statistics on the modern-phone data, and the research notes.

+ + +

7. Decisions for this morning, recommendation first

+
    +
  1. Run the field test first

    Recommend: about an hour, before anything else.

    A current non-LiDAR iPhone, Measure Lab, a 30 ft tape, a real meter wall. Follow experiments/evals/field/FIELD_SHEET.md. One command turns tape readings into a score and the phone's real scale error.

  2. +
  3. How points get measured

    Recommend: AR taps, not depth maps.

    Finding 2: learned depth gets edges to 8 in or worse.

  4. +
  5. How far to walk

    Recommend: outward from the meter on both sides; stop when the nearest spot that fits is fully seen.

    The server prefers the shortest cable, so walking farther cannot find a better spot, and shorter walks drift less.

  6. +
  7. Base's real rules

    Recommend: load them into the private rules file now.

    Confirm with Base which rules the demo follows and which number on the meter plate they need (plates carry several). Otherwise the demo can never show a real "fits here".

  8. +
  9. Merge order

    Recommend: #11 server first, since the others depend on its schemas.

    Then #14 docs, #10 app, #13 verification, #12 evals, then #7, #4, #8, #6, #16, #17, #19.

  10. +
  11. Lane owners

    Recommend: name one owner per lane.

    Capture app, geometry, rules, output. Agents then know whose OK they need.

  12. +
  13. Who checks what is hidden behind things?

    Recommend: the app.

    Before the result, show the homeowner the photo of the chosen spot and ask if anything blocks the wall or ground there. On LiDAR phones, also run a depth test (finding 3).

  14. +
  15. What must the scan see, and what can we just ask?

    Recommend: pool and driveway become one yes/no question, labelled as the homeowner's answer.

    Instead of requiring 8 to 13 ft of photographed ground.

  16. +
+ + +

8. Asks: only a person can do these

+ + + +

9. Reference: what exists, and how to try it

+

Open pull requests. None of these is merged yet. The server (#11) is live.

+ + +

How to try it

+
+

Server health: house-scanning-server.vercel.app/health

+

App in the Simulator: build ios/HouseScan.xcodeproj and run with

+
-replay <folder> -autopilot -serverURL https://house-scanning-server.vercel.app
+

On a real iPhone: set your team in ios/Config/Local.xcconfig (see ios/README.md), plug in, run.

+
+ + + +
+ +