From 8dc50edfcd092fffc809feb4ff231ff749a6c8be Mon Sep 17 00:00:00 2001 From: zyang91 Date: Thu, 15 Oct 2026 15:43:21 -0400 Subject: [PATCH] Document Assignment 4 in the docs site Assignments, How grading works, Rubrics, the home page and Testing now cover Assignment 4: execution: mode: never in every mode, the isolated_functions hidden test that calls only the format_* functions in the network-less sandbox, scraping shown rather than loaded from CSV, and the 10/55/35 point table. API pages are generated for assignments/hw4.py, grader/function_sandbox.py and grader/function_runner.py. Search entries for section headings are now HTML-unescaped, so "Street Networks & Scraping" no longer shows as "&". Co-Authored-By: Claude Opus 5.5 --- docs/_pages/assignments.html | 78 +++++- docs/_pages/how-it-works.html | 7 + docs/_pages/index.html | 14 +- docs/_pages/rubrics.html | 37 ++- docs/_pages/testing.html | 2 +- docs/api/assignments.base.html | 50 ++-- docs/api/assignments.html | 3 + docs/api/assignments.hw1.html | 3 + docs/api/assignments.hw2.html | 3 + docs/api/assignments.hw3.html | 5 +- docs/api/assignments.hw4.html | 371 ++++++++++++++++++++++++++ docs/api/grader.answers.html | 3 + docs/api/grader.charts.html | 3 + docs/api/grader.discovery.html | 3 + docs/api/grader.executor.html | 38 ++- docs/api/grader.feedback.html | 3 + docs/api/grader.function_runner.html | 135 ++++++++++ docs/api/grader.function_sandbox.html | 221 +++++++++++++++ docs/api/grader.html | 3 + docs/api/grader.inspection.html | 3 + docs/api/grader.llm.html | 3 + docs/api/grader.models.html | 3 + docs/api/grader.notebook.html | 5 +- docs/api/grader.notebook_runner.html | 3 + docs/api/grader.probe_runtime.html | 5 +- docs/api/grader.results.html | 3 + docs/api/grader.rubric.html | 30 ++- docs/api/grader.scoring.html | 19 +- docs/api/grader.service.html | 31 ++- docs/api/grader.utils.html | 3 + docs/api/index.html | 8 +- docs/assets/search-index.js | 2 +- docs/assignments.html | 83 +++++- docs/build.py | 5 +- docs/cli.html | 3 + docs/docker.html | 3 + docs/extending.html | 3 + docs/getting-started.html | 3 + docs/how-it-works.html | 10 + docs/index.html | 19 +- docs/results.html | 3 + docs/rubrics.html | 42 ++- docs/ta-workflow.html | 3 + docs/testing.html | 5 +- 44 files changed, 1185 insertions(+), 97 deletions(-) create mode 100644 docs/api/assignments.hw4.html create mode 100644 docs/api/grader.function_runner.html create mode 100644 docs/api/grader.function_sandbox.html diff --git a/docs/_pages/assignments.html b/docs/_pages/assignments.html index 2f66f9d..4b7ce6d 100644 --- a/docs/_pages/assignments.html +++ b/docs/_pages/assignments.html @@ -1,5 +1,5 @@ - +

User guide

Assignments

@@ -9,6 +9,7 @@

Assignments

Assignment 1

The Donut Effect. No answer key, so each step is checked against the student's own data.

 Assignment 2

Exploratory Visualization. No data handed in, so the required elements are graded from saved outputs.

 Assignment 3

Evictions and the NDVI. One shared zip, so there is an answer key for the first time.

 + Assignment 4

Street Networks & Scraping. Everything is downloaded or scraped live, so the notebook is never run.
@@ -215,6 +216,79 @@

Where the 100 points go

-

Assignments 4–6

+

Assignment 4: Street Networks & Scraping

+

rubrics/hw4.yamlassignments/hw4.pyno datanever executed

+ +

Every step of Assignment 4 talks to a server that is not ours. Part 1 downloads the planning districts from ArcGIS Open Data and the street network from the OpenStreetMap Overpass API (osmnx), and Part 2 drives a browser through five pages of Craigslist with a sleep between each. Re-running a class of these notebooks would repeat every one of those requests once per student, which is exactly the load the assignment itself warns will get an IP banned.

+ +

So rubrics/hw4.yaml says execution: mode: never, and the service honours it in every execution mode, local included: no kernel, no notebook container, no browser, no data placement. See Execution. The grade comes from the file the student handed in, read two ways:

+ + + +

The one thing that does run: three pure functions

+ +

format_bedrooms, format_size and format_price turn "2br" into 2, and calling them is far better evidence than reading them. So they, and only they, are cut out of the notebook (grader.function_sandbox) and called with typical Craigslist strings in the Docker sandbox, under the same isolation flags as a notebook run, above all --network none:

+ + + +
+

In local mode the functions are not called

+

The machine has a network, so item 2.5 gets a provisional score from the source and goes to a TA. The sidebar says so before the run.

+
+ +

Scraping has to be shown, not loaded

+ +

A results table read back from a CSV proves nothing about a scraper. Item 2.6 wants the completed loop in the notebook and the log it printed when the student ran it, and the results table only counts when that log is there. A table loaded from a file is recorded as such and goes to a TA.

+ +

Reference values, loosely

+ +

There is no reference solution, and the street network and the listings both change. The only numbers used are the template's own: about 3,750 crashes in Center City (3,000–4,500 accepted), 120 listings a page, and a price per square foot around 1.5. Each only raises confidence. A number far outside its range sends the item to a person; it never costs a point on its own. As everywhere, anything short of full marks reaches a person.

+ +

Where the 100 points go

+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
ItemPoints
Notebook run and submitted with its outputs10
1.1Central planning district polygon4
1.2Drive network from the polygon4
1.3Network edges as a GeoDataFrame3
1.4PennDOT crash data loaded and combined4
1.5Crashes as a GeoDataFrame (latitude-as-x is named)4
1.6Crashes trimmed to Center City5
1.7Graph and crashes projected to EPSG:22725
1.8Nearest edge for each crash5
1.9Crash count per street5
1.10Crash counts merged onto the edges4
1.11Crash index5
1.12Histogram of the crash index3
1.13Interactive map of the crash index4
2.1Selenium driver opens Craigslist3
2.2BeautifulSoup from the page source2
2.3Listing elements extracted4
2.4Price, bedrooms, size and title of the first listing4
2.5format_* functions, called in the sandbox9
2.6Scraping loop completed and shown run7
2.7Histograms of price and price per square foot3
2.8Interactive Altair scatter of price vs size3
Total100
+ +

Execution and output completeness is 10 points, Part 1 is 55 and Part 2 is 35.

+ +

The demo class (python examples/make_hw4_submissions.py) has four submissions: complete, one with the usual mistakes and its Craigslist results loaded from a CSV, one that crashes in Part 1, and the untouched template. Their saved outputs are written in directly; nothing is executed, there or when grading. In Docker mode they score 100, 84, 26.5 and 3. In local mode the format_* item is not called and goes to a TA instead.

+ +
+ +

Assignments 5–6

Their student templates are in assignment_template/, but they are not graded yet. Each needs a rubric YAML and a grader class. See Adding an assignment.

diff --git a/docs/_pages/how-it-works.html b/docs/_pages/how-it-works.html index 101f06e..01dcba6 100644 --- a/docs/_pages/how-it-works.html +++ b/docs/_pages/how-it-works.html @@ -49,6 +49,11 @@

2. Execution

Each submission is copied into its own working directory. The notebook is instrumented with probe cells, and a subprocess runs it. By default that subprocess runs inside a container with no network, capped memory and CPU, a read-only root filesystem and no host environment variables. See Docker sandbox.

+
+

When a rubric says execution: mode: never

+

Assignment 4 scrapes Craigslist and downloads from OpenStreetMap and ArcGIS, so its rubric opts out of execution entirely. The service honours that in every execution mode, local included: no data is placed, no kernel or notebook container is started, and the execution stage is recorded as skipped. The grade comes from the submitted file's code and saved outputs. The only code that runs is a few named pure functions, cut out of the notebook and called in the network-less sandbox, and only in Docker mode. See Assignment 4 and grader.function_sandbox.

+
+

Errors do not stop grading. allow_errors is on, so the notebook runs to the end, the first failing cell and its traceback are recorded, and every check that can still run does. A failed notebook never becomes an automatic zero. The execution item itself awards partial credit for how far the notebook got: failing at the last cell is worth more than failing at the first.

3. Inspection without variable names

@@ -74,6 +79,8 @@

3. Inspection without variable names<

4. Hidden tests

+

Hidden tests normally run inside the student's own kernel, during the notebook run. Assignment 4's notebook is never run, so its hidden test is type: isolated_functions: the named functions are extracted from the notebook and called on their own in the sandbox. See Rubrics.

+

calculate_percent_increase(group_df) takes a DataFrame, so it can't be tested with literal arguments. Instead the probe reads the column names and date dtype of the student's own tidy frame, builds a synthetic group frame from interpolated anchor points using those names, and calls the function with it. The expected answer follows from the anchors, so it is known whatever data the student loaded.

The anchors deliberately imply a different answer for the first and last rows than for the two dates named in the assignment. That lets the grader identify the most common wrong implementation by name instead of just marking it wrong:

diff --git a/docs/_pages/index.html b/docs/_pages/index.html index 1e6f31a..a3eeb12 100644 --- a/docs/_pages/index.html +++ b/docs/_pages/index.html @@ -52,17 +52,17 @@

What it does

-

Three assignments, three ways to grade

+

Four assignments, four ways to grade

-

What the grader can check depends on the dataset. Each of the three assignments gives it a different situation:

+

What the grader can check depends on the dataset. Each of the four assignments gives it a different situation:

- + - - - - + + + +
Assignment 1: The Donut EffectAssignment 2: Exploratory VisualizationAssignment 3: Evictions and the NDVI
Assignment 1: The Donut EffectAssignment 2: Exploratory VisualizationAssignment 3: Evictions and the NDVIAssignment 4: Street Networks & Scraping
DatasetOne Zillow file, downloaded by each studentThe student's own choice, never handed inOne zip, distributed with the assignment
Answer key?No. Zillow revises the file monthlyNo. Every student uses a different datasetYes. Everyone runs the same five files
What is gradedWhether each step worked, checked against the student's own dataWhether the student can produce the required elementsWhether each result is correct
Evidence fromThe grading run's own executionThe outputs saved in the notebookBoth, combined
DatasetOne Zillow file, downloaded by each studentThe student's own choice, never handed inOne zip, distributed with the assignmentDownloaded and scraped live from OSM, ArcGIS and Craigslist
Answer key?No. Zillow revises the file monthlyNo. Every student uses a different datasetYes. Everyone runs the same five filesOnly the template's own rough figures ("about 3,750 crashes")
What is gradedWhether each step worked, checked against the student's own dataWhether the student can produce the required elementsWhether each result is correctWhether each step is written and shown run
Evidence fromThe grading run's own executionThe outputs saved in the notebookBoth, combinedThe code and saved outputs only. The notebook is never run.
diff --git a/docs/_pages/rubrics.html b/docs/_pages/rubrics.html index 2022f94..3c8cca4 100644 --- a/docs/_pages/rubrics.html +++ b/docs/_pages/rubrics.html @@ -13,12 +13,13 @@

Before grading a real class

  • Assignment 1: the center_city_zips list matches the one printed in the student template, and start_date/end_date match the dates in the instructions. Nothing else needs updating when the data changes.
  • Assignment 2: there is no answer key to check. hw2.yaml holds policy: how much a chart that never rendered is worth, what counts as a transformation, how many words make a discussion, and the ceiling on an unreviewed aesthetics score.
  • Assignment 3: every expected count and median was computed from assignment_template/assignment3-data.zip. If you hand out a different zip, they all change together. Re-run the reference solution and update the numbers, or widen the tolerances.
    • +
  • Assignment 4: there is no reference solution. The only numbers are the template's own rough figures (about 3,750 crashes, 120 listings a page), each with a wide tolerance, and they only raise confidence. Keep execution: mode: never, and settings.template_notebook pointing at the template handed out.

    • File layout

    -

    A rubric has six top-level blocks. Only assignment.id and at least one rubric item are required.

    +

    A rubric has seven top-level blocks. Only assignment.id and at least one rubric item are required.

    assignment:            # who this rubric is for
   id: hw1              # required; selects the grader class registered under this id
@@ -31,6 +32,7 @@ 
    File layout
    
 settings: {...}        # grading behaviour
 discovery: {...}       # how to find the notebook in a submission
 data: {...}            # data the instructor supplies
+execution: {...}       # whether submissions are run at all (optional)
 probe: {...}           # hints for the in-kernel probe
 rubric: [...]          # the graded items
    @@ -80,6 +82,15 @@

    data

    max_placements: 12 # cap on target paths per submission (HW3: 40) prefer_student_files: false # grade on the student's own file if they sent one +

    execution

    + +

    Optional. Leave it out and every submission is run in the execution mode the TA selected. Set mode: never and no submission of this assignment is run, in any mode, local included: no data is placed and no kernel or notebook container is started. The grade then comes from the submitted file's code and saved outputs, plus any isolated_functions hidden test. Assignment 4 uses it because its notebooks scrape Craigslist and download from OpenStreetMap and ArcGIS.

    + +
    execution:
+  mode: never      # grade the file as handed in; enforced in grader/service.py
    + +

    The check is Rubric.never_executes.

    +

    probe

    Hints the in-kernel probe uses to recognise columns, plus limits on how much it reports. Hints are matched against column names, case-insensitively.

    @@ -112,14 +123,14 @@

    Item types

    executionDid the notebook run? Handled by the shared check_notebook_execution. Partial credit is proportional to how far it got, scaled by partial_credit_ratio. structuralChecks on the source: imports, read calls, relative paths, functions defined. deterministicChecks on results: probe objects, saved answers, chart specs. This is the default type. - hidden_testA function is called inside the student's kernel with inputs from the rubric. Expected values never leave the host. + hidden_testA function is called with inputs from the rubric: inside the student's kernel, or on its own in the sandbox for isolated_functions. Expected values never leave the host. qualitativeNeeds a person's judgement, or an LLM if enabled. Always reviewed.

    Hidden-test config

    -

    Two call styles are supported. scalar_args passes literal arguments. group_frame asks the probe to build a DataFrame from interpolated anchor points, using the student's own column names.

    +

    Three call styles are supported. scalar_args passes literal arguments. group_frame asks the probe to build a DataFrame from interpolated anchor points, using the student's own column names. Both run inside the student's kernel. isolated_functions runs without the notebook.

    - id: percent_increase_function
   points: 20
@@ -149,6 +160,26 @@ 
    Hidden-test config
    
 
 
    For scalar_args, each case has args and optional kwargs and label.
    
 
+
    isolated_functions
    
+
+
    For an assignment that is never run (execution: mode: never). Only the top-level def of each named function is cut out of the notebook and called in the Docker sandbox with --network none; no other statement of the notebook goes along, and the submission is not mounted. Decorators and annotations are stripped, a function whose default argument is not a plain literal is refused, and only top-level imports of the modules in allowed_imports are sent with it. Each case's input is passed as the single argument; the expected values stay on the host.
    
+
+
    - id: format_functions
+  points: 9
+  type: hidden_test
+  config:
+    type: isolated_functions
+    points_per_function: 3
+    case_timeout_seconds: 5
+    allowed_imports: [re, string, math, decimal, numpy, pandas]   # the default set
+    functions:
+      format_price:
+        cases:
+          - {input: "$950", expected: 950}
+          - {input: "$1,500", expected: 1500}
    
+
+
    In local mode the functions are not called, since the machine has a network. The item is graded provisionally from the source and goes to a TA, and preflight warns before the run. The code is in grader.function_sandbox and grader.function_runner.
    
+
 
    Credit ratios
    
 
 
    Most item configs end in *_credit_ratio or *_penalty keys. These are the named near-misses: each is the fraction of the item's points awarded when the check recognises that particular mistake. They are policy, so tune them freely. The assignment grader reference documents which keys each check reads.
    
diff --git a/docs/_pages/testing.html b/docs/_pages/testing.html
index 5884112..7549c16 100644
--- a/docs/_pages/testing.html
+++ b/docs/_pages/testing.html
@@ -29,7 +29,7 @@ 
    What the suite covers
    
     test_discovery.pyFolder and Canvas layouts, notebook choice, ambiguous submissions
     test_data_supply.pyPlacing instructor data at the paths each notebook reads, path safety
     test_execution.py, test_docker_execution.pyLocal and sandboxed execution, timeouts, probe stripping
-    test_hw1.py, test_hw2.py, test_hw3.pyPer-assignment checks and named near-misses
+    test_hw1.py, test_hw2.py, test_hw3.py, test_hw4.pyPer-assignment checks and named near-misses
     test_hw2_integration.py, test_integration.pyEnd-to-end grading of demo classes
     test_answers.py, test_charts.pyReading answers and chart specs from saved outputs
     test_real_submission_patterns.pyFailure patterns seen in real classes, reproduced synthetically
diff --git a/docs/api/assignments.base.html b/docs/api/assignments.base.html
index c22e077..1fc4462 100644
--- a/docs/api/assignments.base.html
+++ b/docs/api/assignments.base.html
@@ -63,6 +63,8 @@
 
  • grader.executor
    • 
 
  • grader.notebook_runner
    • 
 
  • grader.probe_runtime
    • 
+
  • grader.function_sandbox
    • 
+
  • grader.function_runner
    • 
 
 
  • Evidence
    • 
   
   
   
    
     
    
-
    API reference · assignments/base.py · 297 lines
    
+
    API reference · assignments/base.py · 346 lines
    
 
    assignments.base
    
 
    Assignment grader base class and registry.
    
 
    An assignment module maps rubric ids to check methods. Every check receives the same context and returns one RubricItemResult carrying a score, a status, a confidence and the evidence behind it (design.md §24).
    
 
    Summary
    
 
    NameKindDescription
    GradingContextclassEverything a check may look at.
    AssignmentGraderclassBase grader: dispatches rubric ids to check_<rubric_id> methods.
    register()functionClass decorator: register an AssignmentGrader subclass under its assignment_id.
    get_grader()functionReturn the grader for a rubric, falling back to the generic base.
    registered_assignments()function
    
+
    Constants
    
+
    NameValueNotes
    ISOLATED_FUNCTIONS'isolated_functions'A hidden test that calls functions cut out of the notebook, in the sandbox.
    
 
    Classes
    
 
 
    
   
    class GradingContext
-    #[source]
    
+    #[source]
   
    
     
    dataclass
    
     
    Everything a check may look at.
    
@@ -102,7 +107,7 @@ 
    Classes
    
   
    Fields
    
   
    
-    
+    
    
     
    NameTypeDefaultNotes
    student_idstrrequired
    rubricRubricrequired
    analysisNotebookAnalysisrequired
    executionExecutionRecordrequired
    probedict[str, Any] | NoneNone
    workdirPath | NoneNone
    discovery_warningslist[str]field(default_factory=list)
    qualitative_graderAny | NoneNone
    settingsdict[str, Any]field(default_factory=dict)
    cachedict[str, Any]field(default_factory=dict)Scratch space for work shared between checks on one submission — e.g. HW2 reads the executed notebook once and grades seven items from it.
    student_idstrrequired
    rubricRubricrequired
    analysisNotebookAnalysisrequired
    executionExecutionRecordrequired
    probedict[str, Any] | NoneNone
    workdirPath | NoneNone
    discovery_warningslist[str]field(default_factory=list)
    qualitative_graderAny | NoneNone
    settingsdict[str, Any]field(default_factory=dict)
    cachedict[str, Any]field(default_factory=dict)Scratch space for work shared between checks on one submission — e.g. HW2 reads the executed notebook once and grades seven items from it.
    function_testsdict[str, Any]field(default_factory=dict)Results of functions run on their own in the sandbox, keyed by rubric id (AssignmentGrader.run_isolated_tests).
    
   
    
 
    
     
@@ -110,7 +115,7 @@ 
    Classes
    
 
 
    
   
    property probe_ok: bool
-    #[source]
    
+    #[source]
   
    No description.
    
 
    
   
@@ -118,7 +123,7 @@ 
    Classes
    
 
 
    
   
    class AssignmentGrader(rubric: Rubric)
-    #[source]
    
+    #[source]
   
    
     
    
     
    Base grader: dispatches rubric ids to check_<rubric_id> methods.
    
@@ -135,33 +140,46 @@ 
    Classes
    
 
 
    
   
    probe_config() -> dict[str, Any]
-    #[source]
    
+    #[source]
   
    Config handed to the in-kernel probe (hidden tests included).
    
 
    
 
 
    
   
    hidden_test_specs() -> list[dict[str, Any]]
-    #[source]
    
+    #[source]
   
    Only selectors and call inputs — expected values stay on the host.
    
 
    Two call styles: scalar_args passes literal arguments, group_frame asks the probe to build a DataFrame from interpolated anchor points and pass that. Either way the container is told what to call the function with, never what it should return.
    
 
    
 
+
    
+  
    isolated_test_items() -> list[RubricItem]
+    #[source]
    
+  
    No description.
    
+
    
+
+
    
+  
    run_isolated_tests(analysis: NotebookAnalysis, workdir: Path, config: ExecutionConfig) -> dict[str, Any]
+    #[source]
    
+  
    Call the functions a type: isolated_functions item names.
    
+
    The functions are cut out of the notebook and run on their own in the network-less sandbox (grader/function_sandbox.py); the rest of the notebook is never run. Only the inputs go into the container — the expected values stay in the rubric, on the host.
    
+
    
+
 
    
   
    grade(ctx: GradingContext) -> list[RubricItemResult]
-    #[source]
    
+    #[source]
   
    Run every rubric item's check_<id> method and return one result per item.
    
 
    A missing check returns manual_review; a check that raises returns manual_review with confidence 0 and the error in its evidence, so a broken check never loses a submission.
    
 
    
 
 
    
   
    result(item: RubricItem, score: float, status: str, confidence: float = 1.0, feedback: str = '', evidence: dict[str, Any] | None = None) -> RubricItemResult
-    #[source]
    
+    #[source]
   
    Build a RubricItemResult, rounding the score to the rubric's score_rounding increment and clamping it to [0, points].
    
 
    
 
 
    
   
    no_probe_result(item: RubricItem, ctx: GradingContext) -> RubricItemResult
-    #[source]
    
+    #[source]
   
    Shared response when the kernel never reported its namespace.
    
 
    
 
    Rubric checks 2
    
@@ -169,13 +187,13 @@ 
    Classes
    
 
 
    
   
    check_unimplemented(ctx: GradingContext, item: RubricItem) -> RubricItemResult
-    #[source]
    
+    #[source]
   
    No description.
    
 
    
 
 
    
   
    check_notebook_execution(ctx: GradingContext, item: RubricItem) -> RubricItemResult
-    #[source]
    
+    #[source]
   
    Shared execution check (design.md §19). Never an automatic zero.
    
 
    
   
    
@@ -184,19 +202,19 @@ 
    Functions
    
 
 
    
   
    register(cls: type[AssignmentGrader]) -> type[AssignmentGrader]
-    #[source]
    
+    #[source]
   
    Class decorator: register an AssignmentGrader subclass under its assignment_id.
    
 
    
 
 
    
   
    get_grader(rubric: Rubric) -> AssignmentGrader
-    #[source]
    
+    #[source]
   
    Return the grader for a rubric, falling back to the generic base.
    
 
    
 
 
    
   
    registered_assignments() -> list[str]
-    #[source]
    
+    #[source]
   
    No description.
    
 
    
     
    
@@ -205,7 +223,7 @@ 
    Functions
    
       
    MUSA Grader 0.1.0 · MUSA 5500, University of Pennsylvania
    
     
   
    
-  
+  
 
 
   
    
     
    
     
    No description.
    
@@ -132,46 +135,53 @@ 
    Functions
    
 
 
    
   
    build_probe_cells(probe_config: dict[str, Any]) -> list[Any]
-    #[source]
    
+    #[source]
   
    Return the notebook cells that dump the student's namespace to JSON.
    
 
    
 
 
    
   
    build_url_redirect_cell(redirects: dict[str, str]) -> Any
-    #[source]
    
+    #[source]
   
    A first cell that points data URLs at the supplied local file.
    
 
    The container has no network, by design. A student who loads the data straight from Zillow's URL is following a reasonable pattern, so a URL whose filename matches a file the instructor supplied is read from that local copy; any other URL still fails exactly as it would offline.
    
 
    
 
 
    
   
    instrument_notebook(notebook_path: str | Path, probe_config: dict[str, Any], output_path: str | Path, url_redirects: dict[str, str] | None = None) -> Path
-    #[source]
    
+    #[source]
   
    Append probe cells (and, if needed, prepend URL redirects) to a copy.
    
 
    
 
 
    
   
    docker_available() -> bool
-    #[source]
    
+    #[source]
   
    Whether a Docker daemon answers on this machine.
    
 
    
 
 
    
   
    docker_image_exists(image: str) -> bool
-    #[source]
    
+    #[source]
   
    Whether the sandbox image has been built locally.
    
 
    
 
 
    
   
    execute_submission(workdir: str | Path, probe_config: dict[str, Any], config: ExecutionConfig, run_subdir: str = '') -> tuple[ExecutionRecord, dict[str, Any] | None]
-    #[source]
    
+    #[source]
   
    Execute one prepared submission. Returns (execution record, probe payload).
    
 
    run_subdir is where inside workdir the notebook runs from (see prepare_workdir); artifacts always go to workdir/_musa_artifacts.
    
 
    
 
 
    
   
    strip_probe_cells(notebook_path: str | Path) -> Path
-    #[source]
    
+    #[source]
   
    Remove the injected cells so the saved notebook is the student's work.
    
+
    
+
+
    
+  
    sandbox_isolation_flags(config: ExecutionConfig) -> list[str]
+    #[source]
    
+  
    docker run flags every sandboxed run gets (design.md §18).
    
+
    Shared with grader/function_sandbox.py so the few functions Assignment 4 runs in isolation are held to exactly the same limits as a whole notebook — above all --network none.
    
 
    
     
     
    
@@ -179,7 +189,7 @@ 
    Functions
    
       
    MUSA Grader 0.1.0 · MUSA 5500, University of Pennsylvania
    
     
    
   
-  
+  
 
    
 
@@ -171,7 +181,7 @@ 
    Classes
    
 
 
    
   
    class RubricError(ValueError)
-    #[source]
    
+    #[source]
   
    
     
    
     
    Raised when a rubric file is missing or structurally invalid.
    
@@ -183,14 +193,14 @@ 
    Functions
    
 
 
    
   
    load_rubric(path: str | Path) -> Rubric
-    #[source]
    
+    #[source]
   
    Parse and validate a rubric YAML file.
    
 
    Raises RubricError when the file is missing or unparsable, assignment.id is absent, the rubric is empty, an item id is missing or duplicated, an item type is unknown, or the item points do not add up to total_points.
    
 
    
 
 
    
   
    available_rubrics(directory: str | Path = RUBRIC_DIR) -> dict[str, Path]
-    #[source]
    
+    #[source]
   
    Map assignment id -> rubric path for every parsable rubric on disk.
    
 
    
     
diff --git a/docs/api/grader.scoring.html b/docs/api/grader.scoring.html
index 197de67..f0724cb 100644
--- a/docs/api/grader.scoring.html
+++ b/docs/api/grader.scoring.html
@@ -63,6 +63,8 @@
 
  • grader.executor
    • 
 
  • grader.notebook_runner
    • 
 
  • grader.probe_runtime
    • 
+
  • grader.function_sandbox
    • 
+
  • grader.function_runner
    • 
 
 
  • Evidence
    • 
   
   
   
    
     
    
-
    API reference · grader/scoring.py · 228 lines
    
+
    API reference · grader/scoring.py · 231 lines
    
 
    grader.scoring
    
 
    Review policy and class-level aggregation (design.md §8, §25).
    
 
    Summary
    
@@ -94,7 +97,7 @@ 
    Functions
    
 
 
    
   
    apply_review_policy(result: SubmissionResult, confidence_threshold: float = DEFAULT_CONFIDENCE_THRESHOLD, review_below_full_marks: bool = True) -> SubmissionResult
-    #[source]
    
+    #[source]
   
    Decide what a human still has to look at.
    
 
    Flags are additive and explicit: a TA should be able to read why a submission reached the queue without opening the raw results.
    
 
    With review_below_full_marks on (the default), every deduction goes to the queue: an automated deduction is a claim about a student's work, and a person signs off on it before it becomes a grade. Full marks need no defence, so a clean submission passes straight through.
    
@@ -102,37 +105,37 @@ 
    Functions
    
 
 
    
   
    grade_letter(percentage: float | None) -> str
-    #[source]
    
+    #[source]
   
    No description.
    
 
    
 
 
    
   
    class_summary(results: Iterable[SubmissionResult]) -> dict[str, Any]
-    #[source]
    
+    #[source]
   
    Summary cards, distribution and common issues (design.md §8).
    
 
    
 
 
    
   
    grade_distribution(percentages: list[float], bin_size: int = 10) -> list[dict[str, Any]]
-    #[source]
    
+    #[source]
   
    No description.
    
 
    
 
 
    
   
    common_issues(results: Iterable[SubmissionResult], limit: int = 10) -> list[dict[str, Any]]
-    #[source]
    
+    #[source]
   
    Rank the deductions that affected the most students (design.md §8).
    
 
    
 
 
    
   
    rubric_breakdown(results: Iterable[SubmissionResult]) -> list[dict[str, Any]]
-    #[source]
    
+    #[source]
   
    Mean score per rubric item — shows where the class as a whole struggled.
    
 
    
 
 
    
   
    review_queue(results: Iterable[SubmissionResult]) -> list[dict[str, Any]]
-    #[source]
    
+    #[source]
   
    Unresolved review items, worst first (design.md §14).
    
 
    
     
    
diff --git a/docs/api/grader.service.html b/docs/api/grader.service.html
index 932d6ac..6aa1596 100644
--- a/docs/api/grader.service.html
+++ b/docs/api/grader.service.html
@@ -63,6 +63,8 @@
 
  • grader.executor
    • 
 
  • grader.notebook_runner
    • 
 
  • grader.probe_runtime
    • 
+
  • grader.function_sandbox
    • 
+
  • grader.function_runner
    • 
 
 
  • Evidence
    • 
   
   
   
    
     
    
-
    API reference · grader/service.py · 460 lines
    
+
    API reference · grader/service.py · 517 lines
    
 
    grader.service
    
 
    The grading service the UI talks to.
    
 
    Streamlit never calls the executor or the rubric engine directly (design.md §4): it hands a source and a config to this service and receives progress snapshots and a GradingSession back.
    
@@ -95,7 +98,7 @@ 
    Classes
    
 
 
    
   
    class GraderSettings
-    #[source]
    
+    #[source]
    
   
    
     
    dataclass
    
     
    Everything the Settings page can change (design.md §33).
    
@@ -112,14 +115,14 @@ 
    Classes
    
 
 
    
   
    execution_config(rubric: 'Rubric | None' = None) -> ExecutionConfig
-    #[source]
    
+    #[source]
   
    Execution limits, raised if the assignment declares it needs longer.
    
 
    A rubric's execution_timeout_seconds is assignment knowledge, not a preference: Assignment 3 reads a 24 MB GeoJSON, joins 34,108 points and masks a ten-band raster twice, and cutting it short times out the whole class at once. So it acts as a floor on the configured timeout, never as a cap — a TA who raises the limit keeps their value.
    
 
    
 
 
    
   
    to_dict() -> dict[str, Any]
-    #[source]
    
+    #[source]
   
    The settings as stored in grading_session.json.
    
 
    
   
    
@@ -127,7 +130,7 @@ 
    Classes
    
 
 
    
   
    class LoadedSubmissions
-    #[source]
    
+    #[source]
   
    
     
    dataclass
    
     
    What GradingService.load_submissions found: one SubmissionCandidate per student, plus where a Canvas ZIP was extracted to.
    
@@ -144,13 +147,13 @@ 
    Classes
    
 
 
    
   
    property ok_count: int
-    #[source]
    
+    #[source]
   
    No description.
    
 
    
 
 
    
   
    property warning_count: int
-    #[source]
    
+    #[source]
   
    No description.
    
 
    
   
    
@@ -158,7 +161,7 @@ 
    Classes
    
 
 
    
   
    class GradingService(rubric: Rubric, settings: GraderSettings | None = None, qualitative_grader: Any | None = None)
-    #[source]
    
+    #[source]
   
    
     
    
     
    Orchestrates a grading run: find submissions, place the data, execute each notebook, run the assignment's checks and apply the review policy.
    
@@ -169,39 +172,39 @@ 
    Classes
    
 
 
    
   
    classmethod from_rubric_path(path: str | Path, settings: GraderSettings | None = None, qualitative_grader: Any | None = None) -> 'GradingService'
-    #[source]
    
+    #[source]
   
    Load a rubric YAML and build a service for it.
    
 
    
 
 
    
   
    preflight() -> list[str]
-    #[source]
    
+    #[source]
   
    Problems that would make a grading run misleading rather than wrong.
    
 
    The big one: this assignment's students submit a notebook and nothing else, so without instructor-supplied data every notebook fails at read_csv and the whole class looks broken.
    
 
    
 
 
    
   
    load_submissions(source: str | Path, extract_to: str | Path | None = None) -> LoadedSubmissions
-    #[source]
    
+    #[source]
   
    Accept a folder or a Canvas ZIP (design.md §16).
    
 
    
 
 
    
   
    new_session(source: str) -> GradingSession
-    #[source]
    
+    #[source]
   
    An empty GradingSession for this rubric and these settings, with a fresh timestamped session id.
    
 
    
 
 
    
   
    run(candidates: Iterable[SubmissionCandidate], session: GradingSession | None = None, progress_callback: ProgressCallback | None = None, cancel_event: threading.Event | None = None, preserve_overrides: dict[str, Any] | None = None, workdir_root: str | Path | None = None) -> GradingSession
-    #[source]
    
+    #[source]
   
    Grade every candidate and return the saved session.
    
 
    Submissions are graded one at a time. progress_callback receives a GradingProgress snapshot at every stage (a callback that raises never stops grading); setting cancel_event stops after the current submission. preserve_overrides is a snapshot from GradingSession.collect_overrides to re-apply on top of the fresh automatic scores. Working directories are deleted afterwards unless GraderSettings.keep_workdirs is set.
    
 
    
 
 
    
   
    regrade(session: GradingSession, candidates: Iterable[SubmissionCandidate], preserve_overrides: bool = True, progress_callback: ProgressCallback | None = None, cancel_event: threading.Event | None = None) -> GradingSession
-    #[source]
    
+    #[source]
   
    Grade the same candidates again into the same session id and directory.
    
 
    With preserve_overrides (the default) every manual override is snapshotted first and restored on top of the new automatic scores.
    
 
    
diff --git a/docs/api/grader.utils.html b/docs/api/grader.utils.html
index 02cb21e..65b75de 100644
--- a/docs/api/grader.utils.html
+++ b/docs/api/grader.utils.html
@@ -63,6 +63,8 @@
 
  • grader.executor
    • 
 
  • grader.notebook_runner
    • 
 
  • grader.probe_runtime
    • 
+
  • grader.function_sandbox
    • 
+
  • grader.function_runner
    • 
 
 
  • Evidence
    • 
   
   
diff --git a/docs/api/index.html b/docs/api/index.html
index e31f84b..0eb8282 100644
--- a/docs/api/index.html
+++ b/docs/api/index.html
@@ -63,6 +63,8 @@
 
  • grader.executor
    • 
 
  • grader.notebook_runner
    • 
 
  • grader.probe_runtime
    • 
+
  • grader.function_sandbox
    • 
+
  • grader.function_runner
    • 
 
 
  • Evidence
    • 
   
   
@@ -119,9 +122,11 @@ 
    EngineExecution#
    
 
    
 grader.discoveryFind student submissions in a folder or a Canvas export ZIP (design.md §16-17).1 class · 8 functions
-grader.executorNotebook execution (design.md §18-19).2 classes · 7 functions
+grader.executorNotebook execution (design.md §18-19).2 classes · 8 functions
 grader.notebook_runnerStandalone notebook runner. Copied into the working directory and executed as a subprocess — locally with the host interpreter, or inside the grading container. It must not import anything from the grader package.1 function
 grader.probe_runtimeCode injected into the student's kernel after their notebook runs.module
+grader.function_sandboxRun a few named functions from a notebook, and nothing else, in the sandbox.2 classes · 4 functions
+grader.function_runnerStandalone runner for isolated function tests. Copied into a fresh, otherwise empty directory and executed inside the network-less grading container by grader/function_sandbox.py. It must not import anything from the grader package.1 class · 1 function
 
    
 
    Evidence#
    
 
    
@@ -139,6 +144,7 @@ 
    Assignmentsassignments.hw1Assignment 1 autograder — "The Donut Effect for Philadelphia ZIP Codes".1 class
 assignments.hw2Assignment 2 autograder — exploratory visualization with matplotlib, seaborn and altair.1 class
 assignments.hw3Assignment 3 autograder — evictions, code violations and the NDVI.1 class · 1 function
+assignments.hw4Assignment 4 autograder — street networks and web scraping.3 classes · 3 functions
 
    
     
     
    
diff --git a/docs/assets/search-index.js b/docs/assets/search-index.js
index 121b6c2..7ef0ffb 100644
--- a/docs/assets/search-index.js
+++ b/docs/assets/search-index.js
@@ -1 +1 @@
-window.MUSA_SEARCH_INDEX = [{"t":"Overview","u":"index.html","s":"Introduction","k":"page","d":"Semi-automatic grading for Jupyter Notebook assignments in MUSA 5500: Geospatial Data Science in Python."},{"t":"What it does","u":"index.html#what-it-does","s":"Overview","k":"section"},{"t":"Three assignments, three ways to grade","u":"index.html#three-assignments-three-ways-to-grade","s":"Overview","k":"section"},{"t":"Student data","u":"index.html#student-data","s":"Overview","k":"section"},{"t":"Architecture at a glance","u":"index.html#architecture-at-a-glance","s":"Overview","k":"section"},{"t":"Getting started","u":"getting-started.html","s":"Introduction","k":"page","d":"Install the grader, build the sandbox image, generate demo submissions and run your first grading session."},{"t":"Requirements","u":"getting-started.html#requirements","s":"Getting started","k":"section"},{"t":"Install","u":"getting-started.html#install","s":"Getting started","k":"section"},{"t":"Grade a demo class","u":"getting-started.html#grade-a-demo-class","s":"Getting started","k":"section"},{"t":"Assignment 1","u":"getting-started.html#assignment-1","s":"Getting started","k":"section"},{"t":"Assignment 2","u":"getting-started.html#assignment-2","s":"Getting started","k":"section"},{"t":"Assignment 3","u":"getting-started.html#assignment-3","s":"Getting started","k":"section"},{"t":"Start the web app","u":"getting-started.html#start-the-web-app","s":"Getting started","k":"section"},{"t":"Run everything in Docker","u":"getting-started.html#run-everything-in-docker","s":"Getting started","k":"section"},{"t":"Next steps","u":"getting-started.html#next-steps","s":"Getting started","k":"section"},{"t":"TA workflow","u":"ta-workflow.html","s":"User guide","k":"page","d":"A tour of the Streamlit app, from loading submissions to exporting grades."},{"t":"Start the app","u":"ta-workflow.html#start-the-app","s":"TA workflow","k":"section"},{"t":"The sidebar","u":"ta-workflow.html#the-sidebar","s":"TA workflow","k":"section"},{"t":"Assignment","u":"ta-workflow.html#assignment","s":"TA workflow","k":"section"},{"t":"Submissions","u":"ta-workflow.html#submissions","s":"TA workflow","k":"section"},{"t":"Assignment data","u":"ta-workflow.html#assignment-data","s":"TA workflow","k":"section"},{"t":"Pages","u":"ta-workflow.html#pages","s":"TA workflow","k":"section"},{"t":"The review queue","u":"ta-workflow.html#review-queue","s":"TA workflow","k":"section"},{"t":"Manual overrides","u":"ta-workflow.html#manual-overrides","s":"TA workflow","k":"section"},{"t":"Settings","u":"ta-workflow.html#settings","s":"TA workflow","k":"section"},{"t":"CLI reference","u":"cli.html","s":"User guide","k":"page","d":"The cli.py commands and flags for headless, scripted and CI grading runs."},{"t":"grade","u":"cli.html#grade","s":"CLI reference","k":"section"},{"t":"Example output","u":"cli.html#example-output","s":"CLI reference","k":"section"},{"t":"Examples","u":"cli.html#examples","s":"CLI reference","k":"section"},{"t":"sessions","u":"cli.html#sessions","s":"CLI reference","k":"section"},{"t":"rubrics","u":"cli.html#rubrics","s":"CLI reference","k":"section"},{"t":"Exit codes","u":"cli.html#exit-codes","s":"CLI reference","k":"section"},{"t":"How grading works","u":"how-it-works.html","s":"User guide","k":"page","d":"The grading pipeline: data supply, discovery, sandboxed execution, inspection, hidden tests, confidence and review."},{"t":"0. The data comes from you, not the students","u":"how-it-works.html#data-supply","s":"How grading works","k":"section"},{"t":"1. Discovery","u":"how-it-works.html#discovery","s":"How grading works","k":"section"},{"t":"2. Execution","u":"how-it-works.html#execution","s":"How grading works","k":"section"},{"t":"3. Inspection without variable names","u":"how-it-works.html#inspection-without-variable-names","s":"How grading works","k":"section"},{"t":"4. Hidden tests","u":"how-it-works.html#hidden-tests","s":"How grading works","k":"section"},{"t":"Two readings: saved outputs and the re-run","u":"how-it-works.html#two-readings","s":"How grading works","k":"section"},{"t":"5. Confidence and review","u":"how-it-works.html#confidence-and-review","s":"How grading works","k":"section"},{"t":"Item statuses","u":"how-it-works.html#statuses","s":"How grading works","k":"section"},{"t":"Scores and rounding","u":"how-it-works.html#scores","s":"How grading works","k":"section"},{"t":"Assignments","u":"assignments.html","s":"User guide","k":"page","d":"How Assignments 1, 2 and 3 are graded, and where each one's 100 points go."},{"t":"Assignment 1: The Donut Effect","u":"assignments.html#assignment-1-the-donut-effect","s":"Assignments","k":"section"},{"t":"Assignment 2: Exploratory Visualization","u":"assignments.html#assignment-2-exploratory-visualization","s":"Assignments","k":"section"},{"t":"Facts, graded automatically (70 points)","u":"assignments.html#facts-graded-automatically-70-points","s":"Assignments","k":"section"},{"t":"Judgement, graded by a person (30 points)","u":"assignments.html#judgement-graded-by-a-person-30-points","s":"Assignments","k":"section"},{"t":"Assignment 3: Evictions and the NDVI","u":"assignments.html#assignment-3-evictions-and-the-ndvi","s":"Assignments","k":"section"},{"t":"Plots are graded on the objects","u":"assignments.html#plots-are-graded-on-the-objects","s":"Assignments","k":"section"},{"t":"Named mistakes, not just wrong answers","u":"assignments.html#named-mistakes-not-just-wrong-answers","s":"Assignments","k":"section"},{"t":"Calibrated against a real class","u":"assignments.html#calibrated-against-a-real-class","s":"Assignments","k":"section"},{"t":"Where the 100 points go","u":"assignments.html#where-the-100-points-go","s":"Assignments","k":"section"},{"t":"Assignments 4–6","u":"assignments.html#assignments-4-6","s":"Assignments","k":"section"},{"t":"Rubrics","u":"rubrics.html","s":"User guide","k":"page","d":"The rubric YAML schema: assignment metadata, settings, discovery, data supply, probe hints and rubric items."},{"t":"Before grading a real class","u":"rubrics.html#before-grading-a-real-class","s":"Rubrics","k":"section"},{"t":"File layout","u":"rubrics.html#file-layout","s":"Rubrics","k":"section"},{"t":"assignment","u":"rubrics.html#assignment","s":"Rubrics","k":"section"},{"t":"settings","u":"rubrics.html#settings","s":"Rubrics","k":"section"},{"t":"discovery","u":"rubrics.html#discovery","s":"Rubrics","k":"section"},{"t":"data","u":"rubrics.html#data","s":"Rubrics","k":"section"},{"t":"probe","u":"rubrics.html#probe","s":"Rubrics","k":"section"},{"t":"rubric items","u":"rubrics.html#rubric-items","s":"Rubrics","k":"section"},{"t":"Item types","u":"rubrics.html#item-types","s":"Rubrics","k":"section"},{"t":"Hidden-test config","u":"rubrics.html#hidden-test-config","s":"Rubrics","k":"section"},{"t":"Credit ratios","u":"rubrics.html#credit-ratios","s":"Rubrics","k":"section"},{"t":"Validating a rubric","u":"rubrics.html#validating-a-rubric","s":"Rubrics","k":"section"},{"t":"Docker sandbox","u":"docker.html","s":"User guide","k":"page","d":"How student notebooks are isolated, what is in the sandbox image, and how to run the app itself in a container."},{"t":"Build the image","u":"docker.html#build-the-image","s":"Docker sandbox","k":"section"},{"t":"Isolation flags","u":"docker.html#isolation-flags","s":"Docker sandbox","k":"section"},{"t":"Memory and time","u":"docker.html#memory-and-time","s":"Docker sandbox","k":"section"},{"t":"What is in the image","u":"docker.html#what-is-in-the-image","s":"Docker sandbox","k":"section"},{"t":"Updating the pins","u":"docker.html#updating-the-pins","s":"Docker sandbox","k":"section"},{"t":"Running the app in a container","u":"docker.html#running-the-app-in-a-container","s":"Docker sandbox","k":"section"},{"t":"Results and exports","u":"results.html","s":"User guide","k":"page","d":"What a grading session writes to disk, the result JSON schema, and how to reload, regrade and export."},{"t":"Session directory","u":"results.html#session-directory","s":"Results and exports","k":"section"},{"t":"Result schema","u":"results.html#result-schema","s":"Results and exports","k":"section"},{"t":"Exports","u":"results.html#exports","s":"Results and exports","k":"section"},{"t":"Reloading and regrading","u":"results.html#reloading-and-regrading","s":"Results and exports","k":"section"},{"t":"Adding an assignment","u":"extending.html","s":"Developer guide","k":"page","d":"Write a rubric YAML and a grader class with check methods to grade a new assignment."},{"t":"1. Write the rubric","u":"extending.html#1-write-the-rubric","s":"Adding an assignment","k":"section"},{"t":"2. Write the grader class","u":"extending.html#2-write-the-grader-class","s":"Adding an assignment","k":"section"},{"t":"The check contract","u":"extending.html#the-check-contract","s":"Adding an assignment","k":"section"},{"t":"What a check can see","u":"extending.html#what-a-check-can-see","s":"Adding an assignment","k":"section"},{"t":"Hidden tests","u":"extending.html#hidden-tests","s":"Adding an assignment","k":"section"},{"t":"3. Add tests and demo submissions","u":"extending.html#3-add-tests-and-demo-submissions","s":"Adding an assignment","k":"section"},{"t":"Testing and CI","u":"testing.html","s":"Developer guide","k":"page","d":"Running the test suite, the markers it uses, what CI checks, and how to rebuild these docs."},{"t":"Running the tests","u":"testing.html#running-the-tests","s":"Testing and CI","k":"section"},{"t":"What the suite covers","u":"testing.html#what-the-suite-covers","s":"Testing and CI","k":"section"},{"t":"Continuous integration","u":"testing.html#continuous-integration","s":"Testing and CI","k":"section"},{"t":"Building these docs","u":"testing.html#building-these-docs","s":"Testing and CI","k":"section"},{"t":"API reference","u":"api/index.html","s":"API","k":"page"},{"t":"grader","u":"api/grader.html","s":"Module","k":"module","d":"MUSA Grader — grading engine."},{"t":"grader.service","u":"api/grader.service.html","s":"Module","k":"module","d":"The grading service the UI talks to."},{"t":"GraderSettings","u":"api/grader.service.html#grader.service.GraderSettings","s":"grader.service","k":"class","d":"Everything the Settings page can change (design.md §33)."},{"t":"GraderSettings.execution_config","u":"api/grader.service.html#grader.service.GraderSettings.execution_config","s":"grader.service","k":"method","d":"Execution limits, raised if the assignment declares it needs longer."},{"t":"GraderSettings.to_dict","u":"api/grader.service.html#grader.service.GraderSettings.to_dict","s":"grader.service","k":"method","d":"The settings as stored in grading_session.json."},{"t":"LoadedSubmissions","u":"api/grader.service.html#grader.service.LoadedSubmissions","s":"grader.service","k":"class","d":"What GradingService.load_submissions found: one SubmissionCandidate per student, plus where a Canvas ZIP was extracted to."},{"t":"LoadedSubmissions.ok_count","u":"api/grader.service.html#grader.service.LoadedSubmissions.ok_count","s":"grader.service","k":"property"},{"t":"LoadedSubmissions.warning_count","u":"api/grader.service.html#grader.service.LoadedSubmissions.warning_count","s":"grader.service","k":"property"},{"t":"GradingService","u":"api/grader.service.html#grader.service.GradingService","s":"grader.service","k":"class","d":"Orchestrates a grading run: find submissions, place the data, execute each notebook, run the assignment's checks and apply the review policy."},{"t":"GradingService.from_rubric_path","u":"api/grader.service.html#grader.service.GradingService.from_rubric_path","s":"grader.service","k":"classmethod","d":"Load a rubric YAML and build a service for it."},{"t":"GradingService.preflight","u":"api/grader.service.html#grader.service.GradingService.preflight","s":"grader.service","k":"method","d":"Problems that would make a grading run misleading rather than wrong."},{"t":"GradingService.load_submissions","u":"api/grader.service.html#grader.service.GradingService.load_submissions","s":"grader.service","k":"method","d":"Accept a folder or a Canvas ZIP (design.md §16)."},{"t":"GradingService.new_session","u":"api/grader.service.html#grader.service.GradingService.new_session","s":"grader.service","k":"method","d":"An empty GradingSession for this rubric and these settings, with a fresh timestamped session id."},{"t":"GradingService.run","u":"api/grader.service.html#grader.service.GradingService.run","s":"grader.service","k":"method","d":"Grade every candidate and return the saved session."},{"t":"GradingService.regrade","u":"api/grader.service.html#grader.service.GradingService.regrade","s":"grader.service","k":"method","d":"Grade the same candidates again into the same session id and directory."},{"t":"STAGE_LABELS","u":"api/grader.service.html#grader.service.STAGE_LABELS","s":"grader.service","k":"constant"},{"t":"grader.rubric","u":"api/grader.rubric.html","s":"Module","k":"module","d":"YAML rubric loading (design.md section 20)."},{"t":"RubricItem","u":"api/grader.rubric.html#grader.rubric.RubricItem","s":"grader.rubric","k":"class","d":"One graded line of a rubric, as declared in the YAML rubric: list."},{"t":"RubricItem.is_qualitative","u":"api/grader.rubric.html#grader.rubric.RubricItem.is_qualitative","s":"grader.rubric","k":"property"},{"t":"Rubric","u":"api/grader.rubric.html#grader.rubric.Rubric","s":"grader.rubric","k":"class","d":"A parsed rubric file. Every top-level YAML block is kept as-is, so assignment graders can read their own configuration from settings, probe, ``discovery"},{"t":"Rubric.item","u":"api/grader.rubric.html#grader.rubric.Rubric.item","s":"grader.rubric","k":"method"},{"t":"Rubric.requires_data","u":"api/grader.rubric.html#grader.rubric.Rubric.requires_data","s":"grader.rubric","k":"property"},{"t":"Rubric.offers_data","u":"api/grader.rubric.html#grader.rubric.Rubric.offers_data","s":"grader.rubric","k":"property","d":"Whether the sidebar should let a TA supply data for this assignment."},{"t":"Rubric.declared_points","u":"api/grader.rubric.html#grader.rubric.Rubric.declared_points","s":"grader.rubric","k":"property"},{"t":"Rubric.label","u":"api/grader.rubric.html#grader.rubric.Rubric.label","s":"grader.rubric","k":"method"},{"t":"RubricError","u":"api/grader.rubric.html#grader.rubric.RubricError","s":"grader.rubric","k":"class","d":"Raised when a rubric file is missing or structurally invalid."},{"t":"load_rubric()","u":"api/grader.rubric.html#grader.rubric.load_rubric","s":"grader.rubric","k":"function","d":"Parse and validate a rubric YAML file."},{"t":"available_rubrics()","u":"api/grader.rubric.html#grader.rubric.available_rubrics","s":"grader.rubric","k":"function","d":"Map assignment id -> rubric path for every parsable rubric on disk."},{"t":"RUBRIC_DIR","u":"api/grader.rubric.html#grader.rubric.RUBRIC_DIR","s":"grader.rubric","k":"constant"},{"t":"VALID_TYPES","u":"api/grader.rubric.html#grader.rubric.VALID_TYPES","s":"grader.rubric","k":"constant"},{"t":"grader.models","u":"api/grader.models.html","s":"Module","k":"module","d":"Structured result types shared by the engine, the store and the UI."},{"t":"ExecutionRecord","u":"api/grader.models.html#grader.models.ExecutionRecord","s":"grader.models","k":"class","d":"Outcome of running one student notebook (design.md section 19)."},{"t":"ExecutionRecord.status","u":"api/grader.models.html#grader.models.ExecutionRecord.status","s":"grader.models","k":"property"},{"t":"ExecutionRecord.to_dict","u":"api/grader.models.html#grader.models.ExecutionRecord.to_dict","s":"grader.models","k":"method"},{"t":"ExecutionRecord.from_dict","u":"api/grader.models.html#grader.models.ExecutionRecord.from_dict","s":"grader.models","k":"classmethod"},{"t":"RubricItemResult","u":"api/grader.models.html#grader.models.RubricItemResult","s":"grader.models","k":"class","d":"One graded rubric line (design.md section 24)."},{"t":"RubricItemResult.final_score","u":"api/grader.models.html#grader.models.RubricItemResult.final_score","s":"grader.models","k":"property","d":"The manual score when overridden, otherwise the automatic score."},{"t":"RubricItemResult.needs_review","u":"api/grader.models.html#grader.models.RubricItemResult.needs_review","s":"grader.models","k":"property"},{"t":"RubricItemResult.to_dict","u":"api/grader.models.html#grader.models.RubricItemResult.to_dict","s":"grader.models","k":"method"},{"t":"RubricItemResult.from_dict","u":"api/grader.models.html#grader.models.RubricItemResult.from_dict","s":"grader.models","k":"classmethod"},{"t":"SubmissionResult","u":"api/grader.models.html#grader.models.SubmissionResult","s":"grader.models","k":"class","d":"Everything known about one student's submission."},{"t":"SubmissionResult.max_score","u":"api/grader.models.html#grader.models.SubmissionResult.max_score","s":"grader.models","k":"property","d":"Sum of the items' possible points."},{"t":"SubmissionResult.automatic_total","u":"api/grader.models.html#grader.models.SubmissionResult.automatic_total","s":"grader.models","k":"property"},{"t":"SubmissionResult.total_score","u":"api/grader.models.html#grader.models.SubmissionResult.total_score","s":"grader.models","k":"property","d":"Sum of the items' final scores."},{"t":"SubmissionResult.percentage","u":"api/grader.models.html#grader.models.SubmissionResult.percentage","s":"grader.models","k":"property"},{"t":"SubmissionResult.confidence","u":"api/grader.models.html#grader.models.SubmissionResult.confidence","s":"grader.models","k":"property","d":"Lowest item confidence — the weakest link decides trust in the grade."},{"t":"SubmissionResult.has_manual_override","u":"api/grader.models.html#grader.models.SubmissionResult.has_manual_override","s":"grader.models","k":"property"},{"t":"SubmissionResult.needs_review","u":"api/grader.models.html#grader.models.SubmissionResult.needs_review","s":"grader.models","k":"property"},{"t":"SubmissionResult.open_review_reasons","u":"api/grader.models.html#grader.models.SubmissionResult.open_review_reasons","s":"grader.models","k":"method"},{"t":"SubmissionResult.item","u":"api/grader.models.html#grader.models.SubmissionResult.item","s":"grader.models","k":"method"},{"t":"SubmissionResult.to_dict","u":"api/grader.models.html#grader.models.SubmissionResult.to_dict","s":"grader.models","k":"method"},{"t":"SubmissionResult.from_dict","u":"api/grader.models.html#grader.models.SubmissionResult.from_dict","s":"grader.models","k":"classmethod"},{"t":"GradingProgress","u":"api/grader.models.html#grader.models.GradingProgress","s":"grader.models","k":"class","d":"Snapshot pushed to the UI while grading runs (design.md section 7)."},{"t":"GradingProgress.remaining","u":"api/grader.models.html#grader.models.GradingProgress.remaining","s":"grader.models","k":"property"},{"t":"GradingProgress.to_dict","u":"api/grader.models.html#grader.models.GradingProgress.to_dict","s":"grader.models","k":"method"},{"t":"status_icon()","u":"api/grader.models.html#grader.models.status_icon","s":"grader.models","k":"function"},{"t":"status_color()","u":"api/grader.models.html#grader.models.status_color","s":"grader.models","k":"function"},{"t":"STATUS_PASS","u":"api/grader.models.html#grader.models.STATUS_PASS","s":"grader.models","k":"constant"},{"t":"STATUS_PARTIAL","u":"api/grader.models.html#grader.models.STATUS_PARTIAL","s":"grader.models","k":"constant"},{"t":"STATUS_FAIL","u":"api/grader.models.html#grader.models.STATUS_FAIL","s":"grader.models","k":"constant"},{"t":"STATUS_WARNING","u":"api/grader.models.html#grader.models.STATUS_WARNING","s":"grader.models","k":"constant"},{"t":"STATUS_MANUAL_REVIEW","u":"api/grader.models.html#grader.models.STATUS_MANUAL_REVIEW","s":"grader.models","k":"constant"},{"t":"STATUS_EXECUTION_ERROR","u":"api/grader.models.html#grader.models.STATUS_EXECUTION_ERROR","s":"grader.models","k":"constant"},{"t":"STATUS_NOT_FOUND","u":"api/grader.models.html#grader.models.STATUS_NOT_FOUND","s":"grader.models","k":"constant"},{"t":"ALL_STATUSES","u":"api/grader.models.html#grader.models.ALL_STATUSES","s":"grader.models","k":"constant"},{"t":"STATUS_DISPLAY","u":"api/grader.models.html#grader.models.STATUS_DISPLAY","s":"grader.models","k":"constant"},{"t":"grader.results","u":"api/grader.results.html","s":"Module","k":"module","d":"Grading session persistence and exports (design.md §28-29)."},{"t":"GradingSession","u":"api/grader.results.html#grader.results.GradingSession","s":"grader.results","k":"class","d":"One grading run and everything decided in it, persisted under results//."},{"t":"GradingSession.ordered_results","u":"api/grader.results.html#grader.results.GradingSession.ordered_results","s":"grader.results","k":"method","d":"Results sorted by student id."},{"t":"GradingSession.upsert","u":"api/grader.results.html#grader.results.GradingSession.upsert","s":"grader.results","k":"method","d":"Add or replace one student's result."},{"t":"GradingSession.summary","u":"api/grader.results.html#grader.results.GradingSession.summary","s":"grader.results","k":"method","d":"The class summary (mean, median, distribution, common issues, rubric breakdown)."},{"t":"GradingSession.queue","u":"api/grader.results.html#grader.results.GradingSession.queue","s":"grader.results","k":"method","d":"The review queue: submissions that still need a person, most urgent first."},{"t":"GradingSession.student_ids","u":"api/grader.results.html#grader.results.GradingSession.student_ids","s":"grader.results","k":"method"},{"t":"GradingSession.neighbour","u":"api/grader.results.html#grader.results.GradingSession.neighbour","s":"grader.results","k":"method"},{"t":"GradingSession.set_override","u":"api/grader.results.html#grader.results.GradingSession.set_override","s":"grader.results","k":"method","d":"Override one rubric item's score. The automatic score is kept alongside it; pass enabled=False to remove an override. Returns False if the student or it"},{"t":"GradingSession.mark_reviewed","u":"api/grader.results.html#grader.results.GradingSession.mark_reviewed","s":"grader.results","k":"method","d":"Mark a submission as reviewed, which removes it from the queue."},{"t":"GradingSession.collect_overrides","u":"api/grader.results.html#grader.results.GradingSession.collect_overrides","s":"grader.results","k":"method","d":"Snapshot manual work so a regrade can restore it (design.md §31)."},{"t":"GradingSession.apply_overrides","u":"api/grader.results.html#grader.results.GradingSession.apply_overrides","s":"grader.results","k":"method"},{"t":"GradingSession.to_dict","u":"api/grader.results.html#grader.results.GradingSession.to_dict","s":"grader.results","k":"method"},{"t":"GradingSession.save","u":"api/grader.results.html#grader.results.GradingSession.save","s":"grader.results","k":"method","d":"Write the session and all its exports to disk; returns the session directory."},{"t":"GradingSession.store_executed_notebook","u":"api/grader.results.html#grader.results.GradingSession.store_executed_notebook","s":"grader.results","k":"method","d":"Keep the executed notebook alongside the results (design.md §28)."},{"t":"GradingSession.load","u":"api/grader.results.html#grader.results.GradingSession.load","s":"grader.results","k":"classmethod","d":"Read a session back from its directory."},{"t":"GradingSession.grades_csv","u":"api/grader.results.html#grader.results.GradingSession.grades_csv","s":"grader.results","k":"method","d":"grades.csv: one row per student."},{"t":"GradingSession.rubric_csv","u":"api/grader.results.html#grader.results.GradingSession.rubric_csv","s":"grader.results","k":"method","d":"Long-format per-item scores, useful for item analysis."},{"t":"GradingSession.flagged_csv","u":"api/grader.results.html#grader.results.GradingSession.flagged_csv","s":"grader.results","k":"method","d":"flagged_submissions.csv: the review queue."},{"t":"GradingSession.feedback_zip","u":"api/grader.results.html#grader.results.GradingSession.feedback_zip","s":"grader.results","k":"method","d":"Every student's markdown feedback, zipped."},{"t":"GradingSession.archive_zip","u":"api/grader.results.html#grader.results.GradingSession.archive_zip","s":"grader.results","k":"method","d":"The whole session directory, for handing over or archiving."},{"t":"list_sessions()","u":"api/grader.results.html#grader.results.list_sessions","s":"grader.results","k":"function","d":"Every saved session under results/, in reverse directory-name order (newest first for each assignment)."},{"t":"new_session_id()","u":"api/grader.results.html#grader.results.new_session_id","s":"grader.results","k":"function","d":"__."},{"t":"RESULTS_ROOT","u":"api/grader.results.html#grader.results.RESULTS_ROOT","s":"grader.results","k":"constant"},{"t":"SESSION_FILE","u":"api/grader.results.html#grader.results.SESSION_FILE","s":"grader.results","k":"constant"},{"t":"GRADES_CSV","u":"api/grader.results.html#grader.results.GRADES_CSV","s":"grader.results","k":"constant"},{"t":"SUMMARY_JSON","u":"api/grader.results.html#grader.results.SUMMARY_JSON","s":"grader.results","k":"constant"},{"t":"FLAGGED_CSV","u":"api/grader.results.html#grader.results.FLAGGED_CSV","s":"grader.results","k":"constant"},{"t":"FEEDBACK_DIR","u":"api/grader.results.html#grader.results.FEEDBACK_DIR","s":"grader.results","k":"constant"},{"t":"EXECUTED_DIR","u":"api/grader.results.html#grader.results.EXECUTED_DIR","s":"grader.results","k":"constant"},{"t":"RAW_DIR","u":"api/grader.results.html#grader.results.RAW_DIR","s":"grader.results","k":"constant"},{"t":"grader.scoring","u":"api/grader.scoring.html","s":"Module","k":"module","d":"Review policy and class-level aggregation (design.md §8, §25)."},{"t":"apply_review_policy()","u":"api/grader.scoring.html#grader.scoring.apply_review_policy","s":"grader.scoring","k":"function","d":"Decide what a human still has to look at."},{"t":"grade_letter()","u":"api/grader.scoring.html#grader.scoring.grade_letter","s":"grader.scoring","k":"function"},{"t":"class_summary()","u":"api/grader.scoring.html#grader.scoring.class_summary","s":"grader.scoring","k":"function","d":"Summary cards, distribution and common issues (design.md §8)."},{"t":"grade_distribution()","u":"api/grader.scoring.html#grader.scoring.grade_distribution","s":"grader.scoring","k":"function"},{"t":"common_issues()","u":"api/grader.scoring.html#grader.scoring.common_issues","s":"grader.scoring","k":"function","d":"Rank the deductions that affected the most students (design.md §8)."},{"t":"rubric_breakdown()","u":"api/grader.scoring.html#grader.scoring.rubric_breakdown","s":"grader.scoring","k":"function","d":"Mean score per rubric item — shows where the class as a whole struggled."},{"t":"review_queue()","u":"api/grader.scoring.html#grader.scoring.review_queue","s":"grader.scoring","k":"function","d":"Unresolved review items, worst first (design.md §14)."},{"t":"DEFAULT_CONFIDENCE_THRESHOLD","u":"api/grader.scoring.html#grader.scoring.DEFAULT_CONFIDENCE_THRESHOLD","s":"grader.scoring","k":"constant"},{"t":"grader.feedback","u":"api/grader.feedback.html","s":"Module","k":"module","d":"Student-facing markdown feedback (design.md §30)."},{"t":"build_feedback()","u":"api/grader.feedback.html#grader.feedback.build_feedback","s":"grader.feedback","k":"function","d":"Render one student's feedback as markdown."},{"t":"summary_line()","u":"api/grader.feedback.html#grader.feedback.summary_line","s":"grader.feedback","k":"function","d":"One-line status used in tables and logs."},{"t":"build_class_report()","u":"api/grader.feedback.html#grader.feedback.build_class_report","s":"grader.feedback","k":"function","d":"Markdown version of the class overview, for pasting into a course email."},{"t":"grader.discovery","u":"api/grader.discovery.html","s":"Module","k":"module","d":"Find student submissions in a folder or a Canvas export ZIP (design.md §16-17)."},{"t":"SubmissionCandidate","u":"api/grader.discovery.html#grader.discovery.SubmissionCandidate","s":"grader.discovery","k":"class","d":"One student's submission as found on disk."},{"t":"SubmissionCandidate.ok","u":"api/grader.discovery.html#grader.discovery.SubmissionCandidate.ok","s":"grader.discovery","k":"property","d":"True when a notebook was chosen for this submission."},{"t":"SubmissionCandidate.needs_review","u":"api/grader.discovery.html#grader.discovery.SubmissionCandidate.needs_review","s":"grader.discovery","k":"property"},{"t":"SubmissionCandidate.summary","u":"api/grader.discovery.html#grader.discovery.SubmissionCandidate.summary","s":"grader.discovery","k":"method"},{"t":"choose_notebook()","u":"api/grader.discovery.html#grader.discovery.choose_notebook","s":"grader.discovery","k":"function","d":"Return the most plausible notebook and any review reasons."},{"t":"find_notebooks()","u":"api/grader.discovery.html#grader.discovery.find_notebooks","s":"grader.discovery","k":"function"},{"t":"find_data_files()","u":"api/grader.discovery.html#grader.discovery.find_data_files","s":"grader.discovery","k":"function"},{"t":"extract_zip()","u":"api/grader.discovery.html#grader.discovery.extract_zip","s":"grader.discovery","k":"function","d":"Extract a Canvas export, refusing path traversal entries."},{"t":"parse_canvas_filename()","u":"api/grader.discovery.html#grader.discovery.parse_canvas_filename","s":"grader.discovery","k":"function","d":"Return (student_id, display_name, late) for a flat Canvas filename."},{"t":"discover_submissions()","u":"api/grader.discovery.html#grader.discovery.discover_submissions","s":"grader.discovery","k":"function","d":"Discover submissions under root, handling both supported layouts."},{"t":"place_shared_data()","u":"api/grader.discovery.html#grader.discovery.place_shared_data","s":"grader.discovery","k":"function","d":"Put instructor-supplied data where a notebook expects to find it."},{"t":"prepare_workdir()","u":"api/grader.discovery.html#grader.discovery.prepare_workdir","s":"grader.discovery","k":"function","d":"Copy one submission into an isolated working directory (design.md §17)."},{"t":"NOTEBOOK_SUFFIX","u":"api/grader.discovery.html#grader.discovery.NOTEBOOK_SUFFIX","s":"grader.discovery","k":"constant"},{"t":"DATA_SUFFIXES","u":"api/grader.discovery.html#grader.discovery.DATA_SUFFIXES","s":"grader.discovery","k":"constant"},{"t":"SKIP_DIRS","u":"api/grader.discovery.html#grader.discovery.SKIP_DIRS","s":"grader.discovery","k":"constant"},{"t":"CANVAS_RE","u":"api/grader.discovery.html#grader.discovery.CANVAS_RE","s":"grader.discovery","k":"constant"},{"t":"REASON_MULTIPLE_NOTEBOOKS","u":"api/grader.discovery.html#grader.discovery.REASON_MULTIPLE_NOTEBOOKS","s":"grader.discovery","k":"constant"},{"t":"REASON_NO_NOTEBOOK","u":"api/grader.discovery.html#grader.discovery.REASON_NO_NOTEBOOK","s":"grader.discovery","k":"constant"},{"t":"grader.executor","u":"api/grader.executor.html","s":"Module","k":"module","d":"Notebook execution (design.md §18-19)."},{"t":"ExecutionConfig","u":"api/grader.executor.html#grader.executor.ExecutionConfig","s":"grader.executor","k":"class","d":"Limits for one notebook execution, derived from GraderSettings."},{"t":"ExecutionConfig.hard_timeout","u":"api/grader.executor.html#grader.executor.ExecutionConfig.hard_timeout","s":"grader.executor","k":"property"},{"t":"ExecutionError","u":"api/grader.executor.html#grader.executor.ExecutionError","s":"grader.executor","k":"class"},{"t":"build_probe_cells()","u":"api/grader.executor.html#grader.executor.build_probe_cells","s":"grader.executor","k":"function","d":"Return the notebook cells that dump the student's namespace to JSON."},{"t":"build_url_redirect_cell()","u":"api/grader.executor.html#grader.executor.build_url_redirect_cell","s":"grader.executor","k":"function","d":"A first cell that points data URLs at the supplied local file."},{"t":"instrument_notebook()","u":"api/grader.executor.html#grader.executor.instrument_notebook","s":"grader.executor","k":"function","d":"Append probe cells (and, if needed, prepend URL redirects) to a copy."},{"t":"docker_available()","u":"api/grader.executor.html#grader.executor.docker_available","s":"grader.executor","k":"function","d":"Whether a Docker daemon answers on this machine."},{"t":"docker_image_exists()","u":"api/grader.executor.html#grader.executor.docker_image_exists","s":"grader.executor","k":"function","d":"Whether the sandbox image has been built locally."},{"t":"execute_submission()","u":"api/grader.executor.html#grader.executor.execute_submission","s":"grader.executor","k":"function","d":"Execute one prepared submission. Returns (execution record, probe payload)."},{"t":"strip_probe_cells()","u":"api/grader.executor.html#grader.executor.strip_probe_cells","s":"grader.executor","k":"function","d":"Remove the injected cells so the saved notebook is the student's work."},{"t":"MODE_DOCKER","u":"api/grader.executor.html#grader.executor.MODE_DOCKER","s":"grader.executor","k":"constant"},{"t":"MODE_LOCAL","u":"api/grader.executor.html#grader.executor.MODE_LOCAL","s":"grader.executor","k":"constant"},{"t":"PACKAGE_DIR","u":"api/grader.executor.html#grader.executor.PACKAGE_DIR","s":"grader.executor","k":"constant"},{"t":"RUNNER_SOURCE","u":"api/grader.executor.html#grader.executor.RUNNER_SOURCE","s":"grader.executor","k":"constant"},{"t":"PROBE_SOURCE","u":"api/grader.executor.html#grader.executor.PROBE_SOURCE","s":"grader.executor","k":"constant"},{"t":"INSTRUMENTED_NAME","u":"api/grader.executor.html#grader.executor.INSTRUMENTED_NAME","s":"grader.executor","k":"constant"},{"t":"RUNNER_NAME","u":"api/grader.executor.html#grader.executor.RUNNER_NAME","s":"grader.executor","k":"constant"},{"t":"ARTIFACTS_DIRNAME","u":"api/grader.executor.html#grader.executor.ARTIFACTS_DIRNAME","s":"grader.executor","k":"constant"},{"t":"DEFAULT_IMAGE","u":"api/grader.executor.html#grader.executor.DEFAULT_IMAGE","s":"grader.executor","k":"constant"},{"t":"URL_REDIRECT_TEMPLATE","u":"api/grader.executor.html#grader.executor.URL_REDIRECT_TEMPLATE","s":"grader.executor","k":"constant"},{"t":"grader.notebook_runner","u":"api/grader.notebook_runner.html","s":"Module","k":"module","d":"Standalone notebook runner. Copied into the working directory and executed as a subprocess — locally with the host interpreter, or inside the grading container."},{"t":"main()","u":"api/grader.notebook_runner.html#grader.notebook_runner.main","s":"grader.notebook_runner","k":"function"},{"t":"grader.probe_runtime","u":"api/grader.probe_runtime.html","s":"Module","k":"module","d":"Code injected into the student's kernel after their notebook runs."},{"t":"MUSA_PROBE_MARKER","u":"api/grader.probe_runtime.html#grader.probe_runtime.MUSA_PROBE_MARKER","s":"grader.probe_runtime","k":"constant"},{"t":"grader.notebook","u":"api/grader.notebook.html","s":"Module","k":"module","d":"Static analysis of a student notebook (design.md §23, structural grading)."},{"t":"NotebookAnalysis","u":"api/grader.notebook.html#grader.notebook.NotebookAnalysis","s":"grader.notebook","k":"class","d":"Static facts about a notebook."},{"t":"NotebookAnalysis.function_names","u":"api/grader.notebook.html#grader.notebook.NotebookAnalysis.function_names","s":"grader.notebook","k":"method"},{"t":"NotebookAnalysis.to_dict","u":"api/grader.notebook.html#grader.notebook.NotebookAnalysis.to_dict","s":"grader.notebook","k":"method"},{"t":"empty_analysis()","u":"api/grader.notebook.html#grader.notebook.empty_analysis","s":"grader.notebook","k":"function","d":"Placeholder used when no notebook was submitted at all."},{"t":"read_notebook()","u":"api/grader.notebook.html#grader.notebook.read_notebook","s":"grader.notebook","k":"function"},{"t":"analyze_notebook()","u":"api/grader.notebook.html#grader.notebook.analyze_notebook","s":"grader.notebook","k":"function"},{"t":"extract_markdown_responses()","u":"api/grader.notebook.html#grader.notebook.extract_markdown_responses","s":"grader.notebook","k":"function","d":"Find the most substantial free-text markdown answer in the notebook."},{"t":"render_cells()","u":"api/grader.notebook.html#grader.notebook.render_cells","s":"grader.notebook","k":"function","d":"Flatten a notebook into a read-only structure for the UI (design.md §13)."},{"t":"ABSOLUTE_PATH_RE","u":"api/grader.notebook.html#grader.notebook.ABSOLUTE_PATH_RE","s":"grader.notebook","k":"constant"},{"t":"DATA_SUFFIXES","u":"api/grader.notebook.html#grader.notebook.DATA_SUFFIXES","s":"grader.notebook","k":"constant"},{"t":"READ_FUNCS","u":"api/grader.notebook.html#grader.notebook.READ_FUNCS","s":"grader.notebook","k":"constant"},{"t":"PLOT_HINTS","u":"api/grader.notebook.html#grader.notebook.PLOT_HINTS","s":"grader.notebook","k":"constant"},{"t":"IPYTHON_LINE_PREFIXES","u":"api/grader.notebook.html#grader.notebook.IPYTHON_LINE_PREFIXES","s":"grader.notebook","k":"constant"},{"t":"IMPORT_RE","u":"api/grader.notebook.html#grader.notebook.IMPORT_RE","s":"grader.notebook","k":"constant"},{"t":"READ_CALL_RE","u":"api/grader.notebook.html#grader.notebook.READ_CALL_RE","s":"grader.notebook","k":"constant"},{"t":"STRING_LITERAL_RE","u":"api/grader.notebook.html#grader.notebook.STRING_LITERAL_RE","s":"grader.notebook","k":"constant"},{"t":"INSTRUCTION_RE","u":"api/grader.notebook.html#grader.notebook.INSTRUCTION_RE","s":"grader.notebook","k":"constant"},{"t":"grader.inspection","u":"api/grader.inspection.html","s":"Module","k":"module","d":"Host-side reasoning over the probe payload."},{"t":"confidence_for_candidates()","u":"api/grader.inspection.html#grader.inspection.confidence_for_candidates","s":"grader.inspection","k":"function"},{"t":"dataframes()","u":"api/grader.inspection.html#grader.inspection.dataframes","s":"grader.inspection","k":"function"},{"t":"scalars()","u":"api/grader.inspection.html#grader.inspection.scalars","s":"grader.inspection","k":"function"},{"t":"collections()","u":"api/grader.inspection.html#grader.inspection.collections","s":"grader.inspection","k":"function"},{"t":"hidden_test()","u":"api/grader.inspection.html#grader.inspection.hidden_test","s":"grader.inspection","k":"function"},{"t":"match_column()","u":"api/grader.inspection.html#grader.inspection.match_column","s":"grader.inspection","k":"function","d":"Best column in profile matching any of hints (substring, case-insensitive)."},{"t":"has_datelike_column()","u":"api/grader.inspection.html#grader.inspection.has_datelike_column","s":"grader.inspection","k":"function","d":"A date role can be satisfied by a column name hint or a datetime dtype."},{"t":"numeric_columns()","u":"api/grader.inspection.html#grader.inspection.numeric_columns","s":"grader.inspection","k":"function"},{"t":"zips_in_profile()","u":"api/grader.inspection.html#grader.inspection.zips_in_profile","s":"grader.inspection","k":"function","d":"All ZIP-looking values visible in a dataframe profile."},{"t":"zips_in_collection()","u":"api/grader.inspection.html#grader.inspection.zips_in_collection","s":"grader.inspection","k":"function"},{"t":"find_dataframe_candidates()","u":"api/grader.inspection.html#grader.inspection.find_dataframe_candidates","s":"grader.inspection","k":"function","d":"Return profiles satisfying predicate, best first."},{"t":"find_scalar_close_to()","u":"api/grader.inspection.html#grader.inspection.find_scalar_close_to","s":"grader.inspection","k":"function","d":"Scalars in the namespace within tolerance of the expected answer."},{"t":"describe_dataframe()","u":"api/grader.inspection.html#grader.inspection.describe_dataframe","s":"grader.inspection","k":"function","d":"Compact, TA-readable description used as rubric evidence (design.md §12)."},{"t":"CONFIDENCE_UNIQUE","u":"api/grader.inspection.html#grader.inspection.CONFIDENCE_UNIQUE","s":"grader.inspection","k":"constant"},{"t":"CONFIDENCE_TWO","u":"api/grader.inspection.html#grader.inspection.CONFIDENCE_TWO","s":"grader.inspection","k":"constant"},{"t":"CONFIDENCE_MANY","u":"api/grader.inspection.html#grader.inspection.CONFIDENCE_MANY","s":"grader.inspection","k":"constant"},{"t":"CONFIDENCE_NONE","u":"api/grader.inspection.html#grader.inspection.CONFIDENCE_NONE","s":"grader.inspection","k":"constant"},{"t":"grader.answers","u":"api/grader.answers.html","s":"Module","k":"module","d":"Answers read out of a notebook's own saved outputs (design.md §23, §25)."},{"t":"HoloviewsRepr","u":"api/grader.answers.html#grader.answers.HoloviewsRepr","s":"grader.answers","k":"class","d":"One holoviews object, as its own repr describes it."},{"t":"HoloviewsRepr.is_widget","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.is_widget","s":"grader.answers","k":"property"},{"t":"HoloviewsRepr.is_lazy","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.is_lazy","s":"grader.answers","k":"property","d":"A DynamicMap: the widget exists but no frame was ever computed."},{"t":"HoloviewsRepr.is_layout","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.is_layout","s":"grader.answers","k":"property"},{"t":"HoloviewsRepr.element_types","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.element_types","s":"grader.answers","k":"property"},{"t":"HoloviewsRepr.summary","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.summary","s":"grader.answers","k":"method"},{"t":"CellAnswers","u":"api/grader.answers.html#grader.answers.CellAnswers","s":"grader.answers","k":"class","d":"Everything one code cell's saved output states."},{"t":"CellAnswers.has_output","u":"api/grader.answers.html#grader.answers.CellAnswers.has_output","s":"grader.answers","k":"property"},{"t":"CellAnswers.has_answer","u":"api/grader.answers.html#grader.answers.CellAnswers.has_answer","s":"grader.answers","k":"property","d":"Output that states something, as opposed to a traceback."},{"t":"NotebookAnswers","u":"api/grader.answers.html#grader.answers.NotebookAnswers","s":"grader.answers","k":"class","d":"Every answer a notebook shows, with where each one was seen."},{"t":"NotebookAnswers.shows","u":"api/grader.answers.html#grader.answers.NotebookAnswers.shows","s":"grader.answers","k":"method","d":"The first cell that printed or returned this number."},{"t":"NotebookAnswers.shows_all","u":"api/grader.answers.html#grader.answers.NotebookAnswers.shows_all","s":"grader.answers","k":"method"},{"t":"NotebookAnswers.shows_together","u":"api/grader.answers.html#grader.answers.NotebookAnswers.shows_together","s":"grader.answers","k":"method","d":"A single cell that shows all of these numbers at once."},{"t":"NotebookAnswers.shows_rows","u":"api/grader.answers.html#grader.answers.NotebookAnswers.shows_rows","s":"grader.answers","k":"method","d":"A frame of this many rows, from a repr footer, .shape or .info()."},{"t":"NotebookAnswers.frames_with_columns","u":"api/grader.answers.html#grader.answers.NotebookAnswers.frames_with_columns","s":"grader.answers","k":"method"},{"t":"NotebookAnswers.describes","u":"api/grader.answers.html#grader.answers.NotebookAnswers.describes","s":"grader.answers","k":"method"},{"t":"NotebookAnswers.charts","u":"api/grader.answers.html#grader.answers.NotebookAnswers.charts","s":"grader.answers","k":"method"},{"t":"NotebookAnswers.figures","u":"api/grader.answers.html#grader.answers.NotebookAnswers.figures","s":"grader.answers","k":"property"},{"t":"NotebookAnswers.error_cells","u":"api/grader.answers.html#grader.answers.NotebookAnswers.error_cells","s":"grader.answers","k":"property"},{"t":"NotebookAnswers.cells_with_output","u":"api/grader.answers.html#grader.answers.NotebookAnswers.cells_with_output","s":"grader.answers","k":"property"},{"t":"NotebookAnswers.cells_with_answers","u":"api/grader.answers.html#grader.answers.NotebookAnswers.cells_with_answers","s":"grader.answers","k":"property"},{"t":"NotebookAnswers.note","u":"api/grader.answers.html#grader.answers.NotebookAnswers.note","s":"grader.answers","k":"method"},{"t":"read_answers()","u":"api/grader.answers.html#grader.answers.read_answers","s":"grader.answers","k":"function","d":"Harvest one notebook's saved outputs."},{"t":"collect_answers()","u":"api/grader.answers.html#grader.answers.collect_answers","s":"grader.answers","k":"function","d":"Answers from both notebooks, merged cell by cell."},{"t":"SOURCE_EXECUTED","u":"api/grader.answers.html#grader.answers.SOURCE_EXECUTED","s":"grader.answers","k":"constant"},{"t":"SOURCE_SUBMITTED","u":"api/grader.answers.html#grader.answers.SOURCE_SUBMITTED","s":"grader.answers","k":"constant"},{"t":"SOURCE_MERGED","u":"api/grader.answers.html#grader.answers.SOURCE_MERGED","s":"grader.answers","k":"constant"},{"t":"SOURCE_NONE","u":"api/grader.answers.html#grader.answers.SOURCE_NONE","s":"grader.answers","k":"constant"},{"t":"MAX_SCALAR_CHARS","u":"api/grader.answers.html#grader.answers.MAX_SCALAR_CHARS","s":"grader.answers","k":"constant"},{"t":"NUMBER_RE","u":"api/grader.answers.html#grader.answers.NUMBER_RE","s":"grader.answers","k":"constant"},{"t":"SHAPE_TEXT_RE","u":"api/grader.answers.html#grader.answers.SHAPE_TEXT_RE","s":"grader.answers","k":"constant"},{"t":"SHAPE_HTML_RE","u":"api/grader.answers.html#grader.answers.SHAPE_HTML_RE","s":"grader.answers","k":"constant"},{"t":"ENTRIES_RE","u":"api/grader.answers.html#grader.answers.ENTRIES_RE","s":"grader.answers","k":"constant"},{"t":"INFO_COLUMN_RE","u":"api/grader.answers.html#grader.answers.INFO_COLUMN_RE","s":"grader.answers","k":"constant"},{"t":"DESCRIBE_RE","u":"api/grader.answers.html#grader.answers.DESCRIBE_RE","s":"grader.answers","k":"constant"},{"t":"FIGURE_RE","u":"api/grader.answers.html#grader.answers.FIGURE_RE","s":"grader.answers","k":"constant"},{"t":"HTML_TABLE_RE","u":"api/grader.answers.html#grader.answers.HTML_TABLE_RE","s":"grader.answers","k":"constant"},{"t":"HTML_HEADER_RE","u":"api/grader.answers.html#grader.answers.HTML_HEADER_RE","s":"grader.answers","k":"constant"},{"t":"HOLOVIEWS_LINE_RE","u":"api/grader.answers.html#grader.answers.HOLOVIEWS_LINE_RE","s":"grader.answers","k":"constant"},{"t":"WIDGET_RE","u":"api/grader.answers.html#grader.answers.WIDGET_RE","s":"grader.answers","k":"constant"},{"t":"QUOTED_RE","u":"api/grader.answers.html#grader.answers.QUOTED_RE","s":"grader.answers","k":"constant"},{"t":"NDARRAY_RE","u":"api/grader.answers.html#grader.answers.NDARRAY_RE","s":"grader.answers","k":"constant"},{"t":"STRUCT_CODES","u":"api/grader.answers.html#grader.answers.STRUCT_CODES","s":"grader.answers","k":"constant"},{"t":"IMAGE_MIMES","u":"api/grader.answers.html#grader.answers.IMAGE_MIMES","s":"grader.answers","k":"constant"},{"t":"WIDGET_KINDS","u":"api/grader.answers.html#grader.answers.WIDGET_KINDS","s":"grader.answers","k":"constant"},{"t":"LAZY_KINDS","u":"api/grader.answers.html#grader.answers.LAZY_KINDS","s":"grader.answers","k":"constant"},{"t":"LAYOUT_KINDS","u":"api/grader.answers.html#grader.answers.LAYOUT_KINDS","s":"grader.answers","k":"constant"},{"t":"grader.charts","u":"api/grader.charts.html","s":"Module","k":"module","d":"Chart evidence for visualization assignments (design.md §23, §25)."},{"t":"SpecFacts","u":"api/grader.charts.html#grader.charts.SpecFacts","s":"grader.charts","k":"class","d":"What one Vega-Lite specification says the chart does."},{"t":"SpecFacts.brushes","u":"api/grader.charts.html#grader.charts.SpecFacts.brushes","s":"grader.charts","k":"property","d":"Interval selections a viewer can drag — pan/zoom bindings excluded."},{"t":"SpecFacts.has_brush","u":"api/grader.charts.html#grader.charts.SpecFacts.has_brush","s":"grader.charts","k":"property"},{"t":"SpecFacts.has_transformation","u":"api/grader.charts.html#grader.charts.SpecFacts.has_transformation","s":"grader.charts","k":"property"},{"t":"SpecFacts.is_multi_view","u":"api/grader.charts.html#grader.charts.SpecFacts.is_multi_view","s":"grader.charts","k":"property"},{"t":"SpecFacts.cross_filter_params","u":"api/grader.charts.html#grader.charts.SpecFacts.cross_filter_params","s":"grader.charts","k":"property","d":"Selections that a *different* view filters on — a real cross-filter."},{"t":"SpecFacts.has_cross_filter","u":"api/grader.charts.html#grader.charts.SpecFacts.has_cross_filter","s":"grader.charts","k":"property"},{"t":"SpecFacts.to_dict","u":"api/grader.charts.html#grader.charts.SpecFacts.to_dict","s":"grader.charts","k":"method"},{"t":"ChartRecord","u":"api/grader.charts.html#grader.charts.ChartRecord","s":"grader.charts","k":"class","d":"One chart, as drawn by one cell."},{"t":"ChartRecord.ok","u":"api/grader.charts.html#grader.charts.ChartRecord.ok","s":"grader.charts","k":"property","d":"Drew something, and the cell that drew it did not fail."},{"t":"ChartRecord.has_discussion","u":"api/grader.charts.html#grader.charts.ChartRecord.has_discussion","s":"grader.charts","k":"property"},{"t":"ChartRecord.summary","u":"api/grader.charts.html#grader.charts.ChartRecord.summary","s":"grader.charts","k":"method","d":"Compact, TA-readable evidence (design.md §12)."},{"t":"ChartEvidence","u":"api/grader.charts.html#grader.charts.ChartEvidence","s":"grader.charts","k":"class","d":"Every chart found in a submission, in the order the cells appear."},{"t":"ChartEvidence.from_execution","u":"api/grader.charts.html#grader.charts.ChartEvidence.from_execution","s":"grader.charts","k":"property","d":"Every chart that came out was produced by this grading run."},{"t":"ChartEvidence.verified","u":"api/grader.charts.html#grader.charts.ChartEvidence.verified","s":"grader.charts","k":"method","d":"Were these particular charts produced here, rather than by the student?"},{"t":"ChartEvidence.by_library","u":"api/grader.charts.html#grader.charts.ChartEvidence.by_library","s":"grader.charts","k":"method"},{"t":"ChartEvidence.rendered","u":"api/grader.charts.html#grader.charts.ChartEvidence.rendered","s":"grader.charts","k":"method"},{"t":"ChartEvidence.summary","u":"api/grader.charts.html#grader.charts.ChartEvidence.summary","s":"grader.charts","k":"method"},{"t":"libraries_in_source()","u":"api/grader.charts.html#grader.charts.libraries_in_source","s":"grader.charts","k":"function","d":"Which plotting libraries this cell draws with, most specific first."},{"t":"extract_specs()","u":"api/grader.charts.html#grader.charts.extract_specs","s":"grader.charts","k":"function","d":"Vega-Lite specifications carried by one notebook output."},{"t":"strip_spec_data()","u":"api/grader.charts.html#grader.charts.strip_spec_data","s":"grader.charts","k":"function","d":"Drop inlined rows so a spec can be stored as rubric evidence."},{"t":"analyze_spec()","u":"api/grader.charts.html#grader.charts.analyze_spec","s":"grader.charts","k":"function","d":"Read one Vega-Lite spec, at any level of composition."},{"t":"merge_facts()","u":"api/grader.charts.html#grader.charts.merge_facts","s":"grader.charts","k":"function","d":"One view of a cell that displayed more than one chart."},{"t":"aesthetic_signals()","u":"api/grader.charts.html#grader.charts.aesthetic_signals","s":"grader.charts","k":"function","d":"Presence of the labelling and colour choices a TA would look for."},{"t":"collect_charts()","u":"api/grader.charts.html#grader.charts.collect_charts","s":"grader.charts","k":"function","d":"Chart evidence, taking each chart from whichever notebook drew it."},{"t":"prose_word_count()","u":"api/grader.charts.html#grader.charts.prose_word_count","s":"grader.charts","k":"function","d":"Words of actual discussion — headings and placeholders do not count."},{"t":"MATPLOTLIB","u":"api/grader.charts.html#grader.charts.MATPLOTLIB","s":"grader.charts","k":"constant"},{"t":"SEABORN","u":"api/grader.charts.html#grader.charts.SEABORN","s":"grader.charts","k":"constant"},{"t":"ALTAIR","u":"api/grader.charts.html#grader.charts.ALTAIR","s":"grader.charts","k":"constant"},{"t":"SOURCE_EXECUTED","u":"api/grader.charts.html#grader.charts.SOURCE_EXECUTED","s":"grader.charts","k":"constant"},{"t":"SOURCE_SUBMITTED","u":"api/grader.charts.html#grader.charts.SOURCE_SUBMITTED","s":"grader.charts","k":"constant"},{"t":"SOURCE_MERGED","u":"api/grader.charts.html#grader.charts.SOURCE_MERGED","s":"grader.charts","k":"constant"},{"t":"SOURCE_NONE","u":"api/grader.charts.html#grader.charts.SOURCE_NONE","s":"grader.charts","k":"constant"},{"t":"SEABORN_NON_PLOTTING","u":"api/grader.charts.html#grader.charts.SEABORN_NON_PLOTTING","s":"grader.charts","k":"constant"},{"t":"SEABORN_PLOT_RE","u":"api/grader.charts.html#grader.charts.SEABORN_PLOT_RE","s":"grader.charts","k":"constant"},{"t":"MATPLOTLIB_PLOT_METHODS","u":"api/grader.charts.html#grader.charts.MATPLOTLIB_PLOT_METHODS","s":"grader.charts","k":"constant"},{"t":"MATPLOTLIB_PLOT_RE","u":"api/grader.charts.html#grader.charts.MATPLOTLIB_PLOT_RE","s":"grader.charts","k":"constant"},{"t":"PANDAS_PLOT_RE","u":"api/grader.charts.html#grader.charts.PANDAS_PLOT_RE","s":"grader.charts","k":"constant"},{"t":"ALTAIR_RE","u":"api/grader.charts.html#grader.charts.ALTAIR_RE","s":"grader.charts","k":"constant"},{"t":"ALTAIR_CHART_RE","u":"api/grader.charts.html#grader.charts.ALTAIR_CHART_RE","s":"grader.charts","k":"constant"},{"t":"VEGALITE_MIME_RE","u":"api/grader.charts.html#grader.charts.VEGALITE_MIME_RE","s":"grader.charts","k":"constant"},{"t":"VEGA_EMBED_MARKER","u":"api/grader.charts.html#grader.charts.VEGA_EMBED_MARKER","s":"grader.charts","k":"constant"},{"t":"COMPOSITION_KEYS","u":"api/grader.charts.html#grader.charts.COMPOSITION_KEYS","s":"grader.charts","k":"constant"},{"t":"TRANSFORM_OPS","u":"api/grader.charts.html#grader.charts.TRANSFORM_OPS","s":"grader.charts","k":"constant"},{"t":"ENCODING_TRANSFORM_KEYS","u":"api/grader.charts.html#grader.charts.ENCODING_TRANSFORM_KEYS","s":"grader.charts","k":"constant"},{"t":"AESTHETIC_PATTERNS","u":"api/grader.charts.html#grader.charts.AESTHETIC_PATTERNS","s":"grader.charts","k":"constant"},{"t":"IMAGE_MIMES","u":"api/grader.charts.html#grader.charts.IMAGE_MIMES","s":"grader.charts","k":"constant"},{"t":"ALTAIR_REPR_RE","u":"api/grader.charts.html#grader.charts.ALTAIR_REPR_RE","s":"grader.charts","k":"constant"},{"t":"HEADING_RE","u":"api/grader.charts.html#grader.charts.HEADING_RE","s":"grader.charts","k":"constant"},{"t":"PLACEHOLDER_RE","u":"api/grader.charts.html#grader.charts.PLACEHOLDER_RE","s":"grader.charts","k":"constant"},{"t":"grader.llm","u":"api/grader.llm.html","s":"Module","k":"module","d":"Optional LLM-assisted qualitative grading (design.md §26)."},{"t":"QualitativeGrader","u":"api/grader.llm.html#grader.llm.QualitativeGrader","s":"grader.llm","k":"class","d":"Interface the assignment graders depend on."},{"t":"QualitativeGrader.grade","u":"api/grader.llm.html#grader.llm.QualitativeGrader.grade","s":"grader.llm","k":"method"},{"t":"DisabledQualitativeGrader","u":"api/grader.llm.html#grader.llm.DisabledQualitativeGrader","s":"grader.llm","k":"class","d":"The default. Qualitative items stay with the human."},{"t":"DisabledQualitativeGrader.grade","u":"api/grader.llm.html#grader.llm.DisabledQualitativeGrader.grade","s":"grader.llm","k":"method"},{"t":"LLMQualitativeGrader","u":"api/grader.llm.html#grader.llm.LLMQualitativeGrader","s":"grader.llm","k":"class","d":"Provider-independent grader."},{"t":"LLMQualitativeGrader.grade","u":"api/grader.llm.html#grader.llm.LLMQualitativeGrader.grade","s":"grader.llm","k":"method"},{"t":"anthropic_grader()","u":"api/grader.llm.html#grader.llm.anthropic_grader","s":"grader.llm","k":"function","d":"Anthropic-backed grader, or a disabled one when it cannot be set up."},{"t":"build_grader()","u":"api/grader.llm.html#grader.llm.build_grader","s":"grader.llm","k":"function","d":"Construct a grader from UI settings."},{"t":"DEFAULT_REVIEW_THRESHOLD","u":"api/grader.llm.html#grader.llm.DEFAULT_REVIEW_THRESHOLD","s":"grader.llm","k":"constant"},{"t":"SYSTEM_PROMPT","u":"api/grader.llm.html#grader.llm.SYSTEM_PROMPT","s":"grader.llm","k":"constant"},{"t":"USER_TEMPLATE","u":"api/grader.llm.html#grader.llm.USER_TEMPLATE","s":"grader.llm","k":"constant"},{"t":"grader.utils","u":"api/grader.utils.html","s":"Module","k":"module","d":"Small shared helpers for the grading engine."},{"t":"utc_now_iso()","u":"api/grader.utils.html#grader.utils.utc_now_iso","s":"grader.utils","k":"function"},{"t":"slugify()","u":"api/grader.utils.html#grader.utils.slugify","s":"grader.utils","k":"function","d":"Filesystem-safe identifier."},{"t":"json_default()","u":"api/grader.utils.html#grader.utils.json_default","s":"grader.utils","k":"function","d":"Best-effort JSON encoder for numpy / pandas / arbitrary objects."},{"t":"dump_json()","u":"api/grader.utils.html#grader.utils.dump_json","s":"grader.utils","k":"function"},{"t":"load_json()","u":"api/grader.utils.html#grader.utils.load_json","s":"grader.utils","k":"function"},{"t":"is_close()","u":"api/grader.utils.html#grader.utils.is_close","s":"grader.utils","k":"function"},{"t":"as_float()","u":"api/grader.utils.html#grader.utils.as_float","s":"grader.utils","k":"function"},{"t":"normalize_zips()","u":"api/grader.utils.html#grader.utils.normalize_zips","s":"grader.utils","k":"function","d":"Coerce mixed zip representations (int, float, str, padded) to 5-char strings."},{"t":"jaccard()","u":"api/grader.utils.html#grader.utils.jaccard","s":"grader.utils","k":"function"},{"t":"truncate()","u":"api/grader.utils.html#grader.utils.truncate","s":"grader.utils","k":"function"},{"t":"pluralize()","u":"api/grader.utils.html#grader.utils.pluralize","s":"grader.utils","k":"function"},{"t":"ZIP_RE","u":"api/grader.utils.html#grader.utils.ZIP_RE","s":"grader.utils","k":"constant"},{"t":"assignments","u":"api/assignments.html","s":"Module","k":"module","d":"Per-assignment graders. Importing the package registers every grader."},{"t":"assignments.base","u":"api/assignments.base.html","s":"Module","k":"module","d":"Assignment grader base class and registry."},{"t":"GradingContext","u":"api/assignments.base.html#assignments.base.GradingContext","s":"assignments.base","k":"class","d":"Everything a check may look at."},{"t":"GradingContext.probe_ok","u":"api/assignments.base.html#assignments.base.GradingContext.probe_ok","s":"assignments.base","k":"property"},{"t":"AssignmentGrader","u":"api/assignments.base.html#assignments.base.AssignmentGrader","s":"assignments.base","k":"class","d":"Base grader: dispatches rubric ids to check_ methods."},{"t":"AssignmentGrader.probe_config","u":"api/assignments.base.html#assignments.base.AssignmentGrader.probe_config","s":"assignments.base","k":"method","d":"Config handed to the in-kernel probe (hidden tests included)."},{"t":"AssignmentGrader.hidden_test_specs","u":"api/assignments.base.html#assignments.base.AssignmentGrader.hidden_test_specs","s":"assignments.base","k":"method","d":"Only selectors and call inputs — expected values stay on the host."},{"t":"AssignmentGrader.grade","u":"api/assignments.base.html#assignments.base.AssignmentGrader.grade","s":"assignments.base","k":"method","d":"Run every rubric item's check_ method and return one result per item."},{"t":"AssignmentGrader.result","u":"api/assignments.base.html#assignments.base.AssignmentGrader.result","s":"assignments.base","k":"method","d":"Build a RubricItemResult, rounding the score to the rubric's score_rounding increment and clamping it to [0, points]."},{"t":"AssignmentGrader.check_unimplemented","u":"api/assignments.base.html#assignments.base.AssignmentGrader.check_unimplemented","s":"assignments.base","k":"method"},{"t":"AssignmentGrader.no_probe_result","u":"api/assignments.base.html#assignments.base.AssignmentGrader.no_probe_result","s":"assignments.base","k":"method","d":"Shared response when the kernel never reported its namespace."},{"t":"AssignmentGrader.check_notebook_execution","u":"api/assignments.base.html#assignments.base.AssignmentGrader.check_notebook_execution","s":"assignments.base","k":"method","d":"Shared execution check (design.md §19). Never an automatic zero."},{"t":"register()","u":"api/assignments.base.html#assignments.base.register","s":"assignments.base","k":"function","d":"Class decorator: register an AssignmentGrader subclass under its assignment_id."},{"t":"get_grader()","u":"api/assignments.base.html#assignments.base.get_grader","s":"assignments.base","k":"function","d":"Return the grader for a rubric, falling back to the generic base."},{"t":"registered_assignments()","u":"api/assignments.base.html#assignments.base.registered_assignments","s":"assignments.base","k":"function"},{"t":"assignments.hw1","u":"api/assignments.hw1.html","s":"Module","k":"module","d":"Assignment 1 autograder — \"The Donut Effect for Philadelphia ZIP Codes\"."},{"t":"HW1Grader","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader","s":"assignments.hw1","k":"class"},{"t":"HW1Grader.id_hints","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.id_hints","s":"assignments.hw1","k":"property"},{"t":"HW1Grader.date_hints","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.date_hints","s":"assignments.hw1","k":"property"},{"t":"HW1Grader.value_hints","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.value_hints","s":"assignments.hw1","k":"property"},{"t":"HW1Grader.check_data_loading","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_data_loading","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_philadelphia_subset","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_philadelphia_subset","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_tidy_transformation","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_tidy_transformation","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_center_city_split","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_center_city_split","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_percent_increase_function","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_percent_increase_function","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_donut_effect_comparison","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_donut_effect_comparison","s":"assignments.hw1","k":"method"},{"t":"DEFAULT_CENTER_WORDS","u":"api/assignments.hw1.html#assignments.hw1.DEFAULT_CENTER_WORDS","s":"assignments.hw1","k":"constant"},{"t":"DEFAULT_OUTSIDE_WORDS","u":"api/assignments.hw1.html#assignments.hw1.DEFAULT_OUTSIDE_WORDS","s":"assignments.hw1","k":"constant"},{"t":"DATE_COLUMN_RE","u":"api/assignments.hw1.html#assignments.hw1.DATE_COLUMN_RE","s":"assignments.hw1","k":"constant"},{"t":"assignments.hw2","u":"api/assignments.hw2.html","s":"Module","k":"module","d":"Assignment 2 autograder — exploratory visualization with matplotlib, seaborn and altair."},{"t":"HW2Grader","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader","s":"assignments.hw2","k":"class"},{"t":"HW2Grader.evidence","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.evidence","s":"assignments.hw2","k":"method","d":"Charts found in this submission, read once and reused by every check."},{"t":"HW2Grader.required_count","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.required_count","s":"assignments.hw2","k":"method","d":"How many charts of a library the rubric asks for."},{"t":"HW2Grader.evidence_note","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.evidence_note","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.no_evidence_result","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.no_evidence_result","s":"assignments.hw2","k":"method","d":"No notebook could be read for charts — never an automatic zero."},{"t":"HW2Grader.check_notebook_execution","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_notebook_execution","s":"assignments.hw2","k":"method","d":"Did the student run their notebook through before handing it in?"},{"t":"HW2Grader.check_data_loading","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_data_loading","s":"assignments.hw2","k":"method","d":"Is a dataset loaded, and loaded the way the instructions ask?"},{"t":"HW2Grader.check_matplotlib_chart","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_matplotlib_chart","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_seaborn_chart","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_seaborn_chart","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_altair_charts","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_altair_charts","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_altair_transformation","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_altair_transformation","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_altair_brush_selection","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_altair_brush_selection","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_altair_dashboard_extra_credit","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_altair_dashboard_extra_credit","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_matplotlib_rationale","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_matplotlib_rationale","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_seaborn_rationale","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_seaborn_rationale","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_matplotlib_aesthetics","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_matplotlib_aesthetics","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_chart_conclusions","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_chart_conclusions","s":"assignments.hw2","k":"method"},{"t":"URL_RE","u":"api/assignments.hw2.html#assignments.hw2.URL_RE","s":"assignments.hw2","k":"constant"},{"t":"LIBRARY_LABELS","u":"api/assignments.hw2.html#assignments.hw2.LIBRARY_LABELS","s":"assignments.hw2","k":"constant"},{"t":"assignments.hw3","u":"api/assignments.hw3.html","s":"Module","k":"module","d":"Assignment 3 autograder — evictions, code violations and the NDVI."},{"t":"HW3Grader","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader","s":"assignments.hw3","k":"class"},{"t":"HW3Grader.answers","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.answers","s":"assignments.hw3","k":"method","d":"What the notebooks show, read once per submission and reused."},{"t":"HW3Grader.grade","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.grade","s":"assignments.hw3","k":"method","d":"Grade on the objects this run produced *and* on what the file shows."},{"t":"HW3Grader.charts","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.charts","s":"assignments.hw3","k":"method","d":"matplotlib figures, read once per submission and reused."},{"t":"HW3Grader.check_eviction_data_loading","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_eviction_data_loading","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_philadelphia_tracts","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_philadelphia_tracts","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tidy_evictions","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tidy_evictions","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_evictions_by_year_plot","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_evictions_by_year_plot","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_evictions_choropleth","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_evictions_choropleth","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_violations_choropleth","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_violations_choropleth","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_violations_loading","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_violations_loading","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_violations_subset","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_violations_subset","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_hexbin_map","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_hexbin_map","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_spatial_join","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_spatial_join","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_violations_by_tract","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_violations_by_tract","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_merge_geometries","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_merge_geometries","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_side_by_side","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_side_by_side","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_landsat_loading","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_landsat_loading","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_city_suburb_polygons","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_city_suburb_polygons","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_ndvi_masking","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_ndvi_masking","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_ndvi_medians","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_ndvi_medians","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tree_data_loading","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tree_data_loading","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tree_ndvi","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tree_ndvi","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tree_plots","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tree_plots","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tree_plot_styling","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tree_plot_styling","s":"assignments.hw3","k":"method","d":"The one item on this assignment that a person has to look at."},{"t":"HW3Grader.check_common_violations_extra_credit","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_common_violations_extra_credit","s":"assignments.hw3","k":"method"},{"t":"keys_cover()","u":"api/assignments.hw3.html#assignments.hw3.keys_cover","s":"assignments.hw3","k":"function","d":"Every expected key is present, reading \"e-03\", \"2003\" and 2003 as one year."},{"t":"WIDGET_CONTAINERS","u":"api/assignments.hw3.html#assignments.hw3.WIDGET_CONTAINERS","s":"assignments.hw3","k":"constant"},{"t":"LAZY_CONTAINERS","u":"api/assignments.hw3.html#assignments.hw3.LAZY_CONTAINERS","s":"assignments.hw3","k":"constant"},{"t":"LAYOUT_CONTAINERS","u":"api/assignments.hw3.html#assignments.hw3.LAYOUT_CONTAINERS","s":"assignments.hw3","k":"constant"},{"t":"OVERLAY_CONTAINERS","u":"api/assignments.hw3.html#assignments.hw3.OVERLAY_CONTAINERS","s":"assignments.hw3","k":"constant"},{"t":"MAP_ELEMENTS","u":"api/assignments.hw3.html#assignments.hw3.MAP_ELEMENTS","s":"assignments.hw3","k":"constant"},{"t":"YEAR_KEY_RE","u":"api/assignments.hw3.html#assignments.hw3.YEAR_KEY_RE","s":"assignments.hw3","k":"constant"}];
+window.MUSA_SEARCH_INDEX = [{"t":"Overview","u":"index.html","s":"Introduction","k":"page","d":"Semi-automatic grading for Jupyter Notebook assignments in MUSA 5500: Geospatial Data Science in Python."},{"t":"What it does","u":"index.html#what-it-does","s":"Overview","k":"section"},{"t":"Four assignments, four ways to grade","u":"index.html#four-assignments-four-ways-to-grade","s":"Overview","k":"section"},{"t":"Student data","u":"index.html#student-data","s":"Overview","k":"section"},{"t":"Architecture at a glance","u":"index.html#architecture-at-a-glance","s":"Overview","k":"section"},{"t":"Getting started","u":"getting-started.html","s":"Introduction","k":"page","d":"Install the grader, build the sandbox image, generate demo submissions and run your first grading session."},{"t":"Requirements","u":"getting-started.html#requirements","s":"Getting started","k":"section"},{"t":"Install","u":"getting-started.html#install","s":"Getting started","k":"section"},{"t":"Grade a demo class","u":"getting-started.html#grade-a-demo-class","s":"Getting started","k":"section"},{"t":"Assignment 1","u":"getting-started.html#assignment-1","s":"Getting started","k":"section"},{"t":"Assignment 2","u":"getting-started.html#assignment-2","s":"Getting started","k":"section"},{"t":"Assignment 3","u":"getting-started.html#assignment-3","s":"Getting started","k":"section"},{"t":"Start the web app","u":"getting-started.html#start-the-web-app","s":"Getting started","k":"section"},{"t":"Run everything in Docker","u":"getting-started.html#run-everything-in-docker","s":"Getting started","k":"section"},{"t":"Next steps","u":"getting-started.html#next-steps","s":"Getting started","k":"section"},{"t":"TA workflow","u":"ta-workflow.html","s":"User guide","k":"page","d":"A tour of the Streamlit app, from loading submissions to exporting grades."},{"t":"Start the app","u":"ta-workflow.html#start-the-app","s":"TA workflow","k":"section"},{"t":"The sidebar","u":"ta-workflow.html#the-sidebar","s":"TA workflow","k":"section"},{"t":"Assignment","u":"ta-workflow.html#assignment","s":"TA workflow","k":"section"},{"t":"Submissions","u":"ta-workflow.html#submissions","s":"TA workflow","k":"section"},{"t":"Assignment data","u":"ta-workflow.html#assignment-data","s":"TA workflow","k":"section"},{"t":"Pages","u":"ta-workflow.html#pages","s":"TA workflow","k":"section"},{"t":"The review queue","u":"ta-workflow.html#review-queue","s":"TA workflow","k":"section"},{"t":"Manual overrides","u":"ta-workflow.html#manual-overrides","s":"TA workflow","k":"section"},{"t":"Settings","u":"ta-workflow.html#settings","s":"TA workflow","k":"section"},{"t":"CLI reference","u":"cli.html","s":"User guide","k":"page","d":"The cli.py commands and flags for headless, scripted and CI grading runs."},{"t":"grade","u":"cli.html#grade","s":"CLI reference","k":"section"},{"t":"Example output","u":"cli.html#example-output","s":"CLI reference","k":"section"},{"t":"Examples","u":"cli.html#examples","s":"CLI reference","k":"section"},{"t":"sessions","u":"cli.html#sessions","s":"CLI reference","k":"section"},{"t":"rubrics","u":"cli.html#rubrics","s":"CLI reference","k":"section"},{"t":"Exit codes","u":"cli.html#exit-codes","s":"CLI reference","k":"section"},{"t":"How grading works","u":"how-it-works.html","s":"User guide","k":"page","d":"The grading pipeline: data supply, discovery, sandboxed execution, inspection, hidden tests, confidence and review."},{"t":"0. The data comes from you, not the students","u":"how-it-works.html#data-supply","s":"How grading works","k":"section"},{"t":"1. Discovery","u":"how-it-works.html#discovery","s":"How grading works","k":"section"},{"t":"2. Execution","u":"how-it-works.html#execution","s":"How grading works","k":"section"},{"t":"3. Inspection without variable names","u":"how-it-works.html#inspection-without-variable-names","s":"How grading works","k":"section"},{"t":"4. Hidden tests","u":"how-it-works.html#hidden-tests","s":"How grading works","k":"section"},{"t":"Two readings: saved outputs and the re-run","u":"how-it-works.html#two-readings","s":"How grading works","k":"section"},{"t":"5. Confidence and review","u":"how-it-works.html#confidence-and-review","s":"How grading works","k":"section"},{"t":"Item statuses","u":"how-it-works.html#statuses","s":"How grading works","k":"section"},{"t":"Scores and rounding","u":"how-it-works.html#scores","s":"How grading works","k":"section"},{"t":"Assignments","u":"assignments.html","s":"User guide","k":"page","d":"How Assignments 1, 2, 3 and 4 are graded, and where each one's 100 points go."},{"t":"Assignment 1: The Donut Effect","u":"assignments.html#assignment-1-the-donut-effect","s":"Assignments","k":"section"},{"t":"Assignment 2: Exploratory Visualization","u":"assignments.html#assignment-2-exploratory-visualization","s":"Assignments","k":"section"},{"t":"Facts, graded automatically (70 points)","u":"assignments.html#facts-graded-automatically-70-points","s":"Assignments","k":"section"},{"t":"Judgement, graded by a person (30 points)","u":"assignments.html#judgement-graded-by-a-person-30-points","s":"Assignments","k":"section"},{"t":"Assignment 3: Evictions and the NDVI","u":"assignments.html#assignment-3-evictions-and-the-ndvi","s":"Assignments","k":"section"},{"t":"Plots are graded on the objects","u":"assignments.html#plots-are-graded-on-the-objects","s":"Assignments","k":"section"},{"t":"Named mistakes, not just wrong answers","u":"assignments.html#named-mistakes-not-just-wrong-answers","s":"Assignments","k":"section"},{"t":"Calibrated against a real class","u":"assignments.html#calibrated-against-a-real-class","s":"Assignments","k":"section"},{"t":"Where the 100 points go","u":"assignments.html#where-the-100-points-go","s":"Assignments","k":"section"},{"t":"Assignment 4: Street Networks & Scraping","u":"assignments.html#assignment-4-street-networks-scraping","s":"Assignments","k":"section"},{"t":"The one thing that does run: three pure functions","u":"assignments.html#hw4-isolated-functions","s":"Assignments","k":"section"},{"t":"Scraping has to be shown, not loaded","u":"assignments.html#scraping-has-to-be-shown-not-loaded","s":"Assignments","k":"section"},{"t":"Reference values, loosely","u":"assignments.html#reference-values-loosely","s":"Assignments","k":"section"},{"t":"Where the 100 points go","u":"assignments.html#where-the-100-points-go-2","s":"Assignments","k":"section"},{"t":"Assignments 5–6","u":"assignments.html#assignments-5-6","s":"Assignments","k":"section"},{"t":"Rubrics","u":"rubrics.html","s":"User guide","k":"page","d":"The rubric YAML schema: assignment metadata, settings, discovery, data supply, probe hints and rubric items."},{"t":"Before grading a real class","u":"rubrics.html#before-grading-a-real-class","s":"Rubrics","k":"section"},{"t":"File layout","u":"rubrics.html#file-layout","s":"Rubrics","k":"section"},{"t":"assignment","u":"rubrics.html#assignment","s":"Rubrics","k":"section"},{"t":"settings","u":"rubrics.html#settings","s":"Rubrics","k":"section"},{"t":"discovery","u":"rubrics.html#discovery","s":"Rubrics","k":"section"},{"t":"data","u":"rubrics.html#data","s":"Rubrics","k":"section"},{"t":"execution","u":"rubrics.html#execution","s":"Rubrics","k":"section"},{"t":"probe","u":"rubrics.html#probe","s":"Rubrics","k":"section"},{"t":"rubric items","u":"rubrics.html#rubric-items","s":"Rubrics","k":"section"},{"t":"Item types","u":"rubrics.html#item-types","s":"Rubrics","k":"section"},{"t":"Hidden-test config","u":"rubrics.html#hidden-test-config","s":"Rubrics","k":"section"},{"t":"isolated_functions","u":"rubrics.html#isolated-functions","s":"Rubrics","k":"section"},{"t":"Credit ratios","u":"rubrics.html#credit-ratios","s":"Rubrics","k":"section"},{"t":"Validating a rubric","u":"rubrics.html#validating-a-rubric","s":"Rubrics","k":"section"},{"t":"Docker sandbox","u":"docker.html","s":"User guide","k":"page","d":"How student notebooks are isolated, what is in the sandbox image, and how to run the app itself in a container."},{"t":"Build the image","u":"docker.html#build-the-image","s":"Docker sandbox","k":"section"},{"t":"Isolation flags","u":"docker.html#isolation-flags","s":"Docker sandbox","k":"section"},{"t":"Memory and time","u":"docker.html#memory-and-time","s":"Docker sandbox","k":"section"},{"t":"What is in the image","u":"docker.html#what-is-in-the-image","s":"Docker sandbox","k":"section"},{"t":"Updating the pins","u":"docker.html#updating-the-pins","s":"Docker sandbox","k":"section"},{"t":"Running the app in a container","u":"docker.html#running-the-app-in-a-container","s":"Docker sandbox","k":"section"},{"t":"Results and exports","u":"results.html","s":"User guide","k":"page","d":"What a grading session writes to disk, the result JSON schema, and how to reload, regrade and export."},{"t":"Session directory","u":"results.html#session-directory","s":"Results and exports","k":"section"},{"t":"Result schema","u":"results.html#result-schema","s":"Results and exports","k":"section"},{"t":"Exports","u":"results.html#exports","s":"Results and exports","k":"section"},{"t":"Reloading and regrading","u":"results.html#reloading-and-regrading","s":"Results and exports","k":"section"},{"t":"Adding an assignment","u":"extending.html","s":"Developer guide","k":"page","d":"Write a rubric YAML and a grader class with check methods to grade a new assignment."},{"t":"1. Write the rubric","u":"extending.html#1-write-the-rubric","s":"Adding an assignment","k":"section"},{"t":"2. Write the grader class","u":"extending.html#2-write-the-grader-class","s":"Adding an assignment","k":"section"},{"t":"The check contract","u":"extending.html#the-check-contract","s":"Adding an assignment","k":"section"},{"t":"What a check can see","u":"extending.html#what-a-check-can-see","s":"Adding an assignment","k":"section"},{"t":"Hidden tests","u":"extending.html#hidden-tests","s":"Adding an assignment","k":"section"},{"t":"3. Add tests and demo submissions","u":"extending.html#3-add-tests-and-demo-submissions","s":"Adding an assignment","k":"section"},{"t":"Testing and CI","u":"testing.html","s":"Developer guide","k":"page","d":"Running the test suite, the markers it uses, what CI checks, and how to rebuild these docs."},{"t":"Running the tests","u":"testing.html#running-the-tests","s":"Testing and CI","k":"section"},{"t":"What the suite covers","u":"testing.html#what-the-suite-covers","s":"Testing and CI","k":"section"},{"t":"Continuous integration","u":"testing.html#continuous-integration","s":"Testing and CI","k":"section"},{"t":"Building these docs","u":"testing.html#building-these-docs","s":"Testing and CI","k":"section"},{"t":"API reference","u":"api/index.html","s":"API","k":"page"},{"t":"grader","u":"api/grader.html","s":"Module","k":"module","d":"MUSA Grader — grading engine."},{"t":"grader.service","u":"api/grader.service.html","s":"Module","k":"module","d":"The grading service the UI talks to."},{"t":"GraderSettings","u":"api/grader.service.html#grader.service.GraderSettings","s":"grader.service","k":"class","d":"Everything the Settings page can change (design.md §33)."},{"t":"GraderSettings.execution_config","u":"api/grader.service.html#grader.service.GraderSettings.execution_config","s":"grader.service","k":"method","d":"Execution limits, raised if the assignment declares it needs longer."},{"t":"GraderSettings.to_dict","u":"api/grader.service.html#grader.service.GraderSettings.to_dict","s":"grader.service","k":"method","d":"The settings as stored in grading_session.json."},{"t":"LoadedSubmissions","u":"api/grader.service.html#grader.service.LoadedSubmissions","s":"grader.service","k":"class","d":"What GradingService.load_submissions found: one SubmissionCandidate per student, plus where a Canvas ZIP was extracted to."},{"t":"LoadedSubmissions.ok_count","u":"api/grader.service.html#grader.service.LoadedSubmissions.ok_count","s":"grader.service","k":"property"},{"t":"LoadedSubmissions.warning_count","u":"api/grader.service.html#grader.service.LoadedSubmissions.warning_count","s":"grader.service","k":"property"},{"t":"GradingService","u":"api/grader.service.html#grader.service.GradingService","s":"grader.service","k":"class","d":"Orchestrates a grading run: find submissions, place the data, execute each notebook, run the assignment's checks and apply the review policy."},{"t":"GradingService.from_rubric_path","u":"api/grader.service.html#grader.service.GradingService.from_rubric_path","s":"grader.service","k":"classmethod","d":"Load a rubric YAML and build a service for it."},{"t":"GradingService.preflight","u":"api/grader.service.html#grader.service.GradingService.preflight","s":"grader.service","k":"method","d":"Problems that would make a grading run misleading rather than wrong."},{"t":"GradingService.load_submissions","u":"api/grader.service.html#grader.service.GradingService.load_submissions","s":"grader.service","k":"method","d":"Accept a folder or a Canvas ZIP (design.md §16)."},{"t":"GradingService.new_session","u":"api/grader.service.html#grader.service.GradingService.new_session","s":"grader.service","k":"method","d":"An empty GradingSession for this rubric and these settings, with a fresh timestamped session id."},{"t":"GradingService.run","u":"api/grader.service.html#grader.service.GradingService.run","s":"grader.service","k":"method","d":"Grade every candidate and return the saved session."},{"t":"GradingService.regrade","u":"api/grader.service.html#grader.service.GradingService.regrade","s":"grader.service","k":"method","d":"Grade the same candidates again into the same session id and directory."},{"t":"STAGE_LABELS","u":"api/grader.service.html#grader.service.STAGE_LABELS","s":"grader.service","k":"constant"},{"t":"grader.rubric","u":"api/grader.rubric.html","s":"Module","k":"module","d":"YAML rubric loading (design.md section 20)."},{"t":"RubricItem","u":"api/grader.rubric.html#grader.rubric.RubricItem","s":"grader.rubric","k":"class","d":"One graded line of a rubric, as declared in the YAML rubric: list."},{"t":"RubricItem.is_qualitative","u":"api/grader.rubric.html#grader.rubric.RubricItem.is_qualitative","s":"grader.rubric","k":"property"},{"t":"Rubric","u":"api/grader.rubric.html#grader.rubric.Rubric","s":"grader.rubric","k":"class","d":"A parsed rubric file. Every top-level YAML block is kept as-is, so assignment graders can read their own configuration from settings, probe, ``discovery"},{"t":"Rubric.item","u":"api/grader.rubric.html#grader.rubric.Rubric.item","s":"grader.rubric","k":"method"},{"t":"Rubric.requires_data","u":"api/grader.rubric.html#grader.rubric.Rubric.requires_data","s":"grader.rubric","k":"property"},{"t":"Rubric.offers_data","u":"api/grader.rubric.html#grader.rubric.Rubric.offers_data","s":"grader.rubric","k":"property","d":"Whether the sidebar should let a TA supply data for this assignment."},{"t":"Rubric.never_executes","u":"api/grader.rubric.html#grader.rubric.Rubric.never_executes","s":"grader.rubric","k":"property","d":"The notebook must not be run during grading, by any backend."},{"t":"Rubric.declared_points","u":"api/grader.rubric.html#grader.rubric.Rubric.declared_points","s":"grader.rubric","k":"property"},{"t":"Rubric.label","u":"api/grader.rubric.html#grader.rubric.Rubric.label","s":"grader.rubric","k":"method"},{"t":"RubricError","u":"api/grader.rubric.html#grader.rubric.RubricError","s":"grader.rubric","k":"class","d":"Raised when a rubric file is missing or structurally invalid."},{"t":"load_rubric()","u":"api/grader.rubric.html#grader.rubric.load_rubric","s":"grader.rubric","k":"function","d":"Parse and validate a rubric YAML file."},{"t":"available_rubrics()","u":"api/grader.rubric.html#grader.rubric.available_rubrics","s":"grader.rubric","k":"function","d":"Map assignment id -> rubric path for every parsable rubric on disk."},{"t":"RUBRIC_DIR","u":"api/grader.rubric.html#grader.rubric.RUBRIC_DIR","s":"grader.rubric","k":"constant"},{"t":"VALID_TYPES","u":"api/grader.rubric.html#grader.rubric.VALID_TYPES","s":"grader.rubric","k":"constant"},{"t":"grader.models","u":"api/grader.models.html","s":"Module","k":"module","d":"Structured result types shared by the engine, the store and the UI."},{"t":"ExecutionRecord","u":"api/grader.models.html#grader.models.ExecutionRecord","s":"grader.models","k":"class","d":"Outcome of running one student notebook (design.md section 19)."},{"t":"ExecutionRecord.status","u":"api/grader.models.html#grader.models.ExecutionRecord.status","s":"grader.models","k":"property"},{"t":"ExecutionRecord.to_dict","u":"api/grader.models.html#grader.models.ExecutionRecord.to_dict","s":"grader.models","k":"method"},{"t":"ExecutionRecord.from_dict","u":"api/grader.models.html#grader.models.ExecutionRecord.from_dict","s":"grader.models","k":"classmethod"},{"t":"RubricItemResult","u":"api/grader.models.html#grader.models.RubricItemResult","s":"grader.models","k":"class","d":"One graded rubric line (design.md section 24)."},{"t":"RubricItemResult.final_score","u":"api/grader.models.html#grader.models.RubricItemResult.final_score","s":"grader.models","k":"property","d":"The manual score when overridden, otherwise the automatic score."},{"t":"RubricItemResult.needs_review","u":"api/grader.models.html#grader.models.RubricItemResult.needs_review","s":"grader.models","k":"property"},{"t":"RubricItemResult.to_dict","u":"api/grader.models.html#grader.models.RubricItemResult.to_dict","s":"grader.models","k":"method"},{"t":"RubricItemResult.from_dict","u":"api/grader.models.html#grader.models.RubricItemResult.from_dict","s":"grader.models","k":"classmethod"},{"t":"SubmissionResult","u":"api/grader.models.html#grader.models.SubmissionResult","s":"grader.models","k":"class","d":"Everything known about one student's submission."},{"t":"SubmissionResult.max_score","u":"api/grader.models.html#grader.models.SubmissionResult.max_score","s":"grader.models","k":"property","d":"Sum of the items' possible points."},{"t":"SubmissionResult.automatic_total","u":"api/grader.models.html#grader.models.SubmissionResult.automatic_total","s":"grader.models","k":"property"},{"t":"SubmissionResult.total_score","u":"api/grader.models.html#grader.models.SubmissionResult.total_score","s":"grader.models","k":"property","d":"Sum of the items' final scores."},{"t":"SubmissionResult.percentage","u":"api/grader.models.html#grader.models.SubmissionResult.percentage","s":"grader.models","k":"property"},{"t":"SubmissionResult.confidence","u":"api/grader.models.html#grader.models.SubmissionResult.confidence","s":"grader.models","k":"property","d":"Lowest item confidence — the weakest link decides trust in the grade."},{"t":"SubmissionResult.has_manual_override","u":"api/grader.models.html#grader.models.SubmissionResult.has_manual_override","s":"grader.models","k":"property"},{"t":"SubmissionResult.needs_review","u":"api/grader.models.html#grader.models.SubmissionResult.needs_review","s":"grader.models","k":"property"},{"t":"SubmissionResult.open_review_reasons","u":"api/grader.models.html#grader.models.SubmissionResult.open_review_reasons","s":"grader.models","k":"method"},{"t":"SubmissionResult.item","u":"api/grader.models.html#grader.models.SubmissionResult.item","s":"grader.models","k":"method"},{"t":"SubmissionResult.to_dict","u":"api/grader.models.html#grader.models.SubmissionResult.to_dict","s":"grader.models","k":"method"},{"t":"SubmissionResult.from_dict","u":"api/grader.models.html#grader.models.SubmissionResult.from_dict","s":"grader.models","k":"classmethod"},{"t":"GradingProgress","u":"api/grader.models.html#grader.models.GradingProgress","s":"grader.models","k":"class","d":"Snapshot pushed to the UI while grading runs (design.md section 7)."},{"t":"GradingProgress.remaining","u":"api/grader.models.html#grader.models.GradingProgress.remaining","s":"grader.models","k":"property"},{"t":"GradingProgress.to_dict","u":"api/grader.models.html#grader.models.GradingProgress.to_dict","s":"grader.models","k":"method"},{"t":"status_icon()","u":"api/grader.models.html#grader.models.status_icon","s":"grader.models","k":"function"},{"t":"status_color()","u":"api/grader.models.html#grader.models.status_color","s":"grader.models","k":"function"},{"t":"STATUS_PASS","u":"api/grader.models.html#grader.models.STATUS_PASS","s":"grader.models","k":"constant"},{"t":"STATUS_PARTIAL","u":"api/grader.models.html#grader.models.STATUS_PARTIAL","s":"grader.models","k":"constant"},{"t":"STATUS_FAIL","u":"api/grader.models.html#grader.models.STATUS_FAIL","s":"grader.models","k":"constant"},{"t":"STATUS_WARNING","u":"api/grader.models.html#grader.models.STATUS_WARNING","s":"grader.models","k":"constant"},{"t":"STATUS_MANUAL_REVIEW","u":"api/grader.models.html#grader.models.STATUS_MANUAL_REVIEW","s":"grader.models","k":"constant"},{"t":"STATUS_EXECUTION_ERROR","u":"api/grader.models.html#grader.models.STATUS_EXECUTION_ERROR","s":"grader.models","k":"constant"},{"t":"STATUS_NOT_FOUND","u":"api/grader.models.html#grader.models.STATUS_NOT_FOUND","s":"grader.models","k":"constant"},{"t":"ALL_STATUSES","u":"api/grader.models.html#grader.models.ALL_STATUSES","s":"grader.models","k":"constant"},{"t":"STATUS_DISPLAY","u":"api/grader.models.html#grader.models.STATUS_DISPLAY","s":"grader.models","k":"constant"},{"t":"grader.results","u":"api/grader.results.html","s":"Module","k":"module","d":"Grading session persistence and exports (design.md §28-29)."},{"t":"GradingSession","u":"api/grader.results.html#grader.results.GradingSession","s":"grader.results","k":"class","d":"One grading run and everything decided in it, persisted under results//."},{"t":"GradingSession.ordered_results","u":"api/grader.results.html#grader.results.GradingSession.ordered_results","s":"grader.results","k":"method","d":"Results sorted by student id."},{"t":"GradingSession.upsert","u":"api/grader.results.html#grader.results.GradingSession.upsert","s":"grader.results","k":"method","d":"Add or replace one student's result."},{"t":"GradingSession.summary","u":"api/grader.results.html#grader.results.GradingSession.summary","s":"grader.results","k":"method","d":"The class summary (mean, median, distribution, common issues, rubric breakdown)."},{"t":"GradingSession.queue","u":"api/grader.results.html#grader.results.GradingSession.queue","s":"grader.results","k":"method","d":"The review queue: submissions that still need a person, most urgent first."},{"t":"GradingSession.student_ids","u":"api/grader.results.html#grader.results.GradingSession.student_ids","s":"grader.results","k":"method"},{"t":"GradingSession.neighbour","u":"api/grader.results.html#grader.results.GradingSession.neighbour","s":"grader.results","k":"method"},{"t":"GradingSession.set_override","u":"api/grader.results.html#grader.results.GradingSession.set_override","s":"grader.results","k":"method","d":"Override one rubric item's score. The automatic score is kept alongside it; pass enabled=False to remove an override. Returns False if the student or it"},{"t":"GradingSession.mark_reviewed","u":"api/grader.results.html#grader.results.GradingSession.mark_reviewed","s":"grader.results","k":"method","d":"Mark a submission as reviewed, which removes it from the queue."},{"t":"GradingSession.collect_overrides","u":"api/grader.results.html#grader.results.GradingSession.collect_overrides","s":"grader.results","k":"method","d":"Snapshot manual work so a regrade can restore it (design.md §31)."},{"t":"GradingSession.apply_overrides","u":"api/grader.results.html#grader.results.GradingSession.apply_overrides","s":"grader.results","k":"method"},{"t":"GradingSession.to_dict","u":"api/grader.results.html#grader.results.GradingSession.to_dict","s":"grader.results","k":"method"},{"t":"GradingSession.save","u":"api/grader.results.html#grader.results.GradingSession.save","s":"grader.results","k":"method","d":"Write the session and all its exports to disk; returns the session directory."},{"t":"GradingSession.store_executed_notebook","u":"api/grader.results.html#grader.results.GradingSession.store_executed_notebook","s":"grader.results","k":"method","d":"Keep the executed notebook alongside the results (design.md §28)."},{"t":"GradingSession.load","u":"api/grader.results.html#grader.results.GradingSession.load","s":"grader.results","k":"classmethod","d":"Read a session back from its directory."},{"t":"GradingSession.grades_csv","u":"api/grader.results.html#grader.results.GradingSession.grades_csv","s":"grader.results","k":"method","d":"grades.csv: one row per student."},{"t":"GradingSession.rubric_csv","u":"api/grader.results.html#grader.results.GradingSession.rubric_csv","s":"grader.results","k":"method","d":"Long-format per-item scores, useful for item analysis."},{"t":"GradingSession.flagged_csv","u":"api/grader.results.html#grader.results.GradingSession.flagged_csv","s":"grader.results","k":"method","d":"flagged_submissions.csv: the review queue."},{"t":"GradingSession.feedback_zip","u":"api/grader.results.html#grader.results.GradingSession.feedback_zip","s":"grader.results","k":"method","d":"Every student's markdown feedback, zipped."},{"t":"GradingSession.archive_zip","u":"api/grader.results.html#grader.results.GradingSession.archive_zip","s":"grader.results","k":"method","d":"The whole session directory, for handing over or archiving."},{"t":"list_sessions()","u":"api/grader.results.html#grader.results.list_sessions","s":"grader.results","k":"function","d":"Every saved session under results/, in reverse directory-name order (newest first for each assignment)."},{"t":"new_session_id()","u":"api/grader.results.html#grader.results.new_session_id","s":"grader.results","k":"function","d":"__."},{"t":"RESULTS_ROOT","u":"api/grader.results.html#grader.results.RESULTS_ROOT","s":"grader.results","k":"constant"},{"t":"SESSION_FILE","u":"api/grader.results.html#grader.results.SESSION_FILE","s":"grader.results","k":"constant"},{"t":"GRADES_CSV","u":"api/grader.results.html#grader.results.GRADES_CSV","s":"grader.results","k":"constant"},{"t":"SUMMARY_JSON","u":"api/grader.results.html#grader.results.SUMMARY_JSON","s":"grader.results","k":"constant"},{"t":"FLAGGED_CSV","u":"api/grader.results.html#grader.results.FLAGGED_CSV","s":"grader.results","k":"constant"},{"t":"FEEDBACK_DIR","u":"api/grader.results.html#grader.results.FEEDBACK_DIR","s":"grader.results","k":"constant"},{"t":"EXECUTED_DIR","u":"api/grader.results.html#grader.results.EXECUTED_DIR","s":"grader.results","k":"constant"},{"t":"RAW_DIR","u":"api/grader.results.html#grader.results.RAW_DIR","s":"grader.results","k":"constant"},{"t":"grader.scoring","u":"api/grader.scoring.html","s":"Module","k":"module","d":"Review policy and class-level aggregation (design.md §8, §25)."},{"t":"apply_review_policy()","u":"api/grader.scoring.html#grader.scoring.apply_review_policy","s":"grader.scoring","k":"function","d":"Decide what a human still has to look at."},{"t":"grade_letter()","u":"api/grader.scoring.html#grader.scoring.grade_letter","s":"grader.scoring","k":"function"},{"t":"class_summary()","u":"api/grader.scoring.html#grader.scoring.class_summary","s":"grader.scoring","k":"function","d":"Summary cards, distribution and common issues (design.md §8)."},{"t":"grade_distribution()","u":"api/grader.scoring.html#grader.scoring.grade_distribution","s":"grader.scoring","k":"function"},{"t":"common_issues()","u":"api/grader.scoring.html#grader.scoring.common_issues","s":"grader.scoring","k":"function","d":"Rank the deductions that affected the most students (design.md §8)."},{"t":"rubric_breakdown()","u":"api/grader.scoring.html#grader.scoring.rubric_breakdown","s":"grader.scoring","k":"function","d":"Mean score per rubric item — shows where the class as a whole struggled."},{"t":"review_queue()","u":"api/grader.scoring.html#grader.scoring.review_queue","s":"grader.scoring","k":"function","d":"Unresolved review items, worst first (design.md §14)."},{"t":"DEFAULT_CONFIDENCE_THRESHOLD","u":"api/grader.scoring.html#grader.scoring.DEFAULT_CONFIDENCE_THRESHOLD","s":"grader.scoring","k":"constant"},{"t":"grader.feedback","u":"api/grader.feedback.html","s":"Module","k":"module","d":"Student-facing markdown feedback (design.md §30)."},{"t":"build_feedback()","u":"api/grader.feedback.html#grader.feedback.build_feedback","s":"grader.feedback","k":"function","d":"Render one student's feedback as markdown."},{"t":"summary_line()","u":"api/grader.feedback.html#grader.feedback.summary_line","s":"grader.feedback","k":"function","d":"One-line status used in tables and logs."},{"t":"build_class_report()","u":"api/grader.feedback.html#grader.feedback.build_class_report","s":"grader.feedback","k":"function","d":"Markdown version of the class overview, for pasting into a course email."},{"t":"grader.discovery","u":"api/grader.discovery.html","s":"Module","k":"module","d":"Find student submissions in a folder or a Canvas export ZIP (design.md §16-17)."},{"t":"SubmissionCandidate","u":"api/grader.discovery.html#grader.discovery.SubmissionCandidate","s":"grader.discovery","k":"class","d":"One student's submission as found on disk."},{"t":"SubmissionCandidate.ok","u":"api/grader.discovery.html#grader.discovery.SubmissionCandidate.ok","s":"grader.discovery","k":"property","d":"True when a notebook was chosen for this submission."},{"t":"SubmissionCandidate.needs_review","u":"api/grader.discovery.html#grader.discovery.SubmissionCandidate.needs_review","s":"grader.discovery","k":"property"},{"t":"SubmissionCandidate.summary","u":"api/grader.discovery.html#grader.discovery.SubmissionCandidate.summary","s":"grader.discovery","k":"method"},{"t":"choose_notebook()","u":"api/grader.discovery.html#grader.discovery.choose_notebook","s":"grader.discovery","k":"function","d":"Return the most plausible notebook and any review reasons."},{"t":"find_notebooks()","u":"api/grader.discovery.html#grader.discovery.find_notebooks","s":"grader.discovery","k":"function"},{"t":"find_data_files()","u":"api/grader.discovery.html#grader.discovery.find_data_files","s":"grader.discovery","k":"function"},{"t":"extract_zip()","u":"api/grader.discovery.html#grader.discovery.extract_zip","s":"grader.discovery","k":"function","d":"Extract a Canvas export, refusing path traversal entries."},{"t":"parse_canvas_filename()","u":"api/grader.discovery.html#grader.discovery.parse_canvas_filename","s":"grader.discovery","k":"function","d":"Return (student_id, display_name, late) for a flat Canvas filename."},{"t":"discover_submissions()","u":"api/grader.discovery.html#grader.discovery.discover_submissions","s":"grader.discovery","k":"function","d":"Discover submissions under root, handling both supported layouts."},{"t":"place_shared_data()","u":"api/grader.discovery.html#grader.discovery.place_shared_data","s":"grader.discovery","k":"function","d":"Put instructor-supplied data where a notebook expects to find it."},{"t":"prepare_workdir()","u":"api/grader.discovery.html#grader.discovery.prepare_workdir","s":"grader.discovery","k":"function","d":"Copy one submission into an isolated working directory (design.md §17)."},{"t":"NOTEBOOK_SUFFIX","u":"api/grader.discovery.html#grader.discovery.NOTEBOOK_SUFFIX","s":"grader.discovery","k":"constant"},{"t":"DATA_SUFFIXES","u":"api/grader.discovery.html#grader.discovery.DATA_SUFFIXES","s":"grader.discovery","k":"constant"},{"t":"SKIP_DIRS","u":"api/grader.discovery.html#grader.discovery.SKIP_DIRS","s":"grader.discovery","k":"constant"},{"t":"CANVAS_RE","u":"api/grader.discovery.html#grader.discovery.CANVAS_RE","s":"grader.discovery","k":"constant"},{"t":"REASON_MULTIPLE_NOTEBOOKS","u":"api/grader.discovery.html#grader.discovery.REASON_MULTIPLE_NOTEBOOKS","s":"grader.discovery","k":"constant"},{"t":"REASON_NO_NOTEBOOK","u":"api/grader.discovery.html#grader.discovery.REASON_NO_NOTEBOOK","s":"grader.discovery","k":"constant"},{"t":"grader.executor","u":"api/grader.executor.html","s":"Module","k":"module","d":"Notebook execution (design.md §18-19)."},{"t":"ExecutionConfig","u":"api/grader.executor.html#grader.executor.ExecutionConfig","s":"grader.executor","k":"class","d":"Limits for one notebook execution, derived from GraderSettings."},{"t":"ExecutionConfig.hard_timeout","u":"api/grader.executor.html#grader.executor.ExecutionConfig.hard_timeout","s":"grader.executor","k":"property"},{"t":"ExecutionError","u":"api/grader.executor.html#grader.executor.ExecutionError","s":"grader.executor","k":"class"},{"t":"build_probe_cells()","u":"api/grader.executor.html#grader.executor.build_probe_cells","s":"grader.executor","k":"function","d":"Return the notebook cells that dump the student's namespace to JSON."},{"t":"build_url_redirect_cell()","u":"api/grader.executor.html#grader.executor.build_url_redirect_cell","s":"grader.executor","k":"function","d":"A first cell that points data URLs at the supplied local file."},{"t":"instrument_notebook()","u":"api/grader.executor.html#grader.executor.instrument_notebook","s":"grader.executor","k":"function","d":"Append probe cells (and, if needed, prepend URL redirects) to a copy."},{"t":"docker_available()","u":"api/grader.executor.html#grader.executor.docker_available","s":"grader.executor","k":"function","d":"Whether a Docker daemon answers on this machine."},{"t":"docker_image_exists()","u":"api/grader.executor.html#grader.executor.docker_image_exists","s":"grader.executor","k":"function","d":"Whether the sandbox image has been built locally."},{"t":"execute_submission()","u":"api/grader.executor.html#grader.executor.execute_submission","s":"grader.executor","k":"function","d":"Execute one prepared submission. Returns (execution record, probe payload)."},{"t":"strip_probe_cells()","u":"api/grader.executor.html#grader.executor.strip_probe_cells","s":"grader.executor","k":"function","d":"Remove the injected cells so the saved notebook is the student's work."},{"t":"sandbox_isolation_flags()","u":"api/grader.executor.html#grader.executor.sandbox_isolation_flags","s":"grader.executor","k":"function","d":"docker run flags every sandboxed run gets (design.md §18)."},{"t":"MODE_DOCKER","u":"api/grader.executor.html#grader.executor.MODE_DOCKER","s":"grader.executor","k":"constant"},{"t":"MODE_LOCAL","u":"api/grader.executor.html#grader.executor.MODE_LOCAL","s":"grader.executor","k":"constant"},{"t":"MODE_NOT_EXECUTED","u":"api/grader.executor.html#grader.executor.MODE_NOT_EXECUTED","s":"grader.executor","k":"constant"},{"t":"PACKAGE_DIR","u":"api/grader.executor.html#grader.executor.PACKAGE_DIR","s":"grader.executor","k":"constant"},{"t":"RUNNER_SOURCE","u":"api/grader.executor.html#grader.executor.RUNNER_SOURCE","s":"grader.executor","k":"constant"},{"t":"PROBE_SOURCE","u":"api/grader.executor.html#grader.executor.PROBE_SOURCE","s":"grader.executor","k":"constant"},{"t":"INSTRUMENTED_NAME","u":"api/grader.executor.html#grader.executor.INSTRUMENTED_NAME","s":"grader.executor","k":"constant"},{"t":"RUNNER_NAME","u":"api/grader.executor.html#grader.executor.RUNNER_NAME","s":"grader.executor","k":"constant"},{"t":"ARTIFACTS_DIRNAME","u":"api/grader.executor.html#grader.executor.ARTIFACTS_DIRNAME","s":"grader.executor","k":"constant"},{"t":"DEFAULT_IMAGE","u":"api/grader.executor.html#grader.executor.DEFAULT_IMAGE","s":"grader.executor","k":"constant"},{"t":"URL_REDIRECT_TEMPLATE","u":"api/grader.executor.html#grader.executor.URL_REDIRECT_TEMPLATE","s":"grader.executor","k":"constant"},{"t":"grader.notebook_runner","u":"api/grader.notebook_runner.html","s":"Module","k":"module","d":"Standalone notebook runner. Copied into the working directory and executed as a subprocess — locally with the host interpreter, or inside the grading container."},{"t":"main()","u":"api/grader.notebook_runner.html#grader.notebook_runner.main","s":"grader.notebook_runner","k":"function"},{"t":"grader.probe_runtime","u":"api/grader.probe_runtime.html","s":"Module","k":"module","d":"Code injected into the student's kernel after their notebook runs."},{"t":"MUSA_PROBE_MARKER","u":"api/grader.probe_runtime.html#grader.probe_runtime.MUSA_PROBE_MARKER","s":"grader.probe_runtime","k":"constant"},{"t":"grader.function_sandbox","u":"api/grader.function_sandbox.html","s":"Module","k":"module","d":"Run a few named functions from a notebook, and nothing else, in the sandbox."},{"t":"ExtractedFunctions","u":"api/grader.function_sandbox.html#grader.function_sandbox.ExtractedFunctions","s":"grader.function_sandbox","k":"class","d":"What was cut out of the notebook, and what was refused."},{"t":"ExtractedFunctions.to_dict","u":"api/grader.function_sandbox.html#grader.function_sandbox.ExtractedFunctions.to_dict","s":"grader.function_sandbox","k":"method"},{"t":"FunctionTestRun","u":"api/grader.function_sandbox.html#grader.function_sandbox.FunctionTestRun","s":"grader.function_sandbox","k":"class"},{"t":"FunctionTestRun.case","u":"api/grader.function_sandbox.html#grader.function_sandbox.FunctionTestRun.case","s":"grader.function_sandbox","k":"method"},{"t":"FunctionTestRun.to_dict","u":"api/grader.function_sandbox.html#grader.function_sandbox.FunctionTestRun.to_dict","s":"grader.function_sandbox","k":"method"},{"t":"extract_functions()","u":"api/grader.function_sandbox.html#grader.function_sandbox.extract_functions","s":"grader.function_sandbox","k":"function","d":"Cut the named top-level functions out of a notebook's code cells."},{"t":"build_command()","u":"api/grader.function_sandbox.html#grader.function_sandbox.build_command","s":"grader.function_sandbox","k":"function"},{"t":"run_isolated_functions()","u":"api/grader.function_sandbox.html#grader.function_sandbox.run_isolated_functions","s":"grader.function_sandbox","k":"function","d":"Call each case's function with its arguments, in the sandbox."},{"t":"matches()","u":"api/grader.function_sandbox.html#grader.function_sandbox.matches","s":"grader.function_sandbox","k":"function","d":"Did a call return this number? Any numeric type will do: 2, 2.0, np.int64(2)."},{"t":"RUNNER_SOURCE","u":"api/grader.function_sandbox.html#grader.function_sandbox.RUNNER_SOURCE","s":"grader.function_sandbox","k":"constant"},{"t":"SANDBOX_DIRNAME","u":"api/grader.function_sandbox.html#grader.function_sandbox.SANDBOX_DIRNAME","s":"grader.function_sandbox","k":"constant"},{"t":"RUNNER_NAME","u":"api/grader.function_sandbox.html#grader.function_sandbox.RUNNER_NAME","s":"grader.function_sandbox","k":"constant"},{"t":"DEFAULT_ALLOWED_IMPORTS","u":"api/grader.function_sandbox.html#grader.function_sandbox.DEFAULT_ALLOWED_IMPORTS","s":"grader.function_sandbox","k":"constant"},{"t":"STATUS_RAN","u":"api/grader.function_sandbox.html#grader.function_sandbox.STATUS_RAN","s":"grader.function_sandbox","k":"constant"},{"t":"STATUS_SKIPPED","u":"api/grader.function_sandbox.html#grader.function_sandbox.STATUS_SKIPPED","s":"grader.function_sandbox","k":"constant"},{"t":"STATUS_ERROR","u":"api/grader.function_sandbox.html#grader.function_sandbox.STATUS_ERROR","s":"grader.function_sandbox","k":"constant"},{"t":"grader.function_runner","u":"api/grader.function_runner.html","s":"Module","k":"module","d":"Standalone runner for isolated function tests. Copied into a fresh, otherwise empty directory and executed inside the network-less grading container by ``grader"},{"t":"CaseTimeout","u":"api/grader.function_runner.html#grader.function_runner.CaseTimeout","s":"grader.function_runner","k":"class"},{"t":"main()","u":"api/grader.function_runner.html#grader.function_runner.main","s":"grader.function_runner","k":"function"},{"t":"grader.notebook","u":"api/grader.notebook.html","s":"Module","k":"module","d":"Static analysis of a student notebook (design.md §23, structural grading)."},{"t":"NotebookAnalysis","u":"api/grader.notebook.html#grader.notebook.NotebookAnalysis","s":"grader.notebook","k":"class","d":"Static facts about a notebook."},{"t":"NotebookAnalysis.function_names","u":"api/grader.notebook.html#grader.notebook.NotebookAnalysis.function_names","s":"grader.notebook","k":"method"},{"t":"NotebookAnalysis.to_dict","u":"api/grader.notebook.html#grader.notebook.NotebookAnalysis.to_dict","s":"grader.notebook","k":"method"},{"t":"empty_analysis()","u":"api/grader.notebook.html#grader.notebook.empty_analysis","s":"grader.notebook","k":"function","d":"Placeholder used when no notebook was submitted at all."},{"t":"read_notebook()","u":"api/grader.notebook.html#grader.notebook.read_notebook","s":"grader.notebook","k":"function"},{"t":"analyze_notebook()","u":"api/grader.notebook.html#grader.notebook.analyze_notebook","s":"grader.notebook","k":"function"},{"t":"extract_markdown_responses()","u":"api/grader.notebook.html#grader.notebook.extract_markdown_responses","s":"grader.notebook","k":"function","d":"Find the most substantial free-text markdown answer in the notebook."},{"t":"render_cells()","u":"api/grader.notebook.html#grader.notebook.render_cells","s":"grader.notebook","k":"function","d":"Flatten a notebook into a read-only structure for the UI (design.md §13)."},{"t":"ABSOLUTE_PATH_RE","u":"api/grader.notebook.html#grader.notebook.ABSOLUTE_PATH_RE","s":"grader.notebook","k":"constant"},{"t":"DATA_SUFFIXES","u":"api/grader.notebook.html#grader.notebook.DATA_SUFFIXES","s":"grader.notebook","k":"constant"},{"t":"READ_FUNCS","u":"api/grader.notebook.html#grader.notebook.READ_FUNCS","s":"grader.notebook","k":"constant"},{"t":"PLOT_HINTS","u":"api/grader.notebook.html#grader.notebook.PLOT_HINTS","s":"grader.notebook","k":"constant"},{"t":"IPYTHON_LINE_PREFIXES","u":"api/grader.notebook.html#grader.notebook.IPYTHON_LINE_PREFIXES","s":"grader.notebook","k":"constant"},{"t":"IMPORT_RE","u":"api/grader.notebook.html#grader.notebook.IMPORT_RE","s":"grader.notebook","k":"constant"},{"t":"READ_CALL_RE","u":"api/grader.notebook.html#grader.notebook.READ_CALL_RE","s":"grader.notebook","k":"constant"},{"t":"STRING_LITERAL_RE","u":"api/grader.notebook.html#grader.notebook.STRING_LITERAL_RE","s":"grader.notebook","k":"constant"},{"t":"INSTRUCTION_RE","u":"api/grader.notebook.html#grader.notebook.INSTRUCTION_RE","s":"grader.notebook","k":"constant"},{"t":"grader.inspection","u":"api/grader.inspection.html","s":"Module","k":"module","d":"Host-side reasoning over the probe payload."},{"t":"confidence_for_candidates()","u":"api/grader.inspection.html#grader.inspection.confidence_for_candidates","s":"grader.inspection","k":"function"},{"t":"dataframes()","u":"api/grader.inspection.html#grader.inspection.dataframes","s":"grader.inspection","k":"function"},{"t":"scalars()","u":"api/grader.inspection.html#grader.inspection.scalars","s":"grader.inspection","k":"function"},{"t":"collections()","u":"api/grader.inspection.html#grader.inspection.collections","s":"grader.inspection","k":"function"},{"t":"hidden_test()","u":"api/grader.inspection.html#grader.inspection.hidden_test","s":"grader.inspection","k":"function"},{"t":"match_column()","u":"api/grader.inspection.html#grader.inspection.match_column","s":"grader.inspection","k":"function","d":"Best column in profile matching any of hints (substring, case-insensitive)."},{"t":"has_datelike_column()","u":"api/grader.inspection.html#grader.inspection.has_datelike_column","s":"grader.inspection","k":"function","d":"A date role can be satisfied by a column name hint or a datetime dtype."},{"t":"numeric_columns()","u":"api/grader.inspection.html#grader.inspection.numeric_columns","s":"grader.inspection","k":"function"},{"t":"zips_in_profile()","u":"api/grader.inspection.html#grader.inspection.zips_in_profile","s":"grader.inspection","k":"function","d":"All ZIP-looking values visible in a dataframe profile."},{"t":"zips_in_collection()","u":"api/grader.inspection.html#grader.inspection.zips_in_collection","s":"grader.inspection","k":"function"},{"t":"find_dataframe_candidates()","u":"api/grader.inspection.html#grader.inspection.find_dataframe_candidates","s":"grader.inspection","k":"function","d":"Return profiles satisfying predicate, best first."},{"t":"find_scalar_close_to()","u":"api/grader.inspection.html#grader.inspection.find_scalar_close_to","s":"grader.inspection","k":"function","d":"Scalars in the namespace within tolerance of the expected answer."},{"t":"describe_dataframe()","u":"api/grader.inspection.html#grader.inspection.describe_dataframe","s":"grader.inspection","k":"function","d":"Compact, TA-readable description used as rubric evidence (design.md §12)."},{"t":"CONFIDENCE_UNIQUE","u":"api/grader.inspection.html#grader.inspection.CONFIDENCE_UNIQUE","s":"grader.inspection","k":"constant"},{"t":"CONFIDENCE_TWO","u":"api/grader.inspection.html#grader.inspection.CONFIDENCE_TWO","s":"grader.inspection","k":"constant"},{"t":"CONFIDENCE_MANY","u":"api/grader.inspection.html#grader.inspection.CONFIDENCE_MANY","s":"grader.inspection","k":"constant"},{"t":"CONFIDENCE_NONE","u":"api/grader.inspection.html#grader.inspection.CONFIDENCE_NONE","s":"grader.inspection","k":"constant"},{"t":"grader.answers","u":"api/grader.answers.html","s":"Module","k":"module","d":"Answers read out of a notebook's own saved outputs (design.md §23, §25)."},{"t":"HoloviewsRepr","u":"api/grader.answers.html#grader.answers.HoloviewsRepr","s":"grader.answers","k":"class","d":"One holoviews object, as its own repr describes it."},{"t":"HoloviewsRepr.is_widget","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.is_widget","s":"grader.answers","k":"property"},{"t":"HoloviewsRepr.is_lazy","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.is_lazy","s":"grader.answers","k":"property","d":"A DynamicMap: the widget exists but no frame was ever computed."},{"t":"HoloviewsRepr.is_layout","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.is_layout","s":"grader.answers","k":"property"},{"t":"HoloviewsRepr.element_types","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.element_types","s":"grader.answers","k":"property"},{"t":"HoloviewsRepr.summary","u":"api/grader.answers.html#grader.answers.HoloviewsRepr.summary","s":"grader.answers","k":"method"},{"t":"CellAnswers","u":"api/grader.answers.html#grader.answers.CellAnswers","s":"grader.answers","k":"class","d":"Everything one code cell's saved output states."},{"t":"CellAnswers.has_output","u":"api/grader.answers.html#grader.answers.CellAnswers.has_output","s":"grader.answers","k":"property"},{"t":"CellAnswers.has_answer","u":"api/grader.answers.html#grader.answers.CellAnswers.has_answer","s":"grader.answers","k":"property","d":"Output that states something, as opposed to a traceback."},{"t":"NotebookAnswers","u":"api/grader.answers.html#grader.answers.NotebookAnswers","s":"grader.answers","k":"class","d":"Every answer a notebook shows, with where each one was seen."},{"t":"NotebookAnswers.shows","u":"api/grader.answers.html#grader.answers.NotebookAnswers.shows","s":"grader.answers","k":"method","d":"The first cell that printed or returned this number."},{"t":"NotebookAnswers.shows_all","u":"api/grader.answers.html#grader.answers.NotebookAnswers.shows_all","s":"grader.answers","k":"method"},{"t":"NotebookAnswers.shows_together","u":"api/grader.answers.html#grader.answers.NotebookAnswers.shows_together","s":"grader.answers","k":"method","d":"A single cell that shows all of these numbers at once."},{"t":"NotebookAnswers.shows_rows","u":"api/grader.answers.html#grader.answers.NotebookAnswers.shows_rows","s":"grader.answers","k":"method","d":"A frame of this many rows, from a repr footer, .shape or .info()."},{"t":"NotebookAnswers.frames_with_columns","u":"api/grader.answers.html#grader.answers.NotebookAnswers.frames_with_columns","s":"grader.answers","k":"method"},{"t":"NotebookAnswers.describes","u":"api/grader.answers.html#grader.answers.NotebookAnswers.describes","s":"grader.answers","k":"method"},{"t":"NotebookAnswers.charts","u":"api/grader.answers.html#grader.answers.NotebookAnswers.charts","s":"grader.answers","k":"method"},{"t":"NotebookAnswers.figures","u":"api/grader.answers.html#grader.answers.NotebookAnswers.figures","s":"grader.answers","k":"property"},{"t":"NotebookAnswers.error_cells","u":"api/grader.answers.html#grader.answers.NotebookAnswers.error_cells","s":"grader.answers","k":"property"},{"t":"NotebookAnswers.cells_with_output","u":"api/grader.answers.html#grader.answers.NotebookAnswers.cells_with_output","s":"grader.answers","k":"property"},{"t":"NotebookAnswers.cells_with_answers","u":"api/grader.answers.html#grader.answers.NotebookAnswers.cells_with_answers","s":"grader.answers","k":"property"},{"t":"NotebookAnswers.note","u":"api/grader.answers.html#grader.answers.NotebookAnswers.note","s":"grader.answers","k":"method"},{"t":"read_answers()","u":"api/grader.answers.html#grader.answers.read_answers","s":"grader.answers","k":"function","d":"Harvest one notebook's saved outputs."},{"t":"collect_answers()","u":"api/grader.answers.html#grader.answers.collect_answers","s":"grader.answers","k":"function","d":"Answers from both notebooks, merged cell by cell."},{"t":"SOURCE_EXECUTED","u":"api/grader.answers.html#grader.answers.SOURCE_EXECUTED","s":"grader.answers","k":"constant"},{"t":"SOURCE_SUBMITTED","u":"api/grader.answers.html#grader.answers.SOURCE_SUBMITTED","s":"grader.answers","k":"constant"},{"t":"SOURCE_MERGED","u":"api/grader.answers.html#grader.answers.SOURCE_MERGED","s":"grader.answers","k":"constant"},{"t":"SOURCE_NONE","u":"api/grader.answers.html#grader.answers.SOURCE_NONE","s":"grader.answers","k":"constant"},{"t":"MAX_SCALAR_CHARS","u":"api/grader.answers.html#grader.answers.MAX_SCALAR_CHARS","s":"grader.answers","k":"constant"},{"t":"NUMBER_RE","u":"api/grader.answers.html#grader.answers.NUMBER_RE","s":"grader.answers","k":"constant"},{"t":"SHAPE_TEXT_RE","u":"api/grader.answers.html#grader.answers.SHAPE_TEXT_RE","s":"grader.answers","k":"constant"},{"t":"SHAPE_HTML_RE","u":"api/grader.answers.html#grader.answers.SHAPE_HTML_RE","s":"grader.answers","k":"constant"},{"t":"ENTRIES_RE","u":"api/grader.answers.html#grader.answers.ENTRIES_RE","s":"grader.answers","k":"constant"},{"t":"INFO_COLUMN_RE","u":"api/grader.answers.html#grader.answers.INFO_COLUMN_RE","s":"grader.answers","k":"constant"},{"t":"DESCRIBE_RE","u":"api/grader.answers.html#grader.answers.DESCRIBE_RE","s":"grader.answers","k":"constant"},{"t":"FIGURE_RE","u":"api/grader.answers.html#grader.answers.FIGURE_RE","s":"grader.answers","k":"constant"},{"t":"HTML_TABLE_RE","u":"api/grader.answers.html#grader.answers.HTML_TABLE_RE","s":"grader.answers","k":"constant"},{"t":"HTML_HEADER_RE","u":"api/grader.answers.html#grader.answers.HTML_HEADER_RE","s":"grader.answers","k":"constant"},{"t":"HOLOVIEWS_LINE_RE","u":"api/grader.answers.html#grader.answers.HOLOVIEWS_LINE_RE","s":"grader.answers","k":"constant"},{"t":"WIDGET_RE","u":"api/grader.answers.html#grader.answers.WIDGET_RE","s":"grader.answers","k":"constant"},{"t":"QUOTED_RE","u":"api/grader.answers.html#grader.answers.QUOTED_RE","s":"grader.answers","k":"constant"},{"t":"NDARRAY_RE","u":"api/grader.answers.html#grader.answers.NDARRAY_RE","s":"grader.answers","k":"constant"},{"t":"STRUCT_CODES","u":"api/grader.answers.html#grader.answers.STRUCT_CODES","s":"grader.answers","k":"constant"},{"t":"IMAGE_MIMES","u":"api/grader.answers.html#grader.answers.IMAGE_MIMES","s":"grader.answers","k":"constant"},{"t":"WIDGET_KINDS","u":"api/grader.answers.html#grader.answers.WIDGET_KINDS","s":"grader.answers","k":"constant"},{"t":"LAZY_KINDS","u":"api/grader.answers.html#grader.answers.LAZY_KINDS","s":"grader.answers","k":"constant"},{"t":"LAYOUT_KINDS","u":"api/grader.answers.html#grader.answers.LAYOUT_KINDS","s":"grader.answers","k":"constant"},{"t":"grader.charts","u":"api/grader.charts.html","s":"Module","k":"module","d":"Chart evidence for visualization assignments (design.md §23, §25)."},{"t":"SpecFacts","u":"api/grader.charts.html#grader.charts.SpecFacts","s":"grader.charts","k":"class","d":"What one Vega-Lite specification says the chart does."},{"t":"SpecFacts.brushes","u":"api/grader.charts.html#grader.charts.SpecFacts.brushes","s":"grader.charts","k":"property","d":"Interval selections a viewer can drag — pan/zoom bindings excluded."},{"t":"SpecFacts.has_brush","u":"api/grader.charts.html#grader.charts.SpecFacts.has_brush","s":"grader.charts","k":"property"},{"t":"SpecFacts.has_transformation","u":"api/grader.charts.html#grader.charts.SpecFacts.has_transformation","s":"grader.charts","k":"property"},{"t":"SpecFacts.is_multi_view","u":"api/grader.charts.html#grader.charts.SpecFacts.is_multi_view","s":"grader.charts","k":"property"},{"t":"SpecFacts.cross_filter_params","u":"api/grader.charts.html#grader.charts.SpecFacts.cross_filter_params","s":"grader.charts","k":"property","d":"Selections that a *different* view filters on — a real cross-filter."},{"t":"SpecFacts.has_cross_filter","u":"api/grader.charts.html#grader.charts.SpecFacts.has_cross_filter","s":"grader.charts","k":"property"},{"t":"SpecFacts.to_dict","u":"api/grader.charts.html#grader.charts.SpecFacts.to_dict","s":"grader.charts","k":"method"},{"t":"ChartRecord","u":"api/grader.charts.html#grader.charts.ChartRecord","s":"grader.charts","k":"class","d":"One chart, as drawn by one cell."},{"t":"ChartRecord.ok","u":"api/grader.charts.html#grader.charts.ChartRecord.ok","s":"grader.charts","k":"property","d":"Drew something, and the cell that drew it did not fail."},{"t":"ChartRecord.has_discussion","u":"api/grader.charts.html#grader.charts.ChartRecord.has_discussion","s":"grader.charts","k":"property"},{"t":"ChartRecord.summary","u":"api/grader.charts.html#grader.charts.ChartRecord.summary","s":"grader.charts","k":"method","d":"Compact, TA-readable evidence (design.md §12)."},{"t":"ChartEvidence","u":"api/grader.charts.html#grader.charts.ChartEvidence","s":"grader.charts","k":"class","d":"Every chart found in a submission, in the order the cells appear."},{"t":"ChartEvidence.from_execution","u":"api/grader.charts.html#grader.charts.ChartEvidence.from_execution","s":"grader.charts","k":"property","d":"Every chart that came out was produced by this grading run."},{"t":"ChartEvidence.verified","u":"api/grader.charts.html#grader.charts.ChartEvidence.verified","s":"grader.charts","k":"method","d":"Were these particular charts produced here, rather than by the student?"},{"t":"ChartEvidence.by_library","u":"api/grader.charts.html#grader.charts.ChartEvidence.by_library","s":"grader.charts","k":"method"},{"t":"ChartEvidence.rendered","u":"api/grader.charts.html#grader.charts.ChartEvidence.rendered","s":"grader.charts","k":"method"},{"t":"ChartEvidence.summary","u":"api/grader.charts.html#grader.charts.ChartEvidence.summary","s":"grader.charts","k":"method"},{"t":"libraries_in_source()","u":"api/grader.charts.html#grader.charts.libraries_in_source","s":"grader.charts","k":"function","d":"Which plotting libraries this cell draws with, most specific first."},{"t":"extract_specs()","u":"api/grader.charts.html#grader.charts.extract_specs","s":"grader.charts","k":"function","d":"Vega-Lite specifications carried by one notebook output."},{"t":"strip_spec_data()","u":"api/grader.charts.html#grader.charts.strip_spec_data","s":"grader.charts","k":"function","d":"Drop inlined rows so a spec can be stored as rubric evidence."},{"t":"analyze_spec()","u":"api/grader.charts.html#grader.charts.analyze_spec","s":"grader.charts","k":"function","d":"Read one Vega-Lite spec, at any level of composition."},{"t":"merge_facts()","u":"api/grader.charts.html#grader.charts.merge_facts","s":"grader.charts","k":"function","d":"One view of a cell that displayed more than one chart."},{"t":"aesthetic_signals()","u":"api/grader.charts.html#grader.charts.aesthetic_signals","s":"grader.charts","k":"function","d":"Presence of the labelling and colour choices a TA would look for."},{"t":"collect_charts()","u":"api/grader.charts.html#grader.charts.collect_charts","s":"grader.charts","k":"function","d":"Chart evidence, taking each chart from whichever notebook drew it."},{"t":"prose_word_count()","u":"api/grader.charts.html#grader.charts.prose_word_count","s":"grader.charts","k":"function","d":"Words of actual discussion — headings and placeholders do not count."},{"t":"MATPLOTLIB","u":"api/grader.charts.html#grader.charts.MATPLOTLIB","s":"grader.charts","k":"constant"},{"t":"SEABORN","u":"api/grader.charts.html#grader.charts.SEABORN","s":"grader.charts","k":"constant"},{"t":"ALTAIR","u":"api/grader.charts.html#grader.charts.ALTAIR","s":"grader.charts","k":"constant"},{"t":"SOURCE_EXECUTED","u":"api/grader.charts.html#grader.charts.SOURCE_EXECUTED","s":"grader.charts","k":"constant"},{"t":"SOURCE_SUBMITTED","u":"api/grader.charts.html#grader.charts.SOURCE_SUBMITTED","s":"grader.charts","k":"constant"},{"t":"SOURCE_MERGED","u":"api/grader.charts.html#grader.charts.SOURCE_MERGED","s":"grader.charts","k":"constant"},{"t":"SOURCE_NONE","u":"api/grader.charts.html#grader.charts.SOURCE_NONE","s":"grader.charts","k":"constant"},{"t":"SEABORN_NON_PLOTTING","u":"api/grader.charts.html#grader.charts.SEABORN_NON_PLOTTING","s":"grader.charts","k":"constant"},{"t":"SEABORN_PLOT_RE","u":"api/grader.charts.html#grader.charts.SEABORN_PLOT_RE","s":"grader.charts","k":"constant"},{"t":"MATPLOTLIB_PLOT_METHODS","u":"api/grader.charts.html#grader.charts.MATPLOTLIB_PLOT_METHODS","s":"grader.charts","k":"constant"},{"t":"MATPLOTLIB_PLOT_RE","u":"api/grader.charts.html#grader.charts.MATPLOTLIB_PLOT_RE","s":"grader.charts","k":"constant"},{"t":"PANDAS_PLOT_RE","u":"api/grader.charts.html#grader.charts.PANDAS_PLOT_RE","s":"grader.charts","k":"constant"},{"t":"ALTAIR_RE","u":"api/grader.charts.html#grader.charts.ALTAIR_RE","s":"grader.charts","k":"constant"},{"t":"ALTAIR_CHART_RE","u":"api/grader.charts.html#grader.charts.ALTAIR_CHART_RE","s":"grader.charts","k":"constant"},{"t":"VEGALITE_MIME_RE","u":"api/grader.charts.html#grader.charts.VEGALITE_MIME_RE","s":"grader.charts","k":"constant"},{"t":"VEGA_EMBED_MARKER","u":"api/grader.charts.html#grader.charts.VEGA_EMBED_MARKER","s":"grader.charts","k":"constant"},{"t":"COMPOSITION_KEYS","u":"api/grader.charts.html#grader.charts.COMPOSITION_KEYS","s":"grader.charts","k":"constant"},{"t":"TRANSFORM_OPS","u":"api/grader.charts.html#grader.charts.TRANSFORM_OPS","s":"grader.charts","k":"constant"},{"t":"ENCODING_TRANSFORM_KEYS","u":"api/grader.charts.html#grader.charts.ENCODING_TRANSFORM_KEYS","s":"grader.charts","k":"constant"},{"t":"AESTHETIC_PATTERNS","u":"api/grader.charts.html#grader.charts.AESTHETIC_PATTERNS","s":"grader.charts","k":"constant"},{"t":"IMAGE_MIMES","u":"api/grader.charts.html#grader.charts.IMAGE_MIMES","s":"grader.charts","k":"constant"},{"t":"ALTAIR_REPR_RE","u":"api/grader.charts.html#grader.charts.ALTAIR_REPR_RE","s":"grader.charts","k":"constant"},{"t":"HEADING_RE","u":"api/grader.charts.html#grader.charts.HEADING_RE","s":"grader.charts","k":"constant"},{"t":"PLACEHOLDER_RE","u":"api/grader.charts.html#grader.charts.PLACEHOLDER_RE","s":"grader.charts","k":"constant"},{"t":"grader.llm","u":"api/grader.llm.html","s":"Module","k":"module","d":"Optional LLM-assisted qualitative grading (design.md §26)."},{"t":"QualitativeGrader","u":"api/grader.llm.html#grader.llm.QualitativeGrader","s":"grader.llm","k":"class","d":"Interface the assignment graders depend on."},{"t":"QualitativeGrader.grade","u":"api/grader.llm.html#grader.llm.QualitativeGrader.grade","s":"grader.llm","k":"method"},{"t":"DisabledQualitativeGrader","u":"api/grader.llm.html#grader.llm.DisabledQualitativeGrader","s":"grader.llm","k":"class","d":"The default. Qualitative items stay with the human."},{"t":"DisabledQualitativeGrader.grade","u":"api/grader.llm.html#grader.llm.DisabledQualitativeGrader.grade","s":"grader.llm","k":"method"},{"t":"LLMQualitativeGrader","u":"api/grader.llm.html#grader.llm.LLMQualitativeGrader","s":"grader.llm","k":"class","d":"Provider-independent grader."},{"t":"LLMQualitativeGrader.grade","u":"api/grader.llm.html#grader.llm.LLMQualitativeGrader.grade","s":"grader.llm","k":"method"},{"t":"anthropic_grader()","u":"api/grader.llm.html#grader.llm.anthropic_grader","s":"grader.llm","k":"function","d":"Anthropic-backed grader, or a disabled one when it cannot be set up."},{"t":"build_grader()","u":"api/grader.llm.html#grader.llm.build_grader","s":"grader.llm","k":"function","d":"Construct a grader from UI settings."},{"t":"DEFAULT_REVIEW_THRESHOLD","u":"api/grader.llm.html#grader.llm.DEFAULT_REVIEW_THRESHOLD","s":"grader.llm","k":"constant"},{"t":"SYSTEM_PROMPT","u":"api/grader.llm.html#grader.llm.SYSTEM_PROMPT","s":"grader.llm","k":"constant"},{"t":"USER_TEMPLATE","u":"api/grader.llm.html#grader.llm.USER_TEMPLATE","s":"grader.llm","k":"constant"},{"t":"grader.utils","u":"api/grader.utils.html","s":"Module","k":"module","d":"Small shared helpers for the grading engine."},{"t":"utc_now_iso()","u":"api/grader.utils.html#grader.utils.utc_now_iso","s":"grader.utils","k":"function"},{"t":"slugify()","u":"api/grader.utils.html#grader.utils.slugify","s":"grader.utils","k":"function","d":"Filesystem-safe identifier."},{"t":"json_default()","u":"api/grader.utils.html#grader.utils.json_default","s":"grader.utils","k":"function","d":"Best-effort JSON encoder for numpy / pandas / arbitrary objects."},{"t":"dump_json()","u":"api/grader.utils.html#grader.utils.dump_json","s":"grader.utils","k":"function"},{"t":"load_json()","u":"api/grader.utils.html#grader.utils.load_json","s":"grader.utils","k":"function"},{"t":"is_close()","u":"api/grader.utils.html#grader.utils.is_close","s":"grader.utils","k":"function"},{"t":"as_float()","u":"api/grader.utils.html#grader.utils.as_float","s":"grader.utils","k":"function"},{"t":"normalize_zips()","u":"api/grader.utils.html#grader.utils.normalize_zips","s":"grader.utils","k":"function","d":"Coerce mixed zip representations (int, float, str, padded) to 5-char strings."},{"t":"jaccard()","u":"api/grader.utils.html#grader.utils.jaccard","s":"grader.utils","k":"function"},{"t":"truncate()","u":"api/grader.utils.html#grader.utils.truncate","s":"grader.utils","k":"function"},{"t":"pluralize()","u":"api/grader.utils.html#grader.utils.pluralize","s":"grader.utils","k":"function"},{"t":"ZIP_RE","u":"api/grader.utils.html#grader.utils.ZIP_RE","s":"grader.utils","k":"constant"},{"t":"assignments","u":"api/assignments.html","s":"Module","k":"module","d":"Per-assignment graders. Importing the package registers every grader."},{"t":"assignments.base","u":"api/assignments.base.html","s":"Module","k":"module","d":"Assignment grader base class and registry."},{"t":"GradingContext","u":"api/assignments.base.html#assignments.base.GradingContext","s":"assignments.base","k":"class","d":"Everything a check may look at."},{"t":"GradingContext.probe_ok","u":"api/assignments.base.html#assignments.base.GradingContext.probe_ok","s":"assignments.base","k":"property"},{"t":"AssignmentGrader","u":"api/assignments.base.html#assignments.base.AssignmentGrader","s":"assignments.base","k":"class","d":"Base grader: dispatches rubric ids to check_ methods."},{"t":"AssignmentGrader.probe_config","u":"api/assignments.base.html#assignments.base.AssignmentGrader.probe_config","s":"assignments.base","k":"method","d":"Config handed to the in-kernel probe (hidden tests included)."},{"t":"AssignmentGrader.hidden_test_specs","u":"api/assignments.base.html#assignments.base.AssignmentGrader.hidden_test_specs","s":"assignments.base","k":"method","d":"Only selectors and call inputs — expected values stay on the host."},{"t":"AssignmentGrader.isolated_test_items","u":"api/assignments.base.html#assignments.base.AssignmentGrader.isolated_test_items","s":"assignments.base","k":"method"},{"t":"AssignmentGrader.run_isolated_tests","u":"api/assignments.base.html#assignments.base.AssignmentGrader.run_isolated_tests","s":"assignments.base","k":"method","d":"Call the functions a type: isolated_functions item names."},{"t":"AssignmentGrader.grade","u":"api/assignments.base.html#assignments.base.AssignmentGrader.grade","s":"assignments.base","k":"method","d":"Run every rubric item's check_ method and return one result per item."},{"t":"AssignmentGrader.result","u":"api/assignments.base.html#assignments.base.AssignmentGrader.result","s":"assignments.base","k":"method","d":"Build a RubricItemResult, rounding the score to the rubric's score_rounding increment and clamping it to [0, points]."},{"t":"AssignmentGrader.check_unimplemented","u":"api/assignments.base.html#assignments.base.AssignmentGrader.check_unimplemented","s":"assignments.base","k":"method"},{"t":"AssignmentGrader.no_probe_result","u":"api/assignments.base.html#assignments.base.AssignmentGrader.no_probe_result","s":"assignments.base","k":"method","d":"Shared response when the kernel never reported its namespace."},{"t":"AssignmentGrader.check_notebook_execution","u":"api/assignments.base.html#assignments.base.AssignmentGrader.check_notebook_execution","s":"assignments.base","k":"method","d":"Shared execution check (design.md §19). Never an automatic zero."},{"t":"register()","u":"api/assignments.base.html#assignments.base.register","s":"assignments.base","k":"function","d":"Class decorator: register an AssignmentGrader subclass under its assignment_id."},{"t":"get_grader()","u":"api/assignments.base.html#assignments.base.get_grader","s":"assignments.base","k":"function","d":"Return the grader for a rubric, falling back to the generic base."},{"t":"registered_assignments()","u":"api/assignments.base.html#assignments.base.registered_assignments","s":"assignments.base","k":"function"},{"t":"ISOLATED_FUNCTIONS","u":"api/assignments.base.html#assignments.base.ISOLATED_FUNCTIONS","s":"assignments.base","k":"constant"},{"t":"assignments.hw1","u":"api/assignments.hw1.html","s":"Module","k":"module","d":"Assignment 1 autograder — \"The Donut Effect for Philadelphia ZIP Codes\"."},{"t":"HW1Grader","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader","s":"assignments.hw1","k":"class"},{"t":"HW1Grader.id_hints","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.id_hints","s":"assignments.hw1","k":"property"},{"t":"HW1Grader.date_hints","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.date_hints","s":"assignments.hw1","k":"property"},{"t":"HW1Grader.value_hints","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.value_hints","s":"assignments.hw1","k":"property"},{"t":"HW1Grader.check_data_loading","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_data_loading","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_philadelphia_subset","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_philadelphia_subset","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_tidy_transformation","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_tidy_transformation","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_center_city_split","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_center_city_split","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_percent_increase_function","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_percent_increase_function","s":"assignments.hw1","k":"method"},{"t":"HW1Grader.check_donut_effect_comparison","u":"api/assignments.hw1.html#assignments.hw1.HW1Grader.check_donut_effect_comparison","s":"assignments.hw1","k":"method"},{"t":"DEFAULT_CENTER_WORDS","u":"api/assignments.hw1.html#assignments.hw1.DEFAULT_CENTER_WORDS","s":"assignments.hw1","k":"constant"},{"t":"DEFAULT_OUTSIDE_WORDS","u":"api/assignments.hw1.html#assignments.hw1.DEFAULT_OUTSIDE_WORDS","s":"assignments.hw1","k":"constant"},{"t":"DATE_COLUMN_RE","u":"api/assignments.hw1.html#assignments.hw1.DATE_COLUMN_RE","s":"assignments.hw1","k":"constant"},{"t":"assignments.hw2","u":"api/assignments.hw2.html","s":"Module","k":"module","d":"Assignment 2 autograder — exploratory visualization with matplotlib, seaborn and altair."},{"t":"HW2Grader","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader","s":"assignments.hw2","k":"class"},{"t":"HW2Grader.evidence","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.evidence","s":"assignments.hw2","k":"method","d":"Charts found in this submission, read once and reused by every check."},{"t":"HW2Grader.required_count","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.required_count","s":"assignments.hw2","k":"method","d":"How many charts of a library the rubric asks for."},{"t":"HW2Grader.evidence_note","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.evidence_note","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.no_evidence_result","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.no_evidence_result","s":"assignments.hw2","k":"method","d":"No notebook could be read for charts — never an automatic zero."},{"t":"HW2Grader.check_notebook_execution","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_notebook_execution","s":"assignments.hw2","k":"method","d":"Did the student run their notebook through before handing it in?"},{"t":"HW2Grader.check_data_loading","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_data_loading","s":"assignments.hw2","k":"method","d":"Is a dataset loaded, and loaded the way the instructions ask?"},{"t":"HW2Grader.check_matplotlib_chart","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_matplotlib_chart","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_seaborn_chart","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_seaborn_chart","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_altair_charts","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_altair_charts","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_altair_transformation","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_altair_transformation","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_altair_brush_selection","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_altair_brush_selection","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_altair_dashboard_extra_credit","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_altair_dashboard_extra_credit","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_matplotlib_rationale","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_matplotlib_rationale","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_seaborn_rationale","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_seaborn_rationale","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_matplotlib_aesthetics","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_matplotlib_aesthetics","s":"assignments.hw2","k":"method"},{"t":"HW2Grader.check_chart_conclusions","u":"api/assignments.hw2.html#assignments.hw2.HW2Grader.check_chart_conclusions","s":"assignments.hw2","k":"method"},{"t":"URL_RE","u":"api/assignments.hw2.html#assignments.hw2.URL_RE","s":"assignments.hw2","k":"constant"},{"t":"LIBRARY_LABELS","u":"api/assignments.hw2.html#assignments.hw2.LIBRARY_LABELS","s":"assignments.hw2","k":"constant"},{"t":"assignments.hw3","u":"api/assignments.hw3.html","s":"Module","k":"module","d":"Assignment 3 autograder — evictions, code violations and the NDVI."},{"t":"HW3Grader","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader","s":"assignments.hw3","k":"class"},{"t":"HW3Grader.answers","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.answers","s":"assignments.hw3","k":"method","d":"What the notebooks show, read once per submission and reused."},{"t":"HW3Grader.grade","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.grade","s":"assignments.hw3","k":"method","d":"Grade on the objects this run produced *and* on what the file shows."},{"t":"HW3Grader.charts","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.charts","s":"assignments.hw3","k":"method","d":"matplotlib figures, read once per submission and reused."},{"t":"HW3Grader.check_eviction_data_loading","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_eviction_data_loading","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_philadelphia_tracts","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_philadelphia_tracts","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tidy_evictions","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tidy_evictions","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_evictions_by_year_plot","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_evictions_by_year_plot","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_evictions_choropleth","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_evictions_choropleth","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_violations_choropleth","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_violations_choropleth","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_violations_loading","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_violations_loading","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_violations_subset","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_violations_subset","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_hexbin_map","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_hexbin_map","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_spatial_join","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_spatial_join","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_violations_by_tract","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_violations_by_tract","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_merge_geometries","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_merge_geometries","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_side_by_side","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_side_by_side","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_landsat_loading","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_landsat_loading","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_city_suburb_polygons","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_city_suburb_polygons","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_ndvi_masking","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_ndvi_masking","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_ndvi_medians","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_ndvi_medians","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tree_data_loading","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tree_data_loading","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tree_ndvi","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tree_ndvi","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tree_plots","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tree_plots","s":"assignments.hw3","k":"method"},{"t":"HW3Grader.check_tree_plot_styling","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_tree_plot_styling","s":"assignments.hw3","k":"method","d":"The one item on this assignment that a person has to look at."},{"t":"HW3Grader.check_common_violations_extra_credit","u":"api/assignments.hw3.html#assignments.hw3.HW3Grader.check_common_violations_extra_credit","s":"assignments.hw3","k":"method"},{"t":"keys_cover()","u":"api/assignments.hw3.html#assignments.hw3.keys_cover","s":"assignments.hw3","k":"function","d":"Every expected key is present, reading \"e-03\", \"2003\" and 2003 as one year."},{"t":"WIDGET_CONTAINERS","u":"api/assignments.hw3.html#assignments.hw3.WIDGET_CONTAINERS","s":"assignments.hw3","k":"constant"},{"t":"LAZY_CONTAINERS","u":"api/assignments.hw3.html#assignments.hw3.LAZY_CONTAINERS","s":"assignments.hw3","k":"constant"},{"t":"LAYOUT_CONTAINERS","u":"api/assignments.hw3.html#assignments.hw3.LAYOUT_CONTAINERS","s":"assignments.hw3","k":"constant"},{"t":"OVERLAY_CONTAINERS","u":"api/assignments.hw3.html#assignments.hw3.OVERLAY_CONTAINERS","s":"assignments.hw3","k":"constant"},{"t":"MAP_ELEMENTS","u":"api/assignments.hw3.html#assignments.hw3.MAP_ELEMENTS","s":"assignments.hw3","k":"constant"},{"t":"YEAR_KEY_RE","u":"api/assignments.hw3.html#assignments.hw3.YEAR_KEY_RE","s":"assignments.hw3","k":"constant"},{"t":"assignments.hw4","u":"api/assignments.hw4.html","s":"Module","k":"module","d":"Assignment 4 autograder — street networks and web scraping."},{"t":"CodeCell","u":"api/assignments.hw4.html#assignments.hw4.CodeCell","s":"assignments.hw4","k":"class"},{"t":"SubmittedCode","u":"api/assignments.hw4.html#assignments.hw4.SubmittedCode","s":"assignments.hw4","k":"class"},{"t":"SubmittedCode.student_text","u":"api/assignments.hw4.html#assignments.hw4.SubmittedCode.student_text","s":"assignments.hw4","k":"property"},{"t":"SubmittedCode.part1","u":"api/assignments.hw4.html#assignments.hw4.SubmittedCode.part1","s":"assignments.hw4","k":"method"},{"t":"SubmittedCode.part2","u":"api/assignments.hw4.html#assignments.hw4.SubmittedCode.part2","s":"assignments.hw4","k":"method"},{"t":"SubmittedCode.text","u":"api/assignments.hw4.html#assignments.hw4.SubmittedCode.text","s":"assignments.hw4","k":"method"},{"t":"HW4Grader","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader","s":"assignments.hw4","k":"class"},{"t":"HW4Grader.template_path","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.template_path","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.code","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.code","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.answers","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.answers","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.raw_outputs","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.raw_outputs","s":"assignments.hw4","k":"method","d":"Saved outputs by code-cell number, for what answers does not keep."},{"t":"HW4Grader.answer_cell","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.answer_cell","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.scored","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.scored","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.run_requirements","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.run_requirements","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.part_cells","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.part_cells","s":"assignments.hw4","k":"method","d":"The cells of the Part this step belongs to (part: in the rubric)."},{"t":"HW4Grader.anchor_cells","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.anchor_cells","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.run_state","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.run_state","s":"assignments.hw4","k":"method","d":"How far the student's own run of these cells backs the source up."},{"t":"HW4Grader.source_steps","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.source_steps","s":"assignments.hw4","k":"method","d":"A step graded on what the student wrote, backed by their saved run."},{"t":"HW4Grader.check_notebook_execution","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_notebook_execution","s":"assignments.hw4","k":"method","d":"Was the notebook run through before it was handed in?"},{"t":"HW4Grader.check_crash_geodataframe","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_crash_geodataframe","s":"assignments.hw4","k":"method","d":"The steps, plus the one mistake worth naming: latitude passed as x."},{"t":"HW4Grader.check_trim_crashes","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_trim_crashes","s":"assignments.hw4","k":"method","d":"The steps; the template's \"about 3,750\" only decides who looks again."},{"t":"HW4Grader.check_crash_counts","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_crash_counts","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.hist_cells","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.hist_cells","s":"assignments.hw4","k":"method","d":"Histogram cells, with how many histograms each saved run shows."},{"t":"HW4Grader.check_crash_index_histogram","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_crash_index_histogram","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.check_crash_index_map","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_crash_index_map","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.check_listing_elements","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_listing_elements","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.check_first_listing_fields","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_first_listing_fields","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.check_format_functions","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_format_functions","s":"assignments.hw4","k":"method","d":"Called in the sandbox when possible; otherwise read, and sent to a TA."},{"t":"HW4Grader.check_scraping_loop","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_scraping_loop","s":"assignments.hw4","k":"method","d":"The loop must be written *and* shown running; a CSV is not a scrape."},{"t":"HW4Grader.check_price_histograms","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_price_histograms","s":"assignments.hw4","k":"method"},{"t":"HW4Grader.check_altair_scatter","u":"api/assignments.hw4.html#assignments.hw4.HW4Grader.check_altair_scatter","s":"assignments.hw4","k":"method"},{"t":"strip_comments()","u":"api/assignments.hw4.html#assignments.hw4.strip_comments","s":"assignments.hw4","k":"function","d":"Drop comments without touching a # inside a string (#search=1~)."},{"t":"build_code()","u":"api/assignments.hw4.html#assignments.hw4.build_code","s":"assignments.hw4","k":"function","d":"Each code cell, with any template scaffold it was filled in from removed."},{"t":"requirement_met()","u":"api/assignments.hw4.html#assignments.hw4.requirement_met","s":"assignments.hw4","k":"function"},{"t":"ROOT","u":"api/assignments.hw4.html#assignments.hw4.ROOT","s":"assignments.hw4","k":"constant"},{"t":"PART2_RE","u":"api/assignments.hw4.html#assignments.hw4.PART2_RE","s":"assignments.hw4","k":"constant"},{"t":"HIST_RE","u":"api/assignments.hw4.html#assignments.hw4.HIST_RE","s":"assignments.hw4","k":"constant"},{"t":"READ_CSV_RE","u":"api/assignments.hw4.html#assignments.hw4.READ_CSV_RE","s":"assignments.hw4","k":"constant"},{"t":"SOURCE_CONFIDENCE","u":"api/assignments.hw4.html#assignments.hw4.SOURCE_CONFIDENCE","s":"assignments.hw4","k":"constant"},{"t":"CORROBORATED_CONFIDENCE","u":"api/assignments.hw4.html#assignments.hw4.CORROBORATED_CONFIDENCE","s":"assignments.hw4","k":"constant"},{"t":"DOUBT_CONFIDENCE","u":"api/assignments.hw4.html#assignments.hw4.DOUBT_CONFIDENCE","s":"assignments.hw4","k":"constant"}];
diff --git a/docs/assignments.html b/docs/assignments.html
index 159c8a8..7d82037 100644
--- a/docs/assignments.html
+++ b/docs/assignments.html
@@ -4,7 +4,7 @@
 
 
 Assignments · MUSA Grader
-
+
 
 
 
@@ -63,6 +63,8 @@
 
  • grader.executor
    • 
 
  • grader.notebook_runner
    • 
 
  • grader.probe_runtime
    • 
+
  • grader.function_sandbox
    • 
+
  • grader.function_runner
    • 
 
 
  • Evidence
    • 
   
   
@@ -91,6 +94,7 @@ 
    Assignments
    
   Assignment 1
    The Donut Effect. No answer key, so each step is checked against the student's own data.
    
   Assignment 2
    Exploratory Visualization. No data handed in, so the required elements are graded from saved outputs.
    
   Assignment 3
    Evictions and the NDVI. One shared zip, so there is an answer key for the first time.
    
+  Assignment 4
    Street Networks & Scraping. Everything is downloaded or scraped live, so the notebook is never run.
    
 
    
 
 
    
@@ -297,7 +301,80 @@ 
    Where the 100 points go
 
-
    Assignments 4–6#
    
+
    Assignment 4: Street Networks & Scraping#
    
+
    rubrics/hw4.yamlassignments/hw4.pyno datanever executed
    
+
+
    Every step of Assignment 4 talks to a server that is not ours. Part 1 downloads the planning districts from ArcGIS Open Data and the street network from the OpenStreetMap Overpass API (osmnx), and Part 2 drives a browser through five pages of Craigslist with a sleep between each. Re-running a class of these notebooks would repeat every one of those requests once per student, which is exactly the load the assignment itself warns will get an IP banned.
    
+
+
    So rubrics/hw4.yaml says execution: mode: never, and the service honours it in every execution mode, local included: no kernel, no notebook container, no browser, no data placement. See Execution. The grade comes from the file the student handed in, read two ways:
    
+
+
      
+  
    • The code they wrote. Each code cell is matched to the template cell it was filled in from, and the template's own lines are set aside first. The driver.get(url) and pd.concat(...) inside the provided scraping loop, or a def format_price(price_string): return stub, earn nothing. Each step is a list of requirements in the rubric, one regex each, scoped to its Part, so a TA can widen one without touching Python.
      • 
+  
    • The outputs their own run saved. The crash count, the scraping log (Processing page 3... Number of apartments = 120), the results table, the rendered histograms, the Folium map and the Altair spec, including the scraped rows the spec embeds.
      • 
+
    
+
+
    The one thing that does run: three pure functions#
    
+
+
    format_bedrooms, format_size and format_price turn "2br" into 2, and calling them is far better evidence than reading them. So they, and only they, are cut out of the notebook (grader.function_sandbox) and called with typical Craigslist strings in the Docker sandbox, under the same isolation flags as a notebook run, above all --network none:
    
+
+
      
+  
    • only the top-level def of each named function is extracted. No other statement of the notebook goes anywhere, and the submission itself is not mounted into the container;
      • 
+  
    • decorators and annotations are stripped, and a function whose default argument is not a plain literal is refused, because those run at definition time;
      • 
+  
    • only imports of re, string, math, decimal, numpy and pandas go along. selenium, requests and osmnx do not;
      • 
+  
    • the expected answers stay on the host. The container runs grader.function_runner, which is told what to call, not what should come back.
      • 
+
    
+
+
    
+  
    In local mode the functions are not called
    
+  
    The machine has a network, so item 2.5 gets a provisional score from the source and goes to a TA. The sidebar says so before the run.
    
+
    
+
+
    Scraping has to be shown, not loaded#
    
+
+
    A results table read back from a CSV proves nothing about a scraper. Item 2.6 wants the completed loop in the notebook and the log it printed when the student ran it, and the results table only counts when that log is there. A table loaded from a file is recorded as such and goes to a TA.
    
+
+
    Reference values, loosely#
    
+
+
    There is no reference solution, and the street network and the listings both change. The only numbers used are the template's own: about 3,750 crashes in Center City (3,000–4,500 accepted), 120 listings a page, and a price per square foot around 1.5. Each only raises confidence. A number far outside its range sends the item to a person; it never costs a point on its own. As everywhere, anything short of full marks reaches a person.
    
+
+
    Where the 100 points go#
    
+
+
    
+  
+  
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+  
+
    ItemPoints
    Notebook run and submitted with its outputs10
    1.1Central planning district polygon4
    1.2Drive network from the polygon4
    1.3Network edges as a GeoDataFrame3
    1.4PennDOT crash data loaded and combined4
    1.5Crashes as a GeoDataFrame (latitude-as-x is named)4
    1.6Crashes trimmed to Center City5
    1.7Graph and crashes projected to EPSG:22725
    1.8Nearest edge for each crash5
    1.9Crash count per street5
    1.10Crash counts merged onto the edges4
    1.11Crash index5
    1.12Histogram of the crash index3
    1.13Interactive map of the crash index4
    2.1Selenium driver opens Craigslist3
    2.2BeautifulSoup from the page source2
    2.3Listing elements extracted4
    2.4Price, bedrooms, size and title of the first listing4
    2.5format_* functions, called in the sandbox9
    2.6Scraping loop completed and shown run7
    2.7Histograms of price and price per square foot3
    2.8Interactive Altair scatter of price vs size3
    Total100
    
+
+
    Execution and output completeness is 10 points, Part 1 is 55 and Part 2 is 35.
    
+
+
    The demo class (python examples/make_hw4_submissions.py) has four submissions: complete, one with the usual mistakes and its Craigslist results loaded from a CSV, one that crashes in Part 1, and the untouched template. Their saved outputs are written in directly; nothing is executed, there or when grading. In Docker mode they score 100, 84, 26.5 and 3. In local mode the format_* item is not called and goes to a TA instead.
    
+
+
    
+
+
    Assignments 5–6#
    
 
 
    Their student templates are in assignment_template/, but they are not graded yet. Each needs a rubric YAML and a grader class. See Adding an assignment.
    
 
@@ -307,7 +384,7 @@ 
    Assignments 4–6Edit this page on GitHubMUSA Grader 0.1.0 · MUSA 5500, University of Pennsylvania
     
   
-  
+  
 
 
     
   
-  
+  
 
 
     
   
-  
+