docs: refresh pipeline notes and transport analysis

.env.example

@@ -21,8 +21,18 @@ COMBINE_OUTPUT_LOCATION=/data/vkg_full.ttl
 COMBINE_FORCE=true
 
 # AnzoGraph / SPARQL endpoint settings
+# Use `local` for the Docker Compose AnzoGraph container, or `external` for the
+# remote endpoint below using a runtime-generated bearer token.
+SPARQL_SOURCE_MODE=local
 SPARQL_HOST=http://anzograph:8080
 # SPARQL_ENDPOINT=http://anzograph:8080/sparql
+EXTERNAL_SPARQL_ENDPOINT=https://anzograph.k8s.inf.ufrgs.br/sparql
+KEYCLOAK_TOKEN_ENDPOINT=https://keycloak.k8s.inf.ufrgs.br/realms/INF/protocol/openid-connect/token
+KEYCLOAK_CLIENT_ID=anzograph
+KEYCLOAK_USERNAME=
+KEYCLOAK_PASSWORD=
+KEYCLOAK_SCOPE=openid
+ACCESS_TOKEN=
 SPARQL_USER=admin
 SPARQL_PASS=Passw0rd1
 
@@ -32,7 +42,6 @@ SPARQL_DATA_FILE=file:///opt/shared-files/o3po.ttl # Currently not used.
 
 # Startup behavior for AnzoGraph mode
 SPARQL_LOAD_ON_START=false
-SPARQL_CLEAR_ON_START=false
 SPARQL_READY_TIMEOUT_S=10
 
 # Dev UX
@@ -40,18 +49,69 @@ CORS_ORIGINS=http://localhost:5173
 VITE_BACKEND_URL=http://backend:8000
 
 # Frontend right-pane cosmos.gl layout
+# Colors accept CSS strings or RGBA tuples like [53,214,255,255].
+# Ranges accept comma-separated values like 50,150.
 VITE_COSMOS_ENABLE_SIMULATION=true
 VITE_COSMOS_DEBUG_LAYOUT=false
+VITE_COSMOS_BACKGROUND_COLOR="#05070a"
 VITE_COSMOS_SPACE_SIZE=4096
+VITE_COSMOS_POINT_DEFAULT_COLOR="#b3b3b3"
+VITE_COSMOS_POINT_GREYOUT_COLOR=
+VITE_COSMOS_POINT_GREYOUT_OPACITY=
+VITE_COSMOS_POINT_DEFAULT_SIZE=4
+VITE_COSMOS_POINT_OPACITY=1
+VITE_COSMOS_POINT_SIZE_SCALE=1
+VITE_COSMOS_HOVERED_POINT_CURSOR=pointer
+VITE_COSMOS_HOVERED_LINK_CURSOR=pointer
+VITE_COSMOS_RENDER_HOVERED_POINT_RING=true
+VITE_COSMOS_HOVERED_POINT_RING_COLOR="#35d6ff"
+VITE_COSMOS_FOCUSED_POINT_RING_COLOR=white
+VITE_COSMOS_RENDER_LINKS=true
+VITE_COSMOS_LINK_DEFAULT_COLOR="#666666"
+VITE_COSMOS_LINK_OPACITY=1
+VITE_COSMOS_LINK_GREYOUT_OPACITY=0.1
+VITE_COSMOS_LINK_DEFAULT_WIDTH=1
+VITE_COSMOS_HOVERED_LINK_COLOR="#ffd166"
+VITE_COSMOS_HOVERED_LINK_WIDTH_INCREASE=2.5
+VITE_COSMOS_LINK_WIDTH_SCALE=1
+VITE_COSMOS_SCALE_LINKS_ON_ZOOM=false
 VITE_COSMOS_CURVED_LINKS=true
+VITE_COSMOS_CURVED_LINK_SEGMENTS=19
+VITE_COSMOS_CURVED_LINK_WEIGHT=0.8
+VITE_COSMOS_CURVED_LINK_CONTROL_POINT_DISTANCE=0.5
+VITE_COSMOS_LINK_DEFAULT_ARROWS=false
+VITE_COSMOS_LINK_ARROWS_SIZE_SCALE=1
+VITE_COSMOS_LINK_VISIBILITY_DISTANCE_RANGE=50,150
+VITE_COSMOS_LINK_VISIBILITY_MIN_TRANSPARENCY=0.25
+VITE_COSMOS_USE_CLASSIC_QUADTREE=false
 VITE_COSMOS_FIT_VIEW_PADDING=0.12
+VITE_COSMOS_FIT_VIEW_ON_INIT=false
+VITE_COSMOS_FIT_VIEW_DELAY=250
+VITE_COSMOS_FIT_VIEW_DURATION=250
 VITE_COSMOS_SIMULATION_DECAY=5000
 VITE_COSMOS_SIMULATION_GRAVITY=0
 VITE_COSMOS_SIMULATION_CENTER=0.05
 VITE_COSMOS_SIMULATION_REPULSION=0.5
+VITE_COSMOS_SIMULATION_REPULSION_THETA=1.15
+VITE_COSMOS_SIMULATION_REPULSION_QUADTREE_LEVELS=12
 VITE_COSMOS_SIMULATION_LINK_SPRING=1
 VITE_COSMOS_SIMULATION_LINK_DISTANCE=10
+VITE_COSMOS_SIMULATION_LINK_DISTANCE_RANDOM_VARIATION_RANGE=1,1.2
+VITE_COSMOS_SIMULATION_REPULSION_FROM_MOUSE=2
+VITE_COSMOS_ENABLE_RIGHT_CLICK_REPULSION=false
 VITE_COSMOS_SIMULATION_FRICTION=0.1
+VITE_COSMOS_SIMULATION_CLUSTER=0.1
+VITE_COSMOS_SHOW_FPS_MONITOR=false
+VITE_COSMOS_PIXEL_RATIO=2
+VITE_COSMOS_SCALE_POINTS_ON_ZOOM=false
+VITE_COSMOS_INITIAL_ZOOM_LEVEL=
+VITE_COSMOS_ENABLE_ZOOM=true
+VITE_COSMOS_ENABLE_SIMULATION_DURING_ZOOM=false
+VITE_COSMOS_ENABLE_DRAG=true
+VITE_COSMOS_RANDOM_SEED=
+VITE_COSMOS_POINT_SAMPLING_DISTANCE=150
+VITE_COSMOS_RESCALE_POSITIONS=false
+VITE_COSMOS_ATTRIBUTION=
 
 # Debugging
 LOG_SNAPSHOT_TIMINGS=false

CURRENT_HIERARCHY_PIPELINE.md

@@ -0,0 +1,243 @@
+# Current `subClassOf` / `BFO:entity` Pipeline
+
+This document summarizes how the repository currently builds the hierarchy that ends up in the radial Sugiyama layout, with special attention to the fact that "start from `bfo:entity`" is **not** implemented in the initial `subClassOf` query.
+
+`bfo:entity` here means:
+
+- `http://purl.obolibrary.org/obo/BFO_0000001`
+
+## TL;DR
+
+- The current code does **not** query "all `rdfs:subClassOf` relationships rooted at `bfo:entity`" directly.
+- It first queries the **entire** `rdfs:subClassOf` graph.
+- It builds an in-memory graph from those triples.
+- Only later, in the Rust hierarchy layout bridge, it filters that graph to the descendant closure of the configured root IRI.
+- Because of that, the "rooted at `bfo:entity`" behavior is currently coupled to the layout pipeline instead of existing as a reusable graph-extraction stage.
+
+## Where The Request Starts
+
+The frontend loads the hierarchy through the normal graph endpoint:
+
+1. `frontend/src/App.tsx`
+2. `GET /api/graph?graph_query_id=hierarchy`
+3. `backend_go/server.go` -> `handleGraph`
+4. `backend_go/snapshot_service.go` -> `Get`
+5. `backend_go/graph_snapshot.go` -> `fetchGraphSnapshot`
+
+Important consequence:
+
+- The hierarchy is treated as a graph snapshot mode, not as a dedicated "query descendants of this root" pipeline.
+
+## The Actual SPARQL Query Used For `hierarchy`
+
+The `hierarchy` graph query is defined in:
+
+- `backend_go/graph_queries/hierarchy.go`
+
+It effectively does:
+
+```sparql
+PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
+
+SELECT ?s ?p ?o
+WHERE {
+  VALUES ?p { rdfs:subClassOf }
+  ?s ?p ?o .
+  FILTER(!isLiteral(?o))
+  # optionally also FILTER(!isBlank(?s) && !isBlank(?o))
+}
+ORDER BY ?s ?p ?o
+LIMIT ...
+OFFSET ...
+```
+
+Important facts:
+
+- It queries **all** `rdfs:subClassOf` triples.
+- There is **no root restriction** here.
+- There is **no `bfo:entity` filter** here.
+- Blank nodes are excluded unless `INCLUDE_BNODES=true`.
+- Objects that are literals are excluded.
+
+## How The In-Memory Graph Is Built
+
+Graph construction is handled by:
+
+- `backend_go/graph_export.go`
+
+The accumulator logic works like this:
+
+- Every returned `?s` and `?o` becomes a node if it has not been seen before.
+- There is no separate node query.
+- A class only enters the graph if it appears in at least one fetched edge.
+- Isolated classes with no fetched `subClassOf` edge never appear.
+- If `node_limit` is reached, new nodes stop being added, and edges that depend on them are skipped.
+
+Edge direction at this stage is:
+
+- `Source = subclass (?s)`
+- `Target = superclass (?o)`
+
+So the raw in-memory graph is stored as:
+
+- `subclass -> superclass`
+
+## Where `BFO:entity` Is Actually Applied
+
+The root restriction happens only when the backend chooses the Rust hierarchy layout path.
+
+Relevant files:
+
+- `backend_go/config.go`
+- `.env`
+- `backend_go/graph_snapshot.go`
+- `backend_go/hierarchy_layout_bridge.go`
+- `radial_sugiyama/src/bridge.rs`
+
+Current behavior:
+
+- `.env` sets `HIERARCHY_LAYOUT_ENGINE=rust`.
+- If `graph_query_id == "hierarchy"` and the engine is `rust`, the backend calls the Rust bridge.
+- The root IRI comes from `HIERARCHY_LAYOUT_ROOT_IRI`.
+- If that env var is not set, the checked-in default is `http://purl.obolibrary.org/obo/BFO_0000001`.
+
+This means the current repository behavior is effectively:
+
+- query all `subClassOf`
+- then filter to descendants of `BFO:entity`
+- then lay out the filtered graph
+
+## What Go Sends To Rust
+
+Before calling Rust, Go rewrites the edge orientation in:
+
+- `backend_go/hierarchy_layout_bridge.go`
+
+It converts each stored edge from:
+
+- `subclass -> superclass`
+
+into:
+
+- `parentID = superclass`
+- `childID = subclass`
+
+So the Rust side receives:
+
+- `superclass -> subclass`
+
+Go also:
+
+- de-duplicates repeated parent/child edges
+- sends the configured `root_iri`
+- sends all nodes that were present in the fetched hierarchy graph
+
+## How Rust Filters To Descendants Of The Root
+
+Filtering happens in:
+
+- `radial_sugiyama/src/bridge.rs`
+
+The bridge logic does this:
+
+1. Build an internal graph from the request.
+2. Find the node whose label/IRI matches `root_iri`.
+3. Build adjacency lists in the `parent -> child` direction.
+4. Run a BFS/queue traversal starting at the root.
+5. Keep only the visited nodes.
+6. Keep only edges whose endpoints are both visited.
+7. Run radial Sugiyama layout on that filtered subgraph.
+
+Important consequences:
+
+- Nodes outside the descendant closure of the root are dropped.
+- Disconnected components are dropped.
+- Ancestors of the root are not kept unless they are also reachable as descendants, which normally they are not.
+- If the root is missing, the pipeline errors.
+- If the root has no descendants, the pipeline errors.
+
+So the actual "select only those starting from `bfo:entity`" logic is:
+
+- **graph traversal after fetching the full hierarchy**
+
+not:
+
+- **root-constrained SPARQL**
+
+## What Comes Back From Rust
+
+After Rust finishes:
+
+- only the filtered nodes are returned
+- only edges between retained nodes are returned
+- routed edge segments are returned for drawing
+
+That filtering is applied back onto the original Go snapshot response, so the final `/api/graph?graph_query_id=hierarchy` response only contains the root-descendant subgraph when the Rust path is active.
+
+## Why This Feels Like A Separate Pipeline
+
+The main reason it feels split is that the current behavior crosses multiple stages:
+
+1. SPARQL query stage fetches the whole `subClassOf` graph.
+2. Graph materialization stage builds a generic snapshot graph.
+3. Layout bridge stage applies the root restriction.
+4. Layout stage computes coordinates.
+
+This means the "hierarchy rooted at `BFO:entity`" concept is currently embedded in layout preparation instead of existing as a first-class reusable data pipeline.
+
+In practice, the root filtering is:
+
+- not reusable by itself through a dedicated backend API
+- not expressed in the initial SPARQL query
+- not controlled per request
+- tied to the hierarchy layout engine choice
+
+## Selection Queries Are A Different Mechanism
+
+The repository also has separate selection-query endpoints:
+
+- `backend_go/selection_queries/subclasses.go`
+- `backend_go/selection_queries/superclasses.go`
+- `backend_go/selection_queries/neighbors.go`
+
+Those are used after nodes are already present in a graph snapshot and the user selects node IDs.
+
+They are **not** the mechanism that initially builds the `BFO:entity` hierarchy used by the radial layout.
+
+Their role is more like:
+
+- "given selected node IDs in the current snapshot, query related triples"
+
+not:
+
+- "materialize the hierarchy rooted at `BFO:entity`"
+
+## Current End-To-End Behavior In One Sentence
+
+The current system gets all `rdfs:subClassOf` triples first, constructs a general hierarchy graph, and only then filters it to the descendants of `http://purl.obolibrary.org/obo/BFO_0000001` inside the Rust radial Sugiyama bridge.
+
+## Files To Read When Rewriting
+
+If you want to rewrite this from zero, these are the main files that define the current behavior:
+
+- `backend_go/server.go`
+- `backend_go/snapshot_service.go`
+- `backend_go/graph_snapshot.go`
+- `backend_go/graph_queries/hierarchy.go`
+- `backend_go/graph_export.go`
+- `backend_go/hierarchy_layout_bridge.go`
+- `backend_go/config.go`
+- `.env`
+- `radial_sugiyama/src/bridge.rs`
+- `backend_go/selection_queries/subclasses.go`
+- `backend_go/selection_queries/superclasses.go`
+
+## Rewrite-Oriented Takeaway
+
+If your goal is a cleaner standalone pipeline for:
+
+- query `rdfs:subClassOf`
+- start from `bfo:entity`
+- materialize only the rooted descendant hierarchy
+
+then the current codebase is doing the root restriction too late. Right now, that concern lives in the layout bridge rather than in the query/materialization layer.

GRAPH_TRANSPORT_ALTERNATIVES.md

@@ -0,0 +1,655 @@
+# Graph Transport Alternatives
+
+## Purpose
+
+This document compares alternatives to the current `/api/graph` transport format with two goals:
+
+1. reduce the cost of building, transferring, and decoding very large graph payloads
+2. move the frontend transport shape closer to the renderer/GPU input shape while preserving all data the current frontend and backend pipeline still need
+
+This analysis is based on the current repo state plus official documentation for browser fetch/streaming and candidate transport formats.
+
+## Executive Summary
+
+The current bottleneck is not the renderer's typed-array path. It is the browser's need to fully materialize a huge JSON object graph before the renderer ever runs.
+
+The best candidates for this repo are:
+
+1. **Custom binary columnar payload**
+   - Best fit for the current renderer.
+   - Lowest decode overhead.
+   - Most direct path from backend memory to frontend typed arrays.
+   - Requires custom protocol/versioning work.
+
+2. **Apache Arrow IPC**
+   - Best off-the-shelf columnar binary format.
+   - Very good fit for typed-array-heavy rendering.
+   - Strong option if you want a standard format instead of inventing one.
+   - Heavier conceptual/tooling footprint than a custom binary envelope.
+
+3. **Columnar JSON**
+   - Easiest migration.
+   - Better than today's row-oriented JSON.
+   - Still fundamentally JSON, so it does not remove the browser's JSON parse/object-materialization cost.
+
+4. **NDJSON / streamed chunked JSON**
+   - Good if progressiveness matters.
+   - Better than one giant monolithic JSON document.
+   - Still weaker than a binary/columnar format for this renderer.
+
+The strongest overall recommendation is:
+
+- **Long-term**: custom binary columnar payload or Arrow IPC
+- **Low-risk interim**: columnar JSON, possibly with chunking/streaming
+
+Not recommended as the primary solution for this repo:
+
+- row-oriented MessagePack
+- Protocol Buffers as one giant message
+
+## Verified Current Pipeline
+
+### Backend side
+
+The backend builds a `GraphResponse` and caches it in memory:
+
+- `backend_go/models.go`
+- `backend_go/snapshot_service.go`
+- `backend_go/graph_snapshot.go`
+
+The response shape is:
+
+```go
+type GraphResponse struct {
+    Nodes         []Node
+    Edges         []Edge
+    RouteSegments []RouteSegment
+    Meta          *GraphMeta
+}
+```
+
+and it is currently written as one JSON document with:
+
+```go
+json.NewEncoder(w).Encode(v)
+```
+
+in `backend_go/http_helpers.go`.
+
+### Frontend side
+
+The frontend currently does:
+
+1. `fetch("/api/graph?...")`
+2. `await graphRes.json()`
+3. read `graph.nodes`, `graph.edges`, `graph.route_segments`, `graph.meta`
+4. build:
+   - `Float32Array xs`
+   - `Float32Array ys`
+   - `Uint32Array vertexIds`
+   - `Uint32Array edgeData`
+   - `Float32Array routeLineVertices`
+5. call `renderer.init(xs, ys, vertexIds, edgeData, routeLineVertices)`
+
+Relevant files:
+
+- `frontend/src/App.tsx`
+- `frontend/src/renderer.ts`
+
+This means the current browser path is:
+
+- wire bytes
+- JSON text/body handling
+- JS arrays of node/edge objects
+- typed arrays
+- renderer-side typed arrays/maps/GPU buffers
+
+The expensive part happens before step 4.
+
+## Verified Data Access Audit
+
+This section verifies every field currently produced by the backend and whether it is actually needed by the frontend transport.
+
+### Main graph response fields
+
+| Field | Produced in backend | Used by frontend? | Where used | Required on wire for current UX? | Notes |
+| --- | --- | --- | --- | --- | --- |
+| `nodes[].id` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes | Used to build `vertexIds`, and to map selected renderer indices back to backend IDs for selection queries. |
+| `nodes[].x` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes | Used to build `xs`. |
+| `nodes[].y` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes | Used to build `ys`. |
+| `nodes[].iri` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes, if keeping current hover UX | Used for hover tooltip text. |
+| `nodes[].label` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes, if keeping current hover UX | Used for hover tooltip text. |
+| `nodes[].termType` | `backend_go/models.go` | No frontend use | none in `frontend/src` | No | Still needed internally by backend snapshot/selection index. |
+| `edges[].source` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes | Used to build `edgeData`. |
+| `edges[].target` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes | Used to build `edgeData`. |
+| `edges[].predicate_id` | `backend_go/models.go` | No main-graph frontend use | none in `frontend/src/App.tsx` | No | Still needed internally by backend snapshot and hierarchy layout preparation. |
+| `route_segments[].points` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes when route segments are present | Used to build `routeLineVertices`. |
+| `route_segments[].edge_index` | `backend_go/models.go` | Not used after parsing | `graphRouteSegmentArray` validation only | No | Could be dropped from frontend transport if route lines are pre-flattened. |
+| `route_segments[].kind` | `backend_go/models.go` | Not used after parsing | `graphRouteSegmentArray` validation only | No | Could be dropped from frontend transport if route lines are pre-flattened. |
+| `meta.backend` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes | Displayed in overlay. |
+| `meta.nodes` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes | Displayed in overlay. |
+| `meta.edges` | `backend_go/models.go` | Yes | `frontend/src/App.tsx` | Yes | Displayed in overlay. |
+| `meta.graph_query_id` | `backend_go/models.go` | Yes | `frontend/src/selection_queries/api.ts` | Yes | Sent back on selection endpoints. |
+| `meta.node_limit` | `backend_go/models.go` | Yes | `frontend/src/selection_queries/api.ts` | Yes | Sent back on selection endpoints. |
+| `meta.edge_limit` | `backend_go/models.go` | Yes | `frontend/src/selection_queries/api.ts` | Yes | Sent back on selection endpoints. |
+| `meta.ttl_path` | `backend_go/models.go` | No | none in `frontend/src` | No | Frontend type declares it, but current UI does not use it. |
+| `meta.sparql_endpoint` | `backend_go/models.go` | No | none in `frontend/src` | No | Not used by current UI. |
+| `meta.include_bnodes` | `backend_go/models.go` | No | none in `frontend/src` | No | Not used by current UI. |
+| `meta.layout_engine` | `backend_go/models.go` | No | none in `frontend/src` | No | Not used by current UI. |
+| `meta.layout_root_iri` | `backend_go/models.go` | No | none in `frontend/src` | No | Not used by current UI. |
+| `meta.predicates` | `backend_go/models.go` | No frontend use | none in `frontend/src` | No | Still used internally by backend selection/hierarchy logic. |
+
+### Backend-internal fields that do not need to stay in the frontend transport
+
+This is the most important audit result.
+
+The backend currently reuses one struct for:
+
+- internal cached snapshot
+- HTTP response payload
+
+That is convenient, but it means the frontend receives fields that only the backend needs.
+
+Verified internal-only dependencies:
+
+- `snapshot.Nodes[].TermType` is used in `backend_go/selection_query.go` to build the selection index.
+- `snapshot.Meta.Predicates` is used in `backend_go/selection_query.go`.
+- `Edge.PredicateID` is used internally for hierarchy layout preparation in `backend_go/hierarchy_layout_bridge.go`.
+
+The frontend does **not** need those fields for current behavior.
+
+### What the frontend actually needs
+
+For the current graph view, the hot path can be reduced to:
+
+- `vertexIds[]`
+- `xs[]`
+- `ys[]`
+- `edgeSources[]`
+- `edgeTargets[]`
+- `routeLineVertices[]` or route geometry equivalent
+- `label[]` and `iri[]` by node index
+- `meta.backend`
+- `meta.nodes`
+- `meta.edges`
+- `meta.graph_query_id`
+- `meta.node_limit`
+- `meta.edge_limit`
+
+That is much closer to a columnar or binary payload than to the current array-of-objects JSON.
+
+## Why the Current JSON Path Hurts
+
+`Response.json()` is not just a lightweight decode helper. MDN states that `Response.json()` reads the stream to completion and resolves with the result of parsing the body text as JSON into a JavaScript object.
+
+That matters here because the current payload is row-oriented:
+
+- millions of node objects
+- millions of edge objects
+
+Even though the renderer later wants typed arrays, the browser must first create those JS objects.
+
+This is exactly the part that can stall or run out of memory before `renderer.init(...)` starts.
+
+## Alternatives
+
+### 1. Columnar JSON
+
+#### Idea
+
+Keep JSON, but change the schema from row-oriented objects:
+
+```json
+{
+  "nodes": [{ "id": 1, "x": 0.1, "y": 0.2, ... }],
+  "edges": [{ "source": 1, "target": 2, ... }]
+}
+```
+
+to column-oriented arrays:
+
+```json
+{
+  "vertex_ids": [...],
+  "xs": [...],
+  "ys": [...],
+  "edge_sources": [...],
+  "edge_targets": [...],
+  "node_labels": [...],
+  "node_iris": [...],
+  "route_line_vertices": [...],
+  "meta": { ... }
+}
+```
+
+#### Pros
+
+- easiest migration from the current API contract
+- no schema compiler
+- easy to debug with ordinary tooling
+- much closer to what the renderer already consumes
+- avoids creating per-edge objects in frontend application code
+
+#### Cons
+
+- still goes through JSON parsing
+- still materializes JS arrays before typed arrays are built
+- huge numeric arrays in JSON are still text, not binary
+- string columns are still ordinary JS strings
+
+#### Fit for current pipeline
+
+Good.
+
+No current frontend feature would be lost if the payload includes:
+
+- ids/xs/ys/edge sources/targets
+- labels/iris
+- route line vertices or equivalent
+- the small subset of meta fields currently used
+
+#### Overall assessment
+
+Best low-risk intermediate step.
+
+It is clearly better than today's row-oriented JSON, but it is not the endgame if the goal is to remove the parse bottleneck for 1 GB+ payloads.
+
+### 2. NDJSON / Chunked JSON
+
+#### Idea
+
+Change the backend to stream multiple JSON records instead of one giant JSON object.
+
+Examples:
+
+- one line per chunk of nodes/edges
+- one line for metadata
+- one line per route segment chunk
+
+NDJSON is explicitly designed for transporting multiple JSON texts in a stream protocol.
+
+#### Pros
+
+- can start processing before the whole payload arrives
+- better observability and progress reporting
+- easier cancellation/retry semantics
+- avoids one monolithic `Response.json()` boundary
+
+#### Cons
+
+- record-per-edge NDJSON would still create far too many JS objects
+- to be worth it here, it should be **chunked columnar NDJSON**, not row NDJSON
+- frontend load path must become stream-based
+- renderer still currently expects all arrays at once
+
+#### Fit for current pipeline
+
+Moderate.
+
+It can preserve all current information, but it does not by itself solve the "final representation should look like GPU inputs" goal unless each chunk is already columnar.
+
+#### Best shape if chosen
+
+Not:
+
+- one JSON object per edge
+- one JSON object per node
+
+Better:
+
+- one NDJSON record for metadata
+- then NDJSON records where each record contains columnar chunks:
+  - `vertex_ids_chunk`
+  - `xs_chunk`
+  - `ys_chunk`
+  - `edge_sources_chunk`
+  - `edge_targets_chunk`
+
+#### Overall assessment
+
+Viable, but only attractive if progressiveness is a major goal. On its own, it is weaker than columnar binary formats for this renderer.
+
+### 3. MessagePack
+
+#### Idea
+
+Use a compact binary encoding instead of JSON.
+
+The official JavaScript implementation supports:
+
+- `encode`
+- `decode`
+- `decodeAsync(stream)`
+- `decodeArrayStream(stream)`
+- `decodeMultiStream(stream)`
+
+and even custom extension types for faster handling of large `Float32Array` payloads.
+
+#### Pros
+
+- smaller payload than JSON
+- binary transport
+- async and stream-capable decoding APIs exist
+- mature JS library
+
+#### Cons
+
+- if you keep the current row-oriented schema, you still get one huge object graph after decode
+- therefore MessagePack alone does not remove the fundamental object-allocation problem
+- custom extension types improve typed-array cases, but then you are already halfway to designing a custom binary protocol
+
+#### Fit for current pipeline
+
+Moderate.
+
+It can preserve all current information easily.
+
+But if the schema remains object-heavy, the browser still ends up with millions of JS objects.
+
+#### Overall assessment
+
+Useful if paired with a **columnar** schema. Not compelling as a first move if the schema stays row-oriented.
+
+### 4. Apache Arrow IPC
+
+#### Idea
+
+Use Arrow's columnar binary format and Arrow JS support.
+
+Arrow JS provides:
+
+- `tableFromIPC(...)`
+- support for `fetch(...)`
+- typed-array-backed vectors
+- dictionary-encoded strings
+- a columnar memory model explicitly meant for efficient processing and movement of large in-memory data
+
+#### Pros
+
+- strongest off-the-shelf fit for typed-array-oriented rendering
+- columnar by design
+- binary rather than textual
+- supports large numeric columns very naturally
+- supports dictionary encoding for repeated strings like labels or IRIs
+- much closer to the renderer/GPU input shape than JSON objects
+
+#### Cons
+
+- larger conceptual/tooling jump than columnar JSON
+- route segments are nested/variable-length; representing them cleanly needs design
+- frontend code becomes Arrow-aware unless the decode is hidden behind an adapter
+- backend must serialize Arrow on the Go side or produce Arrow-compatible IPC
+
+#### Fit for current pipeline
+
+Very good.
+
+Current frontend needs can be represented as columns:
+
+- `vertex_ids: uint32`
+- `xs: float32`
+- `ys: float32`
+- `edge_sources: uint32`
+- `edge_targets: uint32`
+- `labels: utf8` or dictionary-encoded utf8
+- `iris: utf8` or dictionary-encoded utf8
+
+Route geometry should probably not stay as nested route-segment objects. It would fit better as:
+
+- pre-flattened `route_line_vertices` float column/buffer
+- or a second Arrow table dedicated to line segments
+
+#### Overall assessment
+
+One of the two best solutions for this repo.
+
+If you want a standard format instead of inventing one, Arrow is the most attractive candidate.
+
+### 5. FlatBuffers
+
+#### Idea
+
+Use a schema-defined binary format designed for direct access without unpacking/parsing.
+
+FlatBuffers explicitly advertises:
+
+- access to serialized data without parsing/unpacking
+- memory efficiency and speed
+- forwards/backwards compatibility
+
+#### Pros
+
+- very strong memory-efficiency story
+- schema evolution support
+- no full parse/unpack step in the same way as JSON
+- can model both scalars and more complex structures
+
+#### Cons
+
+- requires schema/compiler/generated bindings
+- JavaScript integration is more manual than JSON or Arrow
+- ergonomics in app code are not as simple as arrays/objects
+- strings and nested route structures are supported, but the developer experience is more specialized
+
+#### Fit for current pipeline
+
+Good, technically.
+
+It can preserve all current information and remove the giant object-graph parse step.
+
+However, compared with Arrow or a custom binary envelope, it is a less natural conceptual fit for a renderer whose hot path is already columnar/typed-array-based.
+
+#### Overall assessment
+
+A strong technical option, but probably not the most ergonomic option for this specific frontend.
+
+### 6. Protocol Buffers
+
+#### Idea
+
+Use a schema-defined binary format with generated bindings.
+
+#### Pros
+
+- compact binary encoding
+- schema/versioning
+- mature ecosystem
+
+#### Cons
+
+- official docs describe protobuf as a good fit for typed structured messages up to a few megabytes
+- the same docs warn that large data can require loading entire messages into memory and can cause multiple copies
+- large repeated numeric arrays are not protobuf's sweet spot
+- still not especially close to the renderer's typed-array model
+
+#### Fit for current pipeline
+
+Poor for this specific payload size and shape.
+
+#### Overall assessment
+
+Not recommended for this main graph transport.
+
+### 7. Custom Binary Typed-Array Envelope
+
+#### Idea
+
+Define a transport specifically around what the renderer and hover/selection pipeline need.
+
+Example structure:
+
+- small fixed header or small JSON header:
+  - version
+  - counts
+  - offsets/lengths
+  - meta subset
+- then raw binary buffers:
+  - `vertex_ids`
+  - `xs`
+  - `ys`
+  - `edge_sources`
+  - `edge_targets`
+  - `route_line_vertices`
+  - string dictionary / offsets for `label` and `iri`
+
+#### Pros
+
+- closest possible fit to current renderer
+- no schema compiler required
+- no row-object materialization
+- easiest path to zero-copy or near-zero-copy arrays on the frontend
+- easiest path to worker transfer via `ArrayBuffer`
+- can separate hot render data from cold metadata cleanly
+
+#### Cons
+
+- custom protocol to design, version, validate, and document
+- less tooling/interoperability than Arrow
+- backend and frontend both need careful binary codecs
+
+#### Fit for current pipeline
+
+Excellent.
+
+You can preserve all current behavior while only sending the data the frontend actually uses.
+
+#### Overall assessment
+
+The best performance-oriented fit if you are comfortable owning a custom format.
+
+## Comparison Table
+
+| Option | Closeness to GPU shape | Avoids giant object graph | Supports all current frontend data | Streaming-friendly | Implementation cost | Recommendation |
+| --- | --- | --- | --- | --- | --- | --- |
+| Current row JSON | Poor | No | Yes | Poor | Already done | Replace |
+| Columnar JSON | Medium | No | Yes | Medium | Low | Good interim |
+| NDJSON chunked columnar JSON | Medium | Partially | Yes | Good | Medium | Situational |
+| MessagePack row-oriented | Poor | No | Yes | Good | Medium | Not enough alone |
+| MessagePack columnar | Medium | Partially | Yes | Good | Medium | Viable but secondary |
+| Arrow IPC | Very high | Yes or mostly yes | Yes | Good | Medium-high | Strong candidate |
+| FlatBuffers | High | Yes | Yes | Medium | High | Good but specialized |
+| Protobuf | Low-medium | No practical win here | Yes | Medium | Medium-high | Not recommended |
+| Custom binary typed-array envelope | Very high | Yes | Yes | Good | High | Strongest fit |
+
+## Recommended Data Contract Shapes
+
+### Recommended shape for any non-row-oriented solution
+
+The frontend does not need node/edge objects as its primary graph transport.
+
+The main graph payload should be modeled as:
+
+- `vertex_ids`
+- `xs`
+- `ys`
+- `edge_sources`
+- `edge_targets`
+- `route_line_vertices`
+- `node_labels`
+- `node_iris`
+- `meta`
+
+This can be represented as:
+
+- columnar JSON
+- Arrow columns
+- FlatBuffers vectors
+- custom binary sections
+
+### Fields that can be removed from the frontend transport immediately
+
+Without changing current visible behavior, the main graph transport does not need to include:
+
+- `nodes[].termType`
+- `edges[].predicate_id`
+- `meta.predicates`
+- `meta.ttl_path`
+- `meta.sparql_endpoint`
+- `meta.include_bnodes`
+- `meta.layout_engine`
+- `meta.layout_root_iri`
+- `route_segments[].edge_index`
+- `route_segments[].kind`
+
+Important:
+
+Some of those fields are still needed by the backend's **internal snapshot**, especially for selection queries and hierarchy layout. That argues for splitting:
+
+- internal snapshot model
+- frontend transport DTO
+
+instead of continuing to reuse one struct for both.
+
+## Additional Architectural Notes
+
+### A worker is complementary, not a transport format
+
+Web Workers can move parsing/build work off the main thread, and `ArrayBuffer` is transferable. That is useful, but it does not by itself solve the current over-allocation problem if the payload is still a giant row-oriented JSON document.
+
+Workers are most valuable when paired with:
+
+- binary columnar payloads
+- streamed columnar chunks
+- transfer of `ArrayBuffer`s rather than giant JS object graphs
+
+### The backend can keep a richer internal snapshot than it sends
+
+This repo already caches snapshots server-side. Selection and triple queries are built from the backend snapshot and the small `graphMeta` values sent back by the client.
+
+That means the frontend transport can be much slimmer than the backend snapshot representation, as long as the backend retains its richer internal data.
+
+This is the cleanest way to avoid losing information while optimizing the frontend transport.
+
+## Final Recommendation
+
+### Best long-term option
+
+Pick one of:
+
+1. **Custom binary typed-array envelope**
+2. **Apache Arrow IPC**
+
+Reason:
+
+- both map naturally to the renderer's actual input model
+- both avoid the giant row-object parse path
+- both can preserve all current frontend-visible information
+
+### Best low-risk migration path
+
+If you want an incremental step before going binary:
+
+1. split backend internal snapshot from frontend transport DTO
+2. move `/api/graph` to **columnar JSON**
+3. keep only the metadata fields the frontend actually uses
+4. later replace the same columnar DTO with Arrow or custom binary
+
+That path reduces waste immediately and keeps the eventual binary migration straightforward.
+
+## Sources
+
+Official documentation and primary sources used for the comparison:
+
+- MDN `Response.json()`
+  - https://developer.mozilla.org/en-US/docs/Web/API/Response/json
+- MDN `TextDecoderStream`
+  - https://developer.mozilla.org/en-US/docs/Web/API/TextDecoderStream
+- MDN Web Workers
+  - https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers
+- MDN Transferable Objects
+  - https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Transferable_objects
+- Apache Arrow JavaScript
+  - https://arrow.apache.org/js/current/
+  - https://arrow.apache.org/js/main/functions/Arrow.dom.tableFromIPC.html
+- NDJSON specification
+  - https://github.com/ndjson/ndjson-spec
+- MessagePack for JavaScript
+  - https://github.com/msgpack/msgpack-javascript
+- FlatBuffers overview and JavaScript docs
+  - https://flatbuffers.dev/
+  - https://flatbuffers.dev/languages/javascript/
+- Protocol Buffers overview
+  - https://protobuf.dev/overview/
+- Streaming JSON parser references
+  - https://github.com/juanjoDiaz/streamparser-json
+  - https://rictic.github.io/jsonriver/

README.md

@@ -1,108 +1,96 @@
 # Visualizador Instanciados
 
-This repo is a Docker Compose stack for visualizing large RDF/OWL graphs stored in **AnzoGraph**. It includes:
+Docker Compose stack for exploring large RDF/OWL graphs stored in AnzoGraph.
 
-- A **Go backend** that queries AnzoGraph via SPARQL and serves a cached graph snapshot + selection queries.
-- A **React/Vite frontend** that renders nodes/edges with WebGL2 and supports “selection query” + “graph query” modes.
-- A **Python one-shot service** to combine `owl:imports` into a single Turtle file.
-- An **AnzoGraph** container (SPARQL endpoint).
+## What Runs Here
 
-## Quick start (Docker Compose)
+- `anzograph`: SPARQL store
+- `backend`: Go API that queries AnzoGraph and serves cached graph snapshots
+- `frontend`: React/Vite app with a WebGL left graph and a right-side `cosmos.gl` selection graph
+- `owl_imports_combiner`: one-shot Python service that can merge `owl:imports`
+- `radial_sugiyama`: Rust hierarchy layout pipeline used in two ways:
+  - standalone SVG generator through the `radial` Compose profile
+  - optional hierarchy layout engine for the Go backend
 
-1) Put your TTL file(s) in `./data/` (this folder is volume-mounted into AnzoGraph as `/opt/shared-files`).
-2) Optionally configure `.env` (see `.env.example`).
-3) Start the stack:
+## Current Flow
+
+- The backend always builds graph snapshots from SPARQL queries against AnzoGraph.
+- `graph_query_id=default` and `graph_query_id=types` use the Go layout path.
+- `graph_query_id=hierarchy` can use either:
+  - the Go layout path
+  - the Rust radial Sugiyama path when `HIERARCHY_LAYOUT_ENGINE=rust`
+- When the Rust hierarchy path is enabled, the backend sends the hierarchy graph to Rust over JSON, Rust lays it out, returns node positions + routed edge segments, and also rewrites:
+
+```text
+radial_sugiyama/out/layout.svg
+```
+
+That SVG is a debug artifact for the exact Rust layout run used by the backend.
+
+## Quick Start
+
+1. Put your TTL files under `./data/`.
+2. Copy or edit `.env` as needed.
+3. Start the stack:
 
 ```bash
 docker compose up --build
 ```
 
-Then open the frontend:
+Open:
 
-- `http://localhost:5173`
+- Frontend: `http://localhost:5173`
+- Backend health: `http://localhost:8000/api/health`
 
-Stop everything:
+Stop:
 
 ```bash
 docker compose down
 ```
 
-## Services
+## Rust Hierarchy Layout
 
-Defined in `docker-compose.yml`:
+To use Rust for the `hierarchy` graph mode, set this in the repo root `.env`:
 
-- `anzograph` (image `cambridgesemantics/anzograph:latest`)
-  - Ports: `8080`, `8443`
-  - Shared files: `./data → /opt/shared-files`
-- `backend` (`./backend_go`)
-  - Port: `8000` (API under `/api/*`)
-  - Talks to AnzoGraph at `SPARQL_HOST` / `SPARQL_ENDPOINT`
-- `frontend` (`./frontend`)
-  - Port: `5173`
-  - Proxies `/api/*` to `VITE_BACKEND_URL`
-- `owl_imports_combiner` (`./python_services/owl_imports_combiner`)
-  - One-shot: optionally produces a combined TTL by following `owl:imports`
+```env
+HIERARCHY_LAYOUT_ENGINE=rust
+```
 
-Service READMEs:
+The backend also reads `radial_sugiyama/.env` for the Rust layout settings such as:
 
-- `backend_go/README.md`
-- `frontend/README.md`
-- `python_services/owl_imports_combiner/README.md`
-- `anzograph/README.md`
+- `RADIAL_ROOT_CLASS_IRI`
+- `RADIAL_OUTPUT_DIR`
+- `RADIAL_OUTPUT_FILE`
+- `RADIAL_RING_DISTRIBUTION`
 
-## Repo layout
+The debug SVG for backend-driven hierarchy requests is written to:
 
-- `backend_go/` – Go API service (SPARQL → snapshot + selection queries)
-- `frontend/` – React/Vite WebGL renderer
-- `python_services/owl_imports_combiner/` – Python one-shot OWL imports combiner
-- `data/` – local shared volume for TTL inputs/outputs (gitignored)
-- `docker-compose.yml` – service wiring
-- `flake.nix` – optional Nix dev shell
+```text
+radial_sugiyama/out/layout.svg
+```
 
-## Configuration
-
-This repo expects a local `.env` file (not committed). Start from `.env.example`.
-
-Common knobs:
-
-- Backend snapshot size: `DEFAULT_NODE_LIMIT`, `DEFAULT_EDGE_LIMIT`, `MAX_NODE_LIMIT`, `MAX_EDGE_LIMIT`
-- SPARQL connectivity: `SPARQL_HOST` or `SPARQL_ENDPOINT`, plus `SPARQL_USER` / `SPARQL_PASS`
-- Load data on backend startup: `SPARQL_LOAD_ON_START=true` with `SPARQL_DATA_FILE=file:///opt/shared-files/<file>.ttl`
-- Frontend → backend proxy: `VITE_BACKEND_URL`
-
-## API (backend)
-
-Base URL: `http://localhost:8000`
-
-- `GET /api/health` – liveness
-- `GET /api/stats` – snapshot stats (uses default limits)
-- `GET /api/graph` – graph snapshot
-  - Query params: `node_limit`, `edge_limit`, `graph_query_id`
-- `GET /api/graph_queries` – available graph snapshot modes (`graph_query_id` values)
-- `GET /api/selection_queries` – available selection-highlight modes (`query_id` values)
-- `POST /api/selection_query` – run a selection query for highlighted neighbors
-  - Body: `{"query_id":"neighbors","selected_ids":[...],"node_limit":...,"edge_limit":...,"graph_query_id":"default"}`
-- `POST /api/sparql` – raw SPARQL passthrough (debug/advanced)
-- `POST /api/neighbors` – legacy alias (same behavior as `query_id="neighbors"`)
-
-## Frontend UI
-
-- Mouse:
-  - Drag: pan
-  - Scroll: zoom
-  - Click: select nodes
-- **Top-right buttons:** “selection query” mode (how neighbors/highlights are computed for the current selection)
-- **Bottom-right buttons:** “graph query” mode (which SPARQL edge set is used to build the graph snapshot; switching reloads the graph)
-
-## Notes on performance/limits
-
-- The backend caches snapshots in memory; tune `DEFAULT_*_LIMIT` if memory is too high.
-- The frontend renders a sampled subset when zoomed out, and only draws edges when fewer than ~20k nodes are visible.
-
-## Nix dev shell (optional)
-
-If you use Nix, `flake.nix` provides a minimal `devShell`:
+You can still run the standalone Rust SVG pipeline directly with:
 
 ```bash
-nix develop
+docker compose --profile radial up --build radial_sugiyama
 ```
+
+## Main API
+
+- `GET /api/health`
+- `GET /api/stats`
+- `GET /api/graph`
+- `GET /api/graph_queries`
+- `GET /api/selection_queries`
+- `POST /api/selection_query`
+- `POST /api/selection_triples`
+- `POST /api/sparql`
+
+## Repo Layout
+
+- `backend_go/` Go API and SPARQL snapshot logic
+- `frontend/` React/Vite UI
+- `radial_sugiyama/` Rust hierarchy layout and SVG export
+- `python_services/owl_imports_combiner/` import-flattening helper
+- `data/` local shared data mounted into containers
+- `docker-compose.yml` service wiring

backend_go/Dockerfile

@@ -11,12 +11,10 @@ FROM golang:${GO_VERSION}-alpine AS go-builder
 
 WORKDIR /src/backend_go
 
-COPY backend_go/go.mod /src/backend_go/go.mod
-
-RUN go mod download
-
 COPY backend_go /src/backend_go
 
+RUN go mod tidy
+
 RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/backend ./
 
 FROM debian:bookworm-slim

backend_go/README.md

@@ -27,10 +27,14 @@ Important variables:
   - `DEFAULT_NODE_LIMIT`, `DEFAULT_EDGE_LIMIT`
   - `MAX_NODE_LIMIT`, `MAX_EDGE_LIMIT`
 - SPARQL connectivity:
+  - `SPARQL_SOURCE_MODE` (`local` or `external`)
   - `SPARQL_HOST` (default `http://anzograph:8080`) or `SPARQL_ENDPOINT`
+  - `EXTERNAL_SPARQL_ENDPOINT` for external AnzoGraph access
+  - `KEYCLOAK_TOKEN_ENDPOINT`, `KEYCLOAK_CLIENT_ID`, `KEYCLOAK_USERNAME`, `KEYCLOAK_PASSWORD`, `KEYCLOAK_SCOPE`
   - `SPARQL_USER`, `SPARQL_PASS`
+  - External mode fetches a bearer token from Keycloak at startup, sends `Authorization: Bearer ...` to `EXTERNAL_SPARQL_ENDPOINT`, and refreshes once on `401 Unauthorized: Jwt is expired`
 - Startup behavior:
-  - `SPARQL_LOAD_ON_START`, `SPARQL_CLEAR_ON_START`
+  - `SPARQL_LOAD_ON_START`
   - `SPARQL_DATA_FILE` (typically `file:///opt/shared-files/<file>.ttl`)
 - Other:
   - `INCLUDE_BNODES` (include blank nodes in snapshots)
@@ -68,9 +72,9 @@ Stored under `backend_go/graph_queries/` and listed by `GET /api/graph_queries`.
 
 Built-in modes:
 
-- `default` – `rdf:type` (to `owl:Class`) + `rdfs:subClassOf`
-- `hierarchy` – `rdfs:subClassOf` only
-- `types` – `rdf:type` (to `owl:Class`) only
+- `default` – `rdf:type` + `rdfs:subClassOf`
+- `hierarchy` – `rdfs:subClassOf` + `rdf:type`
+- `types` – `rdf:type` only
 
 To add a new mode:
 
@@ -94,5 +98,6 @@ To add a new mode:
 
 ## Performance notes
 
-- Memory usage is dominated by the cached snapshot (`[]Node`, `[]Edge`) and the temporary SPARQL JSON unmarshalling step.
+- Memory usage is dominated by the cached snapshot (`[]Node`, `[]Edge`) and large SPARQL result sets.
+- The backend streams SPARQL JSON bindings for snapshot edge batches to reduce decode overhead.
 - Tune `DEFAULT_NODE_LIMIT`/`DEFAULT_EDGE_LIMIT` first if memory is too high.

backend_go/config.go

@@ -23,15 +23,22 @@ type Config struct {
 	FreeOSMemoryAfterSnapshot bool
 	LogSnapshotTimings        bool
 
+	SparqlSourceMode   string
 	SparqlHost         string
 	SparqlEndpoint     string
+	ExternalSparqlEndpoint string
+	AccessToken        string
+	KeycloakTokenEndpoint string
+	KeycloakClientID   string
+	KeycloakUsername   string
+	KeycloakPassword   string
+	KeycloakScope      string
 	SparqlUser         string
 	SparqlPass         string
 	SparqlInsecureTLS  bool
 	SparqlDataFile     string
 	SparqlGraphIRI     string
 	SparqlLoadOnStart  bool
-	SparqlClearOnStart bool
 
 	SparqlTimeout      time.Duration
 	SparqlReadyRetries int
@@ -60,20 +67,27 @@ func LoadConfig() (Config, error) {
 		FreeOSMemoryAfterSnapshot: envBool("FREE_OS_MEMORY_AFTER_SNAPSHOT", false),
 		LogSnapshotTimings:        envBool("LOG_SNAPSHOT_TIMINGS", false),
 
+		SparqlSourceMode:      envString("SPARQL_SOURCE_MODE", "local"),
 		SparqlHost:         envString("SPARQL_HOST", "http://anzograph:8080"),
 		SparqlEndpoint:     envString("SPARQL_ENDPOINT", ""),
+		ExternalSparqlEndpoint: envString("EXTERNAL_SPARQL_ENDPOINT", ""),
+		AccessToken:        envString("ACCESS_TOKEN", ""),
+		KeycloakTokenEndpoint: envString("KEYCLOAK_TOKEN_ENDPOINT", ""),
+		KeycloakClientID:   envString("KEYCLOAK_CLIENT_ID", ""),
+		KeycloakUsername:   envString("KEYCLOAK_USERNAME", ""),
+		KeycloakPassword:   envString("KEYCLOAK_PASSWORD", ""),
+		KeycloakScope:      envString("KEYCLOAK_SCOPE", "openid"),
 		SparqlUser:         envString("SPARQL_USER", ""),
 		SparqlPass:         envString("SPARQL_PASS", ""),
 		SparqlInsecureTLS:  envBool("SPARQL_INSECURE_TLS", false),
 		SparqlDataFile:     envString("SPARQL_DATA_FILE", ""),
 		SparqlGraphIRI:     envString("SPARQL_GRAPH_IRI", ""),
 		SparqlLoadOnStart:  envBool("SPARQL_LOAD_ON_START", false),
-		SparqlClearOnStart: envBool("SPARQL_CLEAR_ON_START", false),
 
-		HierarchyLayoutEngine:    envString("HIERARCHY_LAYOUT_ENGINE", "go"),
-		HierarchyLayoutBridgeBin: envString("HIERARCHY_LAYOUT_BRIDGE_BIN", "/app/radial_sugiyama_go_bridge"),
+		HierarchyLayoutEngine:       envString("HIERARCHY_LAYOUT_ENGINE", "go"),
+		HierarchyLayoutBridgeBin:    envString("HIERARCHY_LAYOUT_BRIDGE_BIN", "/app/radial_sugiyama_go_bridge"),
 		HierarchyLayoutBridgeWorkdir: envString("HIERARCHY_LAYOUT_BRIDGE_WORKDIR", "/workspace/radial_sugiyama"),
-		HierarchyLayoutRootIRI:   envString("HIERARCHY_LAYOUT_ROOT_IRI", "http://purl.obolibrary.org/obo/BFO_0000001"),
+		HierarchyLayoutRootIRI:      envString("HIERARCHY_LAYOUT_ROOT_IRI", "http://purl.obolibrary.org/obo/BFO_0000001"),
 
 		SparqlReadyRetries: envInt("SPARQL_READY_RETRIES", 30),
 		ListenAddr:         envString("LISTEN_ADDR", ":8000"),
@@ -100,6 +114,35 @@ func LoadConfig() (Config, error) {
 	if cfg.SparqlLoadOnStart && strings.TrimSpace(cfg.SparqlDataFile) == "" {
 		return Config{}, fmt.Errorf("SPARQL_LOAD_ON_START=true but SPARQL_DATA_FILE is not set")
 	}
+	switch strings.ToLower(strings.TrimSpace(cfg.SparqlSourceMode)) {
+	case "local", "external":
+		cfg.SparqlSourceMode = strings.ToLower(strings.TrimSpace(cfg.SparqlSourceMode))
+	default:
+		return Config{}, fmt.Errorf("SPARQL_SOURCE_MODE must be 'local' or 'external'")
+	}
+	if cfg.UsesExternalSparql() {
+		if strings.TrimSpace(cfg.ExternalSparqlEndpoint) == "" {
+			return Config{}, fmt.Errorf("EXTERNAL_SPARQL_ENDPOINT must be set when SPARQL_SOURCE_MODE=external")
+		}
+		if strings.TrimSpace(cfg.KeycloakTokenEndpoint) == "" {
+			return Config{}, fmt.Errorf("KEYCLOAK_TOKEN_ENDPOINT must be set when SPARQL_SOURCE_MODE=external")
+		}
+		if strings.TrimSpace(cfg.KeycloakClientID) == "" {
+			return Config{}, fmt.Errorf("KEYCLOAK_CLIENT_ID must be set when SPARQL_SOURCE_MODE=external")
+		}
+		if strings.TrimSpace(cfg.KeycloakUsername) == "" {
+			return Config{}, fmt.Errorf("KEYCLOAK_USERNAME must be set when SPARQL_SOURCE_MODE=external")
+		}
+		if strings.TrimSpace(cfg.KeycloakPassword) == "" {
+			return Config{}, fmt.Errorf("KEYCLOAK_PASSWORD must be set when SPARQL_SOURCE_MODE=external")
+		}
+		if strings.TrimSpace(cfg.KeycloakScope) == "" {
+			cfg.KeycloakScope = "openid"
+		}
+		if cfg.SparqlLoadOnStart {
+			return Config{}, fmt.Errorf("SPARQL_LOAD_ON_START is not supported when SPARQL_SOURCE_MODE=external")
+		}
+	}
 
 	if cfg.DefaultNodeLimit < 1 {
 		return Config{}, fmt.Errorf("DEFAULT_NODE_LIMIT must be >= 1")
@@ -148,12 +191,19 @@ func LoadConfig() (Config, error) {
 }
 
 func (c Config) EffectiveSparqlEndpoint() string {
+	if c.UsesExternalSparql() {
+		return strings.TrimSpace(c.ExternalSparqlEndpoint)
+	}
 	if strings.TrimSpace(c.SparqlEndpoint) != "" {
 		return strings.TrimSpace(c.SparqlEndpoint)
 	}
 	return strings.TrimRight(c.SparqlHost, "/") + "/sparql"
 }
 
+func (c Config) UsesExternalSparql() bool {
+	return strings.EqualFold(strings.TrimSpace(c.SparqlSourceMode), "external")
+}
+
 func (c Config) corsOriginList() []string {
 	raw := strings.TrimSpace(c.CorsOrigins)
 	if raw == "" || raw == "*" {

backend_go/go.mod

@@ -1,3 +1,5 @@
 module visualizador_instanciados/backend_go
 
 go 1.22
+
+require github.com/apache/arrow/go/v17 v17.0.0

backend_go/graph_arrow.go

@@ -0,0 +1,359 @@
+package main
+
+import (
+	"fmt"
+	"log"
+	"net/http"
+	"time"
+
+	"github.com/apache/arrow/go/v17/arrow"
+	"github.com/apache/arrow/go/v17/arrow/array"
+	"github.com/apache/arrow/go/v17/arrow/ipc"
+	"github.com/apache/arrow/go/v17/arrow/memory"
+)
+
+const graphTransportVersion = "1"
+
+const (
+	graphArrowMetaBackendField = iota
+	graphArrowMetaGraphQueryIDField
+	graphArrowMetaNodeLimitField
+	graphArrowMetaEdgeLimitField
+	graphArrowMetaNodesField
+	graphArrowMetaEdgesField
+	graphArrowMetaRouteLineSegmentsField
+	graphArrowNodeIDField
+	graphArrowNodeXField
+	graphArrowNodeYField
+	graphArrowNodeIRIField
+	graphArrowNodeLabelField
+	graphArrowEdgeSourceField
+	graphArrowEdgeTargetField
+	graphArrowRouteX1Field
+	graphArrowRouteY1Field
+	graphArrowRouteX2Field
+	graphArrowRouteY2Field
+)
+
+var graphArrowSchema = arrow.NewSchema([]arrow.Field{
+	{Name: "meta_backend", Type: arrow.BinaryTypes.String, Nullable: true},
+	{Name: "meta_graph_query_id", Type: arrow.BinaryTypes.String, Nullable: true},
+	{Name: "meta_node_limit", Type: arrow.PrimitiveTypes.Int32, Nullable: true},
+	{Name: "meta_edge_limit", Type: arrow.PrimitiveTypes.Int32, Nullable: true},
+	{Name: "meta_nodes", Type: arrow.PrimitiveTypes.Int32, Nullable: true},
+	{Name: "meta_edges", Type: arrow.PrimitiveTypes.Int32, Nullable: true},
+	{Name: "meta_route_line_segments", Type: arrow.PrimitiveTypes.Int32, Nullable: true},
+	{Name: "node_id", Type: arrow.ListOf(arrow.PrimitiveTypes.Uint32), Nullable: true},
+	{Name: "node_x", Type: arrow.ListOf(arrow.PrimitiveTypes.Float32), Nullable: true},
+	{Name: "node_y", Type: arrow.ListOf(arrow.PrimitiveTypes.Float32), Nullable: true},
+	{Name: "node_iri", Type: arrow.ListOf(arrow.BinaryTypes.String), Nullable: true},
+	{Name: "node_label", Type: arrow.ListOf(arrow.BinaryTypes.String), Nullable: true},
+	{Name: "edge_source", Type: arrow.ListOf(arrow.PrimitiveTypes.Uint32), Nullable: true},
+	{Name: "edge_target", Type: arrow.ListOf(arrow.PrimitiveTypes.Uint32), Nullable: true},
+	{Name: "route_x1", Type: arrow.ListOf(arrow.PrimitiveTypes.Float32), Nullable: true},
+	{Name: "route_y1", Type: arrow.ListOf(arrow.PrimitiveTypes.Float32), Nullable: true},
+	{Name: "route_x2", Type: arrow.ListOf(arrow.PrimitiveTypes.Float32), Nullable: true},
+	{Name: "route_y2", Type: arrow.ListOf(arrow.PrimitiveTypes.Float32), Nullable: true},
+}, nil)
+
+type graphArrowRouteLines struct {
+	X1 []float32
+	Y1 []float32
+	X2 []float32
+	Y2 []float32
+}
+
+func writeGraphArrow(w http.ResponseWriter, snap GraphResponse) error {
+	start := time.Now()
+	routes := flattenGraphRouteLines(snap.RouteSegments)
+	meta := snap.Meta
+	if meta == nil {
+		meta = &GraphMeta{
+			GraphQueryID: "default",
+			NodeLimit:    len(snap.Nodes),
+			EdgeLimit:    len(snap.Edges),
+			Nodes:        len(snap.Nodes),
+			Edges:        len(snap.Edges),
+		}
+	}
+
+	alloc := memory.NewGoAllocator()
+	w.Header().Set("Content-Type", "application/vnd.apache.arrow.stream")
+	w.Header().Set("X-Graph-Transport-Version", graphTransportVersion)
+	w.WriteHeader(http.StatusOK)
+
+	writer := ipc.NewWriter(w, ipc.WithSchema(graphArrowSchema))
+	closed := false
+	defer func() {
+		if !closed {
+			_ = writer.Close()
+		}
+	}()
+
+	if err := writeGraphArrowMetaBatch(writer, alloc, meta, len(routes.X1)); err != nil {
+		return fmt.Errorf("write meta batch failed: %w", err)
+	}
+	if err := writeGraphArrowNodeBatch(writer, alloc, snap.Nodes); err != nil {
+		return fmt.Errorf("write node batch failed: %w", err)
+	}
+	if err := writeGraphArrowEdgeBatch(writer, alloc, snap.Edges); err != nil {
+		return fmt.Errorf("write edge batch failed: %w", err)
+	}
+	if err := writeGraphArrowRouteBatch(writer, alloc, routes); err != nil {
+		return fmt.Errorf("write route batch failed: %w", err)
+	}
+	if err := writer.Close(); err != nil {
+		return fmt.Errorf("close arrow writer failed: %w", err)
+	}
+	closed = true
+
+	log.Printf(
+		"[graph-arrow] encode_done graph_query_id=%s nodes=%d edges=%d route_line_segments=%d encode_time=%s",
+		meta.GraphQueryID,
+		len(snap.Nodes),
+		len(snap.Edges),
+		len(routes.X1),
+		time.Since(start).Truncate(time.Millisecond),
+	)
+	return nil
+}
+
+func writeGraphArrowMetaBatch(writer *ipc.Writer, alloc memory.Allocator, meta *GraphMeta, routeLineSegments int) error {
+	builder := array.NewRecordBuilder(alloc, graphArrowSchema)
+	defer builder.Release()
+
+	appendStringBuilderValue(builder.Field(graphArrowMetaBackendField), meta.Backend)
+	appendStringBuilderValue(builder.Field(graphArrowMetaGraphQueryIDField), meta.GraphQueryID)
+	appendInt32BuilderValue(builder.Field(graphArrowMetaNodeLimitField), int32(meta.NodeLimit))
+	appendInt32BuilderValue(builder.Field(graphArrowMetaEdgeLimitField), int32(meta.EdgeLimit))
+	appendInt32BuilderValue(builder.Field(graphArrowMetaNodesField), int32(meta.Nodes))
+	appendInt32BuilderValue(builder.Field(graphArrowMetaEdgesField), int32(meta.Edges))
+	appendInt32BuilderValue(builder.Field(graphArrowMetaRouteLineSegmentsField), int32(routeLineSegments))
+
+	appendNullTopLevel(builder.Field(graphArrowNodeIDField))
+	appendNullTopLevel(builder.Field(graphArrowNodeXField))
+	appendNullTopLevel(builder.Field(graphArrowNodeYField))
+	appendNullTopLevel(builder.Field(graphArrowNodeIRIField))
+	appendNullTopLevel(builder.Field(graphArrowNodeLabelField))
+	appendNullTopLevel(builder.Field(graphArrowEdgeSourceField))
+	appendNullTopLevel(builder.Field(graphArrowEdgeTargetField))
+	appendNullTopLevel(builder.Field(graphArrowRouteX1Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteY1Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteX2Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteY2Field))
+
+	return writeGraphArrowRecord(writer, builder)
+}
+
+func writeGraphArrowNodeBatch(writer *ipc.Writer, alloc memory.Allocator, nodes []Node) error {
+	builder := array.NewRecordBuilder(alloc, graphArrowSchema)
+	defer builder.Release()
+
+	appendNullTopLevel(builder.Field(graphArrowMetaBackendField))
+	appendNullTopLevel(builder.Field(graphArrowMetaGraphQueryIDField))
+	appendNullTopLevel(builder.Field(graphArrowMetaNodeLimitField))
+	appendNullTopLevel(builder.Field(graphArrowMetaEdgeLimitField))
+	appendNullTopLevel(builder.Field(graphArrowMetaNodesField))
+	appendNullTopLevel(builder.Field(graphArrowMetaEdgesField))
+	appendNullTopLevel(builder.Field(graphArrowMetaRouteLineSegmentsField))
+
+	appendUint32List(builder.Field(graphArrowNodeIDField), func(valueBuilder *array.Uint32Builder) {
+		for _, node := range nodes {
+			valueBuilder.Append(node.ID)
+		}
+	})
+	appendFloat32List(builder.Field(graphArrowNodeXField), func(valueBuilder *array.Float32Builder) {
+		for _, node := range nodes {
+			valueBuilder.Append(float32(node.X))
+		}
+	})
+	appendFloat32List(builder.Field(graphArrowNodeYField), func(valueBuilder *array.Float32Builder) {
+		for _, node := range nodes {
+			valueBuilder.Append(float32(node.Y))
+		}
+	})
+	appendStringList(builder.Field(graphArrowNodeIRIField), func(valueBuilder *array.StringBuilder) {
+		for _, node := range nodes {
+			valueBuilder.Append(node.IRI)
+		}
+	})
+	appendNullableStringList(builder.Field(graphArrowNodeLabelField), func(valueBuilder *array.StringBuilder) {
+		for _, node := range nodes {
+			if node.Label == nil {
+				valueBuilder.AppendNull()
+				continue
+			}
+			valueBuilder.Append(*node.Label)
+		}
+	})
+
+	appendNullTopLevel(builder.Field(graphArrowEdgeSourceField))
+	appendNullTopLevel(builder.Field(graphArrowEdgeTargetField))
+	appendNullTopLevel(builder.Field(graphArrowRouteX1Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteY1Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteX2Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteY2Field))
+
+	return writeGraphArrowRecord(writer, builder)
+}
+
+func writeGraphArrowEdgeBatch(writer *ipc.Writer, alloc memory.Allocator, edges []Edge) error {
+	builder := array.NewRecordBuilder(alloc, graphArrowSchema)
+	defer builder.Release()
+
+	appendNullTopLevel(builder.Field(graphArrowMetaBackendField))
+	appendNullTopLevel(builder.Field(graphArrowMetaGraphQueryIDField))
+	appendNullTopLevel(builder.Field(graphArrowMetaNodeLimitField))
+	appendNullTopLevel(builder.Field(graphArrowMetaEdgeLimitField))
+	appendNullTopLevel(builder.Field(graphArrowMetaNodesField))
+	appendNullTopLevel(builder.Field(graphArrowMetaEdgesField))
+	appendNullTopLevel(builder.Field(graphArrowMetaRouteLineSegmentsField))
+	appendNullTopLevel(builder.Field(graphArrowNodeIDField))
+	appendNullTopLevel(builder.Field(graphArrowNodeXField))
+	appendNullTopLevel(builder.Field(graphArrowNodeYField))
+	appendNullTopLevel(builder.Field(graphArrowNodeIRIField))
+	appendNullTopLevel(builder.Field(graphArrowNodeLabelField))
+
+	appendUint32List(builder.Field(graphArrowEdgeSourceField), func(valueBuilder *array.Uint32Builder) {
+		for _, edge := range edges {
+			valueBuilder.Append(edge.Source)
+		}
+	})
+	appendUint32List(builder.Field(graphArrowEdgeTargetField), func(valueBuilder *array.Uint32Builder) {
+		for _, edge := range edges {
+			valueBuilder.Append(edge.Target)
+		}
+	})
+
+	appendNullTopLevel(builder.Field(graphArrowRouteX1Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteY1Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteX2Field))
+	appendNullTopLevel(builder.Field(graphArrowRouteY2Field))
+
+	return writeGraphArrowRecord(writer, builder)
+}
+
+func writeGraphArrowRouteBatch(writer *ipc.Writer, alloc memory.Allocator, routes graphArrowRouteLines) error {
+	builder := array.NewRecordBuilder(alloc, graphArrowSchema)
+	defer builder.Release()
+
+	appendNullTopLevel(builder.Field(graphArrowMetaBackendField))
+	appendNullTopLevel(builder.Field(graphArrowMetaGraphQueryIDField))
+	appendNullTopLevel(builder.Field(graphArrowMetaNodeLimitField))
+	appendNullTopLevel(builder.Field(graphArrowMetaEdgeLimitField))
+	appendNullTopLevel(builder.Field(graphArrowMetaNodesField))
+	appendNullTopLevel(builder.Field(graphArrowMetaEdgesField))
+	appendNullTopLevel(builder.Field(graphArrowMetaRouteLineSegmentsField))
+	appendNullTopLevel(builder.Field(graphArrowNodeIDField))
+	appendNullTopLevel(builder.Field(graphArrowNodeXField))
+	appendNullTopLevel(builder.Field(graphArrowNodeYField))
+	appendNullTopLevel(builder.Field(graphArrowNodeIRIField))
+	appendNullTopLevel(builder.Field(graphArrowNodeLabelField))
+	appendNullTopLevel(builder.Field(graphArrowEdgeSourceField))
+	appendNullTopLevel(builder.Field(graphArrowEdgeTargetField))
+
+	appendFloat32List(builder.Field(graphArrowRouteX1Field), func(valueBuilder *array.Float32Builder) {
+		for _, value := range routes.X1 {
+			valueBuilder.Append(value)
+		}
+	})
+	appendFloat32List(builder.Field(graphArrowRouteY1Field), func(valueBuilder *array.Float32Builder) {
+		for _, value := range routes.Y1 {
+			valueBuilder.Append(value)
+		}
+	})
+	appendFloat32List(builder.Field(graphArrowRouteX2Field), func(valueBuilder *array.Float32Builder) {
+		for _, value := range routes.X2 {
+			valueBuilder.Append(value)
+		}
+	})
+	appendFloat32List(builder.Field(graphArrowRouteY2Field), func(valueBuilder *array.Float32Builder) {
+		for _, value := range routes.Y2 {
+			valueBuilder.Append(value)
+		}
+	})
+
+	return writeGraphArrowRecord(writer, builder)
+}
+
+func writeGraphArrowRecord(writer *ipc.Writer, builder *array.RecordBuilder) error {
+	record := builder.NewRecord()
+	defer record.Release()
+	return writer.Write(record)
+}
+
+func appendStringBuilderValue(builder array.Builder, value string) {
+	builder.(*array.StringBuilder).Append(value)
+}
+
+func appendInt32BuilderValue(builder array.Builder, value int32) {
+	builder.(*array.Int32Builder).Append(value)
+}
+
+func appendUint32List(builder array.Builder, appendValues func(*array.Uint32Builder)) {
+	listBuilder := builder.(*array.ListBuilder)
+	listBuilder.Append(true)
+	valueBuilder := listBuilder.ValueBuilder().(*array.Uint32Builder)
+	appendValues(valueBuilder)
+}
+
+func appendFloat32List(builder array.Builder, appendValues func(*array.Float32Builder)) {
+	listBuilder := builder.(*array.ListBuilder)
+	listBuilder.Append(true)
+	valueBuilder := listBuilder.ValueBuilder().(*array.Float32Builder)
+	appendValues(valueBuilder)
+}
+
+func appendStringList(builder array.Builder, appendValues func(*array.StringBuilder)) {
+	listBuilder := builder.(*array.ListBuilder)
+	listBuilder.Append(true)
+	valueBuilder := listBuilder.ValueBuilder().(*array.StringBuilder)
+	appendValues(valueBuilder)
+}
+
+func appendNullableStringList(builder array.Builder, appendValues func(*array.StringBuilder)) {
+	listBuilder := builder.(*array.ListBuilder)
+	listBuilder.Append(true)
+	valueBuilder := listBuilder.ValueBuilder().(*array.StringBuilder)
+	appendValues(valueBuilder)
+}
+
+func appendNullTopLevel(builder array.Builder) {
+	switch typed := builder.(type) {
+	case *array.StringBuilder:
+		typed.AppendNull()
+	case *array.Int32Builder:
+		typed.AppendNull()
+	case *array.ListBuilder:
+		typed.AppendNull()
+	default:
+		panic(fmt.Sprintf("unsupported top-level builder type %T", builder))
+	}
+}
+
+func flattenGraphRouteLines(routeSegments []RouteSegment) graphArrowRouteLines {
+	lineCount := 0
+	for _, route := range routeSegments {
+		if len(route.Points) > 1 {
+			lineCount += len(route.Points) - 1
+		}
+	}
+
+	out := graphArrowRouteLines{
+		X1: make([]float32, 0, lineCount),
+		Y1: make([]float32, 0, lineCount),
+		X2: make([]float32, 0, lineCount),
+		Y2: make([]float32, 0, lineCount),
+	}
+	for _, route := range routeSegments {
+		for i := 1; i < len(route.Points); i++ {
+			prev := route.Points[i-1]
+			curr := route.Points[i]
+			out.X1 = append(out.X1, float32(prev.X))
+			out.Y1 = append(out.Y1, float32(prev.Y))
+			out.X2 = append(out.X2, float32(curr.X))
+			out.Y2 = append(out.Y2, float32(curr.Y))
+		}
+	}
+	return out
+}

backend_go/graph_export.go

@@ -5,10 +5,17 @@ type termKey struct {
 	key      string
 }
 
+type edgeKey struct {
+	source      uint32
+	target      uint32
+	predicateID uint32
+}
+
 type graphAccumulator struct {
 	includeBNodes bool
 	nodeLimit     int
 	nodeIDByKey   map[termKey]uint32
+	seenEdges     map[edgeKey]struct{}
 	nodes         []Node
 	edges         []Edge
 	preds         *PredicateDict
@@ -22,6 +29,7 @@ func newGraphAccumulator(nodeLimit int, includeBNodes bool, edgeCapHint int, pre
 		includeBNodes: includeBNodes,
 		nodeLimit:     nodeLimit,
 		nodeIDByKey:   make(map[termKey]uint32),
+		seenEdges:     make(map[edgeKey]struct{}, min(edgeCapHint, 4096)),
 		nodes:         make([]Node, 0, min(nodeLimit, 4096)),
 		edges:         make([]Edge, 0, min(edgeCapHint, 4096)),
 		preds:         preds,
@@ -63,29 +71,29 @@ func (g *graphAccumulator) getOrAddNode(term sparqlTerm) (uint32, bool) {
 	return nid, true
 }
 
-func (g *graphAccumulator) addBindings(bindings []map[string]sparqlTerm) {
-	for _, b := range bindings {
-		sTerm := b["s"]
-		oTerm := b["o"]
-		pTerm := b["p"]
-
-		sid, okS := g.getOrAddNode(sTerm)
-		oid, okO := g.getOrAddNode(oTerm)
-		if !okS || !okO {
-			continue
-		}
-
-		predID, ok := g.preds.GetOrAdd(pTerm.Value)
-		if !ok {
-			continue
-		}
-
-		g.edges = append(g.edges, Edge{
-			Source:      sid,
-			Target:      oid,
-			PredicateID: predID,
-		})
+func (g *graphAccumulator) addTripleBinding(binding sparqlTripleBinding) {
+	sid, okS := g.getOrAddNode(binding.S)
+	oid, okO := g.getOrAddNode(binding.O)
+	if !okS || !okO {
+		return
 	}
+
+	predID, ok := g.preds.GetOrAdd(binding.P.Value)
+	if !ok {
+		return
+	}
+
+	key := edgeKey{source: sid, target: oid, predicateID: predID}
+	if _, seen := g.seenEdges[key]; seen {
+		return
+	}
+	g.seenEdges[key] = struct{}{}
+
+	g.edges = append(g.edges, Edge{
+		Source:      sid,
+		Target:      oid,
+		PredicateID: predID,
+	})
 }
 
 func min(a, b int) int {

backend_go/graph_export_test.go

@@ -0,0 +1,24 @@
+package main
+
+import "testing"
+
+func TestGraphAccumulatorDeduplicatesEdges(t *testing.T) {
+	preds := NewPredicateDict([]string{"http://example.com/p"})
+	acc := newGraphAccumulator(16, false, 16, preds)
+
+	binding := sparqlTripleBinding{
+		S: sparqlTerm{Type: "uri", Value: "http://example.com/s"},
+		P: sparqlTerm{Type: "uri", Value: "http://example.com/p"},
+		O: sparqlTerm{Type: "uri", Value: "http://example.com/o"},
+	}
+
+	acc.addTripleBinding(binding)
+	acc.addTripleBinding(binding)
+
+	if len(acc.nodes) != 2 {
+		t.Fatalf("expected 2 nodes after duplicate bindings, got %d", len(acc.nodes))
+	}
+	if len(acc.edges) != 1 {
+		t.Fatalf("expected 1 deduplicated edge, got %d", len(acc.edges))
+	}
+}

backend_go/graph_queries/default.go

@@ -1,6 +1,10 @@
 package graph_queries
 
-import "fmt"
+import (
+	"fmt"
+
+	"visualizador_instanciados/backend_go/queryscope"
+)
 
 func defaultEdgeQuery(limit int, offset int, includeBNodes bool) string {
 	bnodeFilter := ""
@@ -8,30 +12,33 @@ func defaultEdgeQuery(limit int, offset int, includeBNodes bool) string {
 		bnodeFilter = "FILTER(!isBlank(?s) && !isBlank(?o))"
 	}
 
+	pattern := queryscope.NamedGraph(`
+{
+  VALUES ?p { rdf:type }
+  ?s ?p ?o .
+}
+UNION
+{
+  VALUES ?p { rdfs:subClassOf }
+  ?s ?p ?o .
+}
+`)
+
 	return fmt.Sprintf(`
 PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
 PREFIX owl: <http://www.w3.org/2002/07/owl#>
 
-SELECT ?s ?p ?o
+SELECT DISTINCT ?s ?p ?o
 WHERE {
-  {
-    VALUES ?p { rdf:type }
-    ?s ?p ?o .
-    ?o rdf:type owl:Class .
-  }
-  UNION
-  {
-    VALUES ?p { rdfs:subClassOf }
-    ?s ?p ?o .
-  }
+%s
   FILTER(!isLiteral(?o))
   %s
 }
 ORDER BY ?s ?p ?o
 LIMIT %d
 OFFSET %d
-`, bnodeFilter, limit, offset)
+`, pattern, bnodeFilter, limit, offset)
 }
 
 func defaultPredicateQuery(includeBNodes bool) string {
@@ -40,6 +47,18 @@ func defaultPredicateQuery(includeBNodes bool) string {
 		bnodeFilter = "FILTER(!isBlank(?s) && !isBlank(?o))"
 	}
 
+	pattern := queryscope.NamedGraph(`
+{
+  VALUES ?p { rdf:type }
+  ?s ?p ?o .
+}
+UNION
+{
+  VALUES ?p { rdfs:subClassOf }
+  ?s ?p ?o .
+}
+`)
+
 	return fmt.Sprintf(`
 PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
@@ -47,19 +66,10 @@ PREFIX owl: <http://www.w3.org/2002/07/owl#>
 
 SELECT DISTINCT ?p
 WHERE {
-  {
-    VALUES ?p { rdf:type }
-    ?s ?p ?o .
-    ?o rdf:type owl:Class .
-  }
-  UNION
-  {
-    VALUES ?p { rdfs:subClassOf }
-    ?s ?p ?o .
-  }
+%s
   FILTER(!isLiteral(?o))
   %s
 }
 ORDER BY ?p
-`, bnodeFilter)
+`, pattern, bnodeFilter)
 }

backend_go/graph_queries/hierarchy.go

@@ -1,6 +1,10 @@
 package graph_queries
 
-import "fmt"
+import (
+	"fmt"
+
+	"visualizador_instanciados/backend_go/queryscope"
+)
 
 func hierarchyEdgeQuery(limit int, offset int, includeBNodes bool) string {
 	bnodeFilter := ""
@@ -8,20 +12,31 @@ func hierarchyEdgeQuery(limit int, offset int, includeBNodes bool) string {
 		bnodeFilter = "FILTER(!isBlank(?s) && !isBlank(?o))"
 	}
 
+	pattern := queryscope.NamedGraph(`
+{
+VALUES ?p { rdfs:subClassOf }
+?s ?p ?o .
+}
+UNION
+{
+VALUES ?p { rdf:type }
+?s ?p ?o .
+}
+`)
+
 	return fmt.Sprintf(`
 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
 
-SELECT ?s ?p ?o
+SELECT DISTINCT ?s ?p ?o
 WHERE {
-  VALUES ?p { rdfs:subClassOf }
-  ?s ?p ?o .
+%s
   FILTER(!isLiteral(?o))
   %s
 }
 ORDER BY ?s ?p ?o
 LIMIT %d
 OFFSET %d
-`, bnodeFilter, limit, offset)
+`, pattern, bnodeFilter, limit, offset)
 }
 
 func hierarchyPredicateQuery(includeBNodes bool) string {
@@ -30,16 +45,27 @@ func hierarchyPredicateQuery(includeBNodes bool) string {
 		bnodeFilter = "FILTER(!isBlank(?s) && !isBlank(?o))"
 	}
 
+	pattern := queryscope.NamedGraph(`
+{
+VALUES ?p { rdfs:subClassOf }
+?s ?p ?o .
+}
+UNION
+{
+VALUES ?p { rdf:type }
+?s ?p ?o .
+}
+`)
+
 	return fmt.Sprintf(`
 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
 
 SELECT DISTINCT ?p
 WHERE {
-  VALUES ?p { rdfs:subClassOf }
-  ?s ?p ?o .
+%s
   FILTER(!isLiteral(?o))
   %s
 }
 ORDER BY ?p
-`, bnodeFilter)
+`, pattern, bnodeFilter)
 }

backend_go/graph_queries/named_graph_test.go

@@ -0,0 +1,49 @@
+package graph_queries
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestEdgeQueriesUseNamedGraphsAndDistinct(t *testing.T) {
+	tests := []struct {
+		name  string
+		query string
+	}{
+		{name: "default", query: defaultEdgeQuery(100, 25, false)},
+		{name: "hierarchy", query: hierarchyEdgeQuery(100, 25, false)},
+		{name: "types_only", query: typesOnlyEdgeQuery(100, 25, false)},
+	}
+
+	for _, tt := range tests {
+		if !strings.Contains(tt.query, "SELECT DISTINCT ?s ?p ?o") {
+			t.Fatalf("%s edge query should de-duplicate triples across named graphs:\n%s", tt.name, tt.query)
+		}
+		if !strings.Contains(tt.query, "GRAPH ?g") {
+			t.Fatalf("%s edge query should read from named graphs:\n%s", tt.name, tt.query)
+		}
+		if strings.Contains(tt.query, "owl:Class") {
+			t.Fatalf("%s edge query should no longer require owl:Class declarations:\n%s", tt.name, tt.query)
+		}
+	}
+}
+
+func TestPredicateQueriesUseNamedGraphs(t *testing.T) {
+	tests := []struct {
+		name  string
+		query string
+	}{
+		{name: "default", query: defaultPredicateQuery(false)},
+		{name: "hierarchy", query: hierarchyPredicateQuery(false)},
+		{name: "types_only", query: typesOnlyPredicateQuery(false)},
+	}
+
+	for _, tt := range tests {
+		if !strings.Contains(tt.query, "SELECT DISTINCT ?p") {
+			t.Fatalf("%s predicate query should remain distinct:\n%s", tt.name, tt.query)
+		}
+		if !strings.Contains(tt.query, "GRAPH ?g") {
+			t.Fatalf("%s predicate query should read from named graphs:\n%s", tt.name, tt.query)
+		}
+	}
+}

backend_go/graph_queries/types_only.go

@@ -1,6 +1,10 @@
 package graph_queries
 
-import "fmt"
+import (
+	"fmt"
+
+	"visualizador_instanciados/backend_go/queryscope"
+)
 
 func typesOnlyEdgeQuery(limit int, offset int, includeBNodes bool) string {
 	bnodeFilter := ""
@@ -8,22 +12,25 @@ func typesOnlyEdgeQuery(limit int, offset int, includeBNodes bool) string {
 		bnodeFilter = "FILTER(!isBlank(?s) && !isBlank(?o))"
 	}
 
+	pattern := queryscope.NamedGraph(`
+VALUES ?p { rdf:type }
+?s ?p ?o .
+`)
+
 	return fmt.Sprintf(`
 PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
 PREFIX owl: <http://www.w3.org/2002/07/owl#>
 
-SELECT ?s ?p ?o
+SELECT DISTINCT ?s ?p ?o
 WHERE {
-  VALUES ?p { rdf:type }
-  ?s ?p ?o .
-  ?o rdf:type owl:Class .
+%s
   FILTER(!isLiteral(?o))
   %s
 }
 ORDER BY ?s ?p ?o
 LIMIT %d
 OFFSET %d
-`, bnodeFilter, limit, offset)
+`, pattern, bnodeFilter, limit, offset)
 }
 
 func typesOnlyPredicateQuery(includeBNodes bool) string {
@@ -32,18 +39,21 @@ func typesOnlyPredicateQuery(includeBNodes bool) string {
 		bnodeFilter = "FILTER(!isBlank(?s) && !isBlank(?o))"
 	}
 
+	pattern := queryscope.NamedGraph(`
+VALUES ?p { rdf:type }
+?s ?p ?o .
+`)
+
 	return fmt.Sprintf(`
 PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
 PREFIX owl: <http://www.w3.org/2002/07/owl#>
 
 SELECT DISTINCT ?p
 WHERE {
-  VALUES ?p { rdf:type }
-  ?s ?p ?o .
-  ?o rdf:type owl:Class .
+%s
   FILTER(!isLiteral(?o))
   %s
 }
 ORDER BY ?p
-`, bnodeFilter)
+`, pattern, bnodeFilter)
 }

backend_go/graph_snapshot.go

@@ -2,7 +2,6 @@ package main
 
 import (
 	"context"
-	"encoding/json"
 	"fmt"
 	"log"
 	"runtime"
@@ -12,6 +11,7 @@ import (
 	"time"
 
 	graphqueries "visualizador_instanciados/backend_go/graph_queries"
+	"visualizador_instanciados/backend_go/queryscope"
 )
 
 const (
@@ -56,29 +56,26 @@ func fetchGraphSnapshot(
 	preds, err := func() (*PredicateDict, error) {
 		logStats("predicates_query_start")
 		predQ := def.PredicateQuery(cfg.IncludeBNodes)
-		t0 := time.Now()
-		rawPred, err := sparql.Query(ctx, predQ)
+		var predRes sparqlBindingsResponse[sparqlPredicateBinding]
+		metrics, err := sparql.QueryJSON(ctx, predQ, &predRes)
 		if err != nil {
 			return nil, fmt.Errorf("predicates query failed: %w", err)
 		}
 		if cfg.LogSnapshotTimings {
-			log.Printf("[snapshot] predicates_query_returned bytes=%d query_time=%s", len(rawPred), time.Since(t0).Truncate(time.Millisecond))
-		}
-		var predRes sparqlResponse
-		t1 := time.Now()
-		if err := json.Unmarshal(rawPred, &predRes); err != nil {
-			return nil, fmt.Errorf("predicates unmarshal failed: %w", err)
-		}
-		if cfg.LogSnapshotTimings {
-			log.Printf("[snapshot] predicates_unmarshal_done bindings=%d unmarshal_time=%s", len(predRes.Results.Bindings), time.Since(t1).Truncate(time.Millisecond))
+			log.Printf(
+				"[snapshot] predicates_query_done bytes=%d bindings=%d round_trip_time=%s decode_time=%s",
+				metrics.ResponseBytes,
+				len(predRes.Results.Bindings),
+				metrics.RoundTripTime.Truncate(time.Millisecond),
+				metrics.BodyDecodeTime.Truncate(time.Millisecond),
+			)
 		}
 		predicateIRIs := make([]string, 0, len(predRes.Results.Bindings))
 		for _, b := range predRes.Results.Bindings {
-			pTerm, ok := b["p"]
-			if !ok || pTerm.Type != "uri" || pTerm.Value == "" {
+			if b.P.Type != "uri" || b.P.Value == "" {
 				continue
 			}
-			predicateIRIs = append(predicateIRIs, pTerm.Value)
+			predicateIRIs = append(predicateIRIs, b.P.Value)
 		}
 		logStats("predicates_dict_built")
 		return NewPredicateDict(predicateIRIs), nil
@@ -102,55 +99,48 @@ func fetchGraphSnapshot(
 		}
 
 		logStats(fmt.Sprintf("edges_batch_start batch=%d offset=%d limit=%d", batch, offset, limit))
-		bindings, err := func() ([]map[string]sparqlTerm, error) {
-			edgesQ := def.EdgeQuery(limit, offset, cfg.IncludeBNodes)
-			t0 := time.Now()
-			raw, err := sparql.Query(ctx, edgesQ)
-			if err != nil {
-				return nil, fmt.Errorf("edges query failed: %w", err)
-			}
-			if cfg.LogSnapshotTimings {
-				log.Printf("[snapshot] edges_batch_query_returned batch=%d offset=%d limit=%d bytes=%d query_time=%s", batch, offset, limit, len(raw), time.Since(t0).Truncate(time.Millisecond))
+		edgesQ := def.EdgeQuery(limit, offset, cfg.IncludeBNodes)
+		var batchConvertTime time.Duration
+		metrics, err := sparql.QueryTripleBindingsStream(ctx, edgesQ, func(binding sparqlTripleBinding) error {
+			if !cfg.LogSnapshotTimings {
+				acc.addTripleBinding(binding)
+				return nil
 			}
 
-			var res sparqlResponse
-			t1 := time.Now()
-			if err := json.Unmarshal(raw, &res); err != nil {
-				return nil, fmt.Errorf("edges unmarshal failed: %w", err)
-			}
-			if cfg.LogSnapshotTimings {
-				log.Printf("[snapshot] edges_batch_unmarshal_done batch=%d bindings=%d unmarshal_time=%s", batch, len(res.Results.Bindings), time.Since(t1).Truncate(time.Millisecond))
-			}
-			return res.Results.Bindings, nil
-		}()
+			convertStart := time.Now()
+			acc.addTripleBinding(binding)
+			batchConvertTime += time.Since(convertStart)
+			return nil
+		})
 		if err != nil {
 			return GraphResponse{}, fmt.Errorf("edges batch=%d offset=%d limit=%d: %w", batch, offset, limit, err)
 		}
 
-		got := len(bindings)
+		got := metrics.BindingCount
 		totalBindings += got
 		if got == 0 {
-			bindings = nil
 			logStats(fmt.Sprintf("edges_batch_done_empty batch=%d offset=%d", batch, offset))
 			break
 		}
 
-		convT0 := time.Now()
-		acc.addBindings(bindings)
 		if cfg.LogSnapshotTimings {
 			log.Printf(
-				"[snapshot] edges_batch_convert_done batch=%d got_bindings=%d total_bindings=%d nodes=%d edges=%d convert_time=%s",
+				"[snapshot] edges_batch_stream_done batch=%d offset=%d limit=%d bytes=%d got_bindings=%d total_bindings=%d round_trip_time=%s stream_time=%s decode_overhead_time=%s convert_time=%s nodes=%d edges=%d",
 				batch,
+				offset,
+				limit,
+				metrics.ResponseBytes,
 				got,
 				totalBindings,
+				metrics.RoundTripTime.Truncate(time.Millisecond),
+				metrics.BodyDecodeTime.Truncate(time.Millisecond),
+				maxDuration(metrics.BodyDecodeTime-batchConvertTime, 0).Truncate(time.Millisecond),
+				batchConvertTime.Truncate(time.Millisecond),
 				len(acc.nodes),
 				len(acc.edges),
-				time.Since(convT0).Truncate(time.Millisecond),
 			)
 		}
 
-		// Make the batch eligible for GC.
-		bindings = nil
 		logStats(fmt.Sprintf("edges_batch_done batch=%d offset=%d", batch, offset))
 		if cfg.FreeOSMemoryAfterSnapshot {
 			debug.FreeOSMemory()
@@ -165,6 +155,13 @@ func fetchGraphSnapshot(
 		log.Printf("[snapshot] convert_batches_done total_bindings=%d total_time=%s", totalBindings, time.Since(convAllT0).Truncate(time.Millisecond))
 	}
 	logStats("edges_batched_done")
+	if totalBindings == 0 {
+		log.Printf(
+			"[snapshot] empty_graph_result graph_query_id=%s endpoint=%s hint=app-generated reads now query named graphs only with GRAPH ?g; verify expected triples are present in named graphs and match the graph query shape",
+			graphQueryID,
+			cfg.EffectiveSparqlEndpoint(),
+		)
+	}
 
 	nodes := acc.nodes
 	edges := acc.edges
@@ -283,43 +280,26 @@ func fetchRDFSLabels(
 		}
 		batch := iris[i:end]
 
-		values := make([]string, 0, len(batch))
-		for _, u := range batch {
-			values = append(values, "<"+u+">")
-		}
+		q := rdfsLabelQuery(batch)
 
-		q := fmt.Sprintf(`
-SELECT ?s ?label
-WHERE {
-  VALUES ?s { %s }
-  ?s <%s> ?label .
-}
-`, strings.Join(values, " "), rdfsLabelIRI)
-
-		raw, err := sparql.Query(ctx, q)
+		var res sparqlBindingsResponse[sparqlLabelBinding]
+		_, err := sparql.QueryJSON(ctx, q, &res)
 		if err != nil {
 			return nil, err
 		}
 
-		var res sparqlResponse
-		if err := json.Unmarshal(raw, &res); err != nil {
-			return nil, fmt.Errorf("failed to parse SPARQL JSON: %w", err)
-		}
-
 		for _, b := range res.Results.Bindings {
-			sTerm, ok := b["s"]
-			if !ok || sTerm.Value == "" {
+			if b.S.Value == "" {
 				continue
 			}
-			lblTerm, ok := b["label"]
-			if !ok || lblTerm.Type != "literal" || lblTerm.Value == "" {
+			if b.Label.Type != "literal" || b.Label.Value == "" {
 				continue
 			}
 
-			score := labelScore(lblTerm.Lang)
-			prev, ok := best[sTerm.Value]
+			score := labelScore(b.Label.Lang)
+			prev, ok := best[b.S.Value]
 			if !ok || score > prev.score {
-				best[sTerm.Value] = bestLabel{score: score, value: lblTerm.Value}
+				best[b.S.Value] = bestLabel{score: score, value: b.Label.Value}
 			}
 		}
 	}
@@ -331,6 +311,35 @@ WHERE {
 	return out, nil
 }
 
+func rdfsLabelQuery(iris []string) string {
+	if len(iris) == 0 {
+		return "SELECT ?s ?label WHERE { FILTER(false) }"
+	}
+
+	values := make([]string, 0, len(iris))
+	for _, u := range iris {
+		if strings.TrimSpace(u) == "" {
+			continue
+		}
+		values = append(values, "<"+u+">")
+	}
+	if len(values) == 0 {
+		return "SELECT ?s ?label WHERE { FILTER(false) }"
+	}
+
+	pattern := queryscope.NamedGraph(fmt.Sprintf(`
+VALUES ?s { %s }
+?s <%s> ?label .
+`, strings.Join(values, " "), rdfsLabelIRI))
+
+	return fmt.Sprintf(`
+SELECT DISTINCT ?s ?label
+WHERE {
+%s
+}
+`, pattern)
+}
+
 func labelScore(lang string) int {
 	lang = strings.ToLower(strings.TrimSpace(lang))
 	if lang == "en" {
@@ -357,3 +366,10 @@ func sortIntsUnique(xs []int) []int {
 	}
 	return out
 }
+
+func maxDuration(a time.Duration, b time.Duration) time.Duration {
+	if a > b {
+		return a
+	}
+	return b
+}

backend_go/graph_snapshot_named_graph_test.go

@@ -0,0 +1,23 @@
+package main
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestRDFSLabelQueryUsesNamedGraphs(t *testing.T) {
+	query := rdfsLabelQuery([]string{
+		"http://example.com/A",
+		"http://example.com/B",
+	})
+
+	if !strings.Contains(query, "SELECT DISTINCT ?s ?label") {
+		t.Fatalf("label query should de-duplicate rows across named graphs:\n%s", query)
+	}
+	if !strings.Contains(query, "GRAPH ?g") {
+		t.Fatalf("label query should read from named graphs:\n%s", query)
+	}
+	if !strings.Contains(query, "<"+rdfsLabelIRI+">") {
+		t.Fatalf("label query should still fetch rdfs:label:\n%s", query)
+	}
+}

backend_go/keycloak_token.go

@@ -0,0 +1,149 @@
+package main
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"log"
+	"net/http"
+	"net/url"
+	"strings"
+	"sync"
+	"time"
+)
+
+type keycloakTokenResponse struct {
+	AccessToken string `json:"access_token"`
+}
+
+type keycloakTokenManager struct {
+	cfg    Config
+	client *http.Client
+
+	mu        sync.Mutex
+	token     string
+	refreshCh chan struct{}
+	lastErr   error
+}
+
+func newKeycloakTokenManager(cfg Config, client *http.Client) *keycloakTokenManager {
+	return &keycloakTokenManager{
+		cfg:    cfg,
+		client: client,
+		token:  strings.TrimSpace(cfg.AccessToken),
+	}
+}
+
+func (m *keycloakTokenManager) CurrentToken() string {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	return strings.TrimSpace(m.token)
+}
+
+func (m *keycloakTokenManager) EnsureToken(ctx context.Context, reason string) (string, error) {
+	if token := m.CurrentToken(); token != "" {
+		return token, nil
+	}
+	return m.Refresh(ctx, reason)
+}
+
+func (m *keycloakTokenManager) Refresh(ctx context.Context, reason string) (string, error) {
+	m.mu.Lock()
+	if ch := m.refreshCh; ch != nil {
+		m.mu.Unlock()
+		select {
+		case <-ctx.Done():
+			return "", ctx.Err()
+		case <-ch:
+			m.mu.Lock()
+			token := strings.TrimSpace(m.token)
+			err := m.lastErr
+			m.mu.Unlock()
+			if err != nil {
+				return "", err
+			}
+			if token == "" {
+				return "", fmt.Errorf("keycloak token refresh completed without access_token")
+			}
+			return token, nil
+		}
+	}
+
+	ch := make(chan struct{})
+	m.refreshCh = ch
+	m.mu.Unlock()
+
+	log.Printf("[auth] keycloak_token_refresh_start reason=%s endpoint=%s", reason, m.cfg.KeycloakTokenEndpoint)
+	start := time.Now()
+	token, err := m.fetchToken(ctx)
+	if err != nil {
+		log.Printf("[auth] keycloak_token_refresh_failed reason=%s endpoint=%s err=%v", reason, m.cfg.KeycloakTokenEndpoint, err)
+	} else {
+		log.Printf(
+			"[auth] keycloak_token_refresh_ok reason=%s endpoint=%s elapsed=%s",
+			reason,
+			m.cfg.KeycloakTokenEndpoint,
+			time.Since(start).Truncate(time.Millisecond),
+		)
+	}
+
+	m.mu.Lock()
+	if err == nil {
+		m.token = token
+	}
+	m.lastErr = err
+	close(ch)
+	m.refreshCh = nil
+	currentToken := strings.TrimSpace(m.token)
+	m.mu.Unlock()
+
+	if err != nil {
+		return "", err
+	}
+	return currentToken, nil
+}
+
+func (m *keycloakTokenManager) fetchToken(ctx context.Context) (string, error) {
+	form := url.Values{}
+	form.Set("grant_type", "password")
+	form.Set("client_id", strings.TrimSpace(m.cfg.KeycloakClientID))
+	form.Set("username", strings.TrimSpace(m.cfg.KeycloakUsername))
+	form.Set("password", m.cfg.KeycloakPassword)
+	scope := strings.TrimSpace(m.cfg.KeycloakScope)
+	if scope != "" {
+		form.Set("scope", scope)
+	}
+
+	req, err := http.NewRequestWithContext(ctx, http.MethodPost, strings.TrimSpace(m.cfg.KeycloakTokenEndpoint), strings.NewReader(form.Encode()))
+	if err != nil {
+		return "", err
+	}
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+	req.Header.Set("Accept", "application/json")
+
+	resp, err := m.client.Do(req)
+	if err != nil {
+		return "", err
+	}
+	defer resp.Body.Close()
+
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return "", err
+	}
+	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+		return "", fmt.Errorf("keycloak token request failed: %s: %s", resp.Status, strings.TrimSpace(string(body)))
+	}
+
+	var tokenResp keycloakTokenResponse
+	if err := json.Unmarshal(body, &tokenResp); err != nil {
+		return "", fmt.Errorf("keycloak token parse failed: %w", err)
+	}
+
+	token := strings.TrimSpace(tokenResp.AccessToken)
+	if token == "" {
+		return "", fmt.Errorf("keycloak token response missing access_token")
+	}
+	return token, nil
+}

backend_go/keycloak_token_test.go

@@ -0,0 +1,273 @@
+package main
+
+import (
+	"context"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+func TestKeycloakTokenManagerFetchTokenParsesAccessToken(t *testing.T) {
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path != "/token" {
+			http.NotFound(w, r)
+			return
+		}
+		if got := r.Header.Get("Content-Type"); got != "application/x-www-form-urlencoded" {
+			t.Errorf("unexpected content-type: %s", got)
+			w.WriteHeader(http.StatusInternalServerError)
+			return
+		}
+		_, _ = io.WriteString(w, `{"access_token":"fresh-token"}`)
+	}))
+	defer server.Close()
+
+	cfg := Config{
+		KeycloakTokenEndpoint: server.URL + "/token",
+		KeycloakClientID:      "anzograph",
+		KeycloakUsername:      "user",
+		KeycloakPassword:      "pass",
+		KeycloakScope:         "openid",
+	}
+	manager := newKeycloakTokenManager(cfg, server.Client())
+
+	token, err := manager.fetchToken(context.Background())
+	if err != nil {
+		t.Fatalf("fetchToken returned error: %v", err)
+	}
+	if token != "fresh-token" {
+		t.Fatalf("expected fresh-token, got %q", token)
+	}
+}
+
+func TestAnzoGraphClientStartupFetchesFreshToken(t *testing.T) {
+	var tokenCalls atomic.Int32
+
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/token":
+			tokenCalls.Add(1)
+			_, _ = io.WriteString(w, `{"access_token":"startup-token"}`)
+		case "/sparql":
+			if got := r.Header.Get("Authorization"); got != "Bearer startup-token" {
+				t.Errorf("expected startup bearer token, got %q", got)
+				w.WriteHeader(http.StatusInternalServerError)
+				return
+			}
+			_, _ = io.WriteString(w, `{"head":{},"boolean":true}`)
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	defer server.Close()
+
+	cfg := Config{
+		SparqlSourceMode:       "external",
+		ExternalSparqlEndpoint: server.URL + "/sparql",
+		KeycloakTokenEndpoint:  server.URL + "/token",
+		KeycloakClientID:       "anzograph",
+		KeycloakUsername:       "user",
+		KeycloakPassword:       "pass",
+		KeycloakScope:          "openid",
+		SparqlReadyTimeout:     2 * time.Second,
+		SparqlReadyRetries:     1,
+		SparqlReadyDelay:       1 * time.Millisecond,
+		SparqlTimeout:          2 * time.Second,
+	}
+
+	client := NewAnzoGraphClient(cfg)
+	client.client = server.Client()
+	client.tokenManager.client = server.Client()
+
+	if err := client.Startup(context.Background()); err != nil {
+		t.Fatalf("Startup returned error: %v", err)
+	}
+	if tokenCalls.Load() != 1 {
+		t.Fatalf("expected 1 startup token request, got %d", tokenCalls.Load())
+	}
+}
+
+func TestQueryRetriesOnceWhenJWTExpires(t *testing.T) {
+	var tokenCalls atomic.Int32
+
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/token":
+			call := tokenCalls.Add(1)
+			if call != 1 {
+				t.Errorf("expected exactly 1 refresh call, got %d", call)
+				w.WriteHeader(http.StatusInternalServerError)
+				return
+			}
+			_, _ = io.WriteString(w, `{"access_token":"fresh-token"}`)
+		case "/sparql":
+			switch r.Header.Get("Authorization") {
+			case "Bearer expired-token":
+				w.WriteHeader(http.StatusUnauthorized)
+				_, _ = io.WriteString(w, "Jwt is expired")
+			case "Bearer fresh-token":
+				_, _ = io.WriteString(w, `{"results":{"bindings":[{"s":{"type":"uri","value":"http://example.com/s"},"p":{"type":"uri","value":"http://example.com/p"},"o":{"type":"uri","value":"http://example.com/o"}}]}}`)
+			default:
+				t.Errorf("unexpected authorization header %q", r.Header.Get("Authorization"))
+				w.WriteHeader(http.StatusInternalServerError)
+				return
+			}
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	defer server.Close()
+
+	cfg := Config{
+		SparqlSourceMode:       "external",
+		ExternalSparqlEndpoint: server.URL + "/sparql",
+		KeycloakTokenEndpoint:  server.URL + "/token",
+		KeycloakClientID:       "anzograph",
+		KeycloakUsername:       "user",
+		KeycloakPassword:       "pass",
+		KeycloakScope:          "openid",
+		SparqlTimeout:          2 * time.Second,
+	}
+
+	client := NewAnzoGraphClient(cfg)
+	client.client = server.Client()
+	client.tokenManager.client = server.Client()
+	client.tokenManager.token = "expired-token"
+
+	raw, err := client.Query(context.Background(), "SELECT ?s ?p ?o WHERE { ?s ?p ?o }")
+	if err != nil {
+		t.Fatalf("Query returned error: %v", err)
+	}
+	if string(raw) == "" {
+		t.Fatalf("expected successful response body after refresh")
+	}
+	if tokenCalls.Load() != 1 {
+		t.Fatalf("expected 1 refresh call, got %d", tokenCalls.Load())
+	}
+}
+
+func TestQueryDoesNotRefreshForNonExpiry401(t *testing.T) {
+	var tokenCalls atomic.Int32
+
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/token":
+			tokenCalls.Add(1)
+			_, _ = io.WriteString(w, `{"access_token":"fresh-token"}`)
+		case "/sparql":
+			w.WriteHeader(http.StatusUnauthorized)
+			_, _ = io.WriteString(w, "RBAC: access denied")
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	defer server.Close()
+
+	cfg := Config{
+		SparqlSourceMode:       "external",
+		ExternalSparqlEndpoint: server.URL + "/sparql",
+		KeycloakTokenEndpoint:  server.URL + "/token",
+		KeycloakClientID:       "anzograph",
+		KeycloakUsername:       "user",
+		KeycloakPassword:       "pass",
+		KeycloakScope:          "openid",
+		SparqlTimeout:          2 * time.Second,
+	}
+
+	client := NewAnzoGraphClient(cfg)
+	client.client = server.Client()
+	client.tokenManager.client = server.Client()
+	client.tokenManager.token = "still-bad-token"
+
+	_, err := client.Query(context.Background(), "SELECT ?s ?p ?o WHERE { ?s ?p ?o }")
+	if err == nil {
+		t.Fatalf("expected non-expiry 401 to fail")
+	}
+	if tokenCalls.Load() != 0 {
+		t.Fatalf("expected no token refresh for non-expiry 401, got %d", tokenCalls.Load())
+	}
+}
+
+func TestConcurrentExpiredQueriesShareOneRefresh(t *testing.T) {
+	var tokenCalls atomic.Int32
+	var sparqlCalls atomic.Int32
+	var mu sync.Mutex
+	seenFresh := 0
+
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/token":
+			tokenCalls.Add(1)
+			time.Sleep(50 * time.Millisecond)
+			_, _ = io.WriteString(w, `{"access_token":"fresh-token"}`)
+		case "/sparql":
+			sparqlCalls.Add(1)
+			switch r.Header.Get("Authorization") {
+			case "Bearer expired-token":
+				w.WriteHeader(http.StatusUnauthorized)
+				_, _ = io.WriteString(w, "Jwt is expired")
+			case "Bearer fresh-token":
+				mu.Lock()
+				seenFresh++
+				mu.Unlock()
+				_, _ = io.WriteString(w, `{"head":{},"boolean":true}`)
+			default:
+				t.Errorf("unexpected authorization header %q", r.Header.Get("Authorization"))
+				w.WriteHeader(http.StatusInternalServerError)
+				return
+			}
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	defer server.Close()
+
+	cfg := Config{
+		SparqlSourceMode:       "external",
+		ExternalSparqlEndpoint: server.URL + "/sparql",
+		KeycloakTokenEndpoint:  server.URL + "/token",
+		KeycloakClientID:       "anzograph",
+		KeycloakUsername:       "user",
+		KeycloakPassword:       "pass",
+		KeycloakScope:          "openid",
+		SparqlTimeout:          2 * time.Second,
+	}
+
+	client := NewAnzoGraphClient(cfg)
+	client.client = server.Client()
+	client.tokenManager.client = server.Client()
+	client.tokenManager.token = "expired-token"
+
+	const workers = 5
+	var wg sync.WaitGroup
+	errs := make(chan error, workers)
+	for i := 0; i < workers; i++ {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			_, err := client.Query(context.Background(), "ASK WHERE { ?s ?p ?o }")
+			errs <- err
+		}()
+	}
+	wg.Wait()
+	close(errs)
+
+	for err := range errs {
+		if err != nil {
+			t.Fatalf("concurrent query returned error: %v", err)
+		}
+	}
+	if tokenCalls.Load() != 1 {
+		t.Fatalf("expected exactly 1 shared refresh, got %d", tokenCalls.Load())
+	}
+	if seenFresh != workers {
+		t.Fatalf("expected %d successful retried queries, got %d", workers, seenFresh)
+	}
+	if sparqlCalls.Load() < workers*2 {
+		t.Fatalf("expected each worker to hit sparql before and after refresh, got %d calls", sparqlCalls.Load())
+	}
+}

backend_go/queryscope/scope.go

@@ -0,0 +1,25 @@
+package queryscope
+
+import "strings"
+
+// NamedGraph wraps a read pattern so app-generated queries read from any named graph via GRAPH ?g.
+func NamedGraph(pattern string) string {
+	trimmed := strings.TrimSpace(pattern)
+	if trimmed == "" {
+		return "  GRAPH ?g {\n  }"
+	}
+
+	return indent("GRAPH ?g {\n"+indent(trimmed, "  ")+"\n}", "  ")
+}
+
+func AskAnyTripleQuery() string {
+	return "ASK WHERE {\n" + NamedGraph("?s ?p ?o .") + "\n}"
+}
+
+func indent(text string, prefix string) string {
+	lines := strings.Split(text, "\n")
+	for i, line := range lines {
+		lines[i] = prefix + line
+	}
+	return strings.Join(lines, "\n")
+}

backend_go/selection_queries/named_graph_test.go

@@ -0,0 +1,33 @@
+package selection_queries
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestSelectionQueriesUseNamedGraphs(t *testing.T) {
+	selected := []NodeRef{
+		{ID: 1, TermType: "uri", IRI: "http://example.com/A"},
+	}
+
+	tests := []struct {
+		name  string
+		query string
+	}{
+		{name: "neighbors", query: neighborsQuery(selected, false)},
+		{name: "superclasses", query: superclassesQuery(selected, false)},
+		{name: "subclasses", query: subclassesQuery(selected, false)},
+	}
+
+	for _, tt := range tests {
+		if !strings.Contains(tt.query, "SELECT DISTINCT ?s ?p ?o") {
+			t.Fatalf("%s query should de-duplicate triples across named graphs:\n%s", tt.name, tt.query)
+		}
+		if !strings.Contains(tt.query, "GRAPH ?g") {
+			t.Fatalf("%s query should read from named graphs:\n%s", tt.name, tt.query)
+		}
+		if strings.Contains(tt.query, "owl:Class") {
+			t.Fatalf("%s query should no longer depend on owl:Class:\n%s", tt.name, tt.query)
+		}
+	}
+}

backend_go/selection_queries/neighbors.go

@@ -4,6 +4,8 @@ import (
 	"context"
 	"fmt"
 	"strings"
+
+	"visualizador_instanciados/backend_go/queryscope"
 )
 
 func neighborsQuery(selectedNodes []NodeRef, includeBNodes bool) string {
@@ -26,6 +28,36 @@ func neighborsQuery(selectedNodes []NodeRef, includeBNodes bool) string {
 	}
 
 	values := strings.Join(valuesTerms, " ")
+	pattern := queryscope.NamedGraph(fmt.Sprintf(`
+{
+  VALUES ?sel { %s }
+  BIND(?sel AS ?s)
+  VALUES ?p { rdf:type }
+  ?s ?p ?o .
+}
+UNION
+{
+  VALUES ?sel { %s }
+  VALUES ?p { rdf:type }
+  ?s ?p ?sel .
+  BIND(?sel AS ?o)
+}
+UNION
+{
+  VALUES ?sel { %s }
+  BIND(?sel AS ?s)
+  VALUES ?p { rdfs:subClassOf }
+  ?s ?p ?o .
+}
+UNION
+{
+  VALUES ?sel { %s }
+  VALUES ?p { rdfs:subClassOf }
+  ?s ?p ?sel .
+  BIND(?sel AS ?o)
+}
+`, values, values, values, values))
+
 	return fmt.Sprintf(`
 PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
@@ -33,40 +65,12 @@ PREFIX owl: <http://www.w3.org/2002/07/owl#>
 
 SELECT DISTINCT ?s ?p ?o
 WHERE {
-  {
-    VALUES ?sel { %s }
-    BIND(?sel AS ?s)
-    VALUES ?p { rdf:type }
-    ?s ?p ?o .
-    ?o rdf:type owl:Class .
-  }
-  UNION
-  {
-    VALUES ?sel { %s }
-    VALUES ?p { rdf:type }
-    ?s ?p ?sel .
-    ?sel rdf:type owl:Class .
-    BIND(?sel AS ?o)
-  }
-  UNION
-  {
-    VALUES ?sel { %s }
-    BIND(?sel AS ?s)
-    VALUES ?p { rdfs:subClassOf }
-    ?s ?p ?o .
-  }
-  UNION
-  {
-    VALUES ?sel { %s }
-    VALUES ?p { rdfs:subClassOf }
-    ?s ?p ?sel .
-    BIND(?sel AS ?o)
-  }
+%s
   FILTER(!isLiteral(?o))
   FILTER(?s != ?o)
   %s
 }
-`, values, values, values, values, bnodeFilter)
+`, pattern, bnodeFilter)
 }
 
 func runNeighbors(ctx context.Context, q Querier, idx Index, selectedIDs []uint32, includeBNodes bool) (Result, error) {

backend_go/selection_queries/subclasses.go

@@ -4,6 +4,8 @@ import (
 	"context"
 	"fmt"
 	"strings"
+
+	"visualizador_instanciados/backend_go/queryscope"
 )
 
 func subclassesQuery(selectedNodes []NodeRef, includeBNodes bool) string {
@@ -26,20 +28,33 @@ func subclassesQuery(selectedNodes []NodeRef, includeBNodes bool) string {
 	}
 
 	values := strings.Join(valuesTerms, " ")
+	pattern := queryscope.NamedGraph(fmt.Sprintf(`
+{
+VALUES ?sel { %s }
+VALUES ?p { rdfs:subClassOf }
+?s ?p ?sel .
+BIND(?sel AS ?o)
+}
+UNION
+{
+VALUES ?sel { %s }
+VALUES ?p { <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> }
+?s ?p ?sel .
+BIND(?sel AS ?o)
+}
+`, values, values))
+
 	return fmt.Sprintf(`
 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
 
 SELECT DISTINCT ?s ?p ?o
 WHERE {
-  VALUES ?sel { %s }
-  VALUES ?p { rdfs:subClassOf }
-  ?s ?p ?sel .
-  BIND(?sel AS ?o)
+%s
   FILTER(!isLiteral(?o))
   FILTER(?s != ?o)
   %s
 }
-`, values, bnodeFilter)
+`, pattern, bnodeFilter)
 }
 
 func runSubclasses(ctx context.Context, q Querier, idx Index, selectedIDs []uint32, includeBNodes bool) (Result, error) {

backend_go/selection_queries/superclasses.go

@@ -4,6 +4,8 @@ import (
 	"context"
 	"fmt"
 	"strings"
+
+	"visualizador_instanciados/backend_go/queryscope"
 )
 
 func superclassesQuery(selectedNodes []NodeRef, includeBNodes bool) string {
@@ -26,20 +28,33 @@ func superclassesQuery(selectedNodes []NodeRef, includeBNodes bool) string {
 	}
 
 	values := strings.Join(valuesTerms, " ")
+	pattern := queryscope.NamedGraph(fmt.Sprintf(`
+{
+VALUES ?sel { %s }
+BIND(?sel AS ?s)
+VALUES ?p { rdfs:subClassOf }
+?s ?p ?o .
+}
+UNION
+{
+VALUES ?sel { %s }
+BIND(?sel AS ?s)
+VALUES ?p { <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> }
+?s ?p ?o .
+}
+`, values, values))
+
 	return fmt.Sprintf(`
 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
 
 SELECT DISTINCT ?s ?p ?o
 WHERE {
-  VALUES ?sel { %s }
-  BIND(?sel AS ?s)
-  VALUES ?p { rdfs:subClassOf }
-  ?s ?p ?o .
+%s
   FILTER(!isLiteral(?o))
   FILTER(?s != ?o)
   %s
 }
-`, values, bnodeFilter)
+`, pattern, bnodeFilter)
 }
 
 func runSuperclasses(ctx context.Context, q Querier, idx Index, selectedIDs []uint32, includeBNodes bool) (Result, error) {

backend_go/server.go

@@ -155,7 +155,9 @@ func (s *APIServer) handleGraph(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	writeJSON(w, http.StatusOK, snap)
+	if err := writeGraphArrow(w, snap); err != nil {
+		log.Printf("handleGraph: arrow encode error graph_query_id=%s node_limit=%d edge_limit=%d err=%v", graphQueryID, nodeLimit, edgeLimit, err)
+	}
 }
 
 func (s *APIServer) handleGraphQueries(w http.ResponseWriter, r *http.Request) {

backend_go/snapshot_service.go

@@ -2,6 +2,7 @@ package main
 
 import (
 	"context"
+	"log"
 	"sync"
 )
 
@@ -24,6 +25,8 @@ type GraphSnapshotService struct {
 	sparql *AnzoGraphClient
 	cfg    Config
 
+	fetchSnapshot func(context.Context, *AnzoGraphClient, Config, int, int, string) (GraphResponse, error)
+
 	mu       sync.Mutex
 	cache    map[snapshotKey]GraphResponse
 	inflight map[snapshotKey]*snapshotInflight
@@ -31,10 +34,11 @@ type GraphSnapshotService struct {
 
 func NewGraphSnapshotService(sparql *AnzoGraphClient, cfg Config) *GraphSnapshotService {
 	return &GraphSnapshotService{
-		sparql:    sparql,
-		cfg:      cfg,
-		cache:    make(map[snapshotKey]GraphResponse),
-		inflight: make(map[snapshotKey]*snapshotInflight),
+		sparql:        sparql,
+		cfg:          cfg,
+		fetchSnapshot: fetchGraphSnapshot,
+		cache:        make(map[snapshotKey]GraphResponse),
+		inflight:     make(map[snapshotKey]*snapshotInflight),
 	}
 }
 
@@ -69,7 +73,20 @@ func (s *GraphSnapshotService) Get(ctx context.Context, nodeLimit int, edgeLimit
 	s.inflight[key] = inf
 	s.mu.Unlock()
 
-	snap, err := fetchGraphSnapshot(ctx, s.sparql, s.cfg, nodeLimit, edgeLimit, graphQueryID)
+	log.Printf("[snapshot] build_start graph_query_id=%s node_limit=%d edge_limit=%d detached=true", graphQueryID, nodeLimit, edgeLimit)
+	go s.buildSnapshotInBackground(key, inf, nodeLimit, edgeLimit, graphQueryID)
+
+	select {
+	case <-ctx.Done():
+		log.Printf("[snapshot] requester_canceled graph_query_id=%s node_limit=%d edge_limit=%d err=%v build_continues=true", graphQueryID, nodeLimit, edgeLimit, ctx.Err())
+		return GraphResponse{}, ctx.Err()
+	case <-inf.ready:
+		return inf.snapshot, inf.err
+	}
+}
+
+func (s *GraphSnapshotService) buildSnapshotInBackground(key snapshotKey, inf *snapshotInflight, nodeLimit int, edgeLimit int, graphQueryID string) {
+	snap, err := s.fetchSnapshot(context.Background(), s.sparql, s.cfg, nodeLimit, edgeLimit, graphQueryID)
 
 	s.mu.Lock()
 	inf.snapshot = snap
@@ -81,5 +98,9 @@ func (s *GraphSnapshotService) Get(ctx context.Context, nodeLimit int, edgeLimit
 	close(inf.ready)
 	s.mu.Unlock()
 
-	return snap, err
+	if err != nil {
+		log.Printf("[snapshot] build_done graph_query_id=%s node_limit=%d edge_limit=%d detached=true cached=false err=%v", graphQueryID, nodeLimit, edgeLimit, err)
+		return
+	}
+	log.Printf("[snapshot] build_done graph_query_id=%s node_limit=%d edge_limit=%d detached=true cached=true", graphQueryID, nodeLimit, edgeLimit)
 }

backend_go/snapshot_service_test.go

@@ -0,0 +1,97 @@
+package main
+
+import (
+	"context"
+	"errors"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+func TestSnapshotBuildContinuesAfterRequesterCancellation(t *testing.T) {
+	svc := NewGraphSnapshotService(nil, Config{})
+
+	var fetchCalls atomic.Int32
+	started := make(chan struct{})
+	release := make(chan struct{})
+	expected := GraphResponse{
+		Nodes: []Node{{ID: 1}},
+		Edges: []Edge{{Source: 1, Target: 1, PredicateID: 0}},
+		Meta:  &GraphMeta{GraphQueryID: "default", Nodes: 1, Edges: 1},
+	}
+
+	svc.fetchSnapshot = func(ctx context.Context, _ *AnzoGraphClient, _ Config, nodeLimit int, edgeLimit int, graphQueryID string) (GraphResponse, error) {
+		fetchCalls.Add(1)
+		if nodeLimit != 10 || edgeLimit != 20 || graphQueryID != "default" {
+			t.Fatalf("unexpected fetch args nodeLimit=%d edgeLimit=%d graphQueryID=%s", nodeLimit, edgeLimit, graphQueryID)
+		}
+		close(started)
+		<-release
+		return expected, nil
+	}
+
+	ctx1, cancel1 := context.WithCancel(context.Background())
+	defer cancel1()
+
+	firstErrCh := make(chan error, 1)
+	go func() {
+		_, err := svc.Get(ctx1, 10, 20, "default")
+		firstErrCh <- err
+	}()
+
+	<-started
+	cancel1()
+
+	select {
+	case err := <-firstErrCh:
+		if !errors.Is(err, context.Canceled) {
+			t.Fatalf("first Get error = %v, want context.Canceled", err)
+		}
+	case <-time.After(2 * time.Second):
+		t.Fatal("timed out waiting for first Get to return after cancellation")
+	}
+
+	secondSnapCh := make(chan GraphResponse, 1)
+	secondErrCh := make(chan error, 1)
+	go func() {
+		snap, err := svc.Get(context.Background(), 10, 20, "default")
+		if err != nil {
+			secondErrCh <- err
+			return
+		}
+		secondSnapCh <- snap
+	}()
+
+	time.Sleep(50 * time.Millisecond)
+	if got := fetchCalls.Load(); got != 1 {
+		t.Fatalf("fetchCalls after second waiter start = %d, want 1", got)
+	}
+
+	close(release)
+
+	select {
+	case err := <-secondErrCh:
+		t.Fatalf("second Get error = %v", err)
+	case snap := <-secondSnapCh:
+		if snap.Meta == nil || snap.Meta.Nodes != expected.Meta.Nodes || snap.Meta.Edges != expected.Meta.Edges {
+			t.Fatalf("second Get snapshot meta = %#v, want %#v", snap.Meta, expected.Meta)
+		}
+	case <-time.After(2 * time.Second):
+		t.Fatal("timed out waiting for second Get to return")
+	}
+
+	if got := fetchCalls.Load(); got != 1 {
+		t.Fatalf("fetchCalls after background completion = %d, want 1", got)
+	}
+
+	snap, err := svc.Get(context.Background(), 10, 20, "default")
+	if err != nil {
+		t.Fatalf("cached Get error = %v", err)
+	}
+	if snap.Meta == nil || snap.Meta.Nodes != expected.Meta.Nodes || snap.Meta.Edges != expected.Meta.Edges {
+		t.Fatalf("cached Get snapshot meta = %#v, want %#v", snap.Meta, expected.Meta)
+	}
+	if got := fetchCalls.Load(); got != 1 {
+		t.Fatalf("fetchCalls after cached Get = %d, want 1", got)
+	}
+}

backend_go/sparql.go

@@ -5,50 +5,68 @@ import (
 	"encoding/base64"
 	"fmt"
 	"io"
+	"log"
 	"net/http"
-	"net/url"
 	"strings"
 	"time"
+
+	"visualizador_instanciados/backend_go/queryscope"
 )
 
 type AnzoGraphClient struct {
-	cfg        Config
-	endpoint   string
-	authHeader string
-	client     *http.Client
+	cfg             Config
+	endpoint        string
+	basicAuthHeader string
+	client          *http.Client
+	tokenManager    *keycloakTokenManager
 }
 
 func NewAnzoGraphClient(cfg Config) *AnzoGraphClient {
 	endpoint := cfg.EffectiveSparqlEndpoint()
-	authHeader := ""
-	user := strings.TrimSpace(cfg.SparqlUser)
-	pass := strings.TrimSpace(cfg.SparqlPass)
-	if user != "" && pass != "" {
-		token := base64.StdEncoding.EncodeToString([]byte(user + ":" + pass))
-		authHeader = "Basic " + token
+	client := &http.Client{}
+	basicAuthHeader := ""
+	if !cfg.UsesExternalSparql() {
+		user := strings.TrimSpace(cfg.SparqlUser)
+		pass := strings.TrimSpace(cfg.SparqlPass)
+		if user != "" && pass != "" {
+			token := base64.StdEncoding.EncodeToString([]byte(user + ":" + pass))
+			basicAuthHeader = "Basic " + token
+		}
 	}
 
-	return &AnzoGraphClient{
-		cfg:        cfg,
-		endpoint:   endpoint,
-		authHeader: authHeader,
-		client:     &http.Client{},
+	agc := &AnzoGraphClient{
+		cfg:             cfg,
+		endpoint:        endpoint,
+		basicAuthHeader: basicAuthHeader,
+		client:          client,
 	}
+	if cfg.UsesExternalSparql() {
+		agc.tokenManager = newKeycloakTokenManager(cfg, client)
+	}
+	return agc
 }
 
 func (c *AnzoGraphClient) Startup(ctx context.Context) error {
+	log.Printf(
+		"[sparql] startup source_mode=%s endpoint=%s auth_mode=%s load_on_start=%t",
+		c.cfg.SparqlSourceMode,
+		c.endpoint,
+		c.authMode(),
+		c.cfg.SparqlLoadOnStart,
+	)
+
+	if c.cfg.UsesExternalSparql() {
+		tokenCtx, cancel := context.WithTimeout(ctx, c.cfg.SparqlReadyTimeout)
+		defer cancel()
+		if _, err := c.refreshExternalToken(tokenCtx, "startup"); err != nil {
+			return fmt.Errorf("keycloak startup token fetch failed: %w", err)
+		}
+	}
+
 	if err := c.waitReady(ctx); err != nil {
 		return err
 	}
-
-	if c.cfg.SparqlClearOnStart {
-		if err := c.update(ctx, "CLEAR ALL"); err != nil {
-			return err
-		}
-		if err := c.waitReady(ctx); err != nil {
-			return err
-		}
-	}
+	c.logNamedGraphDatasetProbe(ctx, "startup_initial")
 
 	if c.cfg.SparqlLoadOnStart {
 		df := strings.TrimSpace(c.cfg.SparqlDataFile)
@@ -68,6 +86,7 @@ func (c *AnzoGraphClient) Startup(ctx context.Context) error {
 		if err := c.waitReady(ctx); err != nil {
 			return err
 		}
+		c.logNamedGraphDatasetProbe(ctx, "startup_post_load")
 	}
 
 	return nil
@@ -83,23 +102,7 @@ func (c *AnzoGraphClient) Query(ctx context.Context, query string) ([]byte, erro
 }
 
 func (c *AnzoGraphClient) queryWithTimeout(ctx context.Context, query string, timeout time.Duration) ([]byte, error) {
-	ctx2, cancel := context.WithTimeout(ctx, timeout)
-	defer cancel()
-
-	form := url.Values{}
-	form.Set("query", query)
-
-	req, err := http.NewRequestWithContext(ctx2, http.MethodPost, c.endpoint, strings.NewReader(form.Encode()))
-	if err != nil {
-		return nil, err
-	}
-	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
-	req.Header.Set("Accept", "application/sparql-results+json")
-	if c.authHeader != "" {
-		req.Header.Set("Authorization", c.authHeader)
-	}
-
-	resp, err := c.client.Do(req)
+	resp, _, err := c.queryRequestWithTimeout(ctx, query, timeout)
 	if err != nil {
 		return nil, err
 	}
@@ -109,9 +112,6 @@ func (c *AnzoGraphClient) queryWithTimeout(ctx context.Context, query string, ti
 	if err != nil {
 		return nil, err
 	}
-	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
-		return nil, fmt.Errorf("sparql query failed: %s: %s", resp.Status, strings.TrimSpace(string(body)))
-	}
 	return body, nil
 }
 
@@ -125,8 +125,12 @@ func (c *AnzoGraphClient) update(ctx context.Context, update string) error {
 	}
 	req.Header.Set("Content-Type", "application/sparql-update")
 	req.Header.Set("Accept", "application/json")
-	if c.authHeader != "" {
-		req.Header.Set("Authorization", c.authHeader)
+	authHeader, err := c.authorizationHeader(ctx2, "sparql_update")
+	if err != nil {
+		return err
+	}
+	if authHeader != "" {
+		req.Header.Set("Authorization", authHeader)
 	}
 
 	resp, err := c.client.Do(req)
@@ -144,6 +148,13 @@ func (c *AnzoGraphClient) update(ctx context.Context, update string) error {
 
 func (c *AnzoGraphClient) waitReady(ctx context.Context) error {
 	var lastErr error
+	log.Printf(
+		"[sparql] readiness_wait_start endpoint=%s retries=%d timeout=%s delay=%s query_scope=named_graphs",
+		c.endpoint,
+		c.cfg.SparqlReadyRetries,
+		c.cfg.SparqlReadyTimeout,
+		c.cfg.SparqlReadyDelay,
+	)
 	for i := 0; i < c.cfg.SparqlReadyRetries; i++ {
 		select {
 		case <-ctx.Done():
@@ -154,16 +165,73 @@ func (c *AnzoGraphClient) waitReady(ctx context.Context) error {
 		default:
 		}
 
-		body, err := c.queryWithTimeout(ctx, "ASK WHERE { ?s ?p ?o }", c.cfg.SparqlReadyTimeout)
+		var ask sparqlBooleanResponse
+		_, err := c.queryJSONWithTimeout(ctx, namedGraphAnyTripleAskQuery(), c.cfg.SparqlReadyTimeout, &ask)
 		if err == nil {
-			// Ensure it's JSON, not HTML/text during boot.
-			if strings.HasPrefix(strings.TrimSpace(string(body)), "{") {
-				return nil
-			}
-			err = fmt.Errorf("unexpected readiness response: %s", strings.TrimSpace(string(body)))
+			log.Printf("[sparql] readiness_wait_ok endpoint=%s attempt=%d/%d", c.endpoint, i+1, c.cfg.SparqlReadyRetries)
+			return nil
 		}
 		lastErr = err
+		log.Printf("[sparql] readiness_wait_retry endpoint=%s attempt=%d/%d err=%v", c.endpoint, i+1, c.cfg.SparqlReadyRetries, err)
 		time.Sleep(c.cfg.SparqlReadyDelay)
 	}
 	return fmt.Errorf("anzograph not ready at %s: %w", c.endpoint, lastErr)
 }
+
+func namedGraphAnyTripleAskQuery() string {
+	return queryscope.AskAnyTripleQuery()
+}
+
+func (c *AnzoGraphClient) authMode() string {
+	switch {
+	case c.cfg.UsesExternalSparql():
+		return "bearer"
+	case strings.HasPrefix(c.basicAuthHeader, "Basic "):
+		return "basic"
+	default:
+		return "none"
+	}
+}
+
+func (c *AnzoGraphClient) authorizationHeader(ctx context.Context, reason string) (string, error) {
+	if !c.cfg.UsesExternalSparql() {
+		return c.basicAuthHeader, nil
+	}
+	if c.tokenManager == nil {
+		return "", fmt.Errorf("external sparql mode is enabled but token manager is not configured")
+	}
+	token, err := c.tokenManager.EnsureToken(ctx, reason)
+	if err != nil {
+		return "", err
+	}
+	return "Bearer " + token, nil
+}
+
+func (c *AnzoGraphClient) refreshExternalToken(ctx context.Context, reason string) (string, error) {
+	if !c.cfg.UsesExternalSparql() {
+		return "", nil
+	}
+	if c.tokenManager == nil {
+		return "", fmt.Errorf("external sparql mode is enabled but token manager is not configured")
+	}
+	return c.tokenManager.Refresh(ctx, reason)
+}
+
+func (c *AnzoGraphClient) logNamedGraphDatasetProbe(ctx context.Context, stage string) {
+	var ask sparqlBooleanResponse
+	metrics, err := c.queryJSONWithTimeout(ctx, namedGraphAnyTripleAskQuery(), c.cfg.SparqlReadyTimeout, &ask)
+	if err != nil {
+		log.Printf("[sparql] dataset_probe_failed stage=%s endpoint=%s err=%v", stage, c.endpoint, err)
+		return
+	}
+
+	log.Printf(
+		"[sparql] dataset_probe stage=%s endpoint=%s named_graph_has_triples=%t bytes=%d round_trip_time=%s decode_time=%s",
+		stage,
+		c.endpoint,
+		ask.Boolean,
+		metrics.ResponseBytes,
+		metrics.RoundTripTime.Truncate(time.Millisecond),
+		metrics.BodyDecodeTime.Truncate(time.Millisecond),
+	)
+}

backend_go/sparql_decode.go

@@ -0,0 +1,341 @@
+package main
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"log"
+	"net/http"
+	"net/url"
+	"strings"
+	"time"
+)
+
+type sparqlQueryMetrics struct {
+	ResponseBytes  int64
+	RoundTripTime  time.Duration
+	BodyDecodeTime time.Duration
+	BindingCount   int
+}
+
+type countingReadCloser struct {
+	io.ReadCloser
+	bytesRead int64
+}
+
+func (c *countingReadCloser) Read(p []byte) (int, error) {
+	n, err := c.ReadCloser.Read(p)
+	c.bytesRead += int64(n)
+	return n, err
+}
+
+type cancelOnCloseReadCloser struct {
+	io.ReadCloser
+	cancel context.CancelFunc
+}
+
+func (c *cancelOnCloseReadCloser) Close() error {
+	err := c.ReadCloser.Close()
+	c.cancel()
+	return err
+}
+
+type sparqlHTTPStatusError struct {
+	StatusCode int
+	Status     string
+	Body       string
+}
+
+func (e *sparqlHTTPStatusError) Error() string {
+	return fmt.Sprintf("sparql query failed: %s: %s", e.Status, e.Body)
+}
+
+func (c *AnzoGraphClient) queryRequestWithTimeout(ctx context.Context, query string, timeout time.Duration) (*http.Response, time.Duration, error) {
+	ctx2, cancel := context.WithTimeout(ctx, timeout)
+	resp, roundTripTime, err := c.queryRequest(ctx2, query, true)
+	if err != nil {
+		cancel()
+		return nil, roundTripTime, err
+	}
+	resp.Body = &cancelOnCloseReadCloser{ReadCloser: resp.Body, cancel: cancel}
+	return resp, roundTripTime, nil
+}
+
+func (c *AnzoGraphClient) queryRequest(ctx context.Context, query string, allowRefresh bool) (*http.Response, time.Duration, error) {
+	resp, roundTripTime, err := c.queryRequestAttempt(ctx, query)
+	if err == nil {
+		return resp, roundTripTime, nil
+	}
+
+	var statusErr *sparqlHTTPStatusError
+	if allowRefresh && errors.As(err, &statusErr) && c.shouldRefreshExpiredJWT(statusErr) {
+		log.Printf("[auth] sparql_token_retry endpoint=%s reason=jwt_expired", c.endpoint)
+		if _, refreshErr := c.refreshExternalToken(ctx, "sparql_jwt_expired"); refreshErr != nil {
+			return nil, roundTripTime, fmt.Errorf("%w (token refresh failed: %v)", statusErr, refreshErr)
+		}
+
+		retryResp, retryRoundTripTime, retryErr := c.queryRequest(ctx, query, false)
+		return retryResp, roundTripTime + retryRoundTripTime, retryErr
+	}
+
+	return nil, roundTripTime, err
+}
+
+func (c *AnzoGraphClient) queryRequestAttempt(ctx context.Context, query string) (*http.Response, time.Duration, error) {
+	form := url.Values{}
+	form.Set("query", query)
+
+	req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.endpoint, strings.NewReader(form.Encode()))
+	if err != nil {
+		return nil, 0, err
+	}
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+	req.Header.Set("Accept", "application/sparql-results+json")
+	authHeader, err := c.authorizationHeader(ctx, "sparql_query")
+	if err != nil {
+		return nil, 0, err
+	}
+	if authHeader != "" {
+		req.Header.Set("Authorization", authHeader)
+	}
+
+	start := time.Now()
+	resp, err := c.client.Do(req)
+	if err != nil {
+		return nil, 0, err
+	}
+	roundTripTime := time.Since(start)
+
+	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+		defer resp.Body.Close()
+		body, readErr := io.ReadAll(resp.Body)
+		if readErr != nil {
+			return nil, roundTripTime, readErr
+		}
+		return nil, roundTripTime, &sparqlHTTPStatusError{
+			StatusCode: resp.StatusCode,
+			Status:     resp.Status,
+			Body:       strings.TrimSpace(string(body)),
+		}
+	}
+
+	return resp, roundTripTime, nil
+}
+
+func (c *AnzoGraphClient) shouldRefreshExpiredJWT(err *sparqlHTTPStatusError) bool {
+	if err == nil || !c.cfg.UsesExternalSparql() {
+		return false
+	}
+	return err.StatusCode == http.StatusUnauthorized && strings.Contains(err.Body, "Jwt is expired")
+}
+
+func (c *AnzoGraphClient) QueryJSON(ctx context.Context, query string, target any) (sparqlQueryMetrics, error) {
+	return c.queryJSONWithTimeout(ctx, query, c.cfg.SparqlTimeout, target)
+}
+
+func (c *AnzoGraphClient) queryJSONWithTimeout(ctx context.Context, query string, timeout time.Duration, target any) (sparqlQueryMetrics, error) {
+	resp, roundTripTime, err := c.queryRequestWithTimeout(ctx, query, timeout)
+	if err != nil {
+		return sparqlQueryMetrics{}, err
+	}
+	counter := &countingReadCloser{ReadCloser: resp.Body}
+	defer counter.Close()
+
+	decodeStart := time.Now()
+	if err := json.NewDecoder(counter).Decode(target); err != nil {
+		return sparqlQueryMetrics{
+			ResponseBytes:  counter.bytesRead,
+			RoundTripTime:  roundTripTime,
+			BodyDecodeTime: time.Since(decodeStart),
+		}, wrapSparqlJSONDecodeError(err)
+	}
+
+	return sparqlQueryMetrics{
+		ResponseBytes:  counter.bytesRead,
+		RoundTripTime:  roundTripTime,
+		BodyDecodeTime: time.Since(decodeStart),
+	}, nil
+}
+
+func (c *AnzoGraphClient) QueryTripleBindingsStream(
+	ctx context.Context,
+	query string,
+	visit func(binding sparqlTripleBinding) error,
+) (sparqlQueryMetrics, error) {
+	return c.queryTripleBindingsStreamWithTimeout(ctx, query, c.cfg.SparqlTimeout, visit)
+}
+
+func (c *AnzoGraphClient) queryTripleBindingsStreamWithTimeout(
+	ctx context.Context,
+	query string,
+	timeout time.Duration,
+	visit func(binding sparqlTripleBinding) error,
+) (sparqlQueryMetrics, error) {
+	resp, roundTripTime, err := c.queryRequestWithTimeout(ctx, query, timeout)
+	if err != nil {
+		return sparqlQueryMetrics{}, err
+	}
+	counter := &countingReadCloser{ReadCloser: resp.Body}
+	defer counter.Close()
+
+	decodeStart := time.Now()
+	bindingCount, err := decodeBindingsStream(json.NewDecoder(counter), visit)
+	if err != nil {
+		return sparqlQueryMetrics{
+			ResponseBytes:  counter.bytesRead,
+			RoundTripTime:  roundTripTime,
+			BodyDecodeTime: time.Since(decodeStart),
+			BindingCount:   bindingCount,
+		}, wrapSparqlJSONDecodeError(err)
+	}
+
+	return sparqlQueryMetrics{
+		ResponseBytes:  counter.bytesRead,
+		RoundTripTime:  roundTripTime,
+		BodyDecodeTime: time.Since(decodeStart),
+		BindingCount:   bindingCount,
+	}, nil
+}
+
+func decodeBindingsStream(dec *json.Decoder, visit func(binding sparqlTripleBinding) error) (int, error) {
+	tok, err := dec.Token()
+	if err != nil {
+		return 0, err
+	}
+	if delim, ok := tok.(json.Delim); !ok || delim != '{' {
+		return 0, fmt.Errorf("invalid SPARQL JSON: expected top-level object")
+	}
+
+	foundResults := false
+	bindingCount := 0
+	for dec.More() {
+		keyToken, err := dec.Token()
+		if err != nil {
+			return bindingCount, err
+		}
+
+		key, ok := keyToken.(string)
+		if !ok {
+			return bindingCount, fmt.Errorf("invalid SPARQL JSON: expected top-level field name")
+		}
+
+		switch key {
+		case "results":
+			foundResults = true
+			n, err := decodeTripleBindingsObject(dec, visit)
+			bindingCount += n
+			if err != nil {
+				return bindingCount, err
+			}
+		default:
+			if err := discardJSONValue(dec); err != nil {
+				return bindingCount, err
+			}
+		}
+	}
+
+	tok, err = dec.Token()
+	if err != nil {
+		return bindingCount, err
+	}
+	if delim, ok := tok.(json.Delim); !ok || delim != '}' {
+		return bindingCount, fmt.Errorf("invalid SPARQL JSON: expected top-level object terminator")
+	}
+
+	if !foundResults {
+		return 0, fmt.Errorf("invalid SPARQL JSON: missing results field")
+	}
+
+	return bindingCount, nil
+}
+
+func decodeTripleBindingsObject(dec *json.Decoder, visit func(binding sparqlTripleBinding) error) (int, error) {
+	tok, err := dec.Token()
+	if err != nil {
+		return 0, err
+	}
+	if delim, ok := tok.(json.Delim); !ok || delim != '{' {
+		return 0, fmt.Errorf("invalid SPARQL JSON: expected results object")
+	}
+
+	bindingCount := 0
+	for dec.More() {
+		keyToken, err := dec.Token()
+		if err != nil {
+			return bindingCount, err
+		}
+
+		key, ok := keyToken.(string)
+		if !ok {
+			return bindingCount, fmt.Errorf("invalid SPARQL JSON: expected results field name")
+		}
+
+		if key != "bindings" {
+			if err := discardJSONValue(dec); err != nil {
+				return bindingCount, err
+			}
+			continue
+		}
+
+		tok, err := dec.Token()
+		if err != nil {
+			return bindingCount, err
+		}
+		if delim, ok := tok.(json.Delim); !ok || delim != '[' {
+			return bindingCount, fmt.Errorf("invalid SPARQL JSON: expected bindings array")
+		}
+
+		for dec.More() {
+			var binding sparqlTripleBinding
+			if err := dec.Decode(&binding); err != nil {
+				return bindingCount, err
+			}
+			bindingCount++
+			if err := visit(binding); err != nil {
+				return bindingCount, err
+			}
+		}
+
+		tok, err = dec.Token()
+		if err != nil {
+			return bindingCount, err
+		}
+		if delim, ok := tok.(json.Delim); !ok || delim != ']' {
+			return bindingCount, fmt.Errorf("invalid SPARQL JSON: expected bindings array terminator")
+		}
+	}
+
+	tok, err = dec.Token()
+	if err != nil {
+		return bindingCount, err
+	}
+	if delim, ok := tok.(json.Delim); !ok || delim != '}' {
+		return bindingCount, fmt.Errorf("invalid SPARQL JSON: expected results object terminator")
+	}
+
+	return bindingCount, nil
+}
+
+func discardJSONValue(dec *json.Decoder) error {
+	var discard json.RawMessage
+	return dec.Decode(&discard)
+}
+
+func wrapSparqlJSONDecodeError(err error) error {
+	if err == nil {
+		return nil
+	}
+	if isTruncatedJSONError(err) {
+		return fmt.Errorf("truncated SPARQL JSON: %w", err)
+	}
+	return fmt.Errorf("failed to decode SPARQL JSON: %w", err)
+}
+
+func isTruncatedJSONError(err error) bool {
+	return errors.Is(err, io.ErrUnexpectedEOF) ||
+		errors.Is(err, io.EOF) ||
+		strings.Contains(err.Error(), "unexpected end of JSON input") ||
+		strings.Contains(err.Error(), "unexpected EOF")
+}

backend_go/sparql_decode_test.go

@@ -0,0 +1,144 @@
+package main
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestDecodeBindingsStreamStreamsTripleBindings(t *testing.T) {
+	payload := `{
+  "head": {"vars": ["s", "p", "o"]},
+  "results": {
+    "bindings": [
+      {
+        "s": {"type": "uri", "value": "http://example.com/s1"},
+        "p": {"type": "uri", "value": "http://example.com/p"},
+        "o": {"type": "uri", "value": "http://example.com/o1"}
+      },
+      {
+        "s": {"type": "uri", "value": "http://example.com/s2"},
+        "p": {"type": "uri", "value": "http://example.com/p"},
+        "o": {"type": "uri", "value": "http://example.com/o2"}
+      }
+    ]
+  }
+}`
+
+	var got []sparqlTripleBinding
+	count, err := decodeBindingsStream(json.NewDecoder(strings.NewReader(payload)), func(binding sparqlTripleBinding) error {
+		got = append(got, binding)
+		return nil
+	})
+	if err != nil {
+		t.Fatalf("decodeBindingsStream returned error: %v", err)
+	}
+	if count != 2 {
+		t.Fatalf("expected 2 bindings, got %d", count)
+	}
+	if len(got) != 2 {
+		t.Fatalf("expected 2 streamed bindings, got %d", len(got))
+	}
+	if got[0].S.Value != "http://example.com/s1" || got[1].O.Value != "http://example.com/o2" {
+		t.Fatalf("unexpected streamed bindings: %+v", got)
+	}
+}
+
+func TestQueryJSONDecodesTypedBindings(t *testing.T) {
+	t.Run("predicates", func(t *testing.T) {
+		server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			w.Header().Set("Content-Type", "application/sparql-results+json")
+			_, _ = w.Write([]byte(`{
+  "head": {"vars": ["p"]},
+  "results": {
+    "bindings": [
+      {"p": {"type": "uri", "value": "http://example.com/p"}}
+    ]
+  }
+}`))
+		}))
+		defer server.Close()
+
+		client := &AnzoGraphClient{
+			cfg:      Config{SparqlTimeout: 5 * time.Second},
+			endpoint: server.URL,
+			client:   server.Client(),
+		}
+
+		var res sparqlBindingsResponse[sparqlPredicateBinding]
+		metrics, err := client.QueryJSON(context.Background(), "SELECT ?p WHERE { ?s ?p ?o }", &res)
+		if err != nil {
+			t.Fatalf("QueryJSON returned error: %v", err)
+		}
+		if len(res.Results.Bindings) != 1 || res.Results.Bindings[0].P.Value != "http://example.com/p" {
+			t.Fatalf("unexpected predicate bindings: %+v", res.Results.Bindings)
+		}
+		if metrics.ResponseBytes == 0 {
+			t.Fatalf("expected QueryJSON to record response bytes")
+		}
+	}))
+
+	t.Run("labels", func(t *testing.T) {
+		server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			w.Header().Set("Content-Type", "application/sparql-results+json")
+			_, _ = w.Write([]byte(`{
+  "head": {"vars": ["s", "label"]},
+  "results": {
+    "bindings": [
+      {
+        "s": {"type": "uri", "value": "http://example.com/s"},
+        "label": {"type": "literal", "xml:lang": "en", "value": "Example"}
+      }
+    ]
+  }
+}`))
+		}))
+		defer server.Close()
+
+		client := &AnzoGraphClient{
+			cfg:      Config{SparqlTimeout: 5 * time.Second},
+			endpoint: server.URL,
+			client:   server.Client(),
+		}
+
+		var res sparqlBindingsResponse[sparqlLabelBinding]
+		_, err := client.QueryJSON(context.Background(), "SELECT ?s ?label WHERE { ?s ?p ?label }", &res)
+		if err != nil {
+			t.Fatalf("QueryJSON returned error: %v", err)
+		}
+		if len(res.Results.Bindings) != 1 {
+			t.Fatalf("expected 1 label binding, got %d", len(res.Results.Bindings))
+		}
+		if res.Results.Bindings[0].Label.Value != "Example" || res.Results.Bindings[0].Label.Lang != "en" {
+			t.Fatalf("unexpected label binding: %+v", res.Results.Bindings[0])
+		}
+	}))
+}
+
+func TestQueryTripleBindingsStreamReportsTruncatedJSON(t *testing.T) {
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/sparql-results+json")
+		_, _ = w.Write([]byte(`{"results":{"bindings":[{"s":{"type":"uri","value":"http://example.com/s"}`))
+	}))
+	defer server.Close()
+
+	client := &AnzoGraphClient{
+		cfg:      Config{SparqlTimeout: 5 * time.Second},
+		endpoint: server.URL,
+		client:   server.Client(),
+	}
+
+	_, err := client.QueryTripleBindingsStream(context.Background(), "SELECT ?s ?p ?o WHERE { ?s ?p ?o }", func(binding sparqlTripleBinding) error {
+		return nil
+	})
+	if err == nil {
+		t.Fatalf("expected truncated JSON error")
+	}
+	if !strings.Contains(err.Error(), "truncated SPARQL JSON") {
+		t.Fatalf("expected truncated JSON error, got %v", err)
+	}
+}

backend_go/sparql_named_graph_test.go

@@ -0,0 +1,20 @@
+package main
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestNamedGraphAnyTripleAskQueryUsesGraphVariable(t *testing.T) {
+	query := namedGraphAnyTripleAskQuery()
+
+	if !strings.Contains(query, "ASK WHERE") {
+		t.Fatalf("readiness query should be an ASK query:\n%s", query)
+	}
+	if !strings.Contains(query, "GRAPH ?g") {
+		t.Fatalf("readiness query should probe named graphs:\n%s", query)
+	}
+	if strings.Contains(query, "ASK WHERE { ?s ?p ?o }") {
+		t.Fatalf("readiness query should no longer probe only the default graph:\n%s", query)
+	}
+}

backend_go/sparql_types.go

@@ -6,8 +6,27 @@ type sparqlTerm struct {
 	Lang  string `json:"xml:lang,omitempty"`
 }
 
-type sparqlResponse struct {
+type sparqlBooleanResponse struct {
+	Boolean bool `json:"boolean"`
+}
+
+type sparqlTripleBinding struct {
+	S sparqlTerm `json:"s"`
+	P sparqlTerm `json:"p"`
+	O sparqlTerm `json:"o"`
+}
+
+type sparqlPredicateBinding struct {
+	P sparqlTerm `json:"p"`
+}
+
+type sparqlLabelBinding struct {
+	S     sparqlTerm `json:"s"`
+	Label sparqlTerm `json:"label"`
+}
+
+type sparqlBindingsResponse[T any] struct {
 	Results struct {
-		Bindings []map[string]sparqlTerm `json:"bindings"`
+		Bindings []T `json:"bindings"`
 	} `json:"results"`
 }

docker-compose.yml

@@ -36,14 +36,21 @@ services:
       - MAX_EDGE_LIMIT=${MAX_EDGE_LIMIT:-20000000}
       - INCLUDE_BNODES=${INCLUDE_BNODES:-false}
       - CORS_ORIGINS=${CORS_ORIGINS:-http://localhost:5173}
+      - SPARQL_SOURCE_MODE=${SPARQL_SOURCE_MODE:-local}
       - SPARQL_HOST=${SPARQL_HOST:-http://anzograph:8080}
       - SPARQL_ENDPOINT
+      - EXTERNAL_SPARQL_ENDPOINT
+      - KEYCLOAK_TOKEN_ENDPOINT
+      - KEYCLOAK_CLIENT_ID
+      - KEYCLOAK_USERNAME
+      - KEYCLOAK_PASSWORD
+      - KEYCLOAK_SCOPE=${KEYCLOAK_SCOPE:-openid}
+      - ACCESS_TOKEN
       - SPARQL_USER=${SPARQL_USER:-admin}
       - SPARQL_PASS=${SPARQL_PASS:-Passw0rd1}
       - SPARQL_DATA_FILE=${SPARQL_DATA_FILE:-file:///opt/shared-files/o3po.ttl}
       - SPARQL_GRAPH_IRI
       - SPARQL_LOAD_ON_START=${SPARQL_LOAD_ON_START:-false}
-      - SPARQL_CLEAR_ON_START=${SPARQL_CLEAR_ON_START:-false}
       - SPARQL_TIMEOUT_S=${SPARQL_TIMEOUT_S:-300}
       - SPARQL_READY_RETRIES=${SPARQL_READY_RETRIES:-30}
       - SPARQL_READY_DELAY_S=${SPARQL_READY_DELAY_S:-4}

frontend/package-lock.json

@@ -10,6 +10,7 @@
       "dependencies": {
         "@cosmos.gl/graph": "^2.6.4",
         "@webgpu/types": "^0.1.69",
+        "apache-arrow": "^21.1.0",
         "clsx": "2.1.1",
         "react": "19.2.3",
         "react-dom": "19.2.3",
@@ -1184,6 +1185,15 @@
         "win32"
       ]
     },
+    "node_modules/@swc/helpers": {
+      "version": "0.5.20",
+      "resolved": "https://registry.npmjs.org/@swc/helpers/-/helpers-0.5.20.tgz",
+      "integrity": "sha512-2egEBHUMasdypIzrprsu8g+OEVd7Vp2MM3a2eVlM/cyFYto0nGz5BX5BTgh/ShZZI9ed+ozEq+Ngt+rgmUs8tw==",
+      "license": "Apache-2.0",
+      "dependencies": {
+        "tslib": "^2.8.0"
+      }
+    },
     "node_modules/@tailwindcss/node": {
       "version": "4.1.17",
       "resolved": "https://registry.npmjs.org/@tailwindcss/node/-/node-4.1.17.tgz",
@@ -1501,6 +1511,18 @@
         "@babel/types": "^7.28.2"
       }
     },
+    "node_modules/@types/command-line-args": {
+      "version": "5.2.3",
+      "resolved": "https://registry.npmjs.org/@types/command-line-args/-/command-line-args-5.2.3.tgz",
+      "integrity": "sha512-uv0aG6R0Y8WHZLTamZwtfsDLVRnOa+n+n5rEvFWL5Na5gZ8V2Teab/duDPFzIIIhs9qizDpcavCusCLJZu62Kw==",
+      "license": "MIT"
+    },
+    "node_modules/@types/command-line-usage": {
+      "version": "5.0.4",
+      "resolved": "https://registry.npmjs.org/@types/command-line-usage/-/command-line-usage-5.0.4.tgz",
+      "integrity": "sha512-BwR5KP3Es/CSht0xqBcUXS3qCAUVXwpRKsV2+arxeb65atasuXG9LykC9Ab10Cw3s2raH92ZqOeILaQbsB2ACg==",
+      "license": "MIT"
+    },
     "node_modules/@types/estree": {
       "version": "1.0.8",
       "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz",
@@ -1572,6 +1594,65 @@
       "integrity": "sha512-RPmm6kgRbI8e98zSD3RVACvnuktIja5+yLgDAkTmxLr90BEwdTXRQWNLF3ETTTyH/8mKhznZuN5AveXYFEsMGQ==",
       "license": "BSD-3-Clause"
     },
+    "node_modules/ansi-styles": {
+      "version": "4.3.0",
+      "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
+      "integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
+      "license": "MIT",
+      "dependencies": {
+        "color-convert": "^2.0.1"
+      },
+      "engines": {
+        "node": ">=8"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/ansi-styles?sponsor=1"
+      }
+    },
+    "node_modules/apache-arrow": {
+      "version": "21.1.0",
+      "resolved": "https://registry.npmjs.org/apache-arrow/-/apache-arrow-21.1.0.tgz",
+      "integrity": "sha512-kQrYLxhC+NTVVZ4CCzGF6L/uPVOzJmD1T3XgbiUnP7oTeVFOFgEUu6IKNwCDkpFoBVqDKQivlX4RUFqqnWFlEA==",
+      "license": "Apache-2.0",
+      "dependencies": {
+        "@swc/helpers": "^0.5.11",
+        "@types/command-line-args": "^5.2.3",
+        "@types/command-line-usage": "^5.0.4",
+        "@types/node": "^24.0.3",
+        "command-line-args": "^6.0.1",
+        "command-line-usage": "^7.0.1",
+        "flatbuffers": "^25.1.24",
+        "json-bignum": "^0.0.3",
+        "tslib": "^2.6.2"
+      },
+      "bin": {
+        "arrow2csv": "bin/arrow2csv.js"
+      }
+    },
+    "node_modules/apache-arrow/node_modules/@types/node": {
+      "version": "24.12.0",
+      "resolved": "https://registry.npmjs.org/@types/node/-/node-24.12.0.tgz",
+      "integrity": "sha512-GYDxsZi3ChgmckRT9HPU0WEhKLP08ev/Yfcq2AstjrDASOYCSXeyjDsHg4v5t4jOj7cyDX3vmprafKlWIG9MXQ==",
+      "license": "MIT",
+      "dependencies": {
+        "undici-types": "~7.16.0"
+      }
+    },
+    "node_modules/apache-arrow/node_modules/undici-types": {
+      "version": "7.16.0",
+      "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.16.0.tgz",
+      "integrity": "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==",
+      "license": "MIT"
+    },
+    "node_modules/array-back": {
+      "version": "6.2.3",
+      "resolved": "https://registry.npmjs.org/array-back/-/array-back-6.2.3.tgz",
+      "integrity": "sha512-SGDvmg6QTYiTxCBkYVmThcoa67uLl35pyzRHdpCGBOcqFy6BtwnphoFPk7LhJshD+Yk1Kt35WGWeZPTgwR4Fhw==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=12.17"
+      }
+    },
     "node_modules/baseline-browser-mapping": {
       "version": "2.9.19",
       "resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.9.19.tgz",
@@ -1650,6 +1731,37 @@
       ],
       "license": "CC-BY-4.0"
     },
+    "node_modules/chalk": {
+      "version": "4.1.2",
+      "resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.2.tgz",
+      "integrity": "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==",
+      "license": "MIT",
+      "dependencies": {
+        "ansi-styles": "^4.1.0",
+        "supports-color": "^7.1.0"
+      },
+      "engines": {
+        "node": ">=10"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/chalk?sponsor=1"
+      }
+    },
+    "node_modules/chalk-template": {
+      "version": "0.4.0",
+      "resolved": "https://registry.npmjs.org/chalk-template/-/chalk-template-0.4.0.tgz",
+      "integrity": "sha512-/ghrgmhfY8RaSdeo43hNXxpoHAtxdbskUHjPpfqUWGttFgycUhYPGx3YZBCnUCvOa7Doivn1IZec3DEGFoMgLg==",
+      "license": "MIT",
+      "dependencies": {
+        "chalk": "^4.1.2"
+      },
+      "engines": {
+        "node": ">=12"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/chalk-template?sponsor=1"
+      }
+    },
     "node_modules/clsx": {
       "version": "2.1.1",
       "resolved": "https://registry.npmjs.org/clsx/-/clsx-2.1.1.tgz",
@@ -1659,6 +1771,62 @@
         "node": ">=6"
       }
     },
+    "node_modules/color-convert": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
+      "integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
+      "license": "MIT",
+      "dependencies": {
+        "color-name": "~1.1.4"
+      },
+      "engines": {
+        "node": ">=7.0.0"
+      }
+    },
+    "node_modules/color-name": {
+      "version": "1.1.4",
+      "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
+      "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
+      "license": "MIT"
+    },
+    "node_modules/command-line-args": {
+      "version": "6.0.2",
+      "resolved": "https://registry.npmjs.org/command-line-args/-/command-line-args-6.0.2.tgz",
+      "integrity": "sha512-AIjYVxrV9X752LmPDLbVYv8aMCuHPSLZJXEo2qo/xJfv+NYhaZ4sMSF01rM+gHPaMgvPM0l5D/F+Qx+i2WfSmQ==",
+      "license": "MIT",
+      "dependencies": {
+        "array-back": "^6.2.3",
+        "find-replace": "^5.0.2",
+        "lodash.camelcase": "^4.3.0",
+        "typical": "^7.3.0"
+      },
+      "engines": {
+        "node": ">=12.20"
+      },
+      "peerDependencies": {
+        "@75lb/nature": "latest"
+      },
+      "peerDependenciesMeta": {
+        "@75lb/nature": {
+          "optional": true
+        }
+      }
+    },
+    "node_modules/command-line-usage": {
+      "version": "7.0.4",
+      "resolved": "https://registry.npmjs.org/command-line-usage/-/command-line-usage-7.0.4.tgz",
+      "integrity": "sha512-85UdvzTNx/+s5CkSgBm/0hzP80RFHAa7PsfeADE5ezZF3uHz3/Tqj9gIKGT9PTtpycc3Ua64T0oVulGfKxzfqg==",
+      "license": "MIT",
+      "dependencies": {
+        "array-back": "^6.2.2",
+        "chalk-template": "^0.4.0",
+        "table-layout": "^4.1.1",
+        "typical": "^7.3.0"
+      },
+      "engines": {
+        "node": ">=12.20.0"
+      }
+    },
     "node_modules/convert-source-map": {
       "version": "2.0.0",
       "resolved": "https://registry.npmjs.org/convert-source-map/-/convert-source-map-2.0.0.tgz",
@@ -1980,6 +2148,29 @@
         "node": ">=8"
       }
     },
+    "node_modules/find-replace": {
+      "version": "5.0.2",
+      "resolved": "https://registry.npmjs.org/find-replace/-/find-replace-5.0.2.tgz",
+      "integrity": "sha512-Y45BAiE3mz2QsrN2fb5QEtO4qb44NcS7en/0y9PEVsg351HsLeVclP8QPMH79Le9sH3rs5RSwJu99W0WPZO43Q==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=14"
+      },
+      "peerDependencies": {
+        "@75lb/nature": "latest"
+      },
+      "peerDependenciesMeta": {
+        "@75lb/nature": {
+          "optional": true
+        }
+      }
+    },
+    "node_modules/flatbuffers": {
+      "version": "25.9.23",
+      "resolved": "https://registry.npmjs.org/flatbuffers/-/flatbuffers-25.9.23.tgz",
+      "integrity": "sha512-MI1qs7Lo4Syw0EOzUl0xjs2lsoeqFku44KpngfIduHBYvzm8h2+7K8YMQh1JtVVVrUvhLpNwqVi4DERegUJhPQ==",
+      "license": "Apache-2.0"
+    },
     "node_modules/fsevents": {
       "version": "2.3.3",
       "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
@@ -2037,6 +2228,15 @@
       "dev": true,
       "license": "ISC"
     },
+    "node_modules/has-flag": {
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
+      "integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=8"
+      }
+    },
     "node_modules/internmap": {
       "version": "2.0.3",
       "resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz",
@@ -2086,6 +2286,14 @@
         "node": ">=6"
       }
     },
+    "node_modules/json-bignum": {
+      "version": "0.0.3",
+      "resolved": "https://registry.npmjs.org/json-bignum/-/json-bignum-0.0.3.tgz",
+      "integrity": "sha512-2WHyXj3OfHSgNyuzDbSxI1w2jgw5gkWSWhS7Qg4bWXx1nLk3jnbwfUeS0PSba3IzpTUWdHxBieELUzXRjQB2zg==",
+      "engines": {
+        "node": ">=0.8"
+      }
+    },
     "node_modules/json5": {
       "version": "2.2.3",
       "resolved": "https://registry.npmjs.org/json5/-/json5-2.2.3.tgz",
@@ -2360,6 +2568,12 @@
         "url": "https://opencollective.com/parcel"
       }
     },
+    "node_modules/lodash.camelcase": {
+      "version": "4.3.0",
+      "resolved": "https://registry.npmjs.org/lodash.camelcase/-/lodash.camelcase-4.3.0.tgz",
+      "integrity": "sha512-TwuEnCnxbc3rAvhf/LbG7tJUDzhqXyFnv3dtzLOPgCG/hODL7WFnsbwktkD7yUV0RrreP/l1PALq/YSg6VvjlA==",
+      "license": "MIT"
+    },
     "node_modules/lru-cache": {
       "version": "5.1.1",
       "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-5.1.1.tgz",
@@ -2625,6 +2839,31 @@
         "node": ">=0.10.0"
       }
     },
+    "node_modules/supports-color": {
+      "version": "7.2.0",
+      "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
+      "integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==",
+      "license": "MIT",
+      "dependencies": {
+        "has-flag": "^4.0.0"
+      },
+      "engines": {
+        "node": ">=8"
+      }
+    },
+    "node_modules/table-layout": {
+      "version": "4.1.1",
+      "resolved": "https://registry.npmjs.org/table-layout/-/table-layout-4.1.1.tgz",
+      "integrity": "sha512-iK5/YhZxq5GO5z8wb0bY1317uDF3Zjpha0QFFLA8/trAoiLbQD0HUbMesEaxyzUgDxi2QlcbM8IvqOlEjgoXBA==",
+      "license": "MIT",
+      "dependencies": {
+        "array-back": "^6.2.2",
+        "wordwrapjs": "^5.1.0"
+      },
+      "engines": {
+        "node": ">=12.17"
+      }
+    },
     "node_modules/tailwind-merge": {
       "version": "3.4.0",
       "resolved": "https://registry.npmjs.org/tailwind-merge/-/tailwind-merge-3.4.0.tgz",
@@ -2686,6 +2925,12 @@
         "node": ">=8.0"
       }
     },
+    "node_modules/tslib": {
+      "version": "2.8.1",
+      "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
+      "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
+      "license": "0BSD"
+    },
     "node_modules/tsx": {
       "version": "4.21.0",
       "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz",
@@ -3204,6 +3449,15 @@
         "node": ">=14.17"
       }
     },
+    "node_modules/typical": {
+      "version": "7.3.0",
+      "resolved": "https://registry.npmjs.org/typical/-/typical-7.3.0.tgz",
+      "integrity": "sha512-ya4mg/30vm+DOWfBg4YK3j2WD6TWtRkCbasOJr40CseYENzCUby/7rIvXA99JGsQHeNxLbnXdyLLxKSv3tauFw==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=12.17"
+      }
+    },
     "node_modules/undici-types": {
       "version": "6.21.0",
       "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
@@ -3334,6 +3588,15 @@
         "vite": "^5.4.11 || ^6.0.0 || ^7.0.0"
       }
     },
+    "node_modules/wordwrapjs": {
+      "version": "5.1.1",
+      "resolved": "https://registry.npmjs.org/wordwrapjs/-/wordwrapjs-5.1.1.tgz",
+      "integrity": "sha512-0yweIbkINJodk27gX9LBGMzyQdBDan3s/dEAiwBOj+Mf0PPyWL6/rikalkv8EeD0E8jm4o5RXEOrFTP3NXbhJg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=12.17"
+      }
+    },
     "node_modules/yallist": {
       "version": "3.1.1",
       "resolved": "https://registry.npmjs.org/yallist/-/yallist-3.1.1.tgz",

frontend/package.json

@@ -12,6 +12,7 @@
   "dependencies": {
     "@cosmos.gl/graph": "^2.6.4",
     "@webgpu/types": "^0.1.69",
+    "apache-arrow": "^21.1.0",
     "clsx": "2.1.1",
     "react": "19.2.3",
     "react-dom": "19.2.3",

frontend/src/App.tsx

@@ -4,77 +4,21 @@ import { fetchGraphQueries } from "./graph_queries";
 import type { GraphQueryMeta } from "./graph_queries";
 import { fetchSelectionQueries, runSelectionQuery, runSelectionTripleQuery } from "./selection_queries";
 import { cosmosRuntimeConfig } from "./cosmos_config";
-import type { GraphMeta, GraphRoutePoint, GraphRouteSegment, SelectionQueryMeta, SelectionTriple } from "./selection_queries";
+import type { GraphMeta, SelectionQueryMeta, SelectionTriple } from "./selection_queries";
 import { TripleGraphView } from "./TripleGraphView";
 import { buildTripleGraphModel, type TripleGraphModel } from "./triple_graph";
+import { readGraphArrow } from "./graph_arrow";
 
 function sleep(ms: number): Promise<void> {
   return new Promise((r) => setTimeout(r, ms));
 }
 
-type GraphNodeMeta = {
-  id?: number;
-  iri?: string;
-  label?: string;
-  x?: number;
-  y?: number;
+type GraphNodeLookup = {
+  vertexIds: Uint32Array;
+  labels: (string | undefined)[];
+  iris: string[];
 };
 
-function graphRoutePoint(value: unknown): GraphRoutePoint | null {
-  if (!value || typeof value !== "object") return null;
-  const record = value as Record<string, unknown>;
-  if (typeof record.x !== "number" || typeof record.y !== "number") return null;
-  return {
-    x: record.x,
-    y: record.y,
-  };
-}
-
-function graphRouteSegmentArray(value: unknown): GraphRouteSegment[] {
-  if (!Array.isArray(value)) return [];
-  const out: GraphRouteSegment[] = [];
-  for (const item of value) {
-    if (!item || typeof item !== "object") continue;
-    const record = item as Record<string, unknown>;
-    if (typeof record.edge_index !== "number" || typeof record.kind !== "string") continue;
-    if (!Array.isArray(record.points)) continue;
-    const points: GraphRoutePoint[] = [];
-    for (const point of record.points) {
-      const parsed = graphRoutePoint(point);
-      if (!parsed) continue;
-      points.push(parsed);
-    }
-    if (points.length < 2) continue;
-    out.push({
-      edge_index: record.edge_index,
-      kind: record.kind,
-      points,
-    });
-  }
-  return out;
-}
-
-function buildRouteLineVertices(routeSegments: GraphRouteSegment[]): Float32Array {
-  let lineCount = 0;
-  for (const route of routeSegments) {
-    lineCount += Math.max(0, route.points.length - 1);
-  }
-
-  const out = new Float32Array(lineCount * 4);
-  let offset = 0;
-  for (const route of routeSegments) {
-    for (let i = 1; i < route.points.length; i++) {
-      const previous = route.points[i - 1];
-      const current = route.points[i];
-      out[offset++] = previous.x;
-      out[offset++] = previous.y;
-      out[offset++] = current.x;
-      out[offset++] = current.y;
-    }
-  }
-  return out;
-}
-
 type TripleResultState = {
   status: "idle" | "loading" | "ready" | "error";
   queryId: string;
@@ -92,6 +36,37 @@ function idleTripleResult(queryId: string): TripleResultState {
   };
 }
 
+function formatBytes(bytes: number): string {
+  if (!Number.isFinite(bytes) || bytes <= 0) return "0 B";
+  const units = ["B", "KB", "MB", "GB", "TB"];
+  let value = bytes;
+  let unitIndex = 0;
+  while (value >= 1024 && unitIndex < units.length - 1) {
+    value /= 1024;
+    unitIndex++;
+  }
+  return `${value.toFixed(value >= 100 || unitIndex === 0 ? 0 : 1)} ${units[unitIndex]}`;
+}
+
+function estimateFrontendTypedArrayBytes(nodeCount: number, edgeCount: number, routeLineFloatCount: number): { app: number; renderer: number; total: number } {
+  const app =
+    nodeCount * Float32Array.BYTES_PER_ELEMENT * 2 +
+    nodeCount * Uint32Array.BYTES_PER_ELEMENT +
+    edgeCount * Uint32Array.BYTES_PER_ELEMENT * 2 +
+    routeLineFloatCount * Float32Array.BYTES_PER_ELEMENT;
+
+  const renderer =
+    nodeCount * Float32Array.BYTES_PER_ELEMENT * 2 +
+    nodeCount * Uint32Array.BYTES_PER_ELEMENT * 4 +
+    edgeCount * Uint32Array.BYTES_PER_ELEMENT * 4;
+
+  return {
+    app,
+    renderer,
+    total: app + renderer,
+  };
+}
+
 export default function App() {
   const canvasRef = useRef<HTMLCanvasElement>(null);
   const rendererRef = useRef<Renderer | null>(null);
@@ -121,14 +96,34 @@ export default function App() {
 
   // Store mouse position in a ref so it can be accessed in render loop without re-renders
   const mousePos = useRef({ x: 0, y: 0 });
-  const nodesRef = useRef<GraphNodeMeta[]>([]);
+  const nodeLookupRef = useRef<GraphNodeLookup | null>(null);
 
   async function loadGraph(graphQueryId: string, signal: AbortSignal): Promise<void> {
     const renderer = rendererRef.current;
     if (!renderer) return;
 
-    setStatus("Fetching graph…");
+    const loadStartedAt = performance.now();
+    const logPhase = (phase: string, extra?: Record<string, unknown>) => {
+      console.log(`[graph-load] ${phase}`, {
+        elapsed_ms: Math.round(performance.now() - loadStartedAt),
+        graph_query_id: graphQueryId,
+        ...(extra || {}),
+      });
+    };
+
+    const updateStatus = async (nextStatus: string, extra?: Record<string, unknown>): Promise<void> => {
+      setStatus(nextStatus);
+      logPhase(nextStatus, extra);
+      await sleep(0);
+    };
+
+    await updateStatus("Fetching graph…");
     const graphRes = await fetch(`/api/graph?graph_query_id=${encodeURIComponent(graphQueryId)}`, { signal });
+    logPhase("graph response headers received", {
+      status: graphRes.status,
+      content_type: graphRes.headers.get("content-type"),
+      content_length: graphRes.headers.get("content-length"),
+    });
     if (!graphRes.ok) {
       let detail = "";
       try {
@@ -141,44 +136,57 @@ export default function App() {
       }
       throw new Error(`Failed to fetch graph: ${graphRes.status}${detail ? ` (${detail})` : ""}`);
     }
-    const graph = await graphRes.json();
+    await updateStatus("Streaming Arrow graph…");
+    await updateStatus("Decoding Arrow batches…");
+    const decodeStartedAt = performance.now();
+    let graph: Awaited<ReturnType<typeof readGraphArrow>>;
+    try {
+      graph = await readGraphArrow(graphRes, logPhase);
+    } catch (e) {
+      logPhase("arrow graph decode failed", {
+        error: e instanceof Error ? e.message : String(e),
+      });
+      throw e;
+    }
+    logPhase("arrow graph decoded", {
+      decode_ms: Math.round(performance.now() - decodeStartedAt),
+    });
     if (signal.aborted) return;
 
-    const nodes = Array.isArray(graph.nodes) ? graph.nodes : [];
-    const edges = Array.isArray(graph.edges) ? graph.edges : [];
-    const routeSegments = graphRouteSegmentArray(graph.route_segments);
     const meta = graph.meta || null;
-    const count = nodes.length;
+    const vertexIds = graph.vertexIds;
+    const xs = graph.xs;
+    const ys = graph.ys;
+    const edgeData = graph.edgeData;
+    const routeLineVertices = graph.routeLineVertices;
+    const labels = graph.labels;
+    const iris = graph.iris;
+    const count = vertexIds.length;
+    const edgeCount = edgeData.length / 2;
+    logPhase("graph payload ready", {
+      nodes: count,
+      edges: edgeCount,
+      route_line_segments: routeLineVertices.length / 4,
+      backend_nodes: meta && typeof meta.nodes === "number" ? meta.nodes : undefined,
+      backend_edges: meta && typeof meta.edges === "number" ? meta.edges : undefined,
+    });
 
-    nodesRef.current = nodes;
+    nodeLookupRef.current = { vertexIds, labels, iris };
     graphMetaRef.current = meta && typeof meta === "object" ? (meta as GraphMeta) : null;
     setTripleResult(idleTripleResult(activeSelectionQueryId));
 
-    // Build positions from backend-provided node coordinates.
-    setStatus("Preparing buffers…");
-    const xs = new Float32Array(count);
-    const ys = new Float32Array(count);
-    for (let i = 0; i < count; i++) {
-      const nx = nodes[i]?.x;
-      const ny = nodes[i]?.y;
-      xs[i] = typeof nx === "number" ? nx : 0;
-      ys[i] = typeof ny === "number" ? ny : 0;
-    }
-    const vertexIds = new Uint32Array(count);
-    for (let i = 0; i < count; i++) {
-      const id = nodes[i]?.id;
-      vertexIds[i] = typeof id === "number" ? id >>> 0 : i;
-    }
-
-    // Build edges as vertex-id pairs.
-    const edgeData = new Uint32Array(edges.length * 2);
-    for (let i = 0; i < edges.length; i++) {
-      const s = edges[i]?.source;
-      const t = edges[i]?.target;
-      edgeData[i * 2] = typeof s === "number" ? s >>> 0 : 0;
-      edgeData[i * 2 + 1] = typeof t === "number" ? t >>> 0 : 0;
-    }
-    const routeLineVertices = buildRouteLineVertices(routeSegments);
+    await updateStatus("Preparing buffers…", {
+      nodes: count,
+      edges: edgeCount,
+    });
+    const bufferPrepStartedAt = performance.now();
+    const typedArrayBytes = estimateFrontendTypedArrayBytes(count, edgeCount, routeLineVertices.length);
+    logPhase("buffer prep done", {
+      buffer_prep_ms: Math.round(performance.now() - bufferPrepStartedAt),
+      app_typed_arrays: formatBytes(typedArrayBytes.app),
+      renderer_typed_arrays_estimate: formatBytes(typedArrayBytes.renderer),
+      total_typed_arrays_estimate: formatBytes(typedArrayBytes.total),
+    });
 
     // Use /api/graph meta; don't do a second expensive backend call.
     if (meta && typeof meta.nodes === "number" && typeof meta.edges === "number") {
@@ -188,33 +196,48 @@ export default function App() {
         backend: typeof meta.backend === "string" ? meta.backend : undefined,
       });
     } else {
-      setBackendStats({ nodes: nodes.length, edges: edges.length });
+      setBackendStats({ nodes: count, edges: edgeCount });
     }
 
-    setStatus("Building spatial index…");
-    await new Promise((r) => setTimeout(r, 0));
+    await updateStatus("Building spatial index…", {
+      nodes: count,
+      edges: edgeCount,
+    });
     if (signal.aborted) return;
 
-    const buildMs = renderer.init(
-      xs,
-      ys,
-      vertexIds,
-      edgeData,
-      routeLineVertices.length > 0 ? routeLineVertices : null
-    );
+    let buildMs: number;
+    try {
+      buildMs = renderer.init(
+        xs,
+        ys,
+        vertexIds,
+        edgeData,
+        routeLineVertices.length > 0 ? routeLineVertices : null
+      );
+    } catch (e) {
+      logPhase("renderer.init failed", {
+        error: e instanceof Error ? e.message : String(e),
+      });
+      throw e;
+    }
     setNodeCount(renderer.getNodeCount());
     setSelectedNodes(new Set());
     setStatus("");
-    console.log(`Init complete: ${count.toLocaleString()} nodes, ${edges.length.toLocaleString()} edges in ${buildMs.toFixed(0)}ms`);
+      logPhase("init complete", {
+        renderer_init_ms: Math.round(buildMs),
+        nodes: count,
+        edges: edgeCount,
+      });
   }
 
   function getSelectedIds(renderer: Renderer, selected: Set<number>): number[] {
+    const lookup = nodeLookupRef.current;
+    if (!lookup) return [];
     const selectedIds: number[] = [];
     for (const sortedIdx of selected) {
       const origIdx = renderer.sortedIndexToOriginalIndex(sortedIdx);
       if (origIdx === null) continue;
-      const node = nodesRef.current?.[origIdx];
-      const nodeId = node?.id;
+      const nodeId = lookup.vertexIds[origIdx];
       if (typeof nodeId !== "number") continue;
       selectedIds.push(nodeId);
     }
@@ -370,14 +393,16 @@ export default function App() {
       const hit = renderer.findNodeIndexAt(mousePos.current.x, mousePos.current.y);
       if (hit) {
         const origIdx = renderer.sortedIndexToOriginalIndex(hit.index);
-        const meta = origIdx === null ? null : nodesRef.current[origIdx];
+        const lookup = nodeLookupRef.current;
+        const label = origIdx === null || !lookup ? undefined : lookup.labels[origIdx];
+        const iri = origIdx === null || !lookup ? undefined : lookup.iris[origIdx];
         setHoveredNode({
           x: hit.x,
           y: hit.y,
           screenX: mousePos.current.x,
           screenY: mousePos.current.y,
-          label: meta && typeof meta.label === "string" ? meta.label : undefined,
-          iri: meta && typeof meta.iri === "string" ? meta.iri : undefined,
+          label: typeof label === "string" ? label : undefined,
+          iri: typeof iri === "string" ? iri : undefined,
         });
       } else {
         setHoveredNode(null);

frontend/src/TripleGraphView.tsx

@@ -1,6 +1,6 @@
 import { memo, useEffect, useMemo, useRef, useState } from "react";
 import { Graph, type GraphConfig } from "@cosmos.gl/graph";
-import { cosmosRuntimeConfig } from "./cosmos_config";
+import { cosmosBackgroundCss, cosmosRuntimeConfig } from "./cosmos_config";
 import {
   computeLayoutMetrics,
   type GraphLayoutMetrics,
@@ -194,28 +194,68 @@ export const TripleGraphView = memo(function TripleGraphView({ model }: TripleGr
     };
 
     const config: GraphConfig = {
-      backgroundColor: "#05070a",
+      backgroundColor: cosmosRuntimeConfig.backgroundColor,
       spaceSize: cosmosRuntimeConfig.spaceSize,
       enableSimulation: cosmosRuntimeConfig.enableSimulation,
-      enableDrag: true,
-      enableZoom: true,
-      fitViewOnInit: false,
+      pointDefaultColor: cosmosRuntimeConfig.pointDefaultColor,
+      pointGreyoutColor: cosmosRuntimeConfig.pointGreyoutColor,
+      pointGreyoutOpacity: cosmosRuntimeConfig.pointGreyoutOpacity,
+      pointDefaultSize: cosmosRuntimeConfig.pointDefaultSize,
+      pointOpacity: cosmosRuntimeConfig.pointOpacity,
+      pointSizeScale: cosmosRuntimeConfig.pointSizeScale,
+      hoveredPointCursor: cosmosRuntimeConfig.hoveredPointCursor,
+      hoveredLinkCursor: cosmosRuntimeConfig.hoveredLinkCursor,
+      renderHoveredPointRing: cosmosRuntimeConfig.renderHoveredPointRing,
+      hoveredPointRingColor: cosmosRuntimeConfig.hoveredPointRingColor,
+      focusedPointRingColor: cosmosRuntimeConfig.focusedPointRingColor,
+      renderLinks: cosmosRuntimeConfig.renderLinks,
+      linkDefaultColor: cosmosRuntimeConfig.linkDefaultColor,
+      linkOpacity: cosmosRuntimeConfig.linkOpacity,
+      linkGreyoutOpacity: cosmosRuntimeConfig.linkGreyoutOpacity,
+      linkDefaultWidth: cosmosRuntimeConfig.linkDefaultWidth,
+      hoveredLinkColor: cosmosRuntimeConfig.hoveredLinkColor,
+      hoveredLinkWidthIncrease: cosmosRuntimeConfig.hoveredLinkWidthIncrease,
+      linkWidthScale: cosmosRuntimeConfig.linkWidthScale,
+      scaleLinksOnZoom: cosmosRuntimeConfig.scaleLinksOnZoom,
+      enableDrag: cosmosRuntimeConfig.enableDrag,
+      enableZoom: cosmosRuntimeConfig.enableZoom,
+      enableSimulationDuringZoom: cosmosRuntimeConfig.enableSimulationDuringZoom,
+      fitViewOnInit: cosmosRuntimeConfig.fitViewOnInit,
+      fitViewDelay: cosmosRuntimeConfig.fitViewDelay,
       fitViewPadding: cosmosRuntimeConfig.fitViewPadding,
-      rescalePositions: false,
+      fitViewDuration: cosmosRuntimeConfig.fitViewDuration,
+      initialZoomLevel: cosmosRuntimeConfig.initialZoomLevel,
+      pointSamplingDistance: cosmosRuntimeConfig.pointSamplingDistance,
+      rescalePositions: cosmosRuntimeConfig.rescalePositions,
       curvedLinks: cosmosRuntimeConfig.curvedLinks,
+      curvedLinkSegments: cosmosRuntimeConfig.curvedLinkSegments,
+      curvedLinkWeight: cosmosRuntimeConfig.curvedLinkWeight,
+      curvedLinkControlPointDistance: cosmosRuntimeConfig.curvedLinkControlPointDistance,
+      linkDefaultArrows: cosmosRuntimeConfig.linkDefaultArrows,
+      linkArrowsSizeScale: cosmosRuntimeConfig.linkArrowsSizeScale,
+      linkVisibilityDistanceRange: cosmosRuntimeConfig.linkVisibilityDistanceRange,
+      linkVisibilityMinTransparency: cosmosRuntimeConfig.linkVisibilityMinTransparency,
+      useClassicQuadtree: cosmosRuntimeConfig.useClassicQuadtree,
       simulationDecay: cosmosRuntimeConfig.simulationDecay,
       simulationGravity: cosmosRuntimeConfig.simulationGravity,
       simulationCenter: cosmosRuntimeConfig.simulationCenter,
       simulationRepulsion: cosmosRuntimeConfig.simulationRepulsion,
+      simulationRepulsionTheta: cosmosRuntimeConfig.simulationRepulsionTheta,
+      simulationRepulsionQuadtreeLevels:
+        cosmosRuntimeConfig.simulationRepulsionQuadtreeLevels,
       simulationLinkSpring: cosmosRuntimeConfig.simulationLinkSpring,
       simulationLinkDistance: cosmosRuntimeConfig.simulationLinkDistance,
+      simulationLinkDistRandomVariationRange:
+        cosmosRuntimeConfig.simulationLinkDistRandomVariationRange,
+      simulationRepulsionFromMouse: cosmosRuntimeConfig.simulationRepulsionFromMouse,
+      enableRightClickRepulsion: cosmosRuntimeConfig.enableRightClickRepulsion,
       simulationFriction: cosmosRuntimeConfig.simulationFriction,
-      renderHoveredPointRing: true,
-      hoveredPointRingColor: "#35d6ff",
-      hoveredPointCursor: "pointer",
-      hoveredLinkCursor: "pointer",
-      hoveredLinkColor: "#ffd166",
-      hoveredLinkWidthIncrease: 2.5,
+      simulationCluster: cosmosRuntimeConfig.simulationCluster,
+      randomSeed: cosmosRuntimeConfig.randomSeed,
+      showFPSMonitor: cosmosRuntimeConfig.showFPSMonitor,
+      pixelRatio: cosmosRuntimeConfig.pixelRatio,
+      scalePointsOnZoom: cosmosRuntimeConfig.scalePointsOnZoom,
+      attribution: cosmosRuntimeConfig.attribution,
       onSimulationStart: () => {
         reportLayout("simulation-start", "running", 1);
       },
@@ -320,7 +360,14 @@ export const TripleGraphView = memo(function TripleGraphView({ model }: TripleGr
   }, [activeDetail]);
 
   return (
-    <div style={{ position: "relative", flex: 1, minHeight: 0, background: "#05070a" }}>
+    <div
+      style={{
+        position: "relative",
+        flex: 1,
+        minHeight: 0,
+        background: cosmosBackgroundCss,
+      }}
+    >
       <div
         ref={containerRef}
         style={{ position: "absolute", inset: 0, width: "100%", height: "100%" }}
@@ -468,7 +515,18 @@ function applyGraphModel(graph: Graph, model: TripleGraphModel): void {
   graph.setLinkWidths(model.linkWidths);
   graph.render(0);
   requestAnimationFrame(() => {
-    graph.fitViewByPointPositions(Array.from(model.pointPositions), 0, cosmosRuntimeConfig.fitViewPadding);
+    if (typeof cosmosRuntimeConfig.initialZoomLevel === "number") {
+      graph.setZoomLevel(
+        cosmosRuntimeConfig.initialZoomLevel,
+        cosmosRuntimeConfig.fitViewDuration,
+      );
+    } else {
+      graph.fitViewByPointPositions(
+        Array.from(model.pointPositions),
+        cosmosRuntimeConfig.fitViewDuration,
+        cosmosRuntimeConfig.fitViewPadding,
+      );
+    }
     if (cosmosRuntimeConfig.enableSimulation) {
       graph.start(1);
     }

frontend/src/cosmos_config.ts

@@ -1,3 +1,5 @@
+type CosmosColor = string | [number, number, number, number];
+
 function parseBoolean(value: string | undefined, fallback: boolean): boolean {
   if (value === undefined) return fallback;
   const normalized = value.trim().toLowerCase();
@@ -12,17 +14,167 @@ function parseNumber(value: string | undefined, fallback: number): number {
   return Number.isFinite(parsed) ? parsed : fallback;
 }
 
+function parseOptionalNumber(value: string | undefined): number | undefined {
+  if (value === undefined) return undefined;
+  const trimmed = value.trim();
+  if (trimmed.length === 0) return undefined;
+  const parsed = Number(trimmed);
+  return Number.isFinite(parsed) ? parsed : undefined;
+}
+
+function parseOptionalString(value: string | undefined): string | undefined {
+  if (value === undefined) return undefined;
+  const trimmed = value.trim();
+  return trimmed.length > 0 ? trimmed : undefined;
+}
+
+function parseNumberList(value: string | undefined, fallback: number[]): number[] {
+  return parseOptionalNumberList(value) ?? fallback;
+}
+
+function parseOptionalNumberList(value: string | undefined): number[] | undefined {
+  const raw = parseOptionalString(value);
+  if (!raw) return undefined;
+  const normalized = raw.startsWith("[") && raw.endsWith("]") ? raw.slice(1, -1) : raw;
+  const parts = normalized
+    .split(",")
+    .map((entry) => entry.trim())
+    .filter((entry) => entry.length > 0);
+  if (parts.length === 0) return undefined;
+  const parsed = parts.map((entry) => Number(entry));
+  return parsed.every((entry) => Number.isFinite(entry)) ? parsed : undefined;
+}
+
+function parseColor(value: string | undefined, fallback: CosmosColor): CosmosColor {
+  return parseOptionalColor(value) ?? fallback;
+}
+
+function parseOptionalColor(value: string | undefined): CosmosColor | undefined {
+  const raw = parseOptionalString(value);
+  if (!raw) return undefined;
+  const rgba = parseOptionalNumberList(raw);
+  if (rgba && rgba.length === 4) {
+    return [rgba[0], rgba[1], rgba[2], rgba[3]];
+  }
+  return raw;
+}
+
+function parseOptionalSeed(value: string | undefined): number | string | undefined {
+  const raw = parseOptionalString(value);
+  if (!raw) return undefined;
+  const numeric = Number(raw);
+  return Number.isFinite(numeric) ? numeric : raw;
+}
+
+function toCssColor(color: CosmosColor): string {
+  if (typeof color === "string") return color;
+  return `rgba(${color[0]}, ${color[1]}, ${color[2]}, ${color[3] / 255})`;
+}
+
 export const cosmosRuntimeConfig = {
   enableSimulation: parseBoolean(import.meta.env.VITE_COSMOS_ENABLE_SIMULATION, true),
   debugLayout: parseBoolean(import.meta.env.VITE_COSMOS_DEBUG_LAYOUT, false),
+  backgroundColor: parseColor(import.meta.env.VITE_COSMOS_BACKGROUND_COLOR, "#05070a"),
   spaceSize: parseNumber(import.meta.env.VITE_COSMOS_SPACE_SIZE, 4096),
+  pointDefaultColor: parseOptionalColor(import.meta.env.VITE_COSMOS_POINT_DEFAULT_COLOR),
+  pointGreyoutColor: parseOptionalColor(import.meta.env.VITE_COSMOS_POINT_GREYOUT_COLOR),
+  pointGreyoutOpacity: parseOptionalNumber(import.meta.env.VITE_COSMOS_POINT_GREYOUT_OPACITY),
+  pointDefaultSize: parseNumber(import.meta.env.VITE_COSMOS_POINT_DEFAULT_SIZE, 4),
+  pointOpacity: parseNumber(import.meta.env.VITE_COSMOS_POINT_OPACITY, 1),
+  pointSizeScale: parseNumber(import.meta.env.VITE_COSMOS_POINT_SIZE_SCALE, 1),
+  hoveredPointCursor: parseOptionalString(import.meta.env.VITE_COSMOS_HOVERED_POINT_CURSOR) ?? "pointer",
+  hoveredLinkCursor: parseOptionalString(import.meta.env.VITE_COSMOS_HOVERED_LINK_CURSOR) ?? "pointer",
+  renderHoveredPointRing: parseBoolean(
+    import.meta.env.VITE_COSMOS_RENDER_HOVERED_POINT_RING,
+    true,
+  ),
+  hoveredPointRingColor: parseColor(
+    import.meta.env.VITE_COSMOS_HOVERED_POINT_RING_COLOR,
+    "#35d6ff",
+  ),
+  focusedPointRingColor: parseColor(
+    import.meta.env.VITE_COSMOS_FOCUSED_POINT_RING_COLOR,
+    "white",
+  ),
+  renderLinks: parseBoolean(import.meta.env.VITE_COSMOS_RENDER_LINKS, true),
+  linkDefaultColor: parseOptionalColor(import.meta.env.VITE_COSMOS_LINK_DEFAULT_COLOR),
+  linkOpacity: parseNumber(import.meta.env.VITE_COSMOS_LINK_OPACITY, 1),
+  linkGreyoutOpacity: parseNumber(import.meta.env.VITE_COSMOS_LINK_GREYOUT_OPACITY, 0.1),
+  linkDefaultWidth: parseNumber(import.meta.env.VITE_COSMOS_LINK_DEFAULT_WIDTH, 1),
+  hoveredLinkColor: parseColor(import.meta.env.VITE_COSMOS_HOVERED_LINK_COLOR, "#ffd166"),
+  hoveredLinkWidthIncrease: parseNumber(
+    import.meta.env.VITE_COSMOS_HOVERED_LINK_WIDTH_INCREASE,
+    2.5,
+  ),
+  linkWidthScale: parseNumber(import.meta.env.VITE_COSMOS_LINK_WIDTH_SCALE, 1),
+  scaleLinksOnZoom: parseBoolean(import.meta.env.VITE_COSMOS_SCALE_LINKS_ON_ZOOM, false),
   curvedLinks: parseBoolean(import.meta.env.VITE_COSMOS_CURVED_LINKS, true),
-  fitViewPadding: parseNumber(import.meta.env.VITE_COSMOS_FIT_VIEW_PADDING, 0.12),
+  curvedLinkSegments: parseNumber(import.meta.env.VITE_COSMOS_CURVED_LINK_SEGMENTS, 19),
+  curvedLinkWeight: parseNumber(import.meta.env.VITE_COSMOS_CURVED_LINK_WEIGHT, 0.8),
+  curvedLinkControlPointDistance: parseNumber(
+    import.meta.env.VITE_COSMOS_CURVED_LINK_CONTROL_POINT_DISTANCE,
+    0.5,
+  ),
+  linkDefaultArrows: parseBoolean(import.meta.env.VITE_COSMOS_LINK_DEFAULT_ARROWS, false),
+  linkArrowsSizeScale: parseNumber(import.meta.env.VITE_COSMOS_LINK_ARROWS_SIZE_SCALE, 1),
+  linkVisibilityDistanceRange: parseNumberList(
+    import.meta.env.VITE_COSMOS_LINK_VISIBILITY_DISTANCE_RANGE,
+    [50, 150],
+  ),
+  linkVisibilityMinTransparency: parseNumber(
+    import.meta.env.VITE_COSMOS_LINK_VISIBILITY_MIN_TRANSPARENCY,
+    0.25,
+  ),
+  useClassicQuadtree: parseBoolean(import.meta.env.VITE_COSMOS_USE_CLASSIC_QUADTREE, false),
   simulationDecay: parseNumber(import.meta.env.VITE_COSMOS_SIMULATION_DECAY, 5000),
   simulationGravity: parseNumber(import.meta.env.VITE_COSMOS_SIMULATION_GRAVITY, 0),
   simulationCenter: parseNumber(import.meta.env.VITE_COSMOS_SIMULATION_CENTER, 0.05),
   simulationRepulsion: parseNumber(import.meta.env.VITE_COSMOS_SIMULATION_REPULSION, 0.5),
+  simulationRepulsionTheta: parseNumber(
+    import.meta.env.VITE_COSMOS_SIMULATION_REPULSION_THETA,
+    1.15,
+  ),
+  simulationRepulsionQuadtreeLevels: parseNumber(
+    import.meta.env.VITE_COSMOS_SIMULATION_REPULSION_QUADTREE_LEVELS,
+    12,
+  ),
   simulationLinkSpring: parseNumber(import.meta.env.VITE_COSMOS_SIMULATION_LINK_SPRING, 1),
   simulationLinkDistance: parseNumber(import.meta.env.VITE_COSMOS_SIMULATION_LINK_DISTANCE, 10),
+  simulationLinkDistRandomVariationRange: parseNumberList(
+    import.meta.env.VITE_COSMOS_SIMULATION_LINK_DISTANCE_RANDOM_VARIATION_RANGE,
+    [1, 1.2],
+  ),
+  simulationRepulsionFromMouse: parseNumber(
+    import.meta.env.VITE_COSMOS_SIMULATION_REPULSION_FROM_MOUSE,
+    2,
+  ),
+  enableRightClickRepulsion: parseBoolean(
+    import.meta.env.VITE_COSMOS_ENABLE_RIGHT_CLICK_REPULSION,
+    false,
+  ),
   simulationFriction: parseNumber(import.meta.env.VITE_COSMOS_SIMULATION_FRICTION, 0.1),
-} as const;
+  simulationCluster: parseNumber(import.meta.env.VITE_COSMOS_SIMULATION_CLUSTER, 0.1),
+  showFPSMonitor: parseBoolean(import.meta.env.VITE_COSMOS_SHOW_FPS_MONITOR, false),
+  pixelRatio: parseNumber(import.meta.env.VITE_COSMOS_PIXEL_RATIO, 2),
+  scalePointsOnZoom: parseBoolean(import.meta.env.VITE_COSMOS_SCALE_POINTS_ON_ZOOM, false),
+  initialZoomLevel: parseOptionalNumber(import.meta.env.VITE_COSMOS_INITIAL_ZOOM_LEVEL),
+  enableZoom: parseBoolean(import.meta.env.VITE_COSMOS_ENABLE_ZOOM, true),
+  enableSimulationDuringZoom: parseBoolean(
+    import.meta.env.VITE_COSMOS_ENABLE_SIMULATION_DURING_ZOOM,
+    false,
+  ),
+  enableDrag: parseBoolean(import.meta.env.VITE_COSMOS_ENABLE_DRAG, true),
+  fitViewOnInit: parseBoolean(import.meta.env.VITE_COSMOS_FIT_VIEW_ON_INIT, false),
+  fitViewDelay: parseNumber(import.meta.env.VITE_COSMOS_FIT_VIEW_DELAY, 250),
+  fitViewPadding: parseNumber(import.meta.env.VITE_COSMOS_FIT_VIEW_PADDING, 0.12),
+  fitViewDuration: parseNumber(import.meta.env.VITE_COSMOS_FIT_VIEW_DURATION, 250),
+  randomSeed: parseOptionalSeed(import.meta.env.VITE_COSMOS_RANDOM_SEED),
+  pointSamplingDistance: parseNumber(
+    import.meta.env.VITE_COSMOS_POINT_SAMPLING_DISTANCE,
+    150,
+  ),
+  rescalePositions: parseBoolean(import.meta.env.VITE_COSMOS_RESCALE_POSITIONS, false),
+  attribution: parseOptionalString(import.meta.env.VITE_COSMOS_ATTRIBUTION),
+};
+
+export const cosmosBackgroundCss = toCssColor(cosmosRuntimeConfig.backgroundColor);

frontend/src/graph_arrow.ts

@@ -0,0 +1,301 @@
+import { RecordBatchReader } from "apache-arrow";
+import type { GraphMeta } from "./selection_queries";
+
+export type ArrowGraphLoadResult = {
+  meta: GraphMeta | null;
+  vertexIds: Uint32Array;
+  xs: Float32Array;
+  ys: Float32Array;
+  edgeData: Uint32Array;
+  routeLineVertices: Float32Array;
+  labels: (string | undefined)[];
+  iris: string[];
+};
+
+type ArrowBatchLog = (phase: string, extra?: Record<string, unknown>) => void;
+
+type ArrowLikeVector = {
+  data?: Array<{ valueOffsets?: ArrayLike<number | bigint> }>;
+  getChildAt?: (index: number) => ArrowLikeVector | null;
+  get?: (index: number) => unknown;
+  toArray?: () => ArrayLike<number>;
+};
+
+const graphTransportVersion = "1";
+
+export async function readGraphArrow(response: Response, logPhase?: ArrowBatchLog): Promise<ArrowGraphLoadResult> {
+  validateArrowResponse(response);
+
+  const reader = await openArrowReader(response);
+
+  let meta: GraphMeta | null = null;
+  let vertexIds: Uint32Array | null = null;
+  let xs: Float32Array | null = null;
+  let ys: Float32Array | null = null;
+  let edgeData: Uint32Array | null = null;
+  let routeLineVertices: Float32Array | null = null;
+  let labels: (string | undefined)[] | null = null;
+  let iris: string[] | null = null;
+
+  let batchIndex = 0;
+  for await (const batch of reader) {
+    const batchStartedAt = performance.now();
+
+    switch (batchIndex) {
+      case 0: {
+        meta = decodeMetaBatch(batch);
+        const nodeCount = typeof meta.nodes === "number" ? meta.nodes : 0;
+        const edgeCount = typeof meta.edges === "number" ? meta.edges : 0;
+        const routeLineSegments = readScalarNumber(batch, "meta_route_line_segments") ?? 0;
+
+        vertexIds = new Uint32Array(nodeCount);
+        xs = new Float32Array(nodeCount);
+        ys = new Float32Array(nodeCount);
+        edgeData = new Uint32Array(edgeCount * 2);
+        routeLineVertices = new Float32Array(routeLineSegments * 4);
+        labels = new Array<string | undefined>(nodeCount);
+        iris = new Array<string>(nodeCount);
+
+        logPhase?.("arrow_meta_batch_decoded", {
+          batch_index: batchIndex,
+          rows: batch.numRows,
+          node_count: nodeCount,
+          edge_count: edgeCount,
+          route_line_segments: routeLineSegments,
+          decode_ms: Math.round(performance.now() - batchStartedAt),
+        });
+        break;
+      }
+      case 1: {
+        ensureAllocated(meta, vertexIds, xs, ys, edgeData, routeLineVertices, labels, iris);
+        const nodeIDs = readUint32List(batch, "node_id");
+        const nodeXs = readFloat32List(batch, "node_x");
+        const nodeYs = readFloat32List(batch, "node_y");
+        const nodeIRIs = readStringList(batch, "node_iri");
+        const nodeLabels = readNullableStringList(batch, "node_label");
+
+        if (nodeIDs.length !== vertexIds.length || nodeXs.length !== xs.length || nodeYs.length !== ys.length) {
+          throw new Error("Arrow node batch length mismatch");
+        }
+        if (nodeIRIs.length !== iris.length || nodeLabels.length !== labels.length) {
+          throw new Error("Arrow node metadata batch length mismatch");
+        }
+
+        vertexIds.set(nodeIDs);
+        xs.set(nodeXs);
+        ys.set(nodeYs);
+        for (let i = 0; i < iris.length; i++) {
+          iris[i] = nodeIRIs[i] ?? "";
+          labels[i] = nodeLabels[i];
+        }
+
+        logPhase?.("arrow_node_batch_decoded", {
+          batch_index: batchIndex,
+          rows: batch.numRows,
+          nodes: vertexIds.length,
+          decode_ms: Math.round(performance.now() - batchStartedAt),
+        });
+        break;
+      }
+      case 2: {
+        ensureAllocated(meta, vertexIds, xs, ys, edgeData, routeLineVertices, labels, iris);
+        const edgeSources = readUint32List(batch, "edge_source");
+        const edgeTargets = readUint32List(batch, "edge_target");
+        if (edgeSources.length !== edgeTargets.length) {
+          throw new Error("Arrow edge batch source/target length mismatch");
+        }
+        if (edgeData.length !== edgeSources.length * 2) {
+          throw new Error("Arrow edge batch size mismatch");
+        }
+        for (let i = 0; i < edgeSources.length; i++) {
+          edgeData[i * 2] = edgeSources[i];
+          edgeData[i * 2 + 1] = edgeTargets[i];
+        }
+        logPhase?.("arrow_edge_batch_decoded", {
+          batch_index: batchIndex,
+          rows: batch.numRows,
+          edges: edgeSources.length,
+          decode_ms: Math.round(performance.now() - batchStartedAt),
+        });
+        break;
+      }
+      case 3: {
+        ensureAllocated(meta, vertexIds, xs, ys, edgeData, routeLineVertices, labels, iris);
+        const routeX1 = readFloat32List(batch, "route_x1");
+        const routeY1 = readFloat32List(batch, "route_y1");
+        const routeX2 = readFloat32List(batch, "route_x2");
+        const routeY2 = readFloat32List(batch, "route_y2");
+        if (
+          routeX1.length !== routeY1.length ||
+          routeX1.length !== routeX2.length ||
+          routeX1.length !== routeY2.length
+        ) {
+          throw new Error("Arrow route batch axis length mismatch");
+        }
+        if (routeLineVertices.length !== routeX1.length * 4) {
+          throw new Error("Arrow route batch size mismatch");
+        }
+        for (let i = 0; i < routeX1.length; i++) {
+          routeLineVertices[i * 4] = routeX1[i];
+          routeLineVertices[i * 4 + 1] = routeY1[i];
+          routeLineVertices[i * 4 + 2] = routeX2[i];
+          routeLineVertices[i * 4 + 3] = routeY2[i];
+        }
+        logPhase?.("arrow_route_batch_decoded", {
+          batch_index: batchIndex,
+          rows: batch.numRows,
+          route_line_segments: routeX1.length,
+          decode_ms: Math.round(performance.now() - batchStartedAt),
+        });
+        break;
+      }
+      default:
+        throw new Error(`Unexpected Arrow batch index ${batchIndex}`);
+    }
+
+    batchIndex++;
+  }
+
+  if (batchIndex !== 4) {
+    throw new Error(`Expected 4 Arrow batches, received ${batchIndex}`);
+  }
+  ensureAllocated(meta, vertexIds, xs, ys, edgeData, routeLineVertices, labels, iris);
+
+  return {
+    meta,
+    vertexIds,
+    xs,
+    ys,
+    edgeData,
+    routeLineVertices,
+    labels,
+    iris,
+  };
+}
+
+async function openArrowReader(response: Response) {
+  if (response.body) {
+    return RecordBatchReader.from(response.body);
+  }
+  return RecordBatchReader.from(await response.arrayBuffer());
+}
+
+function validateArrowResponse(response: Response): void {
+  const contentType = response.headers.get("content-type") || "";
+  if (!contentType.includes("application/vnd.apache.arrow.stream")) {
+    throw new Error(`Unexpected graph content type: ${contentType || "(missing)"}`);
+  }
+  const version = response.headers.get("X-Graph-Transport-Version");
+  if (version !== graphTransportVersion) {
+    throw new Error(`Unexpected graph transport version: ${version || "(missing)"}`);
+  }
+}
+
+function decodeMetaBatch(batch: any): GraphMeta {
+  return {
+    backend: readScalarString(batch, "meta_backend"),
+    graph_query_id: readScalarString(batch, "meta_graph_query_id"),
+    node_limit: readScalarNumber(batch, "meta_node_limit"),
+    edge_limit: readScalarNumber(batch, "meta_edge_limit"),
+    nodes: readScalarNumber(batch, "meta_nodes"),
+    edges: readScalarNumber(batch, "meta_edges"),
+  };
+}
+
+function ensureAllocated(
+  meta: GraphMeta | null,
+  vertexIds: Uint32Array | null,
+  xs: Float32Array | null,
+  ys: Float32Array | null,
+  edgeData: Uint32Array | null,
+  routeLineVertices: Float32Array | null,
+  labels: (string | undefined)[] | null,
+  iris: string[] | null
+): asserts meta is GraphMeta & {} & {
+  nodes?: number;
+  edges?: number;
+} {
+  if (!meta || !vertexIds || !xs || !ys || !edgeData || !routeLineVertices || !labels || !iris) {
+    throw new Error("Arrow graph stream is missing the meta batch");
+  }
+}
+
+function readScalarString(batch: any, name: string): string | undefined {
+  const vector = batch.getChild(name);
+  if (!vector || batch.numRows < 1) return undefined;
+  const value = vector.get(0);
+  return typeof value === "string" ? value : undefined;
+}
+
+function readScalarNumber(batch: any, name: string): number | undefined {
+  const vector = batch.getChild(name);
+  if (!vector || batch.numRows < 1) return undefined;
+  const value = vector.get(0);
+  return typeof value === "number" && Number.isFinite(value) ? value : undefined;
+}
+
+function readUint32List(batch: any, name: string): Uint32Array {
+  return readPrimitiveList(batch, name, Uint32Array);
+}
+
+function readFloat32List(batch: any, name: string): Float32Array {
+  return readPrimitiveList(batch, name, Float32Array);
+}
+
+function readPrimitiveList<T extends Uint32Array | Float32Array>(
+  batch: any,
+  name: string,
+  ctor: { new(length: number): T }
+): T {
+  const vector = batch.getChild(name) as ArrowLikeVector | null;
+  if (!vector) return new ctor(0);
+
+  const [begin, end] = getListBounds(vector);
+  const child = vector.getChildAt?.(0);
+  if (!child || typeof child.toArray !== "function") {
+    return new ctor(0);
+  }
+  const values = child.toArray();
+  if (typeof (values as any).subarray === "function") {
+    return (values as T).subarray(begin, end) as T;
+  }
+
+  const out = new ctor(end - begin);
+  for (let i = begin; i < end; i++) {
+    out[i - begin] = Number((values as ArrayLike<number>)[i]) as T[number];
+  }
+  return out;
+}
+
+function readStringList(batch: any, name: string): string[] {
+  return readNullableStringList(batch, name).map((value) => value ?? "");
+}
+
+function readNullableStringList(batch: any, name: string): (string | undefined)[] {
+  const vector = batch.getChild(name) as ArrowLikeVector | null;
+  if (!vector) return [];
+
+  const [begin, end] = getListBounds(vector);
+  const child = vector.getChildAt?.(0);
+  if (!child || typeof child.get !== "function") {
+    return [];
+  }
+
+  const out = new Array<string | undefined>(Math.max(0, end - begin));
+  for (let i = begin; i < end; i++) {
+    const value = child.get(i);
+    out[i - begin] = typeof value === "string" ? value : undefined;
+  }
+  return out;
+}
+
+function getListBounds(vector: ArrowLikeVector): [number, number] {
+  const offsets = vector.data?.[0]?.valueOffsets;
+  if (!offsets || offsets.length < 2) return [0, 0];
+  return [offsetToNumber(offsets[0]), offsetToNumber(offsets[1])];
+}
+
+function offsetToNumber(value: number | bigint | undefined): number {
+  if (typeof value === "bigint") return Number(value);
+  return typeof value === "number" ? value : 0;
+}

frontend/src/renderer.ts

@@ -213,12 +213,23 @@ export class Renderer {
     const count = xs.length;
     const edgeCount = edges.length / 2;
     this.nodeCount = count;
+    console.log("[renderer.init] start", {
+      nodes: count,
+      edges: edgeCount,
+      route_line_vertices: routeLineVertices ? routeLineVertices.length / 2 : 0,
+    });
 
     // Build quadtree (spatially sorts the array)
+    const spatialStart = performance.now();
     const { sorted, leaves, order } = buildSpatialIndex(xs, ys);
     this.leaves = leaves;
     this.sorted = sorted;
     this.sortedToOriginal = order;
+    console.log("[renderer.init] spatial index built", {
+      nodes: count,
+      leaves: leaves.length,
+      spatial_ms: Math.round(performance.now() - spatialStart),
+    });
 
     // Pre-allocate arrays for render loop (zero-allocation rendering)
     this.visibleLeafIndices = new Uint32Array(leaves.length);
@@ -226,12 +237,18 @@ export class Renderer {
     this.countsArray = new Int32Array(leaves.length);
 
     // Upload sorted particles to GPU as STATIC VBO (never changes)
+    const uploadNodesStart = performance.now();
     gl.bindVertexArray(this.vao);
     gl.bindBuffer(gl.ARRAY_BUFFER, this.nodeVbo);
     gl.bufferData(gl.ARRAY_BUFFER, sorted, gl.STATIC_DRAW);
     gl.bindVertexArray(null);
+    console.log("[renderer.init] node buffer uploaded", {
+      upload_ms: Math.round(performance.now() - uploadNodesStart),
+      sorted_bytes: sorted.byteLength,
+    });
 
     // Build vertex ID → original input index mapping
+    const mapsStart = performance.now();
     const vertexIdToOriginal = new Map<number, number>();
     for (let i = 0; i < count; i++) {
       vertexIdToOriginal.set(vertexIds[i], i);
@@ -250,6 +267,10 @@ export class Renderer {
       vertexIdToSortedIndex.set(vertexIds[i], originalToSorted[i]);
     }
     this.vertexIdToSortedIndex = vertexIdToSortedIndex;
+    console.log("[renderer.init] index maps built", {
+      maps_ms: Math.round(performance.now() - mapsStart),
+      vertex_id_map_size: vertexIdToSortedIndex.size,
+    });
 
     this.useRawLineSegments = routeLineVertices !== null && routeLineVertices.length > 0;
     this.rawLineVertexCount = this.useRawLineSegments && routeLineVertices ? routeLineVertices.length / 2 : 0;
@@ -257,14 +278,20 @@ export class Renderer {
       this.edgeCount = edgeCount;
       this.leafEdgeStarts = new Uint32Array(0);
       this.leafEdgeCounts = new Uint32Array(0);
+      const uploadRoutesStart = performance.now();
       gl.bindVertexArray(this.lineVao);
       gl.bindBuffer(gl.ARRAY_BUFFER, this.lineVbo);
       gl.bufferData(gl.ARRAY_BUFFER, routeLineVertices, gl.STATIC_DRAW);
       gl.bindVertexArray(null);
+      console.log("[renderer.init] raw line segments uploaded", {
+        upload_ms: Math.round(performance.now() - uploadRoutesStart),
+        total_ms: Math.round(performance.now() - t0),
+      });
       return performance.now() - t0;
     }
 
     // Remap edges from vertex IDs to sorted indices
+    const remapEdgesStart = performance.now();
     const lineIndices = new Uint32Array(edgeCount * 2);
     let validEdges = 0;
     for (let i = 0; i < edgeCount; i++) {
@@ -278,9 +305,15 @@ export class Renderer {
       validEdges++;
     }
     this.edgeCount = validEdges;
+    console.log("[renderer.init] edges remapped", {
+      remap_ms: Math.round(performance.now() - remapEdgesStart),
+      valid_edges: validEdges,
+      line_indices_bytes: lineIndices.byteLength,
+    });
 
     // Build per-leaf edge index for efficient visible-only edge drawing
     // Find which leaf each sorted index belongs to
+    const edgeIndexStart = performance.now();
     const nodeToLeaf = new Uint32Array(count);
     for (let li = 0; li < leaves.length; li++) {
       const lf = leaves[li];
@@ -314,11 +347,22 @@ export class Renderer {
 
     this.leafEdgeStarts = leafEdgeOffsets;
     this.leafEdgeCounts = leafEdgeCounts;
+    console.log("[renderer.init] leaf edge index built", {
+      leaf_index_ms: Math.round(performance.now() - edgeIndexStart),
+      leaves: leaves.length,
+      sorted_edge_indices_bytes: sortedEdgeIndices.byteLength,
+    });
 
     // Upload sorted edges to GPU
+    const uploadEdgesStart = performance.now();
     gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, this.linesIbo);
     gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, sortedEdgeIndices, gl.STATIC_DRAW);
     gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, null);
+    console.log("[renderer.init] edge buffer uploaded", {
+      upload_ms: Math.round(performance.now() - uploadEdgesStart),
+      total_ms: Math.round(performance.now() - t0),
+      valid_edges: validEdges,
+    });
 
     return performance.now() - t0;
   }

frontend/src/selection_queries/index.ts

@@ -1,8 +1,6 @@
 export { fetchSelectionQueries, runSelectionQuery, runSelectionTripleQuery } from "./api";
 export type {
   GraphMeta,
-  GraphRoutePoint,
-  GraphRouteSegment,
   SelectionQueryMeta,
   SelectionTriple,
   SelectionTripleResult,

frontend/src/selection_queries/types.ts

@@ -1,26 +1,10 @@
 export type GraphMeta = {
   backend?: string;
-  ttl_path?: string | null;
-  sparql_endpoint?: string | null;
-  include_bnodes?: boolean;
   graph_query_id?: string;
   node_limit?: number;
   edge_limit?: number;
   nodes?: number;
   edges?: number;
-  layout_engine?: string;
-  layout_root_iri?: string | null;
-};
-
-export type GraphRoutePoint = {
-  x: number;
-  y: number;
-};
-
-export type GraphRouteSegment = {
-  edge_index: number;
-  kind: string;
-  points: GraphRoutePoint[];
 };
 
 export type SelectionQueryMeta = {

frontend/src/vite-env.d.ts

@@ -4,16 +4,65 @@ interface ImportMetaEnv {
   readonly VITE_BACKEND_URL?: string;
   readonly VITE_COSMOS_ENABLE_SIMULATION?: string;
   readonly VITE_COSMOS_DEBUG_LAYOUT?: string;
+  readonly VITE_COSMOS_BACKGROUND_COLOR?: string;
   readonly VITE_COSMOS_SPACE_SIZE?: string;
+  readonly VITE_COSMOS_POINT_DEFAULT_COLOR?: string;
+  readonly VITE_COSMOS_POINT_GREYOUT_COLOR?: string;
+  readonly VITE_COSMOS_POINT_GREYOUT_OPACITY?: string;
+  readonly VITE_COSMOS_POINT_DEFAULT_SIZE?: string;
+  readonly VITE_COSMOS_POINT_OPACITY?: string;
+  readonly VITE_COSMOS_POINT_SIZE_SCALE?: string;
+  readonly VITE_COSMOS_HOVERED_POINT_CURSOR?: string;
+  readonly VITE_COSMOS_HOVERED_LINK_CURSOR?: string;
+  readonly VITE_COSMOS_RENDER_HOVERED_POINT_RING?: string;
+  readonly VITE_COSMOS_HOVERED_POINT_RING_COLOR?: string;
+  readonly VITE_COSMOS_FOCUSED_POINT_RING_COLOR?: string;
+  readonly VITE_COSMOS_RENDER_LINKS?: string;
+  readonly VITE_COSMOS_LINK_DEFAULT_COLOR?: string;
+  readonly VITE_COSMOS_LINK_OPACITY?: string;
+  readonly VITE_COSMOS_LINK_GREYOUT_OPACITY?: string;
+  readonly VITE_COSMOS_LINK_DEFAULT_WIDTH?: string;
+  readonly VITE_COSMOS_HOVERED_LINK_COLOR?: string;
+  readonly VITE_COSMOS_HOVERED_LINK_WIDTH_INCREASE?: string;
+  readonly VITE_COSMOS_LINK_WIDTH_SCALE?: string;
+  readonly VITE_COSMOS_SCALE_LINKS_ON_ZOOM?: string;
   readonly VITE_COSMOS_CURVED_LINKS?: string;
+  readonly VITE_COSMOS_CURVED_LINK_SEGMENTS?: string;
+  readonly VITE_COSMOS_CURVED_LINK_WEIGHT?: string;
+  readonly VITE_COSMOS_CURVED_LINK_CONTROL_POINT_DISTANCE?: string;
+  readonly VITE_COSMOS_LINK_DEFAULT_ARROWS?: string;
+  readonly VITE_COSMOS_LINK_ARROWS_SIZE_SCALE?: string;
+  readonly VITE_COSMOS_LINK_VISIBILITY_DISTANCE_RANGE?: string;
+  readonly VITE_COSMOS_LINK_VISIBILITY_MIN_TRANSPARENCY?: string;
+  readonly VITE_COSMOS_USE_CLASSIC_QUADTREE?: string;
   readonly VITE_COSMOS_FIT_VIEW_PADDING?: string;
   readonly VITE_COSMOS_SIMULATION_DECAY?: string;
   readonly VITE_COSMOS_SIMULATION_GRAVITY?: string;
   readonly VITE_COSMOS_SIMULATION_CENTER?: string;
   readonly VITE_COSMOS_SIMULATION_REPULSION?: string;
+  readonly VITE_COSMOS_SIMULATION_REPULSION_THETA?: string;
+  readonly VITE_COSMOS_SIMULATION_REPULSION_QUADTREE_LEVELS?: string;
   readonly VITE_COSMOS_SIMULATION_LINK_SPRING?: string;
   readonly VITE_COSMOS_SIMULATION_LINK_DISTANCE?: string;
+  readonly VITE_COSMOS_SIMULATION_LINK_DISTANCE_RANDOM_VARIATION_RANGE?: string;
+  readonly VITE_COSMOS_SIMULATION_REPULSION_FROM_MOUSE?: string;
+  readonly VITE_COSMOS_ENABLE_RIGHT_CLICK_REPULSION?: string;
   readonly VITE_COSMOS_SIMULATION_FRICTION?: string;
+  readonly VITE_COSMOS_SIMULATION_CLUSTER?: string;
+  readonly VITE_COSMOS_SHOW_FPS_MONITOR?: string;
+  readonly VITE_COSMOS_PIXEL_RATIO?: string;
+  readonly VITE_COSMOS_SCALE_POINTS_ON_ZOOM?: string;
+  readonly VITE_COSMOS_INITIAL_ZOOM_LEVEL?: string;
+  readonly VITE_COSMOS_ENABLE_ZOOM?: string;
+  readonly VITE_COSMOS_ENABLE_SIMULATION_DURING_ZOOM?: string;
+  readonly VITE_COSMOS_ENABLE_DRAG?: string;
+  readonly VITE_COSMOS_FIT_VIEW_ON_INIT?: string;
+  readonly VITE_COSMOS_FIT_VIEW_DELAY?: string;
+  readonly VITE_COSMOS_FIT_VIEW_DURATION?: string;
+  readonly VITE_COSMOS_RANDOM_SEED?: string;
+  readonly VITE_COSMOS_POINT_SAMPLING_DISTANCE?: string;
+  readonly VITE_COSMOS_RESCALE_POSITIONS?: string;
+  readonly VITE_COSMOS_ATTRIBUTION?: string;
 }
 
 interface ImportMeta {

radial_sugiyama/GO_PIPELINE_INTERSECTION.md

@@ -1,144 +0,0 @@
-# Radial Sugiyama vs Go Snapshot Pipeline
-
-This note delimits the algorithmic intersection between the Rust pipeline in `radial_sugiyama/` and the Go snapshot/export path in:
-
-- `backend_go/graph_export.go`
-- `backend_go/graph_snapshot.go`
-
-The goal is not to describe integration mechanics yet, but to mark where the two implementations solve the same problem, where they only touch indirectly, and where they are solving different problems.
-
-## Scope
-
-The Rust pipeline is a hierarchy-specific layout pipeline:
-
-1. import ontology hierarchy from Turtle
-2. optionally filter to a rooted descendant subtree
-3. validate DAG structure
-4. assign hierarchy levels
-5. insert dummy nodes for long edges
-6. reduce crossings
-7. assign coordinates
-8. project to radial space
-9. generate routed edge artifacts
-10. export SVG
-
-The Go path is a snapshot/materialization pipeline:
-
-1. query predicates and edges from SPARQL
-2. accumulate nodes and edges
-3. build a graph response
-4. run a lightweight hierarchy layering + radial placement
-5. attach labels
-6. return JSON to the frontend
-
-Because of that, the true intersection is narrow in `graph_export.go` and broader in the layout section of `graph_snapshot.go`.
-
-## Legend
-
-- `Direct overlap`: both sides implement essentially the same algorithmic concern
-- `Adjacent overlap`: one side prepares or consumes the same kind of structure, but the algorithm differs materially
-- `No overlap`: the stage exists only on one side
-
-## Intersection with `graph_export.go`
-
-`graph_export.go` overlaps with the Rust pipeline only at graph materialization time.
-
-| Algorithmic stage | Rust pipeline | `graph_export.go` | Intersection | Notes |
-| --- | --- | --- | --- | --- |
-| Node identity and deduplication | `ttl.rs` maps class IRIs to stable node indices | `graphAccumulator.getOrAddNode` maps SPARQL terms to stable node IDs | Direct overlap | Both build a unique node set from repeated source records. |
-| Edge materialization | `ttl.rs` emits `superclass -> subclass` edges and deduplicates repeats | `graphAccumulator.addBindings` emits `source -> target` edges from SPARQL bindings | Adjacent overlap | Both convert raw triples/bindings into an in-memory graph, but Rust is specialized to `rdfs:subClassOf` while Go is predicate-agnostic. |
-| Literal / blank-node filtering | `ttl.rs` ignores blank/literal hierarchy endpoints | `getOrAddNode` skips literals and optionally keeps blank nodes | Adjacent overlap | Similar sanitation step, but not identical semantics. |
-| Predicate preservation | Rust discards all predicates except `rdfs:subClassOf` | Go preserves predicate IDs through `PredicateDict` | No overlap | This is Go-only in the compared files. |
-| Graph limits / capacity management | Rust does not enforce snapshot-style node and edge caps here | Go enforces `nodeLimit` and preallocates with edge hints | No overlap | This is an operational concern of the Go snapshot path. |
-
-### Boundary for `graph_export.go`
-
-The clean algorithmic seam is:
-
-- Go owns generic SPARQL binding ingestion and generic graph materialization.
-- Rust owns hierarchy-specialized interpretation once a hierarchy graph has already been isolated.
-
-That means `graph_export.go` is not competing with the Rust layout pipeline. It is only producing the kind of node/edge structure that Rust would eventually need as input.
-
-## Intersection with `graph_snapshot.go`
-
-`graph_snapshot.go` intersects with the Rust pipeline in two different regions:
-
-1. graph acquisition and hierarchy preparation
-2. lightweight layout assignment
-
-## Stage-by-stage comparison
-
-| Algorithmic stage | Rust pipeline | `graph_snapshot.go` | Intersection | Notes |
-| --- | --- | --- | --- | --- |
-| Source acquisition | `graph_from_ttl_path` parses Turtle directly | `fetchGraphSnapshot` queries SPARQL in batches | Adjacent overlap | Both acquire a graph, but from different upstream sources. |
-| Hierarchy graph extraction | Rust keeps only `rdfs:subClassOf` during import | Go accepts a `graph_query_id` and accumulates whatever that query returns | Adjacent overlap | The overlap is meaningful only when the Go query is hierarchy-like. |
-| Rooted subtree filtering | `filter_graph_to_descendants` keeps one configured root and its descendants | No equivalent in these two Go files | No overlap | This is currently Rust-only. |
-| Cycle detection / DAG validation | `compute_hierarchy_levels` rejects cyclic graphs | `levelSynchronousKahnLayers` returns a `CycleError` if not all nodes are processed | Direct overlap | Both need a DAG to continue with hierarchy layout. |
-| Level assignment | Rust computes longest-path hierarchy levels | Go computes level-synchronous Kahn layers | Direct overlap | Same problem, different algorithm. Both assign ring depth from DAG structure. |
-| Per-level ordering | Rust later optimizes order for crossings | Go sorts each layer lexicographically by IRI | Adjacent overlap | Both define an order inside a level, but Go is a simple deterministic ordering while Rust is layout-driven. |
-| Radial node placement | Rust projects coordinates to rings after Sugiyama coordinate assignment | Go uses `radialPositionsFromLayers` to place each layer on a ring | Direct overlap | Same output shape, very different sophistication. |
-| Coordinate shifting / scaling controls | Rust has configurable radius, spacing, borders, and positive-coordinate shifting | Go uses a fixed `maxR = 5000.0` radial envelope | Adjacent overlap | Both map levels to 2D coordinates, but only Rust exposes tuned geometry controls. |
-| Label enrichment | Rust keeps node labels as imported IRIs | Go fetches `rdfs:label` after layout | Adjacent overlap | Both carry node naming, but the enrichment algorithm is currently Go-only. |
-| Response packaging | Rust writes SVG and layout artifacts | Go returns `GraphResponse` JSON plus metadata | No overlap | Same graph, different downstream consumers. |
-
-## Rust-only algorithms with no counterpart in the compared Go files
-
-These parts of the Rust pipeline do not currently intersect with `graph_export.go` or `graph_snapshot.go`:
-
-- rooted descendant filtering
-- dummy-node insertion for long edges
-- crossing reduction / sifting
-- coordinate assignment before radial projection
-- adaptive / packed / distributed ring projection modes
-- routed edge generation
-- layout artifact generation
-- SVG rendering and export
-
-These are the parts that make the Rust pipeline a true Sugiyama-style layout engine rather than a simple radial snapshot placer.
-
-## Go-only algorithms with no counterpart in the Rust pipeline
-
-These parts of the compared Go files do not currently exist in Rust:
-
-- predicate dictionary construction from SPARQL results
-- batched SPARQL edge fetching with memory management
-- snapshot limits and backend metadata packaging
-- `rdfs:label` lookup through SPARQL
-- generic graph export over arbitrary predicate sets
-
-These are acquisition and serving concerns rather than layout concerns.
-
-## Algorithmic ownership boundary
-
-If the future integration wants a clean division of responsibility, the strongest ownership boundary is:
-
-### Go-owned stages
-
-- query execution against AnzoGraph / SPARQL
-- predicate-aware graph accumulation
-- generic graph snapshot materialization
-- label fetching and API response orchestration
-
-### Rust-owned stages
-
-- hierarchy-specific filtering
-- hierarchy-level assignment
-- Sugiyama expansion with dummy nodes
-- crossing minimization
-- coordinate assignment
-- radial projection and route generation
-- layout artifact production
-
-## Most important practical conclusion
-
-At algorithm granularity, the Rust pipeline intersects only lightly with `graph_export.go`, but it intersects substantially with the hierarchy-layout portion of `graph_snapshot.go`.
-
-The main replacement candidates in a future integration are therefore not the generic export/materialization routines in `graph_export.go`, but these hierarchy-layout steps currently performed by `graph_snapshot.go`:
-
-1. DAG validation / cycle detection
-2. layer assignment
-3. per-layer ordering
-4. radial coordinate generation
-
-Everything after that depends on how much of the Rust layout artifact model the future integration wants to expose to the frontend.