docs: refresh pipeline notes and transport analysis
This commit is contained in:
Oxy8
150
README.md
150
README.md
@@ -1,108 +1,96 @@
|
||||
# Visualizador Instanciados
|
||||
|
||||
This repo is a Docker Compose stack for visualizing large RDF/OWL graphs stored in **AnzoGraph**. It includes:
|
||||
Docker Compose stack for exploring large RDF/OWL graphs stored in AnzoGraph.
|
||||
|
||||
- A **Go backend** that queries AnzoGraph via SPARQL and serves a cached graph snapshot + selection queries.
|
||||
- A **React/Vite frontend** that renders nodes/edges with WebGL2 and supports “selection query” + “graph query” modes.
|
||||
- A **Python one-shot service** to combine `owl:imports` into a single Turtle file.
|
||||
- An **AnzoGraph** container (SPARQL endpoint).
|
||||
## What Runs Here
|
||||
|
||||
## Quick start (Docker Compose)
|
||||
- `anzograph`: SPARQL store
|
||||
- `backend`: Go API that queries AnzoGraph and serves cached graph snapshots
|
||||
- `frontend`: React/Vite app with a WebGL left graph and a right-side `cosmos.gl` selection graph
|
||||
- `owl_imports_combiner`: one-shot Python service that can merge `owl:imports`
|
||||
- `radial_sugiyama`: Rust hierarchy layout pipeline used in two ways:
|
||||
- standalone SVG generator through the `radial` Compose profile
|
||||
- optional hierarchy layout engine for the Go backend
|
||||
|
||||
1) Put your TTL file(s) in `./data/` (this folder is volume-mounted into AnzoGraph as `/opt/shared-files`).
|
||||
2) Optionally configure `.env` (see `.env.example`).
|
||||
3) Start the stack:
|
||||
## Current Flow
|
||||
|
||||
- The backend always builds graph snapshots from SPARQL queries against AnzoGraph.
|
||||
- `graph_query_id=default` and `graph_query_id=types` use the Go layout path.
|
||||
- `graph_query_id=hierarchy` can use either:
|
||||
- the Go layout path
|
||||
- the Rust radial Sugiyama path when `HIERARCHY_LAYOUT_ENGINE=rust`
|
||||
- When the Rust hierarchy path is enabled, the backend sends the hierarchy graph to Rust over JSON, Rust lays it out, returns node positions + routed edge segments, and also rewrites:
|
||||
|
||||
```text
|
||||
radial_sugiyama/out/layout.svg
|
||||
```
|
||||
|
||||
That SVG is a debug artifact for the exact Rust layout run used by the backend.
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. Put your TTL files under `./data/`.
|
||||
2. Copy or edit `.env` as needed.
|
||||
3. Start the stack:
|
||||
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Then open the frontend:
|
||||
Open:
|
||||
|
||||
- `http://localhost:5173`
|
||||
- Frontend: `http://localhost:5173`
|
||||
- Backend health: `http://localhost:8000/api/health`
|
||||
|
||||
Stop everything:
|
||||
Stop:
|
||||
|
||||
```bash
|
||||
docker compose down
|
||||
```
|
||||
|
||||
## Services
|
||||
## Rust Hierarchy Layout
|
||||
|
||||
Defined in `docker-compose.yml`:
|
||||
To use Rust for the `hierarchy` graph mode, set this in the repo root `.env`:
|
||||
|
||||
- `anzograph` (image `cambridgesemantics/anzograph:latest`)
|
||||
- Ports: `8080`, `8443`
|
||||
- Shared files: `./data → /opt/shared-files`
|
||||
- `backend` (`./backend_go`)
|
||||
- Port: `8000` (API under `/api/*`)
|
||||
- Talks to AnzoGraph at `SPARQL_HOST` / `SPARQL_ENDPOINT`
|
||||
- `frontend` (`./frontend`)
|
||||
- Port: `5173`
|
||||
- Proxies `/api/*` to `VITE_BACKEND_URL`
|
||||
- `owl_imports_combiner` (`./python_services/owl_imports_combiner`)
|
||||
- One-shot: optionally produces a combined TTL by following `owl:imports`
|
||||
```env
|
||||
HIERARCHY_LAYOUT_ENGINE=rust
|
||||
```
|
||||
|
||||
Service READMEs:
|
||||
The backend also reads `radial_sugiyama/.env` for the Rust layout settings such as:
|
||||
|
||||
- `backend_go/README.md`
|
||||
- `frontend/README.md`
|
||||
- `python_services/owl_imports_combiner/README.md`
|
||||
- `anzograph/README.md`
|
||||
- `RADIAL_ROOT_CLASS_IRI`
|
||||
- `RADIAL_OUTPUT_DIR`
|
||||
- `RADIAL_OUTPUT_FILE`
|
||||
- `RADIAL_RING_DISTRIBUTION`
|
||||
|
||||
## Repo layout
|
||||
The debug SVG for backend-driven hierarchy requests is written to:
|
||||
|
||||
- `backend_go/` – Go API service (SPARQL → snapshot + selection queries)
|
||||
- `frontend/` – React/Vite WebGL renderer
|
||||
- `python_services/owl_imports_combiner/` – Python one-shot OWL imports combiner
|
||||
- `data/` – local shared volume for TTL inputs/outputs (gitignored)
|
||||
- `docker-compose.yml` – service wiring
|
||||
- `flake.nix` – optional Nix dev shell
|
||||
```text
|
||||
radial_sugiyama/out/layout.svg
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
This repo expects a local `.env` file (not committed). Start from `.env.example`.
|
||||
|
||||
Common knobs:
|
||||
|
||||
- Backend snapshot size: `DEFAULT_NODE_LIMIT`, `DEFAULT_EDGE_LIMIT`, `MAX_NODE_LIMIT`, `MAX_EDGE_LIMIT`
|
||||
- SPARQL connectivity: `SPARQL_HOST` or `SPARQL_ENDPOINT`, plus `SPARQL_USER` / `SPARQL_PASS`
|
||||
- Load data on backend startup: `SPARQL_LOAD_ON_START=true` with `SPARQL_DATA_FILE=file:///opt/shared-files/<file>.ttl`
|
||||
- Frontend → backend proxy: `VITE_BACKEND_URL`
|
||||
|
||||
## API (backend)
|
||||
|
||||
Base URL: `http://localhost:8000`
|
||||
|
||||
- `GET /api/health` – liveness
|
||||
- `GET /api/stats` – snapshot stats (uses default limits)
|
||||
- `GET /api/graph` – graph snapshot
|
||||
- Query params: `node_limit`, `edge_limit`, `graph_query_id`
|
||||
- `GET /api/graph_queries` – available graph snapshot modes (`graph_query_id` values)
|
||||
- `GET /api/selection_queries` – available selection-highlight modes (`query_id` values)
|
||||
- `POST /api/selection_query` – run a selection query for highlighted neighbors
|
||||
- Body: `{"query_id":"neighbors","selected_ids":[...],"node_limit":...,"edge_limit":...,"graph_query_id":"default"}`
|
||||
- `POST /api/sparql` – raw SPARQL passthrough (debug/advanced)
|
||||
- `POST /api/neighbors` – legacy alias (same behavior as `query_id="neighbors"`)
|
||||
|
||||
## Frontend UI
|
||||
|
||||
- Mouse:
|
||||
- Drag: pan
|
||||
- Scroll: zoom
|
||||
- Click: select nodes
|
||||
- **Top-right buttons:** “selection query” mode (how neighbors/highlights are computed for the current selection)
|
||||
- **Bottom-right buttons:** “graph query” mode (which SPARQL edge set is used to build the graph snapshot; switching reloads the graph)
|
||||
|
||||
## Notes on performance/limits
|
||||
|
||||
- The backend caches snapshots in memory; tune `DEFAULT_*_LIMIT` if memory is too high.
|
||||
- The frontend renders a sampled subset when zoomed out, and only draws edges when fewer than ~20k nodes are visible.
|
||||
|
||||
## Nix dev shell (optional)
|
||||
|
||||
If you use Nix, `flake.nix` provides a minimal `devShell`:
|
||||
You can still run the standalone Rust SVG pipeline directly with:
|
||||
|
||||
```bash
|
||||
nix develop
|
||||
docker compose --profile radial up --build radial_sugiyama
|
||||
```
|
||||
|
||||
## Main API
|
||||
|
||||
- `GET /api/health`
|
||||
- `GET /api/stats`
|
||||
- `GET /api/graph`
|
||||
- `GET /api/graph_queries`
|
||||
- `GET /api/selection_queries`
|
||||
- `POST /api/selection_query`
|
||||
- `POST /api/selection_triples`
|
||||
- `POST /api/sparql`
|
||||
|
||||
## Repo Layout
|
||||
|
||||
- `backend_go/` Go API and SPARQL snapshot logic
|
||||
- `frontend/` React/Vite UI
|
||||
- `radial_sugiyama/` Rust hierarchy layout and SVG export
|
||||
- `python_services/owl_imports_combiner/` import-flattening helper
|
||||
- `data/` local shared data mounted into containers
|
||||
- `docker-compose.yml` service wiring
|
||||
|
||||
Reference in New Issue
Block a user