Add organizations v2 API and registry enrichment

docs/superpowers/plans/2026-05-06-dashboard-registry-enrichment-analytics.md

@@ -0,0 +1,205 @@
+# Dashboard Registry Enrichment Analytics Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Rebuild the dashboard analytics tab around active registry organizations and the enrichment pipeline that fills data for them.
+
+**Architecture:** Keep the existing Django API endpoint `/api/v1/parsers/dashboard/` and template `src/templates/dashboard.html`. Add a focused backend aggregate `registry_enrichment_analytics` beside existing fields, then make the analytics tab render registry coverage, matrix, pipeline, action queue, and secondary technical counters from that aggregate.
+
+**Tech Stack:** Django 3.2, DRF, existing parser/register ORM models, server-rendered HTML with inline vanilla JS/CSS, pytest.
+
+---
+
+### Task 1: Backend Analytics Contract
+
+**Files:**
+- Modify: `src/apps/parsers/views.py`
+- Test: `tests/apps/parsers/test_views.py`
+
+- [ ] **Step 1: Write failing API test**
+
+Add a test that creates two active registry organizations in two registries, FNS data for one organization, industrial data for one organization, and `unfair_suppliers` for one organization. Assert:
+
+- `registry_enrichment_analytics` exists.
+- active population is `2`.
+- `source_coverage` contains FNS/industrial and excludes `unfair_suppliers`.
+- `risk_signals` contains `unfair_suppliers`.
+- `registry_source_matrix` has per-registry source counts.
+- `core_profile_complete` is `1` when one organization has FNS and industrial coverage.
+
+Run:
+
+```bash
+PYTHONPATH=src .venv/bin/pytest tests/apps/parsers/test_views.py::ParsersViewSetTest::test_dashboard_data_exposes_registry_enrichment_analytics -v
+```
+
+Expected: FAIL because `registry_enrichment_analytics` is missing.
+
+- [ ] **Step 2: Implement aggregate helpers**
+
+In `src/apps/parsers/views.py`, add helpers based on the existing registry coverage matching:
+
+- active registry organization identity indexes.
+- source matched organization id sets.
+- source coverage entries.
+- risk signal entries.
+- registry/source matrix rows.
+- pipeline summary from schedules, jobs, and load logs.
+
+Use source matching rules already present:
+
+- default fields: `inn`, `ogrn`.
+- FNS: `ogrn` only.
+- legacy procurements: `customer_inn`, `customer_ogrn`.
+- `unfair_suppliers`: risk signal, not completeness.
+
+- [ ] **Step 3: Add aggregate to dashboard response**
+
+Return `registry_enrichment_analytics` from `ParserDashboardDataView.get()` while keeping existing `registry_data_coverage`.
+
+- [ ] **Step 4: Verify backend test passes**
+
+Run:
+
+```bash
+PYTHONPATH=src .venv/bin/pytest tests/apps/parsers/test_views.py::ParsersViewSetTest::test_dashboard_data_exposes_registry_enrichment_analytics -v
+```
+
+Expected: PASS.
+
+### Task 2: Analytics Template Structure
+
+**Files:**
+- Modify: `src/templates/dashboard.html`
+- Test: `tests/apps/parsers/test_dashboard_page.py`
+
+- [ ] **Step 1: Write failing template test**
+
+Add assertions that `/dashboard` contains:
+
+- `analyticsRegistryKpis`
+- `registrySourceCoverageChart`
+- `registrySourceMatrix`
+- `enrichmentPipelinePanel`
+- `analyticsActionQueue`
+- `technicalSourceCounters`
+- `renderRegistryEnrichmentAnalytics`
+- `renderRegistrySourceMatrix`
+
+Run:
+
+```bash
+PYTHONPATH=src .venv/bin/pytest tests/apps/parsers/test_dashboard_page.py::ParserDashboardPageTest::test_dashboard_prioritizes_registry_enrichment_analytics -v
+```
+
+Expected: FAIL because the new DOM/functions are missing.
+
+- [ ] **Step 2: Replace analytics panel markup**
+
+Update `analyticsPanel` so its first visible blocks are:
+
+- registry organization KPI grid.
+- registry source coverage + matrix.
+- enrichment pipeline + action queue.
+- secondary technical counters below.
+
+- [ ] **Step 3: Add CSS for compact bars and matrix**
+
+Add focused classes:
+
+- `.analytics-hero-grid`
+- `.registry-coverage-layout`
+- `.source-coverage-list`
+- `.registry-matrix`
+- `.matrix-cell`
+- `.action-list`
+- `.technical-counters`
+
+- [ ] **Step 4: Verify template test passes**
+
+Run:
+
+```bash
+PYTHONPATH=src .venv/bin/pytest tests/apps/parsers/test_dashboard_page.py::ParserDashboardPageTest::test_dashboard_prioritizes_registry_enrichment_analytics -v
+```
+
+Expected: PASS.
+
+### Task 3: Frontend Rendering
+
+**Files:**
+- Modify: `src/templates/dashboard.html`
+- Test: `tests/apps/parsers/test_dashboard_page.py`
+
+- [ ] **Step 1: Implement render functions**
+
+Add or update inline JS:
+
+- `renderRegistryEnrichmentAnalytics()`
+- `renderRegistrySourceCoverage()`
+- `renderRegistrySourceMatrix()`
+- `renderEnrichmentPipeline()`
+- `renderAnalyticsActionQueue()`
+- `renderTechnicalSourceCounters()`
+
+Change `renderAnalytics()` to call these functions and keep existing status/source totals as secondary content.
+
+- [ ] **Step 2: Add defensive empty states**
+
+If `dashboardData.registry_enrichment_analytics` is missing, render an empty state and keep secondary counters visible.
+
+- [ ] **Step 3: Syntax-check JS**
+
+Run:
+
+```bash
+perl -0ne 'while (m{<script>(.*?)</script>}sg) { print $1 }' src/templates/dashboard.html > /tmp/mostovik-dashboard-inline.js
+node --check /tmp/mostovik-dashboard-inline.js
+```
+
+Expected: exit code 0.
+
+### Task 4: Verification
+
+**Files:**
+- No new files.
+
+- [ ] **Step 1: Run focused tests**
+
+Run:
+
+```bash
+PYTHONPATH=src .venv/bin/pytest tests/apps/parsers/test_views.py::ParsersViewSetTest::test_dashboard_data_exposes_registry_enrichment_analytics tests/apps/parsers/test_dashboard_page.py::ParserDashboardPageTest::test_dashboard_prioritizes_registry_enrichment_analytics -v
+```
+
+Expected: PASS.
+
+- [ ] **Step 2: Run dashboard/parser regression suite**
+
+Run:
+
+```bash
+PYTHONPATH=src .venv/bin/pytest tests/apps/parsers/test_views.py tests/apps/parsers/test_dashboard_page.py tests/apps/organizations tests/apps/registers/test_services.py
+```
+
+Expected: PASS.
+
+- [ ] **Step 3: Run Django system check**
+
+Run:
+
+```bash
+PYTHONPATH=src .venv/bin/python src/manage.py check
+```
+
+Expected: `System check identified no issues`.
+
+- [ ] **Step 4: Browser smoke test**
+
+Open `/dashboard`, confirm:
+
+- first analytics section is registry coverage, not raw source records.
+- matrix renders.
+- pipeline renders.
+- source totals moved below primary analytics.
+- existing organization/FNS drill-down still works.