feat(organizations): migrate source storage to polymorphic records

docs/superpowers/plans/2026-05-18-polymorphic-organization-sources.md

@@ -0,0 +1,100 @@
+# Polymorphic Organization Sources Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Replace source-centric parser output access with organization-centric polymorphic source extensions.
+
+**Architecture:** Keep `Organization` as the root entity. Add polymorphic source extensions per product source group and a shared subordinate source-record table. Backfill legacy parser tables idempotently, then switch API v2 to the new extension data.
+
+**Tech Stack:** Django 3.2, Django REST Framework, django-filter, django-polymorphic, PostgreSQL, pytest.
+
+---
+
+### Task 1: Dependency And Schema
+
+**Files:**
+- Modify: `pyproject.toml`
+- Modify: `uv.lock`
+- Modify: `src/settings/base.py`
+- Modify: `src/organizations/models.py`
+- Create: `src/organizations/migrations/0006_polymorphic_source_extensions.py`
+- Test: `tests/apps/organizations/test_source_extensions_models.py`
+
+- [ ] Add `django-polymorphic` to project dependencies.
+- [ ] Add `"polymorphic"` to `INSTALLED_APPS` before local apps.
+- [ ] Add source-group and identity-status choices.
+- [ ] Add `identity_status` and `primary_identity` to `Organization`.
+- [ ] Add `OrganizationSourceExtension` as `PolymorphicModel`.
+- [ ] Add source extension subclasses.
+- [ ] Add `OrganizationSourceRecord`.
+- [ ] Add `OrganizationSourceFinancialLine`.
+- [ ] Write tests proving:
+  - one extension per `(organization, source_group)`;
+  - polymorphic queries return subclass instances;
+  - source records are unique by legacy model/pk;
+  - financial lines attach to a source record.
+
+### Task 2: Backfill Service
+
+**Files:**
+- Create: `src/organizations/source_groups.py`
+- Create: `src/organizations/source_backfill.py`
+- Create: `src/organizations/management/commands/backfill_organization_sources.py`
+- Test: `tests/apps/organizations/test_source_backfill.py`
+
+- [ ] Define source group mapping for all legacy parser sources.
+- [ ] Implement organization resolution by `inn + kpp`, `ogrn`, `ogrip`, unique `inn`, then normalized name.
+- [ ] Implement idempotent extension creation/update.
+- [ ] Implement idempotent source record creation/update.
+- [ ] Preserve legacy row payload and `(legacy_model, legacy_pk)`.
+- [ ] Backfill financial report lines into `OrganizationSourceFinancialLine`.
+- [ ] Report scanned, created organizations, created extensions, updated extensions, created records, updated records, unresolved rows.
+
+### Task 3: API v2 Switch
+
+**Files:**
+- Modify: `src/organizations/serializers.py`
+- Modify: `src/organizations/filters.py`
+- Modify: `src/organizations/views.py`
+- Delete or stop using: `src/organizations/api_enrichment.py`
+- Delete or stop using: `src/organizations/services.py` snapshot refresh paths
+- Test: `tests/apps/organizations/test_api_v2.py`
+
+- [ ] Replace embedded `data` JSON with compact `sources`.
+- [ ] Add source extension list/detail serializers.
+- [ ] Add source records endpoint.
+- [ ] Rework source filters to use `OrganizationSourceExtension`.
+- [ ] Remove snapshot dependency from list/retrieve behavior.
+- [ ] Keep old snapshot management command only as deprecated/no-op until cleanup.
+
+### Task 4: Parser Write Path
+
+**Files:**
+- Modify: `src/apps/parsers/tasks.py`
+- Modify: `src/organizations/tasks.py`
+- Test: `tests/apps/parsers/test_tasks.py`
+- Test: `tests/apps/organizations/test_tasks.py`
+
+- [ ] Replace snapshot refresh queueing with source backfill queueing for affected parser batches.
+- [ ] For each parser completion, backfill only the completed source/batch.
+- [ ] Keep full backfill command for initial migration and repair.
+
+### Task 5: Frontend Contract Repair
+
+**Files:**
+- Modify frontend generated API clients after backend OpenAPI changes.
+- Modify source detail table composables to consume `sources` and source records endpoints.
+
+- [ ] Regenerate API client.
+- [ ] Update source pages to request extension records instead of embedded `organization.data[source]`.
+- [ ] Verify planned inspections page loads from source records.
+
+### Task 6: Cleanup Phase
+
+**Files:**
+- Modify migrations only after successful backfill validation.
+
+- [ ] Remove `OrganizationDataSnapshot`.
+- [ ] Remove snapshot refresh schedules.
+- [ ] Decide which legacy parser tables remain as ingestion staging and which can be dropped.
+- [ ] Run full backend and frontend validation.