mostovik-backend/docs/superpowers/plans/2026-05-18-polymorphic-organization-sources.md

Polymorphic Organization Sources Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Replace source-centric parser output access with organization-centric polymorphic source extensions.

Architecture: Keep Organization as the root entity. Add polymorphic source extensions per product source group and a shared subordinate source-record table. Backfill legacy parser tables idempotently, then switch API v2 to the new extension data.

Tech Stack: Django 3.2, Django REST Framework, django-filter, django-polymorphic, PostgreSQL, pytest.

---

Task 1: Dependency And Schema

Files:

• Modify: pyproject.toml

• Modify: uv.lock

• Modify: src/settings/base.py

• Modify: src/organizations/models.py

• Create: src/organizations/migrations/0006_polymorphic_source_extensions.py

• Test: tests/apps/organizations/test_source_extensions_models.py

• Add django-polymorphic to project dependencies.

• Add "polymorphic" to INSTALLED_APPS before local apps.

• Add source-group and identity-status choices.

• Add identity_status and primary_identity to Organization.

• Add OrganizationSourceExtension as PolymorphicModel.

• Add source extension subclasses.

• Add OrganizationSourceRecord.

• Add OrganizationSourceFinancialLine.

• Write tests proving:

  • one extension per (organization, source_group);
  • polymorphic queries return subclass instances;
  • source records are unique by legacy model/pk;
  • financial lines attach to a source record.

Task 2: Backfill Service

Files:

• Create: src/organizations/source_groups.py

• Create: src/organizations/source_backfill.py

• Create: src/organizations/management/commands/backfill_organization_sources.py

• Test: tests/apps/organizations/test_source_backfill.py

• Define source group mapping for all legacy parser sources.

• Implement organization resolution by inn + kpp, ogrn, ogrip, unique inn, then normalized name.

• Implement idempotent extension creation/update.

• Implement idempotent source record creation/update.

• Preserve legacy row payload and (legacy_model, legacy_pk).

• Backfill financial report lines into OrganizationSourceFinancialLine.

• Report scanned, created organizations, created extensions, updated extensions, created records, updated records, unresolved rows.

Task 3: API v2 Switch

Files:

• Modify: src/organizations/serializers.py

• Modify: src/organizations/filters.py

• Modify: src/organizations/views.py

• Delete or stop using: src/organizations/api_enrichment.py

• Delete or stop using: src/organizations/services.py snapshot refresh paths

• Test: tests/apps/organizations/test_api_v2.py

• Replace embedded data JSON with compact sources.

• Add source extension list/detail serializers.

• Add source records endpoint.

• Rework source filters to use OrganizationSourceExtension.

• Remove snapshot dependency from list/retrieve behavior.

• Keep old snapshot management command only as deprecated/no-op until cleanup.

Task 4: Parser Write Path

Files:

• Modify: src/apps/parsers/tasks.py

• Modify: src/organizations/tasks.py

• Test: tests/apps/parsers/test_tasks.py

• Test: tests/apps/organizations/test_tasks.py

• Replace snapshot refresh queueing with source backfill queueing for affected parser batches.

• For each parser completion, backfill only the completed source/batch.

• Keep full backfill command for initial migration and repair.

Task 5: Frontend Contract Repair

Files:

• Modify frontend generated API clients after backend OpenAPI changes.

• Modify source detail table composables to consume sources and source records endpoints.

• Regenerate API client.

• Update source pages to request extension records instead of embedded organization.data[source].

• Verify planned inspections page loads from source records.

Task 6: Cleanup Phase

Files:

• Modify migrations only after successful backfill validation.

• Remove OrganizationDataSnapshot.

• Remove snapshot refresh schedules.

• Decide which legacy parser tables remain as ingestion staging and which can be dropped.

• Run full backend and frontend validation.