Files
mostovik-backend/docs/sources/2026-10-05-implementation-decisions.md
Aleksandr Meshchryakov 897941583e
All checks were successful
Mostovik Backend CI/CD / Tests and lint (push) Successful in 4m22s
Mostovik Backend CI/CD / Build linux/amd64 release images (push) Successful in 2m53s
Mostovik Backend CI/CD / Deploy and verify internal main (push) Has been skipped
Mostovik Backend CI/CD / Deploy dev (push) Successful in 2m55s
fix: include the approved public CA bundle in release images
2026-10-05 19:28:57 +02:00

13 KiB

Implementation decisions, 5 October 2026

Ordered ledger decisions and their costs; runtime gates remain pending.

  • Ruling: stay in existing dev checkout per user branch instruction; no tracked unrelated modifications — avoids new branch/worktree without changing scope — cost if wrong: local dev contains this implementation.

  • Task 1: Ruling: completed dataset has 16 headers, distinct from general 17; use confirmed separate mapping. Scope passport supports ended status; same-date completed observation outranks general while retaining both provenance — cost if wrong: overlapping IP misclassified; contradictory same-dataset rows reject.

  • Task 3: Ruling: add FSSP artifact private-storage prefix; raw large legal datasets must not acquire public media URLs — follows existing private-artifact mechanism — cost if wrong: configured private root required on dev.

  • Task 3: Ruling: settings.test disables all migrations; generate/check with MIGRATION_MODULES={} in the same isolated test-settings process, no user env changes — cost if wrong: migration drift could be missed without actual graph validation.

  • Task 4: Ruling: extend snapshot_metadata and backend build/check-runtime scripts — aggregate history must exclude child CSVs, and the documented --check command treated its flag as a filename — cost if wrong: history/schema gate regressions; covered by aggregate and runtime gates.

  • Task 4: Ruling: preserve publisher p. / p.p. reason text with ambiguous part/point null and warning; parse ISO termination date — verified full CSV labels differ from synthetic ch./p. mapping — cost if wrong: structured legal reason stays incomplete until mapping is verified.

  • Task 4: Ruling: eight-hour task limit, disabled daily interval — two current 4.7GB inputs and streaming validation support bounded operation; full dev publication remains a separate gate — cost if wrong: slow environments can time out safely before publication.

  • Task 4: Ruling: FSSP overflow pages return documented 200/empty while old groups retain existing 404; unsigned money behavior unchanged elsewhere — required frontend contract — cost if wrong: source-specific pagination differs, covered by wire tests.

  • Task 4: Ruling: extend existing demo exclusions to FSSP instead of fabricating proceedings — real source results require snapshot artifacts, matching existing Fedresurs exclusion — cost if wrong: demo commands have no FSSP sample records.

  • Task 1: Ruling: leasing is six verified /fedresurs event types, complete unfiltered pagination then filtering — one source claim, no undocumented contract fields — cost if wrong: richer contract details require another verified channel.

  • Task 1: Ruling: own 100-page/10,000-row cap, provider limit=100 — explicit oversized failures preserve old records and bound quota — cost if wrong: very large issuers need an approved larger cap.

  • Task 1: Ruling: documentation-derived synthetic fixtures suffice for implementation, selected-account access pending approved live probe — no paid requests in tests — cost if wrong: upstream deviations block readiness and require adapter fixes.

  • Task 2: Ruling: CompanyResponse adds defaulted presence/raw flags and contextual license aliases rather than changing global aliases — keep existing client callers compatible while rejecting malformed section envelopes — cost if wrong: a provider schema change blocks collection until reverified.

  • Task 3: Ruling: reuse paid-collector OPK/Rosatom/Roscosmos eligibility, require canonical import, valid ten-digit INN and thirteen/eight-digit OGRN/OKPO, unique head resolution — avoid paying for ambiguous or invalid directory rows — cost if wrong: invalid/branch identities remain preserved and need directory repair.

  • Task 3: Ruling: extend private artifact routing to these sources and store streamed normalized observations with original upstream fields — never publish source details under MEDIA_ROOT — cost if wrong: another raw-envelope audit format may be required.

  • Task 3: Ruling: fix existing publisher retained count independently of successful scope — prior overlapping filters reported zero preserved rows — cost if wrong: shared snapshot diagnostics could change, covered by FSSP/Fedresurs regressions.

  • Task 4: Ruling: keep approved Checko machine IDs and verified provenance intact in new exports, preserve provider-neutral display text and prior-source sanitizer — source IDs/URLs cannot be changed by legacy presentation scrubber — cost if wrong: new provenance fields visibly identify their documented provider.

  • Task 4: Ruling: export all history for these section groups; licence issuance year is not current validity — prevent valid older licences from disappearing in current-year exports — cost if wrong: selected section exports include historic messages too.

  • Task 4: Ruling: exclude new sections from fabricated demo records; empty extensions remain — quota-aware sources need actual adapters rather than invented demo payloads — cost if wrong: demo previews need separately approved explicit fixtures.

  • Task 4: Ruling: successful partial jobs expose counts but complete last-success excludes partial artifact metadata; all live records remain visible regardless batch — cost if wrong: changing eligibility leaves preserved old records until explicit future successful scope refresh.

  • Task 1: Ruling: use issuer emitent_inn from documented securities response, never infer INN from numeric emitent_id — cost if wrong: unmapped issuer is skipped/quarantined rather than name-matched.

  • Task 1: Ruling: replace obsolete XLSX plan with current official UTF-8 semicolon CSV; preserve strict observed 18 headers and use s_RTS_code, never Row/ISIN — verified official client export and 2287 unique codes — cost if wrong: upstream code changes reject/conflict; no guessed workbook channel.

  • Task 1: Ruling: SPB custom CA approved only for probe/download, production collector stays default verified TLS — no extension of consent — cost if wrong: runtime SPB remains blocked until trust approval/configuration.

  • Task 1: Ruling: MOEX shares and bonds reference pagination uses documented terminal empty, parameters/primary-board trading per exact canonical issuer; official init-dt selects dated duration response — cost if wrong: stale publisher date is preserved explicitly rather than guessed current.

  • Task 1: Ruling: preserve effectiveyield raw Decimal string with unit null until field-specific unit proof; do not label current yield — insufficient direct unit proof — cost if wrong: readiness/unit remains pending though confirmed duration/channel implementation continues.

  • Task 2: Ruling: add private stock_exchange_activity storage routing now rather than wait coordinator task — real download regression exposed public file URL — cost if wrong: all new raw/aggregate artifacts share one private prefix; no old source routing changed.

  • Task 3: Ruling: retain complete bounded HTTP envelopes (body_utf8, params, URL, per-response SHA) in one private NDJSON input, including failed partial reads — original precision retained without one DB artifact per request — cost if wrong: replay needs reading NDJSON envelope; checksum/date still explicit.

  • Task 3: Ruling: only exact canonical issuer triggers per-security parameter/trading HTTP, all reference pages still fully read/count; foreign/invalid INN counted — limits cost and no name matching — cost if wrong: missing authoritative issuer mapping remains diagnostic, not fabricated record.

  • Task 4: Ruling: repeated exchange-prefixed vendor IDs reject whole source, including exact duplicate rows — fail closed because reference/current export verifies uniqueness — cost if wrong: harmless upstream duplicates require an explicit dedup policy before publication.

  • Task 5: Ruling: add source refresh typed error envelope and arbitrary details mapping rather than accept generated Zod mutation — actual 403/400/404 differed from stale CommonResponses — cost if wrong: generated frontend error types change to match observed runtime, no runtime error behavior change.

  • Task 5: Ruling: exchange export all current records regardless listing year and private unmodified provenance, exclude fabricated demo records — securities remain listed across years, fixtures do not prove real listing — cost if wrong: export includes older instruments and demo needs explicit separately approved samples.

  • Task 5: Ruling: disabled daily schedule template and eight-hour limit only; local API/code gate separated from live readiness — preserves external authorization boundary — cost if wrong: runtime acceptance still required before enabling source.

  • Task 5: Ruling: finish all authorized local implementation/review now, retain SPB default TLS + MOEX HTML403 + yield-unit uncertainty as readiness gates; paid account and remote DB/deploy await bounded approval — mock/public subset evidence cannot prove complete dev publication — cost if wrong: five sources remain not-ready until exact live acceptance.

  • Final: Ruling: canonical FSSP/exchange OGRN and OKPO require 13/8 digits before publication — enforce existing authoritative requisites contract without repair or new organizations — cost if wrong: incomplete canonical records remain quarantined until directory repair.

  • Final: Ruling: actual Checko account/tariff/live compatibility remains pending bounded authorized acceptance — documentation-derived fixtures prove local logic only — cost if wrong: account or schema mismatch blocks live data.

  • Final: Ruling: SPB runtime trust stays unchanged; approved CA applies to public probe/download only — user scope is explicit — cost if wrong: production collector cannot yet retrieve SPB.

  • Final: Ruling: MOEX HTML403 and effectiveyield unit null remain explicit readiness gates — no WAF bypass or guessed units — cost if wrong: combined snapshot cannot publish until verified channel is available.

  • Final: Ruling: full canonical dev publication, staging scale and live API/export remain unverified — local reader/PostgreSQL evidence cannot establish remote population/performance — cost if wrong: capacity or matching failures require dev acceptance before ready.

  • Final: Ruling: deployment, schedules and production operation remain deferred — push dev triggers deployment and is outside local authorization — cost if wrong: local code exists without operational rollout.

  • Final: Ruling: valid-header empty CSV remains genuine empty under approved plan, with no independent publisher count — no invented completeness signal — cost if wrong: semantic header-only upstream truncation can clear existing rows and is a documented limit.

  • Final: Ruling: frontend rendering, StateCorp and root behavior remain outside scope — frontend only updated from Git per user and no code changes — cost if wrong: frontend-specific acceptance remains with its owner.

  • Final: Ruling: keep existing dev checkout as explicitly chosen by user; adapt finishing-branch menu rather than merge dev into itself or create unrequested PR — no push/deploy implied — cost if wrong: integration awaits explicit external authorization.

Deferred minors: none. All three Important reviewer findings were fixed with watched RED→GREEN regressions.

Subsequent explicit owner instruction

On 2026-10-05 the user requested all five schedules enabled and backend/dev pushed. This supersedes the earlier disabled/deploy-deferred rulings, while live data readiness remains separate from enablement.

  • Ruling: add forward migration 0048 rather than rewrite old seeds — supports already-applied and fresh databases — cost if wrong: 0048 must apply to activate.
  • Ruling: retain daily FSSP/exchange and 30-day Checko intervals/monthly claims — minimal requested change — cost if wrong: 30-day cadence is not a calendar month.

Live acceptance continuation

  • Ruling: accept nullable Checko publisher only when the canonical subject still matches the publisher or an exact participant — observed real upstream null, no invented identifier — cost if wrong: messages without confirmed subject remain rejected; generated clients must accept null.
  • Ruling: preserve specific SnapshotValidationError before its ValueError base handler — real diagnostics otherwise conceal the rejected field — cost if wrong: safe error codes become more specific for existing rejected inputs.
  • Ruling: stop the broken messages run cooperatively, keep all monthly claims, and rerun only eligible unclaimed scopes after release — avoids repeated costs for known adapter failures — cost if wrong: first-month coverage stays partial.
  • Ruling: use the two already verified official FSSP daily files for separate manual import acceptance when dev receives HTTP 474 — no network bypass or invented freshness — cost if wrong: manual publication does not prove the automatic upstream download is operational.
  • Ruling: enable the existing Russian CA only in SPB's dedicated allowlisted session behind a default-off setting and a dev-only host compose flag after explicit user consent — cost if wrong: only SPB gains this extra trust; MOEX/system/main trust remains unchanged, certificate verification stays on.
  • Ruling: whitelist exactly the existing public Russian CA bundle in Docker context and fail release image builds if it is missing — live image omitted it under the generic PEM exclusion — cost if wrong: this public bundle is shipped in web/worker; all other PEM/keys remain excluded.