feat: expand platform APIs, sources, and test coverage
Some checks failed
CI/CD Pipeline / Run Tests (pull_request) Successful in 1m53s
CI/CD Pipeline / Telegram Notify Success (push) Has been cancelled
CI/CD Pipeline / Run Tests (push) Has been cancelled
CI/CD Pipeline / Code Quality Checks (push) Has been cancelled
CI/CD Pipeline / Code Quality Checks (pull_request) Failing after 2m54s
CI/CD Pipeline / Telegram Notify Success (pull_request) Has been skipped
Some checks failed
CI/CD Pipeline / Run Tests (pull_request) Successful in 1m53s
CI/CD Pipeline / Telegram Notify Success (push) Has been cancelled
CI/CD Pipeline / Run Tests (push) Has been cancelled
CI/CD Pipeline / Code Quality Checks (push) Has been cancelled
CI/CD Pipeline / Code Quality Checks (pull_request) Failing after 2m54s
CI/CD Pipeline / Telegram Notify Success (pull_request) Has been skipped
This commit is contained in:
34
docs/adr/ADR-004: Data Ingestion and ETL Strategy.md
Normal file
34
docs/adr/ADR-004: Data Ingestion and ETL Strategy.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# ADR-004: Data Ingestion and ETL Strategy
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
Система интегрируется с нестабильными внешними источниками:
|
||||
- гос API
|
||||
- HTML/JS порталы
|
||||
- файлы (Excel/XML)
|
||||
|
||||
## Decision
|
||||
|
||||
Используется ETL-подход:
|
||||
|
||||
- Extract — парсеры
|
||||
- Transform — сервисный слой
|
||||
- Load — PostgreSQL
|
||||
|
||||
Особенности:
|
||||
- инкрементальная загрузка
|
||||
- потоковый парсинг больших файлов
|
||||
- обработка ошибок
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- контроль над данными
|
||||
- устойчивость к изменениям источников
|
||||
|
||||
### Negative
|
||||
- сложность поддержки парсеров
|
||||
- необходимость ручного восстановления
|
||||
Reference in New Issue
Block a user