feat(parsers): add proverki.gov.ru parser with sync_inspections task
- Add InspectionRecord model with is_federal_law_248, data_year, data_month fields - Add ProverkiClient with Playwright support for JS-rendered portal - Add streaming XML parser for large files (>50MB) - Add sync_inspections task with incremental loading logic - Starts from 01.01.2025 if DB is empty - Loads both FZ-294 and FZ-248 inspections - Stops after 2 consecutive empty months - Add InspectionService methods: get_last_loaded_period, has_data_for_period - Add Minpromtorg parsers (certificates, manufacturers) - Add Django Admin for parser models - Update README with parsers documentation and changelog
This commit is contained in:
22
src/apps/parsers/clients/__init__.py
Normal file
22
src/apps/parsers/clients/__init__.py
Normal file
@@ -0,0 +1,22 @@
|
||||
"""
|
||||
Клиенты для парсинга внешних источников данных.
|
||||
|
||||
Каждый источник имеет изолированный клиент, который:
|
||||
- Принимает настройки (proxy и т.д.) через конструктор
|
||||
- Возвращает типизированные dataclass объекты
|
||||
- Не зависит от Django ORM
|
||||
"""
|
||||
|
||||
from apps.parsers.clients.base import BaseHTTPClient
|
||||
from apps.parsers.clients.minpromtorg import (
|
||||
IndustrialProductionClient,
|
||||
ManufacturesClient,
|
||||
)
|
||||
from apps.parsers.clients.proverki import ProverkiClient
|
||||
|
||||
__all__ = [
|
||||
"BaseHTTPClient",
|
||||
"IndustrialProductionClient",
|
||||
"ManufacturesClient",
|
||||
"ProverkiClient",
|
||||
]
|
||||
Reference in New Issue
Block a user