feat: обновления парсеров, тестов и миграций
Some checks failed
CI/CD Pipeline / Run Tests (push) Failing after 37s
CI/CD Pipeline / Code Quality Checks (push) Failing after 43s
CI/CD Pipeline / Build & Push Images (push) Has been skipped
CI/CD Pipeline / Deploy (dev) (push) Has been skipped
CI/CD Pipeline / Deploy (prod) (push) Has been skipped
CI/CD Pipeline / Code Quality Checks (pull_request) Failing after 0s
CI/CD Pipeline / Run Tests (pull_request) Failing after 0s
CI/CD Pipeline / Build & Push Images (pull_request) Has been skipped
CI/CD Pipeline / Deploy (dev) (pull_request) Has been skipped
CI/CD Pipeline / Deploy (prod) (pull_request) Has been skipped

- Обновлены клиенты парсеров (checko, fns, minpromtorg, proverki, zakupki)
- Добавлены новые миграции для моделей
- Расширено покрытие тестами
- Обновлены конфигурации и настройки проекта
- Добавлены утилиты для тестирования

Co-Authored-By: Warp <agent@warp.dev>
This commit is contained in:
2026-02-10 10:17:47 +01:00
parent 975d019ba5
commit ee95628a0a
59 changed files with 7292 additions and 2876 deletions

View File

@@ -6,6 +6,7 @@ Views для приложения парсеров.
"""
import hashlib
import time
from pathlib import Path
from apps.parsers.models import (
@@ -367,6 +368,24 @@ class FNSReportUploadView(APIView):
from apps.parsers.services import FNSReportService
def _try_create_fns_lock(file_path: Path) -> bool:
lock_path = Path(f"{file_path}.lock")
if lock_path.exists():
try:
age_seconds = time.time() - lock_path.stat().st_mtime
ttl_seconds = getattr(settings, "FNS_LOCK_TTL_SECONDS", 3600)
if age_seconds > ttl_seconds:
lock_path.unlink()
else:
return False
except FileNotFoundError:
pass
try:
lock_path.touch(exist_ok=False)
except FileExistsError:
return False
return True
for file in files:
# Вычисляем хеш файла
file_content = file.read()
@@ -380,12 +399,30 @@ class FNSReportUploadView(APIView):
# Сохраняем файл
file_path = upload_dir / file.name
with open(file_path, "wb") as f:
for chunk in file.chunks():
f.write(chunk)
if not _try_create_fns_lock(file_path):
skipped += 1
continue
lock_path = Path(f"{file_path}.lock")
if file_path.exists():
lock_path.unlink(missing_ok=True)
skipped += 1
continue
try:
with open(file_path, "wb") as f:
for chunk in file.chunks():
f.write(chunk)
except Exception:
lock_path.unlink(missing_ok=True)
raise
# Ставим в очередь
task = process_fns_file.delay(str(file_path))
try:
task = process_fns_file.delay(str(file_path))
except Exception:
lock_path.unlink(missing_ok=True)
raise
task_ids.append(task.id)
queued += 1