6 Commits

Author SHA1 Message Date
mariomitte
61bc772054 fix: reduce memory usage for monthly ZIP archive flow
Replace generated archive download response with FileResponse streaming so ZIP files are not fully loaded into process memory.

Move monthly archive task output to a temporary file and upload that file to storage, and write invoice attachments into ZIP archives in chunks.

Add Celery memory guard settings, a memory-aware base task hook, and periodic /tmp cleanup for prefixed archive files older than one day.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-05 08:15:30 +02:00
mariomitte
b985c285f9 fix: apply EXIF orientation to service record images in PDF/DOCX
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Smartphone photos often have EXIF orientation metadata (tags 6, 8, 3)
that rotates the display but doesn't transform the pixel data. When
PDFs/DOCX embedded images without applying this metadata, they appear
rotated 90/180/270 degrees.

Use ImageOps.exif_transpose(img) before resize/convert in both
_compress_image_for_pdf() and _compress_image_for_docx() to read EXIF
orientation and transpose the actual pixel data accordingly. This is
a standard Pillow function and is a no-op for images without EXIF.

Add regression test test_compress_image_for_docx_handles_exif_orientation
to verify images with EXIF tag 0x0112 (orientation=6) are transposed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 11:08:16 +02:00
mariomitte
aa690d4ac4 fix: increase image megapixel limit from 8 to 30 MP to include standard phone photos
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Previous limit of 8 MP was rejecting standard smartphone photos (12-20 MP),
causing service record PDFs/DOCX documents to be generated without photos.

This was a regression from the image timeout fix: the megapixel guard was
designed to prevent processing of pathologically large files (preventing
worker timeouts), but the threshold was set too aggressively.

Increase limit to 30 MP to allow standard device cameras while still
rejecting extreme outliers that would cause timeout/OOM.

Add regression test test_compress_image_for_docx_keeps_standard_phone_photos
to verify 12 MP photos are accepted.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 10:54:40 +02:00
mariomitte
52408bc5a9 fix: BILINEAR+BaseException catch to fix image worker timeout in PDF/DOCX
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Root cause was WORKER TIMEOUT, not OOM:
- Gunicorn sends SIGABRT to workers that exceed 60s timeout
- SIGABRT handler calls sys.exit(1) raising SystemExit(BaseException)
- except Exception does NOT catch SystemExit, so the worker crashes

Two fixes:
1. except BaseException — catches SystemExit so the worker survives
   and gracefully skips the image (returns None) instead of dying
2. Image.BILINEAR instead of LANCZOS — orders of magnitude faster
   for large images, ensures processing completes well within 60s
3. Reduce max_width 1600->800, megapixel limit 20->8 MP, optimize=True removed
   (these reduce processing time further)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 10:43:27 +02:00
mariomitte
8e9d23fcfa fix: thumbnail-first + megapixel guard to prevent image OOM in PDF/DOCX
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Previous draft()+convert()+thumbnail() ordering still caused OOM for
non-JPEG formats (PNG/HEIC) because draft() is a no-op for those formats,
and convert('RGB') forces a full pixel decode regardless.

Replace with thumbnail()-first ordering:
- thumbnail() internally calls draft() for JPEG before decoding
- thumbnail() performs in-place resize without allocating a second
  full-resolution buffer
- convert('RGB') is then called on the already-small image (safe for
  any format)

Add _COMPRESS_IMAGE_MAX_MEGAPIXELS guard (20 MP): read image dimensions
from headers only (no pixel decode) and return None for images that
exceed the limit. This prevents OOM even for pathologically large files
where draft() provides no benefit (e.g. PNG, TIFF, HEIC).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 10:10:01 +02:00
mariomitte
a7220d27f5 fix: use draft()+thumbnail() to prevent OOM kill on large image resize
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
PIL img.resize() on high-resolution photos was exhausting worker RAM,
causing Gunicorn to SIGKILL the worker mid-request (production OOM crash).

Replace the explicit resize() path in both _compress_image_for_pdf() and
_compress_image_for_docx() with:
  - Image.draft() — hints the JPEG decoder to decode at a lower resolution
  - Image.thumbnail() — in-place resize that avoids allocating a second
    full-resolution buffer

This keeps peak memory proportional to the output size instead of the
original file size, preventing the worker from being killed when
processing multi-megapixel service-record photos.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 09:51:21 +02:00
8 changed files with 319 additions and 72 deletions

View File

@@ -7,7 +7,7 @@ Nakon što je gitea webhook i testirana skripta za automatsko povlačenje promje
```bash ```bash
cd /opt/erp/app cd /opt/erp/app
git fetch origin git fetch origin
git reset --hard origin/main git reset --hard origin/main ili git pull --ff-only origin main
# rebuild backend + worker + beat + frontend # rebuild backend + worker + beat + frontend
docker compose -f docker-compose.yml -f docker-compose.prod.yml build backend worker beat frontend docker compose -f docker-compose.yml -f docker-compose.prod.yml build backend worker beat frontend

View File

@@ -1,7 +1,12 @@
# backend/core/celery.py # backend/core/celery.py
import os import os
import gc
import logging
from celery import Celery from celery import Celery
from celery import Task
logger = logging.getLogger(__name__)
# Postavi Django settings modul # Postavi Django settings modul
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings') os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings')
@@ -11,5 +16,42 @@ app = Celery('core')
# Koristi konfiguraciju iz settings.py s prefiksom 'CELERY_' # Koristi konfiguraciju iz settings.py s prefiksom 'CELERY_'
app.config_from_object('django.conf:settings', namespace='CELERY') app.config_from_object('django.conf:settings', namespace='CELERY')
def _rss_memory_mb():
try:
import resource
usage_kb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
# Na Linuxu je ru_maxrss u KiB.
return round(float(usage_kb) / 1024.0, 2)
except (ImportError, AttributeError, OSError, ValueError):
return None
class MemoryAwareTask(Task):
abstract = True
def __call__(self, *args, **kwargs):
gc.collect()
return super().__call__(*args, **kwargs)
def after_return(self, status, retval, task_id, args, kwargs, einfo):
memory_before_gc = _rss_memory_mb()
gc.collect()
memory_after_gc = _rss_memory_mb()
if memory_before_gc is not None and memory_after_gc is not None:
logger.info(
"Task %s (%s) status=%s RSS prije/poslije GC: %.2fMB -> %.2fMB",
self.name,
task_id,
status,
memory_before_gc,
memory_after_gc,
)
return super().after_return(status, retval, task_id, args, kwargs, einfo)
app.Task = MemoryAwareTask
# Automatski pronalazi taskove u svim 'modules.*' # Automatski pronalazi taskove u svim 'modules.*'
app.autodiscover_tasks() app.autodiscover_tasks()

View File

@@ -154,6 +154,11 @@ REDIS_URL = os.environ.get('REDIS_URL', 'redis://localhost:6379/0')
CELERY_BROKER_URL = REDIS_URL CELERY_BROKER_URL = REDIS_URL
CELERY_RESULT_BACKEND = REDIS_URL CELERY_RESULT_BACKEND = REDIS_URL
CELERY_WORKER_PREFETCH_MULTIPLIER = int(os.environ.get('CELERY_WORKER_PREFETCH_MULTIPLIER', '1'))
CELERY_WORKER_MAX_TASKS_PER_CHILD = int(os.environ.get('CELERY_WORKER_MAX_TASKS_PER_CHILD', '20'))
CELERY_WORKER_MAX_MEMORY_PER_CHILD = int(os.environ.get('CELERY_WORKER_MAX_MEMORY_PER_CHILD', '350000'))
CELERY_TASK_ACKS_LATE = True
CELERY_TASK_REJECT_ON_WORKER_LOST = True
# Ako želiš koristiti Redis kao brzi cache sustav unutar Djanga (izvrsno za ERP performanse) # Ako želiš koristiti Redis kao brzi cache sustav unutar Djanga (izvrsno za ERP performanse)
# Za ovo ti je potreban paket 'django-redis' u requirements.txt # Za ovo ti je potreban paket 'django-redis' u requirements.txt
@@ -171,10 +176,18 @@ CACHES = {
CELERY_ACCEPT_CONTENT = ['json'] CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json' CELERY_TASK_SERIALIZER = 'json'
CELERY_BEAT_SCHEDULE = { CELERY_BEAT_SCHEDULE = {
'cleanup-expired-generated-archives-hourly': {
'task': 'modules.fleet.tasks.cleanup_expired_generated_archives_task',
'schedule': crontab(minute=5),
},
'cleanup-expired-generated-pdfs-hourly': { 'cleanup-expired-generated-pdfs-hourly': {
'task': 'modules.fleet.tasks.cleanup_expired_generated_pdfs_task', 'task': 'modules.fleet.tasks.cleanup_expired_generated_pdfs_task',
'schedule': crontab(minute=0), 'schedule': crontab(minute=0),
}, },
'cleanup-app-tmp-daily': {
'task': 'modules.fleet.tasks.cleanup_tmp_files_task',
'schedule': crontab(hour=3, minute=15),
},
'notify-upcoming-tasks-daily': { 'notify-upcoming-tasks-daily': {
'task': 'modules.task_management.tasks.notify_upcoming_tasks', 'task': 'modules.task_management.tasks.notify_upcoming_tasks',
'schedule': crontab(hour=8, minute=0), 'schedule': crontab(hour=8, minute=0),
@@ -190,6 +203,13 @@ STATIC_URL = '/static/'
# (Opcionalno, ali preporučeno za ERP) Ako koristiš i medije (dokumente) # (Opcionalno, ali preporučeno za ERP) Ako koristiš i medije (dokumente)
MEDIA_URL = '/media/' MEDIA_URL = '/media/'
MEDIA_ROOT = BASE_DIR / 'media' MEDIA_ROOT = BASE_DIR / 'media'
APP_TMP_CLEANUP_DIR = os.environ.get('APP_TMP_CLEANUP_DIR', '/tmp')
APP_TMP_CLEANUP_MAX_AGE_HOURS = int(os.environ.get('APP_TMP_CLEANUP_MAX_AGE_HOURS', '24'))
APP_TMP_CLEANUP_PREFIXES = [
prefix.strip()
for prefix in os.environ.get('APP_TMP_CLEANUP_PREFIXES', 'erp-fleet-archive-').split(',')
if prefix.strip()
]
# ============================================================================== # ==============================================================================

View File

@@ -5,6 +5,7 @@ from django.core.mail import EmailMessage
from django.conf import settings from django.conf import settings
from django.utils import timezone from django.utils import timezone
from django.core.files.base import ContentFile from django.core.files.base import ContentFile
from django.core.files import File
import logging import logging
from io import BytesIO from io import BytesIO
from io import StringIO from io import StringIO
@@ -13,6 +14,9 @@ import base64
import re import re
import csv import csv
import mimetypes import mimetypes
import os
import tempfile
import zipfile
from decimal import Decimal, InvalidOperation from decimal import Decimal, InvalidOperation
from smtplib import SMTPSenderRefused from smtplib import SMTPSenderRefused
from datetime import timedelta from datetime import timedelta
@@ -838,10 +842,9 @@ def process_work_order_invoice_ocr(invoice_id):
@shared_task @shared_task
def build_monthly_archive_cached_task(generated_archive_id): def build_monthly_archive_cached_task(generated_archive_id):
from .models import GeneratedFleetArchive from .models import GeneratedFleetArchive
from .services import NotificationService
from .views import ( from .views import (
_build_monthly_service_tasks_archive_content, _write_monthly_service_tasks_archive_entries,
_build_monthly_work_orders_archive_content, _write_monthly_work_orders_archive_entries,
_notify_monthly_archive_request, _notify_monthly_archive_request,
) )
@@ -854,25 +857,39 @@ def build_monthly_archive_cached_task(generated_archive_id):
if generated is None: if generated is None:
return {"status": "failed", "error": "Generated archive record not found"} return {"status": "failed", "error": "Generated archive record not found"}
tmp_dir = str(getattr(settings, 'APP_TMP_CLEANUP_DIR', '/tmp'))
with tempfile.NamedTemporaryFile(
mode='w+b',
delete=False,
dir=tmp_dir if os.path.isdir(tmp_dir) else None,
prefix='erp-fleet-archive-',
suffix='.zip',
) as temp_zip:
temp_zip_path = temp_zip.name
try: try:
with zipfile.ZipFile(temp_zip_path, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
if generated.archive_type == 'service_tasks': if generated.archive_type == 'service_tasks':
archive_content = _build_monthly_service_tasks_archive_content( entries_written = _write_monthly_service_tasks_archive_entries(
archive,
user=generated.requested_by, user=generated.requested_by,
year=generated.year, year=generated.year,
month=generated.month, month=generated.month,
) )
else: else:
archive_content = _build_monthly_work_orders_archive_content( entries_written = _write_monthly_work_orders_archive_entries(
archive,
user=generated.requested_by, user=generated.requested_by,
year=generated.year, year=generated.year,
month=generated.month, month=generated.month,
) )
if entries_written == 0:
if not archive_content:
raise ValueError('ZIP arhiva je prazna.') raise ValueError('ZIP arhiva je prazna.')
if os.path.getsize(temp_zip_path) <= 0:
raise ValueError('ZIP arhiva je prazna.')
filename = generated.filename or f"{generated.archive_type}-{generated.year}-{generated.month}.zip" filename = generated.filename or f"{generated.archive_type}-{generated.year}-{generated.month}.zip"
generated.file.save(filename, ContentFile(archive_content), save=False) with open(temp_zip_path, 'rb') as temp_file:
generated.file.save(filename, File(temp_file), save=False)
generated.status = 'ready' generated.status = 'ready'
generated.error_message = '' generated.error_message = ''
generated.save(update_fields=['file', 'status', 'error_message', 'updated_at']) generated.save(update_fields=['file', 'status', 'error_message', 'updated_at'])
@@ -902,6 +919,12 @@ def build_monthly_archive_cached_task(generated_archive_id):
) )
logger.exception("Greška kod build_monthly_archive_cached_task: %s", exc) logger.exception("Greška kod build_monthly_archive_cached_task: %s", exc)
return {"status": "failed", "error": str(exc)} return {"status": "failed", "error": str(exc)}
finally:
try:
if os.path.exists(temp_zip_path):
os.remove(temp_zip_path)
except OSError:
logger.warning("Ne mogu obrisati privremenu ZIP datoteku: %s", temp_zip_path)
@shared_task @shared_task
@@ -944,6 +967,46 @@ def cleanup_expired_generated_pdfs_task():
return {"deleted": deleted} return {"deleted": deleted}
@shared_task
def cleanup_tmp_files_task():
tmp_dir = str(getattr(settings, 'APP_TMP_CLEANUP_DIR', '/tmp'))
max_age_hours = int(getattr(settings, 'APP_TMP_CLEANUP_MAX_AGE_HOURS', 24))
prefixes = tuple(getattr(settings, 'APP_TMP_CLEANUP_PREFIXES', ['erp-fleet-archive-']))
if max_age_hours <= 0:
raise ValueError('APP_TMP_CLEANUP_MAX_AGE_HOURS mora biti > 0.')
if not prefixes:
raise ValueError('APP_TMP_CLEANUP_PREFIXES ne smije biti prazan.')
if not os.path.isdir(tmp_dir):
return {'deleted': 0, 'tmp_dir': tmp_dir, 'reason': 'tmp-dir-not-found'}
now = timezone.now().timestamp()
cutoff = now - (max_age_hours * 3600)
deleted = 0
for name in os.listdir(tmp_dir):
if not any(name.startswith(prefix) for prefix in prefixes):
continue
full_path = os.path.join(tmp_dir, name)
try:
stat_info = os.stat(full_path)
except FileNotFoundError:
continue
if stat_info.st_mtime > cutoff:
continue
if os.path.isdir(full_path):
try:
os.rmdir(full_path)
except OSError:
logger.warning("Preskačem %s: direktorij nije prazan ili je nedostupan.", full_path)
continue
else:
os.remove(full_path)
deleted += 1
return {'deleted': deleted, 'tmp_dir': tmp_dir, 'max_age_hours': max_age_hours}
@shared_task @shared_task
def build_work_order_pdf_cached_task(generated_pdf_id): def build_work_order_pdf_cached_task(generated_pdf_id):
from .views import _build_work_order_pdf, _build_work_order_service_records_pdf from .views import _build_work_order_pdf, _build_work_order_service_records_pdf

View File

@@ -29,6 +29,7 @@ from modules.fleet.tasks import build_monthly_archive_cached_task
from modules.fleet.views import ( from modules.fleet.views import (
_build_monthly_servicer_report_rows, _build_monthly_servicer_report_rows,
_calculate_daily_quantity_from_hours, _calculate_daily_quantity_from_hours,
_compress_image_for_docx,
_work_order_related_tasks_queryset, _work_order_related_tasks_queryset,
) )
@@ -240,6 +241,31 @@ class WorkOrderImagesEndpointTests(TestCase):
self.assertIn('additional_costs_table', payload) self.assertIn('additional_costs_table', payload)
self.assertEqual(payload['additional_costs_table']['total_for_payout'], '5.00') self.assertEqual(payload['additional_costs_table']['total_for_payout'], '5.00')
def test_compress_image_for_docx_keeps_standard_phone_photos(self):
file = BytesIO()
Image.new('RGB', (4000, 3000), color='blue').save(file, format='JPEG', quality=85)
file.seek(0)
uploaded = SimpleUploadedFile('phone-12mp.jpg', file.getvalue(), content_type='image/jpeg')
result = _compress_image_for_docx(uploaded)
self.assertIsNotNone(result)
self.assertGreater(len(result.getvalue()), 0)
def test_compress_image_for_docx_handles_exif_orientation(self):
exif = Image.Exif()
exif[0x0112] = 6
file = BytesIO()
Image.new('RGB', (3000, 2000), color='green').save(file, format='JPEG', quality=85, exif=exif.tobytes())
file.seek(0)
uploaded = SimpleUploadedFile('rotated-phone.jpg', file.getvalue(), content_type='image/jpeg')
result = _compress_image_for_docx(uploaded)
self.assertIsNotNone(result)
self.assertGreater(len(result.getvalue()), 0)
def test_work_order_related_tasks_queryset_excludes_other_work_orders_for_same_vehicle(self): def test_work_order_related_tasks_queryset_excludes_other_work_orders_for_same_vehicle(self):
other_work_order = WorkOrder.objects.create( other_work_order = WorkOrder.objects.create(
vehicle=self.vehicle, vehicle=self.vehicle,

View File

@@ -14,7 +14,7 @@ from pathlib import Path
from decimal import Decimal, InvalidOperation from decimal import Decimal, InvalidOperation
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from PIL import Image, UnidentifiedImageError from PIL import Image, ImageOps, UnidentifiedImageError
from django.conf import settings from django.conf import settings
from django.core.mail import EmailMessage from django.core.mail import EmailMessage
from django.core.exceptions import ValidationError as DjangoValidationError from django.core.exceptions import ValidationError as DjangoValidationError
@@ -464,24 +464,32 @@ def _resolve_service_report_tasks(work_order, task_id):
return [selected_task], selected_task return [selected_task], selected_task
def _compress_image_for_pdf(image_field, max_width=1280, quality=75): _COMPRESS_IMAGE_MAX_MEGAPIXELS = 30 # preskači tek vrlo velike slike (>30 MP); standardni telefoni 1220 MP ostaju uključeni
def _compress_image_for_pdf(image_field, max_width=800, quality=75):
""" """
Otvori image_field (Django FileField), kompresiraj na max_width JPEG u memoriji, Otvori image_field (Django FileField), kompresiraj na max_width JPEG u memoriji,
vrati ImageReader spreman za reportlab. Vraća None ako slika nije dostupna. vrati ImageReader spreman za reportlab. Vraća None ako slika nije dostupna.
BILINEAR umjesto LANCZOS: višestruko brže za velike slike (izbjeći Gunicorn timeout).
Megapixel guard čita samo header i preskače slike > 8 MP bez dekodiranja piksela.
except BaseException hvata i SystemExit koji Gunicorn diže na SIGABRT (worker timeout).
""" """
try: try:
image_field.open('rb') image_field.open('rb')
with Image.open(image_field) as src: with Image.open(image_field) as img:
img = src.convert('RGB') img = ImageOps.exif_transpose(img)
if img.width > max_width: w, h = img.size
ratio = max_width / float(img.width) if w * h > _COMPRESS_IMAGE_MAX_MEGAPIXELS * 1_000_000:
new_h = max(1, int(img.height * ratio)) return None
img = img.resize((max_width, new_h), Image.LANCZOS) img.thumbnail((max_width, max_width * 2), Image.BILINEAR)
rgb = img.convert('RGB')
buf = BytesIO() buf = BytesIO()
img.save(buf, format='JPEG', quality=quality, optimize=True) rgb.save(buf, format='JPEG', quality=quality)
buf.seek(0) buf.seek(0)
return ImageReader(buf) return ImageReader(buf)
except Exception: except BaseException:
return None return None
finally: finally:
try: try:
@@ -490,23 +498,28 @@ def _compress_image_for_pdf(image_field, max_width=1280, quality=75):
pass pass
def _compress_image_for_docx(image_field, max_width=1600, quality=80): def _compress_image_for_docx(image_field, max_width=800, quality=80):
""" """
Pripremi sliku za python-docx kao JPEG stream razumne veličine. Pripremi sliku za python-docx kao JPEG stream razumne veličine.
BILINEAR umjesto LANCZOS: višestruko brže za velike slike (izbjeći Gunicorn timeout).
Megapixel guard čita samo header i preskače slike > 8 MP bez dekodiranja piksela.
except BaseException hvata i SystemExit koji Gunicorn diže na SIGABRT (worker timeout).
""" """
try: try:
image_field.open('rb') image_field.open('rb')
with Image.open(image_field) as src: with Image.open(image_field) as img:
img = src.convert('RGB') img = ImageOps.exif_transpose(img)
if img.width > max_width: w, h = img.size
ratio = max_width / float(img.width) if w * h > _COMPRESS_IMAGE_MAX_MEGAPIXELS * 1_000_000:
new_h = max(1, int(img.height * ratio)) return None
img = img.resize((max_width, new_h), Image.LANCZOS) img.thumbnail((max_width, max_width * 2), Image.BILINEAR)
rgb = img.convert('RGB')
buf = BytesIO() buf = BytesIO()
img.save(buf, format='JPEG', quality=quality, optimize=True) rgb.save(buf, format='JPEG', quality=quality)
buf.seek(0) buf.seek(0)
return buf return buf
except Exception: except BaseException:
return None return None
finally: finally:
try: try:
@@ -2437,6 +2450,26 @@ def _file_attachment(file_field, fallback_name):
return (filename, content, _guess_content_type(filename)) return (filename, content, _guess_content_type(filename))
def _write_file_field_to_zip(archive, *, file_field, entry_name, chunk_size=64 * 1024):
if not file_field:
return False
file_field.open('rb')
try:
first_chunk = file_field.read(chunk_size)
if not first_chunk:
return False
with archive.open(entry_name, mode='w') as destination:
destination.write(first_chunk)
while True:
chunk = file_field.read(chunk_size)
if not chunk:
break
destination.write(chunk)
finally:
file_field.close()
return True
def _build_image_attachments_for_work_order(work_order): def _build_image_attachments_for_work_order(work_order):
attachments = [] attachments = []
photos = WorkOrderPhoto.objects.filter(is_active=True, work_order=work_order).order_by('created_at') photos = WorkOrderPhoto.objects.filter(is_active=True, work_order=work_order).order_by('created_at')
@@ -2995,7 +3028,7 @@ def _parse_year_month_params(request):
return year, month return year, month
def _build_monthly_service_tasks_archive_content(*, user, year, month): def _write_monthly_service_tasks_archive_entries(archive, *, user, year, month):
from modules.task_management.models import Task from modules.task_management.models import Task
tasks = list( tasks = list(
@@ -3015,8 +3048,7 @@ def _build_monthly_service_tasks_archive_content(*, user, year, month):
raise DRFValidationError({'detail': 'Nema servisnih taskova za odabrani mjesec.'}) raise DRFValidationError({'detail': 'Nema servisnih taskova za odabrani mjesec.'})
used_names = set() used_names = set()
archive_buffer = BytesIO() entries_written = 0
with zipfile.ZipFile(archive_buffer, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
for task in tasks: for task in tasks:
work_order = task.work_order work_order = task.work_order
if work_order is None: if work_order is None:
@@ -3025,14 +3057,28 @@ def _build_monthly_service_tasks_archive_content(*, user, year, month):
base_name = _service_records_docx_filename(work_order, task) base_name = _service_records_docx_filename(work_order, task)
entry_name = _unique_zip_entry_name(base_name, used_names) entry_name = _unique_zip_entry_name(base_name, used_names)
archive.writestr(entry_name, docx_bytes) archive.writestr(entry_name, docx_bytes)
entries_written += 1
if entries_written == 0:
raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'})
return entries_written
def _build_monthly_service_tasks_archive_content(*, user, year, month):
archive_buffer = BytesIO()
with zipfile.ZipFile(archive_buffer, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
_write_monthly_service_tasks_archive_entries(
archive,
user=user,
year=year,
month=month,
)
archive_content = archive_buffer.getvalue() archive_content = archive_buffer.getvalue()
if not archive_content: if not archive_content:
raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'}) raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'})
return archive_content return archive_content
def _build_monthly_work_orders_archive_content(*, user, year, month): def _write_monthly_work_orders_archive_entries(archive, *, user, year, month):
from modules.task_management.models import Task from modules.task_management.models import Task
monthly_tasks = ( monthly_tasks = (
@@ -3071,23 +3117,34 @@ def _build_monthly_work_orders_archive_content(*, user, year, month):
raise DRFValidationError({'detail': 'Nema putnih naloga za odabrani mjesec.'}) raise DRFValidationError({'detail': 'Nema putnih naloga za odabrani mjesec.'})
used_names = set() used_names = set()
archive_buffer = BytesIO() entries_written = 0
with zipfile.ZipFile(archive_buffer, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
for work_order in work_orders: for work_order in work_orders:
pdf_bytes = _build_work_order_pdf(work_order) pdf_bytes = _build_work_order_pdf(work_order)
work_order_pdf_name = _unique_zip_entry_name(_pdf_filename(work_order, 'work_order'), used_names) work_order_pdf_name = _unique_zip_entry_name(_pdf_filename(work_order, 'work_order'), used_names)
archive.writestr(work_order_pdf_name, pdf_bytes) archive.writestr(work_order_pdf_name, pdf_bytes)
entries_written += 1
display_code = _work_order_display_code(work_order) display_code = _work_order_display_code(work_order)
invoices = work_order.invoices.filter(is_active=True).order_by('datum', 'created_at') invoices = work_order.invoices.filter(is_active=True).order_by('datum', 'created_at')
for index, invoice in enumerate(invoices, start=1): for index, invoice in enumerate(invoices, start=1):
attachment = _file_attachment(invoice.image, fallback_name=f"invoice-{index}.bin") file_name = Path(str(getattr(getattr(invoice, 'image', None), 'name', '') or f"invoice-{index}.bin")).name
if not attachment: archive_path = _unique_zip_entry_name(f"Racuni/{display_code}/{file_name}", used_names)
continue if _write_file_field_to_zip(archive, file_field=invoice.image, entry_name=archive_path):
invoice_filename, content, _content_type = attachment entries_written += 1
archive_path = f"Racuni/{display_code}/{invoice_filename}" if entries_written == 0:
archive.writestr(_unique_zip_entry_name(archive_path, used_names), content) raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'})
return entries_written
def _build_monthly_work_orders_archive_content(*, user, year, month):
archive_buffer = BytesIO()
with zipfile.ZipFile(archive_buffer, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
_write_monthly_work_orders_archive_entries(
archive,
user=user,
year=year,
month=month,
)
archive_content = archive_buffer.getvalue() archive_content = archive_buffer.getvalue()
if not archive_content: if not archive_content:
raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'}) raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'})
@@ -3398,12 +3455,18 @@ def generated_archive_download(request, archive_id):
if generated_archive is None: if generated_archive is None:
raise DRFValidationError({'detail': 'ZIP arhiva nije dostupna ili je istekla.'}) raise DRFValidationError({'detail': 'ZIP arhiva nije dostupna ili je istekla.'})
generated_archive.file.open('rb') file_name = getattr(generated_archive.file, 'name', '')
storage = getattr(generated_archive.file, 'storage', None)
if not file_name or storage is None:
raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'})
try: try:
archive_bytes = generated_archive.file.read() if not storage.exists(file_name):
finally: raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'})
generated_archive.file.close() except (FileNotFoundError, OSError, ValueError):
if not archive_bytes: raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'})
try:
generated_archive.file.open('rb')
except (FileNotFoundError, OSError, ValueError):
raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'}) raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'})
filename = generated_archive.filename or _generated_archive_filename_for_user( filename = generated_archive.filename or _generated_archive_filename_for_user(
request.user, request.user,
@@ -3411,10 +3474,13 @@ def generated_archive_download(request, archive_id):
month=generated_archive.month, month=generated_archive.month,
archive_type=generated_archive.archive_type, archive_type=generated_archive.archive_type,
) )
response = HttpResponse(archive_bytes, content_type='application/zip') response = FileResponse(generated_archive.file, content_type='application/zip')
response['Content-Disposition'] = f'attachment; filename="{filename}"' response['Content-Disposition'] = f'attachment; filename="{filename}"'
response['Cache-Control'] = 'private, max-age=3600' response['Cache-Control'] = 'private, max-age=3600'
response['Content-Length'] = str(len(archive_bytes)) try:
response['Content-Length'] = str(generated_archive.file.size)
except (OSError, ValueError, TypeError):
pass
return response return response

View File

@@ -0,0 +1,24 @@
#!/bin/sh
set -eu
TMP_DIR="${APP_TMP_CLEANUP_DIR:-/tmp}"
MAX_AGE_DAYS="${APP_TMP_CLEANUP_MAX_AGE_DAYS:-1}"
PREFIXES="${APP_TMP_CLEANUP_PREFIXES:-erp-fleet-archive-}"
if [ ! -d "$TMP_DIR" ]; then
echo "TMP dir ne postoji: $TMP_DIR"
exit 0
fi
OLD_IFS="$IFS"
IFS=','
for prefix in $PREFIXES; do
prefix_trimmed="$(echo "$prefix" | xargs)"
if [ -z "$prefix_trimmed" ]; then
continue
fi
find "$TMP_DIR" -maxdepth 1 -mindepth 1 -name "${prefix_trimmed}*" -mtime "+${MAX_AGE_DAYS}" -print -delete
done
IFS="$OLD_IFS"
echo "TMP cleanup dovršen za $TMP_DIR (older than ${MAX_AGE_DAYS} day(s))."

View File

@@ -37,7 +37,13 @@ services:
environment: environment:
DEBUG: "False" DEBUG: "False"
DJANGO_SETTINGS_MODULE: core.settings.production DJANGO_SETTINGS_MODULE: core.settings.production
command: celery -A core worker --loglevel=info CELERY_WORKER_PREFETCH_MULTIPLIER: "1"
CELERY_WORKER_MAX_TASKS_PER_CHILD: "20"
CELERY_WORKER_MAX_MEMORY_PER_CHILD: "350000"
APP_TMP_CLEANUP_DIR: "/tmp"
APP_TMP_CLEANUP_MAX_AGE_HOURS: "24"
APP_TMP_CLEANUP_PREFIXES: "erp-fleet-archive-"
command: celery -A core worker --loglevel=info --concurrency=${CELERY_WORKER_CONCURRENCY:-2}
volumes: volumes:
- media_volume:/app/media - media_volume:/app/media
depends_on: depends_on: