8 Commits

Author SHA1 Message Date
mariomitte
61bc772054 fix: reduce memory usage for monthly ZIP archive flow
Replace generated archive download response with FileResponse streaming so ZIP files are not fully loaded into process memory.

Move monthly archive task output to a temporary file and upload that file to storage, and write invoice attachments into ZIP archives in chunks.

Add Celery memory guard settings, a memory-aware base task hook, and periodic /tmp cleanup for prefixed archive files older than one day.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-05 08:15:30 +02:00
mariomitte
b985c285f9 fix: apply EXIF orientation to service record images in PDF/DOCX
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Smartphone photos often have EXIF orientation metadata (tags 6, 8, 3)
that rotates the display but doesn't transform the pixel data. When
PDFs/DOCX embedded images without applying this metadata, they appear
rotated 90/180/270 degrees.

Use ImageOps.exif_transpose(img) before resize/convert in both
_compress_image_for_pdf() and _compress_image_for_docx() to read EXIF
orientation and transpose the actual pixel data accordingly. This is
a standard Pillow function and is a no-op for images without EXIF.

Add regression test test_compress_image_for_docx_handles_exif_orientation
to verify images with EXIF tag 0x0112 (orientation=6) are transposed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 11:08:16 +02:00
mariomitte
aa690d4ac4 fix: increase image megapixel limit from 8 to 30 MP to include standard phone photos
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Previous limit of 8 MP was rejecting standard smartphone photos (12-20 MP),
causing service record PDFs/DOCX documents to be generated without photos.

This was a regression from the image timeout fix: the megapixel guard was
designed to prevent processing of pathologically large files (preventing
worker timeouts), but the threshold was set too aggressively.

Increase limit to 30 MP to allow standard device cameras while still
rejecting extreme outliers that would cause timeout/OOM.

Add regression test test_compress_image_for_docx_keeps_standard_phone_photos
to verify 12 MP photos are accepted.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 10:54:40 +02:00
mariomitte
52408bc5a9 fix: BILINEAR+BaseException catch to fix image worker timeout in PDF/DOCX
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Root cause was WORKER TIMEOUT, not OOM:
- Gunicorn sends SIGABRT to workers that exceed 60s timeout
- SIGABRT handler calls sys.exit(1) raising SystemExit(BaseException)
- except Exception does NOT catch SystemExit, so the worker crashes

Two fixes:
1. except BaseException — catches SystemExit so the worker survives
   and gracefully skips the image (returns None) instead of dying
2. Image.BILINEAR instead of LANCZOS — orders of magnitude faster
   for large images, ensures processing completes well within 60s
3. Reduce max_width 1600->800, megapixel limit 20->8 MP, optimize=True removed
   (these reduce processing time further)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 10:43:27 +02:00
mariomitte
8e9d23fcfa fix: thumbnail-first + megapixel guard to prevent image OOM in PDF/DOCX
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Previous draft()+convert()+thumbnail() ordering still caused OOM for
non-JPEG formats (PNG/HEIC) because draft() is a no-op for those formats,
and convert('RGB') forces a full pixel decode regardless.

Replace with thumbnail()-first ordering:
- thumbnail() internally calls draft() for JPEG before decoding
- thumbnail() performs in-place resize without allocating a second
  full-resolution buffer
- convert('RGB') is then called on the already-small image (safe for
  any format)

Add _COMPRESS_IMAGE_MAX_MEGAPIXELS guard (20 MP): read image dimensions
from headers only (no pixel decode) and return None for images that
exceed the limit. This prevents OOM even for pathologically large files
where draft() provides no benefit (e.g. PNG, TIFF, HEIC).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 10:10:01 +02:00
mariomitte
a7220d27f5 fix: use draft()+thumbnail() to prevent OOM kill on large image resize
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
PIL img.resize() on high-resolution photos was exhausting worker RAM,
causing Gunicorn to SIGKILL the worker mid-request (production OOM crash).

Replace the explicit resize() path in both _compress_image_for_pdf() and
_compress_image_for_docx() with:
  - Image.draft() — hints the JPEG decoder to decode at a lower resolution
  - Image.thumbnail() — in-place resize that avoids allocating a second
    full-resolution buffer

This keeps peak memory proportional to the output size instead of the
original file size, preventing the worker from being killed when
processing multi-megapixel service-record photos.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 09:51:21 +02:00
mariomitte
1e68c663f2 fix: scope inferred work-order tasks to the current work order
Some checks failed
ERP CI/CD Pipeline / test (push) Has been cancelled
ERP CI/CD Pipeline / Deploy (server git pull + compose) (push) Has been cancelled
Prevent service-record inference from pulling tasks from other work orders
that share the same vehicle. This keeps task-service-context, service
reports, and related work-order widgets isolated to the selected nalog.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 08:32:41 +02:00
mariomitte
2826ee9bab fix: invalidate stale service records PDF cache on service record changes
Ensure admin/API deletes and updates of service records, photos, and
attachments invalidate the cached service-records PDF. Also harden cached
PDF lookup/download paths so a missing file is treated as stale cache
instead of a 500.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-31 07:32:05 +02:00
9 changed files with 503 additions and 77 deletions

View File

@@ -7,7 +7,7 @@ Nakon što je gitea webhook i testirana skripta za automatsko povlačenje promje
```bash
cd /opt/erp/app
git fetch origin
git reset --hard origin/main
git reset --hard origin/main ili git pull --ff-only origin main
# rebuild backend + worker + beat + frontend
docker compose -f docker-compose.yml -f docker-compose.prod.yml build backend worker beat frontend

View File

@@ -1,7 +1,12 @@
# backend/core/celery.py
import os
import gc
import logging
from celery import Celery
from celery import Task
logger = logging.getLogger(__name__)
# Postavi Django settings modul
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings')
@@ -11,5 +16,42 @@ app = Celery('core')
# Koristi konfiguraciju iz settings.py s prefiksom 'CELERY_'
app.config_from_object('django.conf:settings', namespace='CELERY')
def _rss_memory_mb():
try:
import resource
usage_kb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
# Na Linuxu je ru_maxrss u KiB.
return round(float(usage_kb) / 1024.0, 2)
except (ImportError, AttributeError, OSError, ValueError):
return None
class MemoryAwareTask(Task):
abstract = True
def __call__(self, *args, **kwargs):
gc.collect()
return super().__call__(*args, **kwargs)
def after_return(self, status, retval, task_id, args, kwargs, einfo):
memory_before_gc = _rss_memory_mb()
gc.collect()
memory_after_gc = _rss_memory_mb()
if memory_before_gc is not None and memory_after_gc is not None:
logger.info(
"Task %s (%s) status=%s RSS prije/poslije GC: %.2fMB -> %.2fMB",
self.name,
task_id,
status,
memory_before_gc,
memory_after_gc,
)
return super().after_return(status, retval, task_id, args, kwargs, einfo)
app.Task = MemoryAwareTask
# Automatski pronalazi taskove u svim 'modules.*'
app.autodiscover_tasks()

View File

@@ -154,6 +154,11 @@ REDIS_URL = os.environ.get('REDIS_URL', 'redis://localhost:6379/0')
CELERY_BROKER_URL = REDIS_URL
CELERY_RESULT_BACKEND = REDIS_URL
CELERY_WORKER_PREFETCH_MULTIPLIER = int(os.environ.get('CELERY_WORKER_PREFETCH_MULTIPLIER', '1'))
CELERY_WORKER_MAX_TASKS_PER_CHILD = int(os.environ.get('CELERY_WORKER_MAX_TASKS_PER_CHILD', '20'))
CELERY_WORKER_MAX_MEMORY_PER_CHILD = int(os.environ.get('CELERY_WORKER_MAX_MEMORY_PER_CHILD', '350000'))
CELERY_TASK_ACKS_LATE = True
CELERY_TASK_REJECT_ON_WORKER_LOST = True
# Ako želiš koristiti Redis kao brzi cache sustav unutar Djanga (izvrsno za ERP performanse)
# Za ovo ti je potreban paket 'django-redis' u requirements.txt
@@ -171,10 +176,18 @@ CACHES = {
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_BEAT_SCHEDULE = {
'cleanup-expired-generated-archives-hourly': {
'task': 'modules.fleet.tasks.cleanup_expired_generated_archives_task',
'schedule': crontab(minute=5),
},
'cleanup-expired-generated-pdfs-hourly': {
'task': 'modules.fleet.tasks.cleanup_expired_generated_pdfs_task',
'schedule': crontab(minute=0),
},
'cleanup-app-tmp-daily': {
'task': 'modules.fleet.tasks.cleanup_tmp_files_task',
'schedule': crontab(hour=3, minute=15),
},
'notify-upcoming-tasks-daily': {
'task': 'modules.task_management.tasks.notify_upcoming_tasks',
'schedule': crontab(hour=8, minute=0),
@@ -190,6 +203,13 @@ STATIC_URL = '/static/'
# (Opcionalno, ali preporučeno za ERP) Ako koristiš i medije (dokumente)
MEDIA_URL = '/media/'
MEDIA_ROOT = BASE_DIR / 'media'
APP_TMP_CLEANUP_DIR = os.environ.get('APP_TMP_CLEANUP_DIR', '/tmp')
APP_TMP_CLEANUP_MAX_AGE_HOURS = int(os.environ.get('APP_TMP_CLEANUP_MAX_AGE_HOURS', '24'))
APP_TMP_CLEANUP_PREFIXES = [
prefix.strip()
for prefix in os.environ.get('APP_TMP_CLEANUP_PREFIXES', 'erp-fleet-archive-').split(',')
if prefix.strip()
]
# ==============================================================================

View File

@@ -2,10 +2,12 @@ import uuid
import re
from datetime import time
from django.db import models
from django.db.models.signals import post_save, pre_delete
from django.conf import settings
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _
from django.utils import timezone
from django.dispatch import receiver
from core.base_models import BaseModel
from decimal import Decimal
@@ -142,6 +144,67 @@ class VehicleServiceAttachment(BaseModel):
def __str__(self):
return f"Attachment {self.pk} for {self.service_record} ({self.file.name if self.file else 'no-file'})"
def _service_record_work_order_id(instance):
if isinstance(instance, VehicleServiceRecord):
task = getattr(instance, 'task', None)
if task and getattr(task, 'work_order_id', None):
return task.work_order_id
if getattr(instance, 'task_id', None):
return VehicleServiceRecord.objects.filter(pk=instance.pk).values_list('task__work_order_id', flat=True).first()
return None
service_record = getattr(instance, 'service_record', None)
if service_record and getattr(service_record, 'task', None) and getattr(service_record.task, 'work_order_id', None):
return service_record.task.work_order_id
service_record_id = getattr(instance, 'service_record_id', None)
if not service_record_id:
return None
return VehicleServiceRecord.objects.filter(pk=service_record_id).values_list('task__work_order_id', flat=True).first()
def _invalidate_service_records_pdf_cache(work_order_id):
if not work_order_id:
return
cached_pdfs = GeneratedWorkOrderPdf.objects.filter(
is_active=True,
work_order_id=work_order_id,
pdf_type='service_records',
)
for cached in cached_pdfs:
file_name = getattr(cached.file, 'name', '')
if file_name:
try:
if cached.file.storage.exists(file_name):
cached.file.delete(save=False)
except (FileNotFoundError, OSError, ValueError):
pass
cached.file = None
cached.is_active = False
cached.status = 'failed'
cached.error_message = 'PDF cache invalidiran zbog promjene servisnih zapisa.'
cached.save(update_fields=['file', 'is_active', 'status', 'error_message', 'updated_at'])
@receiver(post_save, sender='fleet.VehicleServiceRecord')
@receiver(pre_delete, sender='fleet.VehicleServiceRecord')
def _invalidate_service_records_pdf_for_service_record(sender, instance, **kwargs):
_invalidate_service_records_pdf_cache(_service_record_work_order_id(instance))
@receiver(post_save, sender='fleet.VehicleServicePhoto')
@receiver(pre_delete, sender='fleet.VehicleServicePhoto')
def _invalidate_service_records_pdf_for_service_photo(sender, instance, **kwargs):
_invalidate_service_records_pdf_cache(_service_record_work_order_id(instance))
@receiver(post_save, sender='fleet.VehicleServiceAttachment')
@receiver(pre_delete, sender='fleet.VehicleServiceAttachment')
def _invalidate_service_records_pdf_for_service_attachment(sender, instance, **kwargs):
_invalidate_service_records_pdf_cache(_service_record_work_order_id(instance))
class Vehicle(BaseModel):
ASSET_TYPE_CHOICES = [
('vehicle', _("Vozilo")),

View File

@@ -5,6 +5,7 @@ from django.core.mail import EmailMessage
from django.conf import settings
from django.utils import timezone
from django.core.files.base import ContentFile
from django.core.files import File
import logging
from io import BytesIO
from io import StringIO
@@ -13,6 +14,9 @@ import base64
import re
import csv
import mimetypes
import os
import tempfile
import zipfile
from decimal import Decimal, InvalidOperation
from smtplib import SMTPSenderRefused
from datetime import timedelta
@@ -838,10 +842,9 @@ def process_work_order_invoice_ocr(invoice_id):
@shared_task
def build_monthly_archive_cached_task(generated_archive_id):
from .models import GeneratedFleetArchive
from .services import NotificationService
from .views import (
_build_monthly_service_tasks_archive_content,
_build_monthly_work_orders_archive_content,
_write_monthly_service_tasks_archive_entries,
_write_monthly_work_orders_archive_entries,
_notify_monthly_archive_request,
)
@@ -854,25 +857,39 @@ def build_monthly_archive_cached_task(generated_archive_id):
if generated is None:
return {"status": "failed", "error": "Generated archive record not found"}
tmp_dir = str(getattr(settings, 'APP_TMP_CLEANUP_DIR', '/tmp'))
with tempfile.NamedTemporaryFile(
mode='w+b',
delete=False,
dir=tmp_dir if os.path.isdir(tmp_dir) else None,
prefix='erp-fleet-archive-',
suffix='.zip',
) as temp_zip:
temp_zip_path = temp_zip.name
try:
with zipfile.ZipFile(temp_zip_path, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
if generated.archive_type == 'service_tasks':
archive_content = _build_monthly_service_tasks_archive_content(
entries_written = _write_monthly_service_tasks_archive_entries(
archive,
user=generated.requested_by,
year=generated.year,
month=generated.month,
)
else:
archive_content = _build_monthly_work_orders_archive_content(
entries_written = _write_monthly_work_orders_archive_entries(
archive,
user=generated.requested_by,
year=generated.year,
month=generated.month,
)
if not archive_content:
if entries_written == 0:
raise ValueError('ZIP arhiva je prazna.')
if os.path.getsize(temp_zip_path) <= 0:
raise ValueError('ZIP arhiva je prazna.')
filename = generated.filename or f"{generated.archive_type}-{generated.year}-{generated.month}.zip"
generated.file.save(filename, ContentFile(archive_content), save=False)
with open(temp_zip_path, 'rb') as temp_file:
generated.file.save(filename, File(temp_file), save=False)
generated.status = 'ready'
generated.error_message = ''
generated.save(update_fields=['file', 'status', 'error_message', 'updated_at'])
@@ -902,6 +919,12 @@ def build_monthly_archive_cached_task(generated_archive_id):
)
logger.exception("Greška kod build_monthly_archive_cached_task: %s", exc)
return {"status": "failed", "error": str(exc)}
finally:
try:
if os.path.exists(temp_zip_path):
os.remove(temp_zip_path)
except OSError:
logger.warning("Ne mogu obrisati privremenu ZIP datoteku: %s", temp_zip_path)
@shared_task
@@ -944,6 +967,46 @@ def cleanup_expired_generated_pdfs_task():
return {"deleted": deleted}
@shared_task
def cleanup_tmp_files_task():
tmp_dir = str(getattr(settings, 'APP_TMP_CLEANUP_DIR', '/tmp'))
max_age_hours = int(getattr(settings, 'APP_TMP_CLEANUP_MAX_AGE_HOURS', 24))
prefixes = tuple(getattr(settings, 'APP_TMP_CLEANUP_PREFIXES', ['erp-fleet-archive-']))
if max_age_hours <= 0:
raise ValueError('APP_TMP_CLEANUP_MAX_AGE_HOURS mora biti > 0.')
if not prefixes:
raise ValueError('APP_TMP_CLEANUP_PREFIXES ne smije biti prazan.')
if not os.path.isdir(tmp_dir):
return {'deleted': 0, 'tmp_dir': tmp_dir, 'reason': 'tmp-dir-not-found'}
now = timezone.now().timestamp()
cutoff = now - (max_age_hours * 3600)
deleted = 0
for name in os.listdir(tmp_dir):
if not any(name.startswith(prefix) for prefix in prefixes):
continue
full_path = os.path.join(tmp_dir, name)
try:
stat_info = os.stat(full_path)
except FileNotFoundError:
continue
if stat_info.st_mtime > cutoff:
continue
if os.path.isdir(full_path):
try:
os.rmdir(full_path)
except OSError:
logger.warning("Preskačem %s: direktorij nije prazan ili je nedostupan.", full_path)
continue
else:
os.remove(full_path)
deleted += 1
return {'deleted': deleted, 'tmp_dir': tmp_dir, 'max_age_hours': max_age_hours}
@shared_task
def build_work_order_pdf_cached_task(generated_pdf_id):
from .views import _build_work_order_pdf, _build_work_order_service_records_pdf

View File

@@ -26,7 +26,12 @@ from modules.fleet.models import (
)
from modules.task_management.models import Task, TaskWorkHoursTable
from modules.fleet.tasks import build_monthly_archive_cached_task
from modules.fleet.views import _build_monthly_servicer_report_rows, _calculate_daily_quantity_from_hours
from modules.fleet.views import (
_build_monthly_servicer_report_rows,
_calculate_daily_quantity_from_hours,
_compress_image_for_docx,
_work_order_related_tasks_queryset,
)
def create_test_image(filename='test.jpg', size=(40, 40), color='red'):
@@ -236,6 +241,58 @@ class WorkOrderImagesEndpointTests(TestCase):
self.assertIn('additional_costs_table', payload)
self.assertEqual(payload['additional_costs_table']['total_for_payout'], '5.00')
def test_compress_image_for_docx_keeps_standard_phone_photos(self):
file = BytesIO()
Image.new('RGB', (4000, 3000), color='blue').save(file, format='JPEG', quality=85)
file.seek(0)
uploaded = SimpleUploadedFile('phone-12mp.jpg', file.getvalue(), content_type='image/jpeg')
result = _compress_image_for_docx(uploaded)
self.assertIsNotNone(result)
self.assertGreater(len(result.getvalue()), 0)
def test_compress_image_for_docx_handles_exif_orientation(self):
exif = Image.Exif()
exif[0x0112] = 6
file = BytesIO()
Image.new('RGB', (3000, 2000), color='green').save(file, format='JPEG', quality=85, exif=exif.tobytes())
file.seek(0)
uploaded = SimpleUploadedFile('rotated-phone.jpg', file.getvalue(), content_type='image/jpeg')
result = _compress_image_for_docx(uploaded)
self.assertIsNotNone(result)
self.assertGreater(len(result.getvalue()), 0)
def test_work_order_related_tasks_queryset_excludes_other_work_orders_for_same_vehicle(self):
other_work_order = WorkOrder.objects.create(
vehicle=self.vehicle,
creator=self.user,
display_code='MT160726',
purpose='kontrola',
)
other_task = Task.objects.create(
title='Drugi nalog isti stroj',
assigned_to=self.user,
vehicle=self.vehicle,
work_order=other_work_order,
scheduled_date=date(2033, 12, 25),
)
VehicleServiceRecord.objects.create(
vehicle=self.vehicle,
task=other_task,
performed_by=self.user,
description='Ne smije se pojaviti u prvom nalogu.',
service_title='Krivi nalog',
)
resolved_ids = list(_work_order_related_tasks_queryset(self.work_order).values_list('id', flat=True))
self.assertIn(self.task.pk, resolved_ids)
self.assertNotIn(other_task.pk, resolved_ids)
def test_invoice_upload_immediately_appears_in_additional_costs_table(self):
create_response = self.client.post(
reverse('work-order-invoice-list'),
@@ -298,6 +355,52 @@ class WorkOrderImagesEndpointTests(TestCase):
self.assertFalse(cached.is_active)
self.assertEqual(cached.status, 'failed')
def test_service_record_delete_invalidates_service_records_pdf_cache(self):
cached = GeneratedWorkOrderPdf.objects.create(
work_order=self.work_order,
requested_by=self.user,
pdf_type='service_records',
status='ready',
filename='MT150726.work-order-service-records.pdf',
expires_at=timezone.now() + timedelta(hours=1),
)
cached.file.save(
'MT150726.work-order-service-records.pdf',
ContentFile(b'%PDF-1.4 cached service records'),
save=True,
)
VehicleServiceRecord.objects.filter(pk=self.service_record.pk).delete()
cached.refresh_from_db()
self.assertFalse(cached.is_active)
self.assertEqual(cached.status, 'failed')
self.assertFalse(cached.file.name)
def test_service_records_pdf_rebuilds_when_cached_file_is_missing(self):
cached = GeneratedWorkOrderPdf.objects.create(
work_order=self.work_order,
requested_by=self.user,
pdf_type='service_records',
status='ready',
filename='MT150726.work-order-service-records.pdf',
expires_at=timezone.now() + timedelta(hours=1),
)
cached.file.save(
'MT150726.work-order-service-records.pdf',
ContentFile(b'%PDF-1.4 cached service records'),
save=True,
)
cached.file.storage.delete(cached.file.name)
response = self.client.get(f"/api/fleet/work-orders/{self.work_order.pk}/service-records-pdf/")
self.assertEqual(response.status_code, 200, response.content)
self.assertEqual(response['Content-Type'], 'application/pdf')
cached.refresh_from_db()
self.assertFalse(cached.is_active)
self.assertEqual(cached.status, 'failed')
def test_service_records_docx_contains_embedded_service_photos(self):
VehicleServicePhoto.objects.create(
service_record=self.service_record,

View File

@@ -14,7 +14,7 @@ from pathlib import Path
from decimal import Decimal, InvalidOperation
from django.contrib.auth import get_user_model
from PIL import Image, UnidentifiedImageError
from PIL import Image, ImageOps, UnidentifiedImageError
from django.conf import settings
from django.core.mail import EmailMessage
from django.core.exceptions import ValidationError as DjangoValidationError
@@ -155,6 +155,7 @@ def _work_order_related_tasks_queryset(work_order):
vehicle_id=work_order.vehicle_id,
task_id__isnull=False,
task__is_active=True,
task__work_order=work_order,
).values_list('task_id', flat=True)
)
task_ids = list({*direct_task_ids, *inferred_task_ids})
@@ -463,24 +464,32 @@ def _resolve_service_report_tasks(work_order, task_id):
return [selected_task], selected_task
def _compress_image_for_pdf(image_field, max_width=1280, quality=75):
_COMPRESS_IMAGE_MAX_MEGAPIXELS = 30 # preskači tek vrlo velike slike (>30 MP); standardni telefoni 1220 MP ostaju uključeni
def _compress_image_for_pdf(image_field, max_width=800, quality=75):
"""
Otvori image_field (Django FileField), kompresiraj na max_width JPEG u memoriji,
vrati ImageReader spreman za reportlab. Vraća None ako slika nije dostupna.
BILINEAR umjesto LANCZOS: višestruko brže za velike slike (izbjeći Gunicorn timeout).
Megapixel guard čita samo header i preskače slike > 8 MP bez dekodiranja piksela.
except BaseException hvata i SystemExit koji Gunicorn diže na SIGABRT (worker timeout).
"""
try:
image_field.open('rb')
with Image.open(image_field) as src:
img = src.convert('RGB')
if img.width > max_width:
ratio = max_width / float(img.width)
new_h = max(1, int(img.height * ratio))
img = img.resize((max_width, new_h), Image.LANCZOS)
with Image.open(image_field) as img:
img = ImageOps.exif_transpose(img)
w, h = img.size
if w * h > _COMPRESS_IMAGE_MAX_MEGAPIXELS * 1_000_000:
return None
img.thumbnail((max_width, max_width * 2), Image.BILINEAR)
rgb = img.convert('RGB')
buf = BytesIO()
img.save(buf, format='JPEG', quality=quality, optimize=True)
rgb.save(buf, format='JPEG', quality=quality)
buf.seek(0)
return ImageReader(buf)
except Exception:
except BaseException:
return None
finally:
try:
@@ -489,23 +498,28 @@ def _compress_image_for_pdf(image_field, max_width=1280, quality=75):
pass
def _compress_image_for_docx(image_field, max_width=1600, quality=80):
def _compress_image_for_docx(image_field, max_width=800, quality=80):
"""
Pripremi sliku za python-docx kao JPEG stream razumne veličine.
BILINEAR umjesto LANCZOS: višestruko brže za velike slike (izbjeći Gunicorn timeout).
Megapixel guard čita samo header i preskače slike > 8 MP bez dekodiranja piksela.
except BaseException hvata i SystemExit koji Gunicorn diže na SIGABRT (worker timeout).
"""
try:
image_field.open('rb')
with Image.open(image_field) as src:
img = src.convert('RGB')
if img.width > max_width:
ratio = max_width / float(img.width)
new_h = max(1, int(img.height * ratio))
img = img.resize((max_width, new_h), Image.LANCZOS)
with Image.open(image_field) as img:
img = ImageOps.exif_transpose(img)
w, h = img.size
if w * h > _COMPRESS_IMAGE_MAX_MEGAPIXELS * 1_000_000:
return None
img.thumbnail((max_width, max_width * 2), Image.BILINEAR)
rgb = img.convert('RGB')
buf = BytesIO()
img.save(buf, format='JPEG', quality=quality, optimize=True)
rgb.save(buf, format='JPEG', quality=quality)
buf.seek(0)
return buf
except Exception:
except BaseException:
return None
finally:
try:
@@ -532,10 +546,30 @@ def _get_cached_pdf(work_order, pdf_type):
)
for candidate in candidates:
if str(candidate.filename or '').strip() == expected_filename:
file_name = getattr(candidate.file, 'name', '')
storage = getattr(candidate.file, 'storage', None)
if file_name and storage:
try:
if storage.exists(file_name):
return candidate
except (FileNotFoundError, OSError, ValueError):
pass
candidate.file = None
candidate.is_active = False
candidate.status = 'failed'
candidate.error_message = 'PDF datoteka nije dostupna.'
candidate.save(update_fields=['file', 'is_active', 'status', 'error_message', 'updated_at'])
return None
def _mark_generated_pdf_failed(generated_pdf, error_message):
generated_pdf.file = None
generated_pdf.is_active = False
generated_pdf.status = 'failed'
generated_pdf.error_message = error_message
generated_pdf.save(update_fields=['file', 'is_active', 'status', 'error_message', 'updated_at'])
def _cleanup_expired_generated_pdfs():
now = timezone.now()
expired = GeneratedWorkOrderPdf.objects.filter(
@@ -546,10 +580,11 @@ def _cleanup_expired_generated_pdfs():
for item in expired:
if item.file:
item.file.delete(save=False)
item.file = None
item.is_active = False
item.status = 'failed'
item.error_message = 'PDF cache istekao.'
item.save(update_fields=['is_active', 'status', 'error_message', 'updated_at'])
item.save(update_fields=['file', 'is_active', 'status', 'error_message', 'updated_at'])
def _parse_amount_decimal(value):
@@ -572,10 +607,11 @@ def _invalidate_work_order_pdf_cache(work_order, *, pdf_types=None):
for cached in cache_qs:
if cached.file:
cached.file.delete(save=False)
cached.file = None
cached.is_active = False
cached.status = 'failed'
cached.error_message = 'PDF cache invalidiran zbog promjene podataka.'
cached.save(update_fields=['is_active', 'status', 'error_message', 'updated_at'])
cached.save(update_fields=['file', 'is_active', 'status', 'error_message', 'updated_at'])
def _upsert_additional_cost_row_from_invoice(invoice):
@@ -607,7 +643,11 @@ def _upsert_additional_cost_row_from_invoice(invoice):
def _cached_pdf_file_response(generated_pdf, *, default_filename):
try:
generated_pdf.file.open('rb')
except (FileNotFoundError, OSError, ValueError):
_mark_generated_pdf_failed(generated_pdf, 'PDF datoteka nije dostupna.')
raise DRFValidationError({"detail": "PDF nije dostupan ili je obrisan."})
filename = generated_pdf.filename or default_filename
response = FileResponse(generated_pdf.file, content_type='application/pdf')
response['Content-Disposition'] = f'attachment; filename="{filename}"'
@@ -2410,6 +2450,26 @@ def _file_attachment(file_field, fallback_name):
return (filename, content, _guess_content_type(filename))
def _write_file_field_to_zip(archive, *, file_field, entry_name, chunk_size=64 * 1024):
if not file_field:
return False
file_field.open('rb')
try:
first_chunk = file_field.read(chunk_size)
if not first_chunk:
return False
with archive.open(entry_name, mode='w') as destination:
destination.write(first_chunk)
while True:
chunk = file_field.read(chunk_size)
if not chunk:
break
destination.write(chunk)
finally:
file_field.close()
return True
def _build_image_attachments_for_work_order(work_order):
attachments = []
photos = WorkOrderPhoto.objects.filter(is_active=True, work_order=work_order).order_by('created_at')
@@ -2968,7 +3028,7 @@ def _parse_year_month_params(request):
return year, month
def _build_monthly_service_tasks_archive_content(*, user, year, month):
def _write_monthly_service_tasks_archive_entries(archive, *, user, year, month):
from modules.task_management.models import Task
tasks = list(
@@ -2988,8 +3048,7 @@ def _build_monthly_service_tasks_archive_content(*, user, year, month):
raise DRFValidationError({'detail': 'Nema servisnih taskova za odabrani mjesec.'})
used_names = set()
archive_buffer = BytesIO()
with zipfile.ZipFile(archive_buffer, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
entries_written = 0
for task in tasks:
work_order = task.work_order
if work_order is None:
@@ -2998,14 +3057,28 @@ def _build_monthly_service_tasks_archive_content(*, user, year, month):
base_name = _service_records_docx_filename(work_order, task)
entry_name = _unique_zip_entry_name(base_name, used_names)
archive.writestr(entry_name, docx_bytes)
entries_written += 1
if entries_written == 0:
raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'})
return entries_written
def _build_monthly_service_tasks_archive_content(*, user, year, month):
archive_buffer = BytesIO()
with zipfile.ZipFile(archive_buffer, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
_write_monthly_service_tasks_archive_entries(
archive,
user=user,
year=year,
month=month,
)
archive_content = archive_buffer.getvalue()
if not archive_content:
raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'})
return archive_content
def _build_monthly_work_orders_archive_content(*, user, year, month):
def _write_monthly_work_orders_archive_entries(archive, *, user, year, month):
from modules.task_management.models import Task
monthly_tasks = (
@@ -3044,23 +3117,34 @@ def _build_monthly_work_orders_archive_content(*, user, year, month):
raise DRFValidationError({'detail': 'Nema putnih naloga za odabrani mjesec.'})
used_names = set()
archive_buffer = BytesIO()
with zipfile.ZipFile(archive_buffer, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
entries_written = 0
for work_order in work_orders:
pdf_bytes = _build_work_order_pdf(work_order)
work_order_pdf_name = _unique_zip_entry_name(_pdf_filename(work_order, 'work_order'), used_names)
archive.writestr(work_order_pdf_name, pdf_bytes)
entries_written += 1
display_code = _work_order_display_code(work_order)
invoices = work_order.invoices.filter(is_active=True).order_by('datum', 'created_at')
for index, invoice in enumerate(invoices, start=1):
attachment = _file_attachment(invoice.image, fallback_name=f"invoice-{index}.bin")
if not attachment:
continue
invoice_filename, content, _content_type = attachment
archive_path = f"Racuni/{display_code}/{invoice_filename}"
archive.writestr(_unique_zip_entry_name(archive_path, used_names), content)
file_name = Path(str(getattr(getattr(invoice, 'image', None), 'name', '') or f"invoice-{index}.bin")).name
archive_path = _unique_zip_entry_name(f"Racuni/{display_code}/{file_name}", used_names)
if _write_file_field_to_zip(archive, file_field=invoice.image, entry_name=archive_path):
entries_written += 1
if entries_written == 0:
raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'})
return entries_written
def _build_monthly_work_orders_archive_content(*, user, year, month):
archive_buffer = BytesIO()
with zipfile.ZipFile(archive_buffer, mode='w', compression=zipfile.ZIP_DEFLATED) as archive:
_write_monthly_work_orders_archive_entries(
archive,
user=user,
year=year,
month=month,
)
archive_content = archive_buffer.getvalue()
if not archive_content:
raise DRFValidationError({'detail': 'Nije moguće kreirati ZIP za odabrani mjesec.'})
@@ -3371,12 +3455,18 @@ def generated_archive_download(request, archive_id):
if generated_archive is None:
raise DRFValidationError({'detail': 'ZIP arhiva nije dostupna ili je istekla.'})
generated_archive.file.open('rb')
file_name = getattr(generated_archive.file, 'name', '')
storage = getattr(generated_archive.file, 'storage', None)
if not file_name or storage is None:
raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'})
try:
archive_bytes = generated_archive.file.read()
finally:
generated_archive.file.close()
if not archive_bytes:
if not storage.exists(file_name):
raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'})
except (FileNotFoundError, OSError, ValueError):
raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'})
try:
generated_archive.file.open('rb')
except (FileNotFoundError, OSError, ValueError):
raise DRFValidationError({'detail': 'ZIP arhiva je prazna ili nedostupna.'})
filename = generated_archive.filename or _generated_archive_filename_for_user(
request.user,
@@ -3384,10 +3474,13 @@ def generated_archive_download(request, archive_id):
month=generated_archive.month,
archive_type=generated_archive.archive_type,
)
response = HttpResponse(archive_bytes, content_type='application/zip')
response = FileResponse(generated_archive.file, content_type='application/zip')
response['Content-Disposition'] = f'attachment; filename="{filename}"'
response['Cache-Control'] = 'private, max-age=3600'
response['Content-Length'] = str(len(archive_bytes))
try:
response['Content-Length'] = str(generated_archive.file.size)
except (OSError, ValueError, TypeError):
pass
return response
@@ -3666,6 +3759,18 @@ class WorkOrderViewSet(viewsets.ModelViewSet):
).exclude(file='').exclude(file__isnull=True).first()
if generated_pdf is None:
raise DRFValidationError({"detail": "PDF nije dostupan ili je istekao."})
file_name = getattr(generated_pdf.file, 'name', '')
storage = getattr(generated_pdf.file, 'storage', None)
if not file_name or storage is None:
_mark_generated_pdf_failed(generated_pdf, 'PDF datoteka nije dostupna.')
raise DRFValidationError({"detail": "PDF nije dostupan ili je obrisan."})
try:
if not storage.exists(file_name):
_mark_generated_pdf_failed(generated_pdf, 'PDF datoteka nije dostupna.')
raise DRFValidationError({"detail": "PDF nije dostupan ili je obrisan."})
except (FileNotFoundError, OSError, ValueError):
_mark_generated_pdf_failed(generated_pdf, 'PDF datoteka nije dostupna.')
raise DRFValidationError({"detail": "PDF nije dostupan ili je obrisan."})
return _cached_pdf_file_response(generated_pdf, default_filename=_pdf_filename(work_order, generated_pdf.pdf_type))
@action(detail=True, methods=['get'], url_path='pdf-preview')

View File

@@ -0,0 +1,24 @@
#!/bin/sh
set -eu
TMP_DIR="${APP_TMP_CLEANUP_DIR:-/tmp}"
MAX_AGE_DAYS="${APP_TMP_CLEANUP_MAX_AGE_DAYS:-1}"
PREFIXES="${APP_TMP_CLEANUP_PREFIXES:-erp-fleet-archive-}"
if [ ! -d "$TMP_DIR" ]; then
echo "TMP dir ne postoji: $TMP_DIR"
exit 0
fi
OLD_IFS="$IFS"
IFS=','
for prefix in $PREFIXES; do
prefix_trimmed="$(echo "$prefix" | xargs)"
if [ -z "$prefix_trimmed" ]; then
continue
fi
find "$TMP_DIR" -maxdepth 1 -mindepth 1 -name "${prefix_trimmed}*" -mtime "+${MAX_AGE_DAYS}" -print -delete
done
IFS="$OLD_IFS"
echo "TMP cleanup dovršen za $TMP_DIR (older than ${MAX_AGE_DAYS} day(s))."

View File

@@ -37,7 +37,13 @@ services:
environment:
DEBUG: "False"
DJANGO_SETTINGS_MODULE: core.settings.production
command: celery -A core worker --loglevel=info
CELERY_WORKER_PREFETCH_MULTIPLIER: "1"
CELERY_WORKER_MAX_TASKS_PER_CHILD: "20"
CELERY_WORKER_MAX_MEMORY_PER_CHILD: "350000"
APP_TMP_CLEANUP_DIR: "/tmp"
APP_TMP_CLEANUP_MAX_AGE_HOURS: "24"
APP_TMP_CLEANUP_PREFIXES: "erp-fleet-archive-"
command: celery -A core worker --loglevel=info --concurrency=${CELERY_WORKER_CONCURRENCY:-2}
volumes:
- media_volume:/app/media
depends_on: