CargoForge3D: Port Breakbulk Cargo Reconstruction

Business motivation

Why this is a service system, not a one-image demo

Breakbulk terminals need fast, non-contact reconstruction for cargo archiving, lifting simulation, digital twin visualization, and approximate real-scale placement. The workflow is production-oriented: capture, segment, reconstruct, scale, package, audit, and serve results to downstream twin platforms.

Operational requirement

A crane or yard event can trigger reconstruction. The first usable output is a white mesh for placement and bounding-box estimation; the slower PBR texture stage can finish asynchronously.

Hardware constraint

A single RTX4080 16GB can run the shape stage, but keeping both shape and texture models in one process is fragile. The service separates stages into two GPU-resident workers.

Domain gap

Open Hunyuan3D-2.1 is not specialized for long beams, stacked steel, covered cargo, and irregular breakbulk objects. Domain LoRA adaptation injects cargo-specific geometry priors.

System blueprint

The full loop from port image capture to digital twin asset

The project is organized as a closed engineering loop. Each stage has a measurable contract: what it receives, what it produces, which hardware it occupies, and how the next stage can resume after interruption.

01

Field capture

Cargo photos come from port-side cameras, mobile phones, or event-triggered capture during crane and yard operations. The capture target is practical: enough visual coverage for reconstruction, not a carefully controlled studio dataset.

Output: input image, capture metadata, optional scale reference.

02

Image preprocessing

Background removal and image screening reduce interference from quay ground, vehicles, cargo frames, and sky. This keeps Hunyuan3D focused on the cargo body rather than the surrounding port scene.

Output: source condition image and cleaned prompt image.

03

Fast white mesh

GPU0 runs Hunyuan3D-2.1 shape generation with HiCache++. This is the first useful asset in the workflow because it can be placed into the digital twin before texture is ready.

Output: white mesh, bounding box, mesh statistics, shape profile.

04

Texture generation

GPU1 runs the paint pipeline in low-VRAM mode. Texture is slower and memory-heavy, so it is isolated as a second queue stage instead of blocking shape generation.

Output: textured mesh, PBR maps, texture profile.

05

Asset packaging

The system stores GLB/OBJ artifacts, texture files, previews, and profile JSON under one job directory, then persists artifact metadata into PostgreSQL.

Output: downloadable GLB and auditable artifact record.

06

Evaluation and iteration

Runtime metrics, queue wait time, P95 latency, GPU memory peaks, failure rate, mesh statistics, and LoRA validation metrics feed back into system tuning.

Output: dashboard metrics and training/evaluation records.

Interview framing

A simple demo proves that a model can run once. CargoForge3D proves that a generative 3D model can be adapted to a constrained industrial workflow: limited VRAM, long texture latency, concurrent requests, recoverable jobs, measurable throughput, and domain-specific geometry adaptation.

Base model

How Hunyuan3D-2.1 is used

In this project, Hunyuan3D-2.1 is treated as two linked pipelines: a shape diffusion pipeline that reconstructs a white mesh from one image, and a paint pipeline that generates multi-view PBR textures and bakes them back to the mesh.

CargoForge3D algorithm overview with Hunyuan3D-2.1, HiCache++ and domain LoRA — Algorithm view. HiCache++ is inserted into the Hunyuan3D-2.1 shape DiT sampling loop, while the cargo-domain LoRA adapter is trained from teacher-generated GLB supervision and plugged back into the shape model.

Image-to-3D path single image to deployable GLB asset

Input image

→

DINOv2 image encoder

→

Shape DiT flow matching

→

Shape VAE and mesh extraction

→

White mesh

Normal / position condition views

→

Texture diffusion

→

PBR baking and GLB packing

→

Textured GLB

Windows and local-model adaptation

The local build fixes Windows-specific runtime issues including CUDA device binding, TorchVision compatibility, HuggingFace/hy3dgen local model lookup, dynamic module cache handling, texture-device hardcoding, and safe shape/texture memory release.

Boundary of the claim

Hunyuan3D-2.1 remains a single-image shape model. This project does not claim multi-view shape generation. Multi-GPU support is implemented as staged service parallelism, not tensor-parallel DiT execution.

What happens inside the shape stage

The input image is encoded into visual conditions, and the shape DiT denoises latent shape features through an iterative flow-matching process. After sampling, the shape VAE decodes the latent representation and the mesh extraction step converts it into a white mesh. In port use, this stage is responsible for silhouette, coarse topology, cargo proportions, and whether the object is plausible enough to enter a twin scene.

What happens inside the texture stage

The texture pipeline renders geometry-aware condition views from the generated mesh, including normal and position information, then uses the paint model to synthesize multi-view texture images. The final step is back-projection and PBR baking. This stage is slower because it combines diffusion inference, rendering, inpainting, texture fusion, and GLB repacking.

Why the split matters

For a port system, white-mesh latency and texture latency have different business meanings. White mesh is operational: it supports occupancy, approximate shape, and scale alignment. Texture is archival and visual: it makes the asset recognizable. Splitting them allows the service to return useful intermediate results.

Innovation 1

HiCache++ acceleration for fast white-mesh generation

Port operators often need a fast white mesh before texture is available. HiCache++ targets the expensive shape sampling loop by reusing historical velocity-field information and skipping selected DiT forward passes.

Cache-then-forecast sampling refresh the DiT, forecast intermediate states

DiT

forecast

refresh

forecast

DiT

At refresh steps, the model evaluates the DiT normally. At cached steps, the sampler forecasts the next latent state from historical trajectories. The acceleration is especially useful for shape-only preview, digital twin placeholders, and queue throughput under frequent field captures.

30-step sampling40.0 s7.0 s5.7x

Shape end-to-end67.3 s24.0 s2.8x

Serving modesingle requestGPU0 workerresident

Where it is inserted

HiCache++ is inserted into the shape sampling loop, not into the VAE, mesh extraction, texture model, or renderer. This keeps the output contract unchanged: the downstream mesh extraction and texture stages still receive the same type of latent output.

Serving parameters

The concurrent service starts the shape worker with shape acceleration enabled by default. The local start script exposes the acceleration mode and uses cache interval 3 and history length 5 for the HiCache++ path, while keeping DMD as an experimental alternative.

Validation discipline

Acceleration is evaluated with runtime and quality signals together: sampling time, end-to-end shape time, VRAM peak, face/vertex count, watertightness, visual inspection, and whether the white mesh remains usable for downstream texture generation and twin placement.

Innovation 2

Dual-GPU asynchronous serving for low-VRAM full-chain reconstruction

The engineering bottleneck is not only inference speed. It is queueing, state persistence, GPU isolation, failure recovery, and observability when multiple cargo reconstruction requests arrive.

CargoForge3D FastAPI Redis PostgreSQL dual RTX 4080 serving architecture — Serving architecture. The system uses pipeline parallelism rather than model parallelism: GPU0 keeps the shape worker resident, GPU1 keeps the texture worker resident, Redis handles stage queues, and PostgreSQL records durable job state.

ClientsWeb UI, batch upload, API caller

→

FastAPI gatewaysubmit, status, cancel, download, metrics

→

Redis priority queuesshape: high / normal / batchtexture: high / normal / batch

GPU0 shape workerHunyuan3D-2.1 shape pipelineHiCache++ acceleration

GPU1 texture workerPBR paint pipeline4 views, 512 px, low-VRAM mode

PostgreSQL state storejobs, events, artifacts, profiles, heartbeats

Observability and recoveryDashboard, Prometheus metrics, watchdog, cleanup scripts

Job lifecycle

Jobs move through submitted, queued shape, running shape, queued texture, running texture, packing, completed, failed, cancelled, or retrying states. Every transition is persisted as an event.

Backpressure instead of infinite waiting

Texture is the bottleneck stage. The system reports backlog and ETA, supports priority queues, and prevents unlimited Redis accumulation when GPU memory or texture capacity is saturated.

Recovery and cleanup

Workers write PostgreSQL heartbeats. A watchdog detects stale workers, handles unfinished jobs, and supports operator cleanup of queues and worker state when GPUs must be released.

45.4 sShape P95

321.3 sTexture P95, bottleneck stage

269textured assets/day, P95-derived

1903shape-only assets/day, P95-derived

Runtime repository and one-click startup

The current concurrent-serving build is located at:

D:\tencent Hunyuan3D-2.1(new version)\Hunyuan3D-2.1-main\Hunyuan3D-2.1-main

The startup script checks the Hunyuan3D conda Python, API port 8091, Redis on 6379, PostgreSQL, GPU free memory, then launches shape worker, texture worker, watchdog, and FastAPI.

powershell.exe -ExecutionPolicy Bypass -File "D:\tencent Hunyuan3D-2.1(new version)\Hunyuan3D-2.1-main\Hunyuan3D-2.1-main\start_hy3d_concurrent.ps1"

State and observability design

Redis is deliberately limited to queueing. PostgreSQL stores durable job state, job events, artifacts, per-stage profiles, worker heartbeats, and user quotas. This makes the system recoverable after crashes and gives the dashboard a database-backed source of truth.

Logs: C:\Users\admin\Documents\New project\hy3d_concurrent_logs
Jobs: C:\Users\admin\Documents\New project\hy3d_concurrent_jobs
UI:   http://127.0.0.1:8091

LatencyShape P50/P95/P99, Texture P50/P95/P99, E2E P95, queue waitFinds the real bottleneck instead of guessing from one run.

GPUVRAM peak, utilization, worker busy/idle, CUDA OOM eventsProtects the dual-4080 service from hidden memory pressure.

Redisqueue length, ops/sec, latency, clients, slowlog, error repliesVerifies that Redis is a healthy queue, not a silent bottleneck.

Jobscompleted, failed, cancelled, retrying, artifacts, profile JSONMakes every reconstruction request auditable and reproducible.

Innovation 3

Teacher-supervised LoRA adaptation for port cargo geometry

Since stronger commercial Hunyuan3D versions cannot be deployed locally, the project uses them as teachers: high-quality GLB pseudo labels are generated for real cargo images, then converted into Hunyuan3D-2.1 shape-supervision data for parameter-efficient adaptation.

Dataset distillation pipeline teacher GLB to Hunyuan3D shape supervision

Port cargo photo

+

Teacher GLB

→

Mesh audit and cleaning

→

Blender 24-view condition render

→

Watertight mesh and SDF sampling

→

Rank-16 LoRA training

Training format

Each sample becomes a preprocessed folder containing geometry arrays, watertight OBJ, multi-view render conditions, source condition image, mesh.ply, and transforms.json. This matches the mesh-quality LoRA training pipeline.

A100 training setup

The A100 run uses bf16 mixed precision, DINOv2-Large, Hunyuan3D-2.1 DiT weights, LoRA rank 16, alpha 16, dropout 0.05, TensorBoard logging, periodic adapter export, and validation every 100 steps.

Teacher set100 pairsfield image + high-quality GLB pseudo label

AdapterRank-16 LoRAparameter-efficient domain update

Chamfer-L10.0511 → 0.032736.0% lower

F-score@0.050.670 → 0.793+12.3 percentage points

Source auditinput image, teacher GLB, split_manifest.jsonGuarantees one image-to-mesh pair per sample and creates train/validation splits.

Blender rendering24 RGBA render views, mesh.ply, transforms.jsonReconstructs the multi-view visual conditions expected by the original training pipeline.

Geometry conversionwatertight.obj, surface.npz, sdf.npzConverts teacher GLB geometry into SDF and surface supervision.

Training and selectionLoRA adapters, TensorBoard logs, validation GLBsTracks loss, validates checkpoints, and selects adapters by metrics plus visual inspection.

What the LoRA actually changes

The LoRA adapter updates selected linear projections inside the shape model while keeping the base model mostly frozen. The goal is not to memorize one cargo model, but to bias the shape generator toward port-specific geometry: elongated steel members, stacked cargo, large flat faces, and irregular industrial parts.

Why teacher distillation is used

Real ground-truth 3D scans are expensive in port scenes. Teacher GLBs from a stronger commercial model provide scalable pseudo supervision. They are not treated as perfect truth; they are filtered, cleaned, converted, and checked before training.

How results are judged

The validation loop combines numerical geometry metrics with visual checks. Chamfer-L1 and F-score quantify shape alignment, while GLB inspection catches failure modes that scalar loss can hide, such as collapsed thin structures or over-smoothed industrial edges.

Interactive results

Generated cargo assets

Geometry and texture are inspected separately because the production pipeline is staged: white mesh quality, texture feasibility, runtime profile, and final GLB packaging are evaluated as separate concerns.

Shape 25.3 s Texture 61.7 s 617k faces

LoRA visual check

Base model versus domain adapters

The LoRA comparison is shown as interactive mesh panels so reviewers can rotate the same validation item and inspect geometry changes across checkpoints.

Base Hunyuan3D-2.1

LoRA step 1500

LoRA step 2000

Implementation map

Code modules worth discussing in an interview

hy3d_concurrent/api_server.pyFastAPI gateway, job submission, health checks, metrics JSON, Prometheus endpoint.

hy3d_concurrent/worker.pyGPU0 shape worker, GPU1 texture worker, HiCache++ options, texture compile experiment, CUDA cleanup.

hy3d_concurrent/db.pyPostgreSQL job store, events, artifacts, profiles, worker heartbeat queries, metrics snapshots.

hy3d_concurrent/watchdog.pyStale worker detection, graceful stop signal, force fallback, and safe requeue/cancel handling.

prepare_hunyuan_cargo_dataset.pyGLB/image inventory, train/val split, Blender rendering, watertight/SDF generation, validation reports.

a100_train_cargo_lora.shA100 LoRA training wrapper with local paths, TensorBoard logging, adapter export, and validation settings.

Project outcome

From model adaptation to an operable reconstruction service

CargoForge3D demonstrates an end-to-end engineering path for port cargo assets: field capture, segmentation and preprocessing, accelerated white-mesh generation, low-VRAM texture serving, GLB/PBR export, job audit, capacity profiling, and domain adaptation. The work is connected to a Yantai Port digital twin reconstruction project and is designed around real terminal constraints.

Generative 3D reconstruction for breakbulk cargo.