• Andrey Filippov's avatar
    CLAUDE: item-4 rest-partition - LibTorch-in-carve (TPDNN_GREEN_SM) + the... · 1db6c678
    Andrey Filippov authored
    CLAUDE: item-4 rest-partition - LibTorch-in-carve (TPDNN_GREEN_SM) + the cudaHostAlloc cross-thread stall fix (pinned D2H)
    
    - tp_dnn.cpp: TPDNN_GREEN_SM green-ctx SM carve (spec syntax == tp_jna's
      TP_GREEN_SM) bound before the first torch CUDA call, ctx-drift detector at
      every C-ABI boundary (drift=0 measured: c10 never rebinds primary on a
      single-GPU box); persistent PINNED D2H staging tensors replace the pageable
      .to(kCPU) path - torch's CachingHostAllocator was cudaHostAlloc'ing a fresh
      staging buffer EVERY infer behind the deep async queue, and a hostAlloc/free
      behind a deep queue blocks every OTHER thread's CUDA calls until the queue
      drains (probe2o o8/o9d) = the RT loop stalled ~one full infer (~100 ms).
      Pageable oracle kept behind TPDNN_PAGEABLE_D2H=1; outputs byte-identical.
      TPDNN_ALLOC_DEBUG + TPDNN_BISECT probing hooks.
    - test_dnn_carve: feasibility probe (real L1+L2 models, carve sweep, drift
      gate, --dump for the D2H A/B). Confined: 36 SMs 76.3 / 20 SMs 98.8 /
      8 SMs 202.9 ms/infer, drift 0.
    - test_three_service_jna: MAP v2 concurrent measure - pose-8 + render-8 +
      DNN worker thread in rest-20. Post-fix: pose p50 5.14 solo -> 8.26 +render
      -> 13.5 +DNN continuous (was ~100 before the fix); all arrangements pose
      bit-identical, render tol-0, drift 0; DNN 98.5 solo -> 135 ms/inf under
      full concurrency (BW contention).
    Co-Authored-By: 's avatarClaude Fable 5 <noreply@anthropic.com>
    1db6c678
test_three_service_jna.cu 24.1 KB