Commit a53575f5 authored by Andrey Filippov's avatar Andrey Filippov

docs: Add U of U FOPEN DNN integration plan to project roadmap

parent 69bd7ae6
......@@ -562,6 +562,7 @@ This section captures the latest validated state before pausing Global LMA work
- **Adaptive Integration Windows:** Implement a competitive selection mechanism (e.g., 1x vs 4x integration) where the pipeline automatically selects the motion vector that yields the highest Contrast-to-Noise Ratio (CNR), rather than relying on hardcoded thresholds.
- **Fat Zero Auto-Scaling:** Automatically scale `cuas_rng_fz` (fat zero) down proportionally to the length of the temporal integration (longer integrations lower the stochastic noise floor, allowing closer-to-pure phase correlation).
- **Acceleration Compensation:** Refine the "Virtual Moving Camera" model to handle non-linear motion (like U-turns) that currently "squash" correlation peaks during long integrations.
- **Hybrid Classical/DNN Tracking (U of U FOPEN Collaboration):** Export unnormalized MCLT correlation hyperstacks to train an Attention/SSM-based neural network. The goal is to replace the LMA fitter with a DNN regression head `(X, Y, vX, vY)` that applies learned "soft masks" to 16x16 macroblocks, running inference via TorchScript/DJL inside the Java pipeline. See `03_UU_RongRong_Hybrid_DNN_Architecture.md` for details.
- *Note: These deeper algorithmic optimizations are intentionally deferred. The strategy is to establish a working baseline first, expose the necessary low-bandwidth tile metrics via the MCP server, and then allow AI agents (Codex, Claude, Gemini) to autonomously sweep, analyze, and optimize these specific sub-problems.*
3. **FPGA / Hardware Teaming Roadmap (U of U Collaboration):**
- **MCP for GTKWave:** Develop a Model Context Protocol (MCP) bridge to allow LLMs to natively analyze `.vcd` files. This will enable natural language querying of simulation waveform data (e.g., "Find the memory arbiter hang").
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment