You are browsing as a guest. Sign up (or log in) to start making projects!

11h 42m 22s logged

Hey everyone!Here’s a quick update on VeritasCore.If you haven’t seen it before, VeritasCore is an AI auditing tool. Instead of building another language model, it evaluates existing ones. You point it at a deployed AI model, it automatically launches red-team attacks looking for jailbreaks, bias, prompt injection vulnerabilities, and behavioral drift, then produces a cryptographically signed compliance certificate using Ed25519 so the results can be independently verified.The MVP already works. My goal for this hackathon is something different: making the entire pipeline run on AMD’s AI ecosystem using ROCm 7.2, PyTorch 2.9, and vLLM 0.16.0.Before writing a single line of integration code, I wanted to answer one question:Will this actually run on ROCm?A surprising number of AI projects quietly depend on NVIDIA-only libraries. Things like FlashAttention, xFormers, bitsandbytes, custom CUDA kernels, or even hardcoded cuda:0 references can completely derail a port.So my first task wasn’t coding—it was detective work.I went through the repository dependency by dependency, expecting to uncover compatibility problems.Instead, I found… almost nothing.The reason is surprisingly simple: VeritasCore doesn’t perform inference itself. It’s a black-box auditor that communicates with models over HTTP. The GPU only matters for the model being audited—not for the auditor.That completely changed the migration strategy.Instead of trying to rewrite the auditor, I can simply replace the scripted demo endpoint with a real open-source model served through vLLM on AMD hardware.To prepare for that, I built a small smoke test that will:verify PyTorch is using the HIP/ROCm build,confirm the AMD GPU is visible,and send a real inference request through the smallest practical model before scaling up.So far, I haven’t used a single second of GPU time. Every test has run entirely on CPU while preparing the environment.The next milestone is the exciting one: launching the first inference on the AMD notebook and seeing what actually breaks.That’s usually where the real engineering begins.

2
284

Comments 1

@Riko

sounds interesting .. but way off my league to understand such deep tech stuffs 🥲