Optimize Domain-Specific LLM QA Platform

The prototype of my domain-specific question-answering system is live, with a fine-tuned LoRA/SFT language model sitting in a private Hugging Face repo, served through a FastAPI micro-service and wrapped in a multi-agent pipeline (retriever + analyst). Everything ships in Docker, rolls out under Kubernetes via Helm, and already includes a basic monitoring and evaluation loop. Right now the bottleneck is accuracy—specifically the relevance of each answer. I have a fresh batch of curated domain data ready for ingestion, and I’d like that folded into the workflow along with any architecture or hyper-parameter tweaks that further sharpen responses. Key focus areas for today (hard deadline: 7 pm): • Integrate the new dataset, run a quick SFT/LoRA refresh, and redeploy the updated weights to the private Hugging Face repository. • Tune retrieval strategy and analyst prompts so returned answers stay laser-focused on context. • Profile the FastAPI inference layer, tighten any latency introduced by the new model, and ensure horizontal scaling in the current Kubernetes setup remains intact after the swap. • Extend the existing evaluation script to log per-question relevance scores; surface those in our monitoring dashboard for rapid feedback. Acceptance for this sprint is simple: by 7 pm the cluster serves the new model, passes our internal relevance benchmark (≥10-point lift on the validation set), and all pods stay within current CPU/GPU envelopes under load test. All repos, Helm charts, and sample notebooks are ready to share as soon as work begins.

Project ID: 40188124