[TEST-AI-PROMPT-05] Regression Testing of AI Prompts for Qwen and Llama

Testing: Data Science / ML

Published

October 14, 2026

NoteTask Brief
  • Components: ai_voice_text_parser (Qwen), ai_receipt_vision_gateway (Llama).
  • Tools: Python, pytest, Pydantic validation.
  • Task Type: MLOps / QA.
  • Status: Under Analysis

Task Description

Since local language models are prone to hallucinations, it is necessary to implement automated tests to verify that prompts consistently force the models to output a valid JSON contract, and that the AI does not append any redundant text to the response.

Step-by-Step Instructions

  1. Creating a Golden Dataset: Prepare reference text transcriptions (e.g., “bought two milks and bread”) alongside their ideal, expected JSON responses.
  2. Writing Schema Validation Tests: Develop a Python script that pipes this dataset through the local qwen2.5:3b model.
  3. Success Metrics (Assertion): A test is marked as passed if the model maps the raw text into the JSON fields name, portions, and weight_grams WITHOUT any syntax anomalies or markdown code blocks (```json). If the model hallucinates or skips fields, the test must fail.