[BACK-AI-01] FastAPI Gateway Development for Receipt Recognition via Llama Vision

AI Components (AI Service)

Published

October 14, 2026

NoteTask Brief
  • Repository / Component: ai_receipt_vision_gateway.
  • Language: Python 3.10+ (FastAPI, Pydantic, Requests).
  • Task Type: Backend (AI Integration).
  • Status: Ready for Implementation

Task Description

Implement a lightweight FastAPI-based gateway that accepts receipt images in Base64 format from the main Go backend, forwards them to a local Ollama server running the multimodal llama3.2-vision:11b model, and returns a strictly structured JSON response.

Step-by-Step Instructions (Python)

  1. Create Data Models (Pydantic): Define the schema for the incoming request PromptRequest. The images field must accept an array of strings, where the Go service will inject a single Base64-encoded receipt image. Fix the stream=False and format="json" parameters by default.
  2. Proxy Requests to Ollama: Implement a POST handler for /api/generate. This method must send a synchronous JSON request to the Ollama API (/api/generate) at the address specified in the OLLAMA_HOST environment variable.
  3. Parse and Clean Markdown Tags: Since LLMs often wrap JSON in markdown formatting, apply the regular expression re.sub(r"```json\s*|```", "", raw_text) to clean the response text before deserializing it via json.loads.
  4. Implement CPU Fault Tolerance (Fallback):
    • Set a strict timeout of 40 seconds (timeout=40) for requests to Ollama, as vision model inference on CPU requires significant computational resources.
    • When catching a json.JSONDecodeError, requests.exceptions.RequestException, or receiving a non-200 status code, log the CPU overload error and call the fallback_response helper.
    • The helper must return a fixed, hardcoded list of products (Chicken fillet, Potatoes) structured as a receipt to prevent breaking the processing pipeline on the backend.