[INFRA-AI-02] FastAPI Gateway Containerization and Local Ollama Inference Orchestration

Infrastructure / MLOps

Published

October 14, 2026

NoteTask Brief
  • Repository / Component: ai_receipt_vision_gateway.
  • Stack: Docker, Docker Compose, Ollama API.
  • Task Type: DevOps / Infra.
  • Status: Ready for Implementation

Task Description

It is necessary to package the developed Python gateway into a Docker container and configure the joint launch of the web service and the local Ollama environment via Docker Compose for running inference on the CPU.

Step-by-Step Instructions

  1. Writing the Dockerfile for the Gateway: Prepare a build file for the FastAPI application based on a lightweight python:3.10-slim image. Ensure the installation of fastapi, pydantic, requests, and uvicorn dependencies via pip install. Expose port 11435.
  2. Designing the Docker Compose Manifest: Define two linked services:
    • ai-gateway — the container running FastAPI. Map port 11435:11435 and pass the environment variable OLLAMA_HOST=http://ollama-core:11434.
    • ollama-core — the official ollama/ollama image. Map port 11434:11434. Attach a Docker Volume for the /root/.ollama directory so that downloaded models are not erased upon restart.
  3. Automating Model Downloads (Build Modeling): Write a companion bash script (or entrypoint) that, after the ollama-core container starts, automatically executes a command inside the container: ollama run llama3.2-vision:11b to pre-download the model weights to the disk.