Case Study
Async AI Inference Backend
Based on production systems built at Infidea.
Summary
Designed and built an async job processing architecture at Infidea for long-running AI inference tasks. The system accepts API requests, queues work for background processing via Redis and RabbitMQ, runs inference in worker processes, stores results in object storage (S3/GCS), tracks job status, and delivers results through webhook callbacks. This pattern was reused across multiple AI projects at Infidea.
What This Page Covers
- How the async architecture decouples request submission from processing
- Why object storage was chosen over databases for inference outputs
- How webhook callbacks integrate with downstream systems
Approach
The architecture uses FastAPI for the API layer with endpoints for job submission, status checking, and result retrieval. When a job is submitted, it enters a Redis queue and is picked up by worker processes (managed via RabbitMQ for reliability). Workers run the actual inference, store results to S3 or GCS, and update the job status. Integrating systems receive webhook callbacks when jobs complete.
FastAPI serves the submission and status endpoints. Flask is used for worker processes and internal tooling. Docker containers handle deployment across AWS and GCP, with the same architecture adapting to both clouds.
Key Decisions
- Queue-based decoupling — submission endpoints stay fast regardless of inference duration; workers scale independently
- Object storage for results — inference outputs (images, large JSON, model artifacts) bloat databases; S3/GCS handles them cleanly
- Webhook callbacks — real-time result delivery to integrating systems without polling
- Multi-cloud support — same architecture deploys to both AWS and GCP with S3/GCS swap
Tech Stack
Python, FastAPI, Flask, Redis, RabbitMQ, Docker, AWS, GCP, S3/GCS, webhooks.
Outcome
Reusable async inference pattern deployed across multiple AI projects at Infidea. Each project reuses the queue, storage, and callback infrastructure while swapping only the worker inference logic.
Need something like this?
If you are building a model-serving API, async inference backend, or AI job processing system, contact me to discuss the architecture.