Intelligent inference routing

Route every prompt to the right model.

Clean Food Good Router uses hybrid classical machine learning to classify each request, then LiteLLM routes it to the model best equipped for the work.

View the code See how it works

router.live

ACTIVE

LITELLM ROUTE

> Fix this failing Python function

CLASSIFY

Task detected

Code debugging

Selected model

Kimi K2.7 Code

Answer route ready

01 / Architecture

A deliberate path from request to response.

01
Prompt arrives
A task enters through the inference workflow with its task ID and prompt.
02
Classify the work
The hybrid classical ML classifier identifies one of eight task categories.
03
Route and answer
LiteLLM centralises local and Fireworks models behind one call, then writes answers and diagnostics to JSON.

02 / Task routing

Eight categories. Purposeful model choices.

Routing is task-aware, while model availability and deployment paths stay configurable.

⌘

Code debugging

“Fix this Python error”

{ }

Code generation

“Write a parser”

◌

Factual knowledge

“What causes tides?”

∴

Logical reasoning

“Solve this puzzle”

Math reasoning

“Calculate the derivative”

Entity recognition

“Extract company names”

♥

Sentiment

“Classify this review”

≡

Summarisation

“Condense this report”

03 / Classification

One deployed classifier, built for practical routing.

DEPLOYED / HYBRID CLASSICAL ML

Clean Food Good Router classifies before inference.

The deployed image uses the hybrid classical ML classifier only. It combines a classical model prediction with rule signals to select one of the eight routing categories, keeping the route decision focused, transparent, and lightweight.

The Web GUI uses this same classifier before sending a prompt through LiteLLM.

04 / Developer-ready

One LiteLLM interface for local and hosted models.

Configure Fireworks and the allowed model list through environment variables. LiteLLM makes it straightforward to use the local Ollama model and Fireworks API in the same Docker-oriented workflow.

Llama 3.2 (3B) on local Ollama
Kimi, Minimax, and Gemma via Fireworks API
Structured results plus routing diagnostics

Explore on Docker Hub

configuration.env

# Provider configuration
FIREWORKS_API_KEY=your_key_here
FIREWORKS_BASE_URL=https://api.fireworks.ai/inference/v1

# Models this deployment can route to
ALLOWED_MODELS=Kimi K2.7 Code,Minimax M3,Gemma 4 31B IT NVEP4

$ docker build -f dockerfile-classical .
✓ results.json + debug.json

05 / Web GUI

Explore the router through a focused chat experience.

The optional FastAPI web GUI gives people a simple place to enter a prompt, receive the answer, and see the selected category and model for that response.