Skip to content

Local Image Generation — ComfyUI and FLUX

ComfyUI is the de-facto local diffusion stack — a node-graph GUI and API. It is the primary way to run FLUX locally.

ComfyUI is the de-facto local diffusion stack — a node-graph GUI and API/backend, explicitly "the fastest local inference engine in the world" per its repo. Verified: repo now under Comfy-Org/ComfyUI, commits within hours of Oct 9 2026, still Python, MIT (repo). It is the primary way to run FLUX (incl. FLUX.1 Kontext and the FLUX.2 line) and Stable Diffusion locally, with community nodes and ComfyUI API automation (FLUX.1 Kontext, FLUX.2). A 12GB+ consumer GPU runs FLUX-dev class models; GGUF/quantised variants extend to lower VRAM.

Recommended: ComfyUI + FLUX (dev/schnell) for image gen on a single local GPU; Stable Diffusion XL/1.5 remains the low-VRAM fallback. Note the comfyui PyPI package (v0.0.1, 2024) is not the product — install from the repo.

Sources

  • https://vllm.ai — vLLM: high-throughput OpenAI-compatible local serving engine (docs entry point).
  • https://ollama.readthedocs.io/en/openai — Ollama's experimental OpenAI-compatible /v1/... API and its feature/field gaps.
  • https://github.com/ggml-org/llama.cpp — llama.cpp repo: v0.6.0 (Oct 2026), 130.6k★, commits within hours of Oct 9 2026.
  • https://github.com/ggml-org/llama.cpp/releases — llama.cpp release history confirming v0.6.0 on Oct 5 2026.
  • https://pypi.org/project/vllm/ — vLLM v0.31.0, last upload 2026-10-05 (maintenance verification).
  • https://pypi.org/project/ollama/ — Ollama v0.6.3, last upload 2026-09-29.
  • https://pypi.org/project/litellm/ — LiteLLM v1.104.2, 2026-10-08: proxy layer for normalising local endpoints.
  • https://modelcontextprotocol.io/specification/2026-07-28 — MCP 2026-07-28 stateless spec: JSON-RPC, tools/resources/prompts, security model.
  • https://blog.cloudflare.com/mcp-v2 — Cloudflare on the 2026-07-28 MCP spec and updated SDKs (stateless MCP).
  • https://developers.googleblog.com/scaling-ai-agent-infrastructure-with-the-mcp-stateless-updates — Google on MCP stateless updates as foundational agent infra.
  • https://www.descope.com/learn/post/mcp — MCP security: local-server privilege and startup-command risks.
  • https://pypi.org/project/langchain/ — LangChain v1.4.4, 2026-10-08.
  • https://pypi.org/project/llama-index/ — LlamaIndex v0.14.25, 2026-09-21.
  • https://pypi.org/project/llama-index-vector-stores-qdrant/ — LlamaIndex↔Qdrant integration v0.10.4, 2026-10-08.
  • https://pypi.org/project/crewai/ — CrewAI v1.15.26, 2026-10-08 (multi-agent crews, actively maintained).
  • https://pypi.org/project/pyautogen/ — AutoGen v0.10.0, last upload 2025-07-15 (slowing cadence).
  • https://pypi.org/project/dspy/ — DSPy v3.4.0, 2026-09-25.
  • https://pypi.org/project/haystack-ai/ — Haystack v3.3.0, 2026-10-01.
  • https://github.com/sst/opencode — OpenCode (now anomalyco/opencode): open-source coding agent, commits Oct 8 2026, 212k★.
  • https://github.com/Aider-AI/aider — Aider: v0.86.2 (Feb 2026), last commit May 2026 — mature but slowed.
  • https://platform.claude.com/docs/en/cli-sdks-libraries/libraries/openai-sdk — Anthropic docs: using Claude Code via an OpenAI-compatible SDK/gateway.
  • https://github.com/RichardAtCT/claude-code-openai-wrapper — community wrapper to point Claude Code at OpenAI-compatible endpoints.
  • https://huggingface.co/Qwen/Qwen3-Embedding-8B — Qwen3-Embedding series (0.6B/4B/8B + rerankers); #1 MTEB multilingual 70.58; instruct adds 1–5%.
  • https://huggingface.co/Qwen/Qwen3-VL-Embedding-2B — Qwen3-VL-Embedding/Reranker for multimodal RAG; MMEB-V2 leader.
  • https://pypi.org/project/qdrant-client/ — Qdrant client v1.19.1, 2026-09-16.
  • https://pypi.org/project/chromadb/ — Chroma v1.5.9, 2026-05-05.
  • https://pypi.org/project/faiss-cpu/ — FAISS v1.15.1, 2026-09-16.
  • https://pypi.org/project/weaviate-client/ — Weaviate client v4.23.1, 2026-09-07.
  • https://pypi.org/project/milvus/ — Milvus v2.3.9, last upload 2024-11-12 (PyPI stale).
  • https://pypi.org/project/sentence-transformers/ — sentence-transformers v6.1.0, 2026-09-18 (embedding runtime).
  • https://pypi.org/project/unsloth/ — Unsloth v2026.10.3, 2026-10-08 (fast single-GPU LoRA/QLoRA).
  • https://github.com/axolotl-ai-cloud/axolotl — Axolotl repo: v0.20.0 (Sep 2026), multi-GPU/FSDP2, QAT, MoE LoRA.
  • https://pypi.org/project/axolotl/ — Axolotl v0.20.0, 2026-09-30.
  • https://pypi.org/project/mlx-lm/ — MLX-LM v0.32.0, 2026-10-01 (Apple Silicon fine-tuning).
  • https://pypi.org/project/mlx/ — MLX v0.32.3, 2026-09-29.
  • https://explore.n1n.ai/blog/fine-tune-llm-lora-qlora-guide-2026-2026-04-17 — 2026 VRAM table: QLoRA ~5–8GB for 3–8B, up to 70B at 4-bit.
  • https://gigagpu.com/rtx-4090-24gb-for-fine-tuning — RTX 4090 24GB: LoRA to ~14B FP16, QLoRA to ~70B NF4.
  • https://www.stack-junkie.com/blog/lora-fine-tuning-consumer-gpu — QLoRA on consumer GPU with Unsloth, export to GGUF for llama.cpp/Ollama.
  • https://github.com/EleutherAI/lm-evaluation-harness — lm-evaluation-harness repo (standard open-model benchmarking).
  • https://pypi.org/project/lm-eval/ — lm-eval v0.4.13, 2026-08-31 (maintenance verification).
  • https://github.com/ggml-org/whisper.cpp — whisper.cpp: active local ASR (commits Oct 2026).
  • https://github.com/SYSTRAN/faster-whisper — faster-whisper (CTranslate2) v1.2.1, last upload 2025-10-31.
  • https://pypi.org/project/kokoro-onnx/ — kokoro-onnx v0.6.1, 2026-08-19 (active Kokoro TTS runtime).
  • https://pypi.org/project/piper-tts/ — Piper TTS v1.8.0, 2026-09-04.
  • https://pypi.org/project/mlx-audio/ — mlx-audio v0.5.8, 2026-10-05 (local speech on Apple Silicon).
  • https://github.com/comfyanonymous/ComfyUI — ComfyUI (Comfy-Org/ComfyUI): modular diffusion GUI+API, commits Oct 9 2026.
  • https://bfl.ai/blog/flux-1-kontext — FLUX.1 Kontext announcement (local/ComfyUI image generation).
  • https://bfl.ai/blog/flux-2 — FLUX.2 announcement (Black Forest Labs open weights).
  • https://daily.dev/blog/running-llms-locally-ollama-llama-cpp-self-hosted-ai-developers — 2026 overview: Ollama/llama.cpp/LM Studio/LocalAI all expose OpenAI-compatible endpoints.
  • https://www.spheron.network/blog/openai-compatible-api-self-hosted — vLLM's OpenAI-compatible server endpoints (/v1/chat/completions etc.).
  • https://www.langchain.com/resources/langchain-vs-llamaindex — LangChain vs LlamaIndex capability comparison for retrieval and agents.