feat: add native LangGraph runtime to gateway, replace langgraph-cli#1574
Open
greatmengqi wants to merge 14 commits intobytedance:mainfrom
Open
feat: add native LangGraph runtime to gateway, replace langgraph-cli#1574greatmengqi wants to merge 14 commits intobytedance:mainfrom
greatmengqi wants to merge 14 commits intobytedance:mainfrom
Conversation
added 2 commits
March 30, 2026 01:05
Consolidate the LangGraph CLI server (port 2024) and Gateway API (port 8001) into a single FastAPI service on port 2024. This eliminates the RUN_LOCK serialization bottleneck in langgraph-cli's inmem runtime and reduces the backend from 2 processes to 1. New server implements LangGraph Platform API endpoints (threads, runs/stream, assistants) using DeerFlowClient, while mounting all existing Gateway routers (models, skills, mcp, memory, uploads, artifacts, etc.) on the same app. Key changes: - Add DeerFlowClient stream_mode="dual" for independent messages + values streams - Filter middleware-internal messages (TitleMiddleware etc.) at server layer - Bounded ThreadStore with LRU eviction (max 10k threads) - Client disconnect handling via threading.Event cancel signal - 33 new tests, 1105 existing tests pass
Simplify models, skills, mcp, memory routers by delegating to the shared DeerFlowClient singleton instead of importing from low-level harness modules directly.
greatmengqi
force-pushed
the
feat/unified-server
branch
from
ccf4fa4 to
dd3a2b0
Compare
added 9 commits
March 30, 2026 01:14
- Remove null created_at from empty thread state response - Add SSE end event at stream completion (normal + error paths) - Fix concurrent status race with atomic busy/idle counter - Format with ruff
- Add SSE metadata event at stream start (run_id + thread_id) - Sanitize error messages in SSE (no internal exception leakage) - Fix title update tracking to detect title-clear transitions - Return shallow copies from ThreadStore.to_dict() to prevent mutation
- Add server-layer router implementations for models, skills, mcp, memory (thin wrappers around DeerFlowClient) - Revert gateway routers to original upstream implementations - Feature flag DEERFLOW_USE_GATEWAY_ROUTERS switches between the two - Default: server-layer routers (no gateway dependency for these 4) - Eliminates bidirectional dependency between server and gateway packages
Default (split): server:2024 + gateway:8001 + nginx routes between them. Server handles LangGraph API + models/mcp/skills/memory (via DeerFlowClient). Gateway handles artifacts/uploads/agents/suggestions/channels. DEERFLOW_STANDALONE=true: server:2024 only, mounts all routes. No gateway process needed. - Add nginx.split.conf (original routing, langgraph→server rename only) - Add nginx.standalone.conf (all routes → server) - serve.sh conditionally starts gateway based on DEERFLOW_STANDALONE - Revert gateway routers to original upstream implementations
- server: single process on port 2024, all APIs (LangGraph + REST) - gateway: legacy mode — langgraph-cli(2024) + gateway(8001) - nginx picks matching config (standalone vs split) - No feature flags or standalone/split complexity — just pick a runtime
Instead of a separate server process, add LangGraph Platform API routes directly to the existing gateway behind DEERFLOW_NATIVE_RUNTIME flag. - Flag ON (default): gateway serves everything, no langgraph-cli needed - Flag OFF: legacy mode — langgraph-cli(2024) + gateway(8001) - nginx standalone config points to gateway:8001 - One process, one app, one flag
- Delete app/server/app.py (standalone entry point) - Delete redundant server-layer routers (models, mcp, skills, memory) - Keep threads/runs/assistants routers + store + deps as library - Gateway imports them behind DEERFLOW_NATIVE_RUNTIME flag - Remove make server target
Move deps.py, store.py, and LangGraph routers (threads, runs, assistants) into app/gateway/. Delete app/server/ entirely. All code now lives under one package: app/gateway/.
greatmengqi
force-pushed
the
feat/unified-server
branch
from
30ceb3d to
9747bc2
Compare
greatmengqi
changed the title
greatmengqi
force-pushed
the
feat/unified-server
branch
from
9747bc2 to
0fc4950
Compare
Instead of two nginx configs and port juggling, just change the langgraph upstream from 127.0.0.1:2024 to 127.0.0.1:8001. Gateway on 8001 serves everything. Nginx routes unchanged.
greatmengqi
force-pushed
the
feat/unified-server
branch
from
0fc4950 to
6bc8d9c
Compare
added 2 commits
March 30, 2026 02:04
- start-daemon.sh: replace deleted app.server.app with app.gateway.app - serve.sh: fix log label (langgraph.log → gateway.log) - serve.sh: add "yes" to flag check matching Python's accepted values - serve.sh/start-daemon.sh: remove stale app.server.app pkill refs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DEERFLOW_NATIVE_RUNTIMEfeature flagRUN_LOCKserialization bottleneckstream_mode="dual"provides independent token-level message streaming and state snapshotsArchitecture
Files Changed
gateway/app.pygateway/deps.pygateway/store.pygateway/routers/runtime_threads.pygateway/routers/runs.pygateway/routers/assistants.pydeerflow/client.pystream_mode="dual"+ metadata fieldnginx.local.conf127.0.0.1:2024→:8001nginx.conf(Docker)langgraph:2024→gateway:8001serve.shstart-daemon.shMakefileTest plan
Closes #1595