xiaohongshu
xiaohongshu
chatkit-integration
Foundation skill for integrating OpenAI ChatKit framework with custom backends. This skill should be used for initial ChatKit setup including server implementation, React component integration, authentication, context injection, and database persistence. For streaming UI patterns use chatkit-stre...
Full skill instructions
This skill provides the foundation for ChatKit integration - getting the basic chat working end-to-end with your custom backend and agent. It covers:
For advanced capabilities, see related skills:
chatkit-streaming (Tier 2): Response lifecycle, progress updates, client effectschatkit-actions (Tier 3): Interactive widgets, entity tagging, composer toolsYou are a full-stack engineer integrating OpenAI ChatKit framework with a custom backend and AI agents. You understand that ChatKit provides standardized conversation UI/UX, but requires custom integration to work with domain-specific agents and context.
Backend Integration:
Frontend Integration:
Context Requirements:
Persistence Requirements:
Extend ChatKit Server, Don't Replace
ChatKitServer[RequestContext]respond() method for agent executionContext Injection in Prompt
User Isolation via RequestContext
user_id in RequestContextuser_id automaticallyGraceful Degradation
Connection Pool Warmup
pool_pre_ping=True)Custom Fetch Interceptor
fetch function to useChatKit configX-User-ID)Script Loading Detection
Page Context Extraction
Build-Time Configuration
docusaurus.config.ts (build-time)customFields for client-side accessprocess.env in browser codeAuthentication Gate
httpOnly Cookie Proxy (Next.js)
Next.js Script Loading Strategy
beforeInteractive for web component scripts in layout.tsx<head> elementWhen: Integrating ChatKit with OpenAI Agents SDK
Implementation:
from chatkit.server import ChatKitServer
from agents import Agent, Runner
from chatkit.agents import stream_agent_response
class CustomChatKitServer(ChatKitServer[RequestContext]):
"""Extend ChatKit server with custom agent."""
async def respond(
self,
thread: ThreadMetadata,
input_user_message: UserMessageItem | None,
context: RequestContext,
) -> AsyncIterator[ThreadStreamEvent]:
# Only handle user messages (let base class handle read-only ops)
if not input_user_message:
return
# Load conversation history
previous_items = await self.store.load_thread_items(
thread.id, after=None, limit=10, order="desc", context=context
)
# Build history string for prompt
history_str = "\n".join([
f"{item.role}: {item.content}"
for item in reversed(previous_items.data)
])
# Extract context from metadata
user_info = context.metadata.get('userInfo', {})
page_context = context.metadata.get('pageContext', {})
# Build context strings
user_context_str = f"\nUser: {user_info.get('name')}\n"
page_context_str = f"\nPage: {page_context.get('title')}\n"
# Create agent with tools
agent = Agent(
name="Assistant",
tools=[your_search_tool],
instructions=f"{history_str}\n{user_context_str}{page_context_str}\n{system_prompt}",
)
# Convert message to agent input
converter = YourThreadItemConverter()
agent_input = await converter.to_agent_input(input_user_message)
# Run agent with streaming
agent_context = YourAgentContext(
thread=thread,
store=self.store,
request_context=context,
)
result = Runner.run_streamed(agent, agent_input, context=agent_context)
# Stream results
async for event in stream_agent_response(agent_context, result):
yield event
Evidence: rag-agent/chatkit_server.py:100-270
When: Adding authentication and context to ChatKit requests
Implementation:
const { control, sendUserMessage } = useChatKit({
api: {
url: `${backendUrl}/chatkit`,
domainKey: domainKey,
// Custom fetch to inject auth and context
fetch: async (url: string, options: RequestInit) => {
// Check authentication
if (!isLoggedIn) {
throw new Error('User must be logged in');
}
const userId = session.user.id;
const pageContext = getPageContext();
const userInfo = {
id: userId,
name: session.user.name,
email: session.user.email,
// ... other user fields
};
// Modify request body to add metadata
let modifiedOptions = { ...options };
if (modifiedOptions.body && typeof modifiedOptions.body === 'string') {
const parsed = JSON.parse(modifiedOptions.body);
if (parsed.type === 'threads.create' && parsed.params?.input) {
parsed.params.input.metadata = {
userId,
userInfo,
pageContext,
...parsed.params.input.metadata,
};
modifiedOptions.body = JSON.stringify(parsed);
} else if (parsed.type === 'threads.run' && parsed.params?.input) {
if (!parsed.params.input.metadata) {
parsed.params.input.metadata = {};
}
parsed.params.input.metadata.userInfo = userInfo;
parsed.params.input.metadata.pageContext = pageContext;
modifiedOptions.body = JSON.stringify(parsed);
}
}
// Add authentication header
return fetch(url, {
...modifiedOptions,
headers: {
...modifiedOptions.headers,
'X-User-ID': userId,
'Content-Type': 'application/json',
},
});
},
},
});
Evidence: robolearn-interface/src/components/ChatKitWidget/index.tsx:197-240
When: ChatKit requires external script, component must wait
Implementation:
const [scriptStatus, setScriptStatus] = useState<'pending' | 'ready' | 'error'>(
isBrowser && window.customElements?.get('openai-chatkit') ? 'ready' : 'pending'
);
useEffect(() => {
if (scriptStatus !== 'pending') return;
// Check if already loaded
if (window.customElements?.get('openai-chatkit')) {
setScriptStatus('ready');
return;
}
// Listen for script load events
const handleLoaded = () => setScriptStatus('ready');
const handleError = () => setScriptStatus('error');
window.addEventListener('chatkit-script-loaded', handleLoaded);
window.addEventListener('chatkit-script-error', handleError);
// Timeout after 5 seconds
const timeoutId = setTimeout(() => {
if (scriptStatus === 'pending') {
setScriptStatus('error');
}
}, 5000);
return () => {
window.removeEventListener('chatkit-script-loaded', handleLoaded);
window.removeEventListener('chatkit-script-error', handleError);
clearTimeout(timeoutId);
};
}, [scriptStatus]);
// Only render ChatKit when script ready
{isOpen && scriptStatus === 'ready' && (
<ChatKit control={control} />
)}
Evidence: robolearn-interface/src/components/ChatKitWidget/index.tsx:67-113
When: Agent needs to know what page user is viewing
Implementation:
const getPageContext = useCallback(() => {
if (typeof window === 'undefined') return null;
// Extract meta tags
const metaDescription = document.querySelector('meta[name="description"]')
?.getAttribute('content') || '';
// Find main content
const mainContent = document.querySelector('article') ||
document.querySelector('main') ||
document.body;
// Extract headings
const headings = Array.from(mainContent.querySelectorAll('h1, h2, h3'))
.slice(0, 5)
.map(h => h.textContent?.trim())
.filter(Boolean)
.join(', ');
return {
url: window.location.href,
title: document.title,
path: window.location.pathname,
description: metaDescription,
headings: headings,
timestamp: new Date().toISOString(),
};
}, []);
Evidence: robolearn-interface/src/components/ChatKitWidget/index.tsx:121-151
When: Users want to ask questions about selected content
Implementation:
// Detect text selection
useEffect(() => {
const handleSelection = () => {
const selection = window.getSelection();
if (!selection || selection.rangeCount === 0) {
setSelectedText('');
setSelectionPosition(null);
return;
}
const selectedText = selection.toString().trim();
if (selectedText.length > 0) {
setSelectedText(selectedText);
// Get selection position
const range = selection.getRangeAt(0);
const rect = range.getBoundingClientRect();
setSelectionPosition({
x: rect.left + rect.width / 2,
y: rect.top - 10,
});
}
};
document.addEventListener('selectionchange', handleSelection);
document.addEventListener('mouseup', handleSelection);
return () => {
document.removeEventListener('selectionchange', handleSelection);
document.removeEventListener('mouseup', handleSelection);
};
}, []);
// Send selected text
const handleAskSelectedText = useCallback(async () => {
const pageContext = getPageContext();
const messageText = `Can you explain this from "${pageContext.title}":\n\n"${selectedText}"`;
if (!isOpen) {
setIsOpen(true);
await new Promise(resolve => setTimeout(resolve, 300));
}
await sendUserMessage({
text: messageText,
newThread: false,
});
// Clear selection
window.getSelection()?.removeAllRanges();
setSelectedText('');
setSelectionPosition(null);
}, [selectedText, isOpen, sendUserMessage, getPageContext]);
Evidence: robolearn-interface/src/components/ChatKitWidget/index.tsx:153-187, 273-331
When: Authentication tokens stored in httpOnly cookies (cannot be read by JavaScript)
Implementation:
// app/api/chatkit/route.ts
import { NextRequest, NextResponse } from "next/server";
import { cookies } from "next/headers";
const API_BASE = process.env.NEXT_PUBLIC_API_URL || "http://localhost:8000";
export async function POST(request: NextRequest) {
const cookieStore = await cookies();
// Read httpOnly cookie (only accessible server-side)
const idToken = cookieStore.get("taskflow_id_token")?.value;
if (!idToken) {
return NextResponse.json({ error: "Not authenticated" }, { status: 401 });
}
// Build target URL - note: ChatKit endpoint is at /chatkit, NOT /api/chatkit
const url = new URL("/chatkit", API_BASE);
try {
const body = await request.text();
// Forward request with Authorization header
const response = await fetch(url.toString(), {
method: "POST",
headers: {
Authorization: `Bearer ${idToken}`,
"Content-Type": "application/json",
// Forward custom headers
"X-User-ID": request.headers.get("X-User-ID") || "",
"X-Page-URL": request.headers.get("X-Page-URL") || "",
},
body: body || undefined,
});
// Handle SSE streaming responses
if (response.headers.get("content-type")?.includes("text/event-stream")) {
return new Response(response.body, {
status: response.status,
headers: {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Connection": "keep-alive",
},
});
}
// Return JSON for non-streaming responses
const data = await response.json().catch(() => null);
return NextResponse.json(data, { status: response.status });
} catch (error) {
console.error("[ChatKit Proxy] Error:", error);
return NextResponse.json({ error: "ChatKit proxy request failed" }, { status: 500 });
}
}
Evidence: web-dashboard/src/app/api/chatkit/route.ts
When: ChatKit script must load before React hydration
Implementation:
// app/layout.tsx
import Script from "next/script";
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang="en">
<head>
{/* MUST be in <head> with beforeInteractive for web components */}
<Script
src="https://cdn.platform.openai.com/deployments/chatkit/chatkit.js"
strategy="beforeInteractive"
/>
</head>
<body>{children}</body>
</html>
);
}
In ChatKit Widget - wait for custom element:
// components/chat/ChatKitWidget.tsx
const [scriptStatus, setScriptStatus] = useState<'pending' | 'ready' | 'error'>(
isBrowser && window.customElements?.get('openai-chatkit') ? 'ready' : 'pending'
);
useEffect(() => {
if (!isBrowser) return;
// Already registered
if (window.customElements?.get('openai-chatkit')) {
setScriptStatus('ready');
return;
}
// Wait for custom element to be defined
customElements.whenDefined('openai-chatkit').then(() => {
setScriptStatus('ready');
});
}, []);
// Only render when ready
{isOpen && scriptStatus === 'ready' && <ChatKit control={control} />}
Evidence: web-dashboard/src/app/layout.tsx, web-dashboard/src/components/chat/ChatKitWidget.tsx:42-93
When: Using useChatKit with httpOnly cookie proxy
Implementation:
const chatkitProxyUrl = "/api/chatkit"; // Proxy handles auth
const { control, sendUserMessage } = useChatKit({
api: {
url: chatkitProxyUrl,
domainKey: domainKey,
// Custom fetch - auth handled by proxy, inject context
fetch: async (input: RequestInfo | URL, options?: RequestInit) => {
const url = typeof input === 'string' ? input : input.toString();
// Client-side auth check (proxy will verify token)
if (!isAuthenticated) {
throw new Error('User must be logged in');
}
const userId = user.sub;
const pageContext = getPageContext();
// Inject metadata into request body
let modifiedOptions = { ...options } as RequestInit;
if (modifiedOptions.body && typeof modifiedOptions.body === 'string') {
try {
const parsed = JSON.parse(modifiedOptions.body);
if (parsed.type === 'threads.create' && parsed.params?.input) {
parsed.params.input.metadata = {
userId,
userInfo: { id: userId, name: user.name },
pageContext,
...parsed.params.input.metadata,
};
modifiedOptions.body = JSON.stringify(parsed);
} else if (parsed.type === 'threads.run' && parsed.params?.input) {
parsed.params.input.metadata = {
...parsed.params.input.metadata,
userInfo: { id: userId, name: user.name },
pageContext,
};
modifiedOptions.body = JSON.stringify(parsed);
}
} catch { /* Ignore non-JSON */ }
}
return fetch(url, {
...modifiedOptions,
credentials: 'include', // Include cookies for proxy auth
headers: {
...modifiedOptions.headers,
'X-User-ID': userId,
'X-Page-URL': pageContext?.url || '',
'Content-Type': 'application/json',
},
});
},
},
});
Evidence: web-dashboard/src/components/chat/ChatKitWidget.tsx:168-296
History Not in Prompt: Agent doesn't remember conversation
Context Not Transmitted: Agent doesn't receive user/page context
Script Not Loaded: ChatKit component fails to render
Auth Headers Missing: Backend rejects requests
Database Not Warmed: First request takes 7+ seconds
httpOnly Cookies Not Accessible (Next.js): Can't read auth token from JavaScript
document.cookie returns nothing, "id_token not found"Script Loading Too Late (Next.js): "ChatKit web component is unavailable"
afterInteractive strategy loads after React hydrationbeforeInteractive in <head> within layout.tsxWrong Backend Endpoint: 404 on ChatKit requests
/api/proxy/chatkit returns 404/api/ prefix, but ChatKit is at /chatkitMCP Tools Missing Auth Token: Agent calls MCP tools without credentials
E402 Linter Error: Imports after load_dotenv()
# noqa: E402 to intentional imports after dotenvcontext: dict[str, Any] but ChatKit SDK passes RequestContext objectcontext directly, don't wrap it in RequestContext constructorInput should be a valid dictionary [input_value=RequestContext(...)]--reload flag with uvicorn, but be aware it's not 100% reliableWhen: MCP tools need to call authenticated APIs
Problem: MCP protocol doesn't forward auth headers. Tools need user credentials.
Solution: Pass auth credentials via agent system prompt. LLM includes them in every tool call.
Implementation:
# 1. Extract token at endpoint
@app.post("/chatkit")
async def chatkit_endpoint(request: Request):
auth_header = request.headers.get("Authorization")
access_token = None
if auth_header and auth_header.startswith("Bearer "):
access_token = auth_header[7:] # Remove "Bearer " prefix
# Add to metadata
metadata["access_token"] = access_token
# 2. System prompt template
SYSTEM_PROMPT = """You are Assistant.
## Authentication Context
- User ID: {user_id}
- Access Token: {access_token}
CRITICAL: When calling ANY MCP tool, you MUST ALWAYS include:
- user_id: "{user_id}"
- access_token: "{access_token}"
{rest_of_prompt}
"""
# 3. Format with credentials
instructions = SYSTEM_PROMPT.format(
user_id=context.user_id,
access_token=context.metadata.get("access_token", ""),
...
)
Flow:
Frontend (Authorization: Bearer <jwt>)
→ /chatkit endpoint (extract token)
→ ChatKit server (add to metadata)
→ Agent prompt (include credentials)
→ LLM (includes in tool params)
→ MCP tool (receives token)
→ API call (Authorization header)
Evidence: packages/api/src/taskflow_api/main.py, packages/api/src/taskflow_api/services/chat_agent.py
When: ChatKit needs its own database schema/connection
Implementation:
# config.py - Ignore ChatKit env vars in main Settings
class Settings(BaseSettings):
model_config = SettingsConfigDict(
extra="ignore", # Ignore TASKFLOW_CHATKIT_* vars
)
@property
def chat_enabled(self) -> bool:
return os.getenv("TASKFLOW_CHATKIT_DATABASE_URL") is not None
# chatkit_store/config.py - Separate config for ChatKit
class StoreConfig(BaseSettings):
model_config = SettingsConfigDict(
env_prefix="TASKFLOW_CHATKIT_", # Reads TASKFLOW_CHATKIT_DATABASE_URL
)
database_url: str
schema_name: str = "taskflow_chat" # Separate schema
Evidence: packages/api/src/taskflow_api/config.py, packages/api/src/taskflow_api/chatkit_store/config.py
respond() methoduseChatKit basic configurationonResponseStart / onResponseEnd handlersonEffect for fire-and-forget client updatesProgressUpdateEvent for loading states.widget files)widgets.onAction handleraction() method in ChatKitServersendCustomAction() for widget updatesThreadItemReplacedEventspecs/007-chatkit-server/spec.mdspecs/008-chatkit-ui-widget/spec.mdreferences/nextjs-httponly-cookie-proxy.md, references/nextjs-script-loading.mdblueprints/openai-chatkit-advanced-samples-main/chatkit-streaming - Response lifecycle, progress updates, client effectschatkit-actions - Interactive widgets, entity tagging, composer toolsxiaohongshu
technical spec
product ux expert
database patterns
Conduct multi-agent task orchestration and workflow coordination.
Initialize project with Conductor artifacts (product definition,
Expert in web animations, transitions, and motion design using Framer Motion and CSS
Creates Mermaid and ASCII diagrams for flowcharts, architecture, ERDs, state machines, mindmaps, and more. Use when user mentions diagram, flowchart, mermaid, ASCII diagram, text diagram, terminal diagram, visualize, C4, mindmap, architecture diagram, sequence diagram, ERD, or needs visual docume...
PostgreSQL bindings for H3 hexagonal grid system. Use when working with H3 cells in Postgres, including spatial indexing, geometry/geography integration, and raster analysis.
Context-Driven Development skill for projects using Conductor. Use this skill when you detect a `conductor/` directory in the project, when working on tasks defined in a `plan.md` file, or when the user asks about tracks, specs, or plans. Automatically applies TDD workflow, tracks task completion...
Display project status, active tracks, and next actions
Official Stakpak application containerization standard operating procedure, a step-by-step guidline to properly dockerize applications. This is a rule book curated by the Stakpak Team.
Generate, edit, and beat-sync AI video with leading models in one workspace.
The world's fastest calendar for remote work
Transform Your Design with AI Designer by ImgCreator.ai
Revolutionizing Video Production with AI-Powered Creativity
Extend an image past the frame and let AI fill the new aspect ratio.
Discover your celebrity doppelgänger with StarByFace!
ChainClarity explains 700+ crypto whitepapers in plain English, with layered summaries, comparisons, research tools, alerts, and a $4.99 Pro plan.
Opus.ai: Revolutionize Your Web Experience