How to Build an AI Coding Agent with Streaming Responses: Complete Tutorial

AI coding agents are transforming how developers work. Instead of just autocompleting code, modern agents can read your codebase, understand requirements, write and test code, and even debug issues autonomously.

In this tutorial, you will build a simple but powerful AI coding assistant that streams responses, supports multi-turn conversations, and can be extended with tool calling. We will use the OpenAI-compatible API format -- which means it works with any provider on Qubax AI, including GPT-5.6, Claude, DeepSeek-V4-Flash, and more.

Prerequisites

Node.js 20+ or Python 3.11+
Basic familiarity with async/await
An API key from Qubax AI or any OpenAI-compatible provider

We will build this in both Python and JavaScript so you can choose your preferred language.

What You Will Build

By the end of this tutorial, you will have a terminal-based AI coding assistant that:

Accepts natural language coding questions
Streams responses token-by-token for a real-time feel
Maintains conversation context across multiple turns
Supports system prompts for custom behavior
Can be extended with function calling for tool use

Step 1: Project Setup

Python Version

bash

mkdir ai-coding-agent && cd ai-coding-agent
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install openai

JavaScript Version

bash

mkdir ai-coding-agent && cd ai-coding-agent
npm init -y
npm install openai

Step 2: Basic Streaming Chat

Let us start with the simplest possible streaming implementation. We will use the OpenAI SDK pointed at Qubax AI for maximum model flexibility.

Python

python

from openai import OpenAI

client = OpenAI(
    api_key='your-api-key',
    base_url='https://api.qubax.ai/v1'  # Or any OpenAI-compatible endpoint
)

def chat_stream(prompt, model='gpt-5.6'):
    stream = client.chat.completions.create(
        model=model,
        messages=[{'role': 'user', 'content': prompt}],
        stream=True
    )
    
    for chunk in stream:
        if chunk.choices[0].delta.content is not None:
            print(chunk.choices[0].delta.content, end='', flush=True)
    print()  # New line at end

chat_stream('Write a Python function to check if a number is prime')

JavaScript

javascript

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'your-api-key',
  baseURL: 'https://api.qubax.ai/v1'
});

async function chatStream(prompt, model = 'gpt-5.6') {
  const stream = await client.chat.completions.create({
    model: model,
    messages: [{ role: 'user', content: prompt }],
    stream: true,
  });

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content || '';
    process.stdout.write(content);
  }
  console.log();
}

await chatStream('Write a JavaScript function to check if a number is prime');

Run this and you will see the response appear word by word -- just like ChatGPT. That is streaming in action.

Step 3: Adding Conversation Memory

A single question and answer is not very useful for a coding assistant. Let us add multi-turn conversation support so the AI remembers previous context.

Python

python

from openai import OpenAI
import sys

client = OpenAI(
    api_key='your-api-key',
    base_url='https://api.qubax.ai/v1'
)

class CodingAssistant:
    def __init__(self, model='gpt-5.6'):
        self.model = model
        self.messages = [
            {
                'role': 'system',
                'content': 'You are an expert coding assistant. Provide clear, well-commented code with explanations. Always use best practices.'
            }
        ]
    
    def chat(self, user_input):
        self.messages.append({'role': 'user', 'content': user_input})
        
        stream = client.chat.completions.create(
            model=self.model,
            messages=self.messages,
            stream=True
        )
        
        full_response = ''
        for chunk in stream:
            if chunk.choices[0].delta.content is not None:
                text = chunk.choices[0].delta.content
                full_response += text
                print(text, end='', flush=True)
        
        print()
        self.messages.append({'role': 'assistant', 'content': full_response})
        return full_response

# Interactive loop
assistant = CodingAssistant()
print('AI Coding Assistant (type quit to exit)')
print('-' * 50)

while True:
    user_input = input('\nYou: ').strip()
    if user_input.lower() in ['quit', 'exit', 'q']:
        break
    print('\nAssistant: ', end='')
    assistant.chat(user_input)

Now the assistant remembers the full conversation. You can ask follow-up questions like "now add error handling" or "explain that regex" and it will understand the context.

Step 4: Adding Tool Calling

Real coding agents do not just write code -- they can execute it, read files, and take actions. Let us add function calling so our agent can read files from disk.

Python

python

import json
import os

# Define tools the AI can use
tools = [
    {
        'type': 'function',
        'function': {
            'name': 'read_file',
            'description': 'Read the contents of a file',
            'parameters': {
                'type': 'object',
                'properties': {
                    'path': {
                        'type': 'string',
                        'description': 'The file path to read'
                    }
                },
                'required': ['path']
            }
        }
    },
    {
        'type': 'function',
        'function': {
            'name': 'list_directory',
            'description': 'List files in a directory',
            'parameters': {
                'type': 'object',
                'properties': {
                    'path': {
                        'type': 'string',
                        'description': 'The directory path to list'
                    }
                },
                'required': ['path']
            }
        }
    }
]

def execute_tool(name, args):
    if name == 'read_file':
        try:
            with open(args['path'], 'r') as f:
                return f.read()[:5000]  # Limit to 5000 chars
        except Exception as e:
            return f'Error: {str(e)}'
    elif name == 'list_directory':
        try:
            return json.dumps(os.listdir(args['path']))
        except Exception as e:
            return f'Error: {str(e)}'
    return 'Unknown tool'

class ToolEnabledAssistant(CodingAssistant):
    def chat(self, user_input):
        self.messages.append({'role': 'user', 'content': user_input})
        
        while True:  # Loop until no more tool calls
            response = client.chat.completions.create(
                model=self.model,
                messages=self.messages,
                tools=tools,
                stream=False  # Non-streaming for tool calls
            )
            
            msg = response.choices[0].message
            self.messages.append(msg.model_dump())
            
            if not msg.tool_calls:
                print(msg.content)
                return msg.content
            
            # Execute each tool call
            for tool_call in msg.tool_calls:
                args = json.loads(tool_call.function.arguments)
                result = execute_tool(tool_call.function.name, args)
                print(f'  [Tool: {tool_call.function.name}({args})]')
                self.messages.append({
                    'role': 'tool',
                    'tool_call_id': tool_call.id,
                    'content': str(result)
                })

assistant = ToolEnabledAssistant()
assistant.chat('Read the file main.py in the current directory and suggest improvements')

Now your AI agent can browse your codebase, read files, and provide context-aware suggestions.

Step 5: Choosing the Right Model

One of the great things about using an OpenAI-compatible API is model flexibility. Different models excel at different tasks:

Model	Best For	Speed	Cost
GPT-5.6	General coding, complex reasoning	Fast	Medium
Claude Opus 5	Long-context analysis, careful reasoning	Medium	High
DeepSeek-V4-Flash	High-volume coding, cost-sensitive apps	Very Fast	Low
Gemini 3 Pro	Multimodal tasks, large context	Fast	Medium

You can switch between models instantly by changing one parameter:

python

# Use DeepSeek for cost-effective coding
assistant = CodingAssistant(model='deepseek-v4-flash')

# Switch to Claude for complex reasoning
assistant = CodingAssistant(model='claude-opus-5')

Check Qubax AI models for the full list of available models and pricing.

Step 6: Production Tips

Here are key considerations when deploying your AI coding agent to production:

Error Handling

python

from openai import APIError, RateLimitError, APIConnectionError

def safe_chat(assistant, prompt, retries=3):
    for attempt in range(retries):
        try:
            return assistant.chat(prompt)
        except RateLimitError:
            wait = 2 ** attempt
            print(f'Rate limited. Retrying in {wait}s...')
            time.sleep(wait)
        except APIConnectionError:
            print(f'Connection error. Retrying...')
            time.sleep(1)
        except APIError as e:
            print(f'API error: {e}')
            return None
    return None

Token Management

Long conversations can hit context limits. Implement message pruning:

python

def prune_messages(messages, max_messages=20):
    # Always keep the system prompt
    system = [m for m in messages if m['role'] == 'system']
    conversation = [m for m in messages if m['role'] != 'system']
    return system + conversation[-max_messages:]

Cost Control

Set up usage monitoring and budget alerts. With Qubax AI, you can set spending limits per API key and monitor usage in real-time.

Next Steps

Now that you have a working AI coding agent, here are ideas for extending it:

Add more tools: git operations, running tests, web search
Implement code execution: sandbox and run generated code safely
Add a web UI: build a chat interface with React or Vue
Integrate with your IDE: use Language Server Protocol or editor extensions
Add RAG: let the agent search your documentation before answering

Check out the Qubax AI docs for more tutorials on advanced features like RAG, embeddings, and multi-agent orchestration.

FAQ

Do I need an OpenAI API key for this tutorial?

No. This tutorial uses the OpenAI SDK format, but you can point it at any OpenAI-compatible provider. Qubax AI provides access to 20+ providers with a single API key, so you can try GPT-5.6, Claude, DeepSeek, and others without managing multiple accounts.

Which model is best for coding tasks?

For complex coding tasks, GPT-5.6 and Claude Opus 5 are excellent choices. For cost-sensitive applications, DeepSeek-V4-Flash offers strong coding performance at a fraction of the cost. Compare options on our models page.

Can I use this in production?

Yes. The patterns in this tutorial are production-ready. Just add proper error handling, rate limiting, token management, and cost monitoring. For production deployments, consider using Qubax AI which handles failover and credential pooling automatically.

How do I handle very long conversations?

Implement message pruning (keep only the last N messages), use context window compression, or switch to a model with a larger context window. Most 2026 models support 128K-2M token contexts, but longer contexts cost more.

Can I add custom tools beyond file reading?

Absolutely. You can define any function as a tool -- database queries, API calls, code execution, web scraping. Just define the JSON schema, implement the handler, and add it to the tools list.

Ready to build your own AI coding agent? [Get started with Qubax AI](https://qubax.ai) for instant access to GPT-5.6, Claude, DeepSeek, and 20+ other models through a single API. [Read the docs](https://qubax.ai/docs) for more tutorials.

How to Build an AI Coding Agent with Streaming Responses: Complete Tutorial

Prerequisites

What You Will Build

Step 1: Project Setup

Python Version

JavaScript Version

Step 2: Basic Streaming Chat

Python

JavaScript

Step 3: Adding Conversation Memory

Python

Step 4: Adding Tool Calling

Python

Step 5: Choosing the Right Model

Step 6: Production Tips

Error Handling

Token Management

Cost Control

Next Steps

FAQ

Do I need an OpenAI API key for this tutorial?

Which model is best for coding tasks?

Can I use this in production?

How do I handle very long conversations?

Can I add custom tools beyond file reading?

Article tags

Related articles