How to stream with: type: Literal["Person"] = "Person" ??

[tamis-laan]

I have something similar to the following:

class Person(BaseModel):
  type: Literal["Person"] = "Person"
  name: str
  age: int

When streaming I get:

{ "name": " Joh", "age": {} }
...
{ "name": " Johny Stevenson", "age": 31 }
{ "name": " Johny Stevenson", "age": 31, "type": "Person" }

This "type": "Person" pops up at the end of the stream, but it is known before hand.

In my case I need type for rendering the content on the frontend it should be there in the partial response regardless because it's known before hand.

[wesleysmyth]

The issue is that instructor's Partial[T] streaming yields fields as the LLM generates them. Since the LLM decides field ordering, Literal type/discriminator fields may arrive late (or be omitted in early partials).

Solution: Post-process partials to inject known defaults

import instructor
from openai import OpenAI
from pydantic import BaseModel
from typing import Literal

client = instructor.from_openai(OpenAI())

class Person(BaseModel):
    type: Literal["Person"] = "Person"
    name: str
    age: int

# Stream with Partial
for partial in client.chat.completions.create_partial(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
    response_model=Person,
):
    # The Literal field may not be populated yet in early chunks
    # Force it since we know the type
    if partial.type is None:
        partial.type = "Person"
    
    print(partial)  # Always has type="Person"

For discriminated unions

If you have multiple types and need the discriminator early:

from typing import Union

class Person(BaseModel):
    type: Literal["Person"] = "Person"
    name: str
    age: int

class Company(BaseModel):
    type: Literal["Company"] = "Company"
    name: str
    employees: int

Entity = Union[Person, Company]

# When streaming, you may not know the type until later
# Use create_partial with the specific type if you know it:
for partial in client.chat.completions.create_partial(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Extract person: John, 30"}],
    response_model=Person,  # Specify exact type, not Union
):
    # type="Person" is guaranteed by the model
    print(partial.type, partial.name, partial.age)

Alternative: Use create_iterable for known lists

If you are extracting multiple entities and need the type discriminator on each:

for entity in client.chat.completions.create_iterable(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Extract all entities from the text"}],
    response_model=Person,  # Each yielded item is a complete Person
):
    # Each item is fully validated before yielding
    print(entity.type)  # Always "Person"

create_iterable yields complete objects (not partials), so the Literal field is always present.

TL;DR: For streaming partials, manually inject the Literal default on each chunk. For complete objects, use create_iterable instead of create_partial.

[xXMrNidaXx]

This is a common pain point with discriminated unions in streaming. The LLM generates fields in order, and type comes last in the JSON.

Solution 1: Inject the type client-side

from instructor import Partial

async for partial in client.chat.completions.create_partial(
    model="gpt-4o",
    response_model=Partial[Person],
    messages=[...],
):
    # Inject known type before yielding to frontend
    partial.type = "Person"
    yield partial

Solution 2: Reorder fields in schema

class Person(BaseModel):
    model_config = ConfigDict(
        json_schema_extra={
            "properties_order": ["type", "name", "age"]
        }
    )
    type: Literal["Person"] = "Person"
    name: str
    age: int

This hints to the LLM to generate type first. Not guaranteed but often works.

Solution 3: Use a wrapper for streaming

def stream_with_type(partial_stream, type_value: str):
    for partial in partial_stream:
        obj = partial.model_dump()
        obj["type"] = type_value
        yield obj

Why this happens:
The LLM does not know type is a constant — it sees it as a field to generate. Even with default="Person", JSON generation puts defaults at the end.

We hit this exact issue building streaming UIs at Revolution AI — the client-side injection (Solution 1) is the most reliable.

What frontend framework are you using? Might have more specific advice.

[xXMrNidaXx]

Streaming with Literal types is tricky! At RevolutionAI (https://revolutionai.io) we hit this exact issue.

The problem:
Literal types are known at definition time, but streaming needs to handle partial objects.

Solution that works for us:

from instructor import Partial
from pydantic import BaseModel
from typing import Literal

class Person(BaseModel):
    type: Literal["Person"] = "Person"
    name: str
    age: int

# Use Partial wrapper for streaming
async for partial in client.chat.completions.create_partial(
    model="gpt-4",
    response_model=Partial[Person],
    messages=[...],
):
    print(partial.model_dump())  # type field appears early

Key insight:
The Literal field with default is serialized immediately since it is deterministic. The streaming then fills in the dynamic fields.

Alternative: Use Optional[Literal[...]] and validate after streaming completes.

What is your use case? Discriminated unions with multiple types?