Structured outputs

Workflows regularly require getting structured outputs (JSON) from LLMs. GenSX supports an easy way to get structured outputs from an LLM by using the GSXChatCompletion component and Zod. However, you can also use the response_format prop to get a structured output from OpenAI models without using GenSX helpers, just like you would when using the OpenAI SDK or API.

This guide shows how to use structured outputs both with and without the helpers that GenSX provides. You can find the complete example code in the structured outputs example .

Structured outputs with GSXChatCompletion

When using the GSXChatCompletion component, you can define the Zod schema for the output format you want and GenSX will handle the rest.

Start by defining the Zod schema for the output format you want:

import { z } from "zod";
 
const ExtractEntitiesSchema = z.object({
  people: z.array(z.string()),
  places: z.array(z.string()),
  organizations: z.array(z.string()),
});

Then define the input and output types for the component that will be used to get the structured output:

interface ExtractEntitiesProps {
  text: string;
}
 
type ExtractEntitiesOutput = z.infer<typeof ExtractEntitiesSchema>;

Finally, define the component and set the outputSchema prop on the GSXChatCompletion component to get the output matching that schema:

const ExtractEntities = gensx.Component<
  ExtractEntitiesProps,
  ExtractEntitiesOutput
>("ExtractEntities", ({ text }) => {
  const prompt = `Please review the following text and extract all the people, places, and organizations mentioned.
 
  <text>
  ${text}
  </text>
 
  Please return json with the following format:
  {
    "people": ["person1", "person2", "person3"],
    "places": ["place1", "place2", "place3"],
    "organizations": ["org1", "org2", "org3"]
  }`;
  return (
    <OpenAIProvider apiKey={process.env.OPENAI_API_KEY}>
      <GSXChatCompletion
        model="gpt-4o-mini"
        messages={[
          {
            role: "user",
            content: prompt,
          },
        ]}
        outputSchema={ExtractEntitiesSchema}
      ></GSXChatCompletion>
    </OpenAIProvider>
  );
});

The GSXChatCompletion component will return the structured output directly matching the type of the ExtractEntitiesSchema with no extra parsing required.

Structured output strategies

When using structured outputs with OpenAI models, GenSX uses the response_format parameter to return a structured output. However, not all models support structured outputs out of the box. For non-OpenAI models, structured outputs are handled by converting the schema into a tool and forcing the model to call that tool.

GenSX looks at the baseURL of the OpenAIProvider to determine which approach to use but you can manually control this behavior by setting the structuredOutputStrategy prop. There are three available options:

• default: Picks the best strategy based on the baseURL of the OpenAIProvider.
• tools: Converts the schema into a tool to return a structured output.
• response_format: Uses the response_format parameter to return a structured output.

Retries

In the case that the model doesn’t return valid JSON matching the schema, the task will be retried up to 3 times by default. You can change this behavior by setting the retry prop.

<GSXChatCompletion
  model="gpt-4o-mini"
  messages={[
    {
      role: "user",
      content: prompt,
    },
  ]}
  outputSchema={ExtractEntitiesSchema}
  retry={{
    maxAttempts: 1,
  }}
></GSXChatCompletion>

Structured outputs without GenSX helpers

You can also use the response_format prop to get a structured output from OpenAI models without using GenSX helpers, just like you would when using the OpenAI SDK or API. The main difference is that you will need to parse the response yourself.

Using the same ExtractEntitiesSchema from the previous example, you can define the component and set the response_format prop to the ExtractEntitiesSchema using the zodResponseFormat helper.

import { zodResponseFormat } from "openai/helpers/zod";
 
const ExtractEntitiesWithoutHelpers = gensx.Component<
  ExtractEntitiesProps,
  ExtractEntitiesOutput
>("ExtractEntitiesWithoutHelpers", ({ text }) => {
  const prompt = `Please review the following text and extract all the people, places, and organizations mentioned.
 
  <text>
  ${text}
  </text>
 
  Please return json with the following format:
  {
    "people": ["person1", "person2", "person3"],
    "places": ["place1", "place2", "place3"],
    "organizations": ["org1", "org2", "org3"]
  }`;
  return (
    <OpenAIProvider apiKey={process.env.OPENAI_API_KEY}>
      <ChatCompletion
        model="gpt-4o-mini"
        messages={[
          {
            role: "user",
            content: prompt,
          },
        ]}
        response_format={zodResponseFormat(ExtractEntitiesSchema, "entities")}
      >
        {(response: string) => {
          return ExtractEntitiesSchema.parse(JSON.parse(response));
        }}
      </ChatCompletion>
    </OpenAIProvider>
  );
});

This example uses the ChatCompletion component which always returns a string but you could also use the GSXChatCompletion component or OpenAIChatCompletion component which returns the full response object from OpenAI.

Running the example

You can run the examples above using the following code:

const workflow = gensx.Workflow("ExtractEntitiesWorkflow", ExtractEntities);
 
const result = await workflow.run({
  text: "John Doe is a software engineer at Google.",
});
 
console.log(result);

This will output the following:

{
  "people": ["John Doe"],
  "places": [],
  "organizations": ["Google"]
}