Structured outputs

Workflows regularly require getting structured outputs (JSON) from LLMs. GenSX supports an easy way to get structured outputs from an LLM by using the GSXChatCompletion component and Zod. However, you can also use the response_format prop to get a structured output from OpenAI models without using GenSX helpers, just like you would when using the OpenAI SDK or API.

This guide shows how to use structured outputs both with and without the helpers that GenSX provides. You can find the complete example code in the structured outputs example .

Structured outputs with GSXChatCompletion

When using the GSXChatCompletion component, you can define the Zod schema for the output format you want and GenSX will handle the rest.

Start by defining the Zod schema for the output format you want:


import { z } from "zod";
 
const ExtractEntitiesSchema = z.object({
  people: z.array(z.string()),
  places: z.array(z.string()),
  organizations: z.array(z.string()),
});

Then define the input and output types for the component that will be used to get the structured output:


interface ExtractEntitiesProps {
  text: string;
}
 
type ExtractEntitiesOutput = z.infer<typeof ExtractEntitiesSchema>;

Finally, define the component and set the outputSchema prop on the GSXChatCompletion component to get the output matching that schema:


const ExtractEntities = gensx.Component<
  ExtractEntitiesProps,
  ExtractEntitiesOutput
>("ExtractEntities", ({ text }) => {
  const prompt = `Please review the following text and extract all the people, places, and organizations mentioned.
 
  <text>
  ${text}
  </text>
 
  Please return json with the following format:
  {
    "people": ["person1", "person2", "person3"],
    "places": ["place1", "place2", "place3"],
    "organizations": ["org1", "org2", "org3"]
  }`;
  return (
    <OpenAIProvider apiKey={process.env.OPENAI_API_KEY}>
      <GSXChatCompletion
        model="gpt-4o-mini"
        messages={[
          {
            role: "user",
            content: prompt,
          },
        ]}
        outputSchema={ExtractEntitiesSchema}
      ></GSXChatCompletion>
    </OpenAIProvider>
  );
});

The GSXChatCompletion component will return the structured output directly matching the type of the ExtractEntitiesSchema with no extra parsing required.

In the case that the model doesn’t return valid JSON matching the schema, the task will be retried up to 3 times by default. You can change this behavior by setting the retry prop.


<GSXChatCompletion
  model="gpt-4o-mini"
  messages={[
    {
      role: "user",
      content: prompt,
    },
  ]}
  outputSchema={ExtractEntitiesSchema}
  retry={{
    maxAttempts: 1,
  }}
></GSXChatCompletion>

Structured outputs without GenSX helpers

You can also use the response_format prop to get a structured output from OpenAI models without using GenSX helpers, just like you would when using the OpenAI SDK or API. The main difference is that you will need to parse the response yourself.

Using the same ExtractEntitiesSchema from the previous example, you can define the component and set the response_format prop to the ExtractEntitiesSchema using the zodResponseFormat helper.


import { zodResponseFormat } from "openai/helpers/zod";
 
const ExtractEntitiesWithoutHelpers = gensx.Component<
  ExtractEntitiesProps,
  ExtractEntitiesOutput
>("ExtractEntitiesWithoutHelpers", ({ text }) => {
  const prompt = `Please review the following text and extract all the people, places, and organizations mentioned.
 
  <text>
  ${text}
  </text>
 
  Please return json with the following format:
  {
    "people": ["person1", "person2", "person3"],
    "places": ["place1", "place2", "place3"],
    "organizations": ["org1", "org2", "org3"]
  }`;
  return (
    <OpenAIProvider apiKey={process.env.OPENAI_API_KEY}>
      <ChatCompletion
        model="gpt-4o-mini"
        messages={[
          {
            role: "user",
            content: prompt,
          },
        ]}
        response_format={zodResponseFormat(ExtractEntitiesSchema, "entities")}
      >
        {(response: string) => {
          return ExtractEntitiesSchema.parse(JSON.parse(response));
        }}
      </ChatCompletion>
    </OpenAIProvider>
  );
});

This example uses the ChatCompletion component which always returns a string but you could also use the GSXChatCompletion component or OpenAIChatCompletion component which returns the full response object from OpenAI.

Running the example

You can run the examples above using the following code:


const workflow = gensx.Workflow("ExtractEntitiesWorkflow", ExtractEntities);
 
const result = await workflow.run({
  text: "John Doe is a software engineer at Google.",
});
 
console.log(result);

This will output the following:


{
  "people": ["John Doe"],
  "places": [],
  "organizations": ["Google"]
}