Skip to main content
The optimizer finds the jobs your agents repeat, tests cheaper models on real conversations from each job, and tells you which model does the job as well for less. It only recommends. Nothing about your traffic changes until you change the model you send.

​
Before you start

  • Send traffic through Concentrate. A scan reads your requests from the last 3 days.
  • Turn on logging. The optimizer tests models on the prompts and responses in your logs. In Settings, open Logging and turn on Enable customer-controlled logs. See Zero Data Retention & Logging.
  • In an organization, allow it. An owner turns on Allow the optimizer in the organization’s Settings.

​
Run your first test

1

Open the optimizer

In the app, go to Logs → Optimizer.
2

Scan your requests

Choose Scan requests. In an organization, you can first pick which teams to include.The scan groups your recent requests into workflows: the jobs your agents do again and again.
3

Pick the workflows to test

Workflows are listed biggest spend first, and all of them start ticked. Untick any you don’t want to test.Each row shows what testing that workflow costs. The bar at the bottom shows the total cost and time.
4

Run the test

Choose Run test. The optimizer replays a sample of each workflow’s conversations on cheaper models and grades every reply.
5

Read the results

Each workflow ends with one of three results:Under Evidence, open any conversation to compare your current model’s replies with the recommended model’s.
6

Make the switch

Copy the model ID from the result and send it as model in that workflow’s requests.

​
Adjust a test

The defaults work without changes. Open a workflow before you run the test to adjust it.
Each workflow starts with a shortlist of models that cost less than the one it uses today. Remove any you don’t want, or add any other chat model.To test only models from the maker of your current model, open the settings icon beside Scan requests and turn on Only models by the same maker.
Every reply, from every model, takes the same checks: that your app can read it, that it answers what was asked, and a few rules the scan writes for that workflow.
  • A must pass check fails the reply when missed. A tracked check is reported but doesn’t fail it. You can switch a rule between the two, or remove it.
  • Add your own check: a yes/no question about the reply, a phrase it must or must not include, or a word limit.
  • Edit Judge criteria to describe, in your own words, what a good reply looks like.
Open the settings icon beside Scan requests and choose a number under Requests to scan. Scan more when your jobs don’t repeat often. Scan fewer if the app says the scan was too large.
Choose Optimize one call, open a row in Logs, then choose Optimize this call. The request must have succeeded.
Choose Rerun with changes on a finished test. It reopens with the same workflows, models and checks, so you can change any of them and run it again.

​
Billing

Your first scan creates an API key named Optimizer in API Keys. Scans and tests are billed to that key. The cost shown before a test is an estimate. Pausing or revoking the key stops scans and tests until you resume it.

​
If a scan finds nothing to test

The app shows the reason on the scan box. Find the message you see below.
Last modified on November 12, 2026