Skip to Content

Model

Test a model with questions

Ask questions in the playground, save good answers as definitions and measure how well an AI model answers.

Check that Bryn answers questions about your model correctly before your team and customers ask them.

Before you start

  • Your role has the datamart View permission. To save, run tests or add questions, it also needs Edit.
  • Your plan includes semantic models.
  • Your company has its own AI model in Settings > LLMs, with Custom AI in use and a Default model set. Without one, Playground is greyed out. Databrain AI does not answer here.
  • The datamart is the datamart of a workspace you can open. If no workspace uses it, the playground answers “You do not have access to this semantic model.”

Open the test pages

  • From Data Studio > Model status, select Open playground, or the Playground or Sample set icon in a datamart’s row.
  • From Data Studio > Semantic Models, select the Open playground icon, or Open sample set in the row menu.

The pages share four tabs: Definitions, Sample Set, Playground and Model tests.

Ask a question in the playground

  1. Type a question the way a customer would, for example What is our total revenue and order count?, and send it.

    Playground for Subscription metrics with the question What is our total revenue and order count? in the highlighted message box, and Sonnet 5.5 picked as the model
    Ask in the playground the way a customer would.
  2. Read the answer: Generated SQL and the result rows. Tables used shows below the rows when the answer names its tables.

    Highlighted playground answer card with the Generated SQL for total revenue and order count, the result 434493 and 240, and the thumbs up, thumbs down and Open definitions buttons
    Check the SQL and the rows before you save the answer.
  3. If the answer is right, select the thumbs up (Save as definition). You see Saved as a definition, then Definitions opens with your question marked Unverified.

    Definitions tab of Subscription metrics with the four tabs Definitions, Sample Set, Playground and Model tests, and the highlighted definition What is our total revenue and order count? marked Unverified
    A saved answer becomes a definition, marked Unverified.
  4. If it is wrong, select the thumbs down and pick what went wrong: Reasoning is not accurate, Tables/Columns selected are incorrect, Generated SQL is incorrect or More.... Edit the SQL or add information, then select Save and regenerate.

With Database tenancy, pick a customer first.

Add sample questions

Sample questions pair a question with the SQL you expect. Bryn uses them to fill in table, column and join descriptions.

  1. Open Sample Set and select Add sample question.

    Sample Set page with No sample questions yet and the Add sample question button highlighted, and AutoFill at the top.
    Add sample question pairs a question with the SQL you expect.
  2. Enter the Question and the SQL query (10 to 500 characters). Or select Import from a file to load a CSV or JSON file.

  3. Select Save.

  4. Select AutoFill to update the descriptions from your samples.

AutoFill does not work on semantic models that you edit in the studio.

Review definitions

Definitions holds the questions with validated SQL that Bryn can answer directly.

  • Example questions shows each question as Valid or Unverified. Select Generate with AI to create 10 to 100 questions. Generating takes 1 to 2 minutes.
  • Feedback lists answers your users flagged, with FEEDBACK TYPE and CONFIDENCE.
  • Accuracy tests runs a benchmark. Pick an AI model, then select Run benchmark. Add questions with Add question.

Run model tests

Model tests ask an AI model every saved definition and mark each one Passed or Failed.

  1. Open Model tests.

  2. Pick an AI model in AI model to test with. The list shows the models your company connected in Settings > LLMs, by name.

  3. Select Run tests. It can take a minute.

    Model tests tab with Anthropic picked as the AI model and the Run tests button highlighted, above No model tests yet
    Pick one of your connected models, then Run tests.
  4. Read STATUS for each question. LAST RUN ON shows when the model last answered it.

    Model tests table with the question What is our total revenue and order count? and its highlighted status Passed, run on Nov 19, 2026
    Each saved question gets Passed or Failed for the model you picked.

If something goes wrong

  • Playground is greyed out. Connect your own model in Settings > LLMs, select Use your own keys and pick a Default model.
  • You do not have access to this semantic model. No workspace you can open uses this datamart. Create a workspace with the datamart, or add it to one, then open the playground again.
  • The published model cannot answer this. The question needs a field or sort the model does not have. Ask it another way, or add the field to the model in the studio.
  • Couldn’t save the sample question. Check the SQL and try again.
  • Run tests keeps showing Running... with no result. Reload the page and run the tests again.

On this page