Logging

Start sending data to Gantry to understand the behavior of your model.

The first step to getting value from Gantry is to start gathering data about your model's behavior. Gantry works by accumulating records corresponding to predictions consisting of inputs and outputs. At any subsequent time you can send "feedback," either ground truth any other data, that helps the quality of a prediction. These predictions can also be enriched with tags and projections (derived values) to provide a clearer picture of model behavior.

We will dive into all of those terms in detail in the sections that follow, but the mental model you should for a Gantry record is illustrated below:

12901290

In this example we have a text generation model with a single input and output. Let's review each of the columns along with the role it plays:

ColumnDescription
record_keyA unique and stable identifier for this prediction, can be used to apply feedback at any point in time.
application_nameThe name of this application within Gantry. Roughly speaking, each application has a corresponding "infinite dataframe" consisting of rows like this.
tags.envAn example of how to use our tagging system to indicate that this record was captured in production.
tags.user_typeAnother example of how we can enrich a prediction with data that might not be an input, but helps us understand how the model impacts users.
inputs.prompt.The prompt provided by the user, the model input.
outputs.generationThe output produced by the text generation.
feedback.thumbs_upThis is an example of feedback that is not "ground truth" but merely the opinion of the user, and well suited to assessing how well our users are receiving the model's predictions.
projections.word_countAn example of using projections to "project" a higher dimensional input, raw text, into a scalar that can help us understand the model's behavior more systematically.

Next we will look at each the different types of columns in more detail, starting with predictions.


What’s Next