Cumulative Logprobs (Experimental)

For certain tasks, it can be helpful to inspect the logprobs of an LLM’s output tokens. This can be considered a confidence estimation for the output. For example, if you are generating classification labels, you might want to know how confident the model is in its output.

We store the normalized cumulative logprobs (ranging from 0 to 100) for each output. These can be obtained using the job_results endpoint, and setting include_cumulative_logprobs to True.

This feature is experimental, but we encourage you to utilize it and let us know if you have any feedback. Of course, if you have any questions or need help getting started with this feature, please reach out to us at team@materialized.dev.