Skip to main content

approx_tokens_json

Function approx_tokens_json 

Source
pub fn approx_tokens_json(value: &Value) -> u64
Expand description

Approximate token count for a serialized JSON value.

Reuses a thread-local byte buffer for JSON serialization, so on repeated calls no per-call allocation happens (the buffer grows to the peak size once and then stays). Called on every request through prepare_chat.