API Reference
Interface
IReaderTokenCounter
Counts tokens for deterministic chunk budgeting.
Remarks
Implementations used concurrently must be thread-safe. Counts must be deterministic and non-negative.
Usage
This type appears in these public API surfaces even when no hand-authored example is attached directly to the page.
Returned or exposed by
Methods
public abstract Int32 CountTokens(String text) #Returns:
Int32Counts tokens in the supplied text.
Parameters
- text System.String
Properties
public abstract String Id { get; } #Stable counter identifier retained in chunking evidence.