# RtfTokenizer

Tokenizes RTF text into braces, control words, control symbols, text, and binary payloads.

Class 

`OfficeIMO.Rtf.Syntax`

**Namespace**

`OfficeIMO.Rtf.Syntax`

**Assembly**

`OfficeIMO.Rtf`

**Source revision**

`ba882f6b0d07` with working tree changes

**Source**

[RtfTokenizer.cs:13](https://github.com/EvotecIT/OfficeIMO/blob/ba882f6b0d07acddd884b5e7d3086b0cfff0a05a/OfficeIMO.Rtf/Syntax/RtfTokenizer.cs#L13)

**Modifiers**

static

Tokenizes RTF text into braces, control words, control symbols, text, and binary payloads.

## Inheritance

- object
- RtfTokenizer

Filter members 

Show inherited

## Methods

Tokenize 2 overloads

`public static RtfTokenizeResult Tokenize(string rtf)` [#](#method-tokenize-string)

[RtfTokenizer.cs:13](https://github.com/EvotecIT/OfficeIMO/blob/ba882f6b0d07acddd884b5e7d3086b0cfff0a05a/OfficeIMO.Rtf/Syntax/RtfTokenizer.cs#L13)

Tokenizes RTF content.

Parameters

**rtf string required**

Returns

`RtfTokenizeResult`

`public static RtfTokenizeResult Tokenize(string rtf, RtfLoadOptions? options, CancellationToken cancellationToken = default)` [#](#method-tokenize-string-officeimo-rtf-rtfloadoptions-threading-cancellationtoken)

[RtfTokenizer.cs:20](https://github.com/EvotecIT/OfficeIMO/blob/ba882f6b0d07acddd884b5e7d3086b0cfff0a05a/OfficeIMO.Rtf/Syntax/RtfTokenizer.cs#L20)

Tokenizes RTF content using the configured resource limits and cancellation token.

Parameters

**rtf string required**

**options OfficeIMO.Rtf.RtfLoadOptions? required**

**cancellationToken System.Threading.CancellationToken = default optional**

Returns

`RtfTokenizeResult`

## Inherited Methods

`public virtual bool Equals(object? obj)` [#](#method-equals-object)

Inherited from object

Parameters

**obj object? required**

Returns

`bool`

`public virtual int GetHashCode()` [#](#method-gethashcode)

Inherited from object

Returns

`int`

`public Type GetType()` [#](#method-gettype)

Inherited from object

Returns

`Type`

`public virtual string? ToString()` [#](#method-tostring)

Inherited from object

Returns

`string?`

<!-- PowerForge markdown artifact sha256:C0ECDB1BB835E892FEC6979481C30BFFC05A0EBEC3AA53823206F2CC9B456DF6 -->
