> For the complete documentation index, see [llms.txt](https://docs.kognitos.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.kognitos.com/guides/platform/integrations/idp.md).
# Intelligent Document Processing (IDP)
{% hint style="info" %}
The following documentation is for **Intelligent Document Processing (IDP) v4.19.1**.
{% endhint %}
## Overview
Intelligent Document Processing (IDP) extracts structured data from documents using AI. This integration enables automated document analysis, data extraction, and intelligent processing workflows.
## Setup
The following integrations need to be connected to your Kognitos workspace:
* **Intelligent Document Processing (IDP)**
### Steps
Follow these steps to connect the integration in Kognitos:
{% stepper %}
{% step %}
#### Navigate
Using the left navigation menu, go to **Integrations** → **Explore Integrations**.
{% endstep %}
{% step %}
#### Find
Search for the integration and click on it.
{% endstep %}
{% step %}
#### Connect
Click on **Connect** to add a connection to the integration.
{% endstep %}
{% step %}
#### Configure
Add a name for the connection. You'll be prompted for [**authentication**](#authentication) details if needed. Then, click on **Connect**.
{% endstep %}
{% endstepper %}
### Credentials
#### 1. Anthropic API Key
Follow these steps to obtain your **Anthropic API key**:
{% stepper %}
{% step %}
**Log in to the Anthropic Console**
Go to the [**Anthropic Console**](https://console.anthropic.com) and log in with your credentials.
{% endstep %}
{% step %}
**Navigate to API Keys**
Go to **Settings** > **API Keys**. Then click **+ Create Key** in the top right.
{% endstep %}
{% step %}
**Configuration**
Select a workspace and give your key a descriptive name (e.g., "Development Key" or "Production App"). Then, click **Add** to generate your API key.
{% endstep %}
{% step %}
**Copy and Store Your Key**
Copy your API key immediately and store it securely. You won't be able to view it again after closing the dialog.
{% endstep %}
{% endstepper %}
#### 2. OpenAI API Key
Follow these steps to obtain your **OpenAI API key**:
{% stepper %}
{% step %}
**Log In to OpenAI**
Navigate to the [OpenAI Platform](https://auth.openai.com/log-in) and log in with your credentials.
{% endstep %}
{% step %}
**API Keys**
Open **Account Settings**, then navigate to [**API Keys**](https://platform.openai.com/account/api-keys)**.**
{% endstep %}
{% step %}
**Generate a New API Key**
Click **Create new secret key**. Copy the key immediately — it will only be shown once.
{% endstep %}
{% endstepper %}
## Authentication
Use one of the following authentication methods to connect this integration in Kognitos. Each method has its own configuration requirements.
### Connect using API Key
Connect to Anthropic API for document processing.
| Label | Description | Type |
| ------- | --------------------- | ----------- |
| API Key | The Anthropic API key | `sensitive` |
### Connect using Service Account Credentials and Region
Connect to Google Vertex AI (Gemini) API for document processing.
| Label | Description | Type |
| --------------------------- | ----------------------------------------------------- | ----------- |
| Service Account Credentials | The Google service account credentials JSON as string | `sensitive` |
| Region | The Google Cloud region | `text` |
### Connect using API Key
Connect to OpenAI API for document processing.
| Label | Description | Type |
| ------- | ------------------ | ----------- |
| API Key | The OpenAI API key | `sensitive` |
## Actions
The following actions are available in the **Intelligent Document Processing (IDP)** integration:
### 1. Classify documents
Classify text or documents against user-defined topics.
### 2. Extract fields from documents
Extract structured fields from text or documents.
### 3. Extract subdocuments from a document
Extract subdocuments from a document.
### 4. Extract table records from documents
Extract one recurring table across all pages into a merged dataset.
### 5. Extract tables from documents
Extract structured tables from text or documents.
### 6. Merge subdocuments into a document
Merge multiple subdocuments/pages into a single PDF document.
### 7. Parse layout from a document
Parse the structural layout of a document.
### 8. Read content from documents
Read text content from documents.
## Concepts
### Idp classification options
Options for `classify_text_or_documents`.At least one source of classification topics must be provided: `topics` (explicit list), `classification_rules` (file or string to parse), or `prompt` (natural language description).
| Field Name | Description | Type |
| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`topics`](#idp-classification-topic) | Explicit list of `ClassificationTopic` objects. Each topic has `topic_name` (required), `topic_description` (what the topic represents), and `topic_criteria` (specific criteria to evaluate). Highest priority if provided. Example:: topics=\[ ClassificationTopic( topic\_name="Invoice", topic\_description="A billing document", topic\_criteria="Contains line items, totals, " "and payment terms", ), ] Tip: More specific `topic_criteria` produce better classification accuracy. Vague criteria lead to false positives. | `optional[list of idp classification topic]` |
| `prompt` | Classification instructions or natural language context. If no `topics` are provided, the LLM will extract topics from this prompt automatically. | `optional[text]` |
| `classification_rules` | Domain-specific instructions as a file (.txt, .md, .docx) or plain string. The LLM parses these rules to extract classification topics and criteria. Injected into the system prompt; benefits from prompt caching across multiple calls. Use when topics are easier to express as prose rules than structured `ClassificationTopic` objects. | `optional[file` or `text?]` |
| `examples` | File (.txt, .md, .docx) or plain string with sample classification decisions. Useful when the LLM consistently misclassifies borderline cases. | `optional[file` or `text?]` |
| `llm_model` | LLM model override. The model must belong to the provider configured in the IDP connection (e.g., an OpenAI model requires openai credentials). See `SupportedModel` for valid values. If `None`, uses the provider's default model. | `optional[enum[CLAUDE_HAIKU_4_5, CLAUDE_OPUS_4_6, CLAUDE_OPUS_4_8, CLAUDE_OPUS_5, CLAUDE_SONNET_4_5, CLAUDE_SONNET_4_6, CLAUDE_SONNET_5, GEMINI_2_5_FLASH, GEMINI_2_5_PRO, GEMINI_3_1_FLASH_LITE, GEMINI_3_1_PRO_PREVIEW, GEMINI_3_5_FLASH, GEMINI_3_6_FLASH, GEMINI_3_FLASH_PREVIEW, GPT_5_2, GPT_5_4, GPT_5_4_MINI, GPT_5_5, GPT_5_6_LUNA, GPT_5_6_SOL, GPT_5_6_TERRA]?]` |
| `confidence_threshold` | Minimum average confidence score (0-100). Classifications below this threshold raise `ClassificationError`. | `optional[number]` |
### Idp classification topic
A topic definition for document classification.
| Field Name | Description | Type |
| ------------------- | --------------------------------------------- | ---------------- |
| `topic_name` | Name of the classification topic. | `optional[text]` |
| `topic_description` | Description of what this topic represents. | `optional[text]` |
| `topic_criteria` | Specific criteria to evaluate for this topic. | `optional[text]` |
### Idp classification result
Result of `classify_text_or_documents`.Contains classification results for each input document or text. For document inputs, elements are `DocumentClassification`. For text inputs, elements are `TextClassification`.
| Field Name | Description | Type |
| ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `result_type` | Discriminator constant (`"classification"`) that clients use for UI rendering and response routing — enables distinguishing this result type from other IDP procedure results. | `optional[enum[CLASSIFICATION, FIELDS_EXTRACTION, LAYOUT_PARSING, MERGE_SUBDOCUMENTS, READ_CONTENT, SUBDOCUMENTS_EXTRACTION, TABLES_EXTRACTION, TABLE_RECORDS_EXTRACTION]?]` |
| `classifications` | Per-input classification results. Each element carries its own `element_type` discriminator for further UI dispatch. | `list of idp document classification]` or `optional[list of idp text classification?` |
| [`metrics`](#metrics-idp-classification-result) | Processing metrics (tokens, pages, model used). | `optional[json]` |
### Idp fields extraction options
Options for `extract_fields_from_text_or_documents`.Specify what to extract using either `fields` (structured field definitions) or `prompt` (natural language description). These are mutually exclusive — provide one or the other, not both.
| Field Name | Description | Type |
| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`fields`](#fields-idp-fields-extraction-options) | List of `FieldDefinition` objects describing each field to extract. Each definition has `name` (required), `format` (`"string"`, `"number"`, `"date"`, `"boolean"`, `"list"`), `rule` (per-field extraction instruction), and `default_value` (fallback when the field is missing or below confidence). Mutually exclusive with `prompt`. Max 300 fields per extraction. | `optional[list of json]` |
| `prompt` | Natural language description of what to extract. The LLM parses this into `FieldDefinition` objects automatically. Mutually exclusive with `fields`. Example: `"Extract the invoice number, date, and total amount"`. | `optional[text]` |
| `business_rules` | Domain-specific instructions that guide the LLM during extraction. Accepts a file (.txt, .md, .docx) or a plain string — any text works, no prescribed format. Rules are injected into the system prompt with highest precedence over default behavior. Because they are part of the system prompt, they benefit from prompt caching across multiple LLM calls on the same document set. Use business rules to enforce formatting conventions, domain constraints, or handling instructions for ambiguous content. | `optional[file` or `text?]` |
| `examples` | File (.txt, .md, .docx) or plain string with sample input/output pairs that demonstrate expected extraction behavior. Injected into the system prompt alongside business rules. Particularly useful when the LLM consistently misinterprets a field's format or value. | `optional[file` or `text?]` |
| `llm_model` | LLM model override. The model must belong to the provider configured in the IDP connection (e.g., an OpenAI model requires openai credentials). See `SupportedModel` for valid values. If `None`, uses the provider's default model. | `optional[enum[CLAUDE_HAIKU_4_5, CLAUDE_OPUS_4_6, CLAUDE_OPUS_4_8, CLAUDE_OPUS_5, CLAUDE_SONNET_4_5, CLAUDE_SONNET_4_6, CLAUDE_SONNET_5, GEMINI_2_5_FLASH, GEMINI_2_5_PRO, GEMINI_3_1_FLASH_LITE, GEMINI_3_1_PRO_PREVIEW, GEMINI_3_5_FLASH, GEMINI_3_6_FLASH, GEMINI_3_FLASH_PREVIEW, GPT_5_2, GPT_5_4, GPT_5_4_MINI, GPT_5_5, GPT_5_6_LUNA, GPT_5_6_SOL, GPT_5_6_TERRA]?]` |
| `confidence_threshold` | Minimum confidence score (0-100). Fields below this threshold raise `ExtractionError`. Lower this for noisy or OCR-heavy documents where perfect confidence is unrealistic. | `optional[number]` |
| `verification` | Post-extraction verification level. When enabled, the LLM cross-checks extracted values against the source document. Business rules act as corroborating evidence during verification — values conforming to rules receive a confidence boost, while violations are flagged for review. `None` disables verification. | `optional[enum[HIGH, LOW, MODERATE]?]` |
### Idp fields extraction result
Result of `extract_fields_from_text_or_documents`.Contains all extracted field occurrences. For document inputs, fields are `DocumentField` instances with `page_number` and `bounding_box` (normalized 0-1 coordinates) that clients can use to render overlays, highlight extracted values on the original document, or build customizable review UIs. For text inputs, fields are `TextField` instances with `location` (character offsets) for text highlighting.
| Field Name | Description | Type |
| -------------------------------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `result_type` | Discriminator constant (`"fields_extraction"`). | `optional[enum[CLASSIFICATION, FIELDS_EXTRACTION, LAYOUT_PARSING, MERGE_SUBDOCUMENTS, READ_CONTENT, SUBDOCUMENTS_EXTRACTION, TABLES_EXTRACTION, TABLE_RECORDS_EXTRACTION]?]` |
| `fields` | Extracted field results — one per unique field occurrence. | `list of idp document field]` or `optional[list of idp text field?` |
| [`metrics`](#metrics-idp-fields-extraction-result) | Processing metrics (tokens, pages, model used). | `optional[json]` |
### Idp subdocuments extraction options
Options for `extract_subdocuments_from_a_document`.Supports three extraction strategies: 1. **Page range**: Set `page_range` to extract a specific range. 2. **Page marker**: Set `page_marker` to split at pages matching a text pattern (uses LLM to evaluate each page). 3. **Fixed size**: Set `subdocument_size` to split into N-page chunks with optional overlap.
| Field Name | Description | Type |
| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `page_range` | Tuple of (start\_page, end\_page) for range extraction. Pages are 1-indexed. Example: `(1, 5)` extracts pages 1-5. | `optional[list of number]` |
| `page_marker` | Text pattern to identify subdocument boundaries. Example: `"INVOICE NUMBER"`. The LLM evaluates each page. | `optional[text]` |
| `subdocument_size` | Number of pages per subdocument for chunking. | `optional[number]` |
| `subdocument_overlap_size` | Number of overlapping pages between consecutive chunks. Requires `subdocument_size`. | `optional[number]` |
| `llm_model` | LLM model override (used for marker-based splitting only — page-range and fixed-size modes do not use an LLM). The model must belong to the provider configured in the IDP connection. See `SupportedModel` for valid values. If `None`, uses the provider's default model. | `optional[enum[CLAUDE_HAIKU_4_5, CLAUDE_OPUS_4_6, CLAUDE_OPUS_4_8, CLAUDE_OPUS_5, CLAUDE_SONNET_4_5, CLAUDE_SONNET_4_6, CLAUDE_SONNET_5, GEMINI_2_5_FLASH, GEMINI_2_5_PRO, GEMINI_3_1_FLASH_LITE, GEMINI_3_1_PRO_PREVIEW, GEMINI_3_5_FLASH, GEMINI_3_6_FLASH, GEMINI_3_FLASH_PREVIEW, GPT_5_2, GPT_5_4, GPT_5_4_MINI, GPT_5_5, GPT_5_6_LUNA, GPT_5_6_SOL, GPT_5_6_TERRA]?]` |
### Idp subdocuments extraction result
Result of `extract_subdocuments_from_a_document`.Contains the extracted subdocument pages as separate IO objects.
| Field Name | Description | Type |
| -------------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `result_type` | Discriminator constant (`"subdocuments_extraction"`). | `optional[enum[CLASSIFICATION, FIELDS_EXTRACTION, LAYOUT_PARSING, MERGE_SUBDOCUMENTS, READ_CONTENT, SUBDOCUMENTS_EXTRACTION, TABLES_EXTRACTION, TABLE_RECORDS_EXTRACTION]?]` |
| `subdocuments` | List of subdocument IO objects (PDF format). | `optional[list of file]` |
| `source_document` | Filename of the source document. | `optional[text]` |
| [`metrics`](#metrics-idp-subdocuments-extraction-result) | Processing metrics. | `optional[json]` |
### Idp table records extraction options
Options for `extract_table_records_from_documents`.Describes a single recurring table schema (via `table`) to extract from every page of the input document(s). The document is split into page windows that are extracted in parallel and stitched back into one merged dataset. Use this when the same table layout repeats across many pages (e.g. a multi-page timesheet or line-item packet) and you want all rows in one result. To locate distinct, individual tables with page and bounding-box identity, use `TablesExtractionOptions` instead.
| Field Name | Description | Type |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `table` | Description of the recurring table to extract. Should be descriptive enough for the LLM to identify it on each page — include the table title or expected column headers. Example: `"Daily labor grid with columns Date, Employee, " "Job, Hours"`. | `optional[text]` |
| `business_rules` | Domain-specific instructions that guide the LLM during extraction. Accepts a file (.txt, .md, .docx) or a plain string — any text works. Injected into the system prompt with highest precedence; benefits from prompt caching across the per-chunk calls. Use to enforce column naming, row filtering, or handling of merged cells and annotations. | `optional[file` or `text?]` |
| `examples` | File (.txt, .md, .docx) or plain string with sample extraction pairs. Useful when the LLM misidentifies headers or splits rows incorrectly. | `optional[file` or `text?]` |
| `llm_model` | LLM model override. The model must belong to the provider configured in the IDP connection (e.g., an OpenAI model requires openai credentials). See `SupportedModel` for valid values. If `None`, uses the provider's default model. | `optional[enum[CLAUDE_HAIKU_4_5, CLAUDE_OPUS_4_6, CLAUDE_OPUS_4_8, CLAUDE_OPUS_5, CLAUDE_SONNET_4_5, CLAUDE_SONNET_4_6, CLAUDE_SONNET_5, GEMINI_2_5_FLASH, GEMINI_2_5_PRO, GEMINI_3_1_FLASH_LITE, GEMINI_3_1_PRO_PREVIEW, GEMINI_3_5_FLASH, GEMINI_3_6_FLASH, GEMINI_3_FLASH_PREVIEW, GPT_5_2, GPT_5_4, GPT_5_4_MINI, GPT_5_5, GPT_5_6_LUNA, GPT_5_6_SOL, GPT_5_6_TERRA]?]` |
| `confidence_threshold` | Minimum confidence score (0-100). The aggregated per-column confidence is averaged and compared against this threshold. Set to `0` (default) to disable threshold checking and capture all rows. | `optional[number]` |
| `pages_per_call` | Number of pages sent to the LLM per extraction call. `1` (default) maximizes parallelism and accuracy for dense single-page tables; increase it when a logical row spans page breaks so the model sees both pages at once. | `optional[number]` |
| `page_overlap` | Number of pages shared between consecutive windows. `0` (default) means no overlap. Use a small overlap (e.g. `1`) when rows straddle page boundaries; duplicate rows from the shared pages are de-duplicated during stitching. Must be less than `pages_per_call`. | `optional[number]` |
### Idp table records extraction result
Result of `extract_table_records_from_documents`.Contains every row of one recurring table schema, merged across all pages of the input document(s) into a single dataset. Unlike `TablesExtractionResult` (which returns located table *artifacts*, one per description, each with its own page and bounding box), this returns one merged `records` table with per-chunk page provenance carried in `segments`.
| Field Name | Description | Type |
| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `result_type` | Discriminator constant (`"table_records_extraction"`). | `optional[enum[CLASSIFICATION, FIELDS_EXTRACTION, LAYOUT_PARSING, MERGE_SUBDOCUMENTS, READ_CONTENT, SUBDOCUMENTS_EXTRACTION, TABLES_EXTRACTION, TABLE_RECORDS_EXTRACTION]?]` |
| [`records`](#idp-table) | The merged table — canonical headers plus all rows in document order. `None` only if no input pages were processed. | `optional[idp table]` |
| [`segments`](#idp-table-record-segment) | Per-chunk provenance. Each `TableRecordSegment` maps a half-open row range in `records` back to the source document and page range that produced it. | `optional[list of idp table record segment]` |
| `confidence` | Aggregated per-column confidence scores (0-100), row-count-weighted across the contributing chunks. | `optional[list of number]` |
| [`metrics`](#metrics-idp-table-records-extraction-result) | Processing metrics (tokens, pages, model used). | `optional[json]` |
### Idp table
A structured table extracted from a document or text.Contains the table data (headers + rows) and an optional PyArrow representation for efficient downstream processing.
| Field Name | Description | Type |
| -------------- | ------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `element_type` | Discriminator constant (`"idp_table"`). | `optional[enum[DOCUMENT_CLASSIFICATION, DOCUMENT_CONTENT, DOCUMENT_FIELD, DOCUMENT_IDP_TABLE, DOCUMENT_KEY_VALUE_PAIR, DOCUMENT_LAYOUT, DOCUMENT_TEXT, IDP_TABLE, TEXT_CLASSIFICATION, TEXT_FIELD, TEXT_IDP_TABLE]?]` |
| `title` | Table title or caption, if detected. | `optional[text]` |
| `headers` | Column header names. | `optional[list of text]` |
| `rows` | Table body as a list of rows, each row a list of cell values. | `optional[list of text]` |
| `num_rows` | Number of data rows (excluding header). | `optional[number]` |
| `num_cols` | Number of columns. | `optional[number]` |
| `arrow_table` | PyArrow Table for efficient columnar access. | `optional[table?]` |
### Idp table record segment
Provenance for one contiguous block of rows in a merged record table.Returned as part of `TableRecordsExtractionResult.segments`. Each segment maps a half-open row range in the merged `records` table back to the source document and page range that produced it, so callers can trace any row to its origin without a parallel per-row list.
| Field Name | Description | Type |
| ----------------------- | ------------------------------------------------------------------------------------------------------------ | -------------------------- |
| `source_document` | Filename of the source document. | `optional[text]` |
| `source_document_index` | 0-based index into the input document list. | `optional[number]` |
| `page_start` | 1-indexed first page of the chunk that produced these rows (inclusive). | `optional[number]` |
| `page_end` | 1-indexed last page of the chunk that produced these rows (inclusive). | `optional[number]` |
| `row_start` | 0-indexed offset of the first row of this segment within the merged `records.rows` (inclusive). | `optional[number]` |
| `row_end` | 0-indexed offset one past the last row of this segment within the merged `records.rows` (exclusive). | `optional[number]` |
| `confidence` | Per-column confidence scores (0-100) for this chunk. Each entry corresponds to a column in the merged table. | `optional[list of number]` |
### Idp tables extraction options
Options for `extract_tables_from_text_or_documents`.Specify which tables to extract via `tables` — a list of table descriptions. Each description tells the LLM what table to look for.
| Field Name | Description | Type |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tables` | Descriptions of tables to extract. Each string should be descriptive enough for the LLM to identify the right table — include the table's title or header names. Max 5 tables per extraction. Example: `["Previous Employment History table", "Education and Certifications table"]`. Tip: if a table is not found, make the description more specific by referencing the table's title or column headers (e.g., `"Line items table with columns Item, Qty, Price"` instead of `"table"`). | `optional[list of text]` |
| `business_rules` | Domain-specific instructions that guide the LLM during table extraction. Accepts a file (.txt, .md, .docx) or a plain string — any text works. Injected into the system prompt with highest precedence. Benefits from prompt caching across multiple LLM calls. Use to enforce column naming, row filtering, or handling of merged cells and annotations. | `optional[file` or `text?]` |
| `examples` | File (.txt, .md, .docx) or plain string with sample extraction pairs. Useful when the LLM misidentifies headers or splits rows incorrectly. | `optional[file` or `text?]` |
| `llm_model` | LLM model override. The model must belong to the provider configured in the IDP connection (e.g., an OpenAI model requires openai credentials). See `SupportedModel` for valid values. If `None`, uses the provider's default model. | `optional[enum[CLAUDE_HAIKU_4_5, CLAUDE_OPUS_4_6, CLAUDE_OPUS_4_8, CLAUDE_OPUS_5, CLAUDE_SONNET_4_5, CLAUDE_SONNET_4_6, CLAUDE_SONNET_5, GEMINI_2_5_FLASH, GEMINI_2_5_PRO, GEMINI_3_1_FLASH_LITE, GEMINI_3_1_PRO_PREVIEW, GEMINI_3_5_FLASH, GEMINI_3_6_FLASH, GEMINI_3_FLASH_PREVIEW, GPT_5_2, GPT_5_4, GPT_5_4_MINI, GPT_5_5, GPT_5_6_LUNA, GPT_5_6_SOL, GPT_5_6_TERRA]?]` |
| `confidence_threshold` | Minimum confidence score (0-100). Table confidence is per-column; the average is compared against this threshold. | `optional[number]` |
| `verification` | Post-extraction verification level. `None` disables verification. | `optional[enum[HIGH, LOW, MODERATE]?]` |
### Idp tables extraction result
Result of `extract_tables_from_text_or_documents`.Contains all extracted tables. For document inputs, tables are `DocumentIDPTable` instances with `page_number` and `bounding_box` (normalized 0-1 coordinates) that clients can use to render overlays or highlight table regions on the original document. For text inputs, tables are `TextIDPTable` instances with `location` (character offsets) for text highlighting.
| Field Name | Description | Type |
| -------------------------------------------------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `result_type` | Discriminator constant (`"tables_extraction"`). | `optional[enum[CLASSIFICATION, FIELDS_EXTRACTION, LAYOUT_PARSING, MERGE_SUBDOCUMENTS, READ_CONTENT, SUBDOCUMENTS_EXTRACTION, TABLES_EXTRACTION, TABLE_RECORDS_EXTRACTION]?]` |
| `tables` | Extracted table results. | `list of idp document table]` or `optional[list of idp text table?` |
| [`metrics`](#metrics-idp-tables-extraction-result) | Processing metrics. | `optional[json]` |
### Idp merge subdocuments options
Options for `merge_subdocuments_into_a_document`.
| Field Name | Description | Type |
| --------------- | ---------------------------------------- | ---------------- |
| `document_name` | Filename for the merged output document. | `optional[text]` |
### Idp merge subdocuments result
Result of `merge_subdocuments_into_a_document`.Contains the merged document as a single IO object.
| Field Name | Description | Type |
| --------------------------------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `result_type` | Discriminator constant (`"merge_subdocuments"`). | `optional[enum[CLASSIFICATION, FIELDS_EXTRACTION, LAYOUT_PARSING, MERGE_SUBDOCUMENTS, READ_CONTENT, SUBDOCUMENTS_EXTRACTION, TABLES_EXTRACTION, TABLE_RECORDS_EXTRACTION]?]` |
| `document` | The merged document (PDF format). | `optional[file]` |
| `document_name` | Filename of the merged document. | `optional[text]` |
| `source_count` | Number of input documents/pages that were merged. | `optional[number]` |
| [`metrics`](#metrics-idp-merge-subdocuments-result) | Processing metrics. | `optional[json]` |
### Idp layout parsing options
Options for `parse_layout_from_a_document`.Controls how the document's structural layout is extracted, including tables, key-value pairs, and text segments.
| Field Name | Description | Type |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `business_rules` | Domain-specific instructions that guide the LLM during layout extraction. Accepts a file (.txt, .md, .docx) or a plain string — any text works. Injected into the system prompt with highest precedence. Benefits from prompt caching across multiple LLM calls. | `optional[file` or `text?]` |
| `examples` | File (.txt, .md, .docx) or plain string with sample extraction pairs demonstrating expected output structure. | `optional[file` or `text?]` |
| `llm_model` | LLM model override. The model must belong to the provider configured in the IDP connection (e.g., an OpenAI model requires openai credentials). See `SupportedModel` for valid values. If `None`, uses the provider's default model. | `optional[enum[CLAUDE_HAIKU_4_5, CLAUDE_OPUS_4_6, CLAUDE_OPUS_4_8, CLAUDE_OPUS_5, CLAUDE_SONNET_4_5, CLAUDE_SONNET_4_6, CLAUDE_SONNET_5, GEMINI_2_5_FLASH, GEMINI_2_5_PRO, GEMINI_3_1_FLASH_LITE, GEMINI_3_1_PRO_PREVIEW, GEMINI_3_5_FLASH, GEMINI_3_6_FLASH, GEMINI_3_FLASH_PREVIEW, GPT_5_2, GPT_5_4, GPT_5_4_MINI, GPT_5_5, GPT_5_6_LUNA, GPT_5_6_SOL, GPT_5_6_TERRA]?]` |
| `confidence_threshold` | Minimum confidence score (0-100). Elements below this threshold raise `AnalysisError`. Set to `0` (default) to disable threshold checking. | `optional[number]` |
| `verification` | Post-extraction verification level. `None` disables verification. | `optional[enum[HIGH, LOW, MODERATE]?]` |
| `analysis_mode` | Analysis strategy: - `"single_pass"`: Processes the entire document in one LLM call. Fastest and cheapest — best for simple, short documents. - `"parallel"`: Per-page extraction with smart deduplication. Best quality for multi-page documents — handles cross-page tables and repeated headers. - `"plan_based"`: A planning LLM first analyzes the document, then creates focused extraction tasks that run in parallel. Best for complex documents with varied structure. | `optional[enum[PARALLEL, PLAN_BASED, SINGLE_PASS]?]` |
### Idp layout parsing result
Result of `parse_layout_from_a_document`.Contains the complete structural layout of the document. All layout elements (tables, key-value pairs, texts) carry `page_number` and `bounding_box` (normalized 0-1 coordinates) that clients can use to render overlays, build interactive document viewers, or create customizable review UIs.
| Field Name | Description | Type |
| ----------------------------------------------- | --------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `result_type` | Discriminator constant (`"layout_parsing"`). | `optional[enum[CLASSIFICATION, FIELDS_EXTRACTION, LAYOUT_PARSING, MERGE_SUBDOCUMENTS, READ_CONTENT, SUBDOCUMENTS_EXTRACTION, TABLES_EXTRACTION, TABLE_RECORDS_EXTRACTION]?]` |
| [`layout`](#idp-document-layout) | The parsed document layout with tables, key-value pairs, and text segments. | `optional[idp document layout]` |
| `source_document` | Filename of the source document. | `optional[text]` |
| [`metrics`](#metrics-idp-layout-parsing-result) | Processing metrics. | `optional[json]` |
### Idp document layout
Complete parsed layout of a document.Returned by `parse_layout_from_a_document`. Contains the document's structural elements organized by type: tables, key-value pairs, and text segments. All child elements carry `page_number` and `bounding_box` (normalized 0-1 coordinates) that clients can use to render overlays, build interactive document viewers, or create customizable review UIs.
| Field Name | Description | Type |
| ------------------------------------------------- | ------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `element_type` | Discriminator constant (`"document_layout"`). | `optional[enum[DOCUMENT_CLASSIFICATION, DOCUMENT_CONTENT, DOCUMENT_FIELD, DOCUMENT_IDP_TABLE, DOCUMENT_KEY_VALUE_PAIR, DOCUMENT_LAYOUT, DOCUMENT_TEXT, IDP_TABLE, TEXT_CLASSIFICATION, TEXT_FIELD, TEXT_IDP_TABLE]?]` |
| `document_type` | Detected document type (e.g., `"invoice"`). | `optional[text]` |
| `document_title` | Detected document title, if any. | `optional[text]` |
| [`tables`](#idp-document-table) | Tables found in the document. | `optional[list of idp document table]` |
| [`key_value_pairs`](#idp-document-key-value-pair) | Key-value pairs found in the document. | `optional[list of idp document key value pair]` |
| [`texts`](#idp-document-text) | Text segments found in the document. | `optional[list of idp document text]` |
| `source_document` | Filename of the source document. | `optional[text]` |
| `analysis_mode` | The analysis mode that was used. | `optional[enum[PARALLEL, PLAN_BASED, SINGLE_PASS]?]` |
| `confidence` | Overall confidence score (0-100). | `optional[number]` |
| `verified` | Whether verification was applied. | `optional[boolean]` |
| `hallucinations_detected` | Whether any hallucinations were detected during verification. | `optional[boolean]` |
### Idp read content options
Options for `read_content_from_documents`.Controls how document content is extracted and returned. Digital PDFs use fast PyMuPDF extraction; scanned PDFs and images trigger LLM-based OCR automatically.
| Field Name | Description | Type |
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `conversion` | Output format. `"as_markdown"` preserves headings, lists, and tables as markdown (best for structured documents). `"as_text"` returns plain text (best for downstream text processing or search indexing). Defaults to `"as_markdown"`. | `optional[text]` |
| `max_pages_per_document` | Maximum pages to process per document. Pages beyond this limit are silently skipped. | `optional[number]` |
| `llm_model` | LLM model override (used only for scanned/image documents that require OCR — digital PDFs ignore this setting). The model must belong to the provider configured in the IDP connection. See `SupportedModel` for valid values. If `None`, uses the provider's default model. | `optional[enum[CLAUDE_HAIKU_4_5, CLAUDE_OPUS_4_6, CLAUDE_OPUS_4_8, CLAUDE_OPUS_5, CLAUDE_SONNET_4_5, CLAUDE_SONNET_4_6, CLAUDE_SONNET_5, GEMINI_2_5_FLASH, GEMINI_2_5_PRO, GEMINI_3_1_FLASH_LITE, GEMINI_3_1_PRO_PREVIEW, GEMINI_3_5_FLASH, GEMINI_3_6_FLASH, GEMINI_3_FLASH_PREVIEW, GPT_5_2, GPT_5_4, GPT_5_4_MINI, GPT_5_5, GPT_5_6_LUNA, GPT_5_6_SOL, GPT_5_6_TERRA]?]` |
### Idp read content result
Result of `read_content_from_documents`.Contains extracted text/markdown content for each input document.
| Field Name | Description | Type |
| --------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `result_type` | Discriminator constant (`"read_content"`). | `optional[enum[CLASSIFICATION, FIELDS_EXTRACTION, LAYOUT_PARSING, MERGE_SUBDOCUMENTS, READ_CONTENT, SUBDOCUMENTS_EXTRACTION, TABLES_EXTRACTION, TABLE_RECORDS_EXTRACTION]?]` |
| [`contents`](#idp-document-content) | Extracted content for each document. | `optional[list of idp document content]` |
| [`metrics`](#metrics-idp-read-content-result) | Processing metrics. | `optional[json]` |
### Idp document content
Text content extracted from a single document.Returned as part of `ReadContentResult.content`.
| Field Name | Description | Type |
| ----------------------- | ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `element_type` | Discriminator constant (`"document_content"`). | `optional[enum[DOCUMENT_CLASSIFICATION, DOCUMENT_CONTENT, DOCUMENT_FIELD, DOCUMENT_IDP_TABLE, DOCUMENT_KEY_VALUE_PAIR, DOCUMENT_LAYOUT, DOCUMENT_TEXT, IDP_TABLE, TEXT_CLASSIFICATION, TEXT_FIELD, TEXT_IDP_TABLE]?]` |
| `content` | The extracted text or markdown content. | `optional[text]` |
| `is_handwritten` | Whether the document appears handwritten. | `optional[boolean]` |
| `source_document` | Filename of the source document. | `optional[text]` |
| `source_document_index` | 0-based index into the input document list. | `optional[number]` |
#### Concept attribute specifications
**metrics (idp classification result)**
| Name | Type |
| ----------------------- | ------------------ |
| `num_documents` | `optional[number]` |
| `num_pages` | `optional[number]` |
| `input_tokens` | `optional[number]` |
| `output_tokens` | `optional[number]` |
| `llm_model` | `optional[text]` |
| `data_processed` | `optional[number]` |
| `processing_time` | `optional[number]` |
| `prepass_input_tokens` | `optional[number]` |
| `prepass_output_tokens` | `optional[number]` |
**fields (idp fields extraction options)**
| Name | Type |
| --------------- | ---------------- |
| `name` | `text` |
| `format` | `optional[text]` |
| `rule` | `optional[text]` |
| `default_value` | `optional[any?]` |
**metrics (idp fields extraction result)**
| Name | Type |
| ----------------------- | ------------------ |
| `num_documents` | `optional[number]` |
| `num_pages` | `optional[number]` |
| `input_tokens` | `optional[number]` |
| `output_tokens` | `optional[number]` |
| `llm_model` | `optional[text]` |
| `data_processed` | `optional[number]` |
| `processing_time` | `optional[number]` |
| `prepass_input_tokens` | `optional[number]` |
| `prepass_output_tokens` | `optional[number]` |
**metrics (idp subdocuments extraction result)**
| Name | Type |
| ----------------------- | ------------------ |
| `num_documents` | `optional[number]` |
| `num_pages` | `optional[number]` |
| `input_tokens` | `optional[number]` |
| `output_tokens` | `optional[number]` |
| `llm_model` | `optional[text]` |
| `data_processed` | `optional[number]` |
| `processing_time` | `optional[number]` |
| `prepass_input_tokens` | `optional[number]` |
| `prepass_output_tokens` | `optional[number]` |
**metrics (idp table records extraction result)**
| Name | Type |
| ----------------------- | ------------------ |
| `num_documents` | `optional[number]` |
| `num_pages` | `optional[number]` |
| `input_tokens` | `optional[number]` |
| `output_tokens` | `optional[number]` |
| `llm_model` | `optional[text]` |
| `data_processed` | `optional[number]` |
| `processing_time` | `optional[number]` |
| `prepass_input_tokens` | `optional[number]` |
| `prepass_output_tokens` | `optional[number]` |
**metrics (idp tables extraction result)**
| Name | Type |
| ----------------------- | ------------------ |
| `num_documents` | `optional[number]` |
| `num_pages` | `optional[number]` |
| `input_tokens` | `optional[number]` |
| `output_tokens` | `optional[number]` |
| `llm_model` | `optional[text]` |
| `data_processed` | `optional[number]` |
| `processing_time` | `optional[number]` |
| `prepass_input_tokens` | `optional[number]` |
| `prepass_output_tokens` | `optional[number]` |
**metrics (idp merge subdocuments result)**
| Name | Type |
| ----------------------- | ------------------ |
| `num_documents` | `optional[number]` |
| `num_pages` | `optional[number]` |
| `input_tokens` | `optional[number]` |
| `output_tokens` | `optional[number]` |
| `llm_model` | `optional[text]` |
| `data_processed` | `optional[number]` |
| `processing_time` | `optional[number]` |
| `prepass_input_tokens` | `optional[number]` |
| `prepass_output_tokens` | `optional[number]` |
**metrics (idp layout parsing result)**
| Name | Type |
| ----------------------- | ------------------ |
| `num_documents` | `optional[number]` |
| `num_pages` | `optional[number]` |
| `input_tokens` | `optional[number]` |
| `output_tokens` | `optional[number]` |
| `llm_model` | `optional[text]` |
| `data_processed` | `optional[number]` |
| `processing_time` | `optional[number]` |
| `prepass_input_tokens` | `optional[number]` |
| `prepass_output_tokens` | `optional[number]` |
**metrics (idp read content result)**
| Name | Type |
| ----------------------- | ------------------ |
| `num_documents` | `optional[number]` |
| `num_pages` | `optional[number]` |
| `input_tokens` | `optional[number]` |
| `output_tokens` | `optional[number]` |
| `llm_model` | `optional[text]` |
| `data_processed` | `optional[number]` |
| `processing_time` | `optional[number]` |
| `prepass_input_tokens` | `optional[number]` |
| `prepass_output_tokens` | `optional[number]` |
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.kognitos.com/guides/platform/integrations/idp.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.