November 2024

In the last month Sensible added support for extracting and classifying CSV files, made improvements to the LLM-based List method, added advanced support for extended JsonLogic operations, and added several advanced configuration options.

New feature: Support for CSV documents

Sensible now supports data extraction and classification for comma-separated value (CSV) documents. For more information, see Supported file types.

Improvement: Troubleshoot duplicate or missing List contents

With the new Single LLM Completion parameter, you can troubleshoot incomplete or duplicate results in a multi-page list. When you set this parameter to true, Sensible submits the entire context, or relevant document extract for your prompt, to the LLM in one request rather than in batched calls.

Improvement: Locate lists in documents using LLM page summaries

In addition to supporting the Search By Summarization parameter for the Query Group method, Sensible now supports this parameter for the List method. Use this parameter to troubleshoot situations in which Sensible misidentifies the part of the document that contains the answers to your prompts. With this parameter, Sensible implements a completion-only retrieval-augmented generation (RAG) strategy. Sensible prompts an LLM to summarize each page in the document, prompts a second LLM to return the pages most relevant to your prompt based on the summaries, and extracts the answers to your prompts from those pages.

Improvement: Advanced support for JsonLogic operations

In addition to the base JsonLogic operators, Sensible supports extended operations available in the Json Logic Engine. Newly supported operations include:

• Array operations: "length", "get".
• Miscellaneous operations: "preserve", "keys".
• Higher order operations: "every", "eachKey".

With this improvement, Sensible recommends that you use the syntactically more concise eachKey operator instead of the Object operator where possible.

Improvement: Advanced configuration for Row method and Sections

You can now troubleshoot unusual font sizes that cause the Row method or sections' Stop parameter to fail using the new Tolerance parameter. Use this parameter to adjust Sensible's criteria for recognizing when two lines are in the same "row," or distributed along the same x-axis. For more information, see Row and Sections.

New feature: Define fallback types

With the new Any type, you can define an array of fallback types for a single field. Sensible uses the first-matching type in the array. Use the Any type as a more concise syntactical alternative to defining an array of fallback fields of different types to capture variations in target data's formatting or type.

UX improvement: Sort configurations in document types

In a document type's list of configs, you can now sort the configs by name, date created, and other attributes: