In the last month, we released improvements to NLP methods, to the SenseML and Sensible Instruct editors, and to API key configurability and security. We added the ability to extract from portfolios in the Sensible app's quick extract tab.
You can now configure the context you provide to a large-language model (LLM) through new parameters for the Question method, also known as the Query method. When you pose a question about a document, Sensible provides the LLM with chunks of the document as context for answering the question. You can configure the context in SenseML for the Question method with the following chunk-related parameters:
- Use the Chunk Count, Chunk Size, and Chunk Overlap Percentage parameters to configure the size of the context.
- Use the Chunk Scoring Text parameter to narrow down the page location of the answer to your question. For example, if your question has multiple candidate answers, and the correct answer is located near unique or distinctive text that's difficult to incorporate into your question, then specify the distinctive text in this parameter.
In addition to using the Sensible API, you can now extract data from a single PDF file that contains multiple documents (a PDF "portfolio") using the Sensible quick extract tab.
You can now configure security for your API keys when you create them. Choose from non-recoverable keys (most secure), keys that require an account password and recovery password to recover, and keys that require an account password to recover (least secure). Manage keys on your Sensible account page.
You can now write configurations for large reference documents in the SenseML and Sensible Instruct editors in the Sensible app. The editors' 30-second time-out limit is extended to 15 minutes.