Finds a range of lines in a document that best match a topic as determined by a bag of words NLP approach. Most useful in long, unstructured documents. For example, this method in conjunction with the Summarizer method can extract key-value pairs from free text using ML (machine learning).

Parameters
Examples

Parameters

Note: For the full list of parameters available for this method, see Global parameters for methods. The following table shows parameters most relevant to or specific to this method.

keyvaluedescription
id (required)topic
numParagraphs or numLines (required)numberThe number of paragraphs or consecutive lines to extract, respectively.


If you set the Num Paragraphs parameter, Sensible scores every paragraph in the document and returns the highest-scoring paragraph. For more information about paragraph recognition, see the Paragraph method .

If you set the Num Lines parameter, Sensible scores every group of consecutive lines in the document and returns the highest-scoring group. For information about the definition of "consecutive", see line sorting.

If line groups or paragraphs have equal scores, then Sensible returns the last one.
terms (required)string arrayAn array of terms to score positively during topic recognition. For more information about the NLP approach, see bag of words.
stopTermsstring arrayAn array of terms to score negatively during topic recognition. For more information about the NLP approach, see bag of words.

Examples

The following example shows finding a topic in a licensing legal code.

Config

{
  "fields": [
    {
      "id": "liability_limitation_topic",
      "anchor": {
        "match": {
          "type": "first"
        }
      },
      "method": {
        "id": "topic",
        "numParagraphs": 1,
        "terms": [
          "limitation",
          "liability",
          "extent"
        ],
        "stopTerms": [
          "warranty"
        ]
      }
    }
  ]
}

Example document
The following image shows the example PDF used with this example config:

Click to enlargeClick to enlarge

Example PDFDownload link

Output

{
  "liability_limitation_topic": {
    "type": "string",
    "value": "6. Limitation on Liability. EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES."
  }
}

Did this page help you?