Capturing user input as a custom entity?

[timtutt]

I have a use case where I'm trying to match custom entities that may include patterns like the following:

"[where] [state] [is] [equal] [to] [SOME_USER_INPUT]

Right now I'm replacing [SOME_USER_INPUT] with [|ADJ|ADP|ADV|ADV|AUX|CCONJ|DET|INTJ|NOUN|NUM|PART|PRON|PROPN|PUNCT|SCONJ|SYM|VERB|X], but it doesn't capture cases where the user input is multiple tokens. Is there a best practice for this?

[sanjayaksaxena]

Hello @timtutt

The pattern [|ADJ|ADP|ADV|ADV|AUX|CCONJ|DET|INTJ|NOUN|NUM|PART|PRON|PROPN|PUNCT|SCONJ|SYM|VERB|X] will detect a single token of any PoS and not multiple instances. Currently winkNLP does not support wildcard match. Therefore one way to achieve is to use the following pattern, provided you are only looking to capture only upto a few tokens — say 3 or 4:

const winkNLP = require( 'wink-nlp' );
const model = require( 'wink-eng-lite-web-model' );
const nlp = winkNLP( model );
const its = nlp.its;

const ANY = ' [|ADJ|ADP|ADV|ADV|AUX|CCONJ|DET|INTJ|NOUN|NUM|PART|PRON|PROPN|PUNCT|SCONJ|SYM|VERB|X]';
const patterns = [
  {
    name: 'multi-word-capture',
    patterns: [
    '[where] [state] [is] [equal] [to]' + ANY + ANY + ANY +ANY
    ]
  }
];
// Such pattern will slow down the learning part
// but will not impact further processing i.e. the NLP Pipe so much!
nlp.learnCustomEntities( patterns, { useEntity: false } );
const text = 'Where state is equal to something very amazing. Or Where state is equal to 4 and not 6. And where State is equal to simple';
const doc = nlp.readDoc( text );
console.log( doc.customEntities().out( its.detail ) );

// —> [
// —>   {value: "Where state is equal to something very amazing.", type: "multi-word-capture"},
// —>   {value: "Where state is equal to 4 and not 6", type: "multi-word-capture"},
// —>   {value: "where State is equal to simple", type: "multi-word-capture"}
// —> ]

Note: The useEntity must be set false so that the detected native entities are ignored while pattern matching — for example, without this Where state is equal to 4 and not 6 would not have been detected.

However if you need to capture a larger number of such tokens then you need to code, where you capture until the end of the sentence or text after the occurrence of the pattern — should be simple with winkNLP's API.

Best,
Sanjaya

[timtutt]

Thanks much Sanjaya - this was helpful. Was able to leverage the concept here to get everything squared away - it's worth noting however for anyone that comes across this answer that there is a big cost to doing multiple "ANY" match phrases in terms of heap and time to train. We found between 2 and 3 depending on the complexity of the phrase to be the sweet spot.

[sanjayaksaxena]

Right @timtutt, we need to find a simple way to allow multiple wildcard matches... do share if you have any thoughts/ideas!