NAP Natural Language Management Platform
I developed a platform to manage natural language data for our SaaS chatbot customers. It started as a way to manage the large volumes of FAQs our customers had.
- Organize FAQ question and answers in bulk
- Test matching on input queries
- Manage intents and entities
The main server is built in NodeJS which communicates with a C++ classification engine using subword vectors.
The web front-end is built with VueJS
- Automated test suite runs to double check matches and run regression tests
- A markup area allows intents and entities to be edited
A number of different matching algorithms are implemented:
- For some clients with soft conversational queries the word vector approach works well.
- Other clients in pharma, medical or fintech had much stricter matching requirements, so we built a separate more layer that combined keywords with sentence classification, alllowing much more precise results and avoid false positives.
We used word-vectors to provide multilingual suggestions for entity synonyms.
Coverage tools would show how many responses were prepared for differnet parts of the knowledge graph.
For one of our pharma clients with very strutured medical queries, I wrote a pipeline to generate combinations of question phrases and entities.
I extended the system over time using the classifier for basic matching of the "shape" of phrases, and other methods including detailed control over entity tokens as the signal for matching.
The system was easy enough to use for our clients to manage their FAQbot content themselves.