Chatbot that will answer from the given Information/Documents
Question:
I want to make a chatbot that will answer the questions based on the given document.
E.g, if I have hundreds of documents and I want to get some information from it but don’t know which information is on which line of the page so I have to spend some time and effort to search. I want a chatbot that will learn from those documents and give answers form that documents.
Is there any available service that can full fill my needs?
What if I want to make a model by myself what tools/libraries do I need to make it possible?
Any help will be appreciated.
Answers:
The task you are searching for is called Question Answering. In this particular case, you something that is sometimes called Answer Span Selection or Reading Comprehension.
The problem is typically approached with deep learning. There is dataset called SQuAD created at Stanford University consisting of 100k questions paired with their answers highlighted in Wikipedia articles. This is what the models are typically trained on.
You can try an online demo by Allen Institute for AI to get an idea of how the state-of-the-art looks like. I believe, it should be also able to run it locally with the AllenNLP toolkit.
If you are looking for a service, you can have a look at QnA Maker of Microsoft Bot Framework and here is an article how to create one.
As they mentioned in the documentation-
Some knowledge bots may simply aim to answer frequently asked questions (FAQs). QnA Maker is a powerful tool that’s designed specifically for this use case. QnA Maker has the built-in ability to scrape questions and answers from an existing FAQ site, plus it also allows you to manually configure your own custom list of questions and answers. QnA Maker has natural language processing abilities, enabling it to even provide answers to questions that are worded slightly differently than expected. However, it does not have semantic language understanding abilities. It cannot determine that a puppy is a type of dog, for example.
As a service, QnA maker and bot framework are handy. The only constraint is 10,000 transactions per month, 10 per minute in the free subscription.
Have a look at Custom Question Answering from Microsoft. It is the successor to QNAMaker. It has a portal called Language studio to manage/train files for your bot.
https://language.cognitive.azure.com/
You have to create a Text Analytics(Language) service in Azure and login to Language studio to start uploading files to it and deploy, your bot will be ready.
You can try my application at this URL: https://github.com/muazhari/research-assistant-mini.
Currently, it has a passage search document and long-form-question answering feature based on a given document, URL, or text with the flexibility to change the model used. But beware, it is not yet peer-reviewed.
I want to make a chatbot that will answer the questions based on the given document.
E.g, if I have hundreds of documents and I want to get some information from it but don’t know which information is on which line of the page so I have to spend some time and effort to search. I want a chatbot that will learn from those documents and give answers form that documents.
Is there any available service that can full fill my needs?
What if I want to make a model by myself what tools/libraries do I need to make it possible?
Any help will be appreciated.
The task you are searching for is called Question Answering. In this particular case, you something that is sometimes called Answer Span Selection or Reading Comprehension.
The problem is typically approached with deep learning. There is dataset called SQuAD created at Stanford University consisting of 100k questions paired with their answers highlighted in Wikipedia articles. This is what the models are typically trained on.
You can try an online demo by Allen Institute for AI to get an idea of how the state-of-the-art looks like. I believe, it should be also able to run it locally with the AllenNLP toolkit.
If you are looking for a service, you can have a look at QnA Maker of Microsoft Bot Framework and here is an article how to create one.
As they mentioned in the documentation-
Some knowledge bots may simply aim to answer frequently asked questions (FAQs). QnA Maker is a powerful tool that’s designed specifically for this use case. QnA Maker has the built-in ability to scrape questions and answers from an existing FAQ site, plus it also allows you to manually configure your own custom list of questions and answers. QnA Maker has natural language processing abilities, enabling it to even provide answers to questions that are worded slightly differently than expected. However, it does not have semantic language understanding abilities. It cannot determine that a puppy is a type of dog, for example.
As a service, QnA maker and bot framework are handy. The only constraint is 10,000 transactions per month, 10 per minute in the free subscription.
Have a look at Custom Question Answering from Microsoft. It is the successor to QNAMaker. It has a portal called Language studio to manage/train files for your bot.
https://language.cognitive.azure.com/
You have to create a Text Analytics(Language) service in Azure and login to Language studio to start uploading files to it and deploy, your bot will be ready.
You can try my application at this URL: https://github.com/muazhari/research-assistant-mini.
Currently, it has a passage search document and long-form-question answering feature based on a given document, URL, or text with the flexibility to change the model used. But beware, it is not yet peer-reviewed.