Thursday, May 26, 2016

LUIS and the Bot Framework: A Natural Language Match

My previous Bot Framework posts have concentrated on the sophisticated conversation management facilities of Bot Connector and Bot Builder. While those are amazing capabilities, they don’t offer much more in user interaction than could be handled by a command line utility. For a bot to be truly conversational, it needs to move beyond primitive single word grunts and understand more naturally spoken language.

Language Understanding Intelligent Service (LUIS) is a natural language service that helps your bot be more conversational. It works by building models that classify utterances (language statements) and extracts data from those utterances, providing actionable results for your bot code. The Bot Framework has types that integrate smoothly with LUIS to help your bot understand natural language statements. This post will describe a model I created with LUIS and show you how to integrate that model into a bot.

The LUIS Model

To get started with LUIS, you need to build a model. A LUIS model is a service that uses machine learning to recognize utterances and provide a probability score of how close it thinks that utterance is to Intents and Entities. Intents are things that a user wants to accomplish or obtain information on. Entities are the data points associated with those intents. In your Bot Framework code, you map Intents to methods and use Entities like they were parameters. The LUIS Help pages has an excellent video that explains how to get started, building a LUIS model.

For this demo, I created a ContactInfo model. The purpose of the model is to support a contact management bot. The BotDemos source code for this post includes a LUIS Models folder that contains the ContactInfo.json file that contains the ContactInfo model. This model has a couple intents: None and ChangeInfo. The None intent is the default for catching unrecognized utterances. The ChangeInfo intent supports utterances for when the user wants to change some of their contact information. The ChangeInfo model has a single entity named ContactType, which could be an address, name, phone number, or email. Make sure your train and publish your model before trying to access it via the Bot Framework. After you publish your model, the App Settings on your page will contain an App Id and Subscription Key, which are necessary to add to your Bot code that integrates with LUIS, which is covered next.

Implementing a LuisDialog

To get started with using LUIS, I created a new Bot Framework project. If you don’t know how to do this yet, you can learn from one of my earlier posts, Pig Latin Bot. To start, I created a ContactInfoDialog class that derives from LuisDialog<object> like this:

        "<your App ID goes here>",
        "<your subscription key goes here>")]
    public class ContactInfoDialog : LuisDialog<object>
        public const string ContactType = "ContactType";
        string currentEntity = "";

As is typical of Bot Framework dialogs, you need to make it Serializable if you wish to retain state. The LuisModel attribute allows logic in LuisDialog to communicate with LUIS, using the App ID and Subscription Key from your model, ContactInfo in this case. The LuisDialog type parameter is the result type of the dialog, which contains collected information for a stateful bot. Since we aren’t maintaining state in this simple demo bot, I set the LuisDialog type to object. ContactType is the entity we’ll work with and it’s a const to avoid working with strings. Since later code navigates between methods, I’ll keep the name of the current ContactType entity in the currentEntity field. The ContactInfoDialog class has methods that handle intents and you’ll see what happens when LUIS doesn’t recognize an utterance in the next section.

Handling Unrecognized Utterances

You can set up a method in a LuisDialog to handle cases where your LUIS model doesn’t recognize an utterance. Just create the method and decorate it with a LuisIntent attribute containing an empty string. Here’s an example:

        public async Task None(
             IDialogContext context,
             LuisResult result)
            string userUtterance = result.Query;
            await context.PostAsync(
                $"Sorry, I didn't understand \"{userUtterance}\".");

As mentioned, the LuisIntent attribute with the empty string parameter indicates that LUIS didn’t understand what the user said. The None method has IDialogContext and LuisResult parameters. The IDialogContext parameter is the same as used in all dialogs and you can see how the method uses it to call context.PostAsync to let the user know that the bot couldn’t understand them and then context.Wait to wait for the next message. LuisDialog has an implementation of MessageReceived that it uses to interact with LUIS when the next message from the user arrives. You can use LuisResult to extract information about the user’s utterance, as None does by reading result.Query.

Note: You can visit the LUIS model, click on Review Labels, and classify utterances that aren’t part of your training set. This lets you continuously improve your LUIS model with real data.

Next, we’ll handle an utterance that LUIS does recognize.

Handling Valid Intents

To match methods to intents, decorate the method with a LuisIntent attribute containing a parameter matching the intent name from the LUIS model to handle. Then add the logic you need to handle what the user asks for. Here’s an example for the ChangeInfo intent:

        public async Task ChangeInfo(
            IDialogContext context,
            LuisResult result)
            EntityRecommendation entityRec;
            result.TryFindEntity(ContactType, out entityRec);

            currentEntity = entityRec.Entity;

                context: context,
                resume: ResumeAndHandleTextAsync,
                    $"What would you like to change your” +
                    $” {currentEntity} to?",
                retry: "I didn't understand. Please try again.");

        public async Task ResumeAndHandleTextAsync(
            IDialogContext context,
            IAwaitable<string> argument)
            string newEntityValue = await argument;

            await context.PostAsync(
                $"Your {currentEntity} is now {newEntityValue}");


The ChangeInfo method handles the LUIS ChangeInfo intent. The IDialogContext and LuisResult parameters serve the same purpose as described earlier, except that this time we need to read the associated entity to implement appropriate logic. Here, the code uses result.TryFIndEntity to obtain a reference to the ContactType entity. In real code, you would want to have more logic for recognition and validation of the entity.

The code performs a prompt for the text to change the bit of contact info to and then waterfalls to ResumeAndHandleTextAsync to work with the user’s response. Again, this code calls context.Wait(MessageRecieved) to continually wait for the user’s input.

Here’s an interaction with ContactInfo bot, showing how it responds to various utterances.



Now you know how to integrate a LUIS model with the Microsoft Bot Framework. Create and publish the model, create a class derived from LuisDialog, add App ID and Subscription Key to the LuisModel attribute, and use the LuisIntent attribute to decorate methods that handle LUIS model intents. For more information on running and testing a bot, you can visit previous posts in my blog. You can also find the code (including the ContactInfo model) on my GitHub site.



No comments:

Post a Comment