A First Foray into Alexa Skills Using AWS Lambda's with C#

Tonight I decided to try out something new that I had been meaning to for a while rather than continue poking around on my app.

Since Alexa's release I've been itching to play with it, I decided whilst I was learning one thing why not learn three things so decided to use AWS Lambdas with .Net Core as well.

I've used AWS extensively with work but never used Lambdas although I was aware of them and knew they were similar to Azure Functions, Ben Godwin's show on .Net Rocks also convinced me it was time to give them a proper look.

.Net Core I've also played with and had some great training from the .Net Core guys at NDC London from this year but I've never published anything. Tonight this was all going to change :)

A quick note before I continue, I'm writing this as I go so a few things might feel disjointed but hopefully it will all make sense.

First Steps - 1st Hour

In order to get going I set my self up a personal Amazon account, you can go do this..., once here I decided to get the Lambda setup sorted as I knew from my basic reading Alexa Skills can call out to these or a HTTPS endpoint. Lambda's are much easier than a full HTTPS stack.... well that's pretty easy now but lets not go there yet.

So usually I administer all of our AWS resources via the Web GUI's but others had mentioned the Visual Studio integration was fairly solid so I thought I'd also give this a go. I got the VS extension from https://aws.amazon.com/visualstudio/

Once this is installed follow the directions on creating an IAM account with appropiate keys and permissions so that the extension can be used to administer resources.

VS New Project

The AWS Explorer will then open up and I chose to dock this with my solution explorer etc.

I now wanted to create a basic function just to establish the basics are working. The VS Extension handily installed some new templates so when I clicked new project I got a nice AWS Lambda section. I chose to setup a new project and included tests... although how many I'll end up writing tonight is yet to be seen.

Upon creating the new project you get another wizard (blueprint) dialog to help you get going. I felt confident and chose empty project.

You get two nice projects with a basic hello world created and a test project with associated test. I chose to run this test to ensure my environment was setup correctly.

Boom a Test

Ok test works, excellent, I poke around at what the context gives me and it all looks interesting but rather than try and change anything yet I wanted to see this run in the cloud.... So lets try and deploy.

Publishing the Lambda is again really integrated into Visual Studio, simply right click the project and choose Publish to AWS Lambda. This presents a friendly form for you to complete, I chose the London region and then continued to complete the basic info. You can choose to save the options for late deployments, I suspected this would be one of many so chose to do so. The next page is about roles which looked confusing for a super novice so decided on the AWSLambdaRole... I hope this is OK for now, i'll tighten up the security later once I know what I'm trying to build :)

Magic...

I clicked upload and magic offically happened. Now I know its just use the AWS management API with .Net Core's awesome CLI but knowing it and seeing it is still special.

Once the upload is complete you can laucnch the Lambda Function view which allows you to get an overview of your function but also send test requests at it and and see the response. It also has a drop down of example requests. I chose Hello World and assumed this was for my nice template from the beginning.... I was wrong :(

FAIL

So you will see a lovely exception on the right. A big fat JSON one... hang on whats happened..... So if you look at the sample input you see its an object with some properties, but the json exception is winging about the {. Experience teaches me that this sounds like a method deserialising but only taking a string not an object..... Time to look at that default source code. Low and behold I find: 

public string FunctionHandler(string input, ILambdaContext context)

So the example request is wrong, pass "test" and the response comes back as "TEST". Amazing times, the world has been forever changed. 

Baby Steps - Hour Two

AWS Lambda is working (well kinda) now to hook up Alexa and begin to change the world. Now Alexa is not found within VS, instead you need to access it via the web portal so goto https://developer.amazon.com/edw/home.html#/skills. Once here you can click add a new skill and start the wizard process of building a basic skill. I chose custom skill with a language choice of UK provided and provided a name. One of the things you provide now is the Innocation Name (How Alexa knows its your skill the user wants to use). Choosing a good one of these I think could make or break your integration as it needs to be easier to say but distinct enough to be your skill. It's well worth reading the guidelines for help.

You then need to provide the JSON Schema for your Skill and I was like pants... this could be tricky. Fortunately as I've waited for a bit to try this out theres is now an interactive Skill Builder (although in Beta). I choose this route.....

Very Pretty

Now Skills Builder is a big thing, with lots to talk on so I'm opting to devote how to use this as a separate blog post. However the getting started videos help. I will update this post with a link to my follow up at a later date.

I'm still in Alexa Land - Hour Three

So playing with the Alexa Skills Builder took longer than planned but I believed I had something that might work , it compiled the model so it can't be too bad.... , anyhow the next step is configuration which sounds and looked like it should be easy.... I got stuck here for a little while :(

Easy right...

So when ever I provided the ARN to my lamda I got a super helpful error message:

Error: Invalid ARN for EU region. Please provide a valid ARN in the form 'arn:aws:lambda:eu-west-1:000000000000:function:myFunction:myFunctionVersion'

So immediate thoughts were I've typo'd, whats my function version etc..... 20 mins later nope its all correct. THEN  i chose to look at the tooltips and links next to the ARN selection. Nothing seemed to indicate it until I reread the developing article theres a nice blue section, which i think should be in red or orange) that says 

Lambda functions for Alexa skills can be hosted in either the US East (N. Virginia) or EU (Ireland)region. These are the only regions the Alexa Skills Kit supports.

Now anyone eagle eye'd might have noticed I deployed my lambda to London so long story short... I redeployed to Ireland...  I then got another error :(

Error: Please make sure that "Alexa Skills Kit" is selected for the event source type

So back to tweaking my lambda. The Visual Studio exprience doesn't allow you to setup triggers so instead I had to log into the web console and edited the lambda function's triggers. I'm pleased to say this was an easy intuative step. Click the empty box and choose the trigger, Alexa Skills Kit being what I needed.

Winning...

I had finally made it to testing my skill....

Voice Simulators are Fun... - Hour Three and a bit to  Four 

I finally got to the fun bit where I could see how my sample skill was going to pan out. 

The testing page can seem over whelming but its actually fairly simple. You have a voice simulator bit where you enter sentences and hear how Alexa will say and phrase them to ensure they sound correct, you can also provide help via SSML Now I may or may not have wasted valuable time getting Alexa to say many things that weren't related to my sample app but hey... It's getting late and how much wood would a wood chuck chuck, If a wood chuck could chuck wood? entertained me. Although in seriousness it was interesting to see how punctuation affected the result, try the following:

how much wood would a wood chuck chuck if a wood chuck could chuck wood
how much wood would a wood chuck chuck, If a wood chuck could chuck wood?
how much wood, would a wood chuck chuck, If a wood chuck could chuck wood?

This started me thinking how I might need to think about how I returned my results and ensure the data sounds fluent, not just "the result is XYZ".

The service simulator section is where you get to ensure that your Skills Interaction Model is as you expected and what gets passed to your Lamda function. You have to pass it sample utterances (the things people will say to Alexa) and you get the output. Now whilst I was playing with this I discovered I needed to tweak the interaction model so opened this in a separate tab and made my change. However what I hadn't realised was you can't test your model whilst it's building an update to the current one. Building models takes between 2 - 5 minutes which isn't a huge amount of time but when you aren't expecting it is a pain.

When you use the service simulator its shows the request its generated to send to your service with the JSON payload which is handy. At this point I relised the hello world app doesn't really support the intent's JSON so it was time to switch back to Visual Studio and update the Hello World function to recieve Alexa input and produce a response. Before doing this though grab the generated service request JSON as it can be used to test the Lambda function from within Visual Studio.

Paste as JSON

My first thoughts within JS for handling the Intents JSON response was to see if NuGet could help with the boilerplate code. A quick search found AlexaSkillsKit.NET however it seems its not actively maintained nor does it support .Net Core. Slight.Alexa also looked like an option but it targeted .Net Standard 1.2 but at present AWS Lambda functions are stuck at .Net Core 1.0 .

At this point I gave up on NuGet and decided it was time to see if I could easily slap something together. I remembered there's a feature I've never used much in Visual Studio that came in 2013 (I think.... ) where you could paste JSON as classes, so I took the JSON from the Service Request textarea from earlier and gave it a go.

The resulting classes looked a bit ugly due to the property naming being a bit whack but for a quick hack I just left it. (It did make me rather twitchy as I hate leaving things that are simply wrong but i was running out of time.....)

Let's do this - Hour 4- 5

So I had my classes thanks to paste as JSON so it was time to play with the function to return something meaningful.

Ideally I wanted dyanmic data but given I was almost at my time limit I decided to simplify my task and instead see if I could use the user input to vary the response.

Although I haven't delved into the Skills Builder in this post as thats for later I do want to point out some cool things you can do.

When you make an Alexa Skill you provide utterances for people to say to the device and as part of these utterances you can have variables that get populated by the user, these can be pulled from a list of valid options etc. When I came to take the input data I thought about how I would return it. 

Originally my utterance was the basic "Alexa Order me a Taxi from my location". My basic Lamda then just returned "Your Taxi will arrive in 5 minutes". But I pondered if saying Taxi is correct. Some people might say Cab. So I tweaked my model to have Taxi as a slot (variable ). I could then use this to return Your Taxi.... or Your Cab meaning Alexa could use the same natural language as the user. This I think provides huge impact and its the little difference that would make an app feel more immersive than just a basic computer response.

Once I had my Lambda generating data as I expected I then needed to find out the format Alexa expects the result. I found the JSON schema and hacked together a rough skeleton response.

Once the result object was being correctly populated the service simulator rendered the JSON response and provides a handy play button to hear Alexa read your output. If you implement cards (more on that at a later date) you also get a preview of how that looks.

A few things to note when developing your functions for Alexa.

When you use the Service Simulator you get no error details, simply an error message. This is good as none of your internals are leaked but it really does mean you need to be writing tests or if exploring the API's using the Functions test panel to be posting the JSON request retrieved from the service simulator to see what is actually happening. I know I had an object reference exception as I was expecting something to exist that wasn't always.

It appears the JSON is case sensitive, as I manually wrote the skelton objects for the response I found the simulator either errored or returned random results with no data. I then used paste as JSON on the sample response and updated my property naming and everything started working. I then changed the casing back and found it broke again. When I take this code forward I'll have to see if the JSONSerialiser settings can be tweaked to auto change the casing like JSON.Net allows.

.Net Core doesnt have a ToTitleCase method baked in :(

Testing with Alexa

Up to this point all of my testing has been via text entry into the simulator, no voice input. Now I wanted to see how it worked in real life. Unfortunately I don't yet have an Echo Dot or an Echo but I did find a website you can use instead. https://echosim.io/ allows you to log into your Amazon account and test your skills via your browser and your microphone. 

Unfortunately with it being super late now and with my laptop have a questionable microphone I couldn't get this working but tomorrow I aim to use my Kinect or a headset to see if I can test this fully. Or I may go buy an Echo Dot.... we'll see :)

[Update 6:45 AM]
So when I got up this morning I decided to quickly test it one last time, I found that by having Alexa App on my phone open whilst using echosim I was able to see what Alexa was hearing. In my my case I had wrongly used a . in my Invocation Name. You use . to allow Alexa to under stand BBC (B.B.C) ie how you say it. But if you have something like iCab you don't use i.Cab you simply say i Cab. Lesson learnt :)

What's also excellent is that if your skill returns a card the Alexa app shows it :)

Summary

Now at this point we have a basic Alexa Skill working pretty much end to end. Pretty awesome for a few hours dev, most of the time has been in wiring everything up and understanding the JSON schema's but its very powerful and I can see this turning into a full R&D task for my work and hopefuly become a product feature.  I didn't think this evening I'd get anywhere near as far as I had, having basic voice and card responses is pretty awesome.

Now there's tons of scope that can be added to skills, you can make them more interactive , allow Alexa to provide further details, audio files and all sorts but this has hopefully provided a little insight on how easy it is to get started with Alexa and you don't even have to host the service endpoints yourself if thats a blocker to get started.

One final thing, it's worth clarifying that at this point the Alexa Skill can't be submitted for certification yet as it's missing Help and other requirements to pass certification but its not far off and I could easily see a day's worth of effort on this and you'd have a skill providing some meaningful responses.