Wednesday, 30 August 2017

ADO.Net Async with Transactions... A Lesson Learned

Last night I was testing out a new WebApi Filter I've added to a project, its fairly simple, log some data that comes from a HTTP header to the DB.

This particular project has no Entity Framework or anything, just simple ADO.Net, in fact I think the project has 3 stored proc's in total.

I tend to Async all the things by default so when I originally wrote my DB code I had:

using (var conn = new SqlConnection(_connectionString))
     await conn.OpenAsync();
     using (var transaction = conn.BeginTransaction())
          var command = conn.CreateCommand();
          command.CommandText = "[Api].[LogDeviceRequest]";
          command.CommandType = System.Data.CommandType.StoredProcedure;
          command.Transaction = transaction;

             await command.ExecuteNonQueryAsync();

Looks fairly harmless.

Initial testing of me just poking around with my app showed it to be OK so I was happy and released it.... 20 mins later I had a shock when I looked at my DB looking unhappy.

Uh Oh....
Some very pretty locks there....

So I immediately started to look at my query, its fairly basic, if something exists update a timestamp, else insert new row. I tinkered with it and stopped it doing a key lookup but that wasn't the cause of the issue.

It was getting late so I decided to turn the feature back off and look at it again with fresher eyes this evening.

When looking at it this evening I realised I was calling the code above without awaiting it, I was simply firing and forgetting. Nothing obvious jumped out at me, until I removed the async parts of the code and made it syncronous. The deadlocks then went away. I could have left it like this but I wanted to see if I could make it async again more so for piece of mind.

Whilst sat pondering over this I remembered how async/await will by default try to return to the original syncronisation context after its await. explains alot of these things. I then had a theory (that wasn't based in a lot of knowledge but investigated anyway) that perhaps the transaction context and my syncronisation context was somehow messed up, was it trying to return on the original ASP.Net syncronisation context.

To test this I simply altered my await statements in the code above to use .ConfigureAwait(false);
When you do this you are telling .Net that you don't have to continue on the original SyncronisationContext continue on what ever you want.

Initial testing seemed fine again, so I deployed and triggered a load test (which previously gave deadlocks within minutes) to find the deadlocks all gone.

This does seem to point to the transactions not being committed properly and hanging around for a while, maybe when its trying to get back to the SyncronisationContext. A very weird issue, I wonder if its related to the issue TransactionScope has with async pre .Net 4.5.1. Hopefully someone can tell me :)

Hope this helps someone, it certainly reminds me to test things under load :)

Thursday, 24 August 2017

Playing further with ASP.Net Core whilst on my hols... VSCode fun and cool new Anti Forgery Validation

I'm on vacation, 10 days in Norfolk with extremely limited connectivity and action packed days with the kids. Whilst here on two nights everyone has fallen asleep before me so I decided it would be a good time to poke around further at ASP.Net Core as I've still not produced any production code with it yet, only small samples.

Now I don't have my laptop on me but I do an old one that's here purely for any work emergencies and for childcare emergencies, (you know the ones, its 6am and the 2 year old is awake, peppa pig at 6am is an emergency!). This means I have literally nothing on me and I can only tether internet via my wife's iPhone 6 which has barely a bar of signal, this rules out a cloud VM running VS. So I turned to VSCode, its sooo lightweight its easy to download, install and run on this laptop. Get the .Net Core 2.0 SDK and the VSCode C# extension and you are literally running within 30 mins.

I won't go to much into the projects setup and stuff as its well documented but I want to highlight a few things I've stumbled upon and thought worth sharing.

Installing Packages

In Visual Studio I would just use the Nuget package manager UI to manage all of my dependencies, I know I could use the package manager console but I've become lazy and use my mouse too much..... But in VSCode this doesn't exist without further extensions.

So how do you install packages?

Turns out its really simple, bring up your terminal window (be this console, powershell, bash etc) and use the .Net Core CLI. You simply type:

dotnet add package [packagename]

where [packagename] is what you want to install. This then uses Nuget to find the package and install appropriately. Very quick and powerful.

Anti-Forgery Validation

If you have done any ASP.Net MVC applications you have discovered and used Anti Forgery Validation. I won't go into the details as many places discuss its benefits and importance. Within ASP.Net Core I've found things have improved further. Previously with Razor you had the @HTML.AntiForgeryToken helper syntax and attributes which you start to litter your views and controllers with.

Within ASP.Net Core things have been tidied up, Red Gate have a great post covering the specifics, but essentially if you use the asp tag helpers to generate a form post url (which you almost always will be when building mvc applications) and add a global anti forgery filter you never have to worry about decorating actions with attributes or using the html helpers within your razor views. You get excellent protection pretty much for free which is ace!

That's it for tonight, I'll blog more as I try out more :)

Wednesday, 16 August 2017

Code therapy brought to you this week by .Net Core 2

So the last week has been hard going so tonight I decided to have some code therapy. I've been meaning to explore .Net Core 2 further and decided tonight was the time to port a library I had written in Mono for Raspbian over to .Net Core land.

It's worth noting that .Net Core 2 is only in preview on Raspbian but thats good enough for me ;)

Getting Started

First things first I updated VS2017 to the latest update 3 release. My Visual Studio wasn't telling me the update was available so I simply got the web installer again from and it advised VS was already installed and could be updated. This took about 35 mins on my Dell XPS 13 laptop but on my SurfacePro 3 its taken nearly an hour.

.Net Standard Choices
I then chose to create a new .Net Standard project but found I couldn't target .Net Standard 2. Turns out Update 3 doesn't include .Net Core 2 you need to go get this from:

You can now choose .Net Standard 2.0

I then created Classic .Net Console Application and a .Net Core Console Application to use my class library in.

Adding a reference for the .Net Standard library in Classic .Net Console Application is easy enough,  References > Add Reference but when it came to .Net Core I had to look twice. In .Net Core References is gone, its replaced by Dependencies. Dependencies nicely wraps up project references, service references and Nuget Packages. So right click Dependencies and you get the famiiar Add Reference.

Where's References Gone... Oh There You Are!

Porting My Library

So to start with I just grabbed the code from my Mono project and dropped it into the new Standard Library, it was easier to avoid any complications by simply copy/pasting the files, I have no doubt I could have messed around with the old project to make it a standard one but why gamble.

I then thought lets do a build and see what doesn't work, its a very simple project, basic models, interfaces, but I do use Process.StartNew new to call out and grab some info using Bash on Raspbian, I expected this to have some quirk or API change.

Turns out it built perfectly, 0 build errors, 0 warnings. Crazy.

Next I implemented a very basic console app, again same code in both applications and ran both applications. Spot the difference....

Same output the only difference is how the app is ran, in the classic world we compile an executable where as an net core app is compiled to a dll which is then ran via the dotnet cli.

Next Steps

I now wanted to get this running on my Raspberry Pi, the classic console application can be ran via Mono, this just got a 5.2 release. For the .Net Core Console app I needed to ensure its compiled for ARM32, to do this I used the .Net CLI and published the application. Rather than repeat how to do this Microsoft have all of this on GitHub.


I'm very impressed with how compatible .Net Core 2.0 is , this was a very small library that ported seemlessly but this really opens up the migration path for many of my other ones and begins to make me think I need to rely on Mono far less. I'm looking forward to the future.

Wednesday, 26 July 2017

Its the little things...

In the history of C# we have been spoiled by every version having excellent new language features in. We had Generics, Linq, Tasks, async/await which were massive hitters and transformed the way we worked and wrote code.

C#7 however I found has given us lots of little improvements that make things smoother and easier which has been great. One feature I hadn't really used yet though was the new Pattern Matching, this wasn't because I hadn't wanted to but more I hadn't seen a time to really use it.

That was until tonight.

A year or so ago I had a scenario where I had to take a base type and then1 return the appropiate mobile view to render. Prior to Pattern Matching I ended up with something like:
if (value is HoldingPage)
return new LoadingView();
var listingPage = value as ListingPage;
if (listingPage != null)
return new ListingView(listingPage);
Tonight I needed to add a new view into this code and decided it needed tidying up. Pattern Matching allows us to test that a value has a certain shape, and when its matched provide a variable name for it which we can then continue to use. So rather than have lots of as statements with null checks  we can simply do:
switch (value)
case HoldingPage p:
return new LoadingView();
case ListingPage p:
return new ListingView(p);
case ContentPage p:
var sectionId = parameter as string;
return new DetailsView(p, sectionId);
case IndexPageWithTabledSections p:
return new TabledSectionListingView(p.Sections);

Less code and far more concise and readable. What a great feature!

Thursday, 13 July 2017

Fixing a Quirky "Xamarin.Forms ListView Bug" That Led Me Back to Basics And How Most Bugs Are Written By Users!

For the last two evenings I've been chasing what seemed a random bug. The crux of it came down to scrolling large lists within a ListView, 1000's of items, which were grouped, would muddle up the items and often duplicate an item a couple of times.

An example output:

First page of results:
Test 1
Test 2
Test 3
Test 4
Test 5
Test 6
Test 7
Test 8
Test 9
Test 10
Test 11
Test 12
The second page would then render:

Test 13
Test 14
Test 1
Test 15
Test 16
Test 4
Test 17
Test 18
Test 19
Test 20
Test 6
At first glance this looked like a random error. I considered that maybe my list parsing code (the app takes HTML and makes in easier to use) but my unit tests checked out. So I went over these again with a toothcomb. All checked out.

Next I considered the code that grouped the items, again unit tests all good, toothcomb said still good.

So I sat pondering. Was there something similar about all of the duplicated items. There was, they were all items that didn't then link to a separate page.

I then suspected had I done a rookie mistake thats often easy to miss until you hit larger datasets. Had I overriden GetHashCode and done so poorly.... Let's write some tests to prove it...


So I hadn't modified the Equals and GetHashCode for a long time, I love how CodeLens helps with this. But I had added more properties to the object :(

Code Smells...

The Change

An age ago I had added Section Id to the object but I hadn't added it to the Equals and GetHashCode as it was added to the base class and I just passed it through.

So time to implement Equals and GetHashCode properly and for good measure IEquatable<T> . I've always found a good way to implement these is either let Resharper help you or use the great examples on a popular StackOverflow post. Jon Skeet's example is great but Rick Love's is also quite elegant.

The results...

And then within the app itself....
First page of results:
Test 1
Test 2
Test 3
Test 4
Test 5
Test 6
Test 7
Test 8
Test 9
Test 10
Test 11
Test 12
The second page:

Test 13
Test 14
Test 15
Test 16
Test 17
Test 18
Test 19
Test 20
Test 21
Test 22
Test 23
Test 24

So why would this issue cause the rendering issue? I suspected that the way the recycling listview works is by using HashSet's once an ItemsSource reached a certain size or as part of it's internals. I had a good poke at the Xamarin.Forms source code all on GitHub and I suspect its to do with the ListProxy   but the why isn't super important, the issue is fixed and the issue is relatively easy to spot.

So this was a real reminder how we can't forget about basics and hopefuly offers an insight in how to spot the associated issue. 

Tuesday, 4 July 2017

A First Foray into Alexa Skills Using AWS Lambda's with C#

Tonight I decided to try out something new that I had been meaning to for a while rather than continue poking around on my app.

Since Alexa's release I've been itching to play with it, I decided whilst I was learning one thing why not learn three things so decided to use AWS Lambdas with .Net Core as well.

I've used AWS extensively with work but never used Lambdas although I was aware of them and knew they were similar to Azure Functions, Ben Godwin's show on .Net Rocks also convinced me it was time to give them a proper look.

.Net Core I've also played with and had some great training from the .Net Core guys at NDC London from this year but I've never published anything. Tonight this was all going to change :)

A quick note before I continue, I'm writing this as I go so a few things might feel disjointed but hopefully it will all make sense.

First Steps - 1st Hour

In order to get going I set my self up a personal Amazon account, you can go do this..., once here I decided to get the Lambda setup sorted as I knew from my basic reading Alexa Skills can call out to these or a HTTPS endpoint. Lambda's are much easier than a full HTTPS stack.... well that's pretty easy now but lets not go there yet.

So usually I administer all of our AWS resources via the Web GUI's but others had mentioned the Visual Studio integration was fairly solid so I thought I'd also give this a go. I got the VS extension from

Once this is installed follow the directions on creating an IAM account with appropiate keys and permissions so that the extension can be used to administer resources.

VS New Project
The AWS Explorer will then open up and I chose to dock this with my solution explorer etc.

I now wanted to create a basic function just to establish the basics are working. The VS Extension handily installed some new templates so when I clicked new project I got a nice AWS Lambda section. I chose to setup a new project and included tests... although how many I'll end up writing tonight is yet to be seen.

Upon creating the new project you get another wizard (blueprint) dialog to help you get going. I felt confident and chose empty project.

You get two nice projects with a basic hello world created and a test project with associated test. I chose to run this test to ensure my environment was setup correctly.
Boom a Test
Ok test works, excellent, I poke around at what the context gives me and it all looks interesting but rather than try and change anything yet I wanted to see this run in the cloud.... So lets try and deploy.

Publishing the Lambda is again really integrated into Visual Studio, simply right click the project and choose Publish to AWS Lambda. This presents a friendly form for you to complete, I chose the London region and then continued to complete the basic info. You can choose to save the options for late deployments, I suspected this would be one of many so chose to do so. The next page is about roles which looked confusing for a super novice so decided on the AWSLambdaRole... I hope this is OK for now, i'll tighten up the security later once I know what I'm trying to build :)

I clicked upload and magic offically happened. Now I know its just use the AWS management API with .Net Core's awesome CLI but knowing it and seeing it is still special.

Once the upload is complete you can laucnch the Lambda Function view which allows you to get an overview of your function but also send test requests at it and and see the response. It also has a drop down of example requests. I chose Hello World and assumed this was for my nice template from the beginning.... I was wrong :(
So you will see a lovely exception on the right. A big fat JSON one... hang on whats happened..... So if you look at the sample input you see its an object with some properties, but the json exception is winging about the {. Experience teaches me that this sounds like a method deserialising but only taking a string not an object..... Time to look at that default source code. Low and behold I find: 
public string FunctionHandler(string input, ILambdaContext context)
So the example request is wrong, pass "test" and the response comes back as "TEST". Amazing times, the world has been forever changed. 

Baby Steps - Hour Two

AWS Lambda is working (well kinda) now to hook up Alexa and begin to change the world. Now Alexa is not found within VS, instead you need to access it via the web portal so goto Once here you can click add a new skill and start the wizard process of building a basic skill. I chose custom skill with a language choice of UK provided and provided a name. One of the things you provide now is the Innocation Name (How Alexa knows its your skill the user wants to use). Choosing a good one of these I think could make or break your integration as it needs to be easier to say but distinct enough to be your skill. It's well worth reading the guidelines for help.

You then need to provide the JSON Schema for your Skill and I was like pants... this could be tricky. Fortunately as I've waited for a bit to try this out theres is now an interactive Skill Builder (although in Beta). I choose this route.....

Very Pretty

Now Skills Builder is a big thing, with lots to talk on so I'm opting to devote how to use this as a separate blog post. However the getting started videos help. I will update this post with a link to my follow up at a later date.

I'm still in Alexa Land - Hour Three

So playing with the Alexa Skills Builder took longer than planned but I believed I had something that might work , it compiled the model so it can't be too bad.... , anyhow the next step is configuration which sounds and looked like it should be easy.... I got stuck here for a little while :(
Easy right...
So when ever I provided the ARN to my lamda I got a super helpful error message:

Error: Invalid ARN for EU region. Please provide a valid ARN in the form 'arn:aws:lambda:eu-west-1:000000000000:function:myFunction:myFunctionVersion'

So immediate thoughts were I've typo'd, whats my function version etc..... 20 mins later nope its all correct. THEN  i chose to look at the tooltips and links next to the ARN selection. Nothing seemed to indicate it until I reread the developing article theres a nice blue section, which i think should be in red or orange) that says 

Lambda functions for Alexa skills can be hosted in either the US East (N. Virginia) or EU (Ireland)region. These are the only regions the Alexa Skills Kit supports.

Now anyone eagle eye'd might have noticed I deployed my lambda to London so long story short... I redeployed to Ireland...  I then got another error :(

Error: Please make sure that "Alexa Skills Kit" is selected for the event source type

So back to tweaking my lambda. The Visual Studio exprience doesn't allow you to setup triggers so instead I had to log into the web console and edited the lambda function's triggers. I'm pleased to say this was an easy intuative step. Click the empty box and choose the trigger, Alexa Skills Kit being what I needed.

I had finally made it to testing my skill....

Voice Simulators are Fun... - Hour Three and a bit to  Four 

I finally got to the fun bit where I could see how my sample skill was going to pan out. 
The testing page can seem over whelming but its actually fairly simple. You have a voice simulator bit where you enter sentences and hear how Alexa will say and phrase them to ensure they sound correct, you can also provide help via SSML Now I may or may not have wasted valuable time getting Alexa to say many things that weren't related to my sample app but hey... It's getting late and how much wood would a wood chuck chuck, If a wood chuck could chuck wood? entertained me. Although in seriousness it was interesting to see how punctuation affected the result, try the following:

how much wood would a wood chuck chuck if a wood chuck could chuck wood
how much wood would a wood chuck chuck, If a wood chuck could chuck wood?
how much wood, would a wood chuck chuck, If a wood chuck could chuck wood?

This started me thinking how I might need to think about how I returned my results and ensure the data sounds fluent, not just "the result is XYZ".

The service simulator section is where you get to ensure that your Skills Interaction Model is as you expected and what gets passed to your Lamda function. You have to pass it sample utterances (the things people will say to Alexa) and you get the output. Now whilst I was playing with this I discovered I needed to tweak the interaction model so opened this in a separate tab and made my change. However what I hadn't realised was you can't test your model whilst it's building an update to the current one. Building models takes between 2 - 5 minutes which isn't a huge amount of time but when you aren't expecting it is a pain.

When you use the service simulator its shows the request its generated to send to your service with the JSON payload which is handy. At this point I relised the hello world app doesn't really support the intent's JSON so it was time to switch back to Visual Studio and update the Hello World function to recieve Alexa input and produce a response. Before doing this though grab the generated service request JSON as it can be used to test the Lambda function from within Visual Studio.

Paste as JSON
My first thoughts within JS for handling the Intents JSON response was to see if NuGet could help with the boilerplate code. A quick search found AlexaSkillsKit.NET however it seems its not actively maintained nor does it support .Net Core. Slight.Alexa also looked like an option but it targeted .Net Standard 1.2 but at present AWS Lambda functions are stuck at .Net Core 1.0 .

At this point I gave up on NuGet and decided it was time to see if I could easily slap something together. I remembered there's a feature I've never used much in Visual Studio that came in 2013 (I think.... ) where you could paste JSON as classes, so I took the JSON from the Service Request textarea from earlier and gave it a go.

The resulting classes looked a bit ugly due to the property naming being a bit whack but for a quick hack I just left it. (It did make me rather twitchy as I hate leaving things that are simply wrong but i was running out of time.....)

Let's do this - Hour 4- 5

So I had my classes thanks to paste as JSON so it was time to play with the function to return something meaningful.

Ideally I wanted dyanmic data but given I was almost at my time limit I decided to simplify my task and instead see if I could use the user input to vary the response.

Although I haven't delved into the Skills Builder in this post as thats for later I do want to point out some cool things you can do.

When you make an Alexa Skill you provide utterances for people to say to the device and as part of these utterances you can have variables that get populated by the user, these can be pulled from a list of valid options etc. When I came to take the input data I thought about how I would return it. 

Originally my utterance was the basic "Alexa Order me a Taxi from my location". My basic Lamda then just returned "Your Taxi will arrive in 5 minutes". But I pondered if saying Taxi is correct. Some people might say Cab. So I tweaked my model to have Taxi as a slot (variable ). I could then use this to return Your Taxi.... or Your Cab meaning Alexa could use the same natural language as the user. This I think provides huge impact and its the little difference that would make an app feel more immersive than just a basic computer response.

Once I had my Lambda generating data as I expected I then needed to find out the format Alexa expects the result. I found the JSON schema and hacked together a rough skeleton response.

Once the result object was being correctly populated the service simulator rendered the JSON response and provides a handy play button to hear Alexa read your output. If you implement cards (more on that at a later date) you also get a preview of how that looks.

A few things to note when developing your functions for Alexa.

When you use the Service Simulator you get no error details, simply an error message. This is good as none of your internals are leaked but it really does mean you need to be writing tests or if exploring the API's using the Functions test panel to be posting the JSON request retrieved from the service simulator to see what is actually happening. I know I had an object reference exception as I was expecting something to exist that wasn't always.

It appears the JSON is case sensitive, as I manually wrote the skelton objects for the response I found the simulator either errored or returned random results with no data. I then used paste as JSON on the sample response and updated my property naming and everything started working. I then changed the casing back and found it broke again. When I take this code forward I'll have to see if the JSONSerialiser settings can be tweaked to auto change the casing like JSON.Net allows.

.Net Core doesnt have a ToTitleCase method baked in :(

Testing with Alexa

Up to this point all of my testing has been via text entry into the simulator, no voice input. Now I wanted to see how it worked in real life. Unfortunately I don't yet have an Echo Dot or an Echo but I did find a website you can use instead. allows you to log into your Amazon account and test your skills via your browser and your microphone. 

Unfortunately with it being super late now and with my laptop have a questionable microphone I couldn't get this working but tomorrow I aim to use my Kinect or a headset to see if I can test this fully. Or I may go buy an Echo Dot.... we'll see :)

[Update 6:45 AM]
So when I got up this morning I decided to quickly test it one last time, I found that by having Alexa App on my phone open whilst using echosim I was able to see what Alexa was hearing. In my my case I had wrongly used a . in my Invocation Name. You use . to allow Alexa to under stand BBC (B.B.C) ie how you say it. But if you have something like iCab you don't use i.Cab you simply say i Cab. Lesson learnt :)

What's also excellent is that if your skill returns a card the Alexa app shows it :)


Now at this point we have a basic Alexa Skill working pretty much end to end. Pretty awesome for a few hours dev, most of the time has been in wiring everything up and understanding the JSON schema's but its very powerful and I can see this turning into a full R&D task for my work and hopefuly become a product feature.  I didn't think this evening I'd get anywhere near as far as I had, having basic voice and card responses is pretty awesome.

Now there's tons of scope that can be added to skills, you can make them more interactive , allow Alexa to provide further details, audio files and all sorts but this has hopefully provided a little insight on how easy it is to get started with Alexa and you don't even have to host the service endpoints yourself if thats a blocker to get started.

One final thing, it's worth clarifying that at this point the Alexa Skill can't be submitted for certification yet as it's missing Help and other requirements to pass certification but its not far off and I could easily see a day's worth of effort on this and you'd have a skill providing some meaningful responses.