Last we we highlighted, Jumpstart your Win10 IoT development with the AllJoyn Studio for Visual Studio 2015. This post from Mike Taulty, Official Coding4Fun Friend seemed like a great follow-up.
Not only do we have AllJoyn but how can you go wrong with Cortana too?
Windows 10–Adding Cortana and Voice Control to the UWP/AllJoyn/IoT Core/Lightbulb Demo
This post really follows up on the demo that I built up in this previous post;
Windows 10, UWP, Raspberry PI 2, AllJoyn and Lightbulb Demo
in that screen capture and blog post I walked through a demo of code where I was using AllJoyn in Windows 10 across desktop/phone and IoT devices (Raspberry PI 2 in my case).
It’s a simple scenario where I cooked up an imaginary ‘lightbulb’ interface which I then implement in an app in a couple of ways;
I showed that working in the previous video and the code is all available for it on the blog post.
At the time though, what I really wanted to do was to control it with Cortana but I didn’t have Cortana working on my PC at the time that I wrote that post.
Since then, I’ve moved to Windows 10 build 10240 and I have a working Cortana (albeit after some struggles around UK/US regions and settings) and so I thought that I’d try it out with my goal being to enable this (made up) scenario;
I thought it might be a fun thing to play with and I could certainly see some future scenario of shouting at the XBOX One to turn the lights on outside or similar.
Everything I write here builds on the previous post and so I wouldn’t expect this to make any sense if you haven’t seen that post and I’m only going to write up the additional pieces here rather than run through the whole thing again.
Before I go there though, here’s a quick screencapture/video of me trying out the bits I put together;
from Vimeo.
Additions to the Existing Code
The first thing that I felt that I needed to do was to add a little more ‘functionality’ to my lightbulb interface beyond switch [on/off] and so I added a couple more simple methods;
...
Addition 2 – Linking In Cortana
In trying to do something with Cortana, I made reference to the sample on GitHub but whereas that sample shows two scenarios;
I focused entirely on (2) here and so my app is made up of 2 projects – a blank UWP app and a Windows Runtime Component to host my background task. I made the UWP app reference the component project so that the bits get copied over at build time.
...
challenges…
The Background Task
Where I found writing a background task to be a bit of a challenge here is around the generated types that I seem to get from my AllJoyn interface via the alljoyncodegenerator.exe tool. What I need to do is something like;
However…the class that I get generated from the AllJoyn interface is a lightbulbWatcher class and what it knows how to do is to wait around for lightbulb services to show up on the network.
What it doesn’t seem to be particularly set up to do is to get me all of the lightbulbs that are on the network at a specific point in time.
Meanwhile, Cortana is a demanding client – she’s not going to hang around for ever while I build a list of lightbulbs
...
I found it an interesting thing to experiment with though and I could definitely see this sort of idea being built into apps such that I can control devices around the house through home automation.
I’m fairly certain that companies like Insteon are doing work with both AllJoyn and Cortana so I’m sure it’s something that we’re going to see a lot more of.
If you want the code for all of this (warts and all!) then download from here.
[Click through to read Mike's entire post, see the code and more]
Just a few of the other time's Mike's been on Channel 9;
Follow @CH9
Follow @coding4fun
Follow @gduncan411
Continue reading...
Not only do we have AllJoyn but how can you go wrong with Cortana too?
Windows 10–Adding Cortana and Voice Control to the UWP/AllJoyn/IoT Core/Lightbulb Demo
This post really follows up on the demo that I built up in this previous post;
Windows 10, UWP, Raspberry PI 2, AllJoyn and Lightbulb Demo
in that screen capture and blog post I walked through a demo of code where I was using AllJoyn in Windows 10 across desktop/phone and IoT devices (Raspberry PI 2 in my case).
It’s a simple scenario where I cooked up an imaginary ‘lightbulb’ interface which I then implement in an app in a couple of ways;
- If the app is running on a non Windows IoT device then the interface just changes the colour of a path on screen.
- If the app is running on a Windows IoT device then it will additionally attempt to light up a real LED.
I showed that working in the previous video and the code is all available for it on the blog post.
At the time though, what I really wanted to do was to control it with Cortana but I didn’t have Cortana working on my PC at the time that I wrote that post.
Since then, I’ve moved to Windows 10 build 10240 and I have a working Cortana (albeit after some struggles around UK/US regions and settings) and so I thought that I’d try it out with my goal being to enable this (made up) scenario;
- A lightbulb service comes onto the AllJoyn network and advertises itself in some location (e.g. office, kitchen, etc)
- A user should be able to ask Cortana to show all the lightbulbs that ‘she’ can see.
- A user should be able to ask Cortana to turn on/off lightbulbs.
I thought it might be a fun thing to play with and I could certainly see some future scenario of shouting at the XBOX One to turn the lights on outside or similar.
Everything I write here builds on the previous post and so I wouldn’t expect this to make any sense if you haven’t seen that post and I’m only going to write up the additional pieces here rather than run through the whole thing again.
Before I go there though, here’s a quick screencapture/video of me trying out the bits I put together;
from Vimeo.
Additions to the Existing Code
The first thing that I felt that I needed to do was to add a little more ‘functionality’ to my lightbulb interface beyond switch [on/off] and so I added a couple more simple methods;
...
Addition 2 – Linking In Cortana
In trying to do something with Cortana, I made reference to the sample on GitHub but whereas that sample shows two scenarios;
- Voice commands being used to launch the foreground UI of an app and steer it to navigate to particular content.
- Voice commands being used to speak to a background service of an app and steer it to perform specific tasks with UI hosted on Cortana’s own canvas.
I focused entirely on (2) here and so my app is made up of 2 projects – a blank UWP app and a Windows Runtime Component to host my background task. I made the UWP app reference the component project so that the bits get copied over at build time.
...
challenges…
The Background Task
Where I found writing a background task to be a bit of a challenge here is around the generated types that I seem to get from my AllJoyn interface via the alljoyncodegenerator.exe tool. What I need to do is something like;
- Receive a voice command from Cortana like ‘show all the lights’.
- Execute a query to find all the lightbulb services on the network.
- Format the results back to Cortana.
However…the class that I get generated from the AllJoyn interface is a lightbulbWatcher class and what it knows how to do is to wait around for lightbulb services to show up on the network.
What it doesn’t seem to be particularly set up to do is to get me all of the lightbulbs that are on the network at a specific point in time.
Meanwhile, Cortana is a demanding client – she’s not going to hang around for ever while I build a list of lightbulbs
...
I found it an interesting thing to experiment with though and I could definitely see this sort of idea being built into apps such that I can control devices around the house through home automation.
I’m fairly certain that companies like Insteon are doing work with both AllJoyn and Cortana so I’m sure it’s something that we’re going to see a lot more of.
If you want the code for all of this (warts and all!) then download from here.
[Click through to read Mike's entire post, see the code and more]
Just a few of the other time's Mike's been on Channel 9;
- Conversations with Cortana
- Getting Real with Mike Taulty and RealSense
- Kinect to Fingers, hands, pen and a mouse...
- Hello (Kinect for Windows v2) World Series
- Kinect Studio and a Harmonica Player
- From Zero to Hero, Kinect to Sphero
- Saluting the Visual Gesture Builder - Details and example code
- Just face it... with these two Kinect for Windows v2 SDK Face Examples
- Getting Going with Kinect v2 Development
Follow @CH9
Follow @coding4fun
Follow @gduncan411
Continue reading...
