Sessions is temporarily moving to YouTube, check out all our new videos here.

Machine Learning for Muggles

Martin Kearn speaking at Bristol JS in March, 2017
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

We’ll look at some of the basics around the Azure Machine Learning service, and at some of the Microsoft Cognitive Services APIs, which make it very easy to implement artificial intelligence ‘smarts’ into your application with simple, standard REST calls. We'll focus on JavaScript for any demos, but this technology is largely platform agnostic and can be used for any type of app in any language.


Hi folks, I'm Martin Kearn. I was nearly Michael Kearn, but I'm not, I'm Martin Kearn. I'm here today, I'm from Microsoft. I work in the Evangelism Team there at Microsoft, and I'm here to talk about Machine Learning. So, does anybody self identify as being a boffin in the room today? Any data scientists? Anybody that's really into like, maths and sort of, data science, all that kind of stuff? No, you're all JavaScript Developers? Good, okay, because I'm gonna slag boffins off a little bit, so it's fine. So boffins are quite important. Especially in the world of Machine Learning. In fact, Machine Learning really comes from boffins. We have quite a few boffins at Microsoft. We keep a lot of them in Cambridge. We've got boffin pens all around the world. There's a big pen in Cambridge there. There's another boffin pen in Singapore. There's a boffin pen in Seattle. We call them Microsoft research but really they're boffin pens, where we keep our boffins. And all of the Machine Learning stuff that comes out of Microsoft, mainly comes from Cambridge. So if you're using any of the Machine Learning services, any of the cognitive services, mostly it's been developed in Cambridge, which is quite a cool thing for the UK. I did this talk last night in Oxford. And I have to be a bit careful because there were definitely boffins in the room in Oxford. But I think we're okay in Bristol. They've got this really nice office there. Custom built office, right in the heart of Cambridge. It's one of the most expensive offices Microsoft has, in terms of real estate. On the walls is scientific art. It's not just regular art, it's definitely scientific art. It has some scientific meaning that only boffins can understand. And in the reception, they've got a grand piano, which was donated by one of the leaders of the facility. It's called the facility. I work in Redding, - It is, it really is called the facility, or lab, sometimes they call it. I work in Redding and we've got Xboxes in our reception. So it just goes to show you the kind of different sort of people we're dealing with here. And these boffins do some quite scary things. These are some of the words that boffins use. So we've got things like Bayesian networks. We've got things like artificial neural networks, reinforcement learning, and these are actually Machine Learning algorithms, so they're not technologies. They're mathematical algorithms that have been invented in academia, and they're the basis of Machine Learning. And this diagram here is one of those words in pictorial form. So this is basically, I don't really understand much more about it, other than I know it's one of those algorithms, I don't know which one. It's definitely one of them though. So they did some pretty scary things. And actually, Machine Learning, generally, is a scary thing. If you are just a regular developer, whether it's JavaScript or whatever language you use. The idea of getting into Machine Learning is quite a daunting task, cause it is quite mathematical and it is kind of data science. It's not the sort of thing that a regular developer could ordinarily do, or at least it didn't used to be. But the good news is, it's for muggles too. So because nobody said that they were a boffin, you therefore must all be muggles. You don't possess the power and the magic that these boffins possess. You are just regular JavaScript developers, you don't do some data science nor the mess that goes behind it. Anybody disagree with that? There's one boffin over there. You kind of half put you hand up at the start, didn't you? Okay, so we can use this stuff as regular developers, it's quite straightforward and that's what I'm gonna go through today. So what is Machine Learning? We'll start with that. It's really about finding patterns in data. So if you've got a dataset, it doesn't have to be a big dataset, it can just be a regular-sized dataset or even relatively small. You think there might be some patterns, some connexions in that data. Machine learning is about finding patterns in that data and then provide insights for new data using those patterns. So a classic example, you've got a load of credit card transactions, you know the ones that are fraudulent. Machine learning will find the connexions, the patterns that make a fraudulent transaction what it is. Then when you've a new credit card transaction, it will say well based on what I know about the previous ones, the historical ones, this is 80% likely to be fraudulent or 40% or whatever it might be. So that's what Machine Learning really at it's heart is all about, it's finding patterns in data, that's the key thing. There are lots of use cases for this. So we've just talked about credit card fraud. We've got one about customer churn, so if you're using Machine Learning you can predict when a customer is about to leave your service based on the historical behaviour of customers leaving your service. So if you worked for somebody like Sky or somebody like that where customers leave the service, you should be able to predict that based on behaviour. What are the events that happened leading up to when somebody actually decides to leave the service. So what are their viewing habits? Has Game of Thrones just finished? And then their going to remove their subscription. Has the football season just finished? Something like that. You should be able to predict that using Machine Learning. We can do things like predictive maintenance. So Rolls Royce with their aeroplane engines, they actually Boeing and Airbus, all the aeroplanes that use Rolls Royce, they don't actually buy engines, they just lease them off Rolls Royce. And with that lease comes a maintenance contract. So generally speaking, if you're flying on a Boeing aeroplane, the engine that's built by Rolls Royce is maintained also by Rolls Royce. Boeing don't touch that engine at all. Now Rolls Royce had this kind of, as you would do with cars, I guess, it's like this scheduled maintenance period. So that part needs to be replaced after X number of miles and that part needs to be replaced after these number of miles, and after so many air miles, the whole thing needs to be taken out and cleaned or whatever it might be. And they were doing this on a schedule, just like you would do with a car. But what they found is that using Machine Learning, they were able to predict when the part might need to be changed or replaced, and that would vary by flight, it would vary by the type of aircraft it was associated with, it would vary by airline. And using predict maintenance, they were actually able to cut their cost right down in terms of being able to maintain things just when it needed maintaining rather than what the schedule says, which applied to everything. So there's lots of use cases for Machine Learning, and it's really any scenario where a pattern can be recognised in data, it's all about data. It's all about patterns in data. So let's explore credit card fraud for just a little bit. Can anybody in the room identify the pattern that makes a fraudulent transaction based on this dataset? Just shout out if you can spot it. - [Audience Member] The name starts with P. - Yes, the name begins with P. So is there any Pauls or Peters or Paulines or Pamelas in the room? Cause if there are, you must be a fraudster. Based on this dataset, data science proves that if your name begins with P, you must therefore be a fraud. Now clearly, that's not a very good pattern. We can't say that everybody whose name begins in P is a fraudster and that's because this data isn't big enough to draw any sensible conclusion from it. There is a pattern, but it's not a very good pattern. If we take a bigger dataset, still not very big, it's tiny really in terms of data terms. Can anybody spot the pattern here? There are four indicators in this dataset that shows that somethings a fraudulent transaction. Just shout out if you can see any of them. What was that sorry? - [Audience Member] Russia. - Russia, what is it about Russia? - [Audience Member] Used in Russia. - Used in Russia, okay. Yeah, that's one of them. - [Audience Member] Someone in their 20s. - Somebody in their 20s. Yeah, that's the second one. You were about to say. - [Audience Member] Issued in the USA. - Issued in the US. Yeah, that's three, there's one more. - [Audience] The amount. What's that sorry? - [Audience] The amount. - Nope, or it might be, but it's not one of the ones that I've got on my clicker, so no. - [Audience Member] All under 30. - Yeah, so they're in their 20s and their 30s. The final one is that the amount is over a thousand pounds. So, over a thousand pounds, issued in the USA, used in Russia, and the person is in their 20s. Now that's starting to feel like a relatively sensible set of rules, isn't it. You can make some of that seems about right based on the data that we've got, that seems about right. Now there's a room of about 40, 50 people here and we're all relatively smart programmers, developers, and it took us maybe a minute to figure that out. Humans are not very good at spotting patterns in data. We're just not built, our brains don't work in that way. But machines are very good at this, they can scan data. They can find the connexions really, really quickly. And that's really what Machine Learning is all about. So imagine if you had 10 million records and 60 columns. There's just no way a human or even a 100 humans would be able to find any meaningful patterns in that data, but a machine can. It can do it very, very quickly and very accurately. And it can find patterns without bias as well. So humans have bias, they might have stereotypical views about credit card fraud. Machines don't have any of that. It just goes pure down the data. So the process behind Machine Learning, is you start with some data. It doesn't have to be big data. It doesn't have to be one set of data. It can be multiple datasets, and you're starting to question what you're trying to find out, what's the thing you're trying to prove. You then go through this process of kind of cleansing the data and preparing it for Machine Learning. So an important point of Machine Learning is that it's better to have complete and accurate data, then lots of data. So if you've got rows that have got bad data in them or missing columns, or missing values, or junk data, just get rid of all that stuff. We don't want that. We want a very clean, very correct dataset that conforms to the scheme of what the thing is you're looking at. And that process is called data pre-processing and what you end up with at the end of that, is a prepared dataset. So you take raw data in, you do a load of cleaning and processing work, and you end up with a prepared dataset at the end of that. You then apply one or more algorithms to that dataset. So those words that we saw at the start, those are Machine Learning algorithms. So there's loads of them, and a data scientist would know which algorithms to use, and which scenarios, with which data sets, and you'd experiment with different ones. You'd go through a few cycles, and you end up with a thing called a Candidate Model. The model is kind of like the rule set if you like, it's the thing that machine-learning produces. It's all about producing this model, which is a set of rules that you can compare new data to. So you end up with a Chosen Model at the end of it, and then your application will use that model with new data. So your load of credit card transactions go in, we'll clean them out, we make sure there's no null data, no bad data in there, we apply some Machine Learning algorithms to that data, we get a chosen model, and then our applications can use that with new credit card transactions. That's basically the idea. So in the Azure, we have service for this. You can do this in Azure on the cloud, and it provides you those processing modules, in kind of software form, you just drag and drop them onto your experiment. You don't need to code them or sort of know how they work really, just need to know which ones to use. Same for the algorithms, you just drag and drop those in. Again, you need to know which ones to use, but other than that you don't need to know the maths behind them or how they're coded. The model is exposed as an API, so just a RESTful API, you can call in any language, in any platform, and then we have this thing, called the Azure Machine Learning Studio, which is the kind of interface you use to actually do all of this. So I'm going to take a look now, at trying to do that car price thing, with actual Machine Learning now. So if you go to, it will just search for Azure Machine Learning, this is basically where you'll end up. Now I've got a dataset already in here, of car price data, so this is basically a set of cars with brake horse power, four-wheel drive, lots of data about cars basically. It's not a massive data set, I think it's got about 200 rows in it. So I've already uploaded that, and I'm going to create a new experiment. So in Machine Learning world, a project is an experiment. It basically means the same thing. So I'm just going to do a blank experiment. We get this kind of canvas here, so what we have to do, is we kind of drag things on to the canvas and stitch them all together as like a pipeline that the data goes through. So if I take my saved data set, I can just drag that on, and drop that into the canvas, and I can click on here and see visualise, and it's going to show me what the data looks like. So we can see, we've got a load of data in here, there's 205 rows. We can see that we've got the make, the fuel type, how many doors the cars got, the body, the drive, so there's a load of data about each car. There's some missing data, so. You can see that this normalised losses one seems to have quite a lot of missing data. We can scroll down here and see that some of the other ones might have missing data as well. There's a bit there, so it needs some cleaning up. It's not a good data set for Machine Learning straight away. So let's start that process of cleaning it up. First thing we need to do, is we want to remove that normalised losses column, because it basically has the same values for every car. So it's not of any use to us, in terms of finding patterns. So to do that, we're going to get a module called Select Columns in Dataset, and we just drag and drop that in, and we do this thing of kind of connecting the output of one to the input to it, so the output of my raw data set becomes the input to Select Columns. And I can choose this column selector, and I can choose to exclude, whoops, exclude normalised losses, so like this. I'm just going to run that. Now the output of that module will now be the same dataset, but without that column of data in there. So we're kind of massaging the data as we go through. So let's just have a look at that just to see. Yep, there we go, so we can see that the normalised losses column's no longer in the data set, which is good. So now what we want to do is kind of clean missing data. If there's any cells missing, any values missing, we just want to get rid of that whole row, because it's not useful in terms of Machine Learning. So we're going to use the Clean Missing Data. There we go, Clean Missing Data. And then again I'm going to stitch these two together, and I'm going to say in this one, the cleaning mode is going to be to remove the entire row. So any rows that have got blank value, I'm going to remove that entire row. So again we're ending up with a clean dataset. So this is a simple example, but at this point we've now got a data set that's ready for Machine Learning. We've made sure that we've got the columns that we want. We've made sure that every row has a full set of data associated with it. So now what we need to do is actually start applying some of the Machine Learning experiments to it. So the first thing we need to do, and it's very important with Machine Learning, is that we take this data set, so this has got, I think it's 205 rows in it, it might be 200 now we've cleared some of them out. We need to use a proportion of those rows to learn about, and we need to reserve a proportion to test what we've learned. So what we want to do is split the data set into two parts, and to do that we have to split, Split Data module. So I'm going to stitch that together, there, and I'm going to say that we want to split based on 75%. So this note here has 75% of the data, and the one next to it has 25%. We're going to use the 75% to learn about, and we're going to use the 25% to test what we learned later on. Okay, so now we get to actually apply some Machine Learning algorithms. So we're going to apply something called the Linear Regression Algorithm. Now I don't really know what this does, I just now that in my script that's what I have to do, so this is an important point that, you still have some learning to do. We've made it quite easy, it's all drag and drop and stuff but you still need to know which modules to use. You still need to learn about what's the right algorithm for different datasets. So there is skill for this, in fact data scientist is one of the most sought after jobs in the industry right now, because people that can do this are in high demand. I'm not really one of those people. I'm more of a amateur data scientist, you might say. So anyway, Linear Regression is what we're using for this particular demo for some unknown reason. Next thing we need to do is a thing called train the model. Now what this does is it takes two inputs. The first input is some data. So I'm taking that 75% of my data set, and the second input is the algorithm that I want to train it on. So I'm taking 75% of the data, I'm saying I want to use a Linear Regression Algorithm, and I've got to say what column I'm looking for information about. So we're trying to find a car price, so we're trying to predict what factors influence car price. So I'm going to say Launch Column selector, and I'm just going to add price as the column I'm looking to learn about. So what this is going to do, is it's going to kind of look at all of the other columns, all the connexions between all those columns, and figure out what factors influence the price of the car, based on linear regression. The next thing we need to do is to think about scoring the model. So we drag that on there. Let's take the output of that learning data set, and we need to compare it to the 25% of the data we held back. So we just connect that in as well, and we just click run. And the final step is to do something called evaluate the model, which is where we look for how many errors there are. So how far were the predictions away from the actual values. To do this we use the evaluate, Evaluate Model. Move, I'm just going to make some more space here. Connect them together. Run the experiment at that point, and this is basically the end of the actual training experiment itself. It's the first time we get to see the result of what we've just put together. Now this is obviously a very simple dataset, we have a very simple experiment, which is why it's running through so quickly. In large datasets this obviously takes a long time to do the build and run, and that's what basically, that's the benefit of it being on the cloud, really. So there's a load of stuff here, and if you're a data scientist you'll understand what that means. Again, this is one of the things to learn. But what I do know is that low is good. So we're happy that this is a good dataset, it's a good experiment, it's doing what we needed it to do basically. So what we're going to do now is make this available as an API so that we can start using it in our applications. And to do that we're going to convert it from what's called a training experiment, which is where it's learning and it's producing the model, to a predictive experiment, which is where you can actually call the model in the live application. So I just pressed a button there to say, Create A Predictive Experiment. You wait for that, it does some stuff, it's a bit gratuitous, why it does this animation, I'm not quite sure why it does that, but it does. And then we're going to run that. It's basically just optimising the experiment so that it's optimised for production rather than for learning about stuff. This one takes probably 20 seconds or so to run. And what we're going to do once this is run is we basically publish it as an API. So it's going to use, under the host it's using, a load of Azure services, Azure app services, and other things, but you don't need to worry about any of that. It's just a kind of platform as a service offering. You don't need to worry about what server it's running on, how it scales, or any of that stuff. That's all taken care of. I'm going to deploy that now. And that is now exposed as an API. So I can, it's a REST API, I can use this as any platform. I can go into here and see the details of how I call the API, so I can see there's the end-point, here's some of the headers that I need, this is a sample request, so this is for passing in new car data effectively. We've got this nice little test interface I can use as well to test it out. So I'm going to now put in some data of a car, that's not in the data set, and see what the predicted price of that car is going to be. So let's say that it's going to be an Audi, it's going to be a diesel, it's going to be four doors. The body is a hatchback, the drive is front-wheel drive. The weight is going to be 1,900, engine size 150. So imagine you've kind of, you're looking to buy this car. Brake horse power 150, miles per gallon 55. I don't, no Cortana. I don't need you right now. And the price, let's say 23,000. So this is potentially a car that we're looking to buy. Let's see how the actual price compares to the predicted price for that particular car. So if we zoom in there we can see the Scored Label, this is the predicted price, this is the price that it should be based on all the data that's in our dataset. This price that Machine Learning is predicting it should be. That's 20,261, and the actual price is 23,000. So we can see there's a difference there between, basically it's overpriced is what we can tell. Which of course makes sense, because it's an Audi, so Audi's are always overpriced, aren't they? So we can see that this is definitely working, it's definitely a good experiment. So that's a very quick experiment using Machine Learning and how we can pull it all together. You don't need to write any code. You can write code to this, there's a language called R, there's a language called Python, which if you want to get really in depth with this you can take it to that level, but you can do some really good stuff just by the drag and drop and just by getting to learn what these different modules do. So Machine Learning. It's still a bit for boffins though, right? So you still need to know like linear regression. You still need to know what the different modules do. It's a lot easier than it was a few years ago, but there's still a skill to it. And that's why we created a thing called Cognitive Services. Now Cognitive Services are all about bringing some of the smarts from Machine Learning, some of the clever stuff but without having to understand the actual algorithms and stuff as well. So we've got a load of APIs, they're built on top of Machine Learning, built by Microsoft users, and that's where we just saw. The simple REST API, so you just upload something, you get something back, and they're either free or cheap as well. So most of them have a free trial, to a certain number of transactions per month. So anywhere between five and 10,000 transactions per month, for free, and then to go over that you pay a nominal fee over top that. It's certainly not expensive to use these APIs. There's actually 25 different APIs. So there's ones that do things with vision, so images and photos, those kinds of things. There's ones that do things with speech, so converting audio to text, and text to audio. Language, so understanding the intent of what somebody is saying and also the things that people are talking about when they're using natural language, Text is our natural language processing API. We've got things like text analytics, which can sense sentiment of text, and what the key phrases are in bits of text. We've got things like the recommendations API, which will give you those, kind of, frequently bought together type recommendations that you see on Amazon, and all those kind of things. So there's a whole tonne of APIs. There's actually 66 different methods that you can call across this API set, and it's improving all the time. There's new APIs being added all the time. There's a huge amount of stuff there. But I'm going to try and show you just a few of these APIs in action. It's going to involve emotions, faces, monkey detection, and books. So strap in. So if you got to, this is where you end up. If you go into here, you can see all of our APIs. If we just look at the computer vision one to start off with, this ones all about doing things with images, photos and images. First one it does is analyse an image. So this is kind of a, each of the APIs, you get this kind of little sample app that you can use to understand how the API works, some of the results. So this is obviously just a stock image, but look at some of the data that's been returned back from here. So we can see a man in a pool of water, a man swimming in a pool of water. That sounds about right, that is definitely a man swimming in a pool of water. Some of the tags or keywords are water, sport, swimming, water sports. We can see that this is not clipart. We can see it's not a line drawing. We can see it's not black and white. Thankfully it's not adult content, or racey content. That would be wholly inappropriate. And we can see things like, we can even see that it's detected a face, so it's detected there's a 28 year old male in the photo, and it's also detected the colours and the hue. So dominant colours are background, foreground. So that's all from one image, and so if you post an image to the API, this is the results that it's going to give you back in sort of json. So I know what you're thinking, this is a stock image. So how do we know this actually works for real? And that's why we've got a camera, so everyone smile. Give me your best smile, there we go. So let's upload that image and see what it makes of all you lot. Hopefully there's no adult content or anything like that in there. Where's that gone, pictures. So let's just upload it and see what it thinks about this image, see how it describes this image. Right, a load of stuff. So a group of people sitting on a couch. About right, sounds pretty good. The key words for this are people, indoor, posing, I don't know who's posing, somebody was. Seat, it's giving me the dimensions of the image. It's not a line drawing, it's not a black and white image. Thankfully it's not adult content, which is great. It's detected a bunch of faces in here, and it's given us a gender and an age for each of the faces that are in the image. And it's also got some of the dominant colours. So it's got the red, which is obviously coming from the background there, it's picked up a black, a brown, it's a pretty good job. It's given us a lot of use, so you can imagine how you can use this, and this is just an API call. You send an image and it gives you back all this json. Pretty cool stuff. So that's one of the six functions that the Computer Vision API does. The other things it does is recognise celebrities, analyse videos in real time, read text and images, so OCR type capabilities, and generate smart thumbnails, which is really cool. On this we'll focus on the region of interest in a photo, so in this particular photo, the region of interest is obviously the person on the mountain, but it's off-center, and if you crop it, it will crop on that axis, rather than the central axis. So really good for image editing, that kind of stuff. So that's just one API. We've got a load of other APIs. So the next one I want to talk about is the Emotion API. This is all about detecting emotions in human faces. So we all have eight emotions that we always, that every human on the planet will display at some point, and they are anger, content, disgust, fear, happiness, neutral, sadness, or surprise. Everybody does it, you've probably done this tonight since you've been here, displayed at least some of those emotions. So what this API can do is you upload an image, and it will tell you what the scores are for those various emotions. Now I've got an interesting image here. I've got a daughter called Madison, she's very sweet, and this is her here. So as a father, I'm not quite sure what's going on here, like is she happy, is she sad, if anybody has got young kids they'll relate to this kind of image. This is, you're not really sure what's going on. So thankfully, you know what I'm going to do, I'm going to upload it to the API, and I'm going to get a json result set, that tells me exactly what's going on with my daughter's emotions. Cause that's a really good way to parent. There is a bit of latency is this parenting technique, but it's worth it because you get really accurate data. So there we go, so the first good news is, Madison is human, she has got a face, and it's given me the bounds of the face within the image so I can pull that thing out so that's good. And she's also got a load of scores here about her emotional state. So the primary, so those with the eagle-eyed amongst you will realise that the top score is contempt. So 58%, she's got 58% for contempt in this particular photo. The next score after that is disgust, and that's 26%, and then the third emotion is anger, at 7%. So contempt, disgust, and anger. That's what that looks like. That's contempt, disgust, and anger. So if anybody's got any young kids, and you're not sure what they're thinking, then this is the API for you. Okay, so next thing I want to show you is, this isn't just images. We've got an app here, you can get this. This is a Windows app in this case. But you can get this off GitHub, if you've got Windows. Sorry, if you haven't. You can use a webcam, which I've got here, and we can point at faces and it's going to give us some information. So I'm actually 37 so that's a good result. So I'll point to you guys, and it's going to pick out some faces and give us some information in kind of real time or half real time, at least. It's not a very good camera, which is why it's slowing a little bit. So it's not just images, it's video as well you can do this in, and you can do things like, we can use emotions. So I look happy, happiness. If I look sad, - That was neutral, I obviously can't do a very good sad face. We can do tags which is really cool. People, conference room, crowd, laptop. So in real time it's giving us data and it's describing what we're seeing in real time here which is quite cool. I don't know why it's zooming in, there. Sorry about that. It's got a little mind of it's own. So there we go, so that's kind of video analysis in real time. The next thing we want to look at, is another API called the Face API. So this one it's all about, we're seeing some face stuff already, so we know we can detect faces, but what this will do is it will give us a load of information about a face. So it will give us actually what we call face landmarks. 27 different bits of information about a face like where the pupil is, where the edges of the mouth is, where the nose is, the approximate age and gender of the face. We've got a few websites that we've built to kind of demonstrate this. Has anybody seen this website before, Twins Or Not? So this is built using that API. It'll take two faces and it'll say how similar they are. So if you upload two faces, two pictures of the same person it'll be a 100% match, but you can actually see who you look like. So I've been told I look a little bit like Jake Gyllenhaal, so let's find out if that's true. I'll take it, it's a compliment. So let's upload a picture of Jake, the handsome devil that he is, and now I'm going to upload a picture of myself, and we'll see that based on Machine Learning, and data science, and all of these things, I do look like Jake Gyllenhaal a little bit. Ta-dah, 89%, it must be true. I definitely look like Jake Gyllenhaal, you can't lie if you have data science. So there's another one called How Old, which is a similar kind of thing. You can guess what this is going to do. Let's upload a picture of me, to see how old I look. So I'm 37 when this photo was taken, it's not the best photo in the world, which is why I look 42, so clearly, I've been knocked down a few pegs there. One minute I look like Jake Gyllenhaal, the next minute I look older than I am. But just in the interest of fairness and being scientific about it let's see how old Jake looks, whoops. Try another photo. Use your own photo. Just to see how old, so Jake and I are about the same age. He's about six months younger than I am. So let's just see how old he looks, 43. So he looks even older. Another website we've got here is one that I've built, called You can go to all these websites yourself. What this one does, is it takes a group of faces, and it ranks them by happiness or sadness, or whatever. So we happen to have a photo of a group of faces, don't we? We just took it. So let's upload that photo of all you guys, and see who's the happiest person in the room. I did say to look happy. Now there's some stuff here about resizing the image, which I haven't got around to doing, but we've found 13 faces, we sorted them by happiness. The average happiness in the room is 61%, so that's great. So you can see all the people's, you are, where are you? There you go, you're number three in the room. Number 12 over here. We've got everyone's face in the room, there. Now what I want to know is who's the happiest person in the room. There we go, there's the happiest person in the room. There it goes. So we can do something cool here, so that's great, is sort it by happiness. So we can do something like show me all the angry people. So what I'm doing here is I'm doing a natural language query, so I'm not using buttons or drop downs, I'm just using natural human language, and this is used in a service called LUIS. The Language Understanding Intelligence Service, which is another one of our cognitive services. So it found 13 faces, sorted by anger. So let's see who's the angriest person in this room. Oh there we go, not happy about having his photo taken in such a public way. Sorry about that. Let's see, let's see for example who is the third most surprised person. Okay, so I'm asking these questions in just a natural language way. What this is doing is it's understanding the intent of what I'm asking, so for this particular one I'm looking for a specific number in the list. So we know we've got 13 faces. I'm looking for a specific number, so that's my intent. I've also given it some variable information. So I'm asking for the third, so that could be any number between one and 13, potentially, and I'm looking for an emotion, which is surprised. Now notice I used the word surprised. The emotion actually is surprise. The word is different but it's still working out the connexion between the two. I could say happy, and happiest, and happier, and all those kind of things, it would know what I'm talking about. So this is LUIS, so who is the third most surprised person? There we go, third most surprised person about having his photo taken. And then we can do the one we've all been waiting for, who is the least happiest person. Whoops, it doesn't matter that I've typed that wrong, that's fine. So number one will be here somewhere. No surprise it's Mr. Angry at the front, there. So again the intent there is to show me, to show the least happiest in the room. Is that right? It should deal with that, but. Okay, let's see who the least happiest. Okay, sorry. That's a training error, so it's one thing you have to do with LUIS, is for each of your intent and utterances, you provide sample sentences and you kind of tag them up as, say that's the emotion word, that's the number word. that's the overall intent. And you give it some examples and it trains and learns based on those examples. I've obviously just not trained it very well for that typo. But that's LUIS. The final thing I wanted to show you was this API here. Which is the recommendations API. So if anybody's doing anything in e-commerce, or anything where you're got some kind of inventory, and you've got people using that inventory. It doesn't have to be e-commerce but that's a good example. You can do things like frequently bought together. So users that bought or interacted with that item, also bought, interacted with that item. Item to item recommendations, personalised user recommendations. I've built another website around this called Really catchy title. I've got some data here, which is a load of book data. This is a database of about 2,000 books. So we've got the, for each book we've got an ID, a title, and what we call feature data, things like the publisher, the author, and that kind of thing. It's my inventory, basically. This could be anything. I've worked with company recently called Geek Seeker to do live music discovery. So they inventory is a list of bands and artists, and a list of venues. So it's whatever you're working with basically, but an inventory of data. And then you have these things called usage files. Now most of these are just CS3s, it doesn't have to be any particular format, and this is just mapping a user ID, to an item in the inventory, and how often the user's interacting with that item. So clearly if it's e-commerce, who bought that item, or who viewed that item. If it's the scenario around music and artists, it's who favorited that artist, or who went and bought a tickets for that artist, that kind of thing. It's whatever makes sense in your site. With my example here of books, it's just who read that book, basically. It's all made up data in this case. So using those two files, you can upload them to the API, and it gives you something like this. So this is, don't do this, put 2000 things on one page. It's not a good thing to do but just for the sake of demo, it's fine. So if we click into one of these books, it's going to give me recommendations for that book. So The Piano Man's Daughter has these item to item recommendations. If we click onto one of these other books, it's going to hopefully have more, different recommendations. This one's got some frequently bought together recommendations, as well. So there's no code in this, all I'm doing is calling the API revert, an idea that's in the inventory, it's giving me back a load of data about the recommendations, both frequently bought together, and item to item, and you can also do personalised recommendations, as well. So if anybody's in e-commerce, this is a really good API to use. Probably better than that little function that you've written to kind of, manually do it. So that's the end of the demo. It's not just silly demos. I'm going to show you a video now about a guy that works at Microsoft who's blind, and he's been blind since the age of seven. He's a programmer, I don't know how he can programme if he's blind, but he does, and he's written this application using some of these services to help him, kind of see and visualise the world around him. - I'm Saqib Shaikh. I lost my sight when I was seven, and shortly after that I went to a school for the blind, and that's where I was introduced to talking computers, and that really opened up a whole new world of opportunities. I joined Microsoft 10 years ago as a software engineer. I love making things, which improve peoples lives, and one of the things I've always dreamt of, since I was at university, was this idea of something that could tell you at any moment, what's going on around you. - [Narrator] I think it's a man jumping in the air doing a trick on a skateboard. - I teamed up with like-minded engineers, to make an app which lets you know who, and what is around you. It's based on top of the Microsoft intelligence APIs, which makes it so much easier to make this kind of thing. The app runs on smartphones, but also on the Pivothead Smart Glasses. When you're talking to a bigger group, sometimes you can talk and talk, and there's no response, and you think, "Is everyone "listening really well, or are they half asleep?" and you never know. - [Narrator] I see two faces., 40 year old man with a beard looking surprised, 20 year old woman looking happy. - The app can describe the general age and gender of the people around me, and what their emotions are, which is incredible. One of the things that's most useful about the app, is the ability to read out text. - [Waitress] Hello, good afternoon, here's your menu. - [Saqib] Great, thank you. - [Saqib Voiceover] I can use the app one my phone to take a picture of the menu, and it's going to guide me on how to take that correct photo. - [Narrator] Move camera to the bottom right, and away from the document. - [Sadiq Voiceover] And then it will recognise the text. - Read me the headings. - [Narrator] I see appetisers, salads, paninis, pizzas, pastas. - Hi. - [Sadiq Voiceover] Years ago, this was science fiction. I never thought it would be something that you could actually do, but artificial intelligence is improving at an ever-faster rate, and I'm really excited to see where we can take this. - [Saqib] Hey. - Hi. - [Saqib Voiceover] As engineers, we're always standing on the shoulders of giants, building on top of what went before, and in this case, we've taken years of research from Microsoft research to pull this off. - [Narrator] I think it's a young girl throwing an orange frisbee in the park. - [Sadiq Voiceover] For me, it's about taking that far-off dream and building it, one step at a time. I think this is just the beginning. - There we go I think that's a really compelling story, and the great thing about that is that it's just a few API calls. It seems like it's a lot of code but it's not. All that thing does, it takes a picture, takes a photo, sends it off to a few of these cognitive APIs, and reads the results back, that's all it does, but the impact on Saqib's life, and other people like Saqib, is huge. So that's just so it can hopefully give you some ideas about some of he things you can do with this stuff. So I'm done, so we talked about Machine Learning, the principles of Machine Learning. It's all about data, it's all about patterns in data. It doesn't have to be big data. We've looked at the Azure Machine Learning service, which is the way that you can take your own data, and apply some of these Machine Learning algorithms to it and get some insight, and then we looked at Cognitive Services, which is a way that you can kind of just, kind of cheat, really. You can use the power of Machine Learning, without having to know all the data science behind it. We've got a programme at Microsoft called Ascend. So this is a programme where we look to work with companies and organisations that are trying to do some interesting stuff, with this technology. We do kind of three to five day POC, hack-type projects. I'm on a project this week with Macmillan Council Charity, were looking to build a bot around some of this stuff. We're looking for real-world case studies. So if you've got an idea, that you think some of this technology might be able to help you deliver, then we're interested to talk to you. It's completely free of charge, so we'll come, it'll be somebody like myself, or some of my team will just work with you and some of your team for a few days, and try and see if we can accelerate your project. So if that's of interest, contact me, tweet me, and we'll go from there. But for now, you can contact me, there I am. On twitter, on email, if you go to this URL, you can get these slides and some links to some stuff that's related to this talk. But I'm happy to take any questions, but apart from that I'm done.