Sessions is temporarily moving to YouTube, check out all our new videos here.

Azure Functions

Mark Allan speaking at Dot Net North in March, 2017
701Views
 
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

The natural evolution of Platform As A Service, Functions As A Service is the cornerstone of the new serverless paradigm. Learn how Azure Functions allows you to develop naturally scalable and event-driven "nano-service" solutions. Write exactly the code you need, with infrastructure and scaling taken care of for you, built-in security and integrations and a pay-per-second pricing model.


Transcript


I should particularly point out here, DocumentDB is relatively stable. Azure Functions is not relatively stable. The runtime is actually now RTM, but CLA and all that sort of stuff aren't even released yet, so I will gloss over a lot of that, because it's just changing too fast. So this is a bit of theory, a bit of practise, but functions as a service, going back to the serverless thing. What Azure Functions basically is is you give it a function. You say, here's my function. It's called whatever. Please run this function when it needs to be run. Run this... If this function needs to be run 372 times in a second, please scale it far enough to run it 372 times in a second. If it doesn't need to be run, don't bother running it, don't charge me for it, please. All the functions are triggered by specific events, so you will find it's not... You know, you can, to some extent, lift and shift your data into DocumentDB. You won't be able to do that with Functions or Functions as a Service unless you're already highly event-driven in your backend, which obviously, all the best architectures are, but you're probably on monolith, won't easily lift and shift, so you'll tend to find that you start picking out little bits which lend themselves and start using them. Has anybody used App Service webjobs at all? Yeah. This was built by the WebJobs team, so from that point of view, it's very similar in terms of having a scheduled or what have you function. What you don't get in this is continuous functions, but it's very much the same thing as, you know, you've got your App Service running and you've got little jobs running in the background all the time. The big difference is what they call consumption plans. Normally, when you fire up App Service, you have to tell it how many instances you want, how big those instances are, and what criteria you want to use if you want to change the number of instances in terms of memory and CPU. A consumption plan completely gets rid of all that so that you don't know, don't care about the instances. It handles all that for you completely. Lambda were the first people to provide this, but there's now Google Cloud Functions as well, and Apache have an open source thing called OpenWhisk, which you can run on IBM infrastructure or you can actually take it and run it on your own infrastructure if you want as well. So, yeah. A function, an Azure function, is just a little bit of code, literally just a method call. And that method will be called on a trigger, so you can either have it on a timer schedule trigger. You can have it triggered when something happens data-wise, so usually a queue, something appearing in a queue, but you can also have it if something appears in, a row appears in a table or if something appears in Blob storage, and you can also trigger it via HTTP. Mostly, originally intended for Webhooks, so, for instance, if you do a commit to GitHub or something like that, then you could get a message in saying, please run this function, when it's a commit or a pull request or something. But obviously everybody saw that and said, right, APIs, completely scheduled on APIs, so there's been a lot of work put in recently to making that a reality, so that's where we get the real fun stuff. You also can have extra inputs, so for instance, if you're running on a queue trigger, then your queue trigger returns the ID of a blob in Blob storage or a DocumentDB record, then it can also go off and feed that in as another parameter to your function, and then at the other end, you get outputs, so for instance if you... Quite often you'll want to, for instance, write something to a blob, but also fire off another message to another queue to make something else happen, so you'll have a number of out parameters which you can then easily feed in what you want to happen at that end. The nice thing about it is that all you have to do is give it, you know, a little bit of data, and it will handle all the reading and writing code so you never actually, if you don't want to, have to reference the queue's API or the DocumentDB API. You know, you get it in, you figure it out, just the data, and it handles all the rest of it for you. So, how you'd start putting it all together. This is just a quick example of something that I do with it. I've got the timer, which runs every 15 minutes, and says fire off my poll feed function. That goes off and reads an RSS feed, and sticks into a queue, sticks into a table the details of that RSS feed. What it also does is feed into a queue the URL of the main image for that, and then there's another function which watches that queue, and every time that queue gets a new URI in, it goes and reads that, reads that image, creates thumbnails, and stuffs those into Blob storage. Now, at the other end, I've got an API, so that API calls an Azure function again, that goes and reads the API and the thumbnail and sends it back. I mean the feed, the feed entry and thumbnail, and sends it back. Fairly simple example, but, you know, that's kind of how you start to stick them together. Now, if you've got an event-sourced system already, then you can take that, I say, lift and shift, and you immediately get something where you only pay for the amount you're using, so it's brilliant for startups. It's brilliant for just hobby developers who just want to stick code up and run it and not pay for it. You get a million, I think, at the moment, function invocations for free before you have to start paying at all, so at the moment, it's really, really attractive pricing. Sorry. Going quickly through the things you can trigger it on: Schedule, done using cron format, if anybody knows cron format; an HTTP call, which, like I say, will be a webhook or an API; entries into a storage queue, Azure Storage Queue or an Azure Storage Blob; also a Service Bus queue or a Service Bus topic, or something coming into an Azure Event Hub. The other inputs you can have based on the IDs and those, more tables and blobs, so, you know, again, if it's not that you've created one but you want to just go and look at something, you can fire an ID in one of your triggers. Tables and mobile apps or mobile services or whatever they're calling that these days in Azure, and DocumentDB records. And then what you can write out to all sorts of stuff and probably quite more soon as well, if you triggered by HTTP you can send obviously an HTTP response back. You can output to any of the storage things. You can output to another Service Bus queue. You can write to a table. You can write to DocumentDB and send out an event to Event Hub. You can send out notification to Notification Hub, so, you know, for your mobile devices, and you can also send out Twilio SMSs. Like I said, I believe they'll be working on more and more of these. It also integrates with Flow, which is the Microsoft if this, then that type thing, so you can... If this, then fire this off to an Azure Functions API, and PowerApps, which is... Never used it, but I believe is a sort of build-your-own=app thing based in Azure. Again, you can, you know, you can use functions as an interface to that for your data. Those are both based on the fact that functions will give you a Swagger API definition. These days, I think they just hook into the Swagger, effectively. In terms of what you can actually upload and use for your function, right now, as of today, the first class citizens are Node, C#, and F#. To be precise, CSX and FSX, so script files rather than a C#, .cs file. The reason they're called first-class is because the runtime is already there. If anybody remembers the difference between CGI and FastCGI, this is FastCGI. So the runtime's already there. The function's already loaded up, and they're just ready to go as soon as you're ready. Experimental Python, the PHP, and then the three scripting languages, including Batch. They will just get called, so, you know, if you call PHP, it will literally just say php.exe, this thing, so you'll fire up the runtime every time. Good luck with that. And we're told that at some point in the future, you'll get Java support as well. Don't imagine as many Java people in here, but there's some around somewhere. Now, again, for the sake of speed, I've already created a Functions app, but you won't be surprised see that, you know, it's in New > Compute, somewhere in the... Once you've created your Azure function, it's actually running on top of App Service. This is the view you'll get. Now, basically, at the moment, we have no functions, and we have no proxies. Proxies literally came in last week, but it's very promising. What they're giving you, at the moment, you get basic API routing, so you can say, when you call /apis/thing, you know, these are the parameters, and feed them into my HTTP trigger. Proxies, they're giving you just enough of the API management gateway to actually work that across lots of different function apps and even other things that aren't function apps. So you can put in a route there, and it will actually fire it off to a specific URL. So they're giving you that... This is the new stuff they're giving you to be able to write APIs, which come into one place but then can call lots of different APIs underneath that. So here we go, back to New Function. The thing we saw at the start there is just a bit of prettiness over this, but we get all the languages there which we can choose from, various pre-made scenarios, so there's some actual real code versions in there, and we could choose one of these. So, what I will do... I will see if I can find a C# API. So we'll just call a C# HTTP trigger, and I will call it... What will I call it? Think of a name. I'll call it RequestFamily. Authorization level here, I will set it to anonymous, just for speed. Basically, you can either, you know, if you're just calling this from, again, from your own server and you don't care, you can just use a master key that you pass in as a parameter on the query. If you set it to function level, there is a specific key that you'll get for this function, which again, you pass through on the URL. Anonymous is just quickest, but you've got to watch it obviously if you're anonymous, 'cause then you might get DoS'd and if you don't have a limit set, then you could end up paying rather a lot of money for all the function requests that get hit. So, we end up with a sample function there. Right. What I'm going to do now, in the Integrate tab, we've got the triggers inputs and outputs, so I'm getting in a parameter here called request, and I'm... This one here called return means that I'm just going to return it as the output from the function, so you'll see in here, we've got the request there, so that means it's matched on the name, and then down here, we're returning an HTTP response. Just to be very quick about this. And we'll give this a roots template. Put that in there. Get rid of all that rubbish. All right. So, save that. I'll just pull these out here. So we can see down here we've got a log, which will tell us what's going on, so we run it, and over here, we've got a test thing, so we can either... Yeah, we can call it getPost. We can just call that. Call this AddamsFamily. In this case, we're not posting, so we don't need a body. And this is where we find out what happened. And down here, we'll see HTTP trigger processed a request to AddamsFamily, so that's coming from our log.info there. Now, what I can also do is get a function URL there, paste that in there. AddamsFamily, and we get hello, AddamsFamily. So, that's a very simple API. Now what I'll just quickly show you here is taking that, sticking it into a queue, and then picking up from a queue and another function to actually do something with it. So, we'll go into Integrate here, add a new output. Azure Queue. Right, so... Pick, again, a queue I made earlier to save time. Familyqueue, and I'll call it queueItem. And just know, I'm not actually going to use this here, because it will create a C# when I want to use Javascript for the other one, but you can actually, having created that queue output, then just go and say, okay, I'm going to have another function, which works off this queue, hit the button, and it will create you a new function in the same language. Effectively, what I'm going to get now here is I will give it an out parameter. What do I call it? And then all we have to do there is say queueItem equals, excuse me, new QueueItem, and we'll give it documentId equals familyId, and I'll create up here. We're not getting IntelliSense here. So, I'm calling that QueueItem. That's basically going to get serialised and stuck out into my queue, and then within that, there'll be a documentId set to the familyId which I'll then be able to pick up in the next function and use that to go and fetch a document from the DocumentDB stuff we were doing earlier. Right. So, save that, see if that works, and we'll create a new function, and this time, we'll do a Javascript one. We'll take a queue trigger, and I'll just call it increment, 'cause I'm going to do a really complicated bit of processing with it. All right, so, this is Javascript, 'cause Java's, you know, we're going to get away with DocumentDB, and we're working with JSON. This'll be a lot easier in Javascript. So here, it's slightly different. We get a context in, which is where we get our bindings from, rather than having a number of items, and then we get our input there, which is a key item. Save. Integrate. And here, we'll add a DocumentDB. So, call that. And what was our database called? ApiDemo, not a collection, whoops. Whoops a daisy. And DemoCollection. Right. Again, we could here use the return value, but we'll just show you how the binding's working here. So now all we should have to do is say context.bindings, so this is where the input and output bindings come from, and we called it outputDocument. Whoops a daisy. Bindings.inputDocument, and this is where I remember that I forgot to add the other integration, which is an input. DocumentDB. So I'll call that inputDocument. Again, we're on ApiDemo, and DemoCollection, and our document ID where is DOCUMENTID, which is, it's just going to pick out of that queue message that we sent through. bindings.outputDocument.Counter, and that should be it. Now, it won't run on the first time, but in theory, if I now go and call this, and redo it, and it... And we will go back here, and we'll see. Ah, missing binding argument named queueItem. Cheat here. This is how it actually gets stored in the file system. We've got a folder called RequestFamily, and within that, you've got function.json, which is the bindings definition stuff, and then you've got that .csx file, which is the actual C#, so if I go in here, I can actually see exactly what have been defined there. So what was it saying? Missing binding argument named queueItem. It's definitely called queueItem, so, in the interest of speed, I'm going to cheat. All right. Let's try again. Yay. So, request received. Now, if we're really lucky and we come 'round to this one, yay! Processed work item, function completed. Oops. Go away. So, now, if we go back to here, yay! And our counter is now one, rather than zero. All right, so... That's the basics of how it works. What I will very quickly do here... Where are we? So, we go back to the portal. Because it's based on... Because it's based on Azure App Services, then we get all the nice stuff like Kudu, so we can see here that this is actually what's been stored in our file system, which is actually an Azure File storage thing. We get a host.json there, which we can use to configure things like how often it polls queues and stuff like that, all that, and then each of these, we get those files you saw earlier. So in the case of a Javascript one, you get an index.js. Now, what you can actually do there, because it's App Service, you can start doing things like continuous deployment. If you've got a package.json in there, it will actually instal all the MPM packages for you. If we go back up a level, we can actually create a folder that I called Shared, so if we've got a DLL or a common package that we want to use, we can actually refer up via ../shared to get references to things that'll be used between all the functions. So there's a good amount of stuff around there. So yeah, that is all through the portal, and that all won't change. I think we've hopefully got a decent idea of what's going on. Any quick questions though before I just move onto the last couple of slides while it's fresh in your memory? - [Audience Member] Can you script it so you can put it all into a source control? - Yes, like I say, it's on Azure App Service, so you do get exactly the same thing where you can link it to either... Link it to Bitbucket or GitHub or just add it directly as a remote to your own local Git repository, and then all you have to do is then, when you do a commit, it will see the commit... Well, when you do a push, it will see the push, recompile everything, and hey presto, you've got a new set of functions. The other thing you can do, there is... Not quite at 1.0 yet, but there is an Azure CLI, so you can use that to script the management of everything as well. So, in the same way as there's an overall Azure API which allows you to run services up and down and scale them in and out, Azure Functions has its own CLI which allows you to specifically create functions and stuff. Anyone else? - [Audience Member] So how does it compare or how does it differ from Service Fabric? - Right. How does it differ from Service Fabric? Service Fabric is based around the actor model, so it's, you know, in the sense of being message passing between actors. Functions, you know, it's not a million miles away. You could use it as actors. The main difference is that an Azure function, you know, if it doesn't finish within, I think it's five minutes or whatever other lower limit you put on it in your host.json file, it will get killed, so whereas an actor, you normally expect to stay alive and then maybe die and get spun up again. Functions run. They use what they're going to use. They die again. You get charged for the memory and stuff that you've used, and then you get a fresh one run up. So, although you can do message passing through queues, you're not doing direct message passing between actors the same ways you would in Service Fabric. Roughly. Any more questions? Okay, right. So, yeah, the tooling is the thing that is in a state of flux on this, so I mentioned that there's a CLI now. Now, you can actually run up at the moment what's called a funproj, 'cause obviously functions are the most fun you can have in Visual Studio. And that will allow you to do file > new function. In your project, you'll get the the folder, you'll get the .csx and stuff. Now, what they're actually working on right now, and I didn't see it mentioned today, but I can't imagine it's far away, is full support for Visual Studio so that you can just do a normal C# project in Visual Studio, have unit tests 'round it, all that sort of stuff, have no script files at all, and I guess you'll just define entry points into that, which are your functions, and then deploy that, and, you know, you'll get a much smoother experience than from the point of view of a large project. Doesn't exist yet, but will do soon. There's also a thing called serverless framework, which is actually written, you know, stuff is still labelled AWS Lambda, but the Functions guys have recently set up a provider for Azure Functions. Now, what serverless framework actually does is to add tooling around Lambda, so things like unit testing, again, stuff which aren't part of the standard Lambda offer. That's now available for Functions. Again, I don't know much about it, 'cause it only happened a week or two ago. The challenges you'll hit, there is a bit of latency goes on, not so much with the first-class ones, but, you know, when a new instance gets run up or the first time you run a function for a while, there will be a few seconds of delay. They're getting it down to milliseconds, but they're still working on it, but there is a delay before you actually can start, so if that's really important to your problem. The whole system obviously is stateless and event-driven so if you don't have a stateless and event-driven architecture, you're going to have to think in a different way about that. You'll still get bottlenecks if you've got a SQL server involved in there somewhere. That's still going to be a bottleneck, 'cause it's not going to be serverless and completely scalable. As I mentioned, DDoS and bugs can get quite expensive if you've got some... If your function's running for five minutes 'cause there's a bug in it and you don't notice that, you could end up with quite a lot of compute time going on, and although the serverless framework will address this to some extent, you obviously get a certain amount of vendor lock-in here, 'cause again, you can't really lift and shift stuff that's all very much based 'round Azure queues and Azure functions, Azure storage, off to Amazon Lambda. Right. Made it! Two minutes to nine. Yes. Right. - [Audience Member] The bit around the tooling. - Yeah. - [Audience Member] Does that mean that you get private, proper traceability? As in independent functions with enormous parameters. They're quite difficult to try to review as a whole. - Yeah, when you're developing locally, as well as the full Visual Studio support, the Azure Functions CLI also has the ability to run functions locally so you get the runtime running locally, so you can, you know, you should be able to run everything, debug through stuff, and, you know, get that local development experience without having to deploy and put traces and stuff. Now, obviously, you do have... There is a good amount of logging in there, so you can see all the indications and the login and the logout sort of stuff, but yeah, the plan is that you'll be able to do a full development cycle locally and then deploy rather than having to use the cloud as part of your development process. Yep. - [Audience Member] You say they're all stateless, right, but the batch files are such that there's all sorts of file systems they can access. What happens to that then? - Yeah, when I say they're stateless, you know, each function doesn't get passed in any state other than what you've given it as its trigger or its input. Now, yes, they are on a file system there. You've seen the file system. Like I said, that's actually done using Azure Files, so the version of Blob storage that looks like a file system. So, yes, you could in theory go toward that. You're more likely to use something like Redis or something like that in practise, if only because, you know, the whole point of serverless is you shouldn't care about the server bit. So if you start using the local storage of the server you're running on, you might hit problems in the future, so yeah, what you do get and what is guaranteed is that for your secrets and stuff, you will set those as environment variables in the environment of the App Service. So, for instance, you'll get that system environment in C# or process.end in Node.js, but that's abstracted, so that's being passed into you. You shouldn't really be talking to the files system directly. Yeah. - [Audience Member] Do you know if there are any plans for it to support OAuth, or is kind of the assumption that it's always going to be called from a trusted client? - If you're calling from an HTTP, then, at the moment, you get those three levels. You get anonymous, where anyone can call it; you get function level, where, as long as you know the key for that function, you can pass it in, and you get effectively admin, so it will only work with the master key. You can't validate against Oauth against a particular function, but for some of the stuff that I've done, I have actually used the fact that OAuth calls back on a webhook to OAuth into one function which then returns you, you know, returns HTTP, so you go and do the OAuth, and then when the webhook comes back from the OAuth process, then use the token that you got from that to go and continue the process. So that's how, I don't know whether that will ever become first-party, but, you know, it's easy enough to do yourself.