Sessions is temporarily moving to YouTube, check out all our new videos here.

Agility as Software Science

Andy Wootton speaking at HydraHack in July, 2017
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

Time-travel through the hats software developers have tried on for size in the last 50 years, to discover why they didn't fit, why some of them didn't look right with our ears, if we've ended up back with the one we tried first and if we can relax and stop spending all our time shopping for hats now...


Hello Birmingham Technology. - [Man In Crowd] Whoo! - Whoo, yeah. And hello to Hydrahackers. This is a talk about identity. Partly inspired by the post nominal letters that you might just about be able to see after my name there. But also about the identity of us as a development community. I'm looking at the history of how developers have seen themselves over the 50 years that computing has been active in this country. And I want to find out how we all identify ourselves. Is there anyone here-- there's a list here, you can put your hand up for as many as you like, and I might not be able to see you anyway, because of the bright lights, but who thinks that they're a craftsperson? A scientist? A few scientists. A computer? I think we can probably extend that to in computation. Not many. And you'll note that the BCS is rebranding itself as the Institute of IT Practitioners, so I think they caught onto that as well. An information technologist or in ICT? We really don't know, do we? A software engineer? A ha, we've got a few software engineers. Interesting. Does anyone feel they're an agile, lean startup, Zen Buddhist/Taoist, flow hippy? Yeah, me too. Yeah, okay. That's where I'm coming from. Are there any Europeans in? Yeah, I was hoping for a few more, really, but yeah, okay. Kannst du alle Deutsch sprechen? Can anybody translate that, is what I really want to know. - [Man In Crowd] Progress through technology? - Leap forward, interesting. Yeah, the normal translation is advancement through technology, but I think Stuart's got it; it's leaping forward. That phrase came off the wall of an Audi factory in Germany. Obviously. Sir John Hegarty, who was one of the founders of Bartle Bogle Hegarty, and eventually Saatchi and Saatchi, robbed it off an old poster on the wall of the factory. It was clearly a poster about rebuilding a nation with brand new values. German has two words for tech; Technik, which we see here, which means technique, so Audi were actually saying that they were leaping forward through better processes; it was nothing to do with technology, as we understand it. A German ex-colleague said to me, "SAP helps us concretize our processes". I heard, as you may well do, setting concrete or carving stone. But what he actually meant, was concrete rather than abstract. What German engineers tried to do, is the same as we tried to do with a software project specification; we tried to remove the ambiguity, and they've moved that removal of ambiguity from requirements to implementation. They want everyone to do things in exactly the same way. German engineers are very honest about finding Brits imprecise and unreliable. But they recognise our ability to think on our feet, and to improvise. Under static conditions, we're less efficient, because we're seemingly incapable of following rules, but our lack of rigour allows us to respond faster to change; we're actually culturally more agile. I'm going to look now at some of the identities that we've tried out for size; I mentioned hats in the introduction, I'm sorry if anyone's disappointed they thought they were coming for a talk on hats. When-- Sorry. Computers are computing machines. They were made possible originally by Babbage and Ada, with mechanical computing, then reinvented in the 20th Century with electronics. Computing is about computer machines which process data. Some of the data is software which trains the machine to do a job that it had never done before. Again, we're looking for repeatability of a process. In the early days, you took someone with a PhD in mathematics and you showed them the assembler manual, and eventually they worked out how to do it, by trial and error. So, in this stage, I think that we were treating computing as an art, and an art is made of a concept, which the mathematicians had, and the craftsmanship, which they had picked up from the assembler manual. That is a diagram which you probably can't read the detail of, but it doesn't really matter. That's a UML diagram showing my interpretation of the scientific method. It's surprisingly difficult to find documentation on the scientific method, considering how much we talk about it. I found many and various versions, and this is an attempt to take what I thought were the important things out of everything, but the important thing here is the shape. You'll see it again later. At the top, you see the title of a famous book by Niklaus Wirth, "Algorithms + Data Structures = Programmes". And I've extended that to "The Scientific Method + Scientific Knowledge = Science". You can see here, the process of science. You may just about be able to see here, I've marked a data store of scientific theories. That is what project managers would call a body of knowledge; the facts as we best understand them, and head of-- Sorry, the infinite monkey, and head of UK science, Professor Brian Cox, said there's no such thing as a scientific fact. Facts are theories that haven't been proved wrong yet, because facts change. Computing is so new that our methods are still evolving, as well as the facts. Science has moved to having a basic method, which is being abstracted to a single, stable method, which allows it to change its detail and processes; its techniques. It's-- So science is a process much like code, which is a special type of data that records the best current values of how-to knowledge. Code is codification of "Technik". Science constantly updates itself, but if you write a compiler and use it, it becomes a technology. The emerging data processing industry wasn't really a place for science; industry needed engineering practises to teach less skilled workers to deliver big, complicated, important software to agreed time, quality and cost, to meet a specification. So we needed software engineers. Computer science had naturally provided very much science for them to be working with, so we stole techniques from the real engineers; there's one of them. The famous waterfall. You'll see that this is version two of waterfall; the first one just had the boxes handing over the specifications at each point, and never going back, which of course, didn't work. This is the improved version, in which we see any water that goes the wrong way, gets pumped back up the hill to have another go, which obviously completely destroys the idea of fixed quality, cost and time. The biggest thing that this engineering approach to software had was that the very best we could do was to deliver what we were asked for. Engineering model failed because we copied engineering practises, without thinking why the engineers did them. They solve problems while balancing the constraints of physics. Engineers make things that you can kick, like the van. What they do isn't necessarily appropriate to software. If you think about it, what physics actually constrains software? It constrains the hardware that our software runs on, but that's a different problem. Software's problems are made of complexity, poor communication, and magic, and software runs in a world of zero gravity. Engineers also build things for customers who know what they want. If they don't know what they want, then it's product design, not engineering. They find solutions to known problems we often fail to even identify our customer's real needs. Waterfall enabled us to build the wrong things, and made them hard to change. We needed new techniques to control uncertainty and complexity, and to make our features smaller and fail faster. We tried out phase delivery, prototyping, which was building one to throw away, and we thought eventually, it's software, why are we throwing it away? Why don't we just improve it? So we moved towards incremental delivery. We started using objects, which kept the mess into smaller bundles, which then also enabled us to move to incremental delivery, and finally to agility. Too much stuff to juggle here, sorry. 2001. A lot of practitioners who were working on these new methods got together, they went up a mountain, to a ski resort, to find out what they agreed on. I think they were having problems because they competed with each other, and no one was buying their products, so they felt the need to show that actually they thought approximately the same thing, and you could buy any of them, and they came out with the agile manifesto, which has become-- Well, it's being treated like a bible by many people involved in agility. So let's look at some of the things that were highlighted in the manifesto. We are uncovering better ways of developing software by doing it and helping others to do it. Notice the present tense. A document written in 2001, which has never been updated, about how we're discovering new techniques all the time, and it hasn't been changed. It was backed up by 12 principles, and there's the first two: Our highest priority is to satisfy the customers through early and continuous delivery of valuable software. Doesn't mention developing faster, which is how people often interpret agility. Agilists talk about velocity a lot, but as an estimating tool, not as a metric. Number two was: Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive adventure-- Sorry. Customer's competitive advantage. Agility is not speed, or finishing a fixed scope project faster. It's a fast response to change and discovery; it's about changing direction when you need to. You'll also people-- hear people talking about practitioners who believe in one pure agile. There is no one pure agile. You'll hear people saying agility isn't a process, but processes were there. Explicitly mentioned in principle two. You don't have one process, you have a choice of many. You choose from the infinite set of processes which are agile; it's an adjective, not a noun. An organization's agile operating model is any subset of a current known processes; you choose your own. You can't do all the things, so you pick. A typical agile processing model looks something like this. Do you recognise the shape? Agility is science, but applied to the requirement's discovery. We really are scientists now. We do experiments, we test our assumptions. In addition, we have a design cycle, for software product development, so we're also designers and architects; that didn't go away, and whenever engineering and technical skills are needing, we still have-- needed, we still have to be technologists and engineers. So when you put your hands up for all the things we are, I think it's because we're all of them. There's likely to be discussions about there not having been any trends for a while, so people are asking what happens after agility, and I think the question we ask ourselves is, what came after science? Thank you.