The way we deploy and architect software systems has changed significantly over the years. This talk will take us on an journey from the early days of computing and internet applications, through to the present day with microservices and containers, and ending with a look into the future. Along the way we will explore the business and technical drivers behind these changes and see how everything old is new again.
← All talks 
Build Engineering Turing Fest 2017
The Road to Microservices and Beyond
Auto-generated transcript - may contain errors. Tap a timestamp to jump the video.
So does anybody know what this is? Just shout out if you know. I'm waiting for somebody to shout, our computer. Oh, okay. So it's actually a Thinking Machines CM two hundred. It might look a little bit familiar because the predecessor, the CM five, was used in the film Jurassic Park.
I think just because it had flashing lights in the front. So it's a supercomputer in Jurassic Park. But this was like a cutting edge computer machinery in sort of mid nineties. And it was actually, this exact model, the CM-two hundred, was actually installed in Edinburgh at the University of Edinburgh, Edinburgh Parallel Computing Center, which is just up the road.
In King's buildings, as I'm sure some of you know. So I mentioned that for local connection, but also because, well, it looks cool, and it kind of represents where we used to be in terms of computing hardware and software development. So I'm going try and take us on a journey from requiring to run like large applications.
So you've had back in the nineties, if you want to run a large application, you effectively had to buy your own hardware such as this, and run it on your own premise. And I'm going take some journey from that through towards cloud computing and microservices, hopefully.
Now, admittedly, you probably wouldn't have used a CM-two hundred. These were generally used in like artificial intelligence and high performance computing. So if you were like an Internet startup, you probably wouldn't have used a CM-two hundred. But you may well have used SunKit.
And so in the late nineties, Sun was selling so much hardware that they started calling themselves the dot in dot com. Unfortunately, they also became the dot in the dot com bust, but that's kind of another story. They're also pretty famous for the Java programming language.
And, like, mean, how many people here have programmed Java? Yeah. Quite a lot of you. So, you know, Java and also C became pretty ubiquitous, Especially in sort of enterprise software. And you know, they're pretty good languages, especially when you're working in large teams.
They've got great sort of standard libraries. They work at quite high level of abstraction, and there's great tooling around them and editors, etcetera. But the thing is, the way that we started to develop applications using them has now become known as something as a sort of monolithic style of architecture.
So basically what that means oh, there's no pointer on this. Anyway. So basically what that means is all your functions, your software, ends up as one large process or one large binary. So everything in there goes into this one thing. So even disparate business constraints, so your search functionality or your favorite stuff or you know, the UI for the front end, it's all part of the same process.
And okay, you may have like a separate database for storing all your state and stuff, but you still got one large application in the middle. And this works. There's certainly nothing wrong with it. But you start to see a few issues when you start to add more users.
And you need, you know, you're running on a machine and you start to struggle with the amount of load from extra users or you want to process more data or something. And the sort of first solution to that problem, and simplest solution actually, is to scale up.
So simply run your application on a on a bigger machine. One with, you know, more CPU and more memory. And that's the easiest solution. It's a cheapest solution as well, because it doesn't require any or requires very little engineering time to do. However, there's a couple of fairly obvious constraints.
The first one is you can only go so far. At some point, you're running on the biggest machine with most memory that you can afford. Excuse me. And also, there's no sort of failover scenario here. If that machine goes down, you're in trouble.
There'll be some downtime whilst you migrate to a new machine. So we sort of moved to an architecture for large applications more like this, where we simply run multiple instances of the same monolithic code across several pieces of computing hardware. Or I'll probably refer to them as nodes from now on.
And that works. It gets a bit more complicated because you've got to figure out some way of spreading your users across the different instances of a monolith. And you've also got to figure out some way of keeping the state in sync between the different instances.
And depending on your application, there's different ways to do that. We also start to see another nice feature. This is now assuming you don't have too much traffic, and if you have a hardware failure, there's a good chance that, you know, your site won't go down because the remaining two instances may be enough to handle the load.
So we're starting to balance some resilience there. But the main issue is that we can only scale so far. And in this modern day of Internet applications, we tend to see quite spiky traffic. So I've got this example from the Have I Been Pond website, which is a website run by this guy called Troy Hunt, he's a security researcher.
And if you've not seen it, it's a great website. Basically, you put in your username or email address, and it tells you if that's been involved in any public data breaches. So it's a really useful website. And what happened to him one day was somebody mentioned on this morning, the TV show this morning, the website, just in passing.
It wasn't like a scripted piece. The person they're interviewing just happened to mention his website. And suddenly, he went from serving under a thousand requests a minute to serving over twenty thousand requests a minute. So you had to scale up like twenty times or twenty to forty times.
And that implies if you were running on architecture like this, you require twenty times the amount of compute resources lying there, normally doing nothing just waiting for a peak like this. And that's not just, that's simply not affordable for the majority of companies if you're running on premise.
And so that was one of the drivers, this issue of scalability. It was one of the main drivers towards the move towards cloud computing. Now I'm not going to bore you with like a definition of cloud computing because I reckon most of you here are probably pretty used to the term.
But basically in cloud computing, we rent our computing resources from people like Amazon and Google, rather than buying our own hardware. And because different people or different organizations experience peaks in traffic at different times, you know, the actual provider or computing resource has a fairly level amount of traffic.
Now the interesting thing about cloud computing was in the early days of cloud computing, there's a lot of naysayers. People said that nobody serious would ever run unlike the Amazon cloud. And now this sounds ridiculous, but that was absolutely the case at the time.
There was a lot of negativity around cloud computing. They thought it was just a toy really. And that any proper company would have to have their own data centers and their own computing. And one of the sort of main use cases for cloud computing came out of Netflix.
So I got the slide from Adrian Cockcroft who was, who used to be the, one of the architects of Netflix. And can anybody remember back in the old days when you used Netflix or I think it was Love Film in the UK, and you had this DVD by email?
Does anybody remember that? You went onto a website, you selected like, watch DVDs you wanted to see. Was that a pointer? Oh, thank you very much. You selected which websites you wanted to see. Which DVDs you'd like to watch. And then they sent you in the mail.
And you watched that DVD, you sent it back and you sent it the next one. Did a few of you used to do that? A few of you. Anyway, so that was the old model. Now that did involve a website and a web application.
And it was fairly popular, so it did require a reasonable amount of compute resources to run Netflix at that point. But at some point, people started getting broadband into their homes, and it became plausible to start streaming videos directly to them as opposed to like sending out DVDs in the mail.
The interesting thing though is that, unsurprisingly, requires a lot more workload in terms of data center. And they worked out, it was about a thousand times. So at the point, when just zero point one percent of Netflix's customers had started moving to streaming, they were already using more compute power than everybody that was using the DVD by email stuff.
That turned out to be a big issue for Netflix because they had a data center that was running all the operations, but the data center was almost full. You know, streaming was like taken off like crazy. So they were faced with a very it was a very stark choice, and they had to make a decision very quickly.
They could either build a new data center, which would have cost tens of millions. But you know, they had that money. They could easily do that. They could easily persuade a bank to give them that money. I think they had quite a lot in the bank anyway.
Or they could move to the cloud. Now, it's not an obvious choice because the thing is, even today, it's going be cheaper to run your own data center. Right? You know, the cost of computing Amazon is still significantly more than cost of computing your own data center, even when you take into account stuff like operations.
But the reason, one of the main reasons they went with Amazon, and you've to remember, people were telling it was a bad idea to move to Amazon because they just didn't believe that any serious company could run on top of it. But the main reason they moved to Amazon was they took that money that they saved from building a data center and they put it into content.
So literally the reason we got house of cards when we did is because they decided to move to the cloud and take that capital expenditure and spend it on content instead of on a data center. Another thing that started to come out at around this time is users expectations around sort of reliability.
By this time, I guess we're talking about two thousand and nine sort of time. I think, you know, it took a few more years until House of Cards came out. I guess it takes, you know, some time before your investment in content comes to fruition.
But around the same time, users started getting much more serious and upset when their websites weren't working. I mean, back in the nineties, it wasn't uncommon to see like a website dying for maintenance. Nobody really worried too much about that. But nowadays, people get very upset.
So I looked at some tweets. If you just go and search for Netflix down, you get some quite interesting tweets. I like this one, but this person gets very upset and asks for a rebate. So I think she thinks Netflix is a public service, like, you know, water or electricity.
Yeah. I felt sorry for this girl who like, well, it's Saturday night and what am I gonna do? I would suggest go outside, but Yeah. And lots of people are refusing to pay because Netflix isn't working. So now we just really do not tolerate downtime.
People get very upset and start asking for the money back. And also, one thing they found was they were running a thing in this monolith. And one of the reliability issues with this, there's one day at Netflix where a coder forgot to add a semicolon to the end of an SQL statement.
Literally missed one semicolon in an SQL statement. And it took down the database. And not, you know, not just the database, because they only really had one database with multiple schema and stuff in it. So it took down the entire system. I think they're out for hours if not days, whilst they tried to restore the system from backups.
I mean, thankfully they had good backups, so they were back up and running. But took them down for days because of one very simple mistake. And what they realized was, well, this is kind of silly. If we move away from the sort of monolithic architecture of which they were running, and we split it into separate components, we can actually become more resilient.
So they moved to this sort of architecture, and they also moved like the databases as well. So instead of having all our functions in one application, we split them out into separate independently deployable components that can run on separate VMs or separate containers as we have nowadays.
And what happens is if, say, of these components goes down and you architect things correctly, it doesn't necessarily affect the rest of the system. So for example, if this was search and search goes down, it doesn't necessarily mean that I stopped streaming video or users stopped being able to favorite items or something like that.
The rest of the system can continue working. If billing goes down, it doesn't mean that, you know, I interrupt the user's playback of a video for example. And that was the case before with a semicolon incident. So one of the largest drivers was this idea of reliability.
Another one is that they could scale components independently. So for instance, if this is a UI, maybe that my user interface component requires a lot more resources than maybe my billing component up here. So I can run this across several instances, whereas I can run this over a few instances.
So I can scale things at the level I need to scale as opposed to having to scale the whole monolith each time. Another thing that comes out of this is the speed of change. So I can change any given component within this here, for example.
I can change any given component without requiring to make an update to the entire system. If I want to make this update to, for example, to billing here, I have to redeploy the entire monolith across each node, Which you know is a much bigger deal than just making a single update to a single service which can be deployed independently.
So by moving to a microservice architecture, they're also able to increase their speed of change considerably. And that kind of ties into this idea of two pizza teams. Have people heard of two pizza teams? A few people. So this is more from Amazon rather than Netflix.
Actually, that's an interesting point. One of the interesting things about Netflix within Amazon is they're arguably making better use of AWS than Amazon ad themselves with stuff like Prime and Prime Video, which I always find quite interesting. But anyway, so they are direct competitors in some sense.
Anyway, Amazon started using this idea of two pizza teams. So a two pizza team is basically this idea that any your team size should be limited to a team that can be fed by two pizzas. Do bear in mind we're talking about American pizzas, so we are talking about a lot of people.
So I think like six to ten would be a rough number. And Amazon felt this kind of number was sort of a good team size because any further and you get too much communication, any smaller and you can't do a significant amount of work.
And what you can do is you can give each team a sort of micro service to look after, as opposed to then, you know, requiring to try and break up bits of the monolith, which is more difficult. So Teams now, your organization is reflected in your software, which will be very familiar to anybody that's heard of Conway's Law.
And if you go to any conferences, you usually hear the term Conway's Law at least a dozen times, perhaps not here, but I've mentioned it now. Another thing that came out of this was they found you can use different software and different applications with different parts of your architecture.
So it may be that I run one service. I can write one service in Go if I need it to be more efficient than other parts, or I can write one bit in Python or R if I'm doing some serious data processing. As opposed to with a monolith where I was forced to write everything in the one language like Java or C.
So I can use the language that's appropriate for the task at hand. It's the same thing also with the databases. Generally, you split up your databases and each service has its own data store. And that means you can use a data store appropriate for that use case.
So I might, for example, use a graph database in one area, whereas I'll use a no SQL database elsewhere, and that's traditional like Oracle SQL database in another place. Do be aware that there is a large downside here. If you start using lots of different languages, that means all your developers have to become familiar with several different languages or you might fall foul to the to the bus factor as Mike described.
And there's other issues as well. The primary one being that we've kind of moved all of the complexity from the monolith that was in the software out into the network layer. So this is a diagram that Adrian Cockcroft tried to create off a Netflix architecture.
And it's actually just one small part of Netflix that was simplified when he was trying to draw this diagram. And it's very hard to try and figure out what's happening in your sort of system because all this complexity has moved into the network.
Adrian always refers to these as sort of Death Star diagrams because they always start to look like the Death Star in in Star Wars. And as well as the complexity moving to the network, we've also made things a lot slower. So what was a fast function call in this sort of Java monolith has now become a slow call over the network in our microservice framework, which will be several orders of magnitude slower.
And that's a serious problem if you don't break up your your sort of monolith in the correct way. If I have two microservices that are too chatty, that will incur a serious overhead, and you want to refactor to try and address that problem.
And one thing that I haven't talked about is containers of VMs really. And that's largely because I see that as an implementation detail. Although if you know me, I'm kind of more famous for talking about containers. But in this talk, kind of feel it's an implementation detail.
So when we all start moving towards the cloud, virtual machines really sort of enabled that move. They gave us a form of software packaging and an abstraction layer that we could use to move to the cloud and kept us away from caring about the actual underlying architecture of the hardware.
And it's exactly the same with containers. The advantage of containers however, is that they're a lot smaller and faster than virtual machines. So they've kind of grown up hand in hand with microservices. And a lot of people basically say microservices and containers are synonymous.
But it's worth pointing out when Netflix started moving towards microservices, they were very much using VMs. They weren't using containers. They are starting to use containers now. Okay. I'd like to also talk about a couple of things that I kind of see us go into in the future.
So the first one is UniKernels. This one gets a bit technical. But So when you were talking about virtual machines, we basically have a full copy of a guest operating system and a virtual machine. Lots of code in there and a full copy of your application and they tend to be quite large.
So you know, typically gigabytes in size I would I would guess. And it runs on top of a hypervisor. And with containers, what you have is tends to be a lot smaller. So as some of you are probably aware, in a container you share the kernel from the the host that you're running on with the container.
So the container doesn't have its own kernel. It runs directly on top of the host kernel, and that kernel is also shared between all of the containers, which slims down the size that you require for the container at the cost that you can't do true virtualization.
Although in this case, virtualization has to be an issue. VMs are really used in the cloud as a way of packaging and portability as a, as opposed to you know, translating between machine code. So typically a container may be tens of megabytes or hundreds of megabytes as composed to VMs and gigabytes in size, primarily because we tend to strip down containers to just the sort of operating system essentials that are required to run our application.
But if you think about it, we can take that even further because there's a lot of stuff in the kernel that we don't actually need. And you think about it, Linux kernel is designed to run like multi user, multi process applications. Right? A lot of code in the kernel is just is designed for that purpose of handling multiple users at the same time and multiple processes at the same time.
But then we go and run containers that run a single process and a single user. So we actually don't need most of what's in the kernel. And that's kind of what unique kernels are designed to sort of approach a solution towards. So unique kernel is this idea that we can create a binary that will run directly on top of a hypervisor like a VM, or can run directly on hardware, and just contains the parts of the operating system that we need for our application.
So we can end up with a binary that's perhaps kilobytes in size, but contains like a full web server or something. Because we're able to drop all the stuff from the kernel that we don't need. Like, you know, the multi user stuff, the floppy disk drivers.
I mean who needs a floppy disk driver in these days? And so by dropping all that and combining it with an application code, we end up with a single deployable unit that we can run on bare metal or a hypervisor. And this has big implications for security because we cut down our attack surface, and also potentially for things like IoT.
So we can create like a deployable units that we can put directly in IoT devices. However, most of you probably aren't going to use Unicernels. I see this again as more of an implementation detail. And potentially, it could be used in stuff like serverless.
So we may see applications like serverless beginning to use stuff like Uni kernels underneath the hood. So if you don't know, serverless or function as a service is the term I prefer. It's kind of this idea that rather than rent compute power directly like VMs or containers from the cloud, what we can do is just say to our vendor, okay, I've got this endpoint.
When you want hit, when this endpoint gets hit, start this function up. And you don't see anything about the servers that it runs on. So now my AWS becomes responsible of handling all the servers, and I don't care about the servers. It scales up as much as it needs.
And the magic thing is, if my function isn't being called, I won't be charged at all for it. So I'm only charged when my functions are called. And that's going be a big thing in the future just because it can really drive down costs.
And also, I could stop caring about servers. So just to wrap up, we've kind of moved from this world where we had to buy expensive bespoke hardware and run it on our own premises towards a world of cloud computing. Largely for reasons of scalability and also capital expenditure.
And also because it's not a lot of fuel's core business. People don't want to run their service. They want to run a business. And at the same time, or slightly later, we've also seen this move away from a monolithic style of software architecture towards this microservices style of architecture for reasons again of reliability, but also for scalability.
So I can scale first over this approach and also speed of deployment. So people can deploy things faster in this approach. And if you're part of an organization which is facing challenges around these issues of reliability or scalability, or you're finding it difficult to change it as fast as you like, I would suggest, I strongly suggest that you perhaps consider looking at a microservice architecture or something like serverless.
That's all. Thank you.