Often the first question mobile engineers ask ourselves when sitting down to build an application is what architectural patterns we should use. We try to lay the foundations for our house before we know how many rooms we need to build.
If we want to retain our ability to react to the evolving needs of our users, then we must accept that it’s impossible to know at the beginning of a project what patterns will best suit what is being built. All too often it turns out that the perfect patterns we chose at the beginning don’t work so well with the desired functionality, leading to code that's difficult to understand and maintain, and eventually to expensive rewrites.
So what if we could just... not choose? This talk will outline an incremental approach to application architecture that works in concert with product design, beginning with broad functionality and allowing the design to reveal itself over time as the specifics of our application become clearer.
1 / 36 Use ← → to navigate
Auto-generated transcript - may contain errors.
Tap a timestamp to jump the video.
Hello. Hi. My name is Matthew. I'm a self taught software engineer, and I describe myself as a recovering mathematician. Just because I think it sounds funny. That's not relevant to the talk, but there you go. I've been working in technology for a little over six years.
I started as a tester. I've been a software engineer. I taught software engineering for a little bit. I had a brief run into scrum mastering. But I'm now an engineer again. I care a lot about writing high quality code and building high quality products.
And I really like the places where those things intersect. And hopefully this talk is going be one of them. Currently I work at SoundCloud on the creator networking team, predominantly building mobile features for our creators. So recently I've worked on a redesign of our user profile, implementing profile editing, things like that.
You can tell I'm an engineer because I've tried to go for the award for least creative slides. If you've used Keynote before, you'll know this is one of the default presentation packs. Cool. There's a GIF later on. That will probably be good. How many people in here are engineers?
Wow. Okay. Cool. And for those of you that aren't engineers, presumably you'll know what it's like to work with engineers. So this should hopefully make sense for you. So as engineers, I guess we all know the feeling when you're working on a project in the early days, when things are going really well, when every decision you make seems to be paying off, you feel superhuman, incredible.
It's great. And we probably also know the feeling a little later in the project lifecycle when someone comes to you and says, hey, can you build this feature? And you go, ah, no. Because the perfect architectural decisions that we thought we'd made turned out to not work in every case.
All the things that we thought would speed us up later end up then slowing us down. Products take longer to build. And yeah, these decisions we made earlier are no longer paying off. I don't know the solution to this. That's the end of the talk. No, I'm kidding.
I don't know the one solution to this. What I can talk about are a few things that I've learned working on projects at SoundCloud that have sort of helped. Ways that we can let our architecture evolve rather than fixing it at a point in time.
So to do that, we are going to talk about what is emergent architecture? Specifically what I mean by it. That may be different to what other people mean by it. But yeah, we're going to cover that. We're then going to talk about the concept of being specific and how that can help us in our search for the right abstractions.
And then we're going look at how different approaches to testing can help or hinder us when it comes to making it easy to change our minds. So let's get started. So what even is emergent architecture anyway? Completely unrelated to the talk, I was having coffee with a friend yesterday and I told him what the talk was about and he was like, oh, so it'll be what even is architecture anyway, right?
Which is literally what I had on my slides. So that's how predictable I have become. So to discuss this, I think first we need to answer a simpler question, which is what even is architecture anyway? Who thinks they know a reasonably good definition of what software architecture is?
Like, three people? Cool. I was full of hubris when I started writing this. I thought I definitely know. I started to look around for a formal definition. There's a huge amount of disagreement. Is it overall project organization? Is it the specific design of the software?
Is it the way you have your files organized in a folder? A whole bunch of different opinions. I found a pretty good kind of catchall definition. I'm not comfortable citing Wikipedia in a talk, so I've attributed it to Brian Wikipedia. And Brian says that architecture is the fundamental structure of a software system, which is a pretty clear definition, sums up this idea that it's about the structures that we put in place.
So then when we start to think about emergent architecture, we might think that it's about making those structures easy to change. And this, I think, is actually summed up really nicely in a totally different quote, which is, in many ways, a much less useful definition of architecture.
This is from Ralph Johnson, who's one of the Gang of Four who wrote the Design Patterns book, which many of you may have pretended to read. I know I have. So he says that architecture is about the important stuff, whatever that is. I presume he meant it in a way that's a lot less sassy than I've just implied.
But I think what he's getting at here is pretty sensible. When you're thinking about software architecture, you're thinking about what's important to what you're building. And I would argue that what's important to a lot of us, as product engineers, is change. It's the ability to change our software later.
There may be many other things that are important for your particular situation, but the ability to react to change is one of the most important things that we can do. So with that in mind, this is the definition that I've gone with. So emergent architecture is having the right amount of structure when you need it and not before.
Cool. So with that set, let's talk about mobile application architecture. Who here has worked on mobile projects? Reasonable number. Cool. It's most of what I do. And if you start to look around for mobile architectures you find a lot of three letter acronyms.
Only one of those is a three letter acronym. You find a lot of acronyms. MVC, Model View Controller. MVVM, Model View View Model. You can all read. I'm not reading a Viper. It's too long. And these do what the first definition of architecture said.
Right? They break an app down into fundamental structures. And often when I've worked on teams and we've thought about mobile architecture, we've picked one of these and we've gone, done. That's the big tech architecture phase of the project is finished. But if we think back to the second quote, architecture is about the important stuff.
And all of these describe UI architectures. And if you're building an app, UI is important. It's really important. But it's probably not the only important thing. For us at SoundCloud, we do one thing that is not UI, which is probably the most important thing our app does, which is play audio back to the user.
We may also have background jobs to sync content. And none of these are anything to do with what's happening on screen. I've summed this up in the following way. Just your UI is not your architecture. And this is a mistake that I think a lot of people make when building mobile applications.
I suspect the same is perhaps true on web, but I don't know enough about that to claim it, so don't quote me on that. And if we start to think about our architecture as more than just our user interface, about the way our app works outside of that, we can already start to make our architecture more flexible, even in cases where we essentially are talking about a bit of UI.
So I'll show you that just now using an example from SoundCloud. This is the SoundCloud user profile that I was involved in rebuilding recently. And when we started with this project, we broadly knew the features we wanted to provide to the user. But the lovely UI that you see here wasn't finished.
The designers hadn't done with it. They were going through iterations, were user testing, they were trying out a whole bunch of different things. So when sketching out rough ideas for how this code might work together, when spiking on bits of it, we had to focus on the behaviors and how they might work with any UI that could go on top of it.
One of the big insights that came out of that is a lot of the stuff that's on this screen I'm going to stand over here because that's there. A lot of stuff that's on this screen a fundamental part of a user's profile. Right?
So we have their avatar, we have their name, we have the header image, the content here, we have their tracks. But actually, one thing here, when we think about the domain objects that we're representing, the business objects we're talking about, one thing here is not related to any of that, which is this follow button, currently in the following state there, which has nothing to do with a user's profile.
It's related to a relationship between two users. And when we realized that, we realized that we could, and in fact should, build the following relationship completely abstracted from the rest of the profile. It should have nothing to do with the rest of the profile.
Had we looked at this UI first, I can tell you what we definitely would have ended up building that follow button into the header. And it would have been so much harder to work with, because it has nothing to do with the rest of that object's life cycle, the rest of how these things work.
This is sort of broadly This uses aspects of sort of domain driven design, which I'm going to name check here and not really talk about in the rest of the talk, which is terrible practice. Sorry. There are some wonderful books. It's a pretty common idea in microservice architectures.
It's not so common in mobile architectures, which is unusual because you're representing often the same or similar sets of objects. You're still representing problem domains and finding ways to display them to users. Another benefit that you can get if not thinking UI first is you can do things like this.
This is partway through the project. The header design, the top bit here still wasn't finished. But we knew that we wanted to have follow and play all functionalities. So we built them. We built all the functionality. And then we put them into this temporary profile header in my favorite color, which looks lovely.
I got to write the copy for this. That's why it says, do what unfollow? Because I enjoy poor use of English. Hopefully, of this has started to sell you on an idea of architecture that is more than UI on mobile applications. So how do we go about actually making it easier for this to be emergent?
The answer is sort of counterintuitive, and it's summed up in the next section. If you're doing something specific, be specific. And even if you're not, I couldn't fit this on the slide, but it's important. Even if you're not doing something specific, pretend that you are.
In software engineering, we're taught that reuse is vital. That's the most important thing. I used to teach software engineering, and within the first week, we taught students about dry. The idea of don't repeat yourself. Who knows what dry means? I'm assuming most people. Yeah.
Very quick example for anyone who doesn't. Don't repeat yourself basically means if you have code that is similar, put it in one place. So here we have like a person object. They have a first name and a last name. And in some file, we're creating a full name by adding them together with a space in the middle.
It might be perfectly sensible to go, this code is the same. So let's put it into a function. So far, so simple. By the way, this is Swift. I tried to keep it very simple. If you don't know Swift, hopefully it's still readable.
Anyway, so now we've pulled the function The code here into its function full name. And this is great because if we need to change something that affects every instance of the full name, we change it in one place. Brilliant. Problem solved. So if we, for example, want to add a middle name, and we want to have first name, middle name, last name, every place we use the full name, done.
And when have code that's not dry, we end up with a bunch of issues. We can have correctness problems. We can have things looking totally wrong in one place, and that is a horrible issue for our users. But we can also have problems if our code is too dry.
So let's say, for example, we had similar code to before. We have first name, a middle name, and a last name. In one file, we have the first name plus a space plus a last name. In another file, we have the middle name there.
And this is a deliberate decision. Right? Our designers have said we only show the middle name on whatever screen. We might still look at this and go, well, this code is all the same. I don't want to repeat myself, so I want to bring it into a function.
And you might, with perfectly good intentions, write a function like this. Again, ignore the specifics of the code. I know this is not good code. I was not trying to show off my talents here. Still have the first name. No, I did not highlight it.
We still have the first name and the last name in there. But in the middle, have this conditional that says if we're meant to show the middle name, show it. Otherwise, don't show it. And this is reusing the code. We're pulling the code into one place.
We're not repeating ourselves. We're really happy. And then we get a request that says, hey, in some places, regardless of whether or not we're showing the middle name, we want to show the initials. So we either show first name, middle name, last name, first name, last name, or first initial, last name, or first initial, middle initial, last name.
So we've massively exploded a number of possibilities. And we look at this code and we think, well, there's no how to do this. This is fine. So we add another boolean for should use initials, and the code that we end up with is horribly complex, I haven't even bothered to insert it.
And you may look at this and think, but Matthew, just don't write code this way. And you would be right. That is obviously the right answer. But we have this problem. Because the code was already written in this style, and I have a quote here from Sandy Metz, who is a pretty well known Ruby developer.
Some of you may have seen some of her amazing talks. Who says existing code exerts a powerful influence. Which is to say, if your code solves a problem in a given way, and you need to solve a similar problem or a problem in a similar part of the code, you're going to look to the existing code for an example.
You're going to try and copy that and make it look the same. So it's not quite as simple as just not doing the bad thing when it becomes really bad. It's slightly more complex than that, and then we have to work slightly harder to avoid backing ourselves into this corner.
So the best way that I've found And the way this ties into a sort of idea of emerging architecture. The best way that I've found to avoid negatively influencing the architecture of future code is to actually write less reusable code. To try and write code that is deliberately not reusable.
So that was a toy example. And sometimes you may think, but you see something and you just need to build something general. Sometimes you look at an app or you look at a screen in an app and you go, I know that all of these have a similar representation.
Here's the SoundCloud user profile. Lovely GIF. This is the GIF that I mentioned earlier. Know, top quality slides craft there. It moves. That's exciting. Alright. Lost my train of thought there. I got excited by the movement. It's very distracting. So you might look at all of these content buckets.
We have this Pinto Spotlight, this Top Tracks, this Albums. You may look at each of those and go, surely they're gonna share some sort of representation. Right? They all solve a kind of similar problem. One is a list of tracks. One is a list of playlists.
One is a list of albums. And some are lists of both bits of content. But you can look at that and go, the code behind them is going to be pretty much the same. You might try and do something like this initially. Don't worry too much about the type names here, but these are just kind of rough examples of some code.
You might see we're going have to present an array of content buckets. That's the object we're dealing with. That's the domain. Pretty obvious. And then when we tap on an item, we're going to tap on either a track or a playlist in a specific content bucket.
That would be sensible. But when we were building this, we wanted to just focus on building one thing initially, because building one thing is easier than building three things. So we thought, let's deal with a list of tracks. So when we wrote the code, we implemented this.
We knew we were going to have these kind of content buckets. We knew we were probably heading in that direction, but we really didn't want to bias ourselves. We didn't want to back ourselves into a corner where an assumption that we made on day one of the project changed a couple of days later, a couple of weeks later.
So we had this present tracks and did tap track. And then when we implemented playlists, we did the exact same thing. We deliberately duplicated the code. We made it so that when we came to unify them into sort of tracks and playlists thing, we had to look at both of the existing examples and work out how to abstract them together.
Rather than starting and perhaps making the wrong abstraction, we forced ourselves to constantly make decisions. So we had to look at this and really go, like, okay, is this idea that we had a sensible one? Like, do we want to bring these together in the way that we thought?
And this is one of the sort of key points of what I think of as emergent architecture, which is that you delay big decisions until the last minute by finding ways to force yourselves to reconsider them later. So you write code that seems super counterintuitive, but it forces you later on to write code that ends up better.
Cool. Anyone who knows me will know that I love tests. And I couldn't give a talk on software engineering without talking about testing. That's in my contract. I really like being able to prove that what I'm doing is correct. That makes me really happy.
And you might think, what has testing got to do with architecture? And for me, this is really about the emergence. It's about making things easy to change. In order for things to be easily changeable, we have to know that the changes we're making are correct.
Otherwise, we lose confidence. So with that in mind, I've got this really interesting quote. Please have tests. If you're not an engineer, if you're someone who manages engineers, please make sure your engineers write tests. You might think, we don't have time. We're shipping on Friday.
It will bite you later. Every time I've not written tests, it has bitten me sooner rather than later. Having tests is the number one way to make sure that you can change your code with confidence. Obviously you can do things like manual testing and that can help.
But over time the required is just The time input required is just too big. And so without tests, being able to constantly re architect and rewrite is essentially impossible. And this isn't just conjecture. I'm not just making this up. I've worked on projects that are super well tested and projects that have essentially no tests, and I know which ones are easier to change.
This is despite the fact that you may think something like this. And earlier in my career I did think things like this. If I test a lot, if I write a load of tests, then refactoring is actually harder because every time I change the code, the tests break.
And yeah, that's true. Testing too much can be as much of a problem as not testing at all. Or more accurately, testing in the wrong way. If our tests break every single time we change the structure of our code, then we have no confidence from our tests.
So if we want to enable our architecture to emerge, we need to be able to change our code a lot, which means that as much as possible, our tests have to be completely resistant to that change. If I'm totally honest, know I said at the start I don't have an answer to this.
I really don't have an answer to this. This is something I'm still working on constantly. Every time I write tests, I'm trying to work out new ways of doing this. But here are some kind of concrete tips that I found that may help you get a little bit better.
First one is to focus on behavior. This is an example of a test from the SoundCloud user profile integration test suite that I wrote. So rather than thinking what components does this class interact with, we think about the behavior we're trying to implement.
We think specifically like what am I trying to do? Which feels kind of like a dumb thing to say. It feels very obvious. But the number of times that I've sat down and thought, what classes am I going to call rather than what am I actually trying to do is ridiculous.
Sometimes it's even useful, as I've in this case, to write a facade or a fake interface for testing purposes. So here I wanted the test code to read pretty much like the test name. Because the test name is very unlikely to change here.
When a playlist is tapped, we probably want to open the playlist. Right? Regardless of how that's implemented, that's almost certainly the behavior we want. So here I've got when players are tapped. And I have when item is tapped. We tap on that playlist identifier.
And then it opens that playlist. We can check that the playlist launcher opens that playlist. And with this, the underlying architecture of this can be completely rewritten, can be completely changed. And each of the individual components of this, like the way that we launch a playlist, can be changed, and it only affects that part of the test.
Or ideally, it doesn't affect the test at all. Especially for changing stuff that's internal to the way the screen works, that's not on the interface or not on the boundaries, this will never need to change. And should be able to run and compile and fail or pass, depending on whether or we've got the code right.
I know this is a really extreme example. Would not expect to write this for every single test. This took a bit of time. It took quite a lot of refactoring. But it came about quite naturally as we refactored our tests, alongside refactoring our production code.
But there are simpler things we can do. And one of the simplest things is to try and think about all of our tests or as many of our tests as possible as inputoutput tests. Because if we're The idea here is that if we're looking at the code, it's easiest to look at just what goes in and what comes out.
So if we're testing a plus function, one and one go in, two comes out. That test, no matter how the plus function is implemented, is always going be true. So if we can find tests like this to run-in our code, they're going to have to change a lot less when the implementation changes.
Code often doesn't have this structure, but we can still do our best to think about it in this way. So for example, in the SoundCloud user profile again, we have this presenter object, And the presenter's job is to take business logic objects and turn them into view model objects.
Objects that can be used to build views. It's roughly responsible for deciding how things are shown on the UI. In reality, we're telling it to present. It's doing some internal logic. It's calling out to some mappers that are passed in, maybe a bunch of other components.
And then eventually it's calling methods on a view. But we'd like to think of this only in terms of what we put in and what we get out. So we thought of it just in terms of passing in the user profile model and getting out the view model.
And we wrote the tests to look like it was just doing this, even though it was maybe calling out to five or six different objects in between. Initially we had a problem because these mappers that are responsible for, say, taking a track and turning it into a track view model, or a playlist and turning it into a playlist view model, were mocked out.
And when you have mocks like this, you are very coupled to the structure of the mock. So the name of the code you're testing, or even the number of parameters it takes, things like that. Which means every time you change that, your test won't compile or your test will break.
So what we did was pretty simple. We just didn't mock them. We just stopped mocking, and we said, okay, so we're going to be testing slightly more behavior or there's going be slightly more behavior happening in the body of the test, but if we mock as little as possible, we can change as much as we want and only care about the input and the output here.
So now our tests only care about the behavior of the program rather than how that behavior is implemented. And this lets us change anything we want about the structure. We could completely rewrite the architecture of this presenter layer, and it would still pass.
Which lets us change our architecture all we want. Cool. So in summary, the definition of architecture that we have for mobile applications is often flawed. It often focuses way too much on the UI, on how things look, and not enough on what we're doing.
We can build time for these architectural changes into our processes by being specific, even when we know or we think we know that what we need is a general solution. And in order to make code easy to change, we have to have tests.
And ideally our tests need to be decoupled from the specific implementation that they're testing. And this is important because in an ideal world we just follow all these rules and we never have to worry about architectural changes ever surprising us again. But that's not software engineering.
Everything we build comes with a maintenance cost and the potential that at some point in the future it might be wrong. So if we accept that our architecture is in a constant state of change, that we must constantly be redesigning and redesigning, and we optimise for that, it means that there's no done.
There's never a point where our architecture is finished. We never have to do anything again. We are always ready to redesign it, to change it, and we have the confidence to change our architecture to support user needs that we weren't expecting. That's all I have to say. Here is my LinkedIn.
If you, for any reason, wanted to message me, that's all. Yeah, thanks. Thank you very much. Matthew, Liam, Healy.
