In the past few years much of the code written in government has been either written in the open or later published under open source licences. This has been a huge transformation from the previous government culture. Anna is Open Source Lead and will talk about how this transformation has happened, what it means in practice and the benefits and challenges of coding in the open.
← All talks 
Build Engineering Turing Fest 2017
Coding in the Open in Government
Auto-generated transcript - may contain errors. Tap a timestamp to jump the video.
So it's twenty twelve, and the way you find out information about anything to do with government is looking at this site, DirectGov. And here's a tax calculator on DirectGov. So this is a tax credits calculator. This is where you put in some information about your income and other behaviors, and it will give you an estimate of how much tax credit you might be due.
And if you as a member of the public found an error with this tax calculator, this is what you'd have to do. First, you'd have to get in touch with the DirectGov team. So let's say that was straightforward. You filled in a form on the website and you got talking to the right person.
You explained what the error was. Then the DirectGov team would have two options. If it was a content error, that would be no problem. So you'd notice a typo or something like that. DirectGov is a CMS so they can make that change straight away.
But if it was a code error, if you'd notice some kind of issue with the calculation, that was a different matter. The DirectGov team would have to talk to their supplier because DirectGov is outsourced. And in order to investigate whether the error that you spotted required a code fix.
So this is just for the investigation. The cost would be tens of thousands of pounds. And if they did realize that the error did require a code fix, then the estimate for fixing that would be between fifty and a hundred thousand pounds. And DirectGov was big bang releases once a year.
So if you timed it badly, the error you raised you could maybe be waiting eighteen months for that to get into production. But realistically, if it was a small error, the chances are that wouldn't happen because that's a lot of money to spend on fixing a small code change.
Later in twenty twelve, we launched the gov. Uk website and that replaced direct gov. Let's fast forward now to twenty sixteen and this is another tax calculator this time on gov. Uk. And this is a calculator that tells you how much tax you might have to pay on any child benefit you receive.
And a member of the public did find an error in this calculator. So the error was in the situation where you were claiming tax benefit, you were earning over fifty thousand pounds a year, and you're also contributing to a post tax pension. He found that one of the figures used was the the net instead of the gross.
So he went to the code on GitHub and he made a fix to fix that error. And he raised the poor request. So he raised the poor request last year twenty fourth of January, a Sunday. On Monday the twenty fifth of January, the team came in, they saw his poor request, they discussed it, they agreed that he'd found an error and a code fix was required.
On Wednesday the twenty seventh of January, they merged his code and on Friday the twenty ninth of January, the code was deployed to production. So that's quite a big change from how the direct Gov website was run. And the reason that he could do that was because gov.
Uk is coded in the open. So I'm going to talk to you today about what coding in the open is. Why we do code in the open? What the benefits are and a bit about what the benefits are to government and to industry.
My name is Anna Shipman. I am a technical architect. I used to be technical architect on the gov. Uk website. I was then technical architect on gov. Uk PaaS which is a platform for hosting digital services and I'm now the open source lead.
When we talk about open source in government, we tend to be talking about three different things. Firstly, using open source software. So gov. Uk runs on NGINX. We use MySQL. We write it in Ruby on Rails. So we definitely, we use open source software in government.
The second thing we tend to be talking about is contributing to open source software. So that's you find a bug in NGINX and you submit that patch back to NGINX and we do that as well. The third thing we tend to be talking about is open sourcing our own software and that's what I'm talking about today.
Over the past six years, a huge amount of government code has been published in the open. Either coded in the open or published openly after it's finished. And again that's not as Rachel mentioned, it's not what you might think of as government. So I explain how we got here. I think of a little history.
In twenty eleven, Baroness Marcelane Fox was asked to write a report on DirectGov, the first site we saw. And she actually went wider with her report and looked at digital across the whole of government. And she made several recommendations. One of which was the formation of the government digital service and that's the the that's what I work for.
It's a unit of government within the cabinet office which is in the center of government. And our job is to transform government into a more modern digital organization. First thing we did was launch the gov. Uk website that I've already mentioned. And we're now working on helping government be digital by default.
So a couple of examples of transformations we've done at GDS. So you can now register to vote online. We launched it before we launched this service, The way you would register to vote was fill in a paper form and send it back to your local council.
You can now do that online. Another example is you can now view information about your driving license online. So you can go and see if you have any penalty points. What vehicles you're entitled to drive, that kind of thing. Previously, you'd have to do that by looking at the paper version of your form.
So another thing we have is the digital service standard and all new digital services in government have to meet this digital service standard and that's one thing that GDS helps support. And the digital service standards is a set of principles that digital services have to meet.
And point eight is make all new source code open. So all new government digital services have to make their source code open. So I said that gov. Uk is coded in the open and I'll just explain what I mean about coding in the open rather than open source.
Because coding in the open is technically open source. Open source means the software, the code, the software where the code is freely available and it's published under a license that allows free use modification and sharing. So the GDS code is in our GitHub organization, AlphaGov.
So the code is freely available and we publish it under the MIT license. So it is technically open source, but the reason we refer to it as coding in the open rather than open source is because even though the MIT license, you know, sort of limits warranty, there's sort of certain things that you expect about open source that we don't really support or offer.
So we don't expect that your R code will necessarily work in your environment. If it does that's great but we're not doing anything to support that. We don't make any guarantees that we're going to continue to support it at all. So for example, if we realize there's a better way to write calculators, we'll just retire the calculator code and write new code.
We won't think about who might be using that code. And we haven't worked to build a community around the code. There are some communities around the code but we haven't worked on maintaining that. Community is really important for open source and Mike's going to talk about a bit more about that next.
But what we mean by coding in the open is that it is all, all our code goes to the open straight away. So you make a commit, you push it to GitHub and that code can immediately be seen. You raise a pull request that can be seen publicly.
Our code review takes place in public. Our code is merged in public. And the reason we do that is because one of our design principles is make things open, it makes things better. So I'm gonna tell you now about how coding in the open makes things better.
I'm gonna talk about how it makes the code better, how it makes the organization better and how it makes government better. So the headline about how it makes the code better is basically, it encourages good practice because when you know that somebody could be looking at your code, you you raise your game.
You maybe you try harder to write good commit messages because you know that somebody might be looking to see what the intent of that code was. You're positive and constructive in code reviews because you know that anyone could be looking at that conversation.
You pay attention to security. You think about making sure that you don't commit your AWS credentials for example. You make sure you separate your code and configuration. And you make sure you follow good architectural principles because the point is that even if nobody is looking at your code, people could be.
And so you you everyone works hard to make it a bit better. Second way it makes the code better is it makes it easier to work with other people. So, I'll give you an example that I really like if something happened recently. So this is so all government services are mandated to look like gov.
Uk even if they're produced by a different department. So this is a service from DVLA allowing you to take the registration, the license plate off your vehicle and put it on a new vehicle. But as you can see, it looks like gov. Uk even though it's produced by a different department, it's produced by DVLA.
And the easiest way for other departments to make their code look like the gov. Uk front end code is to use the gov. Uk front end code. So the team that is that maintains the gov. Uk front end code is working on making it easier for other departments to use that code.
And earlier this year they tried an experiment to see if they could transpile some of the code into different languages. And they weren't quite sure how to do it, do it. They were having some problems. And so they discussed it in an open community, design community Slack channel.
And somebody from 18F saw that conversation. So 18F is an organization in the US which is similar to GDS, does similar things. Anyway, so this guy saw the conversation. He thought that looks like an interesting problem. Talked to the team a bit about what exactly they needed to do and then went away and wrote some code that solved their problem.
And so the team were then able to use their code to finish their experiment and see if that would work, use his code. And so the reason that was able to happen was because they coded in the open, because they were open about the problems they were facing, because he coded in the open.
And so those people even though he's in the US and they're in the UK, they were able to work together. They've written a blog post about it. I'll publish these slides afterwards and I'll include links to all the blog posts and things that I mentioned so you can check that out later.
Closer to home, there's another example. So the Ministry of Justice have written a tool to allow you to find out the postcode if you put in an address and vice versa. And HMRC have also written a tool to do pretty much the same thing.
And because both of these teams had written that code in the open, they became aware that they were both solving the same problem. And so now those two people in those two departments are working together to make sure that they both do the same things and then they're gonna turn one off and use the other one.
And again, that was possible because they coded in the open and they can work together on it because it's in the open. They don't have to go you know walk from one department to the other or travel from one department to the other.
And Another way that coding in the open helps make the code better is that other people can do work for you and help make your code better. So I showed you the example at the beginning of somebody fixing an error in the calculator's code.
We've got quite a few more examples of that. So, you might have seen this site, this is the petition site and this is where you can create a petition and if it gets over a hundred thousand signatures then it will be debated in parliament.
And last summer there were some quite high profile petitions that got a lot of attention. And so it used to be the case that when you went to look at a petition, the number at the bottom of people who'd signed it was the number of people who'd signed it when you went to look at that page.
And it wouldn't change as you read about the petition, decided whether you want to sign it or not. That number wouldn't change. But last summer there were some high profile petitions that got lots and lots of signatures. And so somebody added the ability they basically added updating in auto updating in real time.
So after he'd added that and they'd merged his change when you were looking at those petitions you would see the numbers go up as people were signing it. Another, somebody else added So I said that a debate would happen in parliament if there's more than a hundred thousand signatures.
That debate gets scheduled and somebody added to the view the scheduled date. So these are sort of, these are ways that people have helped make this code better. And then this is an example I really like because it's a really tiny example. So this is, those two were people who don't work for the government.
This is somebody who works in a different part of government and was using some of our code. And he noticed that so if you see the top one, the left blob is like chopped off a tiny bit. So the offset was slightly wrong.
So he fixed it to the bottom one. And this crown is used in one place. It's used in Safari pin tabs. So that is a tiny, tiny little favicon. There's just no way that you would have noticed that if you hadn't been looking at the code.
And that's I really like this example because the you know, if you had to raise a bug report and explain that to the team, it would take so long. And then how would it get to the top of the team's priorities? It probably wouldn't be that important.
But you can just if the code is open, you can just scratch those itches. You can make that fix. You can suggest that the team take it on. So I don't wanna, I don't wanna oversell this point. I don't want you to get the idea that you know, you make your code open and suddenly there is a whole army of people who are willing to do all your work for you.
It's not quite like that. These examples, we have a few of them. We have a lot of code. They're few and far between. But the point is, if the code is open, it's possible for people to make those changes. If it's closed, that just can't happen.
Okay. So those are some examples of how it makes the code better. I'm now going to talk about how it makes the organization better. So coding in the open makes it easy to share your learning. We have a format on gov. Uk which is called a smart answer.
And that's a way of presenting complex information so that it's easier for you to find the information you need to know. So an example of that is this is a smart answer that allows you to calculate how much leave or pay you might be entitled to if you have a child.
So that's maternity leave, paternity leave, shared parental leave and pay. And the rules around that are extremely complex. So we could present just a wall of text, lots and lots of information and you would have to go in and work out what's relevant to your particular situation.
But with a smart answer, what we do is we ask you a few questions. When's the baby due? How long have you been with your current employer? How much do you earn? And then at the end of it give you the information that's relevant to you.
So another part of government needed to do something similar. So the Skills Funding Agency needed to present some quite complex information about how much of a levy you might be entitled to if you took on an apprentice. And it was quite complex because there are lots of different kinds of apprenticeships and there are various different rules that apply to them.
And they were told they had to produce a tool to allow people to calculate that. And they started working out how, you know, how they do that and they estimated that the work was going take them about two months. And then somebody in the team pointed out Smart Answers.
And they went to look at the Smart Answer Code on GitHub and it wasn't quite what they needed. It was in Ruby and the rest of their code was in C. They wanted some features that the Smart Answers Code didn't have. So what they did was they used the Smart Answers code as a template and they basically copied the pattern to write their own tool.
So it's not a perfect open source story like a perfect open source story is they go and get the Smart Answers code, they use the Smart Answers code, they make changes and improvements and they contribute them back so that other people can benefit.
But it's a great story because they used the Smart Answers Code and they built this tool in a week. So they were able to get our learning and save themselves a lot of time and money. So another way that it makes the organization better is that you can show off like details about what you're working on.
So next time if it's you up here on stage wouldn't it be really nice if you could like point at some of the code you've written. It helps build up you know pride in what you're working on with specific examples of stuff you've worked on.
And that's good for the organization. And another sort of similar example of that is gov. Uk Puppet code is open. So Puppet is a configuration management tool. It's how we configure our service in production and the GOV. UK Puppet code is open on GitHub.
And now Puppet, the company uses the GOV. UK Puppet code as a reference example. That's really nice, like, it's really nice for us working in the organization and it's good for the organization. So, yes, so basically, if you can share your code then you can, you can get that positive effect.
Open code can also advertise your interesting problems that you're working on. And code review, if you do constructive code review in public, that can act as a positive advert for your culture. So people can see that code review is constructive and think, realize that it's a nice place to work.
So I mean another way of putting that is coding in the open is good for recruitment and it's good for retention. So those are some ways it makes the organization better. I'm now going to talk about how it makes government better. There are three ways that coding in the open makes government better.
Three, three most important things. The first is, it means that you can see what's being done with public money. So it's good for transparency. Coding in the open supports more ways of open working. So this is a GovUK roadmap and that's open. This is a Trello board of one of the teams on GovUK showing the tasks they're working on.
That's open. And on Gov. UK when we have any kind of incident like an outage, we have a post mortem and then we write an incident report and those are also published openly. So you could do all those things without coding in the open but when your code is open that's kind of underpins like a culture of openness across the organization.
And that's good for tax payers because you can see what public money is being spent on. The second thing is coding in the open forces clarity on the ownership of the code. So I mentioned we use the MIT license and the copyright of the MIT license is crowned copyright.
So the queen owns the copyright for all the work that we do in government. And when you when you code in the open from the beginning of the project and you release under a license then it forces clarity around those issues at the beginning of the project.
So you make sure that government owns the IP for the work that's done, the intellectual property. So the advantage of that is let's say a department doesn't have developers and they need to recruit in suppliers to work on something. They could recruit some suppliers to work on the first part of the code and then you know, if they did the first part of the project and then they could maybe recruit in some other suppliers to work on the second part of the project and there's no issue about working on the
same code because government owns the IP for that code. Or you could build a team from multiple suppliers and you can all work on the same code. And the big advantage is it stops us being locked in to sort of massive contracts. Because if you, if government doesn't own the IP for the work that's done in a massive contract.
When that contract comes to the end we've got two choices. We can either renew the contract at whatever cost they say or we can start again from scratch with a new supplier because we don't own the IP for the code. Whereas if the project is coded in the open from the beginning and those IP issues are sorted out then we don't, we're not locked in to that supplier.
And finally, it's cheaper. So access control is complex and so it's expensive. It's complex to work out who can have access to what things and so it costs money. And public code repositories are usually free. So it saves money. So there are two benefits to open source that are usually given as the headlines.
And I haven't mentioned them and I'm just going to talk about why I haven't briefly now. The first is reuse. The idea that you you could not invent the wheel, reinvent the wheel so you could write some code and then it could be easily reused by someone else and so that's a good thing for open source.
And that is, that's great but it's not always that straightforward. You know, we know that sometimes it takes a lot of work to use open code. Sometimes you have to make a lot of changes. And so it's not quite as simple as you know, you can write it once and then reuse it and there's no problems there.
It's great if reuse happens but I hope I've shown that there are lots of benefits to coding in the open even if there's no reuse happening on any significant scale. And the other thing that people often say is this quote, given enough eyeballs all bugs are shallow.
And that's the idea that enough, if enough, if the code is open and enough people look at it, can fix the bugs and make the code more robust. And we know that that's not true because of heart bleed. So this was a massive bug that was in open SSL.
Pretty much everyone was using open SSL for HTTPS and the bug was in the open code for two years before anyone told the maintainers about it. So the fact is that just because the code is open doesn't mean that enough people are going to be looking at it and fixing your bugs for you.
Open doesn't mean that there will be enough eyeballs. But, so the point I want to make is even if those two things aren't happening, it's great if they are, but they're nice to have. Coding in the open has a lot of benefit even if those things aren't happening.
So there are some challenges to coding in the open. I'll talk about two kinds of challenge. The first is cultural. It can be quite difficult to get started. It can be quite stressful if you come from an organization where you don't code in the open to think that anyone could see your code.
It can be stressful to think that you're gonna have a code review in public and people are gonna maybe be criticizing you in public. So it's important to build a strong culture around that. So for example, we have guidance around how to raise pull requests and how to review pull requests.
We do a lot of automation in the pull requests. So for example, we automate linting to check that you're meeting the house style so code reviews don't descend into a political war about tabs versus spaces. That's all done automatically and the code review is just a constructive debate about the, that particular implementation.
So the point is you have to make sure that you have a strong culture around supporting people when you code in the open. And then the other challenge is making sure that you don't publish the wrong thing. So that's kind of publishing you know publishing credentials as I mentioned.
And there's two ways to address that. Firstly, you should have processes that make mistakes harder. So maybe you want to protect your master branch so you can't deploy something to production without realizing. You store your secrets separately and you inject them into the code.
And I said at the beginning that coding in the open raises your game. It forces you to do things properly from the beginning. It forces you to think about keeping your secrets separately and injecting them in. Because the fact is, if your code is closed and you sort of are lax about that and you might have secrets in the wrong place, they will leak anyway even if the code is not made open.
So you should do things right and coding in the open forces you to do that. And the second thing is you need to be able to recover quickly. So even if you have really difficult, know, even if it's really hard to make a mistake, mistakes will happen.
And the important thing is you need to be able to recover from any mistake quickly. So for example, you need to be able to rotate your credentials quickly. This is a nice blog post, it's worth checking out which talks in more detail about this.
About how the important thing to optimize is the mean time to recovery, not the mean time between failures. Okay, so that's all very well for government. We don't make a profit from the software. We're not in the business of selling software. So you might think that's nice if you're not working for government.
You might think that's nice but you know, we can't make our code open. But there is value for commercial companies as well in making their code open. So to explain that, I'm just gonna give you some examples of the commercial companies that do code in the open or release open source.
So The Guardian, they do a lot of coding in the open. You can see their code on GitHub. The Guardian is a print newspaper and their website is free. So maybe that's not a great example but the Financial Times also code in the open and the ft dot com makes more money than the print edition.
And as of last year, it makes more money than advertising. So ft dot com is a major source of their income. Another content provider that releases a lot of open source is Netflix. So you can see a lot of their open source code on GitHub.
And they're pretty obvious about why they release open source. If you go and look at their website about it, the headline before you even see about the open source is do you want to come and work for Netflix? They use it as a recruiting tool.
So those examples are content providers. They're not actually selling software. So let's talk about some examples of companies that are selling software. So GitHub. GitHub also released code in the open and one of the things that GitHub has released is LibGit two which is a tool that you can, it's an engine that you can use to run GitHub.
And GitLab who are a direct competitor of GitHub use LibGit two to run their platform. But, you know, you don't see GitHub going out of business because of that. GitHub is going from strength to strength. And then finally, want to give you the example of FeedBin.
So FeedBin is an RSS reader that allows you to keep up with blogs or news. And FeedBin is a paid for service but it is entirely open source. If you wanted to, you could take the FeedBin code and run your own FeedBin. But people don't, I mean some people do but many people still pay for the FeedBin service.
FeedBin's been entirely open source for four years and is still going strong so it hasn't affected their business. And you should read his blog post because that's a good example of why he thinks it's positive. So hopefully, you are all really enthused about the idea of coding in the open.
If you're not doing it already, you're thinking how can I get involved? So I've got some answers on that. Three things you can do. Firstly, can have a look at our code. So, the government digital service code is on, is on AlphaGov, is on GitHub in our AlphaGov organization.
GitHub have a page of other government organizations that code in the open, so you can go and check out others. The Scottish Government code is on GitHub. You can go and check that out. You can make company make code from your company open and maybe you can use this talk as an example of what the benefits are if you need to persuade anyone.
And finally, you can come and work for government. DDS are hiring. The Scottish government are hiring. Come and work for us. Make things open and make them better. Thank you.