Weeknote 9.0
Smaller is better?
A lot of the excitement around AI recently has been prompted by advances in ‘deep learning’ algorithms (see our glossary) that get better and better as more data and computing power is thrown at them. But from conversations we’ve had recently, it feels as though the mood is starting to shift slightly. Increasingly, experts are emphasising the need to do what we can with small data, and small compute.
In his keynote at CogX a couple of weeks ago, AI doyen Stuart Russell emphasised that as machine learning algorithms continue to improve, they require progressively less data. Which is helpful, because data collection and labelling can be very expensive. Pushing for ever more and bigger data may have diminishing returns, especially when we haven’t exhausted the potential of the data we have already.
At a conference run by the University of Southampton’s Centre for Machine Intelligence on Monday (more below), Professor Tim Norman made a not dissimilar point about computing power. Reliance on massive compute hosted in the cloud is impractical in a world where we expect our devices (and, increasingly, everyday objects) to act intelligently — sending information up into the ether and waiting for it to come back down simply takes too long. Developing AI that can work well on low powered devices is becoming increasingly important as a result, hence all the fuss about edge computing.
Then there’s the environment to consider. Vishal Chatrath, CEO of prowler.io, drew our attention to this paper published the other week, which showed just how astonishingly energy intensive cutting-edge deep learning is (training one of the systems discussed results in c.300x more CO2 emissions than a return flight between NYC and SF). Yes, there are applications of AI that could help address climate change, but we need to make sure that a big data, big compute approach doesn’t undermine that work.
Obviously, this is just a handful of random recent examples. The conversation has been developing for a couple of years, and there’s a lot more to add — on the legal and ethical issues that collecting and processing as much data as possible has already thrown up, on the pollution involved in the production of chips for all these IoT devices, and so on.
But even these brief notes throw up some initial lessons for government. As Jonathan Bright and co. observed in a local government context earlier this year, ‘while “big data” might be desirable, small data is often enough’. In lots of areas, government will already have the data it needs to start making improvements. The barrier to AI entry is lower than many might think.
This also means government shouldn’t collect more data simply for the sake of it. A clear sense of what it is trying to achieve has to be the starting point. Officials can then think about what data could be useful in the given context, and consider whether the financial/environmental/legal/ethical/time/other cost of collecting and processing any data they don’t have is worthwhile. This isn’t exactly new IfG territory — we’ve touched on similar themes before (see here, here, and kind of everywhere in our data work). Maybe we’ll come back to it again sometime…
…did someone say National Data Strategy?
Three things that happened this week
- Invites went out for the fourth edition of Data Bites, which is next Wednesday. A great lineup of speakers, as ever, this time from the Centre for Data Ethics and Innovation, GOV.UK, Newham, and our very kind sponsors for the evening, the Office for Statistics Regulation. Sign up and come along!
- Marcus has been tightening up and developing the outline of the workforce report, based on our interviews so far, with Lewis making a start on some of the introductory sections.
- We’ve been discussing our planned evidence submission to the Committee on Standards in Public Life’s Review on AI in the public sector. Having lots of fun replacing ‘Holders of public office’ with ‘AI systems’ at the start of each of the 7 Nolan Principles and seeing what falls out…
People we chatted to
- Gavin and Marcus had a good conversation with Jonny, Henna, Matthew and Lou from HMRC on Monday. HMRC has featured in previous IfG research as a case study for digital transformation and it was interesting to hear how this work continues with new and emerging technology. There’s a lot of broad thinking there about the applications and impacts of different technologies, both on their customers and their own workers.
- Meanwhile, Lewis was at a mini conference in Southampton about ‘Data Infrastructures for the AI Economy’. Lots of fun presentations that prompted discussions about workforce impacts, ways of automatically tracking goods crossing borders, and who decides what ‘fair’ actually means when it comes to addressing algorithmic bias.
- The team chatted to Vishal Chatrath, CEO of prowler.io on Tuesday — partly about AI and the environment, as above, but also about their work so far applying a combination of computational approaches and economic theory to fields such as logistics and finance to develop models of complex systems and processes.
- And on Wednesday we had lunch with a crowd from Demos/CASM. They were keen to discuss what ‘good’ actually looks like — in terms of how government uses technology, from our perspective, and in terms of the Internet as a whole, from theirs. It’s all very well criticising how things work now, but what we really need is more examples of how things can be done better.
- Marcus also attended a mini-conference at PwC on the future of the workforce. It featured some good discussions about the difference between job augmentation and job automation. He also got to meet a robot called Doris.
What we’re reading and thinking about
- Gavin has been thinking about the National Data Strategy call for evidence, and talking to some other civil society organisations about their responses. More to come on that. And two hardy perennials of his contribution to the project weeknotes: he’s also been thinking a lot about Data Bites — please do come along, tell all your friends! — and about the summer concert he’ll be singing in as part of the New Tottenham Singers. If you’d like to jazz up your July with a ticket, please get in touch.
- Marcus has been getting his head further into the size and shape of the government workforces in different countries: UK, USA, Canada, Australia and New Zealand. Understandably there’s a lot of similarity there, but looking across different countries has turned up some interesting analyses of what it is that government employees actually do, such as this report from New Zealand. He’s also been getting stuck into some interesting papers from Deloitte, including AI-augmented government and The future of work in government.
- Lewis has been reading Tung-Hui Hu’s A Prehistory of the Cloud, which goes beyond simply pointing out that ‘the cloud’ is actually a network of thousands of enormous data centres (as alluded to a few weeks ago), digging deeper into its origins in other network technologies and assessing the militaristic ideologies that underpin it. Fun stuff.
What’s coming up next week?
- Data Bites! Wednesday, 6–7.15pm, followed by drinks and nibbles. Sign up here.
- Lots of interesting-looking conversations in the diary — more on those when we’ve had them
- The Information Commissioner, Elizabeth Denham, and Minister for Digital and the Creative Industries, Margot James, are giving evidence on immersive and addictive technologies to the DCMS Committee on Wednesday morning.
Any last thoughts?
- As ever, hit us up with any thoughts or feedback! Email (digital@instituteforgovernment.org.uk) or Twitter or commenting on this or whatever works best for you — it all works for us.
