Today’s artificial intelligence (AI) systems are built on data generated by humans. They’re trained on huge repositories of writing, images and videos, most of which have been scraped from the Internet without the knowledge or consent of their creators. It’s a vast and sometimes ill-gotten treasure trove of information – but for machine-learning pioneer David Silver, it’s nowhere near enough.

“I think if you provide the knowledge that humans already have, it doesn’t really answer the deepest question for AI, which is how it can learn for itself to solve problems,” Silver told an audience at the 12th Heidelberg Laureate Forum (HLF) in Heidelberg, Germany, on Monday.

Silver’s proposed solution is to move from the “era of human data”, in which AI passively ingests information like a student cramming for an exam, into what he calls the “era of experience” in which it learns like a baby exploring its world. In his HLF talk on Monday, Silver played a sped-up video of a baby repeatedly picking up toys, manipulating them and putting them down while crawling and rolling around a room. To murmurs of appreciation from the audience, he declared, “I think that provides a different perspective of how a system might learn.”

Silver, a computer scientist at University College London, UK, has been instrumental in making this experiential learning happen in the virtual worlds of computer science and mathematics. As head of reinforcement learning at Google DeepMind, he was instrumental in developing AlphaZero, an AI system that taught itself to play the ancient stones-and-grid game of Go. It did this via a so-called “reward function” that pushed it to improve over many iterations, without ever being taught the game’s rules or strategy.

More recently, Silver coordinated a follow-up project called AlphaProof that treats formal mathematics as a game. In this case, AlphaZero’s reward is based on getting correct proofs. While it isn’t yet outperforming the best human mathematicians, in 2024 it achieved silver-medal standard on problems at the International Mathematical Olympiad.

Learning in the physics playroom

Could a similar experiential learning approach work in the physical sciences? At an HLF panel discussion on Tuesday afternoon, particle physicist Thea Klaeboe Åarrestad began by outlining one possible application. Whenever CERN’s Large Hadron Collider (LHC) is running, Åarrestad explained, she and her colleagues in the CMS experiment must control the magnets that keep protons on the right path as they zoom around the collider. Currently, this task is performed by a person, working in real time.

In principle, Åarrestad continued, a reinforcement-learning AI could take over that job after learning by experience what works and what doesn’t. There’s just one problem: if it got anything wrong, the protons would smash into a wall and melt the beam pipe. “You don’t really want to do that mistake twice,” Åarrestad deadpanned.

For Åarrestad’s fellow panellist Kyle Cranmer, a particle physicist who works on data science and machine learning at the University of Wisconsin-Madison, US, this nightmare scenario symbolizes the challenge with using reinforcement learning in physical sciences. In situations where you’re able to do many experiments very quickly and essentially for free – as is the case with AlphaGo and its descendants – you can expect reinforcement learning to work well, Cranmer explained. But once you’re interacting with a real, physical system, even non-destructive experiments require finite amounts of time and money.

Another challenge, Cranmer continued, is that particle physics already has good theories that predict some quantities to multiple decimal places. “It’s not low-hanging fruit for getting an AI to come up with a replacement framework de novo,” Cranmer said. A better option, he suggested, might be to put AI to work on modelling atmospheric fluid dynamics, which are emergent phenomena without first-principles descriptions. “Those are super-exciting places to use ideas from machine learning,” he said.

Not for nuclear arsenals

Silver, who was also on Tuesday’s panel, agreed that reinforcement learning isn’t always the right solution. “We should do this in areas where mistakes are small and it can learn from those small mistakes to avoid making big mistakes,” he said. To general laughter, he added that he would not recommend “letting an AI loose on nuclear arsenals”, either.

Reinforcement learning aside, both Åarrestad and Cranmer are highly enthusiastic about AI. For Cranmer, one of the most exciting aspects of the technology is the way it gets scientists from different disciplines talking to each other. The HLF, which aims to connect early-career researchers with senior figures in mathematics and computer science, is itself a good example, with many talks in the weeklong schedule devoted to AI in one form or another.

For Åarrestad, though, AI’s most exciting possibility relates to physics itself. Because the LHC produces far more data than humans and present-day algorithms can handle, Åarrestad explained, much of it is currently discarded. The idea that, as a result, she and her colleagues could be throwing away major discoveries sometimes keeps her up at night. “Is there new physics below 1 TeV?” Åarrestad wondered.

Someday, maybe, an AI might be able to tell us.

The post The pros and cons of reinforcement learning in physical science appeared first on Physics World.

When researchers at Microsoft released a list of the 40 jobs most likely to be affected by generative artificial intelligence (gen AI), few outsiders would have expected to see “mathematician” among them. Yet according to speakers at this year’s Heidelberg Laureate Forum (HLF), which connects early-career researchers with distinguished figures in mathematics and computer science, computers are already taking over many tasks formerly performed by human mathematicians – and the humans have mixed feelings about it.

One of those expressing disquiet is Yang-Hui He, a mathematical physicist at the London Institute for Mathematical Sciences. In general, He is extremely keen on AI. He’s written a textbook about the use of AI in mathematics, and he told the audience at an HLF panel discussion that he’s been peddling machine-learning techniques to his mathematical physics colleagues since 2017.

More recently, though, He has developed concerns about gen AI specifically. “It is doing mathematics so well without any understanding of mathematics,” he said, a note of wonder creeping into his voice. Then, more plaintively, he added, “Where is our place?”

AI advantages

Some of the things that make today’s gen AI so good at mathematics are the same as the ones that made Google’s DeepMind so good at the game of Go. As the theoretical computer scientist Sanjeev Arora pointed out in his HLF talk, “The reason it’s better than humans is that it’s basically tireless.” Put another way, if the 20th-century mathematician Alfréd Rényi once described his colleagues as “machines for turning coffee into theorems”, one advantage of 21st-century AI is that it does away with the coffee.

Arora, however, sees even greater benefits. In his view, AI’s ability to use feedback to improve its own performance – a technique known as reinforcement learning – is particularly well-suited to mathematics.

In the standard version of reinforcement learning, Arora explains, the AI model is given a large bank of questions, asked to generate many solutions and told to use the most correct ones (as labelled by humans) to refine its model. But because mathematics is so formalized, with answers that are so verifiably true or false, Arora thinks it will soon be possible to replace human correctness checkers with AI “proof assistants”. Indeed, he’s developing one such assistant himself, called Lean, with his colleagues at Princeton University in the US.

Humans in the loop?

But why stop there? Why not use AI to generate mathematical questions as well as producing and checking their solutions? Indeed, why not get it to write a paper, peer review it and publish it for its fellow AI mathematicians – which are, presumably, busy combing the literature for information to help them define new questions?

Arora clearly thinks that’s where things are heading, and many of his colleagues seem to agree, at least in part. His fellow HLF panellist Javier Gómez-Serrano, a mathematician at Brown University in the US, noted that AI is already generating results in a day or two that would previously have taken a human mathematician months. “Progress has been quite quick,” he said.

The panel’s final member, Maia Fraser of the University of Ottawa, Canada, likewise paid tribute to the “incredible things that are possible with AI now”.  But Fraser, who works on mathematical problems related to neuroscience, also sounded a note of caution. “My concern is the speed of the changes,” she told the HLF audience.

The risk, Fraser continued, is that some of these changes may end up happening by default, without first considering whether humans want or need them. While we can’t un-invent AI, “we do have agency” over what we want, she said.

So, do we want a world in which AI mathematicians take humans “out of the loop” entirely? For He, the benefits may outweigh the disadvantages. “I really want to see a proof of the Riemann hypothesis,” he said,  to ripples of laughter. If that means that human mathematicians “become priests to oracles”, He added, so be it.

The post Are we heading for a future of superintelligent AI mathematicians? appeared first on Physics World.

This episode of the Physics World Weekly podcast features Scott Bolton, who is principal investigator on NASA’s Juno mission to Jupiter. Launched in 2011, the mission has delivered important insights into the nature of the gas-giant planet. In this conversation with Physics World’s Margaret Harris, Bolton explains how Juno continues to change our understanding of Jupiter and other gas giants.

Bolton and Harris chat about the mission’s JunoCam, which has produced some gorgeous images of Jupiter and it moons.

Although the Juno mission was expected to last only a few years, the spacecraft is still going strong despite operating in Jupiter’s intense radiation belts. Bolton explains how the Juno team has rejuvenated radiation-damaged components, which has provided important insights for those designing future missions to space.

However Juno’s future is uncertain. Despite its great success, the mission is currently scheduled to end at the end of September, which is something that Bolton also addresses in the conversation.

The post Juno: the spacecraft that is revolutionizing our understanding of Jupiter appeared first on Physics World.

William Phillips is a pioneer in the world of quantum physics. After graduating from Juniata College in Pennsylvania in 1970, he did a PhD with Dan Kleppner at the Massachusetts Institute of Technology (MIT), where he measured the magnetic moment of the proton in water. In 1978 Phillips joined the National Bureau of Standards in Gaithersburg, Maryland, now known as the National Institute of Standards and Technology (NIST), where he is still based.

Phillips shared the 1997 Nobel Prize for Physics with Steven Chu and Claude Cohen-Tannoudji for their work on laser cooling. The technique uses light from precisely tuned laser beams to slow atoms down and cool them to just above absolute zero. As well as leading to more accurate atomic clocks, laser cooling proved vital for the creation of Bose–Einstein condensates – a form of matter where all constituent particles are in the same quantum state.

To mark the International Year of Quantum Science and Technology, Physics World online editor Margaret Harris sat down with Phillips in Gaithersburg to talk about his life and career in physics. The following is an edited extract of their conversation, which you can hear in full on the Physics World Weekly podcast.

How did you become interested in quantum physics?

As an undergraduate, I was invited by one of the professors at my college to participate in research he was doing on electron spin resonance. We were using the flipping of unpaired spins in a solid sample to investigate the structure and behaviour of a particular compound. Unlike a spinning top, electrons can spin only in two possible orientations, which is pretty weird and something I found really fascinating. So I was part of the quantum adventure even as an undergraduate.

What did you do after graduating?

I did a semester at Argonne National Laboratory outside Chicago, working on electron spin resonance with two physicists from Argentina. Then I was invited by Dan Kleppner – an amazing physicist – to do a PhD with him at the Massachusetts Institute of Technology. He really taught me how to think like a physicist. It was in his lab that I first encountered tuneable lasers, another wonderful tool for using the quantum properties of matter to explore what’s going on at the atomic level.

Quantum mechanics is often viewed as being weird, counter-intuitive and strange. Is that also how you felt?

I’m the kind of person entranced by everything in the natural world. But even in graduate school, I don’t think I understood just how strange entanglement is. If two particles are entangled in a particular way, and you measure one to be spin “up”, say, then the other particle will necessarily be spin “down” – even though there’s no connection between them. Not even a signal travelling at the speed of light could get from one particle to the other to tell it, “You’d better be ‘down’ because the first one was measured to be ‘up’.” As a graduate student I didn’t understand how deliciously weird nature is because of quantum mechanics.

Is entanglement the most challenging concept in quantum mechanics?

It’s not that hard to understand entanglement in a formal sense. But it’s hard to get your mind wrapped around it because it’s so weird and distinct from the kinds of things that we experience on a day-to-day basis. The thing that it violates – local realism – seems so reasonable. But experiments done first by John Clauser and then Alain Aspect and Anton Zeilinger, who shared the Nobel Prize for Physics in 2022, basically proved that it happens.

What quantum principle has had the biggest impact on your work?

Superposition has enabled the creation of atomic clocks of incredible precision. When I first came to NIST in 1978, when it was still called the National Bureau of Standards, the very best clock in the world was in our labs in Boulder, Colorado. It was good to one part in 10¹³.

Because of Einstein’s general relativity, clocks run slower if they’re deeper in a gravitational potential. The effect isn’t big: Boulder is about 1.5 km above sea level and a clock there would run faster than a sea level clock by about 1.5 parts in 10¹³. So if you had two such clocks – one at sea level and one in Boulder – you’d barely be able to resolve the difference. Now, at least in part because of the laser cooling and trapping ideas that my group and I have worked on, one can resolve a difference of less than 1 mm with the clocks that exist today. I just find that so amazing.

What research are you and your colleagues at NIST currently involved in?

Our laboratory has been a generator of ideas and techniques that could be used by people who make atomic clocks. Jun Ye, for example, is making clocks from atoms trapped in a so-called optical lattice of overlapping laser beams that are better than one part in 10¹⁸ – two orders of magnitude better than the caesium clocks that define the second. These newer types of clocks could help us to redefine the second.

We’re also working on quantum information. Ordinary digital information is stored and processed using bits that represent 0 or 1. But the beauty of qubits is that they can be in a superposition state, which is both 0 and 1. It might sound like a disaster because one of the great strengths of binary information is there’s no uncertainty; it’s one thing or another. But putting quantum bits into superpositions means you can do a problem in a lot fewer operations than using a classical device.

In 1994, for example, Peter Shor devised an algorithm that can factor numbers quantum mechanically much faster, or using far fewer operations, than with an ordinary classical computer. Factoring is a “hard problem”, meaning that the number of operations to solve it grows exponentially with the size of the number. But if you do it quantum mechanically, it doesn’t grow exponentially – it becomes an “easy” problem, which I find absolutely amazing. Changing the hardware on which you do the calculation changes the complexity class of a problem.

How might that change be useful in practical terms?

Shor’s algorithm is important because of public key encryption, which we use whenever we buy something online with a credit card. A company sends your computer a big integer number that they’ve generated by multiplying two smaller numbers together. That number is used to encrypt your credit card number. Somebody trying to intercept the transmission can’t get any useful information because it would take centuries to factor this big number. But if an evildoer had a quantum computer, they could factor the number, figure out your credit card and use it to buy TVs or whatever evildoers buy.

Now, we don’t have quantum computers that can do this yet – they can’t even do simple problems, let alone factor big numbers. But if somebody did do that, they could decrypt messages that do matter, such as diplomatic or military secrets. Fortunately, quantum mechanics comes to the rescue through something called the no-cloning theorem. These quantum forms of encryption prevent an eavesdropper from intercepting a message, duplicating it and using it – it’s not allowed by the laws of physics.

Quantum processors can be made from different qubits – not just cold atoms but trapped ions, superconducting circuits and others, too. Which do you think will turn out best?

My attitude is that it’s too early to settle on one particular platform. It may well be that the final quantum computer is a hybrid device, where computations are done on one platform and storage is done on another. Superconducting quantum computers are fast, but they can’t store information for long, whereas atoms and ions can store information for a really long time – they’re robust and isolated from the environment, but are slow at computing. So you might use the best features of different platforms in different parts of your quantum computer.

But what do I know? We’re a long way from having quantum computers that can do interesting problems faster than classical device. Sure, you might have heard somebody say they’ve used a quantum computer to solve a problem that would take a classical device a septillion years. But they’ve probably chosen a problem that was easy for a quantum computer and hard for a classical computer – and it was probably a problem nobody cares about.

When do you think we’ll see quantum computers solving practical problems?

People are definitely going to make money from factoring numbers and doing quantum chemistry. Learning how molecules behave could make a big difference to our lives. But none of this has happened yet, and we may still be pretty far away from it. In fact, I have proposed a bet with my colleague Carl Williams, who says that by 2045 we will have a quantum computer that can factor numbers that a classical computer of that time cannot. My view is we won’t. I expect to be dead by then. But I hope the bet will encourage people to solve the problems to make this work, like error correction. We’ll also put up money to fund a scholarship or a prize.

What do you think quantum computers will be most useful for in the nearer term?

What I want is a quantum computer that can tackle problems such as magnetism. Let’s say you have a 1D chain of atoms with spins that can point up or down. Quantum magnetism is a hard problem because with n spins there are 2ⁿ possible states and calculating the overall magnetism of a chain of more than a few tens of spins is impossible for a brute-force classical computer. But a quantum computer could do the job.

There are quantum computers that already have lots of qubits but you’re not going to get a reliable answer from them. For that you have to do error correction by assembling physical qubits into what’s known as a logical qubit.  They let you determine whether an error has happened and fix it, which is what people are just starting to do. It’s just so exciting right now.

What development in quantum physics should we most look out for?

The two main challenges are: how many logical qubits we can entangle with each other; and for how long they can maintain their coherence. I often say we need an “immortal” qubit, one that isn’t killed by the environment and lasts long enough to be used to do an interesting calculation. That’ll determine if you really have a competent quantum computer.

Reflecting on your career so far, what are you most proud of?

Back in around 1988, we were just fooling around in the lab trying to see if laser cooling was working the way it was supposed to. First indications were: everything’s great. But then we discovered that the temperature to which you could laser cool atoms was lower than everybody said was possible based on the theory at that time. This is called sub-Doppler laser cooling, and it was an accidental discovery; we weren’t looking for it.

People got excited and our friends in Paris at the École Normale came up with explanations for what was going on. Steve Chu, who was at that point at Stanford University, was also working on understanding the theory behind it, and that really changed things in an important way. In fact, all of today’s laser-cooled caesium atomic clocks use that feature that the temperature is lower than the original theory of laser cooling said it was.

Another thing that has been particularly important is Bose–Einstein condensation, which is an amazing process that happens because of a purely quantum-mechanical feature that makes atoms of the same kind fundamentally indistinguishable. It goes back to the work of Satyendra Nath Bose, who 100 years ago came up with the idea that photons are indistinguishable and therefore that the statistical mechanics of photons would be different from the usual statistical mechanics of Boltzmann or Maxwell.

Bose–Einstein condensates, where almost all the atoms are in the same quantum state, were facilitated by our discovery that the temperature could be so much lower. To get this state, you’ve got to cool the atoms to a very low temperature – and it helps if the atoms are colder to start with.

Did you make any other accidental discoveries?

We also accidentally discovered optical lattices. In 1968 a Russian physicist named Vladilen Letokhov came up with the idea of trapping atoms in a standing wave of light. This was 10 years before laser cooling arrived and made it possible to do such a thing, but it was a great idea because the atoms are trapped over such a small distance that a phenomenon called Dicke narrowing gets rid of the Doppler shift.

Everybody knew this was a possibility, but we weren’t looking for it. We were trying to measure the temperature of the atoms in the laser-cooling configuration, and the idea we came up with was to look at the Doppler shift of the scattered light. Light comes in, and if it bounces off an atom that’s moving, there’ll be a Doppler shift, and we can measure that Doppler shift and see the distribution of velocities.

So we did that, and the velocity distribution just floored us. It was so odd. Instead of being nice and smooth, there was a big sharp peak right in the middle. We didn’t know what it was. We thought briefly that we might have accidentally made a Bose–Einstein condensate, but then we realized, no, we’re trapping the atoms in an optical lattice so the Doppler shift goes away.

It wasn’t nearly as astounding as sub-Doppler laser cooling because it was expected, but it was certainly interesting, and it is now used for a number of applications, including the next generation of atomic clocks.

How important is serendipity in research?

Learning about things accidentally has been a recurring theme in our laboratory. In fact, I think it’s an important thing for people to understand about the way that science is done. Often, science is done not because people are working towards a particular goal but because they’re fooling around and see something unexpected. If all of our science activity is directed toward specific goals, we’ll miss a lot of really important stuff that allows us to get to those goals. Without this kind of curiosity-driven research, we won’t get where we need to go.

In a nutshell, what does quantum meant to you?

Quantum mechanics was the most important discovery of 20th-century physics. Wave–particle duality, which a lot of people would say was the “ordinary” part of quantum mechanics, has led to a technological revolution that has transformed our daily lives. We all walk around with mobile phones that wouldn’t exist were it not for quantum mechanics. So for me, quantum mechanics is this idea that waves are particles and particles are waves.

The post William Phillips: why quantum physics is so ‘deliciously weird’ appeared first on Physics World.