AI pioneer Jürgen Schmidhuber reveals plan to build a general AI | Industry
Breaking Tech Industry news from the top sources
Artificial general intelligence represents a holy grail for many computer scientists. The concept refers to superintelligent systems that can independently learn to perform any cognitive task that a human can do. It could unleash an unfathomable future known as the singularity where technology advances so rapidly that it transforms civilisation beyond our imagination.
Dr Jürgen Schmidhuber has wanted to build an artificial general intelligence (AGI) since he was a science fiction-loving teenager in his native Bavaria, Germany. His ambitions led him to enroll as a mathematics and computer science student at Munich’s Technical University and then drove him south to Lugano, Switzerland to pursue the goal professionally.
He currently juggles jobs as professor of AI at the University of Italian Switzerland, as the scientific director at the Swiss AI Lab IDSIA (the Dalle Molle Institute for Artificial Intelligence Research), and as the cofounder and chief scientist of NNAISENSE – pronounced “nascence” as it aims to give birth to a general purpose AI.
A founding father of AI to his fans and its enfant terrible to his critics, Schmidhuber’s reputation precedes him, for better and for worse.
His most celebrated contribution to the field is Long Short-Term Memory (LSTM) recurrent neural networks, which learn from experience to translate languages, control robots, predict diseases and compose music.
“General-purpose intelligence goes beyond what you can do with supervised learning, through LSTM,” Schmidhuber tells Techworld in a sunny courtyard of east London’s Tobacco Dock. “The current commercial focus is on supervised learning – for example, you train the LSTM to do speech recognition, or translation from one language to another, by showing it lots of human-created examples.
“But the most exciting thing is what babies do. They don’t have a teacher who tells them, ‘and now, move your left and middle toe just like this, to walk a little bit better’. They figure it out by themselves through increasingly clever trial and error, and quickly learn how to acquire new information that they need to become better problem-solvers.
“And this is something that goes far beyond supervised learning; it’s active unsupervised learning and curiosity. We know in principle how to implement artificial curiosity and creativity – I have been publishing on this since 1990 – but it’s not as commercial yet as supervised LSTM.”
Deep learning in outer space
LSTM didn’t immediately achieve commercial success either. In 1997, a paper that Schmidhuber and his student Sepp Hochreiter published on the concept was rejected by MIT and it was years before the method gained mass adoption.
Things quickly changed when developments in computing and the resurgence of deep learning brought growing interest to the method. LSTM is now used by five of the most valuable public companies in the world and in almost every smartphone.
Facebook uses it to process 4.5 billion automatic translations every day, Apple to improve Siri’s word recommendations, Amazon to convert letters into sounds in Alexa, Google to create automatic email response suggestions, and Microsoft to create a speech recognition system that achieved human parity.
His legacy in the field of AI would seem assured, but Schmidhuber is almost as well-known for interrupting the talks of his colleagues to claim even further credit than what he currently receives.
This reputation divides the AI community but makes him a popular figure on the tech conference circuit, which is what brought him to London today. He just left the CogX festival stage with a promise to the audience: in the coming decades, machines will develop a level of intelligence that animals needed 3.5 million years of evolution to achieve. AGI will lead the entire cosmos to become intelligent, a transcendental moment that Schmidhuber believes could spell the end for human dominion over planet earth.
“We are now like stepping stones on the way of the cosmos towards higher complexity,” he says. “It looks like soon we’ll build true AIs that set themselves their own goals – like they already do in my lab – learning to become much more general problem solvers than humans.”
Schmidhuber has cultivated an otherworldly image to match his space-age theories. Today he’s matched a black Nehru suit and flat cap with a grey goatee. He unfolds his long legs and leans closer to whisper.
“What will they do? Most will emigrate into space, because that’s where almost all of the physical resources are, and within a few tens of billions of years colonise and transform not only the solar system and the milky way but the entire universe in a way infeasible for humans. So humans are not going to remain the crown of creation. But at the moment we are still very important, because we still can mess it all up, by destroying civilisation through a nuclear war or something.”
Back on planet earth
Schmidhuber may envision AI moving to outer space but his current focus is more down to earth. In the NNAISENCE laboratory in the foothills of the Swiss Alps, Schmidhuber’s team is now searching for the next breakthrough in unsupervised learning.
Their recent projects include using AI to optimise the choice of stocks picked for portfolios by a German investment firm called Acatis, and working with Audi to develop the first autonomous vehicles that can learn to park without a teacher.
“There’s no human showing them how it works, and like babies bumping against walls or other cars, just by trying to minimise pain and maximise pleasure through finding a parking spot, they figured out how to park,” says Schmidhuber.
“The focus of our company is on stuff that really nobody knows how to do. The teacher also doesn’t know it. And how can the system learn by itself to do it, and then how can it learn one thing, and then another thing, and then 10 things, and then 100 things, without forgetting the previously learned skills. That’s where we have made a lot of progress.”
They showed how far they had come at the 2017 Neural Information Processing Systems (NIPS) conference in Long Beach, California, during a competition to teach a simulated human body how to run across an obstacle course as quickly as possible. The Swiss company beat a global field of more than 400 competitors from industry and academia to win the contest.
Schmidhuber takes pride in continuing Europe’s legacy of scientific breakthroughs in AI, but adds that the continent has failed to commercialise its research as well as companies in the Pacific Rim.
He argues that stronger industrial policies would help European businesses to compete, pointing to the benefits that companies in the US and China have reaped from their government’s support in subsidies for the likes of Tesla and interventions to prevent foreign rivals such as Tencent from entering their markets.
“Both China and the US are heavily using industrial policy to achieve their goals,” he says. “Only 10 years ago, in 2008, the EU economy was still as big as China’s and America’s taken together. How things have changed within only 10 years!
“Nevertheless, of all three powers, Europe is still by far the biggest exporter. To maintain this export prowess, it also has to be open for imports, and allow the others to come in – for example, Google and Facebook now dominate advertising in Europe – and buy European companies.
“The Chinese went into Germany and bought the world’s third-largest robot maker, Kuka, and some people say they overpaid. But, from their perspective, they didn’t, because they are not only buying the company, they can also replicate the entire structure of the small companies around Kuka, and understand how these little companies that are making parts assembled by Kuka are organised. The idea is, of course, you replicate some of this in China and scale that by a factor of 100 or something. From that perspective, 10 billion [yuan] is nothing.”
European startups also struggle to scale like their rivals in the Pacific Rim. Europe still has more billionaires than any other continent, but investors have less appetite for risk and big businesses rarely spend their savings on acquisitions. This investment landscape led Google to buy DeepMind, meaning IP developed in Europe would be moved to the parent company in the USA.
Schmidhuber believes that this is an area that requires a cultural change beyond the reach of governments and that promoting Europe’s rich history of scientific discoveries would help conservative investors identify with startups and believe in their potential.
He retains hope that this could quickly change in Europe. The continent may have lost DeepMind’s IP to the US, but the company continues to attract talent and attention to the UK and inspire the country’s startups that they could replicate its success. This impact will be boosted by recent national investment strategies announced by the government.
“The current UK plans for AI are encouraging – perhaps small money compared to what the Chinese are spending, but it’s substantial in the European scene, so it’s good,” says Schmidhuber. “And maybe much bigger things will come out of that. I’m trying to convince European governments to just learn a little bit from China here.”
Global threats to the universe
The global battle for AI supremacy is often depicted as an existential threat. Elon Musk believes it is the most likely reason for a third world war, which may not be initiated by a country’s leader but by one of their AIs, if the system decides that a preemptive strike is the most probable path to victory.
Schmidhuber has tried to convince Musk that his fears are overblown. He feels that contemporary conflicts between humans pose a far greater threat to civilisation than the AIs of the future as they will not share our goals and interests.
“Those who share goals are interested in each other,” he says. “Those who don’t share goals are not interested in each other. That’s the reason why you are mostly interested in other humans.
“It’s even more specific. You are mostly interested in other humans quite like yourself, like a CEO is going to be mostly interested in other CEOs of similar companies in the same area, competing for the same market. And 10-year-old girls are mostly interested in other 10-year-old girls, kangaroos mostly in other kangaroos.
“Super smart AIs of the future will be mostly interested in other super smart AIs of the future, and not so much in kangaroos and humans. Sure, as artificial scientists, in the beginning those curious AIs will be fascinated by life and their own origins in our civilization. But once they fully understand that, they will be mostly interested in others like themselves. So, in the long run you can expect some protection through lack of interest on the other side.”
His optimism has led him to embrace the impact of an artificial general intelligence. He has attempted to calculate when it will arrive by analysing a numerical pattern in the history of the universe known as the Omega point.
The theory was developed in the early 20th century by a Jesuit priest called Teilhard de Chardin. He argued that humanity was evolving towards a final state at which it would converge with the supernatural order. He dubbed this evolutionary destination the Omega Point. Some contemporary computer scientists believe that reaching the Omega Point will herald the start of the singularity.
The pattern begins around 13.8 billion years ago with the Big Bang. Divide this number by four and we get to 3.45 billion years ago, around the time that life began on earth. Each resulting figure is in turn divided by four and each result emerges at a key event in human history. They eventually reach zero: the Omega point. The calculation suggests that this will be in around 2050.
“It doesn’t really matter whether true AI will first appear in 10 years or one hundred years or one thousand years, because one thousand years is still less than ten percent of human civilisation,” says Schmidhuber.
“And civilization history is just 13,000 years or so, one-millionth of world history, and so by cosmic standards, it’s going to be really, really soon. But I still hope to live to see it in person – my life has been about making this possible, then retire.”