Will machines acquire knowledge as naturally as children?
Tim Ensor, director of artificial intelligence at Cambridge Consultants, part of Capgemini Invent, wonders if AI-powered machines will learn as naturally as children.
It may seem like a long way off now, but research has shown it could be possible.
Watching a child learn is an amazing experience. As a proud dad, this delights and inspires me, and as an artificial intelligence (AI) professional, it reminds me that our journey to machine learning (ML) has only just begun. What’s especially amazing about babies and toddlers, of course, is that they learn incredibly fast – relying on blocks of information and amazing us as they learn things naturally and incrementally. Is it too much to ask of machines?
For now, the answer is yes. But the extraordinary progress I see in ML convinces me that the ultimate goal of meta-learning – the ability of machines to learn how to learn – is getting closer and closer. The implications are profound, of course. Business opportunities for companies will be propelled to new levels, society will evolve, and ethical, philosophical and moral issues will be on the agenda, in a world where AI and human behavior mirror each other much more closely.
But what is the current state of play in machine learning, and what exciting recent developments are to come that could bring us closer to the notion of “learning to learn”?
Where are we now
For the moment, let me stick to my analogy with the human child. We create never-before-seen machines, with sophisticated specifications, that perform extremely well. But to reach their potential, we need to expose them to hundreds of thousands of training examples for each task. They just don’t understand things like humans.
One way to get machines to learn more naturally is to help them learn from limited data. We can use Generative Adversarial Networks (GANs) to create new examples from a small core of training data rather than having to capture every situation in the real world. It is “adversarial” because one neural network competes with another to generate new synthetic data. Then there’s synthetic data rendering – using game engines or computer graphics to render new scenarios. Finally, there are algorithmic techniques such as Domain Adaption which uses transferable knowledge (using data in summer that you collected in winter for example) or Few Shot Learning which makes predictions from a limited number of samples.
Multi-task learning is about taking another path of limited data, where commonalities and differences are exploited to solve multiple tasks simultaneously. ML is generally supervised (with input and target pairing labels), but advances are being made in unsupervised, semi-supervised, and self-supervised learning. It’s about learning without a human teacher having to label all the examples. With clustering, for example, an algorithm can group items into groups with similarities that may or may not be identified and labeled by a human. Examining the clusters will reveal the thinking of the system.
Human-Machine Understanding: How Technology Helps Us Be More Human
Sally Epstein, Head of Strategic Technology at Cambridge Consultants, explains how human-machine understanding can help us be more human. Read here
But now to the Transformer, one of the new neighborhood kids. Most neural network algorithms need to be tailored to perform a single task. The Transformer architecture makes fewer assumptions about the format of input and output data and can therefore be applied to different tasks – similar to the idea of machines exploiting the building blocks of learning. The transformer initially used self-attention mechanisms as a basic element of machine translation in natural language processing. But now it’s being applied to other tasks, like image recognition and understanding 3D point clouds.
This brings me to the obvious question: what’s the next step for the Transformer? Recent academic work has sought to apply it alongside data-efficient training techniques for protein applications. The Cambridge Consultants team built on this research to create an AI model capable of optimizing protein function for a specific task. We have applied this to fluorescent proteins, particularly if it is possible to recommend protein structures that fluoresce brighter. I don’t have time to go into detail here, but I can say that the results are very encouraging: the model predicted variants with six amino acid changes across the length of the sequence to improve fluorescence protein.
This is just a glimpse of an exciting future. Protein manipulation has the potential to be applied to a range of applications, including medicine, where it could be applied to improving cancer treatments or reducing organ rejection rates. New, more effective antibiotics could also be created using protein manipulation. In the materials space, there could be a role in disposing of plastic waste more efficiently. The technique could also be used to create better performing textiles.
Looping into the future
And what about the process of training AI models in an experiment loop? Essentially, this disrupts the traditional data-driven approach. Instead of saying “here’s the data, what will this solve?”, the idea is to start with the problem and then create the datasets you need. You ask the AI what it would like to know, then run a lab experiment to find information that you feed back into the neural network. Any gaps in knowledge begin to be filled. It’s in a fairly early stage of development, with the goal of closing the loop and automating the whole experimentation process.
As part of the recent AI Summit London 2021, I had a fascinating fireside chat on the subject with Kim Bransonsenior vice president, global head of artificial intelligence and machine learning at GSK. He and his team are applying this concept of experimentation to drug discovery. Their approach – “let’s ask the question first and then get the data we need to answer it” – allows them to build unique sets of data to target the problems they are trying to solve. This is powerful, and indicative of the point I made at the start: the more efficient machines become at learning, the better the outcomes for business, society, and the world. Watch this place.