‘The entire protein universe’: AI predicts shape of nearly every known protein
GM, Database at SolarWinds.
One of the most critical components in machine learning projects is the quality of an organization’s database management system. And as artificial intelligence (AI) continues to grow more complex, access to adequate data is an increasingly important component of a company's success.
For deep learning, forward-thinking companies must choose to upgrade to more robust and efficient databases.
As reported by the World Economic Forum, the “deep” in deep learning refers to the depth of layers in a neural network. A neural network consisting of more than three layers—which would be inclusive of the inputs and the output—can be considered a deep learning algorithm.
Deep learning has the potential to solve big real-world problems, from curing diseases (subscription required) to image analysis to delivering effective cyber defense to ending traffic deaths. But in order to reach this potential, databases must evolve to meet the needs of more advanced AI algorithms.
In general terms, deep learning is a type of machine learning designed to imitate the way humans gain certain types of knowledge. And while machines are capable of processing massive amounts of data at a rate far exceeding that of the human brain—which allows them to help Excellerate productivity, increase retention and drive revenue—sound oversight structures are needed to ensure positive results.
The evolution of next-generation AI to deep learning will require optimized, powerful databases with unlimited throughput, scalable processing power and zero latency. By integrating AI with these more optimized databases, algorithms can be used to train machine learning models, which can run other algorithms.
In addition, increasingly powerful databases can help bridge the gap between current AI models and more advanced and evolving deep learning capacity.
Companies in almost every industry are discovering new opportunities through the connection between AI and machine learning, including retail, banking, healthcare and the hospitality industry. New possibilities are emerging constantly.
With augmented systems, businesses can quickly sort a large amount of data, and leaders can gain meaningful insights from it. Indeed, there’s no avoiding the necessity of upgrading to meet growing database demand and AI infrastructure—one exact study projects the global deep learning market will grow to $526 billion by 2030.
Deep learning has recently become much more popular because of its success in many complex data-driven applications. The database community has worked on data-driven applications for many years and should continue to play a lead role in supporting this new wave.
The most effective solution for modern enterprises seeking to build deep learning solutions is to ensure their strategies begin with addressing database performance and efficiency.
Fully optimized databases are the only way to enable the deep learning applications of tomorrow—applications capable of instantly accessing and understanding data to reach conclusions and make recommendations without human intervention.
Data is increasingly becoming an organization’s—and the world’s—most important asset. In order to unlock the promise of deep learning and solve some of the world’s biggest problems, from energy production to a cure for cancer, we need to start with stronger and more efficient databases.
Forbes Business Development Council is an invitation-only community for sales and biz dev executives. Do I qualify?
The terms machine learning and deep learning can seem interchangeable to most people, but they aren’t. Both considered subdivisions within the world of artificial intelligence (AI), the two have many differences, especially in the architecture and use cases.
Machine learning, for instance, uses structured data and algorithms to train models, with the more data at disposal generally equating with more accurate and better trained models. The idea is to eliminate the need for human intervention. Deep learning, on the other hand, is a subset of machine learning and uses neural networks to imitate the way humans think, meaning the systems designed require even less human intervention.
Differentiating the two, in this way, is crucial to AI research and practical application of both, particularly as businesses attempt to integrate such technologies into their core processes, and recruit for skilled individuals to fill technical roles.
If you have ever communicated with a chatbot, used predictive text, or watched a show Netflix has recommended for you, then chances are you have used something built on machine learning. Machine learning, which itself is a subset of AI, is a general term used to describe machines learning from data.
It uses structured data such as numbers, text, images, financial transactions, and so on, as well as algorithms, to replicate the ways humans learn how to do things. Data is gathered and used as training data to guide the machine learning model. The more data that’s used, in theory, the better the model will work. In essence, machine learning is about letting computers learn to programme themselves through use of training datasets and occasional human intervention.
There are different types of machine learning, such as supervised learning, unsupervised learning, semi-supervised learning and reinforcement learning.
Supervised learning involves data scientists giving labelled training data to algorithms to define variables they need for the algorithm to evaluate for connections. Unsupervised learning, on the other hand, involves unlabelled data being processed by algorithms that instinctively hunt for meaningful correlations. Falling between these two methodologies, semi-supervised learning is used to help the model’s own understanding of the dataset. Reinforcement learning, meanwhile, involves a machine completing a sequence of decisions to achieve a goal in an unknown, complex environment.
Autonomous data management
Requirements, use cases and guidelinesFree Download
Deep learning is a subset of machine learning that uses layers of artificial neural networks to imitate how humans think and learn.
Up until now, it’s been difficult for neural networks to learn much due to the lack of computing power available. This has changed greatly by improvements in big data analytics that allow for larger, sophisticated neural networks that permit machines to perceive, learn, and respond to complex scenarios quicker than humans.
These layers – “deep” neural networks – are constructed to enable data to be transmitted from node to node, like neurons, in a highly connected manner. While huge amounts of data are needed to feed and build such models, they can provide us immediate results with relatively little intervention once the model is in place. There are many ways to perform deep learning.
Convolutional Neural Networks (CNNs): These comprise multiple layers and are mostly used for image processing and object detection.
Recurrent Neural Networks (RNNs): These are types of artificial neural network that use sequential data or time series data. They are frequently used in problems, such as language translation, natural language processing (NLP), speech recognition, and image captioning.
Long Short-Term Memory Networks (LSTMs): These are types of Recurrent Neural Network (RNN) that can learn and remember long-term dependencies. They can be useful for complex problem domains like machine translation, speech recognition, and more.
Generative Adversarial Networks (GANs): These are generative deep learning algorithms that produce new data instances that look like the training data. It comprises two parts; a generator, which learns to generate false data, and a discriminator, which learns from that fake information. These networks have been used to produce fake images of people who have never existed as well as new and unique music.
Radial Basis Function Networks (RBFNs): These networks have an input layer, a hidden layer, and an output layer and are typically used for classification, regression, and time-series predictions.
Multilayer Perceptrons (MLPs): These are a type of feedforward (this means information moves only forward in the network) neural networks. These have an input layer and an output layer that are fully connected. There may also be hidden layers. These are used in speech-recognition, image-recognition, and machine-translation software.
Deep Belief Networks (DBNs): This looks like another feedforward neural network with hidden layers, but isn’t. These are a sequence of restricted boltzmann machines which are sequentially connected. These are used to identify, gather and generate images, video sequences and motion-capture data.
While the two terms often get confused with each other, deep learning is a subset of machine learning. However, deep learning differentiates itself from machine learning by the data types it works with and the methods by which it learns.
Machine learning uses structured, labelled data to predict outcomes. This means a machine learning model’s input data defines specific features and is organised into tables. While it gets progressively better at carrying out the task in hand, there still requires there to be a human to intervene at points to ensure the model is working in the required way. In other words, if the predictions are not accurate, an engineer will make any adjustments needed to get back on track.
A five step blueprint for master data management success
How to create a strategic plan for deploying your MDM initiativeFree Download
On the other hand, deep learning models have algorithms that can figure out if their predictions are accurate using its neural network without human involvement.
Another difference is that where machine learning can use small amounts of data to make predictions, deep learning needs much, much more data to make more accurate predictions.
While machine learning needs little time to train – typically a few seconds to a few hours – deep learning takes far longer as the algorithms used here involve many layers.
Outputs also differ between the two. Machine learning tends to output numerical values, such as a score or classification, while deep learning can output in multiple formats, such as text, scores, or even sounds and images.
Machine learning is already in use in a variety of areas that are considered part of day-to-day life, including on social media, on email platforms and, as mentioned, on streaming services like Netflix. These types of applications lend themselves well to machine learning, because they’re relatively simple and don’t require vast amounts of computational power to process complicated decision-making.
Among some of the more complex uses of machine learning are computer vision, such as facial recognition, where technology can be used to recognise people in crowded areas. Handwriting recognition, too, can be used to identify an individual from documents that are scanned en masse. This would apply, for example, to academic examinations, police records, and so on. Speech recognition, meanwhile, such as those used in voice assistants are another application of machine learning.
Deep learning also has its uses in image recognition, where massive amounts of data is ingested and used to help the model tag, index, and annotate images. Such models are currently in use for generating art, in systems like DALL·E. Similarly to machine learning, deep learning can be used in virtual assistants, in chat bots, and even in image colorisation. Deep learning has also had a particularly exciting impact in the field of medicine, such as in the development of personalised medicines created for somebody’s unique genome.
Cyber security in the retail sector
Retailers need to ensure their business operations and internal data aren't breachedFree Download
Three key steps to modernising legacy applications in the cloud
The challenges and ways to achieve application modernisation successFree Download
The time for cloud MDM is now
Know the differences between cloud-native and cloud-enabled MDMFree Download
Magic quadrant for data quality solutions
Amplifying analytics for better insights and for making trusted, data-driven decisionsFree Download
The role of social media in the context of communication, collaboration and network-building has been well understood. But understanding its potential impact on learning and incorporating it as part of the learning strategy is at a nascent stage. The opportunities that social networks present for learning and the advantages of integrating social networks with online learning are important to recognise while designing the learning plans for individuals and professionals in future.
With a plethora of information available on the internet, searching for the right information or ensuring you are not left behind through timely upgradation of knowledge have become cumbersome tasks. That is where professional learning networks could address these concerns and act as a dynamic feed for continuous learning and upgradation if conceived well. How are these professional learning networks different from the online learning systems set up by companies?
Online learning portals are mostly based on ‘push’ strategy whereas professional learning networks are designed around a ‘pull’ approach and are successful when integrated with social media networks. The starting point is to decide the extent of content curation on the portal. This is because of the humungous amount of content available on the internet and the consequent need for skillfully discerning what content is relevant and how to present it to the learner. Thus, the role of the curator is to artfully filter the information that would suit learners, tastefully present the content, understand the profiles of the learners and their styles of learning.
Once the building blocks of the learning network are defined, the dynamic components of the learning system must be carefully planned. Managing the network for the ongoing feed of content is a crucial success factor for learner adoption and usage of such networks. The types of content that need to be made available in the learning network calls for specialised skills in browsing the web regularly to understand what is ‘in’ and what is ‘out’, listening to peers and industry specialists to identify the best solution for aggregating the content.
The other task that has to be done diligently concerns who to ‘follow’ and who should ideally ‘follow’ the members of the network. Being able to expand the horizons of learning by systematically facilitating engagements with social networks is indeed a great boon for learners, but it comes with some important challenges as well.
Organisations beginning to conceptualise and design customised professional learning networks should also plan to tap into the multitudes of data available for carrying out the required analytics and making the learning process responsive. These are exciting times for training managers, who now have the opportunity to make learning more targeted, customised and real-time.
The writer is chairperson, Global Talent Track, a corporate training solutions company.
Also Read: The increasing importance of learning AI skills and the future of work
Follow us on Twitter, Facebook, LinkedIn
Wi-Fi issues on campus have disrupted students, faculty and staff since the beginning of the semester.
Information Services sent out several notices through email and Spiderbytes to students, faculty and staff about connectivity issues with the on-campus Wi-Fi network, urwin, writing that there had been no current identification of the problem and that it was working with the network vendor, Aruba, to devise a solution.
While there has still not been a root cause identified, Information Services is still looking for the cause in order to prevent similar issues in the future, said Greg Miller, director of enterprise identity and access.
“We’ve heard, definitely from students, and also from staff and faculty that it was disruptive for people, and it was enough people to grab our attention,” Miller said.
He and his team identified the issue on Aug. 23, and received a total of 37 incident reports about slow or no access to the internet. They have not received any new reports or been able to repeat the issue on the devices they have tested since Sept. 6, but announced in an email on Sept. 16, that they have continued receiving reports about the slow or failed connections. Miller and his team have identified macOS users as the primarily affected users.
Information Services recommends using the wired ethernet in the meantime, but some students say it has not fixed the issue. Connecting to the ethernet, restarting her computer, and disconnecting from urwin and reconnecting were some of the solutions that sophomore Lana Vjestica, an employee at the Technology Learning Center, tried, but none of them worked.
“I had trouble just getting onto Blackboard and Google Drive, just doing anything on my laptop,” Vjestica said. “I had been going on for hours, and I couldn’t get any homework done.”
Several people, both through the TLC and personal connections, have asked Vjestica or other employees at the TLC about what they can do to fix internet issues, but the only recommendation is to contact the Help Desk or log on to one of the school computers in the library or one of the academic buildings.
Not all students have been having issues. First-year Paxton Calder, said that she had not been having issues to the same degree that others she had talked to had. While her connection isn’t perfect, she usually only encounters a momentary hiccup in internet connection as she moves from one physical location to another, she said.
Similar Wi-Fi problems occurred during the middle of the pandemic, in spring 2021. A malfunctioning access point caused the internet to have connectivity issues, preventing students from joining online classes. It has been over a year since the last major incident regarding Wi-Fi issues.
If students and staff run into slow loading on websites, or an inability to connect the Wi-Fi while using campus resources such as Blackboard or Google applications, they are recommended to contact the help desk and file a report.
Contact news writer Grace Allen at firstname.lastname@example.org.
You can make a tax-deductible donation by clicking the button below, which takes you to our secure PayPal account. The page is set up to receive contributions in whatever amount you designate. We look forward to using the money we raise to further our mission of providing honest and accurate information to students, faculty, staff, alumni and others in the general public.Donate Now
Researchers at DeepMind in London have shown that artificial intelligence (AI) can find shortcuts in a fundamental type of mathematical calculation, by turning the problem into a game and then leveraging the machine-learning techniques that another of the company’s AIs used to beat human players in games such as Go and chess.
The AI discovered algorithms that break decades-old records for computational efficiency, and the team’s findings, published on 5 October in Nature1, could open up new paths to faster computing in some fields.
“It is very impressive,” says Martina Seidl, a computer scientist at Johannes Kepler University in Linz, Austria. “This work demonstrates the potential of using machine learning for solving hard mathematical problems.”
Advances in machine learning have allowed researchers to develop AIs that generate language, predict the shapes of proteins2 or detect hackers. Increasingly, scientists are turning the technology back on itself, using machine learning to Excellerate its own underlying algorithms.
‘The entire protein universe’: AI predicts shape of nearly every known protein
The AI that DeepMind developed — called AlphaTensor — was designed to perform a type of calculation called matrix multiplication. This involves multiplying numbers arranged in grids — or matrices — that might represent sets of pixels in images, air conditions in a weather model or the internal workings of an artificial neural network. To multiply two matrices together, the mathematician must multiply individual numbers and add them in specific ways to produce a new matrix. In 1969, mathematician Volker Strassen found a way to multiply a pair of 2 × 2 matrices using only seven multiplications3, rather than eight, prompting other researchers to search for more such tricks.
DeepMind’s approach uses a form of machine learning called reinforcement learning, in which an AI ‘agent’ (often a neural network) learns to interact with its environment to achieve a multistep goal, such as winning a board game. If it does well, the agent is reinforced — its internal parameters are updated to make future success more likely.
AlphaTensor also incorporates a game-playing method called tree search, in which the AI explores the outcomes of branching possibilities while planning its next action. In choosing which paths to prioritize during tree search, it asks a neural network to predict the most promising actions at each step. While the agent is still learning, it uses the outcomes of its games as feedback to hone the neural network, which further improves the tree search, providing more successes to learn from.
Each game is a one-player puzzle that starts with a 3D tensor — a grid of numbers — filled in correctly. AlphaTensor aims to get all the numbers to zero in the fewest steps, selecting from a collection of allowable moves. Each move represents a calculation that, when inverted, combines entries from the first two matrices to create an entry in the output matrix. The game is difficult, because at each step the agent might need to select from trillions of moves. “Formulating the space of algorithmic discovery is very intricate,” co-author Hussein Fawzi, a computer scientist at DeepMind, said at a press briefing, but “even harder is, how can we navigate in this space”.
To provide AlphaTensor a leg up during training, the researchers showed it some examples of successful games, so that it wouldn’t be starting from scratch. And because the order of actions doesn’t matter, when it found a successful series of moves, they also presented a reordering of those moves as an example for it to learn from.
The researchers tested the system on input matrices up to 5 × 5. In many cases, AlphaTensor rediscovered shortcuts that had been devised by Strassen and other mathematicians, but in others it broke new ground. When multiplying a 4 × 5 matrix by a 5 × 5 matrix, for example, the previous best algorithm required 80 individual multiplications. AlphaTensor uncovered an algorithm that needed only 76.
“It has got this amazing intuition by playing these games,” said Pushmeet Kohli, a computer scientist at DeepMind, during the press briefing. Fawzi tells Nature that “AlphaTensor embeds no human intuition about matrix multiplication”, so “the agent in some sense needs to build its own knowledge about the problem from scratch”.
DeepMind’s AI helps untangle the mathematics of knots
The researchers tackled larger matrix multiplications by creating a meta-algorithm that first breaks problems down into smaller ones. When crossing an 11 × 12 and a 12 × 12 matrix, their method reduced the number of required multiplications from 1,022 to 990.
AlphaTensor can also optimize matrix multiplication for specific hardware. The team trained the agent on two different processors, reinforcing it not only when it took fewer actions but also when it reduced runtime. In many cases, the AI sped up matrix multiplications by several per cent compared with previous algorithms. And sometimes the fastest algorithms on one processor were not the fastest on the other.
The same general approach could have applications in other kinds of mathematical operation, the researchers say, such as decomposing complex waves or other mathematical objects into simpler ones. “This development would be very exciting if it can be used in practice,” says Virginia Vassilevska Williams, a computer scientist at Massachusetts Institute of Technology in Cambridge. “A boost in performance would Excellerate a lot of applications.”
Grey Ballard, a computer scientist at Wake Forest University in Winston-Salem, North Carolina, sees potential for future human–computer collaborations. “While we may be able to push the boundaries a little further with this computational approach,” he says, “I’m excited for theoretical researchers to start analysing the new algorithms they’ve found to find clues for where to search for the next breakthrough.”
People often need to adapt to unexpected and sudden events, such as a road construction or a road accident while driving, a broken automatic payment or ATM machine, and changes in weather. To effectively deal with these events, they must possess what is known as behavioral flexibility, or the ability to deviate from routine and well-establish behavioral patterns.
To adapt their behavior based on unforeseen events, humans need to encode and retrieve reward-related memories and use them to inform their present or future choices. This process entails the integration of different cognitive abilities that are supported by different regions of the brain.
Past studies found that patients with different neuropsychiatric disorders and those suffering from an addiction tend to have a scarce behavioral flexibility. This often adversely affects their quality of living, as it makes dealing with the uncertainty of daily life particularly challenging.
Researchers at Emory University School of Medicine have recently carried out a study aimed at better understanding the neural underpinnings of behavioral flexibility. Their paper, published in Nature Neuroscience, identifies a specific network of brain regions that could coordinate flexible learning and memory in humans, which could be impaired in individuals with addictions or other neuropsychiatric disorders.
The researchers specifically looked at the orbitofrontal cortex (OFC), a brain region known to decode and represent some sensory reward-based stimuli. The OFC is also believed to be involved in learning that occurs over time, after humans or animals are consistently rewarded or punished for specific behaviors.
"The OFC supports outcome-guided behaviors," Dan C. Li and his colleagues wrote in their paper. "However, the coordinated neural circuitry and cellular mechanisms by which OFC connections sustain flexible learning and memory remain elusive. We demonstrate in mice that basolateral amygdala (BLA)→OFC projections bidirectionally control memory formation when familiar behaviors are unexpectedly not rewarded, whereas OFC→dorsomedial striatum (DMS) projections facilitate memory retrieval."
To identify OFC circuits that mediate the learning and memory processes involved in behavioral flexibility, Li and his colleagues carried out a series of experiments on mice. Specifically, they assessed the ability of mice to adapt their behavior when they unexpectedly did not receive a reward for actions that had previously been rewarded.
The researchers observed the behavior of the mice throughout their experiment and collected measurements that allowed them to determine which brain networks underpinned the mice's behaviors. Their results shed new light on a brain network that could be involved in behavioral flexibility, which includes three areas of the brain, namely the basolateral amygdala (BLA), the OFC, and the dorsomedial striatum (DMS).
"OFC neuronal ensembles store a memory trace for newly learned information, which appears to be facilitated by circuit-specific dendritic spine plasticity and neurotrophin signaling within defined BLA–OFC–DMS connections and obstructed by cocaine," Li and his colleagues wrote in their paper. "Thus, we describe the directional transmission of information within an integrated amygdalo-fronto-striatal circuit across time, whereby novel memories are encoded by BLA→OFC inputs, represented within OFC ensembles and retrieved via OFC→DMS outputs during future choice."
The exact work by this team of researchers offers some new insight about how behavioral flexibility is enabled by the mammalian brain. More specifically, the researchers showed that the learning and memory processes underpinning flexible behavioral adaptation could be the result of specific interactions between the BLA, OFC, and DMS.
These findings could soon pave the way for additional studies aimed at further investigating this newly delineated brain network and how it may promote flexible learning and memory retrieval. In the future, this could potentially help to devise new treatment strategies that might help to increase the behavioral flexibility of people with addictions, anxiety, and other debilitating neuropsychiatric conditions.
© 2022 Science X Network
Citation: Study identifies a neural network supporting flexible learning and memory (2022, October 5) retrieved 17 October 2022 from https://medicalxpress.com/news/2022-10-neural-network-flexible-memory.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
Many of the most innovative ideas and successful companies would not exist if it weren’t for funding from angel investors and venture capital. The traditional approach to business incubators and accelerators also leaves a lot to be desired, though. There is a new player in the market that intends to disrupt the standard model—NumberOne AI.
NumberOne AI announced today that it has raised $13 million to fund a groundbreaking predictive artificial intelligence and machine learning (AI/ML) incubator. Headquartered in Newport Beach, CA, NumberOne AI was founded in 2021 by Stuart McClure as a first-of-its-kind concept—to apply AI/ML to address the challenge of starting and launching successful companies.
NumberOne AI is also focused on fostering AI/ML on a broader scale by investing in and supporting businesses beyond the world of cybersecurity that leverage predictive AI to solve problems in a predictive and preventative way.
NumberOne AI’s founder and CEO, Stuart McClure, is a pioneer in the use of AI/ML for cybersecurity, and has a track record of building and leading successful ventures. He was a co-founder of Foundstone, which was acquired by McAfee in 2004, and he founded Cylance, which was bought by BlackBerry in 2019.
Cylance disrupted the endpoint security market and shifted the conversation from the notion of assumed breach and just trying to detect and respond faster back to the idea that attacks could simply be prevented in the first place. He applied the power of mathematics and predictive analytics to develop Cylance’s deep neural network prevention platform.
Recently, Stuart embarked on a similar mission—but this time to take the concept of predictive AI to the root of the problem. Stuart assumed the role of CEO for ShiftLeft, where he is disrupting the application security market by applying predictive AI to the foundation of cyber threats—the source code itself.
Now, he is taking what he accomplished at Cylance and what he is currently doing at ShiftLeft, and expanding to help other companies leverage AI/ML to solve large-scale problems as well. Stuart explained in a press release, “AI/ML allows the world to learn from the past to predict the future, and that is what we intend to do with each company we create at NumberOne AI.”
“Stuart’s vision for the application of predictive AI into the world beyond cybersecurity will empower companies big and small to solve some of the world’s toughest problems” says Michael Capellas, former CEO Compaq, First Data and WorldCom.
This $13 million will be invested to achieve this mission. NumberOne AI plans to incubate tomorrow’s future by supporting companies built from the ground up with predictive AI at their cores.
Stuart also intends to turn the incubator / foundry / accelerator model upside down with NumberOne AI. I spoke with him about NumberOne AI over the summer, and we discussed that the general perception of the VC world is that they simply throw a ton of money at the wall to see what sticks. It’s purely a numbers game. They know going in that 99 out of 100 will fail, but with the expectation that the one that is left will be wildly successful and they will cash in.
Led by numerous serial CEOs and founders, along with successful venture investors like Miramar Digital Ventures in Newport Beach, CA, NumberOne AI will use the round to build out the core ML learning platform and programmatize the startup building process. “We are thrilled to participate in NumberOne AI. Many company founders and operators often struggle to ﬁnd the right balance of innovation, growth and execution. Stuart has a proven recipe for success that will enable countless entrepreneurs to succeed with a predictive AI/ML core at their heart” says Bruce Hallett, Partner at Miramar Digital Ventures.
With NumberOne AI, Stuart plans to focus on quality over quantity. The goal is to go beyond simply infusing money into the system and provide the support and guidance necessary to help the companies achieve success.
Reinforcement learning broadly describes techniques that use rewards and penalties to guide an AI model through a complex task. A human analogy could be playing any game with a ranking system. Better play (e.g. winning games) is rewarded by moving up the leaderboard while mistakes are met with a drop in rank. Along the way, players will try different tactics and strategies to adapt to what opponents are doing. Of course, some humans may not be bothered to care about a ladder rank, but AI models can be compelled with software.
AlphaTensor is an AI model based on AlphaZero which is tasked with discovering algorithms to solve arbitrary matrix multiplication problems. Matrix multiplications are used to describe transformations in space and the matrices represent a mathematical concept called a tensor, the general term for scalars and vectors. Tensor math is at the heart of linear algebra and has applications in various fields from materials science to machine learning itself.
Matrix multiplications are solved according to specific rules and processes. Like much of math, there are optimizations that can be made to solve these problems in fewer and fewer steps. The refined algorithms enable larger matrix multiplications to be completed at feasible timescales.
AlphaTensor is presented with a game in the form of a single player puzzle. The board consists of a grid of numbers representing a 3D Tensor, which AlphaTensor is then tasked with reducing to zeros through a series of allowable moves, comprised of matrix multiplications. The possible moveset is staggeringly massive, exceeding games like chess and Go by several factors.
The AI uses Monte Carlo tree searches (MCTS) to plan its moves. This is effectively the same system used by AlphaZero to master chess and Go. With MCTS, the AI player looks at a sample of potential moves whose outcomes are tracked as a distribution of potential success. Rounds of the game are capped to a certain number of runs to avoid unnecessarily long games, but successful matches are fed back in to Excellerate the network’s decisionmaking parameters.
Co-author Hussein Fawzi told Nature, “AlphaTensor embeds no human intuition about matrix multiplication,” so “the agent in some sense needs to build its own knowledge about the problem from scratch.” Through this feedback, AlphaTensor learns which moves are more likely to yield success, and successful it has been. In matrix sizes up to 5 x 5 pairs, AlphaTensor has matched or exceeded the efficiency of known algorithms in terms of steps.
Specifically, AlphaTensor has discovered a 47 step solution to paired 4 x 4 matrix multiplication which improves on the known 49 step solution which was found in 1969. It also shaved a couple steps off of paired 5 x 5 matrix multiplication, reducing 98 steps to 96. Its enhancements also extended to a few asymmetrically sized matrix multiplications, with better ways of solving 3 x 4 times 4 x 5 matrices, 4 x 4 times 4 x 5 matrices, and 4 x 5 times 5 x 5 matrices, the last of which it accomplished in four fewer steps than known methods.
The researchers note that the initial predefined moveset sampling does present a limitation. It is possible that more efficient algorithms could be derived from starting moves that are excluded at the start. Apart from brute forcing every possible move—which is computationally expensive—the researchers believe they could adapt AlphaTensor to find better starting sets itself. An AI operating on the fundamental math it is built from and optimizing its conditions? Maybe this is starting to sound like the singularity.
In reality though, it is easy to draw parallels between AlphaTensor learning to solve these immense problems and a fledgling Starcraft player, which DeepMind's AI has coincidentally excelled at too. Through iteration, it learns which moves will maximize its chances of success and which are more likely to end in failure—like having a swarm of zerglings get roasted by the opponent’s clump of fire-spewing hellbats. The zerg player can try sneaky run-bys in future matches to exploit the hellbat’s lack of mobility in future matches rather than engage in a direct fight. Matrix multiplication may not be as thrilling, but the underpinning process the AI uses to learn is all the same.