Automated Reasoning, the New Deep Learning? Could the Two Work Together?

Automated reasoning is based on logic, a branch of mathematics that goes back to the endeavors of Aristotle, which aims to understand and formalize human reasoning. Automated logical reasoning is the oldest and most intensively studied field in the research area of Artificial Intelligence (AI); further fields it is studied within are Knowledge Representation (KR), Machine Learning (ML), and Natural Language Processing (NLP) among others.

There are no strict barriers between these fields. KR and ML techniques are used in NLP. Automated reasoning is used in KR. ML algorithms are part of automated reasoning systems, and vice versa. The combination of all these increasingly powerful techniques is natural and is already happening. Several recent advances in understanding have greatly increased the power of automated reasoning tools. These advances were mostly leveraged to find bugs in device drivers. However, there are now several distinct efforts to introduce automated reasoning into a new range of use case areas. One of the foremost companies productizing automated reasoning is Cambridge-based Diffblue.

Diffblue has developed an automated reasoning software Deeptest, which can automatically generate unit tests for code under development. Deeptest parses the Java bytecode and then passes it off to Diffblue's AI engine, which constructs an exact mathematical model of the codebase. The resulting model allows a deep semantic understanding of a computer program's behavior. Diffblue’s Deeptest interrogates the AI engine to learn groups of distinct classes of behaviors, such as different kinds of “good” behaviors and “bad” (error) behaviors; from this activity, the software can effectively generate unit tests for the code. Behaviors of an Automated Teller Machine (ATM) could be, for example, successfully withdrawing money from your bank account, refusing to give you money because you have exceeded your daily limit, withdrawing from another person's bank account, or swallowing your bank card.

One of the biggest use cases of automated reasoning is checking and verifying the code behind computing systems. Doing so mitigates the prevalence of developer miscalculations and other errors. Of course, automated reasoning requires a developer for programming the technology to understand what problems it should identify and what programing language is needed.

As an example of automated reasoning in action, let's take a look at Amazon Web Services (AWS). AWS uses automated reasoning for keeping customer cloud ecosystems safe and secure. With this solution, the technology is instructed on the intended protocols and code requirements, which are translated into mathematical equations. Using a logic framework, the systems are then cross-referenced to make sure that the mathematical specs are accurate. If any code is inconsistent with the cloud’s intended framework, the user will be alerted because the formal reasoning tools will detect it.

Back to Diffblue, the company claims that its software automatically generates unit tests, has the potential to save development teams hours of time and achieve a much higher coverage of unit testing than is normally achieved. Writing unit tests is often considered a laborious task by developers. While most developers spend one to three hours a day writing them, Deeptest and other automated reasoning algorithms can produce unit tests in minutes. Diffblue has been able to show that its software increases unit test coverage dramatically. Typically, coverage is around 30%, but Deeptest typically achieves coverage anywhere between 50% and 70%.

Unit testing is done by all developers writing software, meaning the resources dedicated to writing the tests could stretch into the many billions of dollars per annum. Deeptest also would allow developers to focus more time on building and improving their products. Moreover, automated reasoning can make for a more creative environment for software developers, empowering them to make better products. Diffblue is unlikely to replace the entire practice of unit test writing without competition, but the company appears to have developed a genuinely unique piece of software to automate the process, giving it a head start over competitors.

In terms of the computing required for Deeptest, Diffblue Chief Technology Officer (CTO) Peter Schrammel said about the software, “AI cannot overturn the laws of nature and principles of mathematics. Solving difficult problems such as reasoning about the behaviors of programs requires significant computational resources. However, we can achieve good performance if we ask the right questions to the AI engine for a target use case. When adding a new feature to an unknown piece of software with several million lines of code, a software developer will first ask, ‘Can you help me understand what I'm doing?’ rather than ‘Can you prove that the entire thing is bug-free?’ It is thus essential to focus on what the (human) end user of the AI system needs most.” Even still, for significant deployments of Deeptest, the software will require clients to have hardware installed on-premise. The company is working with major cloud providers to achieve a more typical Software as a Service (SaaS) type offering. To date, the software is only available in Java, but the company claims it will be expanding the software into other languages as requested by customers.

The mission of Diffblue and other software vendors is to push forward and develop automated reasoning techniques to provide tools that will allow the perfect automated production of software. The end goal for the company is to combine techniques like automated reasoning, NLP, and voice recognition, so anyone will be able to explain to its software what he or she wants a piece of code to do. In turn, it will be able to automatically generate that code to a standard that is ready for market. With that said, Diffblue admits that such a piece of software is still some way off and, in combining several AI techniques, would likely be highly compute-intensive.

AI is broader than just ML, although it has taken much of the limelight with significant and continuing developments in neural networks in the form of Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and now Generative Adversarial Networks (GANs), and Creative Adversarial Networks (CANs). More attention needs to be paid to the market potential in productizing other AI techniques aside from ML, like automated reasoning. Today there is also a lack of exploration even conceptually of the potential of cross-pollinating different AI techniques. AI frameworks are beginning to aid this cross-pollination through framework efforts like ONNX, which allows developers to develop algorithms across multiple frameworks. Frameworks are often specialized and optimized in different ways, favoring different applications; this will more readily enable developers to build software that leverages these different capabilities. Combining multiple layers of different flavors of AI will undoubtedly be heavily compute-intensive, but it also promises to be highly lucrative. Enterprises need to broaden their thinking about the potential of AI and automated reasoning in all its forms and how it can be implemented to truly transform their businesses.