Use Prolog to improve LLM's reasoning - Shchegrikovich LLM

On one side, LLMs show unseen capabilities in reasoning, but on the other - reasoning in LLMs is not ideal. The problem might be in the way LLMs work. LLMs generate answer sequentially in one pass, they can not use loops or conditions. The limit factor here is autoregressive architecture of transformers. In addition, they have problems when reason about the topic which is outside the training set. There are several ways how to improve the reasoning capabilities of LLMs. The first one is to generate many examples and choose the best one. The second is to use special prompting techniques such as CoT. The third technique - is to use programming languages. Suppose we ask a model to solve an equation for us, such as 5+5. The model might write a Python code to find the answer. In this case, python is used as an intermediate language to help the model think. Just recently, developers started to use Prolog as an intermediate language to improve reasoning.