At the 2025 International Conference on Artificial Intelligence and Law, I was very impressed by a demonstration of CATLEX, which includes LLM-assisted extraction of legal doctrines, and a flowchart-like view that diagrams a court’s legal reasoning process in resolving a legal issue.

CATLEX relies on a corpus of disability-claim decisions issued by the Board of Veterans’ Appeals, which was assembled by the Research Laboratory for Law, Logic and Technology (LLT Lab) at Hofstra. CATLEX was presented by Hannes Westermann, and the other two authors of the conference paper were Vern Walker of the LLT Lab, and Jaromir Savelka.

The project depended on a description of the elements of a claim of a service-related disability, in the form of a JSON “rule tree” that I think was created entirely by hand. Then the authors used the litellm Python client to present LLMs, including Claude 3.5 Sonnet, with Board decisions from the LLT Lab’s collection. The LLMs were prompted to determine whether the Board of Veterans’ Appeals found each element to be present or absent in each case. The Board’s determinations could then be plotted onto a diagram showing the steps needed to establish the claim, along with the LLM’s explanations of its interpretations of the Board’s reasoning. The screenshots of the resulting Vue app are neat-looking, but I’m not posting them because the paper is still a preprint and I don’t think the authors have published the screenshots online. Probably the screenshots could be replicated by running the code in the GitHub repo linked above.

The model may oversimplify somewhat, in condensing the range of possible decisions on a factor to just “True” or “False”. In the example data, when the Board assumed a factor was satisfied, that was considered “True”. When the Board didn’t mention a factor, that was considered “False”.

I think CATLEX’s rule visualization by itself could be an interesting avenue for future open source tools, even if the content was created by hand. And the LLM data extraction is also exciting, since it shows how researchers are starting to break down judicial reasoning into modular parts, assigning some of those parts to AI agents, and measuring the accuracy of the results. But in this project the AI hasn’t fully taken over, since the legal analysis is still tethered to human annotations describing the structure of a very specific legal issue.