›
›
11 Important Studies on AI and Law - What Capabilities Does AI Have in the Legal Field?
11 Important Studies on AI and Law - What Capabilities Does AI Have in the Legal Field?
An overview of current research:
AI vs. Humans
1. GPT-4 passes the Bar Exam (Katz et al, 2024).
The classic: The "ancient" GPT-4 passes the US Bar Exam. It remains disputed how well the AI actually performed.
https://lnkd.in/d-Tawyzf
2. AI vs. Judges (Posner/Saran, 2025)
A complex case is presented to both experienced judges and an AI system for decision. The AI adheres more strictly to precedents, judges focus more on the defendant's personality.
https://lnkd.in/dFM-nQng
3. AI deployment in Shenzhen (Liu/Li, 2025)
Chinese courts use AI to draft judgment reasoning. Decisions are still made by humans, but the study suggests that the content of decisions is also changing.
https://lnkd.in/d_mmbe7C
4. AI in Legal Education
AI-assisted legal practice (Schwarcz et al., 2025)
Law students working with advanced AI models not only work faster but also produce higher quality work.
https://lnkd.in/dAaMYPq2
5. AI writes law exams (Fan et al., 2025)
LLMs write 340 German and English law exams - and then grade them too. In German, the AIs perform worse.
https://lnkd.in/d5JnuqEX
My review: https://lnkd.in/dtyUzfyz
6. Can AI take over professor office hours? (Ouellette et al., 2025)
A patent law textbook is fed into leading AI systems. The AI answers 185 questions based on this. But many answers are only "acceptable" or even wrong.
https://lnkd.in/d3X9p_YW
7. Public Perception Objection overruled! (Schneiders et al., 2025)
Laypeople prefer AI-generated legal advice over that from lawyers - when they don't know the source.
https://lnkd.in/dUm_qZvv
8. AI instead of state courts (Kieffaber et al., 2025)
Start-up founders present the concept of a fully AI-based arbitration court as an alternative to state courts.
https://lnkd.in/dwD-wQTb
Limitations and Risks
9. AI and critical thinking (Lee et al., 2025)
Study with 319 knowledge workers suggests: Higher trust in AI leads to less critical thinking.
https://lnkd.in/drihAyJA
10. RAG is no panacea (Magesh et al., 2024)
Legal research tools with Retrieval Augmented Generation cannot completely prevent hallucinations.
https://lnkd.in/d4JmC-Hd
11. Large models, small effects (Humlum/Vestergaard, 2025)
Barely any impact of AI chatbots on the labor market and at most modest productivity gains - including in legal professions.
https://lnkd.in/dH6PfY9G
Source:
https://www.linkedin.com/feed/update/urn:li:activity:7338803493190656000/
Dr. Sebastian Dötterl
Judge at the Higher Regional Court of Munich
Verwandte Ratgeber
Why Swiss Legal AI Needs More Than ChatGPT
Why Swiss legal answers need more than ChatGPT or Claude
10 Reasons: Why ChatGPT & Claude Cannot Reliably Answer Legal Questions
10 Reasons: Why ChatGPT & Claude Cannot Reliably Answer Legal Questions
Why ChatGPT is wrong on 9 out of 10 common employment law questions
Why ChatGPT is wrong on 9 out of 10 common employment law questions
What are "hallucinations" in AI – and why are they dangerous in law?
What are "hallucinations" in AI – and why are they dangerous in law?
Why Prompting and RAG Fail in Law – and Why Jurilo Was Built Differently
Why Prompting and RAG Fail in Law – and Why Jurilo Was Built Differently

