Key Findings
Research Question: Can constraining an LLM with a legal-move list eliminate hallucinated chess moves?
464
Total LLM Calls
295
Legal Moves
169
Illegal Moves
63.6%
Overall Success Rate
Tournament Results
| Tournament | Temperature | Legal Constraint | Legal Move Rate |
|---|---|---|---|
| T1 | 0.1 | No | 49.5% |
| T2 | 0.8 | No | 43.0% |
| T3 | 0.8 | Yes | 97.4% |
Adding the legal-move constraint (injecting the full list of valid UCI moves into the prompt) raised the legal move rate from ~43% to 97.4%.
Hallucination Log (298 entries)
| Timestamp | Game # | Turn # | FEN | Error |
|---|---|---|---|---|
| 2026-02-20 18:16:49 | 1 | 1 | rnbqkbnr/pppppppp/8/8/8/1P6/P1PPPPPP/RNB… | illegal uci: 'e2e4' in rnbqkbnr/pppppppp/8/8/8/1P6/P1PPPPPP/… |
| 2026-02-20 18:17:00 | 2 | 1 | rnbqkbnr/pppppppp/8/8/8/5N2/PPPPPPPP/RNB… | illegal uci: 'e2e4' in rnbqkbnr/pppppppp/8/8/8/5N2/PPPPPPPP/… |
| 2026-02-21 16:53:50 | 1 | 1 | rnbqkbnr/pppppppp/8/8/8/P7/1PPPPPPP/RNBQ… | illegal uci: 'e2e4' in rnbqkbnr/pppppppp/8/8/8/P7/1PPPPPPP/R… |
| 2026-02-21 16:54:02 | 2 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/2N1P3/PPPP1PPP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/2N1P3/PPPP1… |
| 2026-02-21 16:54:13 | 3 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/5NP1/PPPPPP1P/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/5NP1/PPPPPP… |
| 2026-02-21 16:54:25 | 4 | 2 | rnbqkbnr/ppp1pppp/8/3p4/6P1/7N/PPPPPP1P/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/6P1/7N/PPPPPP… |
| 2026-02-21 16:54:36 | 5 | 2 | rnbqkbnr/ppp1pppp/8/3p4/6P1/P7/1PPPPP1P/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/6P1/P7/1PPPPP… |
| 2026-02-21 16:54:47 | 6 | 2 | rnbqkbnr/ppp1pppp/8/3p4/5P2/3P4/PPP1P1PP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/5P2/3P4/PPP1P… |
| 2026-02-21 16:54:57 | 7 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/5PP1/PPPPP2P/R… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/5PP1/PPPPP2… |
| 2026-02-21 16:55:08 | 8 | 2 | rnbqkbnr/ppp1pppp/8/3p4/4P3/5P2/PPPP2PP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/4P3/5P2/PPPP2… |
| 2026-02-21 16:55:19 | 9 | 2 | rnbqkbnr/ppp1pppp/8/3p4/2P5/6P1/PP1PPP1P… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/2P5/6P1/PP1PP… |
| 2026-02-21 16:55:29 | 10 | 2 | rnbqkbnr/ppp1pppp/8/3p4/2P3P1/8/PP1PPP1P… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/2P3P1/8/PP1PP… |
| 2026-02-21 16:55:40 | 11 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/N6P/PPPPPPP1/R… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/N6P/PPPPPPP… |
| 2026-02-21 16:55:51 | 12 | 2 | rnbqkbnr/ppp1pppp/8/3p4/3P4/1P6/P1P1PPPP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/3P4/1P6/P1P1P… |
| 2026-02-21 16:56:02 | 13 | 2 | rnbqkbnr/ppp1pppp/8/3p4/3P4/P7/1PP1PPPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/3P4/P7/1PP1PP… |
| 2026-02-21 16:56:13 | 14 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/2N2N2/PPPPPPPP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/2N2N2/PPPPP… |
| 2026-02-21 16:56:24 | 15 | 2 | rnbqkbnr/ppp1pppp/8/3p4/2P5/3P4/PP2PPPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/2P5/3P4/PP2PP… |
| 2026-02-21 16:56:35 | 16 | 2 | rnbqkbnr/ppp1pppp/8/3p4/5P2/8/PPPPPKPP/R… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/5P2/8/PPPPPKP… |
| 2026-02-21 16:56:46 | 17 | 2 | rnbqkbnr/ppp1pppp/8/3p4/7P/1P6/P1PPPPP1/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/7P/1P6/P1PPPP… |
| 2026-02-21 16:56:57 | 18 | 2 | rnbqkbnr/ppp1pppp/8/3p4/1P4P1/8/P1PPPP1P… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/1P4P1/8/P1PPP… |
| 2026-02-21 16:57:08 | 19 | 2 | rnbqkbnr/ppp1pppp/8/3p4/4P3/6P1/PPPP1P1P… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/4P3/6P1/PPPP1… |
| 2026-02-21 16:57:19 | 20 | 2 | rnbqkbnr/ppp1pppp/8/3p4/6P1/5N2/PPPPPP1P… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/6P1/5N2/PPPPP… |
| 2026-02-21 16:57:30 | 21 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/2N4P/PPPPPPP1/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/2N4P/PPPPPP… |
| 2026-02-21 16:57:41 | 22 | 2 | rnbqkbnr/ppp1pppp/8/3p4/5P2/N7/PPPPP1PP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/5P2/N7/PPPPP1… |
| 2026-02-21 16:57:52 | 23 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/5NP1/PPPPPP1P/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/5NP1/PPPPPP… |
| 2026-02-21 16:58:03 | 24 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/N2P4/PPP1PPPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/N2P4/PPP1PP… |
| 2026-02-21 16:58:15 | 25 | 2 | rnbqkbnr/ppp1pppp/8/3p4/4P3/7P/PPPP1PP1/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/4P3/7P/PPPP1P… |
| 2026-02-21 16:58:25 | 26 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/N3P3/PPPP1PPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/N3P3/PPPP1P… |
| 2026-02-21 16:58:36 | 27 | 2 | rnbqkbnr/ppp1pppp/8/1N1p4/8/8/PPPPPPPP/R… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/1N1p4/8/8/PPPPPPP… |
| 2026-02-21 16:58:47 | 28 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/N6P/PPPPPPP1/R… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/N6P/PPPPPPP… |
| 2026-02-21 16:58:59 | 29 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/3P4/PPPBPPPP/R… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/3P4/PPPBPPP… |
| 2026-02-21 16:59:10 | 30 | 2 | rnbqkbnr/ppp1pppp/8/3p4/6P1/7N/PPPPPP1P/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/6P1/7N/PPPPPP… |
| 2026-02-21 16:59:21 | 31 | 2 | rnbqkbnr/ppp1pppp/8/3p4/7P/7R/PPPPPPP1/R… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/7P/7R/PPPPPPP… |
| 2026-02-21 16:59:32 | 32 | 2 | rnbqkbnr/ppp1pppp/8/3p4/1P6/5P2/P1PPP1PP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/1P6/5P2/P1PPP… |
| 2026-02-21 16:59:43 | 33 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/2N4N/PPPPPPPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/2N4N/PPPPPP… |
| 2026-02-21 16:59:54 | 34 | 2 | rnbqkbnr/ppp1pppp/8/3p4/4P3/5N2/PPPP1PPP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/4P3/5N2/PPPP1… |
| 2026-02-21 17:00:05 | 35 | 2 | rnbqkbnr/ppp1pppp/8/3p4/2P5/2N5/PP1PPPPP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/2P5/2N5/PP1PP… |
| 2026-02-21 17:00:16 | 36 | 2 | rnbqkbnr/ppp1pppp/8/3p4/5P2/P7/1PPPP1PP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/5P2/P7/1PPPP1… |
| 2026-02-21 17:00:28 | 37 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/3P3N/PPP1PPPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/3P3N/PPP1PP… |
| 2026-02-21 17:00:39 | 38 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/N3P3/PPPP1PPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/N3P3/PPPP1P… |
| 2026-02-21 17:00:50 | 39 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/2N4N/PPPPPPPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/2N4N/PPPPPP… |
| 2026-02-21 17:01:00 | 40 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/N6P/PPPPPPP1/R… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/N6P/PPPPPPP… |
| 2026-02-21 17:01:12 | 41 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/2N3P1/PPPPPP1P… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/2N3P1/PPPPP… |
| 2026-02-21 17:01:23 | 42 | 2 | rnbqkbnr/ppp1pppp/8/3p4/1P2P3/8/P1PP1PPP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/1P2P3/8/P1PP1… |
| 2026-02-21 17:01:34 | 43 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/P3P3/1PPP1PPP/… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/P3P3/1PPP1P… |
| 2026-02-21 17:01:44 | 44 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/2N5/PPPPPPPP/1… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/2N5/PPPPPPP… |
| 2026-02-21 17:01:55 | 45 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/8/PPPPPPPP/RNB… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/8/PPPPPPPP/… |
| 2026-02-21 17:02:06 | 46 | 2 | rnbqkbnr/ppp1pppp/8/3p4/3P4/5P2/PPP1P1PP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/3P4/5P2/PPP1P… |
| 2026-02-21 17:02:17 | 47 | 2 | rnbqkbnr/ppp1pppp/8/3p4/8/6PP/PPPPPP2/RN… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p4/8/6PP/PPPPPP2… |
| 2026-02-21 17:02:28 | 48 | 2 | rnbqkbnr/ppp1pppp/8/3p2B1/8/3P4/PPP1PPPP… | illegal uci: 'e2e4' in rnbqkbnr/ppp1pppp/8/3p2B1/8/3P4/PPP1P… |
Engine Telemetry (sample of 100 entries)
| Timestamp | Temp | Latency (ms) | Move | Legal? | Fallback? |
|---|---|---|---|---|---|
| 1771673037 | 0.10 | 6425 | d7d5 |
✅ | — |
| 1771673042 | 0.10 | 4787 | d7d5 |
❌ | ⚡ |
| 1771673048 | 0.10 | 4904 | d7d5 |
✅ | — |
| 1771673053 | 0.10 | 4798 | d7d5 |
❌ | ⚡ |
| 1771673059 | 0.10 | 4848 | d7d5 |
✅ | — |
| 1771673065 | 0.10 | 5092 | d7d5 |
❌ | ⚡ |
| 1771673071 | 0.10 | 5265 | d7d5 |
✅ | — |
| 1771673076 | 0.10 | 4615 | d7d5 |
❌ | ⚡ |
| 1771673082 | 0.10 | 4736 | d7d5 |
✅ | — |
| 1771673087 | 0.10 | 4813 | d7d5 |
❌ | ⚡ |
| 1771673092 | 0.10 | 4278 | d7d5 |
✅ | — |
| 1771673097 | 0.10 | 4694 | d7d5 |
❌ | ⚡ |
| 1771673103 | 0.10 | 4625 | d7d5 |
✅ | — |
| 1771673108 | 0.10 | 4755 | d7d5 |
❌ | ⚡ |
| 1771673113 | 0.10 | 4581 | d7d5 |
✅ | — |
| 1771673119 | 0.10 | 4948 | d7d5 |
❌ | ⚡ |
| 1771673124 | 0.10 | 4722 | d7d5 |
✅ | — |
| 1771673129 | 0.10 | 4799 | d7d5 |
❌ | ⚡ |
| 1771673135 | 0.10 | 4760 | d7d5 |
✅ | — |
| 1771673140 | 0.10 | 4698 | d7d5 |
❌ | ⚡ |
| 1771673146 | 0.10 | 4620 | d7d5 |
✅ | — |
| 1771673151 | 0.10 | 4949 | d7d5 |
❌ | ⚡ |
| 1771673157 | 0.10 | 4763 | d7d5 |
✅ | — |
| 1771673162 | 0.10 | 4764 | d7d5 |
❌ | ⚡ |
| 1771673168 | 0.10 | 4858 | d7d5 |
✅ | — |
| 1771673173 | 0.10 | 4954 | d7d5 |
❌ | ⚡ |
| 1771673179 | 0.10 | 4993 | d7d5 |
✅ | — |
| 1771673184 | 0.10 | 5009 | d7d5 |
❌ | ⚡ |
| 1771673190 | 0.10 | 4880 | d7d5 |
✅ | — |
| 1771673195 | 0.10 | 4898 | d7d5 |
❌ | ⚡ |
| 1771673201 | 0.10 | 4533 | d7d5 |
✅ | — |
| 1771673206 | 0.10 | 4772 | d7d5 |
❌ | ⚡ |
| 1771673212 | 0.10 | 4637 | d7d5 |
✅ | — |
| 1771673217 | 0.10 | 4770 | d7d5 |
❌ | ⚡ |
| 1771673223 | 0.10 | 4884 | d7d5 |
✅ | — |
| 1771673228 | 0.10 | 4668 | d7d5 |
❌ | ⚡ |
| 1771673234 | 0.10 | 4863 | d7d5 |
✅ | — |
| 1771673239 | 0.10 | 4939 | d7d5 |
❌ | ⚡ |
| 1771673244 | 0.10 | 4566 | d7d5 |
✅ | — |
| 1771673250 | 0.10 | 5036 | d7d5 |
❌ | ⚡ |
| 1771673256 | 0.10 | 4765 | d7d5 |
✅ | — |
| 1771673261 | 0.10 | 5580 | d7d5 |
❌ | ⚡ |
| 1771673267 | 0.10 | 4724 | d7d5 |
✅ | — |
| 1771673272 | 0.10 | 4887 | d7d5 |
❌ | ⚡ |
| 1771673278 | 0.10 | 4878 | d7d5 |
✅ | — |
| 1771673283 | 0.10 | 4762 | d7d5 |
❌ | ⚡ |
| 1771673289 | 0.10 | 5100 | d7d5 |
✅ | — |
| 1771673295 | 0.10 | 5079 | d7d5 |
❌ | ⚡ |
| 1771673300 | 0.10 | 4596 | d7d5 |
✅ | — |
| 1771673305 | 0.10 | 4845 | d7d5 |
❌ | ⚡ |
| 1771673311 | 0.10 | 4595 | d7d5 |
✅ | — |
| 1771673316 | 0.10 | 5119 | d7d5 |
❌ | ⚡ |
| 1771673322 | 0.10 | 4905 | d7d5 |
✅ | — |
| 1771673327 | 0.10 | 4763 | d7d5 |
❌ | ⚡ |
| 1771673334 | 0.10 | 4931 | d7d5 |
✅ | — |
| 1771673339 | 0.10 | 5220 | d7d5 |
❌ | ⚡ |
| 1771673345 | 0.10 | 4822 | d7d5 |
✅ | — |
| 1771673350 | 0.10 | 5247 | d7d5 |
❌ | ⚡ |
| 1771673356 | 0.10 | 4554 | d7d5 |
✅ | — |
| 1771673361 | 0.10 | 4954 | d7d5 |
❌ | ⚡ |
| 1771673367 | 0.10 | 4569 | d7d5 |
✅ | — |
| 1771673372 | 0.10 | 4856 | d7d5 |
❌ | ⚡ |
| 1771673378 | 0.10 | 5408 | d7d5 |
✅ | — |
| 1771673383 | 0.10 | 4892 | d7d5 |
❌ | ⚡ |
| 1771673389 | 0.10 | 4691 | d7d5 |
✅ | — |
| 1771673394 | 0.10 | 4880 | d7d5 |
❌ | ⚡ |
| 1771673400 | 0.10 | 4722 | d7d5 |
✅ | — |
| 1771673405 | 0.10 | 5006 | d7d5 |
❌ | ⚡ |
| 1771673411 | 0.10 | 4756 | d7d5 |
✅ | — |
| 1771673416 | 0.10 | 5009 | d7d5 |
❌ | ⚡ |
| 1771673423 | 0.10 | 4947 | d7d5 |
✅ | — |
| 1771673428 | 0.10 | 4751 | d7d5 |
❌ | ⚡ |
| 1771673434 | 0.10 | 4985 | d7d5 |
✅ | — |
| 1771673439 | 0.10 | 4701 | d7d5 |
❌ | ⚡ |
| 1771673444 | 0.10 | 4845 | d7d5 |
✅ | — |
| 1771673450 | 0.10 | 4930 | d7d5 |
❌ | ⚡ |
| 1771673455 | 0.10 | 4610 | d7d5 |
✅ | — |
| 1771673460 | 0.10 | 4847 | d7d5 |
❌ | ⚡ |
| 1771673466 | 0.10 | 4940 | d7d5 |
✅ | — |
| 1771673472 | 0.10 | 5344 | d7d5 |
❌ | ⚡ |
| 1771673478 | 0.10 | 4636 | d7d5 |
✅ | — |
| 1771673483 | 0.10 | 4905 | d7d5 |
❌ | ⚡ |
| 1771673489 | 0.10 | 4648 | d7d5 |
✅ | — |
| 1771673494 | 0.10 | 4854 | d7d5 |
❌ | ⚡ |
| 1771673499 | 0.10 | 4727 | d7d5 |
✅ | — |
| 1771673504 | 0.10 | 4822 | d7d5 |
❌ | ⚡ |
| 1771673510 | 0.10 | 4779 | d7d5 |
✅ | — |
| 1771673515 | 0.10 | 4686 | d7d5 |
❌ | ⚡ |
| 1771673521 | 0.10 | 4684 | d7d5 |
✅ | — |
| 1771673526 | 0.10 | 4891 | d7d5 |
❌ | ⚡ |
| 1771673532 | 0.10 | 4615 | d7d5 |
✅ | — |
| 1771673537 | 0.10 | 5014 | d7d5 |
❌ | ⚡ |
| 1771673543 | 0.10 | 4658 | d7d5 |
✅ | — |
| 1771673548 | 0.10 | 4878 | d7d5 |
❌ | ⚡ |
| 1771673554 | 0.10 | 4941 | d7d5 |
✅ | — |
| 1771673559 | 0.10 | 5088 | d6d5 |
❌ | ⚡ |
| 1771673565 | 0.10 | 4657 | d7d5 |
✅ | — |
| 1771673570 | 0.10 | 5042 | d7d5 |
❌ | ⚡ |
| 1771673576 | 0.10 | 4726 | d7d5 |
✅ | — |
| 1771673581 | 0.10 | 4797 | d7d5 |
❌ | ⚡ |
Architecture
┌──────────────┐ HTTP/REST ┌─────────────────────────────┐
│ Web Browser │◄──────────────────►│ Flask Backend (Python) │
│ (HTML/CSS/ │ │ web/app.py │
│ JavaScript)│ │ │
│ │ │ python-chess game logic │
└──────────────┘ └────────────┬────────────────┘
│ UCI Protocol (stdin/stdout)
┌────────────▼────────────────┐
│ VICE Engine (C) │
│ Original: Bluefever Soft. │
│ │
│ + llm_search.c (added) │──► Legal moves
│ + http_client.c (added) │──► Ollama API
│ + llm_parser.c (added) │──► Move parse
│ + telemetry.c (added) │──► CSV logging
└────────────┬────────────────┘
│ HTTP (libcurl)
┌────────────▼────────────────┐
│ Ollama (local) │
│ llama3:latest │
│ 8B params, Q4_0 │
└─────────────────────────────┘