International Contest 2024 (INLG 2024)
Official Results (Awards)
| Award | Team |
|---|---|
| Best Subjective Evaluation Award (Japanese Track) and Sponsor Award | kanolab |
| Best Subjective Evaluation Award (English Track) | UEC-IL |
| Best Win Rate and Game Action Award | sUper_IL |
| Judge's Special Award: Beginner-like Play | HondaNLP (鳥海審査員) |
| Judge's Special Award: Persuasion as a Werewolf | sUper_IL (原田審査員) |
Results and awards were presented at the AIWolfDial 2024 workshop (September 24, 2024, INLG 2024, Tokyo).
Win Rates
Computed from the logs of all games that finished normally in the main competition. "Overall" is wins divided by games. Per-role cells show the win rate in that role with (wins / games). "Adjusted" re-weights the per-role win rates by the village's role composition, cancelling out uneven role assignments.
English, 5-player village (58 games)
| # | Team | Games | Overall | Villager | Seer | Possessed | Werewolf | Adjusted |
|---|---|---|---|---|---|---|---|---|
| 1 | sUper_IL | 58 | 55.2% | 50.0% (10/20) | 61.5% (8/13) | 50.0% (7/14) | 63.6% (7/11) | 55.0% |
| 2 | kanolab | 58 | 53.4% | 56.5% (13/23) | 42.9% (3/7) | 61.5% (8/13) | 46.7% (7/15) | 52.8% |
| 3 | satozaki | 58 | 51.7% | 47.8% (11/23) | 57.1% (8/14) | 45.5% (5/11) | 60.0% (6/10) | 51.6% |
| 4 | UEC-IL | 58 | 51.7% | 52.4% (11/21) | 50.0% (6/12) | 50.0% (6/12) | 53.8% (7/13) | 51.7% |
| 5 | yuricat | 58 | 37.9% | 44.8% (13/29) | 33.3% (4/12) | 37.5% (3/8) | 22.2% (2/9) | 36.5% |
Japanese, 5-player village (64 games)
| # | Team | Games | Overall | Villager | Seer | Possessed | Werewolf | Adjusted |
|---|---|---|---|---|---|---|---|---|
| 1 | sUper_IL | 40 | 62.5% | 62.5% (10/16) | 50.0% (4/8) | 50.0% (4/8) | 87.5% (7/8) | 62.5% |
| 2 | satozaki | 40 | 60.0% | 37.5% (6/16) | 75.0% (6/8) | 75.0% (6/8) | 75.0% (6/8) | 60.0% |
| 3 | UEC-IL | 40 | 52.5% | 50.0% (8/16) | 37.5% (3/8) | 62.5% (5/8) | 62.5% (5/8) | 52.5% |
| 4 | HondaNLP | 40 | 52.5% | 50.0% (8/16) | 50.0% (4/8) | 75.0% (6/8) | 37.5% (3/8) | 52.5% |
| 5 | kanolab | 40 | 47.5% | 50.0% (8/16) | 25.0% (2/8) | 50.0% (4/8) | 62.5% (5/8) | 47.5% |
| 6 | GPTaku | 40 | 45.0% | 43.8% (7/16) | 37.5% (3/8) | 50.0% (4/8) | 50.0% (4/8) | 45.0% |
| 7 | IS_Lab | 40 | 37.5% | 37.5% (6/16) | 37.5% (3/8) | 25.0% (2/8) | 50.0% (4/8) | 37.5% |
| 8 | yuricat | 40 | 35.0% | 31.2% (5/16) | 50.0% (4/8) | 50.0% (4/8) | 12.5% (1/8) | 35.0% |
Game Metrics
Behavioural metrics computed mechanically from the game logs, independent of dialogue quality. Each value is a ratio against what random play would produce (1.00 = random). "Executed", "divined" and "guarded" show how often the team was targeted relative to chance; lower is better for being executed or divined. "Divination accuracy" and "vote accuracy" show how often the team found a werewolf relative to chance; higher is better. "Possessed avoidance" is the rate at which, as the possessed, the team voted for someone other than a werewolf.
English, 5-player village
| Team | Executed | Divined | Divination accuracy | Vote accuracy | Possessed avoidance |
|---|---|---|---|---|---|
| sUper_IL | 0.67 | 1.14 | 1.44 | 1.05 | 0.67 |
| kanolab | 1.17 | 1.20 | 2.48 | 1.36 | 0.80 |
| satozaki | 1.05 | 0.91 | 0.91 | 1.12 | 0.69 |
| UEC-IL | 0.90 | 0.80 | 0.75 | 0.96 | 0.88 |
| yuricat | 1.25 | 0.95 | 0.80 | 1.10 | 0.44 |
Japanese, 5-player village
| Team | Executed | Divined | Divination accuracy | Vote accuracy | Possessed avoidance |
|---|---|---|---|---|---|
| sUper_IL | 0.34 | 1.46 | 0.55 | 1.10 | 0.69 |
| satozaki | 1.06 | 0.70 | 0.67 | 0.80 | 0.73 |
| UEC-IL | 0.93 | 0.87 | 0.90 | 0.95 | 0.91 |
| HondaNLP | 0.95 | 1.17 | 1.50 | 1.18 | 0.75 |
| kanolab | 1.02 | 0.75 | 1.12 | 1.08 | 0.67 |
| GPTaku | 1.58 | 0.75 | 0.86 | 0.82 | 0.56 |
| IS_Lab | 0.91 | 0.79 | 1.00 | 1.15 | 0.90 |
| yuricat | 1.32 | 1.68 | 0.67 | 0.92 | 0.80 |
Human Evaluation (Relative)
Human judges read selected game logs and evaluate the players of each game on each criterion. From 2024 winter onward the judges rank the players (average rank, lower is better); in the 2024 spring and 2024 international contests they gave 5-point scores (higher is better). In 13-player villages each entry (e.g. team-A, team-B) is evaluated separately.
English, 5-player village
審査員 3 名の 5 段階評価（AIWolfDial 2024 概要論文 Table 2） Scores are on a 5-point scale (higher is better).
| # | Team | Natural expression | Contextual dialogue | Logical consistency | Action consistency | Character consistency | Average |
|---|---|---|---|---|---|---|---|
| 1 | UEC-IL | 4.33 | 4.67 | 4.00 | 4.67 | 4.33 | 4.40 |
| 2 | sUper_IL | 4.00 | 4.00 | 4.00 | 4.67 | 4.00 | 4.13 |
| 3 | satozaki | 3.33 | 3.67 | 3.67 | 3.67 | 3.00 | 3.47 |
| 4 | kanolab | 2.67 | 3.33 | 3.00 | 3.33 | 3.67 | 3.20 |
| 5 | Mille | 1.67 | 2.00 | 1.00 | 2.00 | 1.67 | 1.67 |
Japanese, 5-player village
審査員 4 名の 5 段階評価（AIWolfDial 2024 概要論文 Table 1） Scores are on a 5-point scale (higher is better).
| # | Team | Natural expression | Contextual dialogue | Logical consistency | Action consistency | Character consistency | Average |
|---|---|---|---|---|---|---|---|
| 1 | kanolab | 4.33 | 3.67 | 4.00 | 3.67 | 3.67 | 3.87 |
| 2 | UEC-IL | 3.67 | 3.67 | 3.67 | 3.67 | 3.67 | 3.67 |
| 3 | HondaNLP | 4.00 | 3.00 | 4.00 | 3.33 | 3.67 | 3.60 |
| 4 | IS_Lab | 4.00 | 3.67 | 2.67 | 3.00 | 4.33 | 3.53 |
| 5 | sUper_IL | 3.67 | 3.67 | 3.67 | 4.00 | 2.67 | 3.53 |
| 6 | GPTaku | 3.33 | 3.67 | 2.67 | 3.00 | 2.67 | 3.07 |
| 7 | satozaki | 3.67 | 2.67 | 2.67 | 4.00 | 2.33 | 3.07 |
| 8 | Mille | 2.33 | 3.00 | 2.00 | 2.33 | 2.00 | 2.33 |
LLM-as-a-Judge (Relative Evaluation)
LLM judges rank the players of each game on each evaluation criterion; the ranks are averaged per team (lower is better). The judge models differ by contest and are listed under each track. The judge is published as aiwolf-nlp-llm-judge.
LLM-as-a-Judge relative evaluation was not conducted in this contest (it started with the 2025 international contest).
Game Logs
Logs of the main competition. Only games that finished normally are listed.
| Track | Games | Logs |
|---|---|---|
| English, 5-player village | 58 | Log list |
| Japanese, 5-player village | 64 | Log list |
Official Results (Awards)
| Award | Team |
|---|---|
| Best Subjective Evaluation Award (Japanese Track) and Sponsor Award | kanolab |
| Best Subjective Evaluation Award (English Track) | UEC-IL |
| Best Win Rate and Game Action Award | sUper_IL |
| Judge's Special Award: Beginner-like Play | HondaNLP (鳥海審査員) |
| Judge's Special Award: Persuasion as a Werewolf | sUper_IL (原田審査員) |
Results and awards were presented at the AIWolfDial 2024 workshop (September 24, 2024, INLG 2024, Tokyo).
Win Rates
Computed from the logs of all games that finished normally in the main competition. "Overall" is wins divided by games. Per-role cells show the win rate in that role with (wins / games). "Adjusted" re-weights the per-role win rates by the village's role composition, cancelling out uneven role assignments.
English, 5-player village (58 games)
| # | Team | Games | Overall | Villager | Seer | Possessed | Werewolf | Adjusted |
|---|---|---|---|---|---|---|---|---|
| 1 | sUper_IL | 58 | 55.2% | 50.0% (10/20) | 61.5% (8/13) | 50.0% (7/14) | 63.6% (7/11) | 55.0% |
| 2 | kanolab | 58 | 53.4% | 56.5% (13/23) | 42.9% (3/7) | 61.5% (8/13) | 46.7% (7/15) | 52.8% |
| 3 | satozaki | 58 | 51.7% | 47.8% (11/23) | 57.1% (8/14) | 45.5% (5/11) | 60.0% (6/10) | 51.6% |
| 4 | UEC-IL | 58 | 51.7% | 52.4% (11/21) | 50.0% (6/12) | 50.0% (6/12) | 53.8% (7/13) | 51.7% |
| 5 | yuricat | 58 | 37.9% | 44.8% (13/29) | 33.3% (4/12) | 37.5% (3/8) | 22.2% (2/9) | 36.5% |
Japanese, 5-player village (64 games)
| # | Team | Games | Overall | Villager | Seer | Possessed | Werewolf | Adjusted |
|---|---|---|---|---|---|---|---|---|
| 1 | sUper_IL | 40 | 62.5% | 62.5% (10/16) | 50.0% (4/8) | 50.0% (4/8) | 87.5% (7/8) | 62.5% |
| 2 | satozaki | 40 | 60.0% | 37.5% (6/16) | 75.0% (6/8) | 75.0% (6/8) | 75.0% (6/8) | 60.0% |
| 3 | UEC-IL | 40 | 52.5% | 50.0% (8/16) | 37.5% (3/8) | 62.5% (5/8) | 62.5% (5/8) | 52.5% |
| 4 | HondaNLP | 40 | 52.5% | 50.0% (8/16) | 50.0% (4/8) | 75.0% (6/8) | 37.5% (3/8) | 52.5% |
| 5 | kanolab | 40 | 47.5% | 50.0% (8/16) | 25.0% (2/8) | 50.0% (4/8) | 62.5% (5/8) | 47.5% |
| 6 | GPTaku | 40 | 45.0% | 43.8% (7/16) | 37.5% (3/8) | 50.0% (4/8) | 50.0% (4/8) | 45.0% |
| 7 | IS_Lab | 40 | 37.5% | 37.5% (6/16) | 37.5% (3/8) | 25.0% (2/8) | 50.0% (4/8) | 37.5% |
| 8 | yuricat | 40 | 35.0% | 31.2% (5/16) | 50.0% (4/8) | 50.0% (4/8) | 12.5% (1/8) | 35.0% |
Game Metrics
Behavioural metrics computed mechanically from the game logs, independent of dialogue quality. Each value is a ratio against what random play would produce (1.00 = random). "Executed", "divined" and "guarded" show how often the team was targeted relative to chance; lower is better for being executed or divined. "Divination accuracy" and "vote accuracy" show how often the team found a werewolf relative to chance; higher is better. "Possessed avoidance" is the rate at which, as the possessed, the team voted for someone other than a werewolf.
English, 5-player village
| Team | Executed | Divined | Divination accuracy | Vote accuracy | Possessed avoidance |
|---|---|---|---|---|---|
| sUper_IL | 0.67 | 1.14 | 1.44 | 1.05 | 0.67 |
| kanolab | 1.17 | 1.20 | 2.48 | 1.36 | 0.80 |
| satozaki | 1.05 | 0.91 | 0.91 | 1.12 | 0.69 |
| UEC-IL | 0.90 | 0.80 | 0.75 | 0.96 | 0.88 |
| yuricat | 1.25 | 0.95 | 0.80 | 1.10 | 0.44 |
Japanese, 5-player village
| Team | Executed | Divined | Divination accuracy | Vote accuracy | Possessed avoidance |
|---|---|---|---|---|---|
| sUper_IL | 0.34 | 1.46 | 0.55 | 1.10 | 0.69 |
| satozaki | 1.06 | 0.70 | 0.67 | 0.80 | 0.73 |
| UEC-IL | 0.93 | 0.87 | 0.90 | 0.95 | 0.91 |
| HondaNLP | 0.95 | 1.17 | 1.50 | 1.18 | 0.75 |
| kanolab | 1.02 | 0.75 | 1.12 | 1.08 | 0.67 |
| GPTaku | 1.58 | 0.75 | 0.86 | 0.82 | 0.56 |
| IS_Lab | 0.91 | 0.79 | 1.00 | 1.15 | 0.90 |
| yuricat | 1.32 | 1.68 | 0.67 | 0.92 | 0.80 |
Human Evaluation (Relative)
Human judges read selected game logs and evaluate the players of each game on each criterion. From 2024 winter onward the judges rank the players (average rank, lower is better); in the 2024 spring and 2024 international contests they gave 5-point scores (higher is better). In 13-player villages each entry (e.g. team-A, team-B) is evaluated separately.
English, 5-player village
審査員 3 名の 5 段階評価（AIWolfDial 2024 概要論文 Table 2） Scores are on a 5-point scale (higher is better).
| # | Team | Natural expression | Contextual dialogue | Logical consistency | Action consistency | Character consistency | Average |
|---|---|---|---|---|---|---|---|
| 1 | UEC-IL | 4.33 | 4.67 | 4.00 | 4.67 | 4.33 | 4.40 |
| 2 | sUper_IL | 4.00 | 4.00 | 4.00 | 4.67 | 4.00 | 4.13 |
| 3 | satozaki | 3.33 | 3.67 | 3.67 | 3.67 | 3.00 | 3.47 |
| 4 | kanolab | 2.67 | 3.33 | 3.00 | 3.33 | 3.67 | 3.20 |
| 5 | Mille | 1.67 | 2.00 | 1.00 | 2.00 | 1.67 | 1.67 |
Japanese, 5-player village
審査員 4 名の 5 段階評価（AIWolfDial 2024 概要論文 Table 1） Scores are on a 5-point scale (higher is better).
| # | Team | Natural expression | Contextual dialogue | Logical consistency | Action consistency | Character consistency | Average |
|---|---|---|---|---|---|---|---|
| 1 | kanolab | 4.33 | 3.67 | 4.00 | 3.67 | 3.67 | 3.87 |
| 2 | UEC-IL | 3.67 | 3.67 | 3.67 | 3.67 | 3.67 | 3.67 |
| 3 | HondaNLP | 4.00 | 3.00 | 4.00 | 3.33 | 3.67 | 3.60 |
| 4 | IS_Lab | 4.00 | 3.67 | 2.67 | 3.00 | 4.33 | 3.53 |
| 5 | sUper_IL | 3.67 | 3.67 | 3.67 | 4.00 | 2.67 | 3.53 |
| 6 | GPTaku | 3.33 | 3.67 | 2.67 | 3.00 | 2.67 | 3.07 |
| 7 | satozaki | 3.67 | 2.67 | 2.67 | 4.00 | 2.33 | 3.07 |
| 8 | Mille | 2.33 | 3.00 | 2.00 | 2.33 | 2.00 | 2.33 |
LLM-as-a-Judge (Relative Evaluation)
LLM judges rank the players of each game on each evaluation criterion; the ranks are averaged per team (lower is better). The judge models differ by contest and are listed under each track. The judge is published as aiwolf-nlp-llm-judge.
LLM-as-a-Judge relative evaluation was not conducted in this contest (it started with the 2025 international contest).
Game Logs
Logs of the main competition. Only games that finished normally are listed.
| Track | Games | Logs |
|---|---|---|
| English, 5-player village | 58 | Log list |
| Japanese, 5-player village | 64 | Log list |
How to view logs
Each “Log list” link opens a listing of the logs (.log files) of games that finished normally. A file name starts with the game start time, followed by the names of the participating teams.
- View in the browser: press “▶ 開く” (open) in the “viewerで見る” (view in viewer) column of the row you want to read. aiwolf-nlp-viewer opens with that log loaded.
- Save a file: press “⬇ 保存” (save) in the “ダウンロード” (download) column to download the log file. Clicking the file name shows the raw text of the log instead.
- Save a whole folder: go one level up via “Parent directory/” and press “⬇ zip” in the row of the folder (e.g.
success). All files in the folder are zipped in your browser and downloaded (a dialog confirms the number of files).
How to use the viewer is described in the “Log viewer” section of the Links page.