Link Search Menu Expand Document

Leaderboard


X-CODAH Results

Model Participant Date AVG en de it es fr nl ru vi zh hi pl ar ja pt sw ur
MCP + KG Anonymous
2021-09-20
09/21' 63.7 68.9 66.4 63.7 66.5 66.6 64.6 64.0 66.9 65.4 61.9 64.1 62.1 60.2 66.2 55.5 57.0
MCP(RL) USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 58.3 69.9 60.7 61.9 60.7 61.4 60.7 58.6 62.3 61.9 53.7 59.0 54.1 54.7 60.8 44.6 48.0
XLMR-L USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 56.0 66.4 59.6 59.9 60.9 60.1 59.3 56.3 57.4 57.3 49.1 57.5 51.2 53.8 58.2 42.2 46.6
MCP(RB) USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 43.6 52.2 47.6 46.2 44.4 48.1 44.8 42.9 43.2 45.7 37.8 41.8 41.8 42.9 44.7 37.2 36.4
XLMR-B USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 42.4 50.1 45.8 44.4 44.2 45.2 42.0 44.1 43.2 44.6 38.1 41.9 37.8 42.0 44.1 35.6 34.6
mBERT USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 33.2 42.9 33.1 33.5 33.8 35.2 33.7 31.9 22.8 38.0 26.5 31.0 34.8 34.0 37.2 30.8 31.5
XLM-100 USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 30.4 42.7 31.5 32.2 30.7 34.9 32.6 30.9 24.7 31.4 26.8 27.0 30.0 27.4 33.2 25.3 24.9

X-CSQA Results

Model Participant Date AVG en de it es fr nl ru vi zh hi pl ar ja pt sw ur
MCP + KG Anonymous
2021-09-20
09/21' 59.8 68.2 62.1 61.6 64.4 64.6 61.9 58.2 60.7 58.7 55.5 60.8 56.4 56.2 63.2 51.2 53.3
MCP(RL) USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 56.5 69.5 59.3 60.3 61.4 60.0 61.1 57.5 55.7 56.7 51.3 56.1 52.3 50.2 60.7 43.3 48.8
XLMR-L USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 53.8 66.7 56.1 58.2 59.5 60.3 56.8 52.1 51.4 52.7 48.7 53.9 48.4 50.0 59.9 41.6 45.2
MCP(RB) USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 41.9 52.1 46.2 45.6 44.3 44.7 45.3 42.8 45.3 44.3 36.8 41.4 36.8 37.5 44.9 28.1 33.4
XLMR-B USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 40.6 51.5 44.1 42.1 44.8 44.0 43.3 39.5 42.6 40.6 34.6 40.2 38.4 37.5 43.4 29.6 33.0
mBERT USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 30.4 38.8 29.6 36.4 35.3 33.8 32.6 32.7 22.2 37.8 21.1 27.2 27.7 31.4 34.1 21.8 23.7
XLM-100 USC-INK yuchen.lin@usc.edu
2021-06-01
06/21' 26.7 34.3 26.7 28.5 29.3 28.3 27.2 29.9 21.1 28.6 22.1 26.6 26.3 25.1 30.9 20.1 21.7

Enlarge the Tables

Submission Instruction

If you have a model for solving X-CSR and would like to make a submission, you can create a submission and evaluate your model for the X-CSR dataset.

Submission Format

Please submit a .json file follows the format below

{
    "en": {"be11daf79ced159b": "A", "8abe4f22cf8b2839": "B",..., "7a7bb4af46f9e09c": "C"},
    "fr": {"be11daf79ced159b": "A", "8abe4f22cf8b2839": "B",..., "16b2a162f9656b79": "C"},
    "it": {"be11daf79ced159b": "A", "8abe4f22cf8b2839": "B",..., "34bb7281c8e007f9": "C"},
    ...,
    "jap": {"be11daf79ced159b": "A", "8abe4f22cf8b2839": "B",..., "16b2a162f9656b79": "C"}
}

Your submission will be reviewed by the X-CSR team and added to the leaderboard.