Leaderboard

X-CODAH Results

Model	Participant	Date	AVG	en	de	it	es	fr	nl	ru	vi	zh	hi	pl	ar	ja	pt	sw	ur
TRT	Microsoft Cognitive Services Research yuwfan@microsoft.com 2021-10-15	10/21'	61.9	69.1	65.3	62.5	64.4	64.3	64.5	61.8	64.6	63.3	57.1	62.7	57.6	61.6	64.3	52.5	55.1
MCP(RL)	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	58.3	69.9	60.7	61.9	60.7	61.4	60.7	58.6	62.3	61.9	53.7	59.0	54.1	54.7	60.8	44.6	48.0
XLMR-L	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	56.0	66.4	59.6	59.9	60.9	60.1	59.3	56.3	57.4	57.3	49.1	57.5	51.2	53.8	58.2	42.2	46.6
MCP(RB)	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	43.6	52.2	47.6	46.2	44.4	48.1	44.8	42.9	43.2	45.7	37.8	41.8	41.8	42.9	44.7	37.2	36.4
XLMR-B	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	42.4	50.1	45.8	44.4	44.2	45.2	42.0	44.1	43.2	44.6	38.1	41.9	37.8	42.0	44.1	35.6	34.6
mBERT	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	33.2	42.9	33.1	33.5	33.8	35.2	33.7	31.9	22.8	38.0	26.5	31.0	34.8	34.0	37.2	30.8	31.5
XLM-100	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	30.4	42.7	31.5	32.2	30.7	34.9	32.6	30.9	24.7	31.4	26.8	27.0	30.0	27.4	33.2	25.3	24.9

X-CSQA Results

Model	Participant	Date	AVG	en	de	it	es	fr	nl	ru	vi	zh	hi	pl	ar	ja	pt	sw	ur
TRT	Microsoft Cognitive Services Research yuwfan@microsoft.com 2021-10-15	10/21'	59.8	71.0	61.2	63.0	65.1	65.1	62.8	57.8	58.9	56.3	56.1	59.4	56.2	54.7	64.6	51.0	53.9
MCP(RL)	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	56.5	69.5	59.3	60.3	61.4	60.0	61.1	57.5	55.7	56.7	51.3	56.1	52.3	50.2	60.7	43.3	48.8
XLMR-L	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	53.8	66.7	56.1	58.2	59.5	60.3	56.8	52.1	51.4	52.7	48.7	53.9	48.4	50.0	59.9	41.6	45.2
MCP(RB)	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	41.9	52.1	46.2	45.6	44.3	44.7	45.3	42.8	45.3	44.3	36.8	41.4	36.8	37.5	44.9	28.1	33.4
XLMR-B	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	40.6	51.5	44.1	42.1	44.8	44.0	43.3	39.5	42.6	40.6	34.6	40.2	38.4	37.5	43.4	29.6	33.0
mBERT	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	30.4	38.8	29.6	36.4	35.3	33.8	32.6	32.7	22.2	37.8	21.1	27.2	27.7	31.4	34.1	21.8	23.7
XLM-100	USC-INK yuchen.lin@usc.edu 2021-06-01	06/21'	26.7	34.3	26.7	28.5	29.3	28.3	27.2	29.9	21.1	28.6	22.1	26.6	26.3	25.1	30.9	20.1	21.7

Enlarge the Tables

Submission Instruction

If you have a model for solving X-CSR and would like to make a submission, you can create a submission and evaluate your model for the X-CSR dataset.

Submission Format

Please submit a .json file follows the format below

{
    "en": {"be11daf79ced159b": "A", "8abe4f22cf8b2839": "B",..., "7a7bb4af46f9e09c": "C"},
    "fr": {"be11daf79ced159b": "A", "8abe4f22cf8b2839": "B",..., "16b2a162f9656b79": "C"},
    "it": {"be11daf79ced159b": "A", "8abe4f22cf8b2839": "B",..., "34bb7281c8e007f9": "C"},
    ...,
    "jap": {"be11daf79ced159b": "A", "8abe4f22cf8b2839": "B",..., "16b2a162f9656b79": "C"}
}

Your submission will be reviewed by the X-CSR team and added to the leaderboard.