| Model | Participant | Date | AVG | en | de | it | es | fr | nl | ru | vi | zh | hi | pl | ar | ja | pt | sw | ur |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MCP + KG | Anonymous 2021-09-20 |
09/21' | 63.7 | 68.9 | 66.4 | 63.7 | 66.5 | 66.6 | 64.6 | 64.0 | 66.9 | 65.4 | 61.9 | 64.1 | 62.1 | 60.2 | 66.2 | 55.5 | 57.0 |
| TRT | Microsoft Cognitive Services Research yuwfan@microsoft.com 2021-10-15 |
10/21' | 61.9 | 69.1 | 65.3 | 62.5 | 64.4 | 64.3 | 64.5 | 61.8 | 64.6 | 63.3 | 57.1 | 62.7 | 57.6 | 61.6 | 64.3 | 52.5 | 55.1 |
| MCP(RL) | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 58.3 | 69.9 | 60.7 | 61.9 | 60.7 | 61.4 | 60.7 | 58.6 | 62.3 | 61.9 | 53.7 | 59.0 | 54.1 | 54.7 | 60.8 | 44.6 | 48.0 |
| XLMR-L | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 56.0 | 66.4 | 59.6 | 59.9 | 60.9 | 60.1 | 59.3 | 56.3 | 57.4 | 57.3 | 49.1 | 57.5 | 51.2 | 53.8 | 58.2 | 42.2 | 46.6 |
| MCP(RB) | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 43.6 | 52.2 | 47.6 | 46.2 | 44.4 | 48.1 | 44.8 | 42.9 | 43.2 | 45.7 | 37.8 | 41.8 | 41.8 | 42.9 | 44.7 | 37.2 | 36.4 |
| XLMR-B | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 42.4 | 50.1 | 45.8 | 44.4 | 44.2 | 45.2 | 42.0 | 44.1 | 43.2 | 44.6 | 38.1 | 41.9 | 37.8 | 42.0 | 44.1 | 35.6 | 34.6 |
| mBERT | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 33.2 | 42.9 | 33.1 | 33.5 | 33.8 | 35.2 | 33.7 | 31.9 | 22.8 | 38.0 | 26.5 | 31.0 | 34.8 | 34.0 | 37.2 | 30.8 | 31.5 |
| XLM-100 | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 30.4 | 42.7 | 31.5 | 32.2 | 30.7 | 34.9 | 32.6 | 30.9 | 24.7 | 31.4 | 26.8 | 27.0 | 30.0 | 27.4 | 33.2 | 25.3 | 24.9 |
| Model | Participant | Date | AVG | en | de | it | es | fr | nl | ru | vi | zh | hi | pl | ar | ja | pt | sw | ur |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MCP + KG | Anonymous 2021-09-20 |
09/21' | 59.8 | 68.2 | 62.1 | 61.6 | 64.4 | 64.6 | 61.9 | 58.2 | 60.7 | 58.7 | 55.5 | 60.8 | 56.4 | 56.2 | 63.2 | 51.2 | 53.3 |
| TRT | Microsoft Cognitive Services Research yuwfan@microsoft.com 2021-10-15 |
10/21' | 59.8 | 71.0 | 61.2 | 63.0 | 65.1 | 65.1 | 62.8 | 57.8 | 58.9 | 56.3 | 56.1 | 59.4 | 56.2 | 54.7 | 64.6 | 51.0 | 53.9 |
| MCP(RL) | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 56.5 | 69.5 | 59.3 | 60.3 | 61.4 | 60.0 | 61.1 | 57.5 | 55.7 | 56.7 | 51.3 | 56.1 | 52.3 | 50.2 | 60.7 | 43.3 | 48.8 |
| XLMR-L | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 53.8 | 66.7 | 56.1 | 58.2 | 59.5 | 60.3 | 56.8 | 52.1 | 51.4 | 52.7 | 48.7 | 53.9 | 48.4 | 50.0 | 59.9 | 41.6 | 45.2 |
| MCP(RB) | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 41.9 | 52.1 | 46.2 | 45.6 | 44.3 | 44.7 | 45.3 | 42.8 | 45.3 | 44.3 | 36.8 | 41.4 | 36.8 | 37.5 | 44.9 | 28.1 | 33.4 |
| XLMR-B | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 40.6 | 51.5 | 44.1 | 42.1 | 44.8 | 44.0 | 43.3 | 39.5 | 42.6 | 40.6 | 34.6 | 40.2 | 38.4 | 37.5 | 43.4 | 29.6 | 33.0 |
| mBERT | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 30.4 | 38.8 | 29.6 | 36.4 | 35.3 | 33.8 | 32.6 | 32.7 | 22.2 | 37.8 | 21.1 | 27.2 | 27.7 | 31.4 | 34.1 | 21.8 | 23.7 |
| XLM-100 | USC-INK yuchen.lin@usc.edu 2020-06-01 |
06/21' | 26.7 | 34.3 | 26.7 | 28.5 | 29.3 | 28.3 | 27.2 | 29.9 | 21.1 | 28.6 | 22.1 | 26.6 | 26.3 | 25.1 | 30.9 | 20.1 | 21.7 |