Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CLOSED] Add a two-class variant of the MNLI task #976

Closed
jeswan opened this issue Sep 17, 2020 · 0 comments
Closed

[CLOSED] Add a two-class variant of the MNLI task #976

jeswan opened this issue Sep 17, 2020 · 0 comments
Labels
low-priority Only if you're bored. Ask Sam/Ian/Alex before starting. new-task Adding support for a new task.

Comments

@jeswan
Copy link
Contributor

jeswan commented Sep 17, 2020

Issue by sleepinyourhat
Monday Dec 16, 2019 at 20:11 GMT
Originally opened as nyu-mll/jiant#976

This adds the option to train a model on the three-class MNLI dataset and evaluate it on comparable two-class datasets like RTE or HANS. It does this by overriding the usual accuracy metric with an NLI-specific accuracy metric that collapses 'neutral' and 'contradiction'.

sleepinyourhat included the following code: https://github.com/nyu-mll/jiant/pull/976/commits
@jeswan jeswan added low-priority Only if you're bored. Ask Sam/Ian/Alex before starting. new-task Adding support for a new task. labels Sep 17, 2020
@jeswan jeswan closed this as completed Sep 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
low-priority Only if you're bored. Ask Sam/Ian/Alex before starting. new-task Adding support for a new task.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jeswan