Skip to content

Instable learning with SemiMarkov CRF #117

Open
@urchade

Description

@urchade

HI,

First, thank you for fixing #110 (@da03), the SemiCRF works better now, I was able to get good results on span extraction tasks. However, I still encounter a learning instability where the loss (neg logprob) gets negative after several steps (and the accuracy starts to drop). The same problem occurs with batch_size = 1. Below I put the learning curve (f1_score and log loss).

(Maybe the bug comes from the masking of spans where (length, length + span_with) and length + span_with > length, but I am not sure.)

Edit: I created a test and it seems that the masking is good. Maybe the log_prob computation or the to_parts function ?

train_loss
score

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions