Share Reading Reading "ACL17 DeepSRL"

A new srl model, 4 layer bi-lstm + highway + RNN dropout.

Model

1. applying recent advances in training deep recurrent neural networks such as highway connections (Srivastava et al., 2015)
2. RNN-dropouts (Gal and Ghahramani, 2016).
3. using an A* decoding algorithm (Lewis and Steedman, 2014; Lee et al., 2016) to enforce structural consistency at prediction time without adding more complexity to the training process.

Analysis is complete

• What is the model good at and what kinds of mistakes does it make?
• How well do LSTMs model global structural consistency, despite conditionally indepen- dent tagging decisions?
• Is our model implicitly learning syntax, and could explicitly modeling syntax still help?