On the overlooked pitfalls of weight decay and how to mitigate them a gradient norm perspective. Additional information example for students.

Inveeg split. Hudor Bauang.