adamw

AdamW. Understanding AdamW: Weight decay or L2 regularization? Implementing AdamW; Results of AdamW experiments: does it work? amsgrad. Understanding amsgrad; Implementing amsgrad; Results of amsgrad experiments: a lot of noise for nothing; Appendix: Full results slot demo saldo jutaan