FIXME find params that work for Lamb
64 (optim.AccSGD, {'lr': 0.015}, 800),
65 (build_lookahead, {'lr': 1.0}, 500),
66 (optim.QHAdam, {'lr': 1.0}, 500),
67 # FIXME find params that work for Lamb 68 # (optim.Lamb, {'lr': 0.1, 'betas': (0.9, 0.95)}, 900),
69 # (optim.Lamb, {'lr': 0.1, 'betas': (0.9, 0.95), 'debias': True,
70 # 'adamd_bias_correction': True}, 900),
Description
You have marked this block as FIXME/XXX/TODO. Please make sure this is addressed, or remove this comment as this could be misleading.