Types of gradient-free optimizers

[@pnachtwey]

I like Nelder-Mead. Often REAL data is noisy so that computing an accurate gradient is impossible. You forgot Powell’s method.