Source: Deep Reinforcement Learning Doesn’t Work Yet, by Alex Irpan
here are some of the failure cases of deep RL.
Deep Reinforcement Learning Can Be Horribly Sample Inefficient
…
If You Just Care About Final Performance, Many Problems are Better Solved by Other Methods
…
Reinforcement Learning Usually Requires a Reward Function
…
Reward Function Design is Difficult
…
Even Given a Good Reward, Local Optima Can Be Hard To Escape
…
Even When Deep RL Works, It May Just Be Overfitting to Weird Patterns In the Environment
…
Even Ignoring Generalization Issues, The Final Results Can be Unstable and Hard to Reproduce
…
The way I see it, either deep RL is still a research topic that isn’t robust enough for widespread use, or it’s usable and the people who’ve gotten it to work aren’t publicizing it. I think the former is more likely.
…
My feelings are best summarized by a mindset Andrew Ng mentioned in his Nuts and Bolts of Applying Deep Learning talk – a lot of short-term pessimism, balanced by even more long-term optimism.