This paper provides a simple but effective approach to speeding up an integrated DRL and ES search, incorporating a novel application of previously proposed asynchronous updating rules and presenting experimental results that convincingly show the efficacy of the approach. The paper could use a better justification for its "greater stability" claim, could benefit from some comparison against competing asynchronous update rules, and has a few miscellaneous presentation problems that need to be addressed. But overall this paper makes a solid contribution and the recommendation is to accept. Please take care to address the reviewer comments as you prepare your final version.