This really couldn't be simpler (Mooseburger and Squizzy I am talking to you).
Head out through Sebastopol or Buninyong and turn towards Napoleons. Drive for an eternity (well about 20k) towards Enfield. The turn off to Misery Creek Picnic Ground is called Misery Creek Road. See the map below for the last bit of the trip.
As usual, click on the map to enlarge it.