Yes make reservations. The camp sites in the park have no power and towing in the park can be a challenge. We usually camp outside the park and sightsee in the park during the day in our car/truck.
On the west side of the park the KOA at Luray is nice. Andy Guest State Park is also near Luray and a very popular SP. Near the center of the park on the east side there is a private CG in Standardsville which is OK. Near the south end of the Park just off the BRP is Lake Scherando NFCG they have power at some of their sites. Lake Scherando is fairly convenient to the BRP and Waynesboro for supplies shopping and gas.
If there on the October weekends you may want visit the Graves Mountain Apple Harvest Festivals below the east side of the park near the center. They also have a big buffet in the lodge on Sat. night. Apple Harvest :: Graves Mountain Lodge
Make sure you have a full tank of gas before going up into the park. There are places to buy gas in the park but they are never near when needed.