The drive between Whistler/Pemberton and Lillooet, as noted, has some very steep sections between Pemberton and Lillooet called the Duffy Lake Road. From my experience the steepest section is going south down into Pemberton area. Going north down into Lillooet, although steep, is not as bad. The terrain through the summit of the Duffy Lake area is not bad at all. The scenery is spectacular especially in late May when there is still a lot of snow in the pass. The Duffy Lake route is a winding highway but who is in a hurry when camping and you are busy craning your neck to see the mountains?
Years ago we had the brakes smoking in our old 1989 Ford Aerostar coming south down into Pemberton loaded to the hilt with camping gear pulling a small utility trailer with bikes.
Doing a circle loop from Chilliwack to Whistler to Lillooet and back south via the Fraser River Canyon to Chilliwack would be a great drive during which you would experience several climatic and vegetation zones and of course the mountains. Rainforest like at Squamish to near desert conditions at Lillooet. Sea to Sky it truly is. You could have a great trip doing this circle route in 5 days or so. Total distance about 685 km (425 miles) via Cache Creek. Alternate route from Lillooet is 575 km (357miles) via Hwy 12 to Lytton.
Several Provincial and private camp grounds along the way:
Porteau Cove along Howe Sound already mentioned. About a 1.5 hour drive from Chilliwack area. About 140 km from Chilliwack or 20 km south of Squamish. Beautiful site along Howe Sound between Vancouver and Squamish. Ice-breaker’s photo tells it all. (Except for the occasional train that nearly comes through the park!! Don’t let that stop you from camping there though.) Reservation highly recommended, especially on a weekend.
Alice Lake Provincial Park 13 km north of Squamish.
Whistler RV Park and Campground about 40 km north of Squamish or 20 km south of Whistler.
Riverside Resort campground in Whistler.
Nairn Falls Provincial campground about 2 km south of Pemberton or 31 km north of Whistler.
There are some BC Forestry campground sites along the Duffy Lake section. Minimal facilities offered at these sites. Test out your boon-docking capability at these sites!
Not sure what Lillooet has to offer in regard to campgrounds.
There are two options heading out of Lillooet to complete the loop south. Highway 99 heading northeast joining highway 97 and Trans Canada Hwy 1 (TC1) at Cache Creek. Alternate route is Hwy 12 headed south from Lillooet to Lytton and TC 1. This is a shorter route. It is a coin toss as to which route has the best scenery!!
Marble Canyon Provincial campground. On Highway 99 about 48 km northeast of Lillooet and 40 km northwest of Cache Creek.
Juniper Beach Provincial campground – a side trip about 19 km east of Cache Creek on the Thompson River and TC1. Get up close with the two major train routes, CN & CP. They will definitely wake you up during the night here! Lost count of them during the night after about 7 or so.
Skihist Provincial campground about 6 km northeast of Lytton. A gem of a camp site in the Thompson/Fraser River Canyon area. Once again you can listen to both the CN and CP trains as they travel the Fraser and Thompson River routes but they are much lower down in the river canyon and therefore quieter.
Link to BC Provincial Campgrounds - BC Parks - Province of British Columbia
The experience will be quite different from your Banff/Jasper Canadian Rockies trip.
Just do it.