I've just copied the pertinent parts of my owners' manual into a Word doc and noted that the combination of the gross trailer weight added to the total weight of my Sienna, occupants and vehicle cargo must not exceed a total of 7800 lbs.
That's the GCWR

... but the 7800 pound value is low for a Sienna. Do you have the first generation (1998-2003) or the later (and rare) 4-cylinder, or did you find the GCWR without the towing package? My 2004, which is typical, has about 900 pounds higher GCWR.
At 7800 pounds, 4300 pounds of van plus 3500 pounds of trailer would hit the GCWR limit, without passengers or cargo in the van.
The figures seem to indicate that, as long as I don't overload it, I might even use my Sienna to tow a 19' Escape, which may be a better fit for me in the long run. Do you think this would be pushing the edge of the envelope too far?
That's likely too much trailer (assuming you have some options on the trailer and are carrying some stuff) with a 7800 lb GCWR. With my Sienna and its higher GCWR, I considered a 19', but it would require constraint in options and stuff carried in the trailer (to keep it under 3500 pounds loaded), and tongue weight is an issue...
Most 3500 pound trailer weights also specify 350 pounds tongue weight. The average tongue weight of 9 Escape 17's is 344 pounds with a peak of 500.
For a 19, the average of 12 first generation models is 373 with a peak of 480, 4 2nd generation 19's average 495 pounds of tongue weight with a peak of 650.
Another issue for me is the hitch weight limit, because I don't want a weight-distributing hitch... and can't use one with my receiver. The weight-carrying (non-WD) hitch weight limit is 350 pounds (10% of 3500; for most and perhaps all years of Sienna), but if you do use WD, the limit is 525 pounds (15% of 3500). With WD, the hitch weight limit is high enough for any reasonable 3500 pound trailer.
I have friends that tow a Scamp 16 with a 2011 Sienna van & it works fine with two exceptions. The hitch is so low that it often hits on steep entrances, and they have ended up spinning the front wheels trying to start on a steep gravel road. They did have to switch to a detachable tongue jack to be able to open the rear door.
The Sienna hitch is low (because it is under the bumper rather than in it, and because the bumper is low because the floor is low for easy loading and maximum interior height). Some hitch receivers fit better than others, and one reason that I like my Toyota accessory (made by Reese) 1.25" Class 2 receiver is that the bottom of it is higher than other designs. Air bags added to the rear springs prevent sagging, and loss of clearance. I never had a dragging problem, but I used air bags.
Traction can be an issue with any 2WD vehicle. The van starts quite front-heavy, so even with a trailer is probably has as much traction as a 2WD pickup, but uphill transfers load rearward and makes traction worse for front wheel drive. It hasn't been a problem for me, but it can be in some campgrounds.
I had no issue with the tongue jack and hatch, even with a short drawbar, but trailers vary (and mine is low and doesn't have a power jack head).