I like using my 105mm APO on double stars over my larger scopes just because it gives nicer images even when the seeing is below average. I still get nice round stars but the whole image kind of bounces almost like my clock drive is making it bump up and down a little.
An 8" scope will be affected 4 times as much as a 4" on the same night, so in this case aperture doesn't rule in below average seeing.
If I am going specifically after doubles, I tend towards my 10 inch Dob rather than any of my refractors. One reason is for it to perform it's rock solid best, I need to set out up before sunset and run the fans for considerably more than an hour.
If the seeing is on the good side and I started with a 4 or 5 inch, I'm stuck.
For closer doubles, maybe 1.4" or closer, the 10 inch provides wider splits under most circumstances because it's Airy disk is so much smaller. A 1.14" double is a Dawes limit split in a 4 inch, in the 10 inch it's a wide split and I'm not fighting both the large Airy disk and the seeing.
I'm not sure about your factor of 4 in terms of the effect on the image. I am not looking so much for a pretty image, I'm looking for closer splits. Antares is usually a challenging split in a 4 or 5 inch but last year I made the split in my 22 inch with Antares quite low on the horizon, no way in a small scope.
It wasn't pretty but it was very wide, bright and apparent. It made me realize just how small the airy disks are in a scope that size.
With any scope but a larger scope in particular, the outer seeing aberrated region can as an extended object while it surrounds the region where the airy disk is brighter. Cranking up the magnification can increase the contrast by dimming that outer region.