Here's a little curve ball for the argument: has everyone forgotten that much of the SMiLE sessions were never meant to be heard by all of us? That it took some sneaky, illegal, and perhaps unethical (to the artists) acts to make it possible for us to hear them?
20/20 LP = Illegal
Surf's Up LP = Unethical
GV CD Box = Sneaky
P.S. - I realize you said MUCH of them, but I guess an argument could be made that Brian didn't want any of it heard other than what was on Smiley Smile, but he was overruled, cajoled etc... But if you are referring to bootleggers being the culprit of unleashing the Smile music against everyone's wishes...i think there is solid evidence that Dennis Wilson was the orig. Smile bootlegger, and his transfer is what made up the first Smile bootlegs, which leaked to friends and then in multi generation loss form to the public on cassette circa 1978/1979 or so.