Brian brought so many musical influences including jazz and (especially) classical into his beautifully symphonic work, Smile.
I know there is no way to answer this, but do you think he would have built on this direction had Smile been released to at least moderate success? It's interesting that in the mainstream a lot of bands namingly The Beatles retreated from more of an art rock approach of 1967 to a back-to-the-basics Rock sound in 1968.
I've always felt Brian was still at a point of untapped potential and could have been like a George Gershwin of pop music. I believe he could have brought us some absolutely amazing things beyond what we would ever dream of for pop/rock music. He viewed music so broadly and wasn't confined by conventional pop music constructs. I think Pet Sounds and Smile are unparalleled. He may have continued in that direction of Smile had worked and I think it would have been mesmerizing.
He is!!
...As for the question, that's the same what I've had for a long time. I think, if Brian have finished Smile, he would have gotten into experimental music much more deeply, and soon would have broken down.