I don't think it's fair to tar the other brothers for what one or another did. The brothers haven't been on speaking terms for years. It's also not clear who did what. Blondie now says Stan threatened to beat him up, but Stebbins said that Steve called Blondie the n word and "a fight broke out." It can't be both.
You can say that Steve was only there because his brother was a Beach Boy. On the other hand, no one blames Carl for the fact that his brother offered drugs to his nieces.