I think you can get away with just being a 'group' instead of calling yourself part of it, if you're doing your best to distinguish yourself from what they are.

The Yakuza don't exactly have a good reputation, after all. At the end of the day you can all just be friends.

[ he'll just pat him on the back considering he's about to cry again. what a crybaby this man is. ]