I can't speak for the other groups but I know for a fact this isn't true, do you mind explaining where you got this idea? I say this because I'm in SPEW and we're doing just fine.
Join UK "anarchist federation"
I used to be tangentially involved in Socialist Alliance/TUSC (although I was in the SWP) in Scotland, after the Scottish section fell to bits I had conversations with their former organiser who seemed utterly despondent in terms of their national membership.
SPEW's definitely bigger than the SWP, but given the SWP has (maybe) under 1,500 active members (by their own records) that's not hard. Certainly in terms of Militant, SPEW's a shadow of it's former self, and Socialist Appeal is a joke.
I'm not attacking SPEW or anything, if anything I'm depressed by the state of the British Trotskyist movement at present.
there's your problem
SPEW has something in the region of 3000 members at the moment and growing. Yeah compared to Militant its not close to what it was, but its by no means falling apart or anything.
Keep in mind that engaging in social relations outside of the context of the imposed society is very insurrectionary in of itself.