I agree with the stigma, less so with the fear of legal consequences. If people think that a law is just, the majority won't break it in the first place. If they think it is unjust, they will just flaunt it and come to hold the law itself in contempt (see: Prohibition).
I'd much prefer a situation in which weed were legal, but regulated and perhaps looked down upon, rather than our current situation, in which it serves as a gateway to illegality and lets you come into contact with dealers that have you shove marshmallows up your twat.
The current illegality of weed really only benefits criminals who profit from the black market. We need to drain this fucking swamp by making it legal and, orthogonally to that, we need to delegitimize this whole lol dude weed lmao stoner culture. However, I believe that law enforcement is the fundamentally wrong tool for that.
That's certainly true, but law enforcement can't replace community cohesion. Rebuilding our communities and cleaning out the rot of the boombers and the stoners is a monumental task, but at least each individual and each family can get to work on it - at any rate, it's far easier than getting laws passed.
