I think it is more than just one small wrong step. When you are so unstable, that every small wrong step lets you slip, then there is a bigger problem in my opinion.I feel you, man. What you wrote there is something I would write. Things go well and then BAM! One little thing happens and everything collapses. It's the small rock that overturns the big cart. I've been thinking, maybe it's something more than just one small wrong step.
I think what's really important is a) not sitting around getting bored, but to use your time to build a new, healthy lifestyle and to keep oneself occupied and b) building up negative emotions towards P, the loosers it makes people and the sex industry in general. That is stabilizing you, so one small wrong step won't let you slip.
I tell it like everyone and I don't know how beneficial it is for overs, but for me it was sooo helpfull: In the first two months I read nearly the whole partners section of this forum. Reading from all those women about their addicted husbands/boyfriends helped me to get a view from another perspective on what loosers we addicts become, how we hurt our partners with our behaviour and how laughable it is, to rather sit alone and beat your meat to this pervert and disgusting shit than engage with real women. And this realization keeps me on track, because I never want to go back and be this person I have to be ashamed of.