Hi! I have exactly the same issue. I can easily do 4 PMO sessions per day, but when it comes to real sex, I can only do about one session per day. If I try to do more intercourse sessions, I will probably not reach orgasm even if I manage to get another erection.
But my starting situation was even worse, and I do feel that rebooting has helped me. It is definitely easier to reach orgasm if I abstain from masturbation. The longer I abstain, the easier it gets. I am hoping that by continuing my reboot (it's been over a year already with some relapses) my refractory period will continue to decrease.
You mentioned the possibility of a physical defect, so I wanted to ask, are you circumcised or not? I was circumcised as an adult and I feel that it has decreased my sensitivity quite a bit. But I was unlucky to get quite a radical circumcision where most of my inner skin was removed.