Sound like this guy is being straight up with you, he dont wanna commit, so you know where you stand!
If you are willing to free up sex to him then of course hes gonna take it.
You need to tell him (and be firm) you want more.. if he cant deliver them allow it.