Have a doctor examine it. One thing is for sure, your belly button should not resemble a slightly open or scabbed over wound and it certainly shouldn't ooze puss if you pick the scab, (or whatever it is you are describing.) If your belly button is naturally deep it may just be hard to keep clean and could be a build up of dirt and oil. If it is actually puss, then you have an infection. Only a doctor can determine whether or not you have a more significant problem.