Does the growth have any particular shape? Also, about how large would you describe it as? If it were cancer, most growths/lumps/bumps are harder and don't move as much. Of course you can always see a doctor and have them check to make sure. For example, with breast lumps, a lump for concern is usually very hard. From what you said, it souns like it would NOT be cancer, but still good to see a doctor to rule it out for sure.
Also, have you looked into hernias? Just wondering since I realized I have two and have to get them repaired soon. I've been hearing a lot more about them recently here on the boards. A hernia will often flatten out when you lay flat and it's common for them to bulge out sometimes and go back in other times. You can be born with them (which is likely my case) or you can develop them from strenuous activity. They can be painful, but if they are not bad, you often won't feel them. If anything, you might feel a heaviness in the area. Here are some ways to tell if it can be a hernia- cough and see if the lump slightly bulges as you cough. That's one sign. If the hernia is reducible, this means you can gently push it back in and it kind of goes away. Definitely don't press on it a lot to avoid irritation, but maybe you can try very gently just to get an idea.
It's also possible to have cysts. I'm not sure how large what you feel exactly is, but cysts can vary in size. They can also cause pain or no pain. If they cause trouble, you can look into having them drained.