As long as your garden is a decent size, has appropriate grass and lack of poisonous plants, and is safely fenced with no debris in the lot I see nothing wrong with it.
However, the most important thing is that you need to check with your local zoning laws and rules.