The ground state is the lowest energy state for a given system (and atoms are an example of a system). Since it is the lowest energy state, systems are generally more likely to be in the ground state than any other state and when they are in excited states, they will often decay back to the ground state.
However, when the energy difference between the ground state and the excited states is small compared to the temperature, then you would expect the system to be found in some mixture of the ground state and the excited states. There are also some excited states that are so long lived (some can be live longer than the age of the universe) that they will practically never decay back to the ground state. So you can find situations where you don't find systems in their ground state in nature. In fact, the only way to ensure a system will be in its ground state is to cool the system to absolute zero, which is impossible.