Dude, Adam was a Muslim - Noah was a Muslim, - Abraham was a Muslim, Jesus was a Muslim.
Muslim = Mu-Islam
Mu = "ing" form for the verb. eg: run + ing
similarly, Mu-Islam = Islam + ing
Well what Islam Means - it means "Submit will to God"
Mu-Islam means "SubmitING will to God"
as you can see, Ishmael was NOT the only muslim but all the prophets.
You are surely confused between the Arabs and Muslims.
Probably, Ishmel was the father of Arabs could be more logical - like Abraham was the Father of Jews and Muslims and Christians alike.