Can an Atheists explain how they use different ways of reasoning then religious people?

All Atheists claim that they are identified by not believing in God. They argue that there is no evidence to given credence to the idea of God. But not having evidence of something does not mean that something does not exists. So not believing in God is unfounded given the evidence we have. At best, it seems, they have to be open to the idea of God. So they believe what they believe on faith, right?