Is there a reliable way to detect non-malicious spiders/robots/crawlers in a PHP script? (Malicious ones can easily impersonate browsers, so that's out of scope for my question) I'm looking for something like an open-source User Agent list that's updated periodically. Does any such thing exist? Thanks