[UPHPU] throttling bots and spiders

Joshua Simpson std3rr at gmail.com
Fri Feb 22 15:20:12 MST 2008


On Fri, Feb 22, 2008 at 12:38 PM, Wade Preston Shearer <
lists at wadeshearer.com> wrote:

> > Do you want them to crawl, but just get throttled? Or no spider?
>
> Sorry, I didn't ask my question well. They are free to crawl as much
> as they would like. I don't wan to assign them a session if they are a
> bot/spider. I am looking for a way (besides manually maintaining a
> user-agent list) to automatically disqualify them from getting an
> session assigned to them.
>
>
You could check USER_AGENT and/or use get_browser before initiating the
session.

-- 
-
http://stderr.ws/
"Insert pseudo-insightful quote here." - Some Guy


More information about the UPHPU mailing list