[UPHPU] throttling bots and spiders

Joseph Scott joseph at randomnetworks.com
Fri Feb 22 16:34:53 MST 2008

On Feb 22, 2008, at 1:38 PM, Wade Preston Shearer wrote:

>> Do you want them to crawl, but just get throttled? Or no spider?
> Sorry, I didn't ask my question well. They are free to crawl as  
> much as they would like. I don't wan to assign them a session if  
> they are a bot/spider. I am looking for a way (besides manually  
> maintaining a user-agent list) to automatically disqualify them  
> from getting an session assigned to them.

(making some assumptions)

Sounds like you initiate a session for every HTTP request.  Instead  
of doing that you could only start a session when a specific action  
takes place, like when a user attempts to log in.

Joseph Scott
joseph at randomnetworks.com

More information about the UPHPU mailing list