Allow search bots to crawl your sites without session IDs or arguments that track their path through the site. These techniques are useful for tracking individual user behavior,but the access pattern of bots is entirely different. Using these techniques may result in incomplete indexing of your site,as bots may not be able to eliminate URLs that look different but actually point to the same page.

我的ASP.NET 1.1站点使用自定义身份验证/授权,并且非常依赖会话guids(类似于this approach).我担心允许非会话跟踪流量会破坏我现有的代码或引入安全漏洞.

允许非会话跟踪机器人抓取正常会话跟踪站点的最佳做法是什么？除了检查用户代理之外,还有什么方法可以检测搜索机器人(我不希望人们欺骗自己作为googlebot来绕过我的会话跟踪)？