在做网站SEO优化中robots的编写方法

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 263

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 264

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 290

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 291

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 263

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 264

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 290

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 291

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 263

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 264

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 290

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 291

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 263

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 264

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 290

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 291

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 263

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 264

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 290

Warning: Illegal offset type in isset or empty in /www/wwwroot/www.moyuseo.com/wp-content/themes/justnews/themer/core/wpcom.php on line 291

robots.txt这是搜索引擎中的协议。当你的网站有一些内容不想爬蜘蛛时，你可以使用它robots.txt告诉蜘蛛不要爬这些页面，当然不要爬这些页面。

robots.txt这是搜索引擎访问网站时需要查看的第一个文件。当搜索蜘蛛访问一个网站时，它会首先检查网站的根目录是否存在robots.txt，如果存在，搜索机器人将根据文件中的内容确定访问范围；如果文件不存在，所有搜索蜘蛛都可以访问网站上所有没有密码保护的页面。

robots.txt规则怎么写

新建一个txt文本，命名为robots.txt，比如你不想给蜘蛛爬你网站后台的文件夹，那就这样写吧。文件放在网站根目录中

User-agent: *

Disallow: /admin/

解释：

User-agent: * 这意味着所有的搜索引擎

Disallow: /admin禁止爬行admin这个文件夹

Disallow: 代表根目录

Allow: / 代表允许 的例子webdesgn文件夹 ，但你想让他收录webdesign下的wangzhanjianshe 写法Allow: /webdesign/wangzhanjianshe/*.html 收录webdesign下的wangzhanjianshe下的所有html

假如你只想对百度有效，那就这样写吧。User-agent: Baiduspider 只对百度有效，其他搜索引擎不受限制。

搜索引擎蜘蛛的名字：Baiduspider 百度蜘蛛，Googlebot 谷歌蜘蛛，Bingbot 必须是蜘蛛，360Spider 360蜘蛛、Sogouspider 搜狗蜘蛛