设置教程七牛云存储robots.txt搜索引擎规则

默认robots.txt

# robots.txt generated at http://portal.qiniu.com
User-agent: Baiduspider
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: sogou spider
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Googlebot
Disallow: /
User-agent: Bingbot
Disallow: /
User-agent: Slurp
Disallow: /
User-agent: MSNBot
Disallow: /
User-agent: googlebot-image
Disallow: /
User-agent: googlebot-mobile
Disallow: /
User-agent: yahoo-blogs/v3.9
Disallow: /
User-agent: psbot
Disallow: /
User-agent: *
Disallow: /

修改以后的

# robots.txt generated at http://portal.qiniu.com
# edit for statice domain
User-agent: Baiduspider
Allow: /wp-content/uploads/sites/2/*.jpg$
Allow: /wp-content/uploads/sites/2/*.png$
Disallow: /
User-agent: Baiduspider-image
Allow: /wp-content/uploads/sites/2/*.jpg$
Allow: /wp-content/uploads/sites/2/*.png$
Disallow: /
User-agent: 360Spider
Allow: /wp-content/uploads/sites/2/*.jpg$
Allow: /wp-content/uploads/sites/2/*.png$
Disallow: /
User-agent: 360Spider-Image
Allow: /wp-content/uploads/sites/2/*.jpg$
Allow: /wp-content/uploads/sites/2/*.png$
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: sogou spider
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Googlebot
Disallow: /
User-agent: Bingbot
Disallow: /
User-agent: Slurp
Disallow: /
User-agent: MSNBot
Disallow: /
User-agent: googlebot-image
Disallow: /
User-agent: googlebot-mobile
Disallow: /
User-agent: yahoo-blogs/v3.9
Disallow: /
User-agent: psbot
Disallow: /
User-agent: *
Disallow: /

# robots.txt generated at http://portal.qiniu.com
User-agent: Baiduspider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: 360Spider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: Baiduspider-image
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: 360Spider-Image
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: *
Disallow: /

除了百度360以外还带神马搜索蜘蛛/搜狗/

# robots.txt generated at http://portal.qiniu.com
User-agent: Baiduspider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: Baiduspider-image
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: 360Spider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: YisouSpider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: sogou spider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Googlebot
Disallow: /
User-agent: Bingbot
Disallow: /
User-agent: Slurp
Disallow: /
User-agent: MSNBot
Disallow: /
User-agent: googlebot-image
Disallow: /
User-agent: googlebot-mobile
Disallow: /
User-agent: yahoo-blogs/v3.9
Disallow: /
User-agent: psbot
Disallow: /
User-agent: *
Disallow: /

 

 

版权声明：原创作品，未经允许不得转载，否则将追究法律责任。
本站资源有的自互联网收集整理，如果侵犯了您的合法权益，请联系本站我们会及时删除。
本站资源仅供研究、学习交流之用，若使用商业用途，请购买正版授权，否则产生的一切后果将由下载用户自行承担。
本文链接:源头网https://www.58588885.com/7333.html
许可协议:《署名-非商业性使用-相同方式共享 4.0 国际 (CC BY-NC-SA 4.0)》许可协议授权