在我手提的档案中,使用直截了当的口号,在地址打上以下类型的人......
http://example.com/index.php
......改为......
http://example.com/
我也没有任何索引,在我的所有网站上都有一个字塔。
我的问题是,鉴于间接行为和元数据,如果我的机器人,那么我的主页(即指数.php)的直观器将索引。 file号档案如下......
User-agent: Googlebot
Allow: /index.php
Disallow: /
User-agent: *
Disallow: /
否则,我如何改变我的机器人。 t,只有我的主页才能在山洞里被编入索引,而不再如此?
提前感谢!
它有风险。 确信谷歌确实把你的网页索引成:
User-agent: *
Allow: /index.php
Disallow: /a
Disallow: /b
...
Disallow: /z
Disallow: /0
...
Disallow: /9
因此,你的根基“/”将不符合不规则。
另外,如果你们有爱之士,就不要忘记了。
User-agent: Mediapartners-Google
Allow: /
What does robots.txt file do in PHP project?
Or even forbid indexing the whole site?
We implemented a rating system on a site a while back that involves a link to a script. However, with the vast majority of ratings on the site at 3/5 and the ratings very even across 1-5 we re ...
Ok, I understand the Title didn t make any sense so here I ve tried to explain it in detail. I m using a hosting that gives me space for my domain and lets me "add on" other domains on it. So lets ...
I have been searching around using Google but I can t find an answer to this question. A robots.txt file can contain the following line: Sitemap: http://www.mysite.com/sitemapindex.xml but is it ...
I m creating a forum for a website, and plan on implementing a "Report this content" function. In all honesty, I m not sure how useful (lit. necessary) the feature will be, since a user account (...
I have a MediaWiki, and I don t think I want Google indexing the history of any page. How can a robots.txt disallow URLs with action=history in the query string?
I have a large directory of individual names along with generic publicaly available and category specific information that I want indexed as much as possible in search engines. Listing these names on ...