placement of robots.txt file
Hi all,
I want to disallow search robots from indexing certain directories on a MacOS X Server.
Where would I put the robots.txt file?
According to the web robots pages at http://www.robotstxt.org/wc/exclusion-admin.html it needs to go in the "top-level of your URL space", which depends on the server and software configuration.
Quote: "So, you need to provide the "/robots.txt" in the top-level of your URL space. How to do this depends on your particular server software and configuration."
Quote: "For most servers it means creating a file in your top-level server directory. On a UNIX machine this might be /usr/local/etc/httpd/htdocs/robots.txt".
On a MacOS X Server would the robots.txt go into the "Library" or "WebServer" directory or somewhere else?
Thanxx
monica
G5 Mac OS X (10.4.8)
I want to disallow search robots from indexing certain directories on a MacOS X Server.
Where would I put the robots.txt file?
According to the web robots pages at http://www.robotstxt.org/wc/exclusion-admin.html it needs to go in the "top-level of your URL space", which depends on the server and software configuration.
Quote: "So, you need to provide the "/robots.txt" in the top-level of your URL space. How to do this depends on your particular server software and configuration."
Quote: "For most servers it means creating a file in your top-level server directory. On a UNIX machine this might be /usr/local/etc/httpd/htdocs/robots.txt".
On a MacOS X Server would the robots.txt go into the "Library" or "WebServer" directory or somewhere else?
Thanxx
monica
G5 Mac OS X (10.4.8)
G5, Mac OS X (10.4.8)
