From: robots on
If there is a robots.txt file in the root directory, how can spiders still
list all my files even in the MEMBERS-ONLY section
From: Dennis on
On Tue, 2 Mar 2010 09:37:01 -0800, robots
<robots(a)discussions.microsoft.com> wrote:

>If there is a robots.txt file in the root directory, how can spiders still
>list all my files even in the MEMBERS-ONLY section

Spiders aren't obligated to obey robots.txt.

--

Dennis
From: Wayne-I-M on
ooops - forgot to put the second link in -


Hi

Steep learning curve for me.

I am just start to create a CSS web and have created a simple drop down menu
http://www.ukoutdoorpursuits.org/Thisisatest1.html

Which "seems to work OK when I add in another couple of DIVs below (content
and Footer - note the content DIV will hold the editable region). The Header
DIV, Nav DIV and Footer DIV will only be editable on the DWT.

But as you can see – the “Hidden” is not working like it does in frontpage.
Although you can't see the drop downs they are still there – and clickable.
Drag you mouse over the Contact DIV (white bit) and the drop downs appear -
???? (2nd LINK BELOW)
http://www.ukoutdoorpursuits.org/Thisisatest1.html
Note - the transparency of the drop downs will be sort (this is just a test
:-)

But the main problem is this anything on the Content DIV is always on top -
so the drop downs can't be seen
http://www.ukoutdoorpursuits.org/Thisisatest3.html

Am obviously doing something very basic which is wrong - any ideas would be
really helpful.

Designed in Win7

Thank you

From: Tom Willett on
Why did you hijack someone else's unrelated thread?
--
Tom [Pepper] Willett
Microsoft MVP - FrontPage Since 1997
---------------------------
"Wayne-I-M" <WayneIM(a)discussions.microsoft.com> wrote in message
news:62865494-AF84-4C4B-A1D0-A8402ED4B484(a)microsoft.com...
: ooops - forgot to put the second link in -
:
:
: Hi
:
: Steep learning curve for me.
:
: I am just start to create a CSS web and have created a simple drop down
menu
: http://www.ukoutdoorpursuits.org/Thisisatest1.html
:
: Which "seems to work OK when I add in another couple of DIVs below
(content
: and Footer - note the content DIV will hold the editable region). The
Header
: DIV, Nav DIV and Footer DIV will only be editable on the DWT.
:
: But as you can see - the "Hidden" is not working like it does in
frontpage.
: Although you can't see the drop downs they are still there - and
clickable.
: Drag you mouse over the Contact DIV (white bit) and the drop downs
appear -
: ???? (2nd LINK BELOW)
: http://www.ukoutdoorpursuits.org/Thisisatest1.html
: Note - the transparency of the drop downs will be sort (this is just a
test
::-)
:
: But the main problem is this anything on the Content DIV is always on
top -
: so the drop downs can't be seen
: http://www.ukoutdoorpursuits.org/Thisisatest3.html
:
: Am obviously doing something very basic which is wrong - any ideas would
be
: really helpful.
:
: Designed in Win7
:
: Thank you
:


From: Hot-text on
#
# From HTML-Kit
# This is an empty robots.txt file that can be used to indicate to
# WWW robots which parts of a web site should not be indexed.
#
# Following page has details on how to setup a robots.txt file and
# related security issues:
#
# http://www.chami.com/tips/internet/010198I.html
#


# 1. Uncomment following two lines by removing "#" at the beginning to
# indicate that all robots should stay away from the /cgi-bin/ folder

#User-agent: *
#Disallow: /cgi-bin/


# 2. Following two lines indicate that robots named "WWW_Robot_Name"
# should stay away from this site.

#User-agent: WWW_Robot_Name
#Disallow: /


# 3. Following two lines indicate that robots named "WWW_Wanderer_Name"
# should not index files in the /private/ folder.

#User-agent: WWW_Wanderer_Name
#Disallow: /private/


"robots" <robots(a)discussions.microsoft.com> wrote in message
news:288033DF-3FFE-45FB-85F3-BEAA1AC9FBE6(a)microsoft.com...
> If there is a robots.txt file in the root directory, how can spiders still
> list all my files even in the MEMBERS-ONLY section