?

Log in

No account? Create an account

msilverstar

Google is fixing things.

« previous entry | next entry »
Sep. 14th, 2005 | 04:21 pm

Google knows about the problem with indexing blogs that are marked as "noindex" and they're really sorry. They were leftover from the original test phase and not removed when it went into beta. Dunno when they will remove them from the index, but from what I've been told, it will be very soon. (And I have excellent sources :-)

ETA: Don't expect the fix before tomorrow at the very earliest.

To tell whether or not your journal is in the index, use the "inurl:" syntax, for example:
    http://blogsearch.google.com/blogsearch?q=inurl:billyviggo (OK because my billyviggo comm is public).
Replace that name with your journal name to find out.

To tell whether your journal should be found, edit your Info page. About 3/4 of the way down, there's a checkbox marked "Block Robots/Spiders from indexing your journal". Make sure it's checked.

When it is, every one of your pages will include in the HTML header:
    <meta name="robots" content="noindex, nofollow, noarchive">
Google was very naughty for ignoring those.

NOTE: Friends-locked and private posts cannot be indexed by any search engine. If you want to be sure it is hidden, be sure to lock it.
Tags:

Link | Leave a comment |

Comments {34}

(Deleted comment)

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 12:50 am (UTC)
Link

You're welcome! This time, it was my mac contacts that worked. Estel probably knows some of them!

Reply | Parent | Thread

almaviva

(no subject)

from: almaviva
date: Sep. 14th, 2005 11:51 pm (UTC)
Link

Unfortunately, if you've ever accidently posted as public and then changed it to friendsonly, the text is still viewable through this (not a lot, but then it just happens that there is something there that I would rather not have read publically--not something bad, just something personal, though it probably wouldn't be understandable out of context, which this doesn't provide...I'm rambling.) I'll be happiest when they fix it. Thank you for the information.

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 12:49 am (UTC)
Link

I've gotten pages out of their index, but it takes a while.

Reply | Parent | Thread | Expand

Jassy

(no subject)

from: jasmineskie
date: Sep. 15th, 2005 12:29 am (UTC)
Link

I had checked the 'block robots/spiders' thing and my unlocked posts are still on Google's blog index. (thank you for posting that url thing, sweetie *hugs*) Whether Google was naughty or simply careless, it still makes me glad I decided to lock up all my fiction. *sigh*

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 12:48 am (UTC)
Link

Yeah, we all discovered it was broken today. So did they. They're fixing it even as we speak :-/

Reply | Parent | Thread

I am weasel

(no subject)

from: killerweasel
date: Sep. 15th, 2005 12:49 am (UTC)
Link

I just checked the robot box, but all my unlocked posts still pop up.

*rubs brow*

So this is all supposedly going to get fixed?

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 12:53 am (UTC)
Link

Right now, they're frantically removing the ones in the index that had the robots noindex tag. Next time they index, they'll remove yours. It may take a while, sorry.

Reply | Parent | Thread

tell me your heart doesn't race for a hurricane

(no subject)

from: jocondite
date: Sep. 15th, 2005 05:37 am (UTC)
Link

*dies of relief* You've just eased my mind so much, love. Thank you.

I think I rather panicked, but best to be too careful than too carelss. :)

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 05:39 am (UTC)
Link

I knew there was someone else who was v. worried! Glad I was able to reassure you.

Reply | Parent | Thread

Galadriel

(no subject)

from: caras_galadhon
date: Sep. 15th, 2005 05:52 am (UTC)
Link

(Here via a comment you made in viva_gloria's entry.)

Thanks for the info. Do you mind if I pass the info on in my LJ?

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 05:54 am (UTC)
Link

Please, go ahead! I v. much want to help people calm down :-)

Reply | Parent | Thread | Expand

Gloria Mundi

(no subject)

from: viva_gloria
date: Sep. 15th, 2005 07:50 am (UTC)
Link

many thanks for the calming! I'm still not sure why it bothered me (and lj user="tessabeth">) so much: we are happy for people to discover our little corner, and for it to be recced. But there's something more alarming about the whole thing being searchable (or, as actually happened, three apparently random posts).

Ever so pleased you've alerted them to the problem, and that they're so amenable to fixing it.

Reply | Thread

Gloria Mundi

(no subject)

from: viva_gloria
date: Sep. 15th, 2005 07:55 am (UTC)
Link

when you say 'leftovers from the original test phase', presumably they're not blogs with RSS feeds, as the site implies? ("If you do not publish a site feed for your blog, it will not be included in Blog Search.") This is the bit that threw me -- and set me a-searching for that 'switch off syndication' option -- because I've never knowingly RSS'd any of my blogs, and yet there they were.

Reply | Parent | Thread | Expand

(no subject)

from: ex_salwood
date: Sep. 15th, 2005 08:13 am (UTC)
Link

I have the robotthingy checked and 143 entries are on the googleblog thing.

:(

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 06:05 pm (UTC)
Link

Yeah, they're still there. I know they're trying to get rid of them, but it's not done yet. Hold on a couple more days.

Reply | Parent | Thread | Expand

Nameless Benefactor

(no subject)

from: chenanceou
date: Sep. 15th, 2005 10:38 am (UTC)
Link

Thanks for the info - I didn't even know this could be done and, frankly, it's kinda scary.

Reply | Thread

Alassenya

(no subject)

from: alassenya
date: Sep. 15th, 2005 10:42 am (UTC)
Link

My "no robots" box has been checked ever since I started the journal, yet when I opened "view source" there is no HTML header like the one you describe. This worries me.

Reply | Thread

Alassenya

(no subject)

from: alassenya
date: Sep. 15th, 2005 11:58 am (UTC)
Link

Found problem - the style I was using didn't import the "no robots" info into the header. Have now changed style.

Reply | Parent | Thread | Expand

Kyrre

(no subject)

from: kyrre
date: Sep. 15th, 2005 10:52 am (UTC)
Link

One question because my journal is public: It is my understanding that any public entry that ends on my friends list can be spidered. Not as a separate post but the whole friends page. Correct or not?

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 06:19 pm (UTC)
Link

Any public web page can be spidered and indexed by search engines. Anything. Assume it will be out there forever, for better or worse, and act accordingly.

However, for search engines which behave themselves (unlike this current Google Blogsearch one), if your journal has the Block Robots setting checked, your friendspage will have the ROBOTS NOINDEX, NOFOLLOW meta tag like all your other pages. So there will be less to connect you and your friends, though your comments on their public and unprotected posts will still be indexed.

Reply | Parent | Thread

Stewardess

(no subject)

from: stewardess
date: Sep. 15th, 2005 03:25 pm (UTC)
Link

Thanks for this. I'm not in there, but one of my comms is, and it had robots blocked.

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 06:19 pm (UTC)
Link

Should be removed shortly.

Reply | Parent | Thread

(Deleted comment)

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 06:20 pm (UTC)
Link

That's great, glad to help out.

Reply | Parent | Thread

A Fandom Butterfly

(no subject)

from: chinawolf
date: Sep. 15th, 2005 04:12 pm (UTC)
Link

Thank you very much for this very matter of fact information. At öeast now I know it's temporary.

Reply | Thread

Lotripper

(no subject)

from: msilverstar
date: Sep. 15th, 2005 06:20 pm (UTC)
Link

Nice when my day job lets me help fandom and LJers!

Reply | Parent | Thread

:: an alias foray

(no subject)

from: aliasa
date: Oct. 5th, 2005 02:10 pm (UTC)
Link

Hi, I remember seeing a link to your page from a community. Hope you don't mind the intrusion. Just out of curiosity, do you have an update on where google is with fixing things?

From the beginning, my journal has always been marked as a 'noindex' one. Seems like my recent posts are being indexed again. I see one from Oct 1st.

Thanks for any info you can provide. :)

Reply | Thread