cross-posted at skippy and a veritable cornucopia of other community blogs.
in our recent (and recommended at boomantribune, frontpaged at myleftwing…thank you all…sniff…we couldn’t have done it without the little people) diary the wizard of kos, we tried to prove with real facts that kos is a jerk. well, actually, what we tried to prove was that his purging of his blogroll was objectively a bad thing. we kind of did, too, tho we were pretty much preaching to the choir. can we get an ‘amen’?
however, booman brought up the idea of individual dkos user blogrolls on their own individual diaries:
skippy-
markos gave every diarist their own blogroll, thereby immensely increasing the number of bloglinks emanating from his site. i don’t know how powerful those links are compared to the homepage, but it would seem he has greatly increased the potential for helping google rankings and taking himself out of the decision making process.
in the interest of fairness, you might mention that, since it seems unlikely his intent was to hurt bloggers by jacking up their links by several orders of magnitude. I, for example, used to have one link from there (two if you include my sig) and now I have 80 or something. Overall, it hurt my traffic, but it probably helped by google ranking.
well, bad news for booman: it not only hurt your traffic, it didn’t help your google ranking one iota.
— we tell you why after the jump —
we wrote to bill slawski, the author of the original piece we based our post on, and of the blog seo by the sea – internet marketing and search engine optimization research and services. mr. slawski wrote back:
each individual diary appears to be created on its own unique subdomain at dailykos.com. what that means for purposes of indexing at google is that each is treated as a unique website rather than part of the dailykos site. you can see when you perform a search on google for “dailykos” that in addition to the main dailykos home page, the results are also filled with the diary sites. google normally will only show two results from the same site in their search results page, with the second result being an indented one under the first. since it isn’t filtering out pages from the diary sites, it does appear that it is treating them as separate sites.
[ed. note: don’t get excited, kids. keep reading]
there is a benefit to each of the diary sites in having links to them from the front page of the main dailykos site, and from other pages of the main dailykos domain. hard to tell if the benefit of links appearing in the main content area of the front page is any different than links appearing in the sidebar.
on each of the diary pages, there is a link labeled “blogroll” that points to a page on the main dailykos site. these blogrolls aren’t on the dairy site subdomains, so they aren’t separate blogrolls for each of the diary sites, but rather pages that would be considered by google to be part of the main dailykos site. but…
if you look at the robots.txt file for the dailykos.com site, you’ll see that the user pages are disallowed for a number of robots (http://www.dailykos.com/robots.txt).
user-agent: googlebot
#disallow: /
disallow: /comments
disallow: /user
disallow: /poll
disallow: /print
disallow: /searchoddly, yahoo’s slurp isn’t disallowed, nor is msnbot.
[ed. note: go figure!]
the diary owner’s blogrolls are being disallowed to googlebot by the robots.txt files. the url structure of the diary owners’ blogrolls is like this:
http://www.dailykos.com/user/username/blogroll
because of this line in the robots.txt file – “disallow: /user” – google isn’t allowed to visit those diary users’ blogroll pages, and index them, and follow the links upon them. in terms of ranking value for these user blogrolls, there is none, because google isn’t allowed to visit those pages.
[ed. note: emphasis, and haughty ‘we-told-you-so’ tone, ours]
it is quite possible that the way those blogrolls is set up wasn’t intended to deprive those blogrolls of any value.
[ed. note: yah, right. and duncan black doesn’t mean to be snarky.]
i would disallowed the individual user pages if i had set up this site because the main page of those have the same content as the subdomain main pages for each user.
i also noticed that the main dailykos page that, it has a url canonicalization issue. see matt cutts blog post – seo advice: url canonicalization at http://www.mattcutts.com/blog/seo-advice-url-canonicalization/
if you have the google toolbar installed on your browser, and the pagerank indicator activated, you can look at the main domain address for the site with and without the “www” and see a difference in the numbers that it shows. with the “www” it appears to have a toolbar pagerank of 8. without the “www” it appears to have a toolbar pagerank of 7. it’s possible that google doesn’t recognize that they are in fact the same page, and may be splitting pagerank between the two pages. matt explains in his post how to resolve that issue with a permanent (301) redirect. since it’s likely that most people linking to the site use the “www,” then it may be best to redirect from the non=www version to the www version.
we are in great debt to mr. slawski in helping us prove that kos is a jerk blogroll purges are not healthy for greater blogtopia, and yes, we coined that phrase.
sorry, booman.
