Re: new scholarly content in (vanilla) Google?

Rudy Leon wrote:
> I was searching for a number of books, trying primarily
to get to
> publisher home pages (JSTOR reviews were a useful
surprise, but I'd
> like to know more....)
> 
> One example: slavery, colonialism and economic growth
> 


I found a link to 
<http://lin
ks.jstor.org/sici?sici=0013-0117(198305)2%3A36%3A2%3C318
%3ASCAEGI%3E2.0.CO%3B2-M>

I'm interested to note that <http://links.js
tor.org/robots.txt> 
explicitly prohibits indexers from going into that area. 
Assuming 
googlebot still observes robots.txt, it raises interesting
questions of 
how and why it indexed that page.


-- 
Thomas Dowling
tdowlingohiolink.edu

_______________________________________________
Web4lib mailing list
Web4libwebjunction.org
http://lists.we
bjunction.org/web4lib/