http status code 301 - bots and 301 redirect -
I have changed the structure of my site more than 6 months ago. I find out the use of legacy url and redirect to the new url with a 301 status code. I have verified with the jarring that the status code is returned correctly on request. But bots (yahoo slips, ballbottel, etc.) are still killing old URLs. Is there something I am missing? No, crawlers only take a lot, very long time to get the message.
I have been crawling the bots with leaves that have not been in existence since 2005 - when people make addresses with permanent Weed, they are in fact.
Additionally, depending on your URL being structured, you can reject the old address with robots .txt
Comments
Post a Comment