Robots.txt and extensions

[John]

I want to disallow all HTML pages on my site in robots.txt - accually I want to disallow all pages with html extension

Is there any way to do that?

 

[warrichpk]

Have you got new pages with a different extension instead of HTML? If yes then you should also work with a redirect. If no, and you have no need for these pages and the content and the links that come to these pages then make a folder and put all the pages with the HTML extension in this folder. Now just disllow this folder from being crawled

 

[Lock Down]

Try this :

User-agent: *
disallow: .html$

 

[Occult]

Set it to HTML and then the permission to deny just as Lock Down said so