Is this Robots.txt fine to use for v3.8.5?

Status
Not open for further replies.

JGM

Active Member
1,016
2008
348
5
Code:
#Generated by wgtools.com/seo-tools/robots/
User-agent: *
Disallow: /admincp/
Disallow: /ajax.php
Disallow: /album.php
Disallow: /attachment.php
Disallow: /calendar.php
Disallow: /clientscript/
Disallow: /cron.php
Disallow: /customavatars/
Disallow: /customprofilepics/
Disallow: /editpost.php
Disallow: /faq.php
Disallow: /includes/
Disallow: /inlinemod.php
Disallow: /joinrequests.php
Disallow: /login.php
Disallow: /members/
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /modcp?
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /poll.php
Disallow: /posthistory.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /showgroups.php
Disallow: /showpost.php
Disallow: /subscription.php
Disallow: /threadrate.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /vbseo.php
Disallow: /vbseocp.php
Disallow: /vbseocpform.php

Thanks folks never done this before so I am a little confused. I do have vBSEO installed in case that is important. I do want to improve my SEO but without leaving the site vulnerable to attack by making some daft mistake :'(

JGM.
 
9 comments
For that I need to use a sitemap generator or something as far as I know that's nothing to do with a robots.txt file?

Is the above file OK to use or does it leave stuff out?

JGM.
 
Ah right bxflow thanks for that I don't have a site map atm that is something else I am looking in to making/sorting out (y)

So apart from that one line would you say it's fine to use?

JGM.
 
i am not sure about this line

Disallow: /admincp/

IMO if people open your site like .. yoursite.com/robots.txt they will know your admincp .. -> yoursire.com/admincp

i never put it on my robots..
cmiiw
 
Status
Not open for further replies.
Back
Top