Auto-Indexing WCDDL Internally

[Golden Falcon]

Hey guys,

This probably doesn't exist but I want to experiment with it, I want to create an internal WCDDL (or any good working DDL Script) that indexes my entire site (and only my site) automatically.

I am going to pretty much use it as a secondary search function, for easy searching and listings.

Anyone know if this can be done?

 

[immortalxx]

it can be done

 

[Mdrop]

Yep, a bot can do that, but dont know how .

 

[Nemesis]

This is hmmm, easyish. You'd have to download your database - thread title and url then after that you'd have to be creative :D

 

[SJshah]

it has been done before. this site scrapes and indexes katz downkoads:
http://linklol.com/home/index.php

im sure it can be done for your forum

 

[ChosenOne]

Why don't you use something like sphider?

 

[Nemesis]

I love to create ddl sites but not when I get hacked lol

 

[Golden Falcon]

This is hmmm, easyish. You'd have to download your database - thread title and url then after that you'd have to be creative :D

Thats too manual. That means if I want to updated the listings/searchable content. I'd have to manually download my database, and alter it again.

Was looking for spider/ddl combo type of thing

 

[snowmanrene]

Maybe a script that when a thread is created in a download forum, it will automatically add the title and url to the "ddl" db as well as the forum db.

 

[Golden Falcon]

Maybe a script that when a thread is created in a download forum, it will automatically add the title and url to the "ddl" db as well as the forum db.

Yea that'd work. I doesn't even need to be a continuous running script thats in the forum. The script can just grab the forum thread url and submit them when "activated". To take care of dead links, the scrip can just whipe the db, and re-index the download area manually when activated.

Too bad I suck a programming. Maybe someone can have fun with this concept B-)

 

[CyberJ37]

Make a cron script to get downloads from your forum's DB daily, and make it start from where it stopped.

Will be way faster than crawling.

 

[Lock Down]

Why not just add title and topic id as most link sites do? You would not need urls unless you were trying to monitor for duplicates.

To add the title and topic id to a separate table all you would need it to create it and add a trigger to add a new record to the new table when an insert/update of the topic table is made. Very simple and google has a million examples.

By the way:

I would use topic id as key of new table and use replace rather than insert to add/update any records in that table.

 

[desiboy]

you can use curl to do this

 

[ddlshack]

Or just create a simple script which reads downloads directly from the database? :S