Skip to content
WJunction - Webmaster Forum

Crawl a specific directory for a filetype

Status
Not open for further replies.
I'm looking for a way to crawl a directory for a specific file type. The site requires that the file be read directly, so you can't just crawl the directory literally. I'd like it to remain running, kind of like RSS.

I feel like I'm not being clear so here's essentially what I'm aiming for:
Code:
http://domain.com/fag/fag-*.mp3

I'm trying to crawl a show that gets posted every week but is only available for playing, not downloading. Of course, I used IDM to grab the link but I'm wondering if I can get a script or some kind of crawler that enables me to check every couples days for anything named fag-whatever.mp3. Usually the * has a number as it marks the number of the show released that week.

__________________
Added after 3 Days 5 Hours:

bump... I know this is an easy script, it's just a matter of knowing how to make it.
 
Last edited:

1 comment

Djlatino- ntg? :-P make a php that downloads your target page,several functions there are to do this in php,use regex To extract the url, once your done with php ,put it in cronjob in your linux server. should be done. if you know its easy,y you dont make yourself...lol
 
Status
Not open for further replies.

About the author

D
Active Member · Joined
NIGGA I SAY NIGGA EVEN DOE IM A SPIC BEING WITH MY NIGGAS
2,413
Messages
528
Reactions
113
Points

Advertise on WJunction

Reach 1000's of webmasters, hosts & affiliates. Banner & sponsored-thread slots available.

Contact us
Back
Top Bottom