Crawl a specific directory for a filetype

Status
Not open for further replies.

Djlatino

Active Member
2,413
2010
528
10
I'm looking for a way to crawl a directory for a specific file type. The site requires that the file be read directly, so you can't just crawl the directory literally. I'd like it to remain running, kind of like RSS.

I feel like I'm not being clear so here's essentially what I'm aiming for:
Code:
http://domain.com/fag/fag-*.mp3

I'm trying to crawl a show that gets posted every week but is only available for playing, not downloading. Of course, I used IDM to grab the link but I'm wondering if I can get a script or some kind of crawler that enables me to check every couples days for anything named fag-whatever.mp3. Usually the * has a number as it marks the number of the show released that week.

__________________
Added after 3 Days 5 Hours:

bump... I know this is an easy script, it's just a matter of knowing how to make it.
 
Last edited:
1 comment
Djlatino- ntg? :-P make a php that downloads your target page,several functions there are to do this in php,use regex To extract the url, once your done with php ,put it in cronjob in your linux server. should be done. if you know its easy,y you dont make yourself...lol
 
Status
Not open for further replies.
Back
Top