Latest Announcemnent
New thread

CurlAxel - PHP file download accelerator

Status
Not open for further replies.
@jockerhacker: look into pyLoad. Its meant specially for one-click hosters. The documentation isn't great enough though.
 
I'm also having troubles deciding with what to go :p
atm pyload seems the best, but I lack python experience.
Altough it shouldn't be too hard
 
I just had a major breakthrough, so we've gotten a step closer to integrating this with RL. I managed to to get RL to "echo" the data needed for CurlAxel to do it's job. At the moment I am trying to get in contact with JokerHacker because I need some help integrating cookies in this script.

Here's an example of the data I've managed to get from RL:
Code: 
URL: /file/Rq2MbwF/fCiiHoqngTBgddXZh***********IdyIIFNjbiFp_JM7o./Magic_Numbers_UK_2011-10S370_HQ.pdf

Host: fs5001dm.fileserve.com

Referer: http://www.fileserve.com/file/R*****/Magic_Numbers_UK_2011-10S370_HQ.pdf 

Cookie: PHPSESSID=r29*********466; cookie=467U***********b2ZZAoKh; Rq2MbwF=IfQKt****gsJc%3D
For the people wanting to know exactly at what stage we are, here you go:

  1. CurlAxel class
  2. Getting the needed information from RL
  3. Do some modifications to CurlAxel (Add cookies, and some RL variables)
  4. Sending the data to CurlAxel + Disabling original RL transload
  5. Getting CurlAxel to output the pogress
  6. Getting RL to output CurlAxel's pogress.
If you think you can help us with that, thus not testing, please send me a PM!
 
you would need to integrate the progress bar code to curlaxel...

From the point of post data to the get info and cookie for download it all has to be included into the curlaxel source so once it hits the transload page it can just download the files with the cached information..
 
Anyway, I found out something interesting, if I download this 100mb file the filesize becomes 500MB on my server, although the script has been working fine using a 1GB test file, this smaller file is giving problems. This should be checked...
 
CuraHack said:
I just had a major breakthrough, so we've gotten a step closer to integrating this with RL. I managed to to get RL to "echo" the data needed for CurlAxel to do it's job. At the moment I am trying to get in contact with JokerHacker because I need some help integrating cookies in this script.

Here's an example of the data I've managed to get from RL:
Code: 
URL: /file/Rq2MbwF/fCiiHoqngTBgddXZh***********IdyIIFNjbiFp_JM7o./Magic_Numbers_UK_2011-10S370_HQ.pdf

Host: fs5001dm.fileserve.com

Referer: http://www.fileserve.com/file/R*****/Magic_Numbers_UK_2011-10S370_HQ.pdf 

Cookie: PHPSESSID=r29*********466; cookie=467U***********b2ZZAoKh; Rq2MbwF=IfQKt****gsJc%3D
For the people wanting to know exactly at what stage we are, here you go:

  1. CurlAxel class
  2. Getting the needed information from RL
  3. Do some modifications to CurlAxel (Add cookies, and some RL variables)
  4. Sending the data to CurlAxel + Disabling original RL transload
  5. Getting CurlAxel to output the pogress
  6. Getting RL to output CurlAxel's pogress.
If you think you can help us with that, thus not testing, please send me a PM!
Click to expand...

May i ask why you are implementing it though?
Curl Progressing also requires PHP 5.3 or +
 
CuraHack said:
Anyway, I found out something interesting, if I download this 100mb file the filesize becomes 500MB on my server, although the script has been working fine using a 1GB test file, this smaller file is giving problems. This should be checked...
Click to expand...
Yes, its true. I have checked this with ~250MB , ~400MB files. Effect was the same as above. First parts are downloading fine - stops when they are the size what it should be (~250MB / 5 (if 5 parts)). But size of the last part is like the whole file [~250MB].
 
CuraHack said:
Why not do it if I can? Wouldn't it be great to improve the download speeds in RL?
And if PHP 5.3 is required, then so be it, why would you let that stop you?
Click to expand...
And what makes you think that it will increase the speed? The base is still curl! If you are really looking to increase the speed in rl, then embed the downloading through other programming languages that can help in faster progressing and downloading

Not to mention that the downloading page will lag as usual even with curl since curl progress also outputs the data over page on change!
 
I also think RapidLeech has reached it's limit and a new platform should be thought out.
pyload seems to be the best imo.

But I have 0% expercience in python and don't really have the time to study it.

One should also check Node.js
It has some serious potential aswell.
 
soft2050 said:
And what makes you think that it will increase the speed? The base is still curl! If you are really looking to increase the speed in rl, then embed the downloading through other programming languages that can help in faster progressing and downloading

Not to mention that the downloading page will lag as usual even with curl since curl progress also outputs the data over page on change!
Click to expand...

It will increase the performance, to prove my point here's a screenshot I made while testing a 1GB file download with jokerhacker's script. I only got 33MB/s with a normal WGET while I had speeds of over 100MB/s using CurlAxel.

276107a9fd.png
 
^
I can also show screenshot where axel got me slower speeds than just wget.

I'm ok with this project tho.
But I'd like it more if everyone focused it's energy on something more useful.
 
CuraHack said:
It will increase the performance, to prove my point here's a screenshot I made while testing a 1GB file download with jokerhacker's script. I only got 33MB/s with a normal WGET while I had speeds of over 100MB/s using CurlAxel.

276107a9fd.png
Click to expand...
WGET :O
Check with fsockopen or stream_socket_client 8-)
 
i know that CurlAxel is not complete... yet ;)
you know, i'm busy these days so i can't focus on it, it has many bugs that will be fixed soon. curl is not the perfect library to use, but it's the most portable one, and php has full support for it. I'll study this and see if there is a better method to use. For now, my main update for the next version (will be launched soon) is the Callback function since they are required for progress bars. And i'll see that problem with the last part file :)
thanks for the use and the feedback guys. This will surely help in CurlAxel's development.
 
updated CurlAxel to version 0.2
…
+ added support for various server responses
+ fixed filesize issue when requested server doesn't allow download accelerating
+ added support for cookies (passed as string)
+ added support for standard curl progress callback functon
 
Status
Not open for further replies.
Back
Top