Latest Announcemnent
New thread

how to curl and proxy download

Status
Not open for further replies.
M

musa

Member
22
2011
4
0
hey,

i would like to know how this ( shimory DOT com ) works TECHNICALLY. i suppose first layer is cURL in order to grab the cookie and the downloading link (like http//srv43.fil3s0n1c.com/ef4888/dfeee/file.zip). but HOW is it possible to make the visitor download the file INSTANTLY (shimory does not even grabs the file through like wget() but deservs it immediatly to the visitor, kind of proxy, with a link like http://serv02.sh1m0ry.com/yourfiledearvisitor.zip).
i would like to know how to setup that kind of proxy/instant download.

i dont want to run same service (there so many like this nowadays) but i'm interested in this kind of bypass/workaround for a website i am already running.

is it full PHP or is there some kind of linux (mod_proxy, squid, etc) behind this?

thank you and sorry for my bad english.

looking for some code examples and/or tutorials. i can buy a good tuto up to $50

Musa,
 
16 comments
you need to check PHP: Streams.

here is how it works
1. user give link
2. make script to grab download link
3. stream that content to user.

on 2. you can save data into a file, like on that site its saving a link as (http://shimory.com/static/1234567), 1234567 is either in database or file saved on server, So when user visit that link it will call that ID, Which has file link saved, cookie information etc.

there are a lot of examples on PHP site to work with streams and one open source script for link generation i think named "Vinaleech"? I am not sure but try to search here and look at its code.
 
Last edited:
thanks for the info mRaza, i'll take a look at these.

Santanu18 > i already consider myself an advanced php coder (+10 years of programming), and referring to your topic i don't need a script that grabs links from fast-debrid or so. i can do this by myself to be honest...
i want to make a script that connects to a remote website trough a cookie (i already coded this part), then gets the real download link (already coded this too). And then delivers the file to user INSTANTLY (i dont currently know how to make this, and that is what i will pay for).
what i am doing currently is grab the file to my dedicated server's hard disk drive via wget() then offer the file to the user a few minutes later...
i deliver the file via the x-sendfile mod and notice i already know about readfile and don't want to set up a readfile stream-octet as it consums too much ram/cpu (files i will deliver are between 100 and 300Mb).
if u can help in this then i'd be interested, drop me a pm...

sry for my bad english.
 
The problem is you are using wget to download the file.
If you want to stream the file to your users, you have to:

1) Open the stream to the direct link.
2) Send the required headers ( such as the cookie ) | So now you're writing to the stream.
3) Read the response-headers. | Still the same stream, now you just read from it.
4) Send headers to client ( header() function )
5) Echo rest of response to the screen. | You're reading data from the stream in chunks

But note that PHP isn't the right/best language for this.
 
any small exemple of this plz?

let's take filesonic as an exemple for the rest of this topic discussion.

1) and 2) Open the stream to the direct link. --> ok, opening the direct link (http://srv1337.filesonic.com/45465465/4212/the_desired_file.rar) with php cURL and sending the authorized cookie.txt via CURLOPT_COOKIEFILE
3) why read the response header?
4) ok, sending headers as content-disposition attachment and force-download or stream-octet
5) this is what i don't know how to do. how to stream? how to chunks the data?

thank you.
i only know php. no perl/cgi/c etc :(
 
ok for the third point, grabbing the content-lenght is a good reason.

sending you a pm then, i already know much about the curl lib and the manual but dont get you about the stream...
 
your co-worker did not responded to my pm./

anyway.

fuck about your shit "secrets", here it is, to sum up:

1) cURL login and form and parse the output to get the download link and the member cookie:
curl_setopt($ch, CURLOPT_COOKIEFILE, "the_cookie.txt");

2) open a socket on the download link,:
$fp = @fsockopen($host, $port, $errno, $errstr, 3);
3) putting the right headers/cookies in it:
fputs($fp, $your_headers);
including cookie:
$your_headers .= "Cookie: PHPSESSID=".$session."; key=".$var."\r\n";
(you can read the cookie's keys/values preg_match()'ing from the cookie.txt)
4) then output the stream to the visitor:
while (!feof($sock)){
echo fread($sock, 8192);
}
fclose($sock);



kthxbaï

mfw those who know don't respond to my pm --> :|
mfw i post the response here --> x'D

sry for my bad english

case closed.
 
PyroStrex said:
and yes, I did sign up just to reply this message. But since I'm now a member, I think I will contribute something.
Click to expand...

Great to have a new useful member!

That is what I meant with "PHP isn't the best language for this".
NodeJS however, is (perfect).

@musa; I'm sorry I didn't respond.
If you wanted the fsocket method instead of the curl you should've just said so >_>
Anyway; I made you a little something. Happy x-mas!
It could be that you're only interested in the '//The actual shit' -part.

PHP: 
<?php
	/***************************\
	#	Filesonic stream shit   #
	#	Robin@houtevelts.com    #
	#	http://file-remoter.com #
	#	25-dec-2011			 #
	\****************************/
	
	$username = 'username';
	$password = 'password';
	
	$url = 'http://www.filesonic.com/file/GjRJgk7/FinalWire.AIDA64.Extreme.Edition.v2.00.1700.MULTILINGUAL.Incl.Keygen.WORKING-Lz0.rar';
	
	$regex = '#/file/(.*?)(/|$)#';
	$matches = array();
	preg_match($regex, $url, $matches);
	$id = str_replace('/', '-', $matches[1]);
	unset($matches,$regex);
	
	$fileinfo = file_get_contents('http://api.filesonic.com/link?method=getInfo&format=json&ids='.$id);
	$fileinfo = json_decode($fileinfo,1);
	$fileinfo = $fileinfo['FSApi_Link']['getInfo'];
	
	if($fileinfo['status'] != 'success')
		die('API Error! mofo');
	
	$fileinfo = $fileinfo['response']['links'][0];
	
	if($fileinfo['status'] != 'AVAILABLE')
		die('File is no moar!');

	
	$filename = $fileinfo['filename'];
	$filesize = $fileinfo['size'];

	
	$url = file_get_contents('http://api.filesonic.com/link?method=getDownloadLink&u='.$username.'&p='.$password.'&ids='.$id);
	$url = json_decode($url,1);
	$url = $url['FSApi_Link']['getDownloadLink'];
	
	if($url['status'] != 'success')
		die('API Error! mofo');
	
	$url = $url['response']['links'];
	$url = end($url);
	$url = $url['url'];
	
	
	//The actual shit
	$url = $url;
	$host = parse_url($url);
	$headers = '';
	$rn = "\r\n";
	
	$headers .= 'GET '.$host['path'] . ($host['query'] == true ? '?'.$host['query'] : '').' HTTP/1.1' . $rn;
	$headers .= 'Host: '.$host['host'] . $rn;
	$headers .= 'Accept: */*' . $rn . $rn;
	
	$fs = fsockopen($host['host'],80,$errno,$errstr,30);
	if($errno||$errstr){
		echo 'ERROR: '.$errstr;
		@fclose($fs);
		exit;
	}
	fwrite($fs,$headers);
	unset($headers);
	
	$header = '';
	
	do {
		$header .= fgets ( $fs, 16384 );
	}while(strpos($header,$rn.$rn) === false);
	
	/*
		$header now contains the received headers
		You could fetch the filesize etc from it etc, but we already know it.
		But since I'm such a nice guy:
	*/
	$filesize = preg_match('#Content-Length: (\d+)#',$header,$match);
	$filesize = trim($match[1]);
	unset($match,$header);
	
	header('Content-Length: '.$filesize);
	header('Content-Disposition: attachment; filename="' . $filename . '"');
	header('Connection: Close');
	
	while (!feof($fs)){
		echo fread($fs,8192);
	}
	fclose($fs);
	
?>
 
Last edited:
Robin H said:
@musa; I'm sorry I didn't respond.
If you wanted the fsocket method instead of the curl you should've just said so >_>
Anyway; I made you a little something. Happy x-mas!
It could be that you're only interested in the '//The actual shit' -part.
Click to expand...

hahaha, did u even read my response? you are posting a code i already found without your so called help and i posted just on top of your message :facepalm:

btw i just asked for a clue or a function name that would do the streaming, i dont need your full code as i know about curl'ing and parsing the responses through preg_match and regex...

thanks anyway but the fact that unplease me is that you act like your helping but you just posting way after i found it already and ignoring my pm :)) 8-)

bye.
 
musa why'd you pm me to code you something that is obvious? The method has been said before in many threads, if you can't take the method and TRY to implement it yourself you shouldn't be in this section. Let alone bashing people who did help you.

@RobinH yours looks nicer than mine haha. Although I'm using fopen vs fsockopen... is there a major difference?
 
I don't know the difference between fopen and fsockopen.
They both give a stream, but I'm not sure you can send custom headers with fopen..
 
Status
Not open for further replies.
Back
Top