Best solution for scrape 1M url

[hihotfile]

what will be best/fastest solution for scrape 1 M url.php+CURL , Python

 

[normalperson]

It would be scrapy (Python's framework) - I used it and it seems to be really fast and easy

CURL would make you die if you want to use PHP than use reactphp but really, that would be more difficult than scrapy

 

[hulkcoder]

Guzzle PHP has the option of asynchronous requests as well this might help