接收";内部服务器错误";当尝试使用Curl连接到FTP时


Receive "Internal Server Error" When Trying to Connect to FTP With Curl

好吧,这有点让我抓狂。。。我一直收到一个"内部服务器错误",试图用Curl连接到外部FTP连接。我可以正常访问FTP,但不能通过Curl访问。

我使用CodeIgniter作为包装器,但我真的不认为这会导致这样的问题。我试着增加内存/超时,但仍然无法进入。500内部服务器错误实际上就在我的页面上;我不知道Curl是否返回了任何内容,但我知道,如果我禁用"用户名"或试图添加"密码"(此FTP登录没有密码(,我只会通过Curl收到一个正常错误(而不是内部服务器错误(。

以下是我的主要脚本:

function FTPScrape() {
    parent::Controller();
    $this->load->model("Curl_m");
}
function index() {
    ini_set('memory_limit', '70000000M');
    set_time_limit(0);
    define('COOKIES_DIR', 'cookies');
    define('RANDOM', COOKIES_DIR . '/' . md5(uniqid(mt_rand(), true)));
    $this->Curl_m->init(TRUE, RANDOM . '.txt');
    $this->Curl_m->start();
    $referer = '';
    $url = 'ftp://ftp.server.com/';
    $str = $this->Curl_m->ftp($url, 'user', '', __line__, $referer);
    print "<br><br><textarea cols='"80'" rows='"20'">{$str}</textarea><br><br>";
    $this->Curl_m->close();
}

以下是我使用的"Curl_m"模型函数:

function init($cookies = TRUE, $cookie = 'cookies.txt', $compression = '', $proxy = '') {
if(!is_dir('files')) {
    mkdir('files', 0777);
}
if(!is_dir('cookies')) {
    mkdir('cookies', 0777);
}
else if($dh = opendir('files/')) {
    while(($file = readdir($dh)) !== false) {
        if(is_file('files/'.$file)) {
            unlink('files/'.$file);
        }
    }
}
$this->user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)';
$this->compression = $compression;
$this->proxy = $proxy;
$this->cookies = $cookies;
$this->filenumber = 0;
$this->follow_location = 1;
$this->headers_html = 0;
$this->binary = 0;
if($this->cookies == TRUE) {
    $this->cookie($cookie);
}
}
function start() {
    $this->process = curl_init();
}
function close() {
    curl_close($this->process);
}
function ftp($url, $user = '', $pass = '', $line_no = '', $referer = '') {
    return $this->execute($url, '', $line_no, $referer, $user, $pass);
}
function execute($url, $data = '', $line_no = '', $referer = '', $user = '', $pass = '') {
    if(isset($this->headers)) {
        unset($this->headers);
    }
    if(preg_match('/'w/xsi', $data)) {
        $type = 'POST';
    }
    else {
        $type = 'GET';
    }
    $host = parse_url($url);
    $this->headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
    $this->headers[] = 'Accept-Language: en-us,en;q=0.5';
    $this->headers[] = 'Accept-Encoding: gzip,deflate';
    $this->headers[] = 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7';
    $this->headers[] = 'Keep-Alive: 300';
    $this->headers[] = 'Connection: keep-alive';
    $this->headers[] = 'Expect:';
    if(preg_match('/'w/xsi',  $referer)) {
        $this->headers[] = 'Referer: ' . $referer . '';
    }
    if($type == 'POST') {
        if(isset($this->Content_type_change)) {
            $this->headers[] = 'Content-Type: ' . $this->Content_type_change . '';
            unset($this->Content_type_change);
        }
        else {
            $this->headers[] = 'Content-Type: application/x-www-form-urlencoded';
        }
        if(isset($this->extra_headings)) {
            $this->headers[] = $this->extra_headings;
        }
        if(!preg_match('/https:/xsi', $url)) {
            $this->headers[] = 'Content-Length: ' . strlen($data) . '';
        }
    }
    curl_setopt($this->process, CURLOPT_URL, $url);
    curl_setopt($this->process, CURLOPT_HTTPHEADER, $this->headers);
    curl_setopt($this->process, CURLOPT_HEADER, $this->headers_html);
    curl_setopt($this->process, CURLOPT_USERAGENT, $this->user_agent);
    if($this->cookies == TRUE) {
        curl_setopt($this->process, CURLOPT_COOKIEFILE, $this->cookie_file);
    }
    if($this->cookies == TRUE) {
        curl_setopt($this->process, CURLOPT_COOKIEJAR, $this->cookie_file);
    }
    curl_setopt($this->process, CURLOPT_ENCODING, $this->compression);
    curl_setopt($this->process, CURLOPT_CONNECTTIMEOUT, 0);
    curl_setopt($this->process, CURLOPT_TIMEOUT, 1200);
    curl_setopt($this->process, CURLOPT_SSL_VERIFYPEER, 0);
    curl_setopt($this->process, CURLOPT_RETURNTRANSFER, 1);
    if(strpos("ftp", $url) !== false) {
        curl_setopt($this->process, CURLOPT_FTPLISTONLY, 1);
    }
    if(!empty($user)) {
        $usrpwd = $user . ':' . $pass;
        curl_setopt($this->process, CURLOPT_USERPWD, $usrpwd);
    }
    if($this->binary == 1) {
        curl_setopt($this->process, CURLOPT_BINARYTRANSFER, 1);
    }
    if($this->follow_location == 1) {
        curl_setopt($this->process, CURLOPT_FOLLOWLOCATION, 1);
    }
    else {
        curl_setopt($this->process, CURLOPT_FOLLOWLOCATION, 0);
    }
    if($type == 'POST') {
        curl_setopt($this->process, CURLOPT_POSTFIELDS, $data);
        curl_setopt($this->process, CURLOPT_POST, 1);
    }
    if(preg_match("/'w/", $this->proxy)) {
        curl_setopt($this->process, CURLOPT_PROXY, $this->proxy);
    }
    $return = curl_exec($this->process);
    if($this->file_extension($url,$return) == 'jpg') {
        $fh = fopen('captcha.jpg', "w");
        fwrite($fh, $return);
        fclose($fh);
    }
    unset($this->headers);
    return $return;
}

有人知道我为什么会有这个问题吗?

大部分脚本都是在我开始这个项目之前创建的(即Curl_m模型中的函数(,我只是将该类转换为一个实际的代码点火器模型。

如果我能想出如何防止这导致内部服务器错误,我应该能够很容易地修复其余部分。

好吧,我明白了为什么它不能正常工作。我试图增加允许的内存并允许无限制的超时,但我想我的服务器只是不想让我这么做。

昨天和今天,我都在尝试配置我的客户设置的亚马逊EC2帐户,以便我可以将所有这些信息放在那里。之后,我再次测试了FTP Scraper,它成功地连接并检索了我试图访问的数据。无论如何,我的服务器实际上只是为了测试——最终的脚本将被放在亚马逊EC2上,但由于设置的复杂性,我推迟了它(我以前从未使用过亚马逊EC2(。

不管怎样,问题都是脚本超时或超出了分配的内存。通过在高端服务器上设置它,我可以很好地连接。显然,使用Curl登录FTP所花费的资源/时间比我想象的要多。