为什么我的 CPU 在使用许多等待响应的 CURL 请求时很高


Why is my CPU high when using many CURL requests which are waiting for response?

我有一个大的php脚本,它使用curl和multi-curl请求到我自己的服务器和其他服务器。除此之外,它还在我自己的服务器上执行许多数据库请求和其他操作。

脚本太大,无法放在这里,不会让事情变得更容易。

我测量了整个剧本的时间。我区分了在自己的服务器上进行操作所需的时间和在外部服务器上进行操作所需的时间。

看来,将我自己服务器上所有操作的经过时间加起来,总时间永远不会超过 0.5 秒。因此,我希望我自己的脚本对于我的服务器来说并不繁重。总运行时间平均为 25 秒。

大多数时间:24.5秒,我的服务器正在等待来自外部服务器的响应。我的服务器确实发出了大约 300 个 curl 请求,但 curl 请求本身并不重。

问题:当我运行脚本时,cpu 达到 95%,并且只要此脚本正在运行,服务器(几乎(不会响应其他请求。

这是多卷曲请求的示例:

<?PHP
// Multi curl:
$imax=20;
// build the individual requests as above, but do not execute them
for ($i=1; $i<$imax; $i ++ ):
${'ch_'.$i} = curl_init('http://www.domain.com/');
//$ch_2 = curl_init('http://www.domain.com/');
curl_setopt(${'ch_'.$i}, CURLOPT_RETURNTRANSFER, true);
//curl_setopt($ch_2, CURLOPT_RETURNTRANSFER, true);
endfor;
// build the multi-curl handle, adding both $ch
$mh = curl_multi_init();
for ($i=1; $i<$imax; $i ++ ):
curl_multi_add_handle($mh, ${'ch_'.$i});
endfor;
// execute all queries simultaneously, and continue when all are complete
$running = null;
do {
curl_multi_exec($mh, $running);
} while ($running);
// all of our requests are done, we can now access the results
for ($i=1; $i<$imax; $i ++ ):
${'response_'.$i} = curl_multi_getcontent(${'ch_'.$i});
endfor;
//$response_2 = curl_multi_getcontent($ch_2);
for ($i=1; $i<$imax; $i ++ ):
echo "${'response_'.$i}"; // same output as first example
endfor
?>

httpd.conf:

#
# This is the main Apache HTTP server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See <URL:http://httpd.apache.org/docs/2.2> for detailed information.
# In particular, see 
# <URL:http://httpd.apache.org/docs/2.2/mod/directives.html>
# for a discussion of each configuration directive.
#
# Do NOT simply read the instructions in here without understanding
# what they do.  They're here only as hints or reminders.  If you are unsure
# consult the online docs. You have been warned.  
ServerRoot "/etc/httpd"
Listen 80
#LoadModule dummy_module /usr/lib/apache/mod_dummy.so
#LoadModule php5_module        /usr/lib/apache/libphp5.so
Include /etc/httpd/conf/extra/httpd-phpmodules.conf
User apache
Group apache
KeepAlive On
KeepAliveTimeout 10
ServerAdmin admin@localhost
DocumentRoot "/var/www/html"
MaxClients 10000
MaxRequestsPerChild 50000
<Directory /home/*>
    AllowOverride All
    Options -MultiViews -Indexes FollowSymlinks IncludesNoExec +Includes
<Limit GET POST OPTIONS PROPFIND>
    Order allow,deny
    Allow from all
</Limit>
<LimitExcept GET POST OPTIONS PROPFIND>
    Order deny,allow
    Deny from all
</LimitExcept>
</Directory>
<Directory />
    Options All
    AllowOverride All
</Directory>
<Directory "/var/www/html">
    Options Indexes FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
   <IfModule mod_suphp.c>
        suPHP_Engine On
        suPHP_UserGroup webapps webapps
    SetEnv PHP_INI_SCAN_DIR
   </IfModule>
</Directory>
<IfModule mod_dir.c>
    DirectoryIndex index.html index.htm index.shtml index.php index.php5 index.php4 index.php3 index.phtml index.cgi
</IfModule>
<FilesMatch "^'.ht">
    Order allow,deny
    Deny from all
    Satisfy All
</FilesMatch>
ErrorLog /var/log/httpd/error_log
LogLevel warn
#replace %b with %O for more accurate logging
<IfModule mod_logio.c>
    LogFormat "%h %l %u %t '"%r'" %>s %O '"%{Referer}i'" '"%{User-Agent}i'"" combined
    LogFormat "%h %l %u %t '"%r'" %>s %O" common
    LogFormat "%O %I" bytes
    LogFormat "%h %l %u %t '"%r'" %>s %b '"%{Referer}i'" '"%{User-Agent}i'" %I %O" combinedio
</IfModule>
    CustomLog /var/log/httpd/access_log common
<IfModule mod_alias.c>
    # Include some DirectAdmin alias
    Include conf/extra/httpd-alias.conf
</IfModule>
<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>
DefaultType text/plain
<IfModule mod_mime.c>
    TypesConfig conf/mime.types
    AddType application/x-gzip .tgz
    AddEncoding x-compress .Z
    AddEncoding x-gzip .gz .tgz
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz
    AddHandler cgi-script .cgi
    AddHandler type-map var
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
    AddType video/x-ms-asf .avi
    AddType video/mpeg .mpg
    AddType video/mpeg .mpeg
    AddType video/quicktime .mov
    AddType video/x-ms-wmv .wmv
</IfModule>
#EnableMMAP off
#EnableSendfile off
#######################################################################################
# Do not change anything in included files, because they are rewritten by DirectAdmin #
#######################################################################################
# This is needed for PHP
Include conf/extra/httpd-php-handlers.conf
# Server-pool management (MPM specific)
Include conf/extra/httpd-mpm.conf
# Multi-language error messages
Include conf/extra/httpd-multilang-errordoc.conf
# Fancy directory listings
Include conf/extra/httpd-autoindex.conf
# Language settings
Include conf/extra/httpd-languages.conf
# User home directories
#Include conf/extra/httpd-userdir.conf
# Real-time info on requests and configuration
Include conf/extra/httpd-info.conf
# Virtual hosts
Include conf/extra/httpd-vhosts.conf
# Local access to the Apache HTTP Server Manual
#Include conf/extra/httpd-manual.conf
# Distributed authoring and versioning (WebDAV)
Include conf/extra/httpd-dav.conf
# Various default settings
Include conf/extra/httpd-default.conf
# Secure (SSL/TLS) connections
Include conf/extra/httpd-ssl.conf
# Deflate module settings
Include conf/extra/httpd-deflate.conf
# All the DirectAdmin vhosts
Include conf/extra/directadmin-vhosts.conf
# All suPHP directives
Include conf/extra/httpd-suphp.conf
# For user configurations not maintained by DirectAdmin. Empty by default.
Include conf/extra/httpd-includes.conf
#######################################################################################
# End of included files that are rewritten by DirectAdmin                             #
#######################################################################################
<IfModule mod_ssl.c>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>

我的问题:如何更好地配置服务器以防止过载?

只是一个猜测:curl-multi-exec 返回一个你可以(并且应该(签入 while 块的值

do {
    $mrc = curl_multi_exec($mh, $active);
} while ($active && $mrc == CURLM_CALL_MULTI_PERFORM);

如 PHP 文档中所述