|
|
php.net | support | documentation | report a bug | advanced search | search howto | statistics | random bug | login |
PatchesPull RequestsHistoryAllCommentsChangesGit/SVN commits
[2008-12-04 09:55 UTC] ormandj at gmail dot com
[2009-01-11 15:25 UTC] ormandj at gmail dot com
[2011-01-28 04:22 UTC] mark at hell dot ne dot jp
[2011-02-14 18:20 UTC] chris at ocproducts dot com
[2011-02-14 18:25 UTC] rasmus@php.net
|
|||||||||||||||||||||||||||
Copyright © 2001-2024 The PHP GroupAll rights reserved. |
Last updated: Sat Apr 20 05:01:27 2024 UTC |
Description: ------------ Hi, When utilized on a relatively heavily accessed website, I am encountering hung httpd processes when APC 3.0.19 is in use. Here is the output from gdb to a process, as well as strace output: (gdb) backtrace #0 0x00002b6d41480888 in __lll_mutex_lock_wait () from /lib64/libpthread.so.0 #1 0x00002b6d4147c8a5 in _L_mutex_lock_107 () from /lib64/libpthread.so.0 #2 0x00002b6d4147c333 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00002b6d4d5885b9 in apc_pthreadmutex_lock (lock=0x2b6d51072070) at /root/APC-3.0.19/apc_pthreadmutex.c:72 #4 0x00002b6d4d582d49 in apc_cache_find_slot (cache=0x2b6d5c23bfc0, key= {data = {file = {device = 2053, inode = 3932326}, user = {identifier = 0x805 "\001", identifier_len = 3932326}, fpfile = {fullpath = 0x805 "\001", fullpath_len = 3932326}}, mtime = 1123687688, type = 1 '\001'}, t=1227181723) at /root/APC-3.0.19/apc_cache.c:520 #5 0x00002b6d4d582f10 in apc_cache_find (cache=0x2b6d51072070, key= {data = {file = {device = 2053, inode = 3932326}, user = {identifier = 0x805 "\001", identifier_len = 3932326}, fpfile = {fullpath = 0x805 "\001", fullpath_len = 3932326}}, mtime = 1123687688, type = 1 '\001'}, t=0) at /root/APC-3.0.19/apc_cache.c:568 #6 0x00002b6d4d5876f8 in my_compile_file (h=0x7fff6b138990, type=2) at /root/APC-3.0.19/apc_main.c:343 #7 0x00002b6d4b20d15b in zend_execute_scripts () from /etc/httpd/modules/libphp5.so #8 0x00002b6d4b1d22c7 in php_execute_script () from /etc/httpd/modules/libphp5.so #9 0x00002b6d4b28a6c6 in php_ap2_register_hook () from /etc/httpd/modules/libphp5.so #10 0x00002b6d3f9997ea in ap_run_handler () from /usr/sbin/httpd #11 0x00002b6d3f99cc62 in ap_invoke_handler () from /usr/sbin/httpd #12 0x00002b6d3f9a7618 in ap_process_request () from /usr/sbin/httpd #13 0x00002b6d3f9a48a0 in ap_register_input_filter () from /usr/sbin/httpd #14 0x00002b6d3f9a0a42 in ap_run_process_connection () from /usr/sbin/httpd #15 0x00002b6d3f9ab22b in ap_graceful_stop_signalled () from /usr/sbin/httpd #16 0x00002b6d3f9ab4ba in ap_graceful_stop_signalled () from /usr/sbin/httpd #17 0x00002b6d3f9abd1d in ap_mpm_run () from /usr/sbin/httpd #18 0x00002b6d3f986e04 strace on parent httpd process: select(0, NULL, NULL, NULL, {0, 707000}) = 0 (Timeout) wait4(-1, 0x7fff1bbc933c, WNOHANG|WSTOPPED, NULL) = 0 select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) wait4(-1, 0x7fff1bbc933c, WNOHANG|WSTOPPED, NULL) = 0 *snip snip* strace on child httpd process (hung, all hung processes showed this): Process 9648 attached - interrupt to quit futex(0x2b19a0b01070, FUTEX_WAIT, 2, NULL) = -1 EINTR (Interrupted system call) (I believe the interrupted system call appeared when I restarted apache) httpd -l output: Compiled in modules: core.c prefork.c http_core.c mod_so.c httpd -V Server version: Apache/2.2.3 Server built: Oct 28 2008 07:22:45 Server's Module Magic Number: 20051115:3 Server loaded: APR 1.2.7, APR-Util 1.2.7 Compiled using: APR 1.2.7, APR-Util 1.2.7 Architecture: 64-bit Server MPM: Prefork threaded: no forked: yes (variable process count) Server compiled with.... -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=128 -D HTTPD_ROOT="/etc/httpd" -D SUEXEC_BIN="/usr/sbin/suexec" -D DEFAULT_PIDLOG="logs/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="logs/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="conf/mime.types" -D SERVER_CONFIG_FILE="conf/httpd.conf" ldd /usr/sbin/httpd libm.so.6 => /lib64/libm.so.6 (0x00002b1e7ada9000) libpcre.so.0 => /lib64/libpcre.so.0 (0x00002b1e7b02c000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00002b1e7b248000) libaprutil-1.so.0 => /usr/lib64/libaprutil-1.so.0 (0x00002b1e7b461000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00002b1e7b67e000) libldap-2.3.so.0 => /usr/lib64/libldap-2.3.so.0 (0x00002b1e7b8b6000) liblber-2.3.so.0 => /usr/lib64/liblber-2.3.so.0 (0x00002b1e7baf1000) libdb-4.3.so => /lib64/libdb-4.3.so (0x00002b1e7bcff000) libexpat.so.0 => /lib64/libexpat.so.0 (0x00002b1e7bff4000) libapr-1.so.0 => /usr/lib64/libapr-1.so.0 (0x00002b1e7c217000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b1e7c43e000) libdl.so.2 => /lib64/libdl.so.2 (0x00002b1e7c658000) libc.so.6 => /lib64/libc.so.6 (0x00002b1e7c85d000) libsepol.so.1 => /lib64/libsepol.so.1 (0x00002b1e7cbb0000) /lib64/ld-linux-x86-64.so.2 (0x00002b1e7ab8d000) libuuid.so.1 => /lib64/libuuid.so.1 (0x00002b1e7cdf6000) libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002b1e7cffb000) libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0 (0x00002b1e7d21d000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002b1e7d477000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002b1e7d68d000) libssl.so.6 => /lib64/libssl.so.6 (0x00002b1e7d8a6000) libcrypto.so.6 => /lib64/libcrypto.so.6 (0x00002b1e7daef000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002b1e7de38000) libnsl.so.1 => /lib64/libnsl.so.1 (0x00002b1e7e0cb000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002b1e7e2e4000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002b1e7e512000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002b1e7e714000) libz.so.1 => /usr/lib64/libz.so.1 (0x00002b1e7e93a000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002b1e7eb4e000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002b1e7ed56000) ldd /usr/lib64/php/modules/apc.so.orig librt.so.1 => /lib64/librt.so.1 (0x00002ad3557fb000) libc.so.6 => /lib64/libc.so.6 (0x00002ad355a04000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ad355d57000) /lib64/ld-linux-x86-64.so.2 (0x00000033a8c00000) Linux 2.6.18-92.1.6.el5 #1 SMP Fri Jun 20 02:36:06 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux cat /etc/php.d/apc.ini extension=apc.so apc.enabled=1 apc.shm_segments=1 apc.shm_size=128 apc.ttl=7200 apc.user_ttl=7200 apc.num_files_hint=1024 apc.mmap_file_mask=/tmp/apc.XXXXXX apc.enable_cli=1 cat /proc/sys/kernel/shmmax 68719476736 free -m total used free shared buffers cached Mem: 16121 14696 1425 0 331 11649 -/+ buffers/cache: 2715 13406 Swap: 1992 20 1972 php -m [PHP Modules] apc bz2 calendar ctype curl date dbase dom exif ftp gd gettext gmp hash iconv imap libxml mbstring mime_magic mysql mysqli openssl pcntl pcre PDO pdo_mysql pdo_sqlite posix pspell Reflection session shmop SimpleXML sockets SPL standard sysvmsg sysvsem sysvshm tokenizer wddx xml xmlreader xmlwriter xsl zlib [Zend Modules] ---------- I can not provide code, unfortunately, nor do I know where in the code base this issue occurs. One point of reference, shm_size was increased to 512, and the issue still occurred. Due to the nature of this issue on a production server, it is very difficult to re-enable APC to test further. If I can provide any more information to be of assistance, please feel free to contact me at any time.