Also.... the pie chart will never draw with this many frags...

To fix the graph display, please try out the apc.php from CVS, it has some fixes for this. (let us know if it doesn't help)

Are you using the file upload status at all?  I noticed that this is activated and has a update frequency of 0 which will cause it to update a lot.  May want to set this to some percentage of the uploads so it's not updated every time a little bit of data is received on the socket.

Your cache requests/sec doesn't seem extremely high, is there something that is slow in particular. ie: apc_fetch times are slow, pages load slowly, apc_info(), apc.php, etc.

assuming you have json on your machine,

I'd like you to run

<?php
header("Content-Type: text/json");
echo json_encode(apc_sma_info());
?>

on that server and sort of give me numbers.

Everytime some struct changes size, something like 

http://t3.dotgnu.info/blog/php/apc-and-40bytes.html 

happens (which why I need stats to analyze)

I turned on upload feature the other day when making a progress bar but it was happening before.  I've turned it off and restarted the server, but in a few hours we will see frags again and I will do some timings on fetch and store operations.

I've put the json stuff in a script here:

http://www.tequilasolutions.com/apc_cache_json.php

I had the 40 bytes issue a while ago, I think with 3.0.15 and had mega fragmentation but the gopalv patch seemed to fix this largely.   Its probably just to do with storing loads of slightly different sized small stuff but hopefully you can have a look at the cache in a few hours and sniff something out from the json dump.

Frags are building now up to 4000 could be a good time to check?

yup, save the json data and link it here

Been running 17 hours, now showing 10000+ frags I have saved these here:

http://www.tequilasolutions.com/json_dump.txt

Say bye bye to fragmentation.

Please test out apc-3.1.x from CVS HEAD, I'm aware it may have many bugs in it - but the new allocation redesign will solve the frag issues.

Sorry Gopal, I'm saying hello to frags again, in fact its worse than 3.0.19.

http://www.tequilasolutions.com/apc_cache.php
http://www.tequilasolutions.com/apc_cache_json.php

I'm now running php 5.2.6 and APC 1.0-dev.

Thanks,

J.

Sorry - that first link to my cache should be:

http://web02.tequilasolutions.com/apc_cache.php

and to get a json dump:

http://www.tequilasolutions.com/apc_cache_json.php

I've saved the dump here as I've had to restart the server:

http://web02.tequilasolutons.com/json_dump.txt

I was getting a lot of segfaults, which I have traced to bcompiler, maybe this was causing it, I've restarted the server and will see what happens.  I need to run bcompiler though so any thoughts on why they don't play together nicely would be helpful.

==> /var/log/messages <==
Jun 30 11:00:33 web02 kernel: printk: 2 messages suppressed.
Jun 30 11:00:33 web02 kernel: php[8077]: segfault at 00000000 eip 081d6248 esp bf82cc80 error 4
Jun 30 11:00:35 web02 kernel: php[8088]: segfault at 00000000 eip 081d6248 esp bf8894e0 error 4
Jun 30 11:00:40 web02 kernel: printk: 1 messages suppressed.
Jun 30 11:00:40 web02 kernel: php[8115]: segfault at 00000000 eip 081d6248 esp bfb4b7a0 error 4

Just a quick update, no segfaults anymore after disabling bcompiler, but frags are building as usual.

How specifically are you measuring the slow down due to the fragmentation?  Is general server performance such as requests/second down or something more specific like stores/fetches/deletes etc?  

If you're interested in testing out a patch you can give this a shot, it should improve the memory management for deallocations with high fragmentation: http://tekrat.com/patches/sma_optimize.patch

The frag count is not that bad and in big chunks.

Btw, do you use the upload progress thingy in there? That's on my hitlist for a frag demon of sorts.

Hopefully, that I should be able to fix soon.

I store small user cahce items also, and high fragmentation slows down the server.
I discovered that the fragmentation goes high when negative size user cache entries appear (possibily there is no correlation)

when I manually delete negative size user cache entries the number of fragements start decreasing
after a few minutes when a negative size user cache entry come up (eg. file upload cache entry), number of fragements are 4-5 (somtetimes 10-20) thousands

Is it possible that the problem is with the file upload progress?

yes, uploadprogress does cause a *lot* of fragmentation.

I've got a fix for that in apc-3.1.2, you might want to try that.

I get the same fragmentation behaviour with 3.1.2 ... I run APC on a small webserver this explain the small values :
Request Rate (hits, misses)	0.16 cache requests/second
Hit Rate	0.10 cache requests/second
Miss Rate	0.06 cache requests/second
Insert Rate	0.10 cache requests/second

I use APC to cache page and as store engine for an antiCrawler script (one cache entry per ip with a counter). So I get a high write rate (mainly  cause by the anticrawler), and an high fragmentation rate.

I read that it's cause by writing the same entry at the same time, but as the anticrawler script is based on one entry per IP, there is a low writing conflict probability.

Is there any way to tune APC (with a mutex ... or simple lock) in order to limit fragmentation behaviour ?

Another question is what happen when the cache is totally fragmented ? Is it purge ? Or should I run a daemon (or crontab) which check this case and purge it "manually" ?

I really like using APC as user cache, mainly because I didn't need a distributed memcached (as i only have 1 server), but I am always scared of fragmentation and because this strange behaviour  I need to check my server cache status once ... a day / hour (depends on time/traffic I/my server deal with).

By the way fixing this stuff will save A LOT of time for me and I will be very gratefull for you of it.

Thanks for all the work you have already done !

@petitchevalroux: apc-3.1.2 has an apc_inc/apc_dec which works on long/int vals stored in APC.

So, instead of 

$a = apc_fetch("127.0.0.1");
apc_store("127.0.0.1", $a + 1);

you could just apc_inc("127.0.0.1");

And it'd updated in-place without the "allocate new, copy, deallocate old" approach which the other one requires.

Thanks a lot gopalv, exactly what I need, and thanks to Shire too ;).

Is there any way to use apc_inc and change LM date or created date of the entry without re-alocated it, in order to keep this entry more longer than the duration set using the first apc_store.

By the way, I give my anticrawler class here just in case I made mistake using the undocumented function apc_inc ;)

<?php
/**
 * @author Patrick Poulain
 * @see http://petitchevalroux.net/
 * @licence GPL
 *
 */
class AntiCrawler
{
	const logDuration = 120;
	const limit = 70;
	public static function run()
	{
		$remoteIp = isset($_SERVER['REMOTE_ADDR']) ? $_SERVER['REMOTE_ADDR'] : '127.0.0.1';
		if(isset($_SERVER['SERVER_NAME']) === true && $remoteIp > '192.168.254.254')
		{
			$key = 'ip-'.$remoteIp;
			$count = apc_fetch($key, $valide);
			if ($valide === false || $count === false)
			{
				apc_store($key,$count,self::logDuration);
				return true;
			}
			if($count > self::limit)
			{
				trigger_error(__CLASS__.'::run() ban '.$remoteIp.' '.date('r',$_SERVER['REQUEST_TIME']),E_USER_WARNING);
				echo ErrorPage::getContent(503,'crawler');
			}
			apc_inc($key,1);
		}
	}
}
?>

Oooppps ! I made a mistake just change 
apc_store($key,$count,self::logDuration);
By
apc_store($key,1,self::logDuration);
in the previous code ...

I've updated the latest CVS to update the entry mtime for the atomic inc/dec operations.  Give that a try and hit us up with any problems.

Does this solve your fragmentation issues and this bug, or do we still need to leave this open and fix high fragmentation?

Thanks Shire for fixing inc/dec mtime.

Using apc_inc does not really fix the fragmentation problem, it decrease fragmentation on my APC cache for the anticrawler part, but the fragmentation still pretty high using apc_store in my page caching system.

I think I will code a cron job to check fragmentation rate and clear user cache in case of High Fragmentation.

So if you think Memory Fragmentation is a normal stuff on APC user cache with "high" read/write hits you can close this bug.

Personally I understand that memory fragmentation can appears, but it is hard to say if it is a "bug" or if it is normal. It just depends on which fragmentation amount you accept as normal fragmentation.

... Just to give you some idea about amount of fragmentation : 
User Cache Information
Cached Variables	4271 ( 7.5 MBytes)
Hits	17665
Misses	13630
Request Rate (hits, misses)	0.15 cache requests/second
Hit Rate	0.08 cache requests/second
Miss Rate	0.07 cache requests/second
Insert Rate	0.05 cache requests/second
Fragmentation: 2.61% (532.1 KBytes out of 19.9 MBytes in 344 fragments) 
Uptime	2 days, 10 hours and 32 minutes

The High Miss rate is cause by the last clear (2 Days) because my page are additionnaly cached on disk and as I only cached in apc fresh page my script is serving page from file cache but still checking if the page does not exist in apc.

I delete my file cache today, so if you need more stats whithout high miss rate I can post it in a couple of days.

I'm going to go ahead and close this then as the fragmentation you're reporting doesn't seem that significant to me.  However please let us know if you see a performance drop because of this or if it gets significantly worse.  The total number of fragments should be less of an issue now as that's been optimized, but if you have a lot of space that can't be used due to fragments that may be a concern.  Thanks.

Ok thanks for your attention, and thanks for all your work on APC ;)