View Issue Details

IDProjectCategoryView StatusLast Update
0000521LDMud 3.5Runtimepublic2011-02-13 21:38
Reporterzesstra Assigned Tozesstra  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
Target Version3.5.0Fixed in Version3.5.0 
Summary0000521: Overflows in string statistics
DescriptionSorry for yet another overflow in the statistics. ;-)

String handler:
--------------- Strings Bytes (Data+Overhead)
Total asked for 7786294 281646482 (149279484+132366998)
Total allocated 729111 51079496 ( 38422465+ 12657031)
 - tabled 616083 43650345 ( 32914790+ 10735555)
 - untabled 113028 7429151 ( 5507675+ 1921476)

Space required vs. 'regular C' string implementation: 5% with, 25% without
overhead.
Searches by address: 161432956 - found: 161432956 (100.0%) - avg length: 1.094
Searches by content: 881756876 - found: 4118127974 (467.0%) - avg length: 0.679
Hash chains used: 65529 of 65536 (100.0%)
Distinct strings added: 162049039 - deleted: 161432956
Collisions: 161224279 (99.5% added)
Equality tests: 2628322655 total, 2946161777 by table (112.1%), 638101843 by
hash (24.3%)
Comparisons: 1917844876 total, 2810187620 by table (146.5%)
Table lookups for existence: 513163480, 109167934 by table (21.3%), 403995546 by content (78.7%)

As you see, there are at least 3 overflows (searches, equality tests and comparisons). Additionally the comparison to C strings seems to be flawed (space required should be less without overhead).
The counters are already p_uint so we probably would have to go the way suggested in 0000520 for solving it (giga-searches etc.).

BTW: This is probably an issue to nearly all the statistics counters for heavily used operations in larger muds, e.g. outgoing traffic in network i/o as well, which is limited to 4GB. (Our uptime is only 24 days until now.)
TagsNo tags attached.

Relationships

related to 0000608 resolvedzesstra Overflows in statistics / counters in swap.c 

Activities

zesstra

2008-01-26 15:47

administrator   ~0000589

Mhmm, wouldn't it be the simplest solution to change these counters (I think, a few more overflowed in the meantime in Morgengrauen) to int64/int64_t/long long? I think, that is less complicated and less expensive at runtime than introducing corresponding giga counters...
Is there any serious disadvantage I do not see at the moment?

zesstra

2008-07-16 16:57

administrator   ~0000738

I am not sure how to handle this. Actually, I would like to just change these counters to a larger int. That should be mp_uint, but as long as mp_uint is the same size as p_uint, it won't help on ILP32 platforms. That would only change on ILP32 platforms, if we change mp_uint to uintmax_t, which we can't until 0000556 ist resolved.
So, we could wait with this bug until we solved 0000556 and uintmax_t or ulong long or uint64_t on systems which have one. Or we introduce that Gigacounter strategy, but that is more complicated...

Gnomi

2008-07-17 00:24

manager   ~0000739

I would use (unsigned) long long int. We need a new typedef in port.h for that, so it can be used also on ILP32 platforms.

Gnomi

2008-07-17 00:28

manager   ~0000740

Erm, right, that means waiting for 0000556.

zesstra

2009-06-03 12:12

administrator   ~0001188

Ok, I will introduce a new typedef statcounter_t with corresponding modifiers for printf in port.h and change all the heavily used stat counters to this type.

zesstra

2009-06-03 14:04

administrator   ~0001190

Resolved in r2626.

Issue History

Date Modified Username Field Change
2007-10-18 07:23 zesstra New Issue
2008-01-26 15:47 zesstra Note Added: 0000589
2008-07-16 16:57 zesstra Note Added: 0000738
2008-07-17 00:24 Gnomi Note Added: 0000739
2008-07-17 00:28 Gnomi Note Added: 0000740
2008-07-17 02:53 zesstra Status new => assigned
2008-07-17 02:53 zesstra Assigned To => zesstra
2009-02-15 08:01 zesstra Relationship added related to 0000608
2009-06-03 11:58 zesstra Project LDMud 3.3 => LDMud 3.5
2009-06-03 12:12 zesstra Note Added: 0001188
2009-06-03 14:04 zesstra Note Added: 0001190
2009-06-03 14:04 zesstra Status assigned => resolved
2009-06-03 14:04 zesstra Fixed in Version => 3.5.0
2009-06-03 14:04 zesstra Resolution open => fixed
2011-02-13 21:38 zesstra Target Version => 3.5.0