Question

I have an old structure class like this: typedef vector<vector<string>> VARTYPE_T; which works as a single variable. This variable can hold from one value over a list to data like a table. Most values are long,double, string or double [3] for coordinates (x,y,z). I just convert them as needed. The variables are managed in a map like this : map<string,VARTYPE_T *> where the string holds the variable name. Sure, they are wrapped in classes. Also i have a tree of nodes, where each node can hold one of these variablemaps.

使用VS 2008 SP1进行操作时，我检测到了大量的内存碎片。与stlport进行对比，stlport似乎更快（20%）并且使用的内存更少（对于我的测试案例来说减少了30%）。

So the question is: What is the best implementation to solve this requirement with fast an properly used memory ? Should i write an own allocator like a pool allocator. How would you do this ?

提前道谢。

霍伊

Answer 1

将typedef vector<vector<string>> VARTYPE_T;更改为typedef deque<deque<string>> VARTYPE_T;并检查是否仍存在内存碎片化。

顺便问一下，您如何在VS 2008 SP1中测量它？

Update: I know how HP handles on HP-UX memory fragmentation. After doing some search I found an interesting link Low-fragmentation Heap. This is a quote: The low-fragmentation heap (LFH) helps to reduce heap fragmentation. To me it is interesting since it resembles the approach of HP to memory fragmentation judging of course from its description. I think (besides using deque) it is another good idea to try and check. `

Update 2: 1) Speed. You have told nothing about speed before so I don t have any guidance in order to give you some good advice. I also don t know where your program spends most of the time while executing.

If you think that it mainly spends time in converting strings to longs and doubles then you have to do this conversion only once and use it when accessing values of variables. Probably it is a good idea to store not strings but real values. For example in a union or Boost.Variant. If you think that the variable lacks indexes that let quick access to its values you should add these indexes.

2) 内存使用。您是否正在编写一个运行多天的服务器应用程序，并且应该非常关心其内存消耗？那么我已经告诉您，在Windows上使用低碎片堆，并尝试使用块大小少于16K。同时也要消除内存泄漏。当然，要对此进行检查，但就我所理解的低碎片堆应该可以处理您的申请请求。

3) 如果你编写一个服务器应用程序，那么std::vector有时并不是一个好的选择。它会分配一块内存。如果超过16K，这可能导致内存碎片化。

最后，看一下来自HP-UX的好的报告是什么样子的（为了找到内存碎片）。

Actual Heap Usage:
    Heap Start  =   0x60000000000fea38
    Heap End    =   0x6000000026580000
    Heap Size   =   642258376 bytes

Outstanding Allocations:
    251948524 bytes allocated in 4239425 blocks
                              Detailed Report

-------------------------------------------------------------------------
65343264 bytes in 1361318 blocks (25.94% of all bytes allocated)
These range in size from 48 to 48 bytes and are allocated
#0  stlp_std::__malloc_alloc::allocate(unsigned long&)   from ./libstlport.so.5.1
#1  boost::multi_index::detail::ordered_index<boost::multi_index::identity<csubs::clnt_tax_hist_t>, stlp_std::less<csubs::clnt_tax_hist_t>, boost::multi_index::detail::nth_layer<1, csubs::clnt_tax_hist_t, boost::multi_index::indexed_by<boost::multi_index::ordered_non_unique<boost::multi_index::identity<csubs::clnt_tax_hist_t>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, hrs_allocator::counting_allocator<csubs::clnt_tax_hist_t, (hrs_allocator::allocator_enums)9> >, boost::mpl::vector0<mpl_::na>, boost::multi_index::detail::ordered_non_unique_tag>::insert(csubs::clnt_tax_hist_t const&)   at _alloc.h:381



-------------------------------------------------------------------------
47510512 bytes in 848402 blocks (18.86% of all bytes allocated)
These range in size from 56 to 56 bytes and are allocated
#0  stlp_std::__malloc_alloc::allocate(unsigned long&)   from ./libstlport.so.5.1
#1  csubs::cache_impl<csubs::subs_data_t, csubs::search_policy_range<false>, csubs::erase_policy_range, hrs_allocator::counting_allocator<csubs::subs_data_t, (hrs_allocator::allocator_enums)5>, boost::multi_index::multi_index_container<csubs::subs_data_t, boost::multi_index::indexed_by<boost::multi_index::ordered_non_unique<boost::multi_index::identity<csubs::subs_data_t>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, hrs_allocator::counting_allocator<csubs::subs_data_t, (hrs_allocator::allocator_enums)5> > >::put(csubs::subs_data_t const&)   at _alloc.h:381
#2  csubs::db_cache_loader<csubs::cache_impl<csubs::subs_data_t, csubs::search_policy_range<false>, csubs::erase_policy_range, hrs_allocator::counting_allocator<csubs::subs_data_t, (hrs_allocator::allocator_enums)5>, boost::multi_index::multi_index_container<csubs::subs_data_t, boost::multi_index::indexed_by<boost::multi_index::ordered_non_unique<boost::multi_index::identity<csubs::subs_data_t>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, hrs_allocator::counting_allocator<csubs::subs_data_t, (hrs_allocator::allocator_enums)5> > >, csubs::ctrl_loader_nocheck>::operation()   at cache_subs_loaders.h:121
#3  csubs::group_subs_cache::load_caches_from_db(db_date const&, csubs::cache_options const&, otl_connect&, csubs::cache_stat*)   at ./caches/cache_subs_caches.cpp:1452

等等。

友情链接