English 中文(简体)
Mnesia 数据库的存储容量是多少?
原标题:
  • 时间:2009-01-07 18:39:46
  •  标签:

有些地方规定2GB的期限,有些地方规定取决于节点数量。

最佳回答

如果你的问题是“由大量disc_only_copies表组成的mnesia数据库的存储容量是多少”,那么相当大,你受到可用磁盘空间的严重限制。

一个更容易回答的问题是,不同类型的单个mnesia表的最大容量是多少。ram_copies表受可用内存的限制。disc_copies表受其dets后端的限制(Hakan Mattsson在Mnesia上)-目前这个限制是4Gb的数据。

因此,简单的答案是简单的disc_copies表在运行到问题之前可以存储多达4Gb的数据。(如果超过磁盘大小限制,Mnesia实际上不会崩溃-表的ram_copies部分会继续运行,因此您可以通过删除数据或在运行时进行其他安排来修复此问题

然而,如果考虑其他mnesia的特性,那答案就更加复杂了。

  • local_content tables. If the table is a local_content table, then it can have different contents on each node in the mnesia cluster, so the capacity of the table is 4Gb * <number of nodes>
  • fragmented tables. Mnesia supports user configurable table partitioning or sharding using table fragments. In this case you can effectively distribute and redistribute the data in your table over a number of primitive tables. These primitive tables can each have their own configuration - say one ram_copies table and the rest disc_only_copies tables. These primitive tables have the same size limits as mentioned earlier and now the effective capacity of the fragmented table is 4Gb * <number of fragments>. (Sadly if you fragment your table, you then have to modify your table access code to use mnesia:activity/4 instead of mnesia:write and friends, but if you plan this in advance it s managable)
  • external copies If you like living on the extreme bleeding edge, you could apply the mnesiaex patches to mnesia and store your table data in an external system such as Amazon S3 or Tokyo Cabinet. In this case the capacity of the table is limited by the backend storage.
问题回答

简而言之:Mnesia数据库的存储容量仅受可用RAM的限制。

*假设您使用表类型ram_copiesdisc_copies。此外,如果您在disc_copies表中存储了大量数据,则需要在启动时从磁盘读取,这可能会增加启动时间超出可接受范围。


这个答案与现有的两个答案在 disc_copies 类型的表方面存在矛盾。让我先把几个一般性的观点摆出来:

  • A mnesia table of type ram_copies is only limited by available RAM (except if you re on a 32-bit machine). Data is stored in an ETS table.
  • A mnesia table of type disc_only_copies is stored in a Dets table. Dets tables are limited to 2 GB, because of limits in the file format.
  • The obvious way to circumvent that limit is to create more tables, possibly through table fragmentation.
  • The schema is also stored in a Dets table, so the information describing all existing tables is also limited to 2 GB. You are likely to run into other limits before you hit that one, though.
  • A mnesia table of type disc_copies is stored both in RAM and on disk, so it is limited by available RAM - and perhaps something else?

我将尝试在下面展示,Mnesia对于disc_copies表的大小没有特定的限制。然而,许多Erlang程序员认为disc_copies表的大小限制为2 GB。这在这个问题的被接受的答案中被明确说明,在撰写本文时,该答案的分数比这个答案高出7倍。


disc_copies moved from dets to disk_log in 2001

普遍认为disc_copies表由Dets表支持。据我所知,直到Erlang/OTP R7B-4(于2001年9月30日发布)这种情况才存在。来自README

  -- mnesia -----------------------------------------------------------------

        OTP-3712 - Speed/load improvements disc_copies tables are not 
                   implemented with dets anymore.

查看差异以获取更多详细信息,特别是mnesia_lib.erlmnesia_loader.erl


Sources supporting dets and a 2 / 4 GB limit

archelaus s answer 参考自 http://erlang.org/~hakan/mnesia_consumption.txt,其中解释了 disc_copies 表存在 ets 和 dets 表中。然而,查看 目录索引,我们可以看到该文档的日期为1999年。

[TXT] mnesia_consumption.txt  26-Oct-1999 10:57    10k  

这个说法很有道理,因为它是在变更之前两年写的。

Ray Boosen的答案参考了Erlang FAQ页面:http://erlang.org/faq/mnesia.html。

11.5 How much data can be stored in Mnesia?

Dets使用32位整数作为文件偏移量,因此目前最大可能的mnesia表为4Gb。

在实际使用中,您的机器在达到这个极限之前就会变得非常缓慢。

FAQ自2001年1月(请参见Wayback Machine中最早的版本)起就已经说明了这一点。这意味着此FAQ条目早于切换到disk_log,并且已经很长时间没有更新。(无论如何,Dets表的大小限制是2 GB,而不是4 GB。)我已经提交了一份pull request以更新FAQ。


Sources supporting higher limits

《学习Erlang的Mnesia章节》说:

ram_copies
This option makes it so all data is stored exclusively in ETS, so memory only. Memory should be limited to a theoretical 4GB (and practically around 3GB) for virtual machines compiled on 32 bits, but this limit is pushed further away on 64 bits virtual machines, assuming there is more than 4GB of memory available.

disc_only_copies
This option means that the data is stored only in DETS. Disc only, and as such the storage is limited to DETS 2GB limit.

disc_copies
This option means that the data is stored both in ETS and on disk, so both memory and the hard disk. disc_copies tables are not limited by DETS limits, as Mnesia uses a complex system of transaction logs and checkpoints that allow to create a disk-based backup of the table in memory.

我不确定这是什么时候写的,但是文本上方存在于最早的 Wayback Machine 复制品中,日期为2012年4月。

在2005年11月7日的 erlang-questions的一个名为“打死mnesia(RE:使用Erlang VM的4GB RAM)”的帖子中,Ulf Wiger写道:

在一台16GB的电脑上,您可以:

  • run 6 million simultaneous processes (through use of erlang:hibernate, I was actually able to run 20 million - spawn time: 6.3 us, message passing time: 5.3 us, and I had 1.8 GB to spare.)

  • populate mnesia with at least 12 GB of data, but think through how you want to represent it, since the 64-bit word size blows things up a bit.

  • keep a 10 GB+ disc_copy table in mnesia. The load times and log dump cost seem acceptable (10 minutes to load, dumping takes a while but runs in the background quite nicely.)

Conclusions

混淆似乎源于官方信息缺失或过时。

  • The Mnesia documentation doesn t mention any table size limits
  • The Erlang FAQ says that Mnesia is subject to a 4 GB Dets size limit, but this answer was written before the dets to disk_log change
  • The only other document on the erlang.org domain is Håkan Mattsson s document, dating from before the dets to disk_log change

LYSE似乎是第一个提到 disc_copies 表不受Dets表大小限制的“权威”来源。

根据文档,这是4GB。第11.5节。

将此翻译为中文:http://erlang.org/faq/mnesia.html http://erlang.org/faq/mnesia.html





相关问题
热门标签