Question

我正在使用64个比拉布和32g的援助团(如你所知)。

我有130万份档案(主持人)。我想用同样长的另一种矢量,即每一点是整个第一病媒的加权平均数,加上与这一位置的反向距离(实际上它的位置是^-0.1,而不是^-1,而是为目的)。我可以不使用mat子过滤功能,因为它只能在目前时间点之前平均使用吗? 更清楚的解释,这里就是一个3个要素的例子。

data = [ 2 6 9 ]
weights = [ 1 1/2 1/3; 1/2 1 1/2; 1/3 1/2 1 ]
results=data*weights= [ 8 11.5 12.666 ]
i.e.
8 = 2*1 + 6*1/2 + 9*1/3
11.5 = 2*1/2 + 6*1 + 9*1/2
12.666 = 2*1/3 + 6*1/2 + 9*1

因此,新病媒中的每一个点都是整个第一病媒的加权平均数,加权量为1/(从该位置+1中分离)。

I could just remake the weight vector for each point, then calculate the results vector element by element, but this requires 1.3 million iterations of a for loop, each of which contains 1.3million multiplications. I would rather use straight matrix multiplication, multiplying a 1x1.3mil by a 1.3milx1.3mil, which works in theory, but I can t load a matrix that large.

然后,我试图用手法制作矩阵图,并将其索引在书目中,因此,目前只需要矩阵的相关一栏,但时间也很长。

我无需做这项工作,因此,希望有人就利用这么多的人数和平均人数提出建议。自2006年以来我使用了^-0.1的重量,而不是^-1的重量,但并未放弃这一高点,即与原来的1个加权点相比,第100万点的加权仍然为0.25,因此,我只能将其截断,因为它要么大。

这样做是否足够?

下面是答案的代码(这样可以加以编排):

data = load( /Users/mmanary/Documents/test/insertion.txt );
data=data. ;
total=length(data);
x=1:total;
datapad=[zeros(1,total) data];
weights = ([(total+1):-1:2 1:total]).^(-.4);
weights = weights/sum(weights);
Fdata = fft(datapad);
Fweights = fft(weights);
Fresults = Fdata .* Fweights;
results = ifft(Fresults);
results = results(1:total);
plot(x,results)

Answer 1

这样做的唯一明智方式是FFT convolution,作为filter<> > /code>功能和类似功能的基础。人工操作非常容易:



% Simulate some data
n = 10^6;
x = randi(10,1,n);
xpad = [zeros(1,n) x];

% Setup smoothing kernel
k = 1 ./ [(n+1):-1:2 1:n];

% FFT convolution
Fx = fft(xpad);
Fk = fft(k);

Fxk = Fx .* Fk;

xk = ifft(Fxk);
xk = xk(1:n);


Takes less than half a second for n=10^6!

Answer 2


这可能不是这样做的最佳途径,但记忆很多,你可以肯定地将这一进程平行。

您可以构建由以下原矩阵条目构成的矩阵,其数值为i^(-1)(i = 1  130万),将其与原病媒成倍,并将所有结果汇总在一起。

因此,就你而言,产品基本上是:

a = rand(3,1);
b1 = [1 0 0;
      0 1 0;
      0 0 1];
b2 = [0 1 0;
      1 0 1;
      0 1 0] / 2;
b3 = [0 0 1;
      0 0 0;
      1 0 0] / 3;

c = sparse(b1) * a + sparse(b2) * a + sparse(b3) * a;

当然,你会不以这种方式构筑par。如果你想要减少内部通道的频率,你可以在每个矩阵中拥有一个以上<代码>i。

查阅<代码>parfor loop in MATLAB: 。

Answer 3

I can t use matlab s filter function, because it can only average things before the current point, right?

That is not correct. You can always add samples (i.e, adding or removing zeros) from your data or from the filtered data. Since filtering with filter (you can also use conv by the way) is a linear action, it won t change the result (it s like adding and removing zeros, which does nothing, and then filtering. Then linearity allows you to swap the order to add samples -> filter -> remove sample).

Anyway, in your example, you can take the averaging kernel to be:

weights = 1 ./ [3 2 1 2 3]; % this kernel introduces a delay of 2 samples

之后简单地说:

result =  filter(w,1,[data, zeros(1,3)]); % or conv (data, w)
% removing the delay introduced by the kernel
result = result (3:end-1);

Answer 4

You considered only 2 options: Multiplying 1.3M*1.3M matrix with a vector once or multiplying 2 1.3M vectors 1.3M times.

但是,你可以按照你的愿望,把你的权重矩阵划分到尽可能多的子矩阵上,并且用病媒1.3M/n的频率复制N*1.3M矩阵。

我认为,最快的是,只有数量最少的频率,而且只有这样,才能创造最适合你们记忆的最大次矩阵,而没有使你的电脑开始向你们的硬盘旋。

您的记忆量应从0=5000开始。

您也可通过使用按级(<代码>n>,按加工商数目分列)使其更快。

Answer 5

强势方式很可能对你们有利,在混合过程中稍有选择。

计算重量的^-0.1业务将比+和*业务要长得多,以补偿加权-means,但你在所有100万加权-mean行动中重新使用权重。算法成为:

Create a weightings vector with all the weights any computation would need: weights = (-n:n).^-0.1
For each element in the vector:
- 索引:<代码> 重量/代码>中流放部分,以视现有元素为中心。
- Perform the weighted-mean with the weights portion and the entire vector. This can be done with a fast vector dot-multiply followed by a scalar division.

主要的休息时间是n^2 添加和补充。 <n>等于130万,即3.4万亿美元。现代3GHz CPU的单一核心内容可以说,第二点是60亿增加/重复,这样大约10分钟。添加时间对<代码>重量矢量和间接费用进行索引编制,我仍然估计,你可能半小时以下。

友情链接