English 中文(简体)
re-arrange data by pairs recursively
原标题:

I have dataframe contains ACQ/REL pair recusively as below:

import pandas as pd

data = [
    [ 2023-06-05 16:51:27.561 , ACQ , location ],    
    [ 2023-06-05 16:51:27.564 , ACQ , location ],
    [ 2023-06-05 16:51:27.567 , ACQ , location ],
    [ 2023-06-05 16:51:27.571 , REL , location ],
    [ 2023-06-05 16:51:27.573 , REL , location ],
    [ 2023-06-05 16:51:27.587 , REL , location ],
    [ 2023-06-05 16:51:28.559 , ACQ , location ],
    [ 2023-06-05 16:51:28.561 , ACQ , location ],
    [ 2023-06-05 16:51:28.563 , ACQ , location ],
    [ 2023-06-05 16:51:28.566 , REL , location ],
    [ 2023-06-05 16:51:28.569 , REL , location ],
    [ 2023-06-05 16:51:28.575 , REL , location ]
]

df = pd.DataFrame(data,columns=[ ts , action , name ])

I would re-orgnize it by ACQ/REL pairs, the outer ACQ/REL pairs as a group, so that the output dataframe looks like below:

0   2023-06-05 16:51:27.561    ACQ  location
5   2023-06-05 16:51:27.587    REL  location
1   2023-06-05 16:51:27.564    ACQ  location
4   2023-06-05 16:51:27.573    REL  location
2   2023-06-05 16:51:27.567    ACQ  location
3   2023-06-05 16:51:27.571    REL  location
6   2023-06-05 16:51:28.559    ACQ  location
11  2023-06-05 16:51:28.575    REL  location
7   2023-06-05 16:51:28.561    ACQ  location
10  2023-06-05 16:51:28.569    REL  location
8   2023-06-05 16:51:28.563    ACQ  location
9   2023-06-05 16:51:28.566    REL  location

Current example is 3 pairs a group but it s not constantly the same. What s proper way to get such results?

问题回答

Try this:

df.sort_values( ts ).assign(sortkey=df.groupby( action ).cumcount()).sort_values([ sortkey , action ])

Output:

                         ts action      name  sortkey
0   2023-06-05 16:51:27.561    ACQ  location        0
3   2023-06-05 16:51:27.571    REL  location        0
1   2023-06-05 16:51:27.564    ACQ  location        1
4   2023-06-05 16:51:27.573    REL  location        1
2   2023-06-05 16:51:27.567    ACQ  location        2
5   2023-06-05 16:51:27.587    REL  location        2
6   2023-06-05 16:51:28.559    ACQ  location        3
9   2023-06-05 16:51:28.566    REL  location        3
7   2023-06-05 16:51:28.561    ACQ  location        4
10  2023-06-05 16:51:28.569    REL  location        4
8   2023-06-05 16:51:28.563    ACQ  location        5
11  2023-06-05 16:51:28.575    REL  location        5

If it is always guaranteed that there are the same number of ACQ and REL, and that they are respectively in correct order (i.e., the n-th ACQ should always be paired with the n-th REL), then the solution is to split the original list into two first, by its second element, then loop to output one from each list.

Code:

import pandas as pd

data = [
    [ 2023-06-05 16:51:27.561 ,  ACQ ,  location ],
    [ 2023-06-05 16:51:27.564 ,  ACQ ,  location ],
    [ 2023-06-05 16:51:27.567 ,  ACQ ,  location ],
    [ 2023-06-05 16:51:27.571 ,  REL ,  location ],
    [ 2023-06-05 16:51:27.573 ,  REL ,  location ],
    [ 2023-06-05 16:51:27.587 ,  REL ,  location ],
    [ 2023-06-05 16:51:28.559 ,  ACQ ,  location ],
    [ 2023-06-05 16:51:28.561 ,  ACQ ,  location ],
    [ 2023-06-05 16:51:28.563 ,  ACQ ,  location ],
    [ 2023-06-05 16:51:28.566 ,  REL ,  location ],
    [ 2023-06-05 16:51:28.569 ,  REL ,  location ],
    [ 2023-06-05 16:51:28.575 ,  REL ,  location ]
]

df = pd.DataFrame(data, columns=[ ts ,  action ,  name ])
# print(df)

data_acq = []
data_rel = []

for index, row in df.iterrows():
    if row[ action ] ==  ACQ :
        data_acq.append(row)
    elif row[ action ] ==  REL :
        data_rel.append(row)
assert len(data_acq) == len(data_rel)

df_new = pd.DataFrame([], columns=[ ts ,  action ,  name ])

for j in range(len(data_acq)):
    df_new = pd.concat([
        df_new,
        pd.DataFrame([
            data_acq[j], data_rel[j]
        ])
    ])
print(df_new)

Output:

                         ts action      name
0   2023-06-05 16:51:27.561    ACQ  location
3   2023-06-05 16:51:27.571    REL  location
1   2023-06-05 16:51:27.564    ACQ  location
4   2023-06-05 16:51:27.573    REL  location
2   2023-06-05 16:51:27.567    ACQ  location
5   2023-06-05 16:51:27.587    REL  location
6   2023-06-05 16:51:28.559    ACQ  location
9   2023-06-05 16:51:28.566    REL  location
7   2023-06-05 16:51:28.561    ACQ  location
10  2023-06-05 16:51:28.569    REL  location
8   2023-06-05 16:51:28.563    ACQ  location
11  2023-06-05 16:51:28.575    REL  location

I don t know if this can be translated into pandas equivalents, but in plain Python with accumulate from itertools, you can compute a list of offsets for the REL that indicate how far back the corresponding ACQ is located. Then sort the indices according to the adjusted positions (ACQ remaining at their original position and REL offset back to the ACQ s position) to get the indexes in the new order:

from itertools import accumulate

isREL   = (int(x=="REL") for _,x,_ in data)
offsets = accumulate(isREL,lambda a,b:(a or 1)*b-2*b)
order   = (i for i,_ in sorted(enumerate(offsets),key=sum))
data    = [data[i] for i in order]

print(*data,sep="
")

[ 2023-06-05 16:51:27.561 ,  ACQ ,  location ]
[ 2023-06-05 16:51:27.587 ,  REL ,  location ]
[ 2023-06-05 16:51:27.564 ,  ACQ ,  location ]
[ 2023-06-05 16:51:27.573 ,  REL ,  location ]
[ 2023-06-05 16:51:27.567 ,  ACQ ,  location ]
[ 2023-06-05 16:51:27.571 ,  REL ,  location ]
[ 2023-06-05 16:51:28.559 ,  ACQ ,  location ]
[ 2023-06-05 16:51:28.575 ,  REL ,  location ]
[ 2023-06-05 16:51:28.561 ,  ACQ ,  location ]
[ 2023-06-05 16:51:28.569 ,  REL ,  location ]
[ 2023-06-05 16:51:28.563 ,  ACQ ,  location ]
[ 2023-06-05 16:51:28.566 ,  REL ,  location ]




相关问题
Can Django models use MySQL functions?

Is there a way to force Django models to pass a field to a MySQL function every time the model data is read or loaded? To clarify what I mean in SQL, I want the Django model to produce something like ...

An enterprise scheduler for python (like quartz)

I am looking for an enterprise tasks scheduler for python, like quartz is for Java. Requirements: Persistent: if the process restarts or the machine restarts, then all the jobs must stay there and ...

How to remove unique, then duplicate dictionaries in a list?

Given the following list that contains some duplicate and some unique dictionaries, what is the best method to remove unique dictionaries first, then reduce the duplicate dictionaries to single ...

What is suggested seed value to use with random.seed()?

Simple enough question: I m using python random module to generate random integers. I want to know what is the suggested value to use with the random.seed() function? Currently I am letting this ...

How can I make the PyDev editor selectively ignore errors?

I m using PyDev under Eclipse to write some Jython code. I ve got numerous instances where I need to do something like this: import com.work.project.component.client.Interface.ISubInterface as ...

How do I profile `paster serve` s startup time?

Python s paster serve app.ini is taking longer than I would like to be ready for the first request. I know how to profile requests with middleware, but how do I profile the initialization time? I ...

Pragmatically adding give-aways/freebies to an online store

Our business currently has an online store and recently we ve been offering free specials to our customers. Right now, we simply display the special and give the buyer a notice stating we will add the ...

Converting Dictionary to List? [duplicate]

I m trying to convert a Python dictionary into a Python list, in order to perform some calculations. #My dictionary dict = {} dict[ Capital ]="London" dict[ Food ]="Fish&Chips" dict[ 2012 ]="...

热门标签