By beardc


2013-02-06 16:55:54 8 Comments

How do I access the corresponding groupby dataframe in a groupby object by the key?

With the following groupby:

rand = np.random.RandomState(1)
df = pd.DataFrame({'A': ['foo', 'bar'] * 3,
                   'B': rand.randn(6),
                   'C': rand.randint(0, 20, 6)})
gb = df.groupby(['A'])

I can iterate through it to get the keys and groups:

In [11]: for k, gp in gb:
             print 'key=' + str(k)
             print gp
key=bar
     A         B   C
1  bar -0.611756  18
3  bar -1.072969  10
5  bar -2.301539  18
key=foo
     A         B   C
0  foo  1.624345   5
2  foo -0.528172  11
4  foo  0.865408  14

I would like to be able to access a group by its key:

In [12]: gb['foo']
Out[12]:  
     A         B   C
0  foo  1.624345   5
2  foo -0.528172  11
4  foo  0.865408  14

But when I try doing that with gb[('foo',)] I get this weird pandas.core.groupby.DataFrameGroupBy object thing which doesn't seem to have any methods that correspond to the DataFrame I want.

The best I could think of is:

In [13]: def gb_df_key(gb, key, orig_df):
             ix = gb.indices[key]
             return orig_df.ix[ix]

         gb_df_key(gb, 'foo', df)
Out[13]:
     A         B   C
0  foo  1.624345   5
2  foo -0.528172  11
4  foo  0.865408  14  

but this is kind of nasty, considering how nice pandas usually is at these things.
What's the built-in way of doing this?

5 comments

@meyerson 2015-05-31 16:31:49

I was looking for a way to sample a few members of the GroupBy obj - had to address the posted question to get this done.

create groupby object

grouped = df.groupby('some_key')

pick N dataframes and grab their indicies

sampled_df_i  = random.sample(grouped.indicies, N)

grab the groups

df_list  = map(lambda df_i: grouped.get_group(df_i), sampled_df_i)

optionally - turn it all back into a single dataframe object

sampled_df = pd.concat(df_list, axis=0, join='outer')

@irene 2020-03-25 16:17:44

This doesn't work: sampled_df_i = random.sample(grouped.indicies, N)

@meyerson 2020-03-27 16:00:24

@irene - can you provide a link to a longer example/more context?

@irene 2020-03-31 14:26:14

I get the following error: AttributeError: 'DataFrameGroupBy' object has no attribute 'indicies'

@Andy Hayden 2013-02-06 17:00:27

You can use the get_group method:

In [21]: gb.get_group('foo')
Out[21]: 
     A         B   C
0  foo  1.624345   5
2  foo -0.528172  11
4  foo  0.865408  14

Note: This doesn't require creating an intermediary dictionary / copy of every subdataframe for every group, so will be much more memory-efficient that creating the naive dictionary with dict(iter(gb)). This is because it uses data-structures already available in the groupby object.


You can select different columns using the groupby slicing:

In [22]: gb[["A", "B"]].get_group("foo")
Out[22]:
     A         B
0  foo  1.624345
2  foo -0.528172
4  foo  0.865408

In [23]: gb["C"].get_group("foo")
Out[23]:
0     5
2    11
4    14
Name: C, dtype: int64

@Surya 2016-04-30 06:35:40

gb = df.groupby(['A'])

gb_groups = grouped_df.groups

If you are looking for selective groupby objects then, do: gb_groups.keys(), and input desired key into the following key_list..

gb_groups.keys()

key_list = [key1, key2, key3 and so on...]

for key, values in gb_groups.iteritems():
    if key in key_list:
        print df.ix[values], "\n"

@LegitMe 2015-03-04 05:25:09

Rather than

gb.get_group('foo')

I prefer using gb.groups

df.loc[gb.groups['foo']]

Because in this way you can choose multiple columns as well. for example:

df.loc[gb.groups['foo'],('A','B')]

@Andy Hayden 2016-06-14 21:33:51

Note: You can select different columns using gb[["A", "B"]].get_group("foo").

@JD Margulici 2013-06-25 16:27:52

Wes McKinney (pandas' author) in Python for Data Analysis provides the following recipe:

groups = dict(list(gb))

which returns a dictionary whose keys are your group labels and whose values are DataFrames, i.e.

groups['foo']

will yield what you are looking for:

     A         B   C
0  foo  1.624345   5
2  foo -0.528172  11
4  foo  0.865408  14

@Zhubarb 2014-01-14 13:39:54

Thank you, this is very useful. How can I modify the code to make groups = dict(list(gb)) only store column C? Let's say I am not interested in the other columns and therefore do not want to store them.

@Zhubarb 2014-01-15 13:27:52

Answer: dict(list( df.groupby(['A'])['C'] ))

@Andy Hayden 2014-03-10 22:54:03

Note: it's more efficient (but equivalent) to use dict(iter(g)). (although get_group is the best way / as it doesn't involve creating a dictionary / keeps you in pandas! :D )

@user2476665 2016-03-18 00:20:46

I wasn't able to use groups(dict(list(gb)) but you can create a dictionary the following way: gb_dict = {str(indx): str(val) for indx in gb.indx for val in gb.some_key} and then retrieve value via gb_dict[some_key]

@smci 2019-11-12 23:58:05

Just use get_group() , this recipe has not been needed for years.

Related Questions

Sponsored Content

22 Answered Questions

[SOLVED] How to iterate over rows in a DataFrame in Pandas?

23 Answered Questions

[SOLVED] Adding new column to existing DataFrame in Python pandas

39 Answered Questions

[SOLVED] How do I check whether a file exists without exceptions?

15 Answered Questions

[SOLVED] Delete column from pandas DataFrame

20 Answered Questions

[SOLVED] Accessing the index in 'for' loops?

  • 2009-02-06 22:47:54
  • Joan Venge
  • 2114021 View
  • 3599 Score
  • 20 Answer
  • Tags:   python loops list

17 Answered Questions

[SOLVED] Selecting multiple columns in a pandas dataframe

26 Answered Questions

[SOLVED] Renaming columns in pandas

19 Answered Questions

[SOLVED] Get list from pandas DataFrame column headers

16 Answered Questions

[SOLVED] Check if a given key already exists in a dictionary

  • 2009-10-21 19:05:09
  • Mohan Gulati
  • 3584780 View
  • 2683 Score
  • 16 Answer
  • Tags:   python dictionary

10 Answered Questions

[SOLVED] How to select rows from a DataFrame based on column values?

Sponsored Content