The GroupBy Method#

Using the groupBy object to filter the dataset and calculating its mean.

Importing libraries and packages#

1# Mathematical operations and data manipulation
2import pandas as pd

Set paths#

1# Path to datasets directory
2data_path = "./datasets"
3# Path to assets directory (for saving results to)
4assets_path = "./assets"

Loading dataset#

1dataset = pd.read_csv(f"{data_path}/cleaned_mpi_disagg_by_groups.csv")

Wrangling#

1dataset.head()
Country Type of survey Survey year Ethnic/racial/caste group MPI: Value for the country MPI: Value for the group Headcount (%) Number of multidimensionally poor people by group (thousands) Intensity of deprivation (%) Health (%) ... Cooking fuel (%) Sanitation (%) Drinking water (%) Electricity (%) Housing (%) Assets (%) Population share by group (%) Population size by group (thousands) Population size (thousands) Region
0 Bangladesh MICS 2019 Bengali 0.104060 0.102702 24.384759 39284.990511 42.117223 17.441109 ... 12.484664 8.274627 0.569494 2.308714 12.478603 8.562996 98.809242 161104.688057 163046.173 South Asia
1 Bangladesh MICS 2019 Other 0.104060 0.216783 45.868093 890.521140 47.262356 10.881517 ... 11.733451 10.198139 8.354676 8.593331 11.536150 10.738271 1.190756 1941.482818 163046.173 South Asia
2 Belize MICS 2015/2016 Creole 0.017109 0.003768 1.051818 0.940881 35.820526 52.086931 ... 1.126231 3.964365 1.126231 3.383591 6.162911 4.409921 22.916001 89.452839 390.351 Latin America and the Caribbean
3 Belize MICS 2015/2016 Garifuna 0.017109 0.003887 1.097083 0.224891 35.433114 85.184902 ... 2.963020 2.963020 0.000000 2.963020 2.963020 2.963020 5.251431 20.499014 390.351 Latin America and the Caribbean
4 Belize MICS 2015/2016 Maya 0.017109 0.078922 18.631953 8.557940 42.358151 37.911840 ... 11.931632 7.811719 2.319572 9.465594 11.165109 4.267081 11.766724 45.931523 390.351 Latin America and the Caribbean

5 rows × 26 columns

1dataset_subset = dataset.loc[
2    [i for i in range(20)],
3    ["Country", "MPI: Value for the country", "Intensity of deprivation (%)"],
4]
5dataset_subset
Country MPI: Value for the country Intensity of deprivation (%)
0 Bangladesh 0.104060 42.117223
1 Bangladesh 0.104060 47.262356
2 Belize 0.017109 35.820526
3 Belize 0.017109 35.433114
4 Belize 0.017109 42.358151
5 Belize 0.017109 36.699757
6 Belize 0.017109 39.199564
7 Bolivia, Plurinational State of 0.037754 37.935901
8 Bolivia, Plurinational State of 0.037754 33.333334
9 Bolivia, Plurinational State of 0.037754 41.581705
10 Bolivia, Plurinational State of 0.037754 43.263215
11 Bolivia, Plurinational State of 0.037754 43.184847
12 Burkina Faso 0.523424 55.149454
13 Burkina Faso 0.523424 56.443775
14 Burkina Faso 0.523424 62.004858
15 Burkina Faso 0.523424 53.393632
16 Burkina Faso 0.523424 68.189025
17 Burkina Faso 0.523424 70.047671
18 Burkina Faso 0.523424 59.310508
19 Burkina Faso 0.523424 70.925540 
1byCountry = dataset_subset.groupby(["Country"])

1byCountry.mean()
MPI: Value for the country Intensity of deprivation (%)
Country
Bangladesh 0.104060 44.689789
Belize 0.017109 37.902222
Bolivia, Plurinational State of 0.037754 39.859800
Burkina Faso 0.523424 61.933058 
1byCountry.sum()
MPI: Value for the country Intensity of deprivation (%)
Country
Bangladesh 0.208121 89.379579
Belize 0.085544 189.511111
Bolivia, Plurinational State of 0.188771 199.299002
Burkina Faso 4.187394 495.464462 
1pd.DataFrame(byCountry.describe().loc["Belize"])
Belize
MPI: Value for the country count 5.000000
mean 0.017109
std 0.000000
min 0.017109
25% 0.017109
50% 0.017109
75% 0.017109
max 0.017109
Intensity of deprivation (%) count 5.000000
mean 37.902222
std 2.890254
min 35.433114
25% 35.820526
50% 36.699757
75% 39.199564
max 42.358151 
1byCountryDeprivation = dataset.groupby(
2    ["Country", "MPI: Value for the country"]
3)
4byCountryDeprivation.describe()["Health (%)"]
count mean std min 25% 50% 75% max
Country MPI: Value for the country
Bangladesh 0.104060 2.0 14.161313 4.638332 10.881517 12.521415 14.161313 15.801211 17.441109
Belize 0.017109 5.0 52.424414 20.074087 34.317536 37.911840 52.086931 52.620862 85.184902
Bolivia, Plurinational State of 0.037754 5.0 14.536029 8.448964 0.000000 15.688203 16.573222 18.727330 21.691390
Burkina Faso 0.523424 13.0 19.808747 3.165250 11.212544 18.682662 20.461291 21.819641 23.469443
Central African Republic 0.461348 10.0 20.516219 1.603111 17.638498 20.219176 20.477726 20.797062 24.057694
Chad 0.517011 20.0 19.702261 4.549909 14.092422 16.474577 18.147315 22.312360 29.597738
Colombia 0.019657 6.0 16.508889 24.446927 0.000000 1.933060 10.183126 13.483985 64.919776
Cote d'Ivoire 0.235871 7.0 21.161356 2.962244 15.895532 19.967734 21.848881 22.683231 25.083150
Cuba 0.002689 3.0 10.508758 1.442069 9.268285 9.717630 10.166974 11.128995 12.091015
Ecuador 0.018254 6.0 37.378046 6.760842 29.363000 32.681535 36.213662 42.705620 46.104187
Gabon 0.069695 9.0 34.689746 6.696511 20.443879 33.432665 35.356405 35.701400 45.500579
Gambia 0.203638 7.0 31.119538 7.653808 23.064721 24.173717 31.733188 36.175289 42.340843
Georgia 0.001245 3.0 16.368706 28.351430 0.000000 0.000000 0.000000 24.553059 49.106118
Ghana 0.111218 9.0 22.092419 4.925323 15.389710 17.760051 21.250694 26.110834 29.887124
Guatemala 0.133518 5.0 31.366825 6.569505 26.772781 27.800102 28.915776 30.488324 42.857143
Guinea 0.373222 7.0 25.906650 10.175988 20.994255 21.611236 22.116342 23.050657 48.912169
Guinea-Bissau 0.340689 9.0 18.702386 1.895216 15.572958 17.089683 18.968948 19.653063 21.471031
Guyana 0.006592 5.0 23.903702 14.052001 0.000000 25.023599 27.684335 30.063108 36.747468
India 0.122652 5.0 31.514163 1.694231 29.337359 30.527134 31.322577 32.965096 33.418647
Kazakhstan 0.001611 3.0 88.444520 15.623967 70.668384 82.666780 94.665176 97.332588 100.000000
Kenya 0.170776 15.0 25.393678 6.077790 14.258599 21.043721 27.995216 30.591098 32.420644
Kyrgyzstan 0.001426 4.0 45.206794 31.366488 0.000000 37.500000 54.784643 62.491437 71.257889
Lao People's Democratic Republic 0.108333 5.0 21.799551 3.399659 19.055916 19.861905 20.171678 22.464549 27.443709
Malawi 0.252325 11.0 24.809829 3.296624 20.658263 22.740284 23.732468 26.799370 30.813015
Mali 0.376063 11.0 20.191291 3.597494 13.859468 18.218623 20.493524 22.200528 26.049180
Moldova, Republic of 0.003534 6.0 8.206761 11.701336 0.000000 0.000000 4.760794 9.603490 30.088189
Mongolia 0.028127 3.0 20.748427 2.003589 18.446622 20.072177 21.697732 21.899329 22.100926
Nigeria 0.254390 11.0 34.415321 7.179008 24.512718 29.505580 32.092237 39.213827 46.675504
North Macedonia 0.001422 3.0 48.545156 32.726382 24.056531 29.960591 35.864651 60.789469 85.714287
Paraguay 0.018849 5.0 15.578172 4.237301 11.850100 13.462014 13.998689 15.870105 22.709950
Peru 0.029186 8.0 21.447642 2.102016 18.069342 20.401531 21.877667 22.644197 24.161845
Philippines 0.024249 11.0 34.526625 16.831705 5.798427 21.406204 38.795000 46.543270 61.761850
Senegal 0.262862 7.0 23.878394 4.228989 19.393708 20.589546 21.829581 27.631254 29.483871
Serbia 0.000433 5.0 18.467930 21.463425 0.000000 0.000000 9.088669 41.438395 41.812584
Sierra Leone 0.292899 11.0 24.684252 4.108338 19.350915 22.676755 24.722264 26.112636 34.105322
Sri Lanka 0.011185 7.0 18.581268 17.398381 0.000000 0.000000 31.112409 32.656763 33.642942
Suriname 0.011232 7.0 32.411277 19.074112 9.465703 18.863692 26.965968 45.101759 62.516367
Togo 0.179616 7.0 19.874977 3.231174 14.488444 18.504701 20.616874 21.596494 23.817130
Trinidad and Tobago 0.002418 4.0 32.504629 22.514635 0.000000 28.282044 39.093743 43.316327 51.831028
Uganda 0.281028 14.0 25.879913 2.228499 21.582941 24.974520 25.990269 26.748037 29.726526
Viet Nam 0.019334 2.0 15.239354 1.528644 14.158440 14.698897 15.239354 15.779812 16.320269