{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Sujet 1: Concentration de CO2 dans l'atmosphère depuis 1952" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "#import isomonth\n", "import requests\n", "import os\n", "import urllib.request\n", "data_file = \"weekly_in_situ_co2_mlo.csv\"\n", "data_url = \"https://scrippsco2.ucsd.edu/assets/data/atmospheric/stations/in_situ_co2/weekly/weekly_in_situ_co2_mlo.csv\"\n", "if not os.path.exists(data_file):\n", " urllib.request.urlretrieve(data_url, data_file)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The raw data are rather a mess. The name are not good and are not several lines (with the unity). Hence, the data are stored in a dictionnary with proper keys. Moreover, the date with several format is in the raw data. Only the year and the month is stored. \n", "\n", "Moreover, the data are string and then they are translated as float" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CO2 seasonallyCO2 seasonally adjustCO2 seasonally adjust filledCO2 seasonally filledMonthYearseasonally adjusted fitseasonally fit
2-99.99-99.99-99.99-99.9911958-99.99-99.99
3-99.99-99.99-99.99-99.9921958-99.99-99.99
4315.70314.43314.43315.7031958314.90316.19
5317.45315.16315.16317.4541958314.98317.30
6317.51314.71314.71317.5151958315.06317.86
7-99.99-99.99315.14317.2461958315.14317.24
8315.86315.19315.19315.8671958315.22315.86
9314.93316.19316.19314.9381958315.29313.99
10313.21316.09316.09313.2191958315.35312.45
11-99.99-99.99315.41312.43101958315.41312.43
12313.33315.20315.20313.33111958315.46313.61
13314.67315.43315.43314.67121958315.51314.77
14315.58315.54315.54315.5811959315.57315.62
15316.49315.86315.86316.4921959315.63316.27
16316.65315.38315.38316.6531959315.69316.98
17317.72315.41315.41317.7241959315.77318.09
18318.29315.48315.48318.2951959315.85318.66
19318.15316.03316.03318.1561959315.94318.04
20316.54315.87315.87316.5471959316.03316.67
21314.80316.07316.07314.8081959316.12314.82
22313.84316.73316.73313.8491959316.22313.31
23313.33316.33316.33313.33101959316.31313.32
24314.81316.68316.68314.81111959316.39314.54
25315.58316.35316.35315.58121959316.47315.73
26316.43316.39316.39316.4311960316.56316.61
27316.98316.35316.35316.9821960316.64317.27
28317.58316.28316.28317.5831960316.71318.03
29319.03316.70316.70319.0341960316.79319.15
30320.04317.21317.21320.0451960316.86319.68
31319.58317.47317.47319.5861960316.93319.02
...........................
728408.90408.09408.09408.9072018408.65409.42
729407.10408.63408.63407.1082018408.91407.34
730405.59409.08409.08405.5992018409.18405.67
731405.99409.61409.61405.99102018409.45405.85
732408.12410.38410.38408.12112018409.72407.49
733409.23410.15410.15409.23122018409.98409.08
734410.92410.87410.87410.9212019410.24410.31
735411.66410.90410.90411.6622019410.48411.26
736412.00410.46410.46412.0032019410.70412.25
737413.52410.73410.73413.5242019410.92413.73
738414.83411.43411.43414.8352019411.14414.54
739413.96411.39411.39413.9662019411.36413.91
740411.85411.04411.04411.8572019411.57412.34
741410.08411.62411.62410.0882019411.79410.21
742408.55412.05412.05408.5592019412.01408.49
743408.43412.06412.06408.43102019412.22408.61
744410.29412.55412.55410.29112019412.44410.20
745411.85412.78412.78411.85122019412.65411.74
746413.37413.32413.32413.3712020412.86412.92
747414.09413.33413.33414.0922020413.07413.84
748414.51412.94412.94414.5132020413.26414.85
749416.18413.35413.35416.1842020413.47416.31
750417.16413.75413.75417.1652020413.67417.08
751416.30413.75413.75416.3062020413.89416.42
752414.49413.71413.71414.4972020414.10414.85
753412.59414.16414.16412.5982020414.33412.72
754411.25414.77414.77411.2592020414.57411.03
755411.22414.85414.85411.22102020414.80411.18
756412.95415.21415.21412.95112020-99.99-99.99
757-99.99-99.99-99.99-99.99122020-99.99-99.99
\n", "

756 rows × 8 columns

\n", "
" ], "text/plain": [ " CO2 seasonally CO2 seasonally adjust CO2 seasonally adjust filled \\\n", "2 -99.99 -99.99 -99.99 \n", "3 -99.99 -99.99 -99.99 \n", "4 315.70 314.43 314.43 \n", "5 317.45 315.16 315.16 \n", "6 317.51 314.71 314.71 \n", "7 -99.99 -99.99 315.14 \n", "8 315.86 315.19 315.19 \n", "9 314.93 316.19 316.19 \n", "10 313.21 316.09 316.09 \n", "11 -99.99 -99.99 315.41 \n", "12 313.33 315.20 315.20 \n", "13 314.67 315.43 315.43 \n", "14 315.58 315.54 315.54 \n", "15 316.49 315.86 315.86 \n", "16 316.65 315.38 315.38 \n", "17 317.72 315.41 315.41 \n", "18 318.29 315.48 315.48 \n", "19 318.15 316.03 316.03 \n", "20 316.54 315.87 315.87 \n", "21 314.80 316.07 316.07 \n", "22 313.84 316.73 316.73 \n", "23 313.33 316.33 316.33 \n", "24 314.81 316.68 316.68 \n", "25 315.58 316.35 316.35 \n", "26 316.43 316.39 316.39 \n", "27 316.98 316.35 316.35 \n", "28 317.58 316.28 316.28 \n", "29 319.03 316.70 316.70 \n", "30 320.04 317.21 317.21 \n", "31 319.58 317.47 317.47 \n", ".. ... ... ... \n", "728 408.90 408.09 408.09 \n", "729 407.10 408.63 408.63 \n", "730 405.59 409.08 409.08 \n", "731 405.99 409.61 409.61 \n", "732 408.12 410.38 410.38 \n", "733 409.23 410.15 410.15 \n", "734 410.92 410.87 410.87 \n", "735 411.66 410.90 410.90 \n", "736 412.00 410.46 410.46 \n", "737 413.52 410.73 410.73 \n", "738 414.83 411.43 411.43 \n", "739 413.96 411.39 411.39 \n", "740 411.85 411.04 411.04 \n", "741 410.08 411.62 411.62 \n", "742 408.55 412.05 412.05 \n", "743 408.43 412.06 412.06 \n", "744 410.29 412.55 412.55 \n", "745 411.85 412.78 412.78 \n", "746 413.37 413.32 413.32 \n", "747 414.09 413.33 413.33 \n", "748 414.51 412.94 412.94 \n", "749 416.18 413.35 413.35 \n", "750 417.16 413.75 413.75 \n", "751 416.30 413.75 413.75 \n", "752 414.49 413.71 413.71 \n", "753 412.59 414.16 414.16 \n", "754 411.25 414.77 414.77 \n", "755 411.22 414.85 414.85 \n", "756 412.95 415.21 415.21 \n", "757 -99.99 -99.99 -99.99 \n", "\n", " CO2 seasonally filled Month Year seasonally adjusted fit \\\n", "2 -99.99 1 1958 -99.99 \n", "3 -99.99 2 1958 -99.99 \n", "4 315.70 3 1958 314.90 \n", "5 317.45 4 1958 314.98 \n", "6 317.51 5 1958 315.06 \n", "7 317.24 6 1958 315.14 \n", "8 315.86 7 1958 315.22 \n", "9 314.93 8 1958 315.29 \n", "10 313.21 9 1958 315.35 \n", "11 312.43 10 1958 315.41 \n", "12 313.33 11 1958 315.46 \n", "13 314.67 12 1958 315.51 \n", "14 315.58 1 1959 315.57 \n", "15 316.49 2 1959 315.63 \n", "16 316.65 3 1959 315.69 \n", "17 317.72 4 1959 315.77 \n", "18 318.29 5 1959 315.85 \n", "19 318.15 6 1959 315.94 \n", "20 316.54 7 1959 316.03 \n", "21 314.80 8 1959 316.12 \n", "22 313.84 9 1959 316.22 \n", "23 313.33 10 1959 316.31 \n", "24 314.81 11 1959 316.39 \n", "25 315.58 12 1959 316.47 \n", "26 316.43 1 1960 316.56 \n", "27 316.98 2 1960 316.64 \n", "28 317.58 3 1960 316.71 \n", "29 319.03 4 1960 316.79 \n", "30 320.04 5 1960 316.86 \n", "31 319.58 6 1960 316.93 \n", ".. ... ... ... ... \n", "728 408.90 7 2018 408.65 \n", "729 407.10 8 2018 408.91 \n", "730 405.59 9 2018 409.18 \n", "731 405.99 10 2018 409.45 \n", "732 408.12 11 2018 409.72 \n", "733 409.23 12 2018 409.98 \n", "734 410.92 1 2019 410.24 \n", "735 411.66 2 2019 410.48 \n", "736 412.00 3 2019 410.70 \n", "737 413.52 4 2019 410.92 \n", "738 414.83 5 2019 411.14 \n", "739 413.96 6 2019 411.36 \n", "740 411.85 7 2019 411.57 \n", "741 410.08 8 2019 411.79 \n", "742 408.55 9 2019 412.01 \n", "743 408.43 10 2019 412.22 \n", "744 410.29 11 2019 412.44 \n", "745 411.85 12 2019 412.65 \n", "746 413.37 1 2020 412.86 \n", "747 414.09 2 2020 413.07 \n", "748 414.51 3 2020 413.26 \n", "749 416.18 4 2020 413.47 \n", "750 417.16 5 2020 413.67 \n", "751 416.30 6 2020 413.89 \n", "752 414.49 7 2020 414.10 \n", "753 412.59 8 2020 414.33 \n", "754 411.25 9 2020 414.57 \n", "755 411.22 10 2020 414.80 \n", "756 412.95 11 2020 -99.99 \n", "757 -99.99 12 2020 -99.99 \n", "\n", " seasonally fit \n", "2 -99.99 \n", "3 -99.99 \n", "4 316.19 \n", "5 317.30 \n", "6 317.86 \n", "7 317.24 \n", "8 315.86 \n", "9 313.99 \n", "10 312.45 \n", "11 312.43 \n", "12 313.61 \n", "13 314.77 \n", "14 315.62 \n", "15 316.27 \n", "16 316.98 \n", "17 318.09 \n", "18 318.66 \n", "19 318.04 \n", "20 316.67 \n", "21 314.82 \n", "22 313.31 \n", "23 313.32 \n", "24 314.54 \n", "25 315.73 \n", "26 316.61 \n", "27 317.27 \n", "28 318.03 \n", "29 319.15 \n", "30 319.68 \n", "31 319.02 \n", ".. ... \n", "728 409.42 \n", "729 407.34 \n", "730 405.67 \n", "731 405.85 \n", "732 407.49 \n", "733 409.08 \n", "734 410.31 \n", "735 411.26 \n", "736 412.25 \n", "737 413.73 \n", "738 414.54 \n", "739 413.91 \n", "740 412.34 \n", "741 410.21 \n", "742 408.49 \n", "743 408.61 \n", "744 410.20 \n", "745 411.74 \n", "746 412.92 \n", "747 413.84 \n", "748 414.85 \n", "749 416.31 \n", "750 417.08 \n", "751 416.42 \n", "752 414.85 \n", "753 412.72 \n", "754 411.03 \n", "755 411.18 \n", "756 -99.99 \n", "757 -99.99 \n", "\n", "[756 rows x 8 columns]" ] }, "execution_count": 65, "metadata": {}, "output_type": "execute_result" } ], "source": [ "raw_data_prov = pd.read_csv('./monthly_in_situ_co2_mlo.csv',skiprows=54)\n", "raw_data = pd.DataFrame({\"Year\":raw_data_prov[' Yr'][2:].astype(int), \"Month\":raw_data_prov[' Mn'][2:].astype(int), \n", " \"CO2 seasonally\":raw_data_prov[' CO2'][2:].astype(float), \"CO2 seasonally adjust\":raw_data_prov['seasonally'][2:].astype(float),\n", " \"seasonally fit\":raw_data_prov[' fit'][2:].astype(float),\"seasonally adjusted fit\":raw_data_prov[' seasonally'][2:].astype(float), \n", " \"CO2 seasonally filled\":raw_data_prov[' CO2'][2:].astype(float), \"CO2 seasonally adjust filled\":raw_data_prov[' seasonally'][2:].astype(float)})\n", "#raw_data = raw_data.drop([0,1])\n", "raw_data\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "According to the authors:\n", "\n", "\"CO2 seasonally\" gives monthly Mauna Loa CO2 concentrations in micro-mol CO2 per mole (ppm), reported on the 2008A SIO manometric mole fraction scale.The monthly values have been adjusted to 24:00 hours on the 15th of each month.\n", "\n", "\"CO2 seasonally adjust\" gives the same data after a seasonal adjustment to remove the quasi-regular seasonal cycle. The adjustment involves subtracting from the data a 4-harmonic fit with a linear gain factor.\n", "\n", "\"seasonally fit\" is a smoothed version of the data generated from a stiff cubic spline function plus 4-harmonic functions with linear gain.\n", "\n", "\"seasonally adjusted fit\" is the same smoothed version with the seasonal cycle removed. \n", "\n", "\"CO2 seasonally adjust filled\" and \"CO2 seasonally filled\" are identical to Columns \"CO2 seasonally\" and \"CO2 seasonally adjust\" except that the missing values from have been filled with values from Columns \"fit\" and \"seasonally adjusted fit\"\n", "\n", "Moreover, it is said that the missing data are completed with the value -99.99\n", "\n", "The month and year can be then converted to panda Period " ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [], "source": [ "raw_data['period'] = [pd.Period(str(y)+'-'+str(m)+'-15') for y,m in zip(raw_data['Year'],raw_data['Month'])]\n", "sorted_data = raw_data.set_index('period').sort_index()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to highlight the seasonal and interannual variations, only the raw concentrations stored in 'CO2 seasonally\" and \"CO2 seasonally adjust\" are enough. The first one is the raw concentrations and contains both the seasonal and interannual variations. The second one is a adjustement made in order to get rid of the seasonal variations." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "nb_year = 10\n", "sorted_data=sorted_data[~(sorted_data[\"CO2 seasonally\"] < 0)]\n", "sorted_data[\"CO2 seasonally\"][-12*nb_year:].plot(label='raw data')\n", "sorted_data[\"CO2 seasonally adjust\"][-12*nb_year:].plot(label='interannual variation')\n", "sorted_data=sorted_data[~(sorted_data[\"seasonally fit\"] < 0)]\n", "sorted_data[\"seasonally fit\"][-12*nb_year:].plot(label='smoothed data')\n", "plt.plot([sorted_data[\"period\"][-12*nb_year],sorted_data[\"period\"][-1]],[sorted_data[\"CO2 seasonally adjust\"][-12*nb_year],sorted_data[\"CO2 seasonally adjust\"][-1]])\n", "plt.legend()\n", "#sorted_data = data.set_index('period').sort_index()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We store the difference between the monthly CO2 concentration and the interannual variations of this concentration (obtained by the adjustement). Hence, it represent the seasonnal variations of the concentration " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sorted_data[\"CO2 seasonal variations\"] = sorted_data[\"CO2 seasonally\"] - sorted_data[\"CO2 seasonally adjust\"]\n", "sorted_data[\"CO2 seasonal variations\"][-12*nb_year:].plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This seasonal variations are rather close to a cosinus function with a one year period. In the other hand, the interannual variations are nearly linear. A FAIRE : une régression linéaire pour montrer la proximité et la pente " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" } }, "nbformat": 4, "nbformat_minor": 2 }