pandas_part4.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Handling Missing Data part-1\n",
    "---------------\n",
    "     1)  fillna() method \n",
    "     2)  Forward fill method\n",
    "     3)  Backward fill method\n",
    "     4)  dropna() method\n",
    "     \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dates</th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2/1/2012</td>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>11/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       dates    day  temp     wind-speed\n",
       "0   2/1/2012  sunny  45.0             12\n",
       "1   3/1/2012  rainy  46.0             34\n",
       "2   4/1/2012    hot  47.0             45\n",
       "3   5/1/2012    NaN   NaN             56\n",
       "4   6/1/2012    hot  49.0  Not available\n",
       "5   7/1/2012    NaN   NaN  Not available\n",
       "6   8/1/2012    hot  12.0             45\n",
       "7   9/1/2012  rainy  23.0             41\n",
       "8  10/1/2012    NaN   NaN            NaN\n",
       "9  11/1/2012    NaN   NaN            NaN"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df=pd.read_csv('season.csv')\n",
    "\n",
    "# This is our dataframe\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filling all the NaN with any number or string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If your dataframe contains NaN values then you can not do any data analysis operation because NaN doesnt make any sense to you.Sometimes it is better to fill it with any number or string.\n",
    "* To fill all the cells containing NaN with any number or string we will use \n",
    "   > ##### fillna ( your_value_goes_here ) \n",
    "Here i am replacing all the NaN values with zeros."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dates</th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2/1/2012</td>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>11/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       dates    day  temp     wind-speed\n",
       "0   2/1/2012  sunny  45.0             12\n",
       "1   3/1/2012  rainy  46.0             34\n",
       "2   4/1/2012    hot  47.0             45\n",
       "3   5/1/2012    NaN   NaN             56\n",
       "4   6/1/2012    hot  49.0  Not available\n",
       "5   7/1/2012    NaN   NaN  Not available\n",
       "6   8/1/2012    hot  12.0             45\n",
       "7   9/1/2012  rainy  23.0             41\n",
       "8  10/1/2012    NaN   NaN            NaN\n",
       "9  11/1/2012    NaN   NaN            NaN"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dates</th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2/1/2012</td>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/2012</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7/1/2012</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10/1/2012</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>11/1/2012</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       dates    day  temp     wind-speed\n",
       "0   2/1/2012  sunny  45.0             12\n",
       "1   3/1/2012  rainy  46.0             34\n",
       "2   4/1/2012    hot  47.0             45\n",
       "3   5/1/2012      0   0.0             56\n",
       "4   6/1/2012    hot  49.0  Not available\n",
       "5   7/1/2012      0   0.0  Not available\n",
       "6   8/1/2012    hot  12.0             45\n",
       "7   9/1/2012  rainy  23.0             41\n",
       "8  10/1/2012      0   0.0              0\n",
       "9  11/1/2012      0   0.0              0"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2=df.fillna(0)\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forward Fill Method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In various cases filling all NaN with same value leads to wrong conclusion. So you want to fill all the NaN of respective columns with data similar to that column. In the previous example we can see that filling zero in \"day\" column is not meaningful because zero does not denote any condition of day.What if we can fill the data similar to the respective columns. <br>\n",
    "In the forward fill method we fill the value of NaN with the previous value of the same column like if we have NaN value in temp column in fourth row then the fourth row NaN value of temp column will be filled with the value of third row value of same column in this method \n",
    " * This method can be implemented by \n",
    "         \n",
    "         fillna ( method =\" ffill \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dates</th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2/1/2012</td>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>11/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       dates    day  temp     wind-speed\n",
       "0   2/1/2012  sunny  45.0             12\n",
       "1   3/1/2012  rainy  46.0             34\n",
       "2   4/1/2012    hot  47.0             45\n",
       "3   5/1/2012    hot  47.0             56\n",
       "4   6/1/2012    hot  49.0  Not available\n",
       "5   7/1/2012    hot  49.0  Not available\n",
       "6   8/1/2012    hot  12.0             45\n",
       "7   9/1/2012  rainy  23.0             41\n",
       "8  10/1/2012  rainy  23.0             41\n",
       "9  11/1/2012  rainy  23.0             41"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2=df.fillna(method='ffill')\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can observe the 4th and 6th column of the temp and day, there is a change"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Backward Fill method "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is just opposite of forward fill method.In this method we will fill the NaN value of a dataframe with the value in the same column but next row value like if we have NaN in temp column in 4th row then we will fill it with data in 5th row in the same column.\n",
    "* This can be implemented by   \n",
    "> fillna ( method = \"bfill\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dates</th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2/1/2012</td>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>11/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       dates    day  temp     wind-speed\n",
       "0   2/1/2012  sunny  45.0             12\n",
       "1   3/1/2012  rainy  46.0             34\n",
       "2   4/1/2012    hot  47.0             45\n",
       "3   5/1/2012    hot  49.0             56\n",
       "4   6/1/2012    hot  49.0  Not available\n",
       "5   7/1/2012    hot  12.0  Not available\n",
       "6   8/1/2012    hot  12.0             45\n",
       "7   9/1/2012  rainy  23.0             41\n",
       "8  10/1/2012    NaN   NaN            NaN\n",
       "9  11/1/2012    NaN   NaN            NaN"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2=df.fillna(method=\"bfill\")\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* <b>limit=m :</b> If your dataframe is having \"n\" continuous NaN values and you want only \"m\" continuous NaN to fill by forward fill method or backward fill method then you can fill the m countinuous NaN by passing as extra argument in \n",
    "> <b>fillna(method=\"ffill\",limit=m)</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interpolation Method "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to fill your dataframe column with the value relative to that column but not copied from previous or next row of the same column. Suppose value in a column in row-2 is 30 ,the same in row-4 is 34 and row-3 is NaN in same column. In this case if you apply interpolation you will get 32 in the row-2 in the same column.This is called linear interpolation.There are other types of interpolation exist -\n",
    "* 1) linear\n",
    "* 2) quadratic\n",
    "* 3) polynomial\n",
    "* 4) time etc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dates</th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2/1/2012</td>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>11/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       dates    day  temp     wind-speed\n",
       "0   2/1/2012  sunny  45.0             12\n",
       "1   3/1/2012  rainy  46.0             34\n",
       "2   4/1/2012    hot  47.0             45\n",
       "3   5/1/2012    NaN   NaN             56\n",
       "4   6/1/2012    hot  49.0  Not available\n",
       "5   7/1/2012    NaN   NaN  Not available\n",
       "6   8/1/2012    hot  12.0             45\n",
       "7   9/1/2012  rainy  23.0             41\n",
       "8  10/1/2012    NaN   NaN            NaN\n",
       "9  11/1/2012    NaN   NaN            NaN"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dates</th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2/1/2012</td>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>48.0</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30.5</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>23.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>11/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>23.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       dates    day  temp     wind-speed\n",
       "0   2/1/2012  sunny  45.0             12\n",
       "1   3/1/2012  rainy  46.0             34\n",
       "2   4/1/2012    hot  47.0             45\n",
       "3   5/1/2012    NaN  48.0             56\n",
       "4   6/1/2012    hot  49.0  Not available\n",
       "5   7/1/2012    NaN  30.5  Not available\n",
       "6   8/1/2012    hot  12.0             45\n",
       "7   9/1/2012  rainy  23.0             41\n",
       "8  10/1/2012    NaN  23.0            NaN\n",
       "9  11/1/2012    NaN  23.0            NaN"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#By default it is linear interpolation\n",
    "df2=df.interpolate()\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Note\n",
    "* You can see that only temp column get affected because it contains values\n",
    "* Interpolation only applies to columns which contains values not strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dates</th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2/1/2012</td>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8/1/2012</td>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9/1/2012</td>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>11/1/2012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       dates    day  temp     wind-speed\n",
       "0   2/1/2012  sunny  45.0             12\n",
       "1   3/1/2012  rainy  46.0             34\n",
       "2   4/1/2012    hot  47.0             45\n",
       "3   5/1/2012    NaN   NaN             56\n",
       "4   6/1/2012    hot  49.0  Not available\n",
       "5   7/1/2012    NaN   NaN  Not available\n",
       "6   8/1/2012    hot  12.0             45\n",
       "7   9/1/2012  rainy  23.0             41\n",
       "8  10/1/2012    NaN   NaN            NaN\n",
       "9  11/1/2012    NaN   NaN            NaN"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### \"time\" method:"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "     In time method, you can fill the NaN value with data according to time value it will continue with respect to time.\n",
    "For example, if you have data like this-\n",
    "      _____________________\n",
    "     | dates     |  data   |\n",
    "     |___________|_________|\n",
    "     |01-02-2012 |   10    |\n",
    "     |02-02-2012 |   30    |\n",
    "     |03-02-2012 |   50    |\n",
    "     |05-02-2012 |   NaN   |\n",
    "     |06-02-2012 |   110   |\n",
    "     |___________|_________|\n",
    "\n",
    "You can observe that the data is increasing 20 units per day .If you apply interpolation probably you will get answer around 75 by linear method but if you apply time method you will get 90 instead of 75.Now you can see the differece in the values.\n",
    "But there is a problem if you apply time method in dataframe without conveting its index to timestamp from string.\n",
    "The following code will convert the index of df dataframe into timestamp.Here i have passes the dates column in the to_datetime() function it will return the timestamp value of dates column then after i set the dates column as its index.Now we can apply the \"time\" method successfully."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dates</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-04-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-05-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-06-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-07-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-08-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-09-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-10-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-11-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              day  temp     wind-speed\n",
       "dates                                 \n",
       "2012-02-01  sunny  45.0             12\n",
       "2012-03-01  rainy  46.0             34\n",
       "2012-04-01    hot  47.0             45\n",
       "2012-05-01    NaN   NaN             56\n",
       "2012-06-01    hot  49.0  Not available\n",
       "2012-07-01    NaN   NaN  Not available\n",
       "2012-08-01    hot  12.0             45\n",
       "2012-09-01  rainy  23.0             41\n",
       "2012-10-01    NaN   NaN            NaN\n",
       "2012-11-01    NaN   NaN            NaN"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['dates'] = pd.to_datetime(df['dates'])\n",
    "df = df.set_index('dates')\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dates</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>sunny</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>46.000000</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-04-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-05-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>47.983607</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-06-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>49.000000</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-07-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>30.803279</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-08-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-09-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-10-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-11-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              day       temp     wind-speed\n",
       "dates                                      \n",
       "2012-02-01  sunny  45.000000             12\n",
       "2012-03-01  rainy  46.000000             34\n",
       "2012-04-01    hot  47.000000             45\n",
       "2012-05-01    NaN  47.983607             56\n",
       "2012-06-01    hot  49.000000  Not available\n",
       "2012-07-01    NaN  30.803279  Not available\n",
       "2012-08-01    hot  12.000000             45\n",
       "2012-09-01  rainy  23.000000             41\n",
       "2012-10-01    NaN  23.000000            NaN\n",
       "2012-11-01    NaN  23.000000            NaN"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2=df.interpolate(\"time\")\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Droping the rows containing NaN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Droping rows containing NaN will done in following way -"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " *  Droping or deleting all the rows which contains atleast on NaN <br>\n",
    " dropna() function will drop all the rows which contains atleast one NaN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dates</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-04-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-06-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-08-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-09-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              day  temp     wind-speed\n",
       "dates                                 \n",
       "2012-02-01  sunny  45.0             12\n",
       "2012-03-01  rainy  46.0             34\n",
       "2012-04-01    hot  47.0             45\n",
       "2012-06-01    hot  49.0  Not available\n",
       "2012-08-01    hot  12.0             45\n",
       "2012-09-01  rainy  23.0             41"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2=df.dropna()\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "  *  Droping all the rows which contains all the values NaN <br>\n",
    "To drop all the rows which contains only NaN except index(here date is the index) you need to pass how=\"all\" in dropna() function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dates</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-04-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-05-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-06-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-07-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-08-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-09-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              day  temp     wind-speed\n",
       "dates                                 \n",
       "2012-02-01  sunny  45.0             12\n",
       "2012-03-01  rainy  46.0             34\n",
       "2012-04-01    hot  47.0             45\n",
       "2012-05-01    NaN   NaN             56\n",
       "2012-06-01    hot  49.0  Not available\n",
       "2012-07-01    NaN   NaN  Not available\n",
       "2012-08-01    hot  12.0             45\n",
       "2012-09-01  rainy  23.0             41"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2=df.dropna(how=\"all\")\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*  Droping all the rows which contains n valid values (n is any natural number)\n",
    "If you want to delete those rows which contain only n=2 valid values.In the same way you can delete for n=1,2,3..etc provided n<=number of columns.Here n is called thresold value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>day</th>\n",
       "      <th>temp</th>\n",
       "      <th>wind-speed</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dates</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>sunny</td>\n",
       "      <td>45.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>46.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-04-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>47.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-05-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-06-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-07-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Not available</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-08-01</th>\n",
       "      <td>hot</td>\n",
       "      <td>12.0</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-09-01</th>\n",
       "      <td>rainy</td>\n",
       "      <td>23.0</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              day  temp     wind-speed\n",
       "dates                                 \n",
       "2012-02-01  sunny  45.0             12\n",
       "2012-03-01  rainy  46.0             34\n",
       "2012-04-01    hot  47.0             45\n",
       "2012-05-01    NaN   NaN             56\n",
       "2012-06-01    hot  49.0  Not available\n",
       "2012-07-01    NaN   NaN  Not available\n",
       "2012-08-01    hot  12.0             45\n",
       "2012-09-01  rainy  23.0             41"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2=df.dropna(thresh=1)\n",
    "df2\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "        You can see the row which contains at least on valid values it remains same \n",
    "    and the row doesnt contains atleast one value gets deleted.\"not available\" is not\n",
    "    NaN so it will be considered as valid value for dropna() function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}