examples/13_personalized_bandit_comparison.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "62076295",
   "metadata": {},
   "source": [
    "# RePlay bandit models comparison\n",
    "\n",
    "We will show the main RePlay functionality and compare performance of RePlay models on well-known MovieLens dataset. For simplicity we consider here only the various bandit algorithms, both context-free and context-aware. The list of considered strategies for comparison:\n",
    "\n",
    "Context-free algorithms:\n",
    "* Most popular;\n",
    "* Vanilla UCB algorithm;\n",
    "* Vanilla TS algorithm; (Beta-Binomial);\n",
    "* KL-UCB [Smb et al];\n",
    "\n",
    "Contextual bandits algorithms:\n",
    "* Lin-UCB [Smb et al];\n",
    "* Linear TS [Smb et al] (Thompson sampling with linear feature vectors);\n",
    "\n",
    "\n",
    "### Dataset\n",
    "We will compare RePlay models on __MovieLens 1m__. \n",
    "\n",
    "### Dataset preprocessing: \n",
    "Ratings greater than or equal to 3 are considered as positive interactions.\n",
    "\n",
    "### Data split\n",
    "Dataset is split by date so that 20% of the last interactions as are placed in the test part. Cold items and users are dropped.\n",
    "\n",
    "### Predict:\n",
    "We will predict top-10 most relevant films for each user.\n",
    "\n",
    "### Metrics\n",
    "Quality metrics used:__ndcg@k, hitrate@k, map@k, mrr@k__ for k = 1, 5, 10\n",
    "Additional metrics used: __coverage@k__ and __surprisal@k__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "60dd3ff3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ! pip install rs-datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1dd41af0",
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0bbb8512",
   "metadata": {},
   "outputs": [],
   "source": [
    "%config Completer.use_jedi = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e55939ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "from optuna.exceptions import ExperimentalWarning\n",
    "warnings.filterwarnings(\"ignore\", category=UserWarning)\n",
    "warnings.filterwarnings(\"ignore\", category=ExperimentalWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6e65584",
   "metadata": {},
   "source": [
    "`State` object allows passing existing Spark session or create a new one, which will be used by the all RePlay modules.\n",
    "\n",
    "To create session with custom parameters ``spark.driver.memory`` and ``spark.sql.shuffle.partitions`` use function `get_spark_session` from `session_handler` module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "74edb16a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type='text/css'>\n",
       ".datatable table.frame { margin-bottom: 0; }\n",
       ".datatable table.frame thead { border-bottom: none; }\n",
       ".datatable table.frame tr.coltypes td {  color: #FFFFFF;  line-height: 6px;  padding: 0 0.5em;}\n",
       ".datatable .bool    { background: #DDDD99; }\n",
       ".datatable .object  { background: #565656; }\n",
       ".datatable .int     { background: #5D9E5D; }\n",
       ".datatable .float   { background: #4040CC; }\n",
       ".datatable .str     { background: #CC4040; }\n",
       ".datatable .time    { background: #40CC40; }\n",
       ".datatable .row_index {  background: var(--jp-border-color3);  border-right: 1px solid var(--jp-border-color0);  color: var(--jp-ui-font-color3);  font-size: 9px;}\n",
       ".datatable .frame tbody td { text-align: left; }\n",
       ".datatable .frame tr.coltypes .row_index {  background: var(--jp-border-color0);}\n",
       ".datatable th:nth-child(2) { padding-left: 12px; }\n",
       ".datatable .hellipsis {  color: var(--jp-cell-editor-border-color);}\n",
       ".datatable .vellipsis {  background: var(--jp-layout-color0);  color: var(--jp-cell-editor-border-color);}\n",
       ".datatable .na {  color: var(--jp-cell-editor-border-color);  font-size: 80%;}\n",
       ".datatable .sp {  opacity: 0.25;}\n",
       ".datatable .footer { font-size: 9px; }\n",
       ".datatable .frame_dimensions {  background: var(--jp-border-color3);  border-top: 1px solid var(--jp-border-color0);  color: var(--jp-ui-font-color3);  display: inline-block;  opacity: 0.6;  padding: 1px 10px 1px 5px;}\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import logging\n",
    "import time\n",
    "\n",
    "from pyspark.sql import functions as sf, types as st\n",
    "from pyspark.sql.types import IntegerType\n",
    "\n",
    "from replay.data import Dataset, FeatureHint, FeatureInfo, FeatureSchema, FeatureType\n",
    "from replay.experimental.preprocessing.data_preparator import Indexer, DataPreparator\n",
    "from replay.metrics import Experiment\n",
    "from replay.metrics import Coverage, HitRate, MRR, MAP, NDCG, Surprisal\n",
    "from replay.models import (\n",
    "    PopRec, \n",
    "    RandomRec,\n",
    "    UCB,\n",
    "    Wilson, \n",
    "    ThompsonSampling,\n",
    "    LinUCB, # added LinUCB \n",
    ")\n",
    "\n",
    "from replay.models.base_rec import HybridRecommender\n",
    "from replay.utils.session_handler import State\n",
    "from replay.splitters import TimeSplitter\n",
    "from replay.utils.spark_utils import convert2spark, get_log_info\n",
    "from rs_datasets import MovieLens\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f6917809",
   "metadata": {},
   "outputs": [],
   "source": [
    "spark = State().session\n",
    "spark"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "83fabd91",
   "metadata": {},
   "outputs": [],
   "source": [
    "spark.sparkContext.setLogLevel('ERROR')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2d473bfb",
   "metadata": {},
   "outputs": [],
   "source": [
    "logger = logging.getLogger(\"replay\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "4d9ed2bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "K = 10\n",
    "K_list_metrics = [1, 5, 10]\n",
    "BUDGET = 5\n",
    "BUDGET_NN = 2\n",
    "SEED = 12345"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60b558a9",
   "metadata": {},
   "source": [
    "## 0. Preprocessing <a name='data-preparator'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b749b239",
   "metadata": {},
   "source": [
    "### 0.1 Data loading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4ea48703",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ratings\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>item_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1193</td>\n",
       "      <td>5</td>\n",
       "      <td>978300760</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>661</td>\n",
       "      <td>3</td>\n",
       "      <td>978302109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>914</td>\n",
       "      <td>3</td>\n",
       "      <td>978301968</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  item_id  rating  timestamp\n",
       "0        1     1193       5  978300760\n",
       "1        1      661       3  978302109\n",
       "2        1      914       3  978301968"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "users\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>age</th>\n",
       "      <th>occupation</th>\n",
       "      <th>zip_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>48067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>M</td>\n",
       "      <td>56</td>\n",
       "      <td>16</td>\n",
       "      <td>70072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>M</td>\n",
       "      <td>25</td>\n",
       "      <td>15</td>\n",
       "      <td>55117</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id gender  age  occupation zip_code\n",
       "0        1      F    1          10    48067\n",
       "1        2      M   56          16    70072\n",
       "2        3      M   25          15    55117"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "items\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_id</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Animation|Children's|Comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children's|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   item_id                    title                        genres\n",
       "0        1         Toy Story (1995)   Animation|Children's|Comedy\n",
       "1        2           Jumanji (1995)  Adventure|Children's|Fantasy\n",
       "2        3  Grumpier Old Men (1995)                Comedy|Romance"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "data = MovieLens(\"1m\")\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1f5be67f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>item_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1193</td>\n",
       "      <td>5</td>\n",
       "      <td>978300760</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>661</td>\n",
       "      <td>3</td>\n",
       "      <td>978302109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>914</td>\n",
       "      <td>3</td>\n",
       "      <td>978301968</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3408</td>\n",
       "      <td>4</td>\n",
       "      <td>978300275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>2355</td>\n",
       "      <td>5</td>\n",
       "      <td>978824291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000204</th>\n",
       "      <td>6040</td>\n",
       "      <td>1091</td>\n",
       "      <td>1</td>\n",
       "      <td>956716541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000205</th>\n",
       "      <td>6040</td>\n",
       "      <td>1094</td>\n",
       "      <td>5</td>\n",
       "      <td>956704887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000206</th>\n",
       "      <td>6040</td>\n",
       "      <td>562</td>\n",
       "      <td>5</td>\n",
       "      <td>956704746</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000207</th>\n",
       "      <td>6040</td>\n",
       "      <td>1096</td>\n",
       "      <td>4</td>\n",
       "      <td>956715648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000208</th>\n",
       "      <td>6040</td>\n",
       "      <td>1097</td>\n",
       "      <td>4</td>\n",
       "      <td>956715569</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000209 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         user_id  item_id  rating  timestamp\n",
       "0              1     1193       5  978300760\n",
       "1              1      661       3  978302109\n",
       "2              1      914       3  978301968\n",
       "3              1     3408       4  978300275\n",
       "4              1     2355       5  978824291\n",
       "...          ...      ...     ...        ...\n",
       "1000204     6040     1091       1  956716541\n",
       "1000205     6040     1094       5  956704887\n",
       "1000206     6040      562       5  956704746\n",
       "1000207     6040     1096       4  956715648\n",
       "1000208     6040     1097       4  956715569\n",
       "\n",
       "[1000209 rows x 4 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.ratings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b5d9cfe",
   "metadata": {},
   "source": [
    "#### log preprocessing\n",
    "\n",
    "- converting to spark dataframe\n",
    "- renaming columns\n",
    "- checking for nulls\n",
    "- converting timestamp to Timestamp format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f48a174d",
   "metadata": {},
   "outputs": [],
   "source": [
    "preparator = DataPreparator()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ddc4385e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:32:03, replay, INFO: Columns with ids of users or items are present in mapping. The dataframe will be treated as an interactions log.\n",
      "/home/sudakovcom/Desktop/RePlayHDILab2024/.conda/lib/python3.8/site-packages/pyspark/sql/pandas/conversion.py:471: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.\n",
      "  arrow_data = [[(c, t) for (_, c), t in zip(pdf_slice.iteritems(), arrow_types)]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 18.8 ms, sys: 6.84 ms, total: 25.7 ms\n",
      "Wall time: 2.43 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "log = preparator.transform(columns_mapping={'user_id': 'user_id',\n",
    "                                      'item_id': 'item_id',\n",
    "                                      'relevance': 'rating',\n",
    "                                      'timestamp': 'timestamp'\n",
    "                                     }, \n",
    "                           data=data.ratings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "e7e2623a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "836478"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# will consider ratings >= 3 as positive feedback. A positive feedback is treated with relevance = 1\n",
    "only_positives_log = log.filter(sf.col('relevance') >= 3).withColumn('relevance', sf.lit(1))\n",
    "only_positives_log.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "078ff4ce",
   "metadata": {},
   "source": [
    "<a id='indexing'></a>\n",
    "### 0.2. Indexing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57797c7d",
   "metadata": {},
   "source": [
    "Convert given users' and items' identifiers (\\_id) to integers starting at zero without gaps (\\_idx) with Indexer class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "c6c4308a",
   "metadata": {},
   "outputs": [],
   "source": [
    "indexer = Indexer(user_col='user_id', item_col='item_id')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87098cba",
   "metadata": {},
   "source": [
    "Take all available user and item ids from log and features and pass them to Indexer. The _ids_ could repeat, the indexes will be ordered by label frequencies, so the most frequent label gets index 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "64fdc590",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 12.6 ms, sys: 4.69 ms, total: 17.3 ms\n",
      "Wall time: 1.1 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "indexer.fit(users=log.select('user_id'),\n",
    "           items=log.select('item_id'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "6edcc503",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+---------+-------------------+\n",
      "|user_idx|item_idx|relevance|          timestamp|\n",
      "+--------+--------+---------+-------------------+\n",
      "|    4131|      43|        1|2001-01-01 01:12:40|\n",
      "|    4131|     585|        1|2001-01-01 01:35:09|\n",
      "+--------+--------+---------+-------------------+\n",
      "only showing top 2 rows\n",
      "\n",
      "CPU times: user 17 ms, sys: 3.24 ms, total: 20.3 ms\n",
      "Wall time: 890 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "836478"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "log_replay = indexer.transform(df=only_positives_log)\n",
    "log_replay.show(2)\n",
    "log_replay.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "528bbff3",
   "metadata": {},
   "source": [
    "### 0.2. Data split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "77706881",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train info:\n",
      " total lines: 669181, total users: 5397, total items: 3569\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test info:\n",
      " total lines: 86542, total users: 1139, total items: 3279\n"
     ]
    }
   ],
   "source": [
    "# train/test split \n",
    "train_spl = TimeSplitter(\n",
    "    time_threshold=0.2,\n",
    "    drop_cold_items=True,\n",
    "    drop_cold_users=True,\n",
    "    query_column=\"user_idx\",\n",
    "    item_column=\"item_idx\",\n",
    ")\n",
    "\n",
    "train, test = train_spl.split(log_replay)\n",
    "print('train info:\\n', get_log_info(train))\n",
    "print('test info:\\n', get_log_info(test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "193687a6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(535343, 24241)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# train/test split for hyperparameters selection\n",
    "opt_train, opt_val = train_spl.split(train)\n",
    "opt_train.count(), opt_val.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "5eb1f855",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "798993"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# negative feedback will be used for Wilson and UCB models\n",
    "only_negatives_log = indexer.transform(df=log.filter(sf.col('relevance') < 3).withColumn('relevance', sf.lit(0.)))\n",
    "test_start = test.agg(sf.min('timestamp')).collect()[0][0]\n",
    "\n",
    "# train with both positive and negative feedback\n",
    "pos_neg_train=(train\n",
    "              .withColumn('relevance', sf.lit(1.))\n",
    "              .union(only_negatives_log.filter(sf.col('timestamp') < test_start))\n",
    "             )\n",
    "pos_neg_train.cache()\n",
    "pos_neg_train.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "855797ec",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+---------+-------------------+\n",
      "|user_idx|item_idx|relevance|          timestamp|\n",
      "+--------+--------+---------+-------------------+\n",
      "|     677|    1314|      1.0|2000-12-02 08:30:12|\n",
      "|     677|    1282|      1.0|2000-12-02 08:53:52|\n",
      "|     677|     731|      1.0|2000-12-02 08:41:26|\n",
      "|     677|     234|      1.0|2000-12-02 08:23:47|\n",
      "|     677|     190|      1.0|2000-12-02 08:50:33|\n",
      "|     677|     133|      1.0|2000-12-02 08:27:28|\n",
      "|     677|     546|      1.0|2000-12-02 08:32:02|\n",
      "|     677|    2090|      1.0|2000-12-02 08:53:17|\n",
      "|     677|     421|      1.0|2000-12-02 08:50:13|\n",
      "|     677|     154|      1.0|2000-12-02 08:44:14|\n",
      "|     677|      96|      1.0|2000-12-02 08:11:05|\n",
      "|     677|     221|      1.0|2000-12-02 08:58:08|\n",
      "|     677|     395|      1.0|2000-12-02 08:33:02|\n",
      "|     677|      19|      1.0|2000-12-02 08:46:50|\n",
      "|     677|      73|      1.0|2000-12-02 08:31:30|\n",
      "|     677|     182|      1.0|2000-12-02 08:18:10|\n",
      "|     677|     836|      1.0|2000-12-02 08:47:30|\n",
      "|     677|    1668|      1.0|2000-12-02 08:55:22|\n",
      "|     677|     436|      1.0|2000-12-02 08:55:39|\n",
      "|     677|     586|      1.0|2000-12-02 08:38:27|\n",
      "+--------+--------+---------+-------------------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "pos_neg_train.show(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd32dd01",
   "metadata": {},
   "source": [
    "# 2. Models training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "1a7708d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_predict_add_res(name, model, experiment, train, test, suffix=''):\n",
    "    \"\"\"\n",
    "    Run fit_predict for the `model`, measure time on fit_predict and evaluate metrics\n",
    "    \"\"\"\n",
    "    start_time=time.time()\n",
    "    \n",
    "    dataset = {'dataset': train}\n",
    "    predict_params = {'k': K, 'queries': test.interactions.select('user_idx').distinct()}\n",
    "    \n",
    "    if isinstance(model, (Wilson, UCB, ThompsonSampling, LinUCB)):\n",
    "        dataset['dataset'] = train_neg_dataset\n",
    "    \n",
    "    predict_params.update(dataset)\n",
    "\n",
    "    model.fit(**dataset)\n",
    "    fit_time = time.time() - start_time\n",
    "\n",
    "    pred=model.predict(**predict_params)\n",
    "    pred.show(100)\n",
    "    pred.cache()\n",
    "    predict_time = time.time() - start_time - fit_time\n",
    "\n",
    "    experiment.add_result(name + suffix, pred)\n",
    "    metric_time = time.time() - start_time - fit_time - predict_time\n",
    "    experiment.results.loc[name + suffix, 'fit_time'] = fit_time\n",
    "    experiment.results.loc[name + suffix, 'predict_time'] = predict_time\n",
    "    experiment.results.loc[name + suffix, 'metric_time'] = metric_time\n",
    "    experiment.results.loc[name + suffix, 'full_time'] = (fit_time + \n",
    "                                                          predict_time +\n",
    "                                                          metric_time)\n",
    "    pred.unpersist()\n",
    "    print(experiment.results[['NDCG@{}'.format(K), 'MRR@{}'.format(K), 'Coverage@{}'.format(K), 'fit_time']].sort_values('NDCG@{}'.format(K), ascending=False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "1ba41392",
   "metadata": {},
   "outputs": [],
   "source": [
    "def full_pipeline(models, experiment, train, test, suffix='', budget=BUDGET):\n",
    "    \"\"\"\n",
    "    For each model:\n",
    "        -  if required: run hyperparameters search, set best params and save param values to `experiment`\n",
    "        - pass model to `fit_predict_add_res`        \n",
    "    \"\"\"\n",
    "    \n",
    "    for name, [model, params] in models.items():\n",
    "        model.logger.info(msg='{} started'.format(name))\n",
    "\n",
    "        if params != 'no_opt':\n",
    "            model.logger.info(msg='{} optimization started'.format(name))\n",
    "            best_params = model.optimize(opt_train_dataset, \n",
    "                                         opt_val_dataset, \n",
    "                                         param_borders=params, \n",
    "                                         k=K, \n",
    "                                         budget=budget)\n",
    "            logger.info(msg='best params for {} are: {}'.format(name, best_params))\n",
    "            model.set_params(**best_params)\n",
    "\n",
    "        \n",
    "        logger.info(msg='{} fit_predict started'.format(name))\n",
    "        fit_predict_add_res(name, model, experiment, train, test, suffix)\n",
    "        # here we call protected attribute to get all parameters set during model initialization\n",
    "        experiment.results.loc[name + suffix, 'params'] = str(model._init_args)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a97865b",
   "metadata": {},
   "source": [
    "### 2.1. Contextual bandit models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9843e50a",
   "metadata": {},
   "source": [
    "### 2.1.1 item features preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "25b627cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:32:47, replay, INFO: Column with ids of users or items is absent in mapping. The dataframe will be treated as a users'/items' features dataframe.\n",
      "/home/sudakovcom/Desktop/RePlayHDILab2024/.conda/lib/python3.8/site-packages/pyspark/sql/pandas/conversion.py:471: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.\n",
      "  arrow_data = [[(c, t) for (_, c), t in zip(pdf_slice.iteritems(), arrow_types)]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+----------------+--------------------+\n",
      "|item_idx|           title|              genres|\n",
      "+--------+----------------+--------------------+\n",
      "|      29|Toy Story (1995)|Animation|Childre...|\n",
      "|     393|  Jumanji (1995)|Adventure|Childre...|\n",
      "+--------+----------------+--------------------+\n",
      "only showing top 2 rows\n",
      "\n",
      "+-------------+\n",
      "|max(item_idx)|\n",
      "+-------------+\n",
      "|         3882|\n",
      "+-------------+\n",
      "\n",
      "+-------------+\n",
      "|min(item_idx)|\n",
      "+-------------+\n",
      "|            0|\n",
      "+-------------+\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3883"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "item_features_original = preparator.transform(columns_mapping={'item_id': 'item_id'}, \n",
    "                           data=data.items)\n",
    "item_features = indexer.transform(df=item_features_original)\n",
    "item_features.show(2)\n",
    "#different item features\n",
    "\n",
    "from pyspark.sql.functions import max,min\n",
    "item_features.select(max(item_features.item_idx)).show()\n",
    "item_features.select(min(item_features.item_idx)).show()\n",
    "#just to check that the indexing is dense between 0 and 3882\n",
    "item_features.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "8daa46b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+----+\n",
      "|item_idx|year|\n",
      "+--------+----+\n",
      "|      29|1995|\n",
      "|     393|1995|\n",
      "+--------+----+\n",
      "only showing top 2 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "year = item_features.withColumn('year', sf.substring(sf.col('title'), -5, 4).astype(st.IntegerType())).select('item_idx', 'year')\n",
    "year.show(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "cc2004f5",
   "metadata": {},
   "outputs": [],
   "source": [
    "genres = (\n",
    "    item_features.select(\n",
    "        \"item_idx\",\n",
    "        sf.split(\"genres\", \"\\|\").alias(\"genres\")\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "8713a0db",
   "metadata": {},
   "outputs": [],
   "source": [
    "genres_list = (\n",
    "    genres.select(sf.explode(\"genres\").alias(\"genre\"))\n",
    "    .distinct().filter('genre <> \"(no genres listed)\"')\n",
    "    .toPandas()[\"genre\"].tolist()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "c37ba0a1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Mystery',\n",
       " 'Action',\n",
       " 'Documentary',\n",
       " \"Children's\",\n",
       " 'Drama',\n",
       " 'Adventure',\n",
       " 'Film-Noir',\n",
       " 'Crime',\n",
       " 'Animation',\n",
       " 'Fantasy',\n",
       " 'Comedy',\n",
       " 'Western',\n",
       " 'Romance',\n",
       " 'Thriller',\n",
       " 'War',\n",
       " 'Sci-Fi',\n",
       " 'Musical',\n",
       " 'Horror']"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "genres_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "a62da2a4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DataFrame[item_idx: int, Mystery: int, Action: int, Documentary: int, Children's: int, Drama: int, Adventure: int, Film-Noir: int, Crime: int, Animation: int, Fantasy: int, Comedy: int, Western: int, Romance: int, Thriller: int, War: int, Sci-Fi: int, Musical: int, Horror: int, year: int]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "item_features = genres\n",
    "for genre in genres_list:\n",
    "    item_features = item_features.withColumn(\n",
    "        genre,\n",
    "        sf.array_contains(sf.col(\"genres\"), genre).astype(IntegerType())\n",
    "    )\n",
    "item_features = item_features.drop(\"genres\").cache()\n",
    "item_features.count()\n",
    "item_features = item_features.join(year, on='item_idx', how='inner')\n",
    "item_features.cache()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "6ede60ae",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/sudakovcom/Desktop/RePlayHDILab2024/.conda/lib/python3.8/site-packages/pyspark/sql/pandas/conversion.py:371: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.\n",
      "  for column, series in pdf.iteritems():\n"
     ]
    }
   ],
   "source": [
    "item_features = spark.createDataFrame(pd.concat([item_features.drop(\"year\").toPandas(), pd.get_dummies(item_features.toPandas().year)], axis = 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "2a689fb7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+-------+------+-----------+----------+-----+---------+---------+-----+---------+-------+------+-------+-------+--------+---+------+-------+------+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+\n",
      "|item_idx|Mystery|Action|Documentary|Children's|Drama|Adventure|Film-Noir|Crime|Animation|Fantasy|Comedy|Western|Romance|Thriller|War|Sci-Fi|Musical|Horror|1919|1920|1921|1922|1923|1925|1926|1927|1928|1929|1930|1931|1932|1933|1934|1935|1936|1937|1938|1939|1940|1941|1942|1943|1944|1945|1946|1947|1948|1949|1950|1951|1952|1953|1954|1955|1956|1957|1958|1959|1960|1961|1962|1963|1964|1965|1966|1967|1968|1969|1970|1971|1972|1973|1974|1975|1976|1977|1978|1979|1980|1981|1982|1983|1984|1985|1986|1987|1988|1989|1990|1991|1992|1993|1994|1995|1996|1997|1998|1999|2000|\n",
      "+--------+-------+------+-----------+----------+-----+---------+---------+-----+---------+-------+------+-------+-------+--------+---+------+-------+------+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+\n",
      "|      29|      0|     0|          0|         1|    0|        0|        0|    0|        1|      0|     1|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     393|      0|     0|          0|         1|    0|        1|        0|    0|        0|      1|     0|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     648|      0|     0|          0|         0|    0|        0|        0|    0|        0|      0|     1|      0|      1|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    1574|      0|     0|          0|         0|    1|        0|        0|    0|        0|      0|     1|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    1066|      0|     0|          0|         0|    0|        0|        0|    0|        0|      0|     1|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     233|      0|     1|          0|         0|    0|        0|        0|    1|        0|      0|     0|      0|      0|       1|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     689|      0|     0|          0|         0|    0|        0|        0|    0|        0|      0|     1|      0|      1|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    2293|      0|     0|          0|         1|    0|        1|        0|    0|        0|      0|     0|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    1998|      0|     1|          0|         0|    0|        0|        0|    0|        0|      0|     0|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     259|      0|     1|          0|         0|    0|        1|        0|    0|        0|      0|     0|      0|      0|       1|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     194|      0|     0|          0|         0|    1|        0|        0|    0|        0|      0|     1|      0|      1|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    1635|      0|     0|          0|         0|    0|        0|        0|    0|        0|      0|     1|      0|      0|       0|  0|     0|      0|     1|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    2020|      0|     0|          0|         1|    0|        0|        0|    0|        1|      0|     0|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    1670|      0|     0|          0|         0|    1|        0|        0|    0|        0|      0|     0|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    1706|      0|     1|          0|         0|    0|        1|        0|    0|        0|      0|     0|      0|      1|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     413|      0|     0|          0|         0|    1|        0|        0|    0|        0|      0|     0|      0|      0|       1|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     292|      0|     0|          0|         0|    1|        0|        0|    0|        0|      0|     0|      0|      1|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    1649|      0|     0|          0|         0|    0|        0|        0|    0|        0|      0|     0|      0|      0|       1|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|     815|      0|     0|          0|         0|    0|        0|        0|    0|        0|      0|     1|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "|    1637|      0|     1|          0|         0|    0|        0|        0|    0|        0|      0|     0|      0|      0|       0|  0|     0|      0|     0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   0|   1|   0|   0|   0|   0|   0|\n",
      "+--------+-------+------+-----------+----------+-----+---------+---------+-----+---------+-------+------+-------+-------+--------+---+------+-------+------+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "item_features = item_features.drop(\"year\")\n",
    "item_features.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fcb0ed3",
   "metadata": {},
   "source": [
    "### 2.1.2 User features preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "cfb6442a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>age</th>\n",
       "      <th>occupation</th>\n",
       "      <th>zip_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>48067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>M</td>\n",
       "      <td>56</td>\n",
       "      <td>16</td>\n",
       "      <td>70072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>M</td>\n",
       "      <td>25</td>\n",
       "      <td>15</td>\n",
       "      <td>55117</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>M</td>\n",
       "      <td>45</td>\n",
       "      <td>7</td>\n",
       "      <td>02460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>M</td>\n",
       "      <td>25</td>\n",
       "      <td>20</td>\n",
       "      <td>55455</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id gender  age  occupation zip_code\n",
       "0        1      F    1          10    48067\n",
       "1        2      M   56          16    70072\n",
       "2        3      M   25          15    55117\n",
       "3        4      M   45           7    02460\n",
       "4        5      M   25          20    55455"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.users.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "f6d72c48",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:33:02, replay, INFO: Column with ids of users or items is absent in mapping. The dataframe will be treated as a users'/items' features dataframe.\n",
      "/home/sudakovcom/Desktop/RePlayHDILab2024/.conda/lib/python3.8/site-packages/pyspark/sql/pandas/conversion.py:471: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.\n",
      "  arrow_data = [[(c, t) for (_, c), t in zip(pdf_slice.iteritems(), arrow_types)]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_idx</th>\n",
       "      <th>gender</th>\n",
       "      <th>age</th>\n",
       "      <th>occupation</th>\n",
       "      <th>zip_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4131</td>\n",
       "      <td>F</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>48067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2364</td>\n",
       "      <td>M</td>\n",
       "      <td>56</td>\n",
       "      <td>16</td>\n",
       "      <td>70072</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_idx gender  age  occupation zip_code\n",
       "0      4131      F    1          10    48067\n",
       "1      2364      M   56          16    70072"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#same preprocessing for users as was done in 2.4.1.\n",
    "user_features_original = preparator.transform(columns_mapping={'user_id': 'user_id'}, \n",
    "                           data=data.users)\n",
    "user_features = indexer.transform(df=user_features_original)\n",
    "#switch for a while into pandas\n",
    "user_features = user_features.toPandas()\n",
    "user_features.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "c2862ac6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "22d6e02c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "max ocupation index:  20\n",
      "min ocupation index:  0\n",
      "different zip codes:  3439\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_410703/49124892.py:20: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only.\n",
      "  users_pd = pd.concat([users_pd.drop(columnsToEncode, 1),\n",
      "/home/sudakovcom/Desktop/RePlayHDILab2024/.conda/lib/python3.8/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_idx</th>\n",
       "      <th>agegroup_20-29</th>\n",
       "      <th>agegroup_30-39</th>\n",
       "      <th>agegroup_40-49</th>\n",
       "      <th>agegroup_51-60</th>\n",
       "      <th>agegroup_&lt;20</th>\n",
       "      <th>gender_F</th>\n",
       "      <th>gender_M</th>\n",
       "      <th>occupation_0</th>\n",
       "      <th>occupation_1</th>\n",
       "      <th>...</th>\n",
       "      <th>occupation_11</th>\n",
       "      <th>occupation_12</th>\n",
       "      <th>occupation_13</th>\n",
       "      <th>occupation_14</th>\n",
       "      <th>occupation_15</th>\n",
       "      <th>occupation_16</th>\n",
       "      <th>occupation_17</th>\n",
       "      <th>occupation_18</th>\n",
       "      <th>occupation_19</th>\n",
       "      <th>occupation_20</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4131</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2364</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4217</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5916</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1603</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 29 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_idx  agegroup_20-29  agegroup_30-39  agegroup_40-49  agegroup_51-60  \\\n",
       "0      4131             0.0             0.0             0.0             0.0   \n",
       "1      2364             0.0             0.0             0.0             1.0   \n",
       "2      4217             1.0             0.0             0.0             0.0   \n",
       "3      5916             0.0             0.0             1.0             0.0   \n",
       "4      1603             1.0             0.0             0.0             0.0   \n",
       "\n",
       "   agegroup_<20  gender_F  gender_M  occupation_0  occupation_1  ...  \\\n",
       "0           1.0       1.0       0.0           0.0           0.0  ...   \n",
       "1           0.0       0.0       1.0           0.0           0.0  ...   \n",
       "2           0.0       0.0       1.0           0.0           0.0  ...   \n",
       "3           0.0       0.0       1.0           0.0           0.0  ...   \n",
       "4           0.0       0.0       1.0           0.0           0.0  ...   \n",
       "\n",
       "   occupation_11  occupation_12  occupation_13  occupation_14  occupation_15  \\\n",
       "0            0.0            0.0            0.0            0.0            0.0   \n",
       "1            0.0            0.0            0.0            0.0            0.0   \n",
       "2            0.0            0.0            0.0            0.0            1.0   \n",
       "3            0.0            0.0            0.0            0.0            0.0   \n",
       "4            0.0            0.0            0.0            0.0            0.0   \n",
       "\n",
       "   occupation_16  occupation_17  occupation_18  occupation_19  occupation_20  \n",
       "0            0.0            0.0            0.0            0.0            0.0  \n",
       "1            1.0            0.0            0.0            0.0            0.0  \n",
       "2            0.0            0.0            0.0            0.0            0.0  \n",
       "3            0.0            0.0            0.0            0.0            0.0  \n",
       "4            0.0            0.0            0.0            0.0            1.0  \n",
       "\n",
       "[5 rows x 29 columns]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"max ocupation index: \", user_features['occupation'].max())\n",
    "print(\"min ocupation index: \", user_features['occupation'].min())\n",
    "count_diff_zips = user_features['zip_code'].unique().size\n",
    "print(\"different zip codes: \", count_diff_zips) #ok, too much different zip codes, let us drop them for now\n",
    "users_pd = user_features.drop(columns=['zip_code'])\n",
    "users_pd.head()\n",
    "#binarize age variable\n",
    "bins = [0, 20, 30, 40, 50, 60, np.inf]\n",
    "names = ['<20', '20-29', '30-39','40-49', '51-60', '60+']\n",
    "\n",
    "users_pd['agegroup'] = pd.cut(users_pd['age'], bins, labels=names)\n",
    "users_pd = users_pd.drop([\"age\"], axis = 1)\n",
    "users_pd.head()\n",
    "\n",
    "#binarize following https://github.com/kfoofw/bandit_simulations/tree/master\n",
    "columnsToEncode = [\"agegroup\", \"gender\", \"occupation\"]\n",
    "oheEncoder = OneHotEncoder(sparse=False, handle_unknown='ignore')\n",
    "myEncoder.fit(users_pd[columnsToEncode])\n",
    "\n",
    "users_pd = pd.concat([users_pd.drop(columnsToEncode, 1),\n",
    "                           pd.DataFrame(myEncoder.transform(users_pd[columnsToEncode]), \n",
    "                                        columns = myEncoder.get_feature_names_out(columnsToEncode))], axis=1).reindex()\n",
    "users_pd.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "3d47e5d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- user_idx: integer (nullable = true)\n",
      " |-- agegroup_20-29: double (nullable = true)\n",
      " |-- agegroup_30-39: double (nullable = true)\n",
      " |-- agegroup_40-49: double (nullable = true)\n",
      " |-- agegroup_51-60: double (nullable = true)\n",
      " |-- agegroup_<20: double (nullable = true)\n",
      " |-- gender_F: double (nullable = true)\n",
      " |-- gender_M: double (nullable = true)\n",
      " |-- occupation_0: double (nullable = true)\n",
      " |-- occupation_1: double (nullable = true)\n",
      " |-- occupation_2: double (nullable = true)\n",
      " |-- occupation_3: double (nullable = true)\n",
      " |-- occupation_4: double (nullable = true)\n",
      " |-- occupation_5: double (nullable = true)\n",
      " |-- occupation_6: double (nullable = true)\n",
      " |-- occupation_7: double (nullable = true)\n",
      " |-- occupation_8: double (nullable = true)\n",
      " |-- occupation_9: double (nullable = true)\n",
      " |-- occupation_10: double (nullable = true)\n",
      " |-- occupation_11: double (nullable = true)\n",
      " |-- occupation_12: double (nullable = true)\n",
      " |-- occupation_13: double (nullable = true)\n",
      " |-- occupation_14: double (nullable = true)\n",
      " |-- occupation_15: double (nullable = true)\n",
      " |-- occupation_16: double (nullable = true)\n",
      " |-- occupation_17: double (nullable = true)\n",
      " |-- occupation_18: double (nullable = true)\n",
      " |-- occupation_19: double (nullable = true)\n",
      " |-- occupation_20: double (nullable = true)\n",
      "\n",
      "+--------+--------------+--------------+--------------+--------------+------------+--------+--------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+\n",
      "|user_idx|agegroup_20-29|agegroup_30-39|agegroup_40-49|agegroup_51-60|agegroup_<20|gender_F|gender_M|occupation_0|occupation_1|occupation_2|occupation_3|occupation_4|occupation_5|occupation_6|occupation_7|occupation_8|occupation_9|occupation_10|occupation_11|occupation_12|occupation_13|occupation_14|occupation_15|occupation_16|occupation_17|occupation_18|occupation_19|occupation_20|\n",
      "+--------+--------------+--------------+--------------+--------------+------------+--------+--------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+\n",
      "|    4131|           0.0|           0.0|           0.0|           0.0|         1.0|     1.0|     0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          1.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    2364|           0.0|           0.0|           0.0|           1.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          1.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    4217|           1.0|           0.0|           0.0|           0.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          1.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    5916|           0.0|           0.0|           1.0|           0.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         1.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    1603|           1.0|           0.0|           0.0|           0.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          1.0|\n",
      "|    3594|           0.0|           0.0|           1.0|           0.0|         0.0|     1.0|     0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         1.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    5226|           0.0|           1.0|           0.0|           0.0|         0.0|     0.0|     1.0|         0.0|         1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    2240|           1.0|           0.0|           0.0|           0.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          1.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    2779|           1.0|           0.0|           0.0|           0.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          1.0|          0.0|          0.0|          0.0|\n",
      "|     596|           0.0|           1.0|           0.0|           0.0|         0.0|     1.0|     0.0|         0.0|         1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    2256|           1.0|           0.0|           0.0|           0.0|         0.0|     1.0|     0.0|         0.0|         1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    5705|           1.0|           0.0|           0.0|           0.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          1.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    2725|           0.0|           0.0|           1.0|           0.0|         0.0|     0.0|     1.0|         0.0|         1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    5551|           0.0|           1.0|           0.0|           0.0|         0.0|     0.0|     1.0|         1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    1573|           1.0|           0.0|           0.0|           0.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         1.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    4964|           0.0|           1.0|           0.0|           0.0|         0.0|     1.0|     0.0|         1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    1476|           0.0|           0.0|           1.0|           0.0|         0.0|     0.0|     1.0|         0.0|         1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|     925|           0.0|           0.0|           0.0|           0.0|         1.0|     1.0|     0.0|         0.0|         0.0|         0.0|         1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    1193|           0.0|           0.0|           0.0|           0.0|         1.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          1.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "|    5646|           1.0|           0.0|           0.0|           0.0|         0.0|     0.0|     1.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|         0.0|          0.0|          0.0|          0.0|          0.0|          1.0|          0.0|          0.0|          0.0|          0.0|          0.0|          0.0|\n",
      "+--------+--------------+--------------+--------------+--------------+------------+--------+--------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+\n",
      "only showing top 20 rows\n",
      "\n",
      "total users:  6040\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/sudakovcom/Desktop/RePlayHDILab2024/.conda/lib/python3.8/site-packages/pyspark/sql/pandas/conversion.py:471: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.\n",
      "  arrow_data = [[(c, t) for (_, c), t in zip(pdf_slice.iteritems(), arrow_types)]\n"
     ]
    }
   ],
   "source": [
    "#make it pyspark\n",
    "user_features = spark.createDataFrame(users_pd)\n",
    "user_features.printSchema()\n",
    "user_features.show()\n",
    "print(\"total users: \",user_features.count())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb1fefd9",
   "metadata": {},
   "source": [
    "## 2.2. Fitting various bandits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "4fa54598",
   "metadata": {},
   "outputs": [],
   "source": [
    "bandit_models = {\n",
    "          'Popular': [PopRec(), 'no_opt'], \n",
    "          'Random (uniform)': [RandomRec(seed=SEED, distribution='uniform'), 'no_opt'], \n",
    "          'Random (popularity-based)': [RandomRec(seed=SEED, distribution='popular_based'), {\"alpha\": [-0.5, 100]}],\n",
    "          'UCB': [UCB(exploration_coef=2.0), 'no_opt'], #2.0 as default, 0.5 as original \n",
    "          'Wilson': [Wilson(), 'no_opt'],\n",
    "          'TS (context-free)': [ThompsonSampling(),'no_opt'],\n",
    "          'Linear UCB (eps = -10.0)(disjoint)':[LinUCB(eps = -10.0, alpha = 1.0, is_hybrid = False), 'no_opt'],\n",
    "          'Linear UCB (eps = -10.0)(hybrid)':[LinUCB(eps = -10.0, alpha = 1.0, is_hybrid = True), 'no_opt'],\n",
    "          'Linear UCB (eps = -5.0)(disjoint)':[LinUCB(eps = -5.0, alpha = 1.0, is_hybrid = False), 'no_opt'],\n",
    "          'Linear UCB (eps = -5.0)(hybrid)':[LinUCB(eps = -5.0, alpha = 1.0, is_hybrid = True), 'no_opt'],\n",
    "          'Linear UCB (eps = -2.0)(disjoint)':[LinUCB(eps = -2.0, alpha = 1.0, is_hybrid = False), 'no_opt'],\n",
    "          'Linear UCB (eps = -2.0)(hybrid)':[LinUCB(eps = -2.0, alpha = 1.0, is_hybrid = True), 'no_opt'],\n",
    "          'Linear UCB (disjoint models)':[LinUCB(eps = 0.0, alpha = 1.0, is_hybrid = False), {\"eps\": [-20.0, 10.0], \"alpha\": [0.001, 10.0]}]\n",
    "         }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "3a6e5450",
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_schema = FeatureSchema(\n",
    "    [\n",
    "        FeatureInfo(\n",
    "            column=\"user_idx\",\n",
    "            feature_type=FeatureType.CATEGORICAL,\n",
    "            feature_hint=FeatureHint.QUERY_ID,\n",
    "        ),\n",
    "        FeatureInfo(\n",
    "            column=\"item_idx\",\n",
    "            feature_type=FeatureType.CATEGORICAL,\n",
    "            feature_hint=FeatureHint.ITEM_ID,\n",
    "        ),\n",
    "        FeatureInfo(\n",
    "            column=\"relevance\",\n",
    "            feature_type=FeatureType.NUMERICAL,\n",
    "            feature_hint=FeatureHint.RATING,\n",
    "        ),\n",
    "        FeatureInfo(\n",
    "            column=\"timestamp\",\n",
    "            feature_type=FeatureType.NUMERICAL,\n",
    "            feature_hint=FeatureHint.TIMESTAMP,\n",
    "        ),\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "723b27d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "all_dataset = Dataset(\n",
    "    feature_schema=feature_schema,\n",
    "    interactions=log_replay,\n",
    "    item_features=item_features,\n",
    "    query_features=user_features,\n",
    "    categorical_encoded = True\n",
    ")\n",
    "\n",
    "train_dataset = Dataset(\n",
    "    feature_schema=feature_schema,\n",
    "    interactions=train,\n",
    "    item_features=item_features,\n",
    "    query_features=user_features,\n",
    "    categorical_encoded = True\n",
    ")\n",
    "\n",
    "test_dataset = Dataset(\n",
    "    feature_schema=feature_schema,\n",
    "    interactions=test,\n",
    "    item_features=item_features,\n",
    "    query_features=user_features,\n",
    "    categorical_encoded = True\n",
    ")\n",
    "\n",
    "train_neg_dataset = Dataset(\n",
    "    feature_schema=feature_schema,\n",
    "    interactions=pos_neg_train,\n",
    "    item_features=item_features,\n",
    "    query_features=user_features,\n",
    "    categorical_encoded = True\n",
    ")\n",
    "\n",
    "opt_train_dataset = Dataset(\n",
    "    feature_schema=feature_schema,\n",
    "    interactions=opt_train,\n",
    "    item_features=item_features,\n",
    "    query_features=user_features,\n",
    "    categorical_encoded = True\n",
    ")\n",
    "\n",
    "opt_val_dataset = Dataset(\n",
    "    feature_schema=feature_schema,\n",
    "    interactions=opt_val,\n",
    "    item_features=item_features,\n",
    "    query_features=user_features,\n",
    "    categorical_encoded = True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "eee4ee27",
   "metadata": {},
   "outputs": [],
   "source": [
    "e = Experiment(\n",
    "    [\n",
    "        MAP(K),\n",
    "        NDCG(K),\n",
    "        HitRate(K_list_metrics),\n",
    "        Coverage(K),\n",
    "        Surprisal(K),\n",
    "        MRR(K)\n",
    "    ],\n",
    "    test_dataset.interactions,\n",
    "    train_dataset.interactions,\n",
    "    query_column=train_dataset.feature_schema.query_id_column,\n",
    "    item_column=train_dataset.feature_schema.item_id_column,\n",
    "    rating_column=train_dataset.feature_schema.interactions_rating_column,\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "99ca84a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "8efa70cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:33:38, replay, INFO: Popular started\n",
      "03-Oct-24 17:33:38, replay, INFO: Popular fit_predict started\n",
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+-------------------+\n",
      "|user_idx|item_idx|          relevance|\n",
      "+--------+--------+-------------------+\n",
      "|      18|      14| 0.3258278145695364|\n",
      "|      18|      32| 0.2682119205298013|\n",
      "|      18|      48|  0.223841059602649|\n",
      "|      18|      47| 0.2187086092715232|\n",
      "|      18|      89|0.16258278145695365|\n",
      "|      18|     114|0.15960264900662252|\n",
      "|      18|     101| 0.1490066225165563|\n",
      "|      18|     105| 0.1478476821192053|\n",
      "|      18|     124|0.14718543046357616|\n",
      "|      18|     157|0.14089403973509934|\n",
      "|      46|      12|0.33410596026490064|\n",
      "|      46|      18| 0.3086092715231788|\n",
      "|      46|      19| 0.2905629139072848|\n",
      "|      46|      22| 0.2900662251655629|\n",
      "|      46|      28| 0.2814569536423841|\n",
      "|      46|      30| 0.2682119205298013|\n",
      "|      46|      31|               0.25|\n",
      "|      46|      35|0.24188741721854304|\n",
      "|      46|      34|0.23791390728476822|\n",
      "|      46|      43|0.22996688741721855|\n",
      "|     186|      12|0.33410596026490064|\n",
      "|     186|      17| 0.3130794701986755|\n",
      "|     186|      15| 0.3074503311258278|\n",
      "|     186|      24|0.30099337748344374|\n",
      "|     186|      23| 0.3001655629139073|\n",
      "|     186|      20| 0.2913907284768212|\n",
      "|     186|      26|0.28526490066225163|\n",
      "|     186|      27|0.27864238410596026|\n",
      "|     186|      35|0.24188741721854304|\n",
      "|     186|      43|0.22996688741721855|\n",
      "|     282|       3|0.38211920529801324|\n",
      "|     282|       6| 0.3579470198675497|\n",
      "|     282|       7|0.33625827814569537|\n",
      "|     282|      28| 0.2814569536423841|\n",
      "|     282|      30| 0.2682119205298013|\n",
      "|     282|      31|               0.25|\n",
      "|     282|      37| 0.2445364238410596|\n",
      "|     282|      34|0.23791390728476822|\n",
      "|     282|      43|0.22996688741721855|\n",
      "|     282|      48|  0.223841059602649|\n",
      "|     307|       1|0.40413907284768213|\n",
      "|     307|       2| 0.4013245033112583|\n",
      "|     307|       3|0.38211920529801324|\n",
      "|     307|       5|0.35827814569536426|\n",
      "|     307|       8| 0.3443708609271523|\n",
      "|     307|      11|0.34271523178807944|\n",
      "|     307|      17| 0.3130794701986755|\n",
      "|     307|      15| 0.3074503311258278|\n",
      "|     307|      24|0.30099337748344374|\n",
      "|     307|      23| 0.3001655629139073|\n",
      "|     406|      10| 0.3228476821192053|\n",
      "|     406|      15| 0.3074503311258278|\n",
      "|     406|      29| 0.2781456953642384|\n",
      "|     406|      21|0.24884105960264902|\n",
      "|     406|      34|0.23791390728476822|\n",
      "|     406|      48|  0.223841059602649|\n",
      "|     406|      50|0.22317880794701986|\n",
      "|     406|      39|0.22235099337748343|\n",
      "|     406|      47| 0.2187086092715232|\n",
      "|     406|      42| 0.2140728476821192|\n",
      "|     407|       1|0.40413907284768213|\n",
      "|     407|       3|0.38211920529801324|\n",
      "|     407|       9| 0.3448675496688742|\n",
      "|     407|       7|0.33625827814569537|\n",
      "|     407|      13|0.32466887417218543|\n",
      "|     407|      23| 0.3001655629139073|\n",
      "|     407|      19| 0.2905629139072848|\n",
      "|     407|      26|0.28526490066225163|\n",
      "|     407|      28| 0.2814569536423841|\n",
      "|     407|      32| 0.2682119205298013|\n",
      "|     411|      12|0.33410596026490064|\n",
      "|     411|      13|0.32466887417218543|\n",
      "|     411|      23| 0.3001655629139073|\n",
      "|     411|      19| 0.2905629139072848|\n",
      "|     411|      22| 0.2900662251655629|\n",
      "|     411|      39|0.22235099337748343|\n",
      "|     411|      47| 0.2187086092715232|\n",
      "|     411|      42| 0.2140728476821192|\n",
      "|     411|      33|0.20579470198675498|\n",
      "|     411|      58|0.19850993377483445|\n",
      "|     471|       1|0.40413907284768213|\n",
      "|     471|       2| 0.4013245033112583|\n",
      "|     471|       3|0.38211920529801324|\n",
      "|     471|       5|0.35827814569536426|\n",
      "|     471|       6| 0.3579470198675497|\n",
      "|     471|       8| 0.3443708609271523|\n",
      "|     471|      11|0.34271523178807944|\n",
      "|     471|       7|0.33625827814569537|\n",
      "|     471|       4| 0.3334437086092715|\n",
      "|     471|      14| 0.3258278145695364|\n",
      "|     473|       0| 0.4508278145695364|\n",
      "|     473|       5|0.35827814569536426|\n",
      "|     473|       9| 0.3448675496688742|\n",
      "|     473|       7|0.33625827814569537|\n",
      "|     473|      12|0.33410596026490064|\n",
      "|     473|      14| 0.3258278145695364|\n",
      "|     473|      13|0.32466887417218543|\n",
      "|     473|      10| 0.3228476821192053|\n",
      "|     473|      17| 0.3130794701986755|\n",
      "|     473|      18| 0.3086092715231788|\n",
      "+--------+--------+-------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:33:53, replay, INFO: Random (uniform) started                      \n",
      "03-Oct-24 17:33:53, replay, INFO: Random (uniform) fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "         NDCG@10    MRR@10  Coverage@10  fit_time\n",
      "Popular  0.24367  0.390414     0.033903  1.755369\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+------------------+\n",
      "|user_idx|item_idx|         relevance|\n",
      "+--------+--------+------------------+\n",
      "|      18|     757|0.9993475987324224|\n",
      "|      18|    1948|0.9958625269029056|\n",
      "|      18|    1658|0.9958581137949202|\n",
      "|      18|    1522|0.9951778117229308|\n",
      "|      18|    3706|0.9930875755613394|\n",
      "|      18|     197|0.9930827474156511|\n",
      "|      18|    2644|0.9929310746247295|\n",
      "|      18|    2730|0.9927206244709196|\n",
      "|      18|    3202|0.9926568867577302|\n",
      "|      18|    1782|0.9921495467387857|\n",
      "|      46|    3827|0.9994992249598874|\n",
      "|      46|    3207|0.9991521584153258|\n",
      "|      46|    3222|0.9991089319441182|\n",
      "|      46|    2128|0.9955462962465224|\n",
      "|      46|    1540|0.9949174902180918|\n",
      "|      46|    2273|0.9942066707891665|\n",
      "|      46|    2286|0.9940611391623346|\n",
      "|      46|    3396| 0.991560971355034|\n",
      "|      46|    1596|0.9901570394788072|\n",
      "|      46|     311|0.9865957485430298|\n",
      "|     186|    3637|0.9969224342334095|\n",
      "|     186|     734|0.9964346644226986|\n",
      "|     186|     493|0.9940878250746141|\n",
      "|     186|    1207| 0.991231412296332|\n",
      "|     186|    1960|0.9912269366167428|\n",
      "|     186|    1994|0.9861276366691761|\n",
      "|     186|    2864|  0.98414345907843|\n",
      "|     186|    1294|0.9800578033039643|\n",
      "|     186|     999|0.9791843877089842|\n",
      "|     186|    2186|0.9786302741915067|\n",
      "|     282|    3110|0.9999924508892756|\n",
      "|     282|     218|0.9967621281271072|\n",
      "|     282|    3645|0.9919287459421795|\n",
      "|     282|    1881|0.9864983673687123|\n",
      "|     282|    3304|0.9855750353484244|\n",
      "|     282|    3456| 0.984554738300015|\n",
      "|     282|    1588|0.9793184996761779|\n",
      "|     282|    3238|0.9752485044560053|\n",
      "|     282|     595| 0.973964832762571|\n",
      "|     282|    1271|0.9712694849107345|\n",
      "|     307|     605| 0.998151400211698|\n",
      "|     307|    3501|0.9829954179841366|\n",
      "|     307|    2944|0.9752101458616953|\n",
      "|     307|     742|0.9696743503210883|\n",
      "|     307|    1410|0.9588068715743371|\n",
      "|     307|    2656|0.9445102294198161|\n",
      "|     307|    3072|  0.93566292733357|\n",
      "|     307|    2631|0.9195189834986103|\n",
      "|     307|    2367|0.9136733172335164|\n",
      "|     307|    1359|0.9131067324216879|\n",
      "|     406|     781|0.9990285869966977|\n",
      "|     406|    2729|0.9981078867290428|\n",
      "|     406|    3274|0.9976695313411091|\n",
      "|     406|    1913|0.9946198225841751|\n",
      "|     406|    1532|0.9901362225427118|\n",
      "|     406|    1053|0.9826186879033435|\n",
      "|     406|    2745|0.9795748122727421|\n",
      "|     406|    2196|0.9725694876419173|\n",
      "|     406|     189|0.9719098680740709|\n",
      "|     406|    3006|0.9715036235035567|\n",
      "|     407|     259|0.9957092103508488|\n",
      "|     407|    3449|0.9927428603594716|\n",
      "|     407|    3691|0.9895017485882446|\n",
      "|     407|    2212|0.9886304746642504|\n",
      "|     407|    3742|0.9879571482347481|\n",
      "|     407|     843|0.9871260162443279|\n",
      "|     407|     335|0.9868412674509706|\n",
      "|     407|     858|0.9808660157998816|\n",
      "|     407|    1081|0.9783078575197136|\n",
      "|     407|    2350|0.9744751759827087|\n",
      "|     411|     635|0.9916571121915486|\n",
      "|     411|    3109|0.9915542143424808|\n",
      "|     411|    1902|0.9843689997172859|\n",
      "|     411|    2418|0.9827798046224919|\n",
      "|     411|    3824|0.9790356428675545|\n",
      "|     411|     401| 0.978241464772725|\n",
      "|     411|    3800|0.9714921820198985|\n",
      "|     411|    2057|0.9673950452625205|\n",
      "|     411|     375|0.9664426243658699|\n",
      "|     411|     453|0.9660065661001815|\n",
      "|     471|      60|0.9618229958782423|\n",
      "|     471|    2307|0.9293258304351196|\n",
      "|     471|    2383|0.9018372439168326|\n",
      "|     471|    2082| 0.899292200694718|\n",
      "|     471|    3451|0.8906651136574434|\n",
      "|     471|     665|0.8801433436869942|\n",
      "|     471|    2908|0.8737826587668658|\n",
      "|     471|     776|0.8439221771393712|\n",
      "|     471|     928|0.8113748909692783|\n",
      "|     471|    3782|0.8082766193842912|\n",
      "|     473|    2355|0.9973846610410516|\n",
      "|     473|    1770| 0.992891968515323|\n",
      "|     473|    2954|0.9926631327384833|\n",
      "|     473|    1631|0.9909329333733404|\n",
      "|     473|    1338|0.9900748749744359|\n",
      "|     473|    1925|0.9851680914802496|\n",
      "|     473|    3386|0.9693305691154649|\n",
      "|     473|    1352|0.9664423464870252|\n",
      "|     473|    3710|0.9651021337525588|\n",
      "|     473|    1832|0.9592297758949687|\n",
      "+--------+--------+------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:34:06, replay, INFO: Random (popularity-based) started\n",
      "03-Oct-24 17:34:06, replay, INFO: Random (popularity-based) optimization started\n",
      "[I 2024-10-03 17:34:06,238] A new study created in memory with name: no-name-d4cb8186-229c-4a65-b858-205b272ea80c\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                   NDCG@10    MRR@10  Coverage@10  fit_time\n",
      "Popular           0.243670  0.390414     0.033903  1.755369\n",
      "Random (uniform)  0.022416  0.058170     0.942281  1.586289\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2024-10-03 17:34:15,459] Trial 0 finished with value: 0.024244880309437902 and parameters: {'distribution': 'popular_based', 'alpha': 0.0}. Best is trial 0 with value: 0.024244880309437902.\n",
      "[I 2024-10-03 17:34:24,527] Trial 1 finished with value: 0.03092089599992296 and parameters: {'distribution': 'popular_based', 'alpha': 84.70880898930105}. Best is trial 1 with value: 0.03092089599992296.\n",
      "[I 2024-10-03 17:34:33,465] Trial 2 finished with value: 0.024757409316372374 and parameters: {'distribution': 'popular_based', 'alpha': 60.77960091712146}. Best is trial 1 with value: 0.03092089599992296.\n",
      "[I 2024-10-03 17:34:42,335] Trial 3 finished with value: 0.028328635383958755 and parameters: {'distribution': 'popular_based', 'alpha': 88.11857920761102}. Best is trial 1 with value: 0.03092089599992296.\n",
      "[I 2024-10-03 17:34:50,640] Trial 4 finished with value: 0.03104272914099393 and parameters: {'distribution': 'popular_based', 'alpha': 89.57384480453184}. Best is trial 4 with value: 0.03104272914099393.\n",
      "03-Oct-24 17:34:50, replay, INFO: best params for Random (popularity-based) are: {'distribution': 'popular_based', 'alpha': 89.57384480453184}\n",
      "03-Oct-24 17:34:50, replay, INFO: Random (popularity-based) fit_predict started\n",
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+------------------+\n",
      "|user_idx|item_idx|         relevance|\n",
      "+--------+--------+------------------+\n",
      "|      18|     411|0.9992992462996144|\n",
      "|      18|    3713|0.9982319846137914|\n",
      "|      18|    1279|0.9965086571912344|\n",
      "|      18|    2765|0.9957523607462447|\n",
      "|      18|    1436|0.9951778117229308|\n",
      "|      18|    2624|  0.99367606822507|\n",
      "|      18|     585|0.9932851666199575|\n",
      "|      18|    2558|0.9929310746247295|\n",
      "|      18|    1129|0.9922592942272546|\n",
      "|      18|    1839|0.9911552650414707|\n",
      "|      46|    2709| 0.999130288906365|\n",
      "|      46|     780|0.9986191616427117|\n",
      "|      46|    2830|0.9896916496612517|\n",
      "|      46|    2520| 0.988614939846613|\n",
      "|      46|    1737|0.9884048210773676|\n",
      "|      46|    1730|0.9880037217529459|\n",
      "|      46|    1193|0.9847950920002226|\n",
      "|      46|     716|0.9828603408178712|\n",
      "|      46|    1907|0.9824781310008317|\n",
      "|      46|      40|0.9812112622700344|\n",
      "|     186|    1198|0.9935530949728052|\n",
      "|     186|    1492|0.9927656621572866|\n",
      "|     186|    1210|0.9902055713750425|\n",
      "|     186|    1092|0.9892552824783272|\n",
      "|     186|    1756|0.9869129602548087|\n",
      "|     186|     120|0.9867803643585882|\n",
      "|     186|    1274|0.9852712445630731|\n",
      "|     186|    2275|0.9832771147054762|\n",
      "|     186|    1949|0.9797287223814308|\n",
      "|     186|    2170|0.9786302741915067|\n",
      "|     282|    2219|0.9927092993029493|\n",
      "|     282|    2902|0.9880293321409407|\n",
      "|     282|     257|0.9866755089965221|\n",
      "|     282|    2428| 0.986029658912811|\n",
      "|     282|    1579|0.9793184996761779|\n",
      "|     282|    2573|0.9784233511931525|\n",
      "|     282|    2357|0.9771832998817132|\n",
      "|     282|    1364|0.9764304625233299|\n",
      "|     282|     733|0.9760515695304948|\n",
      "|     282|    3246|0.9649474761449486|\n",
      "|     307|    1189|0.9925269777568764|\n",
      "|     307|    2346|0.9681437933373147|\n",
      "|     307|    3626|0.9543504830458457|\n",
      "|     307|    2259| 0.942219264580318|\n",
      "|     307|    2547|0.9382869433214645|\n",
      "|     307|     278|0.9311027132517629|\n",
      "|     307|     276|0.8953390667840508|\n",
      "|     307|    1129|0.8874192003019074|\n",
      "|     307|    2582|0.8708536588835275|\n",
      "|     307|    2770|0.8591132016517943|\n",
      "|     406|     771|0.9926114883435175|\n",
      "|     406|     982|0.9912111399956273|\n",
      "|     406|    1508|0.9901362225427118|\n",
      "|     406|     443|0.9891730716793562|\n",
      "|     406|     670|0.9882960437667941|\n",
      "|     406|    2334|0.9879737716246599|\n",
      "|     406|    3811|0.9874235916399463|\n",
      "|     406|     954|0.9813588960411669|\n",
      "|     406|    2446|0.9756395550243079|\n",
      "|     406|    3198|0.9748884898400172|\n",
      "|     407|    2256|0.9996345801642781|\n",
      "|     407|    2520| 0.998048003695013|\n",
      "|     407|    3195|0.9976891138020452|\n",
      "|     407|    2342|0.9967491727949436|\n",
      "|     407|    1780|0.9883658339460394|\n",
      "|     407|    3050| 0.983868155067585|\n",
      "|     407|     881|0.9823080785743601|\n",
      "|     407|    3180|0.9787589013973624|\n",
      "|     407|    2329|0.9744751759827087|\n",
      "|     407|    1275|0.9739237833878007|\n",
      "|     411|     260| 0.998792763768093|\n",
      "|     411|    2066|0.9965147951828875|\n",
      "|     411|     839|0.9940840373121376|\n",
      "|     411|    3210|0.9928973687121502|\n",
      "|     411|    2745|0.9898520826549159|\n",
      "|     411|    2669|0.9880827221402779|\n",
      "|     411|    1144|0.9814049948302536|\n",
      "|     411|     494|0.9802012783621672|\n",
      "|     411|    2708|0.9732266700775704|\n",
      "|     411|    1670|0.9724173091025062|\n",
      "|     471|    2627|0.9979749610190378|\n",
      "|     471|     532|0.9912129737212438|\n",
      "|     471|    1954|0.9898281811473721|\n",
      "|     471|    1252|0.9850152597966357|\n",
      "|     471|    2030|0.9797668264570505|\n",
      "|     471|    2580|0.9703575721483204|\n",
      "|     471|     408|0.9656120497992148|\n",
      "|     471|    3024| 0.949761034258036|\n",
      "|     471|    2779|0.9388929735261097|\n",
      "|     471|     755|0.9369150465003578|\n",
      "|     473|    1639|0.9965428631604527|\n",
      "|     473|    2191|0.9932370860129597|\n",
      "|     473|    3451|0.9805484326194326|\n",
      "|     473|    2733|0.9644841389789154|\n",
      "|     473|     883|0.9630150387826152|\n",
      "|     473|    3102| 0.958262274697322|\n",
      "|     473|     369|0.9582529220078986|\n",
      "|     473|    1145|0.9536375954710292|\n",
      "|     473|     710|0.9534482992709721|\n",
      "|     473|    1661|0.9509573458045135|\n",
      "+--------+--------+------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:35:02, replay, INFO: UCB started\n",
      "03-Oct-24 17:35:02, replay, INFO: UCB fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            NDCG@10    MRR@10  Coverage@10  fit_time\n",
      "Popular                    0.243670  0.390414     0.033903  1.755369\n",
      "Random (popularity-based)  0.026008  0.064604     0.839731  1.767257\n",
      "Random (uniform)           0.022416  0.058170     0.942281  1.586289\n",
      "+--------+--------+-----------------+\n",
      "|user_idx|item_idx|        relevance|\n",
      "+--------+--------+-----------------+\n",
      "|      18|    3882|6.213656579361331|\n",
      "|      18|    3881|6.213656579361331|\n",
      "|      18|    3880|6.213656579361331|\n",
      "|      18|    3879|6.213656579361331|\n",
      "|      18|    3878|6.213656579361331|\n",
      "|      18|    3877|6.213656579361331|\n",
      "|      18|    3876|6.213656579361331|\n",
      "|      18|    3875|6.213656579361331|\n",
      "|      18|    3874|6.213656579361331|\n",
      "|      18|    3873|6.213656579361331|\n",
      "|      46|    3882|6.213656579361331|\n",
      "|      46|    3881|6.213656579361331|\n",
      "|      46|    3880|6.213656579361331|\n",
      "|      46|    3879|6.213656579361331|\n",
      "|      46|    3878|6.213656579361331|\n",
      "|      46|    3877|6.213656579361331|\n",
      "|      46|    3876|6.213656579361331|\n",
      "|      46|    3875|6.213656579361331|\n",
      "|      46|    3874|6.213656579361331|\n",
      "|      46|    3873|6.213656579361331|\n",
      "|     186|    3882|6.213656579361331|\n",
      "|     186|    3881|6.213656579361331|\n",
      "|     186|    3880|6.213656579361331|\n",
      "|     186|    3879|6.213656579361331|\n",
      "|     186|    3878|6.213656579361331|\n",
      "|     186|    3877|6.213656579361331|\n",
      "|     186|    3876|6.213656579361331|\n",
      "|     186|    3875|6.213656579361331|\n",
      "|     186|    3874|6.213656579361331|\n",
      "|     186|    3873|6.213656579361331|\n",
      "|     282|    3882|6.213656579361331|\n",
      "|     282|    3881|6.213656579361331|\n",
      "|     282|    3880|6.213656579361331|\n",
      "|     282|    3879|6.213656579361331|\n",
      "|     282|    3878|6.213656579361331|\n",
      "|     282|    3877|6.213656579361331|\n",
      "|     282|    3876|6.213656579361331|\n",
      "|     282|    3875|6.213656579361331|\n",
      "|     282|    3874|6.213656579361331|\n",
      "|     282|    3873|6.213656579361331|\n",
      "|     307|    3882|6.213656579361331|\n",
      "|     307|    3881|6.213656579361331|\n",
      "|     307|    3880|6.213656579361331|\n",
      "|     307|    3879|6.213656579361331|\n",
      "|     307|    3878|6.213656579361331|\n",
      "|     307|    3877|6.213656579361331|\n",
      "|     307|    3876|6.213656579361331|\n",
      "|     307|    3875|6.213656579361331|\n",
      "|     307|    3874|6.213656579361331|\n",
      "|     307|    3873|6.213656579361331|\n",
      "|     406|    3882|6.213656579361331|\n",
      "|     406|    3881|6.213656579361331|\n",
      "|     406|    3880|6.213656579361331|\n",
      "|     406|    3879|6.213656579361331|\n",
      "|     406|    3878|6.213656579361331|\n",
      "|     406|    3877|6.213656579361331|\n",
      "|     406|    3876|6.213656579361331|\n",
      "|     406|    3875|6.213656579361331|\n",
      "|     406|    3874|6.213656579361331|\n",
      "|     406|    3873|6.213656579361331|\n",
      "|     407|    3882|6.213656579361331|\n",
      "|     407|    3881|6.213656579361331|\n",
      "|     407|    3880|6.213656579361331|\n",
      "|     407|    3879|6.213656579361331|\n",
      "|     407|    3878|6.213656579361331|\n",
      "|     407|    3877|6.213656579361331|\n",
      "|     407|    3876|6.213656579361331|\n",
      "|     407|    3875|6.213656579361331|\n",
      "|     407|    3874|6.213656579361331|\n",
      "|     407|    3873|6.213656579361331|\n",
      "|     411|    3882|6.213656579361331|\n",
      "|     411|    3881|6.213656579361331|\n",
      "|     411|    3880|6.213656579361331|\n",
      "|     411|    3879|6.213656579361331|\n",
      "|     411|    3878|6.213656579361331|\n",
      "|     411|    3877|6.213656579361331|\n",
      "|     411|    3876|6.213656579361331|\n",
      "|     411|    3875|6.213656579361331|\n",
      "|     411|    3874|6.213656579361331|\n",
      "|     411|    3873|6.213656579361331|\n",
      "|     471|    3882|6.213656579361331|\n",
      "|     471|    3881|6.213656579361331|\n",
      "|     471|    3880|6.213656579361331|\n",
      "|     471|    3879|6.213656579361331|\n",
      "|     471|    3878|6.213656579361331|\n",
      "|     471|    3877|6.213656579361331|\n",
      "|     471|    3876|6.213656579361331|\n",
      "|     471|    3875|6.213656579361331|\n",
      "|     471|    3874|6.213656579361331|\n",
      "|     471|    3873|6.213656579361331|\n",
      "|     473|    3882|6.213656579361331|\n",
      "|     473|    3881|6.213656579361331|\n",
      "|     473|    3880|6.213656579361331|\n",
      "|     473|    3879|6.213656579361331|\n",
      "|     473|    3878|6.213656579361331|\n",
      "|     473|    3877|6.213656579361331|\n",
      "|     473|    3876|6.213656579361331|\n",
      "|     473|    3875|6.213656579361331|\n",
      "|     473|    3874|6.213656579361331|\n",
      "|     473|    3873|6.213656579361331|\n",
      "+--------+--------+-----------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:35:13, replay, INFO: Wilson started\n",
      "03-Oct-24 17:35:13, replay, INFO: Wilson fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            NDCG@10    MRR@10  Coverage@10  fit_time\n",
      "Popular                    0.243670  0.390414     0.033903  1.755369\n",
      "Random (popularity-based)  0.026008  0.064604     0.839731  1.767257\n",
      "Random (uniform)           0.022416  0.058170     0.942281  1.586289\n",
      "UCB                        0.000000  0.000000     0.000000  1.118119\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+------------------+\n",
      "|user_idx|item_idx|         relevance|\n",
      "+--------+--------+------------------+\n",
      "|      18|     707|0.9754524358354291|\n",
      "|      18|    1209| 0.972226295602107|\n",
      "|      18|     557| 0.971408022009853|\n",
      "|      18|     682|0.9698080855041509|\n",
      "|      18|     234|0.9688400027862992|\n",
      "|      18|    1081|0.9685863803126885|\n",
      "|      18|      32|0.9660186961163115|\n",
      "|      18|     238|0.9647924991594925|\n",
      "|      18|     239|0.9647424995568235|\n",
      "|      18|    1080|0.9643378106230265|\n",
      "|      46|     400|0.9900259255456659|\n",
      "|      46|     106|0.9821985244892515|\n",
      "|      46|     186|0.9795457647582008|\n",
      "|      46|     707|0.9754524358354291|\n",
      "|      46|     569|0.9745366264739433|\n",
      "|      46|     672|0.9737961511019336|\n",
      "|      46|    1209| 0.972226295602107|\n",
      "|      46|     557| 0.971408022009853|\n",
      "|      46|     682|0.9698080855041509|\n",
      "|      46|     191|0.9691067567560038|\n",
      "|     186|     400|0.9900259255456659|\n",
      "|     186|     443|0.9787441584139424|\n",
      "|     186|      23|0.9768577752796562|\n",
      "|     186|     707|0.9754524358354291|\n",
      "|     186|     569|0.9745366264739433|\n",
      "|     186|     672|0.9737961511019336|\n",
      "|     186|    1209| 0.972226295602107|\n",
      "|     186|     268|0.9715273682157255|\n",
      "|     186|     557| 0.971408022009853|\n",
      "|     186|      17|0.9713383387526517|\n",
      "|     282|     400|0.9900259255456659|\n",
      "|     282|     106|0.9821985244892515|\n",
      "|     282|     186|0.9795457647582008|\n",
      "|     282|     443|0.9787441584139424|\n",
      "|     282|     707|0.9754524358354291|\n",
      "|     282|     569|0.9745366264739433|\n",
      "|     282|     672|0.9737961511019336|\n",
      "|     282|    1209| 0.972226295602107|\n",
      "|     282|     268|0.9715273682157255|\n",
      "|     282|     557| 0.971408022009853|\n",
      "|     307|     400|0.9900259255456659|\n",
      "|     307|     186|0.9795457647582008|\n",
      "|     307|      23|0.9768577752796562|\n",
      "|     307|      11|0.9764263212868209|\n",
      "|     307|     707|0.9754524358354291|\n",
      "|     307|     569|0.9745366264739433|\n",
      "|     307|     672|0.9737961511019336|\n",
      "|     307|    1209| 0.972226295602107|\n",
      "|     307|     268|0.9715273682157255|\n",
      "|     307|     557| 0.971408022009853|\n",
      "|     406|     186|0.9795457647582008|\n",
      "|     406|     443|0.9787441584139424|\n",
      "|     406|     707|0.9754524358354291|\n",
      "|     406|     569|0.9745366264739433|\n",
      "|     406|     672|0.9737961511019336|\n",
      "|     406|    1209| 0.972226295602107|\n",
      "|     406|     268|0.9715273682157255|\n",
      "|     406|     557| 0.971408022009853|\n",
      "|     406|     682|0.9698080855041509|\n",
      "|     406|     191|0.9691067567560038|\n",
      "|     407|     400|0.9900259255456659|\n",
      "|     407|     106|0.9821985244892515|\n",
      "|     407|     186|0.9795457647582008|\n",
      "|     407|     443|0.9787441584139424|\n",
      "|     407|     707|0.9754524358354291|\n",
      "|     407|     569|0.9745366264739433|\n",
      "|     407|     672|0.9737961511019336|\n",
      "|     407|    1209| 0.972226295602107|\n",
      "|     407|     268|0.9715273682157255|\n",
      "|     407|     557| 0.971408022009853|\n",
      "|     411|     443|0.9787441584139424|\n",
      "|     411|      23|0.9768577752796562|\n",
      "|     411|     707|0.9754524358354291|\n",
      "|     411|     569|0.9745366264739433|\n",
      "|     411|     672|0.9737961511019336|\n",
      "|     411|    1209| 0.972226295602107|\n",
      "|     411|     268|0.9715273682157255|\n",
      "|     411|     557| 0.971408022009853|\n",
      "|     411|     682|0.9698080855041509|\n",
      "|     411|    1081|0.9685863803126885|\n",
      "|     471|     400|0.9900259255456659|\n",
      "|     471|     106|0.9821985244892515|\n",
      "|     471|     186|0.9795457647582008|\n",
      "|     471|     443|0.9787441584139424|\n",
      "|     471|      11|0.9764263212868209|\n",
      "|     471|     707|0.9754524358354291|\n",
      "|     471|     569|0.9745366264739433|\n",
      "|     471|     672|0.9737961511019336|\n",
      "|     471|    1209| 0.972226295602107|\n",
      "|     471|     268|0.9715273682157255|\n",
      "|     473|     400|0.9900259255456659|\n",
      "|     473|     106|0.9821985244892515|\n",
      "|     473|     443|0.9787441584139424|\n",
      "|     473|      23|0.9768577752796562|\n",
      "|     473|     707|0.9754524358354291|\n",
      "|     473|     569|0.9745366264739433|\n",
      "|     473|     672|0.9737961511019336|\n",
      "|     473|    1209| 0.972226295602107|\n",
      "|     473|     268|0.9715273682157255|\n",
      "|     473|     557| 0.971408022009853|\n",
      "+--------+--------+------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:35:25, replay, INFO: TS (context-free) started\n",
      "03-Oct-24 17:35:25, replay, INFO: TS (context-free) fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            NDCG@10    MRR@10  Coverage@10  fit_time\n",
      "Popular                    0.243670  0.390414     0.033903  1.755369\n",
      "Wilson                     0.092121  0.180976     0.017092  1.060856\n",
      "Random (popularity-based)  0.026008  0.064604     0.839731  1.767257\n",
      "Random (uniform)           0.022416  0.058170     0.942281  1.586289\n",
      "UCB                        0.000000  0.000000     0.000000  1.118119\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+------------------+\n",
      "|user_idx|item_idx|         relevance|\n",
      "+--------+--------+------------------+\n",
      "|      18|    2145| 0.999321272887753|\n",
      "|      18|    2888|0.9982630237251762|\n",
      "|      18|    3038|0.9979932560667897|\n",
      "|      18|    2897|0.9975859876409066|\n",
      "|      18|    2847|0.9967084466736913|\n",
      "|      18|    1209|0.9965098151983792|\n",
      "|      18|    3177| 0.996482216371184|\n",
      "|      18|    2072|0.9957395895496687|\n",
      "|      18|    1880|0.9956008285138828|\n",
      "|      18|    2826|0.9951267068379085|\n",
      "|      46|    2145| 0.999321272887753|\n",
      "|      46|    2888|0.9982630237251762|\n",
      "|      46|    3038|0.9979932560667897|\n",
      "|      46|    2897|0.9975859876409066|\n",
      "|      46|    1611|0.9971124155839757|\n",
      "|      46|    2847|0.9967084466736913|\n",
      "|      46|    1209|0.9965098151983792|\n",
      "|      46|    3177| 0.996482216371184|\n",
      "|      46|     400|0.9957495605321796|\n",
      "|      46|    2072|0.9957395895496687|\n",
      "|     186|    2145| 0.999321272887753|\n",
      "|     186|    2888|0.9982630237251762|\n",
      "|     186|    3038|0.9979932560667897|\n",
      "|     186|    2897|0.9975859876409066|\n",
      "|     186|    1611|0.9971124155839757|\n",
      "|     186|    2847|0.9967084466736913|\n",
      "|     186|    1209|0.9965098151983792|\n",
      "|     186|    3177| 0.996482216371184|\n",
      "|     186|     400|0.9957495605321796|\n",
      "|     186|    2072|0.9957395895496687|\n",
      "|     282|    2145| 0.999321272887753|\n",
      "|     282|    2888|0.9982630237251762|\n",
      "|     282|    3038|0.9979932560667897|\n",
      "|     282|    2897|0.9975859876409066|\n",
      "|     282|    1611|0.9971124155839757|\n",
      "|     282|    2847|0.9967084466736913|\n",
      "|     282|    1209|0.9965098151983792|\n",
      "|     282|    3177| 0.996482216371184|\n",
      "|     282|     400|0.9957495605321796|\n",
      "|     282|    2072|0.9957395895496687|\n",
      "|     307|    2145| 0.999321272887753|\n",
      "|     307|    2888|0.9982630237251762|\n",
      "|     307|    3038|0.9979932560667897|\n",
      "|     307|    2897|0.9975859876409066|\n",
      "|     307|    1611|0.9971124155839757|\n",
      "|     307|    2847|0.9967084466736913|\n",
      "|     307|    1209|0.9965098151983792|\n",
      "|     307|    3177| 0.996482216371184|\n",
      "|     307|     400|0.9957495605321796|\n",
      "|     307|    2072|0.9957395895496687|\n",
      "|     406|    2145| 0.999321272887753|\n",
      "|     406|    2888|0.9982630237251762|\n",
      "|     406|    3038|0.9979932560667897|\n",
      "|     406|    2897|0.9975859876409066|\n",
      "|     406|    1611|0.9971124155839757|\n",
      "|     406|    2847|0.9967084466736913|\n",
      "|     406|    1209|0.9965098151983792|\n",
      "|     406|    3177| 0.996482216371184|\n",
      "|     406|    2072|0.9957395895496687|\n",
      "|     406|    1880|0.9956008285138828|\n",
      "|     407|    2145| 0.999321272887753|\n",
      "|     407|    2888|0.9982630237251762|\n",
      "|     407|    2897|0.9975859876409066|\n",
      "|     407|    2847|0.9967084466736913|\n",
      "|     407|    1209|0.9965098151983792|\n",
      "|     407|    3177| 0.996482216371184|\n",
      "|     407|     400|0.9957495605321796|\n",
      "|     407|    2072|0.9957395895496687|\n",
      "|     407|    1880|0.9956008285138828|\n",
      "|     407|     443|0.9954237732200811|\n",
      "|     411|    2145| 0.999321272887753|\n",
      "|     411|    2888|0.9982630237251762|\n",
      "|     411|    3038|0.9979932560667897|\n",
      "|     411|    2897|0.9975859876409066|\n",
      "|     411|    1611|0.9971124155839757|\n",
      "|     411|    2847|0.9967084466736913|\n",
      "|     411|    1209|0.9965098151983792|\n",
      "|     411|    3177| 0.996482216371184|\n",
      "|     411|    2072|0.9957395895496687|\n",
      "|     411|    1880|0.9956008285138828|\n",
      "|     471|    2145| 0.999321272887753|\n",
      "|     471|    2888|0.9982630237251762|\n",
      "|     471|    3038|0.9979932560667897|\n",
      "|     471|    2897|0.9975859876409066|\n",
      "|     471|    1611|0.9971124155839757|\n",
      "|     471|    2847|0.9967084466736913|\n",
      "|     471|    1209|0.9965098151983792|\n",
      "|     471|    3177| 0.996482216371184|\n",
      "|     471|     400|0.9957495605321796|\n",
      "|     471|    2072|0.9957395895496687|\n",
      "|     473|    2145| 0.999321272887753|\n",
      "|     473|    2888|0.9982630237251762|\n",
      "|     473|    3038|0.9979932560667897|\n",
      "|     473|    2897|0.9975859876409066|\n",
      "|     473|    1611|0.9971124155839757|\n",
      "|     473|    2847|0.9967084466736913|\n",
      "|     473|    1209|0.9965098151983792|\n",
      "|     473|    3177| 0.996482216371184|\n",
      "|     473|     400|0.9957495605321796|\n",
      "|     473|    2072|0.9957395895496687|\n",
      "+--------+--------+------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:35:36, replay, INFO: Linear UCB (eps = -10.0)(disjoint) started\n",
      "03-Oct-24 17:35:36, replay, INFO: Linear UCB (eps = -10.0)(disjoint) fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            NDCG@10    MRR@10  Coverage@10  fit_time\n",
      "Popular                    0.243670  0.390414     0.033903  1.755369\n",
      "Wilson                     0.092121  0.180976     0.017092  1.060856\n",
      "Random (popularity-based)  0.026008  0.064604     0.839731  1.767257\n",
      "Random (uniform)           0.022416  0.058170     0.942281  1.586289\n",
      "TS (context-free)          0.012693  0.027208     0.005884  1.352636\n",
      "UCB                        0.000000  0.000000     0.000000  1.118119\n",
      "+--------+--------+--------------------+\n",
      "|user_idx|item_idx|           relevance|\n",
      "+--------+--------+--------------------+\n",
      "|      18|      14|  0.3461363553113034|\n",
      "|      18|      32| 0.27435792832019157|\n",
      "|      18|      47| 0.15771052776502625|\n",
      "|      18|      48| 0.13450903873946263|\n",
      "|      18|     114| 0.12801077388571092|\n",
      "|      18|      89| 0.08495628340764438|\n",
      "|      18|     146| 0.07927673357502185|\n",
      "|      18|     179|0.005015392648520689|\n",
      "|      18|     161|4.280268349391436E-4|\n",
      "|      18|     105|-0.00684628959884...|\n",
      "|      46|      18|  0.1361872498697485|\n",
      "|      46|      22| 0.07383363680189003|\n",
      "|      46|      19|  0.0644250713023885|\n",
      "|      46|      28|0.013051815491661345|\n",
      "|      46|      43|0.005695501342363185|\n",
      "|      46|      52|0.002297268160284638|\n",
      "|      46|      30|-0.01056825841032...|\n",
      "|      46|      51|-0.04858197305413492|\n",
      "|      46|      35|-0.04873115483799839|\n",
      "|      46|      34|-0.05283451313261711|\n",
      "|     186|      24|-0.02627412210296...|\n",
      "|     186|      12|-0.03773405797423468|\n",
      "|     186|      15|-0.10107116471906208|\n",
      "|     186|      17| -0.1078316910082664|\n",
      "|     186|      35|-0.12391805772191544|\n",
      "|     186|      27| -0.1349344355952209|\n",
      "|     186|      23|-0.13972593004515732|\n",
      "|     186|      43|-0.19373682287727512|\n",
      "|     186|      50|-0.20640093170254614|\n",
      "|     186|      52|-0.22638390542793252|\n",
      "|     282|       3|  -0.867756376259454|\n",
      "|     282|      43|  -0.985854019897749|\n",
      "|     282|     106|  -1.077648924554824|\n",
      "|     282|      45| -1.1717551821629781|\n",
      "|     282|      37| -1.1738622786656312|\n",
      "|     282|      73|  -1.181939167179399|\n",
      "|     282|     142| -1.2779822082968628|\n",
      "|     282|      40| -1.3347514756838346|\n",
      "|     282|      30|  -1.368405773313439|\n",
      "|     282|     205|   -1.38254241098598|\n",
      "|     307|       2| 0.14687944028632893|\n",
      "|     307|       1|  0.1370233813711228|\n",
      "|     307|       5| 0.09271112812483406|\n",
      "|     307|       8| 0.08077775354813799|\n",
      "|     307|      11| 0.07342743663954221|\n",
      "|     307|       3| 0.07199203628708761|\n",
      "|     307|      17| 0.06668890943297467|\n",
      "|     307|      23|0.057055237574673345|\n",
      "|     307|      29|  0.0457284585614498|\n",
      "|     307|      15| 0.03701395808703545|\n",
      "|     406|      39|   -1.15011950899383|\n",
      "|     406|      15| -1.1674726824466635|\n",
      "|     406|      47| -1.1854541718332168|\n",
      "|     406|      29| -1.2032125092187742|\n",
      "|     406|      34| -1.3145055161786225|\n",
      "|     406|      48| -1.3496543288707272|\n",
      "|     406|      66| -1.3955502580922874|\n",
      "|     406|      50| -1.4246400531107009|\n",
      "|     406|     229| -1.4392823919652011|\n",
      "|     406|      87| -1.4555373326734409|\n",
      "|     407|      19| -0.7286530498397328|\n",
      "|     407|      26|  -1.072920836175801|\n",
      "|     407|      13| -1.1533155936352295|\n",
      "|     407|     292| -1.1709833561240042|\n",
      "|     407|      58| -1.1786910634819812|\n",
      "|     407|      92| -1.1863235109445989|\n",
      "|     407|       3| -1.2107435896924237|\n",
      "|     407|     122| -1.2240861065367512|\n",
      "|     407|     188|  -1.234118238498787|\n",
      "|     407|     189| -1.2431292395509232|\n",
      "|     411|      13| -1.0424695022589312|\n",
      "|     411|      23| -1.1652118325442604|\n",
      "|     411|      12| -1.1699591002410268|\n",
      "|     411|      67| -1.2017012804893399|\n",
      "|     411|      19| -1.2798586003003363|\n",
      "|     411|      42| -1.3972574624285337|\n",
      "|     411|      90|   -1.43872001924027|\n",
      "|     411|     141| -1.4425783012990336|\n",
      "|     411|      39|  -1.464800894141474|\n",
      "|     411|      84| -1.4940677204034063|\n",
      "|     471|       1|0.003095149816633...|\n",
      "|     471|       2|-8.62552847212372...|\n",
      "|     471|      18|-0.01585308479247194|\n",
      "|     471|      24|-0.02627412210296...|\n",
      "|     471|      13| -0.0937765683345747|\n",
      "|     471|      11|-0.09717961922569618|\n",
      "|     471|       3|-0.10019675715937704|\n",
      "|     471|       6|-0.10469862846131339|\n",
      "|     471|      17| -0.1078316910082664|\n",
      "|     471|      16|-0.11195783172857332|\n",
      "|     473|       0|  0.1012555458989336|\n",
      "|     473|       9| 0.07346071142769761|\n",
      "|     473|       5| 0.04897365932387299|\n",
      "|     473|      12| 0.04472727711897273|\n",
      "|     473|       7|0.009139887246739864|\n",
      "|     473|      18|-0.00873579356566112|\n",
      "|     473|      13|-0.02563459942288...|\n",
      "|     473|      17|-0.05192145211262...|\n",
      "|     473|      14|-0.05884777973321853|\n",
      "|     473|      23|-0.07425975415058239|\n",
      "+--------+--------+--------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:35:59, replay, INFO: Linear UCB (eps = -10.0)(hybrid) started\n",
      "03-Oct-24 17:35:59, replay, INFO: Linear UCB (eps = -10.0)(hybrid) fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                     NDCG@10    MRR@10  Coverage@10   fit_time\n",
      "Linear UCB (eps = -10.0)(disjoint)  0.256768  0.415579     0.054637  11.314386\n",
      "Popular                             0.243670  0.390414     0.033903   1.755369\n",
      "Wilson                              0.092121  0.180976     0.017092   1.060856\n",
      "Random (popularity-based)           0.026008  0.064604     0.839731   1.767257\n",
      "Random (uniform)                    0.022416  0.058170     0.942281   1.586289\n",
      "TS (context-free)                   0.012693  0.027208     0.005884   1.352636\n",
      "UCB                                 0.000000  0.000000     0.000000   1.118119\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 3883/3883 [00:34<00:00, 113.09it/s]\n",
      "100%|██████████| 3883/3883 [03:14<00:00, 19.98it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+--------------------+\n",
      "|user_idx|item_idx|           relevance|\n",
      "+--------+--------+--------------------+\n",
      "|      18|      14|  0.3461767559008836|\n",
      "|      18|      32|  0.2745079547047511|\n",
      "|      18|      47| 0.15784829769017472|\n",
      "|      18|      48| 0.13456519035999537|\n",
      "|      18|     114| 0.12816958746469875|\n",
      "|      18|      89| 0.08508309371073819|\n",
      "|      18|     146| 0.07933066562451307|\n",
      "|      18|     179|0.005225985552336887|\n",
      "|      18|     161|8.571678339234223E-4|\n",
      "|      18|     105|-0.00632902300429...|\n",
      "|      46|      18| 0.13668638392987287|\n",
      "|      46|      22| 0.07427933264991571|\n",
      "|      46|      19| 0.06471134858708916|\n",
      "|      46|      28| 0.01404128606622812|\n",
      "|      46|      43| 0.00621015894846455|\n",
      "|      46|      52|0.003041434379694...|\n",
      "|      46|      30|-0.00959565334757...|\n",
      "|      46|      51|-0.04730674293550...|\n",
      "|      46|      35|-0.04761687312399765|\n",
      "|      46|      34|-0.05159501876510775|\n",
      "|     186|      24|-0.02644103662143...|\n",
      "|     186|      12|-0.03789462103866359|\n",
      "|     186|      15|-0.10132152936149119|\n",
      "|     186|      17|-0.10808453734100354|\n",
      "|     186|      35| -0.1243033620718681|\n",
      "|     186|      27|-0.13519742716980787|\n",
      "|     186|      23|-0.13963597441557074|\n",
      "|     186|      43| -0.1940598401332324|\n",
      "|     186|      50|-0.20657097473744224|\n",
      "|     186|      52|-0.22659785493715923|\n",
      "|     282|       3| -0.8667120511958228|\n",
      "|     282|      43| -0.9852731460405868|\n",
      "|     282|     106| -1.0776197507647012|\n",
      "|     282|      45| -1.1705040755424938|\n",
      "|     282|      37| -1.1733259827524407|\n",
      "|     282|      73| -1.1807542397555386|\n",
      "|     282|     142| -1.2778711911435834|\n",
      "|     282|      40| -1.3349033952273799|\n",
      "|     282|      30| -1.3664100819837826|\n",
      "|     282|     205| -1.3809483728001801|\n",
      "|     307|       2|  0.1475813748866397|\n",
      "|     307|       1| 0.13780262027663726|\n",
      "|     307|       5| 0.09341416853459872|\n",
      "|     307|       8|   0.081419234182033|\n",
      "|     307|      11|  0.0740936119415182|\n",
      "|     307|       3| 0.07275963245073624|\n",
      "|     307|      17| 0.06751198741137254|\n",
      "|     307|      23|0.057671483391240175|\n",
      "|     307|      29| 0.04645407062352658|\n",
      "|     307|      15|0.037622425968137696|\n",
      "|     406|      39| -1.1462598514756988|\n",
      "|     406|      15|  -1.162606617437969|\n",
      "|     406|      47|  -1.182176321538238|\n",
      "|     406|      29| -1.1990176244860786|\n",
      "|     406|      34| -1.3101393877349428|\n",
      "|     406|      48| -1.3455112699235792|\n",
      "|     406|      66| -1.3909134520057373|\n",
      "|     406|      50| -1.4185115463539422|\n",
      "|     406|     229| -1.4341941933509452|\n",
      "|     406|      87| -1.4511718371827356|\n",
      "|     407|      19| -0.7266304534147341|\n",
      "|     407|      26| -1.0696895865735034|\n",
      "|     407|      13| -1.1494567296398062|\n",
      "|     407|     292| -1.1656175380795453|\n",
      "|     407|      58| -1.1758081209576154|\n",
      "|     407|      92| -1.1822766908922062|\n",
      "|     407|       3|  -1.205498860742098|\n",
      "|     407|     122| -1.2202958710614096|\n",
      "|     407|     188| -1.2316861189338633|\n",
      "|     407|     189| -1.2393310931708512|\n",
      "|     411|      13| -1.0390281836258608|\n",
      "|     411|      23| -1.1628370934536503|\n",
      "|     411|      12| -1.1674483095889074|\n",
      "|     411|      67| -1.1975430363209316|\n",
      "|     411|      19| -1.2752963186042927|\n",
      "|     411|      42| -1.3928497441243204|\n",
      "|     411|      90| -1.4363986278083642|\n",
      "|     411|     141| -1.4376561899179352|\n",
      "|     411|      39| -1.4599628828500126|\n",
      "|     411|      84| -1.4897452303857992|\n",
      "|     471|       1|0.002489712088548...|\n",
      "|     471|       2|-0.00112122145630...|\n",
      "|     471|      18|-0.01601987426837...|\n",
      "|     471|      24|-0.02644103662143...|\n",
      "|     471|      13|  -0.093811544303954|\n",
      "|     471|      11|-0.09706617685307395|\n",
      "|     471|       3|-0.10024017248307071|\n",
      "|     471|       6|-0.10495063515858971|\n",
      "|     471|      17|-0.10808453734100354|\n",
      "|     471|      16| -0.1122863282207891|\n",
      "|     473|       0| 0.10155447635081016|\n",
      "|     473|       9| 0.07358539551174625|\n",
      "|     473|       5| 0.04893154635294972|\n",
      "|     473|      12| 0.04510300115051025|\n",
      "|     473|       7|0.009130245497693923|\n",
      "|     473|      18|-0.00854323504959...|\n",
      "|     473|      13|-0.02541160675860943|\n",
      "|     473|      17| -0.0516883769011085|\n",
      "|     473|      14|-0.05854440188298...|\n",
      "|     473|      23|-0.07376127674373878|\n",
      "+--------+--------+--------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:40:19, replay, INFO: Linear UCB (eps = -5.0)(disjoint) started     \n",
      "03-Oct-24 17:40:19, replay, INFO: Linear UCB (eps = -5.0)(disjoint) fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                     NDCG@10    MRR@10  Coverage@10   fit_time\n",
      "Linear UCB (eps = -10.0)(hybrid)    0.256881  0.415585     0.054637  53.074481\n",
      "Linear UCB (eps = -10.0)(disjoint)  0.256768  0.415579     0.054637  11.314386\n",
      "Popular                             0.243670  0.390414     0.033903   1.755369\n",
      "Wilson                              0.092121  0.180976     0.017092   1.060856\n",
      "Random (popularity-based)           0.026008  0.064604     0.839731   1.767257\n",
      "Random (uniform)                    0.022416  0.058170     0.942281   1.586289\n",
      "TS (context-free)                   0.012693  0.027208     0.005884   1.352636\n",
      "UCB                                 0.000000  0.000000     0.000000   1.118119\n",
      "+--------+--------+--------------------+\n",
      "|user_idx|item_idx|           relevance|\n",
      "+--------+--------+--------------------+\n",
      "|      18|      14|   0.649542231904279|\n",
      "|      18|      32|  0.6281206748210149|\n",
      "|      18|      47|  0.5525380331294993|\n",
      "|      18|     114|  0.5445372581847239|\n",
      "|      18|      48|  0.5351774710095131|\n",
      "|      18|     146|  0.5305180432459238|\n",
      "|      18|     215|  0.4902560887751861|\n",
      "|      18|      89|  0.4896392451855751|\n",
      "|      18|     105|  0.4781346775650069|\n",
      "|      18|     272| 0.44883281149300946|\n",
      "|      46|      18|  0.5421071766363647|\n",
      "|      46|      43|  0.4938265122073314|\n",
      "|      46|      22|  0.4794601733026936|\n",
      "|      46|      52| 0.47476643612590097|\n",
      "|      46|      28|  0.4696753943237524|\n",
      "|      46|      19|  0.4628401599238217|\n",
      "|      46|      30|  0.4538245732203905|\n",
      "|      46|      51|   0.451656994741109|\n",
      "|      46|     106|  0.4500501878811629|\n",
      "|      46|      49|  0.4475531002761074|\n",
      "|     186|      24| 0.48689472947139034|\n",
      "|     186|      12| 0.44222262376681226|\n",
      "|     186|      17| 0.43152180235387183|\n",
      "|     186|      23| 0.41877147862050723|\n",
      "|     186|      35|  0.4045875629999274|\n",
      "|     186|      15| 0.39939986346240597|\n",
      "|     186|      43| 0.39569029657168686|\n",
      "|     186|      50|  0.3883014724488961|\n",
      "|     186|      27|  0.3862693709942223|\n",
      "|     186|     133|  0.3675098601003436|\n",
      "|     282|       3|0.025104211053360914|\n",
      "|     282|      43|-0.02929462913364...|\n",
      "|     282|     106|-0.04668630898850...|\n",
      "|     282|      73|-0.08776475026010866|\n",
      "|     282|      37|-0.09433895829832972|\n",
      "|     282|      45|-0.14589802105424576|\n",
      "|     282|     142|-0.16740529727002218|\n",
      "|     282|      28| -0.2251094723465512|\n",
      "|     282|     113|-0.22694372006961316|\n",
      "|     282|     205|-0.23251441795440497|\n",
      "|     307|       2|  0.5527555330552476|\n",
      "|     307|       1|  0.5485107805267933|\n",
      "|     307|       5|  0.5271408112875411|\n",
      "|     307|      17|  0.5203138924735669|\n",
      "|     307|      11|  0.5173397739609193|\n",
      "|     307|      23|  0.5171972948522661|\n",
      "|     307|      29|  0.4993806297678135|\n",
      "|     307|       8|  0.4991117866341048|\n",
      "|     307|       3|  0.4871862967941192|\n",
      "|     307|      24| 0.48451543612540665|\n",
      "|     406|      39| -0.0754074178549855|\n",
      "|     406|      29|-0.09736543429301547|\n",
      "|     406|      15|-0.10149252071042691|\n",
      "|     406|      47| -0.1088555353979428|\n",
      "|     406|      48|-0.16989203176493772|\n",
      "|     406|      34|-0.17786818609895771|\n",
      "|     406|      66|  -0.199750917506349|\n",
      "|     406|      50|-0.21193679191257875|\n",
      "|     406|     229|-0.23361280423116393|\n",
      "|     406|      87|-0.25520218276228024|\n",
      "|     407|      19| 0.11474882912196027|\n",
      "|     407|      26|-0.07448984439972839|\n",
      "|     407|      13|-0.07643356265709023|\n",
      "|     407|      92|-0.08866492997767605|\n",
      "|     407|     292|-0.10127270624662721|\n",
      "|     407|      58|-0.11263625380038322|\n",
      "|     407|     188|-0.11501373857591846|\n",
      "|     407|     122|-0.12994124788229833|\n",
      "|     407|       3| -0.1307892400099927|\n",
      "|     407|      68|-0.13558496967440536|\n",
      "|     411|      13|-0.01772768308965511|\n",
      "|     411|      12|-0.07872355914307705|\n",
      "|     411|      23|-0.08324832062894572|\n",
      "|     411|      67|-0.09391457658292679|\n",
      "|     411|      19| -0.1316883550155168|\n",
      "|     411|      42|-0.19821450995162349|\n",
      "|     411|      39|-0.22564436514122876|\n",
      "|     411|     141| -0.2276292332108536|\n",
      "|     411|     179| -0.2327286607418304|\n",
      "|     411|      90| -0.2444356882339731|\n",
      "|     471|      24| 0.48689472947139034|\n",
      "|     471|       1|  0.4825292116609562|\n",
      "|     471|       2|  0.4601377836987696|\n",
      "|     471|      18| 0.45878836384011673|\n",
      "|     471|      11|  0.4384645470199833|\n",
      "|     471|      13| 0.43591789006629966|\n",
      "|     471|      17| 0.43152180235387183|\n",
      "|     471|      32| 0.43073118097128593|\n",
      "|     471|      16|  0.4156746404310032|\n",
      "|     471|      35|  0.4045875629999274|\n",
      "|     473|       9|  0.5231624621580157|\n",
      "|     473|      12|  0.5135353645810944|\n",
      "|     473|       0|  0.5131349284170517|\n",
      "|     473|       5|   0.512900649863419|\n",
      "|     473|      13|  0.4808888503908386|\n",
      "|     473|      18|  0.4771762159535649|\n",
      "|     473|       7|  0.4760007446957546|\n",
      "|     473|      24| 0.45495505836538663|\n",
      "|     473|      14|  0.4545658885597972|\n",
      "|     473|      23|  0.4535962281995184|\n",
      "+--------+--------+--------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:40:41, replay, INFO: Linear UCB (eps = -5.0)(hybrid) started\n",
      "03-Oct-24 17:40:41, replay, INFO: Linear UCB (eps = -5.0)(hybrid) fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                     NDCG@10    MRR@10  Coverage@10   fit_time\n",
      "Linear UCB (eps = -10.0)(hybrid)    0.256881  0.415585     0.054637  53.074481\n",
      "Linear UCB (eps = -10.0)(disjoint)  0.256768  0.415579     0.054637  11.314386\n",
      "Linear UCB (eps = -5.0)(disjoint)   0.256633  0.411666     0.055758  11.281787\n",
      "Popular                             0.243670  0.390414     0.033903   1.755369\n",
      "Wilson                              0.092121  0.180976     0.017092   1.060856\n",
      "Random (popularity-based)           0.026008  0.064604     0.839731   1.767257\n",
      "Random (uniform)                    0.022416  0.058170     0.942281   1.586289\n",
      "TS (context-free)                   0.012693  0.027208     0.005884   1.352636\n",
      "UCB                                 0.000000  0.000000     0.000000   1.118119\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 3883/3883 [00:32<00:00, 120.46it/s]\n",
      "100%|██████████| 3883/3883 [03:14<00:00, 20.01it/s]\n",
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+--------------------+\n",
      "|user_idx|item_idx|           relevance|\n",
      "+--------+--------+--------------------+\n",
      "|      18|      14|    0.64958769088108|\n",
      "|      18|      32|  0.6282791601855681|\n",
      "|      18|      47|  0.5527005258783244|\n",
      "|      18|     114|  0.5447097862605865|\n",
      "|      18|      48|  0.5352390084944503|\n",
      "|      18|     146|  0.5305769214041183|\n",
      "|      18|     215| 0.49049194628140186|\n",
      "|      18|      89|  0.4897695408752507|\n",
      "|      18|     105|  0.4786942200194874|\n",
      "|      18|     272|  0.4490487869983859|\n",
      "|      46|      18|  0.5426098299428386|\n",
      "|      46|      43|   0.494354817901255|\n",
      "|      46|      22|  0.4799117213965361|\n",
      "|      46|      52|  0.4755169518485176|\n",
      "|      46|      28| 0.47068393019776944|\n",
      "|      46|      19| 0.46313234865910863|\n",
      "|      46|      30|  0.4548017613299342|\n",
      "|      46|      51|  0.4529389740893929|\n",
      "|      46|     106| 0.45091206062856726|\n",
      "|      46|      49|  0.4483673791356345|\n",
      "|     186|      24| 0.48674175034141454|\n",
      "|     186|      12| 0.44209633555142547|\n",
      "|     186|      17|  0.4312870122721365|\n",
      "|     186|      23|  0.4188925061507698|\n",
      "|     186|      35|  0.4042113169949373|\n",
      "|     186|      15|  0.3991632071006248|\n",
      "|     186|      43|  0.3954024602112981|\n",
      "|     186|      50| 0.38816750729474603|\n",
      "|     186|      27|  0.3860233477873517|\n",
      "|     186|     133|  0.3679563878133799|\n",
      "|     282|       3|0.026242041627861257|\n",
      "|     282|      43|-0.02846659042518329|\n",
      "|     282|     106|-0.04631325924498064|\n",
      "|     282|      73|-0.08636277005465987|\n",
      "|     282|      37| -0.0936725053564007|\n",
      "|     282|      45| -0.1444907763127451|\n",
      "|     282|     142| -0.1671525940764279|\n",
      "|     282|      28|-0.22354835864351363|\n",
      "|     282|     113|-0.22669167627704123|\n",
      "|     282|     205|-0.23071998558914586|\n",
      "|     307|       2|  0.5534623768648974|\n",
      "|     307|       1|  0.5493132084197159|\n",
      "|     307|       5|  0.5278594755194597|\n",
      "|     307|      17|  0.5211441036853908|\n",
      "|     307|      11|  0.5180169468984093|\n",
      "|     307|      23|  0.5178262676728598|\n",
      "|     307|      29|  0.5001103971699365|\n",
      "|     307|       8|  0.4997592561983798|\n",
      "|     307|       3| 0.48795816003501474|\n",
      "|     307|      24|  0.4853588349891066|\n",
      "|     406|      39| -0.0714544517248965|\n",
      "|     406|      29|  -0.093061446207227|\n",
      "|     406|      15|-0.09643438684877992|\n",
      "|     406|      47|-0.10474607434183947|\n",
      "|     406|      48|-0.16553136174349126|\n",
      "|     406|      34|-0.17330214717730763|\n",
      "|     406|      66|-0.19497761832407856|\n",
      "|     406|      50|-0.20537448788575974|\n",
      "|     406|     229| -0.2282965648986819|\n",
      "|     406|      87|-0.25066345993482675|\n",
      "|     407|      19| 0.11690702159605149|\n",
      "|     407|      26|-0.07116493198612461|\n",
      "|     407|      13|-0.07245322121450504|\n",
      "|     407|      92|-0.08447003091253436|\n",
      "|     407|     292| -0.0957109572212771|\n",
      "|     407|      58|-0.10906218702677528|\n",
      "|     407|     188|-0.11230088899456203|\n",
      "|     407|       3| -0.1253428895991101|\n",
      "|     407|     122|-0.12603171097449972|\n",
      "|     407|      68|-0.13037393998883617|\n",
      "|     411|      13|-0.01419768626651...|\n",
      "|     411|      12|-0.07552696549046845|\n",
      "|     411|      23|-0.08049577410056052|\n",
      "|     411|      67|-0.08955651062214787|\n",
      "|     411|      19|-0.12682519165469963|\n",
      "|     411|      42|-0.19366195627511607|\n",
      "|     411|      39|-0.22064781439326686|\n",
      "|     411|     141|-0.22256545352438284|\n",
      "|     411|     179|-0.22838038550176165|\n",
      "|     411|      90|-0.24180916318420298|\n",
      "|     471|      24| 0.48674175034141454|\n",
      "|     471|       1|   0.481973898686034|\n",
      "|     471|       2|  0.4598894781514724|\n",
      "|     471|      18| 0.45862926926676345|\n",
      "|     471|      11|  0.4386037188129902|\n",
      "|     471|      13| 0.43589205049766777|\n",
      "|     471|      17|  0.4312870122721365|\n",
      "|     471|      32|  0.4302358016923683|\n",
      "|     471|      16| 0.41536090881109633|\n",
      "|     471|      35|  0.4042113169949373|\n",
      "|     473|       9|  0.5233077114518228|\n",
      "|     473|      12|  0.5139500167771015|\n",
      "|     473|       0|  0.5134613708269027|\n",
      "|     473|       5|  0.5128839370294105|\n",
      "|     473|      13|   0.481120356041979|\n",
      "|     473|      18|  0.4773779268328222|\n",
      "|     473|       7| 0.47601784394380375|\n",
      "|     473|      24| 0.45517415066985134|\n",
      "|     473|      14|  0.4549020414542222|\n",
      "|     473|      23|  0.4541252009455976|\n",
      "+--------+--------+--------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:44:58, replay, INFO: Linear UCB (eps = -2.0)(disjoint) started     \n",
      "03-Oct-24 17:44:58, replay, INFO: Linear UCB (eps = -2.0)(disjoint) fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                     NDCG@10    MRR@10  Coverage@10   fit_time\n",
      "Linear UCB (eps = -10.0)(hybrid)    0.256881  0.415585     0.054637  53.074481\n",
      "Linear UCB (eps = -5.0)(hybrid)     0.256847  0.412047     0.055478  50.796363\n",
      "Linear UCB (eps = -10.0)(disjoint)  0.256768  0.415579     0.054637  11.314386\n",
      "Linear UCB (eps = -5.0)(disjoint)   0.256633  0.411666     0.055758  11.281787\n",
      "Popular                             0.243670  0.390414     0.033903   1.755369\n",
      "Wilson                              0.092121  0.180976     0.017092   1.060856\n",
      "Random (popularity-based)           0.026008  0.064604     0.839731   1.767257\n",
      "Random (uniform)                    0.022416  0.058170     0.942281   1.586289\n",
      "TS (context-free)                   0.012693  0.027208     0.005884   1.352636\n",
      "UCB                                 0.000000  0.000000     0.000000   1.118119\n",
      "+--------+--------+-------------------+\n",
      "|user_idx|item_idx|          relevance|\n",
      "+--------+--------+-------------------+\n",
      "|      18|      32| 0.8403783227215088|\n",
      "|      18|      14| 0.8315857578600642|\n",
      "|      18|     146| 0.8012628290484649|\n",
      "|      18|     114| 0.7944531487641316|\n",
      "|      18|     215| 0.7894818531273808|\n",
      "|      18|      47| 0.7894345363481833|\n",
      "|      18|     378|  0.778006634051391|\n",
      "|      18|      48| 0.7755785303715433|\n",
      "|      18|     105| 0.7691232578633155|\n",
      "|      18|     239|  0.761065239180516|\n",
      "|      46|      43| 0.7867051187263123|\n",
      "|      46|      18| 0.7856591326963344|\n",
      "|      46|     106| 0.7733858596354098|\n",
      "|      46|      52| 0.7582479369052708|\n",
      "|      46|      49| 0.7529335943151868|\n",
      "|      46|      51| 0.7518003754182554|\n",
      "|      46|      85|  0.747046939519117|\n",
      "|      46|     118| 0.7462389880392188|\n",
      "|      46|      35| 0.7443712422878337|\n",
      "|      46|      28|  0.743649541623007|\n",
      "|     186|      24| 0.7947960404160025|\n",
      "|     186|      17| 0.7551338983711547|\n",
      "|     186|      23|  0.753869923819906|\n",
      "|     186|      43|  0.749346568241064|\n",
      "|     186|      50| 0.7451229149397615|\n",
      "|     186|     133| 0.7402293346243788|\n",
      "|     186|      12| 0.7301966328114404|\n",
      "|     186|      35| 0.7216909354330331|\n",
      "|     186|     191| 0.7206509383390681|\n",
      "|     186|     279| 0.7104856266636747|\n",
      "|     282|     106| 0.5718912603512821|\n",
      "|     282|      73| 0.5687398998914657|\n",
      "|     282|       3| 0.5608205634410499|\n",
      "|     282|      37| 0.5533750339220511|\n",
      "|     282|      43| 0.5446410053248167|\n",
      "|     282|     113|0.49992295326876235|\n",
      "|     282|     142|0.49894084934608224|\n",
      "|     282|      28|0.48259488589373384|\n",
      "|     282|     186| 0.4797261680589148|\n",
      "|     282|      45|0.46961627561099367|\n",
      "|     307|       2| 0.7962811887165988|\n",
      "|     307|       1| 0.7954032200201956|\n",
      "|     307|      23| 0.7932825292188216|\n",
      "|     307|      17| 0.7924888822979222|\n",
      "|     307|      43|  0.788345693186493|\n",
      "|     307|       5| 0.7877986211851653|\n",
      "|     307|      11| 0.7836871763537456|\n",
      "|     307|      29| 0.7715719324916317|\n",
      "|     307|      24| 0.7592325236648299|\n",
      "|     307|     113| 0.7553938842156512|\n",
      "|     406|      39| 0.5694198368283212|\n",
      "|     406|      29| 0.5661428106624398|\n",
      "|     406|      15| 0.5380955763313151|\n",
      "|     406|      48| 0.5379653464985359|\n",
      "|     406|      47| 0.5371036464632216|\n",
      "|     406|      66|  0.517728686845214|\n",
      "|     406|      50| 0.5156851648062946|\n",
      "|     406|      34| 0.5041142119488411|\n",
      "|     406|     129| 0.4986751322146985|\n",
      "|     406|     229| 0.4897889484092583|\n",
      "|     407|      19| 0.6207899564989761|\n",
      "|     407|      92| 0.5699302186024777|\n",
      "|     407|      13| 0.5696956559297933|\n",
      "|     407|     188| 0.5564489613778028|\n",
      "|     407|      68| 0.5461599218225641|\n",
      "|     407|     292| 0.5405536836797991|\n",
      "|     407|      42|  0.534063614162124|\n",
      "|     407|      58| 0.5269966320085755|\n",
      "|     407|     122| 0.5265456673103734|\n",
      "|     407|      26| 0.5245687506659151|\n",
      "|     411|      13| 0.5971174084119106|\n",
      "|     411|      12| 0.5760177655156928|\n",
      "|     411|      67|  0.570757445760921|\n",
      "|     411|      23| 0.5659297865202431|\n",
      "|     411|      19| 0.5572137921553749|\n",
      "|     411|     179| 0.5317076202656089|\n",
      "|     411|     260|  0.522178446851955|\n",
      "|     411|      42| 0.5212112615345226|\n",
      "|     411|      39| 0.5178495522589184|\n",
      "|     411|     431| 0.5064308514481557|\n",
      "|     471|      24| 0.7947960404160025|\n",
      "|     471|      32| 0.7757788097988844|\n",
      "|     471|       1| 0.7701896487675498|\n",
      "|     471|      11| 0.7598510467673909|\n",
      "|     471|      17| 0.7551338983711547|\n",
      "|     471|      13| 0.7537345651068242|\n",
      "|     471|      43|  0.749346568241064|\n",
      "|     471|      50| 0.7451229149397615|\n",
      "|     471|      18| 0.7435732330196699|\n",
      "|     471|     133| 0.7402293346243788|\n",
      "|     473|      12| 0.7948202170583674|\n",
      "|     473|       9| 0.7929835125962065|\n",
      "|     473|       5| 0.7912568441871465|\n",
      "|     473|      13| 0.7848029202790724|\n",
      "|     473|      24| 0.7761064702915002|\n",
      "|     473|      38| 0.7727856604195288|\n",
      "|     473|      23| 0.7703098176095788|\n",
      "|     473|      18| 0.7687234216651005|\n",
      "|     473|      14| 0.7626140895356066|\n",
      "|     473|      52| 0.7619084974859653|\n",
      "+--------+--------+-------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:45:20, replay, INFO: Linear UCB (eps = -2.0)(hybrid) started\n",
      "03-Oct-24 17:45:20, replay, INFO: Linear UCB (eps = -2.0)(hybrid) fit_predict started\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                     NDCG@10    MRR@10  Coverage@10   fit_time\n",
      "Linear UCB (eps = -10.0)(hybrid)    0.256881  0.415585     0.054637  53.074481\n",
      "Linear UCB (eps = -5.0)(hybrid)     0.256847  0.412047     0.055478  50.796363\n",
      "Linear UCB (eps = -10.0)(disjoint)  0.256768  0.415579     0.054637  11.314386\n",
      "Linear UCB (eps = -5.0)(disjoint)   0.256633  0.411666     0.055758  11.281787\n",
      "Popular                             0.243670  0.390414     0.033903   1.755369\n",
      "Linear UCB (eps = -2.0)(disjoint)   0.243607  0.389884     0.061362  11.273261\n",
      "Wilson                              0.092121  0.180976     0.017092   1.060856\n",
      "Random (popularity-based)           0.026008  0.064604     0.839731   1.767257\n",
      "Random (uniform)                    0.022416  0.058170     0.942281   1.586289\n",
      "TS (context-free)                   0.012693  0.027208     0.005884   1.352636\n",
      "UCB                                 0.000000  0.000000     0.000000   1.118119\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 3883/3883 [00:32<00:00, 120.37it/s]\n",
      "100%|██████████| 3883/3883 [03:14<00:00, 20.01it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+-------------------+\n",
      "|user_idx|item_idx|          relevance|\n",
      "+--------+--------+-------------------+\n",
      "|      18|      32| 0.8405418834740582|\n",
      "|      18|      14| 0.8316342518691979|\n",
      "|      18|     146| 0.8013246748718814|\n",
      "|      18|     114| 0.7946339055381191|\n",
      "|      18|     215| 0.7897250048553988|\n",
      "|      18|      47| 0.7896118627912142|\n",
      "|      18|     378|  0.778693120200573|\n",
      "|      18|      48| 0.7756432993751232|\n",
      "|      18|     105| 0.7697081658337579|\n",
      "|      18|     239| 0.7616835109324535|\n",
      "|      46|      43| 0.7872416132729293|\n",
      "|      46|      18| 0.7861638975506179|\n",
      "|      46|     106| 0.7742611966326691|\n",
      "|      46|      52| 0.7590022623298114|\n",
      "|      46|      49| 0.7537616363539821|\n",
      "|      46|      51| 0.7530864043043335|\n",
      "|      46|      85| 0.7477091686899544|\n",
      "|      46|     118| 0.7476233178313462|\n",
      "|      46|      35| 0.7454942001863504|\n",
      "|      46|      28| 0.7446695166766942|\n",
      "|     186|      24| 0.7946514225191219|\n",
      "|     186|      17| 0.7549099420400205|\n",
      "|     186|      23|  0.754009594490574|\n",
      "|     186|      43| 0.7490798404180163|\n",
      "|     186|      50|  0.745010596514059|\n",
      "|     186|     133|  0.740685697521463|\n",
      "|     186|      12| 0.7300909095054788|\n",
      "|     186|      35| 0.7213201244350205|\n",
      "|     186|     191|  0.720610919979755|\n",
      "|     186|     279| 0.7104488708013196|\n",
      "|     282|     106| 0.5724706356668516|\n",
      "|     282|      73| 0.5702721117658673|\n",
      "|     282|       3| 0.5620144973220718|\n",
      "|     282|      37| 0.5541195810812234|\n",
      "|     282|      43| 0.5456173429440588|\n",
      "|     282|     113| 0.5002871493530094|\n",
      "|     282|     142| 0.4992785641638653|\n",
      "|     282|      28|0.48458754715377744|\n",
      "|     282|     186|  0.480045850790556|\n",
      "|     282|      45| 0.4711172032251042|\n",
      "|     307|       2| 0.7969909780518519|\n",
      "|     307|       1| 0.7962195613055629|\n",
      "|     307|      23| 0.7939191382418315|\n",
      "|     307|      17| 0.7933233734498017|\n",
      "|     307|      43|  0.789227222693952|\n",
      "|     307|       5| 0.7885266597103762|\n",
      "|     307|      11| 0.7843709478725438|\n",
      "|     307|      29| 0.7723041930977824|\n",
      "|     307|      24| 0.7600797182071357|\n",
      "|     307|     113| 0.7566494418202462|\n",
      "|     406|      39| 0.5734287881255848|\n",
      "|     406|      29|  0.570512260760084|\n",
      "|     406|      15| 0.5432689515047334|\n",
      "|     406|      48| 0.5424565831645615|\n",
      "|     406|      47| 0.5417120739759995|\n",
      "|     406|      66| 0.5225838818849167|\n",
      "|     406|      50| 0.5225077471951498|\n",
      "|     406|      34| 0.5088001971572734|\n",
      "|     406|     129| 0.5051237904797304|\n",
      "|     406|      42|0.49580562133712247|\n",
      "|     407|      19| 0.6230295066025229|\n",
      "|     407|      92| 0.5742139650752687|\n",
      "|     407|      13| 0.5737488838406757|\n",
      "|     407|     188| 0.5593302489690187|\n",
      "|     407|      68| 0.5514759878916398|\n",
      "|     407|     292| 0.5462329912936839|\n",
      "|     407|      42| 0.5386951197785532|\n",
      "|     407|      58| 0.5309853733317289|\n",
      "|     407|     122| 0.5305267850776463|\n",
      "|     407|      26| 0.5279498607663027|\n",
      "|     411|      13| 0.6007006121490921|\n",
      "|     411|      12| 0.5796258409685949|\n",
      "|     411|      67| 0.5752354047971224|\n",
      "|     411|      23| 0.5689090175112933|\n",
      "|     411|      19| 0.5622574845150563|\n",
      "|     411|     179| 0.5365576019768834|\n",
      "|     411|     260| 0.5278032889785093|\n",
      "|     411|      42| 0.5258507164344065|\n",
      "|     411|      39| 0.5229412266807807|\n",
      "|     411|     431|  0.509564661184541|\n",
      "|     471|      24| 0.7946514225191219|\n",
      "|     471|      32| 0.7753076340768278|\n",
      "|     471|       1| 0.7696644106445251|\n",
      "|     471|      11| 0.7600056562126287|\n",
      "|     471|      17| 0.7549099420400205|\n",
      "|     471|      13| 0.7537142073786408|\n",
      "|     471|      43| 0.7490798404180163|\n",
      "|     471|      50|  0.745010596514059|\n",
      "|     471|      18| 0.7434187553878453|\n",
      "|     471|     133|  0.740685697521463|\n",
      "|     473|      12| 0.7952582261530563|\n",
      "|     473|       9| 0.7931411010158687|\n",
      "|     473|       5|  0.791255371435287|\n",
      "|     473|      13|  0.785039533722332|\n",
      "|     473|      24|  0.776335670843799|\n",
      "|     473|      38| 0.7729119951648793|\n",
      "|     473|      23| 0.7708570875591995|\n",
      "|     473|      18| 0.7689306239622697|\n",
      "|     473|      14| 0.7629699074565455|\n",
      "|     473|      52| 0.7620295706591705|\n",
      "+--------+--------+-------------------+\n",
      "only showing top 100 rows\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "03-Oct-24 17:49:37, replay, INFO: Linear UCB (disjoint models) started          \n",
      "03-Oct-24 17:49:37, replay, INFO: Linear UCB (disjoint models) optimization started\n",
      "[I 2024-10-03 17:49:37,760] A new study created in memory with name: no-name-a72cf692-0069-43b5-bea9-1b70b1f4b9e7\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                     NDCG@10    MRR@10  Coverage@10   fit_time\n",
      "Linear UCB (eps = -10.0)(hybrid)    0.256881  0.415585     0.054637  53.074481\n",
      "Linear UCB (eps = -5.0)(hybrid)     0.256847  0.412047     0.055478  50.796363\n",
      "Linear UCB (eps = -10.0)(disjoint)  0.256768  0.415579     0.054637  11.314386\n",
      "Linear UCB (eps = -5.0)(disjoint)   0.256633  0.411666     0.055758  11.281787\n",
      "Popular                             0.243670  0.390414     0.033903   1.755369\n",
      "Linear UCB (eps = -2.0)(disjoint)   0.243607  0.389884     0.061362  11.273261\n",
      "Linear UCB (eps = -2.0)(hybrid)     0.243178  0.388782     0.061082  50.755872\n",
      "Wilson                              0.092121  0.180976     0.017092   1.060856\n",
      "Random (popularity-based)           0.026008  0.064604     0.839731   1.767257\n",
      "Random (uniform)                    0.022416  0.058170     0.942281   1.586289\n",
      "TS (context-free)                   0.012693  0.027208     0.005884   1.352636\n",
      "UCB                                 0.000000  0.000000     0.000000   1.118119\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2024-10-03 17:49:57,680] Trial 0 finished with value: 0.20628550189639538 and parameters: {'eps': -3.0495855167727406, 'alpha': 8.979674297265841}. Best is trial 0 with value: 0.20628550189639538.\n",
      "[I 2024-10-03 17:50:17,371] Trial 1 finished with value: 0.0034131101044205547 and parameters: {'eps': 0.30618109472649024, 'alpha': 0.7518498633305363}. Best is trial 0 with value: 0.20628550189639538.\n",
      "[I 2024-10-03 17:50:37,143] Trial 2 finished with value: 0.0008720996683991461 and parameters: {'eps': 4.0811330801808126, 'alpha': 1.7822771570303957}. Best is trial 0 with value: 0.20628550189639538.\n",
      "[I 2024-10-03 17:50:56,430] Trial 3 finished with value: 0.20571251265206217 and parameters: {'eps': -10.773006536288307, 'alpha': 4.620065101743285}. Best is trial 0 with value: 0.20628550189639538.\n",
      "[I 2024-10-03 17:51:15,587] Trial 4 finished with value: 0.20623228824713366 and parameters: {'eps': -4.806488909745973, 'alpha': 0.865203088560681}. Best is trial 0 with value: 0.20628550189639538.\n",
      "03-Oct-24 17:51:15, replay, INFO: best params for Linear UCB (disjoint models) are: {'eps': -3.0495855167727406, 'alpha': 8.979674297265841}\n",
      "03-Oct-24 17:51:15, replay, INFO: Linear UCB (disjoint models) fit_predict started\n",
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------+-------------------+\n",
      "|user_idx|item_idx|          relevance|\n",
      "+--------+--------+-------------------+\n",
      "|      18|      14| 0.7694890117395012|\n",
      "|      18|      32| 0.7671931519224076|\n",
      "|      18|     146| 0.7102568089297239|\n",
      "|      18|     114| 0.7091622213419695|\n",
      "|      18|      47|  0.708968114382001|\n",
      "|      18|      48| 0.6955521929385429|\n",
      "|      18|     215| 0.6866091981790141|\n",
      "|      18|     105| 0.6705292239333451|\n",
      "|      18|     378| 0.6572079886344135|\n",
      "|      18|      89| 0.6506418011200548|\n",
      "|      46|      18|  0.698125042723037|\n",
      "|      46|      43| 0.6826614210683251|\n",
      "|      46|     106| 0.6563958014803232|\n",
      "|      46|      52| 0.6553195264830027|\n",
      "|      46|      28| 0.6428811803345469|\n",
      "|      46|      49| 0.6425797963195734|\n",
      "|      46|      51| 0.6389495000741722|\n",
      "|      46|      85|  0.638089585001715|\n",
      "|      46|      22| 0.6378944580559095|\n",
      "|      46|      35| 0.6342055341709173|\n",
      "|     186|      24| 0.6943580745900739|\n",
      "|     186|      17| 0.6518178242235573|\n",
      "|     186|      23|  0.648028048087242|\n",
      "|     186|      12| 0.6381822813479725|\n",
      "|     186|      43| 0.6360117427348655|\n",
      "|     186|      50| 0.6315396734332555|\n",
      "|     186|      35| 0.6247101230139418|\n",
      "|     186|     133| 0.6215603574397752|\n",
      "|     186|      15| 0.6050978812607499|\n",
      "|     186|      27| 0.6007943163069995|\n",
      "|     282|       3| 0.4129674076246608|\n",
      "|     282|     106|0.40526801479354035|\n",
      "|     282|      43|0.39686854044487263|\n",
      "|     282|      73| 0.3787947893159612|\n",
      "|     282|      37|  0.377135409150426|\n",
      "|     282|     142|0.33345713121065246|\n",
      "|     282|     113| 0.3323850943797678|\n",
      "|     282|      45|0.31428424069584404|\n",
      "|     282|     186|0.30738515315578685|\n",
      "|     282|      85|0.30440392095575974|\n",
      "|     307|       2| 0.7071750995054074|\n",
      "|     307|       1| 0.7054569193018901|\n",
      "|     307|      23| 0.6941538655851113|\n",
      "|     307|      17| 0.6941249675819356|\n",
      "|     307|       5|  0.693331657046437|\n",
      "|     307|      11| 0.6877899883981942|\n",
      "|     307|      43| 0.6769326835986077|\n",
      "|     307|      29| 0.6737265998304576|\n",
      "|     307|       8|  0.659949242411239|\n",
      "|     307|      24| 0.6596005350775793|\n",
      "|     406|      29| 0.4000805874502318|\n",
      "|     406|      39|  0.385180937054597|\n",
      "|     406|      15|0.37468442193989493|\n",
      "|     406|      47| 0.3721893469511488|\n",
      "|     406|      48|0.36341008229905725|\n",
      "|     406|      34| 0.3432889132852289|\n",
      "|     406|      66|0.34054588116015627|\n",
      "|     406|      50|0.34007762172281686|\n",
      "|     406|      42|0.30366291542883594|\n",
      "|     406|     129| 0.3035317380065705|\n",
      "|     407|      19| 0.4670486528876508|\n",
      "|     407|      13| 0.3975915635238364|\n",
      "|     407|      92| 0.3761754651403094|\n",
      "|     407|     188|0.37567877648481196|\n",
      "|     407|     292|0.36574137254014727|\n",
      "|     407|      26| 0.3643500656315689|\n",
      "|     407|      58|0.35797721539605165|\n",
      "|     407|      68|0.35488538375980305|\n",
      "|     407|     122| 0.3520820894344996|\n",
      "|     407|       3| 0.3473741394057517|\n",
      "|     411|      13|0.43435163142399713|\n",
      "|     411|      12|0.40613021615590206|\n",
      "|     411|      23|0.39883746153584754|\n",
      "|     411|      19| 0.3808033784094361|\n",
      "|     411|      67|0.37801558373592203|\n",
      "|     411|      42| 0.3446688596055454|\n",
      "|     411|      39| 0.3357503498734019|\n",
      "|     411|     179| 0.3315945626937713|\n",
      "|     411|      22|0.32581355629346476|\n",
      "|     411|      80| 0.3202407430123462|\n",
      "|     471|      24| 0.6943580745900739|\n",
      "|     471|       1| 0.6774780685124694|\n",
      "|     471|      32| 0.6666698658100412|\n",
      "|     471|      11| 0.6579942336235012|\n",
      "|     471|      18| 0.6528538170779086|\n",
      "|     471|      13| 0.6526638090187025|\n",
      "|     471|      17| 0.6518178242235573|\n",
      "|     471|       2| 0.6498255676051299|\n",
      "|     471|      43| 0.6360117427348655|\n",
      "|     471|      16| 0.6345286974720157|\n",
      "|     473|       9| 0.7046397731107925|\n",
      "|     473|      12| 0.7007001661102927|\n",
      "|     473|       5| 0.7001411321721682|\n",
      "|     473|      13| 0.6860541172767463|\n",
      "|     473|       0| 0.6789022589845378|\n",
      "|     473|      18| 0.6740232658650935|\n",
      "|     473|      24| 0.6732011138677345|\n",
      "|     473|      23| 0.6686184720880775|\n",
      "|     473|      38| 0.6661797340791755|\n",
      "|     473|       7| 0.6651220578072443|\n",
      "+--------+--------+-------------------+\n",
      "only showing top 100 rows\n",
      "\n",
      "                                     NDCG@10    MRR@10  Coverage@10   fit_time\n",
      "Linear UCB (eps = -10.0)(hybrid)    0.256881  0.415585     0.054637  53.074481\n",
      "Linear UCB (eps = -5.0)(hybrid)     0.256847  0.412047     0.055478  50.796363\n",
      "Linear UCB (eps = -10.0)(disjoint)  0.256768  0.415579     0.054637  11.314386\n",
      "Linear UCB (eps = -5.0)(disjoint)   0.256633  0.411666     0.055758  11.281787\n",
      "Linear UCB (disjoint models)        0.251921  0.402974     0.055758  11.375094\n",
      "Popular                             0.243670  0.390414     0.033903   1.755369\n",
      "Linear UCB (eps = -2.0)(disjoint)   0.243607  0.389884     0.061362  11.273261\n",
      "Linear UCB (eps = -2.0)(hybrid)     0.243178  0.388782     0.061082  50.755872\n",
      "Wilson                              0.092121  0.180976     0.017092   1.060856\n",
      "Random (popularity-based)           0.026008  0.064604     0.839731   1.767257\n",
      "Random (uniform)                    0.022416  0.058170     0.942281   1.586289\n",
      "TS (context-free)                   0.012693  0.027208     0.005884   1.352636\n",
      "UCB                                 0.000000  0.000000     0.000000   1.118119\n",
      "CPU times: user 18min 37s, sys: 20min 29s, total: 39min 6s\n",
      "Wall time: 17min 59s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "full_pipeline(bandit_models, e, train_dataset, test_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "a8d5970a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MAP@10</th>\n",
       "      <th>NDCG@10</th>\n",
       "      <th>HitRate@1</th>\n",
       "      <th>HitRate@5</th>\n",
       "      <th>HitRate@10</th>\n",
       "      <th>Coverage@10</th>\n",
       "      <th>Surprisal@10</th>\n",
       "      <th>MRR@10</th>\n",
       "      <th>fit_time</th>\n",
       "      <th>predict_time</th>\n",
       "      <th>metric_time</th>\n",
       "      <th>full_time</th>\n",
       "      <th>params</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Linear UCB (eps = -10.0)(hybrid)</th>\n",
       "      <td>0.167964</td>\n",
       "      <td>0.256881</td>\n",
       "      <td>0.311677</td>\n",
       "      <td>0.561896</td>\n",
       "      <td>0.659350</td>\n",
       "      <td>0.054637</td>\n",
       "      <td>0.125022</td>\n",
       "      <td>0.415585</td>\n",
       "      <td>53.074481</td>\n",
       "      <td>199.519035</td>\n",
       "      <td>7.204302</td>\n",
       "      <td>259.797818</td>\n",
       "      <td>{'regression type': 'hybrid', 'seed': None}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Linear UCB (eps = -5.0)(hybrid)</th>\n",
       "      <td>0.168116</td>\n",
       "      <td>0.256847</td>\n",
       "      <td>0.308165</td>\n",
       "      <td>0.568042</td>\n",
       "      <td>0.655838</td>\n",
       "      <td>0.055478</td>\n",
       "      <td>0.126488</td>\n",
       "      <td>0.412047</td>\n",
       "      <td>50.796363</td>\n",
       "      <td>199.043191</td>\n",
       "      <td>7.027678</td>\n",
       "      <td>256.867232</td>\n",
       "      <td>{'regression type': 'hybrid', 'seed': None}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Linear UCB (eps = -10.0)(disjoint)</th>\n",
       "      <td>0.167851</td>\n",
       "      <td>0.256768</td>\n",
       "      <td>0.311677</td>\n",
       "      <td>0.561896</td>\n",
       "      <td>0.659350</td>\n",
       "      <td>0.054637</td>\n",
       "      <td>0.125029</td>\n",
       "      <td>0.415579</td>\n",
       "      <td>11.314386</td>\n",
       "      <td>5.456721</td>\n",
       "      <td>5.704894</td>\n",
       "      <td>22.476000</td>\n",
       "      <td>{'regression type': 'disjoint', 'seed': None}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Linear UCB (eps = -5.0)(disjoint)</th>\n",
       "      <td>0.167926</td>\n",
       "      <td>0.256633</td>\n",
       "      <td>0.307287</td>\n",
       "      <td>0.567164</td>\n",
       "      <td>0.656716</td>\n",
       "      <td>0.055758</td>\n",
       "      <td>0.126491</td>\n",
       "      <td>0.411666</td>\n",
       "      <td>11.281787</td>\n",
       "      <td>5.287628</td>\n",
       "      <td>5.680037</td>\n",
       "      <td>22.249452</td>\n",
       "      <td>{'regression type': 'disjoint', 'seed': None}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Linear UCB (disjoint models)</th>\n",
       "      <td>0.162927</td>\n",
       "      <td>0.251921</td>\n",
       "      <td>0.293240</td>\n",
       "      <td>0.561018</td>\n",
       "      <td>0.657594</td>\n",
       "      <td>0.055758</td>\n",
       "      <td>0.129758</td>\n",
       "      <td>0.402974</td>\n",
       "      <td>11.375094</td>\n",
       "      <td>5.489639</td>\n",
       "      <td>5.716758</td>\n",
       "      <td>22.581491</td>\n",
       "      <td>{'regression type': 'disjoint', 'seed': None}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Popular</th>\n",
       "      <td>0.157257</td>\n",
       "      <td>0.243670</td>\n",
       "      <td>0.284460</td>\n",
       "      <td>0.530290</td>\n",
       "      <td>0.645303</td>\n",
       "      <td>0.033903</td>\n",
       "      <td>0.118354</td>\n",
       "      <td>0.390414</td>\n",
       "      <td>1.755369</td>\n",
       "      <td>5.097414</td>\n",
       "      <td>8.111187</td>\n",
       "      <td>14.963971</td>\n",
       "      <td>{'use_rating': False, 'add_cold_items': True, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Linear UCB (eps = -2.0)(disjoint)</th>\n",
       "      <td>0.155757</td>\n",
       "      <td>0.243607</td>\n",
       "      <td>0.271291</td>\n",
       "      <td>0.558385</td>\n",
       "      <td>0.654083</td>\n",
       "      <td>0.061362</td>\n",
       "      <td>0.134384</td>\n",
       "      <td>0.389884</td>\n",
       "      <td>11.273261</td>\n",
       "      <td>5.326424</td>\n",
       "      <td>5.731244</td>\n",
       "      <td>22.330929</td>\n",
       "      <td>{'regression type': 'disjoint', 'seed': None}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Linear UCB (eps = -2.0)(hybrid)</th>\n",
       "      <td>0.155702</td>\n",
       "      <td>0.243178</td>\n",
       "      <td>0.269535</td>\n",
       "      <td>0.558385</td>\n",
       "      <td>0.653205</td>\n",
       "      <td>0.061082</td>\n",
       "      <td>0.134378</td>\n",
       "      <td>0.388782</td>\n",
       "      <td>50.755872</td>\n",
       "      <td>199.113863</td>\n",
       "      <td>7.220896</td>\n",
       "      <td>257.090631</td>\n",
       "      <td>{'regression type': 'hybrid', 'seed': None}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Wilson</th>\n",
       "      <td>0.045002</td>\n",
       "      <td>0.092121</td>\n",
       "      <td>0.083406</td>\n",
       "      <td>0.345040</td>\n",
       "      <td>0.414399</td>\n",
       "      <td>0.017092</td>\n",
       "      <td>0.262190</td>\n",
       "      <td>0.180976</td>\n",
       "      <td>1.060856</td>\n",
       "      <td>4.335059</td>\n",
       "      <td>6.024863</td>\n",
       "      <td>11.420777</td>\n",
       "      <td>{'alpha': 0.05, 'add_cold_items': True, 'cold_...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Random (popularity-based)</th>\n",
       "      <td>0.009357</td>\n",
       "      <td>0.026008</td>\n",
       "      <td>0.028973</td>\n",
       "      <td>0.110623</td>\n",
       "      <td>0.186128</td>\n",
       "      <td>0.839731</td>\n",
       "      <td>0.501055</td>\n",
       "      <td>0.064604</td>\n",
       "      <td>1.767257</td>\n",
       "      <td>4.451728</td>\n",
       "      <td>5.792844</td>\n",
       "      <td>12.011829</td>\n",
       "      <td>{'distribution': 'popular_based', 'alpha': 89....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Random (uniform)</th>\n",
       "      <td>0.007630</td>\n",
       "      <td>0.022416</td>\n",
       "      <td>0.022827</td>\n",
       "      <td>0.099210</td>\n",
       "      <td>0.171203</td>\n",
       "      <td>0.942281</td>\n",
       "      <td>0.559885</td>\n",
       "      <td>0.058170</td>\n",
       "      <td>1.586289</td>\n",
       "      <td>5.274442</td>\n",
       "      <td>6.210629</td>\n",
       "      <td>13.071360</td>\n",
       "      <td>{'distribution': 'uniform', 'alpha': 0.0, 'see...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TS (context-free)</th>\n",
       "      <td>0.003724</td>\n",
       "      <td>0.012693</td>\n",
       "      <td>0.011414</td>\n",
       "      <td>0.032485</td>\n",
       "      <td>0.113257</td>\n",
       "      <td>0.005884</td>\n",
       "      <td>0.548572</td>\n",
       "      <td>0.027208</td>\n",
       "      <td>1.352636</td>\n",
       "      <td>4.207837</td>\n",
       "      <td>6.046631</td>\n",
       "      <td>11.607105</td>\n",
       "      <td>{'sample': False, 'seed': None}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UCB</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.118119</td>\n",
       "      <td>3.957537</td>\n",
       "      <td>6.115509</td>\n",
       "      <td>11.191166</td>\n",
       "      <td>{'exploration_coef': 2.0, 'sample': False, 'se...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                      MAP@10   NDCG@10  HitRate@1  HitRate@5  \\\n",
       "Linear UCB (eps = -10.0)(hybrid)    0.167964  0.256881   0.311677   0.561896   \n",
       "Linear UCB (eps = -5.0)(hybrid)     0.168116  0.256847   0.308165   0.568042   \n",
       "Linear UCB (eps = -10.0)(disjoint)  0.167851  0.256768   0.311677   0.561896   \n",
       "Linear UCB (eps = -5.0)(disjoint)   0.167926  0.256633   0.307287   0.567164   \n",
       "Linear UCB (disjoint models)        0.162927  0.251921   0.293240   0.561018   \n",
       "Popular                             0.157257  0.243670   0.284460   0.530290   \n",
       "Linear UCB (eps = -2.0)(disjoint)   0.155757  0.243607   0.271291   0.558385   \n",
       "Linear UCB (eps = -2.0)(hybrid)     0.155702  0.243178   0.269535   0.558385   \n",
       "Wilson                              0.045002  0.092121   0.083406   0.345040   \n",
       "Random (popularity-based)           0.009357  0.026008   0.028973   0.110623   \n",
       "Random (uniform)                    0.007630  0.022416   0.022827   0.099210   \n",
       "TS (context-free)                   0.003724  0.012693   0.011414   0.032485   \n",
       "UCB                                 0.000000  0.000000   0.000000   0.000000   \n",
       "\n",
       "                                    HitRate@10  Coverage@10  Surprisal@10  \\\n",
       "Linear UCB (eps = -10.0)(hybrid)      0.659350     0.054637      0.125022   \n",
       "Linear UCB (eps = -5.0)(hybrid)       0.655838     0.055478      0.126488   \n",
       "Linear UCB (eps = -10.0)(disjoint)    0.659350     0.054637      0.125029   \n",
       "Linear UCB (eps = -5.0)(disjoint)     0.656716     0.055758      0.126491   \n",
       "Linear UCB (disjoint models)          0.657594     0.055758      0.129758   \n",
       "Popular                               0.645303     0.033903      0.118354   \n",
       "Linear UCB (eps = -2.0)(disjoint)     0.654083     0.061362      0.134384   \n",
       "Linear UCB (eps = -2.0)(hybrid)       0.653205     0.061082      0.134378   \n",
       "Wilson                                0.414399     0.017092      0.262190   \n",
       "Random (popularity-based)             0.186128     0.839731      0.501055   \n",
       "Random (uniform)                      0.171203     0.942281      0.559885   \n",
       "TS (context-free)                     0.113257     0.005884      0.548572   \n",
       "UCB                                   0.000000     0.000000      1.000000   \n",
       "\n",
       "                                      MRR@10   fit_time  predict_time  \\\n",
       "Linear UCB (eps = -10.0)(hybrid)    0.415585  53.074481    199.519035   \n",
       "Linear UCB (eps = -5.0)(hybrid)     0.412047  50.796363    199.043191   \n",
       "Linear UCB (eps = -10.0)(disjoint)  0.415579  11.314386      5.456721   \n",
       "Linear UCB (eps = -5.0)(disjoint)   0.411666  11.281787      5.287628   \n",
       "Linear UCB (disjoint models)        0.402974  11.375094      5.489639   \n",
       "Popular                             0.390414   1.755369      5.097414   \n",
       "Linear UCB (eps = -2.0)(disjoint)   0.389884  11.273261      5.326424   \n",
       "Linear UCB (eps = -2.0)(hybrid)     0.388782  50.755872    199.113863   \n",
       "Wilson                              0.180976   1.060856      4.335059   \n",
       "Random (popularity-based)           0.064604   1.767257      4.451728   \n",
       "Random (uniform)                    0.058170   1.586289      5.274442   \n",
       "TS (context-free)                   0.027208   1.352636      4.207837   \n",
       "UCB                                 0.000000   1.118119      3.957537   \n",
       "\n",
       "                                    metric_time   full_time  \\\n",
       "Linear UCB (eps = -10.0)(hybrid)       7.204302  259.797818   \n",
       "Linear UCB (eps = -5.0)(hybrid)        7.027678  256.867232   \n",
       "Linear UCB (eps = -10.0)(disjoint)     5.704894   22.476000   \n",
       "Linear UCB (eps = -5.0)(disjoint)      5.680037   22.249452   \n",
       "Linear UCB (disjoint models)           5.716758   22.581491   \n",
       "Popular                                8.111187   14.963971   \n",
       "Linear UCB (eps = -2.0)(disjoint)      5.731244   22.330929   \n",
       "Linear UCB (eps = -2.0)(hybrid)        7.220896  257.090631   \n",
       "Wilson                                 6.024863   11.420777   \n",
       "Random (popularity-based)              5.792844   12.011829   \n",
       "Random (uniform)                       6.210629   13.071360   \n",
       "TS (context-free)                      6.046631   11.607105   \n",
       "UCB                                    6.115509   11.191166   \n",
       "\n",
       "                                                                               params  \n",
       "Linear UCB (eps = -10.0)(hybrid)          {'regression type': 'hybrid', 'seed': None}  \n",
       "Linear UCB (eps = -5.0)(hybrid)           {'regression type': 'hybrid', 'seed': None}  \n",
       "Linear UCB (eps = -10.0)(disjoint)      {'regression type': 'disjoint', 'seed': None}  \n",
       "Linear UCB (eps = -5.0)(disjoint)       {'regression type': 'disjoint', 'seed': None}  \n",
       "Linear UCB (disjoint models)            {'regression type': 'disjoint', 'seed': None}  \n",
       "Popular                             {'use_rating': False, 'add_cold_items': True, ...  \n",
       "Linear UCB (eps = -2.0)(disjoint)       {'regression type': 'disjoint', 'seed': None}  \n",
       "Linear UCB (eps = -2.0)(hybrid)           {'regression type': 'hybrid', 'seed': None}  \n",
       "Wilson                              {'alpha': 0.05, 'add_cold_items': True, 'cold_...  \n",
       "Random (popularity-based)           {'distribution': 'popular_based', 'alpha': 89....  \n",
       "Random (uniform)                    {'distribution': 'uniform', 'alpha': 0.0, 'see...  \n",
       "TS (context-free)                                     {'sample': False, 'seed': None}  \n",
       "UCB                                 {'exploration_coef': 2.0, 'sample': False, 'se...  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "e.results.sort_values('NDCG@10', ascending=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}