docs/chapter3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "![NeuronUnit Logo](https://raw.githubusercontent.com/scidash/assets/master/logos/neuronunit-logo-text.png)\n",
    "# Chapter 3\n",
    "Back to [Chapter 2](chapter2.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In the first chapter we used a dummy model and simple test to illustrate some basic concepts.  Here we'll use NeuronUnit in a more interesting way: let's validate a real ion channel model's IV (current vs voltage) curve against data from a published experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cloning into 'ChannelWorm'...\n",
      "remote: Enumerating objects: 4442, done.\u001b[K\n",
      "remote: Total 4442 (delta 0), reused 0 (delta 0), pack-reused 4442\u001b[K\n",
      "Receiving objects: 100% (4442/4442), 43.65 MiB | 3.17 MiB/s, done.\n",
      "Resolving deltas: 100% (2221/2221), done.\n"
     ]
    }
   ],
   "source": [
    "!git clone https://github.com/openworm/ChannelWorm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# First get ChannelWorm at http://github.com/openworm/channelworm\n",
    "from pathlib import Path\n",
    "import sys\n",
    "CHANNELWORM_HOME = Path('ChannelWorm').resolve()\n",
    "sys.path.insert(1, str(CHANNELWORM_HOME))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/rgerkin/miniconda3/lib/python3.8/site-packages/airspeed/__init__.py:505: FutureWarning: Possible nested set at position 8\n",
      "  KEYVALSEP = re.compile(r'[ \\t]*:[[ \\t]*(.*)$', re.S)\n"
     ]
    }
   ],
   "source": [
    "import channelworm\n",
    "if not getattr(channelworm, 'setup', False):\n",
    "    channelworm.django_setup() # Adds all channelworm subdirectories to the path, including ion_channel.  \n",
    "    channelworm.setup = True\n",
    "from channelworm.ion_channel.models import GraphData # A class to retrieve digitized data from figures.  \n",
    "from neuronunit.tests.channel import IVCurvePeakTest # A test for the current-voltage relationship \n",
    "                                                     # of a neuron or ion channel.\n",
    "from neuronunit.models.channel import ChannelModel # A SciUnit model class that wraps NeuroML ion channel models.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Now let's instantite the ion channel `model`.  \n",
    "It will be one from a calcium channel whose subunit is thought to be encoded by the EGL-19 gene in C. elegans."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "channel_model_name = 'EGL-19.channel' # Name of a NeuroML channel model (corresponds to the name of a NeuroML file).\n",
    "channel_id = 'ca_boyle' # Identifier for the channel in that NeuroML file.\n",
    "channel_dir_path = CHANNELWORM_HOME / 'models' # Location of NeuroML channel files.  \n",
    "channel_file_path = channel_dir_path / ('%s.nml' % channel_model_name) # Location of the current file.  \n",
    "model = ChannelModel(str(channel_file_path), channel_index=0, name=channel_model_name) # Instantiate the model.  \n",
    "                                                                    # channel_index=0 in most cases, unless there\n",
    "                                                                    # are multiple channels of the same type\n",
    "                                                                    # in the model file.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "`ChannelModel` is a NeuronUnit `model` class that wraps an ion channel model expressed using [NeuroML](http://neuroml.org) and simulates its dynamics using [jNeuroML](https://github.com/NeuroML/jNeuroML).  **You should install both.**\n",
    "### As before, all of the interaction with the model is done through its capabilities.  \n",
    "In this case, those capabilities are inherited in the `ChannelModel` class definition, and implemented in the methods of that class as defined in `neuronunit.models`.  \n",
    "\n",
    "Now we get an observation of real ion channel behavior to use for our test:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "ename": "SynchronousOnlyOperation",
     "evalue": "You cannot call this from an async context - use a thread or sync_to_async.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mSynchronousOnlyOperation\u001b[0m                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-2587905dc665>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mdoi\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'10.1083/jcb.200203055'\u001b[0m \u001b[0;31m# A Digital Object Identifier for (in this case) a journal article.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mfig\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'2B'\u001b[0m \u001b[0;31m# A reference to a figure in that article.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m sample_data = GraphData.objects.get(graph__experiment__reference__doi=doi, \n\u001b[0m\u001b[1;32m      6\u001b[0m                                     graph__figure_ref_address=fig) # If that figure has been digitized by the \n\u001b[1;32m      7\u001b[0m                                                                    \u001b[0;31m# ChannelWorm team, we load its data here.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/lib/python3.8/site-packages/django/db/models/manager.py\u001b[0m in \u001b[0;36mmanager_method\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m     83\u001b[0m         \u001b[0;32mdef\u001b[0m \u001b[0mcreate_method\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     84\u001b[0m             \u001b[0;32mdef\u001b[0m \u001b[0mmanager_method\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 85\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_queryset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     86\u001b[0m             \u001b[0mmanager_method\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     87\u001b[0m             \u001b[0mmanager_method\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__doc__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__doc__\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/lib/python3.8/site-packages/django/db/models/query.py\u001b[0m in \u001b[0;36mget\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    423\u001b[0m             \u001b[0mlimit\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mMAX_GET_RESULTS\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    424\u001b[0m             \u001b[0mclone\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mquery\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_limits\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhigh\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mlimit\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 425\u001b[0;31m         \u001b[0mnum\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mclone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    426\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mnum\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    427\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mclone\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_result_cache\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/lib/python3.8/site-packages/django/db/models/query.py\u001b[0m in \u001b[0;36m__len__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    267\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    268\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__len__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 269\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_fetch_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    270\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_result_cache\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    271\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/lib/python3.8/site-packages/django/db/models/query.py\u001b[0m in \u001b[0;36m_fetch_all\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1306\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_fetch_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1307\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_result_cache\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1308\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_result_cache\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_iterable_class\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1309\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_prefetch_related_lookups\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_prefetch_done\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1310\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_prefetch_related_objects\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/lib/python3.8/site-packages/django/db/models/query.py\u001b[0m in \u001b[0;36m__iter__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     51\u001b[0m         \u001b[0;31m# Execute the query. This will also fill compiler.select, klass_info,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     52\u001b[0m         \u001b[0;31m# and annotations.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 53\u001b[0;31m         \u001b[0mresults\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcompiler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute_sql\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchunked_fetch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchunked_fetch\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchunk_size\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchunk_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     54\u001b[0m         select, klass_info, annotation_col_map = (compiler.select, compiler.klass_info,\n\u001b[1;32m     55\u001b[0m                                                   compiler.annotation_col_map)\n",
      "\u001b[0;32m~/miniconda3/lib/python3.8/site-packages/django/db/models/sql/compiler.py\u001b[0m in \u001b[0;36mexecute_sql\u001b[0;34m(self, result_type, chunked_fetch, chunk_size)\u001b[0m\n\u001b[1;32m   1152\u001b[0m             \u001b[0mcursor\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconnection\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchunked_cursor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1153\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1154\u001b[0;31m             \u001b[0mcursor\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconnection\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcursor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1155\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1156\u001b[0m             \u001b[0mcursor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msql\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/miniconda3/lib/python3.8/site-packages/django/utils/asyncio.py\u001b[0m in \u001b[0;36minner\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m     22\u001b[0m                 \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     23\u001b[0m                     \u001b[0;32mif\u001b[0m \u001b[0mevent_loop\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_running\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 24\u001b[0;31m                         \u001b[0;32mraise\u001b[0m \u001b[0mSynchronousOnlyOperation\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     25\u001b[0m             \u001b[0;31m# Pass onwards.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     26\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mSynchronousOnlyOperation\u001b[0m: You cannot call this from an async context - use a thread or sync_to_async."
     ]
    }
   ],
   "source": [
    "import quantities as pq\n",
    "\n",
    "doi = '10.1083/jcb.200203055' # A Digital Object Identifier for (in this case) a journal article.  \n",
    "fig = '2B' # A reference to a figure in that article.  \n",
    "sample_data = GraphData.objects.get(graph__experiment__reference__doi=doi, \n",
    "                                    graph__figure_ref_address=fig) # If that figure has been digitized by the \n",
    "                                                                   # ChannelWorm team, we load its data here.  \n",
    "obs = list(zip(*sample_data.asarray())) # Reformat this to a raw observation of current density and voltage.  \n",
    "observation = {'i/C':obs[1]*pq.A/pq.F, 'v':obs[0]*pq.mV} # Make a dictionary with the appropriately valued units.\n",
    "cell_capacitance = 1e-13 * pq.F # Capacitance is arbitrary if IV curves are scaled.  \n",
    "observation['i'] = observation['i/C']*cell_capacitance\n",
    "\n",
    "test = IVCurvePeakTest(observation, scale=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now that we have an observation, we can instantiate our test.  \n",
    "### How did we know the observation needs the form above?  \n",
    "It is stated in the tests's docstring, or can be inferred from its optional validate_observation method:\n",
    "```python\n",
    "class IVCurveTest(sciunit.Test):\n",
    "    ...\n",
    "    def validate_observation(self, observation):\n",
    "            assert type(observation) is dict\n",
    "            for item in ['v', 'i']:\n",
    "                assert item in observation\n",
    "                assert type(observation[item]) in [list,tuple] \\\n",
    "                    or isinstance(observation[item],np.ndarray)\n",
    "    ...\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "test = IVCurvePeakTest(observation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### With the test instantiated, we can now judge the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# Judge the model output against the experimental data\n",
    "score = test.judge(model)\n",
    "score.summarize()\n",
    "print(\"The score was computed according to '%s' with raw value %s and pass cutoff %s\" \\\n",
    "    % (score.description,score.raw,test.converter.cutoff))\n",
    "print('The scaling factor for the model IV curve was %.3g' % score.related_data['scale_factor'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### A `score` object isn't just a number.###\n",
    "It is also bound to data that allow us to visualize the test result and how the score was reached:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pyplot as plt\n",
    "mpl.rcParams.update({'font.size':14, 'lines.linewidth':3})\n",
    "score.plot()\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "That's a failing score, because those curves don't look too similar.  What was the criterion for passing?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "score.describe()\n",
    "print(\"Raw score is %s\" % score.raw)\n",
    "score.test.converter.description"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Let's change the threshold for a passing score:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "score.test.converter.cutoff = pq.Quantity(10.0,'pA**2') # Make it easier to pass.  \n",
    "score = test.converter.convert(score)\n",
    "score.summarize() # Now shows that the test has passed.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now the score is changed to reflect the new cutoff.  This cutoff is up the person doing the testing!  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### On to [Chapter 4](chapter4.ipynb)!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}