.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tutorials/qualitymetrics/plot_4_curation.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_tutorials_qualitymetrics_plot_4_curation.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_tutorials_qualitymetrics_plot_4_curation.py:


Curation Tutorial
==================

After spike sorting and computing quality metrics, you can automatically curate the spike sorting output using the
quality metrics that you have calculated.

.. GENERATED FROM PYTHON SOURCE LINES 11-12

Import the modules and/or functions necessary from spikeinterface

.. GENERATED FROM PYTHON SOURCE LINES 12-18

.. code-block:: Python


    import spikeinterface.core as si

    from spikeinterface.qualitymetrics import compute_quality_metrics









.. GENERATED FROM PYTHON SOURCE LINES 19-21

Let's generate a simulated dataset, and imagine that the ground-truth
sorting is in fact the output of a sorter.

.. GENERATED FROM PYTHON SOURCE LINES 21-26

.. code-block:: Python


    recording, sorting = si.generate_ground_truth_recording()
    print(recording)
    print(sorting)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    GroundTruthRecording (InjectTemplatesRecording): 4 channels - 25.0kHz - 1 segments 
                          250,000 samples - 10.00s - float32 dtype - 3.81 MiB
    GroundTruthSorting (NumpySorting): 10 units - 1 segments - 25.0kHz




.. GENERATED FROM PYTHON SOURCE LINES 27-32

Create SortingAnalyzer
-----------------------

For this example, we will need a :code:`SortingAnalyzer` and some extensions
to be computed first

.. GENERATED FROM PYTHON SOURCE LINES 32-41

.. code-block:: Python



    analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format="memory")
    analyzer.compute(["random_spikes", "waveforms", "templates", "noise_levels"])

    analyzer.compute("principal_components", n_components=3, mode="by_channel_local")
    print(analyzer)






.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    estimate_sparsity (no parallelization):   0%|          | 0/10 [00:00<?, ?it/s]
    estimate_sparsity (no parallelization): 100%|██████████| 10/10 [00:00<00:00, 417.36it/s]

    compute_waveforms (no parallelization):   0%|          | 0/10 [00:00<?, ?it/s]
    compute_waveforms (no parallelization): 100%|██████████| 10/10 [00:00<00:00, 324.70it/s]

    noise_level (no parallelization):   0%|          | 0/20 [00:00<?, ?it/s]
    noise_level (no parallelization): 100%|██████████| 20/20 [00:00<00:00, 263.04it/s]

    Fitting PCA:   0%|          | 0/10 [00:00<?, ?it/s]
    Fitting PCA: 100%|██████████| 10/10 [00:00<00:00, 180.81it/s]

    Projecting waveforms:   0%|          | 0/10 [00:00<?, ?it/s]
    Projecting waveforms: 100%|██████████| 10/10 [00:00<00:00, 2619.64it/s]
    SortingAnalyzer: 4 channels - 10 units - 1 segments - memory - sparse - has recording
    Loaded 5 extensions: random_spikes, waveforms, templates, noise_levels, principal_components




.. GENERATED FROM PYTHON SOURCE LINES 42-43

Then we compute some quality metrics:

.. GENERATED FROM PYTHON SOURCE LINES 43-47

.. code-block:: Python


    metrics = compute_quality_metrics(analyzer, metric_names=["snr", "isi_violation", "nearest_neighbor"])
    print(metrics)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    calculate pc_metrics:   0%|          | 0/10 [00:00<?, ?it/s]
    calculate pc_metrics:  60%|██████    | 6/10 [00:00<00:00, 51.29it/s]
    calculate pc_metrics: 100%|██████████| 10/10 [00:00<00:00, 51.43it/s]
             snr  isi_violations_ratio  ...  nn_hit_rate  nn_miss_rate
    0   2.081982                   0.0  ...     0.495536      0.068199
    1   1.897748                   0.0  ...     0.496711      0.067587
    2   0.603453                   0.0  ...     0.513793      0.039226
    3  27.721287                   0.0  ...     0.975904      0.004222
    4   2.954370                   0.0  ...     0.493421      0.054506
    5  14.908217                   0.0  ...     0.916176      0.008468
    6   9.532658                   0.0  ...     0.924837      0.005636
    7   3.572693                   0.0  ...     0.743056      0.028360
    8   1.941346                   0.0  ...     0.365809      0.061243
    9  30.585269                   0.0  ...     0.963028      0.002165

    [10 rows x 5 columns]




.. GENERATED FROM PYTHON SOURCE LINES 48-53

We can now threshold each quality metric and select units based on some rules.

The easiest and most intuitive way is to use boolean masking with a dataframe.

Then create a list of unit ids that we want to keep

.. GENERATED FROM PYTHON SOURCE LINES 53-61

.. code-block:: Python


    keep_mask = (metrics["snr"] > 7.5) & (metrics["isi_violations_ratio"] < 0.2) & (metrics["nn_hit_rate"] > 0.90)
    print(keep_mask)

    keep_unit_ids = keep_mask[keep_mask].index.values
    keep_unit_ids = [unit_id for unit_id in keep_unit_ids]
    print(keep_unit_ids)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    0    False
    1    False
    2    False
    3     True
    4    False
    5     True
    6     True
    7    False
    8    False
    9     True
    dtype: bool
    ['3', '5', '6', '9']




.. GENERATED FROM PYTHON SOURCE LINES 62-63

And now let's create a sorting that contains only curated units and save it.

.. GENERATED FROM PYTHON SOURCE LINES 63-70

.. code-block:: Python


    curated_sorting = sorting.select_units(keep_unit_ids)
    print(curated_sorting)


    curated_sorting.save(folder="curated_sorting")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    GroundTruthSorting (UnitsSelectionSorting): 4 units - 1 segments - 25.0kHz


.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div style='border:1px solid #ddd; padding:10px;'><strong>NumpyFolder (NumpyFolderSorting): 4 units - 1 segments - 25.0kHz</strong></div><details style='margin-left: 10px;'>  <summary><strong>Unit IDs</strong></summary><ul>['3' '5' '6' '9'] </details><details style='margin-left: 10px;'>  <summary><strong>Annotations</strong></summary><ul><li> <strong> name </strong>: GroundTruthSorting</li></details><details style='margin-left: 10px;'><summary><strong>Properties</strong></summary><ul><details><summary><strong>gt_unit_locations</strong></summary>[[12.629008  -7.895358   7.6198726]
     [22.504269  29.685955  18.601088 ]
     [29.937254  29.228424  43.268787 ]
     [ 7.276633  -9.420886  33.948456 ]]</details></ul></details>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 71-72

We can also save the analyzer with only theses units

.. GENERATED FROM PYTHON SOURCE LINES 72-76

.. code-block:: Python


    clean_analyzer = analyzer.select_units(unit_ids=keep_unit_ids, format="zarr", folder="clean_analyzer")

    print(clean_analyzer)




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    SortingAnalyzer: 4 channels - 4 units - 1 segments - zarr - sparse - has recording
    Loaded 6 extensions: random_spikes, waveforms, templates, noise_levels, principal_components, quality_metrics





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.527 seconds)


.. _sphx_glr_download_tutorials_qualitymetrics_plot_4_curation.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_4_curation.ipynb <plot_4_curation.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_4_curation.py <plot_4_curation.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_4_curation.zip <plot_4_curation.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_