H01 Dataset#

In this notebook, you can learn how to work with the H01 dataset using NAVis.

The H01 dataset contains 57,000 cells and 150 million synapses from a cubic millimeter of the human temporal cortex, which is proofread using the CAVE ecoystem.

With this interface, you can access both a snapshot of the proofread dataset and the latest dataset using caveclient:

pip install caveclient -U

Authentication

If this is your first time using CAVEclient to access the H01 dataset, you might have to get and set your authentication token:

Go to: https://https//global.brain-wire-test.org/auth/api/v1/create_token to create a new token.
Log in with your Google credentials and copy the token shown afterward.

Save it to your computer with:

from caveclient import CAVEclient
client = CAVEclient(server_address="https://global.brain-wire-test.org", datastack_name='h01_c3_flat', auth_token="PASTE_YOUR_TOKEN_HERE")
client.auth.save_token(token="PASTE_YOUR_TOKEN_HERE")

Note that the H01 dataset uses a server address that is different from the default CAVEClient server. Also be aware that creating a new token by finishing step 2 will invalidate the previous token!

import navis
from navis.interfaces import h01

# Initialize the client
client = h01.get_cave_client()

Traceback (most recent call last):
  File "/home/runner/work/navis/navis/docs/examples/4_remote/tutorial_remote_04_h01.py", line 39, in <module>
    client = h01.get_cave_client()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/navis/navis/navis/interfaces/h01.py", line 24, in get_cave_client
    client.materialize.nucleus_table = NUCLEUS_TABLE
    ^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/caveclient/frameworkclient.py", line 633, in materialize
    self._materialize = MaterializationClient(
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/caveclient/materializationengine.py", line 221, in __init__
    super(MaterializationClient, self).__init__(
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/caveclient/base.py", line 217, in __init__
    self._server_version = self._get_version()
                           ^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/caveclient/base.py", line 246, in _get_version
    version_str = handle_response(response, as_json=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/caveclient/base.py", line 94, in handle_response
    _raise_for_status(response, log_warning=log_warning)
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/caveclient/base.py", line 84, in _raise_for_status
    raise requests.HTTPError(http_error_msg, response=r)
requests.exceptions.HTTPError: 503 Server Error: Service Temporarily Unavailable for url: https://local.brain-wire-test.org/materialize/version content:b'<html>\r\n<head><title>503 Service Temporarily Unavailable</title></head>\r\n<body>\r\n<center><h1>503 Service Temporarily Unavailable</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n'

Query Tables#

client.materialize.get_versions()

client.materialize.get_tables()

Query Materialized Synapse Table#

Query the first few rows in the table

client.materialize.synapse_query(limit=10)

Query specific pre- and/or postsynaptic IDs

syn = client.materialize.synapse_query(
    post_ids=[864691131861340864],
    # pre_ids=[ADD YOUR ROOT ID],
)
syn.head()

print(len(syn))

Live Synapse Queries#

import datetime as dt

# Check if root ID is the most recent root ID
root_id = 864691131861340864
now = dt.datetime.now(dt.timezone.utc)
is_latest = client.chunkedgraph.is_latest_roots([root_id], timestamp=now)
latest_id = client.chunkedgraph.get_latest_roots(root_id, timestamp=now)
print(is_latest, latest_id)

synapse_table = client.info.get_datastack_info()["synapse_table"]
df = client.materialize.query_table(
    synapse_table,
    timestamp=dt.datetime.now(dt.timezone.utc),
    filter_equal_dict={"post_pt_root_id": latest_id[0]},
)
df.head()

Query Cells Table#

ct = client.materialize.query_table(table="cells")
ct.head()

ct.cell_type.unique()

Filter by cell type#

# Get the first 50 interneurons
interneuron_ids = ct[ct.cell_type == "INTERNEURON"].pt_root_id.values[:50]

# Remove 0 IDs
interneuron_ids = interneuron_ids[interneuron_ids != 0]

# What's left?
interneuron_ids

Fetch Neuron Meshes#

interneurons = h01.fetch_neurons(interneuron_ids, lod=2, with_synapses=False)
interneurons_ds = navis.simplify_mesh(interneurons, F=1 / 3)
interneurons_ds

# Plot
import seaborn as sns

colors = {n.id: sns.color_palette("Reds", 7)[i] for i, n in enumerate(interneurons_ds)}
navis.plot3d([interneurons_ds], color=colors)

Fetch Skeletons#

interneurons_sk = navis.skeletonize(interneurons, parallel=True)
interneurons_sk

# Plot
navis.plot3d([interneurons_sk[0], interneurons[0]], color=[(1, 0, 0), (1, 1, 1, 0.5)])

Total running time of the script: ( 0 minutes 2.298 seconds)

Download Python source code: tutorial_remote_04_h01.py

Download Jupyter notebook: tutorial_remote_04_h01.ipynb

Gallery generated by mkdocs-gallery