{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Co-plotting flywire mesh with FAFB autosegments" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### This section contains example of fetching meshes from flywire and co-plotting with automatically segmented data in FAFB" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import neccesary library modules now" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import navis\n", "import fafbseg\n", "import pymaid" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import numpy as np\n", "import os\n", "from copy import deepcopy" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from pyroglancer.layers import create_nglayer\n", "from pyroglancer.localserver import startdataserver, closedataserver\n", "from pyroglancer.ngviewer import openviewer, closeviewer\n", "from pyroglancer.ngspaces import create_ngspace\n", "from pyroglancer.createconfig import createconfig" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fetch meshes from flywire" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "#set the flywire token here if you haven't set before..\n", "#fafbseg.flywire.set_chunkedgraph_secret(\"blabla\")" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "#mesh in flywire for which to find a match..\n", "flywire_id = 720575940633919863" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
typenavis.MeshNeuron
nameNone
id720575940633919863
units1 nanometer
n_vertices785994
n_faces1575396
\n", "
" ], "text/plain": [ "type navis.MeshNeuron\n", "name None\n", "id 720575940633919863\n", "units 1 nanometer\n", "n_vertices 785994\n", "n_faces 1575396\n", "dtype: object" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Fetch the neuron's mesh\n", "putativeDNa02_a = fafbseg.flywire.get_mesh_neuron(flywire_id)\n", "putativeDNa02_a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Transform flywire meshes to fafb" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "DNa02_xyz = pd.DataFrame(list(map(np.ravel, putativeDNa02_a.vertices)), columns=['x', 'y', 'z'])" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Transform path: FLYWIRE -> FLYWIREraw -> FAFB14raw -> FAFB14\n" ] } ], "source": [ "#move the vertices of the flywire mesh to fafb..\n", "fafb_putativeDNa02_xyz = navis.xform_brain(DNa02_xyz, source='FLYWIRE', target='FAFB14')" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "fafb_putativeDNa02_a = putativeDNa02_a.copy()" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "#put the transformed vertices in the new mesh..\n", "fafb_putativeDNa02_a.vertices = fafb_putativeDNa02_xyz.to_numpy()" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#perform a bit of conversion to navis volumes..\n", "fafb_putativeDNa02_a = navis.core.volumes.Volume(fafb_putativeDNa02_a)\n", "fafb_putativeDNa02_a.id = flywire_id\n", "fafb_putativeDNa02_a.name = 'putative_DNa02'\n", "fafb_putativeDNa02_a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fetch neuropils from catmaid for display" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "publicurl = 'https://fafb.catmaid.virtualflybrain.org/'" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO : Global CATMAID instance set. Caching is ON. (pymaid)\n", "INFO - 2021-05-19 23:36:12,510 - client - Global CATMAID instance set. Caching is ON.\n" ] } ], "source": [ "working_rm = pymaid.CatmaidInstance(publicurl, api_token=None, project_id = 1)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "#fetch the LAL neuropil mesh..\n", "LAL_neuropil = pymaid.get_volume('LAL_L', color=(255, 0, 0, .2))" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#perform a bit of conversion to navis volumes..\n", "LAL_neuropil = navis.core.volumes.Volume(LAL_neuropil)\n", "LAL_neuropil.id = 100\n", "LAL_neuropil.name = 'LAL_L'\n", "LAL_neuropil" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO : Cached data used. Use `pymaid.clear_cache()` to clear. (pymaid)\n", "INFO - 2021-05-19 23:36:12,900 - client - Cached data used. Use `pymaid.clear_cache()` to clear.\n" ] } ], "source": [ "#fetch the FB neuropil mesh..\n", "FB_neuropil = pymaid.get_volume('FB', color=(0, 255, 0, .2))" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#perform a bit of conversion to navis volumes..\n", "FB_neuropil = navis.core.volumes.Volume(FB_neuropil)\n", "FB_neuropil.id = 200\n", "FB_neuropil.name = 'FB'\n", "FB_neuropil" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Start dataserver to host precomputed data" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Serving data from: /var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl\n", "Serving directory at http://127.0.0.1:8000\n" ] } ], "source": [ "startdataserver()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Start a basic neuroglancer local session with all FAFB configurations" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "configdata = [dict(\n", " ngspace='FAFB',\n", " dimension=dict(x=1, y=1,z=1,units='um'),\n", " voxelsize=dict(x=4,y=4,z=40,units='nm'),\n", " layers=dict(\n", " fafb_v14_clahe=dict(\n", " type='image',\n", " source='precomputed://gs://neuroglancer-fafb-data/fafb_v14/fafb_v14_clahe'),\n", " fafb_surf=dict(\n", " type='surfacemesh',\n", " source='vtk://https://storage.googleapis.com/neuroglancer-fafb-data/elmr-data/FAFB.surf.vtk.gz'),\n", " synapses_buhmann2019=dict(\n", " type='synapsepred',\n", " source='precomputed://gs://neuroglancer-20191211_fafbv14_buhmann2019_li20190805',\n", " linkedseg='seg_20190805'),\n", " seg_20190805=dict(\n", " type='segmentation',\n", " source='precomputed://gs://fafb-ffn1-20190805/segmentation',\n", " ))\n", " )]" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "configfileloc = '/Users/sri/.pyroglancer/config_temp.yml'" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "adding default config file..\n", "setting default config file loc\n" ] } ], "source": [ "createconfig(configdata, configfileloc, overwrite = True)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "layer_kws = {'ngspace': 'FAFB'}" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Neuroglancer viewer created at: http://127.0.0.1:60881/v/0e89a7ba7e856fff29b23295854df3ec6ac9ab38/\n", "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Dimensions are in : FAFB\n", "Layer created: image\n", "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Using layout : xy-3d\n", "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Dimensions are in : FAFB\n", "Layer created: surfacemesh\n", "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Using layout : xy-3d\n", "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Dimensions are in : FAFB\n", "Layer created: synapsepred\n", "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Using layout : xy-3d\n", "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Dimensions are in : FAFB\n", "Layer created: segmentation\n", "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Using layout : xy-3d\n" ] } ], "source": [ "create_ngspace(layer_kws)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Inject the neuron mesh and neuropils" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Dimensions are in : FAFB\n", "Layer created: volumes\n", "mesh/720575940633919863\n", "Seg id is: 720575940633919863\n", "Full filepath: /private/var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl/precomputed/flywiremeshes/mesh/720575940633919863\n", "creating: /private/var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl/precomputed/flywiremeshes/mesh/segment_properties\n", "creating: /private/var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl/precomputed/flywiremeshes/mesh/segment_names\n", "Using layout : xy-3d\n" ] } ], "source": [ "tmpviewer = create_nglayer(layer_kws = {'type': 'volumes','source': fafb_putativeDNa02_a,\n", " 'name': 'flywiremeshes','color': 'yellow'})" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "config file loc is at: None\n", "using default location at: /Users/sri/.pyroglancer/config_temp.yml\n", "Dimensions are in : FAFB\n", "Layer created: volumes\n", "mesh/100\n", "Seg id is: 100\n", "Full filepath: /private/var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl/precomputed/neuropils/mesh/100\n", "mesh/200\n", "Seg id is: 200\n", "Full filepath: /private/var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl/precomputed/neuropils/mesh/200\n", "creating: /private/var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl/precomputed/neuropils/mesh/segment_properties\n", "creating: /private/var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl/precomputed/neuropils/mesh/segment_names\n", "Using layout : xy-3d\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "127.0.0.1 - - [19/May/2021 23:36:13] \"GET /precomputed/flywiremeshes/mesh/info HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:36:13] \"GET /precomputed/neuropils/mesh/info HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:36:13] \"GET /precomputed/flywiremeshes/mesh/segment_properties/info HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:36:13] \"GET /precomputed/neuropils/mesh/segment_properties/info HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:36:46] \"GET /precomputed/flywiremeshes/mesh/720575940633919863:0 HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:36:46] \"GET /precomputed/flywiremeshes/mesh/720575940633919863 HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:38:04] \"GET /precomputed/neuropils/mesh/100:0 HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:38:04] \"GET /precomputed/neuropils/mesh/200:0 HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:38:04] \"GET /precomputed/neuropils/mesh/100 HTTP/1.1\" 200 -\n", "127.0.0.1 - - [19/May/2021 23:38:04] \"GET /precomputed/neuropils/mesh/200 HTTP/1.1\" 200 -\n" ] } ], "source": [ "tmpviewer = create_nglayer(layer_kws = {'type': 'volumes','source': [LAL_neuropil,FB_neuropil],\n", " 'name': 'neuropils','color': ['magenta', 'cyan'], 'alpha': 0.3})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### After injection poke around the flywire mesh" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### The yellow mesh is the flywire neuron mesh and the green mesh is the fafb poke around match. The magenta region is the LAL, and the cyan region is the FB. The red dots are presynapses, the blue dots are postsynapses. You can see that the putative DNa02_a is indeed getting many synapses from LAL" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![alt text](pics/flywire2fafb.png \"Title\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Close the viewer and dataserver" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "closing already existing ng viewer\n" ] } ], "source": [ "closeviewer()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Closing server at http://127.0.0.1:8000\n", "Cleaning directory at /private/var/folders/_l/lrfvj_8j3ps0c37ncbr3c8dh0000gn/T/tmprbh5_3bl\n" ] } ], "source": [ "closedataserver()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 4 }