Migrating to NAVis 2.0#
NAVis 2.0 is the first major release since 1.0 and the first that is allowed to break things. This page is three things: a summary of what the release is about, a checklist of everything you may have to change in your own code, and - below the fold - the complete list of changes, which is what the changelog points at rather than repeats.
What 2.0 is about#
Rust is the engine. navis-fastcore has been an optional dependency for several releases, with a second, pure-Python implementation kept bit-for-bit in step behind it. That is over: fastcore is now required and the fallbacks are gone. The same move retires the other "install this and NAVis will be faster" paths - the four mesh simplification backends, the mesh smoothing backends, the CMTK/elastix binaries, pycpd, morphops/molesq - each replaced by one implementation that everybody gets. Most of the speed-ups in this release are that, and some of them are large: betweenness_centrality 217 ms 0.49 ms, smooth_skeleton ~200x, smooth_mesh ~50x, alignment 20-28x, xform_brain 4-30x.
One neuron model. Which attributes of a neuron are aligned to its nodes, vertices, points or voxels - and which of them reference those elements - is now declared once, per type, in navis/core/schema.py. subset_neuron was rewritten onto it (three hand-written implementations became one, and four silent bugs fell out), and three things that were not possible before follow from it: navis.masked to work on part of a neuron and fold the edits back in, Neuron.attach to carry your data through every selection, and a Mesh that keeps its skeleton - with the same node IDs - through a subset, a mask or a simplification.
One name per thing. The neuron classes are now Skeleton, Mesh, Voxels and Dotprops; a distance cap is max_dist everywhere; min_size counts elements and min_length is a distance; connected components have one function, one shape and one home; and every morphology function that takes a single neuron has a method, without the type suffix. Old names mostly still work with a DeprecationWarning - see the checklist below.
New capabilities, most of which close a hole rather than open a front: plot_collage and plot3d(snapshot=True) for figures, nblast_knn for k-NN at connectome scale, heal_mesh/fill_holes/Mesh.extra_edges and find_soma_mesh so meshes can do what skeletons could, a proper Voxels toolkit that never materialises the grid, navis.ml for machine-learning inputs, .rds/.rda I/O that replaces the rpy2 interface, Pipeline to fuse a chain of operations into one pass over the neurons, and parallel backends that reach a cluster.
And a lot of fixes. Rather more of them than usual return different numbers than 1.12 did, because that is what a fix to a silent bug looks like - see Results that change.
Migration checklist#
Renamed - old name still works, with a DeprecationWarning#
| Old | New |
|---|---|
navis.TreeNeuron | navis.Skeleton |
navis.MeshNeuron | navis.Mesh |
navis.VoxelNeuron | navis.Voxels |
navis.break_fragments() | navis.split_components() |
navis.split_into_fragments() | navis.split_neurites() |
navis.graph.connected_subgraph() | navis.graph.connecting_nodes() |
Skeleton.subtrees | Skeleton.connected_components() |
Skeleton.n_trees, Skeleton.n_skeletons | Skeleton.n_components |
Skeleton.is_tree | Skeleton.is_acyclic |
bridging_graph(reciprocal=) & co. | inverse_weight= (and the default changed from 0.5 to 1) |
smooth_mesh(L=) | lamb= |
nblast_*(limit_dist=) | max_dist= |
geodesic_matrix(limit=), average_skeletons(limit=) | max_dist= |
cable_overlap(dist=) | max_dist= |
split_neurites(min_size=) | min_length= |
prune_twigs(size=), Skeleton.prune_twigs(size=) | min_length= |
drop_fluff(keep_size=) | min_size= |
heal_skeleton(drop_disc=), heal_mesh(drop_disc=) | keep_largest= |
simplify_mesh(backend=), smooth_mesh(backend=) | gone; there is only one backend now (the argument is ignored) |
The class names are aliases, not subclasses: isinstance(x, navis.TreeNeuron) and class MyNeuron(navis.TreeNeuron) behave exactly as before, and neurons pickled by earlier versions still load. Note that .type follows the class, so it now reads "navis.Skeleton" - code matching on that string, including anything filtering a NeuronList.summary() table by type, needs updating.
The renamed keyword arguments are shimmed too: the old spelling warns and is forwarded, and passing both spellings is a TypeError rather than a coin toss over which one wins.
Python hides DeprecationWarnings by default outside __main__; run with python -W default::DeprecationWarning (or pytest -W default) to find the old names in your code.
Renamed - no alias#
| Old | New | Where |
|---|---|---|
dp.drop_fluff(500) | dp.drop_fluff(epsilon=500) | epsilon is keyword-only on the shared method |
navis.betweeness_centrality | navis.betweenness_centrality | the misspelling is gone |
navis.graph.connected_components_of(x, mask) | navis.connected_components(x, mask=...) | |
break_fragments(..., labels_only=True) | navis.connected_components | |
prune_by_strahler(..., relocate_connectors=) | removed; passing it raises TypeError | the default was False, so nothing changes unless you passed it |
Removed#
| Removed | Use instead |
|---|---|
navis.interfaces.r (and the navis[r] extra) | read_rds/write_rds, read_rda/write_rda, plus NAVis' own nblast/xform_brain |
navis.interfaces.cytoscape | navis.network2nx + networkx.write_graphml |
the vispy plotting backend, navis.Viewer, navis.utils.check_vispy | octarine; plot3d returns an octarine.Viewer |
plot3d arguments combine, shading, shininess, name | vispy-only; no replacement |
navis.meshes.simplify_mesh_fqmr/_open3d/_pyml/_blender, navis.meshes.available_backends | navis.simplify_mesh, navis.smooth_mesh |
navis.find_soma(x)[0], len(navis.find_soma(x)) | find_soma now returns a single node ID or None |
| the non-fastcore graph/geodesic/morphometric/NBLAST fallbacks | nothing to do - fastcore is required |
the private stitching helpers _stitch_edges, _segment_radii, _rewire_from_edges, _component_labels | heal_skeleton, stitch_skeletons |
Deprecated but still working, and scheduled for removal in 3.0: the "binary" CMTK/elastix transform backend and the "python" (morphops/molesq) landmark backend.
Results that change#
These are not renames - the same call returns something different than it did in 1.12. Re-baseline anything pinned to exact output.
| What | Why |
|---|---|
| NBLAST scores (~1e-5) | fastcore now runs on the float32 coordinates NAVis stores, cutting peak memory ~45% |
smooth_skeleton | the window is centred now, not trailing, and branch points are pinned (median 54 nm on the example neuron) |
smooth_mesh | defaults to Taubin's λ|μ filter rather than the plain Laplacian, which cost the example neuron 55% of its volume |
smooth_voxels | treats everything outside the neuron as empty instead of reflecting at the canvas edge |
downsample_neuron | fastcore counts from each segment's distal end, so a different ~7% of nodes survives |
find_soma | returns one node ID (or None), scored by neighbourhood radius, not an array of candidates |
betweenness_centrality(from_=) | sources within one hop of the root now count |
node_label_sorting, rewire_skeleton | ties and leftover roots broken deterministically rather than arbitrarily |
collapse_nodes | no longer re-roots the neuron at the collapsed node |
ivscc_features | neurons are rows now (add a .T for the old layout), and six features were wrong |
| bridging paths | ~29% of routes in navis-flybrains change - mostly NAVis no longer inverting a registration when a purpose-built one exists |
[align_rigid][navis.align.align_rigid]/[align_deform][navis.align.align_deform] | now rcpd registration objects, and four pycpd bugs are fixed |
plot2d with meshes | real back-face culling and depth ordering; plot2d(method="2d") no longer collapses a mesh into a silhouette |
parallel=True | defaults to the joblib backend rather than pathos |
read_precomputed | edges are oriented by traversal, so skeletons written "child first" no longer shatter |
geodesic_matrix, segment_analysis, sholl_analysis, make_dotprops, Voxels | see the fixes below - each was returning something wrong |
Requirements#
navis-fastcore >= 0.13.0is required. Prebuilt wheels exist for macOS (Intel/ARM), Windows and Linux on x86-64, aarch64, i686, armv7l, ppc64le and s390x; only musl-based Linux (Alpine) builds from source.pip install navis[fastcore]still resolves as a no-op aliassparse-cubes[skeleton] >= 0.5.0is required (it backs the sparseVoxels)rcpdis a new optional dependency for alignment:pip install navis[cpd]pathosis no longer needed forparallel=Truesixandpypngare gone;pip install navis[all]no longer drags incloud-volume
Worth adopting#
Nothing here breaks anything - but if you have been working around one of these, stop:
navis.maskedinstead of subsetting, working on the fragment and stitching it back by hand. Inside the block the neuron is the masked region, so every function, property and plot sees only that part; on the way out it is whole again, edits folded in (tutorial)Neuron.attachfor per-node/per-vertex data you currently keep in a parallel array and re-index yourself. Attached data is subset, masked and (where the function can say what happened) carried through rebuilds (tutorial)plot3d(..., snapshot=True)when you want a real render on a matplotlib axes, in the neurons' own coordinates - figures you can annotate, arrange and save as PDFplot_collagefor a page of a few hundred neurons (tutorial)nblast_knninstead of an all-by-all you only wanted the top matches from - 164k neurons is a 107 GB matrix or a 26 MB k-NN graphnavis.Pipelineinstead of calling three functions withparallel=Truein turn, or smuggling the whole chain into oneapplylambda - the neurons make the trip to the workers once for the chain rather than once per function, and the steps stop copying each other's intermediatesset_parallel_backendto pointparallel=Trueatjoblib, adask.distributedcluster, a SLURM array job or anyconcurrent.futures.Executoryou own - andinner_max_num_threadsso the processes and the threads underneath them stop fightingdownsample_neuron(..., method="rdp")to spend the node budget where the neuron curves rather than uniformlynavis.config.strict(orNAVIS_STRICT=1) in servers and pipelines: remote fetches raise instead of returning a partial result, and nothing prompts for inputwrite_parquet(..., format="neurarrow")when the file has to be read by something that is not NAVis
All changes#
Everything below is the release in full, grouped by what it touches.
Naming: one name per thing#
The changes that touch the most code are the ones that make NAVis say the same thing the same way everywhere.
Breaking#
-
one name for a distance cap (
max_dist), andmin_size/min_lengthnow mean count and distance respectively. The same "stop at this distance" argument was spelled five ways depending on which corner of NAVis you were in, andmin_sizemeant a node count in four functions and a cable length in a fifth.Old New Functions limit_distmax_distall five nblast_*limitmax_distgeodesic_matrix,average_skeletonsdistmax_distcable_overlapmin_sizemin_lengthsplit_neuritessizemin_lengthprune_twigs(andSkeleton.prune_twigs())The rule now holds throughout:
min_sizecounts elements (nodes/vertices/voxels -drop_fluff,split_components,heal_skeleton,heal_mesh,stitch_skeletons),min_lengthis a distance, andmax_distcaps a distance.epsilonis deliberately untouched: it does not cap a search but defines the edges of a neighbourhood graph, which is a different question.limitsurvives only where it never meant a distance - restricting which files a reader reads, andguess_radius' count of consecutive missing radii.Every old spelling still works, with a
DeprecationWarning- as do the ones renamed earlier in this cycle (drop_fluff(keep_size=),heal_*(drop_disc=)) andsmooth_mesh(L=), whose hand-rolled shim now goes through the same machinery. Passing both spellings is aTypeError, and each warns once per session so mapping over aNeuronListdoes not produce one warning per neuron.Every one of these now accepts a unit string (e.g.
"5 microns") viamap_units, where it did not already. Newly gained:stitch_skeletons,navis.graph.geodesic_clusters(only whenweightis notNone- withweight=Nonethe radius is a number of hops, which has no units) and thenblast_*family ("auto"andNonekeep their own meaning and pass through). -
every morphology function that takes one neuron now has a method, and the methods drop the type suffix. Which operations were reachable as
neuron.do_this()rather thannavis.do_this(neuron)was arbitrary:prune_twigshad a method,despike_skeletondid not, anddrop_fluffhad one onDotpropsalone despite the function accepting all four neuron types - the same holeconnected_componentshad onVoxels.Shared operations now live on
BaseNeuron, so every type has them:Method Wraps Neuron.drop_fluff()navis.drop_fluffNeuron.subset()navis.subset_neuronNeuron.split_axon_dendrite()navis.split_axon_dendriteType-specific ones live on their class and drop the suffix the function carries, following
reroot_skeleton→.reroot()andresample_skeleton→.resample(), which already worked this way:Breaking:
Dotprops.drop_fluff()tookepsilonas its first positional argument; the shared method takes it as a keyword, sodp.drop_fluff(500)must becomedp.drop_fluff(epsilon=500). Every method also takes**kwargsstraight through to the function it wraps, so the two cannot drift apart.All methods now agree on
inplace:inplace=TruereturnsNoneand mutates,inplace=Falsereturns a copy.Mesh.heal()andMesh.smooth()were the odd ones out while being written and were corrected before landing - note that the olderMesh.fill_holes()andMesh.validate()still hand the neuron back either way. -
connected components have one name, one shape and one home. NAVis had six words for "a connected piece of a neuron" (component, fragment, subtree, tree, skeleton, fluff), three incompatible ways to ask for them, and coverage that depended on which neuron type you happened to hold: a
Skeletonhanded back sets of node IDs, aVoxelsa label array, aMeshnothing at all. All of it now goes through one function.New:
navis.connected_components- the primitive, public at last. It takes any neuron (Skeleton,Mesh,Dotprops,Voxelsor a bareTrimesh) and returns an(N, )array of labels, one per node/vertex/point/voxel and aligned with it:>>> labels = navis.connected_components(mesh) >>> np.bincount(labels)[:3] # component sizes array([17058, 240, 12]) >>> mesh.vertices[labels == 0] # the largest componentLabels are sorted by size, largest first, so
== 0always selects the biggest piece andnp.bincountgives the sizes in descending order (ties break by first element, so the labelling is a function of the neuron rather than of union-find's internals). Where the old function returned a list of sets, an array composes withnp.bincount,groupbyand boolean indexing without materialising anything, and matchesscipy.sparse.csgraph.connected_components. It takesconnectivityandepsilon(asdrop_fluffalways did) plus two new arguments:elementdecides what the labels are of. Only meshes have a choice:"vertex"normally, but under a face-basedconnectivitya pinch vertex belongs to several components at once and only"face"is a partition - so that becomes the default there. Asking for"vertex"anyway is allowed; each such vertex takes the label of the largest component it touches.maskrestricts the answer to the induced sub-neuron: anything outside it neither belongs to a component nor connects two. This replaces the internalgraph.connected_components_of.
New: every neuron type has
.connected_components()and.n_components, offBaseNeuron, so aMeshand aDotpropscan answer the question aSkeletonalways could.Renamed:
Old New Note navis.break_fragments()navis.split_components()labels_only=Trueis gone - that is whatconnected_componentsis for. Gainedconnectivity/epsilonnavis.split_into_fragments()navis.split_neurites()never had anything to do with components: it cuts a connected arbor at longest-neurite branch points Skeleton.subtreesSkeleton.connected_components()Skeleton.n_treesSkeleton.n_componentsSkeleton.n_skeletonsSkeleton.n_componentswas len(.root), i.e. the component count only for an acyclic neuron - a cycle has no root and went uncountedSkeleton.is_treeSkeleton.is_acyclicit is a cycle check, not a connectivity check - it always returned Truefor a forest, which the old name made hard to believenavis.graph.connected_subgraph()navis.graph.connecting_nodes()returns the nodes needed to connect a subset; the old name collides with connected_componentsnavis.graph.connected_components_of()navis.connected_components(mask=...)Voxels.connected_components(connectivity=26)Voxels.connected_components()same name, now the shared one; connectivitydefaults throughNoneParameters normalised:
drop_fluff(keep_size=)is nowmin_size=, matchingsplit_componentsand both healers.heal_skeleton(drop_disc=)/heal_mesh(drop_disc=)are nowkeep_largest=, which says what they do (andheal_skeletonannotated itfloatwhereheal_meshhadbool).The renamed top-level functions still resolve under their old names with a
DeprecationWarning, as do the renamedSkeletonproperties. Run withpython -W default::DeprecationWarningto find them in your code. -
the neuron classes have been renamed. The old names said "neuron" three times over and buried the part that actually distinguishes them; the new ones just name the representation:
Old New navis.TreeNeuronnavis.Skeletonnavis.MeshNeuronnavis.Meshnavis.VoxelNeuronnavis.Voxelsnavis.Dotpropsnavis.Dotprops(unchanged)The old names still work and will be removed in a future version. They are aliases, not subclasses -
isinstance(x, navis.TreeNeuron)andclass MyNeuron(navis.TreeNeuron)behave exactly as before, so downstream packages keep working until they get around to it. Neurons pickled by earlier versions still load.navis.TreeNeuronraises aDeprecationWarningonce per session; Python hides those by default outside__main__, so run withpython -W default::DeprecationWarning(orpytest -W default) to find the old names in your code.The
.typeproperty follows the class, so it now reads"navis.Skeleton"rather than"navis.TreeNeuron". Code matching on that string - including anything filtering aNeuronList.summary()table bytype- needs updating.
Neurons: one schema, masking and attached data#
What a neuron is made of is now declared in one place (navis/core/schema.py) instead of being re-implemented per type. Masking, attached data and a mesh keeping its skeleton all fall out of that.
New#
-
new
navis.masked: work on part of a neuron, then put it back.navis.subset_neuronis a one-way street - you get a fragment, and its relationship to the neuron it came from is gone. A mask keeps that relationship: inside the block the neuron is the masked region, so every function, property and plot sees only that part with no special-casing anywhere; on the way out it becomes whole again, with any edits folded back in. Masking happens in place and masks nest. Works onSkeletons,MeshesandDotprops, and on wholeNeuronLists; the methods underneath (Neuron.mask(),.unmask(),.apply_mask()) are there for when a mask has to outlive a block.Two new primitives underneath, usable on their own:
subset_neuron(..., track=True)records where each surviving element came from, andnavis.merge_subsetfolds an edited subset back in by joining on that record (and raises where the record can no longer be trusted, rather than guessing). Two caveats: merging re-points connectors, tags and the soma at whichever elements survived, so an edit touching none of the elements is silently dropped; and a mask that cuts across branches leaves nodes that look like the ends of the arbour but are not, which anything working from the tips (prune_twigs,strahler_index,.leafs) will act on. See the new masking tutorial. -
new
navis.cast_neuron: convert a neuron's data to a given dtype. What gets cast depends on the neuron type - nodex/y/z/radiusfor skeletons, vertices for meshes, points (and, for float dtypes, tangent vectors and alpha) forDotprops, voxel values forVoxels- plus connectors for all of them. Anything that indexes into those (mesh faces, voxel coordinates, node/parent IDs) is left alone. Handy for e.g. the float32/float64 NBLAST question below
Improvements#
-
a
Meshnow keeps its skeleton through a subset or a mask instead of throwing it away. The vertex-to-node map is declared as a link in the schema (navis/core/schema.py) - one array aligned to the mesh's vertices whose values name skeleton nodes - so a selection carries it the same way it carries anything else, and subsets the skeleton to the nodes that still have vertices:>>> nodes = mesh.skeleton.nodes.node_id.values >>> with navis.masked(mesh, lambda x: x.vertices[:, 0] > 0): ... np.isin(mesh.skeleton.nodes.node_id, nodes).all() # same nodes np.True_Previously the skeleton was a plain cache, dropped by any change to the vertices and re-derived from whatever was left - so it cost a full re-skeletonization (~10x the subset itself on the example neuron) and came back with different node IDs. Anything computed on a masked mesh's skeleton therefore could not be traced back to the whole neuron. Functions that route through the skeleton (
navis.prune_twigs,navis.prune_by_strahler,navis.split_axon_dendrite, ...) get this for free.A skeleton is kept only while NAVis can vouch for it: it is regenerated after any change to the vertices that did not go through the schema (assigning
.vertices, transforming the neuron,navis.merge_subset), and after any change to the skeleton that alters which nodes exist. Moving or rerooting the skeleton does not invalidate it, since links store node IDs. A skeleton attached by hand (mesh.skeleton = ...) behaves as before - there is no correspondence to carry, so it is dropped when the mesh changes.Links compose:
Neuron.get_mapping(source, target)walks the link graph, so a correspondence nobody declared directly is still available. Deliberately one-way - a vertex has one node but a node has many vertices - so useNeuron.select_acrossto go the other way. -
connectors are now elements in their own right, on the same footing as nodes and vertices: they have an axis of their own and reach the thing they sit on through a link rather than a bare reference. Three things follow.
Anything you align to them is carried with them,
get_mappingcomposes across them -connectors → vertices → nodesresolves on a mesh without anyone declaring the shortcut - and one shared declaration (schema.CONNECTOR_AXIS/schema.connector_link) now serves every neuron type instead of three near-copies. Your connector table is left exactly as you handed it over: the axis identifies connectors by position, so NAVis has no reason to write an id column into it. -
new
Neuron.attachandNeuron.attach_linkcarry your own data through selections. Anything attached is subset, filtered and re-indexed bysubset_neuron,maskedand everything built on them, exactly as the neuron's own tables are:>>> n.attach('compartment', labels, axis='vertices') # one per vertex >>> with navis.masked(n, lambda x: x.vertices[:, 0] > 0): ... n.compartment # masked to matchattachalso takes data that brings its own elements (n.attach('mito', table, ids='mito_id')declares an axis), andattach_linksays which elements of another axis that data names - so a mitochondria table can follow the nodes it sits on, and be dropped when they are:>>> n.attach('mito', table, ids='mito_id') >>> n.attach_link('nodes', 'mito', column='node_id', ... source='mito', target_axis='nodes', cascade='keep')Neuron.attached()lists what a neuron carries beyond its type - attached data is otherwise a plain attribute that nothing in the summary mentions - andNeuronList.attached()summarises it over a list, counting how many neurons carry each entry:>>> n.attached() name kind axis names shape 0 embedding aligned nodes (4465, 8) 1 mito axis mito (30, 2) 2 nodes->mito link nodes mito (4465,)Both write per-neuron the same declaration the neuron classes write per-class (
navis/core/schema.py);.connectorsis now itself a thin wrapper overattach. Note that assigning to an axis is not the same as selecting it - replacing.connectorswholesale drops anything aligned to the old connectors, with a warning, since nothing can say where they went. See the new attaching data tutorial, which also covers what no file format persists.Functions that rebuild the elements rather than select from them (
navis.resample_skeleton,navis.downsample_neuron,navis.stitch_skeletons,navis.simplify_mesh, ...) drop attached data with a warning rather than leaving it at the old length. Twoon_rebuildpolicies opt out of that where a rebuild can say enough:>>> n.attach('score', values, axis='nodes', on_rebuild='carry') >>> navis.downsample_neuron(n, 10).score # thinning keeps the nodes it keeps >>> navis.resample_skeleton(n, 1000).score # ... re-sampling does not: dropped AttributeErrorattach_link(..., on_rebuild='snap')is the same choice for something whose values name elements: it does not need a value invented for it, only somewhere to point, so it follows the rebuild to the nearest node of the same branch. That is what connectors, tags and the soma have always done throughresample_skeletonanddownsample_neuron, and it is now the generic path rather than three hand-written blocks - so anything you attach can ask for it too.A rebuild that merges rather than selects can now say so, which is what makes the above work for
navis.simplify_mesh. It reports where every old element went (schema.Rebuild(merged=...), one entry per old element) - several of them possibly to the same place. That is a weaker claim than "this new element is that old one" and a stronger one than "a reference to it should point here": no vertex survives decimation unchanged, but each new one stands for a known group of old ones, and the value of the first of that group is what it takes. So per-vertex data attached withon_rebuild='carry'now comes through simplification, and so - via the newLink(on_rebuild_aligned='carry')- does a mesh's vertex-to-node map: a simplified mesh keeps its skeleton and its node IDs instead of throwing them away and re-skeletonizing into a different set of nodes.
Fixes#
-
navis.subset_neuronhas been rewritten onto a declarative per-type schema (navis/core/schema.py) that says which attributes are aligned to which axis - a skeleton's nodes, a mesh's vertices, a dotprops' points - and what references them. The three hand-written implementations had drifted apart in exactly the places nobody was looking; there is now one, plus a test that fails if any bulk field is undeclared.navis.masked(above) is built on the same schema. Four bugs came out of it:subset_neuronmis-mapped aMesh's connectors: it built its old→new vertex index map from the vertex indices requested, butsubmeshreturns the survivors in sorted order and additionally drops vertices left in no face - so unless the request happened to be sorted and complete, surviving connectors came back attached to the wrong vertex, silentlysubset_neuronleft aDotprops' soma pointing at whatever point had moved into that slot. A dotprops soma is a point index, so a subset has to renumber it; it is now remapped, or dropped if the point did not survivesubset_neurondid not preserve the node table's column order - it movedparent_idto the end as a side effect of how it re-rooted the survivorsDotpropsnever took part in the staleness check: the class attribute naming its core data was misspelled_CORE_DATA, soDotprops.core_md5was alwaysNone
-
rebuilding a neuron left data attached to its old elements exactly where it was. The
.nodes,.verticesand.pointssetters predate the schema and write their private attribute directly, so anything you had attached came out ofnavis.resample_skeleton,navis.downsample_neuron,navis.stitch_skeletons,navis.simplify_meshand friends describing elements that no longer exist - at the old length, so it still indexed cleanly and nothing complained. Such data is now carried where the function can say which elements it kept, and dropped with a warning otherwise; seeon_rebuildabove. -
navis.insert_nodesandnavis.graph.clinic.merge_duplicate_nodeswrote the node table straight past everything that keeps a neuron consistent. Both assigned to the private_nodesrather than going through the setter, so neither the attached-data handling nor the reference repair got a say. Anything attached to the nodes was left at the old length -insert_nodeson the example neuron gave back 4468 nodes and 4465 labels, and the misalignment survived every later operation - andmerge_duplicate_nodesleft connectors sitting on the duplicate it had just folded away. Inserting is a rebuild and now says so (nothing aligned to the nodes can come along: there is no value to give a node that was not there before, so it is dropped with a warning); merging duplicates is a selection and now goes through it, so attached data is carried - and because a folded duplicate is the very same point in space as the node it went into, a connector, tag or soma sitting on it is moved there rather than dropped along with it. A test now fails if a new direct write to_nodes/_vertices/_pointsappears outside the handful that are allowed -
memory_usagedid not count data held inside something else, and cached a number that changes could not clear. It walked the neuron's own attributes for arrays and tables, so a mesh's skeleton, the snapshot a masked neuron holds to be restored from and the arrays provenance keeps per axis were all free - on the example mesh, 239 KB of skeleton reported as 0. Attaching data or masking then did not invalidate the cached total either (masking runs with the neuron locked, which is exactly when_clear_temp_attrdeclines to act), so the size could be reported from before the change. Both fixed; caches built by other libraries (trimesh, igraph, networkx) are still not counted, being temporary and rebuilt on demand. -
memory_usage(estimate=True)raised on any neuron with connectors, andNeuronListreported0.0Bbecause of it. The estimating path prices each column from its dtype usingdtype.itemsize- which pandas' extension dtypes do not all have, and from pandas 3 a text column defaults toStringDtype.NeuronList.memory_usagecaught everything and returned 0, which is why this surfaced only as aNeuronListclaiming to be0.0Bin its own repr. Columns that can be sized from their dtype still are; the rest are now costed by pandas itself, and estimates are exact on the example neurons.NeuronList.memory_usagenow raises instead of returning 0 when it cannot size the neurons - printing aNeuronListstill can't fail (it shows?), but a returned0was indistinguishable from a genuinely empty list
Skeletons: morphology and graph#
Breaking#
-
navis.smooth_skeletonruns on navis-fastcore, and its window is now centred. It used to take a trailing mean (pandas'rolling(window).mean()) along each segment, which lagged the smoothed neurite half a window towards the segment's distal end. It also let branch points move - a branch point is the last node of its segment, so it took a full one-sided mean that the parent segment then read back, dragging the branch's three neurites apart. Both are fixed: the window is centred and shrinks symmetrically as it approaches a segment's ends, and roots, branch points and leafs are pinned. Coordinates change - on the example neuron by a median of 54 nm - so this is not a drop-in match for saved output. It is also ~200x faster (351 ms 1.6 ms for a 4.5k-node skeleton).An even
windownow rounds down to the odd value below, since a centred window can only hold an odd number of nodes. -
new:
navis.smooth_skeletontakes asigmain place ofwindowto smooth with a Gaussian kernel whose width is a distance along the neurite rather than a count of nodes - so the amount of smoothing does not change when the skeleton is resampled, which is usually what you want. The two are mutually exclusive; passing both raises.to_smoothworks as before with either, and note thatsigma's kernel is always measured over the x/y/z coordinates whatever is being smoothed, since a radius is a value and not a geometrysk_smoothed = navis.smooth_skeleton(sk, sigma=2000) # these neurons are in nm -
navis.find_somanow returns a single node ID (orNone) instead of an array of candidates. It used to hand back every node passing the radius/label filter and leave the choice to the caller, which meant a thick primary neurite could be returned alongside - or instead of - the actual soma. Candidates are now scored by the mean radius of their neighbourhood (withindist_factortimes their own radius, new argument) so that the fattest region wins rather than the fattest single node, and the fattest node of that region is returned; the label-only path takes the most central node of the largest connected label component. Nodes whose radius is missing (NaNor<= 0, asguess_radiuswrites) are no longer treated as candidates. Code doingfind_soma(n)[0]orlen(find_soma(n))needs updating -
skeleton graph functions, where several results change - some genuinely, some merely from arbitrary to deterministic:
-
navis.betweenness_centrality(from_=...)now counts sources within one hop of a root. That branch never computed betweenness: it walked root→source paths and tallied every node but the source, which is simply "how many offrom_lie below this node". It additionally discarded paths of two nodes or fewer, so a source sitting on or next to a root contributed nothing. Those sources now count like any other. In practice only the root and its immediate children move - on the example neuron exactly one node changes, by 3.navis.find_main_branchpoint(the one caller) is unaffected, since roots are never branch points -
navis.collapse_nodesno longer re-roots the neuron at the collapsed node. That was a side effect of how it rewired, not a documented behaviour; roots are now left where they were. It also no longer raises on real node IDs - see Fixes -
navis.rewire_skeletonroots leftover components at their lowest node ID rather than at whateverset.pop()returned. Same edge set, same tree; the docstring already promised nothing better than "arbitrary" for those roots, but the choice is now deterministic -
navis.graph.node_label_sortingbreaks ties differently. Where two branches have exactly equal sort keys their order now follows the node table, deterministically; it used to follow the edge-insertion order of the networkx graph underneath, which carried no stability guarantee at all. On the example neuron 14 of 1217 positions move; the node set and the keys themselves are unchanged. This feedsskeleton_adjacency_matrix(sort=True) -
navis.betweenness_centralityis now the only spelling of the function. The oldnavis.betweeness_centralityis gone
-
-
navis.prune_by_strahler'srelocate_connectorsparameter is gone. It walked from each pruned node up the parent chain until it hit a survivor and re-attached the connector there - which, since pruning takes whole branches away, could park a synapse a long way from where it actually was. Pruning removes parts of the neuron, and connectors on the removed parts now simply go with them. This does not change any default behaviour: the parameter defaulted toFalse, so unless you passed it explicitly you already had this. Passing it now raisesTypeError -
navis.downsample_neuronthins skeletons in fastcore now, and picks slightly different nodes. The Python walk that used to do it counted rootwards from every fix point; fastcore counts from each segment's distal end. Roots, branch points and leafs still always survive andpreserve_nodesis unchanged, so the result is the same neuron either way - but the surviving node set differs, by about 7% more nodes atdownsampling_factor=5on the example neuron. Anything pinned to an exact node set needs re-baselining. It is also ~2.3x faster (3.8ms -> 1.7ms on the example neuron) and needsnavis-fastcore >= 0.11.0.Where connectors, tags and the soma end up is unchanged in substance - fastcore hands back a complete map of where every dropped node's data goes, and on the example neuron it agrees with the old geodesic search on 99.7% of dropped connector nodes. The rest are cases where the two surviving nodes are exactly equidistant, which the old code resolved arbitrarily and fastcore now resolves towards the root.
-
navis.downsample_neuron'smethodargument now applies to every type of neuron, and an inapplicable one is an error. It used to be read only byDotpropsand silently ignored by everything else. That was harmless while it named a way of picking points; it is not harmless now that it also names a way of picking nodes, since quietly ignoringmethod="rdp"on a mesh would hand back something simplified by face count and call it shape-aware.navis.downsample_neuron(mesh, 5, method="uniform")now raises aValueErrornaming the methods that type does understand.
New#
-
navis.downsample_neuroncan thin skeletons by shape rather than by counting, viamethod="rdp"(Ramer-Douglas-Peucker) andmethod="vw"(Visvalingam-Whyatt), both from navis-fastcore. RDP drops a node unless removing it would move the traced path by more than the tolerance, so long straight stretches collapse to their two ends while a tight curve keeps every node it needs; Visvalingam-Whyatt repeatedly removes whichever node adds least area, which sheds detail more evenly under aggressive simplification - RDP will happily keep one spike and flatten everything around it. Both spend the same node budget where the neuron actually curves, which buys a much better skeleton per node than a fixed factor does: on the example neuronmethod="rdp"at half a micron keeps 1362 nodes againstdownsampling_factor=5's 1564, for a closer fit.For these,
downsampling_factoris read as a distance tolerance in the neuron's own units rather than as a factor - roughly "how far the simplified neuron may stray from this one" - so it takes a unit string (navis.downsample_neuron(n, "1 micron", method="rdp")) and has no lower bound. Note this includesmethod="vw", whose underlying threshold is an area: navis squares the tolerance for it so thatmethodcan be swapped without also rescaling the number.Worth knowing for both, and for
method="simple": downsampling a skeleton shortens it. Survivors keep their coordinates, so the edges replacing a dropped chain cut its corners and.cable_lengthfalls with them - by 5-6% on the example neuron at the settings above. Usenavis.resample_skeletonif you need the cable length left intact. -
new functions for analyzing the angles in a skeleton (see the new section in the morphometrics tutorial):
navis.branch_angles(between child branches at each branch point),navis.path_angles(in- vs outgoing edge at each continuation node, i.e. how much the path bends),navis.root_angles(how far each edge deviates from pointing radially away from the root) andnavis.soma_exit_angles(between the neurites emanating from the soma). All return a tidy per-node DataFrame, work onMeshesvia their skeleton, and map overNeuronLists -
new
navis.graph.geodesic_clusters(): greedily partitions a skeleton or mesh into connected clusters of bounded geodesic radius. Please read its warning before using it for downsampling - the clusters are guaranteed connected and bounded, but they are not evenly sized and their centroids are not evenly spaced
Improvements#
-
the graph internals now run on navis-fastcore instead of igraph/networkx. Each of these was a general graph algorithm answering a question about a rooted forest, where the answer is a linear pass over the parent vector - so building a graph object cost more than the answer did. Measured on the example neuron (4465 nodes):
Function Before After betweenness_centrality(directed=True)11.1 ms 0.48 ms betweenness_centrality(directed=False)217 ms 0.49 ms betweenness_centrality(from_=...)10.6 ms 0.83 ms find_main_branchpoint("longest_neurite")2.8 ms 0.40 ms find_main_branchpoint("betweenness")9.3 ms 1.4 ms split_neurites(n=5)20.1 ms 7.6 ms reroot_skeleton2.3 ms 0.75 ms cut_skeleton7.4 ms 3.6 ms collapse_nodes10.8 ms 1.9 ms rewire_skeleton9.5 ms 2.9 ms edges2neuron(validate=True)3.5 ms 0.60 ms cell_body_fiber12.3 ms 3.0 ms node_label_sorting14.4 ms 3.5 ms Skeleton.is_acyclic2.3 ms 0.65 ms betweenness_centralitygains the most because shortest paths in a tree are unique, so betweenness has a closed form (descendants × ancestors) and needs neither Brandes nor a graph. Values are unchanged - bit-identical to igraph's, counted inint64since an undirected 100k-node skeleton reaches ~5e9.
Fixes#
-
navis.graph.collapse_nodesnever updated the vertex map it meant to. It guarded onhasattr(x, "_vertex_map"), but the map was stored under the public name, so the branch had never once run and a collapsed skeleton was left with vertices pointing at nodes it no longer had.Skeleton.vertex_mapis now a validated property backed by_vertex_mapand the branch fires (and no longer tries to write into skeletor's read-only array) -
navis.downsample_neuronleft connectors and tags pointing at nodes it had just deleted. It thinned the node table and never touched either, so on the example neuron adownsampling_factor=10produced a skeleton whose 2705 connectors included 1704 referring to node IDs no longer in the table. Both are now moved onto the geodesically nearest surviving node, which is the same stretch of the same branch. Passpreserve_nodes="connectors"to pin connectors exactly instead of letting them move -
navis.prune_by_strahlerleft tags and the soma pointing at nodes it had just pruned - the same class of bug. Tags now lose their pruned nodes (and go away entirely if that empties them) and a soma sitting on a pruned node is cleared -
navis.collapse_nodesraisedMemoryErroron real node IDs. It built an igraph contraction mapping of vertex indices but wrote node IDs into it, so it only held together while IDs happened to run1..N. Given the 7e17-range IDs segmentation backends hand out, igraph tried to reserve a vector sized by the ID and died. Everything now happens in ID space -
navis.geodesic_matrix(directed=True)reported a coincident child as reachable from its parent, at distance 0, whenever bothfrom_andto_were given. A navis-fastcore fix (0.10.0) that NAVis inherits: the partial backend used depth as a proxy for ancestry, which only holds while every edge weight is strictly positive. Coincident nodes are routine in traced and resampled skeletons -
weighted segment lengths were one edge too long.
navis.graph.graph_utils._generate_segments(..., return_lengths=True)summed the weight of every node in a segment including the terminal one, whose own child→parent edge belongs to the parent segment. Another navis-fastcore fix NAVis inherits: lengths now measure first node to last, so they sum to exactly the neuron's cable length.segment_analysis,.segmentsandpersistence_pointsare unaffected - the over-count never reached them -
navis.sholl_analysiswas broken for most of itscenteroptions - including the default. The"centermass"branch reboundcenterfrom a preset name to an x/y/z array before the remaining branches compared it against"soma"/"root", and on numpy >= 1.25array == "soma"is an elementwise comparison, so the nextifraised. Separately, a node ID given as a numpy integer (e.g.center=n.root[0]) failed theisinstance(center, int)check, skipped the node → coordinate lookup and was broadcast as a scalar into the distance computation - returning wrong numbers without raising.centeris now resolved by type before any string comparison. Also fixed in passing:geodesic=True, center="soma"raisedIndexErrorandradiidid not accept numpy integers. Note thatgeodesic=Truenow requires a center that lies on the arbor ("root","soma"or a node ID) and raises for the default"centermass"
Meshes#
Breaking#
-
navis.simplify_meshruns on navis-fastcore and the other four backends are gone.pyfqmr,open3d,pymeshlaband Blender 3D all did the same job, needed installing separately, disagreed about whatFmeant, and - the point - none of them could say what had become of a vertex. fastcore's implementation is the same Garland-Heckbert quadric decimationpyfqmrruns, and it returns a vertex map: for each old vertex, the new one it was merged into. That is what everything below is built on.fastcore is a hard requirement, so there is nothing to install and no backend to choose;
pyfqmrhas been dropped from themeshesextra. Thebackendargument is deprecated and ignored (passing anything raises aDeprecationWarning), andnavis.meshes.simplify_mesh_fqmr,simplify_mesh_open3d,simplify_mesh_pymlandsimplify_mesh_blenderare gone.**kwargsnow go tonavis_fastcore.simplify_mesh-aggressiveness,preserve_borderandlock, the last of which pins vertices so they are never merged away or moved.One thing found along the way and worth reporting on its own:
navis.smooth_meshwithbackend="auto"was silently doing nothing on any machine withpyfqmrinstalled, sincepyfqmrcame first in the shared backend list and smoothing has nopyfqmrbranch. Smoothing no longer has backends at all - see below - so the list is gone with them, andnavis.meshes.available_backendswith it. -
navis.smooth_meshruns on fastcore too, and its default filter has changed.open3d, Blender 3D andtrimeshare gone the same waysimplify_mesh's backends went, andbackendis likewise deprecated and ignored. Smoothing moves vertices and replaces none of them, so the faces, the vertex count and the vertex order all come back untouched - connectors, extra edges, the skeleton correspondence and anything you attached yourself stay attached to the vertex they were attached to, and unlike simplification there is nothing to repair afterwards.The default is now Taubin's λ|μ filter rather than the plain Laplacian, which is the part to know about. The Laplacian step removes high frequencies quickly and low ones slowly, and a closed surface's enclosed volume is a low frequency: at the settings NAVis ships it costs the example neuron 55% of its volume, and a thin neurite proportionally more.
trimeshpapered over that with a rescale (itsvolume_constraint, on by default) anchored at the origin rather than at the mesh, which is not a shape operation - it displaced the example neuron's centroid by 1,963 units, on a neuron 24,442 units across. Taubin alternates a shrinking λ pass with an inflating μ pass tuned so the two cancel below a cut-off frequency, needs no such correction, and holds the same neuron to within 13% while moving its centroid by 0.06 units.method="laplacian"andmethod="humphrey"(the HC filter of Vollmer et al.) are there when you want them, andvolume_correction=Truegets the Laplacian back to 99.6% - this time scaled about the mesh's own centroid.Lis deprecated in favour oflamb: that is whatnavis_fastcore.smooth_meshcalls it, and it leaves room for themuit pairs with. The remaining**kwargsgo straight there -mu,alpha/beta,weights,preserve_border,lock,volume_correctionandthreads. Two worth knowing about:weights="cotangent"is the discrete Laplace-Beltrami operator, which moves vertices along the surface normal instead of sliding them around within the surface, and is usually what you want on meshes out of EM segmentation; andlockpins vertices bitwise, the same argumentsimplify_meshhas.preserve_borderdefaults toTruehere rather than fastcore'sFalse, for the same reason it does insimplify_mesh- NAVis' meshes are routinely fragments cut out of a larger volume, and a boundary vertex's one-ring lies entirely to one side of it, so without this every iteration rolls the cut face a little further inwards.It is also, incidentally, ~50x faster: ten iterations on a 434k-vertex mesh take 2.26 s through
trimeshand 0.04 s here.
New#
-
new
navis.find_soma_mesh: soma detection forMeshes, straight off the mesh. No skeletonization involved: it finds the thickest part of the neuron (the point of largest inscribed sphere) and fits an oriented ellipsoid to the surrounding surface, returning the newnavis.SomaEllipsoid-center,radii, principalaxes,inscribed_radius, plusvolume,equiv_radius,contains()anddistance_to_surface(). Withinplace=Trueit simply sets the neuron's.soma_pos. The approach is inspired by skeliner;min_soma_radius(accepts e.g."1 micron"if the neuron has units) is the main accept/reject knob and should be tuned to your data -
new
navis.heal_mesh: the mesh counterpart tonavis.heal_skeleton. Meshes often consist of several disconnected fragments - because the segmentation had a gap, or because meshing produced separate closed surfaces where the neuron is continuous.heal_meshreconnects them with the set of bridges that minimises the total added length (a true minimum spanning tree over the fragments), subject to the samemax_dist,min_size,maskandkeep_largestknobs as its skeleton sibling. The repair is purely topological: bridges land in.extra_edges(see below) so vertices, faces, area and volume are all untouched. A 100k-vertex mesh takes ~15 ms -
new
Mesh.extra_edges: connectivity that the surface itself does not have. A mesh's topology is implied by its faces, so there is no way to express "these two vertices are connected" without inventing geometry. Extra edges are exactly that: an(N, 2)array of vertex indices that is part of the graph but not of the surface. Everything that derives connectivity from a mesh now sees them -geodesic_matrix,split_components,drop_fluff,.igraph/.graph- while anything describing the surface (e.g..sampling_resolution) does not. They are remapped throughsubset_neuronandcombine_neurons, and dropped whenever the number of vertices changes. Note that mesh file formats have no place for them: they are lost on export
Improvements#
-
navis.drop_fluff(mesh, connectivity=...): a piece that only just touches the rest is now fluff you can drop. Connected components of a mesh have always been components of the vertex graph, where a face joins its three corners - so two blobs pinched together at one vertex are one piece, however little that vertex holds them together. There are now two finer readings to pick from, each dropping a kind of junction:"face"joins two faces wherever they share an edge, which drops the pinch points;"manifold"joins them only across an edge carrying exactly two faces, which also drops the seams, and reproducestrimesh.split(only_watertight=False)exactly - the pieces that are surfaces, each with a well-defined inside. The example mesh has 14 pieces under the default, 24 under"face"and 502 under"manifold"; dropping everything but the largest leaves 17058, 16978 and 15039 vertices respectively. Sizes are counted in vertices whichever you pick, and a vertex that several pieces pinch together stays as long as one of them does.Under the hood this is navis-fastcore 0.13's
mesh_connected_components(..., connectivity=...);graph_utils._connected_componentsand_mesh_component_labelstake the same argument, the latter returning one label per face for the two face readings - a pinch vertex belongs to several face components at once, so there is no per-vertex form of that answer. One wrinkle worth knowing if you useheal_mesh: an extra edge is a bridge between two vertices, so under a face reading it joins the faces at one end to those at the other - and where its endpoint is itself a junction, the pieces meeting there get welded to each other too. That is not a choice so much as arithmetic: they cannot stay apart while both connect to the far end -
new
navis.fill_holescloses the holes in aMesh- the openings it was cut with, and any it came with. Cutting a mesh (vianavis.prune_twigs,navis.prune_by_strahler,navis.subset_neuron, ...) drops every face that loses a corner, which used to leave each severed twig standing open.fill_holestriangulates those cross-sections shut:>>> pruned = navis.prune_twigs(mesh, min_length='5 microns') >>> filled = navis.fill_holes(pruned) # or pruned.fill_holes()Openings are ear-clipped in their own plane rather than filled with a triangle fan, which matters because roughly a third of the cross-sections a prune leaves behind are not convex - a fan spills outside them. This is navis-fastcore
>= 0.11.0's work, so there is nothing extra to install. No vertices are ever added, only faces, so vertex indices - and with them connectors,extra_edgesand any tracked provenance - keep meaning what they meant.navis.subset_neuronalso takes acap_holes=True, which closes only the openings that call itself made and leaves pre-existing ones alone. It is the cheaper of the two - it only inspects the collar of faces around the cut, wherefill_holeshas to group the edges of the whole mesh (3.8 ms against 25 ms on a 2.2M-face mesh) - but it is off by default, so nothing changes for existing code unless you ask. At that price capping adds about 2% to the subset it follows. -
navis.fix_mesh(fill_holes=True)fills holes now. It calledtrimesh.Trimesh.fill_holes, which - in every version NAVis supports, 3.8 through 4.12 - closes single triangle and quad holes and skips everything larger without a word. On a 1M-face mesh cut in half that is 22 of 1030 openings; on the example neuron pruned at 5 µm it is 27% of the cap faces needed. It now goes through the same machinery asfill_holesabove, which is both complete and ~30x faster (278 ms 9 ms on that mesh).Volume.validategets the same treatment, via a newVolume.fill_holesthat overrides the trimesh method of that name.Two consequences worth knowing. Outputs move: a mesh that came back open now comes back closed, so volumes drop and
is_watertightflips. And the fill now runs after the merge/dedupe/degenerate step rather than before it, because a boundary is a topological thing: an unmerged duplicate vertex invents openings that are not there, and a degenerate face sitting on a boundary edge hides one that is. What is left open afterwards is the non-manifold junctions the mesh was already broken at, which no amount of capping can close. -
a mesh's unique edges now come from navis-fastcore (new
navis.utils.mesh_unique_edges) instead oftrimesh.edges_unique, which sorts an(n_faces * 3, 2)array to find them. This sits underneathneuron2nx/neuron2igraphforMeshesand hence everything built on a mesh graph. The results are seeded into trimesh's own cache, so a mesh that has already computed its edges pays nothing
Fixes#
-
navis.simplify_meshleft connectors pointing at vertices it had just deleted. Decimation replaces a mesh's vertices wholesale, so on the example neuron simplifying to 20% left 83% of the connectors naming a vertex index that no longer existed - and the rest naming whatever had come to sit at that index. Connectors now follow the vertex they named to whatever it was merged into (see the rewrite under Breaking). Extra edges survive it too, where they used to be dropped outright with a warning: a bridge names a place on the surface just like a connector does, so it follows its two endpoints (13 fragment-bridging edges on the example neuron come out as 12 - the one that goes is the one whose ends decimation merged into a single vertex, which is no longer an edge at all). -
navis.fix_meshraised anAttributeErrorontrimesh >= 4.10.Trimesh.remove_duplicate_faces/.remove_degenerate_faceswere replaced by.unique_faces()/.nondegenerate_faces()in trimesh 3.23 and finally removed in 4.10;fix_meshnow picks the right pair based on the installed version. This also unbrokeMesh.validate()andMesh(..., validate=True), which route through it -
Mesh(..., validate=True)silently did nothing whenprocess=False: it fixed a copy of the mesh and threw it away
Voxels#
Breaking#
-
Voxelsnow avoid the dense grid wherever possible, which requires sparse-cubes>= 0.5.0- now a core dependency, pulled in assparse-cubes[skeleton]so that its dijkstra3d-sparse accelerator comes along too (skeletonization falls back toscipywithout it, but TEASAR is ~11x slower at 100k voxels and the gap widens with size). Materialising a grid larger thannavis.config.max_grid_size(4 GiB) now raises aMemoryErrorinstead of being silently OOM-killed - a neuron's grid is sized by its bounding box, so a handful of far-apart voxels can imply terabytes. Raise or disable the limit if you hit it on data you know fits -
navis.smooth_voxelstreats everything outside the neuron as empty (scipy'smode="constant"). It previously used scipy's default, which reflects at the canvas boundary and invents signal outside the imaged volume; results change for neurons touching that edge
New#
-
Voxelsgained a proper toolkit, all of it working straight off the sparse voxels:- morphology and set algebra:
dilate,erode,opening,closing,thin,fill_cavities,union,intersection,difference,symmetric_difference. Per-voxel values are carried through; set operations align neurons onto a common lattice and refuse to combine ones that do not line up - measurements:
surface_area,centroid,distance_transform,connected_components,iou,dice,grid_nbytes/voxels_nbytes - shorthands
.mesh()and.skeletonize()
- morphology and set algebra:
-
navis.skeletonizenow acceptsVoxels(via the newnavis.conversion.voxels2skeleton), closing a gap its own docstring used to flag. Defaults tomethod="wavefront"- ~4x faster than"teasar"and radii come free from the ring contraction rather than being snapped to the voxel lattice;"teasar"and"thin"remain available -
existing functions stopped densifying.
navis.drop_fluffandgraph_utils._connected_componentsnow handleVoxels;navis.smooth_voxels,navis.thin_voxelsandnavis.downsample_neuronno longer allocate the grid (the latter could trip the new memory cap on exactly the sparse neurons worth downsampling). Voxel adjacency - behindneuron2nx/neuron2igraph- is ~100x faster and no longer needs the undeclared scikit-learn dependency.smooth_voxels/thin_voxelskeep abackendargument if you want the old scipy/scikit-image route
Fixes#
-
-
a batch of
Voxelsbugs, most of them on the sparse (voxels + values) backing, which until now was barely exercised - values and coordinates were free to drift apart.threshold()filtered the coordinates but not the values;normalize()scaled the coordinates instead of the values, corrupting the geometry outright; the documented(N, 4)constructor input silently discarded its value column; and changing.valuesdid not invalidate a cached.grid. Also fixed:convert_units()resized the neuron instead of re-labelling it (125x too small for 8 nm → µm),.volumesquared the z voxel size and dropped y,.densitycrashed on numpy 2,copy.deepcopy()raised aTypeError,flip()moved the neuron and mirrored connectors in the wrong space, and.bboxdisagreed between the two backings by one voxel -
Voxelswith no filled voxels raisedValueError: zero-size array to reductionon.shape- and hence on.grid,.bbox,repr()andsummary(). Empty neurons are not exotic - an all-zero grid auto-sparsifies to nothing..shapenow falls back to the canvas the neuron was left on, andstrip()/normalize()no-op instead of raising -
assigning
.voxelsa different number of voxels left the old.valuesin place, so.nnz/.volumekept counting voxels that no longer existed and.gridraised a broadcasting error. Mismatched values are now dropped (with a warning); values that still line up row for row are kept. Latent in1.12.0but easy to hit now that grids auto-sparsify -
navis.meshraisedAttributeErroron the(N, 3)voxel arrays it documents (it tested.ndims, which numpy spells.ndim)
-
NBLAST and Dotprops#
Breaking#
-
NBLAST scores shift very slightly with navis-fastcore
>= 0.8.0, which now takes its internal coordinate precision from the dtype of the input rather than always widening to float64. NAVisDotpropsstorepoints/vectas float32, so NBLAST now runs on float32 coordinates - cutting peak memory on a large all-by-all by ~45%, at a cost of ~1e-5 on the scores. Nothing in NAVis changed; upgrading fastcore is enough to see it.The scoring maths itself is untouched (it still accumulates in float64) and this does not change which neurons match - on the example neurons the k-nearest-neighbour identities are unchanged. But it is enough to break a bit-for-bit comparison against previously saved scores: cast
.pointsand.vecttofloat64(seecast_neuronabove) if you need the old numbers exactly.
New#
-
new
navis.nblast_knn: theknearest neighbours of every neuron, without ever building the score matrix. An all-by-all is the wrong shape for a k-NN question at scale - 164k neurons is 2.7e10 pairs and a 107 GB matrix, when what is wanted from it is a 26 MB k-NN graph (typically to feed a UMAP embedding). This computes that graph directly: each neuron is reduced to a coarse voxel-occupancy signature, then_candidatesmost similar neurons per row are shortlisted from those signatures, and the exact NBLAST score is then computed for the shortlisted pairs only. Only the shortlisting is approximate - every returned score is a real NBLAST score. Measured on 163,976 neurons, recall@20 is 0.990 at the defaultn_candidates, having scored 0.16% of pairs. Returns a tidyquery/target/score/rankframe by default;format="wide"gives theextract_matcheslayout andformat="arrays"the raw arrays UMAP'sprecomputed_knnwants. Unlike the other NBLAST functions this one is provided only by navis-fastcore -
navis.make_dotpropsis ~12x faster: the tangent vectors and alpha values (96% of its runtime) now come from one parallel Rust pass instead of ascipy.spatial.cKDTreequery plus N 3x3 SVDs. Same fornavis.Dotprops.recalculate_tangents. The two agree exactly except where the k-nearest-neighbour search hits a tied distance, which grid-quantised coordinates produce readily - there the k-th neighbour is genuinely ambiguous and the two trees may pick differently (~0.3% of points on the example neurons) -
navis.nbl.extract_matchesis much faster and gained amax_matchesguard. All three criteria now go through navis-fastcore:Nis 6-118x faster (the gap widens with matrix size - 2.8 s to 23 ms on a 20k x 20k matrix),threshold1.5-8x,percentage1.3-1.9x.max_matchesrefuses to return more than a given number of matches forthreshold/percentage, whose output size is not knowable in advance - an over-broad cutoff on a large matrix could previously take the machine down. The count is established before anything is allocated -
NBLAST runs on the same backends, so a big one can go to a cluster. The built-in NBLAST backend used to build its own
ProcessPoolExecutor, which meantnavis.set_parallel_backendhad no effect on it; it now dispatches through the same layer as everything else. The unit of work is a block of the score matrix rather than a neuron, sized from a per-block runtime budget so that each is seconds to minutes of work no matter how many neurons you have. Scores are unchanged - bit-identical, on every backend. On a single machine, where the default is nowjoblibrather than a private pool, a repeated NBLAST is ~1.75x faster because the workers are no longer thrown away and rebuilt between calls (150 neurons all-by-all on 8 cores: 6.3s 3.6s); they stay resident for a minute or so afterwards andnavis.compute.shutdown()reclaims them at once.navis-fastcore does not distribute
The navis-fastcore NBLAST backend computes the whole matrix in one Rust call with its own threads, so it ignores the parallel backend entirely and runs everything locally. It is not the default (
navis.config.default_nblast_backendis"builtin"), but it is what"auto"picks where it is installed - so leave that alone, or passbackend="builtin", when you want a distributed NBLAST.
Improvements#
- NBLAST no longer pins its workers to a single thread. It used to force
OMP_NUM_THREADS=1in every worker, from a time when nothing else capped native threading and pykdtree's OpenMP would otherwise claim every core in every one of them. Dividing the machine (above) already prevents that, so the pin was leaving NBLAST at a fraction of the cores it had been asked for - 3 of 14 atn_cores=3, against 12 now. Scores are unchanged. Applies tonavis.nblastand relatives on thebuiltinbackend, and tonavis.nblast_align.
Fixes#
-
NBLAST and
Dotprops:-
navis.nblastwithscores="both"crashed on every multi-core run. With"both", each query occupies two rows of the result, but the code that reassembles the score matrix from its blocks assumed one row per query. Any NBLAST split into more than one block therefore died withValueError: setting an array element with a sequence; only runs that happened to fit in a single block ever worked. While there:navis.nblast_smartandnavis.synblastnow rejectscores="both"rather than accepting it and getting it wrong - neither ever implemented it -
navis.nbl.extract_matchesrankedNaNas the best possible score. numpy sortsNaNto the end, so for a query that had been scored against some targets but not others - which is whatnavis.nbl.update_scoresand any hand-assembled matrix produce - the unscored pair was returned as that query's top match, with aNaNscore.NaNs are now skipped; a query with fewer thanNvalid scores gets emptymatch_k/score_kfor the remainder. Thepercentagecriterion was worse off still: oneNaNanywhere in a row made that row's thresholdNaN, so the query got no matches at all -
extract_matches(..., percentage=...)listed matches worst first for distance matrices - the exact inverse of what it does for similarities, and of what theNcriterion does for either. Matches are now always best first -
navis.make_dotpropssilently produced wrong tangent vectors for point clouds containing duplicate coordinates. Points whoseknearest neighbours are all at distance zero are dropped, but the neighbour indices were then offset by a flatn_dropped- only correct if every duplicate happens to sit at the start of the array. Anywhere else the indices ran past the end or went negative, and because numpy reads negative indices from the back this raised nothing: it just computed each tangent from an unrelated neighbourhood. On a 40-point cloud with a 4-point duplicate block in the middle, 39 of the 40 surviving points came back with the wrong tangent -
navis.Dotprops.recalculate_tangentsreturnedNaNalpha values for points sitting on duplicate coordinates - it has no equivalent ofmake_dotprops' duplicate check and cannot drop points. ThoseNaNs then propagated into every NBLAST score the neuron took part in. Such points now getalpha=0and an arbitrary unit vector, matching navis-fastcore
-
Transforms and alignment#
Breaking#
-
transforms: how NAVis chooses a route between two template brains has been reworked, and the non-fastcore backends are on their way out:
-
bridging graph: edge weights now mean one thing, and lower always wins.
weightwas doing two jobs at once - it set the costnetworkxminimises when choosing a route, and it was also the tie-breaker between several registrations connecting the same two templates (where the old code took the highest-weight edge). The two uses want opposite things, so no weighting could satisfy both. Nowweightonly ever means "what this hop costs" and lower weight = more likely to be used everywhere; which transform serves a hop is decided separately, by the newprefer_forwardargument (see below).This changes ~29% of the bridging paths in
navis-flybrains(no routes gained or lost). Most of that is NAVis no longer inverting a registration when a purpose-built one for that direction was sitting right next to it - e.g.BANC→Cell07used to invertCell07_IS2.listand now simply usesIS2_Cell07.list, at the same path length. 360 fewer routes traverse any transform backwards. -
transforms now declare how expensive they are to invert, via
BaseTransform.inverse_weight_factor. It is1wherever the inverse is stored or exact (AffineTransform,H5transform,TPStransform),2forCMTKtransformand5forElastixTransform- both of which have to solve for the inverse numerically.register_transform'sweight_invnow defaults toweight * inverse_weight_factor; passing it explicitly still overrides that -
reciprocalis deprecated in favour ofinverse_weight(bridging_graph,find_bridging_path,find_all_bridging_paths) - one name for one knob. Its default also changed from0.5to1: NAVis no longer discounts inverse transforms across the board, since each transform now says for itself what inverting it costs. Passingreciprocalstill works but warns -
the non-fastcore transform backends are deprecated and will be removed in 3.0. The CMTK/elastix
"binary"backend (shelling out tostreamxform/transformix) and the"python"landmark backend (morphops/molesq) still work, but selecting either now emits aDeprecationWarning."auto"- the default - has always preferred fastcore, so this only affects code that asked for them by name. Image transforms (xform_image,to_dfield) still need CMTK and are not affected
-
-
alignment runs on rcpd instead of
pycpd, which makes it 20-28x faster and fixes four long-standing bugs.pycpdis the reference Python implementation of coherent point drift and has been unmaintained since 2021;rcpdis a Rust re-implementation of the same algorithms, and a new optional dependency (pip install navis[cpd]). On two ~4,500-node skeletons a rigid fit takes 0.3 s and +69 MB againstpycpd's 9.6 s and +1.4 GB, and a deformable one 2.4 s and +96 MB against 48 s and +2.0 GB. Asked for the same fit, the two agree on where the points land to ~1e-8 of a neuron's extent.[
navis.align.align_pairwise][] fits its whole grid in one batch, one registration per core, for every coherent point drift method - a 5 x 5 alignment of the example neurons takes 6.8 s wherepycpdwould need twenty registrations at ~10 s apiece. A single pair goes the other way and threads inside the fit, worth another ~8x on the align-one-neuron-onto-another case.method="rigid+deform"andmethod="pca"also work at all now: both used to raiseTypeError, as didnavis.nblast_align(align_method="pca").The registration objects you get back are different. [
navis.align.align_rigid][] returnsrcpd.RigidTransformand [navis.align.align_deform][]rcpd.DeformTransform, in place ofpycpd's registration objects. Both move further points with.apply(coords)and carry the residual they left behind as.rms(in input units) and.nrms(as a fraction of the target's own radius, and so comparable between pairs: two neurons of the same type land around 0.06-0.11, two unrelated ones around 0.2). A rigid one also has.scale,.rotation,.translation, a 4x4.matrixand an inverse,~reg.sample=no longer runs a moving-least-squares step over the points it did not fit on: both registrations come back as functions of position rather than as sets of moved points, so they apply to the rest exactly.Four behaviours changed, all of them fixes:
scale=Falseis honoured.pycpd.RigidRegistrationaccepts noscaleargument in any released version, so the value NAVis passed landed in**kwargsand a scale was fitted regardless.align_rigidalso gainsscale_bounds=(lower, upper)to hold a fitted scale inside a range you consider plausible- convergence is tested relatively, on the fractional change in the fitted variance, so the same neurons no longer converge differently in nanometres than in microns. This is what made
align_deformstop after ~3 iterations: NAVis normalised the neurons first, which put the variance belowpycpd's absolutetolerance=1e-3before the fit had done anything wis scale-free.pycpdwrites the uniform outlier density as a literal 1, which is only right for data spanning ~1 unit; on a neuron spanning 1e4 nm anyw > 0collapsed the fit. The escalating-wretry loop inalign_rigidis gone with it, along with theLinAlgErrorit worked around - the 3x3 SVDrcpduses cannot fail to converge, and collinear points complete the rotation basis rather than raisingbetais a fraction of the neuron, not a length, so it means the same thing whatever units the neurons are in. This is what used to pull dense regions into a ball:pycpd's defaultbeta=2, against neurons NAVis had normalised to ~1 unit, is a kernel several times wider than the whole neuron, so every point moved as one
align_deformexposesalpha,beta,w,tolerance,max_iterationsandnum_modesas arguments, and defaults tobeta=0.2wherercpddefaults to 1 - a neuron should be free to deform locally rather than as a block. Measured over the ten pairs of example neurons, mean nearest-neighbour distance to the target (as a fraction of its extent) is 0.0089 unaligned, 0.0085 after a rigid alignment and 0.0043 after a deformable one. The neuron normalisation NAVis used to apply before handing anything topycpdis gone - withbetaandwscale-free, nothing needs it.
New#
-
various improvements to transforms:
- new
navis.transforms.SimilarityTransform: a rotation, a translation and a single uniform scale factor fitted to matched landmarks - the least-squares optimal solution in closed form via SVD (Umeyama, 1991), with optional per-landmarkweights. Setscale=Falsefor a pure rigid transform. Unlike aTPStransformit has only 7 degrees of freedom and so will generally not map the landmarks onto each other exactly - which is the point when you want the landmarks to constrain a global transform rather than warp space to fit them - the landmark transforms now have a fastcore backend.
TPStransformandMovingLeastSquaresTransformuse navis-fastcore's Rust implementation; the deprecatedmorphops/molesqpath agrees with it to ~1e-13. Neither materialises the(n_points, n_landmarks)intermediate those libraries build, sobatch_sizestops mattering and peak memory no longer scales with the landmark count - which is what madeMovingLeastSquaresTransformimpractical at the landmark counts real registrations use (3400 landmarks needed ~23 GB at the default batch size). Transforming points is ~10-15x faster. Note thatTPStransformstill fits its spline withmorphopseven on the fastcore backend, because numpy's LAPACK-backed solve is faster there; only the point transform switches - CMTK and elastix point transforms can now run without the external binaries.
CMTKtransformandElastixTransformnow use navis-fastcore's in-process Rust implementation instead of shelling out tostreamxform/transformix: no CMTK, no elastix, no subprocess and no temporary files. Results match the binaries to ~1e-6 (including which points fail), andxform_brainis 4-30x faster end-to-end. Image transforms (xform_image,to_dfield) still require CMTK. Control this withnavis.config.default_transform_backend("auto"by default, i.e. fastcore) or per transform viabackend= - elastix transforms are now invertible when navis-fastcore is installed - something
transformixcannot do at all. Note the bridging graph does not use this by default: setnavis.config.elastix_invertible = Trueto let it (see the notes onElastixTransformfor why it is off) MovingLeastSquaresTransformgained a.matrix_affineproperty, analogous toTPStransform.matrix_affine: since moving least squares is a locally weighted affine transform, this returns the global (least-squares) affine that it converges to far away from the landmarks
- new
-
find_bridging_path,find_all_bridging_pathsandshortest_bridging_seqgained aprefer_forwardargument (defaultTrue): where two templates are connected by both a purpose-built registration and the inverse of its counterpart, use the purpose-built one - regardless of weight. Set it toFalseto have your graph's weights taken entirely at face value -
new
mirror_axisfor [navis.align.align_rigid][], for when the neurons may be of either handedness - aligning a left-hand-side neuron onto a right-hand-side one, say, without knowing in advance which is which. Given an axis ("x"/"y"/"z"or0/1/2) each neuron is fitted twice, as given and mirrored along that axis, and whichever fit left the smaller residual (RigidTransform.rms) is the one you get back. Both fits go into the same batch as everything else, so across aNeuronListthis costs twice the registrations but not twice the wall-clock. The reflection is folded into the returned transform where it won -np.linalg.det(reg.rotation) < 0says which neurons were flipped, and.apply(),.matrixand~all still hold - and aMeshthat gets mirrored has its face winding flipped with it, so it does not come back inside out.
Fixes#
TransformSequencewas registered as invertible even if it contained a transform that was not
Plotting#
Breaking#
-
the
vispybackend is gone. It was deprecated in1.7.0in favour of octarine, which does the same job on modern WGPU instead of OpenGL.plot3d(..., backend="vispy")now raises, as doesNAVIS_PLOT3D_BACKEND="vispy"; the terminal auto-pick is octarine plotly. This removes ~2.3k lines - 28% of the plotting package - and with them the secondViewerimplementation and thevispy-*install extras.Concretely,
navis.Viewerhas been removed: the viewer is nowoctarine.Viewer, whichnavis.plot3dreturns and whose documentation covers its methods.navis.get_viewer,navis.clear3d,navis.pop3dandnavis.close3dare unaffected - they were never vispy-specific.navis.utils.check_vispyis gone with no replacement, as are the vispy-onlynavis.plot3dargumentscombine,shading,shininessandname(titleis unaffected - it was always plotly-only).navis.matching.matching_pipelinehas been ported to octarine and behaves the same, except that the vispy-only "cycle through neurons by hiding them" mode is not available.
New#
-
new
navis.plot3d(..., snapshot=True): a rendered 3D scene on a matplotlib axes, in data coordinates. Betweennavis.plot2d, which draws real matplotlib artists but has no renderer behind it, andnavis.plot3d, which renders properly but hands back an interactive window, there was no way to get a figure you can annotate, arrange into a panel and save as PDF.snapshot=Truerenders the scene through octarine's offscreen canvas and returns(fig, ax)with the image already placed - and because the image extent is read back off the camera,axis in the neurons' own coordinates, soax.annotate("soma", soma_pos[[0, 2]])lands on the soma and distances are to scale.viewtakes the same("x", "-y")axis pairs asnavis.plot2d, so the two can be mixed in one figure; pass a camera state dict fromviewer.get_view()to reproduce a view you set up interactively, orviewer=to shoot an existing viewer. Only axis-aligned orthographic cameras can be expressed in data coordinates - for anything else the axes fall back to world units in the view plane. Also new alongside it:margin,bgcolor(the render is transparent by default),size/pixel_ratioand the usualax,figsizeanddpi. See the new 3D skeletons tutorial. -
new
navis.plot_collage: a few hundred neurons arranged on one page. A plate of every cell type in a region, a poster, a supplementary "here is the whole dataset" figure - all of it has meant one subplot per neuron and a lot of fiddling with limits, and a regular grid spends most of the paper on the empty space around each arbor.plot_collagedoes the arranging: it scales and moves copies of the neurons onto a page (page_size, A4 by default) and plots them, leaving the originals alone.layout="grid"puts one neuron per cell (cols,margin,uniform_scale,sort,drop_dangling);layout="dense"packs them at one common scale, so their relative sizes survive. Skeletons, meshes and a mix of the two.The dense layout is where the interesting knobs are. By default it packs bounding boxes - fast, but a box is reserved for one neuron whether or not it fills it.
occupancy=Truepacks the rasterised arbors instead, so a neuron may reach into another's empty space - even into the loop of another neuron - as long as no cable actually collides: on 200 central complex skeletons that draws each neuron 1.37x larger for the same page, close to twice the area. The box packing runs first either way and seeds the occupancy search, so it can only improve on it.backfill=takes a second set of neurons to drop into whatever gaps are left - at the scalexsettled on, without influencing it, and silently dropping those that find no room - andmask=confines the neurons to a shape, given either as a bool array covering the page or as a picture of the outline. The packing itself is navis-fastcore's (pack_rectangles/pack_masks), which is what makes searching for the scale affordable.Either renderer will draw it:
backend="matplotlib"gives vector paths you can edit afterwards,backend="octarine"renders the page offscreen vianavis.plot3d(snapshot=True)and places the image, which fixes the file size no matter how dense the page gets (160 neurons: 8.9 MB of SVG against 1.0 MB) and shades meshes the way the interactive viewer does. Both put the neurons in data coordinates, so a page looks the same whichever one drew it and overlays land in the same place. The third return value holds the placed copies: hand them back asplaced=to re-draw that exact page - in different colours, or with the other backend - without packing it again. See the new collage tutorial.
Improvements#
-
the matplotlib, plotly and k3d backends assemble their lines in one pass. Drawing a whole skeleton as a single line wants the segments in one array with the breaks marked, so each backend took the list of per-segment arrays
segments_to_coordshands back and concatenated it straight back together - using[[None] * 3]as the separator, which made the whole array object dtype and pushed the float conversion down to individual coordinates.navis-fastcore0.13 can pad the array as it builds it, andsegments_to_coordspasses that through asflat=True. On 100 example neurons (447k nodes, 61.8k segments) assembling the coordinates goes 120 ms 27 ms. The breaks areNaNrather thanNonenow; all three backends already read that as a gap, and plotly's payload shrinks as a side effect (146 KiB 101 KiB for one example neuron) because a float array serialises to base64 where an object array becomes a JSON list. Renders are pixel-identical.Per-node colours ride along on the same call, so they come back already in segment order instead of being matched up afterwards through a dictionary lookup per vertex. That was the slower half of both 3-D backends: for 100 neurons k3d's coloured path goes 672 ms 38 ms, and plotly's sheds everything except the
rgb(...)strings themselves, which is what it is now bound by.The paths that genuinely want per-segment arrays - depth sorting, per-edge colouring and
method="3d_complex"- are unchanged and still get the list. -
plot3d(..., backend="plotly", hover_id=True)no longer raises aTypeError. It built its hover labels withseg + [None]overnavis.Skeleton.segments, whose entries are numpy arrays rather than lists, so the addition was attempted element-wise againstNone. -
navis.plot2d(method="2d") has a new renderer, which between true surface rendering, shading and depth ordering closes much of the gap to a real 3d view:-
meshes are rendered as surfaces rather than as flat blobs. Faces pointing away from the viewer are dropped and the rest painted furthest-first, so a
MeshorVolumefinally occludes itself. Previously faces went down in whatever order the mesh stored them, which meant a dense arbour collapsed into a single silhouette and the last neuron drawn covered everything under it. There is no switch to turn this off - it is a correctness fix, and because half the polygons never reach the renderer it is also considerably faster (build / draw on the example neurons):Before After one mesh neuron (13k faces) 32 / 28 ms 4.8 / 12 ms five mesh neurons 241 / 107 ms 12 / 19 ms one 209k-face mesh 513 / 278 ms 30 / 28 ms A mesh in a single colour is now filled as one path instead of one polygon per face, which is where most of that speed-up comes from - and it fixes
alpha, which used to composite every triangle separately, so a fold in the surface came out darker than a flat stretch. -
mesh_shadelights meshes and volumes in 2d. It was a bool for the 3d methods only; withmethod="2d"it now takes a mode -True/"lambert","cel","rim"or"ghost"(opacity from the grazing angle instead of brightness, so a neuron inside a neuropil stays visible). Pass a dict to tunelight,ambientandstrength. Shading multiplies into whatever colour a face already has, so it composes withcolor,color_byanddepth_coloringrather than replacing them.style="publication"turns it on - passmesh_shade=Falsealongside it to keep the flat fill. See the new mesh plotting tutorial -
new
depth_sortandhaloarguments, which between them stand in for the depth buffer matplotlib does not have.depth_sort=True/<int>buckets skeleton nodes and mesh faces into that many bins along the axis pointing into the screen and gives each bin its own z-order, so the two kinds of neuron interleave in one stack instead of every mesh landing on top of every skeleton; a negative number flips which end counts as nearest.depth_sort="global"skips the bins and sorts exactly - segment by segment, face by face - which for skeletons is cheaper than the default 10 bins but for meshes costs several times either, since sorting across neurons forces one polygon per face and gives up the fill-oncealpha.halodraws each neuron with a background-coloured outline underneath so that crossings read as one neurite passing in front of another. The two compose everywhere except on the"global"path - a halo has to sit between two neurons of a type, which a single merged artist cannot express, so passing both there warns and falls back to bins -
volume_outlinesdraws its contour opaque. A volume's alpha is a fill alpha - there is nothing to see through a line, so at the 10-20% volumes default to the contour was all but invisible.volume_outlines="both"already did this;Truenow matches it -
Volume.to_2dwarns when the alpha shape falls apart and it retries at a tenth of the alpha. The retry is silent no longer: it is why a too-largevolume_outlines_alphalooks like it did nothing rather than like it was rolled back
-
-
connector plotting has been overhauled:
-
connectors are drawn in one artist per neuron, in a fixed shuffled order. They used to go down one
typeat a time, which meant the type painted last won wherever markers overlap - so a rare type could bury a common one. On the example neuron that is not hypothetical: 232 presynapses sat on top of 1933 postsynapses and the antennal lobe read as an output region. Interleaving the draw order makes the visible mix a fair sample of the real one; the permutation is seeded, so the same data still gives the same figure -
cn_color_bycolours connectors by any column of the connector table (or by an array with one value per connector) instead of by theirtype- ROI, confidence, partner id, whatever the table carries. Numerical data gets a colormap, categorical data one colour per level, andcn_palettepicks the palette. The scale is worked out across all neurons in the call, so the same value is the same colour throughout; missing values are drawn grey. Supported bynavis.plot2dand byplot3d's plotly and k3d backends - the latter two draw markers rather than stalks when it is set, since a line there carries a single colour -
cn_legend=Trueexplains the connector colours - one entry per connector type for the whole axes rather than one per type per neuron, or a colorbar whencn_color_byis numerical.navis.plot2donly; callax.legend()afterwards as usual -
navis.plot2dnow honourscn_layout={"display": "lines"}, which draws a stalk from each connector back to the node it belongs to instead of a free-floating marker. That has always been the default innavis.config.default_connector_colorsand has always been what plotly and k3d drew;matplotlibsilently drew markers regardless. Passcn_layout={"display": "circles"}for the old look. Meshes have no nodes to point at and still get markers -
there is a connectors tutorial covering every
cn_*parameter; connector plotting was previously a three-panel aside in the 3D skeletons page.cn_zorderis documented for the first time - it has always worked, and appeared in no docstring
-
-
the plotting tutorials have been restructured: General (the modes and backends, plus one page on colouring that absorbs the old depth-colouring tutorial), 2D plotting and 3D plotting (one page each for skeletons, meshes and volumes), Other plots (connectors, barcodes, topology, collages, XKCD) and Examples. The styling pages compare settings side by side rather than one render at a time - and the 3D ones do it with
plot3d(..., snapshot=True, ax=...), so every panel is a real octarine render. Old URLs undergenerated/gallery/1_plotting/have moved to1a_plotting_general/,1b_plotting_2d/,1c_plotting_3d/,1d_plotting_misc/and1e_plotting_examples/
Fixes#
-
the plotting backends now share the code that resolves connectors, connector colors, somata and
shade_by(new internalnavis/plotting/_common.py) instead of keeping a copy each - ~240 lines of near-identical code across matplotlib, plotly and k3d, and, as usual, the copies had drifted:connectors=<numpy array of types>raisedValueError: truth value of an array ... is ambiguousin every backend, even though all three explicitly listednp.ndarrayamong the accepted types. The array was being fed tobool()by the guard in front of the filtering- plotly gave somata the wrong color when a per-node colormap was in play (
color_by=...). It indexed the color list built for the line trace - which is ordered by segment and padded with a black sentinel per segment - with a node-table index - a partial
cn_colorsdict (e.g.{"pre": "red"}) brokeplot2d(..., method="3d"): the types the dict didn't cover kept their default rgb tuple, and matplotlib can't build an array from a mixed string/tuple sequence norm_global=FalseraisedTypeError: 'bool' object is not iterablefor bothcolor_byandshade_by, and plotly never forwardednorm_globaltoshade_byat all
Two inconsistencies are resolved by construction rather than fixed: k3d ignored
cn_mesh_colorsentirely and checked acn_colorsdict before"neuron", so it disagreed with the other two on which wins; and the cap on runaway soma detection was applied three different ways (all backends now use plotly/k3d's: 10 or more somata on one neuron are taken as a detection failure and skipped, with a warning). Still outstanding:cn_colors="neuron"combined withcolor_by=<node property>is broken in all three backends, because there is no single "neuron color" to give the connectors in that case. -
more plotting fixes, each of them independent of the shared-code rewrite above:
-
navis.plot2dandnavis.plot3ddocumented several defaults they don't actually use:plot2dadvertisedlinewidth=.5(really1),alpha=1(reallyNone),connectors=True(reallyFalse) andmethod='3d'(really'2d');plot3dadvertisedconnectors=True(reallyFalse) andfig_autosize=False(reallyTrue). The defaults are now checked against theSettingsdataclass fields in the test suite, so they can't drift again -
navis.plot3draised for any skeleton plotted withconnectors=Trueon theplotlybackend. NAVis passedopacityinto plotly'sscatter3d.Line, which has no such property. Plotly's default connector layout isdisplay="lines", so the only way past it wascn_layout={"display": "circles"}or plotting aMesh. Connector transparency now sits on the trace, and an alpha channel on an explicitcn_colorsis honoured too instead of being emitted as a malformedrgb(r,g,b,a)string -
cn_colorswas broken innavis.plot2dfor two of its three documented forms:"neuron"was tested againstcn_layout(a dict, so never equal) and reached matplotlib as a literal color name, and a{type: color}dict replaced each whole per-type layout entry rather than just its color -
color_by=<neuron property>worked only in thematplotlibbackend.plotlyandk3dskipped the "is this a neuron property or a per-node property?" resolution and went straight to the per-node path, so e.g.plot3d(nl, color_by="cell_type", palette="viridis")raisedValueError: Column "cell_type" does not exist. All backends now share one implementation, which also makescolor_byrequire apaletteconsistently everywhere -
radius="auto"was decided once for the wholeNeuronListinstead of per neuron. The auto-detection wrote its verdict back onto the shared settings object, so as soon as one neuron had too few radii every subsequent neuron was forced onto lines regardless of its own radii -
navis.plot_flat:normalize_distance=TrueraisedKeyError(it scaled a key that does not exist) andconnectors=Truecombined withhighlight_connectors=[...]raisedTypeError(the connector angles shadowed the per-node angle lookup). Each worked on its own -
navis.plot2dwithdepth_coloring=Trueraised when there were no neurons to normalize against - e.g. when plotting only aVolume -
navis.plot3dwithbackend="octarine"and an explicitviewer=raisedAttributeErrorunless a NAVis-created viewer already existed in the session -
navis.plot3d'sbackendis now case-insensitive: a capitalised but otherwise valid backend (e.g."Plotly") passed validation and then fell through to "unknown backend"
-
Parallelism and threads#
New#
-
new
navis.Pipeline: chain the operations once, then run the chain. Chaining NAVis functions over aNeuronListwithparallel=Truesends every neuron out to a worker and back once per function, and where the functions are cheap that transfer is the runtime. A pipeline fuses consecutive per-neuron steps into a single task, so each neuron makes the trip once no matter how many steps there are - the more you chain, the better the trade gets, which is the opposite of calling each function withparallel=Truein turn. It also tracks ownership: once a step has handed back something the pipeline made itself, every step after it may modify it in place rather than taking its own defensive copy.Steps can be given up front, added one at a time, or simply named - any NAVis function resolves as a method:
pipe = navis.Pipeline().heal_skeleton().prune_twigs(5000).resample_skeleton(1000) res = pipe(nl, parallel=True, n_cores=2)Pipelines are immutable -
addand friends return a new one, so a base can be kept around and branched off, and|splices two together.add_eachandadd_onceoverride whether a step maps over the neurons or is called once with the whole value. Steps do not have to be NAVis functions, and the input does not have to be neurons - it is whatever the first step accepts, sonavis.Pipeline(neu.fetch_skeletons).resample_skeleton(1000)takes a query object and everything after it runs per neuron. Running one takes the sameparallel,n_cores,chunksize,backend,progressandomit_failuresarguments as everything else, and a step that raises comes back as aPipelineStepErrornaming the step and its position.NeuronList.pipelinebuilds one bound to that list for one-off chains (nl.pipeline.heal_skeleton().prune_twigs(5000).run()). See the multiprocessing tutorial. -
parallel=Truenow runs on a backend you can choose - including one you supply. Where per-neuron work runs used to be a hard-wiredpathospool; it is now pluggable via the newnavis.set_parallel_backend(also usable as a context manager) and a per-callbackend=parameter. Shipspathos,joblib, the standard library's process pool (processes),threadsandserial;navis.list_parallel_backendsshows what's installed and third parties can register their own withnavis.compute.register_backend.NeuronList.apply()takesbackendandchunksizealongside the existingparallel/n_cores/omit_failures, all of them explicit parameters rather than forwarded to the applied function.parallel=Truetherefore no longer needspathos- thoughpathos/joblibare still needed to ship a lambda - andnavis.set_parallel_backendaccepts anyconcurrent.futures.Executor, which is how you point NAVis at a cluster without NAVis ever needing scheduler options of its own.backend="auto"(the default) prefersjoblib, thenpathos, then the standard library's pool. This changes which backend existingparallel=Truecalls use:pathoswas the only option before. The two behave the same, butjoblibkeeps its workers alive between calls wherepathosbuilds a fresh pool each time, which makes a sequence of parallel calls measurably faster; passbackend="pathos"to keep the old one. Note also thatinplace=Falseis only silently turned into an in-place run where the workers get copies of the neurons - a thread-based or in-process backend now honours it. -
parallel=Truecan now run across a cluster, via two new backends:dask(a dask.distributed cluster, from aLocalClusterto a compute centre) andsubmitit(an array job on SLURM). Install withpip install navis[cluster]- deliberately not part ofnavis[all]. Neither backend is ever selected automatically.Nothing about how you call NAVis changes:
parallel=Truestill only means "spread this over the neurons", and all scheduler configuration stays on your own object -navis.set_parallel_backendtakes adask.distributed.Client(or a cluster) and asubmitit.Executordirectly. Both bundle neurons into fewer, larger units of work, aiming for enough units to keep every worker busy while capping each at ~128 MB of neurons;chunksize=still overrides it per call. See the multiprocessing tutorial for both in context.
Improvements#
-
parallel=Trueno longer fights with the threads underneath it. NAVis spreads work over processes; navis-fastcore (and the BLAS/OpenMP pools under numpy) spread it over threads, and by default each takes every core it can see. Nothing told a worker process that it was one of twenty, so the two multiplied -n_cores=20on a 224-core node meant 20 x 224 = 4480 threads over 224 cores. Healing 40 skeletons of 200k nodes that way measured slower than not parallelising at all (6.71 s vs 5.10 s) while burning 2.3x the CPU; one thread per worker did it in 3.60 s at a sixth of the CPU.Each worker is now told what it may use. The default divides the machine up rather than handing all of it to everyone -
cpu_count() // n_coresthreads apiece - and the newinner_max_num_threadsoverrides it (the name is joblib's, for the parameter that does the same job there):# work with little internal parallelism to spread over more than one thread with navis.set_parallel_backend(inner_max_num_threads=1): navis.heal_skeleton(nl, parallel=True, n_cores=20)Cluster backends are left alone:
cpu_count() // n_coresis arithmetic about the submitting machine and says nothing about the node a SLURM job lands on. Pass an explicit value if you want one there. -
new:
navis.set_num_threads, which caps fastcore and BLAS in the current process. For the other direction from the above - when you are the one running the pool and NAVis is the thing inside it, where NAVis cannot help itself:def work(neuron): navis.set_num_threads(1) # or once, in the pool's `initializer` return navis.heal_skeleton(neuron) with mp.Pool(20) as pool: healed = pool.map(work, neurons) -
n_coresdefaults now follow the cores this process may use, viaos.process_cpu_count()(or the affinity mask below Python 3.13) rather thanos.cpu_count(). The two differ under SLURM's--cpus-per-task,tasksetand anything else that pins a process to a subset of the machine - i.e. on exactly the machines where claiming cores you do not have hurts most. The NBLAST family'sn_coresalso picks upnavis.set_parallel_backend(n_workers=...), which it previously ignored: its default was baked in at import time.
Fixes#
-
parallel processing:
-
parallel=Truecould hang forever once the process had done any real work.pathosstarts its workers withforkon every platform - including macOS, where the standard library does not - and forking is unsafe after native thread pools (BLAS, or Accelerate on macOS) have come up: only the forking thread survives into the child, so the first call that touches one blocks forever with no error, no output and no progress bar. Skeletonizing a handful of meshes was enough to trigger it. NAVis now startspathosworkers withforkserver/spawn, as it already did for the standard-library backend -
parallel work printed
resource_tracker: There appear to be N leaked semaphore objects to clean up. Building a progress bar - even a disabled one, since the lock is taken intqdm.__new__- makestqdmallocate a cross-process lock, i.e. a named semaphore that nothing then unlinks. Workers now say up front that their bars aren't shared across processes, so no such lock is allocated -
omit_failures=Truemislabelled dataframes with the wrong neuron. Functions returning a dataframe per neuron (e.g.segment_analysis) paired the results with the input neurons by position. Since failed runs are dropped, every dataframe after the first failure got the wrong ID and the last neuron's results were discarded - silently, and on the serial path as much as the parallel one. Also returns an empty dataframe when everything failed, instead of raising frompd.concat -
can_zip/must_ziparguments were never actually distributed per neuron. They were validated against the neuron count and then passed to every neuron whole. In practice this meantnavis.prune_at_depth- the only user - raised a broadcast error when given onesourceper neuron, rather than pruning each from its own root
-
Morphometrics: IVSCC#
Breaking#
-
navis.ivscc_featuresreturns neurons as rows, like every other NeuronList-level function in NAVis. It used to build itsDataFramestraight out of a{neuron_id: {feature: value}}dict, which lands features on the index and neurons on the columns - so.mean()averaged across neurons per feature only after a transpose nobody remembered. The index is now namedid, and neurons sharing an ID no longer collapse into one row. Add a.Tto get the old layout.Several features changed meaning at the same time (see Fixes below for the ones that were simply wrong):
Old New mean_contractionwas tortuosity,L / R >= 1contraction proper, R / L <= 1max_branch_orderwas the branch point countthe number of bifurcations on the longest root-to-tip path basal_dendrite_calculate_number_of_stems<compartment>_num_stems, and computed for every compartmentmax_euclidean_distancewas the summed distancethe maximum distance Feature classes are now constructed from a shared
NeuronContextrather than from a neuron -feat(ctx)instead offeat(neuron, verbose=...)- so that rerooting and the distance-to-soma pass happen once per neuron instead of once per feature class. Custom classes passed tofeatures=need to subclassFeaturesand take a context._check_compartments, which was never called, is gone, andfrom navis.morpho.ivscc import *no longer raises (__all__named a function that does not exist).
New#
-
navis.ivscc_featuresgained the features it was missing, mostly the ones that need a radius or a third dimension:New feature Description extent_zThe depth extent. Only xandywere measured before, so a neuron's third dimension went unrecordednum_tips,num_branch_pointsTip and branch point counts - num_branches(linear segments) was the only topology metricbifurcation_angle_local,bifurcation_angle_remoteMean angle between child branches, measured at the branch point and to the next branch/tip respectively mean_diameter,total_surface,total_volumeRadius-derived size. total_surface/total_volumemodel the cable as tapered cylinders and matchSkeleton.surface_area/.volumeparent_daughter_ratioMean ratio of daughter to parent radius across branch points early_branch_pathPath length to the first branch point over the maximum path length - how early the arbor starts splitting soma_percentile_x,soma_percentile_yFraction of the compartment's nodes above the soma soma_radius,soma_surface,num_stemsWhole-cell features, from the new SomaFeaturesexit_distanceandexit_thetaare no longer axon-only - every compartment gets them. Features needing a radius are skipped (NaN, plus a warning underverbose=True) where the node table has none: NAVis fills a missingradiuscolumn with zeros, which used to come back astotal_surface = 0.0rather than "not measured".There is also a new IVSCC tutorial - the docstring used to point at one that did not exist. It works through 25 Patch-seq mouse cortical cells from the Brain Image Library, including the part that is easy to get wrong: these features read cortical depth off the
yaxis, and the same cells measured in their unaligned_Rawframe come back withsoma_percentile_yof 0.07 instead of 0.97 andbias_ythe wrong way round. -
a second IVSCC tutorial covers EM data, which the first one dismisses in a sentence. It preps 32 proofread MICrONS cells - two per m-type,
L2athroughL6tall-c- fornavis.ivscc_featuresusing only existing NAVis API. The two things a connectome will not hand you are an apical/basal split and a frame whereymeans cortical depth, and both are short:navis.subset_neuronplusnavis.split_componentsto find the dendritic stem that climbs towards the pia, and one 4x4 matrix throughnavis.xformto undo the ~5° tilt of the cortical column (worth 87 µm of apparent depth per mm ofx- about half a layer across the dataset). Also covers the QC that catches a cell whose axon label is missing entirely, and whatnavis.split_axon_dendritedoes about it.
Fixes#
-
navis.ivscc_featureshad six features that were quietly wrong. Most of them came from the same place: a compartment is measured on a subset of the neuron, and the subset does not know about the cell it was cut out of.max_euclidean_distancewas.sum().max()on the per-node distances..sum()collapses the series to a scalar and.max()on a scalar is a no-op, so the feature was the sum of every node's distance to the soma - growing with node count rather than with reachcalculate_number_of_stemswas always 0. It counted nodes whose parent is the soma, but the soma is not in the compartment subset andsubset_neuronhad already rewired those stems into roots. Stems are now counted on the full neuronmax_branch_orderwasn_branch_points + 1, which is a count, not an order: a balanced binary tree of depth 5 scored 32 instead of 5, and a 10-stem star 2 instead of 1mean_contractionrecordednavis.tortuosity, i.e.L / R. Contraction is the reciprocal,R / L, and is now computed as such - and note thatmean(R / L)is not1 / mean(L / R)(0.940 vs 0.931 on the example neuron)exit_distanceandexit_thetacould describe different roots: the distance took theminover all of the axon's roots, the angle whichever came first in the node table. Both now use the root closest to the soma. The distance is also clamped at 0 (it went negative for a root inside the soma sphere) and falls back to the soma centre where the radius is unknown, instead of returningNaN- branch points, branch order, bifurcation angles and
parent_daughter_ratiomissed a compartment's most proximal fork. Those read the node table'stypecolumn, which types the subset's own root asrooteven where it has two children - so a dendrite splitting as it leaves the soma counted zero branch points. They now count children directly, excluding only the soma
Two crashes are gone with them:
BasicFeatureson a neuron without a soma returnedNonefromextract_featuresand blew up ondict.update(None), and a neuron without alabelcolumn raised a bareValueErrorthatmissing_compartmentsdid not catch - so"ignore"and"skip"did not, in fact, ignore or skip it.missing_compartmentsandxare now validated up front rather than falling through toraise.
Machine learning#
New#
- new
navis.mlmodule: helpers for preparing neurons as machine-learning inputs. These live under their ownnavis.ml.*namespace (deliberately not lifted to the top level) and come with three tutorials:- normalize -
navis.ml.normalize_neuronmaps a neuron into a canonical frame with a single rigid-plus-uniform-scale transform: center on the centroid/bbox/soma/an explicit point, PCA-orient (signs disambiguated deterministically and handedness preserved - neurons are never mirrored), and scale to unit RMS / extent / enclosing sphere.return_matrix=Truereturns the 4x4 matrix so predictions made in the normalized frame can be mapped back - augment - geometry/sampling augmentations, each taking a
random_stateand returning a copy:jitter_neuron,warp_neuron,rotate_neuron,scale_neuron,translate_neuronanddrop_nodes.augment_neuronchains them in one seeded, reproducible call - sample & chunk - turn a variable-sized neuron into a fixed-size model input:
sample_cable(arclength-uniform sampling along a skeleton),sample_surface(area-weighted mesh-surface sampling), both with per-point provenance, pluschunk_neuron(tile a neuron into evenly-sized fragments for batching)
- normalize -
I/O and interfaces#
Breaking#
-
the
rpy2-based R interface is gone. It required a working R installation plusrpy2on top of it, and existed mainly to move data in and out of the natverse. That job is now done by plain files -navis.read_rda,navis.read_rds,navis.write_rdaandnavis.write_rds(see below), which need neither R norrpy2.from navis.interfaces import rnow raises with a pointer to the replacement, and thenavis[r]install extra no longer exists. See the new natverse tutorial for the round trip.Removed Use instead r.neuron2r()/r.neuron2py()navis.write_rds/navis.read_rdsr.load_rda()navis.read_rdar.nblast()/r.nblast_allbyall()navis.nblast/navis.nblast_allbyallr.xform_brain()/r.mirror_brain()navis.xform_brain/navis.mirror_brainr.get_neuropil()/r.get_brain_template_mesh()flybrains r.init_rcatmaid()pymaid -
the Cytoscape interface is gone.
navis.interfaces.cytoscapepushed a network into a running Cytoscape desktop over cyREST, via py2cytoscape - a dependency that is no longer maintained and was never declared as an install extra. It was deprecated in the API docs and nothing in it was navis-specific:generate_network()handed anetworkxGraph or a DataFrame straight to the cyREST client, andget_navis_style()set some colours.from navis.interfaces import cytoscapenow raises with a pointer to the replacement - build the graph withnavis.network2nxand hand it to Cytoscape as a file (networkx.write_graphml(g, "network.graphml")), or drive cyREST yourself. -
navis.interfaces.neuromorpho.find_neuronsno longer prompts. Called with no filters it used to ask"No filters will list all neurons. Continue? [Y/N]"on stdin - a library function blocking on a human who, in a script or a server, is not there. It now raises and tells you to pass either a filter or apage_limit.navis.interfaces.vfb.get_skeletonslikewise returns an emptyNeuronListrather thanNonewhen nothing matches
New#
-
new
navis.write_parquet(..., format="neurarrow"): parquet files other tools can read. neurarrow is a specification for neuroanatomical data on Apache Arrow which started from NAVis' own parquet format and has since gone its own way - different column names,uint64IDs unique across the whole file,float64coordinates, null rather than-1for roots, and required metadata.format="neurarrow"writes all of that, andnavis.read_parquetdetects and reads neurarrow files without being told - including those written by other tools, such as swc2na.Two things don't map cleanly: neurarrow tracks units for the file, not the neuron, so all neurons must share the same units (and dotprops the same
k); and navis' connector table is a set of point annotations where neurarrow'sconnectionsschema describes edges, so connectors are written through thenet.clbarnes.connectorextension, which has no room for extra columns likeroi.format="navis"remains the default and the only lossless round-trip - see the format spec for the full comparison. -
new
navis.write_rdsandnavis.write_rda: hand data to R's natverse withoutrpy2.Skeletonnat::neuron(including nat'sSegList/SubTreestopology, soresample(),prune_*()etc. work on the result),Dotpropsnat::dotprops,Mesh/Volumergl::mesh3d,Voxelsnat::im3dandNeuronListnat::neuronlist(with its meta data attached); plain dicts, lists, arrays andDataFramesbecome the corresponding R types. The serialisation is done by rdata, so no R installation and norpy2are required on either end; read the files in R withreadRDS()/load() -
new
navis.read_rds, the counterpart tonavis.read_rdafor.rdsfiles (a single, unnamed R object rather than a whole workspace) -
neuprintinterface: newdedupoption for fetching synapses. In insect connectomes synapses are polyadic - a single presynapse connects to multiple postsynapses - andneuprint-pythonde-duplicates presynapses so each is reported only once.fetch_synapses(a new thin wrapper overneuprint.fetch_synapses),fetch_skeletonsandfetch_mesh_neuronnow takededup(defaultTrue, i.e. unchanged behaviour). Withdedup=Falseeach presynapse is instead reported once per postsynaptic site it connects to -
new function:
navis.patch_caveclientmonkey patchescaveclientto return NAVis neurons, the same waynavis.patch_cloudvolumedoes forcloud-volume. It wraps the skeleton service -client.skeleton.get_skeleton,get_bulk_skeletonsandfetch_skeletons- giving each anas_navis=Truekeyword argument plus a*_navistwin that always converts. Bothoutput_formats are handled, which matters more than it sounds:"dict"is in nanometres and"swc"in micrometres, and the neurons come back with.unitsset accordingly. The service's per-vertex compartment (1soma,2axon,3dendrite) lands in the node table'slabelcolumn, sonavis.ivscc_featuresandcolor_by="label"work on the result without further ado.Two differences to
patch_cloudvolume: it patches the client class, so it can be called after the client exists and patching twice is a no-op; and because the bulk endpoints cap how many skeletons they return (ten, at time of writing) and drop the rest silently, NAVis warns when it gets fewer skeletons back than you asked for - a truncatedNeuronListlooks a lot like a complete one.
Improvements#
-
the remote data-source interfaces now share their plumbing (new
navis/interfaces/base.py). Each ofmicrons,h01,neuprint,neuromorpho,insectbrain_db,vfb,allen_celltypesandbrain_image_libraryhad grown its own copy of "fan a fetch out over a thread pool", twelve in all - and eight of them ended in the same line,print(f"{id} generated an exception:", exc). A failed neuron was written to stdout and dropped, so a partial result came back looking like a complete one. There is now onefetch_parallel, and with it:- results come back in the order you asked for them. The old loops appended as requests completed, so the order was whatever the network happened to do; two modules re-sorted afterwards to compensate and the rest simply returned neurons shuffled
- a failure is reported through
errors, notprint-"raise","log"(default) or"ignore", matching whatnavis.read_swcand friends already take. Available on every fetch function cave_utils.fetch_neuronstreated a failure differently depending on whetherparallelwasTrueorFalse(swallowed vs raised). Both now followerrors
-
new
navis.config.strictfor server and pipeline contexts (orNAVIS_STRICT=1). It does not change what NAVis computes - only how it behaves when something goes wrong or when it would otherwise want a human: remote fetches default toerrors="raise"so a partial result is loud rather than silent, and nothing prompts for input. Carried into worker processes like the other settings -
importing an interface no longer requires its dependencies - or a network connection.
navis.interfaces.vfbbuilt a liveVfbConnectclient at import time, so merely importing it phoned VirtualFlyBrain;allen_celltypeswrote anallensdkmanifest into your working directory on import, and it andvfbboth raised outright if their dependency was missing (whilemicrons/cave_utilslogged and left aNonebehind to trip over later). Every optional dependency now goes through oneoptional_import, which defers the failure to first use and reports it with thepip installline to fix it.vfb.get_client()andvfb.vcstill give you the client, on demand -
one HTTP session for the whole library (new
navis.utils.http).navis.io, the Brain Image Library and NeuroMorpho each built their own - or, in NeuroMorpho's and part of InsectBrain DB's case, used barerequests.getwith no pooling at all. All now share a pooled, retrying session that identifies NAVis in its User-Agent; URL reads pick up retries on 429/5xx as a side effect. Insect Brain DB keeps its own session, since itsAuthorizationheader must not leak onto anything else -
every interface cache is now reachable from one place:
navis.interfaces.clear_cache(). There were nine - eightlru_caches scattered acrosscave_utils,micronsandinsectbrain_dbplus the Brain Image Library's metadata dict - and only the last of them had a way to clear it
Fixes#
-
navis.write_parquetsilently dropped connectors. It only ever wrote the node table, so round-tripping any neuron with synapses gave it back withconnectors=None- no warning, and neither the docstring nor the format spec mentioned it. Connectors now go into a sidecar file next to the main one (neurons.parquetgets aneurons.connectors.parquet), whichnavis.read_parquetpicks up automatically; two files rather than a zip archive, so both tables keep parquet's column pruning and predicate pushdown. Passwrite_connectors=Falseto opt out. The node table'slabelcolumn was going missing the same way and is now written too -
navis.read_parquetreturned aNeuronListfor a single-neuron file, contradicting its own docstring, because dotprops always carried the neuron-ID column. A file holding exactly one neuron now reads back as a single neuron for both skeletons and dotprops;subset=/limit=still always give you aNeuronList -
reading with a list
limitreturned nothing at all for most sources.limit=["neuron.swc"]is documented as a list of filenames to pick out of a folder or archive, but the readers filtered it withf in limitagainst whatever they happened to be holding -Pathobjects for a folder,ZipInfoobjects for a zip, neither of which compares equal to a string - sonavis.read_swc(dir, limit=["neuron.swc"])came back empty. Tar archives and Google Storage buckets kept full paths, so a plain filename missed there too; of the five call sites only FTP, which happens to store bare filenames, ever worked. A listlimitnow matches either a filename or a full path everywhere. Note that a filename is matched against the last path component, so one entry can pick up files of that name in several subdirectories - give the full path to disambiguate. The lookup is also hashed once rather than scanned per file, which takes filtering a large library from quadratic to linear -
neuprintinterface:fetch_skeletonsshared one client - and hence onerequests.Session- across all its worker threads, whichrequestsdoes not support.neuprint-pythonnormally hands each thread its own deepcopy of the client, but passing an explicit client into a thread bypasses that; NAVis now does the copying itself in the pool's initializer -
navis.interfaces.brain_image_library: a failing root directory listing now raises instead of quietly returning an empty file table, and partial listing failures further down warn that the result is incomplete rather than passing it off as the whole dataset -
navis.conversion.mesh2skeletonignored theprogresskeyword argument it accepts -
navis.read_precomputedsilently shattered skeletons whose edges are not in the column order it expected. The precomputed spec leaves edges undirected - neither column is promised to be the parent - but the reader read column 0 as the parent and column 1 as the child, and built the parent mapping with adictkeyed on column 1. In a file written the other way round, every branch point repeats in column 1 and each repeat overwrites the last, so those edges vanished without a word: the male CNS skeletons, for instance, came out with 378 roots where the graph actually has 4. Edges are now oriented by traversal, which also breaks any cycles. Where one column can be the child column - i.e. it holds each vertex at most once - the vertices missing from it are used as the roots, so a file with a consistent orientation (including anything NAVis wrote) keeps the roots it meant to have
Packaging, dependencies and internals#
Breaking#
-
navis-fastcore is now a required dependency (
>= 0.10.0), and the pure-Python/igraph/scipy fallbacks are gone. NAVis has shipped two implementations of its graph, geodesic, morphometric and NBLAST core for several releases - a fastcore fast path and a fallback for installs without it - and kept them agreeing bit-for-bit. That is over: fastcore is the engine.For anyone installing with
pip, nothing changes but the install: fastcore ships prebuilt wheels for macOS (Intel/ARM), Windows, and Linux on x86-64, aarch64, i686, armv7l, ppc64le and s390x. Only musl-based Linux (Alpine) builds from source and therefore needs a Rust toolchain.pip install navis[fastcore]still resolves - it is now a no-op alias.Two behavioural notes.
navis.geodesic_matrixon aMeshno longer acceptsdirected=Trueas anything but a no-op (it was already ignored for meshes). And the private helpers behind skeleton stitching -_stitch_edges,_segment_radii,_rewire_from_edges,_component_labels- have been removed along with the KDTree stitcher they made up;heal_skeletonandheal_meshare unaffected.
Improvements#
- two dependencies are gone and two are no longer imported up front.
sixwas a Python-2 compatibility shim andpypnghad not been imported anywhere for several releases; both are dropped from the requirements.morphopsandmolesqare now imported on first use rather than at module level, taking ~280 ms offimport navis.molesqis now only reached by the deprecated"python"backend;morphopsis still needed by every thin plate spline transform, on either backend, since it does the fit
Fixes#
-
NAVis' logger could be left silenced for the rest of the session.
arbor_segregation_index,bending_flow,synapse_flow_centralityandflow_centralityquieten the logger while they build a throwaway downsampled copy, and a neuron's HTML thumbnail does the same while it plots; if any of those steps raised, the whole library went quiet with nothing to say why (and the thumbnail additionally left matplotlib's interactive mode off and a stray figure open). All now go through the newnavis.config.quiet_loggercontext manager -
pip install navis[all]pulled incloud-volumeand its dependency tree, which was never the intention - it is a specialized dependency and is meant to be installed on its own (navis[cloudvolume]). The exclusion list named the distribution (cloud-volume) where it needed the extra (cloudvolume), so it silently matched nothing.r,flybrainsandclusterwere unaffected