Teneto Docs

_images/gennet_example1.png

What are temporal networks?

Temporal networks are, quite simply, network representations that flow through time. They are useful for analysing how a connected system develops, changes or evolves through time. This change in time can depict how information spreads along with a social network or how different brain areas cooperate to perform a task.

This page introduces some of the basic concepts of temporal network theory.

Node and edges: the basics of a network

A network is a representation of something using a graph from mathematics. This something can be a representation of an empirical phenomenon or a simulation. A graph contains nodes (sometimes called vertices) and edges (sometimes called links).

Nodes and edges can represent a vast amount of different things in the world. For example, nodes can be friends, cities, or brain regions. Edges between could represent trust relationships, train lines, and neuronal communication.

The flexibility in what nodes are is one of the reasons network theory is very interdisciplinary. The benefits of having network representation are that similar analysis methods can be applied, regardless of what the underlying node or edge represents. This abstractness means that network theory is a very inter-disciplinary subject. However, it also entails that certain concepts have multiple names (e.g. nodes and vertices).

With a network, you can analyse for example, if there is any “hub” node. In transportation networks, there are often hubs which connect many different areas where passengers usually have to change at (e.g. airports like Frankfurt, Heathrow or Denver). In social networks, you can quantify how many steps it is to another person in that network (see the famous six steps to Kevin Bacon).

Mathematically, A network if often referenced as G or \(\mathcal(G)\); i and j are indices of nodes; a tuple (i,j) reference an edge between nodes i and j. G is often expressed in the form of a connectivity matrix (or adjacency matrix) \(A_{ij} = 1\) if a connection is present and \(A_{ij} = 0\) if a connection is not present. The number of nodes if often referenced to as N. Thus, A is a N x N matrix.

Different network types

There are a few different versions of networks. Two key distinctions are:

  1. Are the connections binary or weighted.
  2. Are the connections undirected or directed.

If a connection is binary, then (as in the section above) an edge is either present or not. When adding a weight-value, an edge becomes a 3-tuple (i,j,w) where w is the magnitude of the weight. And in the connectivity matrix, \(A_{ij} = w\). Often the weight is between 0 and 1 or -1 and 1, but this does not have to be the case.

When connections are undirected, it means that both nodes share the connection. Examples of such networks can be if two cities are connected by train lines. For such networks \(A_{ij} = A_{ji}\). With directed edges, it means that the connection goes from i to j. Examples of these types of networks can be citation networks. If a scientific article i cites another article j, it is not common for j to also cite i. So in such cases, \(A_{ij}\) does not need to equal \(A_{ji}\). It is the common notation for the source node (sending the information) to be first and the target node (receiving the information) to be second.

Adding a time dimension

In the above formulation of networks \(A_{ij}\) only has one edge. In a temporal network, a time-stamp is also included in the edge’s tuple. Thus, binary edges are now expressed as 3-tuples (i,j,t) and weighted networks as 4 tuples (i,j,t,w). Connectivity matrices are now three dimensional: \(A_{ijt} = 1\) in binary and \(A_{ijt} = w\) in weighted networks.

The time indices are an ordered sequence. This ordering can now reveal information about what is occurring in the network through time.

For example, using friends’ lists from social network profiles can be used to create a static network about who is friends with who. However, imagine one person enters a group of friends, by seeing when everyone become friends, this gives the network more explanatory power.

Compare the following two figures representing meetings between friends:

(Source code, png, hires.png, pdf)

_images/what_is_tnt-1.png

In the static network, on the left, each person (node) is a circle, and each black line connecting the rings is an edge. In this figure, we can see that everyone has met everyone except Dylan (orange) and Casey (light green).

The slice_plot on the left shows nodes (circles) at multiple “slices” (time-points). Each column represents of nodes represents one time-point. The black line connecting two nodes at a time-point signifies that they met at that time-point.

In the temporal network, we can see a progression of who met who and when. At event 1, Ashley and Blake met. Then A-D all met together at event 2. At event 3, Blake met Dylan. And at event 4, Elliot met Dylan and Ashley (but those two themselves did not attend). This depiction allows for new properties to be quantified that missed in a static network.

What is time-varying connectivity?

Another concept that is often used within fields such as cognitive neuroscience is time-varying connectivity. Time-varying connectivity is a larger domain of methods that analyse distributed patterns over time where temporal network theory is one set of analysis methods within it. Temporal network theory analyses time-varying connectivity representations that consist of time-stamped edges between nodes. There are other alternatives to analyse such representations and other time-varying connectivity representations as well (e.g. temporal ICA).

What is teneto?

Teneto is a python package that can several quantify temporal network measures (more are being added). It can also use methods from time-varying connectivity to derive connectivity estimate from time-series data.

Further reading

Holme, P., & Saramäki, J. (2012). Temporal networks. Physics reports, 519(3), 97-125. [Arxiv link] - Comprehensive introduction about core concepts of temporal networks.

Kivelä, M., Arenas, A., Barthelemy, M., Gleeson, J. P., Moreno, Y., & Porter, M. A. (2014). Multilayer networks. Journal of complex networks, 2(3), 203-271. [Link] - General overview of multilayer networks.

Lurie, D., Kessler, D., Bassett, D., Betzel, R. F., Breakspear, M., Keilholz, S., … & Calhoun, V. (2018). On the nature of resting fMRI and time-varying functional connectivity. [Psyarxiv link] - Review of time-varying connectivity in human neuroimaging.

Masuda, N., & Lambiotte, R. (2016). A Guidance to Temporal Networks. [Link to book’s publisher] - Book that covers a lot of the mathematics of temporal networks.

Nicosia, V., Tang, J., Mascolo, C., Musolesi, M., Russo, G., & Latora, V. (2013). Graph metrics for temporal networks. In Temporal networks (pp. 15-40). Springer, Berlin, Heidelberg. [Arxiv link] - Review of some temporal network metrics.

Thompson, W. H., Brantefors, P., & Fransson, P. (2017). From static to temporal network theory: Applications to functional brain connectivity. Network Neuroscience, 1(2), 69-99. [Link] - Article introducing temporal network’s in cognitive neuroscience context.

Tutorial

Tutorial: Network representation in Teneto

There are three ways that network’s are represented in Teneto:

  1. A TemporalNetwork object
  2. Numpy array/snapshot
  3. Dictionary/contact representation

This tutorial goes through what these different representations. Teneto is migrating towards the TemporalNetwork object. However, it is possible to still use with the other two representations.

TemporalNetwork object

TemporalNetwork is a class in teneto.

>>> from teneto import TemporalNetwork
>>> tnet = TemporalNetwork()
...

As an input, you can pass it a 3D numpy array, a contact representation (see below), a list of edges or a pandas df (see below).

A feature of the TemporalNetwork class is that the different functions such as plotting and networkmeasures can be accessed within the object.

For example, the code below calls the function teneto.generatenetwork.rand_binomial with all subsequent arguments being arguments for the rand_binomial function:

>>> import numpy as np
>>> np.random.seed(2019) # Set random seed for replication
>>> tnet.generatenetwork('rand_binomial',size=(5,3), prob=0.5)

The data this creates is found in tnet.network which is a pandas data frame. To have a peak at the top of the data frame, we can call:

>>> tnet.network.head()
   i  j  t
0  0  1  0
1  0  1  1
2  0  2  0
3  0  2  1
4  0  2  2

Each line in the data frame represents one edge. i and j are both node indexes and t is a temporal index. These column names are always present in data frames made by Teneto. There is no weight column here which indicates this is a binary network.

Exploring the network

You can inspect different parts of the network by calling tnet.get_network_when() and specifying an i, j or t argument.

>>> tnet.get_network_when(i=1)
   i  j  t
6  1  2  0
7  1  2  2
8  1  3  0
9  1  4  1

The different argument can also be combined.

>>> tnet.get_network_when(i=1, t=0)
   i  j  t
6  1  2  0
8  1  3  0

Lists can also be specified as arguments:

>>> tnet.get_network_when(i=[0, 1], t=1)
   i  j  t
1  0  1  1
3  0  2  1
5  0  4  1
9  1  4  1

The logic within each argument is OR (i.e. about get all where i == 1 OR i == 0). The logic between the different arguments, defaults to AND. (i.e. get when i == [0 or 1] AND t == 1). In some cases, you may might the between argument logic to be OR:

>>> tnet.get_network_when(i=1, j=1, logic='or')
   i  j  t
0  0  1  0
1  0  1  1
6  1  2  0
7  1  2  2
8  1  3  0
9  1  4  1

In the above case we select all edges where i == 1 OR j == 1.

Weighted networks

When a network is weighted, the weight appears in its own column in the pandas data frame.

>>> np.random.seed(2019) # For reproducibility
>>> G = np.random.beta(1, 1, [5,5,3]) # Creates 5 nodes and 3 time-points
>>> tnet = TemporalNetwork(from_array=G, nettype='wd', diagonal=True)
>>> tnet.network.head()
   i  j  t    weight
0  0  0  0  0.628820
1  0  0  1  0.059084
2  0  0  2  0.833974
3  0  1  0  0.856509
4  0  1  1  0.518670

Self edges get deleted unless the argument diagonal=True is passed. Above we can see that there are edges when both i and j are 0.

Dense and sparse networks

The example we saw previously was of a sparse network representation. This means that only the active connections are encoded in the representation and all other edges can be assumed to be zero/absent.

There are many weighted networks all edges have a value. These networks are called dense.

In denser networks, tnet.network will be a numpy array with node,node,time dimensions. The reason for this is simply speed. If you do not want a dense network to be created, you can pass a forcesparse=True argument when creating the TemporalNetwork.

If teneto is slow, it could be that creating the sparse network is taking too much time. So one way to ensure the dense representation is forced is to set the parameter dense_threshold. The default value is 0.1 (i.e. 10%), which means that if 10% of the network’s connections are present, teneto will make the network dense. But you can set this to any value.

The TemporalNetwork functions such as get_network_when() still function with the dense representation.

Exporting to a numpy array

You can export the network to a numpy array from the pandas data frame by calling to array:

>>> np.random.seed(2019) # For reproducibility
>>> G = np.random.beta(1, 1, [5,5,3]) # Creates 5 nodes and 3 time-points
>>> tnet = TemporalNetwork(from_array=G, nettype='wd', diagonal=True)
>>> G2 = tnet.to_array()
>>> G == G2
True

Here G2 is a 3D numpy array which is equal to the input G (a numpy array).

Meta-information

Within the object there are multiple bits of information about the network. We, for example, check that the above network create below is binary:

>>> tnet = TemporalNetwork()
>>> tnet.generatenetwork('rand_binomial',size=(3,5), prob=0.5)
>>> tnet.nettype
'bu'
There are 4 different nettypes:
bu, wu, wd and bd.

where b is for binary, w is for weighted, u means undirected and d means directed. Teneto tries to estimate the nettype, but specifying it is good practice.

You can also get the size of the network by using:

>>> tnet.netshape
(3, 5)

Which means there are 3 nodes and 5 time-points.

Certain metainformation is automatically used in the plotting tools. For example, you can add some meta information using the nodelabels (give names to the nodes), timelabels (give names to the time points), and timeunit arguments.

>>> import matplotlib.pyplot as plt
>>> tlabs = ['2014','2015','2016','2017','2018']
>>> tunit = 'years'
>>> nlabs = ['Ashley', 'Blake', 'Casey']
>>> tnet = TemporalNetwork(nodelabels=nlabs, timeunit=tunit, timelabels=tlabs, nettype='bu')
>>> tnet.generatenetwork('rand_binomial',size=(3,5), prob=0.5)
>>> tnet.plot('slice_plot', cmap='Set2')
>>> plt.show()

(Source code, png, hires.png, pdf)

_images/networkrepresentation-1.png
Importing data to TemporalNetwork

There are multiple ways to add data to the TemporalNetwork object. These include:

  1. A 3D numpy array
  2. Contact representation
  3. Pandas data frame
  4. List of edges
Numpy Arrays

For example, here we create a random network based on a beta distribution.

>>> np.random.seed(2019)
>>> G = np.random.beta(1, 1, [5,5,3])
>>> G.shape
(5, 5, 3)

Numpy arrays can get added by using the from_array argument

>>> tnet = TemporalNetwork(from_array=G)

Or for an already defined object:

>>> tnet.network_from_array(G)
Contact representation

The contact representation (see below) is a dictionary which a key called contacts includes a contact list of lists and some additional metadata. Here the argument is from_dict should be called.

>>> C = {'contacts': [[0,1,2],[1,0,0]],
        'nettype': 'bu',
        'netshape': (2,2,3),
        't0': 0,
        'nodelabels': ['A', 'B'],
        'timeunit': 'seconds'}
>>> tnet = TemporalNetwork(from_dict=C)

Or alternatively:

>>> tnet = TemporalNetwork()
>>> tnet.network_from_dict(C)
Pandas data frame

Using a pandas data frame the data can also be imported. Here the required columns are: i, j and t (the first two are nodes, the latter is time index). The column weight is also needed for weighted networks.

>>> import pandas as pd
>>> netin = {'i': [0,0,1,1], 'j': [1,2,2,2], 't': [0,0,0,1], 'weight': [0.5,0.75,0.25,1]}
>>> df = pd.Data Frame(data=netin)
>>> tnet = TemporalNetwork(from_df=df)
>>> tnet.network
   i  j  t  weight
0  0  1  0    0.50
1  0  2  0    0.75
2  1  2  0    0.25
3  1  2  1    1.00
List of edges

Alternatively a list of lists can be given to TemporalNetwork, in such cases each sublist should follow the order [i,j,t,[weight]]. For example:

>>> edgelist = [[0,1,0,0.5], [0,1,1,0.75]]
>>> tnet = TemporalNetwork(from_edgelist=edgelist)
>>> tnet.network
   i  j  t  weight
0  0  1  0    0.50
1  0  1  1    0.75

This creates two edges between nodes 0 and 1 at two different time-points with two weights.

Array/snapshot representation

The array/snapshort representation is a three dimensional numpy array. The dimensions are (node,node,time).

The positives of arrays are that they is easy to understand and manipulate. The downside is that any meta-information about the network is lost and, when the networks are big, can use a lot of memory.

Contact representation

Note, the contact representation is going to be phased out in favour for the TemporalNetwork object with time.

The contact representations is a dictionary that can includes more information about the network than an array.

The keys in the dictionary include ‘contact’ (node,node,timestamp) which define all the edges in te network. A weights key is present in weighted networks containing the weights. Other keys for meta-information include: ‘dimord’ (dimension order), ‘Fs’ (sampling rate), ‘timeunit’, ‘nettype’ (if network is weighted/binary, undirected/directed), ‘timetype’, nodelabels (node labels), t0 (the first time point).

Converting between contact and graphlet representations

Converting between the two different network representations is quite easy. Let us generate a random network that consists of 3 nodes and 5 time points.

import teneto
import numpy as np

# For reproducibility
np.random.seed(2018)
# Number of nodes
N = 3
# Number of time-points
T = 5
# Probability of edge activation
p0to1 = 0.2
p1to1 = .9
G = teneto.generatenetwork.rand_binomial([N,N,T],[p0to1, p1to1],'graphlet','bu')
# Show shape of network
print(G.shape)

You can convert a graphlet representation to contact representation with: teneto.utils.graphlet2contact

C = teneto.utils.graphlet2contact(G)
print(C.keys)

To convert the opposite direction, type teneto.utils.contact2graphlet:

G2 = teneto.utils.contact2graphlet(C)
G==G2

Tutorial: Temporal network measures

The module teneto.networkmeasures includes several functions to quantify different properties of temporal networks. Below are four different types of properties which can be calculated for each node. For all these properties you can generally derive a time-averaged version or one value per time-point.

Many of the functions use a calc argument to specify what type of measure you want to quantify. For example calc=’global’ will return the global version of a measure and calc=’communities’ will return the community version of the function.

Centrality measures

Centrality measures quantify a value per node. These can be useful for finding important nodes in the network.

Community dependent measures

Community measure quantify a value per community or a value for community interactions. Communities are an important part of network theory, where nodes are grouped into groups.

Node measures that are dependent on community vector

Global measures

Global measures try and calculate one value to reflect the entire network. Examples of global measures:

Edge measures

Edge measures quantify a property for each edge.

Community measures

Community measures quantify the community partition instead of the underlying network. These are found in the module teneto.communitymeasures

Workflows

Many analyses can be constructed as a graph to depict all the steps that are made during the analysis. This graph of an analysis is called a workflow. There are many benefits to creating a workflow:

  • Construct entire analysis workflow and view it before running.
  • Carefully records every step, so you know exactly what you did.
  • Can share the entire analysis with someone else (good for reproducibility).

TenetoWorkflow allows you to define a workflow object and then run it. A workflow consists of a directed graph. The nodes of this graph are different Teneto functions. The directed edges of the graph is the sequence the pipeline is run in.

The workflows function around the TenetoBIDS or TemporalNetwork classes. Any analysis made using those classes can be made into a workflow.

There are three different types of nodes in this graph:

root nodes: These are nodes that do not depend on any other nodes in the analysis. These are calls to create a _TenetoBIDS_ or _TemporalNetwork_ object.

non-terminal nodes: These are nodes that are intermediate steps in the analysis.

terminal nodes: These are the final nodes in the analysis. These nodes will include the output of the analysis.

Understanding the concept of root and terminal nodes are useful to understand how the input and output of TenetoWorkflow.

Creating a workflow

We are going to create a workflow that does the following three steps:

  1. Creates a temporal network object (root node)
  2. Generates random data (non-terminal node)
  3. Calculates the temporal degree centrality of each node (terminal node)

We start by creating a workflow object, and defining the first node:

>>> from teneto import TenetoWorkflow
>>> twf = TenetoWorkflow()
>>> nodename = 'create_temporalnetwork'
>>> func = 'TemporalNetwork'
>>> twf.add_node(nodename=nodename, func=func)

Each node in the workflow graph needs a unique name (argument: nodename). If you create two different TemporalNetwork objects in the workflow, these need different names to differentiate them.

The func argument specifies the class that is initiated or the function that is run.

There are two more optional arguments that can be passed to add_node: depends_on and params. We will look at those later though.

By adding a node, this creates an attribute in the workflow object which can be viewed as:

>>> twf.nodes
{'create_temporalnetwork': {'func': 'TemporalNetwork', 'params': {}}}

It also creates a graph (pandas dataframe) which is found in TenetoWorkflow.graph.

>>> twf.graph
    i   j
0   isroot  create_temporalnetwork

Since this is the first node in the workflow, _isroot_ is placed in the _i_ column to signify that _create_temporalnetwork_ is the root node.

Now let us add the next two nodes and we will see the params argument add_node:

>>> # Generate network node
>>> nodename = 'generatenetwork'
>>> func = 'generatenetwork'
>>> params = {
    'networktype': 'rand_binomial',
    'size': (10,5),
    'prob': (0.5,0.25),
    'randomseed': 2019
    }
>>> twf.add_node(nodename, func, params=params)
>>> # Calc temporal degree centrality node
>>> nodename = 'degree'
>>> func = 'calc_networkmeasure'
>>> params = {
    'networkmeasure': 'temporal_degree_centrality',
    'calc': 'time'
    }
>>> twf.add_node(nodename, func, params=params)

Here we see that the params argument is a dictionary of _*kwargs_ for the _TemporalNetwork.generatenetwork_ and _TemporalNetwork.calc_networkmeasure_ functions.

Now we have three nodes defined, so we can look at the TenetoWorkflow.graph:

>>> twf.graph
    i   j
0   isroot  create_temporalnetwork
1   create_temporalnetwork generatenetwork
2   generatenetwork    degree

Each row here shows the new node in the _j_-th column and the step preceding node in the _i_-th column.

The workflow graph can be plotted with:

>>> fig, ax = twf.make_workflow_figure()
>>> fig.show()

(Source code, png, hires.png, pdf)

_images/workflow-1.png

Running a workflow

Now the workflow has been defined, it can be run by typing:

>>> tfw.run()

And this will run all of steps.

Viewing the output

The output of the final step will be found in TenetoWorkflow.output_[<nodename>].

The nodes included here will be all the terminal nodes. However when defining the TenetoWorkflow, you can set the argument, _remove_nonterminal_output_ to False and all node output will be stored.

The output from the above is found in:

>>> tfw.output_['degree']
...

More complicated workflows

The previous example consists of only three steps and occurs linearly. In practice analyses are usually more complex. One typical example is where multiple parameters are run (e.g. to demonstrate that a result is dependent on that parameter).

Here we define a more complex network where we generate two different networks. One where there is a high probability of edges in the network and one where there is a low probability.

When adding a node, the node refers to the last node defined unless depends_on is set. This should point to another preset node.

Example:

First define the object.

>>> from teneto import TenetoWorkflow
>>> twf = TenetoWorkflow()
>>> nodename = 'create_temporalnetwork'
>>> func = 'TemporalNetwork'
>>> twf.add_node(nodename=nodename, func=func)

Then we generate the first network where edges have low probability.

>>> nodename = 'generatenetwork_lowprob'
>>> func = 'generatenetwork'
>>> params = {
    'networktype': 'rand_binomial',
    'size': (10,5),
    'prob': (0.25,0.25),
    'randomseed': 2019
    }
>>> twf.add_node(nodename, func, params=params)

Then add the calculate degree step.

>>> nodename = 'degree_lowprob'
>>> func = 'calc_networkmeasure'
>>> params = {
    'networkmeasure': 'temporal_degree_centrality',
    'calc': 'time'
    }
>>> twf.add_node(nodename, func, params=params)

Now we generate a second network where edges have higher probability. Here depends_on is called and refers back to the create_temporalnetwork node.

>>> nodename = 'generatenetwork_highprob'
>>> func = 'generatenetwork'
>>> depends_on = 'create_temporalnetwork'
>>> params = {
    'networktype': 'rand_binomial',
    'size': (10,5),
    'prob': (0.75,0.1),
    'randomseed': 2019
    }
>>> twf.add_node(nodename, func, depends_on, params)

Now we can calculate temporal degree centrality on this network:

>>> nodename = 'degree_highprob'
>>> func = 'calc_networkmeasure'
>>> params = {
    'networkmeasure': 'temporal_degree_centrality',
    'calc': 'time'
    }
>>> twf.add_node(nodename, func, params=params)

And this workflow can be plotted like before:

>>> fig, ax = twf.make_workflow_figure()
>>> fig.show()

(Source code, png, hires.png, pdf)

_images/workflow-2.png

TenetoBIDS

TenetoBIDS allows use of Teneto functions to analyse entire datasets in just a few lines of code. The output from Teneto is then ready to be placed in statistical models, machine learning algorithms and/or plotted.

Prerequisites

To use TenetoBIDS you need preprocessied fMRI data in the BIDS format. It is tested and optimized for fMRIPrep but other preprocessing software following BIDS should (in theory) work too. For fMRIPrep V1.4 or later is requiresd. This preprocessed data should be in the ~BIDS_dir/derivatives/ directory. The output from teneto will always be found in …/BIDS_dir/derivatives/ in directories that begin with teneto- (depending on the function you use).

Contents of this tutorial

This tutorial will run a complete analysis on some test data.

For this tutorial, we will use some dummy data which is included with teneto. This section details what is in this data.

[1]:
import teneto
import os
dataset_path = teneto.__path__[0] + '/data/testdata/dummybids/'
print(os.listdir(dataset_path))
print(os.listdir(dataset_path + '/derivatives'))

['participants.tsv', 'dataset_description.json', 'sub-001', 'derivatives', 'sub-002']
['teneto-censor-timepoints', 'teneto-derive-temporalnetwork', 'teneto-volatility', 'teneto-exclude-runs', 'teneto-tests', 'teneto-make-parcellation', 'fmriprep', 'teneto-binarize', 'teneto-remove-confounds']

From the above we can see that there are two subjects in our dataset, and there is a fMRIPrep folder in the derivatives section. Only subject 1 has any dummy data, so we will have to select subject 1.

A complete analysis

Below is a complete analysis of this test data. We will go through each step after it.

[2]:

#Imports.
from teneto import TenetoBIDS
from teneto import __path__ as tenetopath
import numpy as np
#Set the path of the dataset.
datdir = tenetopath[0] + '/data/testdata/dummybids/'

# Step 1:
bids_filter = {'subject': '001',
               'run': 1,
               'task': 'a'}
tnet = TenetoBIDS(datdir, selected_pipeline='fmriprep', bids_filter=bids_filter, exist_ok=True)

# Step 2: create a parcellation
parcellation_params = {'atlas': 'Schaefer2018',
                       'atlas_desc': '100Parcels7Networks',
                       'parc_params': {'detrend': True}}
tnet.run('make_parcellation', parcellation_params)

# Step 3: Regress out confounds
remove_params = {'confound_selection': ['confound1']}
tnet.run('remove_confounds', remove_params)

# Step 4: Additonal preprocessing
exclude_params = {'confound_name': 'confound1',
                   'exclusion_criteria': '<-0.99'}
tnet.run('exclude_runs', exclude_params)
censor_params = {'confound_name': 'confound1',
                   'exclusion_criteria': '<-0.99',
                   'replace_with': 'cubicspline',
                   'tol': 0.25}
tnet.run('censor_timepoints', censor_params)

# Step 5: Calculats time-varying connectivity
derive_params = {'params': {'method': 'jackknife',
                            'postpro': 'standardize'}}
tnet.run('derive_temporalnetwork', derive_params)

# Step 6: Performs a binarization of the network
binaraize_params = {'threshold_type': 'percent',
                    'threshold_level': 0.1}
tnet.run('binarize', binaraize_params)

# Step 7: Calculate a network measure
measure_params = {'distance_func': 'hamming'}
tnet.run('volatility', measure_params)

# Step 8: load data
vol = tnet.load_data()
print(vol)

{'sub-001_run-1_task-a_vol.tsv':           0
0  0.103733}

Big Picture

While the above code may seem overwhelming at first. It is quite little code for what it does. It starts with nifti images and ends with a single measure about a time-varying connectivity estimate of the network.

There is one recurring theme used in the code above:

tnet.run(function_name, function_parameters)

function_name is a string and function_parameters is a dictionary function_name can be most functions in teneto if the data is in the correct format. function_parameters are the inputs to that function. You never need to pass the input data (e.g. time series or network), or any functions that have a sidecar input.

TenetoBIDS will also automatically try and find a confounds file in the derivatives when needed, so, this does not need to be specified either.

Once you have grabbed the above, the rest is pretty straight forward. But we will go through each step in turn.

Step 1 - defining the TenetoBIDS object.

[3]:
#Set the path of the dataset.
datdir = tenetopath[0] + '/data/testdata/dummybids/'
# Step 1:
bids_filter = {'subject': '001',
               'run': 1,
               'task': 'a'}
tnet = TenetoBIDS(datdir, selected_pipeline='fmriprep', bids_filter=bids_filter, exist_ok=True)

selected_pipeline

**This states where teneto will go looking for files. This example shows it should look in the fMRIPrep derivative directory. (i.e. in: datadir + ‘/derivatives/fmriprep/’).

bids_filter

teneto uses pybids to select different files. The bids_filter argument is a dictionary of arguments that get passed into the BIDSLayout.get. In the example above, we are saying we want subject 001, run 1 and task a. If you do not provide any arguments for bids_filter, all data found within the derivatives folder gets selected for analyses.

exist_ok (default: False)

This checks that it is ok to overwrite any previous calculations. The output data is saved in a new directory. If the new output directory already exists, the teneto step has previously been run, and an error will be returned because otherwise data may be overwritten. To overrule this error, set exists_ok to True.

We can now look at what files are selected that will be run on the next step.

[4]:
tnet.get_selected_files()

[4]:
[<BIDSDataFile filename='/home/william/anaconda3/lib/python3.6/site-packages/teneto/data/testdata/dummybids/derivatives/fmriprep/sub-001/func/sub-001_task-a_run-01_desc-confounds_regressors.tsv'>,
 <BIDSImageFile filename='/home/william/anaconda3/lib/python3.6/site-packages/teneto/data/testdata/dummybids/derivatives/fmriprep/sub-001/func/sub-001_task-a_run-01_desc-preproc_bold.nii.gz'>]

If there are files here you do not want, you can add to the bids filter with tnet.update_bids_filter Or, you can set tnet.bids_filter to a new dictionary if you want.

Next, you might want to see what functions you can run on these selected files. The following will specify what functions can be run specifically on the selected data. If you want all options, you can add the for_selected=False.

[5]:
tnet.get_run_options()

[5]:
'make_parcellation, exclude_runs'

The output here (exclude_runs and make_parcellation) says which functions that, with the selected files, can be called in tnet.run. Once different functions have been called, the options change.

Step 2 Calling the run function to make a parcellation.

When selecting preprocessed files, these will often be nifti images. From these images, we want to make time-series of regions of interests. This can be done with :py:func:.make_parcellation. This function uses TemplateFlow atlases to make the parcellation.

[6]:
parcellation_params = {'atlas': 'Schaefer2018',
                       'atlas_desc': '100Parcels7Networks',
                       'parc_params': {'detrend': True}}
tnet.run('make_parcellation', parcellation_params)

The atlas and atlas_desc are used to identify TemplateFlow atlases.

Teneto uses nilearn’s NiftiLabelsMasker to mark the parcellation. Any arguments to this function (e.g. preprocessing steps) can be passed in the argument using ‘parc_params’ (here detrend is used).

Step 3 Regress out confounds

[7]:
remove_params = {'confound_selection': ['confound1']}
tnet.run('remove_confounds', remove_params)

Confounds can be removed by calling :py:func:.remove_confounds.

The confounds tsv file is automatically located as long as it is in a derivatives folder and that there is only one

Here ‘confound1’ is a column namn in the confounds tsv file.

Similarly to make parcellation, it uses nilearn (nilean.signal.clean. clean_params is a possible argument, like parc_params these are inputs to the nilearn function.

Step 4: Additonal preprocessing

[8]:
exclude_params = {'confound_name': 'confound1',
                   'exclusion_criteria': '<-0.99'}
tnet.run('exclude_runs', exclude_params)
censor_params = {'confound_name': 'confound1',
                   'exclusion_criteria': '<-0.99',
                   'replace_with': 'cubicspline',
                   'tol': 0.25}
tnet.run('censor_timepoints', censor_params)


These two calls to tnet.run exclude both time-points and runs, which are problematic. The first, exclude_runs, rejects any run where the mean of confound1 is less than 0.99. Excluded runs will no longer be part of the loaded data in later calls of tnet.run().

Centoring time-points here says that whenever there is a time-point that is less than 0.99, it will be “censored” (set to not a number). We have also set argument replace_with to ‘cubicspline’. This argument means that the values that have censored now get simulated using a cubic spline. The parameter tol says what percentage of time-points are allowed to be censored before the run gets ignored.

Step 5: Calculats time-varying connectivity

The code below now derives time-varying connectivity matrices. There are multiple different methods that can be called. See teneto.timeseries.derive_temporalnetwork for more options.

[9]:
derive_params = {'params': {'method': 'jackknife',
                            'postpro': 'standardize'}}
tnet.run('derive_temporalnetwork', derive_params)

Step 6: Performs a binarization of the network

Once you have a network representation, there are multiple ways this can be transformed. One example, is to binarize the network so all values are 0 or 1. The code below converts the top 10% of edges to 1s, the rest 0.

[10]:
binaraize_params = {'threshold_type': 'percent',
                    'threshold_level': 0.1}
tnet.run('binarize', binaraize_params)

Step 7: Calculate a network measure

We are now ready to calculate a property of the temproal network. Here we calculate volatility (i.e. how much the network changes per time-point). This generates one value per subject.

[11]:
measure_params = {'distance_func': 'hamming'}
tnet.run('volatility', measure_params)

Step 8: load data

[12]:
vol = tnet.load_data()
print(vol)

{'sub-001_run-1_task-a_vol.tsv':           0
0  0.103733}

Now that we have a measure of volatility for the network. We can now load it and view the measure.

TCTC

Backgorund

TCTC stands for Temporal Communities by Trajectory Clustering. It is an algorithm designed to find temporal communities on time series data.

The kind of data needed for TCTC are:

  1. Multiple time series.
  2. The time series are from nodes in a network.

Most community detection requires to first create an “edge inference” step where the edges of the different nodes are first calculated.

TCTC first finds clusters of trajectories in the time series without inferring edges. A trajectory is a time series moving through some space. Trajectory clustering tries to group together nodes that have similar paths through a space.

The hyperparameters of TCTC dictate what type of trajectory is found in the data. There are four hyperparameters:

  1. A maximum distance parameter (\(\epsilon\)). The distance between all nodes part of the same trajectory must be \(\epsilon\) or lower.
  2. A minimum size parameter (\(\sigma\)). All trajectories must include at least \(\sigma\) many nodes.
  3. A minimum time parameter (\(\tau\)). All trajectories must persist for \(\tau\) time-points.
  4. A tolerance parameter (\(\kappa\)). \(\kappa\) consecutive “exception” time-points can exist before the trajectory ends.

Outline

This example shows only how TCTC is run and how the different hyperparameters effect the community detection. These hyperparameters can be trained (saved for another example).

Read more

TCTC is outlined in more detail in this article

TCTC - example

We will start by generating some data and importing everything we need.

[1]:
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from teneto.communitydetection import tctc
import pandas as pd

Failed to import duecredit due to No module named 'duecredit'
/home/william/anaconda3/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216, got 192
  return f(*args, **kwds)
/home/william/anaconda3/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/home/william/anaconda3/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
  return f(*args, **kwds)
[2]:
data = np.array([[0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 1, 2, 1],
    [0, 0, 0, 0, 1, 1, 1, 0, 2, 2, 2, 2, 1],
    [1, 0, 1, 1, 1, 1, 1, 1, 2, 2, 1, 0, 0], [-1, 0, 1, 1, 0, -1, 0, -1, 0, 2, 1, 0, -1]], dtype=float)
data = data.transpose()
np.random.seed(2019)
data += np.random.uniform(-0.2, 0.2, data.shape)

[3]:
# Lets have a look at the data
fig, ax = plt.subplots(1)
p = ax.plot(data)
ax.legend(p, [0,1,2,3])
ax.set_xlabel('time')
ax.set_ylabel('amplitude')
print(data.shape)


(13, 4)
_images/tutorial_tctc_4_1.png

There are two different outputs that TCTC can produce. TCTC allows for multilabel communities (i.e. the same node can belong to multiple communities). The output of TCTC can either be:

  1. As a binary array (dimensions: node,node,time) where each 1 designates that two nodes are in the same community.
  2. As a dataframe where each row is a community.

The default output is option one.

So let us run TCTC on the data we have above.

[4]:
parameters = {
    'epsilon': 0.5,
    'tau': 3,
    'sigma': 2,
    'kappa': 0
}
tctc_array = tctc(data, **parameters)
print(tctc_array.shape)

(4, 4, 13)

For now ignore the values in the “parameters” dictionary, we will go through that later.

In order to get the dataframe output, just add output=’df’.

[5]:
parameters = {
    'epsilon': 0.5,
    'tau': 3,
    'sigma': 2,
    'kappa': 0
}
tctc_df = tctc(data, **parameters, output='df')
print(tctc_df.head())


   community  start  end  size  length
0     [0, 1]      0    7   2.0       7
1     [2, 3]      1    4   2.0       3
2  [0, 1, 2]      4    7   3.0       3
3     [0, 2]      4   11   2.0       7
5     [2, 3]      9   12   2.0       3

Here we can see when the different communities start, end, the size, and the length.

Below we define a function which plots each community on the original data.

[6]:

def community_plot(df, data):
    nrows = int(np.ceil((len(df)+1)/2))
    fig, ax = plt.subplots(nrows, 2, sharex=True, sharey=True, figsize=(8, 2+nrows))
    ax = ax.flatten()
    p = ax[0].plot(data)
    ax[0].set_xlabel('time')
    ax[0].set_ylabel('amplitude')
    ax[0].set_title('Original data')

    for i, row in enumerate(df.iterrows()):
        ax[i+1].plot(data, alpha=0.15, color='gray')
        ax[i+1].plot(np.arange(row[1]['start'],row[1]['end']),data[row[1]['start']:row[1]['end'], row[1]['community']],color=plt.cm.Set2.colors[i])
        ax[i+1].set_title('Community: ' + str(i))

    plt.tight_layout()
    return fig, ax

fig, ax = community_plot(tctc_df, data)

_images/tutorial_tctc_11_0.png

The multiple community labels can be seed in 0 and 2 above. Where 2 contains three nodes and community 0 contains 2 nodes.

Changing the hyperparameters

Now we will rerun TCTC but change each of the parameters in turn and then display them on a community plot.

Changing \(\epsilon\)

If we make \(\epsilon\) larger, we will include more time series in a trajectory.

This however can mean that the communities you detect are less “connected” than if \(\epsilon\) was smaller

[7]:
parameters = {
    'epsilon': 1.5,
    'tau': 3,
    'sigma': 2,
    'kappa': 0
}
tctc_df_largeep = tctc(data, **parameters, output='df')
fig, ax = community_plot(tctc_df_largeep, data)


_images/tutorial_tctc_15_0.png

Changing \(\tau\)

If we make \(\tau\) larger, it requires that trajectories persist for more time points.

Shorter trajectories increase the change of more noisey connections.

[8]:
parameters = {
    'epsilon': 0.5,
    'tau': 2,
    'sigma': 2,
    'kappa': 0
}
tctc_df_shorttau = tctc(data, **parameters, output='df')
fig, ax = community_plot(tctc_df_shorttau, data)


_images/tutorial_tctc_17_0.png
[9]:
parameters = {
    'epsilon': 0.5,
    'tau': 5,
    'sigma': 2,
    'kappa': 0
}
tctc_df_longtau = tctc(data, **parameters, output='df')
fig, ax = community_plot(tctc_df_longtau, data)




_images/tutorial_tctc_18_0.png

Changing \(\sigma\)

If we make \(\sigma\) larger, it requires that more nodes are part of the trajectory.

Smaller values of \(\sigma\) will result in possible noiser connections.

[10]:
parameters = {
    'epsilon': 0.5,
    'tau': 3,
    'sigma': 3,
    'kappa': 0
}
tctc_df_longsigma = tctc(data, **parameters, output='df')
fig, ax = community_plot(tctc_df_longsigma, data)


_images/tutorial_tctc_20_0.png

Changing \(\kappa\)

If we make \(\kappa\) larger, it allows for that many number of “noisey” time-points to exist to see if the trajectory continues.

In the data we have been looking at, node 0 and 1 are close to each other except for time-point 7 and 10. If we let \(\kappa\) be 1, if will ignore these time-points and allow the trajectory to continue.

[11]:
parameters = {
    'epsilon': 0.5,
    'tau': 3,
    'sigma': 2,
    'kappa': 1
}
tctc_df_withkappa = tctc(data, **parameters, output='df')
fig, ax = community_plot(tctc_df_withkappa, data)

_images/tutorial_tctc_22_0.png

API

TemporalNetwork, TenetoBIDS and TenetoWorkflow

teneto.classes Package

Classes in Teneto

Classes
TenetoBIDS(bids_dir, selected_pipeline[, …]) Class for analysing data in BIDS.
TemporalNetwork([N, T, nettype, from_df, …]) A class for temporal networks.
TenetoWorkflow([remove_nonterminal_output])
Class Inheritance Diagram
Inheritance diagram of teneto.classes.bids.TenetoBIDS, teneto.classes.network.TemporalNetwork, teneto.classes.workflow.TenetoWorkflow

teneto.communitydetection

Louvain

make_consensus_matrix(com_membership, th=0.5)[source]

Makes the consensus matrix.

From multiple iterations, finds a consensus partition.

.
com_membership : array
Shape should be node, time, iteration.
th : float
threshold to cancel noisey edges
D : array
consensus matrix
make_temporal_consensus(com_membership)[source]

Matches community labels accross time-points.

Jaccard matching is in a greedy fashiong. Matching the largest community at t with the community at t-1.

Parameters:com_membership (array) – Shape should be node, time.
Returns:D – temporal consensus matrix using Jaccard distance
Return type:array
temporal_louvain(tnet, resolution=1, intersliceweight=1, n_iter=100, negativeedge='ignore', randomseed=None, consensus_threshold=0.5, temporal_consensus=True, njobs=1)[source]

Louvain clustering for a temporal network.

Parameters:
  • tnet (array, dict, TemporalNetwork) – Input network
  • resolution (int) – resolution of Louvain clustering ($gamma$)
  • intersliceweight (int) – interslice weight of multilayer clustering ($omega$). Must be positive.
  • n_iter (int) – Number of iterations to run louvain for
  • randomseed (int) – Set for reproduceability
  • negativeedge (str) – If there are negative edges, what should be done with them. Options: ‘ignore’ (i.e. set to 0). More options to be added.
  • consensus (float (0.5 default)) – When creating consensus matrix to average over number of iterations, keep values when the consensus is this amount.
Returns:

communities – node,time array of community assignment

Return type:

array (node,time)

Notes

References

teneto.communitymeasures

teneto.communitymeasures Package

Functions to quantify temporal communities

Functions
flexibility(communities) Amount a node changes community
allegiance(community) Computes allience of communities.
recruitment(temporalcommunities, …) Calculates recruitment in relation to static communities.
integration(temporalcommunities, …) Calculates the integration coefficient for each node.
promiscuity(communities) Calculates promiscuity of communities.
persistence(communities[, calc]) Persistence is the proportion of consecutive time-points that a temporal community is in the same community at the next time-point

teneto.networkmeasures

teneto.networkmeasures Package

Imports from networkmeasures

Functions
temporal_degree_centrality(tnet[, axis, …]) Temporal degree of network.
shortest_temporal_path(tnet[, steps_per_t, …]) Shortest temporal path
temporal_closeness_centrality([tnet, paths]) Returns temporal closeness centrality per node.
intercontacttimes(tnet) Calculates the intercontacttimes of each edge in a network.
volatility(tnet[, distance_func, calc, …]) Volatility of temporal networks.
bursty_coeff(data[, calc, nodes, …]) Calculates the bursty coefficient.[1][2]
fluctuability(netin[, calc]) Fluctuability of temporal networks.
temporal_efficiency([tnet, paths, calc]) Returns temporal efficiency estimate.
temporal_efficiency([tnet, paths, calc]) Returns temporal efficiency estimate.
reachability_latency([tnet, paths, rratio, calc]) Reachability latency.
sid(tnet, communities[, axis, calc, decay]) Segregation integration difference (SID).
temporal_participation_coeff(tnet[, …]) Calculates the temporal participation coefficient
topological_overlap(tnet[, calc]) Topological overlap quantifies the persistency of edges through time.
local_variation(data) Calculates the local variaiont of inter-contact times.
temporal_betweenness_centrality([tnet, …]) Returns temporal betweenness centrality per node.

teneto.plot

teneto.plot Package

Imports when importing plot

Functions
slice_plot(netin, ax[, nodelabels, …]) Fuction draws “slice graph”.
circle_plot(netIn, ax[, nodelabels, …]) Function draws “circle plot” and exports axis handles
graphlet_stack_plot(netin, ax[, q, cmap, …]) Returns matplotlib axis handle for graphlet_stack_plot.

teneto.timeseries

teneto.timeseries Package

Import functions from time series module

Functions
derive_temporalnetwork(data, params) Derives connectivity from the data.
postpro_pipeline(data, pipeline[, report]) Function to call multiple postprocessing steps.
postpro_fisher(data[, report]) Performs fisher transform on everything in data.
postpro_standardize(data[, report]) Standardizes everything in data (along axis -1).
postpro_boxcox(data[, report]) Performs box cox transform on everything in data.
remove_confounds(timeseries, confounds[, …]) Removes specified confounds using nilearn.signal.clean
gen_report(report[, sdir, report_name]) Generates report of derivation and postprocess steps in teneto.timeseries

teneto.trajectory

compression

Calculate compression of trajectory.

create_traj_ranges(start, stop, N)[source]

Fills in the trajectory range.

# Adapted from https://stackoverflow.com/a/40624614

rdp(datin, delta=1, report=10, quiet=True)[source]

Module contents

Trajectory module

teneto.utils

teneto.utils Package

Many helper functions for Teneto

Functions
graphlet2contact(tnet[, params]) Converts array representation to contact representation.
contact2graphlet(C) Converts contact representation to array representation.
binarize_percent(netin, level[, sign, axis]) Binarizes a network proprtionally.
binarize_rdp(netin, level[, sign, axis]) Binarizes a network based on RDP compression.
binarize_magnitude(netin, level[, sign]) Make binary network based on magnitude thresholding.
binarize(netin, threshold_type, threshold_level) Binarizes a network, returning the network.
set_diagonal(tnet[, val]) Generally diagonal is set to 0.
gen_nettype(tnet[, weightonly]) Attempts to identify what nettype input graphlet tnet is.
check_input(netin[, rasie_if_undirected, conmat]) This function checks that netin input is either graphlet (tnet) or contact (C).
get_distance_function(requested_metric) This function returns a specified distance function.
process_input(netin, allowedformats[, …]) Takes input network and checks what the input is.
clean_community_indexes(communityID) Takes input of community assignments.
multiple_contacts_get_values(C) Given an contact representation with repeated contacts, this function removes duplicates and creates a value
df_to_array(df, netshape, nettype[, start_at]) Returns a numpy array (snapshot representation) from thedataframe contact list
check_distance_funciton_input(…) Function checks distance_func_name, if it is specified as ‘default’.
get_dimord(measure[, calc, community]) Get the dimension order of a network measure.
get_network_when(tnet[, i, j, t, ij, logic, …]) Returns subset of dataframe that matches index
create_supraadjacency_matrix(tnet[, …]) Returns a supraadjacency matrix from a temporal network structure
df_drop_ij_duplicates(df)
tnet_to_nx(df[, t]) Creates undirected networkx object
is_jsonable(x) Check if a dict is jsonable.

Contributers

  • William Hedley Thompson
  • Peter Fransson
  • Vatika Harlalka

Contribute to teneto?

Found a bug or want to add a feature? Feel free to contribute! Open up an issue on github with a suggestion/fix and then leave a pull request to submit your code.

At the github page you can find suggested enhancements that you can contribute to: https://github.com/wiheto/teneto/issues

Suggestions of other things that need to be added:

  • Control theory.
  • Weighted shortest paths.
  • More network measures.
  • More derive_temporalnetwork alternatives.
  • Null models.
  • Better documentation/tutorials
  • More plot alternatives
  • Complete HDF5 compatibility
  • Freesurfer output in TenetoBIDS
  • Implement continous time for all networkmeasures (where possible)

FAQ

What is the dimension order for dense arrays in Teneto?

Inputs/outputs in Teneto can be in both Numpy arrays (time series or temporal works) or Pandas Dataframes (time series). The default dimension order runs from node to time. This means that if you have a temporal network array in Teneto, than the array should have the dimension order (node,node,time). If using time series than the dimension order (node,time). This entails that the nodes are the rows in a pandas array and the time-points are the columns. Different software can organize their dimension orders differently (e.g. Nilearn uses a time,node dimension order).

Teneto

Documentation Status PyPI version Build Status Codacy Badge Coverage Status DOI

Temporal network tools.

What is the package

Package includes various tools for analyzing temporal network data. Temporal network measures, temporal network generation, derivation of time-varying/dynamic connectivities, plotting functions.

Some extra focus is placed on neuroimaging data (e.g. compatible with BIDS - NB: currently not compliant with latest release candidate of BIDS Derivatives).

Installation

With pip installed:

pip install teneto

to upgrade teneto:

pip install teneto -U

Requires: Python 3.6+

Installing teneto via pip installs all python package requirements as well.

Documentation

More detailed documentation can be found at teneto.readthedocs.io and includes tutorials.

Outlook

This package is under active development. And a lot of changes will still be made.

Contributers

For a list of contributors to teneto, see: teneto.readthedocs.io

Cite

If using this, please cite us. At present we do not have a dedicated article about teneto, but you can cite the software using the Zenodo DOI and/or the article where teneto is introduced, along with a considerable discussion about many of the measures in teneto:

Thompson et al (2017) “From static to temporal network theory applications to functional brain connectivity.” Network Neuroscience, 2: 1. p.69-99 Link