Brain Simulation Library on HPC Platforms and Brain Experimentation Support

Motivation for Brain Emulation

The United States National Academy of Engineers has already classified brain emulation as one of the Grand Engineering Challenges [1] Brain emulation in-silico is a relevant research field for various reasons:

  • The immediate benefit of brain emulation is the greater understanding of brain behavior by simulations based on biologically plausible models Depending on the complexity of the model it can provide insight on single-cell behavior to network dynamics of whole brain regions without the need for in-vivo experiments This can greatly accelerate brain experimentation and the understanding of the biological mechanisms
  • One important eventual goal of the field is brain rescue If brain function can be emulated in-silico accurately enough and in real time. it can lead to brain prosthetics and implants that can recover brain functionality lost due to health conditions and accidents.

The general goal of the project is to apply high performance. innovative solutions enable large scale. accurate or real-time brain simulations and enhance experimental setups or data analysis of the experimental data concerning brain research Thus. our activities employ a multitude of HPC and other technologies such as FPGAs. Dataflow Computing GPUs and Many-core processors The long term goal and main focus of the effort within this theme is the development of a generic tooling framework for accelerated brain simulations the BrainFrame Framework

The topics currently worked upon within the BrainFrame research theme are:

  • Acceleration of large-scale brain simulations: Focused mainly on cerebellar models, exploring the use of various HPC node technologies such as FPGAs, GPUs, dataflow engines and many-core processors (Xeon Phis) for delivering largely scalable, high-speed simulation platforms.
  • Brain rescue and brain-machine interfaces: Focused mainly on cerebellar models, attempting brain rescue by closing the loop between faulty brain regions and biologically plausible, computational brain models.
  • Powerful, neuroscience-friendly modeling/simulation toolflow: Exploration and development of both the PyNN front-end and hardware-based back-end for the BrainFrame tooling solution.
  • Improved neuronal recordings and feedback: Using hardware acceleration for improved signal analysis of single-neuron recording electrodes as well as for enabling closed-loop brain-machine interfaces (i.e. providing brain feedback in relevant brain time scales).
  • Employing current technologies to serve the neuroscientific effort: Research and development of innovative solutions for brain experiments and advance data analysis of online or offline scientific data.

The BrainFrame Framework

Depending on the desired model characteristics, we identify two general types of simulations that are relevant in neuroscientific experiments. The first one (TYPE-I) has to do with highly accurate (biophysically accurate and even accurate to the molecular level) models of smaller-sized networks (>100 and <1000) that requires real-time or close to real-time performance. The second type involves the simulation of large- or very large-scale networks in which accuracy can often be relaxed.  These experiments attempt to simulate network sizes and connection densities closely resembling their biological counterparts (TYPE-II experiments – over 1000 neurons). This, in combination to the variety of models commonly used, makes for a class of applications that vary greatly in terms of workload, while also, depending on the case, requiring high throughput, low latency or both. A single type of HPC fabric, either software- or hardware-based cannot cover all possible use cases with optimal efficiency.

A better approach is to provide scientists with an acceleration platform that has the ability to adjust to the aforementioned variety of workload characteristics. A heterogeneous system that integrates multiple HPC technologies, instead of just one, would be able to provide this. In addition, a framework for a heterogeneous system using a popular user interface for all integrated technologies can also provide the ability to select a different accelerator, depending on availability, cost and performance desired. Such a hardware back-end must overcome additional challenges to be used in the field. It requires a front-end which should provide two crucial features:

  • An easy and commonly used interface through which neuroscientists can employ the platform, without the constant mediation of an engineer.
  • A front-end that can reuse the vast amount of models already available to the community.

The eventual goal of the acceleration effort is creating such a heterogeneous back-end, based on Maxeler DFE, Xeon PHI and GPGPU technologies. This backend is combined with a PyNN front-end to implement  the BrainFrame tool-flow.  PyNN, a Python-based, simulator-independent language for specification of brain models, is a widely known and used framework by computational neuroscientists. PyNN is capable of achieving high-speed simulation and it already offers a common interface to popular simulation platforms such as NEURON and NEST as well as newly developing ones that show great future potential, such as NeuroML.

Proof-of-concept: The Inferior Olive Model on BrainFrame

The Olivocerebellar is one of the most complex areas of the brain and plays an important role in motor control.

The Olivocerebellar circuitry is a relatively well-charted region of the brain (see figure on the left). Its brain structure is highly repetitive and basically consists of the granule-cell layer (GCL), Purkinje-cell layer (PC), deep-cerebellar-nuclei (DCN), and inferior-olive (IO) nuclei.

To prototype the frameWork we use a state-of-the-art, extended Hodgkin-Huxley (biophysically-meaningful) model of the inferior-olivary nucleus (abbrev. InfOli), made by de Gruijl et al. [2], as a benchmark to evaluate the framework. We chose this model as a respective workload of such neuron representations, as their efficient simulation poses a significant engineering challenge. Even though this model is not the most biophysically accurate represen tation in the field, it is one of the most accepted and widely used models for brain simulations. We evaluated BrainFrame using three distinct instances of the workload, each differentiated by the presence and complexity of the neuron interconnectivity modeling, leading to vastly different computational requirements, while still reflecting realistic neuroscientific experiments.

Our experiments clearly showed the use of a heterogeneous platform. Just with a few use-cases with a single model there was enough variety in efficiency between the platforms so that none was able to provide optimal efficiency for all possible cases on its own. As can be seen below especially for TYPE-II experiments, the efficiency of each technology is heavily dependent on the experiment and all three accelerator fabrics are used to provide the best performance for entirety of networks tested and all considered connectivity densities (%) [3].


[1]  National Academy of Engineering (, Grand Challenges for Engineering, (2010).

[2] De Gruijl JR, Bazzigaluppi P, de Jeu MTG, De Zeeuw CI, “Climbing Fiber Burst Size and Olivary Sub-threshold Oscillations in a Network Setting“, PLoS Comput Biol 8(12): e1002814 (2012).

[3] Georgios SmaragdosGeorgios ChatzikonstantisRahul KukrejaHarry SidiropoulosDimitrios RodopoulosIoannis SourdisZaid Al-ArsChristoforos KachrisDimitrios SoudrisChris de Zeeuw and Christos Strydis, BrainFrame: A node-level heterogeneous accelerator platform for neuron simulations, IOP Journal of Neural Engineering, July 2017