Working bits and pieces of the Neurospaces / GENESIS documentation system

The Neurospaces / GENESIS 3 documentation system consists of a flat set of simple documents that are hyperlinked together. The main entry point into the documentation can be found here. The basic outline of the structure of the documentation can be found here. Most of the documentation is structured according to the GENESIS user workflow, a simple description of operations that a user executes when he simulates a model.

Today we had some fun identifying more flows through the documentation. As such there are vertical flows that go into depth, and horizontal flows that give an overview of concepts or technology. There is also a 'related pages' flow and a hierarchical flow (also known by software engineers as 'breadcrumbs').

The GENESIS documentation system has a convenient tagging system to implement all these flows. For example, the content page for level 1 documentation (high-level overviews), lists all published documents that have the named tag 'level1'. All the pages that are related to the model-container (allows for efficient internal storage of models), are tagged with 'related-model-container'. This makes it easy to identify and name different flows through the documentation and link those documents in a dynamically way using tools that understand the tags and create the appropriate HTML just-in-time.

By design, the documentation system also allows to tag and describe pages and documents that are not directly under our own control. For example, we can tag and link to scientific publications on other websites, or to documentation of software that is not maintained by us.

Anyway, as you can imagine, working on such a system is a lot of fun, especially with this team, and especially when everything starts to work...

Funding for continued GENESIS development

Last week we learned that GENESIS 3 development will receive additional funding. This funding was requested in the form of an administrative supplement to the original GENESIS 3 grant a couple of weeks ago. The money will be used to accelerate the development of documentation specific to GENESIS 3.

It is indeed a shame that there is so much software there, in the form of Neurospaces software components and software components specific to GENESIS 3, but it is still a mystery how to record channel response from a voltage clamp protocol, just one example. Not that this is difficult to do, there is actually a test case for it, but it is just hidden for users who are new to this system (meaning almost everybode out there).

The money of the administrative supplement should come in soon, in a couple of weeks. Both regular documentation, additional documentation infrastructure and tutorials will be developed.

Integration of solvers using Neurospaces

subject: a soliton solver and a compartmental solver


Just wanted to clarify something that was brought up yesterday during our journalclub.

So what I said was that it is a bad idea to integrate a soliton solver (fluid dynamics, hyperbolic partial differential equations) with a compartmental solver (parabolic partial differential equations), because the structure of the equations and their solutions is very different.

This does not mean that integration with neurospaces / G3 is a bad idea. The only thing to pay attention to, is that one has to keep the code base of the two solvers entirely separate, for ease of development / maintenance / testing. During simulation, both are coupled by an appropriate communication framework that comes with neurospaces / G3.

Such a situation is very close to the TMS stimulation engine that Reese and I have worked on.

This is very different from hsolve of G2 where all kinds of solvers and their internal communication mechanisms share a single entangled code base (and development ceased).

So again, it is just a matter of modular design,

Ah and BTW. before I forget to mention: this also facilitates optimization. I recently added a very simple optimizer to the SSP scheduler, and it runs about 30% faster now. Single neuron simulations on G3 run faster than on G2.

This will be posted on the neurospaces blog.

About a week ago, we have made available the monotone repositories of the Neurospaces Studio and the Project Browser. The installer script 'neurospaces_build' can now seemlessly pull from these repositories, and create a directory structure required for Neurospaces development. Preparing a developer PC can now be done in a couple of minutes. The distributed nature of the monotone version control system starts to play a crucial role for such functions to work correctly.

Over last weekend, I have also started working on a simple replacement for the Genesis 2 SLI. This new software component called GShell, allows to construct simple models, and then export them to the Neurospaces NDF format , and an SSP configuration file. This provides a basic interface to the Project Browser.

The GShell is basically agnostic about how to create a model. Instead of having hardcoded constructs for Neuroscience models, it pulls information from the model container, and transforms it using meta programming techniques to perl code that talks to the model container and allows to construct a model. A model container specific to air flow simulations would allow to create models of air flow using the GShell. Obviously the model container of the Neurospaces project only knows about elements of neuroscience models.

Because of the use of meta programming techniques, the source code of the GShell is only about 800 lines of code, and it can instantiate any model supported by the model container.

Mando continues his work on the GENESIS 2 SLI: for some time, the target has been to get the classic Purkinje cell model to work because it uses different types of channels, has a complex morphology, and obviously because its behavior is well described and links nicely with experiments that have been done in the past. It seems like quite soon this model will work, from using the vanilla GENESIS 2 scripts. And for sake of completeness, this model obviously works already for a long time using Neurospaces NDF model files and SSP stimulation configuration files.

Last week I have worked a fair bit on the project browser, such that the morphology analyzer built into the model container can be accessed. This means that a project can have say 10 different morphology files in .p files or NDF files, and your webbrowser allows to examine the number of branchpoints between soma and terminal tips, compute cell surface area and other things. These operations can be applied transparantly on individual morphologies or groups of morphologies. The part to define the groups from the morphologies in the project still needs some work, right now I use a configuration text file in the YAML format.

One thing that became more and more clear over the last few days is that it would be good from a maintenance point of view to merge the project browser and the Neurospaces studio. From the user / installer viewpoint, this means that the Neurospaces project only ships one integrated GUI rather than two, which is obviously a good point too.

Starting to mature

With the Neurospaces project maturing and becoming more useful, I decided to start blogging on tangible progress of the software development. The intent is to give a weekly update, for myself and other people interested and participating in the project.

People somewhat familiar with the project know that we are currently focusing on single neuron modeling. This does not mean that network modeling is not supported, but instead that we are focusing on a complete toolchain that allows scientists to build single neuron models from scratch, explore their behavior, compare the results between alternative models.

Over the last few months, the Neurospaces software toolchain has been used for all the functions mentioned in the previous paragraph by scientists doing scientific research. Multicompartmental models as well as single compartment models have been used, from passive morphologies til active morphologies showing spike frequency adaptation due to the calcium dynamics.

We hope to do a first alpha release early next year. And as said, this alpha release will focus on single neuron modeling.

More to follow.

Application software development

This is what I consider important when doing software development:

1. An application is (often) inherently limited by the technology used to develop the application. So before starting the development of an application, try to assess how flexible the application should be (in the future). Or better, try to anticipate on interfacing, and try to avoid (future) vendor or technology lockin. If interfacing of a technology is a potential future problem -- an example for Java based applications could be the complexity of having to deal with JNI -- then choose another technology. If you have enough money and human resources, interfacing is never a problem.
   
2. A brandnew application is best developed in an organic way. So not first design, then implement, that only works for applications we already know how to implement, and those applications can often be written in one day (so it is a configuration, not an application). An example of a 'configuration' is relational databasing, using Microsoft Studio, the development of such 'applications' is fully automated. If your true application needs good support for workflows, good support for customizability, talk to your (potential) users on a regular basis. Maintaining the appropriate data layering when an application grows, is one of the most important aspects of organic growth. So sometimes, organic style development means splitting one software component into several software components, as indicated by the nature of the data (don't mixin recordsets with workflow configuration data). OTOH, sometimes appropriate data layering is not possible, because there are no efficient data conversion algorithms to do the job (a couple of times I had to develop algorithms to do such conversions -- tricky job). In either case, organically grown applications must always be complemented by good tests, see below.
   
3. An application must be tested on user visible functionality (not: atomic unit tests, I have seen that go wrong quite often). Functional testing and integration testing have several aspects to the inner, like guarding consistency, but also to the outer, like defining workflows and as a guide for user documentation writers. So tests should be self-documenting for the documentation writers. The way I implement this last feature is by using a single set of test specifications, to run the tests, and to convert them to HTML.

I think of these three points as the most important things to keep in mind when developing a (brandnew) software application. I did not talk about backward compatibility, installation, deployment and roll-out. Then, there is obviously also sales and marketing.