With the increasing development of neuromorphic platforms and their related software tools as well as the increasing scale of spiking neural network (SNN) models, there is a pressure for interoperable and scalable representations of network state. In response to this, we discuss a parallel extension of a widely used format for efficiently representing sparse matrices, the compressed sparse row (CSR), in the context of supporting the simulation and serialization of large-scale SNNs. Sparse matrices for graph adjacency structure provide a natural fit for describing the connectivity of an SNN, and prior work in the area of parallel graph partitioning has developed the distributed CSR (dCSR) format for storing and ingesting large graphs. We contend that organizing additional network information, such as neuron and synapse state, in alignment with its adjacency as dCSR provides a straightforward partition-based distribution of network state. For large-scale simulations, this means each parallel process is only responsible for its own partition of state, which becomes especially useful when the size of an SNN exceeds the memory resources of a single compute node. For potentially long-running simulations, this also enables network serialization to and from disk (e.g. for checkpoint/restart fault-tolerant computing) to be performed largely independently between parallel processes. We also provide a potential implementation, and put it forward for adoption within the neural computing community.
翻译:随着神经形态计算平台及其相关的软件工具的不断发展,尖峰神经网络(SNN)模型规模的增加,对网络状态进行互操作和扩展表示的压力也越来越大。为此,我们讨论了广泛使用的用于高效表示稀疏矩阵的格式——压缩稀疏行(CSR)的并行扩展,在大规模SNN的模拟和串行化支持方面进行了阐述。对于描述SNN连接性的稀疏矩阵,其图结构提供了天然的适应性,先前在并行图分区领域的工作开发出了用于存储和摄入大图的分布式CSR(dCSR)格式。我们认为,相对应的神经元和突触状态的其他网络信息组织与其邻接矩阵一起作为dCSR提供了一个简单的基于分区的网络状态分布。对于大规模仿真,这意味着每个并行进程仅负责其自己的状态分区,当一个SNN的大小超过单个计算节点的内存资源时,这一点尤其有用。对于潜在的长时间运行的仿真,这也使得网络串行化到和从磁盘(例如,用于检查点/重启容错计算)可以在并行进程之间大部分独立地完成。我们还提供了一个潜在的实现,并提出将其用于神经计算社区的采用。