A tale of two toolkits, report the first: benchmarking time series classification algorithms for correctness and efficiency

sktime is an open source, Python based, sklearn compatible toolkit for time series analysis developed by researchers at the University of East Anglia, University College London and the Alan Turing Institute. A key initial goal for sktime was to provide time series classification functionality equivalent to that available in a related java package, tsml. We describe the implementation of six such classifiers in sktime and compare them to their tsml equivalents. We demonstrate correctness through equivalence of accuracy on a range of standard test problems and compare the build time of the different implementations. We find that there is significant difference in accuracy on only one of the six algorithms, and this difference was to be expected. We found a much wider range of difference in efficiency. Again, this was not unexpected, but it does highlight ways both toolkits could be improved. PLEASE NOTE THIS PAPER IS NOT COMPLETE. It is a work in progress and we have pushed it early so that we can reference it in another paper. More to follow!