TinyCNN: A Simple Three-Block Convolutional Network for Time Series Classification
Abstract
Time series classification is one of the most broadly applicable problems in machine learning, spanning domains as diverse as medical monitoring, industrial fault detection, speech processing, and astronomy. While state-of-the-art methods such as ROCKET, Inception Time, and temporal Transformers achieve impressive accuracy on standard benchmarks, they do so at the cost of considerable model complexity, high parameter counts, or reliance on large numbers of random convolutional kernels that resist interpretation. This paper proposes TinyCNN, a deliberately minimal three-block one-dimensional convolutional network whose design follows a single guiding principle: each block applies a Conv1D layer followed by batch normalization, ReLU activation, and max pooling, and the three blocks together are succeeded by global average pooling and a two-layer fully connected classifier. The entire model contains approximately 48K trainable parameters and trains in roughly five seconds per epoch on a single GPU. Despite its simplicity, TinyCNN achieves the highest accuracy on all six benchmark datasets tested from the UCR and UEA time series archives, outperforming ROCKET by an average of 1.5 percentage points and Inception Time by 2.1 points while using 9.5× fewer parameters. Wilcoxon signed-rank tests over 30 UCR/UEA datasets confirm that the improvements over all baselines are statistically significant. An ablation study shows that global average pooling is more effective than flattening and that all three convolutional blocks contribute independently to final accuracy.
Downloads
References
[1] Dau, H.A., Bagnall, A., Kamgar, K., Yeh, C.M., Zhu, Y., Gharghabi, S., Ratanamahatana, C.A., Keogh, E. (2019). The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6), 1293–1305.
[2] Ruiz, A.P., Flynn, M., Large, J., Middlehurst, M., Bagnall, A. (2021). The great multivariate time series classification bake off: A review and experimental evaluation of recent algorithmic advances. Data Mining and Knowledge Discovery, 35(2), 401–449.
[3] Sakoe, H., Chiba, S. (1978). Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing, 26(1), 43–49.
[4] Wang, Z., Yan, W., Oates, T. (2017). Time series classification from scratch with deep neural networks: A strong baseline. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), 1578–1585.
[5] Middlehurst, M., Large, J., Flynn, M., Lines, J., Bostrom, A., Bagnall, A. (2021). HIVE-COTE 2.0: A new meta ensemble for time series classification. Machine Learning, 110(11), 3211–3243.
[6] Dempster, A., Petitjean, F., Webb, G.I. (2020). ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, 34(5), 1454–1495.
[7] Fawaz, H.I., Lucas, B., Forestier, G., Pelletier, C., Schmidt, D.F., Weber, J., Webb, G.I., Idoumghar, L., Muller, P.A., Petitjean, F. (2020). InceptionTime: Finding AlexNet for time series classification. Data Mining and Knowledge Discovery, 34(6), 1936–1962.
[8] He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778.
[9] Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., Eickhoff, C. (2021). A transformer-based framework for multivariate time series representation learning. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2114–2124.
[10] Dempster, A., Schmidt, D.F., Webb, G.I. (2021). MiniRocket: A very fast (almost) deterministic transform for time series classification. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 248–257.
[11] Nie, Y., Nguyen, N.H., Sinthong, P., Kalagnanam, J. (2023). A time series is worth 64 words: Long-term forecasting with transformers. International Conference on Learning Representations (ICLR 2023).
[12] Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., Long, M. (2024). iTransformer: Inverted transformers are effective for time series forecasting. International Conference on Learning Representations (ICLR 2024).
[13] Middlehurst, M., Schäfer, P., Bagnall, A. (2024). Bake off redux: A review and experimental evaluation of recent time series classification algorithms. Data Mining and Knowledge Discovery, 38(4), 1958–2031.
[14] Foumani, N.M., Miller, L., Tan, C.W., Webb, G.I., Salehi, M., Mathews, I. (2024). Improving position encoding of transformers for multivariate time series classification. Data Mining and Knowledge Discovery, 38(1), 22–48.
[15] Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., Long, M. (2023). TimesNet: Temporal 2D-variation modeling for general time series analysis. International Conference on Learning Representations (ICLR 2023).
[16] Zhang, T., Zhang, Y., Cao, W., Bian, J., Yi, X., Zheng, S., Li, J. (2023). Less is more: Fast multivariate time series forecasting with light sampling-oriented MLP structures. arXiv:2207.01186. Published in AAAI 2023.
[17] Wang, S., Wu, H., Shi, X., Hu, T., Luo, H., Ma, L., Zhang, J.Y., Zhou, J. (2024). TimeMixer: Decomposable multiscale mixing for time series forecasting. International Conference on Learning Representations (ICLR 2024).
[18] Hendrycks, D., Gimpel, K. (2016). Gaussian error linear units (GELUs). arXiv:1606.08415. Widely adopted since 2020 in transformer-based architectures.
[19] Bagnall, A., Dau, H.A., Lines, J., Flynn, M., Large, J., Bostrom, A., Southam, P., Keogh, E. (2018). The UEA multivariate time series classification archive, 2018. arXiv:1811.00075.
Published
Issue
Section
License
Copyright (c) 2026 Oussama Sadik, Redouane Bessioua, Mohamed Mohidine, Douha Tissir (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
