SpringerLink
Log in

Hybrid CMOS/memristor crossbar structure for implementing hopfield neural network

  • Published:
Analog Integrated Circuits and Signal Processing Aims and scope Submit manuscript

Abstract

Competent hardware implementation of artificial neural networks is still an important contest. In recent literature, memristor has been introduced as a promising candidate for synapses implementation. However, integrating a memristive circuit with neuron hardware is auspicious research with challenging issues. In this paper, a scalable circuit-level hybrid CMOS/memristor hardware is introduced for implementing a Hopfield neural network. The proposed circuit is fully compatible with the crossbar structure. The performance of the proposed hardware is evaluated for different scales and compared with its software-based counterpart. Moreover, the accuracy of large-scale hardware for Hopfield neural network with 45 neurons and 4320 memristors is evaluated. It is demonstrated that the performance of the circuit is in the line of the software simulation. In comparison with similar works, the proposed circuit consumes 2000 times less energy and retrieves patterns 130 times faster. The implemented circuit is a step toward a general and feasible memristive hardware implementation for recurrent neural networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Yegnanarayana, B. (2009). Artificial neural networks. PHI Learning Pvt. Ltd.

  2. Wen, S., Yu, X., Zeng, Z., & Wang, J. (2016). Event-triggering load frequency control for multiarea power systems with communication delays. IEEE Transactions on Industrial Electronics, 63(2), 1308–1317. https://doi.org/10.1109/TIE.2015.2399394.

    Article  Google Scholar 

  3. Wen, S., Zeng, Z., Member, S., & Huang, T. (2015). Lag synchronization of switched neural networks via neural activation function and applications in image encryption. IEEE Transactions on Neural Networks and Learning Systems, 26(7), 1493–1502.

    Article  MathSciNet  Google Scholar 

  4. Pickett, M. D., & Stanley Williams, R. (2013). Phase transitions enable computational universality in neuristor-based cellular automata. Nanotechnology, 24(38), 384002. https://doi.org/10.1088/0957-4484/24/38/384002.

    Article  Google Scholar 

  5. Furber, S., & Temple, S. (2007). Neural systems engineering. Journal of the Royal Society, Interface/the Royal Society, 4(13), 193–206. https://doi.org/10.1098/rsif.2006.0177.

    Article  Google Scholar 

  6. Hopfield, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences, 81(10), 3088–3092. https://doi.org/10.1073/pnas.81.10.3088.

    Article  MATH  Google Scholar 

  7. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8), 2554–2558. https://doi.org/10.1073/pnas.79.8.2554.

    Article  MathSciNet  MATH  Google Scholar 

  8. Hopfield, J. J., & Tank, D. W. (1986). Computing with neural circuits: A model. Science (New York, NY), 233(4764), 625–633. https://doi.org/10.1126/science.3755256.

    Article  Google Scholar 

  9. Hopfield, J. J., & Tank, D. W. (1985). “Neural” computation of decisions in optimization problems. Biological Cybernetics, 52(3), 141–152. https://doi.org/10.1007/BF00339943.

    Article  MathSciNet  MATH  Google Scholar 

  10. Sulehria, H. K., & Zhang, Y. (2007). Hopfield neural networks: A survey. In Proceedings of the 6th Conference on 6th WSEAS Int. Conf. on Artificial Intelligence, Knowledge Engineering and Data Bases (Vol. 6, pp. 125–130). Citeseer.

  11. Shamsi, J., Amirsoleimani, A., Mirzakuchaki, S., & Ahmadi, M. (2017). Modular neuron comprises of memristor-based synapse. Neural Computing and Applications.. https://doi.org/10.1007/s00521-015-2047-0.

    Article  Google Scholar 

  12. Graf, H. P., Jackel, L. D., Howard, R. E., Straughn, B., Denker, J. S., Hubbard, W., et al. (1986). VLSI Implementation of a neural network memory with several hundreds of neurons. Neural Networks for Computing, 182(1986), 182–187. https://doi.org/10.1063/1.36253.

    Article  Google Scholar 

  13. Morishita, T., Tamura, Y., Otsuki, T., & Kano, G. (1992). A BiCMOS analog neural network with dynamically updated weights. IEICE Transactions on Electronics, 75(3), 297–302. https://doi.org/10.1109/ISSCC.1990.110167.

    Article  Google Scholar 

  14. Misra, J., & Saha, I. (2010). Artificial neural networks in hardware: A survey of two decades of progress. Neurocomputing, 74(1–3), 239–255. https://doi.org/10.1016/j.neucom.2010.03.021.

    Article  Google Scholar 

  15. Adhikari, S. P., Yang, C., Kim, H., & Chua, L. O. (2012). Memristor bridge synapse-based neural network and its learning. IEEE Transactions on Neural Networks and Learning Systems, 23(9), 1426–1435. https://doi.org/10.1109/TNNLS.2012.2204770.

    Article  Google Scholar 

  16. Strukov, D. B., Snider, G. S., Stewart, D. R., & Williams, R. S. (2008). The missing memristor found. Nature, 453(7191), 80–83. https://doi.org/10.1038/nature06932.

    Article  Google Scholar 

  17. Hu, S. G., Liu, Y., Chen, T. P., Liu, Z., Yu, Q., Deng, L. J., et al. (2013). Emulating the paired-pulse facilitation of a biological synapse with a NiOx-based memristor. Applied Physics Letters, 102(18), 183510. https://doi.org/10.1063/1.4804374.

    Article  Google Scholar 

  18. Kim, S., Du, C., Sheridan, P., Ma, W., Choi, S., & Lu, W. D. (2015). Experimental demonstration of a second-order memristor and its ability to biorealistically implement synaptic plasticity. Nano Letters, 15(3), 2203–2211. https://doi.org/10.1021/acs.nanolett.5b00697.

    Article  Google Scholar 

  19. Kuzum, D., Yu, S., & Philip Wong, H. S. (2013). Synaptic electronics: Materials, devices and applications. Nanotechnology. https://doi.org/10.1088/0957-4484/24/38/382001.

    Article  Google Scholar 

  20. Li, Y., Zhong, Y., Xu, L., Zhang, J., Xu, X., Sun, H., et al. (2013). Ultrafast synaptic events in a chalcogenide memristor. Scientific Reports, 3(1), 1619. https://doi.org/10.1038/srep01619.

    Article  Google Scholar 

  21. Wang, J. L., Wu, H. N., Huang, T., Ren, S. Y., & Wu, J. (2017). Passivity analysis of coupled reaction-diffusion neural networks with Dirichlet boundary conditions. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(8), 2148–2159. https://doi.org/10.1109/TSMC.2016.2622363.

    Article  Google Scholar 

  22. Wen, S., Zeng, Z., & Huang, T. (2013). Dynamic behaviors of memristor-based delayed recurrent networks. Neural Computing and Applications, 23(3–4), 815–821. https://doi.org/10.1007/s00521-012-0998-y.

    Article  Google Scholar 

  23. Smagulova, K., Krestinskaya, O., & James, A. P. (2018). A memristor-based long short term memory circuit. Analog Integrated Circuits and Signal Processing, 95(3), 467–472. https://doi.org/10.1007/s10470-018-1180-y.

    Article  Google Scholar 

  24. Shamsi, J., Mohammadi, K., & Shokouhi, S. B. (2018). A hardware architecture for columnar-organized memory based on CMOS neuron and memristor crossbar arrays. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 26(12), 2795–2805. https://doi.org/10.1109/TVLSI.2018.2815025.

    Article  Google Scholar 

  25. Krestinskaya, O., Salama, K. N., & James, A. P. (2019). Learning in memristive neural network architectures using analog backpropagation circuits. IEEE Transactions on Circuits and Systems I: Regular Papers, 66(2), 719–732. https://doi.org/10.1109/TCSI.2018.2866510.

    Article  Google Scholar 

  26. Eshraghian, J. K., Cho, K., Zheng, C., Nam, M., Iu, H. H.-C., Lei, W., et al. (2018). Neuromorphic vision Hybrid RRAM-CMOS architecture. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 26(12), 2816–2829. https://doi.org/10.1109/TVLSI.2018.2829918.

    Article  Google Scholar 

  27. Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., & Lu, W. (2010). Nanoscale memristor device as synapse in neuromorphic systems. Nano Letters, 10(4), 1297–1301. https://doi.org/10.1021/nl904092h.

    Article  Google Scholar 

  28. Hu, S. G. G., Liu, Y., Liu, Z., Chen, T. P. P., Wang, J. J. J., Yu, Q., et al. (2015). Associative memory realized by a reconfigurable memristive Hopfield neural network. Nature Communications, 6(1), 7522. https://doi.org/10.1038/ncomms8522.

    Article  Google Scholar 

  29. Yang, J., Wang, L., Wang, Y., & Guo, T. (2017). A novel memristive Hopfield neural network with application in associative memory. Neurocomputing, 227, 142–148. https://doi.org/10.1016/j.neucom.2016.07.065.

    Article  Google Scholar 

  30. Duan, S., Dong, Z., Hu, X., Wang, L., & Li, H. (2016). Small-world Hopfield neural networks with weight salience priority and memristor synapses for digit recognition. Neural Computing and Applications, 27(4), 837–844. https://doi.org/10.1007/s00521-015-1899-7.

    Article  Google Scholar 

  31. Guo, X., Merrikh-Bayat, F., Gao, L., Hoskins, B. D., Alibart, F., Linares-Barranco, B., et al. (2015). Modeling and experimental demonstration of a hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits. Frontiers in Neuroscience, 9(12), 1–12. https://doi.org/10.3389/fnins.2015.00488.

    Article  Google Scholar 

  32. Hebb, D. O. (1949). The organization of behavior. A Neuropsychological Theory.,. https://doi.org/10.1002/1097-4679(195007)6:3<307:AID-JCLP2270060338>3.0.CO;2-K.

    Article  Google Scholar 

  33. Mceliece, R. J., Posner, E. C., Rodemich, E. R., & Venkatesh, S. S. (1987). The capacity of the hopfield associative memory. IEEE Transactions on Information Theory, 33(4), 461–482. https://doi.org/10.1109/TIT.1987.1057328.

    Article  MathSciNet  MATH  Google Scholar 

  34. Verleysen, M., & Jespers, P. G. A. (1989). An analog VLSI implementation of Hopfield’s neural network. IEEE Micro, 9(6), 46–55. https://doi.org/10.1109/40.42986.

    Article  Google Scholar 

  35. Chua, L. O. (1971). Memristor—The missing circuit element. IEEE Transactions on Circuit Theory, 18(5), 507–519. https://doi.org/10.1109/TCT.1971.1083337.

    Article  Google Scholar 

  36. Yakopcic, C., Taha, T. M., Subramanyam, G., & Pino, R. E. (2013). Generalized memristive device SPICE model and its application in circuit design. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 32(8), 1201–1214. https://doi.org/10.1109/TCAD.2013.2252057.

    Article  Google Scholar 

  37. Oblea, A. S., Timilsina, A., Moore, D., & Campbell, K. A. (2010). Silver chalcogenide based memristor devices. In Proceedings of the international joint conference on neural networks (pp. 1–3). IEEE. https://doi.org/10.1109/IJCNN.2010.5596775

  38. Miller, K., Nalwa, K. S., Bergerud, A., Neihart, N. M., & Chaudhary, S. (2010). Memristive behavior in thin anodic Titania. IEEE Electron Device Letters, 31(7), 737–739. https://doi.org/10.1109/LED.2010.2049092.

    Article  Google Scholar 

  39. Miller, K. (2010, January). Fabrication and modeling of thin-film anodic titania memristors. Iowa State University. Retrieved from https://lib.dr.iastate.edu/etd/11426

  40. Chabi, D., Wang, Z., Bennett, C., Klein, J.-O., & Zhao, W. (2015). Ultrahigh Density memristor neural crossbar for on-chip supervised learning. IEEE Transactions on Nanotechnology, 14(6), 954–962. https://doi.org/10.1109/TNANO.2015.2448554.

    Article  Google Scholar 

  41. Chen, L., Li, J., Chen, Y., Deng, Q., Shen, J., Liang, X., & Jiang, L. (2017). Accelerator-friendly neural-network training: Learning variations and defects in RRAM crossbar. In Proceedings of the 2017 design, automation and test in Europe, DATE 2017 (pp. 19–24). IEEE. https://doi.org/10.23919/DATE.2017.7926952

  42. Choi, S., Sheridan, P., & Lu, W. D. (2015). Data clustering using memristor networks. Scientific Reports, 5(1), 10492. https://doi.org/10.1038/srep10492.

    Article  Google Scholar 

  43. Gaba, S., Sheridan, P., Zhou, J., Choi, S., & Lu, W. (2013). Stochastic memristive devices for computing and neuromorphic applications. Nanoscale, 5(13), 5872. https://doi.org/10.1039/c3nr01176c.

    Article  Google Scholar 

  44. Rajendran, J., Karri, R., & Rose, G. S. (2015). Improving tolerance to variations in memristor-based applications using parallel memristors. IEEE Transactions on Computers, 64(3), 733–746. https://doi.org/10.1109/TC.2014.2308189.

    Article  MathSciNet  MATH  Google Scholar 

  45. Chen, C. Y., Shih, H. C., Wu, C. W., Lin, C. H., Chiu, P. F., Sheu, S. S., et al. (2015). RRAM defect modeling and failure analysis based on march test and a novel squeeze-search scheme. IEEE Transactions on Computers, 64(1), 180–190. https://doi.org/10.1109/TC.2014.12.

    Article  MathSciNet  MATH  Google Scholar 

  46. Zidan, M. A., Fahmy, H. A. H., Hussain, M. M., & Salama, K. N. (2013). Memristor-based memory: The sneak paths problem and solutions. Microelectronics Journal, 44(2), 176–183. https://doi.org/10.1016/j.mejo.2012.10.001.

    Article  Google Scholar 

  47. Liu, B., Li, H., Chen, Y., Li, X., Huang, T., Wu, Q., & Barnell, M. (2015). Reduction and IR-drop compensations techniques for reliable neuromorphic computing systems. In IEEE/ACM international conference on computer-aided design, digest of technical papers, ICCAD (Vol. 2015-Janua, pp. 63–70). IEEE. https://doi.org/10.1109/ICCAD.2014.7001330

  48. Liu, B., Li, H., Chen, Y., Li, X., Wu, Q., & Huang, T. (2015). Vortex: Variation-aware training for memristor X-bar. In Proceedings of the 52nd annual design automation conference on—DAC ’15 (pp. 1–6). New York, New York, USA: ACM Press. https://doi.org/10.1145/2744769.2744930

  49. Chakrabarti, B., Lastras-Montaño, M. A., Adam, G., Prezioso, M., Hoskins, B., Payvand, M., et al. (2017). A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuit. Scientific Reports, 7(1), 42429. https://doi.org/10.1038/srep42429.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran

    Mahdiyar Molahasani Majdabadi, Jafar Shamsi & Shahriar Baradaran Shokouhi

Authors
  1. Mahdiyar Molahasani Majdabadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jafar Shamsi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shahriar Baradaran Shokouhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shahriar Baradaran Shokouhi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Molahasani Majdabadi, M., Shamsi, J. & Baradaran Shokouhi, S. Hybrid CMOS/memristor crossbar structure for implementing hopfield neural network. Analog Integr Circ Sig Process 107, 249–261 (2021). https://doi.org/10.1007/s10470-020-01720-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10470-020-01720-y

Keywords

Search

Navigation