SpringerLink
Log in

Language model-accelerated deep symbolic optimization

  • S.I.: Adaptive and Learning Agents 2022 (ALA 2022)
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Symbolic optimization methods have been used to solve varied challenging and relevant problems such as symbolic regression and neural architecture search. However, the current state of the art typically learns each problem from scratch and is unable to leverage pre-existing knowledge and datasets that are available for many applications. Inspired by the similarity between sequence representations learned in natural language processing and the formulation of symbolic optimization as a discrete sequence optimization problem, we propose language model-accelerated deep symbolic optimization (LA-DSO), a method that leverages language models to learn symbolic optimization solutions more efficiently. We demonstrate LA-DSO in two tasks: symbolic regression, which allows us to perform extensive experimentation due to its low computation requirements, and computational antibody optimization, which shows that our proposal accelerates learning in challenging real-world problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://www.wikipedia.org/

  2. https://dumps.wikimedia.org

References

  1. Lu Q, Ren J, Wang Z (2016) Using genetic programming with prior formula knowledge to solve symbolic regression problem. Comput Intell Neurosci 35:33985–33998

    Google Scholar 

  2. Yu K, Sciuto C, Jaggi M, Musat C, Salzmann M (2020) Evaluating the search phase of neural architecture search. In: International conference on learning representations (ICLR)

  3. Kitzelmann E (2009) Inductive programming: a survey of program synthesis techniques. In: Workshop on approaches and applications of inductive programming, pp 50–73

  4. Petersen BK, Landajuela M, Mundhenk TN, Santiago CP, Kim SK, Kim JT (2021) Deep symbolic regression: recovering mathematical expressions from data via risk-seeking policy gradients. In: Proceeding of the international conference on learning representations (ICLR)

  5. Silva FLD, Costa AHR (2019) A survey on transfer learning for multiagent reinforcement learning systems. J Artif Intell Res (JAIR) 64:645–703

    Article  MathSciNet  MATH  Google Scholar 

  6. Barto AG, Thomas PS, Sutton RS (2017) Some recent applications of reinforcement learning. In: Proceedings of the eighteenth Yale workshop on adaptive and learning systems

  7. Udrescu S-M, Tan A, Feng J, Neto O, Wu T, Tegmark M (2020) Ai Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity. Adv Neural Inf Process Syst 33:4860–4871

    Google Scholar 

  8. Brunton SL, Proctor JL, Kutz JN (2016) Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc Natl Acad Sci 113(15):3932–3937

    Article  MathSciNet  MATH  Google Scholar 

  9. Koza JR (1994) Genetic programming as a means for programming computers by natural selection. Stat Comput 4:87–112

    Article  Google Scholar 

  10. Mundhenk T, Landajuela M, Glatt R, Santiago C, Petersen B et al (2021) Symbolic regression via deep reinforcement learning enhanced genetic programming seeding. Adv Neural Inf Process Syst 34:24912

    Google Scholar 

  11. Landajuela M, Lee CS, Yang J, Glatt R, Santiago CP, Aravena I, Mundhenk T, Mulcahy G, Petersen BK (2022) A unified framework for deep symbolic regression. Adv Neural Inf Process Syst 35:33985–33998

    Google Scholar 

  12. Landajuela M, Petersen BK, Kim S, Santiago CP, Glatt R, Mundhenk N, Pettit JF, Faissol D (2021) Discovering symbolic policies with deep reinforcement learning. In: International conference on machine learning (ICML). PMLR, pp 5979–5989

  13. Pettit JF, Petersen BK, Cockrell C, Larie DB, Silva FL, An G, Faissol DM (2021) Learning sparse symbolic policies for sepsis treatment. In: Interpretable machine learning in healthcare workshop at ICML

  14. Glatt R, Silva FLd, Bui VH, Huang C, Xue L, Wang M, Chang F, Murphey Y, Su W (2022) Deep symbolic optimization for electric component sizing in fixed topology power converters. In: Workshop on AI for design and manufacturing (ADAM)

  15. Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Conference of the North American chapter of the association for computational linguistics: human language technologies, (NAACL-HLT), pp 4171–4186

  16. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(140):1–67

    MathSciNet  MATH  Google Scholar 

  17. Pilehvar MT, Camacho-Collados J (2020) Embeddings in natural language processing: theory and advances in vector representations of meaning. Synth Lect Hum Lang Technol 13(4):1–175

    Article  MATH  Google Scholar 

  18. Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big data 3(1):1–40

    Article  Google Scholar 

  19. White DR, Mcdermott J, Castelli M, Manzoni L, Goldman BW, Kronberger G, Jaśkowski W, O’Reilly U-M, Luke S (2013) Better GP benchmarks: community survey results and proposals. Genet Program Evolvable Mach 14(1):3–29

    Article  Google Scholar 

  20. Meurer A, Smith CP, Paprocki M, Čertík O, Kirpichev SB, Rocklin M, Kumar A, Ivanov S, Moore JK, Singh S et al (2017) Sympy: symbolic computing in python. PeerJ Comput Sci 3:103

    Article  Google Scholar 

  21. Mikolov T, Karafiát M, Burget L, Černockỳ J, Khudanpur S (2010) Recurrent neural network based language model. In: Eleventh annual conference of the international speech communication association

  22. Uy NQ, Hoai NX, O’Neill M, McKay RI, Galván-López E (2011) Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genet Program Evolvable Mach 12(2):91–119

    Article  Google Scholar 

  23. Wu TT, Kabat EA (1970) An analysis of the sequences of the variable regions of Bence Jones proteins and myeloma light chains and their implications for antibody complementarity. J Exp Med 132(2):211–250

    Article  Google Scholar 

  24. Carter PJ (2006) Potent antibody therapeutics by design. Nat Rev Immunol 6(5):343–357

    Article  Google Scholar 

  25. Norman RA, Ambrosetti F, Bonvin AMJJ, Colwell LJ, Kelm S, Kumar S, Krawczyk K (2019) Computational approaches to therapeutic antibody design: established methods and emerging trends. Brief Bioinform 21(5):1549–1567

    Article  Google Scholar 

  26. Desautels T, Zemla A, Lau E, Franco M, Faissol D (2020) Rapid in silico design of antibodies targeting SARS-CoV-2 using machine learning and supercomputing. BioRxiv

  27. Leaver-Fay A, Tyka M, Lewis SM, Lange OF, Thompson J, Jacak R, Kaufman KW, Renfrew PD, Smith CA, Sheffler W et al (2011) ROSETTA3: an object-oriented software suite for the simulation and design of macromolecules. Methods Enzymol 487:545–574

    Article  Google Scholar 

  28. Barlow KA, Ó Conchúir S, Thompson S, Suresh P, Lucas JE, Heinonen M, Kortemme T (2018) Flex ddG: Rosetta ensemble-based estimation of changes in protein-protein binding affinity upon mutation. J Phys Chem B 122(21):5389–5399

    Article  Google Scholar 

  29. Snelson E, Ghahramani Z (2006) Sparse Gaussian processes using pseudo-inputs. Adv Neural Inf Process Syst 18:1257

    Google Scholar 

  30. Sui J, Li W, Murakami A, Tamin A, Matthews LJ, Wong SK, Moore MJ, Tallarico ASC, Olurinde M, Choe H et al (2004) Potent neutralization of severe acute respiratory syndrome (SARS) coronavirus by a human mAb to S1 protein that blocks receptor association. Proc Natl Acad Sci 101(8):2536–2541

    Article  Google Scholar 

  31. Walls AC, **ong X, Park Y-J, Tortorici MA, Snijder J, Quispe J, Cameroni E, Gopal R, Dai M, Lanzavecchia A et al (2019) Unexpected receptor functional mimicry elucidates activation of coronavirus fusion. Cell 176(5):1026–1039

    Article  Google Scholar 

  32. Zhu Z, Chakraborti S, He Y, Roberts A, Sheahan T, **ao X, Hensley LE, Prabakaran P, Rockx B, Sidorov IA et al (2007) Potent cross-reactive neutralization of SARS coronavirus isolates by human monoclonal antibodies. Proc Natl Acad Sci 104(29):12123–12128

    Article  Google Scholar 

  33. Suzek BE, Wang Y, Huang H, McGarvey PB, Wu CH (2014) The UniProt Consortium: UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches. Bioinformatics 31(6):926–932

    Article  Google Scholar 

  34. Steinegger M, Mirdita M, Söding J (2018) Protein-level assembly increases protein sequence recovery from metagenomic samples manyfold. bioRxiv

  35. Vashchenko D, Nguyen S, Goncalves A, Silva FLd, Petersen B, Desautels T, Faissol D (2022) AbBERT: learning antibody humanness via masked language modeling. In: Workshop on healthcare AI and Covid-19

  36. Olsen TH, Boyles F, Deane CM (2022) Observed antibody space: a diverse database of cleaned, annotated, and translated unpaired and paired antibody sequences. Protein Sci 31(1):141–146

    Article  Google Scholar 

  37. Azunre P (2021) Transfer learning for natural language processing. Simon and Schuster

  38. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Empirical methods in natural language processing (EMNLP), pp 1532–1543

  39. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. ar**v:1301.3781 [cs.CL]

  40. ...Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler DM, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D (2000) Language models are few-shot learners (2020). ar**v:2005.14165 [cs.CL]

  41. Valipour M, You B, Panju M, Ghodsi A. SymbolicGPT: A generative transformer model for symbolic regression. ar**v:2106.14131

  42. Reid M, Yamada Y, Gu SS (2022) Can wikipedia help offline reinforcement learning? ar**v:2201.12122

  43. Chai D, Wu W, Han Q, Wu F, Li J (2020) Description based text classification with reinforcement learning. In: International conference on machine learning (ICML), pp 1371–1382

  44. Luketina J, Nardelli N, Farquhar G, Foerster J, Andreas J, Grefenstette E, Whiteson S, Rocktäschel T (2019) A survey of reinforcement learning informed by natural language. In: International joint conference on artificial intelligence (IJCAI), pp 6309–6317

  45. Bahdanau D, Hill F, Leike J, Hughes E, Kohli P, Grefenstette E (2019) Learning to understand goal specifications by modelling reward. In: International conference on learning representations (ICRL)

  46. Narasimhan K, Barzilay R, Jaakkola T (2018) Grounding language for transfer in deep reinforcement learning. J Artif Intell Res (JAIR) 63(1):849–874

    Article  MathSciNet  MATH  Google Scholar 

  47. Bahdanau D, Hill F, Leike J, Hughes E, Kohli P, Grefenstette E (2018) Learning to follow language instructions with adversarial reward induction. ar**v:1806.01946

  48. Yu H, Zhang H, Xu W (2018) Interactive grounded language acquisition and generalization in a 2D world. In: International conference on learning representations (ICLR)

  49. Hermann KM, Hill F, Green S, Wang F, Faulkner R, Soyer H, Szepesvari D, Czarnecki WM, Jaderberg M, Teplyashin D, Wainwright M, Apps C, Hassabis D, Blunsom P (2017) Grounded language learning in a simulated 3D world. ar**v:1706.06551

  50. Kim JT, Larma ML, Petersen BK (2021) Distilling wikipedia mathematical knowledge into neural network models. In: Mathematical reasoning in general artificial intelligence workshop

  51. Silva FLd, Goncalves A, Nguyen S, Vashchenko D, Glatt R, Desautels T, Landajuela M, Petersen B, Faissol D (2022) Leveraging language models to efficiently learn symbolic optimization solutions. In: Adaptive and learning agents (ALA) workshop

Download references

Acknowledgements

This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC. LLNL-JRNL-840809. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

Author information

Authors and Affiliations

  1. Lawrence Livermore National Laboratory, Livermore, CA, USA

    Felipe Leno da Silva, Andre Goncalves, Sam Nguyen, Denis Vashchenko, Ruben Glatt, Thomas Desautels, Mikel Landajuela, Daniel Faissol & Brenden Petersen

Authors
  1. Felipe Leno da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andre Goncalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sam Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Denis Vashchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruben Glatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Thomas Desautels
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mikel Landajuela
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Daniel Faissol
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Brenden Petersen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brenden Petersen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Evaluation of simple-LM baseline

Evaluation of simple-LM baseline

Figures 10 and 11 show the comparison between the best and average performance of expressions found by simple-LM against the performance of the DSO baseline and LA-DSO. Simple-LM unsurprisingly greatly underperforms in the best expression found (the main metric) for all the benchmarks. This happens because this baseline algorithm can only sample based on how probable a token is in equations “in general,” without regard to the specific problem being solved at the time. This results in a decent sampling average reward, but the algorithm fails to identify correct expressions for the problem at hand. Therefore, Simple-LM is not useful for our applications of interest.

Fig. 10
figure 10

Learning curves for the symbolic regression domain (Nguyen 1–6). Graphs in the left show the reward from the best token sequence found so far and the ones in the right show the average of rewards on sampled sequences in a particular iteration

Full size image
Fig. 11
figure 11

Learning curves for the symbolic regression domain (Nguyen 7–12). Graphs in the left show the reward from the best token sequence found so far and the ones in the right show the average of rewards on sampled sequences in a particular iteration

Full size image

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

da Silva, F.L., Goncalves, A., Nguyen, S. et al. Language model-accelerated deep symbolic optimization. Neural Comput & Applic (2023). https://doi.org/10.1007/s00521-023-08802-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00521-023-08802-8

Keywords

Search

Navigation