Rational quantitative attribution of beliefs, desires and percepts in human mentalizing

ABSTRACT Social cognition depends on our capacity for ‘mentalizing’, or explaining an agent’s behaviour in terms of their mental states. The development and neural substrates of mentalizing

are well-studied, but its computational basis is only beginning to be probed. Here we present a model of core mentalizing computations: inferring jointly an actor’s beliefs, desires and

percepts from how they move in the local spatial environment. Our Bayesian theory of mind (BToM) model is based on probabilistically inverting artificial-intelligence approaches to rational

planning and state estimation, which extend classical expected-utility agent models to sequential actions in complex, partially observable domains. The model accurately captures the

quantitative mental-state judgements of human participants in two experiments, each varying multiple stimulus dimensions across a large number of stimuli. Comparative model fits with both

simpler ‘lesioned’ BToM models and a family of simpler non-mentalistic motion features reveal the value contributed by each component of our model. Access through your institution Buy or

subscribe This is a preview of subscription content, access via your institution ACCESS OPTIONS Access through your institution Access Nature and 54 other Nature Portfolio journals Get

Nature+, our best-value online-access subscription $32.99 / 30 days cancel any time Learn more Subscribe to this journal Receive 12 digital issues and online access to articles $119.00 per

year only $9.92 per issue Learn more Buy this article * Purchase on SpringerLink * Instant access to full article PDF Buy now Prices may be subject to local taxes which are calculated during

checkout ADDITIONAL ACCESS OPTIONS: * Log in * Learn about institutional subscriptions * Read our FAQs * Contact customer support SIMILAR CONTENT BEING VIEWED BY OTHERS BAYESIAN INFERENCE

WITH INCOMPLETE KNOWLEDGE EXPLAINS PERCEPTUAL CONFIDENCE AND ITS DEVIATIONS FROM ACCURACY Article Open access 29 September 2021 A DISSOCIATION BETWEEN THE USE OF IMPLICIT AND EXPLICIT PRIORS

IN PERCEPTUAL INFERENCE Article Open access 26 November 2024 HUMAN INFERENCE REFLECTS A NORMATIVE BALANCE OF COMPLEXITY AND ACCURACY Article 30 May 2022 REFERENCES * Knill, D. &

Richards, W. _Perception as Bayesian Inference_ (Cambridge Univ. Press, 1996). Book  Google Scholar  * Marr, D . _Vision_ (Freeman, 1982). Google Scholar  * Weiss, Y., Simoncelli, E. P.

& Adelson, E. H . Motion illusions as optimal percepts. _Nat. Neurosci._ 5, 598–604 (2002). Article  CAS  PubMed  Google Scholar  * Carey, S . _The Origin of Concepts_ (Oxford Univ.

Press, 2009). Book  Google Scholar  * Csibra, G., Biró, S., Koós, O. & Gergely, G . One-year-old infants use teleological representations of actions productively. _Cogn. Sci_. 27,

111–133 (2003). Google Scholar  * Hamlin, J. K., Wynn, K. & Bloom, P. Social evaluation by preverbal infants. _Nature_ 450, 557–560 (2007). Article  CAS  PubMed  Google Scholar  *

Leslie, A. M., Friedman, O. & German, T. P . Core mechanisms in ‘theory of mind’. _Trends Cogn. Sci._ 8, 528–533 (2005). Article  Google Scholar  * Onishi, K. H. & Baillargeon, R. Do

15-month-old infants understand false beliefs? _Science_ 308, 255–258 (2005). Article  CAS  PubMed  PubMed Central  Google Scholar  * Woodward, A. L. Infants selectively encode the goal

object of an actor’s reach. _Cognition_ 69, 1–34 (1998). Article  CAS  PubMed  Google Scholar  * Wellman, H. M. _Making Minds: How Theory of Mind Develops_ (Oxford Univ. Press, 2014). Book 

Google Scholar  * Wimmer, H. & Perner, J . Beliefs about beliefs: representation and constraining function of wrong beliefs in young children’s understanding of deception. _Cognition_

13, 103–128 (1983). Article  CAS  PubMed  Google Scholar  * Battaglia, P. W., Hamrick, J. B. & Tenenbaum, J. B . Simulation as an engine of physical scene understanding. _Proc. Natl

Acad. Sci. USA_ 110, 18327–18332 (2013). Article  CAS  PubMed  PubMed Central  Google Scholar  * Körding, K. P. & Wolpert, D. M. Bayesian integration in sensorimotor learning. _Nature_

427, 244–247 (2004). Article  PubMed  Google Scholar  * Baker, C. L., Saxe, R. & Tenenbaum, J. B. Action understanding as inverse planning. _Cognition_ 113, 329–349 (2009). Article 

PubMed  Google Scholar  * Jern, A. & Kemp, C. A decision network account of reasoning about other people’s choices. _Cognition_ 142, 12–38 (2015). Article  PubMed  PubMed Central  Google

Scholar  * Jara-Ettinger, J., Gweon, H., Tenenbaum, J. B. & Schulz, L. E. Children’s understanding of the costs and rewards underlying rational action. _Cognition_ 140, 14–23 (2015).

Article  PubMed  Google Scholar  * Lucas, C. G. et al. The child as econometrician: a rational model of preference understanding in children. _PLoS ONE_ 9, e92160 (2014). Article  PubMed 

Google Scholar  * E. Oztop, D. Wolpert & M. Kawato . Mental state inference using visual control parameters. _Cogn. Brain Res._ 22, 129–151 (2005). Article  Google Scholar  * Pantelis,

P. C. et al. Inferring the intentional states of autonomous virtual agents. _Cognition_ 130, 360–379 (2014). Article  PubMed  Google Scholar  * Blythe, P. W., Todd, P. M. & Miller, G. F.

in _Simple Heuristics that Make Us Smart_ (eds Gigerenzer G., Todd, P. M. & the ABC Research Group) 257–286 (Oxford Univ. Press, 1999). Google Scholar  * Zacks, J. M. Using movement and

intentions to understand simple events. _Cogn. Sci._ 28, 979–1008 (2004). Article  Google Scholar  * Goodman, N. D., Baker, C. L. & Tenenbaum, J. B . Cause and intent: social reasoning

in causal learning. In _Proc. 31st Annu. Conf. Cognitive Science Society_ (eds Taatgen, N. & van Rijn, H.) 2759–2764 (Cognitive Science Society, 2009). * Shafto, P., Goodman, N. D. &

Frank, M. C . Learning from others: the consequences of psychological reasoning for human learning. _Persp. Psychol. Sci._ 7, 341–351 (2012). Article  Google Scholar  * Rafferty A. N.,

LaMar M. M. & Griffiths, T. L. Inferring learners’ knowledge from their actions. _Cogn. Sci._ 39, 584–618 (2015). Article  PubMed  Google Scholar  * Hawthorne-Madell, D. & Goodman,

N. D. So good it has to be true: wishful thinking in theory of mind. In _Proc. 37th Annu. Conf. Cognitive Science Society_ (eds Noelle, D. C. _et al_.) 884–889 (Cognitive Science Society,

2015). * Butterfield, J., Jenkins, O. C., Sobel, D. M. & Schwertfeger, J. Modeling aspects of theory of mind with Markov random fields. _Int. J. Soc. Robot._ 1, 41–51 (2009). Article 

Google Scholar  * Shafto, P., Eaves, B., Navarro, D. J. & Perfors, A . Epistemic trust: modeling children’s reasoning about others’ knowledge and intent. _Dev. Sci._ 15, 436–447 (2012).

Article  PubMed  Google Scholar  * Gergely, G. & Csibra, G . Teleological reasoning in infancy: the naïve theory of rational action. _Trends Cogn. Sci._ 7, 287–292 (2003). Article 

PubMed  Google Scholar  * Perner, J . _Understanding the Representational Mind_ (MIT Press, 1991). Google Scholar  * Kaelbling, L. P., Littman, M. L. & Cassandra, A. R . Planning and

acting in partially observable stochastic domains. _Artificial Intelligence_ 101, 99–134 (1998). Article  Google Scholar  * von Neumann, J. & Morgenstern, O . _Theory of Games and

Economic Behavior_ (Princeton Univ. Press, 1953). Google Scholar  * Gmytrasiewicz, P. J. & Doshi, P. A framework for sequential planning in multi-agent settings. _J. Artif. Intell. Res._

24, 49–79 (2005). Article  Google Scholar  * Davis, L. S. & Benedikt, M. L . Computational models of space: isovists and isovist fields. _Comput. Graph. Image Process._ 11, 49–72

(1979). Article  Google Scholar  * Morariu, V. I., Prasad, V. S. N. & Davis L. S . Human activity understanding using visibility context. In _IEEE/RSJ IROS Workshop: From Sensors to

Human Spatial Concepts (FS2HSC)_ (2007). * Tremoulet, P. D. & Feldman, J . The influence of spatial context and the role of intentionality in the interpretation of animacy from motion.

_Percept. Psychophys._ 29, 943–951 (2006). Article  Google Scholar  * Perner, J. & Ruffman, T. Infants’ insight into the mind: how deep? _Science_ 308, 214–216 (2005). Article  CAS 

PubMed  Google Scholar  * Csibra, G. & Volein, Á . Infants can infer the presence of hidden objects from referential gaze information. _Br. J. Dev. Psychol._ 26, 1–11 (2008). Article 

Google Scholar  * Moll, H. & Tomasello, M . 12- and 18-month-old infants follow gaze to spaces behind barriers. _Dev. Sci._ 7, F1–F9 (2004). Article  PubMed  Google Scholar  * Hsu, D.,

Lee, W. S. & Rong, N . What makes some POMDP problems easy to approximate? In _Advances in Neural Information Processing Systems_ _(NIPS 2007)_ Vol. 20 (eds Platt, J. C. ) (NIPS

Foundation, 2007). Google Scholar  * Silver, D. & Veness, J . Monte-Carlo planning in large POMDPs. In _Advances in Neural Information Processing Systems_ _(NIPS 2010)_ Vol. _23 _ (eds

Lafferty, J. D. et al.) (NIPS Foundation, 2010). Google Scholar  * Somani, A., Ye, N., Hsu, D. & Lee, W. S. Despot: online POMDP planning with regularization. In _Advances in Neural

Information Processing Systems_ _(NIPS 2013)_ Vol. _26_ (eds Burges, C. J. C. et al.) (NIPS Foundation, 2013). Google Scholar  * Lovejoy, W. S . Computationally feasible bounds for partially

observed Markov decision processes. _Oper. Res_. 39, 162–175 (1991). Article  Google Scholar  * Kurniawati, H., Hsu, D. & Lee, W. S. SARSOP: efficient point-based POMDP planning by

approximating optimally reachable belief spaces. In _Proc. Robotics: Science and Systems_ Vol. 4 (eds Brock, O., Trinkle, J. & Ramos, F.) 65–72 (MIT, 2009). Chapter  Google Scholar  *

Kulkarni, T. D., Kohli, P., Tenenbaum, J. B. & Mansinghka, V . Picture: an imperative probabilistic programming language for scene perception. In _IEEE Conf. Computer Vision and Pattern

Recognition_ 4390–4399 (Computer Vision Foundation, 2015). * de Villiers, J. G. & de Villiers, P. A. in _Language Acquisition_ (ed. Foster-Cohen, S. ) Ch. 7, 169–195 (Palgrave Macmillan,

2009). Book  Google Scholar  * Baker, C. L., Goodman, N. D. & Tenenbaum, J. B . Theory-based social goal inference. In _Proc. 30th Annu. Conf. Cognitive Science Society_ 1447–1455

(Cognitive Science Society, 2008). * Hamlin, J. K., Ullman, T. D., Tenenbaum, J. B., Goodman, N. D. & Baker, C. L. The mentalistic basis of core social cognition: experiments in

preverbal infants and a computational model. _Dev. Sci._ 16, 209–226 (2013). Article  PubMed Central  Google Scholar  * Littman, M. L . Markov games as a framework for multi-agent

reinforcement learning. In _Proc. 11th Int. Conf. Machine Learning_ Vol. 9166 (ed. Perner, P.) 157–163 (Springer, 1994). * Yoshida, W., Dolan, R. J. & Friston, K. J. Game theory of mind.

_PLoS Comput. Biol._ 4, 1–14 (2008). Article  Google Scholar  * Doshi, P., Qu, X., Goodie, A. & Young, D. Modeling recursive reasoning by humans using empirically informed interactive

POMDPs. In _Proc. 9th Int. Conf. Autonomous Agents and Multiagent Systems (AAMAS_) (International Foundation for Autonomous Agents and Multiagent Systems, 2010). * Stuhlmüller, A. &

Goodman, N. D . Reasoning about reasoning by nested conditioning: modeling theory of mind with probabilistic programs. _J. Cogn. Syst. Res._ 28, 80–99 (2013). Article  Google Scholar  *

Spelke, E. S. & Kinzler, K. D. Core knowledge. _Dev. Sci._ 10, 89–96 (2007). Article  PubMed  Google Scholar  * Kahneman, D . _Thinking, Fast and Slow_ (Farrar, Straus and Giroux, 2011).

Google Scholar  * Ong, S. C. W., Png, S. W., Hsu, D. & Lee, W. S. POMDPs for robotic tasks with mixed observability. In _Robotics: Science and Systems_ Vol. 5 (MIT Press, 2009). Google

Scholar  * Cohen, P. R. _Empirical Methods in Artificial Intelligence_ (MIT Press, 1995). Google Scholar  Download references ACKNOWLEDGEMENTS This work was supported by the Center for

Brains, Minds & Machines (CBMM), under NSF STC award CCF-1231216; by NSF grant IIS-1227495 and by DARPA grant IIS-1227504. The funders had no role in study design, data collection and

analysis, decision to publish or preparation of the manuscript. AUTHOR INFORMATION AUTHORS AND AFFILIATIONS * Department of Brain and Cognitive Sciences, Massachusetts Institute of

Technology, 77 Massachusetts Avenue, Cambridge, 02139, Massachusetts, USA Chris L. Baker, Julian Jara-Ettinger, Rebecca Saxe & Joshua B. Tenenbaum Authors * Chris L. Baker View author

publications You can also search for this author inPubMed Google Scholar * Julian Jara-Ettinger View author publications You can also search for this author inPubMed Google Scholar * Rebecca

Saxe View author publications You can also search for this author inPubMed Google Scholar * Joshua B. Tenenbaum View author publications You can also search for this author inPubMed Google

Scholar CONTRIBUTIONS C.L.B., R.S. and J.B.T. designed Experiment 1. C.L.B. ran Experiment 1, implemented the models and performed the analyses of Experiment 1. J.J.E., C.L.B. and J.B.T.

designed Experiment 2. J.J.-E. and C.L.B. ran Experiment 2, implemented the models and performed the analyses of Experiment 2. C.L.B. and J.B.T. wrote the manuscript. CORRESPONDING AUTHOR

Correspondence to Joshua B. Tenenbaum. ETHICS DECLARATIONS COMPETING INTERESTS The authors declare no competing interests. SUPPLEMENTARY INFORMATION SUPPLEMENTARY INFORMATION Supplementary

Methods, Supplementary Figures, Supplementary References. (PDF 731 kb) RIGHTS AND PERMISSIONS Reprints and permissions ABOUT THIS ARTICLE CITE THIS ARTICLE Baker, C., Jara-Ettinger, J.,

Saxe, R. _et al._ Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. _Nat Hum Behav_ 1, 0064 (2017). https://doi.org/10.1038/s41562-017-0064 Download

citation * Received: 05 October 2016 * Accepted: 03 February 2017 * Published: 13 March 2017 * DOI: https://doi.org/10.1038/s41562-017-0064 SHARE THIS ARTICLE Anyone you share the following

link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently available for this article. Copy to clipboard Provided by the Springer Nature

SharedIt content-sharing initiative