We propose a new class of probabilistic neural-symbolic models, that have
symbolic functional programs as a latent, stochastic variable. Instantiated in
the context of visual question answering, our probabilistic formulation offers
two key conceptual advantages over prior neural-symbolic models for VQA.
Firstly, the programs generated by our model are more understandable while
requiring lesser number of teaching examples. Secondly, we show that one can
pose counterfactual scenarios to the model, to probe its beliefs on the
programs that could lead to a specified answer given an image. Our results on
the CLEVR and SHAPES datasets verify our hypotheses, showing that the model
gets better program (and answer) prediction accuracy even in the low data
regime, and allows one to probe the coherence and consistency of reasoning
performed.

Source link