A neuro-symbolic ARC parser

Intro

This previous post introduces why I think the abstraction and reasoning corpus (ARC) from this paper is a worthy challenge to pursue for robust ML research. As a quick explainer, the ARC dataset is composed of tasks like this one:

In this task you can produce the correct answer by: finding objects with the same shape as the object contained by the top left grey border and recoloring those objects to grey.

There are more (and less) general solutions to this task (e.g recolor to the same color as the border). This task also shows some the priors that are assumed: the concept of an object, the idea of a relationship between objects and the idea of containment

Two different approaches

Direct programme synthesis

The first approach that comes to mind for ARC is program synthesis. F.Chollet directly suggests this route. Specficially, building a model $f_{\theta}(X, Y) = p$ that attends over each task’s support set of input images $x_i \in X$ and label images $y_i \in Y$, and produces a program $p$, such that $p(X)=Y$. This would likely imply a search step, to select/rank programs produced by $f_{\theta}$.

The design of ARC makes a direct brute force implmentation difficult for a number of reasons, among them:

• Large number of diverse tasks (400)
• Differeing priors required for difering tasks
• Very small support for each task (1-10) samples
• Compositional tasks: the output of one step is the basis for the next

…to be continued