PlaceTally: A Test-Time Memory Model
for Constructing Place Graphs

Alibay Osmanli Shaogang Gong

Queen Mary University of London

One 13-minute stream with six recordings of four rooms: summing visits answers 11 tables, whole-video reading answers 4, PlaceTally answers 5, the true count; right, mean answer against true count for the three methods.
Figure 1. Why do video models miscount when the camera returns? Reading the whole video gives a few small answers, adding up visits counts each return again, and PlaceTally counts each place once. Left: one ARKit-Revisit stream of 13 minutes with six recordings of four rooms, where room D is recorded twice in a row and room A returns at the end; numbers are the same model’s counts. Right: mean answer against binned true count on all 460 test questions, with 95% bands. Every question here has an object seen on a return (Sec. 4).

Abstract

Counting distinct objects in a video is hard when the camera returns to places it has seen. A vision-language model that reads the whole video answers with a few small numbers, and one that reads it in parts counts an object again in every part that shows it. Object trackers need the category before the video is read, or a fixed vocabulary. We observe that a memory for counting does not need to know what it is looking for, only where it has been: objects rarely leave a room between visits, so parts of a video grouped by place share almost no objects, and a place can be recognised before any question is asked. Building on this, we introduce PlaceTally, a question-blind test-time memory model that constructs a place graph for each video. Each video fragment joins the place of its best earlier match or opens a new place, because a strong match is rarely wrong while a true return often matches weakly. A frozen vision-language model counts each place once from at most 128 of its frames, so every read has a bounded input and the memory grows with the number of places; the graph is built 15 times faster than real time. On RevisitCount, our benchmark of real room re-recordings, it scores 62.5% mean relative accuracy on test rooms and 61.2% on held-out rooms, against 51.2% and 44.3% for the same model reading the whole video. It stays level as rooms return more or less often, while a fixed correction of summed visits moves from 38.5% to 61.4%.

Method

Method diagram: a detector cuts the stream into fragments; continuation and return evidence link fragments into places in a place graph built before the question; a frozen reader gates and counts each place once, and the counts are summed.
Figure 2. A PlaceTally memory is constructed per video instance before the question and read after it, shown on one real ARKit-Revisit test stream. (a) The detector cuts the stream into fragments. (b) Continuation and return evidence between fragments, which Eq. 2 would sum. (c) The place graph GV: Eq. 3 takes one decision per fragment (node), the kept links are its edges, and each lane is a place. Read: the category enters only here; each place is gated and counted, and the counts are summed once per place (Eq. 1).

Robustness and cost

Four panels: accuracy as returns are removed, the share of each stay kept, accuracy against memory, and accuracy against answer time, for PlaceTally and the other methods.
Figure 3. Robustness and cost on ARKit test rooms (post hoc, from actual answers). (a) Returns removed from the same 60 test streams; orange: the sum over true visits, as it is and times 0.69 (privileged); band: 95% interval. (b) The 823 test stays; dashed: the cap of 128 frames per place. (c, d) K = 16 to 128 frames per place at 11 minutes (gate refitted for 16 and 32); answer time excludes the one-off build (Table 5). Dotted lines pass through whole-video reading: above and left of them is better on both axes.

Supplementary material

The supplementary material is kept up to date here. Later versions keep the section letters below, so references in the paper such as “Supp. C” still point to the right place. Every version stays available.

VersionDateFile
1 (latest)17 November 2026supplement_v1_2026-11-17.pdf
ARevisitCount Benchmark Details
BMethod Details
CBaselines, Controls and Other Models
DAdditional Results
EExamples and Failure Cases
FPre-Registration Record
GCompute and Reproducibility
HData, Licences and Privacy

Code and benchmark

The GitHub repository has two folders. code/ holds PlaceTally’s code, trained weights and stored results, with scripts that recompute the paper’s numbers. benchmark/ holds RevisitCount: stream definitions, questions and answers, and scripts that rebuild the frames from the source datasets. To recompute the paper’s ARKit numbers on a CPU:

cd code
pip install -e .
python scripts/score_answers.py

The code is under the MIT licence and the benchmark annotations under CC BY-NC 4.0. ARKitScenes, 3RScan and MultiScan keep their own terms; the code and benchmark folders hold no images, video or meshes from them.

Citation

@article{osmanli2026placetally,
  title  = {PlaceTally: A Test-Time Memory Model for Constructing Place Graphs},
  author = {Osmanli, Alibay and Gong, Shaogang},
  journal = {arXiv preprint arXiv:2611.22608},
  year   = {2026}
}