Supervisory control of remote manipulation

TL;DR — In 1967, Ferrell and Sheridan diagnosed why teleoperating a manipulator over a time delay is hopeless — the human degenerates into "move and wait" — and prescribed the fix: the human should send symbolic subgoals to a remote computer that closes its own perception–action loops locally. The prescription was correct and the executive was impossible: no 1967 computer could see, plan, or grasp. Vision-language-action models are, almost exactly, the missing component. The paper reads today like the systems architecture document for language-supervised robot fleets, written 59 years early.

The problem: light is slow and hands are fast

The paper was written against the backdrop of Surveyor landers and pre-Apollo planning: how do you manipulate objects on the Moon from a console on Earth? The round-trip light delay to the Moon is about 2.6 seconds; to Mars, 6 to 44 minutes. Direct teleoperation — the master-slave arms then used in nuclear hot cells, where the operator stands meters away behind leaded glass — depends on a control loop with essentially zero latency. Insert seconds of delay into that loop and it becomes unstable in the ordinary control-theoretic sense: your correction arrives after the error has changed.

Ferrell had already shown experimentally (in work from around 1965 that this paper builds on) what human operators actually do under delay: they stop trying to control continuously. They make a small open-loop move, then wait out the full round trip to see what happened, then make another move. Completion time becomes roughly the number of incremental moves times the round-trip delay — approximately linear in delay, and brutal. A task of a hundred corrective motions at Mars distance takes days. The human's superb 10 Hz sensorimotor loop is throttled to the speed of light and back.

The move-and-wait finding is the empirical heart of the paper, and it's worth internalizing because it recurs everywhere: whenever the feedback loop through the human is slower than the task dynamics, continuous control collapses into discrete, cautious, open-loop chunks. You have felt this on a laggy SSH connection.

The idea: promote the human to supervisor

Their fix is an org-chart change. Stop routing the inner control loop through the delayed channel. Put a computer at the remote site that closes fast local loops — sensing contact, servoing the grasp, avoiding obstacles — and have the human send symbolic instructions across the delay: subgoals like "grasp the rock to the left," not joint torques. The human becomes a planner and monitor; the remote computer becomes what they call a limited but competent local decision-maker. They named the architecture supervisory control.

The elegance is that this is a bandwidth-and-latency argument, not an AI argument. The human's value is concentrated in low-rate, high-level decisions; the robot's needed competence is high-rate, low-level closure. Delay only hurts the channel it sits on. Route the slow stuff through the slow channel and the architecture is delay-invariant — task time stops scaling with round-trip time and starts scaling with the number of escalations to the human. They also saw the second-order consequence clearly: a supervisor who isn't in the inner loop can supervise several remote units, and can do so with degraded, abstracted feedback rather than full telepresence.

Why it could not work in 1967

The paper's own honesty is striking: the architecture is specified, the executive is vaporware. Consider what a "limited local decision-maker" needed and what existed:

Compute. A spacecraft-class computer of the era — the Apollo Guidance Computer is the touchstone — ran at roughly 10⁵ instructions per second with about 2K words of erasable memory. A modern edge module (a Jetson-class GPU) delivers on the order of 10¹³–10¹⁴ FLOP/s with tens of GB of RAM: roughly nine orders of magnitude in compute, seven in memory, in a comparable power envelope.

Perception. Machine vision in 1967 meant Roberts' blocks-world edge detection (1963) on a mainframe, minutes per frame. Shakey, the era's most ambitious attempt to fuse perception with action, took its planning steps off-board and needed a prepared environment. Recognizing an arbitrary rock, estimating its pose, and finding a grasp on it was not slow — it was unformulated as a problem anyone could solve.

Manipulation autonomy. The closest existing artifact was Ernst's MH-1 hand at MIT (1961), which could stack blocks using touch sensors under fixed programs. Autonomous grasping of unmodeled objects was decades out; the reliable general case arguably arrived only in the late 2010s.

The command language. Sheridan and Ferrell imagined the human issuing symbolic instructions, and the era's answer would have been something like a task-specific command grammar — MOVE, GRASP, coordinates. Grounding flexible natural language in perception and action ("the rock to the left of the crater rim, not the loose one") had no plausible mechanism at all until roughly 2022.

So the concept outran the robot by half a century. The paper is a rare case where the systems analysis was complete and correct, and every leaf node of the block diagram was science fiction.

What changed

Three things, and they compose.

First, edge compute crossed the threshold where a full perception–action stack runs at the remote site at control rates.

Second, VLA models — RT-2, OpenVLA, π0 and successors — are, functionally, Ferrell and Sheridan's local executive. They take a camera image and a natural-language subgoal and emit motor actions at tens of Hz, with enough generality to handle objects and scenes not seen in training. Success rates on open-ended manipulation are still well short of deployment-grade unattended autonomy — which is precisely why the supervisory framing matters. A 90%-reliable executive is useless as an autonomous system and extremely useful as a supervised one.

Third, language turned out to be the subgoal channel. Sheridan imagined symbolic commands; what we got is better — a channel that is human-native on one end and grounded in the robot's perception on the other, needing kilobits per second. The delayed link carries text and compressed summaries, not video and torques.

What a serious 2026 revival looks like

The revival is not "teleoperate the Moon," though that's back on the table. It's fleet supervision, and the economics are Sheridan's second-order consequence made primary.

Architecture: N manipulators, each running a local VLA executive at 20–50 Hz. One human supervisor issues language subgoals, monitors abstracted state (a queue of task summaries and confidence-flagged clips, not N live video feeds), and handles escalations. The delay in the loop is no longer light-time — it's attention latency: the seconds-to-minutes before the shared human notices your robot is stuck. Mathematically it's the same regime Ferrell studied, which is why the 1965–67 analyses of move-and-wait and of task time versus delay transfer almost verbatim.

Reuse from the paper: the loop decomposition; the insistence on measuring task completion time as a function of delay (now: as a function of robots-per-supervisor); the notion of the robot maintaining an internal model the human queries rather than raw sensation the human watches. Replace: the command grammar with natural language; the hypothetical executive with a VLA; open-loop trust with calibrated escalation — the executive must know when it doesn't know, which is currently the weakest link, since VLA confidence estimation is immature.

The decisive experiment is boring and important: a real deployment (warehouse kitting, lab automation, retail restocking) measuring robots-per-supervisor, interventions per robot-hour, and time-to-intervention, swept against the executive's autonomy level. The paper's implicit claim, translated: as executive competence rises, supervisor cost per robot falls toward zero while the data from escalations is exactly the curriculum that improves the executive. Supervisory control isn't just a deployment architecture; it's the data flywheel for VLAs.

Has it been vindicated?

Partially, twice. Planetary rovers are the clean vindication: Sojourner through Perseverance operate exactly on this model — daily uplinked activity plans, onboard hazard-avoidance closing local loops, with autonomy (AutoNav, AEGIS target selection) expanding release by release. Sheridan himself developed the framework into the levels-of-automation taxonomy (Sheridan & Verplank, 1978) and the standard text Telerobotics, Automation, and Human Supervisory Control (1992); the whole human-automation-interaction field descends from this paper. The DARPA Robotics Challenge deliberately degraded comms to force supervised autonomy and mostly demonstrated how hard the executive still was in 2015.

The industrial version is happening now, unevenly: Waymo's remote "fleet response" is supervisory control of driving; several manipulation startups run human-in-the-loop teleop with autonomy gradually eating the inner loop. What remains open: escalation calibration (the executive reliably knowing when to ask), delay-tolerant handoff (resuming mid-task after a supervisor intervention without resetting), and honest field numbers for robots-per-supervisor — the metric the 1967 paper implicitly proposed and which almost nobody publishes.

Where to read it

The paper is at doi.org/10.1109/mspec.1967.5217126 (IEEE Spectrum, 1967; bibliographic details verified). Read alongside: Ferrell's 1965 study of remote manipulation with transmission delay (the move-and-wait data); Sheridan & Verplank 1978 on levels of automation; and a current VLA paper — π0 or OpenVLA — read explicitly as a spec for the "limited local decision-maker" of Fig. 1. The 1967 half and the 2024 half snap together with almost no adapter.