Remote GPU (Inference Server)

If you don't have a local GPU but have access to a remote GPU server, run inference there and stream actions back to your machine over an SSH tunnel. Your local smolvla node captures the cameras + joint states and sends them to the server; the server runs the model and returns joint targets.

Your laptop                       Remote GPU server
──────────────────                ──────────────────────────────
Gazebo sim  ──ROS 2──► smolvla ──HTTP──► smolvla-server
RViz2                  (local)    :8000   (remote, has GPU)

Step 1 — Start the inference server on the GPU machine

DATA_DIR is the host directory mounted to /data in the container — point it at the checkpoint output dir. The server auto-selects the latest checkpoint under /data/checkpoints; set SMOLVLA_CHECKPOINT to pin a step. It reads the expected cameras from the checkpoint and validates incoming frames.

If the GPU server has the repo cloned (it will if you trained there, Workflow 2 Option B), start the server from the repo root with the infer-server profile:

# On the GPU server
cd ~/ar4-physical-ai
 
# Auto-select the latest checkpoint under DATA_DIR
DATA_DIR=~/ar4-train/data/checkpoints/smolvla_ar4_moveit \
  docker compose --profile infer-server up -d

To pin a specific checkpoint step (here DATA_DIR maps to /data, so the path is relative to it):

DATA_DIR=~/ar4-train/data/checkpoints/smolvla_ar4_moveit \
SMOLVLA_CHECKPOINT=/data/checkpoints/050000/pretrained_model \
  docker compose --profile infer-server up -d

Prefer that form. It puts the container in the ar4-physical-ai Compose project, so ./scripts/down.sh tears it down with everything else. The older -p ar4 -f docker-compose.infer-server.yaml invocation put it in a project of its own that no teardown from the repo root ever matched, which is how a policy server once stayed up for four days after the stack around it was stopped.

On a machine that has only the gpu-server/ar4-infer/ bundle rather than the whole repo, run it directly instead:

cd ~/ar4-physical-ai/gpu-server/ar4-infer
DATA_DIR=~/ar4-train/data/checkpoints/smolvla_ar4_moveit \
  docker compose -f docker-compose.infer-server.yaml up -d

Check it loaded: curl localhost:8000/health → {"loaded":true,"task":"..."}.

Step 2 — Open an SSH tunnel (keep this terminal open)

# On your local machine — forwards GPU server port 8000 to localhost:8000
ssh -L 8000:localhost:8000 user@gpu-server

Step 3 — Start the sim and point inference at the tunnel

# Terminal 1 — simulation (publishes scene + wrist cameras and joint states)
docker compose up sim-tabletop moveit
 
# Terminal 2 — inference node, offloaded to the remote GPU
SMOLVLA_SERVER_URL=http://localhost:8000 docker compose run --rm smolvla

The local node sends observations to the server over HTTP and receives joint targets back — the arm moves as if the GPU were local. The TASK you set locally is forwarded to the server, so steer it the same way as local inference:

SMOLVLA_SERVER_URL=http://localhost:8000 \
TASK="pick up the teal block and place it on the top shelf" \
  docker compose run --rm smolvla

When you're done, stop the server on the GPU machine with ./scripts/down.sh, which also catches anything else the stack left running.