New here? This is the full pipeline in five commands. All commands must be run from inside the ar4-physical-ai/ folder — open a terminal, cd ar4-physical-ai, and keep that terminal open.
Step
Command
Notes
1. Build images
docker compose build
First time only — takes ~20–30 min
2. Start simulation
docker compose --profile sim up
Opens Gazebo + RViz2. Wait for "MoveIt Servo ready" in logs
3. Collect episodes
docker compose run --rm record
New terminal — arm moves to home, then you drive it
X11 display for the Gazebo GUI — if you get a permission error, run xhost +local:docker once in your terminal before starting
Xbox One (or compatible) controller — USB or Bluetooth; plug it into any USB port, it is detected automatically
ROS 2 Jazzy on your host machine — only needed if you want to use rqt_image_view to preview the wrist camera. If you don't have it, Foxglove works without any local ROS install
# Clone with submodules (the AR4 ROS driver lives under vendor/)git clone --recurse-submodules https://github.com/aegean-ai/ar4-physical-ai.gitcd ar4-physical-ai# Build all Docker images (base → overlay → dev)# This takes a while the first time — grab a coffeedocker compose build
Forgot --recurse-submodules? Run git submodule update --init --recursive to fix it.
Important: Every docker compose command in this README must be run from inside the ar4-physical-ai/ folder. If you open a new terminal, cd back into the repo first:
Every runtime service belongs to a profile, so you always say which stack you want. A bare docker compose up starts nothing. That is deliberate: a --profile flag adds to the profile-less services rather than replacing them, so when the simulation had no profile, --profile hardware up booted Gazebo alongside the real arm and the two sets of /joint_states collided.
Three profiles match the three things this repo does.
Command
What it starts
docker compose --profile sim up
Gazebo GUI + MoveIt2 + RViz2 (simulation)
docker compose --profile dataset up -d
The above plus the Zenoh router and bridge, for episode recording
docker compose --profile infer-server up -d
SmolVLA policy server (run this on the GPU box)
The rest are role profiles you combine with those, or one-shot services you launch by name.
Command
What it starts
docker compose --profile hardware up hardware moveit-hardware
Real AR4 driver + MoveIt2
docker compose --profile headless up sim-tabletop-headless
Gazebo without a display (for servers / CI)
docker compose --profile obs up foxglove-bridge
Foxglove WebSocket bridge
docker compose run --rm record
Xbox teleop + LeRobot recorder
docker compose run --rm smolvla
SmolVLA policy inference
docker compose run demo
C++ pick-and-place demo
Naming a service explicitly activates its profile for you, so docker compose up sim-tabletop moveit still works without a --profile flag.
Model training is not a compose profile. It is a one-shot job with per-run names and hyperparameters, which compose's one-container-per-service model works against, so it stays in scripts/train_local.sh and scripts/train_remote.sh.
Services that run with docker compose run are one-shot interactive containers — they start, do their job, and exit cleanly on Ctrl-C. The simulation (up) stays running in the background while you use them.
Use this rather than docker compose down, which only considers the profiles active for that one invocation and walks silently past the rest. It exits 0 while leaving containers running.
ar4_msgs/ Custom ROS 2 interfacesar4_teleop/ Gamepad teleop + scripted MoveIt recorder (moveit_record_node)ar4_lerobot/ Decoupled numpy-2 LeRobot recorder (camera/joints over Zenoh)ar4_policy_server/ SmolVLA inference backend + ROS bridge (inference_node)ar4_episodes/ Episode config, recording, export, replay helpersar4_skills/ Behavior-tree skills and tuningar4_rerun/ Decoupled Zenoh → Rerun visualisation recorderar4_moveit_cpp_demo/ C++ pick-and-place demo using MoveItgs/ Language-grounded 3D Gaussian-splatting pipelinegpu-server/ Remote train/infer server assets (ar4-train, ar4-infer)configs/ Training configs (e.g. smolvla_ar4.yaml)scripts/ Dataset push, local/remote train, mesh generationdocker/ Dockerfile.gpu (multi-stage), entrypoint.sh, cyclonedds.xmlzenoh/ Zenoh router config, CycloneDDS bridge configdata/ Local data store (gitignored)docs/ Documentation content (MDX)docs-site/ Fumadocs / Next.js documentation sitevendor/ar4_ros_driver/ Git submodule — upstream AR4 ROS driver
Datasets are recorded in LeRobot v3.0 format (Parquet + MP4) and pushed to Hugging Face Hub or
Cloudflare R2; both the human teleop and scripted-expert recorders write the same
image/state/action schema (realised joint positions as action).