Browser Automation with cmux

Browser Automation with cmux

Use this skill for browser tasks inside cmux webviews.

Core Workflow

1. Open or target a browser surface.
2. Verify navigation with get url before waiting or snapshotting.
3. Snapshot (--interactive) to get fresh element refs.
4. Act with refs (click, fill, type, select, press).
5. Wait for state changes.
6. Re-snapshot after DOM/​navigation changes.

cmux --json browser open https://example.com
# use returned surface ref, for example: surface:7

cmux browser surface:7 get url
cmux browser surface:7 wait --load-state complete --timeout-ms 15000
cmux browser surface:7 snapshot --interactive
cmux browser surface:7 fill e1 "hello"
cmux --json browser surface:7 click e2 --snapshot-after
cmux browser surface:7 snapshot --interactive

Surface Targeting

# identify current context
cmux identify --json

# open routed to a specific topology target
cmux browser open https://example.com --workspace workspace:2 --window window:1 --json

Notes:

• CLI output defaults to short refs (surface:N, pane:N, workspace:N, window:N).
• UUIDs are still accepted on input; only request UUID output when needed (--id-format uuids|both).
• Keep using one surface:N per task unless you intentionally switch.

Wait Support

cmux supports wait patterns similar to agent-browser:

cmux browser <surface> wait --selector "#ready" --timeout-ms 10000
cmux browser <surface> wait --text "Success" --timeout-ms 10000
cmux browser <surface> wait --url-contains "/​dashboard" --timeout-ms 10000
cmux browser <surface> wait --load-state complete --timeout-ms 15000
cmux browser <surface> wait --function "document.readyState === 'complete'" --timeout-ms 10000

Common Flows

Form Submit

cmux --json browser open https://example.com/​signup
cmux browser surface:7 get url
cmux browser surface:7 wait --load-state complete --timeout-ms 15000
cmux browser surface:7 snapshot --interactive
cmux browser surface:7 fill e1 "Jane Doe"
cmux browser surface:7 fill e2 "[email protected]"
cmux --json browser surface:7 click e3 --snapshot-after
cmux browser surface:7 wait --url-contains "/​welcome" --timeout-ms 15000
cmux browser surface:7 snapshot --interactive

Clear an Input

cmux browser surface:7 fill e11 "" --snapshot-after --json
cmux browser surface:7 get value e11 --json

Stable Agent Loop (Recommended)

# navigate -> verify -> wait -> snapshot -> action -> snapshot
cmux browser surface:7 get url
cmux browser surface:7 wait --load-state complete --timeout-ms 15000
cmux browser surface:7 snapshot --interactive
cmux --json browser surface:7 click e5 --snapshot-after
cmux browser surface:7 snapshot --interactive

If get url is empty or about:blank, navigate first instead of waiting on load state.

Deep-Dive References

Ready-to-Use Templates

Limits (WKWebView)

These commands currently return not_supported because they rely on Chrome/​CDP-only APIs not exposed by WKWebView:

• viewport emulation
• offline emulation
• trace/​screencast recording
• network route interception/​mocking
• low-level raw input injection

Use supported high-level commands (click, fill, press, scroll, wait, snapshot) instead.

Troubleshooting

js_error on snapshot --interactive or eval

Some complex pages can reject or break the JavaScript used for rich snapshots and ad-hoc evaluation.

Recovery steps:

cmux browser surface:7 get url
cmux browser surface:7 get text body
cmux browser surface:7 get html body

• Use get url first so you know whether the page actually navigated.
• Fall back to get text body or get html body when snapshot --interactive or eval returns js_error.
• If the page is still failing, navigate to a simpler intermediate page, then retry the task from there.