Play Music from YouTube

SKILL.md

---
name: my-play-music-from-yt
description: |
  Play music on YouTube via browser automation with playwright-cli.
  Use when the user wants to: 
  (1) play a specific song (e.g. 'play Money Money Money by ABBA') 
  (2) play songs by an artist as a playlist or mix (e.g. 'play Jay Chou's songs') 
  (3) play genre or mood-based music (e.g. 'play relaxing spa music', 'play 60s Chinese oldies') 
  (4) control playback — next, pause, resume, stop, skip ad, change song, close the player. 
  Also handles song/artist name corrections from voice transcription errors.
allowed-tools: Bash(playwright-cli:*)
metadata: {"openclaw": {"requires": {"bins": ["playwright-cli"]}, "emoji": "🎵"}}
---

# Play Music from YouTube — Clear, Stable, High SNR Instruction Set

This skill controls a **visible browser** using `playwright-cli` to search and play YouTube music.
All actions follow **snapshot → ref → action** pattern. Never guess. Never assume.

---
# CORE PRINCIPLES

### **1. Always use a named session**
```
-s=music_player
```
Every command must include it.

### **2. Always snapshot before interacting**
```
playwright-cli -s=music_player snapshot
```
Never click/fill without fresh refs.

### **3. Only use ref-based actions**
```
click e123
fill e45 "text"
```
Never use CSS, XPaths, or assumptions.

### **4. Browser must be visible**
```
--headed
```
Use headed **by default** unless the user explicitly requests headless.

### **5. Session continues in background**
After playback begins, do not block waiting. Continue responding normally.

### **6. Use persistent browser profile**
```
--persistent
```
Always include `--persistent` when opening a new session. This saves the browser profile (cookies, localStorage, IndexedDB, cache) to disk, so **login state survives session restarts**.

---
# SNAPSHOT STORAGE

Snapshots generated by `playwright-cli` may exist in two possible locations:

1. `$WORKSPACE/.playwright-cli/` ← **primary (search here first)**
2. `~/.playwright-cli/` ← **fallback when the workspace folder is empty**

Use **only these two folders** unless the user explicitly authorizes a broader search.

If no .yml snapshots appear here, it is almost always **MacOS permission-related**.
Ask the user to run:
```
playwright-cli open https://www.youtube.com --headed
```
This will trigger macOS system dialogs for:
- Screen Recording
- Automation
- Accessibility

Approve all to enable snapshot generation.

> **Note:** On Windows and Linux, these permission dialogs do not apply. If snapshots are missing on those platforms, check that the browser launched correctly.

---
# SESSION MANAGEMENT

### Check whether the session exists
```
playwright-cli list
```
- Exists → continue
- Missing → recreate:
```
playwright-cli -s=music_player open https://www.youtube.com --headed --persistent
```

**On any unexpected error**, run `playwright-cli list` first to diagnose before retrying.

### If the session is frozen or blank
```
playwright-cli -s=music_player goto https://www.youtube.com
```
This reliably resets the page without closing the session.

---
# SEARCH QUERY CONSTRUCTION

Build the search query based on user intent:

| Intent | Query pattern | Example |
|---|---|---|
| Specific song | `[Artist] [Song Title]` | `ABBA Money Money Money` |
| Artist playlist | `[Artist]` | `周杰倫` |
| Genre / mood | `[descriptor] music playlist` | `relaxing spa music playlist` |
| Era-based | `[era] [language/genre] playlist` | `華語 60年代 老歌 playlist` |

**Search tips:**
- For artist requests, a simple artist name search usually surfaces "Mix" and playlist results at the top — prefer these for continuous playback.
- Use the original language for non-English songs (e.g. Chinese characters for Chinese songs).

**Voice transcription (ASR) and typo handling:**
- **Do NOT ask the user to confirm** potentially misheard names. YouTube has robust auto-correction and will show results for the intended query even with typos or homophones (e.g. searching "楊成林" will auto-correct to "楊丞琳").
- Always search directly with whatever text you have. After the search, check the results snapshot — if YouTube shows a "Did you mean: ..." or "顯示以下搜尋結果: ..." banner with corrected results, the correction is already applied.
- Only fall back to **web search** or asking the user if the YouTube search results are clearly unrelated or empty.

---
# SEARCH WORKFLOW (HIGH SIGNAL, LOW AMBIGUITY)

## **Step 1 — Reset search state**
A clean search state avoids dropdown obstruction.

Perform the following every time before a new search:

1. Snapshot.
2. If button `"Clear search query"` / `"清除搜尋查詢"` is present → click it.
3. Else press:
```
playwright-cli -s=music_player press Escape
```
4. If suggestions or overlays still block results → reset page:
```
playwright-cli -s=music_player goto https://www.youtube.com
```
Then snapshot again.

---
## **Step 2 — Locate search bar**
Snapshot → find:
- `combobox "搜尋"` or
- `combobox "Search"`

Record its ref (e.g., `e34`).

---
## **Step 3 — Perform search**
```
playwright-cli -s=music_player fill <searchRef> "SEARCH TERM"
playwright-cli -s=music_player press Enter
```
Then snapshot immediately to avoid suggestion-panel obstruction.

---
## **Step 4 — Choose best result**

Select result based on **user intent**:

- **Specific song** → click the `link` whose heading matches the song title.
- **Artist** → prefer `"Mix - [Artist]"` links or playlist links for continuous playback.
- **Genre / mood** → prefer long-duration compilations or playlists (>20 mins).
- **Fallback** if no clear match → prefer: Mix > Long playlist > Official channel upload > Single MV.

For guidance on identifying YouTube result types, see [./references/youtube-guide.md](./references/youtube-guide.md).

**Important:** Do NOT click channel cards (e.g. `link "Artist Name ... @handle•NNNK subscribers ..."` / `"Artist Name ... @handle•NNN萬位訂閱者 ..."`). They navigate to the channel page, not music. Look for Mix or playlist results below them.

Click using its ref:
```
playwright-cli -s=music_player click eXYZ
```

If click is blocked:
1. `press Escape`
2. Retry click once.
3. If still blocked → `goto https://www.youtube.com` and restart from Step 1.

---
# AD HANDLING LOOP

After clicking a result:
```
playwright-cli -s=music_player snapshot
```
Loop up to 4 iterations:

### **1. Playback detected**
If you see:
- `button "Pause (k)"` or
- `button "Play (k)"` (but video advancing)

AND no ad indicators → playback is active → report song title to the user.

### **2. Skip button detected**
Any element whose label contains:
- skip / Skip / 略過
- Skip Ad / Skip Ads / 略過廣告

→ click it immediately.

**Note:** If the skip button is `[disabled]` (countdown not yet elapsed), do NOT click it — wait ~5 seconds instead (see step 3).

Then snapshot again. YouTube often plays **two consecutive ads** — always check for a second ad after skipping.

### **3. Non-skippable ad / countdown**
Wait 5 seconds:
```
playwright-cli -s=music_player eval "await new Promise(r=>setTimeout(r,5000))"
```
Then snapshot again and repeat from step 1.

### If ads persist >20 seconds
- press Escape
- or reload page

---
# PLAYBACK CONTROLS
Always snapshot first.

| Command | Action |
|--------|--------|
| Pause | click `button "Pause (k)"` |
| Resume / Play | click `button "Play (k)"` |
| Next | `press Shift+n` |
| Previous | `press Shift+p` |
| Change song | Start new search from Step 1 |
| Stop | `playwright-cli -s=music_player close` |

**Keyboard shortcuts** (reliable alternative when button refs are hard to locate):

```
k        → play / pause
Shift+n  → next track
Shift+p  → previous track
m        → mute / unmute
f        → fullscreen toggle
j        → rewind 10s
l        → forward 10s
```

Usage: `playwright-cli -s=music_player press <key>`

---
# ERROR RECOVERY & EDGE CASES

### If snapshot empty or missing critical refs
```
playwright-cli -s=music_player goto https://www.youtube.com
```
→ restart from Step 1.

### If browser crashes or session disappears
`playwright-cli list` shows no `music_player`.
Inform the user the browser was closed, then recreate session:
```
playwright-cli -s=music_player open https://www.youtube.com --headed --persistent
```
Because `--persistent` is used, the browser profile (including login state) is restored automatically. No need to log in again.

### YouTube cookie consent / sign-in dialogs
Snapshot → locate dismiss actions:
- "Accept all" / "全部接受"
- "Reject all"
- "No thanks" / "不用了，謝謝"
- Close (X)

Click dismiss.

### YouTube Premium trial popup
Snapshot → look for `"No thanks"`, `"不用了，謝謝"`, or a dismiss/close button → click to dismiss.

### Video unavailable
Snapshot shows "Video unavailable" or similar error.
- `playwright-cli -s=music_player go-back` → snapshot → try the next result link.

### Cannot find elements in search results
- Scroll down: `playwright-cli -s=music_player press PageDown` → snapshot again.

### Page frozen or blank
- `playwright-cli -s=music_player reload` → snapshot.
- If still broken: `playwright-cli -s=music_player close` → reopen and retry from Step 1.

### Persistent profile corrupted or login expired
If the browser opens but login state is lost, or YouTube forces re-authentication:
1. Inform the user they need to log in again in the browser.
2. After login, the persistent profile will be updated automatically.

To completely reset the profile (last resort):
```
playwright-cli -s=music_player close
playwright-cli -s=music_player delete-data
playwright-cli -s=music_player open https://www.youtube.com --headed --persistent
```

---
# REFERENCES
- [Playwright CLI reference](./references/playwright-ref.md)
- [YouTube result identification](./references/youtube-guide.md)