radio/README.md

[![Hippocratic License HL3-FULL](https://img.shields.io/static/v1?label=Hippocratic%20License&message=HL3-FULL&labelColor=5e2751&color=bc8c3d)](https://firstdonoharm.dev/version/3/0/full.html)

# Radio Automation

Self-hosted internet radio built on **liquidsoap** feeding an **Icecast**
server, with podcast subscription management via gPodder.net or manual feeds.

Two parallel implementations are provided:

- **Python** (`fetch_podcasts.py`, `update_playlists.py`) — CPython 3, uses
  `feedparser` and `requests`.
- **JRuby** (`fetch_podcasts.rb`, `update_playlists.rb`) — JRuby 9.x/10.x,
  uses only stdlib plus the `sqlite3` gem.

Pick whichever you prefer; both produce identical behaviour and share the
same `config.json` and `station.liq`.

## Layout

Everything that grows over time (downloaded podcasts, music, jingles,
announcements, state databases, logs, generated playlists) lives under a
single **storage path** chosen at install time and recorded in
`config.json`. The code itself stays in this directory.

    <storage>/
      music/           # background music library (recursive mp3/m4a scan)
      podcasts/<slug>/ # downloaded episodes, one folder per show
      jingles/         # optional bumpers
      announcements/   # optional station IDs
      state/
        subscriptions.db
        played.db
      playlists/       # generated per-show .pls files
      logs/            # fetch.log, update.log, liquidsoap.log, cron.log

The install directory contains only code and configuration:

    fetch_podcasts.py / .rb
    update_playlists.py / .rb
    station.liq
    install_for_python.sh / install_for_jruby.sh
    config.json          # created by the installer (never committed)
    schedule.txt         # cron-style scheduled shows (you author this)

## Installing

Run the installer matching your runtime:

    sudo ./install_for_python.sh
    # or
    sudo ./install_for_jruby.sh

You will be prompted for:

1. **Storage path** — where all media and state live. Defaults to
   `/srv/radio-storage`. Choose a disk with plenty of free space; this is
   what keeps downloads off your boot drive.
2. **Icecast** source port, mount point, username, and password.
3. Optionally, **gPodder.net** credentials for automatic subscription sync.

The installer creates the directory tree, writes `config.json` (mode 600),
installs the `liquidsoap` systemd unit named after this directory, and adds
two cron jobs:

- Hourly at minute 0: fetch new podcast episodes.
- Hourly at minute 30: regenerate per-show playlists from playback history.

Start the station with:

    systemctl start <directory-name>

## Managing shows

List registered shows:

    python3 fetch_podcasts.py --list-shows --detail
    # or
    jruby fetch_podcasts.rb --list-shows --detail

Add a show directly from a feed URL:

    python3 fetch_podcasts.py --add-show https://example.com/feed.xml

Delete a show and its downloaded data:

    python3 fetch_podcasts.py --delete-show some_slug

Import an OPML export (e.g. from gpodder):

    python3 fetch_podcasts.py --import-opml ~/gpodder_export.opml

Shows imported via OPML are flagged and protected from pruning during
gPodder sync.

## Playback model

Background music plays continuously. Scheduled shows interrupt the music at
their appointed times; each scheduled entry in `schedule.txt` needs a
runlength so the scheduler knows when to hand control back to the music bed.
Episode runlengths are extracted from RSS enclosure data when available.

Playlists recursively scan nested directories for `.mp3` and `.m4a` files,
so you can organise your music however you like.

## Notes

- Gems for JRuby should be installed one at a time (separate `gem install`
  commands); bundling several in one command can hit memory limits.
- `config.json` is gitignored and contains secrets — keep it mode 600.
# Radio Automation Stack

A self-hosted internet radio automation system for Linux Mint 22.3 (Ubuntu 24.04 Noble). It continuously plays background music, interrupts it with scheduled podcast shows and external streams, downloads new podcast episodes from RSS feeds, and manages playback history — all fed to an Icecast server via liquidsoap.

Two complete, interchangeable implementations are provided:

- **Python** — `install_for_python`, `fetch_podcasts.py`, `update_playlists.py`
- **Ruby (JRuby)** — `install_for_jruby`, `fetch_podcasts.rb`, `update_playlists.rb`

Both read the same `config.json`, `schedule.txt`, and directory layout, and produce identical output. Pick one stack and stick with it; they share state, so don't run both simultaneously.

## Architecture at a glance

```
                 ┌──────────────┐
   schedule.txt ─▶│              │
   config.json ──▶│  liquidsoap  ├──▶ Icecast /audio.mp3 ──▶ listeners
   station.liq ──▶│  (radio.service)│
                 │              │
   playlists/*.pls ◀── update_playlists.{py,rb}
        ▲                        │
        │ selects unplayed       ▼
   podcasts/<slug>/      state/played.db   (playback history)
        ▲
        │ downloads new episodes
   fetch_podcasts.{py,rb}  ──▶ state/subscriptions.db  (show registry)
        ▲
   OPML file / gpodder.net sync
```

The moving parts:

- **liquidsoap** (`station.liq`) runs as a systemd service, streams MP3 to Icecast, and switches between continuous background music and scheduled content.
- **fetch_podcasts** (cron, hourly) registers shows, pulls new episode metadata from RSS, and downloads audio into `podcasts/<slug>/`.
- **update_playlists** (cron, hourly at :30) picks one unplayed episode per show, writes a `.pls` playlist, and records it as played.

## Directory layout

| Path | Purpose | Tracked in git? |
|------|---------|-----------------|
| `music/` | Background music library (scanned recursively) | No |
| `podcasts/` | Downloaded episodes, one subfolder per show slug | No |
| `jingles/` | Jingle audio | No |
| `announcements/` | Announcement audio | No |
| `playlists/` | Generated `.pls` files, one per show | No |
| `state/` | SQLite databases (`subscriptions.db`, `played.db`) | No |
| `logs/` | Rotated logs for each component | No |
| `config.json` | Credentials and connection settings | No (secret) |
| `schedule.txt` | Human-edited cron schedule of shows/streams | Yes |
| `station.liq` | liquidsoap configuration | Yes |

## Configuration: `config.json`

Generated by the installer, edited by hand afterward. Contains two sections:

```json
{
  "icecast": {
    "host": "localhost",
    "port": 7777,
    "mount": "/audio.mp3",
    "username": "source",
    "password": "SECRET"
  },
  "gpodder": {
    "enable": false,
    "host": "https://gpodder.net",
    "username": "",
    "password": ""
  }
}
```

The `icecast` block holds the source credentials liquidsoap uses to push the stream (the *source* password from your `icecast.xml`, not the admin password). The `gpodder` block enables optional automatic subscription syncing from gpodder.net; when `enable` is `false`, the fetcher works purely from locally registered shows or imported OPML.

Keep this file out of version control — it holds passwords. A `config.example.json` with placeholders can be committed instead if desired.

## The schedule: `schedule.txt`

Each non-comment line defines one recurring trigger. Fields are whitespace-separated:

```
min hour dom mon dow TYPE TARGET [RUNLENGTH_SECONDS]
```

- The first five fields are a standard cron expression.
- `TYPE` is either `show` or `stream`.
- `TARGET` is a show slug (for `show`) or a URL (for `stream`).
- `RUNLENGTH_SECONDS` is required for `stream` entries (how long to play before returning to music) and ignored for `show` entries.

Example:

```
# Weekday morning news, Tuesday 08:00
0 8 * * 2     show    hardcore_history
# Live remote stream, Monday 06:00 for one hour
0 6 * * 1     stream  http://example.org:8000/live.mp3   3600
```

Lines beginning with `#` and blank lines are skipped. Malformed lines are logged and ignored rather than aborting the load.

---

# Installation

There are two installers. Each is idempotent — safe to re-run after changes. Both must be run as root and must be executed from within the target directory (e.g. `/srv/audio/`); the service name is derived from that directory's basename.

## Choosing a stack

Pick based on what you want to maintain. The Python stack uses a virtualenv and CPython libraries; the Ruby stack provisions its own JDK + JRuby under `/opt` and integrates it system-wide via `update-alternatives`. Functionally they are equivalent.

## `install_for_python`

Provisions a Python venv, installs the Python dependencies (feedparser, requests, mutagen, sqlite3), creates the directory tree, generates `config.json`, writes the systemd unit, and sets up the cron jobs. Run it with:

```bash
sudo ./install_for_python
```

It walks through interactive prompts for the Icecast host/port/mount/source username/password and the gPodder sync settings, pre-filled from any existing `config.json` so re-runs preserve current values. It confirms a summary before making changes and exits cleanly if you decline.

## `install_for_jruby`

The JRuby equivalent, with additional provisioning logic:

1. Installs base packages (liquidsoap, icecast2, jq, curl, unzip).
2. **Detects Java.** If a JVM ≥ 21 is already present it is reused; otherwise OpenJDK 21 headless is installed.
3. **Detects JRuby.** Checks the pinned home first, then anything resolvable on `PATH`. Only if neither exists does it download JRuby 10.1.1.0 from Maven Central into `/opt`.
4. **Integrates via `update-alternatives`**, registering `/usr/local/bin/jruby` (plus `bundle`, `gem`, `rake`, `irb`) so the interpreter resolves consistently for cron, systemd, and shells alike.
5. **Installs gems one at a time** (bundler, nokogiri, sqlite3, json) with a raised JVM heap (`JRUBY_OPTS=-J-Xmx1g -J-Xss512k`) to avoid out-of-memory failures during multi-gem resolution on low-RAM hosts.
6. Creates directories, generates `config.json`, writes the systemd unit, and sets up cron — same as the Python installer.

Run it with:

```bash
sudo ./install_for_jruby
```

If you later swap JRuby versions, `update-alternatives --config jruby` switches them without touching any generated config, cron, or service files.

### Installer options and environment

Neither installer takes command-line flags; configuration is collected interactively. Behavior is driven by:

- **Existing `config.json`** — provides defaults for all prompts.
- **Target directory** — determines `INSTALL_DIR` and the resulting service name.
- **System state** — for the JRuby installer, existing Java/JRuby installations are detected and reused rather than overwritten.

After either installer completes, follow the printed next steps: drop music into `music/`, edit `schedule.txt`, review `station.liq`, then start the services.

---

# Tools

## `fetch_podcasts` (`.py` / `.rb`)

Fetches podcast episodes, manages the show registry, and downloads audio. Runs automatically via cron every hour, but supports manual invocation for administration.

Common prefix (adjust per stack):

```bash
# Python
sudo -u liquidsoap <venv>/bin/python /path/fetch_podcasts.py [options]
# Ruby
sudo -u liquidsoap env GEM_HOME=/path/.gems GEM_PATH=/path/.gems \
  /usr/local/bin/jruby -S bundle exec /path/fetch_podcasts.rb [options]
```

### Options

| Option | Argument | Effect |
|--------|----------|--------|
| *(none)* | — | Default mode. If gpodder sync is enabled, syncs subscriptions first, then fetches new episodes for all registered shows. |
| `--list-shows` | — | Prints a table of registered shows (name, slug, protected flag). |
| `--detail` | — | With `--list-shows`, additionally prints each show's feed URL. |
| `--add-show` | `<feed-url>` | Fetches a single feed, registers it, and immediately downloads its episodes. |
| `--delete-show` | `<slug>` | Removes a show from the registry, deletes its downloaded files, and removes its playlist. |
| `--import-opml` | `<file.opml>` | Imports show records from an OPML file (e.g. exported from gpodder). Imported shows are marked protected so they aren't pruned by gpodder sync. |

### Show registration and protection

Shows enter the registry three ways: manual `--add-show`, `--import-opml`, or gpodder.net sync. Shows added via OPML import or manual add carry an `opml_import` protection flag. When gpodder sync runs, it only prunes shows that were *originally* synced from gpodder.net and have since disappeared from the account — protected shows are never auto-deleted, even if absent from the subscription list.

### Playback duration

Episode durations are extracted from the RSS `enclosure length` attribute (RSS) or `media:duration` (Atom) and stored in seconds, so the scheduler knows how long each episode will run.

## `update_playlists` (`.py` / `.rb`)

Regenerates the per-show `.pls` playlist files based on playback history. Runs automatically via cron every hour at :30 (offset from the fetcher so fresh downloads are available).

```bash
# Python
sudo -u liquidsoap <venv>/bin/python /path/update_playlists.py [options]
# Ruby
sudo -u liquidsoap env GEM_HOME=/path/.gems GEM_PATH=/path/.gems \
  /usr/local/bin/jruby -S bundle exec /path/update_playlists.rb [options]
```

### Options

| Option | Argument | Effect |
|--------|----------|--------|
| *(none)* | — | For each show, selects an unplayed episode, writes `playlists/<slug>.pls`, and records it as played. |
| `--json` | — | Emits a JSON summary of each show's total file count and played count, then exits. Useful for monitoring. |

### Selection logic

For each show, the updater scans `podcasts/<slug>/` recursively for audio files, consults `state/played.db` to find which have already aired, and queues one unplayed episode. If everything has been played, it falls back to the least-recently-played episode so the stream never goes silent. The chosen file path is written to the show's `.pls`, and the selection is recorded so the next cycle advances to a different episode.

## `station.liq`

The liquidsoap program itself. Not invoked directly — it runs under the systemd service. Key behaviors:

- Resolves all paths relative to its own location via `configure.bindir()`, so the whole tree can be relocated without editing the file.
- Loads Icecast credentials from `config.json` using an annotated `json.parse` binding.
- Maintains a continuous random background-music playlist drawn from `music/`.
- Uses a request queue as the primary source: when a scheduled show or stream is triggered, it plays ahead of the music fallback; when the queue drains, background music resumes.

To reload after editing, restart the service:

```bash
sudo systemctl restart <servicename>
```

where `<servicename>` is the basename of your install directory.

---

# Operations

## Starting and stopping

```bash
sudo systemctl start icecast2
sudo systemctl start <servicename>
sudo systemctl status <servicename>
tail -f /path/logs/liquidsoap.log
```

Enable both at boot:

```bash
sudo systemctl enable icecast2 <servicename>
```

## Managing shows

```bash
# List all registered shows with details
... fetch_podcasts --list-shows --detail

# Add a show from its feed URL
... fetch_podcasts --add-show https://example.com/feed.xml

# Import a gpodder OPML export
... fetch_podcasts --import-opml ~/gpodder-subscriptions.opml

# Remove a show and all its data
... fetch_podcasts --delete-show some_show_slug
```

(Replace `...` with the appropriate interpreter prefix shown in the Tools section.)

## Monitoring

Check the cron logs for fetch/update activity:

```bash
tail -f /path/logs/fetch.log
tail -f /path/logs/update.log
```

Query playlist state as JSON:

```bash
... update_playlists --json
```

## Re-running an installer

Safe at any time. Existing `config.json` supplies prompt defaults, existing Java/JRuby (Ruby stack) or venv (Python stack) are detected and reused, and cron entries are deduplicated so re-runs don't create duplicates.