ts3j/ts3-client/README.md

# TS3J Client

A desktop TeamSpeak 3 client built on top of the [`ts3j`](../ts3j) reverse-engineered
TS3 protocol library. It looks and behaves like the official TS3 client and focuses
first on the features that matter most: **real-time voice with Opus encoding and
voice-activation detection (VAD)**.

The project is split so the frontend can be replaced (e.g. a future web UI via
TeaVM/CheerpJ) without touching the library:

| Module | Artifact | Responsibility |
|--------|----------|----------------|
| `core` | `ts3-client-core` | Frontend-agnostic library: protocol integration, server model, connection orchestration, audio **abstractions**. No UI, no platform audio, no native code. |
| `desktop` | `ts3-client-desktop` | Desktop audio backend: Java Sound capture/playback + native Opus via the FFM API (project Panama), implementing the core audio interfaces. |
| `swing` | `ts3-client-swing` | Swing desktop UI + entry point. Depends on `core` and `desktop`. |

The core exposes `AudioBackend` / `VoiceInput` / `VoiceOutput`; the frontend injects
a concrete backend (`JavaSoundAudioBackend`) into `TeamspeakConnection`. A different
frontend supplies its own UI and audio backend while reusing `core` unchanged.

## Features

### Voice (the priority)
- **Native Opus codec** via a direct Panama (Foreign Function & Memory API) binding —
  no JNA, no other third-party FFI. The system `libopus` is used when installed,
  otherwise a bundled copy is extracted from the JAR. Only the Windows x86-64 build
  is bundled by default (build with `-Dnatives.all` to package every platform);
  elsewhere install libopus from your package manager. Encoding at 48 kHz, 20 ms frames.
- **Voice Activation Detection (VAD)** with the same three modes as the TS3 client:
  - **Volume Gate** — RMS/dBFS threshold with a live input meter and hangover.
  - **Automatic** — a dependency-free speech detector (short-term energy + spectral
    flatness + dominant frequency against an adaptive noise floor).
  - **Hybrid** — transmit only when loud enough **and** detected as speech.
  - Optional **VAD over Push-To-Talk** (keep detecting voice while PTT is available).
- **Capture pre-processing** — a mini audio-processing chain in the same order as the
  TS3 client's WebRTC APM, applied before activation and encoding (and feeding the VAD):
  input gain → **high-pass filter** (80 Hz, always-on rumble/DC removal) → **noise
  suppression** → **typing attenuation** → **AGC**:
  - **Remove background noise** — spectral denoiser (decision-directed Wiener with
    minimum-statistics noise tracking), with an adjustable removal level.
  - **Typing attenuation** — detects impulsive keystroke transients (short, broadband,
    high-frequency bursts) and ducks them while leaving sustained speech intact.
  - **Automatic gain control (AGC)** — normalises mic loudness to a target level
    (fast attack / slow release, noise-gated so silence is never amplified).
  - Echo cancellation (WebRTC AEC3) is intentionally omitted — it needs the loudspeaker
    reference signal and matters mainly for open speakers, not the typical headset.
- **Push-to-Talk** — bind any key; transmits only while held (while the app is focused).
- **Continuous** transmission mode.
- **Playback mixing** — each speaker gets its own Opus decoder and audio line, so
  multiple simultaneous talkers are mixed and one slow decode never blocks others.
- **Whisper receive** — targeted voice is decoded and played like normal voice.
- Per-client **mute**, master **deafen**, adjustable mic gain / playback volume.
- **On-the-fly Opus tuning** — bitrate, complexity, VBR, FEC and voice/music codec
  can all be changed live from the options dialog and take effect on the running
  encoder immediately (no reconnect); a voice↔music switch transparently rebuilds
  the encoder because Opus fixes its application mode at creation.
- Configurable capture/playback **devices**.

### Server interaction
- Connect to any TS3 server (an identity is generated on first use if none exists).
- **Channel/client tree** styled like TS3, updated live from protocol events
  (joins, leaves, moves, channel create/edit/delete, nickname/mute/away changes).
- **Talk indicators** — speakers turn green live as they talk.
- **Info panel** — selecting a channel shows its topic and **description** (fetched
  on demand); selecting a client shows its **server groups** and **channel group**
  (names resolved from the server's group lists), talk power, platform and version.
- **Server-group badge** next to each client's nickname in the tree.
- **Spacer channels** — TS3 `[spacer]`/`[*spacer]`/`[c/l/r spacer]` names render as
  non-interactive separators (fill, centred, aligned).
- Double-click a channel to **join**; right-click a client to **poke**, open a
  **private chat**, or **locally mute** them.
- **Chat** to the current channel or the whole server; receive channel/server/private
  messages and **pokes**.
- **Server bookmarks** — quick-connect menu with add/edit/remove management, each
  optionally pinned to a specific identity.
- **Identity management** (Tools → Identities) — keep several identities, mark one
  as the default, pick one per server or per bookmark, rename, raise an identity's
  security level, and import/export TeamSpeak-compatible `.ini` identity files.
- **Self status** — Away (with message) and Channel Commander toggles.
- **Status bar** shows the server name, user count and live ping.
- Change your nickname, mute/deafen from the toolbar.

### Notification sounds
- **Sound packs** in TeamSpeak's own format: a folder of waves plus a `settings.ini`
  mapping actions (`CONNECTION_CONNECTED`, `CLIENT_MOVED_TO_CURRENT_CHANNEL_STAYS`, …)
  to `play("file.wav")` entries, with `${clientType}`/`${channelname}`/… placeholders
  resolved per event. Packs are picked up from `~/.ts3jclient/sound`, an installed
  TeamSpeak 3 client (`$TS3_CLIENT_DIR` or the usual install paths) and a folder of
  your choosing, so the official packs work unchanged.
- **Per-action configuration** (Options → Notifications): switch any action's sound
  off, and mark actions as **important** (shown in bold, as in TS3) — important
  actions are the only ones still played while the speakers are muted.
- Sounds are mixed onto one playback line, so overlapping events never fight over
  the device; pack volume is separate from the voice volume.

### Icon packs
- **Icon packs** in TeamSpeak's own format: a `.zip` (or unpacked folder) of SVG/PNG
  artwork plus a `settings.ini` mapping icon keys (`CHANNEL_GREEN`, `PLAYER_ON`,
  `CONNECT`, …) to files. Packs are picked up from `~/.ts3jclient/gfx`, an installed
  TeamSpeak 3 client (`$TS3_CLIENT_DIR` or the usual install paths) and a folder of
  your choosing, so the official packs (`default_colored_2014.zip`,
  `default_mono_2014.zip`, the legacy `default.zip`) work unchanged.
- The pack draws the tree (server, channel, client state), the toolbar, the menus,
  the context menus, the file browser and TeamSpeak's default group icons. Vector
  art is rasterised at the size it is drawn at; a pack's `FALLBACK` option decides
  whether icons it lacks come from `default.zip`, and anything still missing falls
  back to the icons the client draws itself.
- **Pick one in Options → Design**, which also shows a viewer of everything the pack
  contains; switching redraws the window right away.

## Requirements
- Java 26+ (developed/tested on Temurin 26)
- The native Opus library on the system:
  - Debian/Ubuntu: `sudo apt install libopus0`
  - Arch: `sudo pacman -S opus`
  - macOS: `brew install opus`

## Building
The client depends on `ts3j`, so install that into your local Maven repo first:

```bash
cd ../ts3j && mvn -DskipTests install
cd ../ts3-client && mvn -DskipTests package
```

This produces a runnable fat-jar at `swing/target/ts3-client.jar`.

## Running
```bash
java -jar swing/target/ts3-client.jar
# or, during development:
mvn -pl swing exec:java
```

Then use **Connections → Connect…**, enter a server address, port (default 9987),
nickname and identity, and connect. Open **Tools → Options** to pick audio devices and tune
voice activation while watching the live meter.

## Architecture
```
core/   com.ts3client
├── config.Settings          persisted prefs (~/.ts3jclient/settings.properties)
├── config.Bookmarks         persisted server bookmarks (with per-server identity)
├── config.IdentityStore     managed identities (~/.ts3jclient/identities/*.ini)
├── audio                    abstractions + reusable DSP (no platform code)
│   ├── AudioBackend         factory for a platform's VoiceInput/VoiceOutput
│   ├── VoiceInput           capture source (extends ts3j Microphone) + gating controls
│   ├── VoiceOutput          voice-packet playback sink
│   ├── OpusParameters       live-tunable encoder settings
│   └── SpeechDetector       feature-based speech-probability VAD (Automatic/Hybrid)
├── gfx
│   ├── IconPack             an icon pack zip/folder + its settings.ini mapping
│   └── IconPacks            discovery of installed packs
├── sound
│   ├── SoundEvent           catalogue of actions (TeamSpeak's own event ids)
│   ├── SoundPack            a pack folder + its settings.ini mapping
│   ├── SoundPacks           discovery of installed packs
│   ├── NotificationSettings per-action enabled / important flags
│   ├── SoundNotifier        decides what is heard (pack + config + mute state)
│   └── SoundPlayer          platform hook for rendering a sound
└── net
    ├── TeamspeakConnection  ties socket + audio backend + model, translates events
    ├── ServerModel          thread-safe channel/client state
    ├── ChannelNode/ClientEntry  view models
    └── ConnectionListener   frontend callbacks

desktop/  com.ts3client.audio.desktop
├── Opus                     Panama (FFM) binding to native libopus
├── OpusEncoder/OpusDecoder  thin codec wrappers
├── AudioDevices             device enumeration + line opening (48 kHz/16-bit)
├── JavaSoundVoiceInput      capture + VAD/PTT gating + Opus encode
├── JavaSoundVoiceOutput     per-client Opus decode + playback + mixing
├── WavSoundPlayer           sound-pack playback: decode, resample and mix on one line
└── JavaSoundAudioBackend    wires the above into the core AudioBackend

swing/  com.ts3client
├── Main                     entry point (look & feel, settings, backend injection)
└── ui
    ├── MainFrame            window: menu, toolbar, tree | chat, status bar
    ├── ServerTreePanel      TS3-style channel/client tree (group badges, spacers)
    ├── Spacers              TS3 spacer-channel name parsing/rendering
    ├── InfoPanel            channel description / client group + details view
    ├── ChatPanel            chat log + input
    ├── SettingsDialog       audio + VAD options with live meter
    ├── NotificationsPanel   sound pack + per-action sound/important configuration
    ├── IconPackPanel        icon pack chooser + icon viewer (Options → Design)
    ├── ConnectDialog        connect form
    ├── BookmarksDialog      manage saved servers
    ├── IdentitiesDialog     manage identities (new/import/export/improve)
    ├── IdentityChooser      identity drop-down shared by connect/bookmark forms
    ├── LevelMeter           dBFS meter with threshold marker
    ├── IconTheme            active icon pack: rasterising (JSVG) and caching
    ├── Icons                icon lookup by TeamSpeak key, with drawn fallbacks
    └── Theme                palette + fonts
```

## Known limitations / next steps
- The **Automatic/Hybrid** VAD uses a lightweight energy/spectral detector rather
  than the WebRTC GMM model the official client ships; it is intentionally
  dependency-free and reusable in the core library.
- Whisper is received/played but not yet **sendable** from the UI.
- Playback decodes streams as mono; stereo music-bot audio is down-mixed.
- Push-to-talk is captured via Swing key events, so it only works while the app
  window has focus (no global hotkey).
- No file transfer, avatars, or server/channel administration UI yet.