ts3j/ts3-client/README.md

# TS3J Client

A desktop TeamSpeak 3 client built on top of the [`ts3j`](../ts3j) reverse-engineered
TS3 protocol library. It looks and behaves like the official TS3 client and focuses
first on the features that matter most: **real-time voice with Opus encoding and
voice-activation detection (VAD)**.

The project is split so the frontend can be replaced (e.g. a future web UI via
TeaVM/CheerpJ) without touching the library:

| Module | Artifact | Responsibility |
|--------|----------|----------------|
| `core` | `ts3-client-core` | Frontend-agnostic library: protocol integration, server model, connection orchestration, audio **abstractions**. No UI, no platform audio, no JNA. |
| `desktop` | `ts3-client-desktop` | Desktop audio backend: Java Sound capture/playback + native Opus via JNA, implementing the core audio interfaces. |
| `swing` | `ts3-client-swing` | Swing desktop UI + entry point. Depends on `core` and `desktop`. |

The core exposes `AudioBackend` / `VoiceInput` / `VoiceOutput`; the frontend injects
a concrete backend (`JavaSoundAudioBackend`) into `TeamspeakConnection`. A different
frontend supplies its own UI and audio backend while reusing `core` unchanged.

## Features

### Voice (the priority)
- **Native Opus codec** via a direct JNA binding to the system `libopus`
  (no bundled/native-jar dependency). Encoding at 48 kHz, 20 ms frames.
- **Voice Activation Detection (VAD)** with the same three modes as the TS3 client:
  - **Volume Gate** — RMS/dBFS threshold with a live input meter and hangover.
  - **Automatic** — a dependency-free speech detector (short-term energy + spectral
    flatness + dominant frequency against an adaptive noise floor).
  - **Hybrid** — transmit only when loud enough **and** detected as speech.
  - Optional **VAD over Push-To-Talk** (keep detecting voice while PTT is available).
- **Capture pre-processing** — a mini audio-processing chain in the same order as the
  TS3 client's WebRTC APM, applied before activation and encoding (and feeding the VAD):
  input gain → **high-pass filter** (80 Hz, always-on rumble/DC removal) → **noise
  suppression** → **typing attenuation** → **AGC**:
  - **Remove background noise** — spectral denoiser (decision-directed Wiener with
    minimum-statistics noise tracking), with an adjustable removal level.
  - **Typing attenuation** — detects impulsive keystroke transients (short, broadband,
    high-frequency bursts) and ducks them while leaving sustained speech intact.
  - **Automatic gain control (AGC)** — normalises mic loudness to a target level
    (fast attack / slow release, noise-gated so silence is never amplified).
  - Echo cancellation (WebRTC AEC3) is intentionally omitted — it needs the loudspeaker
    reference signal and matters mainly for open speakers, not the typical headset.
- **Push-to-Talk** — bind any key; transmits only while held (while the app is focused).
- **Continuous** transmission mode.
- **Playback mixing** — each speaker gets its own Opus decoder and audio line, so
  multiple simultaneous talkers are mixed and one slow decode never blocks others.
- **Whisper receive** — targeted voice is decoded and played like normal voice.
- Per-client **mute**, master **deafen**, adjustable mic gain / playback volume.
- **On-the-fly Opus tuning** — bitrate, complexity, VBR, FEC and voice/music codec
  can all be changed live from the options dialog and take effect on the running
  encoder immediately (no reconnect); a voice↔music switch transparently rebuilds
  the encoder because Opus fixes its application mode at creation.
- Configurable capture/playback **devices**.

### Server interaction
- Connect to any TS3 server (auto-generates and persists a TS3 identity).
- **Channel/client tree** styled like TS3, updated live from protocol events
  (joins, leaves, moves, channel create/edit/delete, nickname/mute/away changes).
- **Talk indicators** — speakers turn green live as they talk.
- **Info panel** — selecting a channel shows its topic and **description** (fetched
  on demand); selecting a client shows its **server groups** and **channel group**
  (names resolved from the server's group lists), talk power, platform and version.
- **Server-group badge** next to each client's nickname in the tree.
- **Spacer channels** — TS3 `[spacer]`/`[*spacer]`/`[c/l/r spacer]` names render as
  non-interactive separators (fill, centred, aligned).
- Double-click a channel to **join**; right-click a client to **poke**, open a
  **private chat**, or **locally mute** them.
- **Chat** to the current channel or the whole server; receive channel/server/private
  messages and **pokes**.
- **Server bookmarks** — quick-connect menu with add/edit/remove management.
- **Self status** — Away (with message) and Channel Commander toggles.
- **Status bar** shows the server name, user count and live ping.
- Change your nickname, mute/deafen from the toolbar.

## Requirements
- Java 17+ (developed/tested on Temurin 26)
- The native Opus library on the system:
  - Debian/Ubuntu: `sudo apt install libopus0`
  - Arch: `sudo pacman -S opus`
  - macOS: `brew install opus`

## Building
The client depends on `ts3j`, so install that into your local Maven repo first:

```bash
cd ../ts3j && mvn -DskipTests install
cd ../ts3-client && mvn -DskipTests package
```

This produces a runnable fat-jar at `swing/target/ts3-client.jar`.

## Running
```bash
java -jar swing/target/ts3-client.jar
# or, during development:
mvn -pl swing exec:java
```

Then use **Connections → Connect…**, enter a server address, port (default 9987)
and nickname, and connect. Open **Tools → Options** to pick audio devices and tune
voice activation while watching the live meter.

## Architecture
```
core/   com.ts3client
├── config.Settings          persisted prefs (~/.ts3jclient/settings.properties)
├── config.Bookmarks         persisted server bookmarks
├── audio                    abstractions + reusable DSP (no platform code)
│   ├── AudioBackend         factory for a platform's VoiceInput/VoiceOutput
│   ├── VoiceInput           capture source (extends ts3j Microphone) + gating controls
│   ├── VoiceOutput          voice-packet playback sink
│   ├── OpusParameters       live-tunable encoder settings
│   └── SpeechDetector       feature-based speech-probability VAD (Automatic/Hybrid)
└── net
    ├── TeamspeakConnection  ties socket + audio backend + model, translates events
    ├── ServerModel          thread-safe channel/client state
    ├── ChannelNode/ClientEntry  view models
    └── ConnectionListener   frontend callbacks

desktop/  com.ts3client.audio.desktop
├── Opus                     JNA binding to native libopus
├── OpusEncoder/OpusDecoder  thin codec wrappers
├── AudioDevices             device enumeration + line opening (48 kHz/16-bit)
├── JavaSoundVoiceInput      capture + VAD/PTT gating + Opus encode
├── JavaSoundVoiceOutput     per-client Opus decode + playback + mixing
└── JavaSoundAudioBackend    wires the above into the core AudioBackend

swing/  com.ts3client
├── Main                     entry point (look & feel, settings, backend injection)
└── ui
    ├── MainFrame            window: menu, toolbar, tree | chat, status bar
    ├── ServerTreePanel      TS3-style channel/client tree (group badges, spacers)
    ├── Spacers              TS3 spacer-channel name parsing/rendering
    ├── InfoPanel            channel description / client group + details view
    ├── ChatPanel            chat log + input
    ├── SettingsDialog       audio + VAD options with live meter
    ├── ConnectDialog        connect form
    ├── BookmarksDialog      manage saved servers
    ├── LevelMeter           dBFS meter with threshold marker
    ├── Icons                programmatic vector icons (no image assets)
    └── Theme                palette + fonts
```

## Known limitations / next steps
- The **Automatic/Hybrid** VAD uses a lightweight energy/spectral detector rather
  than the WebRTC GMM model the official client ships; it is intentionally
  dependency-free and reusable in the core library.
- Group display shows names; **group icons** are not rendered.
- Whisper is received/played but not yet **sendable** from the UI.
- Playback decodes streams as mono; stereo music-bot audio is down-mixed.
- Push-to-talk is captured via Swing key events, so it only works while the app
  window has focus (no global hotkey).
- No file transfer, avatars, or server/channel administration UI yet.