Compare commits

..

9 Commits

Author SHA1 Message Date
047c89404c Prefer RECORD over XInput2 on X11: raw button releases can go missing
On at least some driver stacks, XI_RawButtonRelease is never delivered
to a client that hasn't grabbed the pointer, even though the matching
XI_RawButtonPress arrives fine — a bound mouse button then looks stuck
down forever, both when recording a hotkey and when using one. Taking
an active XIGrabDevice grab does fix delivery, but even with
owner_events set it blocks clicks from reaching every other window, so
it's not usable. XRecordInputHook taps the same core events xev sees
and isn't affected, so it now goes first on X11; XInput2 stays as the
fallback for servers where RECORD is disabled or missing, and gets a
correctness cleanup (real per-device event selection instead of the
XIAllMasterDevices pseudo-device) along the way.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-17 10:34:19 +00:00
75ae6b89b2 Report mute/hardware status correctly, add Local Mic Mute, fix stale talking indicator
- Mute/deafen status now goes out through clientupdate instead of
  clientedit, which silently rejected client_input_muted/output_muted
  since they are runtime status, not editable client properties.
- Publish client_input_hardware so other clients see "Microphone
  Disabled" instead of silence while another tab holds the capture
  device; track input/output hardware flags for other clients too,
  and show a distinct grey "disabled" icon instead of reusing the red
  "muted" one for both the tree and the info panel.
- Implement TS3's Enable/Disable/Toggle Local Mic Mute hotkeys: they
  silence capture like a real mute, but never touch the published
  mute status or play a sound.
- A speaker's "talking" indicator only ever cleared when its
  zero-length end-of-burst voice packet arrived; if that one UDP
  packet was lost, the indicator stuck until their next burst. Add a
  watchdog that clears it after 200ms of silence from that speaker
  regardless.
2026-08-17 08:43:22 +00:00
e228dd4ad0 Replace RMS voice detection with TeamSpeak's own RNN VAD
Ports WebRTC's rnn_vad (as TS3 embeds it) to Java: LPC, pitch
estimation, spectral features and the RNN itself, feeding a
speech-probability detector that replaces the old SpeechDetector.
Also switches the volume-gate threshold from raw dBFS to
InputLevel's scale, matching TS3's own slider and range, with a
migration for settings saved under the old key.
2026-08-17 07:51:51 +00:00
752676e863 Capture global hotkeys on Windows through raw input
A message-only window registers the keyboard and mouse with
RIDEV_INPUTSINK, so every key and button arrives as WM_INPUT whether or
not the client is in front. Raw input only observes, where a
WH_KEYBOARD_LL hook sits in the path of the system input queue and can
swallow a keystroke; it also reports the side buttons and both edges of
every key, which push-to-talk needs.

Keyboard codes stay in each platform's own numbering — X keycodes on
X11, set-1 scan codes with the E0/E1 escape folded into the high byte on
Windows — since that is what the input API reports and the key-naming
call expects. Bindings are per-machine either way, as TeamSpeak's own
per-OS keydefs are. Mouse buttons are unified on the X numbering, so
"Mouse 4" means the same thing on both.

The RAWINPUT decoding lives in its own class so it can be tested off
Windows; the window and its pump have not been run on Windows yet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-16 18:40:50 +00:00
edc8a0ca3a Capture hotkeys through XInput2, as TeamSpeak does
The bundled TS3 Linux binary dlopens libX11 and libXi and drives
XIQueryVersion/XISelectEvents/XGetEventData — XInput2 raw events, with no
trace of the RECORD extension anywhere in its tree. That is the better
choice for us too: XInput2 is core input, present on any remotely modern
server, where RECORD is a debugging extension that is sometimes disabled
or left out of the build. Both are passive, so the keystroke still
reaches the focused window either way.

XRecordInputHook stays as the fallback behind it. Raw events also explain
why TS3's hotkeys work under KWin's Wayland session despite it using no
Wayland API at all: it runs on Xwayland, and KWin's legacy X11 app
support forwards the keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-16 18:33:32 +00:00
7e4b9671fe Global hotkeys, in TeamSpeak's own shape
Bindings are captured system-wide rather than only while the window has
focus: X11's RECORD extension where the X server sees every key, and a
/dev/input reader as the Wayland fallback. Any key can act as a modifier,
mouse buttons included, as TS3 allows.

The action catalogue, its three categories and the "advanced actions"
split are reverse-engineered from the original client; actions this
client cannot perform are listed but greyed out. Push-to-talk becomes one
of these hotkeys, so the old focus-bound pushToTalkKey setting is gone and
the Voice Activation button edits that binding instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-16 17:08:28 +00:00
3e2c06248f Make the tray menu close, and its window come forward
Swing never grabs the pointer for a popup, so a click on the desktop or
another application is not seen and the menu stays up. The little window
the menu hangs off is focusable now and closes it when the focus goes
elsewhere, Escape closes it, and clicking the icon again toggles it.

Raising the window was ignored for the same reason it usually is: a
window manager refuses a raise from an application that does not have the
focus. The icon asks for _NET_ACTIVE_WINDOW instead — the request meant
for pagers and trays acting for the user — and falls back to Swing's
always-on-top shuffle when it cannot find the window. Windows are matched
by _NET_WM_PID and _NET_WM_NAME, since WM_NAME cannot carry the em dash
in our title.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 22:58:24 +00:00
32e5ffb576 Show the local client's state in the system tray
The icon carries the badge the tree draws next to our own nickname on the
active server — idle, talking, away, commander, microphone or speakers
muted — and clicking it brings the window back to the front. Right click
offers "Show TS3J" and "Quit".

AWT's tray icon cannot be transparent on X11: the toolkit embeds a window
of the screen's default, opaque visual and fills it with a background
colour before drawing the image, so every icon sits in a box. Panels that
can show transparent icons advertise an ARGB visual instead, so the icon
is docked by hand over the system tray protocol, in that visual, with the
image put on the window premultiplied. AWT's own icon stays as the
fallback for everything else.

Two details a panel will not forgive: an icon that publishes no size
hints is allocated a one-pixel sliver, and docking is a request, so an
icon that is never adopted has to hand over to the fallback rather than
sit invisible.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 22:45:32 +00:00
ef95b6305d Report a chosen client version and operating system
Other clients see the version and platform from clientinit in the info
panel, and until now that was whatever ts3j sends. The new Client Version
tab picks one instead.

Servers refuse a version they do not know with "client is modified", so
free text is not enough: the list holds genuine version/platform/signature
triples from released clients, newest first per platform. The fields stay
editable for a version whose signature is known from elsewhere, and a
refused connection now says where to change it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 20:06:09 +00:00
67 changed files with 8591 additions and 331 deletions

View File

@@ -80,6 +80,35 @@ frontend supplies its own UI and audio backend while reusing `core` unchanged.
- **Status bar** shows the server name, user count and live ping.
- Change your nickname, mute/deafen from the toolbar.
### Hotkeys
- **Global hotkeys**, as in TeamSpeak 3: single keys or combinations (any key can act
as the modifier) and mouse buttons including *Mouse 4* / *Mouse 5*, working
system-wide rather than only while the window has focus. Configure them in
**Options → Hotkeys**.
- Per binding you choose whether it **triggers on key down or on key up**, and whether
it applies **on the active server** only or to every connected one (checkbox off).
Push-to-talk style actions ignore the trigger setting and simply last while held.
- The action list is TeamSpeak's own, reverse-engineered from its hotkey dialog:
three categories (*Server*, *Self*, *Misc*) and a **"Show advanced actions"**
checkbox that reveals the rest, exactly as the original hides all but the everyday
actions. Actions this client does not implement are listed but greyed out.
- Push-to-talk is one of those hotkeys; the button in **Options → Voice Activation**
edits that binding.
- On Linux, capture uses **XInput2 raw events**, as TeamSpeak's own client does: no
privileges needed and the keystroke is not swallowed. X11's RECORD extension is kept
as a fallback for servers without XInput2.
- On **Windows**, capture uses **raw input** (`RIDEV_INPUTSINK`) through a message-only
window: it observes rather than intercepts, so unlike a low-level keyboard hook it
cannot swallow a keystroke or stall the system input queue, and it reports the side
buttons and both edges of every key. Bindings are stored as scan codes, so they follow
the physical key rather than the layout.
- On Wayland the hooks run inside Xwayland, so they see every X11 application, and keys
aimed at native Wayland windows only where the compositor forwards them — under KWin
that is *System Settings → Window Management → Legacy X11 App Support*, which is why
TeamSpeak's hotkeys work there. Where it forwards nothing, the client reads
`/dev/input/event*` instead, for which your user has to be in the `input` group.
The Hotkeys tab says which backend is in use, or why none is.
### Notification sounds
- **Sound packs** in TeamSpeak's own format: a folder of waves plus a `settings.ini`
mapping actions (`CONNECTION_CONNECTED`, `CLIENT_MOVED_TO_CURRENT_CHANNEL_STAYS`, …)
@@ -151,6 +180,12 @@ core/ com.ts3client
├── gfx
│ ├── IconPack an icon pack zip/folder + its settings.ini mapping
│ └── IconPacks discovery of installed packs
├── hotkey
│ ├── HotkeyAction catalogue of TeamSpeak's hotkey actions (RE'd from its binary)
│ ├── Hotkey/HotkeyCombo one binding: keys, trigger edge, server scope, argument
│ ├── Hotkeys persisted bindings (~/.ts3jclient/hotkeys.properties)
│ ├── GlobalInputHook platform hook interface: system-wide key/button events
│ └── HotkeyEngine held-key tracking, combination matching, recording
├── sound
│ ├── SoundEvent catalogue of actions (TeamSpeak's own event ids)
│ ├── SoundPack a pack folder + its settings.ini mapping
@@ -164,14 +199,22 @@ core/ com.ts3client
├── ChannelNode/ClientEntry view models
└── ConnectionListener frontend callbacks
desktop/ com.ts3client.audio.desktop
desktop/ com.ts3client.audio.desktop + com.ts3client.hotkey.desktop
├── Opus Panama (FFM) binding to native libopus
├── OpusEncoder/OpusDecoder thin codec wrappers
├── AudioDevices device enumeration + line opening (48 kHz/16-bit)
├── JavaSoundVoiceInput capture + VAD/PTT gating + Opus encode
├── JavaSoundVoiceOutput per-client Opus decode + playback + mixing
├── WavSoundPlayer sound-pack playback: decode, resample and mix on one line
── JavaSoundAudioBackend wires the above into the core AudioBackend
── JavaSoundAudioBackend wires the above into the core AudioBackend
└── hotkey.desktop
├── XInput2InputHook global key/button capture via XInput2 raw events
├── XRecordInputHook the same via X11's RECORD extension, as a fallback
├── EvdevInputHook /dev/input fallback for Wayland sessions
├── WindowsInputHook the same on Windows: raw input into a message-only window
├── RawInput RAWINPUT layout and decoding, split out to be testable
├── X11KeyNamer/WindowsKeyNamer layout-aware key labels per platform
└── DesktopInputHooks picks the backend that suits the session
swing/ com.ts3client
├── Main entry point (look & feel, settings, backend injection)
@@ -182,6 +225,10 @@ swing/ com.ts3client
├── InfoPanel channel description / client group + details view
├── ChatPanel chat log + input
├── SettingsDialog audio + VAD options with live meter
├── HotkeysPanel hotkey list (Options → Hotkeys)
├── HotkeyDialog add/edit one hotkey: action, combination, trigger, scope
├── HotkeyService bindings + engine + input hook, for the dialogs
├── HotkeyActions carries a fired hotkey out on the client
├── NotificationsPanel sound pack + per-action sound/important configuration
├── IconPackPanel icon pack chooser + icon viewer (Options → Design)
├── ConnectDialog connect form
@@ -200,6 +247,15 @@ swing/ com.ts3client
dependency-free and reusable in the core library.
- Whisper is received/played but not yet **sendable** from the UI.
- Playback decodes streams as mono; stereo music-bot audio is down-mixed.
- Push-to-talk is captured via Swing key events, so it only works while the app
window has focus (no global hotkey).
- Hotkey actions TeamSpeak offers but this client does not perform yet (listed but
greyed out in the dialog): capture/playback/hotkey profiles and sound packs,
whisper and push-to-whisper, recording, plugins, server groups, talk power, 3D
sound, hardware ("local") microphone mute and the channel-traversal variants
beyond "Switch to Channel".
- Global hotkeys cover Linux and Windows; **macOS has no backend**, so hotkeys there are
inert (a `CGEventTap`, which needs Accessibility permission, is the way in). The
Windows backend is written but has not been run on Windows — only its decoding is
covered by tests.
- When no backend can start, push-to-talk does not work at all, not even with the window
focused; a focus-scoped fallback hook would restore the pre-hotkey behaviour.
- No file transfer, avatars, or server/channel administration UI yet.

View File

@@ -0,0 +1,17 @@
package com.ts3client.audio;
/** Receives frames of PCM audio as they pass through the capture or playback chain. */
@FunctionalInterface
public interface AudioFrameListener {
/**
* Called with one frame of interleaved samples in [-1, 1].
*
* <p>The array is reused between calls, so copy anything you need to keep. Called from
* the audio thread: do not block.
*
* @param interleaved samples, {@code channels} per sample position
* @param channels number of interleaved channels
*/
void onFrame(float[] interleaved, int channels);
}

View File

@@ -0,0 +1,58 @@
package com.ts3client.audio;
/**
* Capture level on the scale the TeamSpeak 3 client puts on its voice-activation slider.
*
* <p>TS3 stores {@code voiceactivation_level} in decibels but compares powers: its setter
* converts the slider value with {@code 10^(dB/10) * 0.0095} and its getter reports a
* measured power back with {@code 10*log10(power / 0.0095)}. The same reference constant is
* used here so a threshold of &minus;40 means the same thing it does in TeamSpeak.
*
* <p>The resulting scale runs from {@link #MIN_DB} to {@link #MAX_DB}, with 0&nbsp;dB
* corresponding to roughly &minus;20&nbsp;dBFS and the default &minus;40 to about
* &minus;60&nbsp;dBFS.
*/
public final class InputLevel {
/**
* Mean-square level that reads as 0 dB, taken from the client's own dB conversion.
*/
private static final double REFERENCE_POWER = 0.0095;
/** Lower end of the slider range. */
public static final double MIN_DB = -50.0;
/** Upper end of the slider range. */
public static final double MAX_DB = 50.0;
/** Reported when the frame is digital silence. */
public static final double SILENCE_DB = MIN_DB;
private InputLevel() {
}
/** Level of {@code length} mono samples in [-1, 1], in dB on the scale above. */
public static double toDb(float[] frame, int length) {
if (length <= 0) {
return SILENCE_DB;
}
double sumSq = 0;
for (int i = 0; i < length; i++) {
sumSq += (double) frame[i] * frame[i];
}
return powerToDb(sumSq / length);
}
/** Converts a mean-square power to the slider's dB scale. */
public static double powerToDb(double power) {
if (power <= 0) {
return SILENCE_DB;
}
return Math.max(SILENCE_DB, 10.0 * Math.log10(power / REFERENCE_POWER));
}
/** Inverse of {@link #powerToDb}, matching the client's threshold conversion. */
public static double dbToPower(double db) {
return Math.pow(10.0, db / 10.0) * REFERENCE_POWER;
}
}

View File

@@ -1,112 +0,0 @@
package com.ts3client.audio;
/**
* Lightweight speech-presence detector used by the "Automatic" and "Hybrid" voice
* activation modes (the TeamSpeak 3 client uses a WebRTC GMM detector for the same
* purpose). It classifies each 20&nbsp;ms frame from three features — short-term
* energy, spectral flatness and dominant frequency — against an adaptive noise
* floor, following Moattar &amp; Homayounpour's real-time VAD.
*
* <p>Pure DSP with no platform dependencies, so any frontend/backend can reuse it.
* A frame votes for speech when at least two of the three features exceed the
* baseline; the smoothed vote fraction is exposed as a [0,1] probability.
*/
public final class SpeechDetector {
private static final int FFT_SIZE = 1024;
// Primary thresholds from the reference algorithm (16-bit sample scale).
private static final double ENERGY_PRIM = 40.0;
private static final double DOMINANT_FREQ_PRIM = 185.0;
private static final double FLATNESS_PRIM = 5.0;
private final int sampleRate;
private final double[] re = new double[FFT_SIZE];
private final double[] im = new double[FFT_SIZE];
private double minEnergy;
private double minDominantFreq;
private double minFlatness;
private long silenceFrames;
private long frameCount;
private double probability;
public SpeechDetector(int sampleRate) {
this.sampleRate = sampleRate;
}
public void reset() {
frameCount = 0;
silenceFrames = 0;
probability = 0;
}
public double getProbability() {
return probability;
}
/**
* Processes one frame of mono PCM in [-1,1] and returns the smoothed speech
* probability in [0,1].
*/
public double process(float[] frame) {
double energy = 0;
int n = Math.min(frame.length, FFT_SIZE);
for (int i = 0; i < n; i++) {
double s = frame[i] * 32768.0; // emulate 16-bit scale
energy += s * s;
re[i] = frame[i];
im[i] = 0;
}
for (int i = n; i < FFT_SIZE; i++) {
re[i] = 0;
im[i] = 0;
}
Fft.forward(re, im);
int half = FFT_SIZE / 2;
double geoLogSum = 0;
double arithSum = 0;
double maxMag = 0;
int maxBin = 0;
for (int k = 1; k < half; k++) {
double mag = Math.sqrt(re[k] * re[k] + im[k] * im[k]) + 1e-12;
geoLogSum += Math.log(mag);
arithSum += mag;
if (mag > maxMag) {
maxMag = mag;
maxBin = k;
}
}
int bins = half - 1;
double geoMean = Math.exp(geoLogSum / bins);
double arithMean = arithSum / bins;
double flatness = -10.0 * Math.log10(geoMean / arithMean); // high = tonal (speech)
double dominantFreq = (double) maxBin * sampleRate / FFT_SIZE;
if (frameCount == 0) {
minEnergy = Math.max(energy, 1.0);
minDominantFreq = dominantFreq;
minFlatness = flatness;
}
double energyThresh = ENERGY_PRIM * Math.log10(Math.max(minEnergy, 1.0));
int votes = 0;
if (energy - minEnergy >= energyThresh) votes++;
if (dominantFreq - minDominantFreq >= DOMINANT_FREQ_PRIM) votes++;
if (flatness - minFlatness >= FLATNESS_PRIM) votes++;
boolean speech = votes >= 2;
if (!speech) {
// Adapt the noise floor towards the current (silent) energy.
silenceFrames++;
minEnergy = ((silenceFrames * minEnergy) + energy) / (silenceFrames + 1);
}
double instant = votes / 3.0;
probability = 0.6 * probability + 0.4 * instant;
frameCount++;
return probability;
}
}

View File

@@ -0,0 +1,27 @@
package com.ts3client.audio;
/**
* Estimates how likely it is that a frame of captured audio contains speech.
*
* <p>Used by the "Automatic" and "Hybrid" voice-activation modes. Implementations are
* stateful and expect frames in capture order; {@link #reset()} clears that state.
*
* <p>Pure DSP with no platform dependencies, so any frontend or backend can reuse it.
*/
public interface SpeechProbabilityDetector {
/**
* Consumes one frame of mono PCM in [-1, 1] and returns the current speech
* probability in [0, 1].
*
* @param frame samples in capture order
* @param length number of valid samples in {@code frame}
*/
double process(float[] frame, int length);
/** Speech probability from the most recent {@link #process} call. */
double probability();
/** Drops all history. Call on a capture discontinuity or a settings change. */
void reset();
}

View File

@@ -18,11 +18,20 @@ public interface VoiceInput extends Microphone {
void setMuted(boolean muted);
/**
* Silences capture the same way {@link #setMuted} does, but locally only: it does
* not touch {@link #isMuted}, so the mute status published to the server (and the
* mute/unmute sound) is unaffected. TS3's "Local Mic Mute".
*/
void setLocalMuted(boolean muted);
boolean isLocalMuted();
void setMode(Settings.InputMode mode);
void setVadMode(Settings.VadMode mode);
/** Voice-activation volume-gate threshold in dBFS. */
/** Volume-gate threshold on {@link InputLevel}'s scale. */
void setThresholdDb(double db);
/** Speech-probability threshold (0..1) for Automatic/Hybrid modes. */
@@ -50,12 +59,19 @@ public interface VoiceInput extends Microphone {
void setPushToTalk(boolean pressed);
/** Receives the live input level in dBFS, once per captured frame. */
/** Receives the live input level on {@link InputLevel}'s scale, once per captured frame. */
void setLevelListener(Consumer<Double> listener);
/** Receives local transmit-state transitions (talking / silent). */
void setTalkListener(Consumer<Boolean> listener);
/**
* Receives every frame that is actually transmitted, for local monitoring: the audio
* after pre-processing and gating, including the pre-roll replayed when the gate opens.
* Nothing is delivered while the gate is shut or the microphone is muted.
*/
void setMonitorListener(AudioFrameListener listener);
/** Called when speech is detected while the microphone is muted. */
void setMutedTalkListener(Runnable listener);
}

View File

@@ -0,0 +1,99 @@
package com.ts3client.audio.vad;
import static com.ts3client.audio.vad.VadConstants.NUM_LPC_COEFFICIENTS;
/**
* Linear-prediction analysis for the pitch search, ported from WebRTC's
* {@code rnn_vad/lp_residual.cc}.
*
* <p>Whitening the signal before searching for a pitch makes the auto-correlation peak at
* the true period rather than at whichever formant happens to dominate.
*/
final class LinearPrediction {
private final float[] autoCorr = new float[NUM_LPC_COEFFICIENTS];
private final float[] reflection = new float[NUM_LPC_COEFFICIENTS - 1];
/** Computes the 5 post-processed inverse-filter coefficients for {@code x}. */
void computeLpcCoefficients(float[] x, float[] lpcCoeffs) {
for (int lag = 0; lag < NUM_LPC_COEFFICIENTS; lag++) {
float sum = 0;
for (int i = 0; i < x.length - lag; i++) {
sum += x[i] * x[i + lag];
}
autoCorr[lag] = sum;
}
if (autoCorr[0] == 0.0f) { // Empty frame.
java.util.Arrays.fill(lpcCoeffs, 0.0f);
return;
}
// Assume a -40 dB white noise floor.
autoCorr[0] *= 1.0001f;
autoCorr[1] -= autoCorr[1] * 0.000064f;
autoCorr[2] -= autoCorr[2] * 0.000256f;
autoCorr[3] -= autoCorr[3] * 0.000576f;
autoCorr[4] -= autoCorr[4] * 0.001024f;
java.util.Arrays.fill(reflection, 0.0f);
computeInitialInverseFilterCoefficients();
// Bandwidth expansion, then convolution with (1 + 0.8 z^-1).
reflection[0] *= 0.9f;
reflection[1] *= 0.9f * 0.9f;
reflection[2] *= 0.9f * 0.9f * 0.9f;
reflection[3] *= 0.9f * 0.9f * 0.9f * 0.9f;
final float c = 0.8f;
lpcCoeffs[0] = reflection[0] + c;
lpcCoeffs[1] = reflection[1] + c * reflection[0];
lpcCoeffs[2] = reflection[2] + c * reflection[1];
lpcCoeffs[3] = reflection[3] + c * reflection[2];
lpcCoeffs[4] = c * reflection[3];
}
/** Levinson&ndash;Durbin recursion over {@link #autoCorr}, writing {@link #reflection}. */
private void computeInitialInverseFilterCoefficients() {
float error = autoCorr[0];
for (int i = 0; i < NUM_LPC_COEFFICIENTS - 1; i++) {
float reflectionCoeff = 0;
for (int j = 0; j < i; j++) {
reflectionCoeff += reflection[j] * autoCorr[i - j];
}
reflectionCoeff += autoCorr[i + 1];
// Avoid dividing by numbers close to zero.
final float minErrorMagnitude = 1e-6f;
if (Math.abs(error) < minErrorMagnitude) {
error = Math.copySign(minErrorMagnitude, error);
}
reflectionCoeff /= -error;
reflection[i] = reflectionCoeff;
for (int j = 0; j < ((i + 1) >> 1); j++) {
float tmp1 = reflection[j];
float tmp2 = reflection[i - 1 - j];
reflection[j] = tmp1 + reflectionCoeff * tmp2;
reflection[i - 1 - j] = tmp2 + reflectionCoeff * tmp1;
}
error -= reflectionCoeff * reflectionCoeff * error;
if (error < 0.001f * autoCorr[0]) {
break;
}
}
}
/**
* Applies the inverse filter: {@code y[i] = x[i] + sum_j lpc[j] * x[i - 1 - j]},
* with the sum truncated near the start of the buffer.
*/
static void computeLpResidual(float[] lpcCoeffs, float[] x, float[] y) {
for (int i = 0; i < x.length; i++) {
float sum = x[i];
int taps = Math.min(i, NUM_LPC_COEFFICIENTS);
for (int j = 0; j < taps; j++) {
sum += lpcCoeffs[j] * x[i - 1 - j];
}
y[i] = sum;
}
}
}

View File

@@ -0,0 +1,329 @@
package com.ts3client.audio.vad;
import static com.ts3client.audio.vad.VadConstants.BUF_SIZE_12K;
import static com.ts3client.audio.vad.VadConstants.FRAME_20MS_12K;
import static com.ts3client.audio.vad.VadConstants.FRAME_20MS_24K;
import static com.ts3client.audio.vad.VadConstants.INITIAL_NUM_LAGS_24K;
import static com.ts3client.audio.vad.VadConstants.MAX_PITCH_24K;
import static com.ts3client.audio.vad.VadConstants.MAX_PITCH_48K;
import static com.ts3client.audio.vad.VadConstants.MIN_PITCH_24K;
import static com.ts3client.audio.vad.VadConstants.MIN_PITCH_48K;
import static com.ts3client.audio.vad.VadConstants.NUM_LAGS_12K;
import static com.ts3client.audio.vad.VadConstants.REFINE_NUM_LAGS_24K;
/**
* Pitch period estimation for {@code rnn_vad}, ported from WebRTC's
* {@code rnn_vad/pitch_search.cc} and {@code pitch_search_internal.cc}.
*
* <p>Three stages: a coarse search over the 12&nbsp;kHz decimated LP residual picks two
* candidates, those are refined against the 24&nbsp;kHz buffer, and finally sub-harmonics
* of the winner are tested so that an octave error can be corrected. The result is a
* period expressed at 48&nbsp;kHz, which is what the network was trained on.
*
* <p>Lags are stored inverted: index 0 corresponds to the maximum pitch period.
*/
final class PitchEstimator {
/**
* For each divisor k in [2, 15], the multiplier n such that n*T/k is a sub-harmonic
* worth checking alongside T/k, chosen so no multiple is visited twice.
*/
private static final int[] SUB_HARMONIC_MULTIPLIERS =
{3, 2, 3, 2, 5, 2, 3, 2, 3, 2, 5, 2, 3, 2};
/** Thresholds on the initial period, computed as {@code [5*k*k for k in range(16)]}. */
private static final int[] INITIAL_PITCH_PERIOD_THRESHOLDS =
{20, 45, 80, 125, 180, 245, 320, 405, 500, 605, 720, 845, 980, 1125};
/** Number of pitch periods analysed either side of a candidate during refinement. */
private static final int PITCH_NEIGHBORHOOD_RADIUS = 2;
private final float[] pitchBuffer12k = new float[BUF_SIZE_12K];
private final float[] autoCorrelation12k = new float[NUM_LAGS_12K];
private final float[] yEnergy24k = new float[REFINE_NUM_LAGS_24K];
private final float[] autoCorrelation24k = new float[INITIAL_NUM_LAGS_24K];
private final int[] invertedLagsIndex = new int[2 * (2 * PITCH_NEIGHBORHOOD_RADIUS + 1)];
private int invertedLagsCount;
// Pitch tracking: the previous estimate biases the sub-harmonic decision.
private int lastPeriod48k;
private float lastStrength;
void reset() {
lastPeriod48k = 0;
lastStrength = 0;
}
/** Returns the pitch period at 48 kHz for the given 24 kHz LP residual buffer. */
int estimate(float[] pitchBuffer) {
// Coarse search at 12 kHz. WebRTC decimates without an anti-aliasing filter here;
// the refinement stage below is what recovers the accuracy.
for (int i = 0; i < BUF_SIZE_12K; i++) {
pitchBuffer12k[i] = pitchBuffer[2 * i];
}
computeAutoCorrelation12k();
long candidates = computePitchPeriod12kHz();
int best = (int) (candidates >> 32) * 2;
int secondBest = (int) candidates * 2;
computeSlidingFrameSquareEnergies24k(pitchBuffer);
int pitchLag48k = computePitchPeriod48kHz(pitchBuffer, best, secondBest);
computeExtendedPitchPeriod48kHz(pitchBuffer, MAX_PITCH_48K - pitchLag48k);
return lastPeriod48k;
}
/**
* {@code autoCorrelation12k[lag]} is the correlation of the most recent 20 ms with the
* frame starting {@code lag} samples into the buffer.
*/
private void computeAutoCorrelation12k() {
final int refOffset = BUF_SIZE_12K - FRAME_20MS_12K;
for (int lag = 0; lag < NUM_LAGS_12K; lag++) {
float sum = 0;
for (int i = 0; i < FRAME_20MS_12K; i++) {
sum += pitchBuffer12k[refOffset + i] * pitchBuffer12k[lag + i];
}
autoCorrelation12k[lag] = sum;
}
}
/** Returns the best and second-best inverted lags packed into a long. */
private long computePitchPeriod12kHz() {
float denominator = 1.0f;
for (int i = 0; i < FRAME_20MS_12K + 1; i++) {
denominator += pitchBuffer12k[i] * pitchBuffer12k[i];
}
int bestLag = 0;
float bestNumerator = -1.0f;
float bestDenominator = 0.0f;
int secondLag = 1;
float secondNumerator = -1.0f;
float secondDenominator = 0.0f;
for (int invertedLag = 0; invertedLag < NUM_LAGS_12K; invertedLag++) {
// A pitch candidate must have positive correlation.
if (autoCorrelation12k[invertedLag] > 0.0f) {
float numerator = autoCorrelation12k[invertedLag] * autoCorrelation12k[invertedLag];
// Compare numerator/denominator ratios without dividing.
if (numerator * secondDenominator > secondNumerator * denominator) {
if (numerator * bestDenominator > bestNumerator * denominator) {
secondLag = bestLag;
secondNumerator = bestNumerator;
secondDenominator = bestDenominator;
bestLag = invertedLag;
bestNumerator = numerator;
bestDenominator = denominator;
} else {
secondLag = invertedLag;
secondNumerator = numerator;
secondDenominator = denominator;
}
}
}
// Slide the energy window to the next inverted lag.
float yOld = pitchBuffer12k[invertedLag];
float yNew = pitchBuffer12k[invertedLag + FRAME_20MS_12K];
denominator -= yOld * yOld;
denominator += yNew * yNew;
denominator = Math.max(0.0f, denominator);
}
return ((long) bestLag << 32) | (secondLag & 0xffffffffL);
}
/** Energy of the sliding 20 ms frame for every inverted lag. */
private void computeSlidingFrameSquareEnergies24k(float[] pitchBuffer) {
float yy = 0;
for (int i = 0; i < FRAME_20MS_24K; i++) {
yy += pitchBuffer[i] * pitchBuffer[i];
}
yEnergy24k[0] = yy;
for (int invertedLag = 0; invertedLag < MAX_PITCH_24K; invertedLag++) {
yy -= pitchBuffer[invertedLag] * pitchBuffer[invertedLag];
yy += pitchBuffer[invertedLag + FRAME_20MS_24K] * pitchBuffer[invertedLag + FRAME_20MS_24K];
yy = Math.max(1.0f, yy);
yEnergy24k[invertedLag + 1] = yy;
}
}
/** Correlation of the most recent 20 ms with the frame at {@code invertedLag}. */
private static float autoCorrelation(int invertedLag, float[] pitchBuffer) {
float sum = 0;
for (int i = 0; i < FRAME_20MS_24K; i++) {
sum += pitchBuffer[MAX_PITCH_24K + i] * pitchBuffer[invertedLag + i];
}
return sum;
}
/**
* Recomputes the auto-correlation around both candidates at 24 kHz and returns the
* strongest inverted lag, pseudo-interpolated to 48 kHz.
*/
private int computePitchPeriod48kHz(float[] pitchBuffer, int bestCandidate, int secondCandidate) {
invertedLagsCount = 0;
// Order the two neighbourhoods so the first precedes the second.
boolean swap = bestCandidate > secondCandidate;
int firstCenter = swap ? secondCandidate : bestCandidate;
int secondCenter = swap ? bestCandidate : secondCandidate;
int r1min = Math.max(firstCenter - PITCH_NEIGHBORHOOD_RADIUS, 0);
int r1max = Math.min(firstCenter + PITCH_NEIGHBORHOOD_RADIUS, INITIAL_NUM_LAGS_24K - 1);
int r2min = Math.max(secondCenter - PITCH_NEIGHBORHOOD_RADIUS, 0);
int r2max = Math.min(secondCenter + PITCH_NEIGHBORHOOD_RADIUS, INITIAL_NUM_LAGS_24K - 1);
if (r1max + 1 >= r2min) { // Overlapping or adjacent ranges.
computeAutoCorrelationRange(pitchBuffer, r1min, r2max);
} else {
computeAutoCorrelationRange(pitchBuffer, r1min, r1max);
computeAutoCorrelationRange(pitchBuffer, r2min, r2max);
}
int bestInvertedLag = 0;
float bestNumerator = -1.0f;
float bestDenominator = 0.0f;
for (int i = 0; i < invertedLagsCount; i++) {
int invertedLag = invertedLagsIndex[i];
if (autoCorrelation24k[invertedLag] > 0.0f) {
float numerator = autoCorrelation24k[invertedLag] * autoCorrelation24k[invertedLag];
float denominator = yEnergy24k[invertedLag];
if (numerator * bestDenominator > bestNumerator * denominator) {
bestInvertedLag = invertedLag;
bestNumerator = numerator;
bestDenominator = denominator;
}
}
}
if (bestInvertedLag == 0 || bestInvertedLag >= INITIAL_NUM_LAGS_24K - 1) {
return bestInvertedLag * 2; // Cannot interpolate at the boundaries.
}
int offset = pseudoInterpolationOffset(
autoCorrelation24k[bestInvertedLag + 1],
autoCorrelation24k[bestInvertedLag],
autoCorrelation24k[bestInvertedLag - 1]);
return 2 * bestInvertedLag + offset;
}
private void computeAutoCorrelationRange(float[] pitchBuffer, int min, int max) {
// The pseudo-interpolation reads one lag either side, so zero those rather than
// clearing the whole array.
if (min > 0) {
autoCorrelation24k[min - 1] = 0.0f;
}
if (max < INITIAL_NUM_LAGS_24K - 1) {
autoCorrelation24k[max + 1] = 0.0f;
}
for (int invertedLag = min; invertedLag <= max; invertedLag++) {
autoCorrelation24k[invertedLag] = autoCorrelation(invertedLag, pitchBuffer);
invertedLagsIndex[invertedLagsCount++] = invertedLag;
}
}
/** Returns a lag correction in {-1, 0, +1} based on the neighbouring correlations. */
private static int pseudoInterpolationOffset(float prev, float curr, float next) {
if ((next - prev) > 0.7f * (curr - prev)) {
return 1;
} else if ((prev - next) > 0.7f * (curr - next)) {
return -1;
}
return 0;
}
/** Refines {@code lag} (24 kHz) to a 48 kHz lag using pseudo-interpolation. */
private static int pseudoInterpolationLagPitchBuf(int lag, float[] pitchBuffer) {
int offset = 0;
if (lag > 0 && lag < MAX_PITCH_24K) {
int invertedLag = MAX_PITCH_24K - lag;
offset = pseudoInterpolationOffset(
autoCorrelation(invertedLag + 1, pitchBuffer),
autoCorrelation(invertedLag, pitchBuffer),
autoCorrelation(invertedLag - 1, pitchBuffer));
}
return 2 * lag + offset;
}
/** Same as {@code round(multiplier * pitchPeriod / divisor)}. */
private static int alternativePitchPeriod(int pitchPeriod, int multiplier, int divisor) {
return (2 * multiplier * pitchPeriod + divisor) / (2 * divisor);
}
/**
* Tests sub-harmonics of the refined period and keeps the strongest, then stores the
* winner in {@link #lastPeriod48k}/{@link #lastStrength}.
*/
private void computeExtendedPitchPeriod48kHz(float[] pitchBuffer, int initialPeriod48k) {
final float xEnergy = yEnergy24k[MAX_PITCH_24K];
int bestPeriod = Math.min(initialPeriod48k / 2, MAX_PITCH_24K - 1);
float bestXy = autoCorrelation(MAX_PITCH_24K - bestPeriod, pitchBuffer);
float bestYEnergy = yEnergy24k[MAX_PITCH_24K - bestPeriod];
float bestStrength = pitchStrength(bestXy, bestYEnergy, xEnergy);
final int initialPeriod = bestPeriod;
final float initialStrength = bestStrength;
// The tracked pitch is kept at 48 kHz but compared at 24 kHz.
final int lastPeriod = lastPeriod48k / 2;
// Largest divisor for which the alternative period is still above the minimum.
int maxPeriodDivisor = (2 * initialPeriod) / (2 * MIN_PITCH_24K - 1);
for (int periodDivisor = 2; periodDivisor <= maxPeriodDivisor; periodDivisor++) {
int alternativePeriod = alternativePitchPeriod(initialPeriod, 1, periodDivisor);
int dualPeriod = alternativePitchPeriod(
initialPeriod, SUB_HARMONIC_MULTIPLIERS[periodDivisor - 2], periodDivisor);
// Special case: for a divisor of 2 the sub-harmonic can exceed the pitch range.
if (periodDivisor == 2 && dualPeriod > MAX_PITCH_24K) {
dualPeriod = initialPeriod;
}
// Score the candidate together with its sub-harmonic.
float xyPrimary = autoCorrelation(MAX_PITCH_24K - alternativePeriod, pitchBuffer);
float xySecondary = autoCorrelation(MAX_PITCH_24K - dualPeriod, pitchBuffer);
float xy = 0.5f * (xyPrimary + xySecondary);
float yy = 0.5f * (yEnergy24k[MAX_PITCH_24K - alternativePeriod]
+ yEnergy24k[MAX_PITCH_24K - dualPeriod]);
float alternativeStrength = pitchStrength(xy, yy, xEnergy);
if (isAlternativeStronger(lastPeriod, initialPeriod, initialStrength,
alternativePeriod, alternativeStrength, periodDivisor)) {
bestPeriod = alternativePeriod;
bestStrength = alternativeStrength;
bestXy = xy;
bestYEnergy = yy;
}
}
bestXy = Math.max(0.0f, bestXy);
float finalStrength = (bestYEnergy <= bestXy) ? 1.0f : bestXy / (bestYEnergy + 1.0f);
finalStrength = Math.min(bestStrength, finalStrength);
lastPeriod48k = Math.max(MIN_PITCH_48K, pseudoInterpolationLagPitchBuf(bestPeriod, pitchBuffer));
lastStrength = finalStrength;
}
private static float pitchStrength(float xy, float yEnergy, float xEnergy) {
return xy / (float) Math.sqrt(1.0f + xEnergy * yEnergy);
}
private boolean isAlternativeStronger(int lastPeriod, int initialPeriod, float initialStrength,
int alternativePeriod, float alternativeStrength,
int periodDivisor) {
// Pitch tracking: a candidate close to the previous estimate clears a lower bar.
float lowerThresholdTerm = 0.0f;
int distance = Math.abs(alternativePeriod - lastPeriod);
if (distance <= 1) {
lowerThresholdTerm = lastStrength;
} else if (distance == 2
&& initialPeriod > INITIAL_PITCH_PERIOD_THRESHOLDS[periodDivisor - 2]) {
lowerThresholdTerm = 0.5f * lastStrength;
}
// Higher frequencies get a stricter threshold: short-term correlations bias
// the search towards them and cause false positives.
float threshold = Math.max(0.3f, 0.7f * initialStrength - lowerThresholdTerm);
if (alternativePeriod < 3 * MIN_PITCH_24K) {
threshold = Math.max(0.4f, 0.85f * initialStrength - lowerThresholdTerm);
} else if (alternativePeriod < 2 * MIN_PITCH_24K) {
threshold = Math.max(0.5f, 0.9f * initialStrength - lowerThresholdTerm);
}
return alternativeStrength > threshold;
}
}

View File

@@ -0,0 +1,224 @@
package com.ts3client.audio.vad;
/**
* Real-input FFT for the {@code rnn_vad} spectral analysis.
*
* <p>The transform length is 480 (a 20&nbsp;ms frame at 24&nbsp;kHz), which is not a
* power of two, so the shared radix-2 {@code Fft} cannot be used. This is a
* mixed-radix Cooley&ndash;Tukey complex FFT (radices 2, 3, 4 and 5, with a generic
* DFT for any other factor) wrapped in the standard real-input trick: an
* {@code n}-point real transform is computed as an {@code n/2}-point complex one
* followed by a split step.
*
* <p>Output uses the packed layout WebRTC's PFFFT produces, which the band
* correlator indexes directly:
* <pre>
* out[0] = Re(X[0]) (DC)
* out[1] = Re(X[n/2]) (Nyquist)
* out[2k] = Re(X[k])
* out[2k+1] = Im(X[k]) for k in [1, n/2)
* </pre>
*/
final class RealFft {
private final int size;
private final int half;
// Twiddles for the complex sub-transform and for the real split step.
private final float[] halfRe;
private final float[] halfIm;
private final float[] splitCos;
private final float[] splitSin;
private final ComplexFft complexFft;
RealFft(int size) {
if (size < 2 || (size & 1) != 0) {
throw new IllegalArgumentException("Real FFT length must be even: " + size);
}
this.size = size;
this.half = size / 2;
this.halfRe = new float[half];
this.halfIm = new float[half];
this.complexFft = new ComplexFft(half);
this.splitCos = new float[half / 2 + 1];
this.splitSin = new float[half / 2 + 1];
for (int k = 0; k <= half / 2; k++) {
double angle = -2.0 * Math.PI * k / size;
splitCos[k] = (float) Math.cos(angle);
splitSin[k] = (float) Math.sin(angle);
}
}
int size() {
return size;
}
/**
* Transforms {@code in} ({@link #size} real samples) into {@code out}
* ({@link #size} floats in the packed layout described above).
*/
void forward(float[] in, float[] out) {
// Pack the real signal as half/2 complex samples: z[i] = x[2i] + j*x[2i+1].
for (int i = 0; i < half; i++) {
halfRe[i] = in[2 * i];
halfIm[i] = in[2 * i + 1];
}
complexFft.forward(halfRe, halfIm);
// Split Z into the transforms of the even and odd sample sequences, then
// recombine them into the spectrum of the original real signal.
out[0] = halfRe[0] + halfIm[0];
out[1] = halfRe[0] - halfIm[0];
for (int k = 1; k <= half / 2; k++) {
int mirror = half - k;
// Even part: hermitian-symmetric average; odd part: the difference.
float evenRe = 0.5f * (halfRe[k] + halfRe[mirror]);
float evenIm = 0.5f * (halfIm[k] - halfIm[mirror]);
float oddRe = 0.5f * (halfIm[k] + halfIm[mirror]);
float oddIm = -0.5f * (halfRe[k] - halfRe[mirror]);
float c = splitCos[k];
float s = splitSin[k];
float twRe = oddRe * c - oddIm * s;
float twIm = oddRe * s + oddIm * c;
float re = evenRe + twRe;
float im = evenIm + twIm;
out[2 * k] = re;
out[2 * k + 1] = im;
if (mirror != k) {
// X[n/2 - k] is the conjugate-mirrored companion of X[k].
out[2 * mirror] = evenRe - twRe;
out[2 * mirror + 1] = -(evenIm - twIm);
}
}
}
/**
* Mixed-radix complex FFT, decimation in time.
*
* <p>A transform of length {@code n = r * m} is computed by transforming the {@code r}
* interleaved subsequences of length {@code m} and combining them:
* {@code X[k1*m + k2] = sum_j (W_n^(j*k2) * X_j[k2]) * W_r^(j*k1)}.
*
* <p>The recursion is expressed directly rather than as an iterative digit-reversed
* loop, which keeps it correct for any factorisation. Scratch buffers are allocated
* once per recursion level so no transform allocates.
*/
static final class ComplexFft {
private final int n;
private final float[] cosTable;
private final float[] sinTable;
private final float[][] scratchRe;
private final float[][] scratchIm;
private final float[] dftRe;
private final float[] dftIm;
ComplexFft(int n) {
this.n = n;
this.cosTable = new float[n];
this.sinTable = new float[n];
for (int i = 0; i < n; i++) {
double angle = -2.0 * Math.PI * i / n;
cosTable[i] = (float) Math.cos(angle);
sinTable[i] = (float) Math.sin(angle);
}
int depth = 0;
int maxRadix = 1;
for (int rest = n; rest > 1; depth++) {
int r = smallestFactor(rest);
maxRadix = Math.max(maxRadix, r);
rest /= r;
}
this.scratchRe = new float[depth + 1][n];
this.scratchIm = new float[depth + 1][n];
this.dftRe = new float[maxRadix];
this.dftIm = new float[maxRadix];
}
/** Transforms {@code re}/{@code im} of length {@code n} in place. */
void forward(float[] re, float[] im) {
transform(re, im, 0, 1, n, scratchRe[0], scratchIm[0], 0, 1);
System.arraycopy(scratchRe[0], 0, re, 0, n);
System.arraycopy(scratchIm[0], 0, im, 0, n);
}
/**
* Transforms the {@code len} samples starting at {@code inOff} with the given
* {@code stride}, writing the spectrum to {@code outRe}/{@code outIm} at
* {@code outOff}.
*/
private void transform(float[] inRe, float[] inIm, int inOff, int stride, int len,
float[] outRe, float[] outIm, int outOff, int depth) {
if (len == 1) {
outRe[outOff] = inRe[inOff];
outIm[outOff] = inIm[inOff];
return;
}
int r = smallestFactor(len);
int m = len / r;
// Sub-transforms of the r interleaved subsequences, laid out end to end.
float[] subRe = scratchRe[depth];
float[] subIm = scratchIm[depth];
for (int j = 0; j < r; j++) {
transform(inRe, inIm, inOff + j * stride, stride * r, m,
subRe, subIm, outOff + j * m, depth + 1);
}
// Twiddle step for this transform size, expressed in top-level table units.
int unit = n / len;
for (int k2 = 0; k2 < m; k2++) {
for (int j = 0; j < r; j++) {
int idx = outOff + j * m + k2;
int t = (j * k2 * unit) % n;
float c = cosTable[t];
float s = sinTable[t];
dftRe[j] = subRe[idx] * c - subIm[idx] * s;
dftIm[j] = subRe[idx] * s + subIm[idx] * c;
}
combine(outRe, outIm, outOff + k2, m, r);
}
}
/** Length-{@code r} DFT of {@code dftRe}/{@code dftIm}, written with stride {@code m}. */
private void combine(float[] outRe, float[] outIm, int outOff, int m, int r) {
switch (r) {
case 2 -> {
outRe[outOff] = dftRe[0] + dftRe[1];
outIm[outOff] = dftIm[0] + dftIm[1];
outRe[outOff + m] = dftRe[0] - dftRe[1];
outIm[outOff + m] = dftIm[0] - dftIm[1];
}
default -> {
int unit = n / r;
for (int k1 = 0; k1 < r; k1++) {
float sumRe = 0;
float sumIm = 0;
for (int j = 0; j < r; j++) {
int t = (j * k1 % r) * unit;
float c = cosTable[t];
float s = sinTable[t];
sumRe += dftRe[j] * c - dftIm[j] * s;
sumIm += dftRe[j] * s + dftIm[j] * c;
}
outRe[outOff + k1 * m] = sumRe;
outIm[outOff + k1 * m] = sumIm;
}
}
}
}
private static int smallestFactor(int v) {
for (int f = 2; f * f <= v; f++) {
if (v % f == 0) return f;
}
return v;
}
}
}

View File

@@ -0,0 +1,203 @@
package com.ts3client.audio.vad;
/**
* The {@code rnn_vad} network: a 42&rarr;24 dense layer, a 24-unit GRU and a 24&rarr;1
* dense output. Ported from WebRTC's {@code rnn_vad/rnn.cc}, {@code rnn_fc.cc} and
* {@code rnn_gru.cc}, using the activation approximations from RNNoise.
*
* <p>The GRU carries state between frames, so the caller must feed frames in order and
* {@link #reset()} on a discontinuity. Silence resets the state, matching WebRTC.
*/
final class RnnNetwork {
private static final int INPUT_SIZE = 42;
private static final int FC1_SIZE = 24;
private static final int GRU_SIZE = 24;
private static final int NUM_GATES = 3; // update, reset, output
private final float[] fc1Weights; // [output][input]
private final float[] fc1Bias;
private final float[] fc2Weights;
private final float[] fc2Bias;
private final float[] gruWeights; // [gate][output][input]
private final float[] gruRecurrentWeights; // [gate][output][output]
private final float[] gruBias; // [gate][output]
private final float[] fc1Output = new float[FC1_SIZE];
private final float[] state = new float[GRU_SIZE];
private final float[] update = new float[GRU_SIZE];
private final float[] resetGate = new float[GRU_SIZE];
private final float[] resetTimesState = new float[GRU_SIZE];
RnnNetwork() {
fc1Weights = transposeAndScale(RnnVadWeights.INPUT_DENSE_WEIGHTS, INPUT_SIZE, FC1_SIZE);
fc1Bias = scale(RnnVadWeights.INPUT_DENSE_BIAS);
// The output layer is 24x1, so its stored layout already matches.
fc2Weights = scale(RnnVadWeights.OUTPUT_DENSE_WEIGHTS);
fc2Bias = scale(RnnVadWeights.OUTPUT_DENSE_BIAS);
gruWeights = transposeGruTensor(RnnVadWeights.HIDDEN_GRU_WEIGHTS, FC1_SIZE, GRU_SIZE);
gruRecurrentWeights =
transposeGruTensor(RnnVadWeights.HIDDEN_GRU_RECURRENT_WEIGHTS, GRU_SIZE, GRU_SIZE);
gruBias = transposeGruTensor(RnnVadWeights.HIDDEN_GRU_BIAS, 1, GRU_SIZE);
}
void reset() {
java.util.Arrays.fill(state, 0.0f);
}
/**
* Returns the speech probability for a feature vector. When {@code silence} is set the
* state is cleared and zero is returned without running the network.
*/
float computeVadProbability(float[] featureVector, boolean silence) {
if (silence) {
reset();
return 0.0f;
}
for (int o = 0; o < FC1_SIZE; o++) {
float x = fc1Bias[o] + dot(featureVector, fc1Weights, o * INPUT_SIZE, INPUT_SIZE);
fc1Output[o] = tansigApproximated(x);
}
computeGruOutput(fc1Output);
float x = fc2Bias[0] + dot(state, fc2Weights, 0, GRU_SIZE);
return sigmoidApproximated(x);
}
private void computeGruOutput(float[] input) {
final int strideWeights = FC1_SIZE * GRU_SIZE;
final int strideRecurrent = GRU_SIZE * GRU_SIZE;
// u = sigmoid(W_u.i + R_u.s + b_u)
computeGate(input, 0, 0, 0, update);
// r = sigmoid(W_r.i + R_r.s + b_r)
computeGate(input, GRU_SIZE, strideWeights, strideRecurrent, resetGate);
// s' = u.s + (1 - u).ReLU(W_s.i + R_s.(s.r) + b_s)
for (int o = 0; o < GRU_SIZE; o++) {
resetTimesState[o] = state[o] * resetGate[o];
}
for (int o = 0; o < GRU_SIZE; o++) {
float x = gruBias[2 * GRU_SIZE + o];
x += dot(input, gruWeights, 2 * strideWeights + o * FC1_SIZE, FC1_SIZE);
x += dot(resetTimesState, gruRecurrentWeights,
2 * strideRecurrent + o * GRU_SIZE, GRU_SIZE);
state[o] = update[o] * state[o] + (1.0f - update[o]) * Math.max(0.0f, x);
}
}
private void computeGate(float[] input, int biasOffset, int weightsOffset,
int recurrentOffset, float[] gate) {
for (int o = 0; o < GRU_SIZE; o++) {
float x = gruBias[biasOffset + o];
x += dot(input, gruWeights, weightsOffset + o * FC1_SIZE, FC1_SIZE);
x += dot(state, gruRecurrentWeights, recurrentOffset + o * GRU_SIZE, GRU_SIZE);
gate[o] = sigmoidApproximated(x);
}
}
private static float dot(float[] a, float[] weights, int weightsOffset, int len) {
float sum = 0;
for (int i = 0; i < len; i++) {
sum += a[i] * weights[weightsOffset + i];
}
return sum;
}
// ---- weight layout ----
private static float[] scale(byte[] src) {
float[] out = new float[src.length];
for (int i = 0; i < src.length; i++) {
out[i] = RnnVadWeights.SCALE * src[i];
}
return out;
}
/** Stored as {@code [input][output]}; the dot products want {@code [output][input]}. */
private static float[] transposeAndScale(byte[] src, int inputSize, int outputSize) {
float[] out = new float[src.length];
for (int o = 0; o < outputSize; o++) {
for (int i = 0; i < inputSize; i++) {
out[o * inputSize + i] = RnnVadWeights.SCALE * src[i * outputSize + o];
}
}
return out;
}
/**
* GRU tensors are stored as {@code [n][gate][output]} and are wanted as
* {@code [gate][output][n]}, where {@code n} is the input size (1 for the bias).
*/
private static float[] transposeGruTensor(byte[] src, int n, int outputSize) {
float[] out = new float[src.length];
int strideSrc = NUM_GATES * outputSize;
int strideDst = n * outputSize;
for (int g = 0; g < NUM_GATES; g++) {
for (int o = 0; o < outputSize; o++) {
for (int i = 0; i < n; i++) {
out[g * strideDst + o * n + i] =
RnnVadWeights.SCALE * src[i * strideSrc + g * outputSize + o];
}
}
}
return out;
}
// ---- activations (RNNoise approximations, bit-compatible with WebRTC) ----
/** tanh sampled every 0.04, copied verbatim from RNNoise's {@code rnn_activations.h}. */
private static final float[] TANSIG_TABLE = {
0.000000f, 0.039979f, 0.079830f, 0.119427f, 0.158649f, 0.197375f, 0.235496f, 0.272905f,
0.309507f, 0.345214f, 0.379949f, 0.413644f, 0.446244f, 0.477700f, 0.507977f, 0.537050f,
0.564900f, 0.591519f, 0.616909f, 0.641077f, 0.664037f, 0.685809f, 0.706419f, 0.725897f,
0.744277f, 0.761594f, 0.777888f, 0.793199f, 0.807569f, 0.821040f, 0.833655f, 0.845456f,
0.856485f, 0.866784f, 0.876393f, 0.885352f, 0.893698f, 0.901468f, 0.908698f, 0.915420f,
0.921669f, 0.927473f, 0.932862f, 0.937863f, 0.942503f, 0.946806f, 0.950795f, 0.954492f,
0.957917f, 0.961090f, 0.964028f, 0.966747f, 0.969265f, 0.971594f, 0.973749f, 0.975743f,
0.977587f, 0.979293f, 0.980869f, 0.982327f, 0.983675f, 0.984921f, 0.986072f, 0.987136f,
0.988119f, 0.989027f, 0.989867f, 0.990642f, 0.991359f, 0.992020f, 0.992631f, 0.993196f,
0.993718f, 0.994199f, 0.994644f, 0.995055f, 0.995434f, 0.995784f, 0.996108f, 0.996407f,
0.996682f, 0.996937f, 0.997172f, 0.997389f, 0.997590f, 0.997775f, 0.997946f, 0.998104f,
0.998249f, 0.998384f, 0.998508f, 0.998623f, 0.998728f, 0.998826f, 0.998916f, 0.999000f,
0.999076f, 0.999147f, 0.999213f, 0.999273f, 0.999329f, 0.999381f, 0.999428f, 0.999472f,
0.999513f, 0.999550f, 0.999585f, 0.999617f, 0.999646f, 0.999673f, 0.999699f, 0.999722f,
0.999743f, 0.999763f, 0.999781f, 0.999798f, 0.999813f, 0.999828f, 0.999841f, 0.999853f,
0.999865f, 0.999875f, 0.999885f, 0.999893f, 0.999902f, 0.999909f, 0.999916f, 0.999923f,
0.999929f, 0.999934f, 0.999939f, 0.999944f, 0.999948f, 0.999952f, 0.999956f, 0.999959f,
0.999962f, 0.999965f, 0.999968f, 0.999970f, 0.999973f, 0.999975f, 0.999977f, 0.999978f,
0.999980f, 0.999982f, 0.999983f, 0.999984f, 0.999986f, 0.999987f, 0.999988f, 0.999989f,
0.999990f, 0.999990f, 0.999991f, 0.999992f, 0.999992f, 0.999993f, 0.999994f, 0.999994f,
0.999994f, 0.999995f, 0.999995f, 0.999996f, 0.999996f, 0.999996f, 0.999997f, 0.999997f,
0.999997f, 0.999997f, 0.999997f, 0.999998f, 0.999998f, 0.999998f, 0.999998f, 0.999998f,
0.999998f, 0.999999f, 0.999999f, 0.999999f, 0.999999f, 0.999999f, 0.999999f, 0.999999f,
0.999999f, 0.999999f, 0.999999f, 0.999999f, 0.999999f, 0.999999f, 1.000000f, 1.000000f,
1.000000f, 1.000000f, 1.000000f, 1.000000f, 1.000000f, 1.000000f, 1.000000f, 1.000000f,
1.000000f
};
private static float tansigApproximated(float x) {
// Tests are reversed to catch NaNs.
if (!(x < 8.0f)) {
return 1.0f;
}
if (!(x > -8.0f)) {
return -1.0f;
}
float sign = 1.0f;
if (x < 0.0f) {
x = -x;
sign = -1.0f;
}
int i = (int) Math.floor(0.5f + 25 * x);
float y = TANSIG_TABLE[i];
// Newton step back to x's own scale (undoing the table's factor of 25).
x -= 0.04f * i;
y = y + x * (1.0f - y * y) * (1.0f - y * x);
return sign * y;
}
private static float sigmoidApproximated(float x) {
return 0.5f + 0.5f * tansigApproximated(0.5f * x);
}
}

View File

@@ -0,0 +1,140 @@
package com.ts3client.audio.vad;
import com.ts3client.audio.SpeechProbabilityDetector;
/**
* {@link SpeechProbabilityDetector} backed by {@link RnnVad}, adapting an arbitrary
* capture rate and frame size to the detector's fixed 10&nbsp;ms frames at 24&nbsp;kHz.
*
* <p>Capture frames rarely divide into 10&nbsp;ms chunks exactly, so leftover samples are
* carried into the next call. The reported probability is that of the most recent complete
* 10&nbsp;ms frame; a capture frame long enough to span several of them advances the
* detector several times, exactly as feeding it directly would.
*/
public final class RnnSpeechDetector implements SpeechProbabilityDetector {
/** Half-band FIR used when decimating; long enough to keep aliasing out of the passband. */
private static final int FILTER_TAPS = 32;
private final RnnVad vad = new RnnVad();
private final int sampleRate;
private final boolean decimateByTwo;
private final float[] filterCoeffs;
private final float[] filterState;
private int filterPos;
/** Resampled samples not yet consumed as a whole 10 ms frame. */
private final float[] pending = new float[RnnVad.FRAME_SIZE];
private int pendingCount;
/** Fractional read position, for capture rates that are not a multiple of 24 kHz. */
private double resamplePos;
public RnnSpeechDetector(int sampleRate) {
this.sampleRate = sampleRate;
this.decimateByTwo = sampleRate == 2 * RnnVad.SAMPLE_RATE;
if (decimateByTwo) {
this.filterCoeffs = lowPass(FILTER_TAPS, 0.25);
this.filterState = new float[FILTER_TAPS];
} else {
this.filterCoeffs = null;
this.filterState = null;
}
}
@Override
public double process(float[] frame, int length) {
if (sampleRate == RnnVad.SAMPLE_RATE) {
feed(frame, 0, length, 1);
} else if (decimateByTwo) {
decimateAndFeed(frame, length);
} else {
resampleAndFeed(frame, length);
}
return vad.probability();
}
@Override
public double probability() {
return vad.probability();
}
@Override
public void reset() {
vad.reset();
pendingCount = 0;
resamplePos = 0;
filterPos = 0;
if (filterState != null) {
java.util.Arrays.fill(filterState, 0.0f);
}
}
/** Appends samples (taking every {@code step}-th) and runs the detector per full frame. */
private void feed(float[] src, int offset, int length, int step) {
for (int i = offset; i < offset + length; i += step) {
push(src[i]);
}
}
/** 48 kHz (the common case): low-pass, then keep every other sample. */
private void decimateAndFeed(float[] frame, int length) {
for (int i = 0; i < length; i++) {
filterState[filterPos] = frame[i];
filterPos = (filterPos + 1) % FILTER_TAPS;
if ((i & 1) == 1) {
float acc = 0;
int idx = filterPos;
for (int t = 0; t < FILTER_TAPS; t++) {
acc += filterCoeffs[t] * filterState[idx];
idx = (idx + 1) % FILTER_TAPS;
}
push(acc);
}
}
}
/**
* Any other capture rate. Linear interpolation is enough here: the detector's own
* analysis is far coarser than the resampling error.
*/
private void resampleAndFeed(float[] frame, int length) {
double ratio = (double) sampleRate / RnnVad.SAMPLE_RATE;
while (resamplePos < length) {
int i = (int) resamplePos;
double frac = resamplePos - i;
float a = frame[i];
float b = frame[Math.min(i + 1, length - 1)];
push((float) ((1 - frac) * a + frac * b));
resamplePos += ratio;
}
resamplePos -= length;
}
private void push(float sample) {
pending[pendingCount++] = sample;
if (pendingCount == RnnVad.FRAME_SIZE) {
vad.process(pending, 0);
pendingCount = 0;
}
}
/** Windowed-sinc low-pass with the given normalised cutoff, scaled to unit DC gain. */
private static float[] lowPass(int taps, double cutoff) {
float[] h = new float[taps];
double centre = (taps - 1) / 2.0;
double sum = 0;
for (int i = 0; i < taps; i++) {
double t = i - centre;
double sinc = (t == 0) ? 2 * cutoff : Math.sin(2 * Math.PI * cutoff * t) / (Math.PI * t);
double window = 0.54 - 0.46 * Math.cos(2 * Math.PI * i / (taps - 1)); // Hamming
h[i] = (float) (sinc * window);
sum += h[i];
}
for (int i = 0; i < taps; i++) {
h[i] /= (float) sum;
}
return h;
}
}

View File

@@ -0,0 +1,103 @@
package com.ts3client.audio.vad;
import static com.ts3client.audio.vad.VadConstants.BUF_SIZE_24K;
import static com.ts3client.audio.vad.VadConstants.FEATURE_VECTOR_SIZE;
import static com.ts3client.audio.vad.VadConstants.FRAME_10MS_24K;
import static com.ts3client.audio.vad.VadConstants.FRAME_20MS_24K;
import static com.ts3client.audio.vad.VadConstants.MAX_PITCH_24K;
import static com.ts3client.audio.vad.VadConstants.NUM_LPC_COEFFICIENTS;
/**
* WebRTC's {@code rnn_vad} speech-probability detector, ported to Java.
*
* <p>This is the detector the TeamSpeak 3 client uses for its "Automatic" and "Hybrid"
* voice-activation modes: TS3 links the same network with byte-identical weights, so at a
* given threshold this gates on the same frames theirs does.
*
* <p>Feed exactly {@link VadConstants#FRAME_10MS_24K} samples of 24&nbsp;kHz mono audio in
* [-1, 1] per call, in order. Combines {@code features_extraction.cc} and {@code rnn.cc}.
*/
public final class RnnVad {
/** Samples per call: 10 ms at 24 kHz. */
public static final int FRAME_SIZE = FRAME_10MS_24K;
/** Sample rate this detector expects. */
public static final int SAMPLE_RATE = VadConstants.SAMPLE_RATE_24K;
/**
* WebRTC's audio processing carries float samples on the 16-bit integer scale, and the
* network's thresholds (the silence cut-off, the pitch energy floor) are calibrated for
* it. Callers pass [-1, 1], so scale on the way in.
*/
private static final float INT16_SCALE = 32768.0f;
/**
* Rolling window of 24 kHz samples: one maximum pitch period of history followed by
* the most recent 20 ms.
*/
private final float[] pitchBuffer = new float[BUF_SIZE_24K];
private final float[] lpResidual = new float[BUF_SIZE_24K];
private final float[] lpcCoeffs = new float[NUM_LPC_COEFFICIENTS];
private final float[] featureVector = new float[FEATURE_VECTOR_SIZE];
private final LinearPrediction linearPrediction = new LinearPrediction();
private final PitchEstimator pitchEstimator = new PitchEstimator();
private final SpectralFeatures spectralFeatures = new SpectralFeatures();
private final RnnNetwork network = new RnnNetwork();
private float probability;
private int pitchPeriod48k;
/** Clears all history. Call on a capture discontinuity. */
public void reset() {
java.util.Arrays.fill(pitchBuffer, 0.0f);
pitchEstimator.reset();
pitchPeriod48k = 0;
spectralFeatures.reset();
network.reset();
probability = 0.0f;
}
/** Speech probability in [0, 1] from the most recent {@link #process} call. */
public float probability() {
return probability;
}
/**
* Pitch period of the most recent frame, expressed at 48&nbsp;kHz as the network's
* feature is. Exposed for diagnostics and tests.
*/
public int pitchPeriod48kHz() {
return pitchPeriod48k;
}
/**
* Analyses one 10 ms frame and returns the speech probability in [0, 1].
*
* @param frame mono samples in [-1, 1]
* @param offset index of the first sample
*/
public float process(float[] frame, int offset) {
System.arraycopy(pitchBuffer, FRAME_10MS_24K, pitchBuffer, 0, BUF_SIZE_24K - FRAME_10MS_24K);
int start = BUF_SIZE_24K - FRAME_10MS_24K;
for (int i = 0; i < FRAME_10MS_24K; i++) {
pitchBuffer[start + i] = frame[offset + i] * INT16_SCALE;
}
// Search the pitch on the whitened signal so formants don't dominate the peak.
linearPrediction.computeLpcCoefficients(pitchBuffer, lpcCoeffs);
LinearPrediction.computeLpResidual(lpcCoeffs, pitchBuffer, lpResidual);
pitchPeriod48k = pitchEstimator.estimate(lpResidual);
// Normalisation based on the training-set statistics.
featureVector[FEATURE_VECTOR_SIZE - 2] = 0.01f * (pitchPeriod48k - 300);
// Compare the most recent 20 ms against the 20 ms one pitch period earlier.
int laggedOffset = MAX_PITCH_24K - pitchPeriod48k / 2;
boolean silence = spectralFeatures.checkSilenceComputeFeatures(
pitchBuffer, BUF_SIZE_24K - FRAME_20MS_24K, pitchBuffer, laggedOffset, featureVector);
probability = network.computeVadProbability(featureVector, silence);
return probability;
}
}

View File

@@ -0,0 +1,273 @@
package com.ts3client.audio.vad;
/**
* Quantised weights of WebRTC's {@code rnn_vad} network, copied verbatim from
* {@code third_party/rnnoise/src/rnn_vad_weights.cc} (BSD, Xiph/Jean-Marc Valin).
*
* <p>These are the same 4585 bytes the TeamSpeak 3 client links in, so our detector
* produces the same speech probabilities theirs does.
*
* <p>Values are stored quantised; {@link #SCALE} converts them to float.
*/
final class RnnVadWeights {
/** Dequantisation factor applied to every table below. */
static final float SCALE = 1.0f / 256.0f;
/** FC1 weights, laid out as [input][output] (42 x 24). */
static final byte[] INPUT_DENSE_WEIGHTS = {
-10, 0, -3, 1, -8, -6, 3, -13, 1, 0, -3, -7, -5, -3, 6, -1, -6, 0, -6, -4, -1, -2, 1, 1, -7, 2,
21, 10, -5, -20, 24, 23, 37, 8, -2, 33, -6, 22, 13, -2, 50, 8, 13, 1, -15, 30, -10, 30, 0, 3, 5,
27, 1, 4, -3, 41, 56, 35, -2, 49, -13, 11, 13, -2, -47, 5, -16, -60, -15, 77, -17, 26, -3, 14,
-21, 19, -5, -19, -13, 0, 10, 14, 9, 31, -13, -41, -10, 4, 22, 18, -48, -6, -10, 62, -3, -18,
-14, 12, 26, -28, 3, 14, 25, -13, -19, 6, 5, 36, -3, -65, -12, 0, 31, -7, -9, 101, -4, 26, 16,
17, -12, -12, 14, -36, -3, 5, -15, 21, 2, 30, -3, 38, -4, 1, -6, 7, -7, 14, 38, -22, -30, -3,
-7, 3, -39, -70, -126, 25, 34, 94, -67, -22, -33, 83, -47, -118, 4, 70, 33, 25, 62, -128, -76,
-118, -113, 49, -12, -100, -18, -114, -33, 43, 32, 61, 40, -9, -106, 2, 36, -100, -40, -5, 20,
-75, 61, -51, -9, 126, -27, -52, 5, -24, -21, -126, -114, -12, 15, 106, -2, 73, -125, 50, 13,
-120, 35, 35, 4, -61, 29, -124, 6, -53, -69, -125, 64, -89, 36, -107, -103, -7, 27, 121, 69, 77,
-35, 35, 95, -125, -49, 97, -45, -43, -23, 23, -28, -65, -118, 2, 8, -126, 27, -97, 92, 5, 55,
82, 17, -57, -115, 37, 8, -106, -46, 41, -2, 21, -44, 8, -73, -58, -39, 34, 89, -95, 95, -117,
120, -58, 31, 123, 1, -32, -109, -110, 60, -120, -43, -74, 5, 91, 26, 21, 114, 82, -83, -126,
123, 22, -16, -67, 25, -83, 46, 48, -34, -121, -124, -63, -35, -9, 31, 82, 123, 6, -3, 117, 93,
-2, -13, -36, 124, -112, -6, -102, -5, -33, -15, 44, -69, -127, -23, -40, -34, -85, 68, 83, -1,
40, 8, 84, 118, -58, -55, -102, 123, -55, -14, -123, 44, -63, -14, 21, 35, 16, 24, -126, -13,
-114, 35, 20, -36, 61, -9, 97, 34, 19, -32, -109, 76, -104, 99, -119, 45, -125, -51, -28, -8,
-69, -8, 125, -45, -93, 113, 103, -41, -82, 52, 7, 126, 0, -40, 104, 55, -58, 17, -124, -93,
-58, 8, -45, 1, 56, -123, 108, -47, -23, 115, 127, 17, -68, -13, 116, -82, -44, 45, 67, -120,
-101, -15, -125, 120, -113, 17, -48, -73, 126, -64, -86, -118, -19, 112, -1, -66, -27, -62, 121,
-86, -58, 50, 89, -38, -75, 95, -111, 12, -113, 2, -68, 2, -94, -121, 91, -5, 0, 79, 43, -7,
-18, 79, 35, -38, 47, 1, -45, 83, -50, 102, 32, 55, -96, 15, -122, -69, 45, -27, 91, -62, -30,
46, -95, 22, -72, -97, -1, 14, -122, 28, 127, 61, -126, 121, 9, 68, -120, 49, -60, 90, 3, 43,
68, 54, 34, -10, 28, 21, -24, -54, 22, -113, -12, 82, -2, -17, -9, 127, 8, 116, -92, 0, -70,
-33, 123, 66, 116, -74, -4, 74, -72, -22, -47, 1, -83, -60, -124, 1, 122, -57, -43, 49, 40,
-126, -128, -8, -29, 28, -24, -123, -121, -70, -93, -37, -126, 11, -125, -37, 11, -31, -51,
-124, 116, -128, 8, -25, 109, 75, -12, 7, 8, 10, 117, 124, -128, -128, 29, -26, 101, 21, -128,
87, 8, -39, 23, -128, 127, -127, 74, -55, 74, 112, 127, 4, 55, 44, -92, 123, 34, -93, 47, -21,
-92, 17, 49, -121, 92, 7, -126, -125, 124, -74, 3, -59, 18, -91, 3, -9, 9, 56, 116, 7, -29, 33,
87, -21, -128, -13, 57, 74, 9, -29, -61, -97, -21, -95, -12, -114, 16, 82, 125, -7, 10, -24, 9,
77, -128, -102, -25, 3, -126, 10, 13, -18, 51, 26, 127, -79, 35, 51, 12, -50, -24, 1, -7, 22,
81, 65, 120, -30, -38, 85, 122, -4, -106, -11, 27, 53, 41, 8, -104, -66, -38, -124, 10, 12, 76,
117, -109, 9, 11, 2, -18, 3, 113, -16, -79, -39, -123, -20, -128, 2, 13, -33, -58, 10, 84, -104,
13, 64, 109, 1, 54, -12, 28, 24, 63, -126, 118, -82, 46, -12, -15, 14, -43, 60, 22, -32, -19,
-46, 91, -107, 24, -94, 26, -47, 125, 6, 58, -15, -75, -26, -38, -35, 103, -16, -17, -13, 63,
-2, 45, -45, -73, -23, 70, -87, 51, -17, 53, 76, 14, -18, -31, -14, 103, 8, 21, -28, -33, -20,
-47, 6, 39, 40, -30, 7, -76, 55, 31, -20, -21, -59, 1, 25, -11, 17, 5, -13, -39, 0, -76, 50,
-33, -29, -50, -16, -11, -12, -1, -46, 40, -10, 65, -19, 21, -41, -32, -83, -19, -4, 49, -60,
118, -24, -46, 9, 102, -20, 8, -19, 25, 31, -3, -37, 0, 25, 7, 29, 2, -39, 127, -64, -20, 64,
115, -30, 36, 100, 35, 122, 127, 127, -127, 127, -127, 19, 127, -89, -79, -32, 39, -127, 125,
-80, 126, -127, 26, 8, 98, -8, -57, -90, -50, 126, 61, 127, -126, 40, -106, -68, 104, -125,
-119, 11, 10, -127, 66, -56, -12, -126, -104, 27, 75, 38, -124, -126, -125, 84, -123, -45, -114,
-128, 127, 103, -101, -124, 127, -11, -23, -123, 92, -123, 24, 126, 41, -2, -39, -27, -94, 40,
-112, -48, 127, 58, 14, 38, -75, -64, 73, 117, 100, -119, -11, 6, 32, -126, -14, 35, 121, -10,
54, -60, 89, -3, 69, -25, -20, 43, -86, -34, 24, 27, 7, -81, -99, -23, -16, -26, 13, 35, -97,
80, -29, -13, -121, -12, -65, -94, 70, -89, -126, -95, 88, 33, 96, 29, -90, 69, 114, -78, 65,
90, -47, -47, 89, 1, -12, 3, 8, 30, 5, 2, -30, -1, 6, -7, 10, -4, 46, -27, -40, 22, -6, -17, 45,
24, -9, 23, -14, -63, -26, -12, -57, 27, 25, 55, -76, -47, 21, 34, 33, 26, 17, 14, 6, 9, 26, 25,
-25, -25, -18
};
/** FC1 bias (24). */
static final byte[] INPUT_DENSE_BIAS = {
38, -6, 127, 127, 127, -43, -127, 78, 127, 5, 127, 123, 127, 127, -128, -76, -126, 28, 127, 125,
-30, 127, -89, -20
};
/** GRU1 input weights, [input][gate][output] (24 x 3 x 24). */
static final byte[] HIDDEN_GRU_WEIGHTS = {
-124, 23, -123, -33, -95, -4, 8, -84, 4, 101, -119, 116, -4, 123, 103, -51, 29, -124, -114, -49,
31, 9, 75, -128, 0, -49, 37, -50, 46, -21, -63, -104, 54, 82, 33, 21, 70, 127, -9, -79, -39,
-23, -127, 107, 122, -96, -46, -18, -39, 13, -28, -48, 14, 56, -52, 49, -1, -121, 25, -18, -36,
-52, -57, -30, 54, -124, -26, -47, 10, 39, 12, 2, 9, -127, -128, 102, 21, 11, -64, -71, 89,
-113, -111, 54, 31, 94, 121, -40, 30, 40, -109, 73, -9, 108, -92, 2, -127, 116, 127, 127, -122,
95, 127, -37, -127, 28, 89, 10, 24, -104, -62, -67, -14, 38, 14, -71, 22, -41, 20, -50, 39, 63,
86, 127, -18, 79, 4, -51, 2, 33, 117, -113, -78, 56, -91, 37, 34, -45, -44, -22, 21, -16, 56,
30, -84, -79, 38, -74, 127, 9, -25, 2, 82, 61, 25, -26, 26, 11, 117, -65, 12, -58, 42, -62, -93,
11, 11, 124, -123, 80, -125, 11, -90, 42, 94, 4, -109, -1, 85, -52, 45, -26, -27, 77, -5, 30,
90, 0, 95, -7, 53, 29, -82, 22, -9, 74, 2, -12, -73, 114, 97, -64, 122, -77, 43, 91, 86, 126,
106, 72, 90, -43, 46, 96, -51, 21, 22, 68, 22, 41, 79, 75, -46, -105, 23, -116, 127, -123, 102,
57, 85, 10, -29, 34, 125, 126, 124, 81, -15, 54, 96, -128, 39, -124, 103, 74, 126, 127, -50,
-71, -122, -64, 93, -75, 71, 105, 122, 123, 126, 122, -127, 33, -63, -74, 124, -71, 33, 41, -56,
19, 6, 65, 41, 90, -116, -3, -46, 75, -13, 98, -74, -42, 74, -95, -96, 81, 24, 32, -19, -123,
74, 55, 109, 115, 0, 32, 33, 12, -20, 9, 127, 127, -61, 79, -48, -54, -49, 101, -9, 27, -106,
74, 119, 77, 87, -126, -24, 127, 124, 31, 34, 127, 40, 3, -90, 127, 23, 57, -53, 127, -69, -88,
-33, 127, 19, -46, -9, -125, 13, -126, -113, 127, -41, 46, 106, -62, 3, -10, 111, 49, -34, -24,
-20, -112, 11, 101, -50, -34, 50, 65, -64, -106, 70, -48, 60, 9, -122, -45, 15, -112, -26, -4,
1, 39, 23, 58, -45, -80, 127, 82, 58, 30, -94, -119, 51, -89, 95, -107, 30, 127, 125, 58, -52,
-42, -38, -20, -122, 115, 39, -26, 5, 73, 13, -39, 43, -23, -20, -125, 23, 35, 53, -61, -66, 72,
-20, 33, 8, 35, 4, 7, 18, 19, 16, -45, -50, -71, 31, -29, -41, -27, 10, 14, 27, 9, -23, 98, 6,
-94, 92, 127, -114, 59, -26, -100, -62, -127, -17, -85, -60, 126, -42, -6, 33, -120, -26, -126,
-127, -35, -114, -31, 25, -126, -100, -126, -64, -46, -31, 30, 25, -74, -111, -97, -81, -104,
-114, -19, -9, -116, -69, 22, 30, 59, 8, -51, 16, -97, 18, -4, -89, 80, -50, 3, 36, -67, 56, 69,
-26, 107, -10, 58, -28, -4, -57, -72, -111, 0, -75, -119, 14, -75, -49, -66, -49, 8, -121, 22,
-54, 121, 30, 54, -26, -126, -123, 56, 5, 48, 21, -127, -11, 23, 25, -82, 6, -25, 119, 78, 4,
-104, 27, 61, -48, 37, -13, -52, 50, -50, 44, -1, -22, -43, -59, -78, -67, -32, -26, 9, -3, 40,
16, 19, 3, -9, 20, -6, -37, 28, 39, 17, -19, -10, 1, 6, -59, 74, 47, 3, -119, 0, -128, -107,
-25, -22, -69, -23, -111, -42, -93, -120, 90, -85, -54, -118, 76, -79, 124, 101, -77, -75, -17,
-71, -114, 68, 55, 79, -1, -123, -20, 127, -65, -123, -128, -87, 123, 9, -115, -14, 7, -4, 127,
-79, -115, 125, -28, 89, -83, 49, 89, 119, -69, -5, 12, -49, 60, 57, -24, -99, -110, 76, -83,
125, 73, 81, 11, 8, -45, 1, 83, 13, -70, -2, 97, 112, -97, 53, -9, -94, 124, 44, -49, -24, 52,
76, -110, -70, -114, -12, 72, -4, -114, 43, -43, 81, 102, -84, -27, 62, -40, 52, 58, 124, -35,
-51, -123, -43, 56, -75, -34, -35, -106, 93, -43, 14, -16, 46, 62, -97, 21, 30, -53, 21, -11,
-33, -20, -95, 4, -126, 12, 45, 20, 108, 85, 11, 20, -40, 99, 4, -25, -18, -23, -12, -126, -55,
-20, -44, -51, 91, -127, 127, -44, 7, 127, 78, 38, 125, -6, -94, -103, 73, 126, -126, 18, 59,
-46, 106, 76, 116, -31, 75, -4, 92, 102, 32, -31, 73, 42, -21, -28, 57, 127, -8, -107, 115, 124,
-94, -4, -128, 29, -57, 70, -82, 50, -13, -44, 38, 67, -93, 6, -39, -46, 56, 68, 27, 61, 26, 18,
-72, 127, 22, 18, -31, 127, 61, -65, -38, 1, -67, -1, 8, -73, 46, -116, -94, 58, -49, 71, -40,
-63, -82, -20, -60, 93, 76, 69, -106, 34, -31, 4, -25, 107, -18, 45, 4, -61, 126, 54, -126,
-125, 41, 19, 44, 32, -98, 125, -24, 125, -96, -125, 15, 87, -4, -90, 18, -40, 28, -69, 67, 22,
41, 39, 7, -48, -44, 12, 69, -13, 2, 44, -38, 111, -7, -126, -22, -9, 74, -128, -36, -7, -123,
-15, -79, -91, -37, -127, -122, 104, 30, 7, 98, -37, 111, -116, -47, 127, -45, 118, -111, -123,
-120, -77, -64, -125, 124, 77, 111, 77, 18, -113, 117, -9, 67, -77, 126, 49, -20, -124, 39, 41,
-124, -34, 114, -87, -126, 98, -20, 59, -17, -24, 125, 107, 54, 35, 33, -44, 12, -29, 125, -71,
-28, -63, -114, 28, -17, 121, -36, 127, 89, -122, -49, -18, -48, 17, 24, 19, -64, -128, 13, 86,
45, 13, -49, 55, 84, 48, 80, -39, 99, -127, 70, -33, 30, 50, 126, -65, -117, -13, -20, -24, 127,
115, -72, -104, 63, 126, -42, 57, 17, 46, 21, 119, 110, -100, -60, -112, 62, -33, 28, 26, -22,
-60, -33, -54, 78, 25, 32, -114, 86, 44, 26, 43, 76, 121, 19, 97, -2, -3, -73, -68, 6, -116, 6,
-43, -97, 46, -128, -120, -31, -119, -29, 16, 16, -126, -128, -126, -46, -9, -3, 92, -31, -76,
-126, -3, -107, -12, -23, -69, 5, 51, 27, -42, 23, -70, -128, -29, 22, 29, -126, -55, 50, -71,
-3, 127, 44, -27, -70, -63, -66, -70, 104, 86, 115, 29, -92, 41, -90, 44, -11, -28, 20, -11,
-63, -16, 43, 31, 17, -73, -31, -1, -17, -11, -39, 56, 18, 124, 72, -14, 28, 69, -121, -125, 34,
127, 63, 86, -80, -126, -125, -124, -47, 124, 77, 124, -19, 23, -7, -50, 96, -128, -93, 102,
-53, -36, -87, 119, -125, 92, -126, 118, 102, 72, -2, 125, 10, 97, 124, -125, 125, 71, -20, -47,
-116, -121, -4, -9, -32, 79, -124, -36, 33, -128, -74, 125, 23, 127, -29, -115, -32, 124, -89,
32, -107, 43, -17, 24, 24, 18, 29, -13, -15, -36, 62, -91, 4, -41, 95, 28, -23, 6, 46, 84, 66,
77, 68, -70, -1, -23, -6, 65, 70, -21, 9, 77, -12, 2, -118, 4, 9, -108, 84, 52, 2, 52, 13, -10,
58, -110, 18, 66, -95, -23, 70, 31, -3, 56, 56, -3, -7, 1, -27, -48, -61, 41, -4, 10, -62, 32,
-7, -24, 9, -48, -60, -4, 79, -20, -38, -76, 68, -49, -97, 0, -15, 5, -100, -49, -95, -99, -115,
-9, -40, 10, 104, 13, 56, 127, -27, -109, -94, -118, -102, -44, -85, 52, 127, -4, 14, 62, 121,
-122, -26, -79, -42, -34, 1, 25, -38, -79, -58, -31, -31, -90, -30, -123, 32, -56, 125, 66, 124,
-1, 3, 91, -103, -7, 23, 78, -18, 9, 69, -69, 76, -38, -33, -2, -98, 18, 106, 84, 55, 87, -47,
35, -124, 64, 41, -14, 46, 25, -2, 120, -21, 82, 19, -79, -37, -3, -8, -16, 21, 19, -5, -28,
-112, 39, -6, -30, 53, -69, 53, 46, 127, 123, 78, 20, 28, -7, 73, 72, 17, -40, 41, 111, 57, 32,
-95, 29, 28, -39, -65, 54, -20, -63, 29, -67, 3, 44, -57, -47, 11, 61, -22, -44, 61, 48, -100,
20, 125, 96, -24, -16, 3, -69, -126, 74, -125, 9, 45, -67, -123, -59, -72, 118, 69, 45, 50, -57,
67, 13, -66, -106, 47, 62, 22, -1, -22, -25, -40, -125, 3, 125, 32, 102, -56, -25, -75, -30,
122, 60, -13, 36, -73, 7, -84, 124, 40, -118, 17, -87, -118, -8, 3, -27, 111, -40, 40, -51, 127,
125, -45, -30, -54, 46, 80, -1, -30, 101, -17, 18, 26, 54, 7, -12, 1, -127, 123, -122, -27, -75,
64, 10, 25, -15, -44, 127, -127, 5, -84, -81, -7, 19, -26, 126, 15, 116, -126, 14, -76, 44, 62,
-110, -124, 125, -29, -87, -3, -69, 82, 90, 57, -123, 123, 100, -19, -51, -32, 69, 37, -57,
-128, -124, -72, -13, 51, -7, -45, -73, 5, 99, -26, -117, -96, -109, 4, -31, -12, 0, 31, -42,
-27, 12, -81, 118, 39, 83, 14, 41, -126, 107, -82, 94, -116, -122, -47, -109, -84, -128, -35,
-56, 66, 8, -65, 19, 42, -46, -72, -109, 41, 43, -127, -113, 58, 127, 42, -75, -1, 65, 117, -55,
-113, -123, 124, 43, -96, -115, -19, 68, 15, 94, 3, 75, 0, 34, 9, 42, 110, -48, 92, -76, 99,
-17, 27, 32, 13, 125, 50, -17, 56, 4, 53, 34, -8, 99, 80, -126, -21, -65, -11, -46, 44, -81, -3,
-121, 123, 66, -81, -84, 119, 127, 84, 105, 45, -66, -42, -23, 32, -25, 12, 111, 127, 88, 125,
30, 24, -127, -9, -54, 127, -116, -119, 88, 70, 94, -120, 35, -93, 15, 22, -21, 25, -110, -123,
-45, 8, -109, 125, -122, -86, -126, 8, -14, -120, -45, -45, 69, -125, -122, 6, 81, 86, 125, 95,
54, 77, 54, -123, 126, -85, -117, 56, 11, 0, -61, -91, -12, -2, -113, -3, -15, -122, -63, -91,
10, 84, -111, 125, 93, 21, 62, -78, -116, 13, -57, 28, -124, 126, 110, 12, 15, 95, 15, -19,
-125, -97, 52, -7, 101, 9, 20, -125, -26, -56, 72, 77, 12, -126, 22, -29, 47, 62, 95, 112, 69,
32, 97, -83, -8, -5, 67, -63, -123, 79, 59, 0, -6, -17, 4, -111, -52, 27, 65, 0
};
/** GRU1 recurrent weights, same layout (24 x 3 x 24). */
static final byte[] HIDDEN_GRU_RECURRENT_WEIGHTS = {
65, 83, 35, 56, 24, -34, -28, -2, 125, 19, 42, -9, 124, -53, 24, -87, 11, 35, -81, -35, -125,
-31, 123, -21, 33, -91, 113, -93, 45, -6, 53, 38, -92, 8, -27, 87, 4, 43, 43, 10, -128, -128,
-46, 127, -38, -45, 25, -87, 19, 5, 52, -96, -23, -29, 121, -126, -24, -20, -2, 69, -50, 6, 71,
-81, -125, 90, -94, 1, -38, 36, 89, 17, -60, 71, -48, 18, -15, 44, -18, 59, 11, 114, -51, 32,
110, 1, 4, 109, -24, 127, 27, 60, 88, 24, 45, -59, 75, -36, 8, 57, -32, -25, 13, 126, -89, -61,
-76, 127, 18, -62, -68, 23, -113, 5, 126, 43, -88, 26, -78, 18, 75, 21, 9, -74, 20, 41, 126,
-118, -15, 9, 116, 126, -127, 34, -6, 126, -128, -53, -54, -55, -121, 70, 127, -12, -68, 82,
-25, 104, -126, 126, -21, -26, 124, -75, -127, -120, 13, 61, -64, -108, -63, -65, -44, -35, -61,
-39, 109, -74, 113, -3, 108, -30, 125, 120, 39, 125, -128, -95, -99, 111, 9, 25, 114, -75, -92,
-54, -12, -32, -38, 10, 31, 10, 63, 51, 40, -99, 74, 4, 50, -128, -36, -35, -11, -28, -126, -7,
66, -58, -126, -22, -83, -61, -127, 49, 126, -8, 7, 62, 36, -11, -32, -44, 63, 116, 41, 65,
-127, 126, 63, -30, -96, 74, -92, 127, 38, -18, -128, 68, -5, 101, -4, 85, 58, 79, 0, -58, 8,
119, -70, -1, -79, -68, 114, -28, -90, -6, -112, 2, 127, -8, 10, 55, -59, -126, 127, 125, 80,
72, 35, -54, 95, -124, -124, 79, 23, -46, -61, -127, -100, 99, -77, 8, -87, 5, -2, 49, 85, 7,
-71, 82, 53, -41, 22, -22, -93, -103, 6, 52, -56, 14, -8, -111, 85, 16, 54, 32, -118, -24, 61,
-53, 96, -70, -5, -17, -67, -84, -7, -82, -107, -96, 21, -83, -58, 50, 12, -126, -1, -28, 34,
-126, 115, 17, 91, 1, -127, 72, 11, 126, -81, 6, 96, -8, 77, 15, -6, 63, -27, 20, -123, -109,
85, -79, -17, 126, -92, 2, -61, 20, 14, 17, 121, 123, 30, 57, 120, 127, 57, 42, 117, 98, 67, 39,
-20, -70, 100, 7, 125, 122, 40, 16, -79, 125, 83, 41, -106, -57, 24, 55, 27, -66, -111, -44, -7,
-43, -66, 121, 42, -128, -45, 35, 15, -127, 34, -35, -34, -40, -18, -6, 63, 111, 31, 116, 127,
19, 24, -71, -39, 34, 11, 19, -40, 27, 12, 106, -10, 56, -82, -106, -2, -50, -52, 114, -126,
-34, -43, -68, 10, 76, 57, -118, -128, 37, -104, 76, 125, 3, -76, 127, -29, 84, -94, -15, 55,
125, 79, 127, -57, -125, 104, -68, 126, 126, -77, 51, 45, 33, -109, 115, -11, 1, 95, -121, -5,
-9, -126, -114, 39, 68, -126, -107, -51, -42, 24, -8, 51, -27, -43, 66, -45, 62, -98, -109, 69,
67, 0, -125, -128, 49, 31, 126, -122, 2, -55, -67, -126, -70, -128, -125, -77, 25, 16, -8, -102,
11, -75, 82, 38, -5, 5, 19, 34, 47, -127, -93, 21, 24, -97, -18, 31, 39, 34, -20, 22, 123, 7,
-77, -81, -46, -9, 1, 23, 39, -127, -43, -8, -50, 10, -21, 59, -9, -4, -13, -27, 44, 127, 52,
-47, 70, -43, 52, 101, -49, 27, 45, 49, 33, -125, 55, 114, 20, -1, 76, -24, -96, 105, 24, 126,
75, -21, -105, 13, -42, 40, 126, -30, -39, -95, 125, -63, 11, 6, 125, 125, -14, 5, 42, -61, -4,
49, 88, 6, -107, -28, 19, -29, 47, 126, 6, -46, -89, -18, 91, -20, -6, 118, -21, -22, 39, 115,
11, -42, 54, 73, -55, -77, 62, -27, -59, -99, -12, -127, -40, 56, -3, -124, -91, 71, -111, 6,
-19, 82, -24, -35, 102, -42, 7, -126, -126, -125, 18, 98, -52, 127, 105, -52, 40, -83, 126,
-122, 109, 5, 127, 48, 6, 5, -125, 100, -16, 29, 85, -89, 8, 4, 41, 62, -127, 62, 122, 85, 122,
-107, 8, -125, 93, -127, 127, 102, 19, 19, -66, 41, -42, 114, 127, -48, -117, -29, -6, -73,
-102, -3, -19, 0, 88, 42, 87, -117, -20, 2, 122, 28, 63, 71, 66, 120, 93, 124, -43, 49, 103, 31,
90, -91, -22, -126, 26, -24, -21, 51, -126, 87, -103, -69, -10, -66, -23, 20, 97, 36, 25, -127,
30, -20, -63, 30, 51, -116, 23, 40, -39, 36, -83, -77, -25, -50, 110, 14, 13, -109, 125, -65,
-55, -87, 124, -126, -32, -72, -108, 127, 127, -125, -124, 61, 121, 102, -128, -127, 16, 100,
127, -124, -68, 72, -93, -128, 43, -93, -19, -125, -97, -113, -33, 83, 127, -44, 127, -75, 127,
16, 44, 50, -122, 23, 118, 46, 19, 26, -128, 10, 4, 99, -14, -82, -13, 30, 125, 57, 65, 60, -71,
35, 98, 28, 7, 1, 43, 89, 70, 75, 121, -59, 82, -126, -53, -16, -116, -65, 52, -52, 0, 80, 35,
45, -61, 46, 8, 107, 27, -26, -118, 90, 57, -10, 7, -15, 0, -39, -4, 12, 29, -1, 116, 84, 79,
119, 125, -59, 28, -6, -25, -43, 2, 90, 79, 67, 103, -82, 2, -6, 125, 19, 73, 0, -105, 112, -17,
104, 107, 124, 106, 19, 56, -44, 55, -112, 6, -39, -83, 126, -93, -98, 57, -120, -23, -38, 2,
-31, -48, 106, 127, 127, 69, 16, 110, 71, 104, 62, -12, -22, 42, -37, -94, 34, -1, -32, -12,
-124, -47, -13, 60, -75, -66, 58, -127, -2, 64, 76, -106, 73, -49, -31, 127, 126, 31, 16, 127,
-110, 107, -16, -53, 20, 69, -14, -125, 59, -44, 15, 120, 125, 125, 43, 6, 19, -58, 127, 127,
43, 16, 82, 97, -127, 127, -93, -41, 88, 0, 77, -15, 116, 16, -124, -31, -3, 95, -40, -126, -54,
-126, -83, -8, -59, 6, 67, -29, 4, 124, -10, 112, -28, -8, 85, -21, 45, 84, 6, -8, 11, 72, 32,
84, -62, 77, 2, -36, 75, 31, -50, 116, 126, 119, -88, -55, -14, -37, 126, 40, -108, -6, -6, 57,
64, -28, -76, 30, -117, -93, 31, -92, -44, -64, 94, 58, 65, 114, 41, 47, 71, 42, -26, 99, -126,
57, -5, 74, -19, -113, -1, 67, -21, 126, 1, -3, 33, 60, -82, 37, -48, 89, 114, -38, 127, -114,
35, 58, -5, 21, -46, 121, -123, -43, 127, 115, 123, 122, -101, 126, 127, 81, 52, 89, -127, 102,
42, 117, -9, -2, 125, 127, 110, 96, 120, 66, 70, 124, 55, 84, -38, -58, 119, -127, -16, -79,
123, 18, -127, -50, -38, 120, -85, 1, 7, -56, 108, -77, -2, 21, 37, 1, 13, -105, -69, 28, -87,
33, -104, -51, 126, 41, 3, -121, 28, 71, 58, 86, -8, 127, 94, -55, 125, 40, -19, 127, -33, -87,
-23, 7, -111, -68, 9, 84, -119, 55, -82, 78, -37, -20, -9, -23, 53, -13, 15, -46, 116, 126,
-127, 56, -126, 125, -7, -1, 45, 26, 125, 121, 29, 47, -86, 30, 10, 76, -125, -7, 23, 92, -12,
-39, -18, 92, -97, -8, -85, -41, 49, -50, 123, -37, -126, -30, 14, 79, -49, -65, 9, -36, -38,
-96, 85, -24, -13, 37, -25, -5, -64, -127, 55, -60, -18, -61, -63, 127, 56, 67, 15, 124, 72,
120, 127, 40, -10, 114, 24, -23, 46, 78, -53, 125, 86, 124, 86, 0, 38, 93, 21, 127, 123, 75,
-72, 13, 48, 33, 83, -51, 15, -32, -49, -33, 120, 64, 7, 9, 65, 60, 21, -21, -61, -53, -113, 84,
-97, 101, 37, -114, -27, 41, 73, 126, -10, 59, 61, -15, 70, -13, 82, -4, 69, 56, 94, -91, -50,
92, -74, -48, 53, -7, -107, 127, 28, 30, -26, -21, -61, 77, 82, 64, -91, -125, 122, -104, 127,
123, 122, 123, 76, -126, 127, -6, -80, 7, 40, -66, -65, 54, -2, 23, 96, -64, 74, 2, -53, -12,
-123, 39, 60, -20, 16, -17, -97, 23, -4, -53, -122, 32, -16, -54, -95, 43, 71, -1, -67, -33, 41,
18, 72, 28, -83, 31, -100, -91, -27, 10, -128, -106, 2, 76, -13, 42, 34, 112, -19, 44, 40, -9,
-11, 65, 92, -43, -125, 2, 47, -32, 25, 122, -29, 12, 101, -8, -126, -23, 43, 7, 125, -20, -124,
82, -2, 13, -73, -106, 115, 31, 116, -23, -44, -71, 84, 3, 47, 91, 127, 127, -15, 95, 7, 93, 5,
113, -50, 54, 11, 13, -127, 17, 72, 43, -23, 5, -70, 20, 15, -27, 99, 69, -109, -122, -94, 16,
127, 0, 116, 104, 45, 108, -34, 87, 72, -14, 118, 46, 42, 109, -26, 95, 93, 127, 60, 127, -93,
-54, -122, 34, -105, 56, 55, 103, 125, -71, -50, 95, -72, 127, 107, 21, 73, 126, 61, 127, 127,
24, -62, 90, 73, 90, -46, -78, -124, 72, 123, -42, 50, -107, 17, -32, -62, -89, 124, 1, 80, -2,
117, 119, -65, -127, -95, -121, -52, 103, 66, 75, -3, -62, -127, 127, -74, 124, 79, 49, 40, 105,
-67, -71, -70, 43, 127, 119, -4, 66, 43, 23, 91, -126, 15, 63, -119, 112, 103, 15, -99, 31,
-127, 69, 116, -46, -67, 2, -126, -29, 30, 30, -69, -98, -47, -87, -70, -127, 23, -73, 30, -7,
94, -52, -65, 98, -45, 97, 53, 23, -9, -22, -52, -47, 6, -1, -85, -15, -61, -14, 68, 110, -10,
-121, -25, -35, -15, -94, -123, 27, 75, 48, -66, -56, -44, 93, 109, 67, -36, 24, 70, -126, 8,
-127, 126, 52, 11, -32, 120, -13, -26, -28, -125, 127, 106, -50, 124, 36, -126, -12, 0, -23, 76,
-71, -126, -12, -17, -82, 12, 124, 57, 33, 4, 77, -46, 71, -34, 72, 125, -128, 124, -24, -128,
75, -120, 69, -45, 55, 33, 127, -33, 4, -105, -41, -59, -91, 123, 44, -127, 127, -67, 52, 25,
-125, -65, 100, -25, 123, 6, 11, -123, -92, -33, 126, -17, -4, 29, 33, 127, 96, 3, 87, -48, -18,
-70, 123, 58, -127, -3, -52, -1, -36, -41, 127, 51, -52, -27, 46, -83, 57, 9, 126, 127, 94, 79,
-37, -127, -40, 67, 52, 82, -66, 122, -13, -73, 127, -8, -80, 46, -48, 4, -54
};
/** GRU1 bias, [gate][output] (3 x 24). */
static final byte[] HIDDEN_GRU_BIAS = {
124, 125, -57, -126, 53, 123, 127, -75, 68, 102, -2, 116, 124, 127, 124, 125, 126, 123, -16, 48,
125, 126, 78, 85, 11, 126, -30, -30, -64, -3, -105, -29, -17, 69, 63, 2, -32, -10, -62, 113,
-52, 112, -109, 112, 7, -40, 73, 53, 62, 6, -2, 0, 0, 100, -16, 26, -24, 56, 26, -10, -33, 41,
70, 109, -29, 127, 34, -66, 49, 53, 27, 62
};
/** FC2 weights (24 x 1). */
static final byte[] OUTPUT_DENSE_WEIGHTS = {
127, 127, 127, 127, 127, 20, 127, -126, -126, -54, 14, 125, -126, -126, 127, -125, -126, 127,
-127, -127, -57, -30, 127, 80
};
/** FC2 bias (1). */
static final byte[] OUTPUT_DENSE_BIAS = {
-50
};
private RnnVadWeights() {
}
}

View File

@@ -0,0 +1,281 @@
package com.ts3client.audio.vad;
import static com.ts3client.audio.vad.VadConstants.CEPSTRAL_HISTORY;
import static com.ts3client.audio.vad.VadConstants.FRAME_20MS_24K;
import static com.ts3client.audio.vad.VadConstants.NUM_BANDS;
import static com.ts3client.audio.vad.VadConstants.NUM_LOWER_BANDS;
import static com.ts3client.audio.vad.VadConstants.OPUS_BANDS_24K;
/**
* Band-wise spectral features for {@code rnn_vad}, ported from WebRTC's
* {@code rnn_vad/spectral_features.cc} and {@code spectral_features_internal.cc}.
*
* <p>A 20&nbsp;ms Vorbis-windowed FFT is folded into 22 Opus-scale bands, log-compressed
* and DCT'd into a cepstrum. The network is fed the higher cepstral coefficients, the
* running average and first two derivatives of the lower ones, the cepstrum of the
* band-wise correlation between the current frame and the frame one pitch period earlier,
* and a measure of how much the cepstrum has been varying.
*/
final class SpectralFeatures {
/** Total band energy below which the frame is treated as silence. */
private static final float SILENCE_THRESHOLD = 0.04f;
/** FFT bins per Opus band at 24 kHz for a 20 ms frame. */
private static final int[] OPUS_BAND_SIZES =
{4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 16, 16, 16, 24, 24, 32, 48};
/** sqrt(2 / NUM_BANDS), applied after the DCT. */
private static final float DCT_SCALING_FACTOR = 0.301511345f;
private final RealFft fft = new RealFft(FRAME_20MS_24K);
private final float[] halfWindow = new float[FRAME_20MS_24K / 2];
private final float[] bandWeights = new float[FRAME_20MS_24K / 2];
private final float[] dctTable = new float[NUM_BANDS * NUM_BANDS];
private final float[] windowed = new float[FRAME_20MS_24K];
private final float[] referenceFft = new float[FRAME_20MS_24K];
private final float[] laggedFft = new float[FRAME_20MS_24K];
private final float[] referenceBandsEnergy = new float[OPUS_BANDS_24K];
private final float[] laggedBandsEnergy = new float[OPUS_BANDS_24K];
private final float[] bandsCrossCorr = new float[OPUS_BANDS_24K];
private final float[] logBandsEnergy = new float[NUM_BANDS];
private final float[] cepstrum = new float[NUM_BANDS];
private final float[] crossCorrCepstrum = new float[NUM_LOWER_BANDS];
/** Ring buffer of the last {@link VadConstants#CEPSTRAL_HISTORY} cepstra. */
private final float[][] cepstralHistory = new float[CEPSTRAL_HISTORY][NUM_BANDS];
private int cepstralTail;
/**
* Upper-right triangle of pairwise cepstral distances, stored as a square matrix so a
* new frame can be shifted in with a single array copy.
*/
private final float[] cepstralDiffs = new float[(CEPSTRAL_HISTORY - 1) * (CEPSTRAL_HISTORY - 1)];
private final float[] newDistances = new float[CEPSTRAL_HISTORY - 1];
SpectralFeatures() {
int halfSize = FRAME_20MS_24K / 2;
float scaling = 1.0f / FRAME_20MS_24K;
for (int i = 0; i < halfSize; i++) {
double inner = Math.sin(0.5 * Math.PI * (i + 0.5) / halfSize);
halfWindow[i] = (float) (scaling * Math.sin(0.5 * Math.PI * inner * inner));
}
// Each band overlaps its neighbour with a triangular response; the weight is the
// fractional position of the bin within its band.
int bin = 0;
for (int bandSize : OPUS_BAND_SIZES) {
for (int j = 0; j < bandSize; j++) {
bandWeights[bin++] = (float) j / bandSize;
}
}
double k = Math.sqrt(0.5);
for (int i = 0; i < NUM_BANDS; i++) {
for (int j = 0; j < NUM_BANDS; j++) {
dctTable[i * NUM_BANDS + j] = (float) Math.cos((i + 0.5) * j * Math.PI / NUM_BANDS);
}
dctTable[i * NUM_BANDS] *= (float) k;
}
}
void reset() {
for (float[] row : cepstralHistory) {
java.util.Arrays.fill(row, 0.0f);
}
cepstralTail = 0;
java.util.Arrays.fill(cepstralDiffs, 0.0f);
}
/**
* Writes the spectral part of the feature vector and returns {@code true} if the
* reference frame is silence, in which case the vector is only partially written.
*
* @param reference buffer holding the most recent 20 ms at 24 kHz
* @param referenceOffset offset of that frame within {@code reference}
* @param lagged buffer holding the 20 ms one pitch period earlier
* @param laggedOffset offset of that frame within {@code lagged}
*/
boolean checkSilenceComputeFeatures(float[] reference, int referenceOffset,
float[] lagged, int laggedOffset,
float[] featureVector) {
forwardFft(reference, referenceOffset, referenceFft);
crossCorrelation(referenceFft, referenceFft, referenceBandsEnergy);
float totalEnergy = 0;
for (float e : referenceBandsEnergy) {
totalEnergy += e;
}
if (totalEnergy < SILENCE_THRESHOLD) {
return true;
}
forwardFft(lagged, laggedOffset, laggedFft);
crossCorrelation(laggedFft, laggedFft, laggedBandsEnergy);
smoothedLogMagnitudeSpectrum(referenceBandsEnergy, logBandsEnergy);
dct(logBandsEnergy, NUM_BANDS, cepstrum, NUM_BANDS);
// Ad-hoc correction terms for the first two cepstral coefficients.
cepstrum[0] -= 12.0f;
cepstrum[1] -= 4.0f;
pushCepstrum(cepstrum);
// Higher bands go in as-is; the lower ones are summarised over time instead.
System.arraycopy(cepstrum, NUM_LOWER_BANDS, featureVector, NUM_LOWER_BANDS,
NUM_BANDS - NUM_LOWER_BANDS);
computeAverageAndDerivatives(featureVector);
computeNormalizedCepstralCorrelation(featureVector);
featureVector[VadConstants.FEATURE_VECTOR_SIZE - 1] = computeVariability();
return false;
}
/** Windows a 20 ms frame and transforms it, zeroing the unused Nyquist coefficient. */
private void forwardFft(float[] frame, int offset, float[] out) {
int halfSize = FRAME_20MS_24K / 2;
for (int i = 0, j = FRAME_20MS_24K - 1; i < halfSize; i++, j--) {
windowed[i] = frame[offset + i] * halfWindow[i];
windowed[j] = frame[offset + j] * halfWindow[i];
}
fft.forward(windowed, out);
out[1] = 0.0f;
}
/**
* Folds the interleaved spectra into Opus bands. Each bin contributes to the band it
* sits in and to the next one, split by the triangular weight.
*/
private void crossCorrelation(float[] x, float[] y, float[] out) {
int k = 0;
out[0] = 0.0f;
for (int i = 0; i < OPUS_BANDS_24K - 1; i++) {
out[i + 1] = 0.0f;
for (int j = 0; j < OPUS_BAND_SIZES[i]; j++) {
float v = x[2 * k] * y[2 * k] + x[2 * k + 1] * y[2 * k + 1];
float tmp = bandWeights[k] * v;
out[i] += v - tmp;
out[i + 1] += tmp;
k++;
}
}
out[0] *= 2.0f; // The first band only receives half a triangle.
}
/** Log magnitude with smoothing over frequency and a decaying floor. */
private static void smoothedLogMagnitudeSpectrum(float[] bandsEnergy, float[] out) {
final float logOneByHundred = -2.0f;
float logMax = logOneByHundred;
float follow = logOneByHundred;
for (int i = 0; i < out.length; i++) {
float x = (i < bandsEnergy.length)
? (float) Math.log10(0.01f + bandsEnergy[i])
: logOneByHundred;
x = Math.max(logMax - 7.0f, Math.max(follow - 1.5f, x));
logMax = Math.max(logMax, x);
follow = Math.max(follow - 1.5f, x);
out[i] = x;
}
}
/** DCT-II of the first {@code inSize} values of {@code in}, truncated to {@code outSize}. */
private void dct(float[] in, int inSize, float[] out, int outSize) {
for (int i = 0; i < outSize; i++) {
float sum = 0;
for (int j = 0; j < inSize; j++) {
sum += in[j] * dctTable[j * NUM_BANDS + i];
}
out[i] = sum * DCT_SCALING_FACTOR;
}
}
private void pushCepstrum(float[] newCepstrum) {
// Distance from the new cepstrum to each older one, nearest first.
for (int i = 0; i < CEPSTRAL_HISTORY - 1; i++) {
float[] old = cepstralHistoryAt(i);
float sum = 0;
for (int k = 0; k < NUM_BANDS; k++) {
float c = newCepstrum[k] - old[k];
sum += c * c;
}
newDistances[i] = sum;
}
System.arraycopy(newCepstrum, 0, cepstralHistory[cepstralTail], 0, NUM_BANDS);
cepstralTail = (cepstralTail + 1) % CEPSTRAL_HISTORY;
pushCepstralDistances();
}
/** Returns the cepstrum pushed {@code delay} frames ago (0 = most recent). */
private float[] cepstralHistoryAt(int delay) {
int offset = cepstralTail - 1 - delay;
if (offset < 0) {
offset += CEPSTRAL_HISTORY;
}
return cepstralHistory[offset];
}
/** Shifts the distance matrix up-left by one and writes the new column. */
private void pushCepstralDistances() {
final int s = CEPSTRAL_HISTORY;
System.arraycopy(cepstralDiffs, s, cepstralDiffs, 0, cepstralDiffs.length - s);
for (int i = 0; i < newDistances.length; i++) {
cepstralDiffs[(s - 1 - i) * (s - 1) - 1] = newDistances[i];
}
}
private float cepstralDistance(int delay1, int delay2) {
int row = CEPSTRAL_HISTORY - 1 - delay1;
int col = CEPSTRAL_HISTORY - 1 - delay2;
if (row > col) { // Only the upper-right triangle is stored.
int tmp = row;
row = col;
col = tmp;
}
return cepstralDiffs[row * (CEPSTRAL_HISTORY - 1) + (col - 1)];
}
private void computeAverageAndDerivatives(float[] featureVector) {
float[] curr = cepstralHistoryAt(0);
float[] prev1 = cepstralHistoryAt(1);
float[] prev2 = cepstralHistoryAt(2);
for (int i = 0; i < NUM_LOWER_BANDS; i++) {
featureVector[i] = curr[i] + prev1[i] + prev2[i]; // kernel [1, 1, 1]
featureVector[NUM_BANDS + i] = curr[i] - prev2[i]; // kernel [1, 0, -1]
featureVector[NUM_BANDS + NUM_LOWER_BANDS + i] =
curr[i] - 2 * prev1[i] + prev2[i]; // kernel [1, -2, 1]
}
}
/** Band correlation between the current frame and the one a pitch period earlier. */
private void computeNormalizedCepstralCorrelation(float[] featureVector) {
crossCorrelation(referenceFft, laggedFft, bandsCrossCorr);
for (int i = 0; i < bandsCrossCorr.length; i++) {
bandsCrossCorr[i] /= (float) Math.sqrt(
0.001f + referenceBandsEnergy[i] * laggedBandsEnergy[i]);
}
dct(bandsCrossCorr, OPUS_BANDS_24K, crossCorrCepstrum, NUM_LOWER_BANDS);
// Ad-hoc correction terms, as for the frame cepstrum.
crossCorrCepstrum[0] -= 1.3f;
crossCorrCepstrum[1] -= 0.9f;
System.arraycopy(crossCorrCepstrum, 0, featureVector,
NUM_BANDS + 2 * NUM_LOWER_BANDS, NUM_LOWER_BANDS);
}
/**
* How far the current cepstrum sits from its nearest neighbour in the recent history,
* averaged over the history. Steady tones score low, speech scores high.
*/
private float computeVariability() {
float variability = 0;
for (int delay1 = 0; delay1 < CEPSTRAL_HISTORY; delay1++) {
float minDist = Float.MAX_VALUE;
for (int delay2 = 0; delay2 < CEPSTRAL_HISTORY; delay2++) {
if (delay1 == delay2) {
continue; // The distance would be 0.
}
minDist = Math.min(minDist, cepstralDistance(delay1, delay2));
}
variability += minDist;
}
// Normalised against training-set statistics.
return variability / CEPSTRAL_HISTORY - 2.1f;
}
}

View File

@@ -0,0 +1,54 @@
package com.ts3client.audio.vad;
/**
* Sizes and pitch-search bounds of WebRTC's {@code rnn_vad}, mirroring
* {@code modules/audio_processing/agc2/rnn_vad/common.h}.
*
* <p>The detector analyses 10&nbsp;ms frames at 24&nbsp;kHz. Spectral analysis uses the
* most recent 20&nbsp;ms; pitch search additionally needs one maximum pitch period of
* history, hence the 864-sample buffer.
*/
final class VadConstants {
static final int SAMPLE_RATE_24K = 24_000;
static final int FRAME_10MS_24K = SAMPLE_RATE_24K / 100; // 240
static final int FRAME_20MS_24K = FRAME_10MS_24K * 2; // 480
/** Pitch range at 24 kHz: 0.00125 s to 0.016 s. */
static final int MIN_PITCH_24K = SAMPLE_RATE_24K / 800; // 30
static final int MAX_PITCH_24K = 384; // 24000 / 62.5
static final int BUF_SIZE_24K = MAX_PITCH_24K + FRAME_20MS_24K; // 864
/**
* The coarse search skips very short periods; a refinement step recovers them, so the
* initial pass starts three minimum periods in.
*/
static final int INITIAL_MIN_PITCH_24K = 3 * MIN_PITCH_24K; // 90
static final int INITIAL_NUM_LAGS_24K = MAX_PITCH_24K - INITIAL_MIN_PITCH_24K; // 294
static final int REFINE_NUM_LAGS_24K = MAX_PITCH_24K + 1; // 385
// 12 kHz analysis, used for the coarse pitch search.
static final int FRAME_20MS_12K = 240;
static final int BUF_SIZE_12K = BUF_SIZE_24K / 2; // 432
static final int INITIAL_MIN_PITCH_12K = INITIAL_MIN_PITCH_24K / 2; // 45
static final int MAX_PITCH_12K = MAX_PITCH_24K / 2; // 192
static final int NUM_LAGS_12K = MAX_PITCH_12K - INITIAL_MIN_PITCH_12K; // 147
// The pitch period fed to the network is expressed at 48 kHz.
static final int MIN_PITCH_48K = MIN_PITCH_24K * 2; // 60
static final int MAX_PITCH_48K = MAX_PITCH_24K * 2; // 768
// Spectral features.
static final int NUM_BANDS = 22;
static final int NUM_LOWER_BANDS = 6;
static final int CEPSTRAL_HISTORY = 8;
/** Opus bands that fall below the 24 kHz Nyquist frequency. */
static final int OPUS_BANDS_24K = 20;
static final int NUM_LPC_COEFFICIENTS = 5;
static final int FEATURE_VECTOR_SIZE = 42;
private VadConstants() {
}
}

View File

@@ -0,0 +1,88 @@
package com.ts3client.config;
import java.util.List;
/**
* The version, platform and signature a client reports to the server in
* {@code clientinit}. Other clients see the version and platform in the info
* panel ("3.6.2 on Windows").
*
* <p>The signature is TeamSpeak's, over the version and platform strings
* together, so the three only work as a set. Servers answer "client is modified"
* for a version they do not accept — an invented one, or one older than their
* configured minimum — which is why {@link #PRESETS} holds genuine triples taken
* from released clients, newest first per platform.
*
* @param label human-readable name, e.g. "3.6.2 on Windows"
* @param version {@code client_version}, e.g. "3.6.2 [Build: 1695203293]"
* @param platform {@code client_platform}, e.g. "Windows"
* @param sign {@code client_version_sign}, base64
*/
public record ClientVersion(String label, String version, String platform, String sign) {
/** Platform names the official clients report. */
public static final List<String> PLATFORMS =
List.of("Windows", "Linux", "OS X", "Android", "iOS");
/**
* Signed triples from released TeamSpeak clients. "3.?.?" is the version the
* SDK-based clients report; it is signed for every platform and is what this
* client uses by default.
*/
public static final List<ClientVersion> PRESETS = List.of(
new ClientVersion("3.?.? on Windows", "3.?.? [Build: 5680278000]", "Windows",
"DX5NIYLvfJEUjuIbCidnoeozxIDRRkpq3I9vVMBmE9L2qnekOoBzSenkzsg2lC9CMv8K5hkEzhr2TYUYSwUXCg=="),
new ClientVersion("3.6.3 on Windows", "3.6.3 [Build: 1701166057]", "Windows",
"ML3yE6pV0QGyhc8o7k1bgqDSj+5gZIQFMjBNhWJ6L0FtJwniBq8l1wivx3DzgVVCvyRbzfg8MsdV/rjl5sHNCw=="),
new ClientVersion("3.6.2 on Windows", "3.6.2 [Build: 1695203293]", "Windows",
"4BdaZpdgUSMCuIs8qcloJPNxNlJ4o7QKnxMCRO60mSOTtJZyKjOrGLAmeAEtLIJjcjmdSpycMbQOIV92K2vXAw=="),
new ClientVersion("3.6.1 on Windows", "3.6.1 [Build: 1690872913]", "Windows",
"+YmPaYUBL4F9+TgxyFhlT8qIi1Ym+So4NNMcuohla/u4kfJCGabZ1vDF4cIOAghIqWfO+bQ0OkORZrCBdpi2Cw=="),
new ClientVersion("3.6.0 on Windows", "3.6.0 [Build: 1686749764]", "Windows",
"gfzG6CfwNYeqrjT68s7ahPd24A/XS0P8LLAB7hrE54MnrlO12RiflD2w69VZ7EYlqnd9GAiVzdGJHCmCFWtlAQ=="),
new ClientVersion("3.?.? on Linux", "3.?.? [Build: 5680278000]", "Linux",
"Hjd+N58Gv3ENhoKmGYy2bNRBsNNgm5kpiaQWxOj5HN2DXttG6REjymSwJtpJ8muC2gSwRuZi0R+8Laan5ts5CQ=="),
new ClientVersion("3.6.2 on Linux", "3.6.2 [Build: 1695203293]", "Linux",
"p4iF1jZ3ZOz9MEkKSZ2bvnFtm9WmUcQy9mAP//erFE4PF1sB6K1CSANrr+3X4B0aZR0u+K2pjnv8kiKsWKQCBQ=="),
new ClientVersion("3.6.1 on Linux", "3.6.1 [Build: 1690193193]", "Linux",
"UAcE5h5FgPcdChWTtiO816HX8f2qdoDcDEw6crbJ96Yo7rp1Bt+qx/rkKK9bzlVWMhOd4fI7pr++1XaPkEBnAQ=="),
new ClientVersion("3.6.0 on Linux", "3.6.0 [Build: 1686749764]", "Linux",
"TccRBHwn5LSA0lfpa2B42QmfCOiG+MqqyhlOqy5RcuRQNUGPD0e47lUz+Z/8+Gsdc1TPG4+vYlRl8+v2v+26Aw=="),
new ClientVersion("3.5.8 on Linux", "3.5.8 [Build: 1644595825]", "Linux",
"CK+cWJQESZgUdw1AFHINMbwiLjFGSiupour0Dfj/j1/xwx4L3nW3G+alYtiQw/wQwlWm873HS6PvXUBarz/tCw=="),
new ClientVersion("3.?.? on OS X", "3.?.? [Build: 5680278000]", "OS X",
"SttEnjoWE8jqIM6BOHSfiZP9DGjW0EP/ajU4bdKqgGMV4aYq/kzwVA9gxbmdIzV4lbaokvXBqrRjfBHrTVh8Cg=="),
new ClientVersion("3.6.2 on OS X", "3.6.2 [Build: 1695203293]", "OS X",
"SOCkLcsxINIGVc+3OQ5Vrmv2BVNNFA9LX5Jj9tqbIiYv2HU6Z5E8AL5+v5G2LDN0F+U55+uW43ezUoYAr1WuBA=="),
new ClientVersion("3.6.1 on OS X", "3.6.1 [Build: 1690193193]", "OS X",
"P+KOw42V8K0nN5FbGItCaJzrDuWcRgfE3MRzrtbYAhRMTXHzRxvwLkoLM7+vKTW135NtWZlKbZuDAvxgwhSLBg=="),
new ClientVersion("3.6.0 on OS X", "3.6.0 [Build: 1686749764]", "OS X",
"g8H7Op8eCrn0wHutuD/SlmUcZPmkSlFUBhvZpSgm9kuQl4dapzKJq2anI5jDwRs/ooBm/wrIlSiWI6HXuOPZDA=="),
new ClientVersion("3.?.? on Android", "3.?.? [Build: 5680278000]", "Android",
"AWb948BY32Z7bpIyoAlQguSmxOGcmjESPceQe1DpW5IZ4+AW1KfTk2VUIYNfUPsxReDJMCtlhVKslzhR2lf0AA=="),
new ClientVersion("3.8.7 on Android", "3.8.7 [Build: 1785186501]", "Android",
"mbt0WPhXJ+YKJUix5b6t+ZOHTCfvRqhhhrpcS5koM9CUL0BAFQZRG5Di5rJrqipZdk+gEUtpAX9I5nkuL7V9Dw=="),
new ClientVersion("3.8.6 on Android", "3.8.6 [Build: 1774892004]", "Android",
"Hh0oEZupsq8f0BuUHhMKdkXsMmgZZ03dJL8qAwnlH2JAHuefaAZFS/buiH2MLwgdRkZ81ffYk5YOTfc/bJczDw=="),
new ClientVersion("3.?.? on iOS", "3.?.? [Build: 5680278000]", "iOS",
"XrAf+Buq6Eb0ehEW/niFp06YX+nGGOS0Ke4MoUBzn+cX9q6G5C0A/d5XtgcNMe8r9jJgV/adIYVpsGS3pVlSAA=="),
new ClientVersion("3.8.2 on iOS", "3.8.2 [Build: 1764028963]", "iOS",
"Bc57xjumaioWkouTRqoS6XbSIXysrPH+gxynXgL4/nYr38l4OocUTpQaMMHevmgk/FZxJWFNEhU3Hpg5+M+MAA=="),
new ClientVersion("3.8.1 on iOS", "3.8.1 [Build: 1763403218]", "iOS",
"GM/7isPK6LDKTN3Hdelenbr99dnPyWTccLCOCFKK41321XMsdG02putu+NskCLddlDPu2CPYJ4YABRS2pJ88Ag=="));
/** The triple used when the user has not chosen one; matches the library's built-in. */
public static final ClientVersion DEFAULT = PRESETS.get(0);
/** The preset carrying exactly these three values, or null when the user typed their own. */
public static ClientVersion match(String version, String platform, String sign) {
return PRESETS.stream()
.filter(v -> v.version.equals(version) && v.platform.equals(platform) && v.sign.equals(sign))
.findFirst()
.orElse(null);
}
}

View File

@@ -51,6 +51,16 @@ public final class Settings {
/** Id of the identity used when a server doesn't select one of its own. */
public String defaultIdentityId = "";
// ---- reported client version ----
/** Report {@link #clientVersion}/{@link #clientPlatform} instead of the built-in ones. */
public boolean customVersion = false;
/** Version string other clients see, e.g. "3.5.6 [Build: 1603963200]". */
public String clientVersion = ClientVersion.DEFAULT.version();
/** Operating system other clients see, e.g. "Windows". */
public String clientPlatform = ClientVersion.DEFAULT.platform();
/** TeamSpeak's signature over the version and platform; servers may verify it. */
public String clientVersionSign = ClientVersion.DEFAULT.sign();
// ---- audio devices (mixer names; empty = system default) ----
public String inputDevice = "";
public String outputDevice = "";
@@ -59,14 +69,18 @@ public final class Settings {
public InputMode inputMode = InputMode.VOICE_ACTIVATION;
/** Voice-activation strategy used when {@link #inputMode} is VOICE_ACTIVATION. */
public VadMode vadMode = VadMode.HYBRID;
/** VAD threshold in dBFS (RMS). Typical range -60 (sensitive) .. -10 (loud). */
public double vadThresholdDb = -45.0;
/** Speech-probability threshold (0..1) for Automatic/Hybrid modes. */
/**
* Volume-gate threshold on {@link com.ts3client.audio.InputLevel}'s scale, the same one
* the TS3 client's slider uses: -50 (most sensitive) .. +50, defaulting to -40.
*/
public double vadThresholdDb = -40.0;
/**
* Speech-probability threshold (0..1) for Automatic/Hybrid modes. TS3 calls this
* {@code vad_likelihood} and also defaults it to 0.5.
*/
public double speechThreshold = 0.5;
/** Keep voice activation running while in push-to-talk mode. */
public boolean vadOverPtt = false;
/** Push-to-talk key as an AWT virtual-key code; the frontend interprets it. Default: Ctrl (VK_CONTROL). */
public int pushToTalkKey = 17;
/** Opus target bitrate in bits/sec. */
public int bitrate = 48000;
/** Opus complexity 0..10. */
@@ -152,14 +166,18 @@ public final class Settings {
serverPassword = props.getProperty("serverPassword", serverPassword);
identityFile = props.getProperty("identityFile", identityFile);
defaultIdentityId = props.getProperty("defaultIdentityId", defaultIdentityId);
customVersion = parseB(props.getProperty("customVersion"), customVersion);
clientVersion = props.getProperty("clientVersion", clientVersion);
clientPlatform = props.getProperty("clientPlatform", clientPlatform);
clientVersionSign = props.getProperty("clientVersionSign", clientVersionSign);
inputDevice = props.getProperty("inputDevice", inputDevice);
outputDevice = props.getProperty("outputDevice", outputDevice);
inputMode = parseMode(props.getProperty("inputMode"), inputMode);
vadMode = parseVadMode(props.getProperty("vadMode"), vadMode);
vadThresholdDb = parseD(props.getProperty("vadThresholdDb"), vadThresholdDb);
vadThresholdDb = parseD(props.getProperty("vadLevelDb"),
migrateThreshold(props.getProperty("vadThresholdDb"), vadThresholdDb));
speechThreshold = parseD(props.getProperty("speechThreshold"), speechThreshold);
vadOverPtt = parseB(props.getProperty("vadOverPtt"), vadOverPtt);
pushToTalkKey = parseI(props.getProperty("pushToTalkKey"), pushToTalkKey);
bitrate = parseI(props.getProperty("bitrate"), bitrate);
complexity = parseI(props.getProperty("complexity"), complexity);
vbr = parseB(props.getProperty("vbr"), vbr);
@@ -186,14 +204,18 @@ public final class Settings {
props.setProperty("serverPassword", serverPassword);
props.setProperty("identityFile", identityFile);
props.setProperty("defaultIdentityId", defaultIdentityId);
props.setProperty("customVersion", Boolean.toString(customVersion));
props.setProperty("clientVersion", clientVersion);
props.setProperty("clientPlatform", clientPlatform);
props.setProperty("clientVersionSign", clientVersionSign);
props.setProperty("inputDevice", inputDevice);
props.setProperty("outputDevice", outputDevice);
props.setProperty("inputMode", inputMode.name());
props.setProperty("vadMode", vadMode.name());
props.setProperty("vadThresholdDb", Double.toString(vadThresholdDb));
props.setProperty("vadLevelDb", Double.toString(vadThresholdDb));
props.remove("vadThresholdDb");
props.setProperty("speechThreshold", Double.toString(speechThreshold));
props.setProperty("vadOverPtt", Boolean.toString(vadOverPtt));
props.setProperty("pushToTalkKey", Integer.toString(pushToTalkKey));
props.setProperty("bitrate", Integer.toString(bitrate));
props.setProperty("complexity", Integer.toString(complexity));
props.setProperty("vbr", Boolean.toString(vbr));
@@ -232,6 +254,20 @@ public final class Settings {
}
}
/**
* Converts a threshold saved under the old {@code vadThresholdDb} key, which was in
* dBFS ({@code 20*log10(rms)}), to {@link com.ts3client.audio.InputLevel}'s scale, so
* upgrading doesn't silently change how sensitive the gate is.
*/
private static double migrateThreshold(String legacy, double def) {
double dbfs = parseD(legacy, Double.NaN);
if (Double.isNaN(dbfs)) {
return def;
}
double converted = com.ts3client.audio.InputLevel.powerToDb(Math.pow(10.0, dbfs / 10.0));
return Math.min(com.ts3client.audio.InputLevel.MAX_DB, converted);
}
private static double parseD(String v, double def) {
if (v == null) return def;
try {

View File

@@ -0,0 +1,38 @@
package com.ts3client.hotkey;
/**
* A system-wide source of key and mouse-button events: it reports every press and
* release on the machine, whether or not the client has focus, and without swallowing
* the event on its way to the focused application.
*
* <p>Implementations are platform code and live outside the core; the core only ever
* sees this interface.
*/
public interface GlobalInputHook extends AutoCloseable {
interface Listener {
/**
* @param key the key or button
* @param pressed {@code true} for a press, {@code false} for a release
*/
void onInput(HotkeyKey key, boolean pressed);
}
/** Starts delivering events on a background thread. */
void start(Listener listener);
/** Whether events are actually being delivered; false when the platform refused. */
boolean isRunning();
/** Why the hook is not running, for the options dialog to show; empty when it is. */
String unavailableReason();
/**
* The name this system's layout gives a key, or {@code null} to let the caller fall
* back to {@link HotkeyKey#fallbackName()}.
*/
String keyName(HotkeyKey key);
@Override
void close();
}

View File

@@ -0,0 +1,74 @@
package com.ts3client.hotkey;
/**
* A single binding: which combination triggers which action, on which edge of the
* keypress, and whether it reaches only the active server or every connected one.
*/
public final class Hotkey {
/** Which edge of the key press runs the action, as TS3's hotkey mode combo box offers. */
public enum Trigger {
/** Action triggers when the key is pressed down. */
KEY_DOWN("On key down"),
/** Action triggers when the key is released. */
KEY_UP("On key up");
private final String label;
Trigger(String label) {
this.label = label;
}
public String label() {
return label;
}
}
public HotkeyAction action;
public HotkeyCombo combo;
public Trigger trigger = Trigger.KEY_DOWN;
/** Apply to the active server only; when false the action reaches every connection. */
public boolean activeServerOnly = true;
/** Meaning depends on {@link HotkeyAction#argument()}; empty when the action takes none. */
public String argument = "";
public boolean enabled = true;
public Hotkey() {
}
public Hotkey(HotkeyAction action, HotkeyCombo combo) {
this.action = action;
this.combo = combo;
}
/**
* The binding's place in the action tree, as TS3 spells it out:
* {@code Sounds / Activate Soundpack / Default Sound Pack (Male)}.
*/
public String path() {
if (action == null) return "";
StringBuilder sb = new StringBuilder();
if (!action.group().isEmpty()) sb.append(action.group()).append(" / ");
sb.append(action.label());
if (argument != null && !argument.isBlank()) sb.append(" / ").append(argument.trim());
return sb.toString();
}
public boolean isValid() {
return action != null && combo != null && !combo.isEmpty();
}
/** Whether the action is a per-server one, for which {@link #activeServerOnly} matters. */
public boolean isServerScoped() {
return action != null && action.category() != HotkeyAction.Category.MISC;
}
public Hotkey copy() {
Hotkey h = new Hotkey(action, combo);
h.trigger = trigger;
h.activeServerOnly = activeServerOnly;
h.argument = argument;
h.enabled = enabled;
return h;
}
}

View File

@@ -0,0 +1,366 @@
package com.ts3client.hotkey;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
/**
* The catalogue of hotkey actions, reverse-engineered from the official TeamSpeak 3
* client (its hotkey dialog builds exactly this list, in this order).
*
* <p>Every entry keeps the identifiers TS3 itself uses — a numeric action type and a
* parameter keyword — so bindings stay recognisable and could be imported from, or
* exported to, the original client. {@link #supported()} tells whether this client
* can actually carry the action out; the rest are listed but greyed out.
*
* <p>{@link #advanced()} mirrors the dialog's "Show Advanced Actions" checkbox: TS3
* hides all but a handful of everyday actions until it is ticked.
*/
public enum HotkeyAction {
// ----- Server -----
CONNECT_CURRENT_TAB(Category.SERVER, 0x0010, "Server (current tab)",
"Connect to Server on Current Tab", false, Argument.BOOKMARK, true),
CONNECT_NEW_TAB(Category.SERVER, 0x0010, "Server (new tab)",
"Connect to Server in a New Tab", false, Argument.BOOKMARK, true),
DISCONNECT_CURRENT(Category.SERVER, 0x0020, "Current Server",
"Disconnect from Current Server", false, Argument.NONE, true),
DISCONNECT_ALL(Category.SERVER, 0x0020, "All Servers",
"Disconnect from All Servers", false, Argument.NONE, true),
MIC_ACTIVATE(Category.SERVER, 0x0030, "Activate",
"Activate Microphone (current tab)", true, Argument.NONE, true),
MIC_MUTE(Category.SERVER, 0x0030, "Mute",
"Mute Microphone", true, Argument.NONE, true),
MIC_UNMUTE(Category.SERVER, 0x0030, "Unmute",
"Unmute Microphone", true, Argument.NONE, true),
MIC_TOGGLE(Category.SERVER, 0x0030, "Toggle",
"Toggle Microphone Mute", false, Argument.NONE, true),
/**
* TS3's "local" mute silences capture without publishing a mute status, or playing
* a mute/unmute sound — unlike {@link #MIC_MUTE}.
*/
MIC_LOCAL_UNMUTE(Category.SERVER, 0x0030, "ActivateLocalMute",
"Disable Local Mic Mute", true, Argument.NONE, true),
MIC_LOCAL_MUTE(Category.SERVER, 0x0030, "DeactivateLocalMute",
"Enable Local Mic Mute", true, Argument.NONE, true),
MIC_LOCAL_TOGGLE(Category.SERVER, 0x0030, "ToggleLocalMute",
"Toggle Local Mic Mute", true, Argument.NONE, true),
SPEAKER_MUTE(Category.SERVER, 0x0040, "Mute",
"Mute Speaker", true, Argument.NONE, true),
SPEAKER_UNMUTE(Category.SERVER, 0x0040, "Unmute",
"Unmute Speaker", true, Argument.NONE, true),
SPEAKER_TOGGLE(Category.SERVER, 0x0040, "Toggle",
"Toggle Speaker Mute", false, Argument.NONE, true),
AWAY_SET(Category.SERVER, 0x0050, "SetAway",
"Set Away", true, Argument.NONE, true),
AWAY_ONLINE(Category.SERVER, 0x0050, "SetOnline",
"Set Online", true, Argument.NONE, true),
AWAY_TOGGLE(Category.SERVER, 0x0050, "Toggle",
"Toggle Away Status", false, Argument.NONE, true),
AWAY_TOGGLE_WITH_MESSAGE(Category.SERVER, 0x0050, "Toggle With Message",
"Toggle Away Status with Message", true, Argument.TEXT, true),
COMMANDER_ACTIVATE(Category.SERVER, 0x0100, "Activate",
"Activate Channel Commander", true, Argument.NONE, true),
COMMANDER_DEACTIVATE(Category.SERVER, 0x0100, "Deactivate",
"Deactivate Channel Commander", true, Argument.NONE, true),
COMMANDER_TOGGLE(Category.SERVER, 0x0100, "Toggle",
"Toggle Channel Commander", true, Argument.NONE, true),
// ----- Self -----
CAPTURE_PROFILE_ACTIVATE(Category.SELF, 0x0110, "Activate",
"Activate Capture Profile", true, Argument.PROFILE, false),
CAPTURE_PROFILE_DEACTIVATE(Category.SELF, 0x0110, "Deactivate",
"Deactivate Capture Profile", true, Argument.PROFILE, false),
CAPTURE_PROFILE_TOGGLE(Category.SELF, 0x0110, "Toggle",
"Toggle Capture Profile", true, Argument.PROFILE, false),
PLAYBACK_PROFILE_ACTIVATE(Category.SELF, 0x0120, "Activate",
"Activate Playback Profile", true, Argument.PROFILE, false),
PLAYBACK_PROFILE_DEACTIVATE(Category.SELF, 0x0120, "Deactivate",
"Deactivate Playback Profile", true, Argument.PROFILE, false),
PLAYBACK_PROFILE_TOGGLE(Category.SELF, 0x0120, "Toggle",
"Toggle Playback Profile", true, Argument.PROFILE, false),
HOTKEY_PROFILE_ACTIVATE(Category.SELF, 0x0130, "Activate",
"Activate Hotkey Profile", true, Argument.PROFILE, false),
HOTKEY_PROFILE_DEACTIVATE(Category.SELF, 0x0130, "Deactivate",
"Deactivate Hotkey Profile", true, Argument.PROFILE, false),
PTT_ACTIVATE(Category.SELF, 0x0140, "Activate",
"Activate Push-to-Talk", false, Argument.NONE, true, true),
PTT_DEACTIVATE(Category.SELF, 0x0140, "Deactivate",
"Deactivate Push-to-Talk", true, Argument.NONE, true),
PTT_TOGGLE(Category.SELF, 0x0140, "Toggle",
"Toggle Push-to-Talk", true, Argument.NONE, true),
CHANNEL_SWITCH(Category.SELF, 0x0150, "Strict Channel",
"Switch to Channel", true, Argument.CHANNEL, true),
CHANNEL_NEXT(Category.SELF, 0x0150, "Next Channel",
"Switch to Next Channel (Global)", true, Argument.NONE, false),
CHANNEL_PREVIOUS(Category.SELF, 0x0150, "Previous Channel",
"Switch to Previous Channel (Global)", true, Argument.NONE, false),
CHANNEL_LAST_VISITED(Category.SELF, 0x0150, "Last Visited Channel",
"Switch to Last Visited Channel", true, Argument.NONE, false),
CHANNEL_NEXT_FAMILY(Category.SELF, 0x0150, "Next Channel Family",
"Switch to Next Channel (Channel Family)", true, Argument.NONE, false),
CHANNEL_PREVIOUS_FAMILY(Category.SELF, 0x0150, "Previous Channel Family",
"Switch to Previous Channel (Channel Family)", true, Argument.NONE, false),
CHANNEL_NEXT_LEVEL(Category.SELF, 0x0150, "Next Channel Level",
"Switch to Next Channel (Same Level)", true, Argument.NONE, false),
CHANNEL_PREVIOUS_LEVEL(Category.SELF, 0x0150, "Previous Channel Level",
"Switch to Previous Channel (Same Level)", true, Argument.NONE, false),
SERVER_TAB_SELECT(Category.SELF, 0x0160, "Strict Server",
"Select Server Tab", true, Argument.TEXT, true),
SERVER_TAB_NEXT(Category.SELF, 0x0160, "Next Server",
"Select Next Server Tab", true, Argument.NONE, true),
SERVER_TAB_PREVIOUS(Category.SELF, 0x0160, "Previous Server",
"Select Previous Server Tab", true, Argument.NONE, true),
SOUNDPACK_ACTIVATE(Category.SELF, 0x0170, "Activate",
"Activate Soundpack", true, Argument.PROFILE, false),
SOUND_MUTE(Category.SELF, 0x0170, "Mute",
"Mute Sounds", true, Argument.NONE, true),
SOUND_UNMUTE(Category.SELF, 0x0170, "Unmute",
"Unmute Sounds", true, Argument.NONE, true),
SOUND_TOGGLE(Category.SELF, 0x0170, "Toggle",
"Toggle Sound Mute", true, Argument.NONE, true),
WHISPER_PUSH_ACTIVATE(Category.SELF, 0x0180, "ActivatePushToWhisper",
"Activate Push-To-Whisper", true, Argument.TEXT, false, true),
WHISPER_PUSH_DEACTIVATE(Category.SELF, 0x0180, "DeactivatePushToWhisper",
"Deactivate Push-To-Whisper", true, Argument.TEXT, false),
WHISPER_REPLY_PUSH_ACTIVATE(Category.SELF, 0x0180, "ActivateReplyPushToWhisper",
"Activate Push-To-Reply to a Whisper", true, Argument.NONE, false, true),
WHISPER_REPLY_PUSH_DEACTIVATE(Category.SELF, 0x0180, "DeactivateReplyPushToWhisper",
"Deactivate Push-To-Reply to a Whisper", true, Argument.NONE, false),
WHISPERLIST_ACTIVATE(Category.SELF, 0x0180, "Activate",
"Activate Whisperlist", true, Argument.TEXT, false),
WHISPERLIST_DEACTIVATE(Category.SELF, 0x0180, "Deactivate",
"Deactivate Whisperlist", true, Argument.TEXT, false),
WHISPER_REPLY_ACTIVATE(Category.SELF, 0x0180, "ActivateReply",
"Activate Reply to a Whisper", true, Argument.NONE, false),
WHISPER_REPLY_DEACTIVATE(Category.SELF, 0x0180, "DeactivateReply",
"Deactivate Reply to a Whisper", true, Argument.NONE, false),
WHISPER_BLOCK_ACTIVATE(Category.SELF, 0x0180, "ActivateBlock",
"Activate Block Incoming Whispers", true, Argument.NONE, false),
WHISPER_BLOCK_DEACTIVATE(Category.SELF, 0x0180, "DeactivateBlock",
"Deactivate Block Incoming Whispers", true, Argument.NONE, false),
WHISPER_BLOCK_TOGGLE(Category.SELF, 0x0180, "ToggleBlock",
"Toggle Block Incoming Whispers", true, Argument.NONE, false),
RECORDING_START(Category.SELF, 0x01c0, "Activate",
"Start Recording", true, Argument.NONE, false),
RECORDING_START_MULTITRACK(Category.SELF, 0x01c0, "ActivateMT",
"Start Multitrack Recording", true, Argument.NONE, false),
RECORDING_STOP(Category.SELF, 0x01c0, "Deactivate",
"Stop Recording", true, Argument.NONE, false),
VOLUME_INCREASE(Category.SELF, 0x01e0, "Increase",
"Increase Master Volume", true, Argument.NONE, true),
VOLUME_DECREASE(Category.SELF, 0x01e0, "Decrease",
"Decrease Master Volume", true, Argument.NONE, true),
PLUGIN_ACTIVATE(Category.SELF, 0x01f0, "Activate",
"Activate Plugin", true, Argument.TEXT, false),
PLUGIN_DEACTIVATE(Category.SELF, 0x01f0, "Deactivate",
"Deactivate Plugin", true, Argument.TEXT, false),
PLUGIN_TOGGLE(Category.SELF, 0x01f0, "Toggle",
"Toggle Plugin", true, Argument.TEXT, false),
PLUGIN_COMMAND(Category.SELF, 0x01f0, "Run",
"Run Plugin Command", true, Argument.TEXT, false),
PLUGIN_HOTKEY(Category.SELF, 0x01f0, "Hotkey",
"Plugin Hotkey", true, Argument.TEXT, false),
SERVER_GROUP_ASSIGN(Category.SELF, 0x0270, "AssignSG",
"Assign Server Group", true, Argument.TEXT, false),
SERVER_GROUP_REVOKE(Category.SELF, 0x0270, "RevokeSG",
"Revoke Server Group", true, Argument.TEXT, false),
SERVER_GROUP_TOGGLE(Category.SELF, 0x0270, "ToggleSG",
"Toggle Server Group", true, Argument.TEXT, false),
STYLESHEET_HELPER_ON(Category.SELF, 0x0240, "Activate",
"Stylesheet Helper (on)", true, Argument.NONE, false),
STYLESHEET_HELPER_OFF(Category.SELF, 0x0240, "Deactivate",
"Stylesheet Helper (off)", true, Argument.NONE, false),
SOUND_3D_ACTIVATE(Category.SELF, 0x0250, "Activate",
"Activate 3D Sound", true, Argument.NONE, false),
SOUND_3D_DEACTIVATE(Category.SELF, 0x0250, "Deactivate",
"Deactivate 3D Sound", true, Argument.NONE, false),
SOUND_3D_TOGGLE(Category.SELF, 0x0250, "Toggle",
"Toggle 3D Sound", true, Argument.NONE, false),
NICKNAME_CHANGE(Category.SELF, 0x01a0, "Rename",
"Change Nickname", true, Argument.TEXT, true),
// ----- Misc -----
TALK_POWER_GRANT_NEXT(Category.MISC, 0x01b0, "GrantNextTalkPower",
"Grant Next User Talk Power", true, Argument.NONE, false),
TALK_POWER_REQUEST(Category.MISC, 0x01b0, "RequestTalkPower",
"Request Talk Power", true, Argument.NONE, false),
TALK_POWER_REVOKE_ALL_GRANT_NEXT(Category.MISC, 0x01b0, "RevokeAllAndGrantNext",
"Revoke All And Grant Next User Talk Power", true, Argument.NONE, false),
FILEBROWSER(Category.MISC, 0x01d0, "Filebrowser",
"Open Filebrowser on Channel", true, Argument.NONE, true),
AUTOCONNECT_DISABLE(Category.MISC, 0x0200, "Autoconnect",
"Disable Autoconnect", true, Argument.NONE, false),
SKIN_RELOAD(Category.MISC, 0x0210, "SkinReload",
"Reload Skin", true, Argument.NONE, true),
BRING_TO_FRONT(Category.MISC, 0x0220, "Bring2Front",
"Bring Client to Front", true, Argument.NONE, true),
SEND_TO_BACK(Category.MISC, 0x0230, "Send2Back",
"Send Client to Background", true, Argument.NONE, true);
/** The three groups the TS3 hotkey dialog separates its list into. */
public enum Category {
SERVER("Server"),
SELF("Self"),
MISC("Misc");
private final String label;
Category(String label) {
this.label = label;
}
public String label() {
return label;
}
}
/**
* The group an action's type belongs to in TeamSpeak's hotkey tree, taken from the
* table its dialog builds. Types absent here are ungrouped: TS3 shows them as a
* single item directly under their category.
*/
private static final Map<Integer, String> GROUPS = Map.ofEntries(
Map.entry(0x0010, "Connect to Server"),
Map.entry(0x0020, "Disconnect from Server"),
Map.entry(0x0030, "Microphone"),
Map.entry(0x0040, "Speaker"),
Map.entry(0x0050, "Away Status"),
Map.entry(0x0100, "Channel Commander"),
Map.entry(0x0110, "Capture Profile"),
Map.entry(0x0120, "Playback Profile"),
Map.entry(0x0130, "Hotkey Profile"),
Map.entry(0x0140, "Push-to-Talk"),
Map.entry(0x0150, "Switch to Channel"),
Map.entry(0x0160, "Select Server Tab"),
Map.entry(0x0170, "Sounds"),
Map.entry(0x0180, "Whisper"),
Map.entry(0x01b0, "Talk Power"),
Map.entry(0x01c0, "Recording"),
Map.entry(0x01e0, "Master Volume"),
Map.entry(0x01f0, "Plugins"),
Map.entry(0x0250, "3D Sound"),
Map.entry(0x0270, "Permissions"));
/** What the action's free-text parameter means, when it takes one. */
public enum Argument {
NONE,
/** Label of a bookmark to connect to. */
BOOKMARK,
/** Channel path, "/"-separated. */
CHANNEL,
/** Name of a capture/playback/hotkey profile or sound pack. */
PROFILE,
/** Anything else: nickname, away message, tab number, plugin command. */
TEXT
}
private final Category category;
private final int type;
private final String parameter;
private final String label;
private final boolean advanced;
private final Argument argument;
private final boolean supported;
private final boolean momentary;
HotkeyAction(Category category, int type, String parameter, String label,
boolean advanced, Argument argument, boolean supported) {
this(category, type, parameter, label, advanced, argument, supported, false);
}
HotkeyAction(Category category, int type, String parameter, String label,
boolean advanced, Argument argument, boolean supported, boolean momentary) {
this.category = category;
this.type = type;
this.parameter = parameter;
this.label = label;
this.advanced = advanced;
this.argument = argument;
this.supported = supported;
this.momentary = momentary;
}
public Category category() {
return category;
}
/** TS3's numeric action type; actions sharing one differ only by {@link #parameter()}. */
public int type() {
return type;
}
/**
* The node this action hangs under in the hotkey tree ("Sounds", "Microphone", …),
* or an empty string for the actions TS3 lists on their own.
*/
public String group() {
return GROUPS.getOrDefault(type, "");
}
/** TS3's parameter keyword, e.g. {@code Toggle} or {@code ActivateReply}. */
public String parameter() {
return parameter;
}
/** The name shown in the hotkey list. */
public String label() {
return label;
}
/** Hidden until "Show advanced actions" is ticked, as in TS3. */
public boolean advanced() {
return advanced;
}
public Argument argument() {
return argument;
}
/** Whether this client implements the action; unsupported ones cannot be bound. */
public boolean supported() {
return supported;
}
/**
* Whether the action lasts only while the hotkey is held (push-to-talk and the
* push-to-whisper variants), in which case the trigger setting does not apply and
* the release is reported as well.
*/
public boolean momentary() {
return momentary;
}
public static List<HotkeyAction> of(Category category, boolean includeAdvanced) {
List<HotkeyAction> out = new ArrayList<>();
for (HotkeyAction a : values()) {
if (a.category == category && (includeAdvanced || !a.advanced)) out.add(a);
}
return out;
}
/** Looks an action up by its enum name, falling back to {@code null}. */
public static HotkeyAction byName(String name) {
for (HotkeyAction a : values()) {
if (a.name().equals(name)) return a;
}
return null;
}
}

View File

@@ -0,0 +1,94 @@
package com.ts3client.hotkey;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Set;
import java.util.function.Function;
/**
* A key combination: one key or button, or several that have to be held together.
* There is no distinguished modifier — as in TS3, any key can take that role, and the
* combination fires once every one of its keys is down.
*/
public final class HotkeyCombo {
/** Recording order, which is also the order the combination is displayed in. */
private final List<HotkeyKey> keys;
public HotkeyCombo(Collection<HotkeyKey> keys) {
List<HotkeyKey> copy = new ArrayList<>();
for (HotkeyKey k : keys) {
if (k != null && !copy.contains(k)) copy.add(k);
}
this.keys = Collections.unmodifiableList(copy);
}
public List<HotkeyKey> keys() {
return keys;
}
public boolean isEmpty() {
return keys.isEmpty();
}
public boolean contains(HotkeyKey key) {
return keys.contains(key);
}
/** Whether every key of the combination is currently held down. */
public boolean isSatisfiedBy(Set<HotkeyKey> pressed) {
return !keys.isEmpty() && pressed.containsAll(keys);
}
/** Serialised form, e.g. {@code Keyboard:37+Keyboard:45}. */
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
for (HotkeyKey k : keys) {
if (sb.length() > 0) sb.append('+');
sb.append(k);
}
return sb.toString();
}
public static HotkeyCombo parse(String text) {
List<HotkeyKey> keys = new ArrayList<>();
if (text != null && !text.isBlank()) {
for (String part : text.split("\\+")) {
HotkeyKey k = HotkeyKey.parse(part);
if (k != null) keys.add(k);
}
}
return new HotkeyCombo(keys);
}
/**
* Renders the combination for the user, e.g. {@code Ctrl+Mouse 4}.
*
* @param namer resolves a key to its name on this system, typically from the
* keyboard layout; may return {@code null} to fall back
*/
public String display(Function<HotkeyKey, String> namer) {
if (keys.isEmpty()) return "No hotkey assigned";
StringBuilder sb = new StringBuilder();
for (HotkeyKey k : keys) {
String name = namer == null ? null : namer.apply(k);
if (name == null || name.isBlank()) name = k.fallbackName();
if (sb.length() > 0) sb.append(" + ");
sb.append(name);
}
return sb.toString();
}
@Override
public boolean equals(Object o) {
return o instanceof HotkeyCombo other && keys.equals(other.keys);
}
@Override
public int hashCode() {
return keys.hashCode();
}
}

View File

@@ -0,0 +1,154 @@
package com.ts3client.hotkey;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;
/**
* Turns the raw stream of global key events into hotkey activations: it tracks which
* keys are down, decides when a binding's combination becomes (or stops being)
* satisfied, and reports that on the configured edge.
*
* <p>While a {@link Recorder} is installed the engine records combinations instead of
* firing them, which is how the hotkey dialog captures a new binding without the old
* ones going off.
*/
public final class HotkeyEngine implements GlobalInputHook.Listener {
/** Receives activations; called on the input hook's thread. */
public interface Handler {
/**
* @param hotkey the binding that fired
* @param active {@code true} on activation; a {@code false} call follows on release
* for {@link HotkeyAction#momentary()} actions only
*/
void onHotkey(Hotkey hotkey, boolean active);
}
/** Collects a combination as the user presses it, for the "set hotkey" button. */
public interface Recorder {
/** The keys held so far, in press order; called on every change. */
void onRecording(HotkeyCombo combo);
/** The user let go of everything: this is the final combination. */
void onRecorded(HotkeyCombo combo);
}
private final Hotkeys hotkeys;
private final Handler handler;
private final Set<HotkeyKey> pressed = new LinkedHashSet<>();
/** Bindings whose combination is currently held, so a release can end them. */
private final Set<Hotkey> active = new HashSet<>();
private volatile Recorder recorder;
private final Set<HotkeyKey> recording = new LinkedHashSet<>();
public HotkeyEngine(Hotkeys hotkeys, Handler handler) {
this.hotkeys = hotkeys;
this.handler = handler;
}
/**
* Diverts every event into {@code recorder} until {@link #stopRecording()}; the
* bindings themselves stay silent meanwhile.
*/
public void record(Recorder recorder) {
synchronized (this) {
recording.clear();
releaseActive();
pressed.clear();
}
this.recorder = recorder;
}
public void stopRecording() {
recorder = null;
synchronized (this) {
recording.clear();
pressed.clear();
}
}
@Override
public void onInput(HotkeyKey key, boolean down) {
Recorder rec = recorder;
if (rec != null) {
recordKey(rec, key, down);
return;
}
List<Runnable> fired = new ArrayList<>();
synchronized (this) {
if (down) {
if (!pressed.add(key)) return;
} else if (!pressed.remove(key)) {
return;
}
synchronized (hotkeys.all()) {
for (Hotkey h : hotkeys.all()) {
if (!h.enabled || !h.isValid() || !h.action.supported()) continue;
if (!h.combo.contains(key)) continue;
boolean satisfied = h.combo.isSatisfiedBy(pressed);
if (satisfied && active.add(h)) {
if (h.action.momentary() || h.trigger == Hotkey.Trigger.KEY_DOWN) {
fired.add(() -> handler.onHotkey(h, true));
}
} else if (!satisfied && active.remove(h)) {
if (h.action.momentary()) {
fired.add(() -> handler.onHotkey(h, false));
} else if (h.trigger == Hotkey.Trigger.KEY_UP) {
fired.add(() -> handler.onHotkey(h, true));
}
}
}
}
}
for (Runnable r : fired) r.run();
}
private void recordKey(Recorder rec, HotkeyKey key, boolean down) {
HotkeyCombo combo;
boolean finished;
synchronized (this) {
if (down) {
recording.add(key);
pressed.add(key);
} else {
pressed.remove(key);
}
combo = new HotkeyCombo(recording);
finished = !down && pressed.isEmpty() && !recording.isEmpty();
if (finished) recording.clear();
}
if (finished) {
rec.onRecorded(combo);
} else {
rec.onRecording(combo);
}
}
/** Ends every held binding, so nothing stays stuck when the hook stops. */
public void releaseAll() {
List<Runnable> fired = new ArrayList<>();
synchronized (this) {
for (Hotkey h : active) {
if (h.action.momentary()) fired.add(() -> handler.onHotkey(h, false));
}
active.clear();
pressed.clear();
}
for (Runnable r : fired) r.run();
}
private void releaseActive() {
for (Hotkey h : new ArrayList<>(active)) {
if (h.action.momentary()) handler.onHotkey(h, false);
}
active.clear();
}
}

View File

@@ -0,0 +1,88 @@
package com.ts3client.hotkey;
/**
* One physical key or mouse button. Keys are identified by their position rather than by
* the character they produce, so a binding is independent of the keyboard layout in force
* when it was recorded.
*
* <p>The keyboard numbering is whatever the platform's own is — X keycodes (Linux evdev
* code + 8) on X11, set-1 scan codes with the {@code E0}/{@code E1} escape in the high
* byte on Windows — because that is what its input API reports and its key-naming call
* expects. Bindings are therefore per-machine, which is also how TeamSpeak stores them.
* Mouse buttons are unified on the X numbering everywhere, side buttons included.
*
* @param device which input device the code belongs to
* @param code the platform's key code for {@link Device#KEYBOARD}, an X-style button
* number for {@link Device#MOUSE}
*/
public record HotkeyKey(HotkeyKey.Device device, int code) {
public enum Device {
KEYBOARD("Keyboard"),
MOUSE("Mouse");
private final String prefix;
Device(String prefix) {
this.prefix = prefix;
}
String prefix() {
return prefix;
}
}
public static HotkeyKey keyboard(int keycode) {
return new HotkeyKey(Device.KEYBOARD, keycode);
}
public static HotkeyKey mouse(int button) {
return new HotkeyKey(Device.MOUSE, button);
}
/** Serialised form, matching TS3's {@code Keyboard:9} / {@code Mouse:9} notation. */
@Override
public String toString() {
return device.prefix() + ":" + code;
}
public static HotkeyKey parse(String text) {
if (text == null) return null;
int colon = text.indexOf(':');
if (colon <= 0) return null;
String prefix = text.substring(0, colon).trim();
int code;
try {
code = Integer.parseInt(text.substring(colon + 1).trim());
} catch (NumberFormatException e) {
return null;
}
for (Device d : Device.values()) {
if (d.prefix().equalsIgnoreCase(prefix)) return new HotkeyKey(d, code);
}
return null;
}
/**
* A readable name that needs no help from the platform: mouse buttons carry the
* numbering users know them by, keys fall back to their raw code.
*/
public String fallbackName() {
if (device == Device.MOUSE) {
return switch (code) {
case 1 -> "Mouse Left";
case 2 -> "Mouse Middle";
case 3 -> "Mouse Right";
case 4 -> "Wheel Up";
case 5 -> "Wheel Down";
case 6 -> "Wheel Left";
case 7 -> "Wheel Right";
// X hands the side buttons out as 8 and 9; everyone else calls them 4 and 5.
case 8 -> "Mouse 4";
case 9 -> "Mouse 5";
default -> "Mouse " + code;
};
}
return "Key " + code;
}
}

View File

@@ -0,0 +1,109 @@
package com.ts3client.hotkey;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Properties;
/**
* The user's hotkey bindings, stored alongside the settings file.
* Frontend-agnostic: no UI dependencies.
*/
public final class Hotkeys {
private static final File DIR = new File(System.getProperty("user.home"), ".ts3jclient");
private static final File FILE = new File(DIR, "hotkeys.properties");
private final List<Hotkey> entries = Collections.synchronizedList(new ArrayList<>());
/** Live view; iterate under {@code synchronized (all())} when the engine may be running. */
public List<Hotkey> all() {
return entries;
}
public void add(Hotkey hotkey) {
entries.add(hotkey);
}
public void set(int index, Hotkey hotkey) {
if (index >= 0 && index < entries.size()) entries.set(index, hotkey);
}
public void remove(int index) {
if (index >= 0 && index < entries.size()) entries.remove(index);
}
/** Replaces every binding at once, e.g. when the options dialog is applied. */
public void replaceAll(List<Hotkey> hotkeys) {
synchronized (entries) {
entries.clear();
for (Hotkey h : hotkeys) {
if (h != null && h.isValid()) entries.add(h);
}
}
}
public static Hotkeys load() {
Hotkeys h = new Hotkeys();
if (!FILE.isFile()) return h;
Properties p = new Properties();
try (FileInputStream in = new FileInputStream(FILE)) {
p.load(in);
} catch (Exception e) {
return h;
}
int count = parseInt(p.getProperty("count"), 0);
for (int i = 0; i < count; i++) {
String prefix = "hotkey." + i + ".";
HotkeyAction action = HotkeyAction.byName(p.getProperty(prefix + "action", ""));
if (action == null) continue;
Hotkey entry = new Hotkey(action, HotkeyCombo.parse(p.getProperty(prefix + "keys", "")));
entry.trigger = "KEY_UP".equals(p.getProperty(prefix + "trigger"))
? Hotkey.Trigger.KEY_UP : Hotkey.Trigger.KEY_DOWN;
entry.activeServerOnly = !"false".equals(p.getProperty(prefix + "activeServerOnly"));
entry.argument = p.getProperty(prefix + "argument", "");
entry.enabled = !"false".equals(p.getProperty(prefix + "enabled"));
if (entry.isValid()) h.entries.add(entry);
}
return h;
}
public void save() {
Properties p = new Properties();
synchronized (entries) {
p.setProperty("count", Integer.toString(entries.size()));
for (int i = 0; i < entries.size(); i++) {
Hotkey e = entries.get(i);
String prefix = "hotkey." + i + ".";
p.setProperty(prefix + "action", e.action.name());
p.setProperty(prefix + "keys", e.combo.toString());
p.setProperty(prefix + "trigger", e.trigger.name());
p.setProperty(prefix + "activeServerOnly", Boolean.toString(e.activeServerOnly));
p.setProperty(prefix + "argument", e.argument == null ? "" : e.argument);
p.setProperty(prefix + "enabled", Boolean.toString(e.enabled));
}
}
try {
if (!DIR.isDirectory()) {
//noinspection ResultOfMethodCallIgnored
DIR.mkdirs();
}
try (FileOutputStream out = new FileOutputStream(FILE)) {
p.store(out, "TS3J client hotkeys");
}
} catch (Exception ignored) {
}
}
private static int parseInt(String v, int def) {
if (v == null) return def;
try {
return Integer.parseInt(v.trim());
} catch (NumberFormatException e) {
return def;
}
}
}

View File

@@ -21,6 +21,10 @@ public final class ClientEntry {
public boolean talking;
public boolean inputMuted; // microphone muted (client_input_muted)
public boolean outputMuted; // speakers muted / deafened (client_output_muted)
/** False while the capture device is unavailable: another tab holds the microphone. */
public boolean inputHardware = true;
/** False while the playback device is unavailable. */
public boolean outputHardware = true;
public boolean away;
/** The message published with the away state, empty when there is none. */
public String awayMessage = "";

View File

@@ -153,6 +153,22 @@ public final class TeamspeakConnection implements TS3Listener {
} catch (Exception e) {
ui.onError("Microphone unavailable: " + rootMessage(e));
}
pushInputHardware();
}
/**
* Reports whether the capture device is actually available on this connection, so
* other clients see "Microphone Disabled" rather than plain silence while another
* tab holds it. {@code clientinit} always claims hardware input, so this is what
* corrects that once tabs start sharing one microphone.
*/
private void pushInputHardware() {
boolean hardware = microphoneActive;
updateSelf(self -> self.inputHardware = hardware);
if (client == null || !connected) return;
selfUpdate(cmd ->
cmd.add(new CommandSingleParameter("client_input_hardware", hardware ? "1" : "0")),
"Microphone hardware status update failed");
}
// ---- connection lifecycle ----
@@ -226,6 +242,7 @@ public final class TeamspeakConnection implements TS3Listener {
ui.onError("Network error: " + rootMessage(t));
});
applyClientVersion();
applyDefaultChannel(channel, channelPassword);
ui.onStatus("Connecting to " + address + ":" + port + "");
@@ -256,7 +273,7 @@ public final class TeamspeakConnection implements TS3Listener {
applyMicrophoneState();
} catch (Exception e) {
connected = false;
ui.onError("Connection failed: " + rootMessage(e));
ui.onError("Connection failed: " + rootMessage(e) + versionHint(e));
safeCleanup();
// Report the failed attempt as a disconnect too, so the UI leaves the
// "connecting" state and the tab becomes reusable.
@@ -265,6 +282,38 @@ public final class TeamspeakConnection implements TS3Listener {
}
}
/**
* "client is modified" is what a server answers when it does not accept the reported
* version: too old for its minimum, not a version it knows for that platform, or a
* signature that does not belong to the two. Easy to run into after editing them by
* hand, and the bare message does not point anywhere.
*/
private String versionHint(Exception e) {
if (!settings.customVersion) return "";
String message = rootMessage(e);
return message != null && message.contains("client is modified")
? " — the server did not accept the custom client version; pick a newer one"
+ " of the listed versions in Options."
: "";
}
/**
* Reports the version and operating system the user configured, which is what
* other clients see in the info panel. Each field falls back to the library's
* built-in one when left empty; the signature has to match the version and
* platform it was issued for, or a server may refuse the connection.
*/
private void applyClientVersion() {
if (!settings.customVersion) return;
setIfPresent("client.version_string", settings.clientVersion);
setIfPresent("client.version_platform", settings.clientPlatform);
setIfPresent("client.version_sign", settings.clientVersionSign);
}
private void setIfPresent(String option, String value) {
if (value != null && !value.isBlank()) client.setOption(option, value.trim());
}
/**
* Asks the server, as part of {@code clientinit}, to place us in a specific
* channel right away instead of the default one. This is the path a real client
@@ -370,6 +419,8 @@ public final class TeamspeakConnection implements TS3Listener {
e.talkPower = cl.getTalkPower();
e.inputMuted = cl.isInputMuted();
e.outputMuted = cl.isOutputMuted();
e.inputHardware = cl.isInputHardware();
e.outputHardware = cl.isOutputHardware();
e.away = cl.isAway();
e.awayMessage = orEmpty(cl.get("client_away_message"));
e.uniqueId = cl.getUniqueIdentifier();
@@ -472,6 +523,14 @@ public final class TeamspeakConnection implements TS3Listener {
pushSelfFlags();
}
/**
* TS3's "Local Mic Mute": silences capture without touching the {@code client_input_muted}
* status the server (and other clients) see, and without a mute/unmute sound.
*/
public void setMicLocalMuted(boolean muted) {
if (microphone != null) microphone.setLocalMuted(muted);
}
public void setDeafened(boolean deaf) {
// Announce muting while we can still be heard, and unmuting once we can again.
if (deaf) sound(SoundEvent.SOUND_PLAYBACK_MUTED);
@@ -491,15 +550,14 @@ public final class TeamspeakConnection implements TS3Listener {
self.outputMuted = deaf;
});
// Best-effort: publish input/output muted flags to the server so others see them.
// Publish the input/output muted flags to the server so others see them. These
// are runtime status, not editable client-database properties, so they go out
// through clientupdate (like nickname/away), not clientedit.
if (client == null || !connected) return;
try {
java.util.Map<String, String> props = new java.util.HashMap<>();
props.put("client_input_muted", microphone.isMuted() ? "1" : "0");
props.put("client_output_muted", playback.isDeafened() ? "1" : "0");
client.editClient(selfClientId, props);
} catch (Exception ignored) {
}
selfUpdate(cmd -> {
cmd.add(new CommandSingleParameter("client_input_muted", micMuted ? "1" : "0"));
cmd.add(new CommandSingleParameter("client_output_muted", deaf ? "1" : "0"));
}, "Mute status update failed");
}
public void joinChannel(int channelId, String password) {
@@ -725,6 +783,8 @@ public final class TeamspeakConnection implements TS3Listener {
c.talkPower = e.getClientTalkPower();
c.inputMuted = e.isClientInputMuted();
c.outputMuted = e.isClientOutputMuted();
c.inputHardware = e.isClientUsingHardwareInput();
c.outputHardware = e.isClientUsingHardwareOutput();
c.away = e.isClientAway();
c.awayMessage = orEmpty(e.get("client_away_message"));
c.uniqueId = orEmpty(e.getUniqueClientIdentifier());
@@ -893,6 +953,8 @@ public final class TeamspeakConnection implements TS3Listener {
if (renamed) c.nickname = e.get("client_nickname");
if (has(e, "client_input_muted")) c.inputMuted = e.getBoolean("client_input_muted");
if (has(e, "client_output_muted")) c.outputMuted = e.getBoolean("client_output_muted");
if (has(e, "client_input_hardware")) c.inputHardware = e.getBoolean("client_input_hardware");
if (has(e, "client_output_hardware")) c.outputHardware = e.getBoolean("client_output_hardware");
if (has(e, "client_away")) {
c.away = e.getBoolean("client_away");
// Both fields travel together, so an absent message here means "no message"

View File

@@ -24,6 +24,7 @@ public final class SoundNotifier {
private volatile SoundPlayer player;
private volatile SoundPack pack;
private volatile List<SoundPack> available = List.of();
private volatile boolean soundsMuted;
public SoundNotifier(Settings settings) {
this.settings = settings;
@@ -55,6 +56,19 @@ public final class SoundNotifier {
settings.soundPack = pack == null ? "" : pack.id();
}
/**
* Silences every notification sound until turned back on — TS3's "Mute Sounds"
* hotkey action. Unlike deafening, this is not tied to a server and even important
* actions stay quiet.
*/
public void setMuted(boolean muted) {
this.soundsMuted = muted;
}
public boolean isMuted() {
return soundsMuted;
}
/**
* Plays the sound for an action.
*
@@ -70,6 +84,7 @@ public final class SoundNotifier {
* {@code clientname} or {@code channelname}
*/
public void fire(SoundEvent event, boolean muted, Map<String, String> variables) {
if (soundsMuted) return;
NotificationSettings notifications = settings.notifications;
if (!notifications.isEnabled(event)) return;
if (muted && !notifications.isImportant(event)) return;

View File

@@ -0,0 +1,58 @@
package com.ts3client.audio.vad;
import org.junit.jupiter.api.Test;
import java.util.Random;
import static org.junit.jupiter.api.Assertions.assertEquals;
class RealFftTest {
/**
* The rnn_vad transform length is 480 = 2^5 * 3 * 5, so the mixed-radix path has to be
* right for factors other than powers of two.
*/
@Test
void matchesDirectDftForMixedRadixLengths() {
for (int n : new int[]{480, 240, 512, 64, 60, 210, 100}) {
RealFft fft = new RealFft(n);
Random random = new Random(n);
float[] in = new float[n];
float[] out = new float[n];
for (int i = 0; i < n; i++) {
in[i] = (float) random.nextGaussian();
}
fft.forward(in, out);
for (int k = 0; k <= n / 2; k++) {
double re = 0;
double im = 0;
for (int t = 0; t < n; t++) {
double angle = -2 * Math.PI * k * t / n;
re += in[t] * Math.cos(angle);
im += in[t] * Math.sin(angle);
}
double gotRe;
double gotIm;
if (k == 0) {
gotRe = out[0];
gotIm = 0;
} else if (k == n / 2) {
gotRe = out[1]; // Packed layout keeps Nyquist in slot 1.
gotIm = 0;
} else {
gotRe = out[2 * k];
gotIm = out[2 * k + 1];
}
assertEquals(re, gotRe, 1e-2, "n=" + n + " bin=" + k + " real");
assertEquals(im, gotIm, 1e-2, "n=" + n + " bin=" + k + " imag");
}
}
}
@Test
void rejectsOddLengths() {
org.junit.jupiter.api.Assertions.assertThrows(
IllegalArgumentException.class, () -> new RealFft(481));
}
}

View File

@@ -0,0 +1,129 @@
package com.ts3client.audio.vad;
import org.junit.jupiter.api.Test;
import java.util.Random;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* Behavioural checks on the {@code rnn_vad} port. The network weights are the ones the TS3
* client links, so these also pin the detector's agreement with it.
*/
class RnnVadTest {
private static final int FRAMES = 60;
@Test
void silenceScoresZero() {
float[] x = new float[RnnVad.FRAME_SIZE * FRAMES];
double[] probabilities = run(x);
for (double p : probabilities) {
assertEquals(0.0, p, 0.0, "silence must not score above zero");
}
}
@Test
void voicedSpeechLikeSignalScoresHigh() {
float[] x = synth(FRAMES, t -> {
double sum = 0;
for (int h = 1; h <= 30 && 120 * h < 8000; h++) {
double f = 120 * h;
double gain = formant(f, 700, 90) + formant(f, 1220, 110) + formant(f, 2600, 160);
sum += gain * Math.sin(2 * Math.PI * f * t + h);
}
// A slow tremolo keeps it from looking like a stationary tone.
return 0.25 * (0.6 + 0.4 * Math.sin(2 * Math.PI * 5 * t)) * sum / 6.0;
});
// Skip the first frames: the pitch buffer and GRU need history.
assertTrue(meanFrom(run(x), 20) > 0.8, "a voiced signal should score high");
}
@Test
void quietPinkNoiseScoresLow() {
Random random = new Random(3);
float[] x = new float[RnnVad.FRAME_SIZE * FRAMES];
double b0 = 0;
double b1 = 0;
double b2 = 0;
for (int i = 0; i < x.length; i++) {
double w = random.nextGaussian();
b0 = 0.99765 * b0 + w * 0.0990460;
b1 = 0.96300 * b1 + w * 0.2965164;
b2 = 0.57000 * b2 + w * 1.0526913;
// About -45 dBFS: representative of room tone rather than loud hiss.
x[i] = (float) ((b0 + b1 + b2 + w * 0.1848) * 0.0011);
}
assertTrue(meanFrom(run(x), 20) < 0.2, "quiet room tone should not read as speech");
}
@Test
void estimatesPitchPeriodExactly() {
// The pitch feature is fed to the network as 0.01 * (period48k - 300).
for (double f0 : new double[]{80, 100, 120, 150, 200, 250, 300, 400}) {
RnnVad vad = new RnnVad();
float[] x = synth(FRAMES, t -> {
double sum = 0;
for (int h = 1; h <= 40 && f0 * h < 11000; h++) {
sum += Math.sin(2 * Math.PI * f0 * h * t) / h;
}
return 0.2 * sum;
});
for (int f = 0; f < FRAMES; f++) {
vad.process(x, f * RnnVad.FRAME_SIZE);
}
assertEquals(48000 / f0, vad.pitchPeriod48kHz(), 1e-9,
"pitch period for f0=" + f0);
}
}
@Test
void resetClearsState() {
float[] x = synth(FRAMES, t -> 0.3 * Math.sin(2 * Math.PI * 200 * t));
RnnVad vad = new RnnVad();
for (int f = 0; f < FRAMES; f++) {
vad.process(x, f * RnnVad.FRAME_SIZE);
}
assertTrue(vad.probability() > 0.0);
vad.reset();
assertEquals(0.0, vad.probability(), 0.0);
}
// ---- helpers ----
private interface Signal {
double at(double t);
}
private static double formant(double f, double centre, double bandwidth) {
double d = (f - centre) / bandwidth;
return 1.0 / (1.0 + d * d);
}
private static float[] synth(int frames, Signal signal) {
float[] x = new float[RnnVad.FRAME_SIZE * frames];
for (int i = 0; i < x.length; i++) {
x[i] = (float) signal.at((double) i / RnnVad.SAMPLE_RATE);
}
return x;
}
private static double[] run(float[] x) {
RnnVad vad = new RnnVad();
int frames = x.length / RnnVad.FRAME_SIZE;
double[] probabilities = new double[frames];
for (int i = 0; i < frames; i++) {
probabilities[i] = vad.process(x, i * RnnVad.FRAME_SIZE);
}
return probabilities;
}
private static double meanFrom(double[] values, int start) {
double sum = 0;
for (int i = start; i < values.length; i++) {
sum += values[i];
}
return sum / (values.length - start);
}
}

View File

@@ -0,0 +1,120 @@
package com.ts3client.hotkey;
import org.junit.jupiter.api.Test;
import java.util.ArrayList;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
class HotkeyEngineTest {
private static final HotkeyKey CTRL = HotkeyKey.keyboard(37);
private static final HotkeyKey K = HotkeyKey.keyboard(45);
private static final HotkeyKey MOUSE4 = HotkeyKey.mouse(8);
private final List<String> fired = new ArrayList<>();
private final Hotkeys hotkeys = new Hotkeys();
private final HotkeyEngine engine = new HotkeyEngine(hotkeys,
(hotkey, active) -> fired.add(hotkey.action + (active ? ":on" : ":off")));
private Hotkey bind(HotkeyAction action, Hotkey.Trigger trigger, HotkeyKey... keys) {
Hotkey h = new Hotkey(action, new HotkeyCombo(List.of(keys)));
h.trigger = trigger;
hotkeys.add(h);
return h;
}
@Test
void firesOnKeyDownOnlyWhenEveryKeyIsHeld() {
bind(HotkeyAction.MIC_TOGGLE, Hotkey.Trigger.KEY_DOWN, CTRL, K);
engine.onInput(K, true);
assertTrue(fired.isEmpty(), "one key of the combination is not enough");
engine.onInput(CTRL, true);
assertEquals(List.of("MIC_TOGGLE:on"), fired);
// Releasing and re-pressing arms it again, but only once per completion.
engine.onInput(CTRL, false);
engine.onInput(CTRL, true);
assertEquals(List.of("MIC_TOGGLE:on", "MIC_TOGGLE:on"), fired);
}
@Test
void keyUpTriggerFiresOnRelease() {
bind(HotkeyAction.SPEAKER_TOGGLE, Hotkey.Trigger.KEY_UP, MOUSE4);
engine.onInput(MOUSE4, true);
assertTrue(fired.isEmpty());
engine.onInput(MOUSE4, false);
assertEquals(List.of("SPEAKER_TOGGLE:on"), fired);
}
@Test
void momentaryActionReportsPressAndRelease() {
bind(HotkeyAction.PTT_ACTIVATE, Hotkey.Trigger.KEY_UP, MOUSE4);
engine.onInput(MOUSE4, true);
engine.onInput(MOUSE4, false);
// The trigger setting does not apply: push-to-talk always spans the hold.
assertEquals(List.of("PTT_ACTIVATE:on", "PTT_ACTIVATE:off"), fired);
}
@Test
void disabledAndUnsupportedBindingsStaySilent() {
bind(HotkeyAction.MIC_TOGGLE, Hotkey.Trigger.KEY_DOWN, K).enabled = false;
bind(HotkeyAction.RECORDING_START, Hotkey.Trigger.KEY_DOWN, MOUSE4);
engine.onInput(K, true);
engine.onInput(MOUSE4, true);
assertTrue(fired.isEmpty());
}
@Test
void recordingCapturesTheCombinationInsteadOfFiring() {
bind(HotkeyAction.MIC_TOGGLE, Hotkey.Trigger.KEY_DOWN, K);
List<HotkeyCombo> recorded = new ArrayList<>();
engine.record(new HotkeyEngine.Recorder() {
@Override
public void onRecording(HotkeyCombo combo) {
}
@Override
public void onRecorded(HotkeyCombo combo) {
recorded.add(combo);
}
});
engine.onInput(CTRL, true);
engine.onInput(K, true);
engine.onInput(K, false);
engine.onInput(CTRL, false);
assertTrue(fired.isEmpty(), "bindings must not fire while recording");
assertEquals(1, recorded.size());
assertEquals(List.of(CTRL, K), recorded.get(0).keys());
}
@Test
void actionsCarryTeamSpeaksTreeGrouping() {
assertEquals("Sounds", HotkeyAction.SOUNDPACK_ACTIVATE.group());
assertEquals("Microphone", HotkeyAction.MIC_TOGGLE.group());
// TS3 leaves a handful of actions ungrouped, listed straight under their category.
assertEquals("", HotkeyAction.NICKNAME_CHANGE.group());
assertEquals("", HotkeyAction.BRING_TO_FRONT.group());
Hotkey pack = new Hotkey(HotkeyAction.SOUNDPACK_ACTIVATE, new HotkeyCombo(List.of(K)));
pack.argument = "Default Sound Pack (Male)";
assertEquals("Sounds / Activate Soundpack / Default Sound Pack (Male)", pack.path());
assertEquals("Bring Client to Front",
new Hotkey(HotkeyAction.BRING_TO_FRONT, new HotkeyCombo(List.of(K))).path());
}
@Test
void combinationsSurviveARoundTripThroughText() {
HotkeyCombo combo = new HotkeyCombo(List.of(CTRL, MOUSE4));
assertEquals("Keyboard:37+Mouse:8", combo.toString());
assertEquals(combo, HotkeyCombo.parse(combo.toString()));
}
}

View File

@@ -2,9 +2,12 @@ package com.ts3client.audio.desktop;
import com.github.manevolent.ts3j.enums.CodecType;
import com.ts3client.audio.AudioEnhancer;
import com.ts3client.audio.AudioFrameListener;
import com.ts3client.audio.InputLevel;
import com.ts3client.audio.OpusParameters;
import com.ts3client.audio.SpeechDetector;
import com.ts3client.audio.SpeechProbabilityDetector;
import com.ts3client.audio.VoiceInput;
import com.ts3client.audio.vad.RnnSpeechDetector;
import com.ts3client.config.Settings;
import java.util.concurrent.ConcurrentLinkedQueue;
@@ -28,10 +31,27 @@ import java.util.function.Consumer;
*/
public final class DesktopVoiceInput implements VoiceInput {
private static final int HANGOVER_FRAMES = 15; // ~300 ms of tail after level drops
/**
* How long the gate stays open after the last active frame. TS3 counts 64 of its
* 10&nbsp;ms preprocessor frames before closing, so the tail of a word is never clipped.
*/
private static final int HANGOVER_MS = 640;
/**
* Audio replayed when the gate opens, so a word's onset is not swallowed by the frame
* that detected it. TS3 keeps up to three 10&nbsp;ms buffers ({@code vad_extrabuffersize}
* defaults to 2, plus one).
*/
private static final int PREROLL_MS = 30;
private static final int HANGOVER_FRAMES =
Math.max(1, HANGOVER_MS * AudioDevices.SAMPLE_RATE / 1000 / AudioDevices.FRAME_SIZE);
private static final int PREROLL_FRAMES =
Math.max(1, PREROLL_MS * AudioDevices.SAMPLE_RATE / 1000 / AudioDevices.FRAME_SIZE);
private final ConcurrentLinkedQueue<byte[]> queue = new ConcurrentLinkedQueue<>();
private final AtomicBoolean muted = new AtomicBoolean(false);
private final AtomicBoolean localMuted = new AtomicBoolean(false);
private final AtomicBoolean transmitting = new AtomicBoolean(false);
private final AtomicBoolean pttDown = new AtomicBoolean(false);
private final AtomicBoolean running = new AtomicBoolean(false);
@@ -44,12 +64,14 @@ public final class DesktopVoiceInput implements VoiceInput {
private volatile double inputGain;
private volatile CodecType codec = CodecType.OPUS_VOICE;
private final SpeechDetector speechDetector = new SpeechDetector(AudioDevices.SAMPLE_RATE);
private final SpeechProbabilityDetector speechDetector =
new RnnSpeechDetector(AudioDevices.SAMPLE_RATE);
private final AudioEnhancer enhancer = new AudioEnhancer(AudioDevices.SAMPLE_RATE);
private volatile Consumer<Double> levelListener; // input level in dBFS
private volatile Consumer<Double> levelListener; // input level, InputLevel scale
private volatile Consumer<Boolean> talkListener; // local talk-state changes
private volatile Runnable mutedTalkListener; // speech detected while muted
private volatile AudioFrameListener monitorListener; // local monitoring of what is sent
private boolean mutedTalking;
private final String deviceName;
@@ -68,6 +90,11 @@ public final class DesktopVoiceInput implements VoiceInput {
private int hangover;
private boolean lastTransmitting;
// Ring of recent frames captured while the gate was shut.
private final float[][] preroll = new float[PREROLL_FRAMES][];
private int prerollTail;
private int prerollCount;
public DesktopVoiceInput(Settings settings) {
this.deviceName = settings.inputDevice;
this.mode = settings.inputMode;
@@ -195,6 +222,11 @@ public final class DesktopVoiceInput implements VoiceInput {
this.talkListener = l;
}
@Override
public void setMonitorListener(AudioFrameListener l) {
this.monitorListener = l;
}
@Override
public void setMutedTalkListener(Runnable l) {
this.mutedTalkListener = l;
@@ -208,6 +240,16 @@ public final class DesktopVoiceInput implements VoiceInput {
this.muted.set(m);
}
@Override
public void setLocalMuted(boolean m) {
this.localMuted.set(m);
}
@Override
public boolean isLocalMuted() {
return localMuted.get();
}
// ---- lifecycle ----
public synchronized void start() {
@@ -305,36 +347,76 @@ public final class DesktopVoiceInput implements VoiceInput {
// stream bypasses them and is transmitted as captured.
if (!stereo) enhancer.process(mono, frameSamples);
double sumSq = 0;
for (int i = 0; i < frameSamples; i++) {
sumSq += (double) mono[i] * mono[i];
}
double rms = Math.sqrt(sumSq / frameSamples);
double db = (rms <= 1e-9) ? -100.0 : 20.0 * Math.log10(rms);
double db = InputLevel.toDb(mono, frameSamples);
Consumer<Double> ll = levelListener;
if (ll != null) ll.accept(db);
boolean wasOpen = transmitting.get();
boolean open = decideGate(db, mono);
setTransmitting(open);
if (open && !muted.get() && encoder != null) {
try {
packet = encoder.encode(stereo ? pcm : mono);
// On the opening edge, send the buffered lead-in first so the
// word's onset isn't lost to the frame that detected it.
if (!wasOpen) {
flushPreroll(stereo);
}
float[] sent = stereo ? pcm : mono;
packet = encoder.encode(sent);
monitor(sent, stereo ? encoderChannels : 1);
} catch (Exception ignored) {
}
}
if (!open) {
rememberForPreroll(stereo ? pcm : mono);
}
}
if (packet != null && packet.length > 0) {
queue.offer(packet);
// Guard against unbounded growth if the network stalls.
while (queue.size() > 10) queue.poll();
offer(packet);
}
}
}
private void offer(byte[] packet) {
queue.offer(packet);
// Guard against unbounded growth if the network stalls.
while (queue.size() > 10) queue.poll();
}
/** Keeps the most recent frames while the gate is shut, for {@link #flushPreroll}. */
private void rememberForPreroll(float[] frame) {
float[] slot = preroll[prerollTail];
if (slot == null || slot.length != frame.length) {
slot = new float[frame.length];
preroll[prerollTail] = slot;
}
System.arraycopy(frame, 0, slot, 0, frame.length);
prerollTail = (prerollTail + 1) % PREROLL_FRAMES;
if (prerollCount < PREROLL_FRAMES) prerollCount++;
}
/** Encodes and queues the buffered lead-in, oldest first. Call under {@link #encoderLock}. */
private void flushPreroll(boolean stereo) {
int expected = stereo ? AudioDevices.FRAME_SIZE * encoderChannels : AudioDevices.FRAME_SIZE;
for (int i = 0; i < prerollCount; i++) {
int idx = (prerollTail - prerollCount + i + PREROLL_FRAMES) % PREROLL_FRAMES;
float[] frame = preroll[idx];
// A channel-count change between capture and flush invalidates the buffer.
if (frame == null || frame.length != expected) continue;
try {
byte[] p = encoder.encode(frame);
if (p != null && p.length > 0) offer(p);
monitor(frame, stereo ? encoderChannels : 1);
} catch (Exception ignored) {
}
}
prerollCount = 0;
}
private boolean decideGate(double db, float[] pcm) {
if (muted.get()) {
if (muted.get() || localMuted.get()) {
hangover = 0;
detectMutedSpeech(db);
return false;
@@ -353,8 +435,10 @@ public final class DesktopVoiceInput implements VoiceInput {
}
/**
* Applies the selected VAD mode (volume gate, speech probability, or both) with
* a deactivation-delay hangover so trailing syllables aren't clipped.
* Applies the selected VAD mode with a hangover so trailing syllables aren't clipped.
*
* <p>The modes match the TS3 client's: the volume gate alone, the speech detector
* alone, or both together. Volume Gate skips the detector entirely, as TS3 does.
*/
private boolean voiceActivated(double db, float[] pcm) {
boolean detected;
@@ -363,11 +447,11 @@ public final class DesktopVoiceInput implements VoiceInput {
detected = db >= thresholdDb;
break;
case AUTOMATIC:
detected = speechDetector.process(pcm) >= speechThreshold;
detected = speechDetector.process(pcm, AudioDevices.FRAME_SIZE) >= speechThreshold;
break;
case HYBRID:
default:
double probability = speechDetector.process(pcm);
double probability = speechDetector.process(pcm, AudioDevices.FRAME_SIZE);
detected = db >= thresholdDb && probability >= speechThreshold;
break;
}
@@ -395,6 +479,12 @@ public final class DesktopVoiceInput implements VoiceInput {
mutedTalking = talking;
}
/** Hands a transmitted frame to the monitor, if one is attached. */
private void monitor(float[] frame, int channels) {
AudioFrameListener l = monitorListener;
if (l != null) l.onFrame(frame, channels);
}
private void setTransmitting(boolean t) {
transmitting.set(t);
if (t != lastTransmitting) {

View File

@@ -10,6 +10,8 @@ import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.function.BiConsumer;
/**
@@ -24,6 +26,16 @@ public final class DesktopVoiceOutput implements VoiceOutput {
/** Longest Opus frame (120 ms @ 48 kHz) a packet may decode to, per channel. */
private static final int MAX_FRAME = 5760;
/**
* A speaker is only meant to stop when the empty voice packet marking the end of a
* talk burst arrives — but that packet is UDP too, and a lost one otherwise leaves
* the talking indicator stuck until the speaker's next burst. Opus frames are 20&nbsp;ms
* apart while someone is actually talking, so a gap several times that long, with no
* packet of either kind, is unambiguous: force the indicator off rather than trust
* the one packet that could go missing.
*/
private static final long TALK_TIMEOUT_NANOS = TimeUnit.MILLISECONDS.toNanos(200);
/** One speaker's decode + playback pipeline. */
private final class ClientStream {
final int clientId;
@@ -81,8 +93,32 @@ public final class DesktopVoiceOutput implements VoiceOutput {
/** Notified (clientId, talking) on the EDT-agnostic worker thread when a speaker starts/stops. */
private volatile BiConsumer<Integer, Boolean> talkListener;
/** Catches a talk burst whose end packet never arrived; see {@link #TALK_TIMEOUT_NANOS}. */
private final ScheduledExecutorService watchdog = Executors.newSingleThreadScheduledExecutor(r -> {
Thread t = new Thread(r, "ts3j-talk-watchdog");
t.setDaemon(true);
return t;
});
public DesktopVoiceOutput(String outputDevice) {
this.outputDevice = outputDevice;
watchdog.scheduleWithFixedDelay(this::checkTalkTimeouts, 50, 50, TimeUnit.MILLISECONDS);
}
private void checkTalkTimeouts() {
long now = System.nanoTime();
for (ClientStream s : streams.values()) {
if (s.talking && now - s.lastPacketNanos > TALK_TIMEOUT_NANOS) {
s.worker.submit(() -> {
try {
s.line.drain();
} catch (Exception ignored) {
}
if (s.decoder != null) s.decoder.reset();
markTalking(s, false);
});
}
}
}
public void setTalkListener(BiConsumer<Integer, Boolean> l) {
@@ -226,6 +262,7 @@ public final class DesktopVoiceOutput implements VoiceOutput {
}
public void shutdown() {
watchdog.shutdownNow();
for (ClientStream s : streams.values()) {
s.close();
}

View File

@@ -0,0 +1,102 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.GlobalInputHook;
import com.ts3client.hotkey.HotkeyKey;
import java.util.ArrayList;
import java.util.List;
/**
* Picks the global input hook that suits the running session and starts it.
*
* <p>Windows has one answer, raw input. On X11 it is XInput2, with RECORD behind it for
* servers that lack it: neither needs privileges or devices to open. On Wayland the X
* server sees only what the compositor forwards, so reading evdev is the surest way to
* catch every key — and when that is not permitted either, the returned hook is simply
* not running and says why.
*/
public final class DesktopInputHooks {
private DesktopInputHooks() {
}
/** @return a hook, started when it could be; check {@link GlobalInputHook#isRunning()} */
public static GlobalInputHook start(GlobalInputHook.Listener listener) {
List<String> reasons = new ArrayList<>();
for (GlobalInputHook hook : candidates()) {
hook.start(listener);
if (hook.isRunning()) return hook;
reasons.add(hook.getClass().getSimpleName() + ": " + hook.unavailableReason());
hook.close();
}
return new Unavailable(reasons.isEmpty()
? "no global input backend for this session"
: String.join("; ", reasons));
}
private static List<GlobalInputHook> candidates() {
List<GlobalInputHook> hooks = new ArrayList<>();
if (Win32.isWindows()) {
if (WindowsInputHook.isSupported()) hooks.add(new WindowsInputHook());
return hooks;
}
if (isWayland()) {
// Evdev first: it is the only backend guaranteed to see keys aimed at native
// Wayland windows. Failing that, Xwayland still covers every X11 app, and on
// compositors that forward the rest — KWin, with legacy X11 app support on —
// it covers those too.
if (EvdevInputHook.isSupported()) hooks.add(new EvdevInputHook());
addX11Hooks(hooks);
} else {
addX11Hooks(hooks);
if (EvdevInputHook.isSupported()) hooks.add(new EvdevInputHook());
}
return hooks;
}
/**
* RECORD before XInput2: on at least some driver stacks, XInput2's raw
* {@code XI_RawButtonRelease} never reaches a client that hasn't grabbed the pointer —
* see {@link XInput2InputHook} — which makes a bound mouse button look stuck down.
* RECORD taps the same core events {@code xev} sees and is not affected. XInput2 stays
* as the fallback for servers where RECORD is disabled or missing.
*/
private static void addX11Hooks(List<GlobalInputHook> hooks) {
if (XRecordInputHook.isSupported()) hooks.add(new XRecordInputHook());
if (XInput2InputHook.isSupported()) hooks.add(new XInput2InputHook());
}
private static boolean isWayland() {
return System.getenv("WAYLAND_DISPLAY") != null
|| "wayland".equalsIgnoreCase(System.getenv("XDG_SESSION_TYPE"));
}
/** Stands in when no backend could start, so callers need no null checks. */
private record Unavailable(String reason) implements GlobalInputHook {
@Override
public void start(Listener listener) {
}
@Override
public boolean isRunning() {
return false;
}
@Override
public String unavailableReason() {
return reason;
}
@Override
public String keyName(HotkeyKey key) {
// Bindings recorded earlier still deserve their real names in the dialog,
// even with no hook running to have produced them.
return Win32.isWindows() ? WindowsKeyNamer.name(key) : X11KeyNamer.name(key);
}
@Override
public void close() {
}
}
}

View File

@@ -0,0 +1,151 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.GlobalInputHook;
import com.ts3client.hotkey.HotkeyKey;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.ArrayList;
import java.util.List;
/**
* System-wide input hook that reads the kernel's evdev devices directly, for sessions
* where {@link XRecordInputHook} cannot see everything — Wayland above all, where the X
* server is only told about events aimed at X clients.
*
* <p>It needs read access to {@code /dev/input/event*}, which normally means membership
* of the {@code input} group; without it the hook reports itself unavailable and the
* client carries on without global hotkeys.
*
* <p>Key codes are reported as X keycodes (the kernel code plus 8) and buttons as X
* button numbers, so bindings mean the same thing whichever backend recorded them.
*/
public final class EvdevInputHook implements GlobalInputHook {
private static final File INPUT_DIR = new File("/dev/input");
/** {@code struct input_event} on 64-bit Linux: two 8-byte time fields, then type/code/value. */
private static final int EVENT_SIZE = 24;
private static final int EV_KEY = 1;
/** Kernel key codes below this are keyboard keys; from here up they are BTN_* buttons. */
private static final int BTN_MISC = 0x100;
private static final int BTN_LEFT = 0x110;
private final List<InputStream> streams = new ArrayList<>();
private final List<Thread> threads = new ArrayList<>();
private volatile Listener listener;
private volatile boolean running;
private volatile String unavailable = "not started";
public static boolean isSupported() {
File[] devices = INPUT_DIR.listFiles((dir, name) -> name.startsWith("event"));
if (devices == null) return false;
for (File device : devices) {
if (device.canRead()) return true;
}
return false;
}
@Override
public void start(Listener listener) {
this.listener = listener;
File[] devices = INPUT_DIR.listFiles((dir, name) -> name.startsWith("event"));
if (devices == null || devices.length == 0) {
unavailable = "no /dev/input devices";
return;
}
for (File device : devices) {
try {
InputStream in = new FileInputStream(device);
streams.add(in);
Thread t = new Thread(() -> read(in), "ts3j-hotkeys-evdev-" + device.getName());
t.setDaemon(true);
threads.add(t);
} catch (Exception ignored) {
// Devices we may not read are simply skipped.
}
}
if (streams.isEmpty()) {
unavailable = "no readable /dev/input device — add your user to the \"input\" group";
return;
}
running = true;
unavailable = "";
for (Thread t : threads) t.start();
}
private void read(InputStream in) {
byte[] buffer = new byte[EVENT_SIZE * 16];
ByteBuffer view = ByteBuffer.wrap(buffer).order(ByteOrder.nativeOrder());
while (running) {
try {
int read = in.read(buffer);
if (read < 0) return;
for (int offset = 0; offset + EVENT_SIZE <= read; offset += EVENT_SIZE) {
int type = view.getShort(offset + 16) & 0xffff;
int code = view.getShort(offset + 18) & 0xffff;
int value = view.getInt(offset + 20);
// 2 is auto-repeat, which must not look like a fresh press.
if (type == EV_KEY && value != 2) dispatch(code, value == 1);
}
} catch (Exception e) {
return;
}
}
}
private void dispatch(int code, boolean pressed) {
Listener l = listener;
if (l == null) return;
HotkeyKey key = code < BTN_MISC ? HotkeyKey.keyboard(code + 8) : mouse(code);
if (key != null) l.onInput(key, pressed);
}
/** Maps the kernel's BTN_* codes onto the X button numbering. */
private static HotkeyKey mouse(int code) {
return switch (code) {
case BTN_LEFT -> HotkeyKey.mouse(1);
case BTN_LEFT + 1 -> HotkeyKey.mouse(3); // BTN_RIGHT
case BTN_LEFT + 2 -> HotkeyKey.mouse(2); // BTN_MIDDLE
case BTN_LEFT + 3 -> HotkeyKey.mouse(8); // BTN_SIDE, "mouse 4"
case BTN_LEFT + 4 -> HotkeyKey.mouse(9); // BTN_EXTRA, "mouse 5"
case BTN_LEFT + 5 -> HotkeyKey.mouse(10); // BTN_FORWARD
case BTN_LEFT + 6 -> HotkeyKey.mouse(11); // BTN_BACK
case BTN_LEFT + 7 -> HotkeyKey.mouse(12); // BTN_TASK
default -> null; // joysticks, lid switches, …
};
}
@Override
public boolean isRunning() {
return running;
}
@Override
public String unavailableReason() {
return running ? "" : unavailable;
}
@Override
public String keyName(HotkeyKey key) {
// The kernel knows scancodes, not layouts; let X name the key when it can.
return X11KeyNamer.name(key);
}
@Override
public void close() {
running = false;
listener = null;
// Closing the descriptor is what breaks the readers out of their blocking read.
for (InputStream in : streams) {
try {
in.close();
} catch (Exception ignored) {
}
}
streams.clear();
threads.clear();
}
}

View File

@@ -0,0 +1,47 @@
package com.ts3client.hotkey.desktop;
import java.util.Map;
/** Turns X keysym names into the labels users expect on a hotkey button. */
final class KeyNames {
private KeyNames() {
}
private static final Map<String, String> SPECIAL = Map.ofEntries(
Map.entry("Control_L", "Left Ctrl"),
Map.entry("Control_R", "Right Ctrl"),
Map.entry("Shift_L", "Left Shift"),
Map.entry("Shift_R", "Right Shift"),
Map.entry("Alt_L", "Left Alt"),
Map.entry("Alt_R", "Right Alt"),
Map.entry("ISO_Level3_Shift", "AltGr"),
Map.entry("Super_L", "Left Super"),
Map.entry("Super_R", "Right Super"),
Map.entry("Meta_L", "Left Meta"),
Map.entry("Meta_R", "Right Meta"),
Map.entry("Prior", "Page Up"),
Map.entry("Next", "Page Down"),
Map.entry("Return", "Enter"),
Map.entry("space", "Space"),
Map.entry("BackSpace", "Backspace"),
Map.entry("Escape", "Esc"),
Map.entry("Caps_Lock", "Caps Lock"),
Map.entry("Num_Lock", "Num Lock"),
Map.entry("Scroll_Lock", "Scroll Lock"),
Map.entry("Menu", "Menu"),
Map.entry("Print", "Print Screen"));
/**
* @param keysym the X name of the keysym, e.g. {@code a}, {@code Control_L}, {@code KP_Add}
* @return a display name, or {@code null} when there is nothing sensible to show
*/
static String pretty(String keysym) {
if (keysym == null || keysym.isBlank()) return null;
String special = SPECIAL.get(keysym);
if (special != null) return special;
if (keysym.startsWith("KP_")) return "Numpad " + pretty(keysym.substring(3));
if (keysym.length() == 1) return keysym.toUpperCase();
return keysym.replace('_', ' ');
}
}

View File

@@ -0,0 +1,109 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.HotkeyKey;
import java.util.ArrayList;
import java.util.List;
/**
* Layout of a {@code RAWINPUT} structure and the reading of the two kinds we ask for,
* split out from {@link WindowsInputHook} so the decoding can be tested anywhere.
*
* <p>Keyboards report a set-1 scan code, which is what a binding stores: it names the
* physical key, so it survives a layout change exactly as an X keycode does on Linux.
* Mice are renumbered to the X button numbering the rest of the client already speaks,
* so {@link HotkeyKey#fallbackName()} and the saved bindings mean the same thing on
* both platforms.
*/
final class RawInput {
private RawInput() {
}
/** {@code RAWINPUTHEADER.dwType}. */
static final int TYPE_MOUSE = 0;
static final int TYPE_KEYBOARD = 1;
/** Field offsets in {@code RAWINPUT} under x64, past the 24-byte header. */
static final long HEADER_TYPE = 0;
static final long HEADER_SIZE = 24;
static final long KEYBOARD_MAKE_CODE = 24;
static final long KEYBOARD_FLAGS = 26;
static final long KEYBOARD_VKEY = 30;
static final long MOUSE_BUTTON_FLAGS = 28;
/** Both event kinds fit comfortably; the larger, {@code RAWMOUSE}, needs 48 bytes. */
static final long BUFFER_SIZE = 64;
/** {@code RAWKEYBOARD.Flags}. */
static final int RI_KEY_BREAK = 0x01;
static final int RI_KEY_E0 = 0x02;
static final int RI_KEY_E1 = 0x04;
/** {@code VK_NO_VKEY}: the filler half of an escaped sequence, carrying no key. */
private static final int VKEY_NONE = 0xFF;
/** {@code RAWMOUSE.usButtonFlags}, in down/up pairs. */
private static final int[] BUTTON_FLAGS = {
0x0001, 0x0002, 1, // left
0x0010, 0x0020, 2, // middle
0x0004, 0x0008, 3, // right
0x0040, 0x0080, 8, // X1, "Mouse 4"
0x0100, 0x0200, 9, // X2, "Mouse 5"
};
private static final int RI_MOUSE_WHEEL = 0x0400;
private static final int RI_MOUSE_HWHEEL = 0x0800;
/**
* The scan code identifying a key, with the escape prefix folded into the high byte
* so that e.g. right Ctrl ({@code E0 1D}) stays distinct from left ({@code 1D}).
*
* @return the code, or -1 when the event names no key of its own
*/
static int scanCode(int makeCode, int flags, int vkey) {
if (vkey == VKEY_NONE) return -1;
int code = makeCode & 0xFF;
if (code == 0) return -1;
if ((flags & RI_KEY_E0) != 0) code |= 0xE000;
else if ((flags & RI_KEY_E1) != 0) code |= 0xE100;
return code;
}
static boolean isRelease(int flags) {
return (flags & RI_KEY_BREAK) != 0;
}
/** One button transition, in the order the flags word packs them. */
record ButtonEvent(HotkeyKey key, boolean pressed) {
}
/**
* Unpacks a mouse event, which may carry several transitions at once.
*
* @param buttonFlags {@code usButtonFlags}
* @param wheelDelta {@code usButtonData}, read as a signed delta when a wheel bit is set
*/
static List<ButtonEvent> buttons(int buttonFlags, short wheelDelta) {
List<ButtonEvent> events = new ArrayList<>();
for (int i = 0; i < BUTTON_FLAGS.length; i += 3) {
HotkeyKey key = HotkeyKey.mouse(BUTTON_FLAGS[i + 2]);
if ((buttonFlags & BUTTON_FLAGS[i]) != 0) events.add(new ButtonEvent(key, true));
if ((buttonFlags & BUTTON_FLAGS[i + 1]) != 0) events.add(new ButtonEvent(key, false));
}
// A wheel notch has no release of its own; X reports it as a button tap, and the
// hotkey engine expects the same shape, so synthesise both edges.
if ((buttonFlags & RI_MOUSE_WHEEL) != 0 && wheelDelta != 0) {
tap(events, wheelDelta > 0 ? 4 : 5);
}
if ((buttonFlags & RI_MOUSE_HWHEEL) != 0 && wheelDelta != 0) {
tap(events, wheelDelta > 0 ? 7 : 6);
}
return events;
}
private static void tap(List<ButtonEvent> events, int button) {
events.add(new ButtonEvent(HotkeyKey.mouse(button), true));
events.add(new ButtonEvent(HotkeyKey.mouse(button), false));
}
}

View File

@@ -0,0 +1,210 @@
package com.ts3client.hotkey.desktop;
import java.lang.foreign.Arena;
import java.lang.foreign.FunctionDescriptor;
import java.lang.foreign.Linker;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.SymbolLookup;
import java.lang.foreign.ValueLayout;
import java.lang.invoke.MethodHandle;
import java.nio.charset.StandardCharsets;
/**
* Raw binding to {@code user32} and {@code kernel32}, with just the calls
* {@link WindowsInputHook} needs to own a message-only window and read raw input from it.
*
* <p>Raw input is the polite way to watch the whole machine on Windows: registering with
* {@code RIDEV_INPUTSINK} delivers every key and button even while another application is
* in the foreground, and — unlike a {@code WH_KEYBOARD_LL} hook — it observes rather than
* intercepts, so nothing can be swallowed and nothing serialises the system input queue.
*
* <p>Loading is lazy and failure is expected off Windows, where {@link #isAvailable()}
* returns {@code false} and the caller falls back to another backend.
*/
final class Win32 {
private Win32() {
}
private static final ValueLayout.OfInt INT = ValueLayout.JAVA_INT;
private static final ValueLayout.OfLong LONG = ValueLayout.JAVA_LONG;
private static final java.lang.foreign.AddressLayout PTR = ValueLayout.ADDRESS;
/** Window messages we care about. */
static final int WM_DESTROY = 0x0002;
static final int WM_CLOSE = 0x0010;
static final int WM_INPUT = 0x00FF;
/** {@code HWND_MESSAGE}: parent for a window that only ever receives messages. */
static final long HWND_MESSAGE = -3L;
/** {@code RIDEV_INPUTSINK}: deliver input even when the window is not in front. */
static final int RIDEV_INPUTSINK = 0x00000100;
/** HID usages for the two devices we register (usage page 1, "generic desktop"). */
static final int USAGE_PAGE_GENERIC = 0x01;
static final int USAGE_MOUSE = 0x02;
static final int USAGE_KEYBOARD = 0x06;
/** {@code RID_INPUT}: ask {@code GetRawInputData} for the payload, not the header. */
static final int RID_INPUT = 0x10000003;
/** {@code WNDCLASSEXW} size and the field offsets we fill, under x64. */
static final long WNDCLASS_SIZE = 80;
static final long WNDCLASS_CBSIZE = 0;
static final long WNDCLASS_WNDPROC = 8;
static final long WNDCLASS_HINSTANCE = 24;
static final long WNDCLASS_CLASSNAME = 64;
/** {@code RAWINPUTDEVICE}: usage page, usage, flags, target window. */
static final long RAWINPUTDEVICE_SIZE = 16;
static final long RAWINPUTDEVICE_USAGE_PAGE = 0;
static final long RAWINPUTDEVICE_USAGE = 2;
static final long RAWINPUTDEVICE_FLAGS = 4;
static final long RAWINPUTDEVICE_TARGET = 8;
/** {@code MSG} is 48 bytes under x64. */
static final long MSG_SIZE = 48;
/** The C signature of a {@code WNDPROC}. */
static final FunctionDescriptor WND_PROC =
FunctionDescriptor.of(LONG, PTR, INT, LONG, LONG);
private static final Linker LINKER = Linker.nativeLinker();
private static final class Libs {
static final SymbolLookup USER32 = SymbolLookup.libraryLookup("user32.dll", Arena.global());
static final SymbolLookup KERNEL32 = SymbolLookup.libraryLookup("kernel32.dll", Arena.global());
static final MethodHandle GET_MODULE_HANDLE =
downcall(KERNEL32, "GetModuleHandleW", FunctionDescriptor.of(PTR, PTR));
static final MethodHandle REGISTER_CLASS =
downcall(USER32, "RegisterClassExW", FunctionDescriptor.of(INT, PTR));
static final MethodHandle UNREGISTER_CLASS =
downcall(USER32, "UnregisterClassW", FunctionDescriptor.of(INT, PTR, PTR));
static final MethodHandle CREATE_WINDOW =
downcall(USER32, "CreateWindowExW", FunctionDescriptor.of(PTR,
INT, PTR, PTR, INT, INT, INT, INT, INT, PTR, PTR, PTR, PTR));
static final MethodHandle DESTROY_WINDOW =
downcall(USER32, "DestroyWindow", FunctionDescriptor.of(INT, PTR));
static final MethodHandle DEF_WINDOW_PROC =
downcall(USER32, "DefWindowProcW", FunctionDescriptor.of(LONG, PTR, INT, LONG, LONG));
static final MethodHandle GET_MESSAGE =
downcall(USER32, "GetMessageW", FunctionDescriptor.of(INT, PTR, PTR, INT, INT));
static final MethodHandle DISPATCH_MESSAGE =
downcall(USER32, "DispatchMessageW", FunctionDescriptor.of(LONG, PTR));
static final MethodHandle POST_MESSAGE =
downcall(USER32, "PostMessageW", FunctionDescriptor.of(INT, PTR, INT, LONG, LONG));
static final MethodHandle POST_QUIT_MESSAGE =
downcall(USER32, "PostQuitMessage", FunctionDescriptor.ofVoid(INT));
static final MethodHandle REGISTER_RAW_INPUT =
downcall(USER32, "RegisterRawInputDevices", FunctionDescriptor.of(INT, PTR, INT, INT));
static final MethodHandle GET_RAW_INPUT_DATA =
downcall(USER32, "GetRawInputData",
FunctionDescriptor.of(INT, PTR, INT, PTR, PTR, INT));
static final MethodHandle GET_KEY_NAME_TEXT =
downcall(USER32, "GetKeyNameTextW", FunctionDescriptor.of(INT, INT, PTR, INT));
}
private static MethodHandle downcall(SymbolLookup lookup, String symbol,
FunctionDescriptor descriptor) {
return LINKER.downcallHandle(
lookup.find(symbol).orElseThrow(() ->
new UnsatisfiedLinkError("unresolved symbol " + symbol)),
descriptor);
}
static boolean isWindows() {
return System.getProperty("os.name", "").toLowerCase().startsWith("windows");
}
static boolean isAvailable() {
if (!isWindows()) return false;
try {
return Libs.USER32 != null && Libs.KERNEL32 != null;
} catch (Throwable t) {
return false;
}
}
/** Allocates a null-terminated UTF-16 string, as every {@code ...W} entry point wants. */
static MemorySegment wide(Arena arena, String text) {
return arena.allocateFrom(text, StandardCharsets.UTF_16LE);
}
static MemorySegment moduleHandle() {
return (MemorySegment) call(Libs.GET_MODULE_HANDLE, MemorySegment.NULL);
}
/** @return the class atom, or 0 when registration failed */
static int registerClass(MemorySegment wndClass) {
return (int) call(Libs.REGISTER_CLASS, wndClass);
}
static void unregisterClass(MemorySegment className, MemorySegment instance) {
call(Libs.UNREGISTER_CLASS, className, instance);
}
/** Creates a message-only window: no pixels, but it has a queue and can be a target. */
static MemorySegment createMessageWindow(MemorySegment className, MemorySegment instance) {
return (MemorySegment) call(Libs.CREATE_WINDOW, 0, className, MemorySegment.NULL, 0,
0, 0, 0, 0, MemorySegment.ofAddress(HWND_MESSAGE),
MemorySegment.NULL, instance, MemorySegment.NULL);
}
static void destroyWindow(MemorySegment window) {
call(Libs.DESTROY_WINDOW, window);
}
static long defWindowProc(MemorySegment window, int message, long wParam, long lParam) {
return (long) call(Libs.DEF_WINDOW_PROC, window, message, wParam, lParam);
}
/** @return 1 for a message, 0 for {@code WM_QUIT}, -1 on error */
static int getMessage(MemorySegment message) {
return (int) call(Libs.GET_MESSAGE, message, MemorySegment.NULL, 0, 0);
}
static void dispatchMessage(MemorySegment message) {
call(Libs.DISPATCH_MESSAGE, message);
}
static void postMessage(MemorySegment window, int message, long wParam, long lParam) {
call(Libs.POST_MESSAGE, window, message, wParam, lParam);
}
static void postQuitMessage(int exitCode) {
call(Libs.POST_QUIT_MESSAGE, exitCode);
}
/** @return whether the devices were registered for background delivery */
static boolean registerRawInputDevices(MemorySegment devices, int count) {
return (int) call(Libs.REGISTER_RAW_INPUT, devices, count, (int) RAWINPUTDEVICE_SIZE) != 0;
}
/** @return bytes written into {@code buffer}, or -1 on failure */
static int getRawInputData(MemorySegment handle, MemorySegment buffer, MemorySegment size) {
return (int) call(Libs.GET_RAW_INPUT_DATA, handle, RID_INPUT, buffer, size,
(int) RawInput.HEADER_SIZE);
}
/**
* The layout's name for a key.
*
* @param lParam scan code in bits 16..23, the extended flag in bit 24, as the API wants
*/
static String keyName(int lParam, Arena arena) {
MemorySegment buffer = arena.allocate(128);
int length = (int) call(Libs.GET_KEY_NAME_TEXT, lParam, buffer, 64);
if (length <= 0) return null;
return buffer.getString(0, StandardCharsets.UTF_16LE);
}
private static Object call(MethodHandle handle, Object... args) {
try {
return handle.invokeWithArguments(args);
} catch (Throwable t) {
throw new IllegalStateException("Win32 call failed", t);
}
}
}

View File

@@ -0,0 +1,234 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.GlobalInputHook;
import com.ts3client.hotkey.HotkeyKey;
import java.lang.foreign.Arena;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.ValueLayout;
import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles;
import java.lang.invoke.MethodType;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
/**
* System-wide input hook for Windows, built on raw input: a message-only window
* registers the keyboard and mouse with {@code RIDEV_INPUTSINK}, so every key and button
* on the machine arrives as {@code WM_INPUT} whether or not this application is in front,
* and the event still reaches the foreground window untouched.
*
* <p>Chosen over a {@code WH_KEYBOARD_LL} hook because raw input only observes: it cannot
* swallow a keystroke, it does not put this process in the path of the system input queue
* — where a slow callback stalls typing everywhere — and it reports the side buttons and
* both edges of every key, which push-to-talk needs.
*
* <p>A window's messages belong to the thread that created it, so the window is created
* on the pump thread and {@link #start} waits to hear whether that succeeded.
*/
public final class WindowsInputHook implements GlobalInputHook {
private static final String WINDOW_CLASS = "Ts3jHotkeyInputSink";
/** How long to wait for the pump thread to stand its window up before giving up. */
private static final long STARTUP_TIMEOUT_SECONDS = 5;
private static final MethodHandle WND_PROC;
static {
try {
WND_PROC = MethodHandles.lookup().findStatic(WindowsInputHook.class, "onMessage",
MethodType.methodType(long.class, MemorySegment.class, int.class,
long.class, long.class));
} catch (ReflectiveOperationException e) {
throw new ExceptionInInitializerError(e);
}
}
/** The running hook, for the static window procedure to find its way back. */
private static volatile WindowsInputHook current;
private volatile MemorySegment window = MemorySegment.NULL;
private Thread thread;
private volatile Listener listener;
private volatile boolean running;
private volatile String unavailable = "not started";
public static boolean isSupported() {
return Win32.isAvailable();
}
@Override
public void start(Listener listener) {
this.listener = listener;
CountDownLatch ready = new CountDownLatch(1);
thread = new Thread(() -> pump(ready), "ts3j-hotkeys-rawinput");
thread.setDaemon(true);
thread.start();
try {
if (!ready.await(STARTUP_TIMEOUT_SECONDS, TimeUnit.SECONDS)) {
unavailable = "the raw input window did not come up";
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
unavailable = "interrupted while starting";
}
}
/** Owns the window for its whole life: creates it, pumps it, tears it down. */
private void pump(CountDownLatch ready) {
boolean signalled = false;
try (Arena arena = Arena.ofConfined()) {
MemorySegment className = Win32.wide(arena, WINDOW_CLASS);
MemorySegment instance = Win32.moduleHandle();
MemorySegment stub = java.lang.foreign.Linker.nativeLinker()
.upcallStub(WND_PROC, Win32.WND_PROC, arena);
MemorySegment wndClass = arena.allocate(Win32.WNDCLASS_SIZE);
wndClass.set(ValueLayout.JAVA_INT, Win32.WNDCLASS_CBSIZE, (int) Win32.WNDCLASS_SIZE);
wndClass.set(ValueLayout.ADDRESS, Win32.WNDCLASS_WNDPROC, stub);
wndClass.set(ValueLayout.ADDRESS, Win32.WNDCLASS_HINSTANCE, instance);
wndClass.set(ValueLayout.ADDRESS, Win32.WNDCLASS_CLASSNAME, className);
// A leftover registration from an earlier run in this process is harmless:
// CreateWindowEx only needs the class to exist.
Win32.registerClass(wndClass);
MemorySegment hwnd = Win32.createMessageWindow(className, instance);
if (hwnd.equals(MemorySegment.NULL)) {
unavailable = "cannot create the raw input window";
return;
}
window = hwnd;
if (!Win32.registerRawInputDevices(rawInputDevices(arena, hwnd), 2)) {
unavailable = "RegisterRawInputDevices failed";
Win32.destroyWindow(hwnd);
window = MemorySegment.NULL;
return;
}
current = this;
running = true;
unavailable = "";
ready.countDown();
signalled = true;
MemorySegment message = arena.allocate(Win32.MSG_SIZE);
while (running) {
int result = Win32.getMessage(message);
// 0 is WM_QUIT, -1 an error; either way the window is finished.
if (result <= 0) break;
Win32.dispatchMessage(message);
}
} catch (Throwable t) {
unavailable = describe(t);
} finally {
running = false;
window = MemorySegment.NULL;
if (current == this) current = null;
if (!signalled) ready.countDown();
}
}
/** The keyboard and the mouse, both asked for in the background. */
private static MemorySegment rawInputDevices(Arena arena, MemorySegment hwnd) {
MemorySegment devices = arena.allocate(Win32.RAWINPUTDEVICE_SIZE * 2);
int[] usages = {Win32.USAGE_KEYBOARD, Win32.USAGE_MOUSE};
for (int i = 0; i < usages.length; i++) {
long base = i * Win32.RAWINPUTDEVICE_SIZE;
devices.set(ValueLayout.JAVA_SHORT, base + Win32.RAWINPUTDEVICE_USAGE_PAGE,
(short) Win32.USAGE_PAGE_GENERIC);
devices.set(ValueLayout.JAVA_SHORT, base + Win32.RAWINPUTDEVICE_USAGE,
(short) usages[i]);
devices.set(ValueLayout.JAVA_INT, base + Win32.RAWINPUTDEVICE_FLAGS,
Win32.RIDEV_INPUTSINK);
devices.set(ValueLayout.ADDRESS, base + Win32.RAWINPUTDEVICE_TARGET, hwnd);
}
return devices;
}
/** Upcall target: the window procedure. */
@SuppressWarnings("unused")
private static long onMessage(MemorySegment hwnd, int message, long wParam, long lParam) {
WindowsInputHook hook = current;
try {
if (message == Win32.WM_INPUT && hook != null) {
hook.readInput(lParam);
} else if (message == Win32.WM_DESTROY) {
Win32.postQuitMessage(0);
return 0;
}
} catch (Throwable ignored) {
}
// WM_INPUT must reach DefWindowProc too, so the system can release the event.
return Win32.defWindowProc(hwnd, message, wParam, lParam);
}
/** Copies one {@code RAWINPUT} out of the system and turns it into hotkey events. */
private void readInput(long handle) {
Listener l = listener;
if (l == null) return;
try (Arena arena = Arena.ofConfined()) {
MemorySegment buffer = arena.allocate(RawInput.BUFFER_SIZE);
MemorySegment size = arena.allocate(ValueLayout.JAVA_INT);
size.set(ValueLayout.JAVA_INT, 0, (int) RawInput.BUFFER_SIZE);
if (Win32.getRawInputData(MemorySegment.ofAddress(handle), buffer, size) <= 0) return;
switch (buffer.get(ValueLayout.JAVA_INT, RawInput.HEADER_TYPE)) {
case RawInput.TYPE_KEYBOARD -> {
int makeCode = buffer.get(ValueLayout.JAVA_SHORT, RawInput.KEYBOARD_MAKE_CODE) & 0xFFFF;
int flags = buffer.get(ValueLayout.JAVA_SHORT, RawInput.KEYBOARD_FLAGS) & 0xFFFF;
int vkey = buffer.get(ValueLayout.JAVA_SHORT, RawInput.KEYBOARD_VKEY) & 0xFFFF;
int code = RawInput.scanCode(makeCode, flags, vkey);
if (code >= 0) l.onInput(HotkeyKey.keyboard(code), !RawInput.isRelease(flags));
}
case RawInput.TYPE_MOUSE -> {
int flags = buffer.get(ValueLayout.JAVA_SHORT, RawInput.MOUSE_BUTTON_FLAGS) & 0xFFFF;
short data = buffer.get(ValueLayout.JAVA_SHORT, RawInput.MOUSE_BUTTON_FLAGS + 2);
for (RawInput.ButtonEvent e : RawInput.buttons(flags, data)) {
l.onInput(e.key(), e.pressed());
}
}
default -> {
}
}
} catch (Throwable ignored) {
}
}
@Override
public boolean isRunning() {
return running;
}
@Override
public String unavailableReason() {
return running ? "" : unavailable;
}
@Override
public String keyName(HotkeyKey key) {
return WindowsKeyNamer.name(key);
}
@Override
public void close() {
listener = null;
running = false;
MemorySegment hwnd = window;
try {
// Closing has to happen on the pump thread; ask it to, then let it unwind.
if (!hwnd.equals(MemorySegment.NULL)) Win32.postMessage(hwnd, Win32.WM_CLOSE, 0, 0);
if (thread != null) thread.join(2000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (Throwable ignored) {
}
thread = null;
}
private static String describe(Throwable t) {
String message = t.getMessage();
return (message == null || message.isBlank()) ? t.getClass().getSimpleName() : message;
}
}

View File

@@ -0,0 +1,37 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.HotkeyKey;
import java.lang.foreign.Arena;
import java.util.HashMap;
import java.util.Map;
/**
* Names scan codes through the active Windows layout, so a hotkey button shows "A"
* rather than "Key 30" — the counterpart to {@link X11KeyNamer}.
*
* <p>{@code GetKeyNameTextW} wants the scan code where a {@code WM_KEYDOWN} would carry
* it: bits 16..23, with bit 24 marking the {@code E0} escape.
*/
final class WindowsKeyNamer {
private WindowsKeyNamer() {
}
private static final Map<Integer, String> CACHE = new HashMap<>();
static synchronized String name(HotkeyKey key) {
if (key.device() != HotkeyKey.Device.KEYBOARD) return null;
if (!Win32.isAvailable()) return null;
return CACHE.computeIfAbsent(key.code(), code -> {
try (Arena arena = Arena.ofConfined()) {
int lParam = (code & 0xFF) << 16;
if ((code & 0xE000) == 0xE000) lParam |= 1 << 24;
String name = Win32.keyName(lParam, arena);
return (name == null || name.isBlank()) ? null : name;
} catch (Throwable t) {
return null;
}
});
}
}

View File

@@ -0,0 +1,47 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.HotkeyKey;
import java.lang.foreign.MemorySegment;
import java.util.HashMap;
import java.util.Map;
/**
* Names keycodes using the X keyboard layout, so a hotkey button shows "A" rather than
* "Key 38". Bindings themselves stay layout-independent; only the label comes from here.
*
* <p>Keeps one long-lived display connection of its own, opened on first use, so it can
* be asked from any thread without racing the input hooks' own connections.
*/
final class X11KeyNamer {
private X11KeyNamer() {
}
private static final Map<Integer, String> CACHE = new HashMap<>();
private static MemorySegment display;
private static boolean tried;
static synchronized String name(HotkeyKey key) {
if (key.device() != HotkeyKey.Device.KEYBOARD) return null;
if (!tried) {
tried = true;
try {
if (Xlib.isCoreAvailable() && System.getenv("DISPLAY") != null) {
display = Xlib.openDisplay();
}
} catch (Throwable ignored) {
display = null;
}
}
if (display == null || display.equals(MemorySegment.NULL)) return null;
return CACHE.computeIfAbsent(key.code(), code -> {
try {
long sym = Xlib.keysym(display, code);
return sym == 0 ? null : KeyNames.pretty(Xlib.keysymName(sym));
} catch (Throwable t) {
return null;
}
});
}
}

View File

@@ -0,0 +1,206 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.GlobalInputHook;
import com.ts3client.hotkey.HotkeyKey;
import java.lang.foreign.Arena;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.ValueLayout;
/**
* System-wide input hook built on XInput2 raw events, the way TeamSpeak's own Linux
* client does it: {@code XI_RawKeyPress} and friends are selected on the root window, so
* every key and button on the machine arrives here regardless of which window has focus,
* and the event still reaches that window untouched.
*
* <p>Raw events report the device's own view of the key, before focus routing and before
* any layout mapping, which is precisely what a hotkey wants — the detail field is the
* keycode, matching how {@link HotkeyKey} identifies keys.
*
* <p>On at least some driver stacks, {@code XI_RawButtonRelease} is never delivered here
* at all: the X server hands the initiating {@code XI_RawButtonPress} to every raw
* listener, but the matching release only reaches whichever client holds the pointer's
* implicit per-click grab, which is never us. An active {@code XIGrabDevice} grab makes
* this hook that owner and does fix delivery, but even with {@code owner_events} set it
* stopped clicks from reaching other windows at all — worse than the bug it fixes — so it
* is not used here. {@link XRecordInputHook} is unaffected and takes over first on X11 as
* a result; this hook stays as its fallback for servers without RECORD.
*
* <p>Under Wayland this hook lives inside Xwayland and therefore sees what Xwayland is
* given: every X11 client, plus — on compositors that allow it, KWin's "legacy X11 app
* support" among them — keys aimed at native Wayland windows too. Where the compositor
* forwards nothing, {@link EvdevInputHook} is the way out.
*/
public final class XInput2InputHook implements GlobalInputHook {
/** How long the event loop sleeps on the socket before rechecking {@link #running}. */
private static final int POLL_TIMEOUT_MILLIS = 200;
private Arena arena;
private MemorySegment display = MemorySegment.NULL;
private int opcode = -1;
private Thread thread;
private volatile Listener listener;
private volatile boolean running;
private volatile String unavailable = "not started";
public static boolean isSupported() {
return Xlib.isInputAvailable() && System.getenv("DISPLAY") != null;
}
@Override
public void start(Listener listener) {
this.listener = listener;
try {
open();
} catch (Throwable t) {
unavailable = describe(t);
closeQuietly();
}
}
private void open() {
arena = Arena.ofShared();
display = Xlib.openDisplay();
if (display.equals(MemorySegment.NULL)) {
throw new IllegalStateException("cannot open the X display");
}
opcode = Xlib.queryExtension(display, "XInputExtension", arena);
if (opcode < 0) throw new IllegalStateException("the X server has no XInput extension");
if (!Xlib.queryInputVersion(display, arena)) {
throw new IllegalStateException("the X server does not speak XInput 2");
}
long root = Xlib.defaultRootWindow(display);
int maskLen = maskLen();
MemorySegment bits = rawMaskBits(maskLen);
int[] masters = Xlib.queryMasterDeviceIds(display, arena);
Xlib.selectInputEvents(display, root, masters, bits, maskLen, arena);
Xlib.sync(display);
running = true;
unavailable = "";
thread = new Thread(this::pump, "ts3j-hotkeys-xinput2");
thread.setDaemon(true);
thread.start();
}
private static int maskLen() {
// XIMaskLen(event): the byte length of a mask covering event types up to this one.
return (Xlib.XI_RAW_BUTTON_RELEASE >> 3) + 1;
}
/** The four raw event types, as a byte mask in {@code XISetMask} order. */
private MemorySegment rawMaskBits(int maskLen) {
MemorySegment bits = arena.allocate(maskLen);
for (int type : new int[]{Xlib.XI_RAW_KEY_PRESS, Xlib.XI_RAW_KEY_RELEASE,
Xlib.XI_RAW_BUTTON_PRESS, Xlib.XI_RAW_BUTTON_RELEASE}) {
long index = type / 8;
byte bit = (byte) (1 << (type % 8));
bits.set(ValueLayout.JAVA_BYTE, index,
(byte) (bits.get(ValueLayout.JAVA_BYTE, index) | bit));
}
return bits;
}
private void pump() {
try (Arena loop = Arena.ofConfined()) {
MemorySegment event = loop.allocate(Xlib.EVENT_SIZE);
MemorySegment pollFd = Xlib.pollFd(display, loop);
while (running) {
if (Xlib.pending(display) <= 0) {
Xlib.awaitEvent(pollFd, POLL_TIMEOUT_MILLIS);
continue;
}
Xlib.nextEvent(display, event);
handle(event);
}
} catch (Throwable t) {
unavailable = describe(t);
} finally {
running = false;
}
}
/** Unpacks one event, which is ours only when it is a generic event from XInput2. */
private void handle(MemorySegment event) {
if (event.get(ValueLayout.JAVA_INT, Xlib.COOKIE_TYPE) != Xlib.GENERIC_EVENT) return;
if (event.get(ValueLayout.JAVA_INT, Xlib.COOKIE_EXTENSION) != opcode) return;
if (!Xlib.getEventData(display, event)) return;
try {
int evtype = event.get(ValueLayout.JAVA_INT, Xlib.COOKIE_EVTYPE);
MemorySegment raw = event.get(ValueLayout.ADDRESS, Xlib.COOKIE_DATA);
if (raw.equals(MemorySegment.NULL)) return;
int detail = raw.reinterpret(Xlib.RAW_EVENT_SIZE)
.get(ValueLayout.JAVA_INT, Xlib.RAW_EVENT_DETAIL);
dispatch(evtype, detail);
} finally {
Xlib.freeEventData(display, event);
}
}
private void dispatch(int evtype, int detail) {
Listener l = listener;
if (l == null) return;
switch (evtype) {
case Xlib.XI_RAW_KEY_PRESS -> l.onInput(HotkeyKey.keyboard(detail), true);
case Xlib.XI_RAW_KEY_RELEASE -> l.onInput(HotkeyKey.keyboard(detail), false);
case Xlib.XI_RAW_BUTTON_PRESS -> l.onInput(HotkeyKey.mouse(detail), true);
case Xlib.XI_RAW_BUTTON_RELEASE -> l.onInput(HotkeyKey.mouse(detail), false);
default -> {
}
}
}
@Override
public boolean isRunning() {
return running;
}
@Override
public String unavailableReason() {
return running ? "" : unavailable;
}
@Override
public String keyName(HotkeyKey key) {
return X11KeyNamer.name(key);
}
@Override
public void close() {
listener = null;
running = false;
try {
if (thread != null) thread.join(POLL_TIMEOUT_MILLIS * 5L);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
closeQuietly();
}
private synchronized void closeQuietly() {
// The pump owns the display connection; closing it underneath would crash Xlib.
boolean pumpDone = thread == null || !thread.isAlive();
if (pumpDone) {
try {
if (!display.equals(MemorySegment.NULL)) Xlib.closeDisplay(display);
} catch (Throwable ignored) {
}
display = MemorySegment.NULL;
if (arena != null) {
try {
arena.close();
} catch (Throwable ignored) {
}
arena = null;
}
thread = null;
}
}
private static String describe(Throwable t) {
String message = t.getMessage();
return (message == null || message.isBlank()) ? t.getClass().getSimpleName() : message;
}
}

View File

@@ -0,0 +1,212 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.GlobalInputHook;
import com.ts3client.hotkey.HotkeyKey;
import java.lang.foreign.Arena;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.ValueLayout;
import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles;
import java.lang.invoke.MethodType;
/**
* System-wide input hook built on X11's RECORD extension: it taps the core key and
* button events the server delivers to every client, so hotkeys work no matter which
* window has focus, and the keystroke still reaches that window untouched.
*
* <p>Two connections are needed, as RECORD demands: a control one that creates and
* later tears the context down, and a data one that blocks inside
* {@code XRecordEnableContext} handing us events.
*
* <p>Under Wayland the X server only ever sees events aimed at X clients, so this hook
* is not truly global there; {@link EvdevInputHook} is the way out.
*/
public final class XRecordInputHook implements GlobalInputHook {
private static final MethodHandle CALLBACK;
static {
try {
CALLBACK = MethodHandles.lookup().findStatic(XRecordInputHook.class, "onRecorded",
MethodType.methodType(void.class, MemorySegment.class, MemorySegment.class));
} catch (ReflectiveOperationException e) {
throw new ExceptionInInitializerError(e);
}
}
/** The running hook, for the static upcall to find its way back. */
private static volatile XRecordInputHook current;
private Arena arena;
private MemorySegment control = MemorySegment.NULL;
private MemorySegment data = MemorySegment.NULL;
private long context;
private Thread thread;
private volatile Listener listener;
private volatile boolean running;
private volatile String unavailable = "not started";
public static boolean isSupported() {
return Xlib.isAvailable() && System.getenv("DISPLAY") != null;
}
@Override
public void start(Listener listener) {
this.listener = listener;
try {
open();
} catch (Throwable t) {
unavailable = describe(t);
closeQuietly();
}
}
private void open() {
arena = Arena.ofShared();
control = Xlib.openDisplay();
if (control.equals(MemorySegment.NULL)) {
throw new IllegalStateException("cannot open the X display");
}
if (!Xlib.queryRecordVersion(control, arena)) {
throw new IllegalStateException("the X server has no RECORD extension");
}
MemorySegment range = Xlib.allocRange();
if (range.equals(MemorySegment.NULL)) throw new IllegalStateException("XRecordAllocRange failed");
MemorySegment r = range.reinterpret(64);
r.set(ValueLayout.JAVA_BYTE, Xlib.RANGE_DEVICE_EVENTS_FIRST, (byte) Xlib.KEY_PRESS);
r.set(ValueLayout.JAVA_BYTE, Xlib.RANGE_DEVICE_EVENTS_LAST, (byte) Xlib.BUTTON_RELEASE);
MemorySegment clients = arena.allocate(ValueLayout.JAVA_LONG);
clients.set(ValueLayout.JAVA_LONG, 0, Xlib.ALL_CLIENTS);
MemorySegment ranges = arena.allocate(ValueLayout.ADDRESS);
ranges.set(ValueLayout.ADDRESS, 0, range);
context = Xlib.createContext(control, clients, 1, ranges, 1);
if (context == 0) throw new IllegalStateException("XRecordCreateContext failed");
// The context id is allocated client-side and the request is not a round trip:
// without this the second connection can enable a context the server has yet to
// create, which it answers with BadContext.
Xlib.sync(control);
data = Xlib.openDisplay();
if (data.equals(MemorySegment.NULL)) {
throw new IllegalStateException("cannot open the second X display connection");
}
MemorySegment stub = java.lang.foreign.Linker.nativeLinker()
.upcallStub(CALLBACK, Xlib.INTERCEPT_PROC, arena);
current = this;
running = true;
unavailable = "";
thread = new Thread(() -> {
try {
// Blocks until close() disables the context from the control connection.
Xlib.enableContext(data, context, stub);
} catch (Throwable ignored) {
} finally {
running = false;
}
}, "ts3j-hotkeys-xrecord");
thread.setDaemon(true);
thread.start();
}
/** Upcall target: one recorded protocol datum. */
@SuppressWarnings("unused")
private static void onRecorded(MemorySegment closure, MemorySegment recorded) {
XRecordInputHook hook = current;
MemorySegment d = recorded.reinterpret(48);
try {
if (hook == null || d.get(ValueLayout.JAVA_INT, Xlib.INTERCEPT_CATEGORY) != Xlib.FROM_SERVER) {
return;
}
long length = d.get(ValueLayout.JAVA_LONG, Xlib.INTERCEPT_DATA_LEN);
MemorySegment event = d.get(ValueLayout.ADDRESS, Xlib.INTERCEPT_DATA);
// data_len counts 4-byte units; a core event is always 32 bytes.
if (event.equals(MemorySegment.NULL) || length < 2) return;
event = event.reinterpret(32);
hook.dispatch(event.get(ValueLayout.JAVA_BYTE, 0) & 0x7f,
event.get(ValueLayout.JAVA_BYTE, 1) & 0xff);
} catch (Throwable ignored) {
} finally {
Xlib.freeData(recorded);
}
}
private void dispatch(int type, int detail) {
Listener l = listener;
if (l == null) return;
switch (type) {
case Xlib.KEY_PRESS -> l.onInput(HotkeyKey.keyboard(detail), true);
case Xlib.KEY_RELEASE -> l.onInput(HotkeyKey.keyboard(detail), false);
case Xlib.BUTTON_PRESS -> l.onInput(HotkeyKey.mouse(detail), true);
case Xlib.BUTTON_RELEASE -> l.onInput(HotkeyKey.mouse(detail), false);
default -> {
}
}
}
@Override
public boolean isRunning() {
return running;
}
@Override
public String unavailableReason() {
return running ? "" : unavailable;
}
@Override
public String keyName(HotkeyKey key) {
return X11KeyNamer.name(key);
}
@Override
public void close() {
listener = null;
running = false;
try {
if (!control.equals(MemorySegment.NULL) && context != 0) {
Xlib.disableContext(control, context);
Xlib.flush(control);
}
if (thread != null) thread.join(1000);
} catch (Throwable ignored) {
}
closeQuietly();
}
private synchronized void closeQuietly() {
if (current == this) current = null;
try {
if (!control.equals(MemorySegment.NULL) && context != 0) Xlib.freeContext(control, context);
} catch (Throwable ignored) {
}
context = 0;
for (MemorySegment display : new MemorySegment[]{data, control}) {
try {
if (!display.equals(MemorySegment.NULL)) Xlib.closeDisplay(display);
} catch (Throwable ignored) {
}
}
data = control = MemorySegment.NULL;
// The arena owns the upcall stub: freeing it while the recording thread could
// still call into it would take the JVM down, so a stuck thread keeps it alive.
if (arena != null && (thread == null || !thread.isAlive())) {
try {
arena.close();
} catch (Throwable ignored) {
}
arena = null;
}
thread = null;
}
private static String describe(Throwable t) {
String message = t.getMessage();
return (message == null || message.isBlank()) ? t.getClass().getSimpleName() : message;
}
}

View File

@@ -0,0 +1,400 @@
package com.ts3client.hotkey.desktop;
import java.lang.foreign.Arena;
import java.lang.foreign.FunctionDescriptor;
import java.lang.foreign.Linker;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.SymbolLookup;
import java.lang.foreign.ValueLayout;
import java.lang.invoke.MethodHandle;
import java.nio.charset.StandardCharsets;
/**
* Raw binding to {@code libX11}, the XInput2 extension in {@code libXi} and the RECORD
* extension in {@code libXtst}, with just the calls the X11 hooks need.
*
* <p>Both extensions let a client watch input the server delivers to everyone else
* without intercepting it, which is exactly what a global hotkey needs: the keystroke
* still reaches the focused application. See {@link DesktopInputHooks#addX11Hooks} for
* which one is preferred and why.
*
* <p>Each library loads lazily and failure is expected — with no X11 around, the
* {@code is*Available()} probes return {@code false} and the caller falls back to
* another backend.
*/
final class Xlib {
private Xlib() {
}
static final ValueLayout.OfInt INT = ValueLayout.JAVA_INT;
static final ValueLayout.OfLong LONG = ValueLayout.JAVA_LONG;
private static final java.lang.foreign.AddressLayout PTR = ValueLayout.ADDRESS;
/** Core event types (X.h) as they appear in the recorded protocol data. */
static final int KEY_PRESS = 2;
static final int KEY_RELEASE = 3;
static final int BUTTON_PRESS = 4;
static final int BUTTON_RELEASE = 5;
/** {@code XRecordAllClients}: record every client, present and future. */
static final long ALL_CLIENTS = 3L;
/** {@code XRecordInterceptData.category} values we care about. */
static final int FROM_SERVER = 0;
/**
* {@code XRecordRange}: only {@code device_events} is filled in. Its two bytes sit
* after the request/reply and delivered-event ranges, at a fixed LP64 offset.
*/
static final long RANGE_DEVICE_EVENTS_FIRST = 18;
static final long RANGE_DEVICE_EVENTS_LAST = 19;
/** {@code XRecordInterceptData}: the protocol bytes and the reason we were called. */
static final long INTERCEPT_CATEGORY = 24;
static final long INTERCEPT_DATA = 32;
static final long INTERCEPT_DATA_LEN = 40;
/** {@code GenericEvent}, the envelope every extension event arrives in. */
static final int GENERIC_EVENT = 35;
/** XInput2 raw event types (XI2.h): the device's own view, before any focus routing. */
static final int XI_RAW_KEY_PRESS = 13;
static final int XI_RAW_KEY_RELEASE = 14;
static final int XI_RAW_BUTTON_PRESS = 15;
static final int XI_RAW_BUTTON_RELEASE = 16;
/** {@code XIAllMasterDevices}: raw events reach us through the master devices. */
static final int XI_ALL_MASTER_DEVICES = 1;
/** An {@code XEvent} is a union of 24 longs; a cookie is the widest member we read. */
static final long EVENT_SIZE = 192;
/** {@code XGenericEventCookie} field offsets under LP64. */
static final long COOKIE_TYPE = 0;
static final long COOKIE_EXTENSION = 32;
static final long COOKIE_EVTYPE = 36;
static final long COOKIE_DATA = 48;
/**
* {@code XIRawEvent.detail} — the keycode or button number — past the generic header,
* {@code time}, {@code deviceid} and {@code sourceid}.
*/
static final long RAW_EVENT_DETAIL = 56;
static final long RAW_EVENT_SIZE = 152;
/** {@code XIEventMask}: {@code deviceid}, {@code mask_len}, then the mask pointer. */
static final long EVENT_MASK_SIZE = 16;
static final long EVENT_MASK_DEVICEID = 0;
static final long EVENT_MASK_LEN = 4;
static final long EVENT_MASK_MASK = 8;
/** {@code XIDeviceInfo.deviceid} — the only field {@link #queryMasterDeviceIds} needs. */
static final long DEVICE_INFO_SIZE = 40;
static final long DEVICE_INFO_DEVICEID = 0;
private static final Linker LINKER = Linker.nativeLinker();
private static final String[] X11_NAMES = {"libX11.so.6", "libX11.so"};
private static final String[] XI_NAMES = {"libXi.so.6", "libXi.so"};
private static final String[] XTST_NAMES = {"libXtst.so.6", "libXtst.so"};
/** libX11 alone: enough to open a display and name keys. */
private static final class Core {
static final SymbolLookup LIB = load(X11_NAMES, "libX11");
static final MethodHandle X_OPEN_DISPLAY =
downcall(LIB, "XOpenDisplay", FunctionDescriptor.of(PTR, PTR));
static final MethodHandle X_CLOSE_DISPLAY =
downcall(LIB, "XCloseDisplay", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_FLUSH =
downcall(LIB, "XFlush", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_SYNC =
downcall(LIB, "XSync", FunctionDescriptor.of(INT, PTR, INT));
static final MethodHandle XKB_KEYCODE_TO_KEYSYM =
downcall(LIB, "XkbKeycodeToKeysym", FunctionDescriptor.of(LONG, PTR, INT, INT, INT));
static final MethodHandle X_KEYSYM_TO_STRING =
downcall(LIB, "XKeysymToString", FunctionDescriptor.of(PTR, LONG));
static final MethodHandle X_QUERY_EXTENSION =
downcall(LIB, "XQueryExtension", FunctionDescriptor.of(INT, PTR, PTR, PTR, PTR, PTR));
static final MethodHandle X_DEFAULT_ROOT_WINDOW =
downcall(LIB, "XDefaultRootWindow", FunctionDescriptor.of(LONG, PTR));
static final MethodHandle X_CONNECTION_NUMBER =
downcall(LIB, "XConnectionNumber", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_PENDING =
downcall(LIB, "XPending", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_NEXT_EVENT =
downcall(LIB, "XNextEvent", FunctionDescriptor.of(INT, PTR, PTR));
static final MethodHandle X_GET_EVENT_DATA =
downcall(LIB, "XGetEventData", FunctionDescriptor.of(INT, PTR, PTR));
static final MethodHandle X_FREE_EVENT_DATA =
downcall(LIB, "XFreeEventData", FunctionDescriptor.ofVoid(PTR, PTR));
}
/** XInput2, which ships separately in libXi. */
private static final class Input {
static final SymbolLookup LIB = load(XI_NAMES, "libXi");
static final MethodHandle XI_QUERY_VERSION =
downcall(LIB, "XIQueryVersion", FunctionDescriptor.of(INT, PTR, PTR, PTR));
static final MethodHandle XI_SELECT_EVENTS =
downcall(LIB, "XISelectEvents", FunctionDescriptor.of(INT, PTR, LONG, PTR, INT));
static final MethodHandle XI_QUERY_DEVICE =
downcall(LIB, "XIQueryDevice", FunctionDescriptor.of(PTR, PTR, INT, PTR));
static final MethodHandle XI_FREE_DEVICE_INFO =
downcall(LIB, "XIFreeDeviceInfo", FunctionDescriptor.ofVoid(PTR));
}
/** {@code poll(2)}, so the event loop can wait on the X socket and still be woken. */
private static final class Poll {
static final MethodHandle POLL = LINKER.downcallHandle(
LINKER.defaultLookup().find("poll").orElseThrow(() ->
new UnsatisfiedLinkError("unresolved symbol poll")),
FunctionDescriptor.of(INT, PTR, LONG, INT));
}
/** {@code struct pollfd} is an int and two shorts, and {@code POLLIN} is bit 0. */
private static final long POLLFD_SIZE = 8;
private static final long POLLFD_EVENTS = 4;
private static final short POLLIN = 1;
/** The RECORD extension, which ships separately in libXtst. */
private static final class Record {
static final SymbolLookup LIB = load(XTST_NAMES, "libXtst");
static final MethodHandle QUERY_VERSION =
downcall(LIB, "XRecordQueryVersion", FunctionDescriptor.of(INT, PTR, PTR, PTR));
static final MethodHandle ALLOC_RANGE =
downcall(LIB, "XRecordAllocRange", FunctionDescriptor.of(PTR));
static final MethodHandle CREATE_CONTEXT =
downcall(LIB, "XRecordCreateContext",
FunctionDescriptor.of(LONG, PTR, INT, PTR, INT, PTR, INT));
static final MethodHandle ENABLE_CONTEXT =
downcall(LIB, "XRecordEnableContext", FunctionDescriptor.of(INT, PTR, LONG, PTR, PTR));
static final MethodHandle DISABLE_CONTEXT =
downcall(LIB, "XRecordDisableContext", FunctionDescriptor.of(INT, PTR, LONG));
static final MethodHandle FREE_CONTEXT =
downcall(LIB, "XRecordFreeContext", FunctionDescriptor.of(INT, PTR, LONG));
static final MethodHandle FREE_DATA =
downcall(LIB, "XRecordFreeData", FunctionDescriptor.ofVoid(PTR));
}
private static SymbolLookup load(String[] names, String what) {
IllegalArgumentException last = null;
for (String name : names) {
try {
return SymbolLookup.libraryLookup(name, Arena.global());
} catch (IllegalArgumentException e) {
last = e;
}
}
throw (last != null) ? last : new IllegalArgumentException(what + " not found");
}
private static MethodHandle downcall(SymbolLookup lookup, String symbol,
FunctionDescriptor descriptor) {
return LINKER.downcallHandle(
lookup.find(symbol).orElseThrow(() ->
new UnsatisfiedLinkError("unresolved symbol " + symbol)),
descriptor);
}
/** The C signature of {@code XRecordInterceptProc}. */
static final FunctionDescriptor INTERCEPT_PROC = FunctionDescriptor.ofVoid(PTR, PTR);
/** Whether libX11 loaded, which is all that naming keys needs. */
static boolean isCoreAvailable() {
try {
return Core.LIB != null;
} catch (Throwable t) {
return false;
}
}
static boolean isAvailable() {
try {
return Core.LIB != null && Record.LIB != null;
} catch (Throwable t) {
return false;
}
}
/** Whether libX11 and libXi both loaded, which is what the XInput2 hook needs. */
static boolean isInputAvailable() {
try {
return Core.LIB != null && Input.LIB != null && Poll.POLL != null;
} catch (Throwable t) {
return false;
}
}
static MemorySegment openDisplay() {
return (MemorySegment) call(Core.X_OPEN_DISPLAY, MemorySegment.NULL);
}
static void closeDisplay(MemorySegment display) {
call(Core.X_CLOSE_DISPLAY, display);
}
static void flush(MemorySegment display) {
call(Core.X_FLUSH, display);
}
/** Flushes and waits for the server to have processed everything sent so far. */
static void sync(MemorySegment display) {
call(Core.X_SYNC, display, 0);
}
/** @return whether the server has the RECORD extension */
static boolean queryRecordVersion(MemorySegment display, Arena arena) {
MemorySegment major = arena.allocate(INT);
MemorySegment minor = arena.allocate(INT);
return (int) call(Record.QUERY_VERSION, display, major, minor) != 0;
}
static MemorySegment allocRange() {
return (MemorySegment) call(Record.ALLOC_RANGE);
}
static long createContext(MemorySegment display, MemorySegment clients, int clientCount,
MemorySegment ranges, int rangeCount) {
return (long) call(Record.CREATE_CONTEXT, display, 0, clients, clientCount, ranges, rangeCount);
}
static int enableContext(MemorySegment display, long context, MemorySegment callback) {
return (int) call(Record.ENABLE_CONTEXT, display, context, callback, MemorySegment.NULL);
}
static void disableContext(MemorySegment display, long context) {
call(Record.DISABLE_CONTEXT, display, context);
}
static void freeContext(MemorySegment display, long context) {
call(Record.FREE_CONTEXT, display, context);
}
static void freeData(MemorySegment data) {
call(Record.FREE_DATA, data);
}
/** @return the extension's major opcode, or -1 when the server does not have it */
static int queryExtension(MemorySegment display, String name, Arena arena) {
MemorySegment opcode = arena.allocate(INT);
MemorySegment event = arena.allocate(INT);
MemorySegment error = arena.allocate(INT);
int found = (int) call(Core.X_QUERY_EXTENSION, display, arena.allocateFrom(name),
opcode, event, error);
return found != 0 ? opcode.get(INT, 0) : -1;
}
/** @return whether the server speaks at least XInput 2.0 */
static boolean queryInputVersion(MemorySegment display, Arena arena) {
MemorySegment major = arena.allocate(INT);
MemorySegment minor = arena.allocate(INT);
major.set(INT, 0, 2);
minor.set(INT, 0, 0);
// Success is 0; anything else means the server would not agree on 2.x.
return (int) call(Input.XI_QUERY_VERSION, display, major, minor) == 0;
}
static long defaultRootWindow(MemorySegment display) {
return (long) call(Core.X_DEFAULT_ROOT_WINDOW, display);
}
/**
* Asks for the events set in {@code bits} on {@code window}, once per device in
* {@code deviceIds}: selecting against {@code XIAllMasterDevices} itself does not
* reliably deliver every raw event type, so callers resolve it to real device ids first
* with {@link #queryMasterDeviceIds}.
*/
static void selectInputEvents(MemorySegment display, long window, int[] deviceIds,
MemorySegment bits, int maskLen, Arena arena) {
MemorySegment masks = arena.allocate(EVENT_MASK_SIZE * deviceIds.length);
for (int i = 0; i < deviceIds.length; i++) {
long base = i * EVENT_MASK_SIZE;
masks.set(INT, base + EVENT_MASK_DEVICEID, deviceIds[i]);
masks.set(INT, base + EVENT_MASK_LEN, maskLen);
masks.set(PTR, base + EVENT_MASK_MASK, bits);
}
call(Input.XI_SELECT_EVENTS, display, window, masks, deviceIds.length);
}
/**
* The real device ids behind the {@code XIAllMasterDevices} pseudo-device — selecting
* raw events against each one individually, rather than the pseudo-device itself, is
* the more correct form (some drivers reportedly need it), even though it alone does
* not fix the {@code XI_RawButtonRelease} gap {@link XInput2InputHook} documents.
*/
static int[] queryMasterDeviceIds(MemorySegment display, Arena arena) {
MemorySegment count = arena.allocate(INT);
MemorySegment infos = (MemorySegment) call(Input.XI_QUERY_DEVICE, display,
XI_ALL_MASTER_DEVICES, count);
int n = count.get(INT, 0);
if (infos.equals(MemorySegment.NULL) || n <= 0) return new int[0];
try {
MemorySegment array = infos.reinterpret(DEVICE_INFO_SIZE * n);
int[] ids = new int[n];
for (int i = 0; i < n; i++) {
ids[i] = array.get(INT, i * DEVICE_INFO_SIZE + DEVICE_INFO_DEVICEID);
}
return ids;
} finally {
call(Input.XI_FREE_DEVICE_INFO, infos);
}
}
static int pending(MemorySegment display) {
return (int) call(Core.X_PENDING, display);
}
static void nextEvent(MemorySegment display, MemorySegment event) {
call(Core.X_NEXT_EVENT, display, event);
}
/** Fetches a generic event's payload; the caller must free it when this succeeds. */
static boolean getEventData(MemorySegment display, MemorySegment cookie) {
return (int) call(Core.X_GET_EVENT_DATA, display, cookie) != 0;
}
static void freeEventData(MemorySegment display, MemorySegment cookie) {
call(Core.X_FREE_EVENT_DATA, display, cookie);
}
/**
* A {@code struct pollfd} watching the display connection, to be allocated once and
* handed to {@link #awaitEvent} for the life of an event loop.
*/
static MemorySegment pollFd(MemorySegment display, Arena arena) {
MemorySegment fds = arena.allocate(POLLFD_SIZE);
fds.set(INT, 0, (int) call(Core.X_CONNECTION_NUMBER, display));
fds.set(ValueLayout.JAVA_SHORT, POLLFD_EVENTS, POLLIN);
return fds;
}
/**
* Waits until the connection has something to read or the timeout runs out, so an
* event loop can stay asleep yet still notice it has been asked to stop.
*/
static void awaitEvent(MemorySegment pollFd, int timeoutMillis) {
call(Poll.POLL, pollFd, 1L, timeoutMillis);
}
/** The unshifted keysym of a keycode in the first group, or 0 when unbound. */
static long keysym(MemorySegment display, int keycode) {
return (long) call(Core.XKB_KEYCODE_TO_KEYSYM, display, keycode, 0, 0);
}
/** The keysym's X name ("a", "Control_L", "F5"), or {@code null}. */
static String keysymName(long keysym) {
MemorySegment name = (MemorySegment) call(Core.X_KEYSYM_TO_STRING, keysym);
if (name == null || name.equals(MemorySegment.NULL)) return null;
return name.reinterpret(Long.MAX_VALUE).getString(0, StandardCharsets.US_ASCII);
}
private static Object call(MethodHandle handle, Object... args) {
try {
return handle.invokeWithArguments(args);
} catch (Throwable t) {
throw new IllegalStateException("X11 call failed", t);
}
}
}

View File

@@ -0,0 +1,92 @@
package com.ts3client.hotkey.desktop;
import com.ts3client.hotkey.HotkeyKey;
import org.junit.jupiter.api.Test;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/** Decoding of Windows raw input, which needs no Windows to check. */
class RawInputTest {
@Test
void plainKeyKeepsItsScanCode() {
// 'A' is scan code 0x1E, no escape.
assertEquals(0x1E, RawInput.scanCode(0x1E, 0, 0x41));
}
@Test
void escapedKeysStayDistinctFromTheirUnescapedTwins() {
// Left Ctrl is 1D; right Ctrl is E0 1D, and the two must not collide.
int left = RawInput.scanCode(0x1D, 0, 0xA2);
int right = RawInput.scanCode(0x1D, RawInput.RI_KEY_E0, 0xA3);
assertEquals(0x1D, left);
assertEquals(0xE01D, right);
}
@Test
void pauseUsesTheOtherEscape() {
assertEquals(0xE11D, RawInput.scanCode(0x1D, RawInput.RI_KEY_E1, 0x13));
}
@Test
void fillerHalfOfAnEscapedSequenceIsIgnored() {
assertEquals(-1, RawInput.scanCode(0x2A, RawInput.RI_KEY_E0, 0xFF));
assertEquals(-1, RawInput.scanCode(0, 0, 0x41));
}
@Test
void breakFlagMarksTheRelease() {
assertFalse(RawInput.isRelease(0));
assertTrue(RawInput.isRelease(RawInput.RI_KEY_BREAK));
assertTrue(RawInput.isRelease(RawInput.RI_KEY_BREAK | RawInput.RI_KEY_E0));
}
@Test
void sideButtonsMapToTheNumbersUsersKnow() {
// X1 down: "Mouse 4" everywhere else in the client, X button 8.
List<RawInput.ButtonEvent> down = RawInput.buttons(0x0040, (short) 0);
assertEquals(List.of(new RawInput.ButtonEvent(HotkeyKey.mouse(8), true)), down);
assertEquals("Mouse 4", HotkeyKey.mouse(8).fallbackName());
List<RawInput.ButtonEvent> up = RawInput.buttons(0x0200, (short) 0);
assertEquals(List.of(new RawInput.ButtonEvent(HotkeyKey.mouse(9), false)), up);
assertEquals("Mouse 5", HotkeyKey.mouse(9).fallbackName());
}
@Test
void primaryButtonsUseTheXNumbering() {
assertEquals(List.of(new RawInput.ButtonEvent(HotkeyKey.mouse(1), true)),
RawInput.buttons(0x0001, (short) 0));
assertEquals(List.of(new RawInput.ButtonEvent(HotkeyKey.mouse(2), true)),
RawInput.buttons(0x0010, (short) 0));
assertEquals(List.of(new RawInput.ButtonEvent(HotkeyKey.mouse(3), true)),
RawInput.buttons(0x0004, (short) 0));
}
@Test
void oneEventCanCarrySeveralTransitions() {
// Left up and right down in the same report.
assertEquals(List.of(new RawInput.ButtonEvent(HotkeyKey.mouse(1), false),
new RawInput.ButtonEvent(HotkeyKey.mouse(3), true)),
RawInput.buttons(0x0002 | 0x0004, (short) 0));
}
@Test
void wheelBecomesATapSoTheEngineSeesBothEdges() {
assertEquals(List.of(new RawInput.ButtonEvent(HotkeyKey.mouse(4), true),
new RawInput.ButtonEvent(HotkeyKey.mouse(4), false)),
RawInput.buttons(0x0400, (short) 120));
assertEquals(List.of(new RawInput.ButtonEvent(HotkeyKey.mouse(5), true),
new RawInput.ButtonEvent(HotkeyKey.mouse(5), false)),
RawInput.buttons(0x0400, (short) -120));
}
@Test
void mouseMovementAloneReportsNothing() {
assertEquals(List.of(), RawInput.buttons(0, (short) 0));
}
}

View File

@@ -35,6 +35,12 @@
<groupId>com.github.weisj</groupId>
<artifactId>jsvg</artifactId>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.13.4</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>

View File

@@ -0,0 +1,190 @@
package com.ts3client.ui;
import com.ts3client.config.ClientVersion;
import com.ts3client.config.Settings;
import javax.swing.BorderFactory;
import javax.swing.DefaultListCellRenderer;
import javax.swing.JCheckBox;
import javax.swing.JComboBox;
import javax.swing.JComponent;
import javax.swing.JLabel;
import javax.swing.JList;
import javax.swing.JPanel;
import javax.swing.JTextField;
import java.awt.BorderLayout;
import java.awt.Component;
import java.awt.GridBagConstraints;
import java.awt.GridBagLayout;
import java.awt.Insets;
/**
* Options page for the version and operating system reported to the server, which
* is what other clients see next to the nickname ("3.6.2 on Windows").
*
* <p>Servers verify TeamSpeak's signature over the version and platform and refuse
* the connection ("client is modified") when it does not match, so the page is
* built around the signed versions in {@link ClientVersion#PRESETS}. The fields
* behind them can still be edited by hand for a version whose signature is known
* from somewhere else. Changes take effect on the next connect.
*/
final class ClientVersionPanel extends JPanel {
/** Combo entry that leaves the three fields to the user. */
private static final String CUSTOM = "Custom…";
private final Settings settings;
private final JCheckBox enabled = new JCheckBox("Report a custom version and operating system");
private final JComboBox<Object> presetCombo = new JComboBox<>();
private final JTextField versionField = new JTextField(24);
private final JComboBox<String> platformCombo = new JComboBox<>();
private final JTextField signField = new JTextField(24);
/** Set while a preset is being copied into the fields, so that does not count as an edit. */
private boolean filling;
ClientVersionPanel(Settings settings) {
super(new BorderLayout(0, 8));
this.settings = settings;
setBorder(BorderFactory.createEmptyBorder(12, 12, 12, 12));
add(buildForm(), BorderLayout.NORTH);
load();
enabled.addActionListener(e -> updateEnabled());
updateEnabled();
}
/** Copies the edited version into the settings; the caller saves them. */
void apply() {
settings.customVersion = enabled.isSelected();
settings.clientVersion = versionField.getText().trim();
settings.clientPlatform = String.valueOf(platformCombo.getEditor().getItem()).trim();
settings.clientVersionSign = signField.getText().trim();
}
private JComponent buildForm() {
JPanel p = new JPanel(new GridBagLayout());
GridBagConstraints c = new GridBagConstraints();
c.insets = new Insets(2, 2, 2, 2);
c.anchor = GridBagConstraints.WEST;
c.fill = GridBagConstraints.HORIZONTAL;
presetCombo.addItem(CUSTOM);
for (ClientVersion preset : ClientVersion.PRESETS) presetCombo.addItem(preset);
presetCombo.setRenderer(new DefaultListCellRenderer() {
@Override
public Component getListCellRendererComponent(JList<?> list, Object value, int index,
boolean selected, boolean focus) {
super.getListCellRendererComponent(list, value, index, selected, focus);
setText(value instanceof ClientVersion v ? v.label() : String.valueOf(value));
return this;
}
});
presetCombo.setToolTipText("Versions with a signature TeamSpeak issued for them");
presetCombo.addActionListener(e -> onPresetSelected());
platformCombo.setEditable(true);
for (String platform : ClientVersion.PLATFORMS) platformCombo.addItem(platform);
versionField.setToolTipText("Version string, e.g. 3.6.2 [Build: 1695203293]");
platformCombo.setToolTipText("Operating system shown to other clients");
signField.setToolTipText("TeamSpeak's signature for this version and platform");
versionField.getDocument().addDocumentListener(new EditListener());
signField.getDocument().addDocumentListener(new EditListener());
platformCombo.addActionListener(e -> onFieldEdited());
int row = 0;
c.gridx = 0;
c.gridy = row++;
c.gridwidth = 2;
p.add(enabled, c);
c.gridwidth = 1;
addRow(p, c, row++, new JLabel("Version:"), presetCombo);
addRow(p, c, row++, new JLabel("Version string:"), versionField);
addRow(p, c, row++, new JLabel("Operating system:"), platformCombo);
addRow(p, c, row++, new JLabel("Signature:"), signField);
c.gridx = 0;
c.gridy = row;
c.gridwidth = 2;
p.add(new JLabel("<html><body style='width:320px'>The three fields belong together: the "
+ "signature is TeamSpeak's, over that version and operating system. A server "
+ "answers \"client is modified\" and refuses the connection when it does not "
+ "accept the version — made up, or older than the server's minimum — so prefer "
+ "a recent one from the list. Takes effect the next time you connect."
+ "</body></html>"), c);
return p;
}
private void load() {
enabled.setSelected(settings.customVersion);
ClientVersion preset = ClientVersion.match(
settings.clientVersion, settings.clientPlatform, settings.clientVersionSign);
fill(preset != null ? preset
: new ClientVersion("", settings.clientVersion, settings.clientPlatform, settings.clientVersionSign));
presetCombo.setSelectedItem(preset != null ? preset : CUSTOM);
}
private void onPresetSelected() {
if (presetCombo.getSelectedItem() instanceof ClientVersion preset) fill(preset);
}
/** A hand-edited field no longer describes the selected preset. */
private void onFieldEdited() {
if (filling) return;
ClientVersion preset = ClientVersion.match(versionField.getText().trim(),
String.valueOf(platformCombo.getEditor().getItem()).trim(), signField.getText().trim());
presetCombo.setSelectedItem(preset != null ? preset : CUSTOM);
}
private void fill(ClientVersion version) {
filling = true;
try {
versionField.setText(version.version());
platformCombo.setSelectedItem(version.platform());
platformCombo.getEditor().setItem(version.platform());
signField.setText(version.sign());
} finally {
filling = false;
}
}
private void updateEnabled() {
boolean on = enabled.isSelected();
presetCombo.setEnabled(on);
versionField.setEnabled(on);
platformCombo.setEnabled(on);
signField.setEnabled(on);
}
/** Notices any typing in the version or signature field. */
private final class EditListener implements javax.swing.event.DocumentListener {
@Override
public void insertUpdate(javax.swing.event.DocumentEvent e) {
onFieldEdited();
}
@Override
public void removeUpdate(javax.swing.event.DocumentEvent e) {
onFieldEdited();
}
@Override
public void changedUpdate(javax.swing.event.DocumentEvent e) {
onFieldEdited();
}
}
private static void addRow(JPanel p, GridBagConstraints c, int row, JLabel label, Component field) {
c.gridx = 0;
c.gridy = row;
c.weightx = 0;
p.add(label, c);
c.gridx = 1;
c.weightx = 1;
p.add(field, c);
}
}

View File

@@ -0,0 +1,160 @@
package com.ts3client.ui;
import com.ts3client.hotkey.Hotkey;
import com.ts3client.hotkey.HotkeyEngine;
import javax.swing.SwingUtilities;
import java.util.List;
/**
* Carries a fired hotkey out on the client.
*
* <p>Activations arrive on the input hook's thread, so everything is handed to the
* event dispatch thread first. Which connections an action reaches is the binding's
* "on active server" flag: set, only the selected tab; clear, every connected one.
*/
final class HotkeyActions implements HotkeyEngine.Handler {
/** How much one press of the master-volume hotkeys moves the slider. */
private static final double VOLUME_STEP = 0.05;
private final MainFrame frame;
/** Latched push-to-talk, driven by the "Toggle Push-to-Talk" action. */
private boolean pttLatched;
HotkeyActions(MainFrame frame) {
this.frame = frame;
}
@Override
public void onHotkey(Hotkey hotkey, boolean active) {
SwingUtilities.invokeLater(() -> perform(hotkey, active));
}
private void perform(Hotkey hotkey, boolean active) {
List<ServerTab> targets = frame.hotkeyTargets(hotkey.activeServerOnly);
switch (hotkey.action) {
case CONNECT_CURRENT_TAB -> frame.connectBookmark(hotkey.argument, false);
case CONNECT_NEW_TAB -> frame.connectBookmark(hotkey.argument, true);
case DISCONNECT_CURRENT -> {
ServerTab tab = frame.selectedTab();
if (tab != null) tab.disconnect();
}
case DISCONNECT_ALL -> {
for (ServerTab tab : frame.allTabs()) tab.disconnect();
}
case MIC_ACTIVATE -> frame.moveMicrophoneToSelectedTab();
case MIC_MUTE -> setMicMuted(targets, true);
case MIC_UNMUTE -> setMicMuted(targets, false);
case MIC_TOGGLE -> setMicMuted(targets, !anyMicMuted(targets));
case MIC_LOCAL_MUTE -> setMicLocalMuted(targets, true);
case MIC_LOCAL_UNMUTE -> setMicLocalMuted(targets, false);
case MIC_LOCAL_TOGGLE -> setMicLocalMuted(targets, !anyMicLocalMuted(targets));
case SPEAKER_MUTE -> setDeafened(targets, true);
case SPEAKER_UNMUTE -> setDeafened(targets, false);
case SPEAKER_TOGGLE -> setDeafened(targets, !anyDeafened(targets));
case AWAY_SET -> setAway(targets, true, "");
case AWAY_ONLINE -> setAway(targets, false, "");
case AWAY_TOGGLE -> setAway(targets, !anyAway(targets), "");
case AWAY_TOGGLE_WITH_MESSAGE -> setAway(targets, !anyAway(targets), hotkey.argument);
case COMMANDER_ACTIVATE -> setCommander(targets, true);
case COMMANDER_DEACTIVATE -> setCommander(targets, false);
case COMMANDER_TOGGLE -> setCommander(targets, !anyCommander(targets));
// Momentary: the engine reports the release too, so the key simply holds it open.
case PTT_ACTIVATE -> frame.setPushToTalk(active || pttLatched);
case PTT_DEACTIVATE -> {
pttLatched = false;
frame.setPushToTalk(false);
}
case PTT_TOGGLE -> {
pttLatched = !pttLatched;
frame.setPushToTalk(pttLatched);
}
case CHANNEL_SWITCH -> {
for (ServerTab tab : targets) tab.joinChannelPath(hotkey.argument);
}
case SERVER_TAB_SELECT -> frame.selectTabNumber(parseIndex(hotkey.argument));
case SERVER_TAB_NEXT -> frame.stepTab(1);
case SERVER_TAB_PREVIOUS -> frame.stepTab(-1);
case SOUND_MUTE -> frame.setSoundsMuted(true);
case SOUND_UNMUTE -> frame.setSoundsMuted(false);
case SOUND_TOGGLE -> frame.setSoundsMuted(!frame.areSoundsMuted());
case VOLUME_INCREASE -> frame.adjustMasterVolume(VOLUME_STEP);
case VOLUME_DECREASE -> frame.adjustMasterVolume(-VOLUME_STEP);
case NICKNAME_CHANGE -> frame.changeNickname(hotkey.argument);
case FILEBROWSER -> frame.browseCurrentChannel();
case SKIN_RELOAD -> frame.reloadSkin();
case BRING_TO_FRONT -> frame.bringToFront();
case SEND_TO_BACK -> frame.sendToBack();
default -> {
// Listed for completeness in the action catalogue, but not implemented here.
}
}
}
private void setMicMuted(List<ServerTab> targets, boolean muted) {
for (ServerTab tab : targets) tab.setMicMuted(muted);
frame.refreshAfterHotkey();
}
private void setMicLocalMuted(List<ServerTab> targets, boolean muted) {
for (ServerTab tab : targets) tab.setMicLocalMuted(muted);
frame.refreshAfterHotkey();
}
private void setDeafened(List<ServerTab> targets, boolean deaf) {
for (ServerTab tab : targets) tab.setDeafened(deaf);
frame.refreshAfterHotkey();
}
private void setAway(List<ServerTab> targets, boolean away, String message) {
for (ServerTab tab : targets) tab.setAway(away, message == null ? "" : message);
frame.refreshAfterHotkey();
}
private void setCommander(List<ServerTab> targets, boolean commander) {
for (ServerTab tab : targets) tab.setCommander(commander);
frame.refreshAfterHotkey();
}
private static boolean anyMicMuted(List<ServerTab> tabs) {
return tabs.stream().anyMatch(ServerTab::isMicMuted);
}
private static boolean anyMicLocalMuted(List<ServerTab> tabs) {
return tabs.stream().anyMatch(ServerTab::isMicLocalMuted);
}
private static boolean anyDeafened(List<ServerTab> tabs) {
return tabs.stream().anyMatch(ServerTab::isDeafened);
}
private static boolean anyAway(List<ServerTab> tabs) {
return tabs.stream().anyMatch(ServerTab::isAway);
}
private static boolean anyCommander(List<ServerTab> tabs) {
return tabs.stream().anyMatch(ServerTab::isCommander);
}
/** @return the 1-based tab number in the argument, or 1 when it is not a number */
private static int parseIndex(String argument) {
try {
return Math.max(1, Integer.parseInt(argument.trim()));
} catch (RuntimeException e) {
return 1;
}
}
}

View File

@@ -0,0 +1,16 @@
package com.ts3client.ui;
import com.ts3client.hotkey.HotkeyAction;
import java.util.List;
/**
* Supplies the concrete values an action's parameter can take — the bookmarks, sound
* packs and channels the hotkey tree hangs under an action as its leaves, so a binding
* reads as "Sounds / Activate Soundpack / Default Sound Pack (Male)".
*/
interface HotkeyArguments {
/** @return the choices for this action, or an empty list when it is free-form */
List<String> choices(HotkeyAction action);
}

View File

@@ -0,0 +1,385 @@
package com.ts3client.ui;
import com.ts3client.hotkey.Hotkey;
import com.ts3client.hotkey.HotkeyAction;
import com.ts3client.hotkey.HotkeyCombo;
import com.ts3client.hotkey.HotkeyEngine;
import javax.swing.BorderFactory;
import javax.swing.Box;
import javax.swing.DefaultListCellRenderer;
import javax.swing.JButton;
import javax.swing.JCheckBox;
import javax.swing.JComboBox;
import javax.swing.JComponent;
import javax.swing.JDialog;
import javax.swing.JLabel;
import javax.swing.JList;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextField;
import javax.swing.JTree;
import javax.swing.SwingUtilities;
import javax.swing.tree.DefaultMutableTreeNode;
import javax.swing.tree.DefaultTreeCellRenderer;
import javax.swing.tree.DefaultTreeModel;
import javax.swing.tree.TreePath;
import javax.swing.tree.TreeSelectionModel;
import java.awt.BorderLayout;
import java.awt.Component;
import java.awt.Dimension;
import java.awt.Font;
import java.awt.GridBagConstraints;
import java.awt.GridBagLayout;
import java.awt.Insets;
import java.awt.Window;
import java.util.List;
/**
* Adds or edits one hotkey, following the official client's dialog: pick the action
* from the tree — category, action group, action, and where the action takes one, the
* concrete bookmark, sound pack or channel — press the key combination to bind, and
* choose the edge it triggers on and whether it applies to the active server only.
*/
final class HotkeyDialog extends JDialog {
/**
* A tree node: a category or group heading ({@code action == null}), an action, or
* one of the values an action's parameter can take ({@code argument != null}).
*/
private record Node(HotkeyAction action, String argument, String label) {
static Node heading(String label) {
return new Node(null, null, label);
}
static Node of(HotkeyAction action) {
return new Node(action, null, action.label());
}
static Node value(HotkeyAction action, String argument) {
return new Node(action, argument, argument);
}
boolean isHeading() {
return action == null;
}
@Override
public String toString() {
return label;
}
}
private final HotkeyService service;
private final Hotkey hotkey;
private final DefaultMutableTreeNode root = new DefaultMutableTreeNode("Actions");
private final DefaultTreeModel treeModel = new DefaultTreeModel(root);
private final JTree tree = new JTree(treeModel);
private final JCheckBox advancedCheck = new JCheckBox("Show advanced actions");
private final JButton keyButton = new JButton();
private final JComboBox<Hotkey.Trigger> triggerCombo = new JComboBox<>(Hotkey.Trigger.values());
private final JCheckBox activeServerCheck = new JCheckBox("On active server");
private final JLabel argumentLabel = new JLabel();
private final JTextField argumentField = new JTextField();
private final JLabel hint = new JLabel();
private HotkeyCombo combo;
private boolean recording;
private boolean confirmed;
HotkeyDialog(Window owner, HotkeyService service, Hotkey existing) {
// Any window may open this: the options dialog's hotkey tab as much as a frame.
super(owner, existing == null ? "Add hotkey" : "Edit hotkey", ModalityType.APPLICATION_MODAL);
this.service = service;
this.hotkey = existing == null ? new Hotkey() : existing.copy();
this.combo = hotkey.combo;
advancedCheck.setSelected(hotkey.action != null && hotkey.action.advanced());
advancedCheck.addActionListener(e -> rebuildTree());
tree.setRootVisible(false);
tree.setShowsRootHandles(true);
tree.getSelectionModel().setSelectionMode(TreeSelectionModel.SINGLE_TREE_SELECTION);
tree.setCellRenderer(new NodeRenderer());
tree.addTreeSelectionListener(e -> selectionChanged());
rebuildTree();
keyButton.addActionListener(e -> startRecording());
triggerCombo.setRenderer(new DefaultListCellRenderer() {
@Override
public Component getListCellRendererComponent(JList<?> list, Object value, int index,
boolean selected, boolean focused) {
super.getListCellRendererComponent(list, value, index, selected, focused);
if (value instanceof Hotkey.Trigger t) setText(t.label());
return this;
}
});
triggerCombo.setSelectedItem(hotkey.trigger);
activeServerCheck.setSelected(hotkey.activeServerOnly);
argumentField.setText(hotkey.argument == null ? "" : hotkey.argument);
hint.setFont(hint.getFont().deriveFont(Font.ITALIC, hint.getFont().getSize2D() - 1f));
getContentPane().setLayout(new BorderLayout(8, 8));
((JComponent) getContentPane()).setBorder(BorderFactory.createEmptyBorder(8, 8, 8, 8));
getContentPane().add(buildActionPane(), BorderLayout.CENTER);
getContentPane().add(buildForm(), BorderLayout.SOUTH);
updateKeyButton();
updateForAction();
Dialogs.closeOnEscape(this, this::cancel);
setDefaultCloseOperation(DISPOSE_ON_CLOSE);
setSize(new Dimension(470, 560));
setLocationRelativeTo(owner);
}
private JPanel buildActionPane() {
JPanel p = new JPanel(new BorderLayout(0, 4));
p.add(new JLabel("Action:"), BorderLayout.NORTH);
p.add(new JScrollPane(tree), BorderLayout.CENTER);
p.add(advancedCheck, BorderLayout.SOUTH);
return p;
}
private JPanel buildForm() {
JPanel p = new JPanel(new GridBagLayout());
GridBagConstraints c = new GridBagConstraints();
c.insets = new Insets(3, 3, 3, 3);
c.anchor = GridBagConstraints.WEST;
c.fill = GridBagConstraints.HORIZONTAL;
int row = 0;
addRow(p, c, row++, new JLabel("Hotkey:"), keyButton);
addRow(p, c, row++, argumentLabel, argumentField);
addRow(p, c, row++, new JLabel("Trigger:"), triggerCombo);
c.gridx = 1;
c.gridy = row++;
p.add(activeServerCheck, c);
c.gridx = 0;
c.gridy = row++;
c.gridwidth = 2;
p.add(hint, c);
JPanel buttons = new JPanel();
JButton ok = new JButton("OK");
JButton cancel = new JButton("Cancel");
ok.addActionListener(e -> confirm());
cancel.addActionListener(e -> cancel());
buttons.add(Box.createHorizontalGlue());
buttons.add(ok);
buttons.add(cancel);
c.gridy = row;
p.add(buttons, c);
getRootPane().setDefaultButton(ok);
return p;
}
private static void addRow(JPanel p, GridBagConstraints c, int row, JComponent left, JComponent right) {
c.gridwidth = 1;
c.gridx = 0;
c.gridy = row;
c.weightx = 0;
p.add(left, c);
c.gridx = 1;
c.weightx = 1;
p.add(right, c);
}
/**
* Builds category → group → action → value, collapsing the groups TS3 leaves
* unnamed so their actions sit directly under the category.
*/
private void rebuildTree() {
boolean advanced = advancedCheck.isSelected();
root.removeAllChildren();
for (HotkeyAction.Category category : HotkeyAction.Category.values()) {
List<HotkeyAction> actions = HotkeyAction.of(category, advanced);
if (actions.isEmpty()) continue;
DefaultMutableTreeNode categoryNode =
new DefaultMutableTreeNode(Node.heading(category.label()));
DefaultMutableTreeNode groupNode = null;
String groupName = null;
for (HotkeyAction action : actions) {
DefaultMutableTreeNode parent = categoryNode;
if (!action.group().isEmpty()) {
if (groupNode == null || !action.group().equals(groupName)) {
groupName = action.group();
groupNode = new DefaultMutableTreeNode(Node.heading(groupName));
categoryNode.add(groupNode);
}
parent = groupNode;
}
DefaultMutableTreeNode actionNode = new DefaultMutableTreeNode(Node.of(action));
parent.add(actionNode);
for (String value : service.argumentChoices(action)) {
actionNode.add(new DefaultMutableTreeNode(Node.value(action, value)));
}
}
root.add(categoryNode);
}
treeModel.reload();
// Categories open, groups closed: the whole action set at a glance, as in TS3.
for (int i = 0; i < root.getChildCount(); i++) {
tree.expandPath(new TreePath(((DefaultMutableTreeNode) root.getChildAt(i)).getPath()));
}
selectCurrent();
}
/** Reveals and selects the node matching the binding being edited. */
private void selectCurrent() {
if (hotkey.action == null) return;
DefaultMutableTreeNode match = null;
var nodes = root.depthFirstEnumeration();
while (nodes.hasMoreElements()) {
DefaultMutableTreeNode node = (DefaultMutableTreeNode) nodes.nextElement();
if (!(node.getUserObject() instanceof Node n) || n.action() != hotkey.action) continue;
boolean sameArgument = n.argument() != null && n.argument().equals(hotkey.argument);
if (sameArgument) {
match = node;
break;
}
if (n.argument() == null && match == null) match = node;
}
if (match == null) return;
TreePath path = new TreePath(match.getPath());
tree.setSelectionPath(path);
SwingUtilities.invokeLater(() -> tree.scrollPathToVisible(path));
}
private void selectionChanged() {
if (!(tree.getLastSelectedPathComponent() instanceof DefaultMutableTreeNode node)
|| !(node.getUserObject() instanceof Node selected)) {
return;
}
if (selected.isHeading()) {
// Headings only structure the tree; keep the action that was chosen before.
tree.clearSelection();
selectCurrent();
return;
}
hotkey.action = selected.action();
if (selected.argument() != null) argumentField.setText(selected.argument());
updateForAction();
}
/** Syncs the form to the selected action: parameter row, trigger, scope and hint. */
private void updateForAction() {
HotkeyAction action = hotkey.action;
boolean takesArgument = action != null && action.argument() != HotkeyAction.Argument.NONE;
argumentLabel.setText(takesArgument ? argumentLabel(action) : "");
argumentLabel.setVisible(takesArgument);
argumentField.setVisible(takesArgument);
boolean momentary = action != null && action.momentary();
triggerCombo.setEnabled(!momentary);
activeServerCheck.setEnabled(action != null && action.category() != HotkeyAction.Category.MISC);
if (action != null && !action.supported()) {
hint.setText("This action is part of TeamSpeak's hotkey set but is not implemented yet.");
} else if (momentary) {
hint.setText("Held down: the action lasts as long as the hotkey is pressed.");
} else {
hint.setText(service.isRunning() ? " " : service.status());
}
}
private static String argumentLabel(HotkeyAction action) {
return switch (action.argument()) {
case BOOKMARK -> "Bookmark:";
case CHANNEL -> "Channel path:";
case PROFILE -> "Profile:";
default -> "Parameter:";
};
}
private void startRecording() {
if (recording) return;
if (!service.isRunning()) {
hint.setText(service.status());
return;
}
recording = true;
keyButton.setText("Press hotkey combination…");
service.record(new HotkeyEngine.Recorder() {
@Override
public void onRecording(HotkeyCombo partial) {
SwingUtilities.invokeLater(() -> keyButton.setText(service.display(partial) + ""));
}
@Override
public void onRecorded(HotkeyCombo recorded) {
SwingUtilities.invokeLater(() -> {
combo = recorded;
stopRecording();
});
}
});
}
private void stopRecording() {
if (!recording) return;
recording = false;
service.stopRecording();
updateKeyButton();
}
private void updateKeyButton() {
keyButton.setText(combo == null || combo.isEmpty()
? "No hotkey assigned" : service.display(combo));
}
private void confirm() {
stopRecording();
if (hotkey.action == null || combo == null || combo.isEmpty()) {
hint.setText("Pick an action and press a key combination first.");
return;
}
if (!hotkey.action.supported()) {
hint.setText("This action is not implemented yet — pick another one.");
return;
}
hotkey.combo = combo;
hotkey.trigger = (Hotkey.Trigger) triggerCombo.getSelectedItem();
hotkey.activeServerOnly = activeServerCheck.isSelected();
hotkey.argument = argumentField.getText().trim();
confirmed = true;
dispose();
}
private void cancel() {
stopRecording();
dispose();
}
boolean isConfirmed() {
return confirmed;
}
Hotkey result() {
return hotkey;
}
/** Draws headings in bold and greys out the actions this client cannot perform. */
private static final class NodeRenderer extends DefaultTreeCellRenderer {
@Override
public Component getTreeCellRendererComponent(JTree tree, Object value, boolean selected,
boolean expanded, boolean leaf, int row,
boolean focused) {
super.getTreeCellRendererComponent(tree, value, selected, expanded, leaf, row, focused);
setIcon(null);
if (!(value instanceof DefaultMutableTreeNode node)
|| !(node.getUserObject() instanceof Node n)) {
return this;
}
if (n.isHeading()) {
setFont(getFont().deriveFont(Font.BOLD));
setToolTipText(null);
} else {
setEnabled(n.action().supported());
setToolTipText(n.action().supported() ? null : "Not implemented by this client");
}
return this;
}
}
}

View File

@@ -0,0 +1,94 @@
package com.ts3client.ui;
import com.ts3client.hotkey.GlobalInputHook;
import com.ts3client.hotkey.Hotkey;
import com.ts3client.hotkey.HotkeyAction;
import com.ts3client.hotkey.HotkeyCombo;
import com.ts3client.hotkey.HotkeyEngine;
import com.ts3client.hotkey.HotkeyKey;
import com.ts3client.hotkey.Hotkeys;
import com.ts3client.hotkey.desktop.DesktopInputHooks;
import java.util.List;
/**
* Owns the hotkey machinery for the UI: the stored bindings, the matching engine and
* the platform input hook they are fed from. Everything the dialogs need — recording a
* combination, naming keys, telling the user why hotkeys are dead — goes through here.
*/
final class HotkeyService {
private final Hotkeys hotkeys = Hotkeys.load();
private final HotkeyEngine engine;
private final GlobalInputHook hook;
private final HotkeyArguments arguments;
HotkeyService(HotkeyEngine.Handler handler, HotkeyArguments arguments) {
this.arguments = arguments;
engine = new HotkeyEngine(hotkeys, handler);
hook = DesktopInputHooks.start(engine);
}
/** The values this action's parameter can take right now, for the action tree. */
List<String> argumentChoices(HotkeyAction action) {
if (arguments == null || action.argument() == HotkeyAction.Argument.NONE) return List.of();
try {
return arguments.choices(action);
} catch (RuntimeException e) {
return List.of();
}
}
List<Hotkey> all() {
return hotkeys.all();
}
void replaceAll(List<Hotkey> updated) {
engine.releaseAll();
hotkeys.replaceAll(updated);
hotkeys.save();
}
/** The binding for an action with no argument, or {@code null} when unbound. */
Hotkey find(HotkeyAction action) {
synchronized (hotkeys.all()) {
for (Hotkey h : hotkeys.all()) {
if (h.action == action) return h;
}
}
return null;
}
boolean isRunning() {
return hook.isRunning();
}
/** One line for the options dialog: either working, or why it is not. */
String status() {
return hook.isRunning()
? "Global hotkeys are active."
: "Global hotkeys are unavailable (" + hook.unavailableReason() + ").";
}
String display(HotkeyCombo combo) {
return combo == null ? "No hotkey assigned" : combo.display(this::keyName);
}
private String keyName(HotkeyKey key) {
return hook.keyName(key);
}
/** Captures the next combination instead of firing bindings; see {@link #stopRecording()}. */
void record(HotkeyEngine.Recorder recorder) {
engine.record(recorder);
}
void stopRecording() {
engine.stopRecording();
}
void dispose() {
engine.releaseAll();
hook.close();
}
}

View File

@@ -0,0 +1,170 @@
package com.ts3client.ui;
import com.ts3client.hotkey.Hotkey;
import javax.swing.BorderFactory;
import javax.swing.JButton;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTable;
import javax.swing.ListSelectionModel;
import javax.swing.SwingUtilities;
import javax.swing.table.AbstractTableModel;
import java.awt.BorderLayout;
import java.awt.Font;
import java.awt.Window;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import java.util.ArrayList;
import java.util.List;
/**
* The options dialog's hotkey tab: the list of bindings with the buttons to add, edit
* and remove them. Edits happen on a working copy and only reach the running engine
* when the dialog is applied.
*/
final class HotkeysPanel extends JPanel {
private static final String[] COLUMNS = {"Action", "Hotkey", "Trigger", "Active server", "On"};
private final HotkeyService service;
private final List<Hotkey> working = new ArrayList<>();
private final Model model = new Model();
private final JTable table = new JTable(model);
HotkeysPanel(HotkeyService service) {
super(new BorderLayout(6, 6));
this.service = service;
setBorder(BorderFactory.createEmptyBorder(8, 8, 8, 8));
synchronized (service.all()) {
for (Hotkey h : service.all()) working.add(h.copy());
}
table.setSelectionMode(ListSelectionModel.SINGLE_SELECTION);
table.setRowHeight(table.getRowHeight() + 4);
table.getColumnModel().getColumn(0).setPreferredWidth(240);
table.getColumnModel().getColumn(1).setPreferredWidth(150);
table.addMouseListener(new MouseAdapter() {
@Override
public void mouseClicked(MouseEvent e) {
if (e.getClickCount() == 2) edit();
}
});
JButton add = new JButton("Add…");
JButton edit = new JButton("Edit…");
JButton remove = new JButton("Remove");
add.addActionListener(e -> add());
edit.addActionListener(e -> edit());
remove.addActionListener(e -> remove());
JPanel buttons = new JPanel();
buttons.add(add);
buttons.add(edit);
buttons.add(remove);
JLabel status = new JLabel(service.status());
status.setFont(status.getFont().deriveFont(Font.ITALIC, status.getFont().getSize2D() - 1f));
add(new JScrollPane(table), BorderLayout.CENTER);
JPanel south = new JPanel(new BorderLayout());
south.add(buttons, BorderLayout.WEST);
south.add(status, BorderLayout.SOUTH);
add(south, BorderLayout.SOUTH);
}
private void add() {
HotkeyDialog dlg = new HotkeyDialog(owner(), service, null);
dlg.setVisible(true);
if (!dlg.isConfirmed()) return;
working.add(dlg.result());
model.fireTableDataChanged();
}
private void edit() {
int row = table.getSelectedRow();
if (row < 0) return;
HotkeyDialog dlg = new HotkeyDialog(owner(), service, working.get(row));
dlg.setVisible(true);
if (!dlg.isConfirmed()) return;
working.set(row, dlg.result());
model.fireTableRowsUpdated(row, row);
}
private void remove() {
int row = table.getSelectedRow();
if (row < 0) return;
working.remove(row);
model.fireTableDataChanged();
}
private Window owner() {
return SwingUtilities.getWindowAncestor(this);
}
/** Commits the edited list to the engine and to disk. */
void apply() {
service.replaceAll(working);
}
/** Picks the stored bindings up again after something else changed them. */
void reload() {
working.clear();
synchronized (service.all()) {
for (Hotkey h : service.all()) working.add(h.copy());
}
model.fireTableDataChanged();
}
private final class Model extends AbstractTableModel {
@Override
public int getRowCount() {
return working.size();
}
@Override
public int getColumnCount() {
return COLUMNS.length;
}
@Override
public String getColumnName(int column) {
return COLUMNS[column];
}
@Override
public Class<?> getColumnClass(int column) {
return column >= 3 ? Boolean.class : String.class;
}
@Override
public boolean isCellEditable(int row, int column) {
return column == 4 || (column == 3 && working.get(row).isServerScoped());
}
@Override
public Object getValueAt(int row, int column) {
Hotkey h = working.get(row);
return switch (column) {
case 0 -> h.path();
case 1 -> service.display(h.combo);
case 2 -> h.action.momentary() ? "While held" : h.trigger.label();
case 3 -> h.isServerScoped() && h.activeServerOnly;
default -> h.enabled;
};
}
@Override
public void setValueAt(Object value, int row, int column) {
Hotkey h = working.get(row);
if (column == 3) {
h.activeServerOnly = Boolean.TRUE.equals(value);
} else if (column == 4) {
h.enabled = Boolean.TRUE.equals(value);
}
}
}
}

View File

@@ -55,7 +55,7 @@ public final class Icons {
private static ImageIcon themed(String key, int size, Painter fallback) {
ImageIcon icon = IconTheme.icon(key, size);
return icon != null ? icon : make(fallback);
return icon != null ? icon : make(size, fallback);
}
/**
@@ -71,10 +71,16 @@ public final class Icons {
}
private static ImageIcon make(Painter p) {
BufferedImage img = new BufferedImage(SZ, SZ, BufferedImage.TYPE_INT_ARGB);
return make(SZ, p);
}
/** Runs a painter (which draws in a 16&times;16 box) scaled to {@code size}. */
private static ImageIcon make(int size, Painter p) {
BufferedImage img = new BufferedImage(size, size, BufferedImage.TYPE_INT_ARGB);
Graphics2D g = img.createGraphics();
g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);
g.setRenderingHint(RenderingHints.KEY_STROKE_CONTROL, RenderingHints.VALUE_STROKE_PURE);
g.scale(size / (double) SZ, size / (double) SZ);
p.paint(g);
g.dispose();
return new ImageIcon(img);
@@ -111,25 +117,37 @@ public final class Icons {
// ---- client status icons ----
public static ImageIcon clientIdle() {
return themed("PLAYER_OFF", g -> paintPerson(g, Theme.IDLE_CLIENT));
return clientIdle(SZ);
}
public static ImageIcon clientIdle(int size) {
return themed("PLAYER_OFF", size, g -> paintPerson(g, Theme.IDLE_CLIENT));
}
public static ImageIcon clientTalking() {
return themed("PLAYER_ON", g -> paintPerson(g, Theme.TALKING));
return clientTalking(SZ);
}
public static ImageIcon clientTalking(int size) {
return themed("PLAYER_ON", size, g -> paintPerson(g, Theme.TALKING));
}
public static ImageIcon clientAway() {
return themed("AWAY", g -> paintPerson(g, Theme.AWAY));
return clientAway(SZ);
}
public static ImageIcon clientAway(int size) {
return themed("AWAY", size, g -> paintPerson(g, Theme.AWAY));
}
/** A channel commander that is not talking. */
public static ImageIcon clientCommander() {
return themed("PLAYER_COMMANDER_OFF", g -> paintPerson(g, Theme.IDLE_CLIENT));
return clientCommander(SZ);
}
/** A channel commander that is talking. */
public static ImageIcon clientCommanderTalking() {
return themed("PLAYER_COMMANDER_ON", g -> paintPerson(g, Theme.TALKING));
return clientCommanderTalking(SZ);
}
public static ImageIcon clientQuery() {
@@ -137,11 +155,56 @@ public final class Icons {
}
public static ImageIcon micMuted() {
return themed("INPUT_MUTED", Icons::paintMicMuted);
return micMuted(SZ);
}
public static ImageIcon micMuted(int size) {
return themed("INPUT_MUTED", size, Icons::paintMicMuted);
}
public static ImageIcon speakerMuted() {
return themed("OUTPUT_MUTED", Icons::paintSpeakerMuted);
return speakerMuted(SZ);
}
public static ImageIcon speakerMuted(int size) {
return themed("OUTPUT_MUTED", size, Icons::paintSpeakerMuted);
}
/** The capture device is unavailable (another tab holds it), as opposed to muted. */
public static ImageIcon micDisabled() {
return micDisabled(SZ);
}
public static ImageIcon micDisabled(int size) {
return themed("HARDWARE_INPUT_MUTED", size, Icons::paintMicDisabled);
}
/** The playback device is unavailable, as opposed to muted/deafened. */
public static ImageIcon speakerDisabled() {
return speakerDisabled(SZ);
}
public static ImageIcon speakerDisabled(int size) {
return themed("HARDWARE_OUTPUT_MUTED", size, Icons::paintSpeakerDisabled);
}
/** TS3's "Local Mic Mute": silenced, but not reported as muted. */
public static ImageIcon micLocalMuted() {
return micLocalMuted(SZ);
}
public static ImageIcon micLocalMuted(int size) {
return themed("INPUT_MUTED_LOCAL", size, Icons::paintMicLocalMuted);
}
/** A channel commander that is not talking. */
public static ImageIcon clientCommander(int size) {
return themed("PLAYER_COMMANDER_OFF", size, g -> paintPerson(g, Theme.IDLE_CLIENT));
}
/** A channel commander that is talking. */
public static ImageIcon clientCommanderTalking(int size) {
return themed("PLAYER_COMMANDER_ON", size, g -> paintPerson(g, Theme.TALKING));
}
// ---- toolbar / action icons ----
@@ -190,7 +253,11 @@ public final class Icons {
}
public static ImageIcon app() {
return make(Icons::paintApp);
return app(SZ);
}
public static ImageIcon app(int size) {
return make(size, Icons::paintApp);
}
// ---- built-in painters ----
@@ -256,6 +323,35 @@ public final class Icons {
g.drawLine(2, 2, 14, 14);
}
/** Same shape as {@link #paintMicMuted}, but grey: unavailable, not muted by choice. */
private static void paintMicDisabled(Graphics2D g) {
g.setColor(Theme.IDLE_CLIENT);
g.fillRoundRect(6, 2, 4, 7, 2, 2);
g.setStroke(new BasicStroke(1.4f));
g.drawArc(4, 6, 8, 6, 200, 140);
g.drawLine(8, 12, 8, 14);
g.setStroke(new BasicStroke(2f));
g.drawLine(2, 2, 14, 14);
}
/** Same shape as {@link #paintSpeakerMuted}, but grey: unavailable, not muted by choice. */
private static void paintSpeakerDisabled(Graphics2D g) {
g.setColor(Theme.IDLE_CLIENT);
g.fillRect(2, 6, 3, 4);
int[] xs = {5, 9, 9, 5};
int[] ys = {6, 3, 13, 10};
g.fillPolygon(xs, ys, 4);
g.setStroke(new BasicStroke(2f));
g.drawLine(2, 2, 14, 14);
}
/** {@link #paintMicMuted}, marked with a small dot: silenced, but not reported as muted. */
private static void paintMicLocalMuted(Graphics2D g) {
paintMicMuted(g);
g.setColor(Theme.ACCENT);
g.fillOval(11, 10, 4, 4);
}
private static void paintConnect(Graphics2D g) {
g.setColor(new Color(0x2E8B57));
g.setStroke(new BasicStroke(2f));

View File

@@ -96,8 +96,10 @@ public final class InfoPanel extends JScrollPane {
if (!cl.platform.isEmpty()) row(sb, "Platform", esc(cl.platform));
if (!cl.version.isEmpty()) row(sb, "Version", esc(cl.version));
if (cl.away) row(sb, "Status", "Away");
if (cl.inputMuted) row(sb, "Microphone", "muted");
if (cl.outputMuted) row(sb, "Speakers", "muted");
if (!cl.inputHardware) row(sb, "Microphone", "disabled");
else if (cl.inputMuted) row(sb, "Microphone", "muted");
if (!cl.outputHardware) row(sb, "Speakers", "disabled");
else if (cl.outputMuted) row(sb, "Speakers", "muted");
if (!cl.uniqueId.isEmpty()) row(sb, "Unique ID", esc(cl.uniqueId));
if (cl.description != null && !cl.description.isEmpty()) {

View File

@@ -8,21 +8,28 @@ import java.awt.Graphics2D;
import java.awt.RenderingHints;
/**
* Horizontal audio level meter (dBFS) with an optional VAD threshold marker.
* The filled portion turns green once the level crosses the threshold, giving
* immediate visual feedback while tuning voice activation.
* Horizontal audio level meter with an optional VAD threshold marker. The filled portion
* turns green once the level crosses the threshold, giving immediate visual feedback while
* tuning voice activation.
*
* <p>Scaled to {@link com.ts3client.audio.InputLevel}, matching the TS3 client's slider.
*/
public final class LevelMeter extends JComponent {
private static final double MIN_DB = -70.0;
private static final double MAX_DB = 0.0;
private static final double MIN_DB = com.ts3client.audio.InputLevel.MIN_DB;
private static final double MAX_DB = com.ts3client.audio.InputLevel.MAX_DB;
private volatile double levelDb = MIN_DB;
private volatile double thresholdDb = -45.0;
private volatile double thresholdDb = -40.0;
private volatile boolean showThreshold = true;
private volatile boolean transmitting;
public LevelMeter() {
setPreferredSize(new Dimension(240, 18));
// A bare JComponent reports no minimum of its own, so a layout tight on space
// collapses the bar to nothing. Keep the height and let only the width give.
setMinimumSize(new Dimension(60, 18));
setMaximumSize(new Dimension(Integer.MAX_VALUE, 18));
}
public void setLevel(double db) {
@@ -40,6 +47,16 @@ public final class LevelMeter extends JComponent {
repaint();
}
/**
* Whether the gate is currently open. This is the real transmit decision, which in the
* Automatic and Hybrid modes depends on the speech detector and the hangover as well as
* on the level, so the bar cannot infer it from the threshold alone.
*/
public void setTransmitting(boolean transmitting) {
this.transmitting = transmitting;
repaint();
}
private int dbToX(double db, int w) {
double clamped = Math.max(MIN_DB, Math.min(MAX_DB, db));
return (int) ((clamped - MIN_DB) / (MAX_DB - MIN_DB) * w);
@@ -56,8 +73,7 @@ public final class LevelMeter extends JComponent {
g.fillRoundRect(0, 0, w - 1, h - 1, 6, 6);
int level = dbToX(levelDb, w - 2);
boolean over = levelDb >= thresholdDb;
g.setColor(over ? Theme.TALKING : new Color(0x5A9BD4));
g.setColor(transmitting ? Theme.TALKING : new Color(0x5A9BD4));
g.fillRoundRect(1, 1, Math.max(0, level), h - 3, 5, 5);
if (showThreshold) {

View File

@@ -7,6 +7,7 @@ import com.ts3client.config.Bookmark;
import com.ts3client.config.Bookmarks;
import com.ts3client.config.IdentityStore;
import com.ts3client.config.Settings;
import com.ts3client.net.ChannelNode;
import com.ts3client.sound.SoundNotifier;
import com.ts3client.sound.SoundPlayer;
@@ -31,9 +32,6 @@ import javax.swing.SwingUtilities;
import java.awt.BorderLayout;
import java.awt.Dimension;
import java.awt.GridLayout;
import java.awt.KeyEventDispatcher;
import java.awt.KeyboardFocusManager;
import java.awt.event.KeyEvent;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.util.ArrayList;
@@ -76,6 +74,9 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
private final JLabel statusLabel = new JLabel("Not connected");
private final JLabel codecLabel = new JLabel();
/** Mirrors the active server's own client state next to the clock. */
private TrayController tray;
private JToolBar toolbar;
private JButton connectButton;
private JButton disconnectButton;
@@ -86,6 +87,9 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
private boolean pttPressed;
/** Global hotkeys: the bindings, the matching engine and the platform input hook. */
private final HotkeyService hotkeys;
/** Guards {@link #shutdown()} so the window listener and JVM hook don't both run it. */
private final AtomicBoolean shuttingDown = new AtomicBoolean(false);
private final Thread shutdownHook = new Thread(this::shutdown, "ts3j-shutdown");
@@ -97,6 +101,7 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
this.sounds = new SoundNotifier(settings);
this.soundPlayer = audio.createSoundPlayer(settings);
this.sounds.setPlayer(soundPlayer);
this.hotkeys = new HotkeyService(new HotkeyActions(this), this::hotkeyArgumentChoices);
setIconImage(Icons.app().getImage());
// We tear the connections down ourselves on close, so don't let Swing kill the JVM.
@@ -104,14 +109,14 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
addWindowListener(new WindowAdapter() {
@Override
public void windowClosing(WindowEvent e) {
shutdown();
System.exit(0);
quit();
}
});
// Catches Ctrl+C / SIGTERM so we still leave the servers cleanly.
Runtime.getRuntime().addShutdownHook(shutdownHook);
setMinimumSize(new Dimension(720, 480));
tray = new TrayController(this, this::quit);
setJMenuBar(buildMenuBar());
toolbar = buildToolbar();
@@ -127,8 +132,8 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
selectTab(first);
first.chat().appendSystem("Welcome to the TS3J Swing client.");
first.chat().appendSystem("Use Connections → Connect to join a server.");
first.chat().appendSystem(tray.status());
installPushToTalk();
statusTimer = new javax.swing.Timer(3000, e -> updateConnectionStatus());
statusTimer.start();
setSize(880, 560);
@@ -158,10 +163,7 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
closeTab.setAccelerator(KeyStroke.getKeyStroke("control W"));
closeTab.addActionListener(e -> closeTab(selected));
JMenuItem quit = new JMenuItem("Quit", Icons.of("QUIT"));
quit.addActionListener(e -> {
shutdown();
System.exit(0);
});
quit.addActionListener(e -> quit());
connections.add(connect);
connections.add(disconnect);
connections.add(closeTab);
@@ -307,6 +309,7 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
add(toolbar, BorderLayout.NORTH);
updateToolbar();
refreshTabs();
if (tray != null) tray.refreshIcon();
revalidate();
repaint();
}
@@ -376,6 +379,7 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
ServerTab previous = micTab;
micTab = tab;
refreshTabs();
updateTray();
// Closing and reopening the capture line can block briefly; keep it off the EDT.
new Thread(() -> {
if (previous != null) previous.setMicrophoneActive(false);
@@ -411,6 +415,12 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
updateToolbar();
updateStatusLabel();
}
updateTray();
}
/** The local client on {@code tab} started or stopped talking, muted itself, … */
void selfStateChanged(ServerTab tab) {
if (tabs.contains(tab)) updateTray();
}
void tabConnected(ServerTab tab) {
@@ -427,26 +437,151 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
tabUpdated(tab);
}
// ---- push to talk ----
// ---- hotkeys ----
private void installPushToTalk() {
KeyboardFocusManager.getCurrentKeyboardFocusManager().addKeyEventDispatcher(new KeyEventDispatcher() {
@Override
public boolean dispatchKeyEvent(KeyEvent e) {
if (settings.inputMode != Settings.InputMode.PUSH_TO_TALK) return false;
ServerTab tab = micTab;
if (tab == null || !tab.isConnected() || tab.connection().getMicrophone() == null) return false;
if (e.getKeyCode() != settings.pushToTalkKey) return false;
if (e.getID() == KeyEvent.KEY_PRESSED && !pttPressed) {
pttPressed = true;
tab.connection().getMicrophone().setPushToTalk(true);
} else if (e.getID() == KeyEvent.KEY_RELEASED) {
pttPressed = false;
tab.connection().getMicrophone().setPushToTalk(false);
}
return false;
/**
* Which connections a fired hotkey reaches: with "on active server" ticked only the
* selected tab, otherwise every connected one.
*/
List<ServerTab> hotkeyTargets(boolean activeServerOnly) {
List<ServerTab> out = new ArrayList<>();
if (activeServerOnly) {
if (selected != null && selected.isConnected()) out.add(selected);
return out;
}
for (ServerTab tab : tabs) {
if (tab.isConnected()) out.add(tab);
}
return out;
}
ServerTab selectedTab() {
return selected;
}
List<ServerTab> allTabs() {
return new ArrayList<>(tabs);
}
/** Opens the microphone while a push-to-talk hotkey is held. */
void setPushToTalk(boolean talking) {
pttPressed = talking;
ServerTab tab = micTab;
if (tab == null || !tab.isConnected() || tab.connection().getMicrophone() == null) return;
tab.connection().getMicrophone().setPushToTalk(talking);
}
void moveMicrophoneToSelectedTab() {
if (selected != null && selected.isConnected()) setMicTab(selected);
updateToolbar();
}
/** Connects to the bookmark with this label; no label means the first one. */
void connectBookmark(String label, boolean newTab) {
Bookmark match = null;
for (Bookmark b : bookmarks.all()) {
if (label == null || label.isBlank() || label.equalsIgnoreCase(b.label)) {
match = b;
break;
}
});
}
if (match == null) return;
if (newTab) selectTab(newTab());
connectToBookmark(match);
}
/** Selects the tab with this 1-based number, as the "Select Server Tab" action names it. */
void selectTabNumber(int number) {
if (number >= 1 && number <= tabs.size()) selectTab(tabs.get(number - 1));
}
void stepTab(int delta) {
if (tabs.isEmpty()) return;
int index = Math.max(0, tabs.indexOf(selected));
selectTab(tabs.get(Math.floorMod(index + delta, tabs.size())));
}
void setSoundsMuted(boolean muted) {
sounds.setMuted(muted);
}
boolean areSoundsMuted() {
return sounds.isMuted();
}
void adjustMasterVolume(double delta) {
settings.outputVolume = Math.max(0, Math.min(2.0, settings.outputVolume + delta));
settings.save();
applyOutputSettingsToAllTabs();
updateStatusLabel();
}
/** Applies a new nickname, or asks for one when the hotkey carries none. */
void changeNickname(String nickname) {
if (nickname == null || nickname.isBlank()) {
changeNickname();
return;
}
settings.nickname = nickname.trim();
settings.save();
for (ServerTab tab : tabs) tab.setNickname(settings.nickname);
}
void browseCurrentChannel() {
if (selected == null || !selected.isConnected()) return;
ChannelNode channel = selected.currentChannel();
if (channel != null) selected.browseFiles(channel);
}
void reloadSkin() {
IconTheme.get().reload(settings);
}
void bringToFront() {
setVisible(true);
setExtendedState(getExtendedState() & ~JFrame.ICONIFIED);
toFront();
requestFocus();
}
void sendToBack() {
setExtendedState(getExtendedState() | JFrame.ICONIFIED);
}
/**
* The values a parameterised hotkey action can take, for the tree in the hotkey
* dialog: the saved bookmarks, the installed sound packs and the channels of the
* server on screen.
*/
private List<String> hotkeyArgumentChoices(com.ts3client.hotkey.HotkeyAction action) {
List<String> out = new ArrayList<>();
switch (action.argument()) {
case BOOKMARK -> {
for (Bookmark b : bookmarks.all()) {
if (b.label != null && !b.label.isBlank()) out.add(b.label);
}
}
case PROFILE -> {
if (action == com.ts3client.hotkey.HotkeyAction.SOUNDPACK_ACTIVATE) {
for (com.ts3client.sound.SoundPack pack : sounds.availablePacks()) out.add(pack.name());
}
}
case CHANNEL -> {
if (selected != null && selected.isConnected()) {
out.addAll(selected.channelPaths());
}
}
default -> {
}
}
return out;
}
/** Repaints the chrome after a hotkey changed the local client's state. */
void refreshAfterHotkey() {
updateToolbar();
updateTray();
refreshTabs();
}
// ---- actions ----
@@ -618,6 +753,14 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
tab.shutdown();
}
soundPlayer.shutdown();
hotkeys.dispose();
if (tray != null) tray.dispose();
}
/** Leaves every server and ends the process. */
private void quit() {
shutdown();
System.exit(0);
}
private void showIdentities() {
@@ -630,7 +773,7 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
SettingsDialog dlg = new SettingsDialog(this, settings,
micTab == null ? null : micTab.connection().getMicrophone(),
selected == null ? null : selected.connection().getPlayback(),
sounds,
sounds, hotkeys,
this::applyOutputSettingsToAllTabs);
dlg.setVisible(true);
}
@@ -690,6 +833,26 @@ public final class MainFrame extends JFrame implements ServerTabPane.Listener {
awayItem.setSelected(away);
awayButton.setSelected(away);
commanderItem.setSelected(connected && selected.isCommander());
updateTray();
}
/**
* The server the tray icon speaks for: the one holding the microphone, or the
* visible one when nobody is capturing.
*/
private ServerTab trayTab() {
if (micTab != null && micTab.isConnected()) return micTab;
return selected != null && selected.isConnected() ? selected : null;
}
private void updateTray() {
if (tray == null) return;
ServerTab tab = trayTab();
if (tab == null) {
tray.update(SelfState.DISCONNECTED, null);
} else {
tray.update(tab.selfState(), tab.title());
}
}
private void updateStatusLabel() {

View File

@@ -0,0 +1,181 @@
package com.ts3client.ui;
import com.ts3client.audio.VoiceInput;
import com.ts3client.audio.desktop.AudioDevices;
import com.ts3client.audio.desktop.AudioPlayback;
import com.ts3client.audio.desktop.DesktopVoiceInput;
import com.ts3client.config.Settings;
import javax.swing.SwingUtilities;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.function.Consumer;
/**
* Drives the settings dialog's microphone test from a real capture chain.
*
* <p>The dialog runs its own {@link DesktopVoiceInput} rather than borrowing the connected
* one, whose listeners belong to the connection. Because it is the same class that feeds
* the server, the level and the gate shown here are exactly what would be transmitted —
* pre-processing, voice detection, hangover and pre-roll included.
*
* <p>Loopback is optional: when enabled, the frames that would be sent are played back
* locally so you can hear precisely what the other side would.
*/
final class MicrophoneTest {
/** Frames buffered for loopback before the oldest is dropped (~100 ms). */
private static final int LOOPBACK_QUEUE_FRAMES = 5;
private final Consumer<Double> onLevel;
private final Consumer<Boolean> onTransmitting;
private DesktopVoiceInput mic;
private final ArrayBlockingQueue<byte[]> loopbackQueue =
new ArrayBlockingQueue<>(LOOPBACK_QUEUE_FRAMES);
private volatile boolean loopbackEnabled;
private volatile boolean loopbackRunning;
private Thread loopbackThread;
private String outputDevice = "";
MicrophoneTest(Consumer<Double> onLevel, Consumer<Boolean> onTransmitting) {
this.onLevel = onLevel;
this.onTransmitting = onTransmitting;
}
/** Whether a capture chain is currently running. */
boolean isRunning() {
return mic != null;
}
/**
* (Re)starts the test chain against {@code settings}' devices and voice options.
*
* @return false if the capture device could not be opened
*/
boolean start(Settings settings) {
stop();
DesktopVoiceInput input = new DesktopVoiceInput(settings);
input.setLevelListener(db -> SwingUtilities.invokeLater(() -> onLevel.accept(db)));
input.setTalkListener(talking -> SwingUtilities.invokeLater(() -> onTransmitting.accept(talking)));
input.setMonitorListener(this::enqueueForLoopback);
this.outputDevice = settings.outputDevice;
try {
input.start();
} catch (RuntimeException e) {
return false; // Device busy or gone; leave the test switched off.
}
this.mic = input;
return true;
}
/**
* Stops the capture chain. The caller owns the UI reset: doing it here would race with
* a restart, whose own state has already been put on screen.
*/
void stop() {
setLoopback(false);
if (mic != null) {
mic.stop();
mic = null;
}
}
/** Applies a change to the test chain, if it is running. */
void configure(Consumer<VoiceInput> change) {
VoiceInput m = mic;
if (m != null) change.accept(m);
}
/** The playback device to loop back through; takes effect on the next enable. */
void setOutputDevice(String deviceId) {
this.outputDevice = deviceId == null ? "" : deviceId;
}
void setLoopback(boolean enabled) {
if (enabled == loopbackEnabled) {
return;
}
loopbackEnabled = enabled;
if (enabled) {
loopbackRunning = true;
loopbackThread = new Thread(this::loopbackLoop, "settings-loopback");
loopbackThread.setDaemon(true);
loopbackThread.start();
} else {
loopbackRunning = false;
if (loopbackThread != null) {
loopbackThread.interrupt();
loopbackThread = null;
}
loopbackQueue.clear();
}
}
/**
* Converts a transmitted frame to 16-bit PCM and queues it. Runs on the capture thread,
* so it must not block: a full queue means playback has fallen behind and the oldest
* frame is dropped instead.
*/
private void enqueueForLoopback(float[] interleaved, int channels) {
if (!loopbackEnabled) {
return;
}
byte[] pcm = toPcm16(interleaved);
if (!loopbackQueue.offer(pcm)) {
loopbackQueue.poll();
loopbackQueue.offer(pcm);
}
}
private void loopbackLoop() {
AudioPlayback line = null;
try {
// The monitored frames are mono for voice; ask for a matching line so no
// channel juggling is needed, and up-mix only if the device insists on stereo.
line = AudioDevices.openPlayback(outputDevice, 1);
line.start();
int channels = line.channels();
while (loopbackRunning) {
byte[] frame = loopbackQueue.poll(100, java.util.concurrent.TimeUnit.MILLISECONDS);
if (frame == null) {
continue;
}
byte[] out = (channels == 1) ? frame : upmix(frame, channels);
line.write(out, 0, out.length);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (Exception ignored) {
// The device may be busy or gone; the test simply runs without loopback.
} finally {
if (line != null) line.close();
}
}
/** Converts float samples in [-1, 1] to 16-bit little-endian PCM. */
static byte[] toPcm16(float[] samples) {
byte[] pcm = new byte[samples.length * 2];
for (int i = 0; i < samples.length; i++) {
float f = samples[i];
if (f > 1f) f = 1f;
else if (f < -1f) f = -1f;
int s = Math.round(f * 32767f);
pcm[2 * i] = (byte) (s & 0xFF);
pcm[2 * i + 1] = (byte) ((s >> 8) & 0xFF);
}
return pcm;
}
/** Copies a mono frame across {@code channels} interleaved channels. */
static byte[] upmix(byte[] mono, int channels) {
byte[] out = new byte[mono.length * channels];
for (int i = 0, frames = mono.length / 2; i < frames; i++) {
for (int c = 0; c < channels; c++) {
out[2 * (i * channels + c)] = mono[2 * i];
out[2 * (i * channels + c) + 1] = mono[2 * i + 1];
}
}
return out;
}
}

View File

@@ -0,0 +1,55 @@
package com.ts3client.ui;
import javax.swing.ImageIcon;
/**
* The state of the local client on one server, as the tree shows it for any other
* client — the states are listed in the order the official client gives them
* priority, the strongest one first.
*/
enum SelfState {
DISCONNECTED("Not connected"),
DEAFENED("Speakers muted"),
MIC_MUTED("Microphone muted"),
MIC_LOCAL_MUTED("Microphone locally muted"),
AWAY("Away"),
COMMANDER_TALKING("Talking (channel commander)"),
COMMANDER("Channel commander"),
TALKING("Talking"),
IDLE("Connected");
private final String label;
SelfState(String label) {
this.label = label;
}
String label() {
return label;
}
/** The icon this state is drawn with, from the active icon pack. */
ImageIcon icon(int size) {
switch (this) {
case DISCONNECTED:
return Icons.app(size);
case DEAFENED:
return Icons.speakerMuted(size);
case MIC_MUTED:
return Icons.micMuted(size);
case MIC_LOCAL_MUTED:
return Icons.micLocalMuted(size);
case AWAY:
return Icons.clientAway(size);
case COMMANDER_TALKING:
return Icons.clientCommanderTalking(size);
case COMMANDER:
return Icons.clientCommander(size);
case TALKING:
return Icons.clientTalking(size);
default:
return Icons.clientIdle(size);
}
}
}

View File

@@ -15,6 +15,8 @@ import javax.swing.JComponent;
import javax.swing.JOptionPane;
import javax.swing.JSplitPane;
import javax.swing.SwingUtilities;
import java.util.ArrayList;
import java.util.List;
import java.awt.Component;
/**
@@ -47,6 +49,7 @@ final class ServerTab implements ConnectionListener, ServerTreePanel.Actions {
private String identityId = "";
private boolean micMuted;
private boolean micLocalMuted;
private boolean deafened;
private boolean away;
/** Away message currently published, empty when away carries no message. */
@@ -137,6 +140,10 @@ final class ServerTab implements ConnectionListener, ServerTreePanel.Actions {
return micMuted;
}
boolean isMicLocalMuted() {
return micLocalMuted;
}
boolean isDeafened() {
return deafened;
}
@@ -153,6 +160,19 @@ final class ServerTab implements ConnectionListener, ServerTreePanel.Actions {
return commander;
}
/** What the local client looks like on this server, for the tray icon. */
SelfState selfState() {
if (!conn.isConnected()) return SelfState.DISCONNECTED;
if (deafened) return SelfState.DEAFENED;
if (micMuted) return SelfState.MIC_MUTED;
if (micLocalMuted) return SelfState.MIC_LOCAL_MUTED;
if (away) return SelfState.AWAY;
ClientEntry self = conn.getModel().getClient(conn.getSelfClientId());
boolean talking = self != null && self.talking;
if (commander) return talking ? SelfState.COMMANDER_TALKING : SelfState.COMMANDER;
return talking ? SelfState.TALKING : SelfState.IDLE;
}
/** Path of the channel we are in, or empty when not connected. */
String currentChannelPath() {
if (!conn.isConnected()) return "";
@@ -217,6 +237,13 @@ final class ServerTab implements ConnectionListener, ServerTreePanel.Actions {
chatPanel.appendSystem(muted ? "Microphone muted." : "Microphone active.");
}
/** TS3's "Local Mic Mute": silences capture without publishing a status change. */
void setMicLocalMuted(boolean muted) {
micLocalMuted = muted;
conn.setMicLocalMuted(muted);
chatPanel.appendSystem(muted ? "Microphone locally muted." : "Microphone locally unmuted.");
}
void setDeafened(boolean deaf) {
deafened = deaf;
conn.setDeafened(deaf);
@@ -307,6 +334,33 @@ final class ServerTab implements ConnectionListener, ServerTreePanel.Actions {
return c != null ? c.nickname : "Client " + clientId;
}
/** Joins the channel at a "/"-separated path, as the hotkey action names it. */
void joinChannelPath(String path) {
if (!conn.isConnected() || path == null || path.isBlank()) return;
ChannelNode target = conn.getModel().findChannelByPath(path);
if (target != null) conn.joinChannel(target.id, null);
}
/** Every channel of this server as a "/"-separated path, in the tree's own order. */
List<String> channelPaths() {
List<String> out = new ArrayList<>();
if (!conn.isConnected()) return out;
for (ChannelNode root : conn.getModel().buildTree()) collectPaths(root, out);
return out;
}
private void collectPaths(ChannelNode channel, List<String> out) {
out.add(conn.getModel().channelPath(channel.id));
for (ChannelNode child : channel.children) collectPaths(child, out);
}
/** The channel we are in, or null when not connected. */
ChannelNode currentChannel() {
if (!conn.isConnected()) return null;
ClientEntry self = conn.getModel().getClient(conn.getSelfClientId());
return self == null ? null : conn.getModel().getChannel(self.channelId);
}
// ---- ServerTreePanel.Actions ----
@Override
@@ -443,6 +497,7 @@ final class ServerTab implements ConnectionListener, ServerTreePanel.Actions {
connecting = false;
treePanel.setSelfClientId(conn.getSelfClientId());
micMuted = false;
micLocalMuted = false;
deafened = false;
away = false;
awayMessage = "";
@@ -511,7 +566,10 @@ final class ServerTab implements ConnectionListener, ServerTreePanel.Actions {
@Override
public void onTalkStateChanged(int clientId, boolean talking) {
SwingUtilities.invokeLater(treePanel::refreshVisual);
SwingUtilities.invokeLater(() -> {
treePanel.refreshVisual();
if (clientId == conn.getSelfClientId()) host.selfStateChanged(this);
});
}
@Override

View File

@@ -650,7 +650,9 @@ public final class ServerTreePanel extends JScrollPane {
/** The client's state, in the order the official client gives them priority. */
private ImageIcon iconFor(ClientEntry cl) {
if (cl.isQuery()) return Icons.clientQuery();
if (!cl.outputHardware) return Icons.speakerDisabled();
if (cl.outputMuted) return Icons.speakerMuted();
if (!cl.inputHardware) return Icons.micDisabled();
if (cl.inputMuted) return Icons.micMuted();
if (cl.away) return Icons.clientAway();
if (cl.channelCommander) {

View File

@@ -1,11 +1,13 @@
package com.ts3client.ui;
import com.ts3client.audio.InputLevel;
import com.ts3client.audio.OpusParameters;
import com.ts3client.audio.VoiceInput;
import com.ts3client.audio.VoiceOutput;
import com.ts3client.audio.desktop.AudioCapture;
import com.ts3client.audio.desktop.AudioDevices;
import com.ts3client.config.Settings;
import com.ts3client.hotkey.Hotkey;
import com.ts3client.hotkey.HotkeyAction;
import com.ts3client.sound.SoundNotifier;
import javax.swing.BorderFactory;
@@ -21,6 +23,7 @@ import javax.swing.JPanel;
import javax.swing.JRadioButton;
import javax.swing.JSlider;
import javax.swing.JTabbedPane;
import javax.swing.JToggleButton;
import javax.swing.Scrollable;
import javax.swing.SwingUtilities;
import java.awt.BorderLayout;
@@ -30,8 +33,6 @@ import java.awt.GridBagConstraints;
import java.awt.GridBagLayout;
import java.awt.Insets;
import java.awt.Rectangle;
import java.awt.event.KeyAdapter;
import java.awt.event.KeyEvent;
import java.util.List;
/**
@@ -53,10 +54,13 @@ public final class SettingsDialog extends JDialog {
private final VoiceInput liveMic;
private final VoiceOutput livePlayback;
private final SoundNotifier sounds;
private final HotkeyService hotkeys;
private final Runnable onApply;
private NotificationsPanel notificationsPanel;
private IconPackPanel iconPackPanel;
private ClientVersionPanel clientVersionPanel;
private HotkeysPanel hotkeysPanel;
private JComboBox<AudioDevices.Device> inputCombo;
private JComboBox<AudioDevices.Device> outputCombo;
@@ -77,8 +81,10 @@ public final class SettingsDialog extends JDialog {
private JLabel thresholdLabel;
private JLabel speechLabel;
private LevelMeter meter;
private JToggleButton testButton;
private JCheckBox loopbackCheck;
private JLabel talkIndicator;
private JButton pttKeyButton;
private int pttKey;
private JSlider bitrateSlider;
private JLabel bitrateLabel;
private JSlider complexitySlider;
@@ -86,19 +92,18 @@ public final class SettingsDialog extends JDialog {
private JCheckBox fecCheck;
private JCheckBox musicCheck;
private volatile boolean meterRunning;
private Thread meterThread;
private final MicrophoneTest micTest = new MicrophoneTest(this::onTestLevel, this::onTestTalking);
public SettingsDialog(Frame owner, Settings settings,
VoiceInput liveMic, VoiceOutput livePlayback,
SoundNotifier sounds, Runnable onApply) {
SoundNotifier sounds, HotkeyService hotkeys, Runnable onApply) {
super(owner, "Options", true);
this.settings = settings;
this.liveMic = liveMic;
this.livePlayback = livePlayback;
this.sounds = sounds;
this.onApply = onApply;
this.pttKey = settings.pushToTalkKey;
this.hotkeys = hotkeys;
JTabbedPane tabs = new JTabbedPane();
tabs.addTab("Playback / Capture", scrollable(buildDevicesTab()));
@@ -107,6 +112,10 @@ public final class SettingsDialog extends JDialog {
tabs.addTab("Notifications", notificationsPanel);
iconPackPanel = new IconPackPanel(settings);
tabs.addTab("Design", iconPackPanel);
hotkeysPanel = new HotkeysPanel(hotkeys);
tabs.addTab("Hotkeys", hotkeysPanel);
clientVersionPanel = new ClientVersionPanel(settings);
tabs.addTab("Client Version", scrollable(clientVersionPanel));
JPanel buttons = new JPanel(new BorderLayout());
JPanel right = new JPanel();
@@ -130,7 +139,7 @@ public final class SettingsDialog extends JDialog {
addWindowListener(new java.awt.event.WindowAdapter() {
@Override
public void windowClosed(java.awt.event.WindowEvent e) {
stopMeter();
micTest.stop();
}
});
@@ -138,7 +147,6 @@ public final class SettingsDialog extends JDialog {
setSize(new Dimension(480, 540));
setMinimumSize(new Dimension(420, 360));
setLocationRelativeTo(owner);
startMeter();
}
private JPanel buildDevicesTab() {
@@ -175,10 +183,10 @@ public final class SettingsDialog extends JDialog {
outputVol.addChangeListener(e -> {
if (livePlayback != null) livePlayback.setMasterVolume(outputVol.getValue() / 100.0);
});
inputGain.addChangeListener(e -> {
if (liveMic != null) liveMic.setInputGain(inputGain.getValue() / 100.0);
});
inputCombo.addActionListener(e -> restartMeter());
inputGain.addChangeListener(e ->
applyLive(m -> m.setInputGain(inputGain.getValue() / 100.0)));
inputCombo.addActionListener(e -> restartTest());
outputCombo.addActionListener(e -> micTest.setOutputDevice(comboValue(outputCombo)));
c.gridx = 0;
c.gridy = row++;
@@ -217,19 +225,18 @@ public final class SettingsDialog extends JDialog {
Runnable syncNoise = () -> {
denoiseLevel.setEnabled(denoiseCheck.isSelected());
if (liveMic != null) {
liveMic.setNoiseSuppression(denoiseCheck.isSelected());
liveMic.setDenoiserLevel(denoiseLevel.getValue() / 100.0);
liveMic.setTypingAttenuation(typingCheck.isSelected());
liveMic.setAgc(agcCheck.isSelected());
}
applyLive(m -> {
m.setNoiseSuppression(denoiseCheck.isSelected());
m.setDenoiserLevel(denoiseLevel.getValue() / 100.0);
m.setTypingAttenuation(typingCheck.isSelected());
m.setAgc(agcCheck.isSelected());
});
};
denoiseCheck.addActionListener(e -> syncNoise.run());
typingCheck.addActionListener(e -> syncNoise.run());
agcCheck.addActionListener(e -> syncNoise.run());
denoiseLevel.addChangeListener(e -> {
if (liveMic != null) liveMic.setDenoiserLevel(denoiseLevel.getValue() / 100.0);
});
denoiseLevel.addChangeListener(e ->
applyLive(m -> m.setDenoiserLevel(denoiseLevel.getValue() / 100.0)));
syncNoise.run();
// filler
@@ -279,26 +286,30 @@ public final class SettingsDialog extends JDialog {
c.gridy = row;
c.gridwidth = 2;
c.insets = new Insets(10, 4, 2, 4);
p.add(new JLabel("Input level (speak to test):"), c);
p.add(new JLabel("Input level:"), c);
c.gridy = ++row;
p.add(meter, c);
c.insets = new Insets(4, 4, 4, 4);
c.gridy = ++row;
p.add(buildTestControls(), c);
c.gridwidth = 1;
row++;
vadModeCombo = new JComboBox<>(new String[]{"Automatic", "Volume Gate", "Hybrid"});
vadModeCombo.setSelectedIndex(vadModeIndex(settings.vadMode));
vadModeCombo.addActionListener(e -> micTest.configure(m -> m.setVadMode(currentVadMode())));
vadModeCombo.setToolTipText("<html><b>Automatic</b>: intelligent speech detection.<br>"
+ "<b>Volume Gate</b>: transmit when loud enough.<br>"
+ "<b>Hybrid</b>: loud enough <i>and</i> detected as speech.</html>");
addRow(p, c, row++, new JLabel("Detection:"), vadModeCombo);
thresholdSlider = new JSlider(-70, 0, (int) Math.round(settings.vadThresholdDb));
thresholdSlider = new JSlider((int) InputLevel.MIN_DB, (int) InputLevel.MAX_DB,
(int) Math.round(settings.vadThresholdDb));
thresholdLabel = new JLabel(Math.round(settings.vadThresholdDb) + " dB");
thresholdSlider.addChangeListener(e -> {
meter.setThreshold(thresholdSlider.getValue());
thresholdLabel.setText(thresholdSlider.getValue() + " dB");
if (liveMic != null) liveMic.setThresholdDb(thresholdSlider.getValue());
applyLive(m -> m.setThresholdDb(thresholdSlider.getValue()));
});
addRow(p, c, row++, new JLabel("Volume gate:"), sliderWithLabel(thresholdSlider, thresholdLabel));
@@ -306,15 +317,18 @@ public final class SettingsDialog extends JDialog {
speechLabel = new JLabel(Math.round(settings.speechThreshold * 100) + "%");
speechSlider.addChangeListener(e -> {
speechLabel.setText(speechSlider.getValue() + "%");
if (liveMic != null) liveMic.setSpeechThreshold(speechSlider.getValue() / 100.0);
applyLive(m -> m.setSpeechThreshold(speechSlider.getValue() / 100.0));
});
addRow(p, c, row++, new JLabel("Speech threshold:"), sliderWithLabel(speechSlider, speechLabel));
pttKeyButton = new JButton(keyName(pttKey));
pttKeyButton.addActionListener(e -> capturePttKey());
addRow(p, c, row++, new JLabel("Push-to-talk key:"), pttKeyButton);
pttKeyButton = new JButton(pushToTalkHotkeyText());
pttKeyButton.setToolTipText("Push-to-talk is a global hotkey; this opens its binding");
pttKeyButton.addActionListener(e -> editPushToTalkHotkey());
addRow(p, c, row++, new JLabel("Push-to-talk hotkey:"), pttKeyButton);
vadOverPttCheck = new JCheckBox("Also detect voice while push-to-talk", settings.vadOverPtt);
vadOverPttCheck.addActionListener(e ->
micTest.configure(m -> m.setVadOverPtt(vadOverPttCheck.isSelected())));
c.gridx = 0;
c.gridy = row++;
c.gridwidth = 2;
@@ -508,64 +522,94 @@ public final class SettingsDialog extends JDialog {
if (liveMic != null) liveMic.setOpusParameters(currentOpusParameters());
}
private void capturePttKey() {
pttKeyButton.setText("Press a key…");
pttKeyButton.requestFocusInWindow();
KeyAdapter ka = new KeyAdapter() {
@Override
public void keyPressed(KeyEvent e) {
pttKey = e.getKeyCode();
pttKeyButton.setText(keyName(pttKey));
pttKeyButton.removeKeyListener(this);
}
};
pttKeyButton.addKeyListener(ka);
private String pushToTalkHotkeyText() {
Hotkey ptt = hotkeys.find(HotkeyAction.PTT_ACTIVATE);
return ptt == null ? "No hotkey assigned" : hotkeys.display(ptt.combo);
}
private static String keyName(int code) {
String t = KeyEvent.getKeyText(code);
return (t == null || t.isEmpty()) ? ("Key " + code) : t;
/**
* Push-to-talk is an ordinary hotkey, so this shortcut edits that binding — adding
* it when there is none — rather than keeping a key of its own.
*/
private void editPushToTalkHotkey() {
Hotkey existing = hotkeys.find(HotkeyAction.PTT_ACTIVATE);
HotkeyDialog dlg = new HotkeyDialog(this, hotkeys,
existing == null ? new Hotkey(HotkeyAction.PTT_ACTIVATE, null) : existing);
dlg.setVisible(true);
if (!dlg.isConfirmed()) return;
List<Hotkey> updated = new java.util.ArrayList<>();
synchronized (hotkeys.all()) {
for (Hotkey h : hotkeys.all()) {
if (h != existing) updated.add(h.copy());
}
}
updated.add(dlg.result());
hotkeys.replaceAll(updated);
pttKeyButton.setText(pushToTalkHotkeyText());
hotkeysPanel.reload();
}
/** Copies the audio form into {@code target}, without touching anything else. */
private void writeAudioSettings(Settings target) {
target.inputDevice = comboValue(inputCombo);
target.outputDevice = comboValue(outputCombo);
target.inputVolume = inputGain.getValue() / 100.0;
target.outputVolume = outputVol.getValue() / 100.0;
target.denoise = denoiseCheck.isSelected();
target.denoiserLevel = denoiseLevel.getValue() / 100.0;
target.typingAttenuation = typingCheck.isSelected();
target.agc = agcCheck.isSelected();
target.inputMode = pttRadio.isSelected() ? Settings.InputMode.PUSH_TO_TALK
: contRadio.isSelected() ? Settings.InputMode.CONTINUOUS
: Settings.InputMode.VOICE_ACTIVATION;
target.vadMode = currentVadMode();
target.vadThresholdDb = thresholdSlider.getValue();
target.speechThreshold = speechSlider.getValue() / 100.0;
target.vadOverPtt = vadOverPttCheck.isSelected();
target.bitrate = bitrateSlider.getValue() * 1000;
target.complexity = complexitySlider.getValue();
target.vbr = vbrCheck.isSelected();
target.fec = fecCheck.isSelected();
target.music = musicCheck.isSelected();
}
/**
* The audio form as a standalone {@link Settings}, so the test chain runs with the
* values currently on screen rather than the ones last saved.
*
* <p>The test always runs voice activation: it exists to tune the gate, and push-to-talk
* would need the global hotkey, which belongs to the connected microphone.
*/
private Settings audioSnapshot() {
Settings snapshot = new Settings();
writeAudioSettings(snapshot);
if (snapshot.inputMode == Settings.InputMode.PUSH_TO_TALK) {
snapshot.inputMode = Settings.InputMode.VOICE_ACTIVATION;
}
return snapshot;
}
private void apply() {
settings.inputDevice = comboValue(inputCombo);
settings.outputDevice = comboValue(outputCombo);
settings.inputVolume = inputGain.getValue() / 100.0;
settings.outputVolume = outputVol.getValue() / 100.0;
settings.denoise = denoiseCheck.isSelected();
settings.denoiserLevel = denoiseLevel.getValue() / 100.0;
settings.typingAttenuation = typingCheck.isSelected();
settings.agc = agcCheck.isSelected();
settings.inputMode = pttRadio.isSelected() ? Settings.InputMode.PUSH_TO_TALK
: contRadio.isSelected() ? Settings.InputMode.CONTINUOUS
: Settings.InputMode.VOICE_ACTIVATION;
settings.vadMode = currentVadMode();
settings.vadThresholdDb = thresholdSlider.getValue();
settings.speechThreshold = speechSlider.getValue() / 100.0;
settings.vadOverPtt = vadOverPttCheck.isSelected();
settings.pushToTalkKey = pttKey;
settings.bitrate = bitrateSlider.getValue() * 1000;
settings.complexity = complexitySlider.getValue();
settings.vbr = vbrCheck.isSelected();
settings.fec = fecCheck.isSelected();
settings.music = musicCheck.isSelected();
writeAudioSettings(settings);
notificationsPanel.apply();
iconPackPanel.apply();
clientVersionPanel.apply();
hotkeysPanel.apply();
settings.save();
if (liveMic != null) {
liveMic.setMode(settings.inputMode);
liveMic.setVadMode(settings.vadMode);
liveMic.setThresholdDb(settings.vadThresholdDb);
liveMic.setSpeechThreshold(settings.speechThreshold);
liveMic.setVadOverPtt(settings.vadOverPtt);
liveMic.setInputGain(settings.inputVolume);
liveMic.setNoiseSuppression(settings.denoise);
liveMic.setDenoiserLevel(settings.denoiserLevel);
liveMic.setTypingAttenuation(settings.typingAttenuation);
liveMic.setAgc(settings.agc);
liveMic.setOpusParameters(OpusParameters.from(settings));
}
applyLive(m -> {
m.setMode(settings.inputMode);
m.setVadMode(settings.vadMode);
m.setThresholdDb(settings.vadThresholdDb);
m.setSpeechThreshold(settings.speechThreshold);
m.setVadOverPtt(settings.vadOverPtt);
m.setInputGain(settings.inputVolume);
m.setNoiseSuppression(settings.denoise);
m.setDenoiserLevel(settings.denoiserLevel);
m.setTypingAttenuation(settings.typingAttenuation);
m.setAgc(settings.agc);
m.setOpusParameters(OpusParameters.from(settings));
});
if (livePlayback != null) {
livePlayback.setMasterVolume(settings.outputVolume);
livePlayback.setOutputDevice(settings.outputDevice);
@@ -581,64 +625,91 @@ public final class SettingsDialog extends JDialog {
}
private void close() {
stopMeter();
micTest.stop();
dispose();
}
// ---- live meter ----
// ---- microphone test ----
private void startMeter() {
meterRunning = true;
meterThread = new Thread(this::meterLoop, "settings-meter");
meterThread.setDaemon(true);
meterThread.start();
/**
* The test row: a toggle that runs the capture chain, an indicator showing whether the
* gate is open, and an optional loopback so you can hear what is being sent.
*/
private JPanel buildTestControls() {
JPanel row = new JPanel(new java.awt.FlowLayout(java.awt.FlowLayout.LEFT, 8, 0));
testButton = new JToggleButton("Begin Test");
testButton.setToolTipText("Run the capture chain exactly as it runs while connected, "
+ "so the bar and the indicator show what would actually be transmitted.");
testButton.addActionListener(e -> setTesting(testButton.isSelected()));
loopbackCheck = new JCheckBox("Hear myself");
loopbackCheck.setToolTipText("Play the transmitted audio back through the playback "
+ "device. Use headphones to avoid feedback.");
loopbackCheck.setEnabled(false);
loopbackCheck.addActionListener(e -> micTest.setLoopback(loopbackCheck.isSelected()));
talkIndicator = new JLabel("Not transmitting", Icons.clientIdle(), JLabel.LEFT);
talkIndicator.setToolTipText("Lights up while your microphone is open, exactly as "
+ "other users would see you in the channel list.");
row.add(testButton);
row.add(loopbackCheck);
row.add(talkIndicator);
return row;
}
private void restartMeter() {
stopMeter();
startMeter();
private void setTesting(boolean on) {
if (on && !micTest.start(audioSnapshot())) {
testButton.setSelected(false);
testButton.setText("Begin Test");
loopbackCheck.setEnabled(false);
resetTestIndicators();
talkIndicator.setText("Capture device unavailable");
return;
}
if (!on) {
micTest.stop();
loopbackCheck.setSelected(false);
resetTestIndicators();
}
testButton.setText(on ? "Stop Test" : "Begin Test");
loopbackCheck.setEnabled(on);
}
private void stopMeter() {
meterRunning = false;
if (meterThread != null) {
meterThread.interrupt();
meterThread = null;
private void resetTestIndicators() {
onTestTalking(false);
onTestLevel(InputLevel.SILENCE_DB);
}
/** Restarts the test chain, if running, so a device change takes effect. */
private void restartTest() {
if (micTest.isRunning()) {
boolean loopback = loopbackCheck.isSelected();
if (micTest.start(audioSnapshot())) {
micTest.setLoopback(loopback);
} else {
setTesting(false);
testButton.setSelected(false);
}
}
}
private void meterLoop() {
String device = comboValue(inputCombo);
AudioCapture line = null;
try {
line = AudioDevices.openCapture(device, AudioDevices.MAX_CHANNELS);
line.start();
int frame = AudioDevices.FRAME_SIZE;
int channels = line.channels();
byte[] buf = new byte[frame * 2 * channels];
double gain = (inputGain != null ? inputGain.getValue() / 100.0 : 1.0);
while (meterRunning) {
if (line.read(buf, 0, buf.length) < buf.length) break;
// Metering follows the capture chain: level is taken off the downmix.
double sumSq = 0;
for (int i = 0; i < frame; i++) {
double mono = 0;
for (int c = 0; c < channels; c++) {
int k = i * channels + c;
short s = (short) ((buf[2 * k + 1] << 8) | (buf[2 * k] & 0xFF));
mono += s / 32768.0;
}
double f = mono / channels * gain;
sumSq += f * f;
}
double rms = Math.sqrt(sumSq / frame);
double db = rms <= 1e-9 ? -100 : 20 * Math.log10(rms);
final double fdb = db;
if (meter != null) SwingUtilities.invokeLater(() -> meter.setLevel(fdb));
}
} catch (Exception ignored) {
} finally {
if (line != null) line.close();
/** Applies a live change to the connected microphone and to the test one alike. */
private void applyLive(java.util.function.Consumer<VoiceInput> change) {
if (liveMic != null) change.accept(liveMic);
micTest.configure(change);
}
private void onTestLevel(double db) {
if (meter != null) meter.setLevel(db);
}
private void onTestTalking(boolean talking) {
if (meter != null) meter.setTransmitting(talking);
if (talkIndicator != null) {
talkIndicator.setIcon(talking ? Icons.clientTalking() : Icons.clientIdle());
talkIndicator.setText(talking ? "Transmitting" : "Not transmitting");
}
}

View File

@@ -0,0 +1,231 @@
package com.ts3client.ui;
import com.ts3client.ui.tray.AwtTray;
import com.ts3client.ui.tray.TrayBackend;
import com.ts3client.ui.tray.X11Tray;
import javax.swing.JComponent;
import javax.swing.JDialog;
import javax.swing.JMenuItem;
import javax.swing.JPopupMenu;
import javax.swing.KeyStroke;
import javax.swing.SwingUtilities;
import javax.swing.event.PopupMenuEvent;
import javax.swing.event.PopupMenuListener;
import java.awt.Frame;
import java.awt.Image;
import java.awt.MenuItem;
import java.awt.PopupMenu;
import java.awt.Window;
import java.awt.event.KeyEvent;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.util.function.IntFunction;
/**
* The system tray icon. It shows the local client's state on the active server —
* the same badge the tree draws next to a nickname, so it turns green while
* speaking and into the muted microphone or speaker when muted — and brings the
* window back to the front when clicked.
*
* <p>The icon is docked by {@link X11Tray} where the panel can show a transparent
* one, and by {@link AwtTray} everywhere else. On a desktop with no tray at all
* nothing is installed and every call here is a no-op.
*/
final class TrayController {
private final MainFrame frame;
private final Runnable onQuit;
private final TrayBackend tray;
/** What is on screen now, so repeated updates don't touch the tray. */
private SelfState state;
private String tooltip = "";
/** The context menu while it is on screen, and the window it hangs off. */
private JPopupMenu openMenu;
private JDialog openAnchor;
TrayController(MainFrame frame, Runnable onQuit) {
this.frame = frame;
this.onQuit = onQuit;
this.tray = install();
}
/** What happened when the icon was installed, for the diagnostics in the chat log. */
String status() {
return tray == null
? "System tray: no icon could be installed (run with -Dts3j.tray.debug=true for why)."
: "System tray: " + tray.description() + ".";
}
/** @return the backend that took the icon, or {@code null} when none could */
private TrayBackend install() {
TrayBackend.Listener listener = new TrayBackend.Listener() {
@Override
public void activated() {
SwingUtilities.invokeLater(TrayController.this::show);
}
@Override
public void menuRequested(int x, int y) {
SwingUtilities.invokeLater(() -> showMenu(x, y));
}
};
// -Dts3j.tray=awt|x11 pins the backend; by default the transparent one is
// tried first and AWT's picks up whatever it leaves.
String choice = System.getProperty("ts3j.tray", "auto");
IntFunction<Image> initial = size -> SelfState.DISCONNECTED.icon(size).getImage();
TrayBackend backend = choice.equals("awt") ? null : X11Tray.create(listener, initial);
if (backend == null && !choice.equals("x11")) {
backend = AwtTray.create(listener, buildAwtMenu(), initial);
}
return backend;
}
/** Shows the state of the given server, named by {@code serverName} in the tooltip. */
void update(SelfState newState, String serverName) {
if (tray == null) return;
String newTooltip = "TS3J — " + (serverName == null || serverName.isBlank()
? newState.label()
: serverName + ": " + newState.label());
if (newState == state && newTooltip.equals(tooltip)) return;
state = newState;
tooltip = newTooltip;
tray.setIcon(size -> newState.icon(size).getImage());
tray.setTooltip(newTooltip);
}
/** Redraws after the icon pack changed. */
void refreshIcon() {
if (tray == null || state == null) return;
SelfState current = state;
tray.setIcon(size -> current.icon(size).getImage());
}
void dispose() {
if (tray != null) tray.dispose();
}
// ---- menus ----
/** The menu for AWT's icon, which shows one of its own. */
private PopupMenu buildAwtMenu() {
PopupMenu menu = new PopupMenu();
MenuItem showItem = new MenuItem("Show TS3J");
showItem.addActionListener(e -> SwingUtilities.invokeLater(this::show));
MenuItem quitItem = new MenuItem("Quit");
quitItem.addActionListener(e -> SwingUtilities.invokeLater(onQuit));
menu.add(showItem);
menu.addSeparator();
menu.add(quitItem);
return menu;
}
/**
* The same menu for backends that have none — a Swing popup needs a component to
* hang off, so it is given an empty window at the pointer.
*
* <p>Swing dismisses a popup from events it sees itself, and it sees none of the
* clicks that land on the desktop or another application, so the little window is
* made focusable and the menu closed as soon as it loses the focus. Escape and a
* second click on the icon close it too.
*/
private void showMenu(int x, int y) {
if (openMenu != null) { // a second click on the icon: close it again
hideMenu();
return;
}
JDialog anchor = new JDialog(frame);
anchor.setUndecorated(true);
anchor.setFocusableWindowState(true);
anchor.setAlwaysOnTop(true);
anchor.setLocation(x, y);
anchor.setSize(1, 1);
anchor.setVisible(true);
anchor.toFront();
anchor.requestFocus();
anchor.addWindowFocusListener(new WindowAdapter() {
@Override
public void windowLostFocus(WindowEvent e) {
// The menu's own popup window may take the focus instead; only the
// focus moving away from the menu altogether closes it.
if (!ownedBy(e.getOppositeWindow(), anchor)) hideMenu();
}
});
anchor.getRootPane().registerKeyboardAction(e -> hideMenu(),
KeyStroke.getKeyStroke(KeyEvent.VK_ESCAPE, 0),
JComponent.WHEN_IN_FOCUSED_WINDOW);
JPopupMenu menu = new JPopupMenu();
JMenuItem showItem = new JMenuItem("Show TS3J", Icons.app());
showItem.addActionListener(e -> show());
JMenuItem quitItem = new JMenuItem("Quit", Icons.of("QUIT"));
quitItem.addActionListener(e -> onQuit.run());
menu.add(showItem);
menu.addSeparator();
menu.add(quitItem);
menu.addPopupMenuListener(new PopupMenuListener() {
@Override
public void popupMenuWillBecomeVisible(PopupMenuEvent e) {
}
@Override
public void popupMenuWillBecomeInvisible(PopupMenuEvent e) {
closed(anchor);
}
@Override
public void popupMenuCanceled(PopupMenuEvent e) {
closed(anchor);
}
});
openMenu = menu;
openAnchor = anchor;
menu.show(anchor, 0, 0);
}
/** @return whether {@code window} is {@code owner} or one of its child windows */
private static boolean ownedBy(Window window, Window owner) {
for (Window w = window; w != null; w = w.getOwner()) {
if (w == owner) return true;
}
return false;
}
private void hideMenu() {
if (openMenu != null) openMenu.setVisible(false); // the listener cleans up
closed(openAnchor);
}
private void closed(JDialog anchor) {
openMenu = null;
openAnchor = null;
if (anchor != null) anchor.dispose();
}
/**
* Brings the window back from the taskbar or an iconified state.
*
* <p>A window manager ignores {@link Frame#toFront()} from an application that is
* not focused, which is every time the tray icon is used, so the backend is asked
* first — it can make the request the window manager does honour.
*/
private void show() {
hideMenu();
frame.setVisible(true);
frame.setExtendedState(frame.getExtendedState() & ~Frame.ICONIFIED);
// Deferred so the window is mapped before anything tries to raise it.
SwingUtilities.invokeLater(() -> {
if (tray != null && tray.activateWindow(frame.getTitle())) return;
// Whatever Swing can manage: asking to be on top for a moment gets the
// window in front even where a plain raise is refused.
frame.setAlwaysOnTop(true);
frame.toFront();
frame.requestFocus();
frame.setAlwaysOnTop(false);
});
}
}

View File

@@ -0,0 +1,90 @@
package com.ts3client.ui.tray;
import java.awt.AWTException;
import java.awt.Dimension;
import java.awt.Image;
import java.awt.PopupMenu;
import java.awt.SystemTray;
import java.awt.TrayIcon;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import java.util.function.IntFunction;
/**
* The tray icon AWT itself provides — used where {@link X11Tray} cannot help:
* Windows and macOS, and X11 panels that do not offer a transparent visual.
*
* <p>On X11 the toolkit draws the icon onto an opaque background, which is the
* price of this fallback; the menu, on the other hand, is the desktop's own.
*/
public final class AwtTray implements TrayBackend {
private final TrayIcon icon;
private final int size;
/**
* @param menu the icon's context menu, or {@code null} to have right clicks
* reported to {@code listener} instead
* @param initial the image to dock with, rendered at the tray's icon size
* @return the icon, or {@code null} when this desktop has no tray at all
*/
public static AwtTray create(Listener listener, PopupMenu menu, IntFunction<Image> initial) {
if (!SystemTray.isSupported()) {
TrayLog.debug("this desktop has no system tray");
return null;
}
try {
AwtTray tray = new AwtTray(listener, menu, initial);
TrayLog.debug("using AWT's tray icon, " + tray.size + "px (it draws a background)");
return tray;
} catch (AWTException | UnsupportedOperationException e) {
TrayLog.debug("AWT would not add the icon: " + e.getMessage());
return null;
}
}
private AwtTray(Listener listener, PopupMenu menu, IntFunction<Image> initial)
throws AWTException {
Dimension traySize = SystemTray.getSystemTray().getTrayIconSize();
size = Math.max(16, Math.min(traySize.width, traySize.height));
// Added with its final image and tooltip: some panels take the icon they are
// handed at that moment and ignore later changes.
icon = new TrayIcon(initial.apply(size), "TS3J");
if (menu != null) icon.setPopupMenu(menu);
// The action event only fires on a double click (and not at all on some
// desktops), so listen for the click itself.
icon.addMouseListener(new MouseAdapter() {
@Override
public void mouseClicked(MouseEvent e) {
if (e.getButton() == MouseEvent.BUTTON1) {
listener.activated();
} else if (menu == null && e.isPopupTrigger()) {
listener.menuRequested(e.getXOnScreen(), e.getYOnScreen());
}
}
});
SystemTray.getSystemTray().add(icon);
}
@Override
public void setIcon(IntFunction<Image> render) {
Image image = render.apply(size);
if (image != null) icon.setImage(image);
}
@Override
public void setTooltip(String tooltip) {
icon.setToolTip(tooltip);
}
@Override
public String description() {
return "AWT's own icon, " + size + "px (the desktop draws a background behind it)";
}
@Override
public void dispose() {
SystemTray.getSystemTray().remove(icon);
}
}

View File

@@ -0,0 +1,51 @@
package com.ts3client.ui.tray;
import java.awt.Image;
import java.util.function.IntFunction;
/** A tray icon, however the desktop happens to provide one. */
public interface TrayBackend {
/**
* Sets the icon to show.
*
* @param render draws it at the size the tray asks for — panels differ, and the
* size can change while the icon is docked
*/
void setIcon(IntFunction<Image> render);
/** Sets the text the panel shows on hover, where it supports one. */
void setTooltip(String tooltip);
/** Takes the icon out of the tray again. */
void dispose();
/** How this icon is being shown, for the client's own diagnostics. */
String description();
/**
* Asks the window manager to bring a window of this application to the front.
* Swing cannot: a window manager ignores a raise from an application that does
* not have the focus, which is exactly the case when the tray icon is used.
*
* @param title the window's title
* @return {@code false} when this backend cannot ask, and the caller should try
* whatever Swing can do
*/
default boolean activateWindow(String title) {
return false;
}
/** What the user did with the icon. */
interface Listener {
/** The icon was clicked (or activated from the keyboard). */
void activated();
/**
* A context menu was asked for at a screen position. Only called by backends
* that have no menu of their own.
*/
void menuRequested(int x, int y);
}
}

View File

@@ -0,0 +1,20 @@
package com.ts3client.ui.tray;
/**
* Diagnostics for the tray icon, off unless {@code -Dts3j.tray.debug=true} is given.
*
* <p>Which backend ends up with the icon depends on what the panel offers, and a
* panel that quietly ignores a docking request leaves nothing to see, so it is
* worth being able to ask.
*/
final class TrayLog {
private static final boolean ENABLED = Boolean.getBoolean("ts3j.tray.debug");
private TrayLog() {
}
static void debug(String message) {
if (ENABLED) System.err.println("tray: " + message);
}
}

View File

@@ -0,0 +1,470 @@
package com.ts3client.ui.tray;
import java.lang.foreign.AddressLayout;
import java.lang.foreign.Arena;
import java.lang.foreign.FunctionDescriptor;
import java.lang.foreign.Linker;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.SymbolLookup;
import java.lang.foreign.ValueLayout;
import java.lang.invoke.MethodHandle;
import java.nio.charset.StandardCharsets;
/**
* Raw binding to {@code libX11} through the Foreign Function &amp; Memory API, with
* just the calls {@link X11Tray} needs.
*
* <p>This class is the only place that knows about Xlib's ABI: the entry points, the
* protocol constants and the offsets of the few structures that are read or written
* directly. Loading is lazy and failure is expected — on a machine with no X11 (or a
* pure Wayland session) {@link #isAvailable()} returns {@code false} and the caller
* falls back to AWT's own tray icon.
*
* <p>Offsets are those of the LP64 ABI, in which {@code XID}, {@code Atom} and
* {@code long} are 64 bits wide and every field is naturally aligned.
*/
final class X11 {
private X11() {
}
static final AddressLayout PTR = ValueLayout.ADDRESS;
static final ValueLayout.OfInt INT = ValueLayout.JAVA_INT;
static final ValueLayout.OfLong LONG = ValueLayout.JAVA_LONG;
// ---- protocol constants (X.h) ----
static final int INPUT_OUTPUT = 1;
static final int ALLOC_NONE = 0;
static final int PROP_MODE_REPLACE = 0;
static final int Z_PIXMAP = 2;
static final int SUCCESS = 0;
/** Fields of {@code XSetWindowAttributes} passed to {@code XCreateWindow}. */
static final long CW_BACK_PIXEL = 1L << 1;
static final long CW_BORDER_PIXEL = 1L << 3;
static final long CW_EVENT_MASK = 1L << 11;
static final long CW_COLORMAP = 1L << 13;
static final long BUTTON_PRESS_MASK = 1L << 2;
static final long SUBSTRUCTURE_NOTIFY_MASK = 1L << 19;
static final long SUBSTRUCTURE_REDIRECT_MASK = 1L << 20;
static final long EXPOSURE_MASK = 1L << 15;
static final long STRUCTURE_NOTIFY_MASK = 1L << 17;
static final long NO_EVENT_MASK = 0L;
static final int BUTTON_PRESS = 4;
static final int EXPOSE = 12;
static final int MAP_NOTIFY = 19;
static final int REPARENT_NOTIFY = 21;
static final int CONFIGURE_NOTIFY = 22;
static final int CLIENT_MESSAGE = 33;
/** Predefined atoms (Xatom.h). */
static final long XA_STRING = 31;
static final long XA_VISUALID = 32;
static final long VISUAL_ID_MASK = 0x1;
/** {@code AnyPropertyType}: read a property whatever its type. */
static final long ANY_PROPERTY_TYPE = 0;
// ---- structure layout ----
/** {@code XEvent} is a union; 192 bytes is its size on every LP64 platform. */
static final long EVENT_SIZE = 192;
static final long EVENT_TYPE = 0;
/** {@code XExposeEvent} / {@code XConfigureEvent} / {@code XButtonEvent} share this prefix. */
static final long EVENT_WINDOW = 32;
/** {@code XConfigureEvent}: the reconfigured window, then its new geometry. */
static final long CONFIGURE_WINDOW = 40;
static final long CONFIGURE_WIDTH = 56;
static final long CONFIGURE_HEIGHT = 60;
/** {@code XReparentEvent}: the reparented window and its new parent. */
static final long REPARENT_WINDOW = 40;
static final long REPARENT_PARENT = 48;
/** {@code XButtonEvent}. */
static final long BUTTON_X_ROOT = 72;
static final long BUTTON_Y_ROOT = 76;
static final long BUTTON_BUTTON = 84;
/** {@code XClientMessageEvent}. */
static final long CLIENT_MESSAGE_TYPE = 40;
static final long CLIENT_MESSAGE_FORMAT = 48;
static final long CLIENT_MESSAGE_DATA = 56;
/** {@code XSetWindowAttributes}. */
static final long ATTRIBUTES_SIZE = 112;
static final long ATTRIBUTES_BACKGROUND_PIXEL = 8;
static final long ATTRIBUTES_BORDER_PIXEL = 24;
static final long ATTRIBUTES_EVENT_MASK = 72;
static final long ATTRIBUTES_COLORMAP = 96;
/** {@code XSizeHints} fields we set, and its {@code flags} bits (Xutil.h). */
static final long SIZE_HINTS_SIZE = 80;
static final long SIZE_HINTS_FLAGS = 0;
static final long SIZE_HINTS_MIN_WIDTH = 24;
static final long SIZE_HINTS_MIN_HEIGHT = 28;
static final long SIZE_HINTS_MAX_WIDTH = 32;
static final long SIZE_HINTS_MAX_HEIGHT = 36;
static final long SIZE_HINTS_BASE_WIDTH = 64;
static final long SIZE_HINTS_BASE_HEIGHT = 68;
static final long P_MIN_SIZE = 1L << 4;
static final long P_MAX_SIZE = 1L << 5;
static final long P_BASE_SIZE = 1L << 8;
/** {@code XVisualInfo}. */
static final long VISUAL_INFO_SIZE = 64;
static final long VISUAL_INFO_VISUAL = 0;
static final long VISUAL_INFO_VISUALID = 8;
static final long VISUAL_INFO_DEPTH = 20;
/** {@code XImage}: only the data pointer is touched, to unhook it before freeing. */
static final long IMAGE_DATA = 16;
private static final Linker LINKER = Linker.nativeLinker();
private static final String[] LIBRARY_NAMES = {"libX11.so.6", "libX11.so"};
private static final class Handles {
static final SymbolLookup LOOKUP = load();
static final MethodHandle X_OPEN_DISPLAY =
downcall("XOpenDisplay", FunctionDescriptor.of(PTR, PTR));
static final MethodHandle X_CLOSE_DISPLAY =
downcall("XCloseDisplay", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_INTERN_ATOM =
downcall("XInternAtom", FunctionDescriptor.of(LONG, PTR, PTR, INT));
static final MethodHandle X_GET_SELECTION_OWNER =
downcall("XGetSelectionOwner", FunctionDescriptor.of(LONG, PTR, LONG));
static final MethodHandle X_DEFAULT_SCREEN =
downcall("XDefaultScreen", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_ROOT_WINDOW =
downcall("XRootWindow", FunctionDescriptor.of(LONG, PTR, INT));
static final MethodHandle X_GET_WINDOW_PROPERTY =
downcall("XGetWindowProperty", FunctionDescriptor.of(INT, PTR, LONG, LONG, LONG, LONG,
INT, LONG, PTR, PTR, PTR, PTR, PTR));
static final MethodHandle X_GET_VISUAL_INFO =
downcall("XGetVisualInfo", FunctionDescriptor.of(PTR, PTR, LONG, PTR, PTR));
static final MethodHandle X_FREE =
downcall("XFree", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_CREATE_COLORMAP =
downcall("XCreateColormap", FunctionDescriptor.of(LONG, PTR, LONG, PTR, INT));
static final MethodHandle X_CREATE_WINDOW =
downcall("XCreateWindow", FunctionDescriptor.of(LONG, PTR, LONG, INT, INT, INT, INT,
INT, INT, INT, PTR, LONG, PTR));
static final MethodHandle X_DESTROY_WINDOW =
downcall("XDestroyWindow", FunctionDescriptor.of(INT, PTR, LONG));
static final MethodHandle X_CHANGE_PROPERTY =
downcall("XChangeProperty", FunctionDescriptor.of(INT, PTR, LONG, LONG, LONG, INT,
INT, PTR, INT));
static final MethodHandle X_SELECT_INPUT =
downcall("XSelectInput", FunctionDescriptor.of(INT, PTR, LONG, LONG));
static final MethodHandle X_SEND_EVENT =
downcall("XSendEvent", FunctionDescriptor.of(INT, PTR, LONG, INT, LONG, PTR));
static final MethodHandle X_QUERY_TREE =
downcall("XQueryTree", FunctionDescriptor.of(INT, PTR, LONG, PTR, PTR, PTR, PTR));
static final MethodHandle X_SET_WM_NORMAL_HINTS =
downcall("XSetWMNormalHints", FunctionDescriptor.ofVoid(PTR, LONG, PTR));
static final MethodHandle X_RESIZE_WINDOW =
downcall("XResizeWindow", FunctionDescriptor.of(INT, PTR, LONG, INT, INT));
static final MethodHandle X_MAP_WINDOW =
downcall("XMapWindow", FunctionDescriptor.of(INT, PTR, LONG));
static final MethodHandle X_UNMAP_WINDOW =
downcall("XUnmapWindow", FunctionDescriptor.of(INT, PTR, LONG));
static final MethodHandle X_FLUSH =
downcall("XFlush", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_PENDING =
downcall("XPending", FunctionDescriptor.of(INT, PTR));
static final MethodHandle X_NEXT_EVENT =
downcall("XNextEvent", FunctionDescriptor.of(INT, PTR, PTR));
static final MethodHandle X_CREATE_GC =
downcall("XCreateGC", FunctionDescriptor.of(PTR, PTR, LONG, LONG, PTR));
static final MethodHandle X_FREE_GC =
downcall("XFreeGC", FunctionDescriptor.of(INT, PTR, PTR));
static final MethodHandle X_CREATE_IMAGE =
downcall("XCreateImage", FunctionDescriptor.of(PTR, PTR, PTR, INT, INT, INT, PTR,
INT, INT, INT, INT));
static final MethodHandle X_PUT_IMAGE =
downcall("XPutImage", FunctionDescriptor.of(INT, PTR, LONG, PTR, PTR, INT, INT, INT,
INT, INT, INT));
private static SymbolLookup load() {
IllegalArgumentException last = null;
for (String name : LIBRARY_NAMES) {
try {
return SymbolLookup.libraryLookup(name, Arena.global());
} catch (IllegalArgumentException e) {
last = e;
}
}
throw (last != null) ? last : new IllegalArgumentException("libX11 not found");
}
private static MethodHandle downcall(String symbol, FunctionDescriptor descriptor) {
return LINKER.downcallHandle(
LOOKUP.find(symbol).orElseThrow(() ->
new UnsatisfiedLinkError("libX11: unresolved symbol " + symbol)),
descriptor);
}
}
/** {@code true} when libX11 could be loaded and every symbol we need resolved. */
static boolean isAvailable() {
try {
return Handles.LOOKUP != null;
} catch (Throwable t) {
return false;
}
}
// ---- display ----
/** @return the display, or {@link MemorySegment#NULL} when it cannot be opened */
static MemorySegment openDisplay() {
return (MemorySegment) call(Handles.X_OPEN_DISPLAY, MemorySegment.NULL);
}
static void closeDisplay(MemorySegment display) {
call(Handles.X_CLOSE_DISPLAY, display);
}
static int defaultScreen(MemorySegment display) {
return (int) call(Handles.X_DEFAULT_SCREEN, display);
}
static long rootWindow(MemorySegment display, int screen) {
return (long) call(Handles.X_ROOT_WINDOW, display, screen);
}
static long internAtom(MemorySegment display, Arena arena, String name) {
return (long) call(Handles.X_INTERN_ATOM, display, arena.allocateFrom(name), 0);
}
static long selectionOwner(MemorySegment display, long atom) {
return (long) call(Handles.X_GET_SELECTION_OWNER, display, atom);
}
static void flush(MemorySegment display) {
call(Handles.X_FLUSH, display);
}
// ---- properties ----
/**
* Reads a single 32-bit property value (which Xlib hands back as a C {@code long}).
*
* @return the value, or {@code 0} when the window has no such property
*/
static long cardinalProperty(MemorySegment display, Arena arena, long window, long property,
long type) {
MemorySegment actualType = arena.allocate(LONG);
MemorySegment actualFormat = arena.allocate(INT);
MemorySegment items = arena.allocate(LONG);
MemorySegment bytesAfter = arena.allocate(LONG);
MemorySegment data = arena.allocate(PTR);
int status = (int) call(Handles.X_GET_WINDOW_PROPERTY, display, window, property, 0L, 1L, 0,
type, actualType, actualFormat, items, bytesAfter, data);
MemorySegment values = data.get(PTR, 0);
if (status != SUCCESS || values.equals(MemorySegment.NULL)) return 0;
long value = items.get(LONG, 0) > 0
? values.reinterpret(Long.BYTES).get(LONG, 0)
: 0;
call(Handles.X_FREE, values);
return value;
}
static void setCardinals(MemorySegment display, Arena arena, long window, long property,
long type, long... values) {
MemorySegment data = arena.allocate(LONG, values.length);
for (int i = 0; i < values.length; i++) data.setAtIndex(LONG, i, values[i]);
call(Handles.X_CHANGE_PROPERTY, display, window, property, type, 32, PROP_MODE_REPLACE,
data, values.length);
}
static void setText(MemorySegment display, Arena arena, long window, long property, long type,
String text) {
byte[] bytes = text.getBytes(StandardCharsets.UTF_8);
MemorySegment data = arena.allocate(bytes.length + 1);
MemorySegment.copy(bytes, 0, data, ValueLayout.JAVA_BYTE, 0, bytes.length);
call(Handles.X_CHANGE_PROPERTY, display, window, property, type, 8, PROP_MODE_REPLACE,
data, bytes.length);
}
/** @return the children of a window, oldest first */
static long[] children(MemorySegment display, Arena arena, long window) {
MemorySegment rootReturn = arena.allocate(LONG);
MemorySegment parentReturn = arena.allocate(LONG);
MemorySegment childrenReturn = arena.allocate(PTR);
MemorySegment countReturn = arena.allocate(INT);
int status = (int) call(Handles.X_QUERY_TREE, display, window, rootReturn, parentReturn,
childrenReturn, countReturn);
MemorySegment list = childrenReturn.get(PTR, 0);
if (status == 0 || list.equals(MemorySegment.NULL)) return new long[0];
int count = countReturn.get(INT, 0);
long[] ids = new long[count];
MemorySegment windows = list.reinterpret((long) count * Long.BYTES);
for (int i = 0; i < count; i++) ids[i] = windows.getAtIndex(LONG, i);
call(Handles.X_FREE, list);
return ids;
}
/**
* Reads a text property.
*
* @return its value as UTF-8, or {@code null} when the window has no such property
*/
static String textProperty(MemorySegment display, Arena arena, long window, long property) {
MemorySegment actualType = arena.allocate(LONG);
MemorySegment actualFormat = arena.allocate(INT);
MemorySegment items = arena.allocate(LONG);
MemorySegment bytesAfter = arena.allocate(LONG);
MemorySegment data = arena.allocate(PTR);
int status = (int) call(Handles.X_GET_WINDOW_PROPERTY, display, window, property, 0L, 1024L,
0, ANY_PROPERTY_TYPE, actualType, actualFormat, items, bytesAfter, data);
MemorySegment value = data.get(PTR, 0);
if (status != SUCCESS || value.equals(MemorySegment.NULL)) return null;
long length = items.get(LONG, 0);
byte[] bytes = value.reinterpret(length).toArray(ValueLayout.JAVA_BYTE);
call(Handles.X_FREE, value);
return new String(bytes, StandardCharsets.UTF_8);
}
/** @return {@code true} when the window carries the property at all */
static boolean hasProperty(MemorySegment display, Arena arena, long window, long property) {
MemorySegment actualType = arena.allocate(LONG);
MemorySegment actualFormat = arena.allocate(INT);
MemorySegment items = arena.allocate(LONG);
MemorySegment bytesAfter = arena.allocate(LONG);
MemorySegment data = arena.allocate(PTR);
int status = (int) call(Handles.X_GET_WINDOW_PROPERTY, display, window, property, 0L, 0L, 0,
ANY_PROPERTY_TYPE, actualType, actualFormat, items, bytesAfter, data);
MemorySegment value = data.get(PTR, 0);
if (!value.equals(MemorySegment.NULL)) call(Handles.X_FREE, value);
return status == SUCCESS && actualType.get(LONG, 0) != 0;
}
// ---- visuals ----
/**
* Looks a visual up by id.
*
* @return its {@code XVisualInfo} (owned by Xlib, to be released with {@link #free})
* or {@link MemorySegment#NULL} when the id is unknown
*/
static MemorySegment visualInfo(MemorySegment display, Arena arena, long visualId) {
MemorySegment template = arena.allocate(VISUAL_INFO_SIZE);
template.set(LONG, VISUAL_INFO_VISUALID, visualId);
MemorySegment count = arena.allocate(INT);
MemorySegment info = (MemorySegment) call(Handles.X_GET_VISUAL_INFO, display,
VISUAL_ID_MASK, template, count);
if (info.equals(MemorySegment.NULL) || count.get(INT, 0) < 1) return MemorySegment.NULL;
return info.reinterpret(VISUAL_INFO_SIZE);
}
static void free(MemorySegment pointer) {
call(Handles.X_FREE, pointer);
}
// ---- windows ----
static long createColormap(MemorySegment display, long window, MemorySegment visual) {
return (long) call(Handles.X_CREATE_COLORMAP, display, window, visual, ALLOC_NONE);
}
static long createWindow(MemorySegment display, long parent, int width, int height, int depth,
MemorySegment visual, MemorySegment attributes, long valueMask) {
return (long) call(Handles.X_CREATE_WINDOW, display, parent, 0, 0, width, height, 0, depth,
INPUT_OUTPUT, visual, valueMask, attributes);
}
static void destroyWindow(MemorySegment display, long window) {
call(Handles.X_DESTROY_WINDOW, display, window);
}
static void selectInput(MemorySegment display, long window, long mask) {
call(Handles.X_SELECT_INPUT, display, window, mask);
}
/**
* Tells the panel how big the icon wants to be. XEmbed trays lay their children
* out from these hints; without them a panel is free to allocate a sliver.
*/
static void setSizeHints(MemorySegment display, Arena arena, long window, int size) {
MemorySegment hints = arena.allocate(SIZE_HINTS_SIZE);
hints.fill((byte) 0);
hints.set(LONG, SIZE_HINTS_FLAGS, P_MIN_SIZE | P_MAX_SIZE | P_BASE_SIZE);
hints.set(INT, SIZE_HINTS_MIN_WIDTH, size);
hints.set(INT, SIZE_HINTS_MIN_HEIGHT, size);
hints.set(INT, SIZE_HINTS_MAX_WIDTH, size);
hints.set(INT, SIZE_HINTS_MAX_HEIGHT, size);
hints.set(INT, SIZE_HINTS_BASE_WIDTH, size);
hints.set(INT, SIZE_HINTS_BASE_HEIGHT, size);
call(Handles.X_SET_WM_NORMAL_HINTS, display, window, hints);
}
static void resizeWindow(MemorySegment display, long window, int width, int height) {
call(Handles.X_RESIZE_WINDOW, display, window, width, height);
}
static void mapWindow(MemorySegment display, long window) {
call(Handles.X_MAP_WINDOW, display, window);
}
static void unmapWindow(MemorySegment display, long window) {
call(Handles.X_UNMAP_WINDOW, display, window);
}
// ---- events ----
static int pending(MemorySegment display) {
return (int) call(Handles.X_PENDING, display);
}
static void nextEvent(MemorySegment display, MemorySegment event) {
call(Handles.X_NEXT_EVENT, display, event);
}
static void sendEvent(MemorySegment display, long window, long mask, MemorySegment event) {
call(Handles.X_SEND_EVENT, display, window, 0, mask, event);
}
// ---- drawing ----
static MemorySegment createGC(MemorySegment display, long drawable) {
return (MemorySegment) call(Handles.X_CREATE_GC, display, drawable, 0L, MemorySegment.NULL);
}
static void freeGC(MemorySegment display, MemorySegment gc) {
call(Handles.X_FREE_GC, display, gc);
}
/** Wraps a caller-owned pixel buffer in an {@code XImage} of 32-bit ARGB pixels. */
static MemorySegment createImage(MemorySegment display, MemorySegment visual, int depth,
MemorySegment data, int width, int height) {
MemorySegment image = (MemorySegment) call(Handles.X_CREATE_IMAGE, display, visual, depth,
Z_PIXMAP, 0, data, width, height, 32, 0);
return image.equals(MemorySegment.NULL) ? image : image.reinterpret(IMAGE_DATA + 8);
}
/** Frees the {@code XImage} without touching the pixel buffer, which is ours. */
static void destroyImage(MemorySegment image) {
image.set(PTR, IMAGE_DATA, MemorySegment.NULL);
call(Handles.X_FREE, image);
}
static void putImage(MemorySegment display, long drawable, MemorySegment gc,
MemorySegment image, int width, int height) {
call(Handles.X_PUT_IMAGE, display, drawable, gc, image, 0, 0, 0, 0, width, height);
}
private static Object call(MethodHandle handle, Object... args) {
try {
return handle.invokeWithArguments(args);
} catch (Throwable t) {
throw new IllegalStateException("libX11 call failed", t);
}
}
}

View File

@@ -0,0 +1,508 @@
package com.ts3client.ui.tray;
import java.awt.Graphics2D;
import java.awt.Image;
import java.awt.RenderingHints;
import java.awt.image.BufferedImage;
import java.awt.image.DataBufferInt;
import java.lang.foreign.Arena;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.ValueLayout;
import java.util.function.IntFunction;
import java.util.function.LongPredicate;
/**
* A tray icon docked into the panel's notification area by hand, over the
* freedesktop system tray protocol.
*
* <p>AWT can do this too, but its icon is never transparent on X11: the toolkit
* embeds an ordinary window of the screen's default (opaque) visual and fills it
* with a background colour before drawing, so every icon sits on a coloured box.
* Panels that support transparency — MATE's, Xfce's, GNOME's — advertise an ARGB
* visual in {@code _NET_SYSTEM_TRAY_VISUAL} and composite the icon themselves;
* all that is needed is to dock a window of <em>that</em> visual, which is what
* this class does.
*
* <p>Everything here runs on one thread of its own, which owns the display
* connection and pumps the X event queue; the rest of the client only posts new
* icons to it.
*/
public final class X11Tray implements TrayBackend {
/** The size to ask for; the panel resizes us to whatever it wants. */
private static final int DEFAULT_SIZE = 24;
private static final int POLL_MS = 40;
/** How long to give the panel to adopt the icon before falling back to AWT. */
private static final int DOCK_TIMEOUT_MS = 1500;
/** Anything narrower than this is a sliver, not an icon the panel meant to give us. */
private static final int MIN_SENSIBLE_SIZE = 8;
/** A tall panel should still not get a huge icon. */
private static final int MAX_ICON_SIZE = 64;
/**
* Panels that composite the icon themselves need not send us a single expose, so
* the icon is also repainted at this interval — it costs one 24&times;24 image.
*/
private static final int REPAINT_MS = 2000;
/** {@code _NET_SYSTEM_TRAY_OPCODE} message: dock this window. */
private static final long SYSTEM_TRAY_REQUEST_DOCK = 0;
/** {@code _XEMBED_INFO} flag: the icon wants to be mapped. */
private static final long XEMBED_MAPPED = 1;
private final Listener listener;
/** Lives as long as the icon: the display connection and every buffer we pass to it. */
private final Arena arena = Arena.ofShared();
private final MemorySegment display;
private final long root;
private final long window;
private final MemorySegment visual;
private final int depth;
private final MemorySegment gc;
private final long selectionAtom;
private final long opcodeAtom;
private final long managerAtom;
private final Thread thread;
private volatile boolean running = true;
/** Set once the panel has reparented the icon into itself. */
private volatile boolean docked;
/** The square size last asked of the panel, so we ask only once per size. */
private int requested;
/** The icon and the size the panel gave us, both read by the event thread. */
private volatile IntFunction<Image> render;
private volatile int width = DEFAULT_SIZE;
private volatile int height = DEFAULT_SIZE;
private volatile boolean dirty = true;
/** The pixel buffer handed to X, kept until the icon's size changes. */
private MemorySegment pixels;
private long lastDrawn;
private MemorySegment image;
private int imageWidth;
private int imageHeight;
/**
* Docks an icon into the notification area.
*
* @return the icon, or {@code null} when this is not an X11 session, there is no
* notification area, or it cannot show transparent icons — the caller should then
* fall back to AWT's tray, which works everywhere but always draws a background
*/
public static X11Tray create(Listener listener, IntFunction<Image> initial) {
if (!X11.isAvailable()) {
TrayLog.debug("libX11 is not available");
return null;
}
try {
X11Tray tray = new X11Tray(listener, initial);
TrayLog.debug("docked into the notification area, " + tray.width + "x" + tray.height
+ " in an ARGB visual");
return tray;
} catch (RuntimeException e) {
TrayLog.debug("not usable: " + e.getMessage());
return null;
}
}
private X11Tray(Listener listener, IntFunction<Image> initial) {
this.listener = listener;
this.render = initial;
display = X11.openDisplay();
if (display.equals(MemorySegment.NULL)) throw new Unsupported("no display");
boolean ok = false;
try {
int screen = X11.defaultScreen(display);
root = X11.rootWindow(display, screen);
selectionAtom = X11.internAtom(display, arena, "_NET_SYSTEM_TRAY_S" + screen);
opcodeAtom = X11.internAtom(display, arena, "_NET_SYSTEM_TRAY_OPCODE");
managerAtom = X11.internAtom(display, arena, "MANAGER");
long manager = X11.selectionOwner(display, selectionAtom);
if (manager == 0) throw new Unsupported("no notification area");
visual = argbVisual(manager);
depth = 32; // argbVisual() accepts nothing else
window = createWindow();
gc = X11.createGC(display, window);
// Panels announce themselves with a MANAGER message when they (re)start,
// which is our cue to dock again.
X11.selectInput(display, root, X11.STRUCTURE_NOTIFY_MASK);
requestDock(manager);
// Docking is a request, not a call: a panel that ignores it would leave us
// with an invisible window and the user with no icon at all.
if (!awaitDock()) throw new Unsupported("the notification area did not adopt the icon");
thread = new Thread(this::pump, "x11-tray");
thread.setDaemon(true);
thread.start();
ok = true;
} finally {
if (!ok) {
X11.closeDisplay(display);
arena.close();
}
}
}
// ---- setup ----
/**
* @return the visual the panel wants transparent icons drawn in
* @throws Unsupported when it offers none, so nothing would be gained over AWT
*/
private MemorySegment argbVisual(long manager) {
long visualAtom = X11.internAtom(display, arena, "_NET_SYSTEM_TRAY_VISUAL");
long visualId = X11.cardinalProperty(display, arena, manager, visualAtom, X11.XA_VISUALID);
if (visualId == 0) throw new Unsupported("the notification area is not transparent");
MemorySegment info = X11.visualInfo(display, arena, visualId);
if (info.equals(MemorySegment.NULL)) throw new Unsupported("unknown visual " + visualId);
try {
int visualDepth = info.get(ValueLayout.JAVA_INT, X11.VISUAL_INFO_DEPTH);
if (visualDepth != 32) {
throw new Unsupported("visual " + visualId + " has depth " + visualDepth);
}
// The XVisualInfo itself is Xlib's, but the Visual it points at outlives it.
return info.get(X11.PTR, X11.VISUAL_INFO_VISUAL);
} finally {
X11.free(info);
}
}
private long createWindow() {
MemorySegment attributes = arena.allocate(X11.ATTRIBUTES_SIZE);
// Fully transparent background, and no border pixmap inherited from the root:
// a window whose depth differs from its parent's must name its own.
attributes.set(ValueLayout.JAVA_LONG, X11.ATTRIBUTES_BACKGROUND_PIXEL, 0);
attributes.set(ValueLayout.JAVA_LONG, X11.ATTRIBUTES_BORDER_PIXEL, 0);
attributes.set(ValueLayout.JAVA_LONG, X11.ATTRIBUTES_COLORMAP,
X11.createColormap(display, root, visual));
attributes.set(ValueLayout.JAVA_LONG, X11.ATTRIBUTES_EVENT_MASK,
X11.EXPOSURE_MASK | X11.STRUCTURE_NOTIFY_MASK | X11.BUTTON_PRESS_MASK);
long mask = X11.CW_BACK_PIXEL | X11.CW_BORDER_PIXEL | X11.CW_COLORMAP | X11.CW_EVENT_MASK;
long id = X11.createWindow(display, root, DEFAULT_SIZE, DEFAULT_SIZE, depth, visual,
attributes, mask);
if (id == 0) throw new Unsupported("could not create the icon window");
// Panels lay their icons out from these; without them MATE's allocates a
// one-pixel-wide sliver, which looks exactly like no icon at all.
X11.setSizeHints(display, arena, id, DEFAULT_SIZE);
X11.setText(display, arena, id, X11.internAtom(display, arena, "WM_NAME"),
X11.XA_STRING, "TS3J");
// XEMBED protocol version 0, and "please map me".
long xembedInfo = X11.internAtom(display, arena, "_XEMBED_INFO");
X11.setCardinals(display, arena, id, xembedInfo, xembedInfo, 0, XEMBED_MAPPED);
return id;
}
/** Asks the notification area to adopt our window. */
private void requestDock(long manager) {
MemorySegment event = arena.allocate(X11.EVENT_SIZE);
event.fill((byte) 0);
event.set(ValueLayout.JAVA_INT, X11.EVENT_TYPE, X11.CLIENT_MESSAGE);
event.set(ValueLayout.JAVA_LONG, X11.EVENT_WINDOW, manager);
event.set(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_TYPE, opcodeAtom);
event.set(ValueLayout.JAVA_INT, X11.CLIENT_MESSAGE_FORMAT, 32);
event.set(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_DATA, 0); // CurrentTime
event.set(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_DATA + 8, SYSTEM_TRAY_REQUEST_DOCK);
event.set(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_DATA + 16, window);
X11.sendEvent(display, manager, X11.NO_EVENT_MASK, event);
X11.flush(display);
}
/**
* Waits for the panel to reparent the icon into itself.
*
* @return {@code false} when it did not within {@link #DOCK_TIMEOUT_MS}
*/
private boolean awaitDock() {
MemorySegment event = arena.allocate(X11.EVENT_SIZE);
long deadline = System.currentTimeMillis() + DOCK_TIMEOUT_MS;
while (!docked && System.currentTimeMillis() < deadline) {
while (!docked && X11.pending(display) > 0) {
X11.nextEvent(display, event);
handle(event);
}
if (docked) break;
try {
Thread.sleep(POLL_MS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return false;
}
}
return docked;
}
// ---- TrayBackend ----
@Override
public void setIcon(IntFunction<Image> render) {
this.render = render;
dirty = true;
}
@Override
public void setTooltip(String tooltip) {
// Panels take the tooltip from the icon window's name; those that don't show
// nothing, which is what AWT's own icon does on X11 anyway.
synchronized (this) {
if (!running) return;
X11.setText(display, arena, window, X11.internAtom(display, arena, "WM_NAME"),
X11.XA_STRING, tooltip);
X11.setText(display, arena, window, X11.internAtom(display, arena, "_NET_WM_NAME"),
X11.internAtom(display, arena, "UTF8_STRING"), tooltip);
X11.flush(display);
}
}
/**
* Sends the window manager an {@code _NET_ACTIVE_WINDOW} request for the named
* window — the EWMH way of saying "the user asked for this window", which is
* honoured where a plain raise is not.
*/
@Override
public boolean activateWindow(String title) {
synchronized (this) {
if (!running) return false;
long target = findWindow(title);
if (target == 0) {
TrayLog.debug("no window named \"" + title + "\" to activate");
return false;
}
TrayLog.debug("activating window 0x" + Long.toHexString(target));
MemorySegment event = arena.allocate(X11.EVENT_SIZE);
event.fill((byte) 0);
event.set(ValueLayout.JAVA_INT, X11.EVENT_TYPE, X11.CLIENT_MESSAGE);
event.set(ValueLayout.JAVA_LONG, X11.EVENT_WINDOW, target);
event.set(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_TYPE,
X11.internAtom(display, arena, "_NET_ACTIVE_WINDOW"));
event.set(ValueLayout.JAVA_INT, X11.CLIENT_MESSAGE_FORMAT, 32);
// Source indication 2: a pager or tray acting for the user, which window
// managers accept without their focus-stealing rules getting in the way.
event.set(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_DATA, 2);
X11.sendEvent(display, root,
X11.SUBSTRUCTURE_NOTIFY_MASK | X11.SUBSTRUCTURE_REDIRECT_MASK, event);
X11.flush(display);
return true;
}
}
/**
* Finds one of this process' own top-level windows: the one with the given title
* for choice, else any window the window manager is managing for us.
*
* <p>Windows are matched by {@code _NET_WM_PID} rather than by name alone, and
* the title is read from {@code _NET_WM_NAME} because {@code WM_NAME} cannot
* carry anything outside Latin-1.
*/
private long findWindow(String title) {
long pid = ProcessHandle.current().pid();
long pidAtom = X11.internAtom(display, arena, "_NET_WM_PID");
long nameAtom = X11.internAtom(display, arena, "_NET_WM_NAME");
long stateAtom = X11.internAtom(display, arena, "WM_STATE");
long found = findWindow(root, 0, w ->
X11.cardinalProperty(display, arena, w, pidAtom, X11.ANY_PROPERTY_TYPE) == pid
&& title.equals(X11.textProperty(display, arena, w, nameAtom)));
if (found != 0) return found;
return findWindow(root, 0, w ->
X11.cardinalProperty(display, arena, w, pidAtom, X11.ANY_PROPERTY_TYPE) == pid
&& X11.hasProperty(display, arena, w, stateAtom));
}
/**
* Walks the window tree below {@code parent}. Only the top few levels are looked
* at: a reparenting window manager puts client windows one or two frames below
* the root.
*/
private long findWindow(long parent, int depth, LongPredicate match) {
if (depth > 3) return 0;
for (long child : X11.children(display, arena, parent)) {
if (match.test(child)) return child;
long found = findWindow(child, depth + 1, match);
if (found != 0) return found;
}
return 0;
}
@Override
public String description() {
return "docked into the notification area, " + width + "x" + height + ", transparent";
}
@Override
public void dispose() {
if (!running) return;
running = false;
try {
thread.join(500);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
synchronized (this) {
if (image != null) X11.destroyImage(image);
X11.freeGC(display, gc);
X11.destroyWindow(display, window);
X11.closeDisplay(display);
}
arena.close();
}
// ---- event thread ----
private void pump() {
MemorySegment event = arena.allocate(X11.EVENT_SIZE);
while (running) {
try {
synchronized (this) {
while (running && X11.pending(display) > 0) {
X11.nextEvent(display, event);
handle(event);
}
boolean due = System.currentTimeMillis() - lastDrawn >= REPAINT_MS;
if (running && (dirty || due)) {
dirty = false;
lastDrawn = System.currentTimeMillis();
draw();
}
}
Thread.sleep(POLL_MS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return;
} catch (RuntimeException e) {
TrayLog.debug("the display connection failed: " + e.getMessage());
return; // the display died with the panel or the session
}
}
}
private void handle(MemorySegment event) {
switch (event.get(ValueLayout.JAVA_INT, X11.EVENT_TYPE)) {
case X11.EXPOSE -> dirty = true;
case X11.CONFIGURE_NOTIFY -> {
if (event.get(ValueLayout.JAVA_LONG, X11.CONFIGURE_WINDOW) != window) return;
int w = event.get(ValueLayout.JAVA_INT, X11.CONFIGURE_WIDTH);
int h = event.get(ValueLayout.JAVA_INT, X11.CONFIGURE_HEIGHT);
if (w == width && h == height) return;
width = w;
height = h;
dirty = true;
askForSquare();
}
case X11.REPARENT_NOTIFY -> {
if (event.get(ValueLayout.JAVA_LONG, X11.REPARENT_WINDOW) != window) return;
if (event.get(ValueLayout.JAVA_LONG, X11.REPARENT_PARENT) == root) {
// The panel went away and handed the window back to the root; hide
// it so it does not turn up in the middle of the screen.
docked = false;
X11.unmapWindow(display, window);
} else {
// Adopted. The panel is meant to map us because of _XEMBED_INFO,
// but mapping ourselves costs nothing and does not depend on it.
docked = true;
X11.mapWindow(display, window);
dirty = true;
}
}
case X11.MAP_NOTIFY -> dirty = true;
case X11.BUTTON_PRESS -> {
long button = event.get(ValueLayout.JAVA_INT, X11.BUTTON_BUTTON);
int x = event.get(ValueLayout.JAVA_INT, X11.BUTTON_X_ROOT);
int y = event.get(ValueLayout.JAVA_INT, X11.BUTTON_Y_ROOT);
if (button == 1) {
listener.activated();
} else if (button == 3) {
listener.menuRequested(x, y);
}
}
case X11.CLIENT_MESSAGE -> {
// A panel started: dock into it.
if (event.get(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_TYPE) != managerAtom) return;
if (event.get(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_DATA + 8) != selectionAtom) {
return;
}
long manager = event.get(ValueLayout.JAVA_LONG, X11.CLIENT_MESSAGE_DATA + 16);
if (manager != 0) {
requestDock(manager);
X11.mapWindow(display, window);
dirty = true;
}
}
default -> {
}
}
}
/**
* Asks for a square icon when the panel has given us a sliver — its idea of the
* icon size is then the other side, and a panel that lays out from the size hints
* grants it. Each size is asked for once, so a panel that refuses cannot put us
* in a loop.
*/
private void askForSquare() {
int narrow = Math.min(width, height);
int wide = Math.max(width, height);
// Only a sliver is worth arguing about; any sane allocation is left alone.
if (narrow >= MIN_SENSIBLE_SIZE) return;
int wanted = Math.min(wide, MAX_ICON_SIZE);
if (wanted <= 0 || wanted == requested) return;
requested = wanted;
X11.setSizeHints(display, arena, window, wanted);
X11.resizeWindow(display, window, wanted, wanted);
X11.flush(display);
}
/** Paints the current icon over the whole window, alpha and all. */
private void draw() {
IntFunction<Image> painter = render;
if (painter == null) return;
int w = width;
int h = height;
if (w <= 0 || h <= 0) return;
if (image == null || imageWidth != w || imageHeight != h) {
if (image != null) X11.destroyImage(image);
pixels = arena.allocate((long) w * h * Integer.BYTES);
image = X11.createImage(display, visual, depth, pixels, w, h);
imageWidth = w;
imageHeight = h;
if (image.equals(MemorySegment.NULL)) {
image = null;
return;
}
}
// X wants the pixels premultiplied, in the server's own byte order — which
// for a 32-bit ARGB visual on a local display is an int per pixel.
BufferedImage buffer = new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB_PRE);
Graphics2D g = buffer.createGraphics();
g.setRenderingHint(RenderingHints.KEY_INTERPOLATION,
RenderingHints.VALUE_INTERPOLATION_BILINEAR);
Image icon = painter.apply(Math.min(w, h));
if (icon != null) g.drawImage(icon, 0, 0, w, h, null);
g.dispose();
int[] data = ((DataBufferInt) buffer.getRaster().getDataBuffer()).getData();
MemorySegment.copy(data, 0, pixels, ValueLayout.JAVA_INT, 0, data.length);
X11.putImage(display, window, gc, image, w, h);
X11.flush(display);
}
/** Thrown while setting up when this desktop cannot do what we need. */
private static final class Unsupported extends RuntimeException {
Unsupported(String message) {
super(message);
}
}
}

View File

@@ -0,0 +1,48 @@
package com.ts3client.ui;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
class MicrophoneTestTest {
private static int sampleAt(byte[] pcm, int index) {
return (short) (((pcm[2 * index + 1] & 0xFF) << 8) | (pcm[2 * index] & 0xFF));
}
@Test
void convertsFloatSamplesToLittleEndianPcm() {
byte[] pcm = MicrophoneTest.toPcm16(new float[]{0f, 1f, -1f, 0.5f});
assertEquals(8, pcm.length);
assertEquals(0, sampleAt(pcm, 0));
assertEquals(32767, sampleAt(pcm, 1));
assertEquals(-32767, sampleAt(pcm, 2));
assertEquals(16384, sampleAt(pcm, 3), 1);
}
@Test
void clampsSamplesOutsideRange() {
byte[] pcm = MicrophoneTest.toPcm16(new float[]{2f, -2f});
assertEquals(32767, sampleAt(pcm, 0));
assertEquals(-32767, sampleAt(pcm, 1));
}
@Test
void upmixRepeatsMonoAcrossChannels() {
byte[] mono = MicrophoneTest.toPcm16(new float[]{0.25f, -0.25f});
byte[] stereo = MicrophoneTest.upmix(mono, 2);
assertEquals(mono.length * 2, stereo.length);
// Each mono sample appears once per channel, in order.
assertEquals(sampleAt(mono, 0), sampleAt(stereo, 0));
assertEquals(sampleAt(mono, 0), sampleAt(stereo, 1));
assertEquals(sampleAt(mono, 1), sampleAt(stereo, 2));
assertEquals(sampleAt(mono, 1), sampleAt(stereo, 3));
}
@Test
void upmixIsIdentityForOneChannel() {
byte[] mono = MicrophoneTest.toPcm16(new float[]{0.1f, 0.2f, 0.3f});
assertArrayEquals(mono, MicrophoneTest.upmix(mono, 1));
}
}