myTeamSpeak: two-way sync of bookmarks and identities

MyTeamSpeakSync compares each item against what the last run left: a
change on one side goes to the other, deletions too, and an item changed
on both sides is a conflict for the user. The first run pairs items up by
identity key or bookmark name and address. Edits are written over the
item's Item_Data field by field, so what we do not model survives.

The protocol was probed live and checked against the official client:
the server stores whatever revisions it is sent and never reports a
collision, so the client re-reads the cursors before writing. Edits carry
last_known_version like the official client's, which applies them within
its one-minute poll. All requests now share one session; signing in for
each got the account's logins throttled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-15 12:37:50 +00:00
co-authored by Claude Opus 5.5
parent ab6bf30003
commit 756e3850a3
19 changed files with 1910 additions and 156 deletions
@@ -161,6 +161,23 @@ public final class IdentityStore {
return LocalIdentity.read(new ByteArrayInputStream(ini.getBytes(StandardCharsets.US_ASCII)));
}
/** The identity in the {@code <offset>V<obfuscated key>} form {@link #parseIdentityString} reads. */
public static String identityString(LocalIdentity identity) throws IOException {
Ini ini = new Ini(new ByteArrayInputStream(identity.export().getBytes(StandardCharsets.UTF_8)));
String value = unquote(ini.get("Identity", "identity"));
if (value == null) throw new IOException("Identity did not export");
return value;
}
/** Takes the key offset of another copy of the same key, raising or lowering its security level. */
public synchronized void setKeyOffset(IdentityEntry entry, LocalIdentity sameKey) throws IOException {
if (!sameKey.getUid().toBase64().equals(entry.getUniqueId())) throw new IOException("Not the same key");
entry.getIdentity().setKeyOffset(sameKey.getKeyOffset());
entry.getIdentity().setLastCheckedKeyOffset(
Math.max(entry.getIdentity().getLastCheckedKeyOffset(), sameKey.getLastCheckedKeyOffset()));
write(entry, entry.getFile());
}
/** Files an existing key pair as a managed identity under the given name. */
public IdentityEntry add(String name, LocalIdentity identity, String nickname, String phoneticNickname)
throws IOException {
@@ -11,12 +11,16 @@ import java.net.URI;
import java.security.GeneralSecurityException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TreeMap;
/**
* Reads the bookmarks, identities and contacts a myTeamSpeak account synchronises, the way
* the official client pulls them on a fresh install: sign in, ask for every item
* of the wanted classes against an empty local state, decrypt them, and sign out.
* Nothing is ever written to the account. Staying signed in is {@link MyTeamSpeakLogin}'s.
* {@link #pull} and {@link #push} are the two halves of synchronising, which
* {@link MyTeamSpeakSync} drives. Staying signed in is {@link MyTeamSpeakLogin}'s.
*
* <p>The protocol is described in {@code docs/myteamspeak/PROTOCOL.md}.
*/
@@ -47,7 +51,7 @@ public final class MyTeamSpeak {
}
}
private static final String NO_VERSION = "00000000-0000-0000-0000-000000000000";
static final String NO_VERSION = "00000000-0000-0000-0000-000000000000";
private static final int SYNC_VERSION_1_1 = 1;
private static final int CONTACTS = 8;
private static final int[] ITEM_CLASSES = {0 /* BOOKMARK */, 1 /* IDENTITY */, 6 /* ITEM_FOLDER */, CONTACTS};
@@ -104,38 +108,32 @@ public final class MyTeamSpeak {
public Account download(Credentials credentials) throws IOException, MyTsException {
Session session = signIn(credentials);
try {
if (session.keyPackage == null) throw new MyTsException("The account has no synchronisation key", 0);
byte[] itemKey;
try {
itemKey = MyTsCrypto.unwrapItemKey(credentials.accountKey(), session.keyPackage);
} catch (GeneralSecurityException e) {
throw new MyTsException("Could not unlock the account's encrypted data", e);
}
return new Account(session.info, pull(session.id, itemKey));
return read(session, credentials);
} finally {
signOut(session.id);
}
}
/** Reads the account in a signed-in session. */
Account read(Session session, Credentials credentials) throws IOException, MyTsException {
return new Account(session.info, pull(session.id, itemKey(credentials, session)));
}
/** A signed-in session: the bearer token later requests carry, and who it belongs to. */
record Session(String id, AccountInfo info, byte[] keyPackage, byte[] userId, long createdAt,
byte[] publicKey, byte[] encryptedPrivateKey, byte[] publicSignature) {
}
/** Reads only the server identity; does not download cloud items. */
MyTeamSpeakIdentity serverIdentity(Credentials credentials) throws IOException, MyTsException {
Session session = signIn(credentials);
byte[] itemKey = null;
MyTeamSpeakIdentity serverIdentity(Session session, Credentials credentials) throws MyTsException {
byte[] itemKey = itemKey(credentials, session);
try {
if (session.keyPackage == null) throw new MyTsException("The account has no encryption key", 0);
itemKey = MyTsCrypto.unwrapItemKey(credentials.accountKey(), session.keyPackage);
return new MyTeamSpeakIdentity(session.userId, session.createdAt, session.publicKey,
session.encryptedPrivateKey, session.publicSignature, itemKey);
} catch (GeneralSecurityException e) {
throw new MyTsException("Could not unlock the myTeamSpeak server identity", e);
} finally {
if (itemKey != null) java.util.Arrays.fill(itemKey, (byte) 0);
signOut(session.id);
java.util.Arrays.fill(itemKey, (byte) 0);
}
}
@@ -185,61 +183,217 @@ public final class MyTeamSpeak {
}
private List<SyncItem> pull(String session, byte[] itemKey) throws IOException, MyTsException {
ProtobufWriter request = new ProtobufWriter().string(1, session);
for (int itemClass : ITEM_CLASSES) {
request.message(2, new ProtobufWriter().varint(1, itemClass).string(2, NO_VERSION));
}
request.string(3, NO_VERSION).varint(4, SYNC_VERSION_1_1);
ProtobufReader reply = new ProtobufReader(
transport.call("synchronization", "requestServerItems", request.toByteArray()));
Map<Integer, String> cursors = new TreeMap<>();
for (int itemClass : ITEM_CLASSES) cursors.put(itemClass, NO_VERSION);
SyncReply reply = exchange(session, "requestServerItems", NO_VERSION, cursors, Map.of());
List<SyncItem> items = new ArrayList<>();
for (int wanted : ITEM_CLASSES) {
SyncReply.ItemClass itemClass = reply.classes.get(wanted);
if (itemClass == null) continue;
for (SyncReply.Detail detail : itemClass.details) {
if (detail.deleted || detail.blob == null || detail.blob.length == 0) continue;
SyncItem item = decode(wanted, detail.blob, itemKey);
if (item != null) items.add(item);
}
}
return SyncOrder.sorted(items);
}
/** One item as the account holds it: its class, identity, the server's revision of it and its plain {@code Item_Data}. */
record CloudItem(int itemClass, String uuid, String version, byte[] data) {
}
/**
* What changed in the account since the given cursors.
*
* @param global the account-wide cursor, which the official client sends back with every request
* @param cursors every class's cursor now, to send next time
* @param changed the classes asked for that changed, each with all of its items; a class
* not in here is as it was
*/
record Pull(String global, Map<Integer, String> cursors, Map<Integer, List<CloudItem>> changed) {
}
/**
* Reads the classes that changed since {@code cursors} (all of them for an empty map),
* decrypting the items of the {@code wanted} ones.
*/
Pull pull(Session session, Credentials credentials, String global, Map<Integer, String> cursors,
Set<Integer> wanted) throws IOException, MyTsException {
byte[] itemKey = itemKey(credentials, session);
SyncReply reply = exchange(session.id, "requestServerItems", global, advertised(cursors, wanted), Map.of());
Map<Integer, String> now = new TreeMap<>(cursors);
Map<Integer, List<CloudItem>> changed = new TreeMap<>();
for (SyncReply.ItemClass itemClass : reply.classes.values()) {
if (itemClass.notInDb) {
now.remove(itemClass.id);
if (wanted.contains(itemClass.id) && cursors.containsKey(itemClass.id)) {
changed.put(itemClass.id, List.of());
}
continue;
}
now.put(itemClass.id, itemClass.version);
if (!wanted.contains(itemClass.id)) continue;
List<CloudItem> items = new ArrayList<>();
for (SyncReply.Detail detail : itemClass.details) {
if (detail.deleted || detail.blob == null || detail.blob.length == 0) continue;
try {
items.add(new CloudItem(itemClass.id, detail.uuid, detail.version,
MyTsCrypto.decryptItem(detail.blob, itemKey)));
} catch (GeneralSecurityException e) {
throw new MyTsException("Could not decrypt the account's items", e);
}
}
changed.put(itemClass.id, items);
}
return new Pull(reply.global.isEmpty() ? global : reply.global, now, changed);
}
/** New contents for one class: its new revision, and the items to store; one without data is deleted. */
record ClassUpload(String version, List<CloudItem> items) {
}
/**
* Stores items in the account, but only if none of the {@code watched} classes changed since
* {@code cursors}: the server takes whatever it is sent, so the check is the client's.
*
* @return the account-wide cursor after the change
* @throws MyTsException {@link MyTsException#CHANGED_MEANWHILE} when one did
*/
String push(Session session, Credentials credentials, String global, Map<Integer, String> cursors,
Set<Integer> watched, Map<Integer, ClassUpload> uploads) throws IOException, MyTsException {
byte[] itemKey = itemKey(credentials, session);
SyncReply now = exchange(session.id, "requestServerItems", global, advertised(cursors, watched), Map.of());
for (int itemClass : watched) {
SyncReply.ItemClass reply = now.classes.get(itemClass);
boolean known = cursors.containsKey(itemClass);
if (reply != null && (known || !reply.notInDb)) {
throw new MyTsException("myTeamSpeak changed meanwhile; synchronize again", MyTsException.CHANGED_MEANWHILE);
}
}
Map<Integer, String> versions = new TreeMap<>();
Map<Integer, List<SyncReply.Detail>> details = new TreeMap<>();
for (Map.Entry<Integer, ClassUpload> upload : uploads.entrySet()) {
versions.put(upload.getKey(), upload.getValue().version());
List<SyncReply.Detail> list = new ArrayList<>();
for (CloudItem item : upload.getValue().items()) {
try {
list.add(item.data() == null
? new SyncReply.Detail(item.uuid(), item.version(), null, true)
: new SyncReply.Detail(item.uuid(), item.version(),
MyTsCrypto.encryptItem(item.data(), itemKey), false));
} catch (GeneralSecurityException e) {
throw new MyTsException("This platform lacks the ciphers myTeamSpeak needs", e);
}
}
details.put(upload.getKey(), list);
}
SyncReply stored = exchange(session.id, "synchronizeItems", global, versions, details);
if (stored.status != IN_SYNC) {
throw new MyTsException("myTeamSpeak did not store the changes (status " + stored.status + ")",
stored.status);
}
return stored.global;
}
/** The cursors to send: every known one, and none yet for a wanted class never seen, so it comes whole. */
private static Map<Integer, String> advertised(Map<Integer, String> cursors, Set<Integer> wanted) {
Map<Integer, String> out = new TreeMap<>(cursors);
for (int itemClass : wanted) out.putIfAbsent(itemClass, NO_VERSION);
return out;
}
private static byte[] itemKey(Credentials credentials, Session session) throws MyTsException {
if (session.keyPackage == null) throw new MyTsException("The account has no synchronisation key", 0);
try {
return MyTsCrypto.unwrapItemKey(credentials.accountKey(), session.keyPackage);
} catch (GeneralSecurityException e) {
throw new MyTsException("Could not unlock the account's encrypted data", e);
}
}
/** A {@code Sync_Reply_ItemClasses}. */
private record SyncReply(int status, String global, Map<Integer, ItemClass> classes) {
record ItemClass(int id, String version, boolean notInDb, List<Detail> details) {
}
record Detail(String uuid, String version, byte[] blob, boolean deleted) {
}
}
/**
* One synchronisation call: the cursor of each class and the details to send under it.
* Fails unless the server answered with a synchronisation status.
*/
private SyncReply exchange(String session, String method, String global, Map<Integer, String> versions,
Map<Integer, List<SyncReply.Detail>> details) throws IOException, MyTsException {
ProtobufWriter request = new ProtobufWriter().string(1, session);
for (Map.Entry<Integer, String> cursor : versions.entrySet()) {
ProtobufWriter itemClass = new ProtobufWriter().varint(1, cursor.getKey()).string(2, cursor.getValue());
for (SyncReply.Detail d : details.getOrDefault(cursor.getKey(), List.of())) {
ProtobufWriter detail = new ProtobufWriter().string(1, d.uuid()).string(2, d.version());
if (d.blob() != null) detail.bytes(3, d.blob());
if (d.deleted()) detail.varint(4, 1);
itemClass.message(3, detail);
}
request.message(2, itemClass);
}
request.string(3, global).varint(4, SYNC_VERSION_1_1);
ProtobufReader reply = new ProtobufReader(transport.call("synchronization", method, request.toByteArray()));
List<Frame> frames = new ArrayList<>();
int status = 0;
String replyGlobal = "";
Map<Integer, SyncReply.ItemClass> classes = new TreeMap<>();
while (reply.next()) {
switch (reply.fieldNumber()) {
case 1 -> status = (int) reply.readVarint();
case 2 -> collectFrames(reply.readMessage(), frames);
case 2 -> {
SyncReply.ItemClass itemClass = itemClass(reply.readMessage());
classes.put(itemClass.id(), itemClass);
}
case 3 -> replyGlobal = reply.readString();
default -> reply.skip();
}
}
if (status < IN_SYNC || status > NOT_IN_DB) {
throw new MyTsException("myTeamSpeak refused to synchronise (status " + status + ")", status);
}
List<SyncItem> items = new ArrayList<>();
for (Frame frame : frames) {
SyncItem item = decode(frame, itemKey);
if (item != null) items.add(item);
}
return SyncOrder.sorted(items);
return new SyncReply(status, replyGlobal, classes);
}
/**
* The encrypted items of one {@code Sync_ItemClasses_Data}; tombstones carry none.
* The server answers with every class the account holds, whatever was asked for,
* and not all of them are encrypted (add-on flags come in the clear), so only the
* wanted classes are kept.
*/
private static void collectFrames(ProtobufReader itemClass, List<Frame> frames) throws IOException {
int classId = 0;
List<byte[]> blobs = new ArrayList<>();
while (itemClass.next()) {
switch (itemClass.fieldNumber()) {
case 1 -> classId = (int) itemClass.readVarint();
case 3 -> {
byte[] blob = itemBlob(itemClass.readMessage());
if (blob != null) blobs.add(blob);
}
default -> itemClass.skip();
private static SyncReply.ItemClass itemClass(ProtobufReader r) throws IOException {
int id = 0;
String version = "";
boolean notInDb = false;
List<SyncReply.Detail> details = new ArrayList<>();
while (r.next()) {
switch (r.fieldNumber()) {
case 1 -> id = (int) r.readVarint();
case 2 -> version = r.readString();
case 3 -> details.add(detail(r.readMessage()));
case 4 -> notInDb = r.readBool();
default -> r.skip();
}
}
for (int wanted : ITEM_CLASSES) {
if (wanted != classId) continue;
for (byte[] blob : blobs) frames.add(new Frame(classId, blob));
}
return new SyncReply.ItemClass(id, version, notInDb, details);
}
private record Frame(int itemClass, byte[] blob) {
private static SyncReply.Detail detail(ProtobufReader r) throws IOException {
String uuid = "", version = "";
byte[] blob = null;
boolean deleted = false;
while (r.next()) {
switch (r.fieldNumber()) {
case 1 -> uuid = r.readString();
case 2 -> version = r.readString();
case 3 -> blob = r.readBytes();
case 4 -> deleted = r.readBool();
default -> r.skip();
}
}
return new SyncReply.Detail(uuid, version, blob, deleted);
}
/**
@@ -247,33 +401,20 @@ public final class MyTeamSpeak {
* decrypt is taken for plaintext, like add-on flags, or else left out — not allowed to
* fail the bookmarks and identities.
*/
private static SyncItem decode(Frame frame, byte[] itemKey) throws IOException, MyTsException {
private static SyncItem decode(int itemClass, byte[] blob, byte[] itemKey) throws IOException, MyTsException {
try {
return SyncItemDecoder.decode(MyTsCrypto.decryptItem(frame.blob(), itemKey));
return SyncItemDecoder.decode(MyTsCrypto.decryptItem(blob, itemKey));
} catch (GeneralSecurityException e) {
if (frame.itemClass() != CONTACTS) throw new MyTsException("Could not decrypt the account's items", e);
if (itemClass != CONTACTS) throw new MyTsException("Could not decrypt the account's items", e);
}
try {
SyncItem item = SyncItemDecoder.decode(frame.blob());
SyncItem item = SyncItemDecoder.decode(blob);
return item instanceof SyncItem.Contact ? item : null;
} catch (IOException | RuntimeException notPlaintext) {
return null;
}
}
private static byte[] itemBlob(ProtobufReader detail) throws IOException {
byte[] blob = null;
boolean deleted = false;
while (detail.next()) {
switch (detail.fieldNumber()) {
case 3 -> blob = detail.readBytes();
case 4 -> deleted = detail.readBool();
default -> detail.skip();
}
}
return !deleted && blob != null && blob.length > 0 ? blob : null;
}
/** Ends the session rather than leave it for the server to expire; nothing to do if that fails. */
void signOut(String session) {
if (session.isEmpty()) return;
@@ -14,7 +14,8 @@ import java.util.Properties;
* The myTeamSpeak account this client stays signed in to. Signing in keeps the
* {@link MyTeamSpeak.Credentials} in the profile, private to the user, so the account
* can be read again later without asking for the password; signing out forgets them.
* Each read signs in afresh; only {@link MyTsFiles} keeps a session.
* Requests share one session, as the official client's do: signed in on first use, again
* when the service lets it run out, and ended on sign-out or {@link #endSession}.
*/
public final class MyTeamSpeakLogin {
@@ -25,6 +26,8 @@ public final class MyTeamSpeakLogin {
private MyTeamSpeak.Credentials credentials;
private String username = "";
private MyTeamSpeakIdentity serverIdentity;
private MyTeamSpeak.Session session;
private MyTeamSpeak.Credentials sessionCredentials;
private final Object identityLoadLock = new Object();
private long revision;
@@ -63,15 +66,27 @@ public final class MyTeamSpeakLogin {
*/
public MyTeamSpeak.Account signIn(String email, String password) throws IOException, MyTsException {
MyTeamSpeak.Credentials fresh = MyTeamSpeak.Credentials.derive(email.trim(), password);
MyTeamSpeak.Account account = service.download(fresh);
MyTeamSpeak.Session started = service.signIn(fresh);
MyTeamSpeak.Account account;
try {
account = service.read(started, fresh);
} catch (IOException | MyTsException | RuntimeException e) {
service.signOut(started.id());
throw e;
}
MyTeamSpeak.Session previous;
synchronized (this) {
if (serverIdentity != null) serverIdentity.close();
serverIdentity = null;
previous = session;
session = started;
sessionCredentials = fresh;
credentials = fresh;
revision++;
username = account.username();
write();
}
if (previous != null) service.signOut(previous.id());
return account;
}
@@ -82,13 +97,8 @@ public final class MyTeamSpeakLogin {
* @throws IllegalStateException when signed out
*/
public MyTeamSpeak.Account fetch() throws IOException, MyTsException {
MyTeamSpeak.Credentials current;
synchronized (this) {
if (credentials == null) throw new IllegalStateException("Not signed in to myTeamSpeak");
current = credentials;
}
try {
MyTeamSpeak.Account account = service.download(current);
return request((s, current) -> {
MyTeamSpeak.Account account = service.read(s, current);
synchronized (this) {
if (credentials == current && !account.username().equals(username)) {
username = account.username();
@@ -96,6 +106,37 @@ public final class MyTeamSpeakLogin {
}
}
return account;
});
}
/** A request made in the kept session. */
interface Request<T> {
T run(MyTeamSpeak.Session session, MyTeamSpeak.Credentials credentials) throws IOException, MyTsException;
}
/**
* Makes a request in the kept session, signing in when there is none or it ran out, and
* signing out if the server no longer takes the credentials.
*
* @throws IllegalStateException when signed out
*/
<T> T request(Request<T> request) throws IOException, MyTsException {
MyTeamSpeak.Credentials current;
synchronized (this) {
if (credentials == null) throw new IllegalStateException("Not signed in to myTeamSpeak");
current = credentials;
}
try {
MyTeamSpeak.Session s = session(current);
try {
return request.run(s, current);
} catch (MyTsException e) {
if (!e.sessionExpired()) throw e;
synchronized (this) {
if (session == s) session = null;
}
return request.run(session(current), current);
}
} catch (MyTsException e) {
if (!e.credentialsRejected()) throw e;
synchronized (this) {
@@ -106,6 +147,30 @@ public final class MyTeamSpeakLogin {
}
}
private synchronized MyTeamSpeak.Session session(MyTeamSpeak.Credentials current) throws IOException, MyTsException {
if (session != null && sessionCredentials == current) return session;
endSession();
session = service.signIn(current);
sessionCredentials = current;
return session;
}
/** Ends the kept session, as on exit; the next request signs in again. Blocks for a request. */
public void endSession() {
MyTeamSpeak.Session ended;
synchronized (this) {
ended = session;
session = null;
sessionCredentials = null;
}
if (ended != null) service.signOut(ended.id());
}
/** Where {@link MyTeamSpeakSync} keeps what it last synchronised; it goes with the sign-in. */
File syncStateFile() {
return new File(file.getAbsoluteFile().getParentFile(), SyncState.FILE_NAME);
}
/** What signs in, or {@code null} when signed out. */
synchronized MyTeamSpeak.Credentials credentials() {
return credentials;
@@ -124,15 +189,7 @@ public final class MyTeamSpeakLogin {
if (serverIdentity != null) return serverIdentity;
current = credentials;
}
MyTeamSpeakIdentity loaded;
try {
loaded = service.serverIdentity(current);
} catch (MyTsException e) {
synchronized (this) {
if (e.credentialsRejected() && credentials == current) signOut();
}
throw e;
}
MyTeamSpeakIdentity loaded = request(service::serverIdentity);
synchronized (this) {
if (credentials != current) {
loaded.close();
@@ -144,7 +201,16 @@ public final class MyTeamSpeakLogin {
}
}
/** Forgets the sign-in; the session ends in the background. */
public synchronized void signOut() {
MyTeamSpeak.Session ended = session;
session = null;
sessionCredentials = null;
if (ended != null) {
Thread end = new Thread(() -> service.signOut(ended.id()), "myteamspeak-sign-out");
end.setDaemon(true);
end.start();
}
if (serverIdentity != null) serverIdentity.close();
serverIdentity = null;
credentials = null;
@@ -152,6 +218,8 @@ public final class MyTeamSpeakLogin {
username = "";
//noinspection ResultOfMethodCallIgnored
file.delete();
//noinspection ResultOfMethodCallIgnored
syncStateFile().delete();
}
private void read() {
@@ -0,0 +1,498 @@
package eu.lixko.ts3j.client.myts;
import eu.lixko.ts3j.client.config.Bookmarks;
import eu.lixko.ts3j.client.config.IdentityStore;
import eu.lixko.ts3j.client.config.Settings;
import eu.lixko.ts3j.client.proto.ProtobufMessage;
import eu.lixko.ts3j.client.proto.ProtobufReader;
import eu.lixko.ts3j.client.teamspeak.SyncItem;
import eu.lixko.ts3j.client.teamspeak.SyncItemDecoder;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TreeMap;
import java.util.UUID;
/**
* Two-way synchronisation of bookmarks and identities with the signed-in myTeamSpeak account.
* Against what both sides held after the last run, a change on one side is carried to the other,
* deletions included; an item changed on both sides is a {@link Conflict} the user settles.
* The first run, with nothing to go by, pairs items up by identity key or bookmark name and
* address, and carries over whatever has no partner.
*
* <p>A run is four steps, so that the user can be asked in between and so that the stores are
* touched only on the thread that owns them: {@link #fetch} and {@link #upload} block on the
* network, {@link #plan} and {@link #apply} read and change the stores. Nothing changes on
* either side before {@link #upload}, and here only once it succeeded.
*
* <p>The server takes whatever it is sent, so detecting a conflict is up to the client; see
* {@code docs/myteamspeak/IMPLEMENTATION.md}.
*/
public final class MyTeamSpeakSync {
private static final int BOOKMARK = 0;
private static final int IDENTITY = 1;
private static final Set<Integer> CLASSES = Set.of(BOOKMARK, IDENTITY);
private static final int ITEM_ADDED = 0;
/** What the official client writes on every change it sends, though the flag reads "not manipulated". */
private static final int ITEM_COMMITTED = 3;
private final MyTeamSpeakLogin login;
private final IdentityStore identities;
private final Bookmarks bookmarks;
private final Settings settings;
private final File stateFile;
public MyTeamSpeakSync(MyTeamSpeakLogin login, IdentityStore identities, Bookmarks bookmarks, Settings settings) {
this.login = login;
this.identities = identities;
this.bookmarks = bookmarks;
this.settings = settings;
this.stateFile = login.syncStateFile();
}
/** The account as it is now, against how it was after the last run. */
public static final class Remote {
final String account;
final SyncState base;
final MyTeamSpeak.Pull pull;
Remote(String account, SyncState base, MyTeamSpeak.Pull pull) {
this.account = account;
this.base = base;
this.pull = pull;
}
}
/** What a run did. */
public record Result(int sent, int received) {
}
/** Step 1: reads what changed in the account. Blocks on the network. */
public Remote fetch() throws IOException, MyTsException {
return login.request((session, credentials) -> {
SyncState base = SyncState.load(stateFile, credentials.email());
return new Remote(credentials.email(), base,
login.service().pull(session, credentials, base.global, base.cursors, CLASSES));
});
}
/** Step 2: works out what goes which way, on the thread that owns the stores. */
public Plan plan(Remote remote) throws IOException {
SyncKind.Links links = new SyncKind.Links();
SyncKind identityKind = new SyncKind.IdentityKind(identities, bookmarks, settings, links);
SyncKind bookmarkKind = new SyncKind.BookmarkKind(identities, bookmarks, settings, links);
Map<Integer, List<MyTeamSpeak.CloudItem>> cloud = new HashMap<>();
Map<Integer, List<SyncState.Entry>> base = new HashMap<>();
for (int itemClass : CLASSES) {
base.put(itemClass, new ArrayList<>());
List<MyTeamSpeak.CloudItem> changed = remote.pull.changed().get(itemClass);
if (changed != null) cloud.put(itemClass, changed);
}
for (SyncState.Entry entry : remote.base.items.values()) {
if (!CLASSES.contains(entry.itemClass())) continue;
base.get(entry.itemClass()).add(entry);
if (!remote.pull.changed().containsKey(entry.itemClass())) {
cloud.computeIfAbsent(entry.itemClass(), k -> new ArrayList<>()).add(
new MyTeamSpeak.CloudItem(entry.itemClass(), entry.uuid(), entry.version(), entry.data()));
}
}
// Bookmarks name identities the cloud holds, now or before; the current ones take precedence.
for (MyTeamSpeak.CloudItem item : cloud.getOrDefault(IDENTITY, List.of())) {
if (decode(item.data()) instanceof SyncItem.Identity identity) links.addCloudIdentity(identity);
}
for (SyncState.Entry entry : base.get(IDENTITY)) {
if (decode(entry.data()) instanceof SyncItem.Identity identity) links.addCloudIdentity(identity);
}
Plan plan = new Plan(remote, links);
reconcile(plan, identityKind, base.get(IDENTITY), cloud.getOrDefault(IDENTITY, List.of()));
for (Step step : plan.allSteps()) {
if (step.kind == identityKind && step.localId != null && !step.localId.isEmpty()) {
links.uuidByLocal.put(step.localId, step.uuid);
}
}
reconcile(plan, bookmarkKind, base.get(BOOKMARK), cloud.getOrDefault(BOOKMARK, List.of()));
return plan;
}
/** Step 3: stores this side's changes in the account. Blocks on the network. */
public void upload(Plan plan) throws IOException, MyTsException {
if (!plan.isResolved()) throw new IllegalStateException("Unsettled conflicts");
Map<Integer, MyTeamSpeak.ClassUpload> uploads = new TreeMap<>();
Map<Integer, String> versions = new TreeMap<>();
Map<Integer, List<MyTeamSpeak.CloudItem>> items = new TreeMap<>();
for (Step step : plan.effectiveSteps()) {
if (step.upload == null) continue;
items.computeIfAbsent(step.kind.itemClass(), k -> new ArrayList<>()).add(step.upload);
}
for (Map.Entry<Integer, List<MyTeamSpeak.CloudItem>> e : items.entrySet()) {
String version = UUID.randomUUID().toString();
versions.put(e.getKey(), version);
uploads.put(e.getKey(), new MyTeamSpeak.ClassUpload(version, e.getValue()));
}
if (!uploads.isEmpty()) {
login.request((session, credentials) -> {
if (!credentials.email().equalsIgnoreCase(plan.remote.account)) {
throw new MyTsException("Signed in to another account meanwhile", MyTsException.CHANGED_MEANWHILE);
}
plan.uploadedGlobal = login.service().push(session, credentials, plan.remote.pull.global(),
plan.remote.pull.cursors(), CLASSES, uploads);
return null;
});
}
plan.uploadedVersions = versions;
}
/** Step 4: brings the account's changes here and remembers the outcome for the next run. */
public Result apply(Plan plan) throws IOException {
if (plan.uploadedVersions == null) throw new IllegalStateException("Not uploaded");
SyncState state = new SyncState();
state.cursors.putAll(plan.remote.pull.cursors());
state.cursors.putAll(plan.uploadedVersions);
state.global = plan.uploadedGlobal != null ? plan.uploadedGlobal : plan.remote.pull.global();
// Unmodelled classes keep what the last run knew of them.
for (SyncState.Entry entry : plan.remote.base.items.values()) {
if (!CLASSES.contains(entry.itemClass())) state.items.put(entry.uuid(), entry);
}
int sent = 0, received = 0;
for (Step step : plan.effectiveSteps()) {
int itemClass = step.kind.itemClass();
switch (step.action) {
case KEEP -> keep(state, step.cloud, step.localId);
case PUSH -> {
state.items.put(step.uuid, new SyncState.Entry(itemClass, step.uuid, step.upload.version(),
step.localId, step.upload.data()));
sent++;
}
case PUSH_DELETE -> sent++;
case PULL -> {
SyncItem item = decode(step.cloud.data());
String localId = step.localId == null || step.localId.isEmpty()
? step.kind.add(item) : step.kind.update(step.localId, item);
keep(state, step.cloud, localId);
received++;
}
case PULL_DELETE -> {
step.kind.remove(step.localId);
received++;
}
case FORGET -> {
}
}
if (itemClass == IDENTITY && state.items.containsKey(step.uuid)) {
plan.links.localByUuid.put(step.uuid, state.items.get(step.uuid).localId());
}
}
bookmarks.save();
settings.save();
state.save(stateFile, plan.remote.account);
return new Result(sent, received);
}
private static void keep(SyncState state, MyTeamSpeak.CloudItem item, String localId) {
state.items.put(item.uuid(), new SyncState.Entry(item.itemClass(), item.uuid(), item.version(),
localId == null ? "" : localId, item.data()));
}
/**
* The three-way comparison of one class: what the last run left (base), what is here
* (local) and what is in the account (cloud).
*/
private void reconcile(Plan plan, SyncKind kind, List<SyncState.Entry> base, List<MyTeamSpeak.CloudItem> cloud)
throws IOException {
Map<String, MyTeamSpeak.CloudItem> cloudByUuid = new LinkedHashMap<>();
for (MyTeamSpeak.CloudItem item : cloud) cloudByUuid.put(item.uuid(), item);
Set<String> claimed = new HashSet<>();
for (SyncState.Entry entry : base) {
MyTeamSpeak.CloudItem now = cloudByUuid.remove(entry.uuid());
SyncFields baseFields = fieldsOf(kind, entry.data());
if (baseFields == null || entry.localId().isEmpty()) {
// Not something this client models: left as the account has it.
if (now != null) plan.add(Step.keep(kind, now, ""));
continue;
}
SyncFields local = kind.localFields(entry.localId());
if (local != null) claimed.add(entry.localId());
SyncFields remote = now == null ? null : fieldsOf(kind, now.data());
if (now != null && remote == null) {
plan.add(Step.keep(kind, now, ""));
continue;
}
boolean localChanged = local != null && !local.sameAs(baseFields);
boolean cloudChanged = now != null && !now.version().equals(entry.version());
if (now == null && local == null) {
plan.add(new Step(kind, Action.FORGET, entry.uuid(), entry.localId(), null, null));
} else if (now == null) {
Step delete = new Step(kind, Action.PULL_DELETE, entry.uuid(), entry.localId(), null, null);
if (!localChanged) {
plan.add(delete);
} else {
plan.conflict(kind, local, null, push(kind, entry.uuid(), entry.localId(),
new MyTeamSpeak.CloudItem(kind.itemClass(), entry.uuid(), entry.version(), entry.data()),
true), delete);
}
} else if (local == null) {
Step delete = new Step(kind, Action.PUSH_DELETE, entry.uuid(), entry.localId(), null,
new MyTeamSpeak.CloudItem(kind.itemClass(), entry.uuid(), UUID.randomUUID().toString(), null));
if (!cloudChanged) plan.add(delete);
else plan.conflict(kind, null, remote, delete, Step.pull(kind, now, null));
} else if (!localChanged && !cloudChanged) {
plan.add(Step.keep(kind, now, entry.localId()));
} else if (!cloudChanged) {
plan.add(push(kind, entry.uuid(), entry.localId(), now, false));
} else if (!localChanged || local.sameAs(remote)) {
plan.add(local.sameAs(remote) ? Step.keep(kind, now, entry.localId())
: Step.pull(kind, now, entry.localId()));
} else {
plan.conflict(kind, local, remote, push(kind, entry.uuid(), entry.localId(), now, false),
Step.pull(kind, now, entry.localId()));
}
}
// New on either side: pair them up, carry over the rest.
Map<String, String> unpaired = new LinkedHashMap<>();
for (String localId : kind.localIds()) {
if (claimed.contains(localId)) continue;
SyncFields local = kind.localFields(localId);
if (local != null) unpaired.putIfAbsent(kind.matchKey(local), localId);
}
Set<String> paired = new HashSet<>();
for (MyTeamSpeak.CloudItem now : cloudByUuid.values()) {
SyncFields remote = fieldsOf(kind, now.data());
if (remote == null) {
plan.add(Step.keep(kind, now, ""));
continue;
}
String localId = unpaired.remove(kind.matchKey(remote));
if (localId == null) {
plan.add(Step.pull(kind, now, null));
continue;
}
paired.add(localId);
SyncFields local = kind.localFields(localId);
if (local.sameAs(remote)) {
plan.add(Step.keep(kind, now, localId));
} else {
plan.conflict(kind, local, remote, push(kind, now.uuid(), localId, now, false),
Step.pull(kind, now, localId));
}
}
for (String localId : kind.localIds()) {
if (claimed.contains(localId) || paired.contains(localId)) continue;
plan.add(push(kind, UUID.randomUUID().toString(), localId, null, true));
}
}
/**
* Sending the local item as {@code uuid}: written over {@code onto}, keeping whatever this
* client does not model, or from scratch when there is nothing to write over.
*/
private static Step push(SyncKind kind, String uuid, String localId, MyTeamSpeak.CloudItem onto, boolean added)
throws IOException {
String version = UUID.randomUUID().toString();
SyncFields local = kind.localFields(localId);
ProtobufMessage item;
ProtobufMessage content;
Set<String> labels;
if (onto == null) {
item = new ProtobufMessage().string(2, uuid).string(3, version).varint(4, ITEM_ADDED)
.varint(6, kind.itemClass()).string(7, "").string(8, "");
content = new ProtobufMessage();
labels = new LinkedHashSet<>(local.values().keySet());
} else {
// Like the official client: the revision written over becomes the last known one, and a
// client holding that revision takes the change.
item = new ProtobufMessage(onto.data());
ProtobufReader previous = item.get(3);
if (previous != null) item.string(5, previous.readString());
item.string(3, version).varint(4, added ? ITEM_ADDED : ITEM_COMMITTED);
ProtobufReader existing = item.get(kind.contentField());
content = existing == null ? new ProtobufMessage() : new ProtobufMessage(existing.readBytes());
SyncFields before = fieldsOf(kind, onto.data());
labels = before == null ? new LinkedHashSet<>(local.values().keySet()) : local.changedFrom(before);
}
item.varint(9, System.currentTimeMillis() / 1000);
kind.write(localId, content, labels);
item.message(kind.contentField(), content);
return new Step(kind, Action.PUSH, uuid, localId, null,
new MyTeamSpeak.CloudItem(kind.itemClass(), uuid, version, item.toByteArray()));
}
private static SyncFields fieldsOf(SyncKind kind, byte[] data) {
SyncItem item = decode(data);
return item == null ? null : kind.cloudFields(item);
}
private static SyncItem decode(byte[] data) {
try {
return SyncItemDecoder.decode(data);
} catch (IOException | RuntimeException unreadable) {
return null;
}
}
private enum Action { KEEP, PUSH, PUSH_DELETE, PULL, PULL_DELETE, FORGET }
/**
* What happens to one item. {@code cloud} is the account's copy where there is one to keep or
* bring here; {@code upload} what is sent, without data for a deletion.
*/
private record Step(SyncKind kind, Action action, String uuid, String localId,
MyTeamSpeak.CloudItem cloud, MyTeamSpeak.CloudItem upload) {
static Step keep(SyncKind kind, MyTeamSpeak.CloudItem cloud, String localId) {
return new Step(kind, Action.KEEP, cloud.uuid(), localId, cloud, null);
}
static Step pull(SyncKind kind, MyTeamSpeak.CloudItem cloud, String localId) {
return new Step(kind, Action.PULL, cloud.uuid(), localId, cloud, null);
}
boolean sends() {
return action == Action.PUSH || action == Action.PUSH_DELETE;
}
boolean receives() {
return action == Action.PULL || action == Action.PULL_DELETE;
}
}
/** What a run will do, once its conflicts are settled. */
public static final class Plan {
final Remote remote;
final SyncKind.Links links;
private final List<Step> steps = new ArrayList<>();
private final List<Conflict> conflicts = new ArrayList<>();
/** The class revisions sent; set once {@link #upload} succeeded. */
volatile Map<Integer, String> uploadedVersions;
/** The account-wide cursor the upload left, {@code null} when nothing was sent. */
volatile String uploadedGlobal;
Plan(Remote remote, SyncKind.Links links) {
this.remote = remote;
this.links = links;
}
void add(Step step) {
steps.add(step);
}
void conflict(SyncKind kind, SyncFields here, SyncFields cloud, Step keepHere, Step keepCloud) {
conflicts.add(new Conflict(kind.title(here != null ? here : cloud),
kind instanceof SyncKind.IdentityKind, here, cloud, keepHere, keepCloud));
}
/** Every step either choice of a conflict could take, for the identity links. */
List<Step> allSteps() {
List<Step> out = new ArrayList<>(steps);
for (Conflict c : conflicts) {
out.add(c.keepHere);
out.add(c.keepCloud);
}
return out;
}
/** The steps, the conflicts' as settled, identities before the bookmarks that use them. */
List<Step> effectiveSteps() {
List<Step> out = new ArrayList<>(steps);
for (Conflict c : conflicts) out.add(c.choice == Conflict.Choice.HERE ? c.keepHere : c.keepCloud);
out.sort((a, b) -> Integer.compare(b.kind.itemClass(), a.kind.itemClass()));
return out;
}
public List<Conflict> conflicts() {
return List.copyOf(conflicts);
}
public boolean isResolved() {
return conflicts.stream().allMatch(c -> c.choice != null);
}
/** Items to send to the account, not counting conflicts. */
public int toSend() {
return (int) steps.stream().filter(Step::sends).count();
}
/** Items to bring here from the account, not counting conflicts. */
public int toReceive() {
return (int) steps.stream().filter(Step::receives).count();
}
}
/** An item changed both here and in the account since the last run. */
public static final class Conflict {
public enum Choice { HERE, CLOUD }
/** One synchronised field of the item, as each side has it; empty on a side where the item is gone. */
public record Difference(String label, String here, String cloud, boolean differs) {
}
private final String title;
private final boolean identity;
private final SyncFields here;
private final SyncFields cloud;
final Step keepHere;
final Step keepCloud;
private volatile Choice choice;
Conflict(String title, boolean identity, SyncFields here, SyncFields cloud, Step keepHere, Step keepCloud) {
this.title = title;
this.identity = identity;
this.here = here;
this.cloud = cloud;
this.keepHere = keepHere;
this.keepCloud = keepCloud;
}
public String title() {
return title;
}
/** An identity; otherwise a bookmark. */
public boolean isIdentity() {
return identity;
}
public boolean deletedHere() {
return here == null;
}
public boolean deletedInCloud() {
return cloud == null;
}
public List<Difference> differences() {
SyncFields any = here != null ? here : cloud;
List<Difference> out = new ArrayList<>();
for (String label : any.values().keySet()) {
String h = here == null ? "" : here.display(label);
String c = cloud == null ? "" : cloud.display(label);
boolean differs = here == null || cloud == null || !here.compare(label).equals(cloud.compare(label));
out.add(new Difference(label, h, c, differs));
}
return out;
}
public Choice choice() {
return choice;
}
public void choose(Choice choice) {
this.choice = choice;
}
}
}
@@ -8,6 +8,7 @@ import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.GeneralSecurityException;
import java.security.MessageDigest;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Base64;
@@ -28,6 +29,7 @@ final class MyTsCrypto {
private static final int KEY_PACKAGE_VERSION = 2;
private static final int CTR_BLOCK_BYTES = 16;
private static final int ITEM_HASH_BYTES = 64;
private static final SecureRandom RANDOM = new SecureRandom();
private MyTsCrypto() {
}
@@ -68,23 +70,14 @@ final class MyTsCrypto {
/**
* Decrypts an item frame, {@code iv[16] || ciphertext || SHA-512(plaintext)}, into
* its {@code Item_Data} message. The cipher is AES-256-CTR, but with the counter
* incremented little-endian, so JCA's own CTR mode cannot be used.
* its {@code Item_Data} message.
*/
static byte[] decryptItem(byte[] frame, byte[] itemKey) throws GeneralSecurityException {
int textLength = frame.length - CTR_BLOCK_BYTES - ITEM_HASH_BYTES;
if (textLength <= 0) throw new GeneralSecurityException("Truncated item");
Cipher aes = Cipher.getInstance("AES/ECB/NoPadding");
aes.init(Cipher.ENCRYPT_MODE, new SecretKeySpec(itemKey, "AES"));
byte[] counter = Arrays.copyOf(frame, CTR_BLOCK_BYTES);
byte[] plain = new byte[textLength];
for (int off = 0; off < textLength; off += CTR_BLOCK_BYTES) {
byte[] keyStream = aes.doFinal(counter);
int n = Math.min(CTR_BLOCK_BYTES, textLength - off);
for (int i = 0; i < n; i++) plain[off + i] = (byte) (frame[CTR_BLOCK_BYTES + off + i] ^ keyStream[i]);
incrementLittleEndian(counter);
}
ctr(itemKey, Arrays.copyOf(frame, CTR_BLOCK_BYTES), frame, CTR_BLOCK_BYTES, plain, 0, textLength);
byte[] hash = MessageDigest.getInstance("SHA-512").digest(plain);
byte[] expected = Arrays.copyOfRange(frame, frame.length - ITEM_HASH_BYTES, frame.length);
@@ -92,6 +85,34 @@ final class MyTsCrypto {
return plain;
}
/** The inverse of {@link #decryptItem}, under a fresh random IV. */
static byte[] encryptItem(byte[] plain, byte[] itemKey) throws GeneralSecurityException {
byte[] frame = new byte[CTR_BLOCK_BYTES + plain.length + ITEM_HASH_BYTES];
byte[] iv = new byte[CTR_BLOCK_BYTES];
RANDOM.nextBytes(iv);
System.arraycopy(iv, 0, frame, 0, CTR_BLOCK_BYTES);
ctr(itemKey, iv, plain, 0, frame, CTR_BLOCK_BYTES, plain.length);
byte[] hash = MessageDigest.getInstance("SHA-512").digest(plain);
System.arraycopy(hash, 0, frame, CTR_BLOCK_BYTES + plain.length, ITEM_HASH_BYTES);
return frame;
}
/**
* AES-256-CTR with the counter incremented little-endian, which JCA's own CTR mode
* cannot do. {@code counter} is consumed.
*/
private static void ctr(byte[] key, byte[] counter, byte[] in, int inAt, byte[] out, int outAt, int length)
throws GeneralSecurityException {
Cipher aes = Cipher.getInstance("AES/ECB/NoPadding");
aes.init(Cipher.ENCRYPT_MODE, new SecretKeySpec(key, "AES"));
for (int off = 0; off < length; off += CTR_BLOCK_BYTES) {
byte[] keyStream = aes.doFinal(counter);
int n = Math.min(CTR_BLOCK_BYTES, length - off);
for (int i = 0; i < n; i++) out[outAt + off + i] = (byte) (in[inAt + off + i] ^ keyStream[i]);
incrementLittleEndian(counter);
}
}
/** PBKDF2-HMAC-SHA512 keyed by the password, salted with {@code lower(email) + purpose + password}. */
private static byte[] derive(String email, String password, String purpose, int length)
throws GeneralSecurityException {
@@ -5,6 +5,11 @@ public final class MyTsException extends Exception {
/** {@code ERROR_LOGIN_FAILED}: wrong email or password. */
static final int LOGIN_FAILED = 202;
/** A session the service no longer knows: {@code SyncStatus.ERROR_SESSION_EXPIRED}, the user service's own code. */
private static final int SYNC_SESSION_EXPIRED = 102;
private static final int USER_SESSION_EXPIRED = 109;
/** Ours, not the service's: the account changed between reading it and writing to it. */
static final int CHANGED_MEANWHILE = 1;
private final int code;
@@ -23,6 +28,16 @@ public final class MyTsException extends Exception {
return code;
}
/** The account changed while a synchronisation was under way; reading it again resolves it. */
public boolean changedMeanwhile() {
return code == CHANGED_MEANWHILE;
}
/** The session ran out; signing in again resolves it. */
public boolean sessionExpired() {
return code == SYNC_SESSION_EXPIRED || code == USER_SESSION_EXPIRED;
}
/** The server no longer accepts the email and password, for instance after a password change. */
public boolean credentialsRejected() {
return code == LOGIN_FAILED;
@@ -13,8 +13,7 @@ import java.net.URL;
/**
* Files kept in myTeamSpeak cloud storage, as TeamSpeak 6 shares them in chat. The service
* hands out signed storage addresses: one to put a new file under this account's ID and a
* room, one to get a file someone put under theirs. Both need a signed-in account; the
* session is kept for later requests and renewed when it runs out.
* room, one to get a file someone put under theirs. Both need a signed-in account.
*
* <p>The storage addresses are signed for {@code Content-Type: text/plain} whatever the file
* is, both ways; a request without it is refused.
@@ -29,8 +28,6 @@ public final class MyTsFiles {
private final MyTeamSpeakLogin login;
private final MyTeamSpeak service;
private MyTeamSpeak.Session session;
private MyTeamSpeak.Credentials sessionCredentials;
public MyTsFiles(MyTeamSpeakLogin login) {
this(login, login.service());
@@ -106,15 +103,9 @@ public final class MyTsFiles {
}
}
/** Ends the kept session, e.g. on sign-out or exit; the next request signs in again. */
/** Ends the account's session, e.g. on exit; the next request signs in again. */
public void close() {
MyTeamSpeak.Session ended;
synchronized (this) {
ended = session;
session = null;
sessionCredentials = null;
}
if (ended != null) service.signOut(ended.id());
login.endSession();
}
private interface Request<T> {
@@ -122,28 +113,11 @@ public final class MyTsFiles {
}
private <T> T withSession(Request<T> request) throws IOException, MyTsException {
MyTeamSpeak.Session s = session();
try {
return request.run(s);
} catch (MyTsException e) {
if (e.code() != ERROR_SESSION_EXPIRED) throw e;
synchronized (this) {
if (session == s) session = null;
}
return request.run(session());
}
}
private synchronized MyTeamSpeak.Session session() throws IOException, MyTsException {
MyTeamSpeak.Credentials credentials = login.credentials();
if (session != null && sessionCredentials == credentials) return session;
if (session != null) service.signOut(session.id());
session = null;
if (credentials == null) throw new MyTsException("Sign in to myTeamSpeak to share files", 0);
session = service.signIn(credentials);
sessionCredentials = credentials;
if (session.info().userId().isEmpty()) throw new MyTsException("The account has no myTeamSpeak ID", 0);
return session;
if (!login.isSignedIn()) throw new MyTsException("Sign in to myTeamSpeak to share files", 0);
return login.request((session, credentials) -> {
if (session.info().userId().isEmpty()) throw new MyTsException("The account has no myTeamSpeak ID", 0);
return request.run(session);
});
}
private static HttpURLConnection open(String url) throws IOException {
@@ -0,0 +1,67 @@
package eu.lixko.ts3j.client.myts;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.Map;
import java.util.Set;
/**
* The parts of a bookmark or identity that synchronise, by label, in the same terms whichever
* side they come from, so that two copies compare equal exactly when they agree.
*/
final class SyncFields {
/**
* @param compare what decides equality
* @param display what the user is shown; for a secret, only whether it is set
*/
record Value(String compare, String display) {
}
private final Map<String, Value> values = new LinkedHashMap<>();
SyncFields put(String label, String value) {
return put(label, value, value);
}
SyncFields put(String label, String compare, String display) {
values.put(label, new Value(compare == null ? "" : compare, display == null ? "" : display));
return this;
}
SyncFields secret(String label, String value) {
String v = value == null ? "" : value;
return put(label, v, v.isEmpty() ? "" : "••••••");
}
SyncFields flag(String label, boolean value) {
return put(label, value ? "Yes" : "No");
}
Map<String, Value> values() {
return values;
}
String compare(String label) {
Value v = values.get(label);
return v == null ? "" : v.compare();
}
String display(String label) {
Value v = values.get(label);
return v == null ? "" : v.display();
}
/** The labels whose values differ from {@code other}'s. */
Set<String> changedFrom(SyncFields other) {
Set<String> out = new LinkedHashSet<>();
for (String label : values.keySet()) {
if (other == null || !compare(label).equals(other.compare(label))) out.add(label);
}
return out;
}
boolean sameAs(SyncFields other) {
return changedFrom(other).isEmpty();
}
}
@@ -0,0 +1,366 @@
package eu.lixko.ts3j.client.myts;
import com.github.manevolent.ts3j.identity.LocalIdentity;
import eu.lixko.ts3j.client.config.Bookmark;
import eu.lixko.ts3j.client.config.Bookmarks;
import eu.lixko.ts3j.client.config.IdentityEntry;
import eu.lixko.ts3j.client.config.IdentityStore;
import eu.lixko.ts3j.client.config.Settings;
import eu.lixko.ts3j.client.proto.ProtobufMessage;
import eu.lixko.ts3j.client.teamspeak.SyncItem;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
/**
* One class of synchronised items and its local store: how both sides read as {@link SyncFields},
* how a local item is written into an {@code Item_Data}, and how a cloud item lands here.
*/
abstract sealed class SyncKind {
/** How identities are referred to across the two sides; bookmarks need it. */
static final class Links {
/** Cloud identity item UUID by local identity id. */
final Map<String, String> uuidByLocal = new HashMap<>();
/** Local identity id by cloud identity item UUID. */
final Map<String, String> localByUuid = new HashMap<>();
/** The cloud's identities by item UUID and by name, as bookmarks may use either. */
final Map<String, SyncItem.Identity> cloudIdentities = new HashMap<>();
/** Adds an identity the cloud holds; the first one added under a UUID or name stays. */
void addCloudIdentity(SyncItem.Identity identity) {
cloudIdentities.putIfAbsent(identity.uuid(), identity);
if (!identity.name().isEmpty()) cloudIdentities.putIfAbsent(identity.name(), identity);
}
}
final IdentityStore identities;
final Bookmarks bookmarks;
final Settings settings;
final Links links;
SyncKind(IdentityStore identities, Bookmarks bookmarks, Settings settings, Links links) {
this.identities = identities;
this.bookmarks = bookmarks;
this.settings = settings;
this.links = links;
}
abstract int itemClass();
/** The {@code Item_Data} field holding the content. */
abstract int contentField();
abstract List<String> localIds();
/** {@code null} when the item is gone. */
abstract SyncFields localFields(String localId);
/** {@code null} when the item is not of this kind or cannot be read. */
abstract SyncFields cloudFields(SyncItem item);
/** What makes a local and a cloud item the same one when neither knows of the other yet. */
abstract String matchKey(SyncFields fields);
abstract String title(SyncFields fields);
/** Writes the labelled fields of the local item into the content message. */
abstract void write(String localId, ProtobufMessage content, Set<String> labels) throws IOException;
/** Creates the cloud item here; returns its local id. */
abstract String add(SyncItem cloud) throws IOException;
/** Makes the local item what the cloud holds; returns its local id, which may change. */
abstract String update(String localId, SyncItem cloud) throws IOException;
abstract void remove(String localId);
static final class IdentityKind extends SyncKind {
static final String NAME = "Name";
static final String NICKNAME = "Nickname";
static final String PHONETIC = "Phonetic nickname";
static final String KEY = "Unique ID";
static final String DEFAULT = "Default identity";
IdentityKind(IdentityStore identities, Bookmarks bookmarks, Settings settings, Links links) {
super(identities, bookmarks, settings, links);
}
@Override
int itemClass() {
return 1;
}
@Override
int contentField() {
return 17;
}
@Override
List<String> localIds() {
return identities.all().stream().map(IdentityEntry::getId).toList();
}
@Override
SyncFields localFields(String localId) {
IdentityEntry e = identities.byId(localId);
if (e == null) return null;
return fields(e.getName(), e.getNickname(), e.getPhoneticNickname(), e.getIdentity(),
localId.equals(settings.defaultIdentityId.get()));
}
@Override
SyncFields cloudFields(SyncItem item) {
if (!(item instanceof SyncItem.Identity i)) return null;
try {
return fields(i.name(), i.nickname(), i.phoneticNickname(),
IdentityStore.parseIdentityString(i.identity()), i.isDefault());
} catch (IOException malformed) {
return null;
}
}
private static SyncFields fields(String name, String nickname, String phonetic, LocalIdentity key,
boolean isDefault) {
String uid = key.getUid().toBase64();
return new SyncFields().put(NAME, name).put(NICKNAME, nickname).put(PHONETIC, phonetic)
.put(KEY, uid + "#" + key.getKeyOffset(), uid + " (level " + key.getSecurityLevel() + ")")
.flag(DEFAULT, isDefault);
}
@Override
String matchKey(SyncFields fields) {
String key = fields.compare(KEY);
return key.substring(0, key.indexOf('#'));
}
@Override
String title(SyncFields fields) {
return fields.display(NAME);
}
@Override
void write(String localId, ProtobufMessage content, Set<String> labels) throws IOException {
IdentityEntry e = identities.byId(localId);
if (labels.contains(KEY)) content.string(1, IdentityStore.identityString(e.getIdentity()));
if (labels.contains(NAME)) content.string(2, e.getName());
if (labels.contains(NICKNAME)) content.string(3, e.getNickname());
if (labels.contains(PHONETIC)) content.string(4, e.getPhoneticNickname());
if (labels.contains(DEFAULT)) content.varint(5, localId.equals(settings.defaultIdentityId.get()) ? 1 : 0);
}
@Override
String add(SyncItem cloud) throws IOException {
SyncItem.Identity i = (SyncItem.Identity) cloud;
LocalIdentity key = IdentityStore.parseIdentityString(i.identity());
IdentityEntry existing = identities.byUniqueId(key.getUid().toBase64());
if (existing != null) return update(existing.getId(), cloud);
String name = i.name().isBlank() ? (i.nickname().isBlank() ? "TeamSpeak identity" : i.nickname()) : i.name();
IdentityEntry e = identities.add(name, key, i.nickname(), i.phoneticNickname());
if (i.isDefault()) settings.defaultIdentityId.set(e.getId());
return e.getId();
}
@Override
String update(String localId, SyncItem cloud) throws IOException {
SyncItem.Identity i = (SyncItem.Identity) cloud;
IdentityEntry e = identities.byId(localId);
if (e == null) return add(cloud);
LocalIdentity key = IdentityStore.parseIdentityString(i.identity());
if (!key.getUid().toBase64().equals(e.getUniqueId())) {
// Another key altogether: a new identity takes the old one's place.
List<Bookmark> using = new ArrayList<>();
for (Bookmark b : bookmarks.all()) {
if (localId.equals(b.identityId)) using.add(b);
}
boolean wasDefault = localId.equals(settings.defaultIdentityId.get());
IdentityEntry replacement = identities.add(i.name().isBlank() ? e.getName() : i.name(), key,
i.nickname(), i.phoneticNickname());
identities.remove(e, settings, null);
for (Bookmark b : using) b.identityId = replacement.getId();
if (wasDefault || i.isDefault()) settings.defaultIdentityId.set(replacement.getId());
return replacement.getId();
}
if (key.getKeyOffset() != e.getIdentity().getKeyOffset()) identities.setKeyOffset(e, key);
if (!i.name().isBlank() && !i.name().equals(e.getName())) identities.rename(e, i.name());
if (!i.nickname().equals(e.getNickname()) || !i.phoneticNickname().equals(e.getPhoneticNickname())) {
identities.setNicknames(e, i.nickname(), i.phoneticNickname());
}
if (i.isDefault()) settings.defaultIdentityId.set(localId);
return localId;
}
@Override
void remove(String localId) {
IdentityEntry e = identities.byId(localId);
if (e != null) identities.remove(e, settings, bookmarks);
}
}
static final class BookmarkKind extends SyncKind {
static final String NAME = "Name";
static final String ADDRESS = "Address";
static final String NICKNAME = "Nickname";
static final String PASSWORD = "Server password";
static final String IDENTITY = "Identity";
static final String CHANNEL = "Default channel";
static final String CHANNEL_PASSWORD = "Channel password";
static final String AUTOCONNECT = "Connect on startup";
static final String MYTS = "myTeamSpeak features";
private static final Pattern CHANNEL_ID = Pattern.compile("/(\\d+)");
BookmarkKind(IdentityStore identities, Bookmarks bookmarks, Settings settings, Links links) {
super(identities, bookmarks, settings, links);
}
@Override
int itemClass() {
return 0;
}
@Override
int contentField() {
return 16;
}
@Override
List<String> localIds() {
return bookmarks.all().stream().map(b -> b.id).toList();
}
@Override
SyncFields localFields(String localId) {
Bookmark b = bookmarks.find(localId);
if (b == null) return null;
IdentityEntry identity = b.identityId == null || b.identityId.isBlank() ? null : identities.byId(b.identityId);
return fields(b.label, b.address, b.port, b.nickname,
identity == null ? "" : identity.getUniqueId(), identity == null ? "Default" : identity.getName(),
b.password, b.channel, b.channelPassword, b.connectOnStartup, b.myTeamSpeakFeatures);
}
@Override
SyncFields cloudFields(SyncItem item) {
if (!(item instanceof SyncItem.Bookmark b)) return null;
// One the cloud does not hold connects with the default, as it does once brought here.
SyncItem.Identity identity = b.identityUuid().isEmpty() ? null : links.cloudIdentities.get(b.identityUuid());
String uid = "", name = "Default";
if (identity != null) {
name = identity.name();
try {
uid = IdentityStore.parseIdentityString(identity.identity()).getUid().toBase64();
} catch (IOException malformed) {
uid = "?" + identity.uuid();
}
}
return fields(b.name(), b.address(), b.port(), b.nickname(), uid, name, b.serverPassword(), channelOf(b),
b.defaultChannelPassword(), b.autoconnect(), b.myTeamSpeakFeatures());
}
private static SyncFields fields(String label, String address, int port, String nickname, String identityUid,
String identityName, String password, String channel, String channelPassword,
boolean autoconnect, boolean myts) {
String hostPort = Bookmark.hostPort(address == null ? "" : address.trim(), port);
return new SyncFields().put(NAME, label).put(ADDRESS, hostPort.toLowerCase(java.util.Locale.ROOT), hostPort)
.put(NICKNAME, nickname).put(IDENTITY, identityUid, identityName).secret(PASSWORD, password).put(CHANNEL, channel)
.secret(CHANNEL_PASSWORD, channelPassword).flag(AUTOCONNECT, autoconnect).flag(MYTS, myts);
}
private static String channelOf(SyncItem.Bookmark b) {
if (b.defaultChannel().isBlank() && b.defaultChannelId() != 0) return "/" + b.defaultChannelId();
return b.defaultChannel();
}
@Override
String matchKey(SyncFields fields) {
return fields.compare(NAME) + "\n" + fields.compare(ADDRESS);
}
@Override
String title(SyncFields fields) {
String name = fields.display(NAME);
return name.isBlank() ? fields.display(ADDRESS) : name;
}
@Override
void write(String localId, ProtobufMessage content, Set<String> labels) {
Bookmark b = bookmarks.find(localId);
if (labels.contains(NAME)) content.string(1, nonNull(b.label));
if (labels.contains(ADDRESS)) {
content.string(2, nonNull(b.address).trim());
content.varint(3, b.port);
}
if (labels.contains(NICKNAME)) content.string(4, nonNull(b.nickname));
if (labels.contains(IDENTITY)) {
String uuid = b.identityId == null ? null : links.uuidByLocal.get(b.identityId);
content.string(6, uuid == null ? "" : uuid);
}
if (labels.contains(PASSWORD)) content.string(7, nonNull(b.password));
if (labels.contains(CHANNEL)) {
String channel = nonNull(b.channel);
Matcher id = CHANNEL_ID.matcher(channel);
if (id.matches()) {
content.string(8, "");
content.varint(10, Long.parseLong(id.group(1)));
} else {
content.string(8, channel);
content.varint(10, 0);
}
}
if (labels.contains(CHANNEL_PASSWORD)) content.string(9, nonNull(b.channelPassword));
if (labels.contains(AUTOCONNECT)) content.varint(14, b.connectOnStartup ? 1 : 0);
if (labels.contains(MYTS)) content.varint(22, b.myTeamSpeakFeatures ? 1 : 0);
}
@Override
String add(SyncItem cloud) {
Bookmark b = new Bookmark();
if (bookmarks.find(cloud.uuid()) == null) b.id = cloud.uuid();
copy((SyncItem.Bookmark) cloud, b);
bookmarks.add(b);
return b.id;
}
@Override
String update(String localId, SyncItem cloud) {
Bookmark b = bookmarks.find(localId);
if (b == null) return add(cloud);
copy((SyncItem.Bookmark) cloud, b);
return localId;
}
/** Everything that synchronises; what only this client keeps, like whisper controls, stays. */
private void copy(SyncItem.Bookmark from, Bookmark to) {
to.label = from.name();
to.address = from.address();
to.port = from.port();
to.nickname = from.nickname();
to.password = from.serverPassword();
to.channel = channelOf(from);
to.channelPassword = from.defaultChannelPassword();
to.connectOnStartup = from.autoconnect();
to.myTeamSpeakFeatures = from.myTeamSpeakFeatures();
to.identityId = "";
if (from.identityUuid().isEmpty()) return;
SyncItem.Identity identity = links.cloudIdentities.get(from.identityUuid());
String local = identity == null ? null : links.localByUuid.get(identity.uuid());
if (local != null) to.identityId = local;
}
@Override
void remove(String localId) {
Bookmark b = bookmarks.find(localId);
if (b != null) bookmarks.all().remove(b);
}
private static String nonNull(String s) {
return s == null ? "" : s;
}
}
}
@@ -0,0 +1,87 @@
package eu.lixko.ts3j.client.myts;
import eu.lixko.ts3j.client.config.ProfileFiles;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Base64;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Properties;
import java.util.TreeMap;
/**
* What the account held after the last synchronisation, which tells a change made here
* from one made elsewhere: each class's cursor, and each synchronised item with the
* revision it had, its contents and the bookmark or identity it is here. Holds identity
* keys, so it lives in the private profile like they do.
*/
final class SyncState {
static final String FILE_NAME = "myteamspeak-sync.properties";
/**
* @param localId the {@link eu.lixko.ts3j.client.config.Bookmark#id} or
* {@link eu.lixko.ts3j.client.config.IdentityEntry#getId()} the item is here
*/
record Entry(int itemClass, String uuid, String version, String localId, byte[] data) {
}
String global = MyTeamSpeak.NO_VERSION;
final Map<Integer, String> cursors = new TreeMap<>();
final Map<String, Entry> items = new LinkedHashMap<>();
/** The state kept for {@code account}; empty when there is none or it is another account's. */
static SyncState load(File file, String account) {
SyncState state = new SyncState();
if (!file.isFile()) return state;
Properties p = new Properties();
try (InputStream in = new FileInputStream(file)) {
p.load(in);
} catch (IOException unreadable) {
return state;
}
if (!account.equalsIgnoreCase(p.getProperty("account", ""))) return state;
try {
state.global = p.getProperty("global", MyTeamSpeak.NO_VERSION);
for (String key : p.stringPropertyNames()) {
if (key.startsWith("cursor.")) {
state.cursors.put(Integer.parseInt(key.substring("cursor.".length())), p.getProperty(key));
}
}
int count = Integer.parseInt(p.getProperty("items", "0"));
for (int i = 0; i < count; i++) {
String prefix = "item." + i + ".";
Entry entry = new Entry(Integer.parseInt(p.getProperty(prefix + "class")),
p.getProperty(prefix + "uuid"), p.getProperty(prefix + "version"),
p.getProperty(prefix + "local", ""),
Base64.getDecoder().decode(p.getProperty(prefix + "data")));
state.items.put(entry.uuid(), entry);
}
} catch (RuntimeException corrupt) {
// Starts over: the next synchronisation matches items up again, as the first one does.
return new SyncState();
}
return state;
}
void save(File file, String account) throws IOException {
Properties p = new Properties();
p.setProperty("account", account);
p.setProperty("global", global);
cursors.forEach((itemClass, cursor) -> p.setProperty("cursor." + itemClass, cursor));
p.setProperty("items", Integer.toString(items.size()));
int i = 0;
for (Entry entry : items.values()) {
String prefix = "item." + i++ + ".";
p.setProperty(prefix + "class", Integer.toString(entry.itemClass()));
p.setProperty(prefix + "uuid", entry.uuid());
p.setProperty(prefix + "version", entry.version());
p.setProperty(prefix + "local", entry.localId());
p.setProperty(prefix + "data", Base64.getEncoder().encodeToString(entry.data()));
}
ProfileFiles.write(file, out -> p.store(out, "myTeamSpeak synchronisation state"));
}
}
@@ -0,0 +1,72 @@
package eu.lixko.ts3j.client.proto;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.function.Consumer;
/**
* A decoded message whose fields can be replaced one by one while every other field,
* known or not, is written back byte for byte. For editing messages whose schema is
* only partly modelled.
*/
public final class ProtobufMessage {
private record Field(int number, byte[] raw) {
}
private final List<Field> fields = new ArrayList<>();
public ProtobufMessage() {
}
public ProtobufMessage(byte[] encoded) throws IOException {
ProtobufReader r = new ProtobufReader(encoded);
while (r.next()) fields.add(new Field(r.fieldNumber(), r.readRawField()));
}
/** The first occurrence of the field, as a reader positioned on it, or {@code null}. */
public ProtobufReader get(int number) throws IOException {
for (Field f : fields) {
if (f.number != number) continue;
ProtobufReader r = new ProtobufReader(f.raw);
r.next();
return r;
}
return null;
}
public ProtobufMessage string(int number, String value) {
return set(number, w -> w.string(number, value));
}
public ProtobufMessage varint(int number, long value) {
return set(number, w -> w.varint(number, value));
}
public ProtobufMessage message(int number, ProtobufMessage value) {
return set(number, w -> w.bytes(number, value.toByteArray()));
}
/** Replaces every occurrence of the field with what {@code write} writes, in the first one's place. */
public ProtobufMessage set(int number, Consumer<ProtobufWriter> write) {
ProtobufWriter w = new ProtobufWriter();
write.accept(w);
Field replacement = new Field(number, w.toByteArray());
int at = -1;
for (int i = fields.size() - 1; i >= 0; i--) {
if (fields.get(i).number != number) continue;
fields.remove(i);
at = i;
}
if (at < 0) fields.add(replacement);
else fields.add(at, replacement);
return this;
}
public byte[] toByteArray() {
ProtobufWriter w = new ProtobufWriter();
for (Field f : fields) w.raw(f.raw);
return w.toByteArray();
}
}
@@ -21,6 +21,7 @@ public final class ProtobufReader {
private int pos;
private int fieldNumber;
private int wireType;
private int fieldStart;
public ProtobufReader(byte[] data) {
this(data, 0, data.length);
@@ -35,6 +36,7 @@ public final class ProtobufReader {
/** Advances to the next field; {@code false} at the end of the message. */
public boolean next() throws IOException {
if (pos >= end) return false;
fieldStart = pos;
long tag = readVarint();
fieldNumber = (int) (tag >>> 3);
wireType = (int) (tag & 7);
@@ -98,6 +100,14 @@ public final class ProtobufReader {
if (pos > end) throw new IOException("Truncated protobuf message");
}
/** The current field exactly as encoded, tag included, for writing back unchanged. */
public byte[] readRawField() throws IOException {
skip();
byte[] out = new byte[pos - fieldStart];
System.arraycopy(data, fieldStart, out, 0, out.length);
return out;
}
private int readLength() throws IOException {
if (wireType != LENGTH_DELIMITED) throw new IOException("Field " + fieldNumber + " is not length-delimited");
long length = readVarint();
@@ -29,6 +29,12 @@ public final class ProtobufWriter {
return bytes(field, message.toByteArray());
}
/** Appends a field as {@link ProtobufReader#readRawField()} returned it. */
public ProtobufWriter raw(byte[] field) {
out.write(field, 0, field.length);
return this;
}
public byte[] toByteArray() {
return out.toByteArray();
}
@@ -86,7 +86,7 @@ class MyTeamSpeakIdentityTest {
}
@Test
void parsesLoginIdentityAndEndsSessionWithoutDownloadingItems() throws Exception {
void parsesLoginIdentityWithoutDownloadingItems() throws Exception {
List<String> calls = new ArrayList<>();
MyTeamSpeak service = new MyTeamSpeak((endpoint, method, request) -> {
calls.add(method);
@@ -97,10 +97,11 @@ class MyTeamSpeakIdentityTest {
.message(2, new ProtobufWriter().bytes(1, ENCRYPTED))
.varint(3, 1790376353L).bytes(4, ID).bytes(5, CERT)).toByteArray();
});
try (MyTeamSpeakIdentity identity = service.serverIdentity(MyTeamSpeakTest.credentials(MyTsCryptoTest.PASSWORD))) {
MyTeamSpeak.Credentials credentials = MyTeamSpeakTest.credentials(MyTsCryptoTest.PASSWORD);
try (MyTeamSpeakIdentity identity = service.serverIdentity(service.signIn(credentials), credentials)) {
assertEquals(6, identity.authenticate(new byte[32]).size());
}
assertEquals(List.of("login", "deleteSession"), calls);
assertEquals(List.of("login"), calls);
}
@Test
@@ -56,7 +56,9 @@ class MyTeamSpeakLoginTest {
MyTeamSpeakLogin login = new MyTeamSpeakLogin(file, new MyTeamSpeak(server));
login.signIn(MyTsCryptoTest.EMAIL, MyTsCryptoTest.PASSWORD);
// The next run signs in afresh, and the service turns the saved token away.
server.loginError = MyTsException.LOGIN_FAILED;
login = new MyTeamSpeakLogin(file, new MyTeamSpeak(server));
MyTsException e = assertThrows(MyTsException.class, login::fetch);
assertTrue(e.credentialsRejected());
assertFalse(login.isSignedIn());
@@ -0,0 +1,384 @@
package eu.lixko.ts3j.client.myts;
import eu.lixko.ts3j.client.config.AppDirs;
import eu.lixko.ts3j.client.config.Bookmark;
import eu.lixko.ts3j.client.config.Bookmarks;
import eu.lixko.ts3j.client.config.IdentityEntry;
import eu.lixko.ts3j.client.config.IdentityStore;
import eu.lixko.ts3j.client.config.Settings;
import eu.lixko.ts3j.client.proto.ProtobufReader;
import eu.lixko.ts3j.client.proto.ProtobufWriter;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import java.io.File;
import java.io.IOException;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.UUID;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
class MyTeamSpeakSyncTest {
/**
* The account as the live server behaves (probed 2026-10-15): a class whose cursor matches
* comes back empty, any other whole; it stores whatever revisions it is sent.
*/
static final class Cloud implements MyTsTransport {
record Stored(String version, byte[] blob) {
}
final Map<Integer, String> versions = new TreeMap<>();
final Map<Integer, Map<String, Stored>> items = new TreeMap<>();
int stores;
int logins;
@Override
public byte[] call(String endpoint, String method, byte[] request) throws IOException {
return switch (method) {
case "login" -> {
logins++;
yield new ProtobufWriter().bytes(1, MyTsCryptoTest.KEY_PACKAGE).string(2, "session")
.varint(5, 200).string(8, "tester").toByteArray();
}
case "requestServerItems" -> pull(request);
case "synchronizeItems" -> store(request);
default -> new byte[0];
};
}
private byte[] pull(byte[] request) throws IOException {
Map<Integer, String> asked = new TreeMap<>();
ProtobufReader r = new ProtobufReader(request);
while (r.next()) {
if (r.fieldNumber() != 2) {
r.skip();
continue;
}
ProtobufReader c = r.readMessage();
int itemClass = 0;
String version = "";
while (c.next()) {
if (c.fieldNumber() == 1) itemClass = (int) c.readVarint();
else if (c.fieldNumber() == 2) version = c.readString();
else c.skip();
}
asked.put(itemClass, version);
}
ProtobufWriter reply = new ProtobufWriter().varint(1, 301);
for (Map.Entry<Integer, String> e : asked.entrySet()) {
if (!versions.containsKey(e.getKey())) {
reply.message(2, new ProtobufWriter().varint(1, e.getKey()).varint(4, 1));
}
}
for (Map.Entry<Integer, String> e : versions.entrySet()) {
if (e.getValue().equals(asked.get(e.getKey()))) continue;
ProtobufWriter c = new ProtobufWriter().varint(1, e.getKey()).string(2, e.getValue());
items.get(e.getKey()).forEach((uuid, stored) -> c.message(3, new ProtobufWriter()
.string(1, uuid).string(2, stored.version()).bytes(3, stored.blob())));
reply.message(2, c);
}
return reply.string(3, UUID.randomUUID().toString()).toByteArray();
}
private byte[] store(byte[] request) throws IOException {
stores++;
ProtobufReader r = new ProtobufReader(request);
while (r.next()) {
if (r.fieldNumber() != 2) {
r.skip();
continue;
}
ProtobufReader c = r.readMessage();
int itemClass = 0;
while (c.next()) {
switch (c.fieldNumber()) {
case 1 -> itemClass = (int) c.readVarint();
case 2 -> versions.put(itemClass, c.readString());
case 3 -> {
ProtobufReader d = c.readMessage();
String uuid = "", version = "";
byte[] blob = null;
boolean delete = false;
while (d.next()) {
switch (d.fieldNumber()) {
case 1 -> uuid = d.readString();
case 2 -> version = d.readString();
case 3 -> blob = d.readBytes();
case 4 -> delete = d.readBool();
default -> d.skip();
}
}
Map<String, Stored> stored = items.computeIfAbsent(itemClass, k -> new LinkedHashMap<>());
if (delete) stored.remove(uuid);
else stored.put(uuid, new Stored(version, blob));
}
default -> c.skip();
}
}
}
return new ProtobufWriter().varint(1, 300).string(3, UUID.randomUUID().toString()).toByteArray();
}
/** Puts an item there as another client would. */
void seed(int itemClass, String uuid, byte[] itemData) throws Exception {
items.computeIfAbsent(itemClass, k -> new LinkedHashMap<>())
.put(uuid, new Stored(UUID.randomUUID().toString(), MyTsCrypto.encryptItem(itemData, MyTsCryptoTest.ITEM_KEY)));
versions.put(itemClass, UUID.randomUUID().toString());
}
byte[] plain(int itemClass, String uuid) throws Exception {
return MyTsCrypto.decryptItem(items.get(itemClass).get(uuid).blob(), MyTsCryptoTest.ITEM_KEY);
}
}
/** One installation: its own profile, signed in to the shared cloud. */
final class Client {
final File profile;
Settings settings;
IdentityStore identities;
Bookmarks bookmarks;
MyTeamSpeakSync sync;
Client(File profile) throws Exception {
this.profile = profile;
in(() -> {
settings = new Settings();
identities = IdentityStore.load(settings);
bookmarks = new Bookmarks();
MyTeamSpeakLogin login = new MyTeamSpeakLogin(new File(profile, "myteamspeak.properties"),
new MyTeamSpeak(cloud));
login.signIn(MyTsCryptoTest.EMAIL, MyTsCryptoTest.PASSWORD);
sync = new MyTeamSpeakSync(login, identities, bookmarks, settings);
});
}
interface Work {
void run() throws Exception;
}
void in(Work work) throws Exception {
AppDirs.init(profile, profile.toPath().resolve("chats"));
work.run();
}
MyTeamSpeakSync.Plan plan() throws Exception {
MyTeamSpeakSync.Plan[] plan = new MyTeamSpeakSync.Plan[1];
in(() -> plan[0] = sync.plan(sync.fetch()));
return plan[0];
}
MyTeamSpeakSync.Result finish(MyTeamSpeakSync.Plan plan) throws Exception {
MyTeamSpeakSync.Result[] result = new MyTeamSpeakSync.Result[1];
in(() -> {
sync.upload(plan);
result[0] = sync.apply(plan);
});
return result[0];
}
MyTeamSpeakSync.Result sync() throws Exception {
MyTeamSpeakSync.Plan plan = plan();
assertEquals(List.of(), plan.conflicts());
return finish(plan);
}
Bookmark bookmark(String label) {
return bookmarks.all().stream().filter(b -> label.equals(b.label)).findFirst().orElse(null);
}
}
final Cloud cloud = new Cloud();
@AfterEach
void defaultProfile() {
AppDirs.init(null, null);
}
@Test
void firstRunMergesBothWaysAndKeepsWhatItDoesNotModel(@TempDir File dir) throws Exception {
String official = "43c5d058-4803-3cd8-b844-15fdcc92e730";
cloud.seed(0, official, MyTsCryptoTest.ITEM_DATA);
Client a = new Client(new File(dir, "a"));
a.in(() -> {
a.identities.generate("Mine", 1);
a.bookmarks.add(new Bookmark("Mine", "mine.example", 9987, "me", ""));
// Already there: paired up, not duplicated.
a.bookmarks.add(new Bookmark("EričQov TeamSpeak server", "server.lixko.eu", 9987, "niger", ""));
});
int logins = cloud.logins;
MyTeamSpeakSync.Result first = a.sync();
assertEquals(logins, cloud.logins, "a run works in the session signing in opened");
assertEquals(2, first.sent(), "the identity and the new bookmark");
assertEquals(0, first.received());
assertEquals(2, a.bookmarks.all().size());
assertEquals(2, cloud.items.get(0).size());
assertEquals(1, cloud.items.get(1).size());
assertEquals(new MyTeamSpeakSync.Result(0, 0), a.sync());
a.bookmark("EričQov TeamSpeak server").nickname = "renamed";
assertEquals(new MyTeamSpeakSync.Result(1, 0), a.sync());
byte[] edited = cloud.plain(0, official);
Map<Integer, String> bookmark = strings(content(edited, 16));
assertEquals("renamed", bookmark.get(4));
assertEquals("Default", bookmark.get(6), "the identity reference is left as the official client wrote it");
assertEquals("Default", bookmark.get(11), "the capture profile, which this client does not model");
assertEquals("+Tyg2JtxE8vRNZp+JiUBnmBh0MY=", bookmark.get(15), "the server UID");
assertEquals(official, strings(edited).get(2));
assertEquals("d76cf761-1ac3-744f-6b41-66638cbd1e03", strings(edited).get(5),
"the revision written over is the last known one, which the official client checks");
}
@Test
void changesAndDeletionsTravelBetweenInstallations(@TempDir File dir) throws Exception {
Client a = new Client(new File(dir, "a"));
Client b = new Client(new File(dir, "b"));
String[] identityId = new String[1];
a.in(() -> {
IdentityEntry identity = a.identities.generate("Work", 1);
identityId[0] = identity.getId();
Bookmark x = new Bookmark("X", "x.example", 9988, "", "secret");
x.identityId = identity.getId();
x.channel = "/42";
a.bookmarks.add(x);
});
a.sync();
assertEquals(new MyTeamSpeakSync.Result(0, 2), b.sync());
Bookmark x = b.bookmark("X");
assertNotNull(x);
assertEquals("secret", x.password);
assertEquals("/42", x.channel);
IdentityEntry work = b.identities.byId(x.identityId);
assertEquals("Work", work.getName());
assertEquals(a.identities.byId(identityId[0]).getUniqueId(), work.getUniqueId());
x.nickname = "from b";
b.in(() -> b.identities.setNicknames(work, "worker", ""));
assertEquals(new MyTeamSpeakSync.Result(2, 0), b.sync());
assertEquals(new MyTeamSpeakSync.Result(0, 2), a.sync());
assertEquals("from b", a.bookmark("X").nickname);
assertEquals("worker", a.identities.byId(identityId[0]).getNickname());
a.bookmarks.all().remove(a.bookmark("X"));
assertEquals(new MyTeamSpeakSync.Result(1, 0), a.sync());
assertEquals(new MyTeamSpeakSync.Result(0, 1), b.sync());
assertNull(b.bookmark("X"));
b.in(() -> b.identities.remove(work, b.settings, b.bookmarks));
b.sync();
a.sync();
assertNull(a.identities.byId(identityId[0]));
assertEquals(0, cloud.items.get(1).size());
}
@Test
void anItemChangedOnBothSidesIsTheUsersToSettle(@TempDir File dir) throws Exception {
Client a = new Client(new File(dir, "a"));
Client b = new Client(new File(dir, "b"));
a.bookmarks.add(new Bookmark("X", "x.example", 9987, "", ""));
a.sync();
b.sync();
a.bookmark("X").nickname = "a";
b.bookmark("X").nickname = "b";
a.sync();
MyTeamSpeakSync.Plan plan = b.plan();
assertEquals(1, plan.conflicts().size());
MyTeamSpeakSync.Conflict conflict = plan.conflicts().get(0);
assertEquals("X", conflict.title());
assertFalse(conflict.isIdentity());
List<MyTeamSpeakSync.Conflict.Difference> differs =
conflict.differences().stream().filter(MyTeamSpeakSync.Conflict.Difference::differs).toList();
assertEquals(List.of(new MyTeamSpeakSync.Conflict.Difference("Nickname", "b", "a", true)), differs);
assertThrows(IllegalStateException.class, () -> b.finish(plan));
conflict.choose(MyTeamSpeakSync.Conflict.Choice.CLOUD);
assertEquals(new MyTeamSpeakSync.Result(0, 1), b.finish(plan));
assertEquals("a", b.bookmark("X").nickname);
a.bookmark("X").nickname = "c";
b.bookmark("X").nickname = "d";
a.sync();
MyTeamSpeakSync.Plan again = b.plan();
again.conflicts().get(0).choose(MyTeamSpeakSync.Conflict.Choice.HERE);
assertEquals(new MyTeamSpeakSync.Result(1, 0), b.finish(again));
a.sync();
assertEquals("d", a.bookmark("X").nickname);
// Edited here, deleted elsewhere.
a.bookmarks.all().remove(a.bookmark("X"));
a.sync();
b.bookmark("X").nickname = "e";
MyTeamSpeakSync.Conflict gone = b.plan().conflicts().get(0);
assertTrue(gone.deletedInCloud());
assertFalse(gone.deletedHere());
}
@Test
void doesNotOverwriteWhatChangedSinceItLooked(@TempDir File dir) throws Exception {
Client a = new Client(new File(dir, "a"));
Client b = new Client(new File(dir, "b"));
a.bookmarks.add(new Bookmark("X", "x.example", 9987, "", ""));
a.sync();
b.sync();
a.bookmark("X").nickname = "a";
MyTeamSpeakSync.Plan stale = a.plan();
b.bookmark("X").nickname = "b";
b.sync();
int stores = cloud.stores;
MyTsException e = assertThrows(MyTsException.class, () -> a.finish(stale));
assertTrue(e.changedMeanwhile());
assertEquals(stores, cloud.stores);
assertEquals("a", a.bookmark("X").nickname);
MyTeamSpeakSync.Plan fresh = a.plan();
assertEquals(1, fresh.conflicts().size());
}
@Test
void signingOutForgetsWhatWasSynchronised(@TempDir File dir) throws Exception {
Client a = new Client(new File(dir, "a"));
a.bookmarks.add(new Bookmark("X", "x.example", 9987, "", ""));
a.sync();
File state = new File(a.profile, SyncState.FILE_NAME);
assertTrue(state.isFile());
a.in(() -> new MyTeamSpeakLogin(new File(a.profile, "myteamspeak.properties"), new MyTeamSpeak(cloud)).signOut());
assertFalse(state.exists());
}
private static byte[] content(byte[] itemData, int field) throws IOException {
ProtobufReader r = new ProtobufReader(itemData);
while (r.next()) {
if (r.fieldNumber() == field) return r.readBytes();
r.skip();
}
throw new AssertionError("no field " + field);
}
/** The string fields of a message by number; other fields read as text too, harmlessly. */
private static Map<Integer, String> strings(byte[] message) throws IOException {
Map<Integer, String> out = new TreeMap<>();
ProtobufReader r = new ProtobufReader(message);
while (r.next()) {
try {
out.putIfAbsent(r.fieldNumber(), r.readString());
} catch (IOException notLengthDelimited) {
r.skip();
}
}
return out;
}
}
@@ -46,7 +46,7 @@ class MyTsFilesTest {
}
@Test
void asksUnderTheAccountsIdAndKeepsTheSession(@TempDir File dir) throws Exception {
void asksUnderTheAccountsIdInItsSession(@TempDir File dir) throws Exception {
FileServer server = new FileServer();
MyTsFiles files = signedIn(dir, server);
@@ -58,11 +58,11 @@ class MyTsFilesTest {
String session = "00000000-1111-2222-3333-444444444444";
assertEquals(List.of("requestUploadFile 1=" + session + " 2=" + upload.senderUserId() + " 3=room=",
"requestDownloadFile 1=" + session + " 2=owner 3=room= 4=file9"), server.requests);
assertEquals(List.of("authentication/login", "user/requestUploadFile", "user/requestDownloadFile"),
server.accounts.calls);
assertEquals(List.of("user/requestUploadFile", "user/requestDownloadFile"), server.accounts.calls,
"the session signing in opened");
files.close();
assertEquals("authentication/deleteSession", server.accounts.calls.get(3));
assertEquals("authentication/deleteSession", server.accounts.calls.get(2));
}
@Test
+37 -8
View File
@@ -13,11 +13,14 @@ that Android 13 lacks (hence `HttpURLConnection`, not `java.net.http`).
private profile file `myteamspeak.properties`; never the password. `signIn` derives them, reads the
account and only then saves them; `fetch` reads the account again with the saved ones; `signOut`
deletes the file. When the server rejects saved credentials (`ERROR_LOGIN_FAILED`, e.g. the password
was changed elsewhere) `fetch` signs out.
was changed elsewhere) `fetch` signs out. All requests share one session (`MyTeamSpeakLogin.request`),
as the official client's do: signed in on first use, again when the service reports it expired
(102/109), ended on sign-out or exit. Signing in for every request gets the account's logins throttled
within a few dozen (seen 2026-10-15: `login` timed out for minutes).
- `MyTeamSpeak.download(Credentials)` — signs in (`authentication/login`), unwraps the item key, pulls
BOOKMARK, IDENTITY and ITEM_FOLDER with one `synchronization/requestServerItems` against an empty
local state (all-zero global and class versions, no details), decrypts every item, then ends the
session (`authentication/deleteSession`). Each read signs in afresh; no server session is kept.
session (`authentication/deleteSession`); `MyTeamSpeakLogin` does the same in its shared session.
- `MyTsCrypto` — PBKDF2-HMAC-SHA512 login token and account key, `LoginSession.key` unwrap (AES-256-GCM),
item frame decryption (little-endian AES-CTR + SHA-512 trailer). See `CRYPTO_RE_SALT.md`.
- `MyTsTransport` — the `application/ts3cloud` framing over HTTP/1.1; an interface so tests replay a server.
@@ -35,13 +38,39 @@ of its item UUID; the importer resolves both.
Tests: `MyTsCryptoTest` (captured vectors), `MyTeamSpeakTest` (download against a replayed server built from
the captured login and item), `MyTeamSpeakLoginTest` (persistence, failed sign-in, rejected credentials).
## Two-way sync (not done)
## Two-way sync
Would additionally need the server's global and per-class version cursors and each item's
`item_uuid`/`sync_version_uuid` persisted, and a mapping between our bookmarks/identities and those items. The request/reply state machine,
mutation table, and item encryption (`IV || CTR || SHA-512`, random IV) are in `CRYPTO_RE_SALT.md` section 5. Open points: when edits rotate `sync_version_uuid`, the role of
`last_known_version`, the collision merge policy, and confirming the deletion tombstone before anything
destructive is sent.
`MyTeamSpeakSync` (with `SyncKind`, `SyncFields`, `SyncState`): bookmarks and identities, run by hand
(Swing: Options → myTeamSpeak → Synchronize; conflicts in `SyncConflictsDialog`). A run is
`fetch` (network) → `plan` (stores' thread) → user settles `Plan.conflicts()` → `upload` (network) → `apply`
(stores' thread). Nothing changes on either side before `upload`, and here only after it succeeded.
The comparison is three-way per item, against `SyncState` (profile file `myteamspeak-sync.properties`: the
global cursor, each class's cursor, and per item its server revision, plaintext `Item_Data` and local id).
Changed on one side → carried over (deletions too); on both → a `Conflict` with a field-by-field diff;
first run → items paired by identity UID or bookmark name+address, the rest carried over. Edits are
written over the item's current `Item_Data` field by field (`ProtobufMessage`), so fields this client does not
model survive.
### How the service and the official client behave (live, 2026-10-15)
Probed against the test account and by capturing the official 3.6.2 client's traffic.
- `requestServerItems`: a class whose cursor matches comes back empty; any other comes back whole. Item
details in the request are ignored; there are no deltas. Deleted items simply vanish.
- `synchronizeItems` stores what it is sent: the class revision (client-made, a fresh UUID), each item's
wire revision and blob; replies `300` and a new global cursor. It never reports `COLLISION`, whatever is
stale — conflicts are the client's to find. `push` therefore re-reads the cursors first and fails with
`changedMeanwhile` when any moved.
- The official client polls about once a minute with its cursors and the global cursor, and on startup with
random class cursors (a full read). It applies what it reads live; it writes `settings.db` only later, so
read its state from the UI, not the database, while it runs. Our pushes do not reach its push channel.
- Its change upload: plaintext `sync_version_uuid` new, `last_known_version` := the previous one,
`manipulated` = 3; a fresh wire revision; only the changed items; the global cursor echoed. Adds carry
`manipulated` = 0. Deletion: the detail's `item_delte_on_server` = 1 under a fresh revision (it also
sends the last blob; one without is accepted too). We write the same.
- It reports "myTeamSpeak account collision detected" and stops applying when a remote change meets an item
it still holds as locally changed. Seen once, after mixed probing; not with the writes above.
## Etiquette
One sign-in is three requests. Don't loop sign-ins while testing; the service sits behind Cloudflare and
+2 -6
View File
@@ -84,9 +84,5 @@ Recovered constructions:
## Remaining reverse-engineering gaps
Password login, key unwrap, item encryption/integrity framing, two-phase comparison, and the local
manipulation flags are recovered. The remaining uncertainties are narrower: where ordinary edits rotate
`sync_version_uuid`, the exact lifecycle of `last_known_version`, the official collision winner/merge
policy, and instruction-level confirmation of the deletion-detail builder. Recovery-key import is also
not yet recovered. The read-only pull is implemented in `eu.lixko.ts3j.client.myts` (see IMPLEMENTATION.md); destructive upload
should remain gated until the deletion mapping has a fixture or controlled integration test.
The synchronisation protocol is settled by live probing and capture; see IMPLEMENTATION.md "How the
service and the official client behave". Recovery-key import is not recovered.