ericek111/Nonemm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Do not trust the engine's count as the pace

Browse Source 
The second probe run read TxBufLen as 0 one millisecond and again fifty
milliseconds after twenty-one characters had been pushed, while those characters
were already going out: the first of them was decoded back off the air 420 ms
later. The number is right when it is a second old and wrong when it is fresh,
so it runs behind the engine.

A pump that fed on it would have handed over the whole message in half a second
and put all of it beyond reach, which is worse than the clock it replaced. So
the clock is the pace again, and the count is a check on it: the engine is never
given more than Lead + Slack characters however fast the clock says to feed, and
a message ends when the count and the clock estimate both say it has. Characters
fed since the last answer are added to it, so a stale answer cannot be spent
twice.

The engine is set to Character out: ABCD with no space after it went out at
once. Word out would have held it, so the holding case stays, but it now needs
the count to be neither going down nor being added to, since an engine kept
exactly at the cap has a count that does not move either.

The probe writes down every reading for two seconds after a push and does not
believe an empty one in that window, which is what the run needed to measure the
lag and did not do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtspmWmS7f8kUvcyaHpRWZ
This commit is contained in:
ericek111
2026-09-01 23:13:28 +00:00
parent 4118cd6d33
commit 5370ad3f6d
4 changed files with 142 additions and 192 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
docs/unfinished.md
View File

@@ -50,32 +50,30 @@ so it never runs dry and never transmits idle in the middle of a message. Those
two characters cannot be taken back; everything behind them can still be two characters cannot be taken back; everything behind them can still be
rewritten. rewritten.
How far the engine has got comes from the engine. MMTTY answers `TxBufLen` with The pace comes from the clock at the baud rate in the digital settings. The
the number of characters it still has to transmit, so the pump asks, feeds while engine's own count of what it has left is a check on that rather than the pace
the answer is under `Lead`, and asks again, 50 ms at a time. An engine that will itself, for the reason in the table below: it is used as a cap, so the engine is
not answer is paced by the clock at the baud rate in the digital settings never given more than `Lead + Slack` characters however fast the clock says to
instead, which drifts; that is what the baud setting is still for. feed, and as the end of a message, together with the clock estimate. A count
that neither goes down nor is being added to means the engine is holding what it
has, and the pump feeds a character every `HoldingPatience` until it moves.
What the engine probe established on 2026-09-01, with MMTTY 1.70 under Wine on a What the engine probe established, with MMTTY 1.70 under Wine on a machine with
machine with a sound card: a sound card:
| What | Answer | | What | Answer |
|---|---| |---|---|
| Does the control answer `TxBufLen`? | Yes, and a name it does not know fails with `DISP_E_UNKNOWNNAME`, so the two are distinguishable. | | Does the control answer `TxBufLen`? | Yes, and a name it does not know fails with `DISP_E_UNKNOWNNAME`, so the two are distinguishable. |
| What does it count? | Characters left to transmit. Seventeen were pushed; a second later, with five decoded back off the air, it read 12, and it counted down to 0 as the message went out. | | What does it count? | Characters left to transmit. Seventeen were pushed; a second later, with five decoded back off the air, it read 12, and it counted down to 0 as the message went out. |
| How current is that number? | Not current. Read a millisecond and again fifty milliseconds after twenty-one characters had been pushed — and were already going out, the first of them decoded back a moment later — it still said 0. So a pump that fed on the count alone would hand over a whole message in half a second and put all of it beyond reach. How far behind it runs has not been measured; the probe now writes down every reading for the two seconds after a push, which is where the number will come from. |
| Does a backspace take back a character the engine has not transmitted? | No. Pushed in as a character it made the count go **up** by four, and the text went out unchanged. MMTTY's help describes its backspace key doing this in its own window; through `PostMmttyMessage(4, 8)` it is just another character. So text in the engine cannot be retracted, only aborted. | | Does a backspace take back a character the engine has not transmitted? | No. Pushed in as a character it made the count go **up** by four, and the text went out unchanged. MMTTY's help describes its backspace key doing this in its own window; through `PostMmttyMessage(4, 8)` it is just another character. So text in the engine cannot be retracted, only aborted. |
| How long after a push does the count include it? | Not immediately. A count read a millisecond after the push still read 0, so one empty answer is not the end of a message. | | Way to send | This engine is set to Character out: `ABCD` with no space after it went out at once, a character every 200 ms. MMTTY's help says Word out is the usual setting, which would hold that word, so the pump copes with both. |
Two things the probe has still to answer, and it now asks both: The report is worth reading for the receive side as well. With MMTTY's sound
loopback off, its help says the receive window is fed from the transmit window,
- **Way to send.** MMTTY set to Word out holds a word until the space after it, which would be a per-character report of what has gone out with no polling at
which the first run appeared to show but could not confirm, because the engine all. With loopback on, which is how this station runs it, the decoded text
was not keyed for that step. The pump handles it either way: a count that arrives about 420 ms after the character goes out and matches what was sent.
stops going down means the engine is holding what it has, so the pump feeds
one character per look until it moves again.
- **Whether `{RX}` sends a held word.** `SetMmttyPTT(1)` stops once the buffer is
empty. If a held word does not count as buffer, a macro whose last word has no
space after it loses that word.
```sh ```sh
./build.sh run --project tools/Nonemm.EngineProbe -- \ ./build.sh run --project tools/Nonemm.EngineProbe -- \
@@ -85,13 +83,6 @@ Two things the probe has still to answer, and it now asks both:
It transmits on the sound card for about twenty seconds and keys no serial port It transmits on the sound card for about twenty seconds and keys no serial port
unless `--ptt` names one. `EngineProbe` states what each answer means. unless `--ptt` names one. `EngineProbe` states what each answer means.
The receive side is worth watching in the report as well. With MMTTY's sound
loopback off, its help says the receive window is fed from the transmit window,
which would be a per-character report of what has gone out with no polling at
all. With loopback on it is the demodulator hearing the transmission, which
arrives about half a second late and can carry decode errors, so the count is
what the pump uses.
**Voice keying.** `MessageSender` was written to cover a voice keyer playing a **Voice keying.** `MessageSender` was written to cover a voice keyer playing a
recording, and nothing implements it. Each operator's recordings folder is recording, and nothing implements it. Each operator's recordings folder is
stored with the rest of his settings and waits on the same thing. No DVK support either, so the second radio stored with the rest of his settings and waits on the same thing. No DVK support either, so the second radio

195
src/Nonemm.Digital/TypeAhead.cs
View File

@@ -19,16 +19,23 @@ namespace Nonemm.Digital;
/// which is where the operator is typing: the pump stops when it reaches the /// which is where the operator is typing: the pump stops when it reaches the
/// cursor, because the operator has not finished the word yet. /// cursor, because the operator has not finished the word yet.
/// ///
/// How far the engine has got comes from the engine when it can say. MMTTY /// The pump is paced by the clock at the baud rate, and the engine's own count
/// answers `TxBufLen` with the number of characters it still has to transmit, /// of what it has left is a check on it rather than the pace. MMTTY answers
/// so the pump asks, feeds while the answer is under `Lead`, and asks again. An /// `TxBufLen` with the number of characters it still has to transmit, but the
/// engine that will not answer is paced by the clock at the baud rate instead, /// answer is behind what the engine is really doing: the probe read 0 fifty
/// which drifts and is what the baud setting is for. /// milliseconds after twenty-one characters had been pushed and were already
/// going out, and read a right-looking 12 a second after a push. A pump that
/// believed a stale 0 would hand over the whole message in half a second and
/// put all of it beyond reach, so the count is used two ways only:
///
/// - as a cap. The engine holding more than `Lead + Slack` characters stops the
/// pump until it comes down, whatever the clock thinks.
/// - as the end of a message, together with the clock estimate.
/// ///
/// A count that stops going down means the engine is holding what it has: /// A count that stops going down means the engine is holding what it has:
/// MMTTY set to Word out keeps a word until the space after it arrives. The /// MMTTY set to Word out keeps a word until the space after it arrives. The
/// pump feeds one character per look while that lasts, so the space gets there /// pump feeds a character anyway while that lasts, so the space gets there and
/// and the word goes out. /// the word goes out.
public sealed class TypeAhead : IDisposable public sealed class TypeAhead : IDisposable
{ {
/// A RTTY character is a start bit, five data bits and a stop bit and a /// A RTTY character is a start bit, five data bits and a stop bit and a
@@ -52,6 +59,11 @@ public sealed class TypeAhead : IDisposable
/// stands while the operator is not typing into the pane. /// stands while the operator is not typing into the pane.
public const int NoCursor = int.MaxValue; public const int NoCursor = int.MaxValue;
/// How many characters the engine's count may be behind what it is really
/// doing. The count is what stops the pump when the engine falls behind the
/// clock, so it has to allow for the lag rather than fight it.
public const int Slack = 3;
/// How often the engine is asked how much it has left. /// How often the engine is asked how much it has left.
public static readonly TimeSpan PollInterval = TimeSpan.FromMilliseconds(50); public static readonly TimeSpan PollInterval = TimeSpan.FromMilliseconds(50);
@@ -60,9 +72,6 @@ public sealed class TypeAhead : IDisposable
/// character going out, so it is short against a character time. /// character going out, so it is short against a character time.
private static readonly TimeSpan LongestTick = TimeSpan.FromMilliseconds(10); private static readonly TimeSpan LongestTick = TimeSpan.FromMilliseconds(10);
/// An engine that has not answered by now is not going to.
private static readonly TimeSpan AnswerPatience = TimeSpan.FromSeconds(2);
private readonly Func<char, CancellationToken, Task> send; private readonly Func<char, CancellationToken, Task> send;
private readonly EngineBuffer? counter; private readonly EngineBuffer? counter;
private readonly Lock gate = new(); private readonly Lock gate = new();
@@ -71,9 +80,16 @@ public sealed class TypeAhead : IDisposable
private CancellationTokenSource? stopping; private CancellationTokenSource? stopping;
private Task pump = Task.CompletedTask; private Task pump = Task.CompletedTask;
private TaskCompletionSource<int>? asking;
private int cursor = NoCursor; private int cursor = NoCursor;
/// The engine's last answer: how many characters it still had to transmit,
/// or -1 before it has answered at all.
private int counted = -1;
/// How many answers have arrived, so the pump can tell a new one from the
/// one it has already added its own characters to.
private int answers;
public TypeAhead(Func<char, CancellationToken, Task> send, double baud = DefaultBaud) public TypeAhead(Func<char, CancellationToken, Task> send, double baud = DefaultBaud)
{ {
this.send = send; this.send = send;
@@ -81,7 +97,7 @@ public sealed class TypeAhead : IDisposable
} }
/// An engine that holds a buffer of its own and can say how much of it is /// An engine that holds a buffer of its own and can say how much of it is
/// left, which is what paces the pump instead of the clock. /// left, which is what the pump checks the clock against.
public TypeAhead(EngineBuffer engine, double baud = DefaultBaud) public TypeAhead(EngineBuffer engine, double baud = DefaultBaud)
: this((character, cancellation) => engine.TypeAsync(character, cancellation), baud) : this((character, cancellation) => engine.TypeAsync(character, cancellation), baud)
{ {
@@ -89,16 +105,15 @@ public sealed class TypeAhead : IDisposable
engine.Buffered += WhenBuffered; engine.Buffered += WhenBuffered;
} }
/// The speed the engine transmits at, which paces the pump when the engine /// The speed the engine transmits at, which is what paces the pump.
/// will not say how much it holds.
public double Baud { get; set; } public double Baud { get; set; }
/// How many characters may sit in the engine at once. /// How many characters may sit in the engine at once.
public int Lead { get; set; } = DefaultLead; public int Lead { get; set; } = DefaultLead;
/// False once the engine has refused to say how much it holds, which puts /// True once the engine has said how much it holds. An engine that will not
/// the pump back on the clock. /// say is paced by the clock alone.
public bool Counts { get; private set; } = true; public bool Counts { get; private set; }
public TimeSpan CharacterTime => public TimeSpan CharacterTime =>
TimeSpan.FromSeconds(BitsPerCharacter / (Baud > 0 ? Baud : DefaultBaud)); TimeSpan.FromSeconds(BitsPerCharacter / (Baud > 0 ? Baud : DefaultBaud));
@@ -259,81 +274,20 @@ public sealed class TypeAhead : IDisposable
} }
} }
/// Feeds the engine on the clock, holds off when its own count says it is
/// further behind than that, and stops when both agree there is nothing
/// left.
private async Task RunAsync(CancellationToken cancellation) private async Task RunAsync(CancellationToken cancellation)
{ {
int inEngine = 0;
int last = int.MaxValue;
int fedSince = 0;
int seen = -1;
DateTime nextOut = DateTime.MinValue;
DateTime lastAsked = DateTime.MinValue;
DateTime moved = DateTime.UtcNow;
try try
{ {
if (counter is not null && Counts && await CountedAsync(cancellation).ConfigureAwait(false))
{
return;
}
await PacedAsync(cancellation).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
}
}
/// Asks the engine how much it has left, feeds it up to `Lead`, and asks
/// again. The pump is the only thing that writes to the engine, so an
/// answer is about characters it has already been given.
///
/// Returns false if the engine will not say, which puts the pump on the
/// clock instead.
private async Task<bool> CountedAsync(CancellationToken cancellation)
{
DateTime moved = DateTime.UtcNow;
int last = int.MaxValue;
bool wasEmpty = false;
while (!cancellation.IsCancellationRequested)
{
int left = await AskAsync(cancellation).ConfigureAwait(false);
if (left < 0)
{
Counts = false;
return false;
}
if (left < last)
{
moved = DateTime.UtcNow;
}
bool holding = DateTime.UtcNow - moved > HoldingPatience;
while ((left < Lead || holding) && Take() is { } next)
{
await send(next, cancellation).ConfigureAwait(false);
Changed?.Invoke(this, EventArgs.Empty);
left++;
holding = false;
moved = DateTime.UtcNow;
}
last = left;
// the engine takes a moment to count what it has just been given,
// so one empty answer is not the end of the message
if (left == 0 && !IsSending)
{
if (wasEmpty)
{
Drained?.Invoke(this, EventArgs.Empty);
return true;
}
wasEmpty = true;
}
else
{
wasEmpty = false;
}
await Task.Delay(PollInterval, cancellation).ConfigureAwait(false);
}
return true;
}
/// The fallback: count character times off the clock at the baud rate, so
/// the engine is fed at the speed it transmits. It drifts, which shows up
/// as a gap between characters near the end of a long message.
private async Task PacedAsync(CancellationToken cancellation)
{
int inEngine = 0;
DateTime nextOut = DateTime.MinValue;
while (!cancellation.IsCancellationRequested) while (!cancellation.IsCancellationRequested)
{ {
DateTime now = DateTime.UtcNow; DateTime now = DateTime.UtcNow;
@@ -342,18 +296,45 @@ public sealed class TypeAhead : IDisposable
inEngine--; inEngine--;
nextOut += CharacterTime; nextOut += CharacterTime;
} }
if (inEngine < Lead && Take() is { } next) if (counter is not null && now - lastAsked >= PollInterval)
{
lastAsked = now;
await counter.AskBufferedAsync("", cancellation).ConfigureAwait(false);
}
int answered = Volatile.Read(ref answers);
if (answered != seen)
{
seen = answered;
fedSince = 0;
}
int left = Volatile.Read(ref counted);
if (left < last)
{
moved = now;
}
last = left;
// the answer says nothing about the characters fed since it
int held = left < 0 ? -1 : left + fedSince;
bool holding = left > 0 && now - moved > HoldingPatience;
bool room = holding
|| (inEngine < Lead && (held < 0 || held < Lead + Slack));
if (room && Take() is { } next)
{ {
if (inEngine == 0) if (inEngine == 0)
{ {
nextOut = now + CharacterTime; nextOut = now + CharacterTime;
} }
inEngine++; inEngine++;
fedSince++;
// feeding explains a count that is not going down, so it
// counts as movement: an engine is only holding what it has
// if it neither transmits nor is given anything
moved = now;
await send(next, cancellation).ConfigureAwait(false); await send(next, cancellation).ConfigureAwait(false);
Changed?.Invoke(this, EventArgs.Empty); Changed?.Invoke(this, EventArgs.Empty);
continue; continue;
} }
if (inEngine == 0 && !IsSending) if (inEngine == 0 && left <= 0 && !IsSending)
{ {
Drained?.Invoke(this, EventArgs.Empty); Drained?.Invoke(this, EventArgs.Empty);
return; return;
@@ -361,6 +342,10 @@ public sealed class TypeAhead : IDisposable
await Task.Delay(Tick, cancellation).ConfigureAwait(false); await Task.Delay(Tick, cancellation).ConfigureAwait(false);
} }
} }
catch (OperationCanceledException)
{
}
}
private TimeSpan Tick private TimeSpan Tick
{ {
@@ -371,35 +356,11 @@ public sealed class TypeAhead : IDisposable
} }
} }
/// One question and its answer, or -1 when the engine would not say.
private async Task<int> AskAsync(CancellationToken cancellation)
{
if (counter is null)
{
return -1;
}
TaskCompletionSource<int> answer = new(TaskCreationOptions.RunContinuationsAsynchronously);
lock (gate)
{
asking = answer;
}
await counter.AskBufferedAsync("", cancellation).ConfigureAwait(false);
try
{
return await answer.Task.WaitAsync(AnswerPatience, cancellation).ConfigureAwait(false);
}
catch (TimeoutException)
{
return -1;
}
}
private void WhenBuffered(object? sender, int left) private void WhenBuffered(object? sender, int left)
{ {
lock (gate) Counts = left >= 0;
{ Volatile.Write(ref counted, left);
asking?.TrySetResult(left); Interlocked.Increment(ref answers);
}
} }
/// The next character to send, or null when there is none to send now: /// The next character to send, or null when there is none to send now:

42
tests/Nonemm.Digital.Tests/TypeAheadTests.cs
View File

@@ -212,39 +212,29 @@ public class TypeAheadTests
Assert.Equal(2, Sent.Length); Assert.Equal(2, Sent.Length);
} }
/// An engine that says how much it holds is asked rather than timed, so /// The engine's count of what it has left runs behind what it is really
/// nothing depends on the baud rate being right. /// doing: the probe read 0 fifty milliseconds after twenty-one characters
/// had been pushed and were already going out. A pump that believed that
/// would hand over the whole message at once, so the count only ever stops
/// it, and the engine is never given more than `Lead` plus `Slack`.
[Fact] [Fact]
public async Task ACountingEngineIsAskedHowMuchItHolds() public async Task AnEngineBehindTheClockIsNotFedPastTheCap()
{ {
FakeEngine engine = new(); FakeEngine engine = new();
using TypeAhead buffer = new(engine, baud: Slow); using TypeAhead buffer = new(engine, baud: Slow);
// half the speed of the baud rate the pump is paced by
using Timer transmitting = new(_ => engine.Transmit(1), null, 0, 200);
buffer.Append("CQ TEST"); buffer.Append("CQ TEST DE OM5M");
await WaitForAsync(() => engine.Waiting.Length >= 2); int most = 0;
await Task.Delay(30); for (int look = 0; look < 50; look++)
Assert.Equal("CQ", engine.Waiting); {
most = Math.Max(most, engine.Waiting.Length);
engine.Transmit(2); await Task.Delay(20);
await WaitForAsync(() => engine.Transmitted.Length + engine.Waiting.Length >= 4);
Assert.Equal("CQ T", engine.Transmitted + engine.Waiting);
} }
Assert.True(most <= TypeAhead.DefaultLead + TypeAhead.Slack, $"the engine was given {most}");
/// MMTTY set to Word out holds a word until the space after it, so the Assert.True(engine.Transmitted.Length > 0, "nothing was transmitted at all");
/// count stops going down. Waiting for it to move would leave the message
/// sitting there, so the pump feeds on.
[Fact]
public async Task AnEngineHoldingWhatItHasIsFedAnyway()
{
FakeEngine engine = new();
using TypeAhead buffer = new(engine, baud: Slow);
buffer.Append("CQ TEST");
await WaitForAsync(() => engine.Waiting == "CQ TEST");
Assert.Equal("CQ TEST", engine.Waiting);
} }
/// The clock is the fallback for an engine that will not answer. /// The clock is the fallback for an engine that will not answer.

18
tools/Nonemm.EngineProbe/EngineProbe.cs
View File

@@ -41,6 +41,11 @@ public sealed class EngineProbe
/// A count that has not moved for this long is not going to move. /// A count that has not moved for this long is not going to move.
private static readonly TimeSpan Still = TimeSpan.FromSeconds(5); private static readonly TimeSpan Still = TimeSpan.FromSeconds(5);
/// How long after a push the count is written down every time it is read,
/// and an empty answer is not believed. The count runs behind the engine,
/// and this is where the report says by how much.
private static readonly TimeSpan Settling = TimeSpan.FromSeconds(2);
/// How long the engine is given to key up and to drop again. /// How long the engine is given to key up and to drop again.
private static readonly TimeSpan Keying = TimeSpan.FromSeconds(3); private static readonly TimeSpan Keying = TimeSpan.FromSeconds(3);
@@ -126,6 +131,7 @@ public sealed class EngineProbe
await KeyAsync(cancellation).ConfigureAwait(false); await KeyAsync(cancellation).ConfigureAwait(false);
await TypeAsync(Message, cancellation).ConfigureAwait(false); await TypeAsync(Message, cancellation).ConfigureAwait(false);
log.Write($"pushed {Message.Length} characters"); log.Write($"pushed {Message.Length} characters");
log.Write("every reading for the next two seconds: the first one that is not 0 says how far behind the count is");
await DrainAsync(cancellation).ConfigureAwait(false); await DrainAsync(cancellation).ConfigureAwait(false);
log.Write($"received while transmitting: \"{TakeReceived()}\""); log.Write($"received while transmitting: \"{TakeReceived()}\"");
await UnkeyAsync(cancellation).ConfigureAwait(false); await UnkeyAsync(cancellation).ConfigureAwait(false);
@@ -173,20 +179,22 @@ public sealed class EngineProbe
/// registered, not that the message has gone. /// registered, not that the message has gone.
private async Task DrainAsync(CancellationToken cancellation) private async Task DrainAsync(CancellationToken cancellation)
{ {
DateTime giveUp = DateTime.UtcNow + Patience; DateTime started = DateTime.UtcNow;
DateTime moved = DateTime.UtcNow; DateTime giveUp = started + Patience;
DateTime moved = started;
int last = int.MinValue; int last = int.MinValue;
bool wasEmpty = false; bool wasEmpty = false;
while (DateTime.UtcNow < giveUp) while (DateTime.UtcNow < giveUp)
{ {
int left = await AskAsync("", cancellation).ConfigureAwait(false); int left = await AskAsync("", cancellation).ConfigureAwait(false);
if (left != last) bool settling = DateTime.UtcNow - started < Settling;
if (left != last || settling)
{ {
log.Write($"TxBufLen: {Answer(left)}"); log.Write($"TxBufLen: {Answer(left)}");
moved = left != last ? DateTime.UtcNow : moved;
last = left; last = left;
moved = DateTime.UtcNow;
} }
if (left == 0 && wasEmpty) if (left == 0 && wasEmpty && !settling)
{ {
return; return;
} }