Nonemm/README.md

# Nonemm

A contest logger for amateur radio: a from-scratch reimplementation of
[N1MM Logger+](https://n1mmwp.hamdocs.com/) in C#, running on Linux and Windows.

The contest rules, log formats and database schema are written from their
published definitions. What Nonemm keeps is **interoperability**: the log
database is N1MM's `.s3db` in N1MM's schema, user-defined contests are N1MM's
`.udc` files, and a log either program writes opens in the other.

## Running it

The SDK lives at `~/.dotnet` on this machine, so a shell that has not been set
up needs:

```sh
export DOTNET_ROOT=$HOME/.dotnet PATH="$HOME/.dotnet:$PATH"
```

Then:

```sh
dotnet run --project src/Nonemm.App     # start the logger
dotnet test Nonemm.slnx                 # run every test
```

`./build.sh` does the same with the environment already set: `./build.sh test
Nonemm.slnx`.

To work a contest: **Config → Station** for your callsign and zones, then
**File → New Database** and **File → New Contest**. In the entry window, type a
callsign and press **space** to move to the exchange, then **Enter** to log.
Type a frequency into the callsign box and press Enter to change band. **View**
opens the log, check, bandmap, score and telnet windows.

Everything that judges a station — the bar under the callsign box, a row in the
check window, a spot on the bandmap, a line in the log — is coloured by the same
scorer: red for a dupe, green for a new multiplier, blue for points.

## What it does

| | |
|---|---|
| Contests | CQ WW (CW, SSB, RTTY), CQ WPX (CW, SSB, RTTY), WAE (CW, SSB, RTTY), ARRL DX, IARU HF, Sweepstakes, RTTY Roundup, NAQP, general logging, and user-defined `.udc` contests |
| Log | N1MM `.s3db`, Cabrillo 3.0 out, ADIF in and out |
| While typing | dupe check, multiplier check, points, country and zone from the country file, exchange filled from a call history file |
| Windows | entry, log, check, bandmap, available mults and Qs, score summary, telnet |
| Editing | double-click a cell in the log, or open the whole contact with Ctrl+Y; Delete removes it. All of it goes out to the other stations |
| Radio | one or two radios over hamlib `rigctld`, split, reconnecting on its own |
| Cluster | DX cluster over telnet: a telnet window with the node's traffic, the published list of nodes, command buttons and spot filters, spots feeding the bandmap, Alt+P to spot a station |
| Bandmap | drawn like N1MM's: a frequency scale with the receiver on it and callsigns beside it, joined by leader lines |
| Network | contacts shared with the other stations of a multi-operator entry, in N1MM's own contact message |
| Keying | CW through `cwdaemon` or a WinKeyer, with N1MM's message macros |

### The country file and the callsign database

Neither is bundled. **Config → Download Country File** and **Download Check
Partial File** fetch them from where they are published, which is where N1MM
fetches them from too —
[`country-files.com/cty/wl_cty.dat`](https://www.country-files.com/cty/wl_cty.dat)
and
[`supercheckpartial.com/MASTER.SCP`](https://www.supercheckpartial.com/MASTER.SCP).
The country file is `wl_cty.dat` rather than plain `cty.dat`: same format, with
the WAE entities listed separately, which is what CQ WW counts.

A download that fails changes nothing. The file is fetched, checked that it
parses as what it claims to be, and only then put in place, so a site that
answers with an apology page instead of a country file cannot cost an operator
their multipliers mid-contest.

Both files can also be dropped into `SupportFiles` under the configuration
directory (`~/.config/nonemm` on Linux, `Documents\Nonemm` on Windows).
User-defined contests go in `UserDefinedContests` under the same directory.

Without a country file the program still runs; country- and continent-scored
contests lose accuracy.

### Radio control

The logger reads and tunes the radio through
[hamlib](https://hamlib.github.io/)'s `rigctld`, started separately for whichever
radio is on the desk:

```sh
rigctld -m 2028 -r /dev/ttyUSB0     # -m is the hamlib model number; rigctl -l lists them
```

Then **Config → Radio**. With no radio connected nothing changes: frequency and
mode stay where they were last typed.

### CW

**Config → Keyer and messages** picks `cwdaemon` (a UDP port, usually 6789) or a
WinKeyer (a serial port) and sets the speed. Escape stops sending.

The messages are edited as their file, in a plain text editor: right-click any
of the function key buttons under the entry window, or use the two buttons in
Config → Keyer and messages. One line per key — the button label, a comma, then
the message — and a line starting with `#` is a comment. Which key a line
belongs to is decided by where it is in the file: the first twelve lines are F1
to F12 while running, and twelve more give the same keys while searching, with
the ones left out keeping their running message. The buttons say what the file
says, and they change as you move between running and searching.

This is N1MM's `.mc` format. **Import…** reads a file somebody published for
N1MM, **Export…** writes one out, and **Back to the defaults** puts the built-in
messages back.

The macros are N1MM's, spelled the way N1MM's function-key documentation spells
them, so a `.mc` file written for either program says the same thing. The text
macros that are filled in:

| | |
|---|---|
| `*`, `{MYCALL}` | your callsign |
| `!`, `{CALL}` | the call being worked, or the last one logged when the box is empty |
| `#` | the serial number for this contact, or the last one when the box is empty |
| `{LASTCALL}`, `{PREVNR}` | the call and the serial of the contact just logged |
| `{EXCH}` | the sent exchange from the contest setup |
| `{SENTRST}`, `{SENTRSTCUT}` | the report you send, plain or in cut numbers |
| `{NAME}`, `{NAMEANDSPACE}`, `{CHNAME}` | the other operator's name, from the entry window or the call history file |
| `{GRID}` / `{GRIDSQUARE}` | your grid / theirs |
| `{GRIDBEARING}`, `{REVGRIDBEARING}`, `{KMGRIDDISTANCE}` | bearing and distance between the two |
| `{FREQ}`, `{FREQROUND}` | this radio's frequency in kilohertz, with `R` for the decimal point on CW |
| `{OTHERFREQ}`, `{OTHERFREQROUND}`, `{OTHERMHZ}`, `{OTHERBAND}`, `{LRMHZ}`, `{RRMHZ}` | the other radio, for passing a station |
| `{TIMESTAMP}`, `{TIME2}` | the time now |

The action macros run rather than standing for text: `{WIPE}`, `{LOG}`,
`{RUN}`, `{S&P}`, `{SPACE}`, `{SPOTME}`, `{STOPTX}`, `{FREQUP}`, `{FREQDN}`,
`{PGUP}`, `{PGDN}`, `{JUMPRX}`, `{JUMPRXTX}`, `{CTRLF1}` to `{CTRLF12}` to send
on the other radio, `{TELNET sh/dx}` to put a command on the cluster, and
`{OTRSP TX2}` to send one to the SO2R box. An action runs before the message
goes out, unless it stands after `{END}`, and then it runs once the message has
been sent — so `TU {LOG}{END}{WIPE}` logs the contact, sends TU, and clears the
boxes when the key has finished. Text after `{END}` is dropped, as it is in
N1MM.

How far `{FREQUP}` and the rest move the radio is `FrequencyStepHertz` and
`PageStepHertz` in `settings.json`, 100 Hz and 1 kHz out of the box; N1MM asks
for the same two numbers in its Configurer.

The rest of N1MM's action macros — the CAT families, audio, rotators, stereo,
call stacking, the digital TNC macros — are read and passed over, so a message
that holds one still sends the right characters.

### ESM — Enter sends the message

**Config → ESM** turns it on, and it stays on between runs. Enter then sends
whatever the contact has got to, instead of only logging, which is how most
people run N1MM. The keys Enter would send next are highlighted, so what is
about to happen is on the screen.

Searching: type a call, Enter sends your call (F4), space moves to the exchange,
and when the exchange is filled in Enter sends yours (F2) and logs the contact.
Running: Enter calls CQ (F1); a call in the box makes Enter send his call and
your exchange (F5 and F2); once the exchange you copied is in the boxes, Enter
sends the message that ends the contact (F3) and logs it. A station already in
the log gets QSO B4 (F6) while running, and nothing at all while searching.
Pressing F1 while searching puts you into run mode, as it does in N1MM.

`=` sends whatever Enter last sent, without deciding again. Escape and F12 clear
the boxes and put ESM back to the start of a contact.

Three switches live in `settings.json`, and are N1MM's own:
`EsmSendsCallOnce` sends your call once while searching and then moves to the
exchange (N1MM's "big gun" switch), `EsmWorksDupes` works a station that calls
in again while you are running, and `EsmSendsCorrectedCall` sends the call again
in front of the last message when you have corrected it — copy `SM3AB`, fix it
to `SM3ABC`, and the key sends `SM3ABC TU DL1ABC`.

### Editing the log

Double-click a cell in the log window to change it. The columns follow the
contest exchange, so CQ WW shows a Zone column and Sweepstakes shows Nr, Prec,
Ck and Sec. A value the contest does not accept — zone 41, an ARRL section that
does not exist, a callsign with a space in it — is refused and the reason
appears under the log. Changing the callsign looks the country up again.

Delete, or the right-click menu, removes the selected contact after a
confirmation. Either way the whole log is scored again, so a multiplier the
removed contact was holding passes to the next contact that claims it.

**Edit → Edit Last Contact**, Ctrl+Y, or Enter on a row in the log opens the
whole contact in one dialog. That is where the fields with no log column live:
QSX frequency, name, QTH, comment, grid, power, radio number, run position, and
the country and WPX prefixes. ◀ and ▶ walk the log without closing, offering to
save first. Points and the multiplier flags are shown but cannot be typed in —
they are worked out from the rules every time the log changes, so anything typed
there would be overwritten on the next edit.

Correcting the country prefix by hand changes the score. The country file is a
best guess for calls it has no rule for, so what the contact says now wins over
what the file says.

### Call history files

**Config → Call History File…** points at one of the files published before a
contest. It is N1MM's format, so a file written for N1MM is read as it is:
`#` comments, `!!Order!!` naming the columns of the lines after it, semicolons
in preference to commas, and the directives that change what is stored —
`!!FourCharGridSq!!`, `!!NoLoc2AltGrid!!`, `!!MapStateToSect!!`,
`!!AppendUserText!!` and `!!NoAppendUserText!!`.

Leaving the callsign box fills what the file knows into the exchange boxes that
are still empty. A box with something in it is left alone: what the other
station actually sends beats what somebody published months ago. The check
window gets a History column of matching calls.

The section-validating directives — `!!Validate50State!!`,
`!!ValidateArrlSection!!`, `!!MapOnSection!!`, `!!GTA2GH_NT2TER!!` — are read
and passed over rather than acted on. Throwing data away on a section list this
program does not hold would be worse than keeping it.

### The check window

Five columns, kept apart because they answer different questions: **Log** is
what this station copied itself, **Master** is a guess about the whole world,
**History** is what was published before the contest, **Bandmap** is somebody
else's claim, and **Exchange** offers the values the exchange box with the
cursor in it can hold — the ARRL sections, or the states and provinces. A
serial number or a report has no list behind it, so that column stays empty
rather than guessing.

### Available mults and Qs

**View → Available Mults and Qs** lists what is spotted and not worked yet, a
column per band, multipliers first and then the stations that are only points.
Clicking one puts the radio there with the call in the entry window, the same as
clicking a spot on the bandmap. **Mults only** hides everything that brings no
multiplier.

A spot carries no exchange, so a station is only known to be a multiplier where
the multiplier follows from the callsign — a DXCC entity — or where the call
history file says what the station sends. That is why CQ WW shows the country
but not the zone: the zone that counts is the one the station sends, and the
country file's guess is wrong for the large countries. A call history file with
a `CQZone` column fills that in.

A spot carries no mode either. Stations are judged in the mode the radio is in,
so in a mixed-mode contest the answer follows the operator.

### Radios

**Config → Radios** takes a `rigctld` address per radio. Each radio needs its
own `rigctld`, started for whichever rig is on that port:

    rigctld -m 2028 -r /dev/ttyUSB0 -t 4532
    rigctld -m 1035 -r /dev/ttyUSB1 -t 4533

Commands go out with a `+` in front, which asks `rigctld` for its extended
answer: named fields ended by an `RPRT` line. The raw answer is bare values with
no terminator, so the client has to know how many lines each command returns,
and one wrong count leaves the connection reading every later answer against
the wrong command.

Split is read from the radio and recorded: the contact stores where we
transmitted in N1MM's QSX column, the entry window shows `14008.00 ▸ 14020.0`,
and the bandmap draws a red bar at the transmit frequency. A radio that cannot
do split answers `RPRT -11`; that is an answer, not a broken connection, and
everything else it reported still counts.

A second radio makes the station SO2R and opens a second entry window. The two
windows share one log, one score and one run of serial numbers, so a station
worked on radio 1 is a dupe on radio 2. What is typed, the frequency, the mode
and whether you are running belong to each radio on its own.

| Key | What it does |
|---|---|
| Ctrl+Tab | moves you to the other radio: the keyboard, the entry window and the box all follow |
| Ctrl+Shift+Tab | puts both radios in the headphones, and back |

Both radios show on the bandmap — the one you are on in green, the other in
orange — but only the one you are on drives its entry window, so the second
radio moving cannot drag you off the station you are working.

An SO2R box speaking OTRSP routes the key and the headphones. Name its serial
port in **Config → Radios** and it follows: `TX2` when you move to radio 2,
`RX1S` when you ask for both radios, and `TX` again before every message goes
out, so a message cannot go out of the radio you have just left. A command that
would change nothing is not sent, because the box works relays. Without a box
the logger just keeps track of which radio it is on and you switch by hand.

Not there yet: voice keying on the second radio.

### Alternating CQ

**Ctrl+B** calls CQ on one radio, and when that message has gone out, moves to
the other radio and calls there, until Escape or another Ctrl+B. The keyboard,
the entry window and the SO2R box follow each turn. N1MM calls this dueling CQs
and puts it on the same key.

It runs off the keyer saying the message has gone out, not off a guess at how
long the text takes: `cwdaemon` answers the `<ESC>h` reply request, and a
WinKeyer clears the busy bit in its status byte. A keyer that cannot report this
cannot drive alternating CQ, and says so rather than keying the second radio
while the first is still sending. **Config → Keyer and messages** sets the gap
between the two, which is 100 ms by default because an SO2R box works relays.

CW only for now: it sends the F1 message, and there is no voice keyer.

Why the keying is `cwdaemon`'s job rather than the logger's, and what it would
take to key the port directly as N1MM does, is in
[`docs/keying.md`](docs/keying.md).

### The bandmap

A frequency scale down the left with the stations written out beside it. Each
callsign sits level with its frequency, and when two stations are too close to
write one above the other the lower one moves down and a leader line runs back
to where it really is. That is what N1MM does, and the stacking follows its
rule: place the label centred on its frequency, and push it below the one above
if it would cover it.

The slice shown follows the receiver — 10, 20, 40 or 100 kHz, or the whole band
— and stretches past the band edge when the radio is past it.
Callsigns are coloured by the same scorer as the entry window, so a dupe reads
as a dupe here too. Clicking a callsign puts the radio there with the call
already in the entry window; clicking anywhere else just moves the radio.

The bars on the scale are as wide as the mode passes: green where you are
listening, red where the radio transmits when working split, orange for the
other radio of a two-radio station.

Behind the scale the CW part of the band is one shade and the phone part
another, with a third shade for a digital block where one is set, and the CW
boundary named in the status line. Anything past a band edge is shaded red,
which is where the edges show. N1MM paints the same blocks from its `CWBands`,
`DigiBands` and `SSBBands` tables.

**Config → Sub bands** edits them, one row per band, in kilohertz, with a
Defaults button. The digital pair starts empty, as N1MM's does, so nothing is
painted for it until it is filled in. Only the rows that differ from the
defaults are written to the settings file.

The defaults are one worldwide set, as N1MM's are, not one per ITU region. They
are a drawing aid: a contact logs either side of a boundary just the same.
Bands where N1MM's own numbers contradict themselves — 60M, and everything
above 2M — get no shading rather than a guess.

### Worked All Europe

WAE is scored differently from everything else here. A multiplier counts four
times on 80 metres, three times on 40 and twice on 20, 15 and 10, and half the
score is QTC traffic: one station reports contacts it has already made to the
station it is working, and every line reported is a point for both of them.

The QTC lines live in the log as rows of their own, which is where N1MM keeps
them — `Exchange1` says `SQTC` or `RQTC`, the time, callsign and serial number
of the contact being reported go in the sent report, received report and sent
number columns, and the series such as `9/10` goes in the misc text column. A
log written by either program therefore opens in the other with its QTCs
intact, and a QTC row is written to the Cabrillo file as a `QTC:` record.

**Ctrl+Z** opens the QTC window for the station being worked, which is the key
N1MM uses. The window is laid out the same way: the header, ten lines of time,
callsign and serial number, an Agn and a Cfm button beside each, and Clear,
Close and Cancel underneath. Green is a line that will be saved, red one that is
still empty, yellow one that cannot be read. Enter and Tab move forward, space
moves within a line.

Receiving, the lines are typed as they arrive. Sending, they are filled from the
log — the contacts not reported to anybody yet, oldest first, never the station
being worked, and never more than the ten any station may have — and cannot be
edited. On CW and SSB the direction is fixed by the rules; on RTTY traffic goes
both ways and the window has a switch.

Right-click the window for **Setup**: how many QTCs a series carries, whether
the RX Ready and Cfm steps are stopped at, and whether Hdr Agn and Agn clear
what they ask for again. The defaults are N1MM's — both steps skipped, the
header cleared, the line left alone.

Nothing in the window transmits. On CW the operator sends from the entry window,
on SSB by voice, and RTTY has no digital window yet, so Agn, Cfm and RX Ready
move the cursor rather than putting anything on the air. N1MM's QTC settings for
sending — the CW messages for Agn and TU, the field spacing, the SSB recordings
and the RTTY templates — are left out with them.

The scoring was checked against six real WAE logs, and agrees with N1MM on
points and on weighted multipliers to the contact.

### Band edges

A contact just outside a band edge keeps its band. `Bands.ForFrequency` first
checks the real edges, and for anything outside them falls back to the whole
megahertz: 14400 kHz is 20M, 7305 kHz is 40M. This is N1MM's rule, and it is
there because contest operating goes past the edges — someone answers a CQ at
14352, or a radio reports a frequency a few hundred hertz off — and a contact
with no band scores nothing and exports wrong. Below 1 MHz the megahertz is
always 0, so 2190M and 630M are told apart by their edges alone.

### The DX cluster

**View → Telnet**, or **Config → Cluster**, opens the telnet window. It has four
tabs, the ones N1MM has:

- **Telnet** shows the node's traffic as it arrived, spot lines in green and
  what went out in blue. Type a command at the bottom, or press one of the
  buttons; up and down walk back through what has been typed. Double clicking a
  spot line, or "Jump to this spot" on the right-click menu, puts the radio
  there with the call in the entry window. Scrolling stops while the pointer is
  over the traffic, so a line can be read while the node keeps sending.
- **Clusters** keeps a list of nodes with their names, ports, passwords and the
  commands to send after login. Connect and Disconnect are there, with automatic
  logon, the call to log on with, and the keep-alive interval. **Download** under
  it fetches the published list of nodes from
  [NG3K](https://www.ng3k.com/misc/cluster.html) — around fifty nodes with the
  sysop's call, the address and a note about each — and clicking one fills the
  boxes in. N1MM downloads its list from its own web service, which asks the
  operator to opt in to data collection; this reads a public page instead.
- **Filters** says which spots reach the bandmap: bands, modes, beacons, busted
  calls, stations outside the call history file, blacklisted spotters and calls,
  whether to take spots only from your country, your continent or a list of
  prefixes, and how long a spot stays on the bandmap. It also holds the preferred
  spotters and the switch that randomises incoming CW spot frequencies. Nothing
  is filtered out of the traffic itself; the operator sees everything the node
  sends.
- **Buttons** edits the twelve command buttons: a label and its commands, or
  back to the defaults.
- **Spot comment** is what goes out with the spots you send.

The buttons come from the settings; without any, they are `sh/dx`, `sh/dx/20`,
`sh/wwv`, `sh/users`, `sh/c/n`, `help` and `bye`. A button holds what N1MM's
buttons hold: several commands separated by semicolons, the function-key message
macros — `sh/dx {MYCALL}` sends the station callsign — or `{CONN}` and the name
of a favourite, which connects to that node instead of sending anything. The
label takes the macros too, expanded once when the buttons load, so a label of
`{MYCALL}` reads as the callsign. Right-clicking a button opens the editor, and
its tooltip shows the command behind it. What is typed on the command line goes
out as typed, again as N1MM sends it. The one place this differs from N1MM: the
commands of one button go out together rather than a quarter of a second apart.

Three of the filters are worth spelling out:

- **Busted spots.** A spotted call the callsign database has never heard, but
  which is one character away from a call it does know, is a miscopy and is
  dropped. A call nothing resembles is kept, because that is what a new station
  looks like. Without `MASTER.SCP` nothing is called busted.
- **Preferred spotters.** Spot lines from those spotters are written in bold, so
  the spotters worth believing stand out. They are matched on the start of the
  call, so `W3LPL` also covers `W3LPL-#`. Nothing is filtered by this.
- **Randomised frequencies.** Each incoming CW spot is moved thirty or sixty
  hertz either way, which is what N1MM does, so the operator has to find the
  station by ear. Phone and digital spots are left alone.

The client speaks telnet properly: it answers the option negotiation instead of
letting the control bytes turn up in the first lines of text, and it reads the
login prompt out of a partial line, because nodes write `login: ` with no line
ending. If no prompt arrives within ten seconds the callsign goes out anyway,
which is what N1MM does. A connection that has heard nothing for four minutes
gets a blank line, so the node does not drop it as idle. A dropped connection is
retried. The last two hundred lines are kept, so a window opened mid-contest is
not blank.

Alt+P, or **Edit → Spot It**, puts the call being typed on the cluster at the
current frequency, with the comment from the Spot comment tab — which takes the
message macros too; with nothing
typed it spots the last contact logged. The spot also goes straight onto our own
bandmap rather than waiting to come back round from the node.

Filters are applied to the spots, not to the node: it is still worth filtering
at the node as well, because that is less traffic on the wire.

### Networked stations

**Config → Network** names this station and lists the others. Each contact is
sent to them as it is logged, in N1MM's `contactinfo` message, so an N1MM
station on the same network sees them too. Editing a contact sends
`contactreplace` and deleting one sends `contactdelete`, the same as N1MM. With
no addresses listed the messages are broadcast. A contact that arrives is scored
again here from the rules rather than trusted.

An incoming edit or delete is matched by contact id first. N1MM does not know
our ids, so the call and the timestamp are the fallback — that is the pair N1MM
itself keys a contact on.

## Layout

| Project | What it holds |
|---|---|
| `Nonemm.Core` | Frequencies, bands, modes, callsigns, grid squares, the country file and the callsign database |
| `Nonemm.Contests` | Contest rules, the scoring engine, `.udc` files |
| `Nonemm.Formats` | Cabrillo out, ADIF in and out |
| `Nonemm.Storage` | The N1MM-compatible `.s3db` |
| `Nonemm.Rig` | Radio control over `rigctld`, and the SO2R box |
| `Nonemm.Spotting` | Spots, the bandmap, the DX cluster client |
| `Nonemm.Network` | Contacts shared between the stations of a multi-operator entry |
| `Nonemm.Keying` | CW through `cwdaemon` or a WinKeyer |
| `Nonemm.Session` | The contest in progress, and one entry position per radio — no UI toolkit |
| `Nonemm.App` | The Avalonia windows |

The split at `Nonemm.Session` is the important one: it references no UI
framework, so what space does, when a dupe fires and what a contact scores are
covered by plain unit tests.

## Where it stands

Working: logging a contest end to end, live dupe and multiplier checking, eight
built-in contests plus user-defined ones, Cabrillo and ADIF export, ADIF import,
the log, check, graphical bandmap, score and telnet windows, editing and deleting logged
contacts, radio control, DX cluster spots, contacts shared between networked
stations, and CW keying.

Checked against N1MM 1.0.11031: a log this program wrote opens in N1MM, which
reads the contest, its categories and the contacts. See
[`docs/n1mm-interop.md`](docs/n1mm-interop.md) for what that took.

Nothing here has been run against a real radio, keyer, SO2R box or cluster node.
Every one of those is tested against a fake that speaks the same protocol, which
proves the two agree with each other and nothing more.

[`docs/unfinished.md`](docs/unfinished.md) lists what is half-built and what has
never been tested against the real thing.