Carry out the call history section directives

A call history file is built from years of logs, so its section and
state columns carry old names, and the file says at the top what to do
about that. The four lines that say it were read and passed over.

All four are now carried out, in N1MM's order: !!MapOnSection!! stores
the old Ontario sections as ON, !!ValidateArrlSection!! drops a section
that is not one of N1MM's, !!GTA2GH_NT2TER!! stores GTA as GH and NT as
TER, and !!Validate50State!! keeps a state, turns a section into the
state it is in, or drops the value.

The section list is the one N1MM's call history reader validates
against, which is not the list its entry window offers: it holds the
retired GTA and NT, and leaves out ON. It sits beside the reader in
Nonemm.Core rather than with the contest lists, which are the entry
window's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoGtneMQaz4M9w7Kk49AVD

README.md

@@ -390,17 +390,30 @@ contest. It is N1MM's format, so a file written for N1MM is read as it is:
 `#` comments, `!!Order!!` naming the columns of the lines after it, semicolons
 in preference to commas, and the directives that change what is stored —
 `!!FourCharGridSq!!`, `!!NoLoc2AltGrid!!`, `!!MapStateToSect!!`,
-`!!AppendUserText!!` and `!!NoAppendUserText!!`.
+`!!AppendUserText!!`, `!!NoAppendUserText!!` and the four that clean up the
+section and state columns.
 
 Leaving the callsign box fills what the file knows into the exchange boxes that
 are still empty. A box with something in it is left alone: what the other
 station actually sends beats what somebody published months ago. The check
 window gets a History column of matching calls.
 
-The section-validating directives — `!!Validate50State!!`,
+These files are built from years of logs, so the section and state columns
-`!!ValidateArrlSection!!`, `!!MapOnSection!!`, `!!GTA2GH_NT2TER!!` — are read
+carry old names. Four directives say what to do about that, and all four are
-and passed over rather than acted on. Throwing data away on a section list this
+carried out as they are read:
-program does not hold would be worse than keeping it.
+
+| Directive | What it does |
+| --- | --- |
+| `!!MapOnSection!!` | `GTA`, `GH`, `ONE`, `ONN` and `ONS` are stored as `ON` |
+| `!!ValidateArrlSection!!` | a section that is not one of N1MM's is dropped |
+| `!!GTA2GH_NT2TER!!` | `GTA` is stored as `GH`, `NT` as `TER` |
+| `!!Validate50State!!` | the state column keeps one of the 50 states, turns a section into the state it is in, and drops the rest |
+
+The section list is N1MM's, which is not the same list its entry window offers:
+it holds the retired `GTA` and `NT` so a file carrying an old name is fixed
+rather than thrown away, and it leaves out `ON`, so a file asking for both
+`!!MapOnSection!!` and `!!ValidateArrlSection!!` ends with no section — which
+is what N1MM does, and what the file's own author asked for.
 
 ### The check window
 

docs/unfinished.md

@@ -29,12 +29,6 @@ of an SO2R station cannot call CQ by voice. Alternating CQ therefore works on CW
 only, though nothing in it is CW-specific: a voice keyer that reports when the
 recording has finished would drive it as it stands.
 
-**Call history: the section-validating directives.** `!!Validate50State!!`,
-`!!ValidateArrlSection!!`, `!!MapOnSection!!` and `!!GTA2GH_NT2TER!!` are read
-and passed over. Acting on them means holding N1MM's section lists and its
-rules about which sections are retired, and getting that wrong throws away good
-data. The file is read without them.
-
 **The QTC window transmits on CW only.** The header, the lines, the QRV, the TU
 and the again messages go out through the keyer, with N1MM's messages and
 N1MM's defaults. Nothing goes out on SSB or RTTY: N1MM plays four recordings on

src/Nonemm.Core/Calls/CallHistory.cs

@@ -39,11 +39,7 @@ public sealed class CallHistory
     public static CallHistory Parse(string text)
     {
         Dictionary<string, CallHistoryEntry> found = new(StringComparer.Ordinal);
-        string[] order = DefaultOrder;
+        Directives said = new();
-        bool fourCharacterGrid = false;
-        bool useAlternateGrid = true;
-        bool mapStateToSection = false;
-        bool appendUserText = true;
 
         foreach (string line in text.Split('\n'))
         {
@@ -56,40 +52,81 @@ public sealed class CallHistory
             string first = fields[0].Trim().ToUpperInvariant();
             if (first.StartsWith("!!", StringComparison.Ordinal))
             {
-                switch (first)
+                said.Read(first, fields);
-                {
-                    case "!!ORDER!!":
-                        order = [.. fields.Skip(1).Select(f => f.Trim().ToUpperInvariant())];
-                        break;
-                    case "!!FOURCHARGRIDSQ!!":
-                        fourCharacterGrid = true;
-                        break;
-                    case "!!NOLOC2ALTGRID!!":
-                        useAlternateGrid = false;
-                        break;
-                    case "!!MAPSTATETOSECT!!":
-                        mapStateToSection = true;
-                        break;
-                    case "!!APPENDUSERTEXT!!":
-                        appendUserText = true;
-                        break;
-                    case "!!NOAPPENDUSERTEXT!!":
-                        appendUserText = false;
-                        break;
-                }
-                // the rest are validation directives, and validating a section
-                // list we do not hold would throw data away
                 continue;
             }
-            CallHistoryEntry? entry = Read(fields, order, fourCharacterGrid, useAlternateGrid, mapStateToSection);
+            CallHistoryEntry? entry = Read(fields, said);
             if (entry is not null)
             {
-                found[entry.Call] = Merge(found.GetValueOrDefault(entry.Call), entry, appendUserText);
+                found[entry.Call] = Merge(
+                    found.GetValueOrDefault(entry.Call),
+                    entry,
+                    said.AppendUserText);
             }
         }
         return new CallHistory(found);
     }
 
+    /// The `!!…!!` lines at the top of a file, which say how to read the rest
+    /// of it. A line the format does not define is passed over.
+    private sealed class Directives
+    {
+        public string[] Order { get; private set; } = DefaultOrder;
+
+        public bool FourCharacterGrid { get; private set; }
+
+        public bool UseAlternateGrid { get; private set; } = true;
+
+        public bool MapStateToSection { get; private set; }
+
+        public bool AppendUserText { get; private set; } = true;
+
+        public bool ValidateArrlSection { get; private set; }
+
+        public bool MapOnSection { get; private set; }
+
+        public bool RenameGtaAndNt { get; private set; }
+
+        public bool ValidateFiftyState { get; private set; }
+
+        public void Read(string directive, string[] fields)
+        {
+            switch (directive)
+            {
+                case "!!ORDER!!":
+                    Order = [.. fields.Skip(1).Select(f => f.Trim().ToUpperInvariant())];
+                    break;
+                case "!!FOURCHARGRIDSQ!!":
+                    FourCharacterGrid = true;
+                    break;
+                case "!!NOLOC2ALTGRID!!":
+                    UseAlternateGrid = false;
+                    break;
+                case "!!MAPSTATETOSECT!!":
+                    MapStateToSection = true;
+                    break;
+                case "!!APPENDUSERTEXT!!":
+                    AppendUserText = true;
+                    break;
+                case "!!NOAPPENDUSERTEXT!!":
+                    AppendUserText = false;
+                    break;
+                case "!!VALIDATEARRLSECTION!!":
+                    ValidateArrlSection = true;
+                    break;
+                case "!!MAPONSECTION!!":
+                    MapOnSection = true;
+                    break;
+                case "!!GTA2GH_NT2TER!!":
+                    RenameGtaAndNt = true;
+                    break;
+                case "!!VALIDATE50STATE!!":
+                    ValidateFiftyState = true;
+                    break;
+            }
+        }
+    }
+
     /// A call can appear more than once. Later lines win, except that user text
     /// is added to rather than replaced unless the file says otherwise.
     private static CallHistoryEntry Merge(CallHistoryEntry? earlier, CallHistoryEntry later, bool append)
@@ -105,29 +142,25 @@ public sealed class CallHistory
         return later with { UserText = $"{earlier.UserText} {later.UserText}" };
     }
 
-    private static CallHistoryEntry? Read(
+    private static CallHistoryEntry? Read(string[] fields, Directives said)
-        string[] fields,
-        string[] order,
-        bool fourCharacterGrid,
-        bool useAlternateGrid,
-        bool mapStateToSection)
     {
+        string[] order = said.Order;
         string call = At(fields, order, "CALL").ToUpperInvariant();
         if (call.Length == 0)
         {
             return null;
         }
-        string section = At(fields, order, "SECT");
+        string section = Section(At(fields, order, "SECT"), said);
-        string state = At(fields, order, "STATE");
+        string state = State(At(fields, order, "STATE"), said);
         return new CallHistoryEntry
         {
             Call = call,
             Name = At(fields, order, "NAME"),
-            GridSquare = Grid(At(fields, order, "LOC1"), fourCharacterGrid),
+            GridSquare = Grid(At(fields, order, "LOC1"), said.FourCharacterGrid),
-            AlternateGridSquare = useAlternateGrid
+            AlternateGridSquare = said.UseAlternateGrid
-                ? Grid(At(fields, order, "LOC2"), fourCharacterGrid)
+                ? Grid(At(fields, order, "LOC2"), said.FourCharacterGrid)
                 : "",
-            Section = section.Length == 0 && mapStateToSection ? state : section,
+            Section = section.Length == 0 && said.MapStateToSection ? state : section,
             State = state,
             Check = Number(At(fields, order, "CK")),
             Exchange1 = At(fields, order, "EXCH1"),
@@ -139,6 +172,48 @@ public sealed class CallHistory
         };
     }
 
+    /// The section as the file's directives ask for it. The three run in
+    /// N1MM's order, so a file asking for both `!!MapOnSection!!` and
+    /// `!!ValidateArrlSection!!` ends up with no section at all: the first
+    /// makes `ON`, and `ON` is not on the list the second checks against.
+    private static string Section(string section, Directives said)
+    {
+        if (section.Length == 0)
+        {
+            return section;
+        }
+        string name = section.ToUpperInvariant();
+        if (said.MapOnSection && name is "GTA" or "GH" or "ONE" or "ONN" or "ONS")
+        {
+            name = "ON";
+        }
+        if (said.ValidateArrlSection && !SectionNames.IsArrlSection(name))
+        {
+            return "";
+        }
+        if (said.RenameGtaAndNt)
+        {
+            name = name switch { "GTA" => "GH", "NT" => "TER", _ => name };
+        }
+        return name;
+    }
+
+    /// `!!Validate50State!!` keeps a value that is one of the 50 states, turns
+    /// a section into the state it is in, and drops anything else.
+    private static string State(string state, Directives said)
+    {
+        if (state.Length == 0 || !said.ValidateFiftyState)
+        {
+            return state;
+        }
+        if (SectionNames.IsFiftyState(state))
+        {
+            return state.ToUpperInvariant();
+        }
+        string fromSection = SectionNames.StateFor(state);
+        return SectionNames.IsFiftyState(fromSection) ? fromSection : "";
+    }
+
     /// N1MM takes semicolons in preference to commas, so a file whose user text
     /// holds commas can still be read.
     private static string[] Split(string line) =>

src/Nonemm.Core/Calls/SectionNames.cs

@@ -0,0 +1,70 @@
+namespace Nonemm.Core.Calls;
+
+/// The section and state names a call history file is cleaned up against.
+///
+/// The section list is N1MM's `IsArrlSection`, which is not the same as the
+/// list its entry window offers: it holds the retired `GTA` and `NT` and the
+/// odd `MD`, `HI` and `MAR` as well, so a file carrying an old name is fixed
+/// rather than thrown away. It leaves out `ON`, which is why
+/// `!!ValidateArrlSection!!` removes a section a `!!MapOnSection!!` line has
+/// just made.
+public static class SectionNames
+{
+    private static readonly IReadOnlySet<string> Sections = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
+    {
+        "AB", "AK", "AL", "AR", "AZ", "BC", "CO", "CT", "DE", "EB", "EMA",
+        "ENY", "EPA", "EWA", "GA", "GH", "GTA", "HI", "IA", "ID", "IL", "IN",
+        "KS", "KY", "LA", "LAX", "MAR", "MB", "MD", "MDC", "ME", "MI", "MN",
+        "MO", "MS", "MT", "NB", "NC", "ND", "NE", "NFL", "NH", "NL", "NLI",
+        "NM", "NNJ", "NNY", "NS", "NT", "NTX", "NV", "OH", "OK", "ONE", "ONN",
+        "ONS", "OR", "ORG", "PAC", "PE", "PR", "QC", "RI", "SB", "SC", "SCV",
+        "SD", "SDG", "SF", "SFL", "SJV", "SK", "SNJ", "STX", "SV", "TER", "TN",
+        "UT", "VA", "VI", "VT", "WCF", "WI", "WMA", "WNY", "WPA", "WTX", "WV",
+        "WWA", "WY",
+    };
+
+    private static readonly IReadOnlySet<string> FiftyStates = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
+    {
+        "AK", "AL", "AR", "AZ", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "IA",
+        "ID", "IL", "IN", "KS", "KY", "LA", "MA", "MD", "ME", "MI", "MN", "MO",
+        "MS", "MT", "NC", "ND", "NE", "NH", "NJ", "NM", "NV", "NY", "OH", "OK",
+        "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VA", "VT", "WA", "WI",
+        "WV", "WY",
+    };
+
+    /// The state each section that splits one belongs to. A section that is a
+    /// state on its own is not here, and neither is a Canadian one.
+    private static readonly IReadOnlyDictionary<string, string> StatesBySection =
+        new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
+        {
+            ["EMA"] = "MA", ["WMA"] = "MA",
+            ["SNJ"] = "NJ", ["NNJ"] = "NJ",
+            ["EWA"] = "WA", ["WWA"] = "WA",
+            ["EB"] = "CA", ["LAX"] = "CA", ["ORG"] = "CA", ["SB"] = "CA",
+            ["SCV"] = "CA", ["SDG"] = "CA", ["SF"] = "CA", ["SJV"] = "CA",
+            ["SV"] = "CA",
+            ["NTX"] = "TX", ["STX"] = "TX", ["WTX"] = "TX",
+            ["NFL"] = "FL", ["SFL"] = "FL", ["WCF"] = "FL",
+            ["ENY"] = "NY", ["NLI"] = "NY", ["NNY"] = "NY", ["WNY"] = "NY",
+            ["EPA"] = "PA", ["WPA"] = "PA",
+            ["MDC"] = "MD",
+            ["PAC"] = "HI",
+        };
+
+    public static bool IsArrlSection(string section) => Sections.Contains(section.Trim());
+
+    public static bool IsFiftyState(string state) => FiftyStates.Contains(state.Trim());
+
+    /// The state a section is in. A name that is not a section of a state comes
+    /// back as it went in, which is what N1MM's `StateForSection` returns, and
+    /// `DX` comes back empty.
+    public static string StateFor(string section)
+    {
+        string wanted = section.Trim();
+        if (wanted.Equals("DX", StringComparison.OrdinalIgnoreCase))
+        {
+            return "";
+        }
+        return StatesBySection.TryGetValue(wanted, out string? state) ? state : wanted;
+    }
+}

tests/Nonemm.Core.Tests/Calls/CallHistoryTests.cs

@@ -170,9 +170,9 @@ public class CallHistoryTests
         Assert.Equal("second", history.Find("OM5M")?.UserText);
     }
 
-    /// A directive we do not act on must not be read as a callsign.
+    /// A directive must not be read as a callsign.
     [Fact]
-    public void ADirectiveWeIgnoreIsStillNotData()
+    public void ADirectiveIsNotData()
     {
         CallHistory history = CallHistory.Parse("""
             !!ValidateArrlSection!!
@@ -184,6 +184,70 @@ public class CallHistoryTests
         Assert.Null(history.Find("!!ValidateArrlSection!!"));
     }
 
+    /// `!!MapOnSection!!`: the old Ontario sections are stored as `ON`.
+    [Theory]
+    [InlineData("ONN", "ON")]
+    [InlineData("GTA", "ON")]
+    [InlineData("EMA", "EMA")]
+    public void MapOnSectionStoresTheOntarioSectionsAsOne(string section, string expected) =>
+        Assert.Equal(
+            expected,
+            Section("!!MapOnSection!!", section));
+
+    /// `!!ValidateArrlSection!!`: a section that is not one of N1MM's is
+    /// dropped rather than stored.
+    [Theory]
+    [InlineData("EMA", "EMA")]
+    [InlineData("XYZ", "")]
+    [InlineData("ON", "")]
+    public void ValidateArrlSectionDropsWhatIsNotASection(string section, string expected) =>
+        Assert.Equal(expected, Section("!!ValidateArrlSection!!", section));
+
+    /// `!!GTA2GH_NT2TER!!`: the two renamed sections.
+    [Theory]
+    [InlineData("GTA", "GH")]
+    [InlineData("NT", "TER")]
+    [InlineData("ONN", "ONN")]
+    public void Gta2GhRenamesTheTwoSections(string section, string expected) =>
+        Assert.Equal(expected, Section("!!GTA2GH_NT2TER!!", section));
+
+    /// `!!Validate50State!!`: a state stays, a section becomes the state it is
+    /// in, and anything else goes.
+    [Theory]
+    [InlineData("OH", "OH")]
+    [InlineData("EMA", "MA")]
+    [InlineData("SCV", "CA")]
+    [InlineData("ONN", "")]
+    [InlineData("XYZ", "")]
+    public void Validate50StateKeepsOnlyStates(string state, string expected) =>
+        Assert.Equal(
+            expected,
+            CallHistory.Parse($"""
+                !!Validate50State!!
+                !!Order!!,Call,State
+                OM5M,{state}
+                """).Find("OM5M")?.State);
+
+    /// Without the directive the column is stored as the file wrote it.
+    [Fact]
+    public void AFileThatAsksForNoCleanupIsStoredAsItIs()
+    {
+        CallHistoryEntry? found = CallHistory.Parse("""
+            !!Order!!,Call,Sect,State
+            VE3ABC,ONN,XYZ
+            """).Find("VE3ABC");
+
+        Assert.Equal("ONN", found?.Section);
+        Assert.Equal("XYZ", found?.State);
+    }
+
+    private static string? Section(string directive, string section) =>
+        CallHistory.Parse($"""
+            {directive}
+            !!Order!!,Call,Sect
+            VE3ABC,{section}
+            """).Find("VE3ABC")?.Section;
+
     [Fact]
     public void ALineWithNoCallIsNotAnEntry() =>
         Assert.Equal(0, CallHistory.Parse(",Erik,JN88").Count);