Eric Ball Wrote
Hrm, the best advice I can give anyone is to write out a palette+sprite bit to color register index for the various modes based on the 7800 Software Guide.
For example 320B is:
P2|0|0|S7|S3 P2|0|0|S6|S2 P2|0|0|S5|S1 P2|0|0|S4|S0
So for 320B the MSB of the palette bits in the sprite header selects either palette 4 or 0, while the sprite bits select one of the colors (remembering the rules for background/transparency).
160A:
P2|P1|P0|S7|S6 P2|P1|P0|S5|S4 P2|P1|P0|S3|S2 P2|P1|P0|S1|S0
160B:
P2|S3|S2|S7|S6 P2|S3|S2|S7|S6
320A:
P2|P1|P0|S7|0 P2|P1|P0|S6|0 P2|P1|P0|S5|0 P2|P1|P0|S4|0
P2|P1|P0|S3|0 P2|P1|P0|S2|0 P2|P1|P0|S1|0 P2|P1|P0|S0|0
320C:
P2|S3|S2|S7|0 P2|S3|S2|S6|0 P2|S1|S0|S5|0 P2|S1|S0|S4|0
320D:
P2|0|0|S7|P1 P2|0|0|S6|P0 P2|0|0|S5|P1 P2|0|0|S4|P0
P2|0|0|S3|P1 P2|0|0|S2|P0 P2|0|0|S1|P1 P2|0|0|S0|P0
I just had an AH-HH! moment about this. MARIA has two 800 bit line RAMs, so each 160H pixel has 5 bits as a color LUT index. The 320H modes don't change how the line RAM is written to, just how it is read. Which, unfortunately, means there are only 2.5 bits per 320H pixel, or max 8 colors per pixel with one bit shared between the two half pixels.
I've also come up with a theory about how this came to be when MARIA was first developed. My guess is 160A was the original graphics mode which MARIA was designed for, which resulted in the 800 bit line RAMs. Then someone had the bright idea that since MARIA was operating at 2x colorburst (probably in order to increase the number of sprites per line) that it would be possible to increase the output pixel clock rate to allow 320H. Unfortunately, there weren't enough transistors left in the budget to increase the line RAMs. But with some minor tweaks to the output stages 320A was born. 320B was put in as a way to provide more colors, but required further modifications to the output and input stages. The remaining modes are just combinations of the input (write) and output (read) modes.