I referred to the bII chord as a subdominant function, not dominant. It has its origins in the minor mode as a "Neapolitan 6," but has become a common SD chord in both major and minor modes; and it is now done in all inversions. Check out Piston's entire chapter on this chord (I just re-read it). It makes sense when you think about it, since it has both of the SD notes: 2 and 4. The chromatic lowering of the supertonic only increases its need to resolve, making it even more active in a subdominant way.
Therefore, Fmaj13Q / B7#9 / in the key of Em would be SD to D. Indeed, in the key of C Major it would also be a SD to D cadence, since it would be IVMA7 to the leading-tone Dominant chord (or V7/iii, V7 of a Tonic chord).
Regarding the chord oscillation || Am9 - - - | F∆ - - - | Am9 - - - | F∆ - - - |:
Related to your observation about the characteristic harmonic minor-ish motive of the A section, perhaps Miles was trying to give momentary relief from that H- sound, since this is a "diatonic" phrase which temporarily leaves that behind. It seems to suggest this pc (however you term it): A B C D E F G A.
Also, the progression is temporarily weakened (in tonal terms) by its root progression in thirds, rather than the stronger tonal fourths. This makes the direct modulation to the key of C all the stronger by comparison, with its straight-forward cadence and four-measure duration.
Getting back to your motive:
I too view it as the salient characteristic of the entire composition. Therefore, without caring what it is called, I start with developing that in etudes throughout the entire form--with possible exception of the "diatonic" section. This can be done by adopting certain of its notes to facilitate its incorporation into other contexts throughout the progression. This is the place where I begin to develop "pseudo improvisations" on the specific tune in the woodshed.
With regard to chord scales:
Also in the woodshed, I will learn to sing and improvise on all of these choices. But I prefer to do this separately on each pc at length, and to not worry at all about which chords they "go with" locally.
Once I have them internalized, I start running choruses and they find their own--more melodic--ways of combining into lines in their natural course through the singing process. This is how I employ any new vocabulary, in addressing a specific tune. When you think about it, there are tens of thousands of ways that these scales or their subsets can be used. I find that a "left brain" intellectual process at this point is not only tedious, but (more importantly) that it impedes the improvisation process.
Ed, given your observation of those alternate scales with similar motives, I view them all as living options in performance in the now. I feel virtually no need to justify any such choices. Moreover, depending on how they are actually sounded, many of those notes could function as mere non-harmonic tones as well.
Therefore, these names of scales, while helpful in their identification in first learning them, are to be forgotten once internalized. They are meaningless in actual performance.
Regarding "giving up" something: I never view the learning of new vocabulary as giving anything up: "Make new friends and keep the old. One is silver and the other, gold." Same thing goes for learning new methods or concepts: Your time was never wasted: you are just moving on. I once complained to Mick Goodrick that I had "wasted a month" working on something that ended up sucking, and he said: "At least you learned what you didn't like."