ENA - Create maps on a progressive basis

[jtniehof]

Summary

Dave has given direction to proceed with making partial maps: when data are downlinked, all the canonical maps should be automatically produced, including all available data up to the recent downlink. The rest will be fill and the maps will gradually "fill in".

This still needs additional definition with IMWG, @jtniehof and @ahotasu will work that.

Once the feature is developed, should be integration-tested in dev and then deployed to prod immediately.

Requirements

• Currently the plan is for the file to be named like the final map, and increment versions as additional data are filled in; can be revisted.
  • Only the final version will be made public; we may want to consider archive scrubbing capabilities.
• Triggering needs to be evaluated; possibilities include:
  • Many-to-one triggering from pointing sets to maps, and ensuring this doesn't "thrash" if we download multiple pointings' worth of data in one downlink
  • Time-based processing: make everything daily (may result in version thrash on days where there is no downlink). Can potentially use mapper tool code for gathering inputs (it is likely a thin layer over imap-data-access).
  • Time-based processing: trigger n hours after downlink; again, can use mapper tool.
  • Time-based checks: trigger daily, but the code triggered is a more elaborate dependency checker that evaluates possible maps to make (similar to GLOWS L3b+ code)--this hides some dependency/triggering from the main system.

Related tickets

This may supersede #2774 but they have slightly different emphases, so making a new issue.

IMAP-Science-Operations-Center/imap_L3_processing#60 is a component of this for the SP-corrected L3 maps.

It's expected there will be child issues for this.

[jtniehof]

@ahotasu , am I missing anything from our conversation?

[jtniehof]

There may be work to be done, both pipeline and mapper, in order to use predict ephemeris (which is needed because reconstruct can be delayed by a few weeks)