In AnVIL, disable the export to Terra functionality for the four datasets listed here with an explanation as to why. That comment also links to the Slack thread with stakeholders.
To summarize, the only problematic datasets are
* `ANVIL_eMERGE_GWAS_HMB_MDS`
* `ANVIL_eMERGE_GWAS_GRU`
* `ANVIL_eMERGE_GWAS_HMB_GSO`
* `ANVIL_eMERGE_GWAS_HMB_PUB_GSO`
As we established above, these are the same datasets that also cause aggregation issues (#8017).
And as I surmise above, a permanent solution would likely require a substantial revamp of the verbatim PFB manifest generation. We were told that Terra is unable to import three of the four datasets, so there wouldn't be any point in spending time on that.
We should, however, prevent users from even attempting to export these four datasets to Terra. This would reduce user frustration (each attempt takes 15min before it eventually fails) and save internal team resources (each failed attempt raises an alarm that the operator has to spend 5-10 min investigating). The best place to do implement these preventative measures is the Data Browser. While Azul could be modified to fail early, refusing to even start the futile 15min attempt at manifest generation, the Data Browser could disable the respective button and display an explanation as to why it is disabled.
I proposed this on Slack to stakeholders.
In AnVIL, disable the export to Terra functionality for the four datasets listed here with an explanation as to why. That comment also links to the Slack thread with stakeholders.