Skip to content

Add logic to copy historical data - #2187

Merged
vish-cs merged 1 commit into
datacommonsorg:masterfrom
vish-cs:history
Aug 28, 2026
Merged

Add logic to copy historical data#2187
vish-cs merged 1 commit into
datacommonsorg:masterfrom
vish-cs:history

Conversation

@vish-cs

@vish-cs vish-cs commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a feature to ingest historic data from GCS into the versioned output directory, controlled by a new configuration flag ingest_historic_data. However, the changes also comment out the virtual environment creation (_create_venv), replacing it with a dummy process and sys.executable, which will bypass dependency installation and likely cause runtime failures. Additionally, the GCS blob listing logic for historic data should be refined to use a slash-terminated prefix to prevent matching unintended directories and to simplify path parsing.

Comment thread import-automation/executor/app/executor/import_executor.py Outdated
Comment thread import-automation/executor/app/executor/import_executor.py
Comment thread import-automation/executor/app/executor/import_executor.py Outdated
Comment thread import-automation/executor/app/executor/import_executor.py
Comment thread import-automation/executor/app/executor/import_executor.py
Comment thread import-automation/executor/app/executor/import_executor.py Outdated
Comment thread import-automation/executor/app/executor/import_executor.py
@vish-cs
vish-cs force-pushed the history branch 5 times, most recently from 49a8960 to ccd9a2b Compare August 28, 2026 07:44
@vish-cs
vish-cs merged commit 095a3c0 into datacommonsorg:master Aug 28, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants