Skip to content

Allow for an agnostic column name to be used for object identifier. #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bdgregg wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Allow for an agnostic column name to be used for object identifier. #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ google_credentials_file: /path/to/google-service-account.json
google_sheet_id: "your_google_sheet_id"
google_sheet_name: "Sheet1"

id_column: id

environments:
production:
peertube_instance: https://your-peertube-instance.example.com
Expand All @@ -55,15 +57,17 @@ environments:

Multiple environments can be defined under `environments:`. The `--env` argument selects which one to use at runtime.

id_column sets the name of the column in the Google Sheet used as the unique identifier for each video record. Defaults to id if omitted. Can be overridden at runtime with --id-column.

---

## Google Sheet Format

The script reads from a Google Sheet with the following columns:

| Column | Required | Description |
|--------|----------|-------------|
| `id` | Yes | A local identifier for the video record |
|:----|:----|:----|
| (configurable, default `id`) | Yes | A local identifier for the video record. Column name is set via id_column in the config file or --id-column on the command line. |
| `title` | Yes | Title of the video |
| `file` | Yes | Path to the local video/media file |
| `field_media_oembed_video` | Yes | Full URL to the PeerTube video (used to extract the short UUID) |
Expand Down Expand Up @@ -98,6 +102,7 @@ python upload-media-transcripts.py \
| `--peertube-username` | No | Override the PeerTube username |
| `--peertube-password` | No | Override the PeerTube password |
| `--peertube-channel` | No | Override the PeerTube channel number |
| `--id-column` | No | Override the name of the identifier column from config (default: `id`)|

### Dry Run Example

Expand Down
16 changes: 12 additions & 4 deletions upload-media-transcripts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,7 @@ def parse_arguments():
parser.add_argument('--peertube-username', dest="peertube_username", help='PeerTube account username.')
parser.add_argument('--peertube-password', dest="peertube_password", help='PeerTube account password.')
parser.add_argument('--peertube-channel', dest="peertube_channel", help='PeerTube channel number.')
parser.add_argument('--id-column', dest="id_column", help='Name of the column to use as the unique row/media identifier (default: "id").')

# Parse the arguments:
args = parser.parse_args()
Expand Down Expand Up @@ -524,7 +525,7 @@ def peertube_add_caption(

def process_row(df: pd.DataFrame, row) -> pd.DataFrame:
# Required Vars for all media objects.
video_id = row['id']
video_id = row[id_column]
video_title = row['title']
video_file = row['file']

Expand Down Expand Up @@ -599,7 +600,7 @@ def process_sheet(sheet_id: str, sheet_name: str, credentials_file):
df = read_google_sheet(sheet_id, sheet_name, credentials_file)

# Check for Required columns
required_columns = ['id','title','file','field_media_oembed_video']
required_columns = [id_column,'title','file','field_media_oembed_video']
optional_columns = ['description','language','transcript','transcript_language']

row_count = 1
Expand All @@ -609,7 +610,7 @@ def process_sheet(sheet_id: str, sheet_name: str, credentials_file):
# Process each row in the sheet
for index,row in df.iterrows():
# Get Row information.
video_id = row['id']
video_id = row[id_column]
oembed_url = row['field_media_oembed_video']
transcript = row['transcript']

Expand Down Expand Up @@ -678,6 +679,11 @@ def main():
google_sheet_id = None
google_sheet_name = None

# Set id_column variable (default to 'id' if not specified anywhere).
id_column = 'id'
if cfg.get('id_column') and cfg['id_column'] is not None:
id_column = cfg['id_column']

# Override config file variable with command line parameters if they exist.
if args.in_gs_creds is not None:
google_credentials = args.in_gs_creds
Expand All @@ -696,7 +702,6 @@ def main():
else:
if cfg['google_sheet_name'] and cfg['google_sheet_name'] is not None:
google_sheet_name = cfg['google_sheet_name']


print(f"PeerTube Channel: {peertube_channel}")

Expand All @@ -715,9 +720,12 @@ def main():
google_sheet_name = args.in_gs_name
if args.log_file is not None:
log_file = args.log_file
if args.id_column is not None:
id_column = args.id_column

globals()['peertube_channel'] = peertube_channel
globals()['peertube_instance'] = peertube_instance
globals()['id_column'] = id_column

# Create the Log file.
print(f"Creating log file: {args.log_file}")
Expand Down