[Feature] Support overriding conf and logs directories in upstream startup scripts

[AvanishShukla20]

Feature Description (功能描述)

[Feature] Support overriding conf and logs directories in upstream startup scripts

Description

Currently, HugeGraph startup scripts (start-hugegraph.sh, start-hugegraph-pd.sh, and start-hugegraph-store.sh) resolve configuration and log paths relative to the installation directory ($TOP).

This assumption fails to work in distributed deployments where the installation directory is separate from the runtime CONF and LOGS directories.

This proposal introduces command-line flags (and optional environment variable fallbacks) to allow operators to explicitly override the target paths for CONF and LOGS at startup.

Background

Each startup script ( start-hugegraph.sh, hugegraph-server.sh, start-hugegraph-pd.sh and start-hugegraph-store.sh ) derives its working paths relative to the script location:

BIN=$(abs_path)
TOP="$(cd "$BIN"/../ && pwd)"
CONF="$TOP/conf"
LOGS="$TOP/logs"

This enforces that:

• Config files must reside under <install-dir>/conf.
• Logs must be written to <install-dir>/logs.

Currently, there is no native mechanism to specify custom runtime directories (such as configuration, logs) outside the unpacked apache-hugegraph-* distribution folder.

Problem

In a distributed cluster deployment, the different roles — Server, Store, and PD may share a common, versioned, immutable installation of the apache-hugegraph-1.7.0 distribution, while their runtime state (configuration, logs) can be managed separately per-node.

Common situations where the current behavior is a blocker:

Multiple roles / instances sharing one install.

Running multiple roles (Server, Store, PD) or distinct instances from a single shared binary distribution, each process requires an isolated configuration workspace and log directory. Forcing all roles to share $TOP/conf and $TOP/logs leads to configuration overwrites and log truncation.

Separation of binaries from configuration.

Operators frequently follow the convention of keeping immutable binaries in one location (e.g. /opt/...) and mutable, node-specific configuration and logs in another (e.g. /etc/... for config and /var/log/... for logs. The current scripts cannot honor this layout.

Read-only / immutable install directory.

In enterprise environments, distribution binaries are frequently installed on read-only filesystems (e.g., shared NFS mounts, OS package directories like /usr/lib/, or immutable container layers). Because HugeGraph scripts attempt to write logs to $TOP/logs and validate write access to $TOP/conf, startup fails on read-only filesystems due to permission errors.

Externally-generated configuration.

Configuration is often generated/templated by an external system into a dedicated writable runtime directory. Today the only way to use it is to copy or symlink files back into <install-dir>/conf, which mutates the installation and defeats the read-only goal.

The net effect: to run HugeGraph roles on a distributed cluster with a read-only install, users have to patch the startup scripts or overlay files into the install tree — both of which are fragile and hard to maintain across upgrades.

Proposed Solution

Add optional command-line flags to each startup script (parsed via the existing getopts blocks) that let the caller override the default directories. When a flag is not supplied, the script falls back to the current behavior, so this change is fully backward compatible.

Proposed flags

Flag	Overrides	Falls back to
-c <conf_dir>	Configuration directory	$TOP/conf
-l <logs_dir>	Log directory	$TOP/logs

Flags Overview per script file

start-hugegraph.sh (Server)

Flag	Meaning	Status
-c	Override conf directory	New (proposed)
-l	Override logs directory	New (proposed)
-d	Run as daemon (true/false)	Existing
-g	GC option (e.g. g1)	Existing
-j	Extra user/JVM option	Existing
-y	Enable/disable telemetry (true/false)	Existing
-m	Enable/disable monitor	Existing
-p	Preload option	Existing
-s	Enable/disable security check	Existing
-t	Server startup timeout (seconds)	Existing

hugegraph-server.sh (invoked by start-hugegraph.sh )

Input	Kind	Meaning	Status
CONF_OVERRIDE	env var	Override conf directory (CONF="${CONF_OVERRIDE:-$TOP/conf}")	New (proposed) — must be exported by start-hugegraph.sh
LOGS_OVERRIDE	env var	Override logs directory (LOGS="${LOGS_OVERRIDE:-$TOP/logs}")	New (proposed) — must be exported by start-hugegraph.sh
$1 (positional)	arg	GREMLIN_SERVER_CONF path	Existing
$2 (positional)	arg	REST_SERVER_CONF path	Existing
$3 (positional)	arg	OPEN_SECURITY_CHECK (true/false)	Existing
$4 (positional)	arg	USER_OPTION	Existing
$5 (positional)	arg	GC_OPTION	Existing
$6 (positional)	arg	OPEN_TELEMETRY (true/false)	Existing

start-hugegraph-store.sh (Store)

Flag	Meaning	Status
-c	Override conf directory	New (proposed)
-l	Override logs directory	New (proposed)
-d	Run as daemon (true/false)	Existing
-g	GC option (e.g. g1)	Existing
-j	Extra user/JVM option	Existing
-y	Enable/disable telemetry (true/false)	Existing

start-hugegraph-pd.sh (PD)

Flag	Meaning	Status
-c	Override conf directory	New (proposed)
-l	Override logs directory	New (proposed)
-d	Run as daemon (true/false)	Existing
-g	GC option (e.g. g1)	Existing
-j	Extra user/JVM option	Existing
-y	Enable/disable telemetry (true/false)	Existing

The path resolution then becomes:

CONF="${CONF_OVERRIDE:-$TOP/conf}"

LOGS="${LOGS_OVERRIDE:-$TOP/logs}"

Example usage

# Run a role from a read-only install, with writable runtime dirs elsewhere:
./start-hugegraph.sh \
  -c /etc/hugegraph/server/conf \
  -l /var/log/hugegraph/server

# PD and Store follow the same pattern:
./start-hugegraph-pd.sh    -c /etc/hugegraph/pd/conf    -l /var/log/hugegraph/pd
./start-hugegraph-store.sh -c /etc/hugegraph/store/conf -l /var/log/hugegraph/store

Backward compatibility: The new behavior is entirely opt-in. With no flags provided, every script behaves exactly as it does today. Existing users and the default distribution layout remain unaffected

Affected Files

• hugegraph-server/hugegraph-dist/src/assembly/static/bin/start-hugegraph.sh
• hugegraph-server/hugegraph-dist/src/assembly/static/bin/hugegraph-server.sh
• hugegraph-pd/hg-pd-dist/src/assembly/static/bin/start-hugegraph-pd.sh
• hugegraph-store/hg-store-dist/src/assembly/static/bin/start-hugegraph-store.sh
• Related util.sh USAGE/help text (where applicable)

Benefits

• Enables running HugeGraph roles (Server, Store, PD) on distributed clusters where binaries and runtime state are intentionally kept in separate locations.
• Allows multiple roles or instances to share a single install while keeping isolated, per-instance conf/logs/plugins.
• Supports read-only / immutable installation directories — a common hardening and reproducibility practice.
• Cleanly integrates with externally-generated configuration without mutating the install tree.
• Backward compatible — no behavior change for existing single-directory deployments.

[AvanishShukla20]

I would like to raise a PR if community aligns with this proposal

[bitflicker64]

@AvanishShukla20 had a few questions on this. The log4j2.xml files for PD, Store and Server all have LOG_PATH set to plain "logs", and the scripts cd into the install folder before starting Java, so would -l actually move where the logs get written, or just the folder the script checks? Store also writes its gc log to ./logs/gc.log. The pid file goes into bin/pid too, and the stop scripts read it from there, so on a read-only install that write would fail, and two instances from the same install would end up sharing it. Are you planning to cover the stop scripts and the pid file as well? Also rest-server.properties has graphs=./conf/graphs, which resolves against the install folder, so I'm guessing that would need to follow -c as well. And since PD, Store and Server already ship as separate folders with their own conf and logs, is the main case running more than one instance of the same role from one install?

[AvanishShukla20]

@bitflicker64

The log4j2.xml files for PD, Store and Server all have LOG_PATH set to plain "logs", and the scripts cd into the install folder before starting Java, so would -l actually move where the logs get written, or just the folder the script checks?

Thanks for pointing this out , Enabling log overrides through -l should move the folders where logs are written. i have addressed this and will include in the scope.

Store also writes its gc log to ./logs/gc.log

The Store script's GC logging option is a literal relative path ./logs/gc.log, embedded when JAVA_OPTIONS is unset . We can set the JAVA_OPTIONS environment variable before invoking the startup scripts — then this bypasses that block (including GC logging). So you'd need to supply your own -Xlog:gc=...:file= argument yourself gc log reference

The pid file goes into bin/pid too, and the stop scripts read it from there, so on a read-only install that write would fail, and two instances from the same install would end up sharing it. Are you planning to cover the stop scripts and the pid file as well?

Thanks for pointing this, I have addressed this and i think we should provide flags to override the $BIN/pid file as well.

Also rest-server.properties has graphs=./conf/graphs, which resolves against the install folder, so I'm guessing that would need to follow -c as well

We can set the graphs directory to either relative or absolute in rest-server.properties .
Eg:
In rest-server.properties we can use
graphs=<GRAPHS_DIR>
In gremlin-server.yaml , similarly
graphs: {
hugegraph: <GRAPH_DIR>/hugegraph.properties
}

And since PD, Store and Server already ship as separate folders with their own conf and logs, is the main case running more than one instance of the same role from one install?

Yes, that is one of the reasons and there are other ones also that can be found in the proposal