Implement Perl-compatible pseudofork with cloned runtimes

[fglock]

Summary

Implement Perl-compatible pseudofork by cloning a PerlRuntime and resuming the parent and child continuations on separate Java threads. Advertise d_pseudofork, not d_fork.

This should support common CPAN patterns such as:

my $pid = fork;
die "fork failed: $!" unless defined $pid;

if ($pid == 0) {
    # child
    exit 0;
}

waitpid $pid, 0;

True OS process isolation is not possible on the JVM and is not proposed.

Motivation

PerlOnJava already supports independent PerlRuntime instances, snapshot cloning, ithreads, pipes, sockets, loopback listeners, and process enumeration. fork() still returns undef, which excludes otherwise-supported process-oriented code.

Immediate ecosystem targets:

• Mojolicious: Mojo::IOLoop::Subprocess, t/mojo/subprocess.t, t/mojo/subprocess_ev.t, and controlled prefork/process-management paths.
• Catalyst::Runtime: t/live_fork.t and fork-based live/development test servers.
• Common parent/child worker patterns using pipes, loopback sockets, exit, wait, waitpid, $?, and SIGCHLD.

Mojolicious and Catalyst already work in supported single-process configurations. Pseudofork extends process-oriented modes; it is not required for ordinary PSGI request handling.

Related roadmap: docs/about/roadmap.md, Objective 6 / Fork Emulation. Related compatibility work: #1115.

Semantic contract

Phase 1 should provide:

• fork() returns a positive synthetic PID in the parent and 0 in the child.
• Failure returns undef and sets $!.
• Parent and child resume immediately after the same fork expression.
• Mutable Perl runtime state is isolated from the snapshot point, following Perl fork expectations rather than ithread CLONE semantics.
• exit and uncaught termination end only the pseudo-child, record status, notify the parent, and never terminate the host JVM.
• wait, waitpid, WNOHANG, $?, and a minimal SIGCHLD work for pseudo-children.
• Nested pseudofork is either supported with the same ownership rules or rejected deterministically with a documented error during the first phase.
• $Config{d_fork} remains false/undefined; $Config{d_pseudofork} is defined only when the implementation is enabled.
• Unsupported operations fail explicitly instead of silently sharing unsafe process-global state.

The implementation must behave consistently on JVM and interpreter execution backends.

Architecture

1. Make fork a resumable control-flow boundary

Cloning PerlRuntime is insufficient because Java cannot copy the current Java stack. Treat fork as a compiler/runtime control-flow operation with a resumable continuation:

• Capture the Perl instruction position, operand/value stack, lexical pads, call frames, dynamic localization state, warning/context state, regex state, and active exception/unwind metadata needed after the call.
• Resume the parent continuation with the synthetic child PID.
• Resume a cloned child continuation with 0 on a virtual thread by default, respecting the configured carrier policy.
• For the interpreter, snapshot the bytecode frame and continue at the instruction following fork.
• For the JVM backend, initially lower fork-containing scopes to the resumable bytecode/interpreter path or an explicit state-machine continuation. Do not attempt to clone a Java stack. A later optimization can add native JVM continuation lowering without changing semantics.

Compiler-owned focused tests must cover fork in expressions, conditionals, loops, nested calls, eval, localized state, exception paths, and both scalar/list-adjacent contexts.

2. Clone runtime state

Build on the existing multiplicity/ithread snapshot infrastructure, but define a distinct fork snapshot policy:

• Copy mutable globals, pads, symbol tables, module/runtime state, %ENV, signal handlers, $!, $?, random state, and process metadata.
• Preserve immutable compiled code and safe read-only resources by sharing.
• Do not invoke ithread CLONE hooks for pseudofork unless Perl pseudofork compatibility requires it.
• Give the child a synthetic PID, parent PID, lifecycle state, exit status, and cancellation token managed by a runtime-local process table.
• Keep process-global JVM state out of the child model. Working directory and environment changes must be runtime-local or explicitly unsupported.

Introduce a PseudoProcessManager (name illustrative) owned by the parent runtime/session to allocate PIDs, track parent/child relationships, deliver status, and reap children.

3. Define external-resource inheritance

Perl variables are copied, but external resources need explicit inheritance policies:

• Pipes and loopback sockets: inherit duplicate Perl handle wrappers backed by shared, reference-counted endpoints; closing one runtime's wrapper must not prematurely close the sibling's endpoint.
• Listening sockets: allow the controlled parent/child patterns required by live server tests, with documented shared-accept behavior.
• Regular files: duplicate logical handle state where safe; document shared file-position behavior if the underlying channel is shared.
• Database handles, Java objects, native handles, locks, thread primitives, and non-duplicable channels: reject, detach, or mark unsupported through typed snapshot policies rather than shallow-copying accidentally.
• Event-loop watchers, timers, and selectors: rebuild them in the child against inherited handles; never share a mutable selector/reactor instance across runtimes.

Resource ownership must integrate with the existing RuntimeIO/scope-exit ownership work so parent exit, child exit, and handle transfer are deterministic.

4. Process operations and signals

Implement pseudoprocess-aware paths before delegating to OS process APIs:

• getpid/$$ and getppid
• exit, child exception termination, and status encoding
• wait and waitpid, including WNOHANG
• $? updates in the waiting runtime
• kill 0, cooperative TERM/INT, and pseudoprocess lookup
• SIGCHLD delivery at safe Perl execution boundaries

Java cannot safely force-kill arbitrary threads. KILL should be documented as cooperative cancellation unless the child is at a runtime safepoint; uninterruptible Java/native work is outside the guarantee.

5. Isolation and safety

Pseudofork runs inside one JVM and must not claim OS security or crash isolation:

• No privilege, namespace, UID/GID, resource-limit, or memory-isolation guarantee.
• exec may replace only the pseudo-child runtime when implemented internally; launching an external process is a separate operation.
• Avoid deadlocks when a snapshot is taken while another runtime/thread owns a lock. Initially permit fork only from a single active Perl execution thread, or fail with a clear diagnostic.
• Bound unreaped-child records and ensure abandoned children are cancelled/reaped during parent runtime shutdown.

Implementation phases

Phase 1: compiler contract and lifecycle

• Add permanent system-Perl-validated compiler regressions before implementation.
• Add the resumable fork operation on both backends.
• Clone pure Perl state and run a child virtual thread.
• Implement synthetic PID, exit, waitpid, $?, and cleanup.
• Support simple no-I/O parent/child programs.

Phase 2: pipes, handles, and signals

• Add inherited pipe and loopback-socket ownership.
• Rebuild event-loop watchers in the child.
• Implement WNOHANG, minimal SIGCHLD, and cooperative kill.
• Add stress tests for close order, parent-first/child-first exit, failures, and orphan cleanup.

Phase 3: ecosystem acceptance

• Pass unchanged Mojolicious subprocess tests on Poll and EV where available.
• Pass Catalyst::Runtime t/live_fork.t unchanged.
• Evaluate Mojolicious prefork tests; document unsupported OS-isolation assumptions separately.
• Run Mojolicious, Catalyst::Runtime, and DBIx::Class acceptance suites to ensure the new snapshot/resource behavior does not regress single-process operation.

Phase 4: hardening

• Nested pseudofork policy.
• Additional signal/status parity.
• Performance and leak testing under repeated child creation.
• Windows CI coverage and platform-specific handle behavior.

Required permanent tests

Validate Perl-level tests with system Perl first and retain unfixed PerlOnJava evidence.

• Parent receives PID; child receives 0; both continue after one fork point.
• Global, lexical, localized, closure-captured, tied, blessed, weak, and cyclic values isolate correctly.
• Parent and child exceptions and exit N produce correct wait status.
• wait, targeted waitpid, WNOHANG, no-children behavior, and $?.
• Parent/child pipe exchange and EOF after each close ordering.
• Loopback listener inherited across the fork boundary without leaked sockets.
• Child cannot terminate the host JVM or unrelated runtimes.
• No zombie pseudo-children, orphan Java threads, retained runtime snapshots, or leaked handles after stress loops.
• JVM/interpreter parity for every compiler/runtime behavior.

Acceptance criteria

• make passes.
• New project-owned fork regressions pass on JVM and interpreter.
• Existing ithread, multiplicity, socket, weak-reference, destruction, and process tests remain green.
• Unchanged Mojolicious subprocess tests pass, or any remaining failures are proven to require true OS isolation and documented.
• Unchanged Catalyst::Runtime t/live_fork.t passes.
• $Config{d_pseudofork} is advertised only when all baseline semantics are enabled; $Config{d_fork} remains false.
• Documentation clearly distinguishes pseudofork from true OS fork and lists unsupported isolation/resource cases.

Non-goals

• True copy-on-write address-space cloning.
• OS-level security or fault isolation.
• Exact Unix signal delivery during arbitrary Java/native calls.
• Silently emulating resources that cannot be safely inherited.
• Patching Mojolicious or Catalyst to bypass Perl semantics.

[fglock]

The overall direction is viable—Windows Perl uses the same broad model of cloning an interpreter into another thread—but this issue is not yet implementation-ready.

Verdict: cloning PerlRuntime plus starting a virtual thread is necessary, but does not by itself implement fork. The hard requirement is cloning the complete continuation after the fork expression.

Existing foundations

PerlOnJava already provides useful infrastructure:

• RuntimeGraphCloner preserves aliases, cycles, closures, weak references, and many handle types.
• PerlThreadControlBlock provides virtual/platform thread lifecycle machinery.
• SuspendedInterpreterFrame already stores an interpreter PC, registers, eval stacks, localization state, and regex state for await.
• WaitpidOperator already multiplexes native and Java child processes.

These components should be generalized, not replaced.

Blocking correctness problems

1. Only the current interpreter frame is resumable. Calls still use synchronous RuntimeCode.apply() Java recursion. If fork occurs in a nested subroutine, resuming only that frame loses every caller continuation. Merely changing the existing FORK opcode to clone the runtime would work only for trivial top-level cases.

2. Lowering only the fork-containing scope is insufficient. A JVM-compiled caller can invoke an interpreted subroutine containing fork. The child must eventually return through the compiled caller, whose Java stack cannot be cloned. V1 therefore needs either:

   • a whole-runtime fork-capable interpreter mode; or
   • resumable state-machine/CPS support at every possible JVM call boundary.

   The first option is substantially safer and cheaper for an initial implementation.

3. The existing snapshot is specifically an ithread snapshot. It runs CLONE_SKIP and CLONE, preserves threads::shared storage by identity, and deliberately starts execution, lifecycle, signal, native, and I/O registry state fresh. Fork needs a separate ForkSnapshotPolicy; reusing snapshotCloneForThread unchanged would produce ithread semantics, not fork semantics.

4. Unknown Java resources currently have a shallow-copy fallback. Fork snapshots must reject, detach, or explicitly adapt unclassified Java/native resources. Accidental sharing is unsafe.

5. Process identity is not centralized. PerlRuntime.pid, POSIX::getpid, getppid, kill, File::Temp, syslog, and other paths still consult ProcessHandle or native APIs directly. Synthetic PID/PPID lookup needs one runtime-local process identity service.

6. POSIX::_exit currently calls Runtime.halt. This would terminate the entire JVM from a pseudo-child. Both exit and _exit need pseudo-child-specific termination paths, with _exit skipping END/global destruction while still releasing manager-owned Java resources.

Mojolicious acceptance contradiction

Mojolicious 9.49 explicitly refuses to start a subprocess when $Config{d_pseudofork} is true:

return $self->$parent('Subprocesses do not support fork emulation')
  if $Config{d_pseudofork};

See Mojo::IOLoop::Subprocess. Therefore the current acceptance criteria cannot simultaneously:

• advertise d_pseudofork;
• avoid modifying Mojolicious; and
• pass its unchanged subprocess tests.

This is a policy gate in Mojo, not proof that PerlOnJava’s emulation cannot support its pipe/wait behavior. Once the implementation is proven safe, Mojo would need a finer-grained capability check or an explicit PerlOnJava exception. Hiding d_pseudofork would be misleading and should not be shipped.

Catalyst’s t/live_fork.t is a realistic early ecosystem target because its child exits directly and the parent only needs PID, exit, and waitpid.

Efficient implementation sequence

1. Add system-Perl-validated regressions for top-level fork, fork inside nested calls, child return through callers, eval, localization, exit, _exit, and waitpid. Keep d_pseudofork disabled.
2. Implement a ForkSuspension that unwinds and records every active interpreted frame. At the outer execution boundary, clone the runtime and captured continuation once, insert the synthetic PID/zero results, and resume parent and child through a continuation trampoline.
3. Initially expose pseudofork through a whole-runtime fork-capable interpreter mode. Do not claim JVM parity by lowering only the innermost scope.
4. Add ForkSnapshotPolicy beside the ithread policy: no ithread CLONE hooks, explicit mutable-state isolation, and strict resource classification.
5. Add a dedicated pseudo-process table, reusing the existing Java thread execution policy where useful. Consult it before native/Java-process paths in waitpid, kill, PID lookup, and shutdown.
6. Deliver pure-state lifecycle behavior first. Then enable inherited pipes/sockets using the existing independently closable handle leases and add signals/event-loop acceptance.
7. Advertise d_pseudofork only after the baseline lifecycle and continuation tests pass. Treat Mojolicious enablement as a separately reviewed compatibility decision.

The implementation should not begin with the current SystemOperator.fork() stub. The first technical spike should prove that a nested interpreted call chain can be unwound, cloned, and resumed twice without losing dynamic state or running cleanup prematurely.

Reference behavior: Perl’s perlfork documentation.