exitStatusCode:0 The return code of spark-submit is 0

[matianhe3]

spark-submit task failed, but exitStatusCode is 0 , task manager unrecognized.

[arham0254A]

Hi, I recently picked this up to investigate and opened a draft PR, but after tracing the control flow and testing locally, it appears the core Scala code is already returning a failure code correctly.

In SparkSubmitArguments.scala, the handleUnknown method calls error(). This function throws a SparkException. Because SparkSubmit.main only catches SparkUserAppException, the SparkException remains uncaught, crashing the JVM and inherently returning an OS exit code of 1.

If you are seeing an exit code of 0 after an unrecognized argument, the error is likely being swallowed by an external execution wrapper. Are you running spark-submit inside a specific orchestration tool (like Apache Airflow), a CI/CD pipeline, a Docker container, or a custom wrapper script?

Could you provide the exact execution environment and the raw command being used to reproduce the 0 exit code?

[matianhe3]

In terminal :

spark-submit \
  --master spark://spark-219:7077 \
  --deploy-mode cluster \
  --class com.pulse.spark.SqlRunner \
  --name Test_Insert_ODS \
  --driver-cores 1 \
  --driver-memory 512m \
  --executor-cores 1 \
  --executor-memory 1g \
  --total-executor-cores 1 \
  --conf spark.standalone.submit.waitAppCompletion=true \
  s3a://jars/spark-runner_2.13-1.0.0.jar \
  --sql "INSERT INTO lk.ods.test (col) VALUES ('test')"

exit_code=$?

if [ $exit_code -eq 0 ]; then
  echo "success : $exit_code"
else
  echo "failed : $exit_code"
fi
WARNING: Using incubator modules: jdk.incubator.vector
26/06/24 18:02:59 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
26/06/24 18:02:59 INFO SecurityManager: Changing view acls to: admin
26/06/24 18:02:59 INFO SecurityManager: Changing modify acls to: admin
26/06/24 18:02:59 INFO SecurityManager: Changing view acls groups to: admin
26/06/24 18:02:59 INFO SecurityManager: Changing modify acls groups to: admin
26/06/24 18:02:59 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: admin groups with view permissions: EMPTY; users with modify permissions: admin; groups with modify permissions: EMPTY; RPC SSL disabled
26/06/24 18:02:59 INFO Utils: Successfully started service 'driverClient' on port 53018.
26/06/24 18:02:59 INFO TransportClientFactory: Successfully created connection to spark-219/192.168.16.219:7077 after 8 ms (0 ms spent in bootstraps)
26/06/24 18:02:59 INFO ClientEndpoint: ... waiting before polling master for driver state
26/06/24 18:02:59 INFO ClientEndpoint: Driver successfully submitted as driver-20260624180300-0054
26/06/24 18:03:04 INFO ClientEndpoint: State of driver-20260624180300-0054 is RUNNING
26/06/24 18:03:04 INFO ClientEndpoint: Driver running on 192.168.16.219:39181 (worker-20260615171840-192.168.16.219-39181)
26/06/24 18:03:14 INFO ClientEndpoint: State of driver driver-20260624180300-0054 is FAILED, exiting spark-submit JVM.
success : 0

[arham0254A]

Thank you for providing the script and the terminal screenshots! This completely clarifies the issue.

I initially thought this was related to argument parsing, but your logs show that is not the case here.

This is definitely a valid bug that breaks external orchestration pipelines. I am going to pivot my investigation away from the argument parser and look directly into the Standalone client's shutdown sequence to ensure it properly bubbles up the failure code. I'll work on getting a PR open for this!