[BUG] Spark application status is displayed as failed when driver exit with 143 code

[houyuting]

What happened?

My scenario is spark on kubernetes, The driver business logic has already completed, which then triggers the driver Pod to exit. the pod exit times out(we set deletionGracePeriodSeconds: 30 ), so that driver pod exit code is 143. and the Spark Operator check non-zero exit as Failed, so spark application is marked as Failed.

However, in normal, exit code of 143 for a pod means that the container/process was normally terminated due to receiving a SIGTERM, rather than crashing because of an application error. Therefore, can we handle the driver Pod exit code 143 as a DriverStateCompleted status ?

sparkapplication.go

[houyuting]

@ChenYi015 Hi, excuse me, Could you please take a look at this when you have time? thanks

[ChenYi015]

@houyuting Are you trying to delete the driver pod manually?

[houyuting]

@houyuting Are you trying to delete the driver pod manually?

No, no any action manually

[ChenYi015]

@houyuting Are you trying to delete the driver pod manually?

No, no any action manually

Then why would the driver pocess receive a SIGTERM signal?

[houyuting]

@houyuting Are you trying to delete the driver pod manually?

No, no any action manually

Then why would the driver pocess receive a SIGTERM signal?

because driver pod did not exit within 30 seconds, so k8s delete pod. this maybe shuttdown hook to cleanup slowly, Additionally, other situations include high cpu load, network io and so on.