HistoryServer
a inner join with one empty side and one large side, the plan would finish and the large side is still running.
- Use SparkListenerSQLExecutionEnd to track final SQL status in UI
https://issues.apache.org/jira/browse/SPARK-40834
Spark HistoryServer 3.4 以后修复了这个问题
JsonProtocol
JsonProtocol 类为 HistoryServer 提供解析 eventLog 的基本工具类库。
other 选项为其提供了 基本反射功能,可以解决各类 event 事件
FsHistoryProvider
override def start(): Unit = {initThread = initialize()}
private[history] def initialize(): Thread = {if (!isFsInSafeMode()) {startPolling()null} else {startSafeModeCheckThread(None)}}