这个脚本是Apache Doris的编译脚本，用于管理整个项目的编译流程。以下是对每一行代码的详细分析：
脚本基础设置
#!/usr/bin/env bash
# Licensed to the Apache Software Foundation (ASF) under one
# ... 省略许可证信息 ...
• 作用: 声明脚本使用Bash解释器，并添加Apache 2.0许可证信息。
环境初始化
set -eo pipefail
• 作用: 设置脚本执行选项：
• -e: 任何命令失败（返回值非零）立即退出。
• -o pipefail: 管道中任意命令失败，整个管道视为失败。
路径定义
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
export DORIS_HOME="${ROOT}"
. "${DORIS_HOME}/env.sh"
• 作用:
ROOT获取脚本所在目录的绝对路径。DORIS_HOME设置为ROOT，即Doris源码根目录。- 加载
env.sh环境配置（可能定义编译工具路径、环境变量等）。
帮助函数 usage()
usage() {echo "
Usage: $0 <options>... 详细选项说明 ..."exit 1
}
• 作用: 打印脚本使用说明，包括支持的编译选项、环境变量等。
清理函数
clean_gensrc() { ... }
clean_be() { ... }
clean_fe() { ... }
• 作用:
• clean_gensrc: 清理生成的代码（如Thrift生成的源码）。
• clean_be: 删除后端构建目录和输出目录。
• clean_fe: 使用Maven清理前端代码。
公共文件复制函数
function copy_common_files() { ... }
• 作用: 将许可证文件（NOTICE.txt、LICENSE等）复制到输出目录。
参数解析
if ! OPTS="$(getopt ... )"; thenusage
fi
eval set -- "${OPTS}"
• 作用: 使用 getopt 解析命令行参数，支持长选项（如 --fe, --be）和短选项（如 -j）。
第三方库编译（注意：这一步坑很多，不是少yum安装包，就是包的hash值不对，还有主机环境奇怪的问题，尤其是离线构建问题巨多）
if [[ ! -f "${DORIS_THIRDPARTY}/installed/lib/libbacktrace.a" ]]; then"${DORIS_THIRDPARTY}/build-thirdparty.sh" ...
fi
• 作用: 检查第三方库是否已编译，若未编译则调用 build-thirdparty.sh 脚本编译。
子模块更新
update_submodule() { ... }
• 作用: 更新Git子模块，若失败则从指定URL下载并解压代码（应对网络问题）。
编译选项设置
WITH_MYSQL='OFF'
GLIBC_COMPATIBILITY='ON'
USE_AVX2='ON'
...
• 作用: 设置默认编译选项（如是否启用MySQL支持、AVX2指令集等）。
后端编译逻辑
if [[ "${BUILD_BE}" -eq 1 ]]; thenupdate_submodule "be/src/apache-orc" ...CMAKE_BUILD_DIR="${DORIS_HOME}/be/build_${CMAKE_BUILD_TYPE}""${CMAKE_CMD}" -G "${GENERATOR}" ..."${BUILD_SYSTEM}" -j "${PARALLEL}""${BUILD_SYSTEM}" install
fi
• 作用:
- 更新子模块（如ORC、Clucene）。
- 创建构建目录，调用CMake生成构建配置。
- 使用Make/Ninja编译并安装后端。
前端UI构建
function build_ui() {npm install ...npm run buildcp -r "${ui_dist}"/* "${FE_STATIC_DIR}"
}
• 作用: 使用npm安装依赖并构建前端UI，将生成的静态文件复制到FE资源目录。
前端Java模块构建
if [[ "${FE_MODULES}" != '' ]]; thencd "${DORIS_HOME}/fe""${MVN_CMD}" package ...
fi
• 作用: 使用Maven编译指定的FE模块（如FE核心、Spark DPP等）。
输出目录准备
DORIS_OUTPUT=${DORIS_OUTPUT:="${DORIS_HOME}/output/"}
mkdir -p "${DORIS_OUTPUT}"
• 作用: 创建输出目录，用于存放编译后的二进制文件、配置文件等。
组件复制到输出目录
if [[ "${BUILD_FE}" -eq 1 ]]; thencp -r -p "${DORIS_HOME}/bin"/*_fe.sh "${DORIS_OUTPUT}/fe/bin"/...
fi
• 作用: 将FE、BE、Broker等组件的构建结果复制到输出目录，包括：
• 启动脚本（如start_fe.sh）
• 配置文件（如fe.conf）
• JAR包和依赖库
• 静态资源（Web UI）
Broker编译
if [[ "${BUILD_BROKER}" -eq 1 ]]; thencd "${DORIS_HOME}/fs_brokers/apache_hdfs_broker"./build.shcp -r -p ... "${DORIS_OUTPUT}/apache_hdfs_broker"/
fi
• 作用: 调用Broker的独立编译脚本，并将结果复制到输出目录。
后处理
echo "Successfully build Doris"
if [[ -n "${DORIS_POST_BUILD_HOOK}" ]]; theneval "${DORIS_POST_BUILD_HOOK}"
fi
• 作用: 打印成功信息，执行用户自定义的后处理命令（通过环境变量 DORIS_POST_BUILD_HOOK）。
关键逻辑总结
- 参数解析: 支持灵活选择编译组件（FE、BE、Broker等）。
- 依赖管理: 自动编译第三方库和子模块。
- 构建隔离: 支持清理操作（
--clean），确保干净的构建环境。 - 多平台支持: 处理MacOS和Linux的差异（如JVM路径、库文件兼容性）。
- 性能优化: 并行编译（
-j参数）、CCache加速。 - 输出管理: 统一输出目录结构，便于分发和部署。
使用示例
• 全量编译: ./build.sh
• 仅编译BE: ./build.sh --be
• 清理并重新编译: ./build.sh --clean --fe --be
通过分析此脚本，可以全面理解Apache Doris的编译流程和关键配置点。
脚本内容
#!/usr/bin/env bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.##############################################################
# This script is used to compile Apache Doris
# Usage:
# sh build.sh --help
#
# You need to make sure all thirdparty libraries have been
# compiled and installed correctly.
############################################################### 命令行失败退出，管道中命令失败退出
set -eo pipefail
# 获取doris home目录
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
# 设置doris home目录到环境变量
export DORIS_HOME="${ROOT}"
# 检查和初始化 doris环境
. "${DORIS_HOME}/env.sh"# 打印脚本使用说明，包括支持的编译选项、环境变量等。
usage() {echo "
Usage: $0 <options>Optional options:[no option] build all components--fe build Frontend and Spark DPP application. Default ON.--be build Backend. Default ON.--meta-tool build Backend meta tool. Default OFF.--index-tool build Backend inverted index tool. Default OFF.--broker build Broker. Default ON.--spark-dpp build Spark DPP application. Default ON.--hive-udf build Hive UDF library for Spark Load. Default ON.--be-java-extensions build Backend java extensions. Default ON.--be-extension-ignore build be-java-extensions package, choose which modules to ignore. Multiple modules separated by commas.--clean clean and build target--output specify the output directory-j build Backend parallelEnvironment variables:USE_AVX2 If the CPU does not support AVX2 instruction set, please set USE_AVX2=0. Default is ON.STRIP_DEBUG_INFO If set STRIP_DEBUG_INFO=ON, the debug information in the compiled binaries will be stored separately in the 'be/lib/debug_info' directory. Default is OFF.DISABLE_BE_JAVA_EXTENSIONS If set DISABLE_BE_JAVA_EXTENSIONS=ON, we will do not build binary with java-udf,hudi-scanner,jdbc-scanner and so on Default is OFF.DISABLE_JAVA_CHECK_STYLE If set DISABLE_JAVA_CHECK_STYLE=ON, it will skip style check of java code in FE.Eg.$0 build all$0 --be build Backend$0 --meta-tool build Backend meta tool$0 --index-tool build Backend inverted index tool$0 --fe --clean clean and build Frontend and Spark Dpp application$0 --fe --be --clean clean and build Frontend, Spark Dpp application and Backend$0 --spark-dpp build Spark DPP application alone$0 --broker build Broker$0 --be --fe build Backend, Frontend, Spark Dpp application and Java UDF library$0 --be --coverage build Backend with coverage enabled$0 --be --output PATH build Backend, the result will be output to PATH(relative paths are available)$0 --be-extension-ignore avro-scanner build be-java-extensions, choose which modules to ignore. Multiple modules separated by commas, like --be-extension-ignore avro-scanner,hudi-scannerUSE_AVX2=0 $0 --be build Backend and not using AVX2 instruction.USE_AVX2=0 STRIP_DEBUG_INFO=ON $0 build all and not using AVX2 instruction, and strip the debug info for Backend"exit 1
}
# 清理FE生成的代码（如Thrift生成的源码）。
clean_gensrc() {pushd "${DORIS_HOME}/gensrc"make cleanrm -rf "${DORIS_HOME}/gensrc/build"rm -rf "${DORIS_HOME}/fe/fe-common/target"rm -rf "${DORIS_HOME}/fe/fe-core/target"popd
}
# 删除BE构建目录和输出目录。
clean_be() {# 进入DORIS_HOME目录pushd "${DORIS_HOME}"# "build.sh --clean" just cleans and exits, however CMAKE_BUILD_DIR is set# while building be.CMAKE_BUILD_TYPE="${BUILD_TYPE:-Release}"CMAKE_BUILD_DIR="${DORIS_HOME}/be/build_${CMAKE_BUILD_TYPE}"rm -rf "${CMAKE_BUILD_DIR}"rm -rf "${DORIS_HOME}/be/output"# 返回执行pushd ${DORIS_HOME}之前的目录popd
}
# 使用Maven清理FE代码
clean_fe() {pushd "${DORIS_HOME}/fe""${MVN_CMD}" cleanpopd
}# 将许可证文件（NOTICE.txt、LICENSE等）复制到输出目录。
function copy_common_files() {cp -r -p "${DORIS_HOME}/NOTICE.txt" "$1/"cp -r -p "${DORIS_HOME}/dist/LICENSE-dist.txt" "$1/"cp -r -p "${DORIS_HOME}/dist/licenses" "$1/"
}
# 使用 getopt 解析命令行参数，支持长选项（如 --fe, --be）和短选项（如 -j）。
if ! OPTS="$(getopt \-n "$0" \-o '' \-l 'fe' \-l 'be' \-l 'broker' \-l 'meta-tool' \-l 'index-tool' \-l 'spark-dpp' \-l 'hive-udf' \-l 'be-java-extensions' \-l 'be-extension-ignore:' \-l 'clean' \-l 'coverage' \-l 'help' \-l 'output:' \-o 'hj:' \-- "$@")"; thenusage
fieval set -- "${OPTS}"# CPU核数除以4加1
PARALLEL="$(($(nproc) / 4 + 1))"
BUILD_FE=0
BUILD_BE=0
BUILD_BROKER=0
BUILD_META_TOOL='OFF'
BUILD_INDEX_TOOL='OFF'
BUILD_SPARK_DPP=0
BUILD_BE_JAVA_EXTENSIONS=0
BUILD_HIVE_UDF=0
CLEAN=0
HELP=0
PARAMETER_COUNT="$#"
PARAMETER_FLAG=0
DENABLE_CLANG_COVERAGE='OFF'
BUILD_UI=1
if [[ "$#" == 1 ]]; then# defaultBUILD_FE=1BUILD_BE=1BUILD_BROKER=1BUILD_META_TOOL='OFF'BUILD_INDEX_TOOL='OFF'BUILD_SPARK_DPP=1BUILD_HIVE_UDF=1BUILD_BE_JAVA_EXTENSIONS=1CLEAN=0
elsewhile true; docase "$1" in--fe)BUILD_FE=1BUILD_SPARK_DPP=1BUILD_HIVE_UDF=1BUILD_BE_JAVA_EXTENSIONS=1shift;;--be)BUILD_BE=1BUILD_BE_JAVA_EXTENSIONS=1shift;;--broker)BUILD_BROKER=1shift;;--meta-tool)BUILD_META_TOOL='ON'shift;;--index-tool)BUILD_INDEX_TOOL='ON'shift;;--spark-dpp)BUILD_SPARK_DPP=1shift;;--hive-udf)BUILD_HIVE_UDF=1shift;;--be-java-extensions)BUILD_BE_JAVA_EXTENSIONS=1shift;;--clean)CLEAN=1shift;;--coverage)DENABLE_CLANG_COVERAGE='ON'shift;;-h)HELP=1shift;;--help)HELP=1shift;;-j)PARALLEL="$2"PARAMETER_FLAG=1shift 2;;--output)DORIS_OUTPUT="$2"shift 2;;--be-extension-ignore)BE_EXTENSION_IGNORE="$2"shift 2;;--)shiftbreak;;*)echo "Internal error"exit 1;;esacdone#only ./build.sh -j xx then build allif [[ "${PARAMETER_COUNT}" -eq 3 ]] && [[ "${PARAMETER_FLAG}" -eq 1 ]]; thenBUILD_FE=1BUILD_BE=1BUILD_BROKER=1BUILD_META_TOOL='ON'BUILD_INDEX_TOOL='ON'BUILD_SPARK_DPP=1BUILD_HIVE_UDF=1BUILD_BE_JAVA_EXTENSIONS=1CLEAN=0fi
fiif [[ "${HELP}" -eq 1 ]]; thenusage
fi
# 检查第三方库是否已编译，若未编译则调用 build-thirdparty.sh 脚本编译。
if [[ ! -f "${DORIS_THIRDPARTY}/installed/lib/libbacktrace.a" ]]; thenecho "Thirdparty libraries need to be build ..."# 需要删除所有已安装的PKGS，因为一些库，如lz4，如果它的库已经存在，将抛出错误rm -rf "${DORIS_THIRDPARTY}/installed"if [[ "${CLEAN}" -eq 0 ]]; then"${DORIS_THIRDPARTY}/build-thirdparty.sh" -j "${PARALLEL}"else"${DORIS_THIRDPARTY}/build-thirdparty.sh" -j "${PARALLEL}" --cleanfi
fi# 更新Git子模块，若失败则从指定URL下载并解压代码（应对网络问题）。
update_submodule() {local submodule_path=$1local submodule_name=$2local archive_url=$3set +ecd "${DORIS_HOME}"echo "Update ${submodule_name} submodule ..."git submodule update --init --recursive "${submodule_path}"exit_code=$?if [[ "${exit_code}" -eq 0 ]]; thencd "${submodule_path}"submodule_commit_id=$(git rev-parse HEAD)cd -expect_submodule_commit_id=$(git ls-tree HEAD "${submodule_path}" | awk '{print $3}')echo "Current commit ID of ${submodule_name} submodule: ${submodule_commit_id}, expected is ${expect_submodule_commit_id}"fiset -eif [[ "${exit_code}" -ne 0 ]]; thenset +e# try to get submodule's current commitsubmodule_commit=$(git ls-tree HEAD "${submodule_path}" | awk '{print $3}')exit_code=$?if [[ "${exit_code}" = "0" ]]; thencommit_specific_url=$(echo "${archive_url}" | sed "s/refs\/heads/${submodule_commit}/")elsecommit_specific_url="${archive_url}"fiset -eecho "Update ${submodule_name} submodule failed, start to download and extract ${commit_specific_url}"mkdir -p "${DORIS_HOME}/${submodule_path}"curl -L "${commit_specific_url}" | tar -xz -C "${DORIS_HOME}/${submodule_path}" --strip-components=1fi
}# 根据传入参数--clean，判断是否清理
if [[ "${CLEAN}" -eq 1 && "${BUILD_BE}" -eq 0 && "${BUILD_FE}" -eq 0 && "${BUILD_SPARK_DPP}" -eq 0 ]]; thenclean_gensrcclean_beclean_feexit 0
fi# 设置默认编译选项（如是否启用MySQL支持、AVX2指令集等）。
if [[ -z "${WITH_MYSQL}" ]]; thenWITH_MYSQL='OFF'
fi
# GLIBC兼容性
if [[ -z "${GLIBC_COMPATIBILITY}" ]]; thenif [[ "$(uname -s)" != 'Darwin' ]]; thenGLIBC_COMPATIBILITY='ON'elseGLIBC_COMPATIBILITY='OFF'fi
fi
# 是否支持AVX2
if [[ -z "${USE_AVX2}" ]]; thenUSE_AVX2='ON'
fi
if [[ -z "${USE_LIBCPP}" ]]; thenif [[ "$(uname -s)" != 'Darwin' ]]; thenUSE_LIBCPP='OFF'elseUSE_LIBCPP='ON'fi
fi
if [[ -z "${STRIP_DEBUG_INFO}" ]]; thenSTRIP_DEBUG_INFO='OFF'
fi
if [[ -z "${USE_MEM_TRACKER}" ]]; thenif [[ "$(uname -s)" != 'Darwin' ]]; thenUSE_MEM_TRACKER='ON'elseUSE_MEM_TRACKER='OFF'fi
fi
# 是否启用内存分配器，优化内存
if [[ -z "${USE_JEMALLOC}" ]]; thenUSE_JEMALLOC='ON'
fi
if [[ -z "${USE_BTHREAD_SCANNER}" ]]; thenUSE_BTHREAD_SCANNER='OFF'
fi
if [[ -z "${ENABLE_STACKTRACE}" ]]; thenENABLE_STACKTRACE='ON'
fiif [[ -z "${USE_DWARF}" ]]; thenUSE_DWARF='OFF'
fiif [[ -z "${USE_UNWIND}" ]]; thenif [[ "$(uname -s)" != 'Darwin' ]]; thenUSE_UNWIND='ON'elseUSE_UNWIND='OFF'fi
fiif [[ -z "${DISPLAY_BUILD_TIME}" ]]; thenDISPLAY_BUILD_TIME='OFF'
fiif [[ -z "${OUTPUT_BE_BINARY}" ]]; thenOUTPUT_BE_BINARY=${BUILD_BE}
fiif [[ -n "${DISABLE_BE_JAVA_EXTENSIONS}" ]]; thenif [[ "${DISABLE_BE_JAVA_EXTENSIONS}" == "ON" ]]; thenBUILD_BE_JAVA_EXTENSIONS=0elseBUILD_BE_JAVA_EXTENSIONS=1fi
fiif [[ -n "${DISABLE_BUILD_UI}" ]]; thenif [[ "${DISABLE_BUILD_UI}" == "ON" ]]; thenBUILD_UI=0fi
fiif [[ -n "${DISABLE_BUILD_SPARK_DPP}" ]]; thenif [[ "${DISABLE_BUILD_SPARK_DPP}" == "ON" ]]; thenBUILD_SPARK_DPP=0fi
fiif [[ -n "${DISABLE_BUILD_HIVE_UDF}" ]]; thenif [[ "${DISABLE_BUILD_HIVE_UDF}" == "ON" ]]; thenBUILD_HIVE_UDF=0fi
fiif [[ -z "${DISABLE_JAVA_CHECK_STYLE}" ]]; thenDISABLE_JAVA_CHECK_STYLE='OFF'
fiif [[ -z "${RECORD_COMPILER_SWITCHES}" ]]; thenRECORD_COMPILER_SWITCHES='OFF'
fiif [[ "${BUILD_BE_JAVA_EXTENSIONS}" -eq 1 && "$(uname -s)" == 'Darwin' ]]; thenif [[ -z "${JAVA_HOME}" ]]; thenCAUSE='the environment variable JAVA_HOME is not set'elseLIBJVM="$(find -L "${JAVA_HOME}/" -name 'libjvm.dylib')"if [[ -z "${LIBJVM}" ]]; thenCAUSE="the library libjvm.dylib is missing"elif [[ "$(file "${LIBJVM}" | awk '{print $NF}')" != "$(uname -m)" ]]; thenCAUSE='the architecture which the library libjvm.dylib is built for does not match'fifiif [[ -n "${CAUSE}" ]]; thenecho -e "\033[33;1mWARNNING: \033[37;1mSkip building with BE Java extensions due to ${CAUSE}.\033[0m"BUILD_BE_JAVA_EXTENSIONS=0BUILD_BE_JAVA_EXTENSIONS_FALSE_IN_CONF=1fi
fiecho "Get params:BUILD_FE -- ${BUILD_FE}BUILD_BE -- ${BUILD_BE}BUILD_BROKER -- ${BUILD_BROKER}BUILD_META_TOOL -- ${BUILD_META_TOOL}BUILD_INDEX_TOOL -- ${BUILD_INDEX_TOOL}BUILD_SPARK_DPP -- ${BUILD_SPARK_DPP}BUILD_BE_JAVA_EXTENSIONS -- ${BUILD_BE_JAVA_EXTENSIONS}BUILD_HIVE_UDF -- ${BUILD_HIVE_UDF}PARALLEL -- ${PARALLEL}CLEAN -- ${CLEAN}WITH_MYSQL -- ${WITH_MYSQL}GLIBC_COMPATIBILITY -- ${GLIBC_COMPATIBILITY}USE_AVX2 -- ${USE_AVX2}USE_LIBCPP -- ${USE_LIBCPP}USE_DWARF -- ${USE_DWARF}USE_UNWIND -- ${USE_UNWIND}STRIP_DEBUG_INFO -- ${STRIP_DEBUG_INFO}USE_MEM_TRACKER -- ${USE_MEM_TRACKER}USE_JEMALLOC -- ${USE_JEMALLOC}USE_BTHREAD_SCANNER -- ${USE_BTHREAD_SCANNER}ENABLE_STACKTRACE -- ${ENABLE_STACKTRACE}DENABLE_CLANG_COVERAGE -- ${DENABLE_CLANG_COVERAGE}DISPLAY_BUILD_TIME -- ${DISPLAY_BUILD_TIME}ENABLE_PCH -- ${ENABLE_PCH}
"# 清理并构建生成的代码
if [[ "${CLEAN}" -eq 1 ]]; thenclean_gensrc
fi
"${DORIS_HOME}"/generated-source.sh noclean# Assesmble FE modules
FE_MODULES=''
BUILD_DOCS='OFF'
modules=("")
if [[ "${BUILD_FE}" -eq 1 ]]; thenmodules+=("fe-common")modules+=("fe-core")BUILD_DOCS='ON'
fi
if [[ "${BUILD_SPARK_DPP}" -eq 1 ]]; thenmodules+=("fe-common")modules+=("spark-dpp")
fi
if [[ "${BUILD_HIVE_UDF}" -eq 1 ]]; thenmodules+=("fe-common")modules+=("hive-udf")
fi
if [[ "${BUILD_BE_JAVA_EXTENSIONS}" -eq 1 ]]; thenmodules+=("fe-common")modules+=("be-java-extensions/hudi-scanner")modules+=("be-java-extensions/java-common")modules+=("be-java-extensions/java-udf")modules+=("be-java-extensions/jdbc-scanner")modules+=("be-java-extensions/paimon-scanner")modules+=("be-java-extensions/max-compute-scanner")modules+=("be-java-extensions/avro-scanner")modules+=("be-java-extensions/preload-extensions")# If the BE_EXTENSION_IGNORE variable is not empty, remove the modules that need to be ignored from FE_MODULESif [[ -n "${BE_EXTENSION_IGNORE}" ]]; thenIFS=',' read -r -a ignore_modules <<<"${BE_EXTENSION_IGNORE}"for module in "${ignore_modules[@]}"; domodules=("${modules[@]/be-java-extensions\/${module}/}")donefi
fi
FE_MODULES="$(IFS=','echo "${modules[*]}"
)"# 清理和编译Backend
if [[ "${BUILD_BE}" -eq 1 ]]; then# 更新子模块（如ORC、Clucene）。update_submodule "be/src/apache-orc" "apache-orc" "https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz"update_submodule "be/src/clucene" "clucene" "https://github.com/apache/doris-thirdparty/archive/refs/heads/clucene.tar.gz"if [[ -e "${DORIS_HOME}/gensrc/build/gen_cpp/version.h" ]]; thenrm -f "${DORIS_HOME}/gensrc/build/gen_cpp/version.h"fiCMAKE_BUILD_TYPE="${BUILD_TYPE:-Release}"echo "Build Backend: ${CMAKE_BUILD_TYPE}"CMAKE_BUILD_DIR="${DORIS_HOME}/be/build_${CMAKE_BUILD_TYPE}"if [[ "${CLEAN}" -eq 1 ]]; thenclean_befiMAKE_PROGRAM="$(command -v "${BUILD_SYSTEM}")"if [[ -z "${BUILD_FS_BENCHMARK}" ]]; thenBUILD_FS_BENCHMARK=OFFfiecho "-- Make program: ${MAKE_PROGRAM}"echo "-- Use ccache: ${CMAKE_USE_CCACHE}"echo "-- Extra cxx flags: ${EXTRA_CXX_FLAGS:-}"echo "-- Build fs benchmark tool: ${BUILD_FS_BENCHMARK}"# 创建构建目录，调用CMake生成构建配置。mkdir -p "${CMAKE_BUILD_DIR}"cd "${CMAKE_BUILD_DIR}"# 使用Make/Ninja编译并安装后端。"${CMAKE_CMD}" -G "${GENERATOR}" \-DCMAKE_MAKE_PROGRAM="${MAKE_PROGRAM}" \-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \-DCMAKE_BUILD_TYPE="${CMAKE_BUILD_TYPE}" \-DMAKE_TEST=OFF \-DBUILD_FS_BENCHMARK="${BUILD_FS_BENCHMARK}" \${CMAKE_USE_CCACHE:+${CMAKE_USE_CCACHE}} \-DWITH_MYSQL="${WITH_MYSQL}" \-DUSE_LIBCPP="${USE_LIBCPP}" \-DBUILD_META_TOOL="${BUILD_META_TOOL}" \-DBUILD_INDEX_TOOL="${BUILD_INDEX_TOOL}" \-DSTRIP_DEBUG_INFO="${STRIP_DEBUG_INFO}" \-DUSE_DWARF="${USE_DWARF}" \-DUSE_UNWIND="${USE_UNWIND}" \-DDISPLAY_BUILD_TIME="${DISPLAY_BUILD_TIME}" \-DENABLE_PCH="${ENABLE_PCH}" \-DUSE_MEM_TRACKER="${USE_MEM_TRACKER}" \-DUSE_JEMALLOC="${USE_JEMALLOC}" \-DENABLE_STACKTRACE="${ENABLE_STACKTRACE}" \-DUSE_AVX2="${USE_AVX2}" \-DGLIBC_COMPATIBILITY="${GLIBC_COMPATIBILITY}" \-DEXTRA_CXX_FLAGS="${EXTRA_CXX_FLAGS}" \-DENABLE_CLANG_COVERAGE="${DENABLE_CLANG_COVERAGE}" \-DDORIS_JAVA_HOME="${JAVA_HOME}" \"${DORIS_HOME}/be"if [[ "${OUTPUT_BE_BINARY}" -eq 1 ]]; then"${BUILD_SYSTEM}" -j "${PARALLEL}""${BUILD_SYSTEM}" installficd "${DORIS_HOME}"
fi# 生成帮助文档
if [[ "${BUILD_DOCS}" = "ON" ]]; then# Build docs, should be built before Frontendecho "Build docs"cd "${DORIS_HOME}/docs"./build_help_zip.shcd "${DORIS_HOME}"
fi# 使用npm安装依赖并构建前端UI，将生成的静态文件复制到FE资源目录。
function build_ui() {NPM='npm'if ! ${NPM} --version; thenecho "Error: npm is not found"exit 1fiif [[ -n "${CUSTOM_NPM_REGISTRY}" ]]; then"${NPM}" config set registry "${CUSTOM_NPM_REGISTRY}"npm_reg="$("${NPM}" get registry)"echo "NPM registry: ${npm_reg}"fiecho "Build Frontend UI"ui_dist="${DORIS_HOME}/ui/dist"if [[ -n "${CUSTOM_UI_DIST}" ]]; thenui_dist="${CUSTOM_UI_DIST}"elsecd "${DORIS_HOME}/ui""${NPM}" cache clean --force"${NPM}" install --legacy-peer-deps"${NPM}" run buildfiecho "ui dist: ${ui_dist}"rm -rf "${DORIS_HOME}/fe/fe-core/src/main/resources/static"mkdir -p "${DORIS_HOME}/fe/fe-core/src/main/resources/static"cp -r "${ui_dist}"/* "${DORIS_HOME}/fe/fe-core/src/main/resources/static"/
}# 必须在构建FE之前构建FE UI
if [[ "${BUILD_FE}" -eq 1 ]]; thenif [[ "${BUILD_UI}" -eq 1 ]]; thenbuild_uifi
fi# Clean and build Frontend
# 使用Maven编译指定的FE模块（如FE核心、Spark DPP等）。
if [[ "${FE_MODULES}" != '' ]]; thenecho "Build Frontend Modules: ${FE_MODULES}"cd "${DORIS_HOME}/fe"if [[ "${CLEAN}" -eq 1 ]]; thenclean_fefiif [[ "${DISABLE_JAVA_CHECK_STYLE}" = "ON" ]]; then# Allowed user customer set env param USER_SETTINGS_MVN_REPO means settings.xml file pathif [[ -n ${USER_SETTINGS_MVN_REPO} && -f ${USER_SETTINGS_MVN_REPO} ]]; then"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} -Dskip.doc=true -DskipTests -Dcheckstyle.skip=true ${MVN_OPT:+${MVN_OPT}} -gs "${USER_SETTINGS_MVN_REPO}" -T 1Celse"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} -Dskip.doc=true -DskipTests -Dcheckstyle.skip=true ${MVN_OPT:+${MVN_OPT}} -T 1Cfielseif [[ -n ${USER_SETTINGS_MVN_REPO} && -f ${USER_SETTINGS_MVN_REPO} ]]; then"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} -Dskip.doc=true -DskipTests ${MVN_OPT:+${MVN_OPT}} -gs "${USER_SETTINGS_MVN_REPO}" -T 1Celse"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} -Dskip.doc=true -DskipTests ${MVN_OPT:+${MVN_OPT}} -T 1Cfificd "${DORIS_HOME}"
fi# 创建输出目录，用于存放编译后的二进制文件、配置文件等。
DORIS_OUTPUT=${DORIS_OUTPUT:="${DORIS_HOME}/output/"}
echo "OUTPUT DIR=${DORIS_OUTPUT}"
mkdir -p "${DORIS_OUTPUT}"# 将FE、BE、Broker等组件的构建结果复制到输出目录，包括：# 启动脚本（如start_fe.sh）# 配置文件（如fe.conf）# JAR包和依赖库# 静态资源（Web UI）
if [[ "${BUILD_FE}" -eq 1 ]]; theninstall -d "${DORIS_OUTPUT}/fe/bin" "${DORIS_OUTPUT}/fe/conf" \"${DORIS_OUTPUT}/fe/webroot" "${DORIS_OUTPUT}/fe/lib"cp -r -p "${DORIS_HOME}/bin"/*_fe.sh "${DORIS_OUTPUT}/fe/bin"/cp -r -p "${DORIS_HOME}/conf/fe.conf" "${DORIS_OUTPUT}/fe/conf"/cp -r -p "${DORIS_HOME}/conf/ldap.conf" "${DORIS_OUTPUT}/fe/conf"/cp -r -p "${DORIS_HOME}/conf/mysql_ssl_default_certificate" "${DORIS_OUTPUT}/fe/"/rm -rf "${DORIS_OUTPUT}/fe/lib"/*cp -r -p "${DORIS_HOME}/fe/fe-core/target/lib"/* "${DORIS_OUTPUT}/fe/lib"/cp -r -p "${DORIS_HOME}/fe/fe-core/target/doris-fe.jar" "${DORIS_OUTPUT}/fe/lib"/cp -r -p "${DORIS_HOME}/docs/build/help-resource.zip" "${DORIS_OUTPUT}/fe/lib"/cp -r -p "${DORIS_HOME}/minidump" "${DORIS_OUTPUT}/fe"/cp -r -p "${DORIS_HOME}/webroot/static" "${DORIS_OUTPUT}/fe/webroot"/cp -r -p "${DORIS_THIRDPARTY}/installed/webroot"/* "${DORIS_OUTPUT}/fe/webroot/static"/copy_common_files "${DORIS_OUTPUT}/fe/"mkdir -p "${DORIS_OUTPUT}/fe/log"mkdir -p "${DORIS_OUTPUT}/fe/doris-meta"mkdir -p "${DORIS_OUTPUT}/fe/conf/ssl"
fi
# 编译
if [[ "${BUILD_SPARK_DPP}" -eq 1 ]]; theninstall -d "${DORIS_OUTPUT}/fe/spark-dpp"rm -rf "${DORIS_OUTPUT}/fe/spark-dpp"/*cp -r -p "${DORIS_HOME}/fe/spark-dpp/target"/spark-dpp-*-jar-with-dependencies.jar "${DORIS_OUTPUT}/fe/spark-dpp"/
fiif [[ "${OUTPUT_BE_BINARY}" -eq 1 ]]; then# 如果多次使用$DORIS_OUTPUT，需要删除旧版本的hadoop jar，否则会导致jar冲突rm -rf "${DORIS_OUTPUT}/be/lib/hadoop_hdfs"install -d "${DORIS_OUTPUT}/be/bin" \"${DORIS_OUTPUT}/be/conf" \"${DORIS_OUTPUT}/be/lib" \"${DORIS_OUTPUT}/be/www" \"${DORIS_OUTPUT}/be/tools/FlameGraph"cp -r -p "${DORIS_HOME}/be/output/bin"/* "${DORIS_OUTPUT}/be/bin"/cp -r -p "${DORIS_HOME}/be/output/conf"/* "${DORIS_OUTPUT}/be/conf"/cp -r -p "${DORIS_HOME}/be/output/dict" "${DORIS_OUTPUT}/be/"if [[ -d "${DORIS_THIRDPARTY}/installed/lib/hadoop_hdfs/" ]]; thencp -r -p "${DORIS_THIRDPARTY}/installed/lib/hadoop_hdfs/" "${DORIS_OUTPUT}/be/lib/"fiif [[ -f "${DORIS_THIRDPARTY}/installed/lib/libz.so" ]]; thencp -r -p "${DORIS_THIRDPARTY}/installed/lib/libz.so"* "${DORIS_OUTPUT}/be/lib/"fiif [[ "${BUILD_BE_JAVA_EXTENSIONS_FALSE_IN_CONF}" -eq 1 ]]; thenecho -e "\033[33;1mWARNNING: \033[37;1mDisable Java UDF support in be.conf due to the BE was built without Java UDF.\033[0m"cat >>"${DORIS_OUTPUT}/be/conf/be.conf" <<EOF# Java UDF and BE-JAVA-EXTENSION support
enable_java_support = false
EOFfi# Fix Killed: 9 error on MacOS (arm64).# See: https://stackoverflow.com/questions/67378106/mac-m1-cping-binary-over-another-results-in-crashif [[ -f "${DORIS_HOME}/be/output/lib/doris_be" ]]; thenrm -f "${DORIS_OUTPUT}/be/lib/doris_be"cp -r -p "${DORIS_HOME}/be/output/lib/doris_be" "${DORIS_OUTPUT}/be/lib"/fiif [[ -d "${DORIS_HOME}/be/output/lib/doris_be.dSYM" ]]; thenrm -rf "${DORIS_OUTPUT}/be/lib/doris_be.dSYM"cp -r "${DORIS_HOME}/be/output/lib/doris_be.dSYM" "${DORIS_OUTPUT}/be/lib"/fiif [[ -f "${DORIS_HOME}/be/output/lib/fs_benchmark_tool" ]]; thencp -r -p "${DORIS_HOME}/be/output/lib/fs_benchmark_tool" "${DORIS_OUTPUT}/be/lib"/fiif [[ "${BUILD_META_TOOL}" = "ON" ]]; thencp -r -p "${DORIS_HOME}/be/output/lib/meta_tool" "${DORIS_OUTPUT}/be/lib"/fiif [[ "${BUILD_INDEX_TOOL}" = "ON" ]]; thencp -r -p "${DORIS_HOME}/be/output/lib/index_tool" "${DORIS_OUTPUT}/be/lib"/ficp -r -p "${DORIS_HOME}/webroot/be"/* "${DORIS_OUTPUT}/be/www"/cp -r -p "${DORIS_HOME}/tools/FlameGraph"/* "${DORIS_OUTPUT}/be/tools/FlameGraph"/if [[ "${STRIP_DEBUG_INFO}" = "ON" ]]; thencp -r -p "${DORIS_HOME}/be/output/lib/debug_info" "${DORIS_OUTPUT}/be/lib"/fiif [[ "${BUILD_FS_BENCHMARK}" = "ON" ]]; thencp -r -p "${DORIS_HOME}/bin/run-fs-benchmark.sh" "${DORIS_OUTPUT}/be/bin/"/fiextensions_modules=("java-udf")extensions_modules+=("jdbc-scanner")extensions_modules+=("hudi-scanner")extensions_modules+=("paimon-scanner")extensions_modules+=("max-compute-scanner")extensions_modules+=("avro-scanner")extensions_modules+=("preload-extensions")if [[ -n "${BE_EXTENSION_IGNORE}" ]]; thenIFS=',' read -r -a ignore_modules <<<"${BE_EXTENSION_IGNORE}"new_modules=()for module in "${extensions_modules[@]}"; domodule=${module// /}if [[ -n "${module}" ]]; thenignore=0for ignore_module in "${ignore_modules[@]}"; doif [[ "${module}" == "${ignore_module}" ]]; thenignore=1breakfidoneif [[ "${ignore}" -eq 0 ]]; thennew_modules+=("${module}")fifidoneextensions_modules=("${new_modules[@]}")fiBE_JAVA_EXTENSIONS_DIR="${DORIS_OUTPUT}/be/lib/java_extensions/"rm -rf "${BE_JAVA_EXTENSIONS_DIR}"mkdir "${BE_JAVA_EXTENSIONS_DIR}"for extensions_module in "${extensions_modules[@]}"; domodule_jar="${DORIS_HOME}/fe/be-java-extensions/${extensions_module}/target/${extensions_module}-jar-with-dependencies.jar"module_proj_jar="${DORIS_HOME}/fe/be-java-extensions/${extensions_module}/target/${extensions_module}-project.jar"mkdir "${BE_JAVA_EXTENSIONS_DIR}"/"${extensions_module}"if [[ -f "${module_jar}" ]]; thencp "${module_jar}" "${BE_JAVA_EXTENSIONS_DIR}"/"${extensions_module}"fiif [[ -f "${module_proj_jar}" ]]; thencp "${module_proj_jar}" "${BE_JAVA_EXTENSIONS_DIR}"/"${extensions_module}"fidonecp -r -p "${DORIS_THIRDPARTY}/installed/webroot"/* "${DORIS_OUTPUT}/be/www"/copy_common_files "${DORIS_OUTPUT}/be/"mkdir -p "${DORIS_OUTPUT}/be/log"mkdir -p "${DORIS_OUTPUT}/be/log/tracing"mkdir -p "${DORIS_OUTPUT}/be/storage"
fi# 调用Broker的独立编译脚本，并将结果复制到输出目录。
if [[ "${BUILD_BROKER}" -eq 1 ]]; theninstall -d "${DORIS_OUTPUT}/apache_hdfs_broker"cd "${DORIS_HOME}/fs_brokers/apache_hdfs_broker"./build.shrm -rf "${DORIS_OUTPUT}/apache_hdfs_broker"/*cp -r -p "${DORIS_HOME}/fs_brokers/apache_hdfs_broker/output/apache_hdfs_broker"/* "${DORIS_OUTPUT}/apache_hdfs_broker"/copy_common_files "${DORIS_OUTPUT}/apache_hdfs_broker/"cd "${DORIS_HOME}"
fiecho "***************************************"
echo "Successfully build Doris"
echo "***************************************"if [[ -n "${DORIS_POST_BUILD_HOOK}" ]]; theneval "${DORIS_POST_BUILD_HOOK}"
fiexit 0
