Skip to main content

Ding Zhiyu Week 2 Study Report: Compilation Tools

​ Author: Ding Zhiyu Date: 2024.1.27

Table of Contents

[TOC]

Main Content

CMake Study​

Introduction to CMake​

  1. What is CMake? CMake (short for Cross platform Make) is not a build system itself, but rather a build system generator. It is a cross-platform build tool that generates Makefile files for the make build system on Linux, and generates Visual Studio or MSVC projects on Windows. Therefore, the actual build work still needs to be delegated to build systems such as Make, Ninja, MSVC, etc.

  2. What is CMakeLists.txt? CMakeLists.txt is an important file used by CMake to describe project build rules and processes. It is like a build instruction list that tells CMake how to compile and link code. In CMakeLists.txt, we can set the project name, specify source files to compile, add library dependencies, and more. CMake reads this file and generates the corresponding build files to guide the entire build process. Writing CMakeLists.txt can make cross-platform compilation much simpler and more convenient.

CMake Basic Syntax​

cmake_minimum_required Sets the minimum CMake version requirement Example:

cmake_minimum_required(VERSION 3.22)

project Used to specify the project name and supported languages. By default, all languages are supported. PROJECT(HELLO) specifies the project name and supports all languages (highly recommended). PROJECT(HELLO CXX) specifies the project name and supports C++. PROJECT(HELLO C CXX) specifies the project name and supports C and C++. Example:

project(Test)

set keyword Used to set the value of a variable. The variable can be a normal variable, cache variable, or environment variable.

SET(<variable> <value>... [PARENT_SCOPE]) #设置普通变量
SET(<variable> <value>... CACHE <type> <docstring> [FORCE]) #设置缓存变量
SET(ENV{<variable>} [<value>]) #设置环境变量

add_executable Generates an executable file.

ADD_EXECUTABLE(hello ${SRC_LIST})

In this line of code, hello is the filename of the generated executable, and the file content is taken from the variable SRC_LIST.

MESSAGE keyword

Outputs user-defined information to the terminal (personally feels similar to printf() in C).

Mainly includes three types of messages:

SEND_ERROR: Produces an error, and the generation process is skipped.

STATUS: Outputs information prefixed with -.

FATAL_ERROR: Immediately terminates all CMake processes.

Example: Message( STATUS "module path = ${CMAKE_MODULE_PATH}" ) This code in CMake is used to output a status message to the console. Specifically:

  • Message is a CMake command that allows displaying messages to the user during the build process.
  • The STATUS parameter specifies the message type. Here it is a "status" message, which is typically informational and does not interrupt the build process, but is displayed in standard output (stdout) or the CMake GUI interface.
  • "module path = ${CMAKE_MODULE_PATH}" is the message content to display, where ${CMAKE_MODULE_PATH} is a predefined CMake variable that contains the list of search paths CMake uses when looking for module files.

CMake Practice​

Using CMake to compile OpenMP, MPI, and CUDA programs

cmake_minimum_required(VERSION 3.8 FATAL_ERROR)
project(HelloPrograms LANGUAGES CXX CUDA)

# 寻找OpenMP
find_package(OpenMP REQUIRED)

# CUDA程序
add_executable(hello_cuda hello_cuda.cu)

# MPI程序
find_package(MPI REQUIRED)
include_directories(SYSTEM ${MPI_INCLUDE_PATH})
add_executable(hello_mpi hello_mpi.cpp)
target_link_libraries(hello_mpi MPI::MPI_CXX)

# OpenMP程序
add_executable(hello_omp hello_omp.cpp)
target_link_libraries(hello_omp OpenMP::OpenMP_CXX)

Makefile Study​

Introduction to Makefile​

Makefile is the configuration file used by the make tool for automated builds on Linux and Unix-like systems. It defines how to compile and link program object files (.o), specifying dependencies and command rules. By parsing the Makefile, make can automatically detect which source files have changed and execute the corresponding compilation operations, avoiding the need to manually enter complex compilation commands and greatly improving development efficiency.

Makefile Basic Syntax​

  1. Targets:
  • Targets usually correspond to an executable file or intermediate file, such as: target: dependencies. For example, hello: hello.o means the "hello" target depends on "hello.o".
  1. Dependencies:
  • Dependency files are a set of files that must exist before generating the target. When a dependency file is newer than the target file or the target does not exist, make executes the relevant rules to update.
  1. Commands:
  • Command lines are written after the dependencies, each line starting with a tab character. For example: 
    target: dependencies
        command1
        command2
    Here, command1 and command2 are commands used to rebuild the target when dependency files have changed.
  1. Variables:
  • Variables are used to store reusable values, such as compiler names, compilation options, etc. 
    CC = gcc
    CFLAGS = -Wall -g
    target: dependencies
    $(CC) $(CFLAGS) -c source.c -o target.o
  1. Implicit Rules and Pattern Rules:
  • Make comes with some predefined implicit rules, such as .c.o: for compiling C source files to object files.
  • Pattern rules allow you to describe a set of files with the same compilation pattern in a concise way.
  1. Clean Target:
  • Usually a clean target is defined to remove all generated intermediate files and final target files, such as: 
    clean:
    rm -f *.o target

More Makefile Usage Methods and Tips​

About $^​

In Makefile, $^ is an automatic variable that represents all the prerequisites (dependencies) in the current rule, i.e., everything listed to the right of the colon :. These dependencies are separated by spaces and do not include duplicate dependencies.

For example, if you have a Makefile rule like this:

myprogram: main.o utils.o
    gcc -o myprogram $^

Here $^ will be expanded to main.o utils.o. So the above rule is equivalent to:

myprogram: main.o utils.o
    gcc -o myprogram main.o utils.o

The benefit of using $^ is that you don't have to manually repeat all the dependencies, making the Makefile more concise and easier to maintain. If the dependency list changes, you only need to update the dependency list rather than modifying every occurrence.

Common Phony Targets:​

These phony targets need a .PHONY declaration before them to tell make that these targets are not filenames:

.PHONY: all clean install uninstall distclean test help

  1. all: Usually the default target, used to compile all programs. If you just type make, the all target is typically executed.

    all: prog1 prog2

  2. clean: As mentioned earlier, used to delete all files created by make.

    clean:
    rm -f *.o prog1 prog2

  3. check or test: Used to run the program's test suite.

    test: all
    ./test_script.sh

Note: All of these phony targets, except all, require make target_name to trigger their use.

Handling Multi-Level Folder Structures in Projects​

  1. Source file paths:

    • If source files are in different subdirectories, you can explicitly write the full path in the dependencies. For example, if source1.c is in the src/subdir1 directory, you can write the rule like this: 
      target: src/subdir1/source1.o
          $(CC) $(CFLAGS) -o target $^

  2. Relative and absolute paths:

    • When referencing files in command lines, you can use relative paths (relative to the Makefile's location) or absolute paths. If the target file and dependency files are not in the same directory, you may need to adjust the output directory or the -o parameter in the compilation command to specify the output location.

Makefile Practice​

Using Makefile to compile OpenMP, MPI, and CUDA programs

# 变量定义
CC=gcc
CXX=g++
NVCC=nvcc
MPICC=mpicxx
CFLAGS=-fopenmp

# 默认目标
all: hello_cuda hello_mpi hello_omp

# CUDA程序
hello_cuda: hello_cuda.cu
	$(NVCC) -o hello_cuda hello_cuda.cu

# MPI程序
hello_mpi: hello_mpi.cpp
	$(MPICC) -o hello_mpi hello_mpi.cpp

# OpenMP程序
hello_omp: hello_omp.cpp
	$(CC) $(CFLAGS) -o hello_omp hello_omp.cpp

# 清理目标
.PHONY: clean
clean:
	rm -f hello_cuda hello_mpi hello_omp

# 测试生成的可执行文件
.PHONY: test
test:
    ./hello_omp
    ./hello_mpi
    ./hello_cuda

Simplified Makefile Practice​

Using Automatic Variables to Simplify Basic Command Blocks​

Before simplification

myprogram: main.o utils.o
    gcc -o myprogram main.o utils.o

After simplification

myprogram: main.o utils.o
    gcc -o $@ $^

The automatic variable $@ represents the target file (in this case myprogram), and $^ represents all dependency files (in this case main.o utils.o).

Compiling Each .c File into an Executable​

If you need to compile each .c file in a folder into a separate executable, you can use pattern rules in the Makefile. Here is a simplified Makefile example that finds all .c files in the current directory and compiles each into an executable with the same name (without the .c extension):

# 定义编译器
CC := gcc

# 定义编译选项
CFLAGS := -Wall

# 查找当前目录下所有的.c文件
SOURCES := $(wildcard *.c)

# 将.c文件列表转换成可执行文件列表
EXECUTABLES := $(SOURCES:.c=)

# 默认目标
all: $(EXECUTABLES)

# 模式规则：如何从.c文件生成可执行文件
%: %.c
    $(CC) $(CFLAGS) -o $@ $<

# 伪目标，比如清理
.PHONY: clean

clean:
    rm -f $(EXECUTABLES)

Let me explain this Makefile:

  1. The CC variable defines the compiler used, here it is gcc.
  2. The CFLAGS variable defines the compilation options, here using -Wall to enable all warnings.
  3. The SOURCES variable uses the wildcard function to get all .c files in the current directory.
  4. The EXECUTABLES variable converts the file suffixes in the SOURCES variable from .c to no suffix, i.e., the executable filenames.
  5. all is the default target, which depends on all executables.
  6. %: %.c is a pattern rule that tells make how to generate an executable from each .c file. $@ represents the target filename, and $< represents the first dependency, i.e., the corresponding .c file.
  7. .PHONY defines a phony target clean to ensure that even if a file named clean exists, executing make clean will still run the cleanup command, which deletes all executables.

Using this Makefile, simply run make on the command line, and it will automatically compile each .c file into a corresponding executable. Running make clean will delete these executables.

Real Makefile Practice​

Let me explain the following Makefile in detail.

# We will benchmark you against Intel MKL implementation, the default processor vendor-tuned implementation.
# This makefile is intended for the Intel C compiler.
# Your code must compile (with icc) with the given CFLAGS. You may experiment with the OPT variable to invoke additional compiler options.

CC = gcc 
# OPT = -no-multibyte-chars
FLAGS = -O3
# -fopt-info
LDLIBS = -lpthread -lm -llapack -lopenblas -lrt -lcblas -I/usr/include/openblas -lgptl -lpapi -O
targets = benchmark-test benchmark-naive benchmark-blocked benchmark-blas
objects = benchmark-test.o benchmark.o gemm-naive.o gemm-blocked.o gemm-blas.o


.PHONY : default
default : all

.PHONY : all
all : clean $(targets)

benchmark-test : benchmark-test.o gemm-blocked.o
	$(CC) -o $@ $^ $(FLAGS) $(LDLIBS)

benchmark-naive : benchmark.o gemm-naive.o
	$(CC) -o $@ $^ $(FLAGS)  $(LDLIBS)
benchmark-blocked : benchmark.o gemm-blocked.o
	$(CC) -o $@ $^ $(FLAGS) $(LDLIBS)
benchmark-blas : benchmark.o gemm-blas.o
	$(CC) -o $@ $^ $(FLAGS) $(LDLIBS)
%.o : %.c
	$(CC) -c $(CFLAGS) $(FLAGS) $<

.PHONY : clean
clean:
	rm -f $(targets) $(objects)

Analysis

This Makefile is used to compile and link a benchmark program based on matrix multiplication, with several different versions of matrix multiplication implementations (such as naive, blocked, and BLAS library). Below is a detailed explanation of each part of the Makefile:

First, the comments in the Makefile indicate that it is designed for the Intel C compiler, but the actual compiler specified in the Makefile is gcc. It also mentions that you can experiment with different compiler options by modifying the OPT variable.

CC = gcc 
# OPT = -no-multibyte-chars
FLAGS = -O3
# -fopt-info
LDLIBS = -lpthread -lm -llapack -lopenblas -lrt -lcblas -I/usr/include/openblas -lgptl -lpapi -O
  • CC: Specifies the compiler as gcc.
  • OPT: This is a commented-out variable that can be used to add additional compiler options, but it is not used in the current Makefile.
  • FLAGS: Specifies the compiler optimization level as -O3, a commonly used compiler optimization option for enabling high-level optimizations.
  • LDLIBS: Specifies linker options, including libraries to link. Here it includes the multi-threaded library pthread, math library m, LAPACK library, OpenBLAS library, real-time library rt, CBLAS library, GPTL performance profiling library, and PAPI performance API library. -I/usr/include/openblas is a compiler option for specifying the header file location of the OpenBLAS library. Note that the -O here is likely a mistake, as it is usually a compiler optimization flag rather than a linker flag.

Next, the Makefile defines targets and object files:

targets = benchmark-test benchmark-naive benchmark-blocked benchmark-blas
objects = benchmark-test.o benchmark.o gemm-naive.o gemm-blocked.o gemm-blas.o
  • targets: These are the final executables to be built by the Makefile.
  • objects: These are the intermediate object files generated after compiling source files.

The Makefile also has some phony targets that do not correspond to filenames but instead execute specific commands:

.PHONY : default
default : all

.PHONY : all
all : clean $(targets)
  • .PHONY: This declaration tells make that these targets are not filenames.
  • default: This is the default target, executed when you just type make. Here it depends on the all target.
  • all: This target first executes the clean command to delete all target and object files, then builds all targets.

Below are the specific rules for building each target:

benchmark-test : benchmark-test.o gemm-blocked.o
	$(CC) -o $@ $^ $(FLAGS) $(LDLIBS)

benchmark-naive : benchmark.o gemm-naive.o
	$(CC) -o $@ $^ $(FLAGS) $(LDLIBS)
benchmark-blocked : benchmark.o gemm-blocked.o
	$(CC) -o $@ $^ $(FLAGS) $(LDLIBS)
benchmark-blas : benchmark.o gemm-blas.o
	$(CC) -o $@ $^ $(FLAGS) $(LDLIBS)

Here each target has its own dependencies, and then uses $(CC) (gcc) to link the object files and libraries to generate the final executable.

For the compilation rule for each .c file:

%.o : %.c
	$(CC) -c $(CFLAGS) $(FLAGS) $<
  • % is a wildcard. This rule tells make how to generate .o files from .c files. Here $< represents the first item in the dependency list, i.e., the corresponding .c file.

Finally, the clean target is used to clean up all files generated during the build process:

.PHONY : clean
clean:
	rm -f $(targets) $(objects)

This rule deletes all target and object files, ensuring the next build starts from a clean state.

Summary: This Makefile defines a series of rules to compile and link benchmark programs based on different matrix multiplication implementations. It uses phony targets to organize the build process and provides functionality to clean up build artifacts.

Question

In a Makefile, the order of rules does not always determine the order in which they are executed. Makefile uses dependency relationships to determine which rules need to be executed first. When you try to build a target (such as an executable), make examines the target's dependencies (usually object files) and then looks for rules on how to build those dependencies. This means that even if the rules for compiling .o files appear later in the Makefile, as long as some target needs those object files, make will first execute the relevant rules to generate them.

In the Makefile provided, when we try to build targets like benchmark-test, benchmark-naive, benchmark-blocked, or benchmark-blas, make looks for the .o files they depend on. For example:

benchmark-test : benchmark-test.o gemm-blocked.o
	$(CC) -o $@ $^ $(FLAGS) $(LDLIBS)

Before attempting to build benchmark-test, make needs the benchmark-test.o and gemm-blocked.o object files. If these files don't exist or the source files are newer than the existing object files, make looks for rules on how to build these object files. In this Makefile, that rule is:

%.o : %.c
	$(CC) -c $(CFLAGS) $(FLAGS) $<

This rule tells make how to generate .o files from .c files. % is a pattern matching character that matches any common part between the target and dependency. Here, $< represents the first of all dependencies, i.e., the corresponding .c file.

Therefore, even if this rule is at the end of the Makefile, it will still be executed before building the executable, because the executable's build depends on the object files. This is a fundamental feature of make: it determines which rules need to be executed based on dependency relationships and file timestamps, not just based on their order of appearance in the Makefile.

Compiling with Shell Scripts​

Basic Shell Script Compilation​

#!/bin/bash

# 指定源文件名
SOURCE="my_program.cpp"

# 指定输出可执行文件名
EXECUTABLE="my_program"

# 使用mpicxx编译器（通常为OpenMPI或MPICH提供的MPI版本的C++编译器）
# 同时添加OpenMP支持（-fopenmp），链接MPI库（-lmpi）
mpicxx -o $EXECUTABLE $SOURCE -fopenmp -lmpi -std=c++11

# 检查编译是否成功
if [ $? -eq 0 ]; then
    echo "Compilation successful."
else
    echo "Compilation failed."
fi

# 如果编译成功，可以运行程序（假设是并行执行）
# mpirun -np <num_procs> ./my_program
# 其中<num_procs>是你想要使用的处理器核心数

More Advanced Shell Script Compilation​

#!/bin/bash

# Set the paths for MPI, OpenMP, and CUDA compilers
# Please adjust these variables according to your environment
MPI_COMPILER=mpicc
OPENMP_COMPILER=gcc
CUDA_COMPILER=nvcc

# Set the source file names
MPI_SOURCE="your_mpi_program.c"
OPENMP_SOURCE="your_openmp_program.c"
CUDA_SOURCE="your_cuda_program.cu"

# Set the output names
MPI_OUTPUT="mpi_program"
OPENMP_OUTPUT="openmp_program"
CUDA_OUTPUT="cuda_program"

# Compile MPI program
echo "Compiling MPI program..."
$MPI_COMPILER $MPI_SOURCE -o $MPI_OUTPUT
if [ $? -eq 0 ]; then
echo "MPI program compiled successfully."
else
echo "Error compiling MPI program."
exit 1
fi

# Compile OpenMP program
echo "Compiling OpenMP program..."
$OPENMP_COMPILER -fopenmp $OPENMP_SOURCE -o $OPENMP_OUTPUT
if [ $? -eq 0 ]; then
echo "OpenMP program compiled successfully."
else
echo "Error compiling OpenMP program."
exit 1
fi

# Compile CUDA program
echo "Compiling CUDA program..."
$CUDA_COMPILER $CUDA_SOURCE -o $CUDA_OUTPUT
if [ $? -eq 0 ]; then
echo "CUDA program compiled successfully."
else
echo "Error compiling CUDA program."
exit 1
fi

echo "All programs compiled successfully."

Save this script as compile.sh, then run chmod +x compile.sh in the terminal to grant execution permissions. Before running the script, make sure you have correctly set the compiler paths and source filenames in the script.

In this shell script, the $? variable contains the exit status of the last command. If the command executed successfully, it returns 0; otherwise it returns a non-zero value. The if statements in the script check whether each compilation command succeeded. If any compilation fails, the script prints an error message and exits.

This script assumes that the source code files do not require any additional libraries or specific compiler options. If the program depends on specific libraries or needs additional compilation options, you need to modify the compilation commands accordingly. For example, if a CUDA program needs to link to certain libraries, you may need to add the -l option to the nvcc command.

Makefile-Like Build Script​

#!/bin/bash

# 变量定义
CC=gcc
CXX=g++
NVCC=nvcc
MPICC=mpicxx
CFLAGS=-fopenmp

# 默认目标
function build_all {
    build_hello_cuda
    build_hello_mpi
    build_hello_omp
}

# CUDA程序
function build_hello_cuda {
    $NVCC -o hello_cuda hello_cuda.cu
}

# MPI程序
function build_hello_mpi {
    $MPICC -o hello_mpi hello_mpi.cpp
}

# OpenMP程序
function build_hello_omp {
    $CC $CFLAGS -o hello_omp hello_omp.cpp
}

# 清理目标
function clean {
    rm -f hello_cuda hello_mpi hello_omp
}

# 测试生成的可执行文件
function test {
    ./hello_omp
    ./hello_mpi
    ./hello_cuda
}

# 解析命令行参数
function main {
    case "$1" in
        clean)
            clean
            ;;
        test)
            test
            ;;
        *)
            build_all
            ;;
    esac
}

# 调用主函数
main $1

Grant it execution permissions:

chmod +x build.sh

Run with the following commands:

./build.sh         # 编译所有程序
./build.sh clean   # 清理编译产生的文件
./build.sh test    # 测试生成的可执行文件

This script provides the same functionality as a Makefile, including compiling CUDA, MPI, and OpenMP programs, cleaning build artifacts, and testing the compiled programs.