 |
P4C
The P4 Compiler
|
|
P4C
The P4 Compiler
|
P4C is a reference compiler for the P4 programming language. It supports both P4-14 and P4-16; you can find more information about P4 here and the specifications for both versions of the language here. One fact attesting to the level of quality and completeness of P4C's code is that its front-end code, mid-end code, and P4C-graphs back end are used as the basis for at least one commercially supported P4 compiler.
P4C is modular; it provides a standard frontend and midend which can be combined with a target-specific backend to create a complete P4 compiler. The goal is to make adding new backends easy.
The P4C compiler is a compiler infrastructure for the P4 compiler designed with the following goals:
P4C includes seven sample backends, catering to different target architectures and use cases:
simple_switch written using the BMv2 behavioral model,Compile P4_16 or P4_14 source code. If your program successfully compiles, the command will create files with the same base name as the P4 program you supplied, and the following suffixes instead of the .p4:
.p4i, which is the output from running the preprocessor on your P4 program..json that is the JSON file format expected by BMv2 behavioral model simple_switch.By adding the option --p4runtime-files <filename>.txt as shown in the example commands below, P4C will also create a file <filename>.txt. This is a text format "P4Info" file, containing a description of the tables and other objects in your P4 program that have an auto-generated control plane API.
All of these commands take the --help argument to show documentation of supported command line options. p4c --target-help shows the supported "target, arch" pairs.
Auto-translate P4_14 source to P4_16 source:
Check syntax of P4_16 or P4_14 source code, without limitations that might be imposed by any particular compiler back end. There is no output for these commands other than error and/or warning messages.
Generate GraphViz ".dot" files for parsers and controls of a P4_16 or P4_14 source program.
Generate PDF of parser instance named "ParserImpl" generated by the p4c-graphs command above (search for graphviz below for its install instructions):
P4C has package support for several Ubuntu and Debian distributions.
A P4C package is available in the following repositories for Ubuntu 20.04 and newer.
For Debian 11 (Bullseye) it can be installed as follows:
If you cannot use a repository to install P4C, you can download the .deb file for your release and install it manually. You need to download a new file each time you want to upgrade P4C.
--recursive to pull them in: git clone --recursive https://github.com/p4lang/p4c.git If you forgot --recursive, you can update the submodules at any time using: git submodule update --init --recursive Build. Building should also take place in a subdirectory named build. mkdir build cd build cmake .. <optional arguments> make -j4 make -j4 check The cmake command takes the following optional arguments to further customize the build (see file CMakeLists.txt for the full list):
-DCMAKE_BUILD_TYPE=Release|Debug – set CMAKE_BUILD_TYPE to Release or Debug to build with optimizations or with debug symbols to run in gdb. Default is Release.-DCMAKE_INSTALL_PREFIX=<path> – set the directory where make install installs the compiler. Defaults to /usr/local.-DENABLE_BMV2=ON|OFF. Enable the bmv2 backend. Default ON.-DENABLE_EBPF=ON|OFF. Enable the ebpf backend. Default ON.-DENABLE_P4TC=ON|OFF. Enable the TC backend. Default ON.-DENABLE_UBPF=ON|OFF. Enable the ubpf backend. Default ON.-DENABLE_DPDK=ON|OFF. Enable the DPDK backend. Default ON.-DENABLE_P4C_GRAPHS=ON|OFF. Enable the p4c-graphs backend. Default ON.-DENABLE_P4FMT=ON|OFF. Enable the p4fmt backend. Default ON.-DENABLE_P4TEST=ON|OFF. Enable the p4test backend. Default ON.-DENABLE_TEST_TOOLS=ON|OFF. Enable the p4tools backend. Default OFF.-DENABLE_DOCS=ON|OFF. Build documentation. Default is OFF.-DENABLE_GC=ON|OFF. Enable the use of the garbage collection library. Default is ON.-DENABLE_GTESTS=ON|OFF. Enable building and running GTest unit tests. Default is ON.-DP4C_USE_PREINSTALLED_ABSEIL=ON|OFF. Try to find a system version of Abseil instead of a fetched one. Default is OFF.-DP4C_USE_PREINSTALLED_PROTOBUF=ON|OFF. Try to find a system version of Protobuf instead of a CMake version. Default is OFF.-DENABLE_ABSEIL_STATIC=ON|OFF. Enable the use of static abseil libraries. Default is ON. Only has an effect when P4C_USE_PREINSTALLED_ABSEIL is enabled.-DENABLE_PROTOBUF_STATIC=ON|OFF. Enable the use of static protobuf libraries. Default is ON. Only has an effect when P4C_USE_PREINSTALLED_PROTOBUF is enabled.-DENABLE_MULTITHREAD=ON|OFF. Use multithreading. Default is OFF.-DBUILD_LINK_WITH_GOLD=ON|OFF. Use Gold linker for build if available.-DBUILD_LINK_WITH_LLD=ON|OFF. Use LLD linker for build if available (overrides BUILD_LINK_WITH_GOLD).-DENABLE_LTO=ON|OFF. Use Link Time Optimization (LTO). Default is OFF.-DENABLE_WERROR=ON|OFF. Treat warnings as errors. Default is OFF.-DCMAKE_UNITY_BUILD=ON|OFF. Enable unity builds for faster compilation. Default is OFF.If adding new targets to this build system, please see instructions.
sudo make install The compiler driver p4c and binaries for each of the backends are installed in /usr/local/bin by default; the P4 headers are placed in /usr/local/share/p4c. p4c -b bmv2-ss-p4org program.p4 -o program.bmv2.json If you plan to contribute to P4C, you'll find more useful information here.
Ubuntu 20.04 is the officially supported platform for P4C. There's also unofficial support for macOS 11. Other platforms are untested; you can try to use them, but YMMV.
git for version controlBackends may have additional dependencies. The dependencies for the backends included with P4C are documented here:
Most dependencies can be installed using apt-get install:
For documentation building:
Tools
P4C also depends on Google Protocol Buffers (Protobuf). P4C requires version 3.0 or higher, so the packaged version provided in Ubuntu 20.04 should work. However, P4C typically installs its own version of Protobuf using CMake's FetchContent module (at the moment, 3.25.3). If you are experiencing issues with the Protobuf version shipped with your OS distribution, we recommend that to install Protobuf 3.25.3 from source. You can find instructions here. After cloning Protobuf and before you build, check-out version 3.25.3:
git checkout v3.25.3
Please note that while all Protobuf versions newer than 3.0 should work for P4C itself, you may run into trouble with Abseil, some extensions and other p4lang projects unless you install version 3.25.3.
P4C also depends on Google Abseil library. This library is also a pre-requisite for Protobuf of any version newer than 3.21. Therefore the use of Protobuf of suitable version automatically fulfils Abseil dependency. P4C typically installs its own version of Abseil using CMake's FetchContent module (Abseil LTS 20240116.1 at the moment).
P4C requires a CMake version of at least 3.16.3 or higher. On older systems, a newer version of CMake can be installed using pip3 install --user cmake==3.16.3. We have a CI test on Ubuntu 18.04 that uses this option, but there is no guarantee that this will lead to a successful build.
For documentation building:
Tools
You can also look at the dependencies installation script for a fresh Fedora instance.
Installing on macOS:
/usr/local/bin/ to your $PATH.Install dependencies using Homebrew:
or with MacPorts
By default, Homebrew doesn't link programs into /usr/local/bin if they would conflict with a version provided by the base system. This includes Bison, since an older version ships with macOS. make check depends on the newer Bison we just installed from Homebrew (see #83), so you'll want to add it to your $PATH one way or another. One simple way to do that is to request that Homebrew link it into /usr/local/bin:
Optional documentation building tools:
Homebrew offers a protobuf formula. It installs version 3.2, which should work for P4C itself but may cause problems with some extensions. It's preferable to use the version of Protobuf which is supplied with CMake's fetchcontent (3.25.3).
The protobuf formula requires the following CMake variables to be set, otherwise CMake does not find the libraries or fails in linking. It is likely that manually installed Protobuf will require similar treatment.
P4c relies on BDW garbage collector to manage its memory. By default, the P4C executables are linked with the garbage collector library. When the GC causes problems, this can be disabled by setting ENABLE_GC cmake option to OFF. However, this will dramatically increase the memory usage by the compiler, and may become impractical for compiling large programs. Do not disable the GC, unless you really have to. We have noticed that this may be a problem on MacOS.
P4c will use libbacktrace to produce readable crash dumps if it is available. This is an optional dependency; if it is not available everything should build just fine, but crash dumps will not be very readable.
There is a variety of design and development documentation here.
We recommend using clang++ with no optimizations for speeding up compilation and simplifying debugging.
We recommend installing a new version of gdb, because older gdb versions do not always handle C++11 or newer correctly.
We recommend exuberant ctags for navigating source code in Emacs and vi. sudo apt-get install exuberant-ctags. The Makefile targets make ctags and make etags generate tags for vi and Emacs respectively. (Make sure that you are using the correct version of ctags; there are several competing programs with the same name in existence.)
To build code documentation, after installing Doxygen and the other required packages:
The HTML output is available in docs/doxygen/build/html/index.html.
Occasionally formatting commits are applied to P4C. These pollute the git history. To ignore these commits in git blame, run this command
The P4C code base is subject to a series of linter checks which are checked by CI. To avoid failing these checks and wasting unnecessary CI cycles and resources, you can install git commit hooks by running
These commit hooks will run on every commit and check the files you are planning to commit with cpplint and clang-format.
A Dockerfile is included. You can generate an image which contains a copy of P4C in /p4c/build by running:
On some platforms Docker limits the memory usage of any container, even containers used during the docker build process. On macOS in particular the default is 2GB, which is not enough to build P4C. Increase the memory limit to at least 4GB via Docker preferences or you are likely to see "internal compiler
errors" from GCC which are caused by low memory.
The project can also be build using Bazel:
We run continuous integration to ensure this works with the latest version of Bazel.
We also provide a p4_library rule for invoking P4C during the build process of 3rd party Bazel projects.
See bazel/example for an example of how to use or extend P4C in your own Bazel project. You may use it as a template to get you started.
The build system is based on cmake. This section describes how it can be customized.
When building a new backend target, add it into the development tree in the extensions subdirectory. The cmake-based build system will automatically include it if it contains a CMakeLists.txt file.
For a new backend, the cmake file should contain the following rules:
Backend specific IR definition files should be added to the global list of IR_DEF_FILES as they are processed together with the core IR files. Use the following rule:
where MY_IR_DEF_FILES is a list of file names with absolute path (for example, use ${CMAKE_CURRENT_SOURCE_DIR}).
If in addition you have additional supporting source files, they should be added to the ir sources, as follows:
Again, MY_IR_SRCS is a list of file names with absolute path.
Sources (.cpp and .h) should be added to the cpplint and clang-format target using the following rule:
Python files should be added to the black and isort target using the following rule:
The P4C CMakeLists.txt will use that name to figure the full path of the files to lint.
clang-format, black, and isort need to be installed before the linter can be used. They can be installed with the following command:
clang-format can be checked using the make clang-format command. Complaints can be fixed by running make clang-format-fix-errors. black and isort can be checked using the make black or make isort command respectively. Complaints can be fixed by running make black-fix-errors or make isort-fix-errors.
cpplint, clang-format, and black/isort run as checks as port of P4C's continuous integration process. To make sure that these tests pass, we recommend installing the appropriate git hooks. This can be done by running
clang-format, cpplint, and black/isort checks will be enforced on every branch commit. In cases where checks are failing but the commit is sound, one can bypass the hook enforcement using git commit --no-verify.
Define a target for your executable. The target should link against the core P4C_LIBRARIES and P4C_LIB_DEPS. P4C_LIB_DEPS are package dependencies. If you need additional libraries for your project, add them to P4C_LIB_DEPS.
In addition, your target should depend on the genIR target, since you need all the IR generation to happen before you start compiling your backend. If you chose to have your backend as a library (seem the backends/bmv2 example), the library should depend on genIR, and there is no longer necessary for your executable to depend on it.
We implemented support equivalent to the automake make check rules. All tests should be included in make check and in addition, we support make check-* rules. To enable this support, add the following rules:
In addition, you can add individual tests to a suite using the following macro:
See the documentation for p4c_add_test_with_args and p4c_add_tests for more information on the arguments to these macros.
To pass custom arguments to P4C, you can set the environment variable P4C_ARGS:
When making changes to P4C, it is sometimes useful to be able to run the tests while overwriting the expected output files that are saved in this repository. One such situation is when your changes to P4C cause the names of compiler-generated local variables to change. To force the expected output files to be rewritten while running the tests, assign a value to the shell environment variable P4TEST_REPLACE. Here is one example Bash command to do so:
Define rules to install your backend. Typically you need to install the binary, the additional architecture headers, and the configuration file for the P4C driver.
The lib directory contains miscellaneous utilities that are generally useful and not specific to any part of the compiler. Most are not even compiler specific. The files in the lib folder should only depend on each other; they cannot depend on any other compiler files.
| File(s) | Description |
|---|---|
algorithm.h | Wrapper around <algorithm> that contains several useful additional algorithms. |
bitops.h | Bit manipulation operations. |
bitvec.h, bitvec.cpp | Dynamic bitvectors with useful operations. The standard types std::vector<bool> and std::bitset are missing crucial functionality, making them generally useless. |
cstring.h, cstring.cpp | Constant strings. The standard library std::string type is mutable, allowing the string to be changed dynamically. cstring keeps the memory for all constant strings in a single global pool, allowing constant time comparisons. |
default.h | Synthesizing default values of various types (e.g., 0 for integers, nullptr for pointers, etc.). |
enumerator.h, enumerator.cpp | C#-like enumerator interface. |
error.h, error.cpp, expressions.h | Error reporting functions. |
gc.cpp | Overrides global operator new and delete to use the Boehm/Demers/Weiser conservative collector, so all memory allocations are garbage collected. |
hex.h, hex.cpp | Adaptor for more conveniently printing hexadecimal strings with ostreams. |
indent.h, indent.cpp | Adaptor for managing indentation on ostreams. |
log.h, log.cpp | Macros and support for logging that can be managed on a per-source-file basis. |
ltbitmatrix.h | Adaptor using a bitvec as a lower-triangular bit matrix. |
map.h | Wrapper around map, adding some useful functions that are missing from std::map. |
nullstream.h | A simple ostream that does nothing. |
options.h, options.cpp | Represents compiler command-line options. |
range.h | Iterators over numeric ranges. |
source_file.h, source_file.cpp | Represents the input source of the compiler and source file position information used for error reporting and generating debugging information. |
stringify.h, stringify.cpp | Conversion of various types to strings. |
sourceCodeBuilder.h | Support for emitting programs in source (works for P4 and C). |
Issues with the compiler are tracked on GitHub. Before opening a new issue, please check whether a similar issue is already opened. Opening issues and submitting a pull request with fixes for those issues is much appreciated.
In addition to the list of issues on Github, there are a number of currently unsupported features listed below:
See also unsupported P4_16 language features.
We welcome and appreciate new contributions. Please take a moment to review our Contribution Guidelines to get started.
Educational material on P4:
We appreciate your contributions and look forward to working with you to improve the P4 Compiler Project (P4C)!
1.12.0