This is a short guide to using compilers and some scripting languages for numerical calculations on OpenBSD.
Not all languages and scripts are included, mostly because I don’t know about all of them.
(This guide is valid for OpenBSD release 7.1 and should be considered obsolete after October 2023 unless updated. It is mostly about the AMD64 and compatible platforms.)
This is not historically one of OpenBSD’s reputational strengths. Mostly it is known for security and excellent documentation.
Yet, many computational tools are available including recent releases:
It also has libraries such as boost, eigen, fftw, netcdf and others.
Numerical computations are sometime used with OpenBSD as part of a larger system such as an embedded computer or, for example, a web service using machine-learning or statistical calculations.
Some people use OpenBSD for development of numerical code or for calculating.
You can find numerical codes in all languages; here we focus on C/C++ using clang and GNU tools. We also describe a bit of Fortran.
Clang is included in “base” on AMD64, meaning it comes with the operating system. GNU languages are added with packages, including C, C++, Fortran and others.
Additional languages such as Forth, Go, Haskell, Ocaml and Rust are also available for various problems. These are not covered here.
OpenBSD on the AMD 64-bit platform (which also supports the Intel emulations known as EMT64) is configured to support all compatible processors including older ones. Newer processors have improved instruction sets, resulting in a variety of options for possibly gaining performance.
By default, the compiler will not optimize for your particular CPU. As
a result, your code might go faster if you specify some
compiler switches like -Ofast or -O3, or various loop-specific
code-generation options. Or not: you will have to experiment.
CPU-model specific options are also available. The easiest way
is -march=native.
Most compiled code automatically produces dynamic-linked executables. The executable file thus includes only the names of shared libraries (located in /usr/lib or /usr/local/lib). This reduces disk space use and forces convergence of library file versions to the same version. However it may be slower by up to 10%.
Using the --static option in compiler commands, or referencing static
object libraries for most code may help speed. For example
libset="-I/usr/local/include -L/usr/local/lib"
boostset="-lboost_timer -lboost_chrono -lboost_system"
c++ -o test.x --static $libset $boostset test.cpp
This results in the essential parts of the three boost libraries (and default libraries) being included within the test.x executable.
See my examples page for further information.
AMD64 platforms can use 80-bit floating-point arithmetic internally, and 64-bit arithmetic for double-precision values that are written to memory. Some compile options (vector instructions like SSE3, AVX, fast-math) may change the conversion of 80-bit numbers or avoid them altogether. Numerical results of calcuations with these options may therefore vary from calculations without these options.
Be careful when testing code changes or compiler changes for effectiveness when searching for faster programs. Here are some brief suggestions:
-O3 is often the best optimization you can do,
within a few percent.
If more is needed, try -march=<cpumodel> especially for later models
of Intel processors. -march=nosuchmodel gives a list of possibilities.
Also, try -march=native -mcpu=native on GCC compilers.You can use C and Fortran code compiled with either compiler together in the same executable. We know this because code compiled with the GCC C compiler successfully links with the OpenBSD base (built with clang). Note only GCC Fortran (g95) is available as of OpenBSD 6.8.
C++ is not compatible across clang and GCC because the run-time libraries are not interchangeable. If you want to use C++ libraries from ports you have to use clang C++.
If you need to use GNU C++ then you will have to compile any needed C++ libraries with GNU C++ as well. Examples are: boost, tbb, uuid. If COMPILER_LIBCXX is in the ports Makefile for that library, then that library uses clang C++ and would have to be built separately with GCC (or avoided).
Most numerically useful interpreted or scripting languages are available.
Python 3.9 is available. A number of python packages are available to be added as you need them. Python virtual environments work as well.
Your shell (ksh) has a built-in “time” command; so does OpenBSD. Both
report user and system CPU time; /usr/bin/time -l
also reports memory and IO.
See style(9) for the OpenBSD style guide.
The
clang-format -style="{BasedOnStyle: Mozilla, IndentWidth: 4}"
command generates OpenBSD-like source
from your ragged nasty C/C++ original. Recommended. See the manpage.
For Fortran, try fprettify available for Python 3.x. This program will
change files in-place by default. See fprettify --help.
A software SCM tool provides a reliable way of tracking minor and major code changes to your software. You (and your teammates) record change events with the SCM tool as you create or test or revise code. The SCM tracks differences at each event and can report on these. The SCM can also reproduce older versions of files or sets of files, according to event ids, dates or named markers.
Use one of: CVS (included with OpenBSD), git (or got), or Subversion.
You can find lots of help online with a web search.
Numerical codes are difficult to test because of design issues (few or no subroutines or functions), numerical stability questions, lack of reference solutions for some problems, uncertain approximations or unknown assumptions. It’s not, however, impossible. Attention to accuracy and precision informs the user in addition to the numerical results.
The purpose of computation is insight, not numbers. –Hamming.
Test tools come in many shapes; here are some options:
Automated testing has additional benefits: it helps protect against bugs introduced by code changes (or compiler or operating system updates); it documents the tests for subsequent review; it serves as an educational tool for new programmers learning a large code base.
Links:
OpenBSD Numerics |
OpenBSD Numerics - Parallelization
OpenBSD Numerics - Clusters |
OpenBSD Numerics - Examples
OpenBSD Numerics - Experiences pages