How to Assess Code Performance, Benchmark in julia

Overview

The primary reason for using Julia is its speed and performance, so writing code that is optimized from an engineering perspective is extremely important. BenchmarkTools.jl provides an interface for easy, convenient, and accurate performance evaluation¹.

Code

julia> sinpi(0.5), cospi(0.5)
(1.0, 0.0)

julia> sincospi(0.5)
(1.0, 0.0)

Let’s compare the performance between computing a value at once with sincospi and using sinpi and cospi separately.

@btime

The evaluation shows that sincospi is slower than using sinpi alone, but it is faster than calculating sinpi and cospi separately. This result is more accurate and reliable than @time because it’s not based on a single run but on repeated measurements made by the function itself.

julia> x = rand()
0.49058418112141866

julia> @btime SC = sincospi($x);
  7.500 ns (0 allocations: 0 bytes)

julia> @btime S = sinpi($x);
  4.500 ns (0 allocations: 0 bytes)

julia> @btime SC = sinpi($x), cospi($x);
  8.300 ns (0 allocations: 0 bytes)

Here, putting $ in front of the variable x ensures that the benchmarking is measuring only the function’s execution time, not the time it takes to interpolate the variable x into the expression. This leads to more accurate benchmarking results.

@benchmark

For more detailed performance evaluation results, the @benchmark macro is used. It provides visually superior result analysis, including histograms, as shown below.

Differences with @time, @elapsed

While @btime is intended for accurate performance evaluation, @time simply measures time, and @elapsed returns that time as a Float64.

It’s a good habit to measure the execution time of code with @time for writing high-performance code², but fundamentally, @time includes all sorts of overheads, from compilation time onwards. Although it provides an accurate measure of the time invested, it is not suitable for evaluating the performance of the code itself.

Full Code

sinpi(0.5), cospi(0.5)
sincospi(0.5)

using BenchmarkTools
x = rand()
@btime SC = sincospi($x);
@btime S = sinpi($x);
@btime SC = sinpi($x), cospi($x);
@benchmark SC = sincospi($x)
@benchmark SC = sinpi($x), cospi($x)

Environment

• OS: Windows
• julia: v1.10.0