Being lazy in C++

Context

Again, I haven’t posted in a long time. I suppose I’ll have to start all my new articles with this sentence, ahaha. I can say I was a bit lazy, and that’s a good thing because we’re going to see how to be lazy in C++.

Lazy initialization is the practice of delaying initialization until it is needed. It can be used to optimize performance (which will be the subject of the next article), but it popped up when I was asked an interesting question a while ago while working for one of my clients.

int complex_computation()
{
    std::cout << "Compute" << std::endl;
    return 2;
}

int main()
{
    std::optional<int> a = 50;
    std::optional<int> b;

    std::cout << a.value_or(complex_computation()) << "\n";
    std::cout << b.value_or(complex_computation()) << "\n";
}

Why is the result of this not:

50
Compute
2

but is:

Compute
50
Compute
2

Shouldn’t the mantra for C++ be “You don’t pay for what you don’t use”? In this case, the computation isn’t needed for the first case…

Attentive readers will have noticed that the value_or is a function, and each argument of a function must be evaluated before the call.

In C++23, std::optional::or_else(f) solves this specific case since it takes a callable. However, value_or is far from the only function with this problem, so it is worth having a generic solution. For example, std::map::try_emplace suffers from this exact problem.

Tackling the problem

The first thing to do is to understand what value_or does. STL implementation from MSVC is something similar to:

template<typename T> class optional {
    template <class U>
    constexpr T value_or(U&& value) const& {
        if (this->has_value()) {
            return **this;
        }

        return static_cast<T>(std::forward<U>(value));
    }
};

The conversion from U to T occurs only in the fallback branch, so the computation should be triggered there.

Said another way, the T object must be materialized by converting U to T.

Let’s create a simple helper now!

template<typename F>
struct Lazy { // C++17 users will need a deduction guide (aggregate CTAD is C++20)
    template<typename T>
    operator T() const {
        return initializer();
    }

    F initializer;
};

int main()
{
    std::optional<int> a = 50;
    std::optional<int> b;

    std::cout << a.value_or(Lazy{complex_computation}) << "\n";
    std::cout << b.value_or(Lazy{complex_computation}) << "\n";
}

Now the result is exactly what we expected.

50
Compute
2

Limitation

No memoization:

int main()
{
    std::optional<int> a = 50;
    std::optional<int> b;
    std::optional<int> c;
    Lazy value{complex_computation};

    std::cout << a.value_or(value) << "\n";
    std::cout << b.value_or(value) << "\n";
    std::cout << c.value_or(value) << "\n";
}

Since Lazy does not memoize the result of the operation, the computation runs twice, once for b and once for c.

No constraint: if (Lazy{...}) will compile and may not do what you think it does.

Conclusion

std::optional is not to blame for the behavior we started with: C++ evaluates function arguments before the call, so value_or(complex_computation()) runs the computation whether we need it or not. What makes value_or interesting is that it only converts its argument in the fallback branch. Our Lazy helper exploits exactly that: it hides the computation behind a conversion operator, so the caller only performs the work when it actually materializes the value, and skips it otherwise.

We kept the helper deliberately minimal, and it shows two weaknesses: it recomputes the result on every conversion, and its unconstrained conversion operator silently converts to anything, bool included. In the next article, we will build a more robust lazy type that memoizes its result and converts only to the type its initializer returns, and we will measure its runtime cost.

I hope you enjoyed this article!

Reference:

MSVC STL: optional::value_or