Introduction

A 17-bit square produces up to 34 bits, so the right AVX2 implementation depends almost entirely on which result bits you need to keep. If the algorithm only needs the low part of the product, the problem is much simpler than full-precision multiplication. In that case, AVX2 can process several values per register with ordinary 32-bit lane operations.

Start from the Width Analysis

Each input fits in 17 bits, so each square fits in 34 bits. AVX2 does not have a direct “vector 34-bit integer” type, so the normal mapping is to store each value in a 32-bit lane and use a multiply that naturally gives you a truncated low result.

That makes the first design question:

• do you need the full 34-bit square
• or do you only need the low n bits

If truncation is acceptable, _mm256_mullo_epi32 is usually the right instruction because it computes eight independent 32-bit products and returns the low 32 bits of each.

Mask Inputs Before Squaring

If the incoming data is packed into 32-bit integers but only the low 17 bits are meaningful, mask the values first. This prevents garbage in the upper bits from corrupting the product.

1#include <immintrin.h>
2#include <stdint.h>
3
4static inline __m256i square17_low32(__m256i x) {
5    const __m256i mask17 = _mm256_set1_epi32((1u << 17) - 1);
6    x = _mm256_and_si256(x, mask17);
7    return _mm256_mullo_epi32(x, x);
8}

This function squares eight lanes in parallel and keeps the low 32 bits of each result. For many DSP-style and hashing-style uses, that is already the exact answer.

Truncate Further with a Final Mask

Sometimes the algorithm wants only the low 17, 18, or 24 bits rather than the low 32. In that case, keep the AVX2 multiply and apply a second mask afterward.

1static inline __m256i square17_low17(__m256i x) {
2    const __m256i mask17 = _mm256_set1_epi32((1u << 17) - 1);
3    __m256i product = square17_low32(x);
4    return _mm256_and_si256(product, mask17);
5}

That remains efficient because masking is cheap compared with trying to reconstruct a full-width result that you do not actually need.

End-to-End Example

This complete program loads eight unsigned inputs, squares them, and stores the low 32 bits of each result.

1#include <immintrin.h>
2#include <stdint.h>
3#include <stdio.h>
4
5static inline __m256i square17_low32(__m256i x) {
6    const __m256i mask17 = _mm256_set1_epi32((1u << 17) - 1);
7    x = _mm256_and_si256(x, mask17);
8    return _mm256_mullo_epi32(x, x);
9}
10
11int main(void) {
12    uint32_t input[8] = {1, 2, 3, 1000, 50000, 70000, 100000, 131071};
13    uint32_t output[8];
14
15    __m256i x = _mm256_loadu_si256((const __m256i *)input);
16    __m256i y = square17_low32(x);
17    _mm256_storeu_si256((__m256i *)output, y);
18
19    for (int i = 0; i < 8; i++) {
20        printf("%u\n", output[i]);
21    }
22
23    return 0;
24}

Compile with AVX2 enabled, for example -mavx2 on GCC or Clang.

Signed and Unsigned Interpretation

For a pure 17-bit square, signedness often disappears because x * x is non-negative. But signedness still matters if the source values are sign-extended into the upper bits of each lane before masking. If you intend unsigned 17-bit data, mask before multiplication. If you intend signed 17-bit data, first normalize the representation so the values are what your algorithm expects.

That detail is easy to miss when data arrives from packed bitfields or from a previous SIMD stage.

Know When _mm256_mullo_epi32 Is Not Enough

If you need the full 34-bit result, _mm256_mullo_epi32 alone is insufficient because it drops the high two bits of the mathematical square. In that case you need a wider reconstruction strategy, often by decomposing the value into smaller pieces or by using 64-bit lanes with lower throughput.

That is a different optimization problem from truncated squaring. The most efficient solution depends on whether your downstream algorithm consumes full precision or just the low part.

Memory Layout Still Matters

The arithmetic is only half the story. If inputs are not already arranged as eight 32-bit values, the cost of unpacking can dominate the multiply. AVX2 shines when the data is already lane-friendly or when unpacking can be amortized across a larger pipeline.

So benchmark the whole loop, not just the intrinsic in isolation. A beautiful multiply kernel can still be irrelevant if the surrounding shuffles cost more than the arithmetic.

Common Pitfalls

The biggest mistake is optimizing for full precision when the algorithm only needs the low result bits. That adds complexity for no benefit.

Another mistake is forgetting to mask inputs down to 17 bits. If upper bits are dirty, the product is wrong even though the intrinsic itself is behaving correctly.

Developers also assume _mm256_mullo_epi32 returns a full 34-bit square. It does not. It returns only the low 32 bits of each 32-bit lane product.

Summary

• A 17-bit square needs 34 bits in full precision, but truncation changes the implementation strategy.
• For low-part results, AVX2 maps naturally to eight 32-bit lanes.
• '_mm256_mullo_epi32 is the core instruction for an efficient truncated implementation.'
• Mask inputs before multiplying, and mask outputs afterward if you need fewer than 32 low bits.
• If you need the full 34-bit result, treat that as a separate design problem rather than stretching a truncated approach beyond its limits.