DSPRelated.com
Blogs

Fixed-Point Simulation in GNU Octave—Without MATLAB

AHMED SHAHEINApril 11, 2026

The Itch I Couldn't Scratch

If you have spent any time doing DSP algorithm development that eventually targets an FPGA or ASIC, you know the drill. You prototype in floating-point — clean, fast, forgiving — and then comes the moment of reckoning: you have to port the whole thing to fixed-point and suddenly the comfortable world of real numbers collapses into overflow, rounding errors, and saturation surprises.

MATLAB handles this gracefully with its fi class. You wrap a value in fi(), tell it your word length and fractional length, pick a rounding mode, and it just works. You can inspect the binary representation, check for overflow, and run the exact same algorithm in floating-point and fixed-point side by side.

GNU Octave, which is free and open-source, has always been a close cousin of MATLAB. But one gap has been painful: there has been no equivalent of fi. You could approximate it with scaling tricks, but those fall apart quickly when you need to model saturation, configurable rounding, or 2's complement binary output for bit-true verification.

This article is available in PDF format for easy printing

That gap is what I set out to close.

Introducing pkg-fxp

pkg-fxp is a fixed-point arithmetic class for GNU Octave that mirrors MATLAB's fi workflow as closely as possible. It is free, open-source (MIT-licensed), and — as of a recent patch — also runs on MATLAB 2023 without modification.

The class is constructed around the same three-parameter philosophy MATLAB uses: signed/unsigned flag (S), word length (WL), and fractional length (FL). So a signed 16-bit number with 8 fractional bits — equivalent to MATLAB's numerictype(1,16,8) — looks like this:

x = fxp(0.75, 1, 16, 8);

That is it. From there, you get automatic quantization, configurable rounding (truncation, round-half-away-from-zero, ceiling, floor), saturation and wrap-around overflow modes, element-wise vectorized operations, and 2's complement binary display with the binary point shown explicitly — something I always found awkward to produce manually.

A Quick Walkthrough

Format and Quantization

Let's say you are building a biquad IIR filter and your coefficients are computed in double precision. You want to know what happens when you drop them to a signed 1.15 format (S=1, WL=16, FL=15):

b0_float = 0.12345;

b0_fxp = fxp(b0_float, 1, 16, 15);

disp(b0_fxp)

 Fixed-Point (fxp) Object:
 Config                    : 8-bit, Sign=1, Int=1, Frac=6
 Range                   : [-2, 1.984375]
 Resolution             : 0.015625
 Dynamic Range:    : 12.04 dB
 Value                     : -2.000000
 Quantization Error: 0
 Binary                   : 10.000000
 Overflow               : 1

The quantization error is immediately visible. No scaling gymnastics, no manual bit-shifting — just a direct comparison between your design intent and what the hardware will actually compute.

Fig. 1 shows an FIR frequency response comparing float to fxp outcomes. While, Fig. 2 is just a zoom-in around the filter's passband.Fig. 3 shows the quantized filter coefficients (right column) compared to the floating coefficients (the left column).

Fig. 1

Fig. 2


Fig. 3


Seeing Overflow in Action

One of the most useful things a fixed-point model can do is show you exactly when and how overflow occurs. With pkg-fxp you can choose between saturation (clamp to max/min) and wrap-around, and watch the effect:

a = fxp(1.99, 1, 8, 6, 'ovf_action', 'sat'); % saturates at ~1.984

b = fxp(1.99, 1, 8, 6, 'ovf_action', 'wrap'); % wraps into negative range -2.0

The Binary Representation You Actually Want

A feature I personally missed for a long time: proper 2's complement display with a visible binary point. When you are debugging an RTL simulation and comparing against a reference model, you want to see the bits as the hardware sees them:

x = fxp(1.56872, 1, 8, 6);

disp(x.bin) % displays: 01.100100
x.bin_str % displays: 01.100100

Why This Matters for RTL Verification

Here is the workflow that motivated this project in the first place:

  • Design an algorithm in floating-point Octave.
  • Gradually convert it to fixed-point using fxp, observing where precision is lost and where overflow lurks.
  • Generate golden reference vectors — hex files — from the Octave model.
  • Drive a SystemVerilog testbench with those vectors and compare RTL output against the reference.

Step 4 is where bit-true accuracy matters most. If the Octave model does not match the RTL's rounding and saturation behavior exactly, your verification is worthless. pkg-fxp was built with that contract in mind — the numbers it produces are designed to match what a properly coded RTL module produces.

Comparison with MATLAB fi

For those coming from MATLAB, the mapping is intentional and direct:

MATLAB: x = fi(0.75, 1, 16, 8)

Octave: x = fxp(0.75, 1, 16, 8)

MATLAB: x.bin → binary string

Octave: x.bin → binary string with decimal point

MATLAB: fimath rounding modes

Octave: 'round', 'floor', 'ceil', 'fix' string arguments

It is not a clone — there are deliberate differences, especially around the binary display and how rounding modes are specified. But if you know fi, you will not be lost. And if you do not have a MATLAB license, you now have a free alternative that runs the same regression tests and produces bit-identical results for the cases I have tested.

Current Status and How to Get It

The package is actively developed and tested. The regression suite runs against Python's fxpmath library as a golden reference, and as of version 5.1.0, the code is 100% compatible with both GNU Octave and MATLAB (verified on MATLAB 2023).

To install in GNU Octave (from Octave itself):

pkg install -forge fxp

pkg load fxp

Or clone from:

https://github.com/ahmedshahein/pkg-fxp.git

addpath('/path/to/pkg-fxp');

What's Next

A few things are on the roadmap:

  • Integration examples: FIR/IIR filter quantization, CORDIC, and FFT fixed-point analysis
  • SystemC and SystemVerilog co-verification flow documentation

If there is a specific DSP building block you would like to see demonstrated, drop a comment below. That is genuinely the best way to guide what gets built next.


To post reply to a comment, click on the 'reply' button attached to each comment. To post a new comment (not a reply to a comment) check out the 'Write a Comment' tab at the top of the comments.

Please login (on the right) if you already have an account on this platform.

Otherwise, please use this form to register (free) an join one of the largest online community for Electrical/Embedded/DSP/FPGA/ML engineers: