You are browsing as a guest. Sign up (or log in) to start making projects!

14h 5m 48s logged

calc82: Devlog #2 08/07/2026

calc82: because calculators are for everyone

This devlog follows the handling of nasty floating point errors in my web calculator - inspired by and designed to work with Casio’s fx-82au.

I realised that relying purely on the Javascript math system was unsuitable for a scientific calculator that works with decimal numbers. This is due to binary floating point not translating well to decimal.

0.1+0.2=0.30000000000000004 >:(

I researched and considered a few approaches.

🥧 Symbolic simplifier?

The idea behind this is to perform algebra and avoid evaluating constants and functions to values until strictly necessary. However, the scope of this seemed massive, and I did not observe this behaviour in the Casio.

🔟 Decimal arithmetic

Instead, I stumbled upon the fact that decimal arithmetic can be implemented in a computer. While IEEE floats use binary representation, there is no reason we cannot use decimals internally, avoiding errors that arise from conversion. Decimal floats can be stored by:

  • Sign: +/-
  • Mantissa: the number itself, in my case an array of digits
  • Exponent: what power of 10 the number should be multiplied by.

I started implementation, made decent progress, but found out that I was re-doing work that was already done by other people.

🗄️ Decimal.js

I found the decimal.js library after researching how to handle decimal addition. Implementing this quickly fixed my floating point errors, but did not fix trig approximation.

👀 Observing the calculator

I observed some of the behaviour of the real calculator to work out how it functions.

  • Handles small-angle approximation based on a threshold
  • Drops precision as the trig function argument value increases
  • Appears to use a 15-digit mantissa
  • Refuses to evaluate trig functions with argument above a certain limit

I tried my best to implement the behaviour I observed, some of which required editing the code of the decimal.js library.

📝 Editing decimal.js

The decimal.js library uses more than the set precision for some internal calculations. However, arithmetic with pi is otherwise only the set precision, causing drift.

My fix for this was to adjust the internal pi value to match the mantissa size of the real calculator.

Troubles with the library

When given a less accurate value for pi (in line with that of the real calculator) sine quadrant is incorrectly calculated. I found that the isOdd function is flawed!

function isOdd(n) {
return n.d[n.d.length - 1] & 1;
}

I replaced it with one that works for my use case (if less efficient):

function isOdd(n) {
    return n.mod(2).eq(1);
}

I need to see if this is just due to my low precision pi, or an actual problem in the library, in the future. But for now my issue is fixed.

I also found an error in the cosine function: if (x.isZero()) return x needed to be changed to if (x.isZero()) return new Ctor(1).

Again, I’m going to have to look into this more before submitting a pr or anything, since decimal.js is not made for such an imprecise pi.

🗒️ Notes

Deviations
There are still small deviations between my calculator and the real one at really, really large trig angles.

One idea to reduce error could be optimising the order of operations in the AST to prevent dropping precision unnecessarily. I might explore this later

However, I want to press on with other features, instead of tiny inaccuracies that would never show up in real-world use. I have slightly rearranged the to-do list below.

📈 Next steps

Immediate:

  • Implement the other calculator buttons and functions
  • Add fractional I/O

Medium:

  • Create algebraic display functionality
  • Multi-line calculation (Ans) and variables

Long-term:

  • Build UI
  • Ship :)

Thanks so much for reading! If you have any suggestions, please leave me a comment. Until next time

0
2

Comments 0

No comments yet. Be the first!