# 6502 - detecting ADC overflow with unsigned + signed

## Recommended Posts

Is there a simple way to detect overflow in case of addition (ADC) of unsigned and signed value? If I read description of overflow flag it works only for both signed values.

I'd like to detect overflow of unsigned value when signed one is added e.g.:

unsigned \$1 + signed \$FE (-2) -> overflow

unsigned \$1 + signed \$1 -> no overflow

unsigned \$FF + signed \$1 -> overflow

unsigned \$FF + signed \$FE (-2) -> no overflow

##### Share on other sites

Just to be sure, you want [0...255] adc [-128...127] and detect what exactly? I assume you mean the answer should be in the range of [0...255] and how to detect when it's not?

##### Share on other sites

Note the 6502 maths instructions have no distinction between siqned/unsigned, the flags are just there to help.

Also note that behaviours can be different if you're using decimal (SED) mode.

The carry bit is set if you ADC and an unsigned # goes beyond \$FF

The carry bit is clear if you SBC and an unsigned # goes below \$00

From memory, the overflow bit is set if the sign (bit 7) changes in either operation.

##### Share on other sites

Consider: A = unsigned, B = signed

if (A >= 129) and (B is +ve) then for A + B, 'C'arry indicates overflow

if (A >= 129) and (B is -ve) then for A + B, there cannot be an overflow

if (A < 129) and (B is +ve) then for A + B, there cannot be an overflow

if (A < 129) and (B is -ve) then for A + B, 'C'arry indicates overflow

So because the test on A is not a nice BPL/BMI a CMP #129/BC[CS] would have to be employed so might be be so efficient.

##### Share on other sites

I know that according to ADC specification added values need to be of the same type, but I was wondering if there is some trick that could be used to identify such case and treat one as unsigned and other signed. The situation can be identified and handled with conditional jumps, but maybe there is some faster way (preset flags?). To explain the use case - I have a page and unsigned byte Index for accessing its data. I move the index forward or backward by adding signed Step value. I need to identify case where Index is "virtually" wrapping or crossing byte boundary, therefore e.g. if Index is \$F1 and I add \$FF, by 6502 it's overflow, however I consider here \$FF as -1 and output value of Index is \$F0, therefore for my use-case there is no "wrapping". I'm going through such page in a loop and need to break the loop at the moment when Index moving forward or backward by Step is crossing page boundary.

Where it's needed - e.g. raycaster where I move a ray by step on "upscalled grid" to 256*256 resolution (for calc precision), where I need to detect if a ray is crossing 256*256 boundary.

Edited by ilmenit

##### Share on other sites

Rebiasing the unsigned value to signed might work:

```LDA unsigned_value
EOR #\$80
EOR #\$80
BVS overflow```

It's a trick I occasionally use in vectorized code since SSE2 is annoying non-orthogonal for byte ops.

• 4
• 4

##### Share on other sites
2 hours ago, phaeron said:

Rebiasing the unsigned value to signed might work:

```LDA unsigned_value
EOR #\$80
EOR #\$80
BVS overflow```

It's a trick I occasionally use in vectorized code since SSE2 is annoying non-orthogonal for byte ops.

Oh wow, it seems to work! Man, you are amazing! 🙂

##### Share on other sites

I've been thinking about this for hours, and it was driving me crazy .. but in the darkest hour, Phaeron comes to the rescue !

##### Share on other sites
8 hours ago, ilmenit said:

e.g. raycaster where I move a ray by step on "upscalled grid" to 256*256 resolution (for calc precision), where I need to detect if a ray is crossing 256*256 boundary.

That's the correct "generic" way of doing it, but you also could have different logic for different "quadrants" for the ray (centered in the viewpoint position).

So, for example, a ray moving in a "top-left" direction, could check X against the 0 limit and Y against the 256 limit only (assuming coordinate Y grows from "bottom" to "top").

Also, you normally trace rays from left to right ("clockwise"), so you don't change "quadrants" that often.

##### Share on other sites
19 minutes ago, NRV said:

That's the correct "generic" way of doing it, but you also could have different logic for different "quadrants" for the ray (centered in the viewpoint position).

So, for example, a ray moving in a "top-left" direction, could check X against the 0 limit and Y against the 256 limit only (assuming coordinate Y grows from "bottom" to "top").

Also, you normally trace rays from left to right ("clockwise"), so you don't change "quadrants" that often.

Correct. I was considering doing different procedures for quadrants (or even octants) or use self-modified code, but in my case I have more space for precalculated data and I can store it for all the "angles", which greatly simplifies the code.

Phaeron's proposal is great for some other "clipping" purposes too.

Edited by ilmenit

##### Share on other sites
1 hour ago, Wrathchild said:

Nice... never used it, but always believed the common myth about it, that the overflow bit indicates a carry (or overflow) from bit 6.. x)

## Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

×   Pasted as rich text.   Paste as plain text instead

Only 75 emoji are allowed.

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.