RMAC/RLN

swapd0 · August 8, 2020

I've done a full compile for BurgerTime and my engine library and It looks that there are something wrong.

I'm using gcc and rmac, I've a some .c and .s files for the gcc boot code (brownboot.s, zerocrtfini.c, zerolib.cpp, etc) I've included into this "library" the skunk.s, also I've my c/asm library common to all my games, and then I've the current game that I'm developing.

With the latest rmac version I've the following errors when I compile BurgerTime.

/opt/local/bin/gcc68k/lib/libcxx-jaguar/zerocrtfini.o: In function `_exit':
zerocrtfini.c:(.text.exit+0x0): multiple definition of `_exit'
/opt/local/bin/gcc68k/lib/libcxx-jaguar/tinyalloc.o: In function `_ta_init':
tinyalloc.c:(.text.ta_init+0x0): multiple definition of `_ta_init'
/opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o: In function `_skunkRESET':
(.text+0x0): multiple definition of `_skunkRESET'
/opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o: In function `_skunkNOP':
(.text+0x3a): multiple definition of `_skunkNOP'
/opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o: In function `_skunkCONSOLEWRITE':
(.text+0x6e): multiple definition of `_skunkCONSOLEWRITE'
obj/main.o: In function `_main':
main.c:(.text.startup.main+0x0): multiple definition of `_main'
/Users/Shared/apps/Jaguar/projects/lib/libkatu.a(debug_asm.o): In function `_skunkCONSOLEWRITE':
(*ABS*+0x0): multiple definition of `_skunkFILEOPEN'
/opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o:(.text+0x170): first defined here
/Users/Shared/apps/Jaguar/projects/lib/libkatu.a(debug_asm.o): In function `_skunkCONSOLEWRITE':
(*ABS*+0x0): multiple definition of `_skunkFILECLOSE'
/opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o:(.text+0x2a0): first defined here
/Users/Shared/apps/Jaguar/projects/lib/libkatu.a(debug_asm.o): In function `_skunkCONSOLEWRITE':
(*ABS*+0x0): multiple definition of `_skunkFILEWRITE'
/opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o:(.text+0x1ca): first defined here
/Users/Shared/apps/Jaguar/projects/lib/libkatu.a(gpu_asm.o): In function `DRAM':
(*ABS*+0x0): multiple definition of `_gpu_events_list'
/Users/Shared/apps/Jaguar/projects/lib/libkatu.a(sprite.o):(.bss._op_start+0x0): multiple definition of `__op_start'

Ok, let's have a look at _skunkRESET, it's defined in skunk.s like this.

; skunkRESET()
; Resets the library, waits for the PC, and marks the console up or down
_skunkRESET::
		movem.l	a1-a2,-(sp)

		move.w	#$4001,_skunkReadMode	; ensure normal 4MB mode set

continueRESET:
		move.l	#-1,_skunkConsoleUp		; optimistic!

		bsr		setAddresses		; get HPI addresses into a1 & a2
		; try and get both buffers, that tells us the console is up
		bsr		getBothBuffers		; also sets skunkConsoleUp
		bsr		restoreMode			; set correct flash mode
		movem.l (sp)+,a1-a2			; Restore regs
		rts

And it's referenced in brownboot.s with a .extern _skunkRESET and somewhere in the same file I've a jsr _skunkRESET, it's not called from my engine library, neither from my game, and it's only defined in skunk.s... so why I've a "multiple definition" error?

edit:

After some testing looks like the symbol must be referenced from at least two different files to get the error of multiple definitions, and must be defined in a third file.

nm dump for skunk.o and brownboot.o

swapd0$ nm skunk.o
000000b0 T _skunkCONSOLECLOSE
000000f6 T _skunkCONSOLEREAD
0000006e T _skunkCONSOLEWRITE
000004a4 t _skunkConsoleUp
000002a0 T _skunkFILECLOSE
00000170 T _skunkFILEOPEN
00000214 T _skunkFILEREAD
000001ca T _skunkFILEWRITE
000002dc T _skunkMEMREAD
00000356 T _skunkMEMWRITE
0000003a T _skunkNOP
00000000 T _skunkRESET
00000028 T _skunkRESET6MB
000004a4 t _skunkReadMode
000004a4 t _skunkTimeout
000003dc t checkBuffer1
000003e8 t checkBuffer2
000003ba t checkbuffer
0000000c t continueRESET
00000436 t getBothBuffers
000003f4 t getBuffer
000003b2 t restoreMode
000003a0 t setAddresses
0000047e t waitforbufferack

swapd0$ nm brownboot.o
00f02200 a A1_BASE
00f02208 a A1_CLIP
...
00000152 T __exit
00000000 A _exit
000004c8 T _init_debugger
000004c2 T _jaguar_reset
00000000 A _main
00000154 T _memcpy
000002e2 T _memmove
00000272 T _memset
00000466 T _putchar
0000047c T _rand
00000000 A _skunkCONSOLEWRITE
00000000 A _skunkNOP
00000000 A _skunkRESET
0000049e T _srand

If I use the old rmac I get an 'U' for _skunkRESET instead of an 'A'

Edited August 8, 2020 by swapd0

ggn · August 9, 2020

Ok, so what probably happens now is that rmac optimistically exports all global symbols as Absolute. Reading your post (thanks for bearing with this btw) I think the problem isn't in skunk.s but brownboot.s, or rather the way symbols are exported in brownboot.s.

Last time I asked you to remove these two lines in object.c:

    else if (globflag)
        st_shndx = 0;        // Global, not absolute

This time I ask of you to replace them with

	else if (globflag && !(w1 & DEFINED) && (w1 & REFERENCED))
	{
		st_shndx = SHN_UNDEF;
	}

This will hopefully fix all your "multiple definition" issues (unless there's some other case I'm not considering right now, which is possible!). So give it a go and let us know what happens.

Thanks

swapd0 · August 9, 2020

Now it compiles but the game doesn't start.

swapd0 · August 9, 2020

With the older version it runs, at the start I print this through the skunk board, with the new version it hangs after the Starting console... message.

I'll have a look later at the symbol table.

Starting console...
init memory alloc at 00032c00
find_file_system at 00802000 Found at 0080209c
init_file_system at 0080209c with 26 files
NTSC Jaguar

ggn · August 9, 2020

If it's only rmac that's different between the 2 builds you could even binary compare the two different output files. Also, producing a map file ("-Xlinker -Map=output.map" for gcc) when linking might also shed some clues. Or, even better, binary diffing the object files produced by the 2 rmacs would help tremendously.

swapd0 · August 10, 2020

I've only changed the rmac version, same compiler options.

old rmac output size is 309,872 bytes

new rmac output size is 290,760 bytes

There are a lot of labels that are shifted.

I don't understand this, old rmac

00f0323c A _GPU_MAX_VIDEO_X
00f03240 A _GPU_MAX_VIDEO_Y
00f03234 a _GPU_SPRITE_OFFSETS
00f03234 A _GPU_SPRITE_OFFSET_X
00f03238 A _GPU_SPRITE_OFFSET_Y

New rmac

00f03228 A _GPU_MAX_VIDEO_X
00f03228 A _GPU_MAX_VIDEO_Y
00f03228 a _GPU_SPRITE_OFFSETS
00f03228 A _GPU_SPRITE_OFFSET_X
00f03228 A _GPU_SPRITE_OFFSET_Y

On the new version the labels are in a lower memory address and they are from an assembly file, so they are using the same instructions (same size). Also, why _GPU_SPRITE_OFFSET_Y are at the same address as _GPU_SPRITE_OFFSET_X? They are defined like this.

	.long
_gpu_sprite_dst::		dc.l 0,0		; destination address

_GPU_SPRITE_OFFSETS:
_GPU_SPRITE_OFFSET_X::		dc.l 0
_GPU_SPRITE_OFFSET_Y::		dc.l 0

_GPU_MAX_VIDEO_X::		dc.l 0
_GPU_MAX_VIDEO_Y::		dc.l 0

Any tool for the binary-diff? I've my own but the output it's something like this.

out-new.bin and out-old.bin has different sizes.
differs at offset 0x22 5 4f
differs at offset 0x23 38 e0
differs at offset 0x46 8 53
differs at offset 0x47 c4 64
differs at offset 0x4a a6 eb
differs at offset 0x4b ac 4c
differs at offset 0x201f b8 bc
differs at offset 0x2061 0 1
differs at offset 0x2062 0 e5
differs at offset 0x2063 0 48
differs at offset 0x213d d0 d4
differs at offset 0x2142 49 4b
differs at offset 0x2143 50 20
differs at offset 0x2148 49 4b
differs at offset 0x2149 8a 5a
differs at offset 0x214e 73 7d
differs at offset 0x214f 4e 2
differs at offset 0x2158 90 9a
differs at offset 0x2159 62 16
differs at offset 0x2471 0 1
differs at offset 0x2472 0 e5
differs at offset 0x2473 0 50
differs at offset 0x2477 0 1
differs at offset 0x2478 0 e5
differs at offset 0x2479 0 50
differs at offset 0x247e 49 4b

out-new.map.txt out-new.txt out-old.map.txt out-old.txt

ggn · August 10, 2020

Hi there,

Thanks a lot for the files. From the little I can gather it seems like the linker is rejecting the data from the symbols. So my guess is that while it's importing the symbols correctly, it thinks that the data associated with the symbols is 0 in length, maybe that's the reason you see symbol addresses clashing.

Also, since we get such a large difference in the final binaries, it makes little sense to binary compare them. Instead, it'd be really useful if you could binary diff the object files generated by the two rmacs. For example brownboot.o. At least there I more or less expect to see the same file sizes but some bytes will be different, most likely in the symbol table. So it'd be great if I could have such a diff.

Also, as for a visual hex diff client you can try https://winmerge.org - just go to edit->options under "binary" and append the file extension you'll diff (most likely ".o"). Then drag the two object files and it'll do its job.

Thanks for going through with this so far!

ggn · August 10, 2020

I just realised that I actually have some of these files since they're from dml's tiny startup code - d'oh!

So I tried assembling brownboot.s using a pre-2.0.0 version of rmac, as well as the latest with all the the above proposed fixed applied. Then I binary diffed them and something stood out:

Seems like the name of the section names was turned to lower case at some point and nobody complained until now. So, current theory: the linker is anal about case sensitivity in section names, sees the label names, loads them into a different section, then sees that the sections aren't used, and then zaps them from orbit.

So let's try something. Around line 561 of object.c there should be a line like

			shstSize += DepositELFSHSTEntry(&shstPtr, ".text");

and a similar line below for ".data" and ".bss". Change those to "TEXT', "DATA", and "BSS" (note: no leading dot "."). Recompile rmac and test things out. I have a good feeling about this

swapd0 · August 10, 2020

Done, ".text" changed for "TEXT", and the same for data and bss

Now the file size is sightly bigger (290,932 bytes) but still smaller than the one using old rmac. The labels are still clashing.

ggn · August 10, 2020

11 minutes ago, swapd0 said:

Done, ".text" changed for "TEXT", and the same for data and bss

Now the file size is sightly bigger (290,932 bytes) but still smaller than the one using old rmac. The labels are still clashing.

Are you able to diff a few object files now?

swapd0 · August 10, 2020

They are quite different. I think that it's better if I send you several object files.

In the gpu.o is where _GPU_SPRITE_OFFSET_X, _GPU_SPRITE_OFFSET_Y, _GPU_MAX_VIDEO_X are defined

gpu-new.o gpu-old.o main-new.o main-old.o video-new.o video-old.o brownboot-new.o brownboot-old.o

ggn · August 10, 2020

Without going any further I noticed some weird things.

For starters, brownboot.o:

That's 68000 code in there, and it seems that after that 4E75 (RTS) where everything starts mismatching very badly there is a "10a8 60c1 4e75" (which is move.b $6061(a0),a0) / rts) snippet inserted in the "old" version instead of just a "4ef9" which appears on the "new" version.

Similarly, in gpu.o:

A "0000 0000" is inserted at offset $78 which seems to derail the whole binary afterwards.

The other two sets of .o files seem to be an identical match, but I'm puzzled about these two. It seems like they're assembled from a different version of the source codes. Is it possible that this is the case?

swapd0 · August 10, 2020

20 minutes ago, ggn said:

The other two sets of .o files seem to be an identical match, but I'm puzzled about these two. It seems like they're assembled from a different version of the source codes. Is it possible that this is the case?

I have only a version of the source code, and I've made a make clean of each project. Anyway I've done this just to be sure.

swapd0$ rmac-old -fe brownboot.s -o brownboot-old.o
swapd0$ rmac-new -fe brownboot.s -o brownboot-new.o

brownboot-new.o brownboot-old.o

ggn · August 10, 2020

Sorry, but the invocation is wrong, it should be:

swapd0$ rmac-old -fe -o brownboot-old.o brownboot.s
swapd0$ rmac-new -fe -o brownboot-new.o brownboot.s

(yes, it's a remnant from the old MadMac times, the idea behind this is that you can assemble multiple files from the command line that have different options you can switch on and off. So whenever rmac encounters an input file, it goes and assembles without parsing the rest of the arguments)

(I still believe there's an issue since the register clashing is real fwiw)

Edited August 10, 2020 by ggn

swapd0 · August 10, 2020

34 minutes ago, ggn said:

Similarly, in gpu.o:

A "0000 0000" is inserted at offset $78 which seems to derail the whole binary afterwards.

I'm 100% that the extra "0000 0000" are from a .long directive to align a data. It' this part of the code.

    movei #_gpu_events_list,list_ptr
    movei #.execute_list,r1
    jump (r1)
    load (list_ptr),events

    .long
_gpu_events_list::
    dc.l 0  ; pointer to events list

"D020" it's the jump (r1) and the "A672" it's the load (list_ptr),events, then two bytes with zeros for the alignment but in the new version it appends 4 extra bytes.

swapd0 · August 10, 2020

9 minutes ago, ggn said:
Sorry, but the invocation is wrong, it should be:
swapd0$ rmac-old -fe -o brownboot-old.o brownboot.s
swapd0$ rmac-new -fe -o brownboot-new.o brownboot.s
(yes, it's a remnant from the old MadMac times, the idea behind this is that you can assemble multiple files from the command line that have different options you can switch on and off. So whenever rmac encounters an input file, it goes and assembles without parsing the rest of the arguments)

(I still believe there's an issue since the register clashing is real fwiw)

Ok, I will fix all the makefiles

Edit:It was correct in the makefiles...

Edited August 10, 2020 by swapd0

ggn · August 13, 2020

Oh hi, I lost track of this thread because I missed the edit in the last post.

So, my guess that now you should be able to make equal sized brownboot.o files but with some slight differences. Could you try that?

swapd0 · August 13, 2020

I've downloaded the latest version 2.0.20 (sorry but I think that I was using 2.0.16), and I've done the modifications.

Almost the same size.

swapd0$ rmac-old -fe -o brownboot-old.o brownboot.s
swapd0$ rmac-new -fe -o brownboot-new.o brownboot.s
swapd0$ ls -l brownboot*
-rw-------  1 swapd0  wheel  18548 13 ago 16:59 brownboot-new.o
-rw-------  1 swapd0  wheel  18552 13 ago 16:58 brownboot-old.o

brownboot-new.o brownboot-old.o

ggn · August 13, 2020

Oh, I see. Well, v2.0.20 doesn't have the proposed fixes of this thread yet (because we're still investigating). So what you could do is to grab the latest snapshot of the source code (currently http://shamusworld.gotdns.org/cgi-bin/gitweb.cgi?p=rmac;a=tree;h=234f42879ceda29cd6a37a50ff287730f4ac632e;hb=HEAD - just click "snapshot") and then apply the things on this thread.

This means posts #102 and #108. Then recompile and generate brownboot.o. I expect that with this you'll get equal sized files but with a few diffs.

I hope we're getting to the bottom of this soon!

swapd0 · August 13, 2020

Done, downloaded and patched, I think that I get the same result.

brownboot-new.o brownboot-old.o

swapd0 · August 13, 2020

By the way I get a compiler error with SHN_UNDEF but I've googled it and changed to 0.

ggn · August 13, 2020

Hmm, that's odd

The differences start yet again on the name of the segments - old has ".text", new has "TEXT" etc. Did you apply that fix?

[EDIT]

On second thoughts, could you attach brownboot.s? I'll take a look at it here and not have to bother you.

Edited August 13, 2020 by ggn

swapd0 · August 13, 2020

Yes, I changed .text to TEXT and the same for .data and .bss (I hope that I didn't understand in the wrong way)

Maybe you can remove the include "debugger.s" and the jsr _init_debugger to fix the bug.

brownboot.s debugger.s

ggn · August 15, 2020

Okay, my computer gave me the day off yesterday as it needed to install some very important updates, but I'm back at the helm now.

What I did just now was to try and match my brownboot.o with the brownboot-old.o in post #120. I assume that one is linking fine for you. Skipping swiftly past the fact that jaguar.inc was different than the stock one and had to be recreated, I'm now able to produce a 1:1 copy of brownboot-old.o.

My version is built from head (http://shamusworld.gotdns.org/cgi-bin/gitweb.cgi?p=rmac;a=snapshot;h=821279dcb7b5d069d175767edcd413b833b0cae3;sf=tgz) with the following changes:

object.c: line 235 replaced with:

	uint16_t st_shndx = SHN_ABS;	// Assume absolute (equated) number

lines 243 and 244 replaced with:

	else if (globflag && !(w1 & DEFINED) && (w1 & REFERENCED))
	{
		st_shndx = SHN_UNDEF;
	}

object.h: the following inserted in the file (around line 35 but shouldn't matter)

//
// ELF special section indices (field st_shndx)
// Lifted from glibc (https://sourceware.org/git/?p=glibc.git;a=blob;f=elf/elf.h)
//
#define SHN_UNDEF       0               /* Undefined section */
#define SHN_ABS         0xFFF1          /* Associated symbol is absolute */
#define SHN_COMMON      0xFFF2          /* Associated symbol is common */

and that's it. If I want to match brownboot-new.o from post #120 I can do so by replacing line 561 in object.c with

			shstSize += DepositELFSHSTEntry(&shstPtr, "TEXT");

and "DATA", "BSS" in lines 570 and 579 respectively.

Now, again, if brownboot-old.o links ok for you then this should also link as it should produce the same object file!

swapd0 · August 15, 2020

Ok, I've done all the changes except .text -> TEXT, .data -> DATA and .bss -> BSS.

After a clean make both versions of the game are the same!!!

Thanks a lot for all the work!

Edited August 15, 2020 by swapd0

RMAC/RLN

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation

Recently Browsing 0 members