Seems like you're making a leap of faith to assume that Motorola's docs would clearly outline which instructions were futureproof or not - you'd have to check really.
In any case, maybe the compiler version, or the build flags to the compiler happened to use instructions or alignments that turned out to be incompatible with the later chips. The source code to TOS was IIRC about 60%-70% C and the rest assembly. This was apparently one reason why the VDI/AES routines were so slow in GEM applications, leading to third-party improvements like QuickST (or maybe it was TurboST) and NVDI which used optimised assembly for the bottleneck operations.
Also IIRC from the superb Dadhacker blog and other articles, wasn't TOS a port of CP/M, with DR's GEM thrown on top? Note that GEM was released standalone for PC DOS as well, and was eventually crippled by a nonsensical Apple lawsuit, although its modern descendants look nice today